Expired ssl and bad gateway on reboot

debugger008 · June 10, 2019, 12:17pm

let’s encrypt ssl is expired today on mib 0.30 ( ubuntu 14.04 )

each time I run

sudo mailinabox

these rows in nginx log increases and the trouble is still here.

2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:09 [warn] 10175#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found
2019/06/10 12:59:10 [warn] 10179#0: "ssl_stapling" ignored, issuer certificate not found

I tried to restore ssl deleting the folder and recreating with

root@box:~/mailinabox/management# python3 ssl_certificates.py 

skipped: www.ANOTHER.DOMAIN:
The domain name does not resolve to this machine: x.x.x.x (A).

error: BOX.HIDDEN.REAL.DOMAIN:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.31.0', 'console_scripts', 'certbot')()
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 560, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2648, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2302, in load
    return self.resolve()
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2308, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 14, in <module>
    from acme.magic_typing import Union  # pylint: disable=unused-import, no-name-in-module
ImportError: No module named 'acme.magic_typing'

but no luck

please put me on the right way to set a new ssl and to make the mib working again.

alento · June 10, 2019, 1:53pm

The short answer is …

Upgrade to Ubuntu 18.04 and migrate.

I am still thinking through if there is a reasonable long answer.

I would highly suggest grabbing a backup before 3AM…

debugger008 · June 10, 2019, 2:46pm

Frankly speaking I think is something “easy” to fix but very hard to find and is related to let’s encrypt.

the weird thing is that i have 3 different vps with the same software/versions on it and just this has been blocked this morning.

( 3 different domain for 3 different customers )
I’m now trying to create manually a let’s encrypt cert with certbot :-/

alento · June 10, 2019, 2:50pm

I do NOT disagree at all … but the thing is as Ubuntu 14.04 becomes more out of date things are going to start going wrong.

Did you keep the contents of the old /ssl directory?

debugger008 · June 10, 2019, 2:52pm

sure i backupped it before deleting

alento · June 10, 2019, 3:02pm

@debugger008

Ok … something you did threw me for a loop so I did a test on my test install.

Restore to the previous condition with the /ssl directory intact as it was originally.

then from ~/mailinabox/management do the following on the command line as the root user.

./ssl_certificates.py

You did it with “python3 ssl_certificates.py” on the command line. When I do that it throws errors. Running it without calling python3 works fine for me.

~~You may even be able to do this without restoring the /ssl directory as this should create a new LE account.~~ No, this script does not appear to install the LE client itself. So restoring the directory is likely necessary.

debugger008 · June 10, 2019, 3:11pm

root@box:/home/user-data/ssl# cd ~/mailinabox/management
root@box:~/mailinabox/management# ./ssl_certificates.py 
-bash: ./ssl_certificates.py: /usr/local/lib/mailinabox/env/bin/python: bad interpreter: No such file or directory
root@box:~/mailinabox/management# ls -al
total 276
drwxr-xr-x 4 root root  4096 May 21 15:44 .
drwxr-xr-x 9 root root  4096 May 21 15:44 ..
-rw-r--r-- 1 root root  5199 Apr 26  2017 auth.py
-rwxr-xr-x 1 root root 21042 May 21 15:44 backup.py
-rw-r--r-- 1 root root  4449 Apr 26  2017 csr_country_codes.tsv
-rwxr-xr-x 1 root root 20948 May 21 15:44 daemon.py
-rwxr-xr-x 1 root root  1066 May 21 15:44 daily_tasks.sh
-rwxr-xr-x 1 root root 38832 May 21 15:44 dns_update.py
-rwxr-xr-x 1 root root  1370 May 21 15:44 email_administrator.py
-rwxr-xr-x 1 root root 21681 May 21 15:44 mailconfig.py
-rwxr-xr-x 1 root root 30441 May 21 15:44 mail_log.py
drwxr-xr-x 2 root root  4096 Jun 10 12:48 __pycache__
-rwxr-xr-x 1 root root 25313 May 21 15:44 ssl_certificates.py
-rwxr-xr-x 1 root root 44281 May 21 15:44 status_checks.py
drwxr-xr-x 2 root root  4096 May 21 15:44 templates
-rw-r--r-- 1 root root  6347 Apr 26  2017 utils.py
-rw-r--r-- 1 root root  9368 May 21 15:44 web_update.py

root@box:~/mailinabox/management# sudo ./ssl_certificates.py 
sudo: unable to execute ./ssl_certificates.py: No such file or directory

root@box:~/mailinabox/management# python3 ssl_certificates.py 
Provisioning TLS certificates for box.A_DOMAIN.SAFE.COM.
Traceback (most recent call last):
  File "ssl_certificates.py", line 660, in <module>
    provision_certificates_cmdline()
  File "ssl_certificates.py", line 372, in provision_certificates_cmdline
    status = provision_certificates(env, limit_domains=domains)
  File "ssl_certificates.py", line 348, in provision_certificates
    ret.extend(post_install_func(env))
  File "ssl_certificates.py", line 458, in post_install_func
    if cert and os.readlink(system_ssl_certificate) != cert['certificate']:
OSError: [Errno 22] Invalid argument: '/home/user-data/ssl/ssl_certificate.pem'


root@box:~/mailinabox/management# ls -al /home/user-data/ssl/ssl_certificate.pem
-rw------- 1 root root 3559 Jun 10 18:06 /home/user-data/ssl/ssl_certificate.pem

murgero · June 10, 2019, 3:12pm

The python scripts in miab have a special line 1 comment to specify what binary is called when you “./” Launch via bash. In this case, the SSL python script you mentioned uses a special python install that was configured directly during install of miab.

alento · June 10, 2019, 3:15pm

@murgero

Which would or would not mean that running the python scripts using the command line

python3 ssl_certificates.py

would be correct or not?

alento · June 10, 2019, 3:18pm

Why sudo? You are already root.

debugger008 · June 10, 2019, 3:20pm

just to double check
I’m not anymore sure of everything

anyway i guess that the trouble is

-bash: ./ssl_certificates.py: /usr/local/lib/mailinabox/env/bin/python: bad interpreter: No such file or directory

curiously also if try the standalone certbot everything goes bad

root@box:~# certbot certonly --standalone -d box.JUST_A_DOMAIN.com
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.31.0', 'console_scripts', 'certbot')()
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 560, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2648, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2302, in load
    return self.resolve()
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2308, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 14, in <module>
    from acme.magic_typing import Union  # pylint: disable=unused-import, no-name-in-module
ImportError: No module named 'acme.magic_typing'

alento · June 10, 2019, 3:22pm

Yeah, looks like exactly where you started.

alento · June 10, 2019, 3:27pm

@debugger008

If your goal is just to get things up and running until you can reinstall may I suggest obtaining a certificate using https://gethttpsforfree.com/ ?

You’d have to manually install it to the correct location in /home/user-data/ssl but it would be a temporary fix.

debugger008 · June 10, 2019, 3:33pm

50% has been done.
i manually created a certificate with certbot-auto ( so i get 3 month to migrate to ubuntu lts 18.04 )

i put it manually in /home/user-data/ and ssl is working fine!!

what’s wrong is /admin

uhm…

502 Bad Gateway

nginx

alento · June 10, 2019, 3:34pm

@debugger008

Did you reload nginx?

service nginx reload

debugger008 · June 10, 2019, 3:35pm

maybe i should just try again

 sudo mailinabox

while keeping fingers crossed… uhm… I’m checking inside logs first…

murgero · June 10, 2019, 3:36pm

The file on line one says:
#!/usr/local/lib/mailinabox/env/bin/python which tells bash to use /usr/local/lib/mailinabox/env/bin/python as the executable program for the script.

Try reinstalling MIAB, if that file is missing or corrupt nothing can get done.

With the MIAB installed Python (/usr/local/lib/mailinabox/env/bin/python) not existing or being corrupt, the admin panel will not work.

Looks like a reinstall is the way to go.

murgero · June 10, 2019, 3:41pm

Not 100% related to issue, but Just to note for new users looking for answers here on the forums, on 18.04 Systemd is the default service / system manager in 18.04. So if service is not an available command for you (likely it will be, but for the few out there that it isn’t) the command to restart nginx would be:

systemctl restart nginx

debugger008 · June 10, 2019, 3:43pm


Mail-in-a-Box Version:  v0.30

Updating system packages...
Installing system packages...
Initializing system random number generator...
Firewall is active and enabled on system startup
Installing nsd (DNS server)...
Installing Postfix (SMTP server)...
Installing Dovecot (IMAP server)...
Installing OpenDKIM/OpenDMARC...
Installing SpamAssassin...
Installing Nginx (web server)...

FAILED: apt-get -y -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confnew install nginx php7.0-cli php7.0-fpm
-----------------------------------------
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package php7.0-cli
E: Couldn't find any package by regex 'php7.0-cli'
E: Unable to locate package php7.0-fpm
E: Couldn't find any package by regex 'php7.0-fpm'
-----------------------------------------

rebooted

SSL is ok

/admin still had 502 bad gateway

murgero · June 10, 2019, 3:58pm

Can you give me the output of:

apt update && apt-cache search php7