Errors running status_checks.py in vers .54

Maintenance
1

I have been having issues again with the system status checks page in the browser. When I tried to run it manually, I was getting a syntax error message for line 480. I ran “sudo mailinabox” to see if that would fix things. After the install script was completed I got this message:

Your Mail-in-a-Box is running.

Please log in to the control panel for further instructions at:

management/status_checks.py: line 8: import: command not found
management/status_checks.py: line 9: import: command not found
management/status_checks.py: line 11: import: command not found
management/status_checks.py: line 12: import: command not found
management/status_checks.py: line 13: import: command not found
management/status_checks.py: line 14: import: command not found
management/status_checks.py: line 15: import: command not found
from: can't read /var/mail/dns_update
from: can't read /var/mail/web_update
from: can't read /var/mail/ssl_certificates
from: can't read /var/mail/mailconfig
from: can't read /var/mail/utils
management/status_checks.py: line 24: syntax error near unexpected token `('
management/status_checks.py: line 24: `def get_services():'
https://159.65.182.128/admin

You will be alerted that the website has an invalid certificate. Check that
the certificate fingerprint matches:

FE:1D:90:FC:08:A8:47:5D:00:56:0E:D8:EC:FB:E1:10:A5:FC:4B:44:A2:A7:AC:A6:14:C9:1C:A1:7E:70:88:C6

Then you can confirm the security exception and continue.

Is this a known issue?

2

That doesn’t look like anything that makes any sense. I can get to your admin area just fine though, and email seems to be flowing OK as well.

Have you removed any domains completely from your MiaB? (potentially not related, but checking a hunch)

3

No domains, just a subdomain recently. The error message for line 480 of status_checks.py is:

  File "status_checks.py", line 480
    mail_aliases = dict([(address, receivers) for address, receivers, *_ in get_mail_aliases(env)])
                                                                      ^
SyntaxError: invalid syntax

That’s with the utf-8 encoding line included in the file. How were you able to get to the admin area?

4

Check the file /home/user-data/dns/custom.yaml for any references to the deleted subdomain.

Via the IP address in your OP.

5

I checked my custom.yaml file and I couldn’t find any stray domains or subdomains. Just to make sure, what would happen if I were to rename or delete that file and run mailinabox again? Would that rewrite the file with the existing information?

6

Did you use a custom STORAGE_ROOT when you set up your MiaB server?

I ask as these make absolutely no sense…

MiaB does not use those directories/files.

7

I tried redoing “sudo mailinabox” and no longer get those funny errors. However, I am back to getting the syntax error on line 480:

 File "status_checks.py", line 480
    mail_aliases = dict([(address, receivers) for address, receivers, *_ in get_mail_aliases(env)])
                                                                      ^
SyntaxError: invalid syntax

I have no idea why I keep having this issue! And no, I didn’t use a custom anything when I originally installed MIAB. The issue with the “Something went wrong” message loading the System Status page has been an off-again on-again thing with the last two updates at least.

8

Just a hunch, and might lead to nowhere, but can you show the output of the following?

ls /usr/local/lib/mailinabox/env/bin/python*

and

cat /usr/local/bin/mailinabox
9 
/usr/local/lib/mailinabox/env/bin/python         /usr/local/lib/mailinabox/env/bin/python3
/usr/local/lib/mailinabox/env/bin/python-config  /usr/local/lib/mailinabox/env/bin/python3.6

#!/bin/bash
cd /home/ubuntu/mailinabox
source setup/start.sh
10

You mentioned issues with utf-8 earlier. Are your locales correctly set up?

sudo apt install locales
sudo dpkg-reconfigure locales

Then select en_US.UTF-8 and set it as default.

11

Did as you suggested. I still get a syntax error on line 480 of status_checks.py manually and the System Status Checks page still returns “Something went wrong. Sorry.”

12

Odd. Was the file somehow accidentally modified? What do you get when running:

cd /home/ubuntu/mailinabox
git status
13

git status returns the following:

HEAD detached at v0.54
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        management/sudo

nothing added to commit but untracked files present (use "git add" to track)

I have made sure that the default encoding for .py files is UTF-8, but I am still getting this error with status_checks.py:

File "status_checks.py", line 420
SyntaxError: Non-ASCII character '\xe2' in file status_checks.py on line 420, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Should I add back the following line to the top of the file?

# -*- coding: utf-8 -*-
14

You probably shouldn’t have to change any files, but the error suggests that the file is now not interpreted as utf-8. How did you change the default encoding?
The management/sudo file should not be there. I wonder if it does any harm. Can you show the contents?

(my hunch was about the *_ being something new in python 3, if the interpreter used was pyhton 2, such an error might be expected. Also, the errors you had earlier (import not found etc) point at some python interpreter issue)

15

I followed @davness suggestion by running

sudo dpkg-reconfigure locales

I simply selected the first option (install all) then set en_US.UTF-8 as the default encoding. However, that didn’t correct the encoding errors or the syntax errors.

Contents of management/sudo:

#!/bin/bash
#########################################################
# This script is intended to be run like this:
#
#   curl https://mailinabox.email/setup.sh | sudo bash
#
#########################################################

if [ -z "$TAG" ]; then
	# If a version to install isn't explicitly given as an environment
	# variable, then install the latest version. But the latest version
	# depends on the operating system. Existing Ubuntu 14.04 users need
	# to be able to upgrade to the latest version supporting Ubuntu 14.04,
	# in part because an upgrade is required before jumping to Ubuntu 18.04.
	# New users on Ubuntu 18.04 need to get the latest version number too.
	#
	# Also, the system status checks read this script for TAG = (without the
	# space, but if we put it in a comment it would confuse the status checks!)
	# to get the latest version, so the first such line must be the one that we
	# want to display in status checks.
	if [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/18\.04\.[0-9]/18.04/' `" == "Ubuntu 18.04 LTS" ]; then
		# This machine is running Ubuntu 18.04.
		TAG=v0.42b

	elif [ "`lsb_release -d | sed 's/.*:\s*//' | sed 's/14\.04\.[0-9]/14.04/' `" == "Ubuntu 14.04 LTS" ]; then
		# This machine is running Ubuntu 14.04.
		echo "You are installing the last version of Mail-in-a-Box that will"
		echo "support Ubuntu 14.04. If this is a new installation of Mail-in-a-Box,"
		echo "stop now and switch to a machine running Ubuntu 18.04. If you are"
		echo "upgrading an existing Mail-in-a-Box --- great. After upgrading this"
		echo "box, please visit https://mailinabox.email for notes on how to upgrade"
		echo "to Ubuntu 18.04."
		echo ""
		TAG=v0.30

	else
		echo "This script must be run on a system running Ubuntu 18.04 or Ubuntu 14.04."
		exit
	fi
fi

# Are we running as root?
if [[ $EUID -ne 0 ]]; then
	echo "This script must be run as root. Did you leave out sudo?"
	exit
fi

# Clone the Mail-in-a-Box repository if it doesn't exist.
if [ ! -d $HOME/mailinabox ]; then
	if [ ! -f /usr/bin/git ]; then
		echo Installing git . . .
		apt-get -q -q update
		DEBIAN_FRONTEND=noninteractive apt-get -q -q install -y git < /dev/null
		echo
	fi

	echo Downloading Mail-in-a-Box $TAG. . .
	git clone \
		-b $TAG --depth 1 \
		https://github.com/mail-in-a-box/mailinabox \
		$HOME/mailinabox \
		< /dev/null 2> /dev/null

	echo
fi

# Change directory to it.
cd $HOME/mailinabox

# Update it.
if [ "$TAG" != `git describe` ]; then
	echo Updating Mail-in-a-Box to $TAG . . .
	git fetch --depth 1 --force --prune origin tag $TAG
	if ! git checkout -q $TAG; then
		echo "Update failed. Did you modify something in `pwd`?"
		exit
	fi
	echo
fi

# Start setup script.
setup/start.sh
16

This morning I deleted that sudo file then ran apt upgrade/apt update and then sudo mailinabox again. THIS time for some reason, I got the following message when trying to install Nextcloud:

Installing Nextcloud (contacts/calendar)...
Cannot write into "config" directory!
This can usually be fixed by giving the webserver write access to the config directory

Or, if you prefer to keep config.php file read only, set the option "config_is_read_only" to true in it.
See https://docs.nextcloud.com/server/20/go.php?to=admin-config
Cannot write into "config" directory!
This can usually be fixed by giving the webserver write access to the config directory

Or, if you prefer to keep config.php file read only, set the option "config_is_read_only" to true in it.
See https://docs.nextcloud.com/server/20/go.php?to=admin-config

Everything else appears to have installed correctly. After the updated DNS message, the install script completes with the following:

-----------------------------------------------

Your Mail-in-a-Box is running.

Please log in to the control panel for further instructions at:

https://emmons.uniquelyyoursmail.com/admin

If you have a DNS problem put the box's IP address in the URL
(https://159.65.182.128/admin) but then check the TLS fingerprint:
FE:1D:90:FC:08:A8:47:5D:00:56:0E:D8:EC:FB:E1:10:A5:FC:4B:44:A2:A7:AC:A6:14:C9:1C:A1:7E:70:88:C6

However, I still get the “Something went wrong. Sorry.” message when I try to load the System Status Checks page. And if I run status_checks.py manually I get the Line 420 syntax error about no encoding. Stumped.

17

Who is your VPS provider?

18

I use Digital Ocean droplets.

19

For the nextcloud config issue you can check if the user www-data has rwx access to the /usr/local/lib/owncloud/config folder.

For the syntax error issue, unfortunately I’m out of my depth. Anything from me is just googling stuff, trying to collect information which in turn might trigger to check something else.
Following Python 3, UTF-8 and Locale I would try:

cd /usr/local/lib/mailinabox/env
python

This will open a python shell where you can:

import sys
sys.getfilesystemencoding()
sys.getdefaultencoding()
exit()

But maybe I’m just leading towards a wild goose chase. Perhaps it’s better to take a step back. You already mentioned this is a Digital Ocean droplet. This is on a fresh ubuntu 18.04 install? Or was the ubuntu upgraded? Same question for mailinabox. Fresh install or upgraded? Anything interesting you might remark on the history of the box?

20

When MAIB v.0.40 was released, I created a fresh droplet with a fresh install of Ubuntu 18.04 and then a fresh install of MAIB 0.40. I followed the instructions for moving all the MAIB data over to the new droplet. Everything (including the system status page) worked beautifully. The issues I have now with status_checks.py (and a couple of other files) throwing syntax errors and the System Status Checks page not loading began when I upgraded to v. 0.53. These same issues continued through v. 0.53a and 0.54.

I checked the /usr/local/lib/owncloud/config folder. I made sure everything was writable for www-data but still get the “can’t write” message. Since none of my users use Nextcloud, I’m going to ignore that for the moment.

As for the python encoding, THAT was interesting! I ran the script in the python shell. Here is what it gave me: the file system encoding is ANSI_X3.4-1968 and the default encoding is “ascii.” The default version of Python appears to be 2.7.17. I guess I’ll need to figure out how to set the encoding to URF-8?