500 Internal Server Errors

Everything has been working for a couple of years. I started having issues ever since I updated to v70. I’d get 500 errors when trying to add new users, add custom DNS and when accessing the External DNS page. I also noticed some DNS zone files were missing while others would not display.

After a few hours of trying to figure out what was causing the issues, I decided to just re-install Ubuntu and start all over. After doing so, I get the following error right out of the gate:

Okay. I'm about to set up jeff@xxxxx.com for you. This account will also
have access to the box's control panel.
password:
 (again):
<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

So now, with a fresh install, I get a 500 error during setup, when it’s time to add my admin account credentials for the web UI.

I don’t know where to begin troubleshooting this. Any ideas?

root@mail:~# systemctl status nsd.service
× nsd.service - Name Server Daemon
     Loaded: loaded (/lib/systemd/system/nsd.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sun 2024-10-20 19:59:49 EDT; 14s ago
       Docs: man:nsd(8)
    Process: 4350 ExecStart=/usr/sbin/nsd -d -P  (code=exited, status=1/FAILURE)
   Main PID: 4350 (code=exited, status=1/FAILURE)
        CPU: 90ms

Oct 20 19:59:49 mail.xxx.com systemd[1]: nsd.service: Main process exited, code=exited, status=1/FAILURE
Oct 20 19:59:49 mail.xxx.com systemd[1]: nsd.service: Failed with result 'exit-code'.
Oct 20 19:59:49 mail.xxx.com systemd[1]: Failed to start Name Server Daemon.
Oct 20 19:59:49 mail.xxx.com systemd[1]: nsd.service: Scheduled restart job, restart counter is at 5.
Oct 20 19:59:49 mail.xxx.com systemd[1]: Stopped Name Server Daemon.
Oct 20 19:59:49 mail.xxx.com systemd[1]: nsd.service: Start request repeated too quickly.
Oct 20 19:59:49 mail.xxx.com systemd[1]: nsd.service: Failed with result 'exit-code'.
Oct 20 19:59:49 mail.xxx.com systemd[1]: Failed to start Name Server Daemon.
root@mail:~# journalctl -xeu nsd.service
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit nsd.service has begun execution.
░░
░░ The job identifier is 2236.
Oct 20 20:05:47 mail.xxx.com nsd[5358]: [2024-10-20 20:05:47.361] nsd[5358]: error: Cannot open /var/log/nsd.log for appending (Read-only file system), logging to stderr
Oct 20 20:05:47 mail.xxx.com nsd[5358]: [2024-10-20 20:05:47.363] nsd[5358]: notice: nsd starting (NSD 4.3.9)
Oct 20 20:05:47 mail.xxx.com nsd[5358]: [2024-10-20 20:05:47.367] nsd[5358]: error: can't bind tcp socket: Cannot assign requested address
Oct 20 20:05:47 mail.xxx.com nsd[5358]: [2024-10-20 20:05:47.368] nsd[5358]: error: cannot open control interface ::1 8952
Oct 20 20:05:47 mail.xxx.com nsd[5358]: [2024-10-20 20:05:47.369] nsd[5358]: error: could not open remote control port
Oct 20 20:05:47 mail.xxx.com nsd[5358]: [2024-10-20 20:05:47.370] nsd[5358]: error: could not perform remote control setup
Oct 20 20:05:47 mail.xxx.com systemd[1]: nsd.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ An ExecStart= process belonging to unit nsd.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Oct 20 20:05:47 mail.xxx.com systemd[1]: nsd.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit nsd.service has entered the 'failed' state with result 'exit-code'.
Oct 20 20:05:47 mail.xxx.com systemd[1]: Failed to start Name Server Daemon.
░░ Subject: A start job for unit nsd.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit nsd.service has finished with a failure.
░░
░░ The job identifier is 2236 and the job result is failed.
Oct 20 20:05:47 mail.xxx.com systemd[1]: nsd.service: Scheduled restart job, restart counter is at 5.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ Automatic restarting of the unit nsd.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Oct 20 20:05:47 mail.xxx.com systemd[1]: Stopped Name Server Daemon.
░░ Subject: A stop job for unit nsd.service has finished
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A stop job for unit nsd.service has finished.
░░
░░ The job identifier is 2319 and the job result is done.
Oct 20 20:05:47 mail.xxx.com systemd[1]: nsd.service: Start request repeated too quickly.
Oct 20 20:05:47 mail.xxx.com systemd[1]: nsd.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit nsd.service has entered the 'failed' state with result 'exit-code'.
Oct 20 20:05:47 mail.xxx.com systemd[1]: Failed to start Name Server Daemon.
░░ Subject: A start job for unit nsd.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit nsd.service has finished with a failure.
░░
░░ The job identifier is 2319 and the job result is failed.
lines 2911-2974/2974 (END)

root@mail:~# ufw status
Status: active

To                         Action      From
--                         ------      ----
22/tcp                     LIMIT       Anywhere
53                         ALLOW       Anywhere
25/tcp                     ALLOW       Anywhere
465/tcp                    ALLOW       Anywhere
587/tcp                    ALLOW       Anywhere
993/tcp                    ALLOW       Anywhere
995/tcp                    ALLOW       Anywhere
4190/tcp                   ALLOW       Anywhere
80/tcp                     ALLOW       Anywhere
443                        ALLOW       Anywhere
22/tcp (v6)                LIMIT       Anywhere (v6)
53 (v6)                    ALLOW       Anywhere (v6)
25/tcp (v6)                ALLOW       Anywhere (v6)
465/tcp (v6)               ALLOW       Anywhere (v6)
587/tcp (v6)               ALLOW       Anywhere (v6)
993/tcp (v6)               ALLOW       Anywhere (v6)
995/tcp (v6)               ALLOW       Anywhere (v6)
4190/tcp (v6)              ALLOW       Anywhere (v6)
80/tcp (v6)                ALLOW       Anywhere (v6)
443 (v6)                   ALLOW       Anywhere (v6)

can’t bind tcp socket: Cannot assign requested address

This is usually due to the port (53 probably) already being used by a different program, like systemd-networkd. Run ss -lptun as root to see the programs listening on ports. For port 53 it should be bind9’s named on 127.0.0.1 and nsd on the external ip.

Make sure you are on Ubuntu 22.04.

Thank you!

However, after more testing, I believe the issue is caused by not having an IPv6 address.

I get nothing back when I run ip -6 addr

If I run the following commands…

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=0
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=0
sudo service nsd start

…the NSD service starts but I run into the same problem when the server is rebooted.

I never manually disabled IPv6 so I don’t know why I have to run those commands to get a v6 IP address. Where should I look to see why I’m not getting a v6 IP address until I run those commands?

These settings through sysctl -w don’t persist, that’d be in /etc/sysctl.conf (and others, see man sysctl.conf). You should find something there disabling it since the default is on.

Maybe the miab scripts don’t configure bind9/nsd to be separate properly if ipv6 is disabled, they should be isolated by the former listening on localhost and the latter on the exposed ip.

Thanks, again.

It seems the Ubuntu 22.04 template the VPS provider uses has IPv6 disabled by default.

I had to change all three from 1 to 0

net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0

I still get the following error:

and MTA-STS policy is missing: STSFetchResult.NONE

Check if this works from an external machine: dig @box.example.com mta-sts.box.example.com TXT - replace the domain ofc

I ended running the curl -s https://mailinabox.email/setup.sh | sudo -E bash command and that seems to have solved the MTA-STS policy is missing: STSFetchResult.NONE issue.

However, I’m still getting Something went wrong, sorry. error when accessing the MIAB External DNS page. I also get 500 Internal Server Errors when adding users but it seems they are being added. I get the same results when I let MIAB handle the DNS on the box and when I use external DNS on a third-party provider. I have, of course, allowed enough time for the DNS to propagate when switching back and forth between internal and external DNS.

The first thing to check with http 500 is the nginx error log at /var/log/nginx/error.log for more detail