Speed up backups or migration proccess

Finally, I have completed the migration… using the traditional method (creating a backup with the MIAB tool and then restoring it on the new machine).

Everything went well, but let me share what I did. What you mentioned about the database format was what worried me the most (and indeed, what you said about database versions makes a lot of sense). I couldn’t find out if the database versions vary between the installation on Ubuntu 18 and the installation on Ubuntu 22, so I preferred to play it safe.

The problem, as I feared, was the copying time. I tried to minimize the volume of what I was going to copy (deleted old emails from many accounts and even removed some that were no longer in use), but still, it took 12 hours for the backup and nearly the same amount of time for the restoration on the new machine. In total, it took 24 hours.

I know such a large-scale installation is not common, but as a suggestion (and I say this with the humility of understanding that this is an open-source project, and the developers will have to evaluate if this proposal is beneficial for the community or not), it might be good to have an option to perform a direct copy between two machines. Perhaps utilizing the existing script, but instead of compressing everything into multiple files, it would directly copy to the correct folders on the destination machine. This way, it would save time on file compression (copying is always faster than compressing), and there would be no need for decompression. Such an approach would accelerate migration by more than twice the time.

Regarding particularities of the installation, let me share some anecdotes of things that happened to me, so if anyone else encounters these issues, they know how to solve them.

The first issue I encountered was that immediately after restoring the backup, nothing was working (not even the system’s configuration page). I simply had to run the installation script again, and everything was resolved with a restart.

But immediately after, I noticed that no emails were showing up in the mailboxes of the accounts. I remembered this from previous migrations. After migrating, I always have to remember to reconfigure the compression of the emails (as I mentioned in my third message in this thread), because otherwise, the system will encounter compressed mails it can’t read. Once those changes were made, the system was functioning correctly.

A couple of days after completing the migration, the certificate of one of the domains was about to expire. However, when the system attempted to renew the domain, the renewal process failed and presented the following message:

Provisioning TLS certificates for correo.domain.com, autoconfig.correo.domain.com, autodiscover.correo.domain.com, mta-sts.correo.domain.com, [www.correo.domain.com](http://www.correo.domain.com).
error: correo.domain.com, autoconfig.correo.domain.com, autodiscover.correo.domain.com, mta-sts.correo.domain.com, [www.correo.domain.com:](http://www.correo.domain.com:)
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Missing command line flag or config entry for this setting:
Please choose an account
Choices: ['box.otherdomain.com@2018-10-24T07:09:56Z (1665)', 'box.otherdomain.com@2019-03-16T16:19:18Z (235f)', 'box.otherdomain.com@2018-10-24T08:18:53Z (7cfb)']
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

I tried to manually renew the certificate through the terminal, but I couldn’t manage to renew it. I found the solution on these same forums at this link:

https://discourse.mailinabox.email/t/missingcommandlineflag-msg-certbot-errors-missingcommandlineflag-missing-command-line-flag-or-config-entry-for-this-setting-please-choose-an-account/7281

I simply had to go to the folder /home/user-data/ssl/lets_encrypt/accounts/acme-v02.api.letsencrypt.org/directory and delete the older certificate folders (there were three folders, and I kept only the latest one). With this change made, I restarted the machine, launched the certificate renewal through the web again, and the certificate was successfully renewed without any issues.

Another thing that I always forget (yes… my memory is a mess) is that I must always modify fail2ban after any MIAB update (after running the installation script). Many people who use the server access it from the same location, so they access it from the same public IP address. Fail2ban interprets this as an attack, and after a while, the server gets blocked, preventing anyone from that IP from accessing it. To fix this, I just need to access these two files:

/etc/fail2ban/jail.d/mailinabox.conf
/etc/fail2ban/jail.conf

…and in the line “ignore IP,” I just need to put the public IP addresses from which we access the server.

Lastly, I had to make another small adjustment (I forgot to mention this in my third message as well). Since most messages are received through aliases from other domains, it’s common for some messages to end up in the SPAM folder incorrectly (because the mail server sending the message to MIAB is one of our servers, but the sender doesn’t use that server). With this setup, sometimes SPF and DMARC records get marked as incorrect by SpamAssassin, causing the message to be sent to the spam folder. This can be resolved by editing the miab_spf_dmark_.cf file located in the /etc/spamassassin directory and modifying the scores assigned when there are issues. I’ve left the file as follows:

# Evaluate DMARC Authentication-Results
header DMARC_PASS Authentication-Results =~ /box\.correo\.tabigal\.com; dmarc=pass/
describe DMARC_PASS DMARC check passed
score DMARC_PASS -0.1

header DMARC_NONE Authentication-Results =~ /box\.correo\.tabigal\.com; dmarc=none/
describe DMARC_NONE DMARC record not found
score DMARC_NONE 0.1

header DMARC_FAIL_NONE Authentication-Results =~ /box\.correo\.tabigal\.com; dmarc=fail \(p=none/
describe DMARC_FAIL_NONE DMARC check failed (p=none)
score DMARC_FAIL_NONE 1.0

header DMARC_FAIL_QUARANTINE Authentication-Results =~ /box\.correo\.tabigal\.com; dmarc=fail \(p=quarantine/
describe DMARC_FAIL_QUARANTINE DMARC check failed (p=quarantine)
score DMARC_FAIL_QUARANTINE 1.0

header DMARC_FAIL_REJECT Authentication-Results =~ /box\.correo\.tabigal\.com; dmarc=fail \(p=reject/
describe DMARC_FAIL_REJECT DMARC check failed (p=reject)
score DMARC_FAIL_REJECT 0.0

# Evaluate SPF Authentication-Results
header SPF_PASS Authentication-Results =~ /box\.correo\.tabigal\.com; spf=pass/
describe SPF_PASS SPF check passed
score SPF_PASS -0.1

header SPF_NONE Authentication-Results =~ /box\.correo\.tabigal\.com; spf=none/
describe SPF_NONE SPF record not found
score SPF_NONE 1.0

header SPF_FAIL Authentication-Results =~ /box\.correo\.tabigal\.com; spf=fail/
describe SPF_FAIL SPF check failed
score SPF_FAIL 1.0

And yes… I know you’re going to say that this will cause more actual SPAM to go to the inbox. I know. But over time, I’ve observed that a message that is genuinely SPAM is usually detected correctly by spamassassin through other means. At least, no one has complained about having an excessive volume of spam in the inbox (especially when the emails are already filtered and marked at the main domain on our hosting).
Perhaps someone may find some of the things I’ve done useful (sharing knowledge is always best).

1 Like