Speed up backups or migration proccess

Perfect. That reassures me. It’s a feature that I know wasn’t ready in the migration and that I consider very necessary.

No. Let me explain a little how we have set up the MIAB server. I get the impression that you may be interested in the way we are using it and how we have it set up.

I’ve never liked hosting things on external “clouds.” I always preferred “on-premise” installations, meaning with our own hardware. Our first MIAB server was a small Pentium IV, but gradually, we made it grow. First, we upgraded it to a better machine, and later, when we virtualized some servers with VMWare ESXi, we included it as another virtual machine. Now, it coexists with a few other servers on the same machine. Currently, we perform backups using Veeam Backup to a FreeNAS that is connected to our network (as I told you, we don’t like “the cloud” and we want everything local). We have the entire machine fully backed up, so there is no need to use the mail-in-a-box backup tool (besides, the initial and incremental backups are significantly faster with Veeam).

Currently, we have assigned 4 processor cores, 4 gigabytes of memory, and 700 GB of hard disk space to the MIAB server. There are 184 email accounts on it, spread across 10 different domains. In reality, they are third-level subdomains of the type “correo.domain.com.” We use second-level domains for the main email, which are hosted by our regular email provider. The issue is that the space that our email provider offer is limited. It’s not a problem for computers using “POP” accounts, but people using mobile devices need IMAP accounts, so we use a workaround explained on this website:

Basically, what we do in these cases is the following: for the person who needs IMAP email on their mobile device, we forward a copy of their received emails to the account hosted on mail-in-a-box. This way, incoming emails can be viewed on the mobile device. As for outgoing emails, we configure the SMTP settings on the mobile device to use our regular email provider.

The problem with this method is that sometimes a user requests full synchronization of incoming and outgoing emails. In such cases, what I do is create an “alias” on our domain provider’s server, which is a “virtual” account that directly forwards received emails to the mail-in-a-box. I configure both the computer and the mobile device with IMAP settings pointing to the mail-in-a-box server, and for sending emails, I set up a generic account that allows sending emails through our regular provider. Essentially, the third-level domains on mail-in-a-box remain hidden from view, and the user is unaware that they exist in the background.

This may seem tedious and crazy. You’re probably thinking, “Why not let MIAB handle all the emails directly?” Well, it’s because of the issues with blacklists, Microsoft bans, and similar problems. Matters like the ones explained on this website:

With this method, I let my domain provider handle those issues, and we only have to ensure that the communication between the domain provider and our server is flawless. We never use MIAB to send emails; we only use it to receive emails from our domain provider, and always we use it via IMAP. This way, we can focus on seamless email reception while leaving the complexities of deliverability and blacklisting to the domain provider to handle.

Another issue we face is the disk space: It might seem a lot of disk space, but if we don’t put a stop to it, it would be around five times more (I’m not exaggerating). We receive enormous amounts of email, some of which are very large in size. So, I’ve been forced to make a small “hack” to MIAB and enable email compression. Well… since this is becoming lengthy, I’ll also explain how I do it:

  1. Change the file /etc/dovecot/dovecot.conf to add this lines (I added all of them at top of the file)

mail_plugins = $mail_plugins zlib

plugin {
zlib_save_level = 6 # 1…9; default is 6
zlib_save = gz # or bz2, xz or lz4
}

  1. change the file /etc/dovecot/conf.d/20-pop3.conf . You have to find this line

mail_plugins = $mail_plugins antispam

And change it to this line

mail_plugins = $mail_plugins antispam zlib

  1. Edit the file /etc/dovecot/conf.d/20-imap.conf

In the file you have to put the following:

protocol imap {
mail_plugins = $mail_plugins antispam imap_zlib
}

  1. Edit the file /etc/dovecot/conf.d/20-lmtp.conf
    and put:
    protocol lmtp {
    mail_plugins = $mail_plugins sieve zlib
    }
    And then restart the server

But even so, I have some “challenging” users who have an extremely high volume of emails. Since most of them fetch their emails via POP and keep a local copy on their computers, and they don’t need to access emails older than a year, what I do is run a script periodically (launched with cronab) that deletes emails older than a year for these users. Here’s how I do it:

cd /home/user-data/mail/mailboxes/correo.mydomain.com/mailoftheuser
find * -mtime +365 -exec rm {} \;

Now we are planning to take advantage of setting up a new server with Proxmox to host MIAB there, allocating it more resources and disk space. However, the issue is that migrating “only” those nearly 600 gigabytes is proving to be incredibly slow (we have some virtual machines that occupy much more, and managing them is much faster).

I assume that if I move the content of the “/home/user-data/” folder from one server to another, restore the secret key and enable email compression again, I should have everything working, but I would rather make sure before I start doing things that will probably take a long time. I have previously performed updates and migrated machines with MIAB (from this same installation with everything I have commented) and everything has always gone well, but I have always used the official MIAB backup tool for it. And back then the machine didn’t weigh as much as it does now. That’s why my question about whether there is any other method I can use for the migration.

By the way… I am very clear that MIAB is focused on personal use, but I have to be honest: it works wonderfully and does not fail at all when using it in this way for larger projects.

1 Like