Using S3 to store all mailboxes data

Improvements
1

Is it possible to use S3 to store all mailboxes data. Perhaps using S3QL?

Can someone please help me on finding ifo on how to move the mailboxes to a custom directory?

Thanks!
Mauro W

2

I don’t think there is such option by default. But there is a backup to S3 option. If you go to box.yourdomain.com/admin and then to system -> backup status, you will see the option to backup to S3 the mailbox data.

3

Thanks Jack, but my issue is that my users have large email boxes and I would like to have an “unlimited” mail box via S3.
I saw the backup option but it does not helps me on making a large mailbox for my users.

4

There’s no reason you couldn’t mount the entire MIAB mailbox dir using S3FS. Just rename /home/user-data/mail/mailboxes to something like /home/user-data/mail/mailboxes-old, then

# service postfix stop
# mkdir /home/user-data/mail/mailboxes
# s3fs bucketname /home/user-data/mail/mailboxes
# mv /home/user-data/mail/mailboxes-old/* /home/user-data/mail/mailboxes/
# chown -R mail.mail /home/user-data/mail/mailboxes
# service postfix start

This will upload all of your mail to the S3 bucket of your choosing and store new messages there as well. You’ll need to add the S3 mountpoint to your /etc/fstab file as well.

Keep an eye on the price. If you get lots of large email or users retrieve lots of email, it could get expensive fast. It also opens you to abuse, because if you get a sudden torrent of email, you’re paying for both transmitting and storing every unwanted message.

5

@JoshData do you think this is update Safe ? If yes, why not offer this in the Advanced Guide?

6

If yes, why not offer this in the Advanced Guide?

When someone demonstrates it’s safe, then we can consider recommending it to others.

Adding network latency and glitches to your primary email store seems like a bad idea to me, fwiw.

7

Agreed. This seems like a terrible idea, and there are a zillion better ways to get more space on the host.

8

So, after playing around with S3 and mailboxes, here are my findings:

What I do is to install some software that will make S3 look like a drive to linux. After this, I move the mailbox folder to the S3 drive and create a symlink at the original place so it is transparent to mailbox.

  1. Using S3QL (https://bitbucket.org/nikratio/s3ql/overview) - Don’t work with Dovecot. I have a lot of permission issues after moving and creating the symlink. Unsusable.

  2. Using s3fs (https://github.com/s3fs-fuse/s3fs-fuse) - Works, but it is slooooow as hell. It takes almost 1 minute to open a mailbox in roundcube webmail.

  3. Using objectivefs (https://objectivefs.com/) - Works perfectly. It is fast - maybe because the cache is doing its part and there is absolutely no issues with Dovecot or Owncloud. I use it also to store documents from Owncloud and it was really good. The only downside is that they objectivefs is not free. They were charging US$ 14.00 per month (that I think is acceptable) and now they have a new price and are charging US$ 29.00 per month that is a little bit expensive.

I will test other s3 drive emulators and post my findings here.
The best part on doing this, is that there is no need to change any settings from mailinabox, since I am moving the mailbox and then using a symlink to connect it.

Now about the comment from mokiejovis. I have a customer that have about 250 mailboxes and some of them are with > 10gb of mail. I could not find any better and cheaper way to get more space on AWS, Digital Ocean, Azure, etc. If you know a cloud provider that has servers with a couple Terabytes for a low price, please let us know.

9

[quote=“dwidman, post:8, topic:1128”]
I have a customer that have about 250 mailboxes and some of them are with > 10gb of mail.[/quote]
The thing to consider is the saying “pay now, or pay later,” because you’re offloading the cost of local storage to transfer. A basic box with DigitalOcean gives you 1TB per month of transfer. You’re easily going to exceed that (and pay for the excess) if you’re hosting all of your data offsite.

Obviously this doesn’t exist. You pay for what you get. If you need the space, you’re going to end up paying for it one way or the other.

A better choice for your 250-user customer would be to pay for a static IP from their ISP and run MIAB from a local VM they control, which allows them to have all the disk they want.

10

I agree. I was thinking of using AWS so the transfer between S3 buckets and the EC2 instance will be free. All input and output will be in charge of the EC2.
The actual price of S3 gb is about US$ 0.03 per GB per month. So 1TB would cost us about US$ 30.00. Not too expensive.
So a box on a EC2 t2.medium @ US$ 40.00/month + 3 TB of S3 storage @ US$ 90.00/month will cost us something on the average of US$ 130.00/month.
Please correct me if I’m wrong.

I also thought about having a local device, but a decent box with mirrored raid 3TB hard drives and a decent link would cost a lot initially and not give us the reliability that we can have at AWS.

11

Don’t forget the monthly cost of objectivefs (either $19 or $39 per month, I’m not sure which … are you being grandfathered on the $19/mo plan? Or would your bill go up?) as well as the increased cost of transfer over 2TB on DigitalOcean (I checked; the box with recommended specs for MIAB gets 2TB of free transfer per month and additional transfer is billed).

So you’re talking a $130/mo PLUS the costs for objectivefs and DigitalOcean. And if you get hit with a torrent of spam? You stand to get an eye-popping bill.

12

This topic was automatically closed after 61 days. New replies are no longer allowed.