I’ve managed to get my backup to Backblaze B2 storage and I thought I would share it here…
Shamelessly edited the script found over at autoize > original one found here, which is intended for Nextcloud.
Made some adjustments and got it working for Mailinabox.
#!/bin/sh
Path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
# EDITED by Mellow for use with Mailinabox
# Originally a NextCloud to BackBlaze B2 Backup Script
# Original Author: Autoize (autoize.com)
# This script creates an incremental backup of your Mailinabox instance at BackBlaze's off-site location.
# BackBlaze B2 is an object storage service that is much less expensive than using Amazon S3 for the same purpose, with similar versioning and lifecycle management features.
# Uploads are free, and storage costs only $0.005/GB/month compared to S3's $0.022/GB/month.
# Requirements
# - BackBlaze B2 account (10 GB Free) - Create one at https://www.backblaze.com/b2/sign-up.html
# - BackBlaze B2 CLI installed from PyPI - sudo pip3 install b2
# Instructions
# 1. Create a bucket and obtain your Account ID and Application Key from your B2 account.
# 2. Authenticate your CLI using the b2 authorize_account command.
# 4. Save this script to a safe directory such as /srv/backupToB2.sh and make it executable with the following command.
# sudo chmod +x backupToB2.sh
# 5. This script must be run as root. To run a backup now:
# sudo ./backupToB2.sh
# 6. Set up a cron job to run this backup on a predefined schedule (optional).
# sudo crontab -u root -e
# Add the following line to the crontab to conduct a weekly backup every Saturday at 2:00am.
# 0 2 * * sat /srv/backupToB2.sh > /srv/backupToB2.log 2>&1
# Save, quit and check that the crontab has been installed using the following command.
# sudo crontab -u root -l
# Name of BackBlaze B2 Bucket
b2_bucket='yourbucketname'
# Path to box backupdata directory
data_dir='/home/user-data/backup/encrypted'
# Check if running as root
if [ "$(id -u)" != "0" ]; then
echo "This script must be run as root" 1>&2
exit 1
fi
echo 'Started'
date +'%a %b %e %H:%M:%S %Z %Y'
# Sync data to B2
b2 sync $data_dir b2://$b2_bucket$data_dir
date +'%a %b %e %H:%M:%S %Z %Y'
echo 'Finished'
Been meaning to enable this on my box while we wait for maybe having it implemented in vanilla MiaB. This is great to share a detailed step by step manual setup and a script for it!
I would appreciate having B2 support as well. I currently run a cron job that runs independent of MiaB and syncs MiaB’s backups to a B2 bucket using rclone. This has worked well for 6 months, but having B2 as an option in MiaB would be the preferred way to go.
I tried that a few weeks back and it did not work. It seems like some s3 api calls that duplicity uses were missing/different enough in b2 that it did not work properly. Might have changed already though
I looked into why the S3 option of MiaB wouldn’t work with B2 back when B2 offered S3 compatibility. From what I could see, the API calls were the same, but the endpoint URLs to B2 nodes are slightly different that S3.
https://s3.us-west-001.backblazeb2.com/bucketname
vs https://s3.us-west-1.amazonaws.com/bucketname
The scripts MiaB uses hardcodes parts of the AWS endpoint, the “us-west-1” part, and thus it can’t talk to B2, simply because of the missing ‘00’.
yes that is a problem. We would require post-processing in order to detect whether b2 or amz is used in the backend. This came up in the PR as well and I agree with the author’s stance that a separate b2 option would be preferable both from a usabilty and code quality point of view.
There is another problem however:
the duplicity version used in miab relies on boto2 s3 bindings which do not support b2’s s3-compatible API due to it requiring the v4 signature algorithm.
So at the minimum, we would have to upgrade duplicity so that we can use boto3. As mentioned, I tried all that on my dev box a few weeks back and still ran into trouble when verifying backups.
Ok, so update from me: I use the PR since I made it and I also run into verification problems. However, I feel that this is a problem with the verification process or duplicity, but not with the implementation.
If I look into the logs I see for example:
“Difference found: File mail/mailboxes/hilko.eu/mail/.Mailinabox has mtime Thu Nov 19 23:21:00 2020, expected Wed Nov 18 03:46:08 2020”
For me that states that the version on backblaze is newer than the local version. So this is no problem for me (since newer should be better) and I think the reason is probably that we are comparing the backup with an outdated cache. Has anyone tried running the verification command on an amazon aws backup? I would expect the same behaviour since I did (at least to my knowledge) no changes to the backup procedure, i.e. which files are updates, how the cache is stored etc.
Just had a look again. There is an older issue with exactly the same problem (backup data is newer than the data compared against):
As mentioned earlier, i guess this has something to do with the way we verify our backup or how we create the cache directory against we which compare our backup.
@fspoettel are you using AWS Backup on your miab production system? Would you mind running “sudo .management/backup.py --verify”? Do you get verification errors or does it run fine?
