Our MiaB v57a VPS hosted by DigitalOcean has been reliably saving Duplicity backup files to a DigitalOcean Space. The Digital Ocean Spaces service provides AWS S3-compatible object storage. I built and updated a new Ubuntu 22.04 LTS 64-bit VPS (Droplet), installed MiaB v60 on the new VPS, and copied the secret_key.txt file from the former MiaB v57a instance to the new v60 instance. The next step of restoring from the Duplicity backup fails with error: “botocore.exceptions.ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden” The commands being run as the System user (root) are:
I am certain that the Access Key and Secret Access Key used were correct. I tried creating and testing a new Spaces access key, which made no difference. I consulted the duplicity stable version 1.0.1 man page https://duplicity.us/stable/duplicity.1.html and the Spaces API Reference https://docs.digitalocean.com/reference/api/spaces-api/. Based upon information gleaned from those pages and the Moving to a New Box section of the Mail-in-a-Box Maintenance Guide, I tried several variations of the source URL:
Neither worked nor provided any useful additional information. Other Duplicity commands such as “collection-status” also fail similarly. If anyone has successfully restored MiaB from DigitalOcean Spaces, I would much appreciate knowing what worked.
Since no one has replied with a report of a successful restore to MiaB v60 directly from a DigitalOcean Space, I tried manually copying new full Duplicity backup files from the Space to a directory on the new MiaB v60 instance, and restoring from there. This also failed with error: “Local and Remote metadata are synchronized, no sync needed.” This error makes little sense since the MiaB v60 instance was fresh. This error left the Duplicity backup to S3-compatible storage useless for restoring MiaB.
To work around this error, I reverted the VPS (Droplet) to MiaB v57a from a previously created Snapshot, ran a new full Duplicity backup to the default local storage on the MiaB v57a VPS, downloaded the /home/user-data/backup/encrypted/ directory including the new Duplicity files, uploaded the “encrypted” directory with the contained files to the new MiaB v60 instance, and restored from there using the file:// prefix. That worked properly with no problems.
This experience indicates that automatic Duplicity backup of a MiaB instance to S3-compatible storage can be useless for restoring MiaB even though the automatic backups run properly for for a long period of time. This makes it very advisable to test Duplicity backups to S3e-compatible storage. Running nondestructive Duplicity commands such as “collection-status” seem a good way to begin testing. See the duplicity stable version 1.0.1 man page for more information.
I just now noticed that DigitalOcean has reported issues involving Spaces that might have prevented the restore operation from accessing the Duplicity files. For example, see: DigitalOcean Status - App Platform Deployments / Container Registry / Spaces. These issues might be unrelated, yet reinforce the need for testing the backup.
I had exactly the same experience upgrading from v57a on Ubuntu 18.04 to v60.1 on Ubuntu 22.04. When I tried to restore from my AWS S3 backup (so canonical S3, not DigitalOcean Spaces or another S3-compatible service):
$ export AWS_ACCESS_KEY_ID=[my key]
$ export AWS_SECRET_ACCESS_KEY="[my secret key]"
$ export PASSPHRASE="$(cat secret_key.v5x.txt)"
$ sudo -E duplicity restore --force s3://s3.amazonaws.com/my-bucket/backups-v5x /home/user-data/
Attempt of list Nr. 1 failed. ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden
Attempt of list Nr. 2 failed. AttributeError: 'NoneType' object has no attribute 'objects'
Attempt of list Nr. 3 failed. AttributeError: 'NoneType' object has no attribute 'objects'
Attempt of list Nr. 4 failed. AttributeError: 'NoneType' object has no attribute 'objects'
Giving up after 5 attempts. AttributeError: 'NoneType' object has no attribute 'objects'
(I didn’t try the local recovery workaround steps, I just restored my 18.04 snapshot and called it a day.)