Problem with duplicity v57a and v60: boto3

#1

I’ve been trying to upgrade to v60. Using a new Ubuntu 20.04, I failed to restore by old machine’s data using duplicity, both from an AWS bucket as well as from a backup uploaded to the machine. It was throwing a lot of errors. I will not bother with the details here.

Thus, I decided to test my backups on the old machine aftering passing the secrets from AWS:
sudo -E duplicity restore --force s3://s3.us-east-2.amazonaws.com/xyz-mailinabox-20oct2022 /home/ubuntu/user-data/ (keep in mind, this is only to test a restore, not an acutal one).

I get the following error:
Attempt of list Nr. 1 failed. ModuleNotFoundError: No module named ‘boto3’

These modules are installed:
python-boto/bionic-updates,bionic-security 2.44.0-1ubuntu2.18.04.1 all
python-boto3/bionic 1.4.2-1 all
python-botocore/bionic-updates 1.16.19+repack-1ubuntu0.18.04.1 all

Has anyone else tried a restore with duplicity in ubuntu 18.04 and succeeded recently? Clearly I can run a backup (did a full one today to check) to S3 with no problems. Perhaps I have the syntax wrong?

Thanks for any pointers.

#2

Hi, wondering if you overcame this issue? I’ve been putting off and putting off the move to 22.04 as I was nervous about my knowledge around Digital Ocean droplets and retaining the IP address but I understand that much better now.

I’ve taken the plunge and am testing out the steps on some dummy boxes but the duplicity restore is failing on 18.04 with the same error as you had:

Attempt of list Nr. 1 failed. ModuleNotFoundError: No module named 'boto3'
Attempt of list Nr. 2 failed. ModuleNotFoundError: No module named 'boto3'
Attempt of list Nr. 3 failed. ModuleNotFoundError: No module named 'boto3'
Attempt of list Nr. 4 failed. ModuleNotFoundError: No module named 'boto3'
Giving up after 5 attempts. ModuleNotFoundError: No module named 'boto3'

and on 22.04 with a different error

Attempt of list Nr. 1 failed. ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden
Attempt of list Nr. 2 failed. AttributeError: 'NoneType' object has no attribute 'objects'
Attempt of list Nr. 3 failed. AttributeError: 'NoneType' object has no attribute 'objects'
Attempt of list Nr. 4 failed. AttributeError: 'NoneType' object has no attribute 'objects'

Wondering whether it will be easier to change the backup to local to finish the upgrade and then re-instate S3 but if it’s not working now then no guarantee it will work after the upgrade.

Thanks

#3

Have you tried to install boto3 with pip?

pip3 install boto3

#4

Thanks, that resolved the boto3 error on the 18.04 instance but now I have the same 403 error I have on the 22.04 instance which makes sense

The ClientError: An error occurred (403) … is clearly AWS permission related (access policy/ACL) but for the life of me can’t work out why as my experience of AWS is pretty limited.

Time to start from scratch with a test box and a test bucket I think, thanks for the advice.