Backup script failing

bobaxx · September 2, 2023, 6:57am

Hi,

i got the same problem and did the patch to backup.py, upgraded duplicity then restart.

Now i get this error message :

Error processing remote manifest (duplicity-inc.20230830T010310Z.to.20230831T010310Z.manifest.gpg): GPG Failed, see log below:
===== Begin GnuPG log =====
gpg: Note: ‘–decrypt’ is not considered an option
gpg: WARNING: no command supplied. Trying to guess what you mean …
usage: gpg [options] [filename]
===== End GnuPG log =====

GPGError: GPG Failed, see log below:
===== Begin GnuPG log =====
gpg: Note: ‘–force-mdc’ is not considered an option
gpg: Note: ‘–symmetric’ is not considered an option
gpg: WARNING: no command supplied. Trying to guess what you mean …
usage: gpg [options] [filename]
===== End GnuPG log =====

Traceback (most recent call last):
File “/root/mailinabox/management/backup.py”, line 625, in
perform_backup(full_backup)
File “/root/mailinabox/management/backup.py”, line 317, in perform_backup
shell(‘check_call’, [
File “/root/mailinabox/management/utils.py”, line 126, in shell
ret = getattr(subprocess, method)(cmd_args, **kwargs)
File “/usr/lib/python3.10/subprocess.py”, line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command ‘[’/usr/bin/duplicity’, ‘incr’, ‘–verbosity’, ‘warning’, ‘–no-print-statistics’, ‘–archive-dir’, ‘/home/user-data/backup/cache’, ‘–exclude’, ‘/home/user-data/backup’, ‘–volsize’, ‘250’, ‘–gpg-options’, “‘–cipher-algo=AES256’”, ‘/home/user-data’, ‘file:///home/user-data/backup/encrypted’, ‘–allow-source-mismatch’]’ returned non-zero exit status 31.

am i the only one to get this message ?

stevep · September 3, 2023, 6:12am

I’m getting the same error in Version 64.

ssh in and I did the usual apt update && apt upgrade
Ran the script to upgrade from version 63 to 64 which completed without incident.
Rebooted
ssh into the box as my non root user, cd mailinabox, sudo maintenance/backup.py and got the same error.
But, the admin backup status page shows no error.

Did I do something wrong?

Thank you.

jusoaresf · September 3, 2023, 8:20am

Hi @stevep
When I checked my backup.py after installing v 64 it was already ‘patched’. I assumed the new version was shipping it with the patch but now you make me think it kept my ‘patched’ version from previous version.
If that’s true then you also need to edit the one at /root/mailinabox/management/backup.py
and/or restart webmin

lebidule · September 3, 2023, 8:25am

FYI duplicity 2.1.1 just released
Don’t know if it fixes “old” MIAB backup script as I patched mine

stevep · September 3, 2023, 9:24pm

Thanks. I will open that script and see if it’s been patched. I would expect it to be patched, with Version 64, right?

After the backup issues on this post, I’ve spent quite a bit of time to learn duplicity, which is what miab uses for backups. It seems robust. I would be interested in general, what folders or files miab does backup. That way, I have the option to just tar them and encrypt that tar file with gpg.

I just want to thank Josh for the great system MIAB is! Thanks for the hard work you do in your unpaid, spare time.

Thank you.

mokiejovis · September 4, 2023, 3:49pm

If your setup is sufficiently old, odds are you have a local non-root user with a mailinabox dir, and then a current release in /root/mailinabox that is the up-to-date version. The one in your non-root user directory hasn’t been updated to v64, will not work with a current release of duplicity, and should probably go. The current one is in /root and will work with the latest release of duplicity as long as you upgrade to v64.

stevep · September 5, 2023, 12:30am

This is spot on, and incredible catch. Plus, I should’ve been aware of dates on the backup.py file as that would’ve helped me understand the error.

my non root user shows backup.py dated Aug 27
my root user shows backup.py dated Sep 3

Running /root/mailinabox/management/backup.py as root worked without an error this time, and the backup status page lists the just made backup.

I haven’t gone back to the MIAB maint section to see the path to the backup.py is, when we run it.

Thank you!

amazzi44 · September 5, 2023, 5:53am

Hello everyone,

yesterday I upgraded to v64.

This morning my backup failed.

My root backup.py was already patched

My non-root backup.py was not, and I patched it, but backup still fails.

My duplicity version is 0.8.21 and I don’t know how to upgrade it besides apt-get upgrade, which does not work.

Does anybody have some advices?

Thank you very much.

andrew · September 5, 2023, 8:21am

If you put duplicity on hold, as some posts suggested, you’ll need to remove the hold. Try sudo apt-mark unhold duplicity, then normal sudo apt update and sudo apt upgrade should work.

amazzi44 · September 5, 2023, 9:06am

Thank you for your reply.

duplicity was already not on hold.

I am using Ubuntu 22

jusoaresf · September 5, 2023, 9:09am

sudo apt install duplicity

amazzi44 · September 5, 2023, 9:54am

duplicity is already the newest version (0.8.21-1build1).

Somehow a problem with repositories? (Of course I did apt update before apt update)

amazzi44 · September 5, 2023, 10:59am

Maybe should I unpatch both my backup.py files until a duplicity upgrade is officially made available?

eXTric · September 5, 2023, 12:33pm

Duplicity should be on 2.x. The issues started with v2. V0.8 is only available when you downgraded

darond · September 5, 2023, 12:37pm

To view the versions available from your repository:

apt list -a duplicity

For me I have current and older available, those are the only versions accessible from DO repo:

duplicity/jammy,now 2.1.1-ppa202309021553~ubuntu22.04.1 amd64 [installed]
duplicity/jammy 0.8.21-1build1 amd64

At least that way you can confirm whether the newer version is available to you which might explain why it is not updating.

amazzi44 · September 5, 2023, 12:53pm

apt list -a duplicity Listing... Done duplicity/jammy,now 0.8.21-1build1 arm64 [installed]

darond · September 5, 2023, 12:59pm

OK at least that explains why your version is not upgrading. If you monitor your overnight emails to see when the latest version becomes available for you, you can proceed at that point with the upgrade.

In the meantime:

You should still be able to get v0.8.21 working - as I did. For me it took a server restart after the backup.py was patched with the single quotes before it would work with that version - have you tried a restart?
If you have and it’s still failing what is the error message you’re getting?
Does the backup screen on the status page also error or display correctly?

Thanks

Daron

mokiejovis · September 5, 2023, 1:08pm

An upgrade for duplicity is officially available. Your Mail-in-a-Box installation should have installed the duplicity PPA.

Just to make sure we’re completely clear:

You’re doing:

sudo apt update

followed by

sudo apt upgrade

Correct?

The update command updates repository data. The upgrade command upgrades the packages themselves.

You should check to make sure the duplicity PPA is installed:

$ ls -al /etc/apt/sources.list.d/
total 24
drwxr-xr-x 2 root root 4096 Apr  3 11:26 .
drwxr-xr-x 8 root root 4096 Mar 16 22:10 ..
-rw-r--r-- 1 root root  192 Sep  4 10:31 duplicity-team-ubuntu-duplicity-release-git-jammy.list
-rw-r--r-- 1 root root  140 Sep  4 10:31 ondrej-ubuntu-php-jammy.list

If it doesn’t exist, you can manually install the duplicity PPA with:
sudo apt-add-repository ppa:duplicity-team/duplicity-release-git

After that, you should be able to update to the latest release of duplicity.

amazzi44 · September 5, 2023, 1:13pm

Thank you both!

I’m at work right now: will put my hands on it later today.

SHAdmin · September 5, 2023, 8:56pm

Version 64 of MiaB has been released which includes an updated version of the backup.py script to work with the new version of Duplicity. If the patch referenced in this discussion has been applied manually, then running curl -s https://mailinabox.email/setup.sh | sudo bash returns:

curl -s https://mailinabox.email/setup.sh | sudo bash
Updating Mail-in-a-Box to v64 . . .
remote: Enumerating objects: 72, done.
remote: Counting objects: 100% (58/58), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 16 (delta 14), reused 7 (delta 6), pack-reused 0
Unpacking objects: 100% (16/16), 2.51 KiB | 214.00 KiB/s, done.
From https://github.com/mail-in-a-box/mailinabox
 * [new tag]         v64        -> v64
error: Your local changes to the following files would be overwritten by checkout:
	management/backup.py
Please commit your changes or stash them before you switch branches.
Aborting
Update failed. Did you modify something in /root/mailinabox?

Since the manually edited backup.py script file should be overwritten, it seems likely safe to manually run the setup/start.sh script to complete the upgrade process. The setup.sh script does not include any way to command the script to continue upgrading beyond the if ! git checkout -q $TAG; then condition. An authoritative reply about whether it is appropriate to now manually run the setup/start.sh script to complete the upgrade process would be much appreciated. Thanks in advance.