Backup problems after upgrading 0.65 -> 0.66

informatom · December 20, 2023, 8:15am

Edit: Problem solved (see my list update for the resolution))

I upgraded to 0.66 yesterday. Today I see (via my automated monitoring over the api) that a full backup and incremental ones are not cleared up:
2023-07-19 03:30:09 CET 5 months, 1 days ago full 1.9 GB on next daily backup

It was not there yesterday before the upgrade - my montoring checks daily.
So far so good, the daily backup job had suceeded. I thought, I give manual backup a check, as described in the maintenance guide for the backup before upgrade

./management/backup.py 
CommandLineError: argument --gpg-options: expected one argument
Something is wrong with the backup: ERROR 23 CommandLineError
. CommandLineError: argument --gpg-options: expected one argument

But this now fails (I remember we had this problem earlier, but with a diffent duplicity version).

$ duplicity --version
duplicity 2.1.4

So has there been a coincidential upgrade of duplicity as well? I checked Ubuntu’u package page duplicity package : Ubuntu and indeed, Published on 2023-12-18 might indicate this.

Edit: I just realized, that mailinabox uses a ppa for duplicity, so I checked there as well and that makes even more sense, it’s a release from today (just a little bit funny, that for my time zone, the timestamp is from the future):
https://ppa.launchpadcontent.net/duplicity-team/duplicity-release-git/ubuntu/pool/main/d/duplicity/

Anybody else experiencing that?

gezza · December 20, 2023, 8:41am

Hi
I know it may not help but I did the exact same upgrade yesterday 65 to 66 and all is working well, backup went through without issue.

My duplicity ver: 2.1.4

Had any other modifications been done before hand?

informatom · December 20, 2023, 8:49am

Thanks for the quick reply!
No modifications made on my instance.

Version 2.1.4 from the ppa has been updated within 2.1.4 today in the morning, so nightly backups have succeeded so far on my side and also your backup will fail tonight in case you run automatic updates.
So it even depends on the duplicity version timestamp and mine is:

$ apt list --installed | grep duplicity
duplicity/jammy,now 2.1.4-ppa202310201613~ubuntu22.04.1 amd64 [installed]

gezza · December 20, 2023, 9:14am

Hello

Not sure whats happening here but I’ve just run an “apt update” and all I have is:

2 packages can be upgraded. Run ‘apt list --upgradable’ to see them.
root@xxxxx:/home/gez# apt list --upgradable
Listing… Done
python3-update-manager/jammy-updates 1:22.04.17 all [upgradable from: 1:22.04.10]
update-manager-core/jammy-updates 1:22.04.17 all [upgradable from: 1:22.04.10]

my duplicity: duplicity/jammy,now 2.1.4-ppa202310201613~ubuntu22.04.1 amd64 [installed]

Last nights backup went through no problems and this mornings manual backup no problems.

Sorry I seem to be at a loss to help, maybe re-running the setup again may overwrite the existing binaries and fix the issue.

Please keep us updated.

gezza · December 20, 2023, 9:22am

Just looking at that link for the ppa I can’t see any Published on 2023-12-18

Am I looking in the wrong place?

vele · December 20, 2023, 11:46am

You might purge duplicity ppa version and install a snap duplicity see if it is working like the example for the arm64 problem here: Using MIAB On ARM

informatom · December 20, 2023, 4:20pm

No, you are looking in the right place, update was today morning.
First I was writing about the standard Ubuntu duplicity package, that was updated on 18th. I included the link where I was looking into in my original post.
Then I realized Mailinabox introduced the ppa. That’s why I put the ‘Edit:’ section in.

informatom · December 20, 2023, 4:23pm

So you re-ran the backup and it was fine, interesting!
I also re-ran the installer, and then the backup, but that didn’t change anything.

informatom · December 20, 2023, 4:29pm

Tried this approach, had to add a symlink for the backup script to start
sudo ln -s /snap/bin/duplicity /usr/bin/duplicity
but then again

$ sudo ./management/backup.py 
CommandLineError: argument --gpg-options: expected one argument
Something is wrong with the backup: ERROR 23 CommandLineError
. CommandLineError: argument --gpg-options: expected one argument

Nice try, but didn’t help. So I’m going back to the ppa and standard mailinabox.

gezza · December 20, 2023, 4:43pm

Yep all ran ok. If i send you my “mailinabox/management/backup.py” file you could run a diff against yours to see if there’s any difference. Let me know if you want the copy

FYI: Manual backup made about 08:42 this morning

informatom · December 20, 2023, 4:46pm

Ok, problem solved:
My installation was missing the patched version for the backup script that was discussed in

and can be found in

github.com

mail-in-a-box/mailinabox/blob/main/management/backup.py

#!/usr/local/lib/mailinabox/env/bin/python

# This script performs a backup of all user data:
# 1) System services are stopped.
# 2) STORAGE_ROOT/backup/before-backup is executed if it exists.
# 3) An incremental encrypted backup is made using duplicity.
# 4) The stopped services are restarted.
# 5) STORAGE_ROOT/backup/after-backup is executed if it exists.

import os, os.path, shutil, glob, re, datetime, sys
import dateutil.parser, dateutil.relativedelta, dateutil.tz
import rtyaml
from exclusiveprocess import Lock

from utils import load_environment, shell, wait_for_service

def backup_status(env):
	# If backups are dissbled, return no status.
	config = get_backup_config(env)
	if config["target"] == "off":

This file has been truncated. show original

Copied in the latest version, ran backup manually and now I’m fine.
I have no Idea, why the reinstall didn’t update the backup script.

gezza · December 20, 2023, 4:53pm

Glad to hear that.

Thanks for letting me know.

daniel1 · December 25, 2023, 3:30pm

I’m just chiming in to say that I had the exact same problem. Today I upgraded from v63 to v67. I don’t think I ever manually modified anything in the mailinabox directory so I was surprised that the backup.py didn’t update.

I had the duplicity issue before. I thought it was a duplicity bug so I manually downgraded duplicity and told apt to block that specific version. When the next duplicity build was released it automatically updated and my backups have been broken since, i.e. for 3 months. Today I decided to fix it. My first step was to update to the newest mailinabox. I manually ran the backup.py and I kept getting the same ERROR 23 until I manually copied in the latest backup.py from git.

system · January 1, 2024, 3:30pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.