[Solved] Setting up rsync backups

I’m trying to set up rsync backups, but I keep getting the error message that it cannot resolve the hostname that I am using. if I do a nslookup from the box it resolves fine. Any ideas?

Thanks

Use the IP address instead?

ah, sorry, forgot to mention, oddly that gives the exact same message

Is the backup path correct on the remote server as well as the username/pass?

I set up a test rsync the other day, and until I had everything correct it errored out. So the key is copied, etc?

yep. and the error message is specifically that the hostname cannot be resolved…

Connection to rsync host failed: The hostname xxx.xxx.xxx.xxx cannot be resolved.

I think there are two things going on here. First is an actual coding bug that was causing the misleading error message…

in backup.py:list_target_files()

elif ‘Could not resolve hostname’:

should be

elif ‘Could not resolve hostname’ in listing:

Second,

Once the above was done, then it was giving me an unknown error and the message says to run the backup from the command line with the --verify option. Fine, except that since it failed to set the backup options it’s wasn’t verifying the rsync settings. So I had to comment out the part of backup.py:backup_set_custom() that validates the settings. This caused the config to get written into the custom.yaml file in the user_data/backup directory. Once that was done, then running the backup command with the --verify option worked and identified what the real problem was.

So the real problem… and this is truly embarrassing… rsync wasn’t installed on the target because it’s not standard in a minimal debian 9 image.

Why are you running Debian 9? MiaB requires Ubuntu 18.04 LTS

I’m running debian 9 on the target of the rsync, not where MiaB is running. It’s just a tiny vm with storage attached.

Gotcha!

Do me a favor please … take a look at /home/user-data/backup/encrypted/ and also your backup directory … make a note of what is there, and then look again in a few days.

People are seeing and reporting oddities such as the backups saving each file in a directory and duplicating themselves, etc.

Personally, I don’t trust the rsync built into MiaB. I much prefer pulling myself via a cron job and rsync from the backup server.

will do. Thanks for your help.

edit: looking at the other thread, I’ll also try restoring from the backup. (which is kind of what started all this, I want to migrate the MiaB server to a different provider.

just as a baseline, after running backup.py --full from the CLI I have the following:

nothing in /home/user-data/backup/encrypted, but there is some stuff in /home/user-data/backup/cache that seems right.

on the rsync target:

 ls -la *
-rw------- 1 backups backups 1679 Aug 11 16:54 id_rsa_miab

duplicity-full.20190811T233719Z.manifest.gpg:
total 16
drwxr-xr-x 2 backups backups 4096 Aug 11 16:42 .
drwxr-xr-x 8 backups backups 4096 Aug 11 16:54 ..
-rw------- 1 backups backups  477 Aug 11 16:42 duplicity-full.20190811T233719Z.manifest.gpg
-rw------- 1 backups backups 1679 Aug 11 16:42 id_rsa_miab

duplicity-full.20190811T233719Z.vol1.difftar.gpg:
total 256296
drwxr-xr-x 2 backups backups      4096 Aug 11 16:38 .
drwxr-xr-x 8 backups backups      4096 Aug 11 16:54 ..
-rw------- 1 backups backups      1679 Aug 11 16:38 id_rsa_miab
-rw------- 1 backups backups 262170221 Aug 11 16:38 mktemp-ZfR3Lp-2

duplicity-full.20190811T233719Z.vol2.difftar.gpg:
total 256288
drwxr-xr-x 2 backups backups      4096 Aug 11 16:39 .
drwxr-xr-x 8 backups backups      4096 Aug 11 16:54 ..
-rw------- 1 backups backups      1679 Aug 11 16:39 id_rsa_miab
-rw------- 1 backups backups 262163516 Aug 11 16:39 mktemp-ks5mh3-3

duplicity-full.20190811T233719Z.vol3.difftar.gpg:
total 256268
drwxr-xr-x 2 backups backups      4096 Aug 11 16:41 .
drwxr-xr-x 8 backups backups      4096 Aug 11 16:54 ..
-rw------- 1 backups backups      1679 Aug 11 16:40 id_rsa_miab
-rw------- 1 backups backups 262142975 Aug 11 16:41 mktemp-nFwCjk-4

duplicity-full.20190811T233719Z.vol4.difftar.gpg:
total 249280
drwxr-xr-x 2 backups backups      4096 Aug 11 16:42 .
drwxr-xr-x 8 backups backups      4096 Aug 11 16:54 ..
-rw------- 1 backups backups      1679 Aug 11 16:42 id_rsa_miab
-rw------- 1 backups backups 254994690 Aug 11 16:42 mktemp-uRUg0w-5

duplicity-full-signatures.20190811T233719Z.sigtar.gpg:
total 34952
drwxr-xr-x 2 backups backups     4096 Aug 11 16:42 .
drwxr-xr-x 8 backups backups     4096 Aug 11 16:54 ..
-rw------- 1 backups backups 35736934 Aug 11 16:42 duplicity-full-signatures.20190811T233719Z.sigtar.gpg
-rw------- 1 backups backups     1679 Aug 11 16:42 id_rsa_miab

which seems a little odd, every directory has a file in it with the same name as the directory plus the ssh key.

@alento looking at this I believe the reason it’s getting a full backup every time is because it’s not expecting the file to be inside directories so it’s not able to see that a full backup has been done.

I’m kinda suspicious at that point that the issue is a duplicity bug (but I’m still looking at it)

Definitely not duplicity. I think I’ve got it fixed. It’s a problem with the rsync and ssl options specified at the top of the file. basically the quoting is wrong. Checked in a fix at github

Excellent, thank you. Just curious, which issue # is it under?

I submitted 3 different PRs - 1625, 1628, 1629

They are referenced from issues - 1624 & 1627