Has anyone got the Backblaze B2 setup for backup?
I kept getting 500 internal server error message.
After further investigation. The problem is not on Backblaze b2 because I was able to run duplicity backup to b2 on my other linux box. But when i ran the same command on the MIAB i’m getting some AttributeError.
root@mailvps:/# duplicity ~ b2://xxxxx:xxxxx@MailServerJW
Traceback (innermost last):
File “/usr/bin/duplicity”, line 87, in
with_tempdir(main)
File “/usr/bin/duplicity”, line 70, in with_tempdir
fn()
File “/usr/lib/python3/dist-packages/duplicity/dup_main.py”, line 1564, in main
action = commandline.ProcessCommandLine(sys.argv[1:])
File “/usr/lib/python3/dist-packages/duplicity/commandline.py”, line 1254, in ProcessCommandLine
backup, local_pathname = set_backend(args[0], args[1])
File “/usr/lib/python3/dist-packages/duplicity/commandline.py”, line 1125, in set_backend
config.backend = backend.get_backend(bend)
File “/usr/lib/python3/dist-packages/duplicity/backend.py”, line 223, in get_backend
obj = get_backend_object(url_string)
File “/usr/lib/python3/dist-packages/duplicity/backend.py”, line 209, in get_backend_object
return factory(pu)
File “/usr/lib/python3/dist-packages/duplicity/backends/b2backend.py”, line 78, in init
from b2sdk.v2 import B2Api
File “/usr/local/lib/python3.6/dist-packages/b2sdk/v2/init.py”, line 11, in
from b2sdk._v3 import * # noqa
File “/usr/local/lib/python3.6/dist-packages/b2sdk/_v3/init.py”, line 170, in
from b2sdk.sync.folder import AbstractFolder
File “/usr/local/lib/python3.6/dist-packages/b2sdk/sync/folder.py”, line 21, in
from .scan_policies import DEFAULT_SCAN_MANAGER, ScanPoliciesManager
File “/usr/local/lib/python3.6/dist-packages/b2sdk/sync/scan_policies.py”, line 108, in
class ScanPoliciesManager:
File “/usr/local/lib/python3.6/dist-packages/b2sdk/sync/scan_policies.py”, line 131, in ScanPoliciesManager
exclude_uploaded_after: Optional[int] = None,
AttributeError: module ‘re’ has no attribute ‘Pattern’
1 Like
I’ve just setup my MIAB today and I’m receiving this same error. I’m using a linode nanode, could lack of resources be the issue?
The back-blaze configuration seems self explanatory:
- Setup your backblaze account
- create your buckup
- create you app key
- Copy the keyid, application key, and the bucket name and fill in the appropriate fields and click save.
I can see the same python errors when running journalctl -u mailinabox.service
I also couldn’t find any documentation how to setup a backblaze backup configuration. Once i get this working I’d be willing write up a guide to share for others if one doesn’t already exist.
I don’t think resources is the issue. My VPS is hosted at OVH which has 2GB of RAM which is plenty. Maybe some compatibility issue between Python and duplicity command?
On my box it shows
Python version is 2.7.17
duplicity version is 0.8.23
I tried to use python 3.6.9 it gave me the same error.
It looks like re.Pattern was added in Python 3.7 and the b2sdk doesn’t support anything lower than that.
Looks like they dropped support for 3.6 in April so the b2sdk dependency would need to be pinned to 1.14.1.
1 Like
It looks like you’re onto something. I tested installing the b2 package with python3 on a vanilla ubuntu 18.0.4 installation:
sudo apt update -y
sudo apt install python3-pip -y
pip3 install b2
The package installs successfully
Collecting b2
Downloading https://files.pythonhosted.org/packages/8a/f2/d2ab885c30c17344d40bf0f6ad13ba12dc05aec5db358662c68981b9fb27/b2-3.4.0.tar.gz (86kB)
100% |████████████████████████████████| 92kB 10.5MB/s
Collecting arrow<2.0.0,>=1.0.2 (from b2)
Downloading https://files.pythonhosted.org/packages/36/e7/3043959c8e3e3d6e346b69417e85daa591f9c018b99c383cc3f316bbf269/arrow-1.2.2-py3-none-any.whl (64kB)
100% |████████████████████████████████| 71kB 12.2MB/s
Collecting b2sdk==1.16.0 (from b2)
Downloading https://files.pythonhosted.org/packages/3d/be/61b8631fc00c4036136cb0b31359f0aa25da1e35b64676b79e7d0637372b/b2sdk-1.16.0.tar.gz (290kB)
100% |████████████████████████████████| 296kB 4.6MB/s
Collecting docutils==0.16 (from b2)
Downloading https://files.pythonhosted.org/packages/81/44/8a15e45ffa96e6cf82956dd8d7af9e666357e16b0d93b253903475ee947f/docutils-0.16-py2.py3-none-any.whl (548kB)
100% |████████████████████████████████| 552kB 2.5MB/s
Collecting phx-class-registry==3.0.5 (from b2)
Downloading https://files.pythonhosted.org/packages/32/70/666e3ccf2e8b49c3cabd807570b3e410d9fbda9dcae15421f656a770d377/phx_class_registry-3.0.5-py3-none-any.whl
Collecting rst2ansi==0.1.5 (from b2)
Downloading https://files.pythonhosted.org/packages/46/cb/fdb216f2b8bbec9c43655a79f2f280b2ba7822b2c8396ecceafa0c232320/rst2ansi-0.1.5-py3-none-any.whl
Collecting importlib-metadata>=3.3.0 (from b2)
Downloading https://files.pythonhosted.org/packages/a0/a1/b153a0a4caf7a7e3f15c2cd56c7702e2cf3d89b1b359d1f1c5e59d68f4ce/importlib_metadata-4.8.3-py3-none-any.whl
Collecting typing-extensions; python_version < "3.8" (from arrow<2.0.0,>=1.0.2->b2)
Downloading https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl
Collecting python-dateutil>=2.7.0 (from arrow<2.0.0,>=1.0.2->b2)
Downloading https://files.pythonhosted.org/packages/36/7a/87837f39d0296e723bb9b62bbb257d0355c7f6128853c78955f57342a56d/python_dateutil-2.8.2-py2.py3-none-any.whl (247kB)
100% |████████████████████████████████| 256kB 5.3MB/s
Collecting logfury<2.0.0,>=1.0.1 (from b2sdk==1.16.0->b2)
Downloading https://files.pythonhosted.org/packages/04/c7/8a2cfdf39f4cbbb307603aeb03ca89a4765b96be310a0848b20eb4d17e76/logfury-1.0.1-py3-none-any.whl
Requirement already satisfied: requests<3.0.0,>=2.9.1 in /usr/lib/python3/dist-packages (from b2sdk==1.16.0->b2)
Collecting tqdm<5.0.0,>=4.5.0 (from b2sdk==1.16.0->b2)
Downloading https://files.pythonhosted.org/packages/8a/c4/d15f1e627fff25443ded77ea70a7b5532d6371498f9285d44d62587e209c/tqdm-4.64.0-py2.py3-none-any.whl (78kB)
100% |████████████████████████████████| 81kB 11.8MB/s
Collecting zipp>=0.5 (from importlib-metadata>=3.3.0->b2)
Downloading https://files.pythonhosted.org/packages/bd/df/d4a4974a3e3957fd1c1fa3082366d7fff6e428ddb55f074bf64876f8e8ad/zipp-3.6.0-py3-none-any.whl
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.7.0->arrow<2.0.0,>=1.0.2->b2)
Collecting importlib-resources; python_version < "3.7" (from tqdm<5.0.0,>=4.5.0->b2sdk==1.16.0->b2)
Downloading https://files.pythonhosted.org/packages/24/1b/33e489669a94da3ef4562938cd306e8fa915e13939d7b8277cb5569cb405/importlib_resources-5.4.0-py3-none-any.whl
Building wheels for collected packages: b2, b2sdk
Running setup.py bdist_wheel for b2 ... done
Stored in directory: /root/.cache/pip/wheels/be/1c/3e/712b8f822d66ee571d4879ec29514a510febd539625d8eb1b9
Running setup.py bdist_wheel for b2sdk ... done
Stored in directory: /root/.cache/pip/wheels/7d/10/c0/f8b6c4e4e2293b150576355d32e911dd3dbb2c20afd0dd8b33
Successfully built b2 b2sdk
Installing collected packages: typing-extensions, python-dateutil, arrow, logfury, zipp, importlib-resources, tqdm, importlib-metadata, b2sdk, docutils, phx-class-registry, rst2ansi, b2
Successfully installed arrow-1.2.2 b2-3.4.0 b2sdk-1.16.0 docutils-0.16 importlib-metadata-4.8.3 importlib-resources-5.4.0 logfury-1.0.1 phx-class-registry-3.0.5 python-dateutil-2.8.2 rst2ansi-0.1.5 tqdm-4.64.0 typing-extensions-4.1.1 zipp-3.6.0
Running the b2 package on its own should result in the normal command --help output:
This program provides command-line access to the B2 service.
Usages:
b2 authorize-account [<applicationKeyId>] [<applicationKey>]
b2 cancel-all-unfinished-large-files <bucketName>
b2 cancel-large-file <fileId>
b2 clear-account
b2 copy-file-by-id [--metadataDirective [copy|replace]] [--contentType <contentType>] \
[--info <key>=<value>]* [--range start,end] \
<sourceFileId> <destinationBucketName> <b2FileName>
b2 create-bucket [--bucketInfo <json>] [--corsRules <json>] [--lifecycleRules <json>] <bucketName> [allPublic | allPrivate]
b2 create-key [--duration <validDurationSeconds>] [--bucket <bucketName>] [--namePrefix <namePrefix>] <keyName> <capabilities>
b2 delete-bucket <bucketName>
b2 delete-file-version [<fileName>] <fileId>
b2 delete-key <applicationKeyId>
b2 download-file-by-id [--noProgress] <fileId> <localFileName>
b2 download-file-by-name [--noProgress] <bucketName> <fileName> <localFileName>
b2 get-account-info
b2 get-bucket [--showSize] <bucketName>
b2 get-download-auth [--prefix <fileNamePrefix>] [--duration <durationInSeconds>] <bucketName>
b2 get-download-url-with-auth [--duration <durationInSeconds>] <bucketName> <fileName>
b2 get-file-info <fileId>
b2 help [commandName]
b2 hide-file <bucketName> <fileName>
b2 list-buckets
b2 list-keys
b2 list-parts <largeFileId>
b2 list-unfinished-large-files <bucketName>
b2 ls [--long] [--versions] [--recursive] <bucketName> [<folderName>]
b2 make-url <fileId>
b2 sync [--delete] [--keepDays N] [--skipNewer] [--replaceNewer] \
[--compareVersions <option>] [--compareThreshold N] \
[--threads N] [--noProgress] [--dryRun ] [--allowEmptySource ] \
[--excludeRegex <regex> [--includeRegex <regex>]] \
[--excludeDirRegex <regex>] \
[--excludeAllSymlinks ] \
<source> <destination>
b2 update-bucket [--bucketInfo <json>] [--corsRules <json>] [--lifecycleRules <json>] <bucketName> [allPublic | allPrivate]
b2 upload-file [--sha1 <sha1sum>] [--contentType <contentType>] \
[--info <key>=<value>]* [--minPartSize N] \
[--noProgress] [--threads N] <bucketName> <localFilePath> <b2FileName>
b2 version
The environment variable B2_ACCOUNT_INFO specifies the sqlite
file to use for caching authentication information.
The default file to use is: ~/.b2_account_info
For more details on one command: b2 help <command>
When authorizing with application keys, this tool requires that the key
have the 'listBuckets' capability so that it can take the bucket names
you provide on the command line and translate them into bucket IDs for the
B2 Storage service. Each different command may required additional
capabilities. You can find the details for each command in the help for
that command.
But i get an exception instead:
Traceback (most recent call last):
File "/usr/local/bin/b2", line 7, in <module>
from b2.console_tool import main
File "/usr/local/lib/python3.6/dist-packages/b2/console_tool.py", line 28, in <module>
from b2sdk.v2 import (
File "/usr/local/lib/python3.6/dist-packages/b2sdk/v2/__init__.py", line 11, in <module>
from b2sdk._v3 import * # noqa
File "/usr/local/lib/python3.6/dist-packages/b2sdk/_v3/__init__.py", line 16, in <module>
from b2sdk.api import B2Api
File "/usr/local/lib/python3.6/dist-packages/b2sdk/api.py", line 13, in <module>
from .account_info.abstract import AbstractAccountInfo
File "/usr/local/lib/python3.6/dist-packages/b2sdk/account_info/__init__.py", line 11, in <module>
from .in_memory import InMemoryAccountInfo
File "/usr/local/lib/python3.6/dist-packages/b2sdk/account_info/in_memory.py", line 14, in <module>
from .upload_url_pool import UrlPoolAccountInfo
File "/usr/local/lib/python3.6/dist-packages/b2sdk/account_info/upload_url_pool.py", line 15, in <module>
from .abstract import AbstractAccountInfo
File "/usr/local/lib/python3.6/dist-packages/b2sdk/account_info/abstract.py", line 14, in <module>
from b2sdk.raw_api import ALL_CAPABILITIES
File "/usr/local/lib/python3.6/dist-packages/b2sdk/raw_api.py", line 20, in <module>
from .replication.setting import ReplicationConfiguration
File "/usr/local/lib/python3.6/dist-packages/b2sdk/replication/setting.py", line 14, in <module>
from dataclasses import dataclass, field
Is the solution to upgrade python to version 3.7?
Upgrading to Python 3.7+ or downgrading b2sdk to 1.14.1 would do it, I think.
EDIT: Updating python, that turned out to be far more complicated than downgrading the b2sdk package to 1.14.1.
Updating the b2sdk package is also not as easy as running sudo pip3 install b2sdk==1.14.1.
I found that you have to use the management.sh script to have it install the python package in the appropriate location.
SOLUTION: I’ve been able to get backblaze working by updating the mailinabox/setup/management.sh script at line 33 to specify that it install the b2sdk package at version 1.14.1:
hide_output pip3 install --upgrade b2sdk==1.14.1 boto
after updating this file, run sudo mailinabox again and it will install the b2sdk package in the appropriate location for mailinabox.
After that configure your backblaze settings and perform the backup.
4 Likes
brilliant. I hope to not see MIAB get dusty and out of date glad to see progress and solutions starting to rise.
I’d like to request making a change to the management.sh script in the github repo to ensure this fix is included in new setups. Do i have to open a github issue first?
In looking through GitHub, I think a lot of people open an issue first to see response from others, but other people just submit a PR directly without posting an issue.
Do i have to create a fork of the repo and then open a PR? I’m not sure what the procedure is for something like this. This is my first time contributing to a open source project like this.
If you want to submit a PR, yes, you should fork, make changes, then submit PR.
The exception is extremely simple changes that can be made using the very basic editor in GitHub. If you’re not sure, fork.
Thank you so much. I followed your instructions and it works perfectly.
Oops I say it too fast. duplicity command is working now when I manually ran it at the command prompt but the web use interface where I setup the B2 backup is still give me the same 500 internal server error.
When i run journalctl this is what i get.
May 21 18:48:13 vps.readytech.us start[15296]: File “/root/mailinabox/management/backup.py”, line 498, in backup_set_custom
May 21 18:48:13 vps.readytech.us start[15296]: list_target_files(config)
May 21 18:48:13 vps.readytech.us start[15296]: File “/root/mailinabox/management/backup.py”, line 460, in list_target_files
May 21 18:48:13 vps.readytech.us start[15296]: from b2sdk.v1 import InMemoryAccountInfo, B2Api
May 21 18:48:13 vps.readytech.us start[15296]: File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/v1/init.py”, line 11, in
May 21 18:48:13 vps.readytech.us start[15296]: from b2sdk.v2 import * # noqa
May 21 18:48:13 vps.readytech.us start[15296]: File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/v2/init.py”, line 11, in
May 21 18:48:13 vps.readytech.us start[15296]: from b2sdk._v3 import * # noqa
May 21 18:48:13 vps.readytech.us start[15296]: File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/_v3/init.py”, line 170, in
May 21 18:48:13 vps.readytech.us start[15296]: from b2sdk.sync.folder import AbstractFolder
May 21 18:48:13 vps.readytech.us start[15296]: File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/sync/folder.py”, line 21, in
May 21 18:48:13 vps.readytech.us start[15296]: from .scan_policies import DEFAULT_SCAN_MANAGER, ScanPoliciesManager
May 21 18:48:13 vps.readytech.us start[15296]: File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/sync/scan_policies.py”, line 108, in
May 21 18:48:13 vps.readytech.us start[15296]: class ScanPoliciesManager:
May 21 18:48:13 vps.readytech.us start[15296]: File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/sync/scan_policies.py”, line 131, in ScanPoliciesManager
May 21 18:48:13 vps.readytech.us start[15296]: exclude_uploaded_after: Optional[int] = None,
May 21 18:48:13 vps.readytech.us start[15296]: AttributeError: module ‘re’ has no attribute ‘Pattern’
May 21 18:48:13 vps.readytech.us python[15296]: Exception on /system/backup/config [POST]
Traceback (most recent call last):
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/flask/app.py”, line 2073, in wsgi_app
response = self.full_dispatch_request()
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/flask/app.py”, line 1518, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/flask/app.py”, line 1516, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/flask/app.py”, line 1502, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File “/root/mailinabox/management/daemon.py”, line 75, in newview
return viewfunc(*args, **kwargs)
File “/root/mailinabox/management/daemon.py”, line 637, in backup_set_custom
request.form.get(‘min_age’, ‘’)
File “/root/mailinabox/management/backup.py”, line 498, in backup_set_custom
list_target_files(config)
File “/root/mailinabox/management/backup.py”, line 460, in list_target_files
from b2sdk.v1 import InMemoryAccountInfo, B2Api
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/v1/init.py”, line 11, in
from b2sdk.v2 import * # noqa
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/v2/init.py”, line 11, in
from b2sdk._v3 import * # noqa
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/_v3/init.py”, line 170, in
from b2sdk.sync.folder import AbstractFolder
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/sync/folder.py”, line 21, in
from .scan_policies import DEFAULT_SCAN_MANAGER, ScanPoliciesManager
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/sync/scan_policies.py”, line 108, in
class ScanPoliciesManager:
File “/usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk/sync/scan_policies.py”, line 131, in ScanPoliciesManager
exclude_uploaded_after: Optional[int] = None,
AttributeError: module ‘re’ has no attribute ‘Pattern’
May 21 18:48:13 vps.readytech.us start[15296]: 127.0.0.1 - - [21/May/2022 18:48:13] “POST /system/backup/config HTTP/1.0” 500 -
It looks like the newer version of the b2sdk package is installed and currently being used by mailinabox. If you run the command ls -lah /usr/local/lib/mailinabox/env/lib/python3.6/site-packages do you see b2sdk-1.16.0.dist-info directory listed?
If so i’d recommend deleting the b2sdk package files and reinstalling them using the pip3 command.
- stop the mailinabox service:
sudo systemctl stop mailinabox - delete the b2sdk python packages used by mailinabox:
sudo rm -rfv /usr/local/lib/mailinabox/env/lib/python3.6/site-packages/b2sdk* - Install the b2sdk package in mailinabox’s site-packages directory at the required version and upgrade any other packages:
sudo pip3 install b2sdk==1.14.1 --target /usr/local/lib/mailinabox/env/lib/python3.6/site-packages --upgrade - restart the mailinabox service:
sudo systemctl restart mailinabox
1 Like
I see the b2sdk-1.16.0.dist-info directory and I followed your steps 1~4 and it did the trick.
THANK YOU VERY VERY MUCH.
1 Like
Same erors as above but didn’t want to mess up my python installation. Instead,
pip install b2==3.1.0
Has done the trick for me.
1 Like