I’m trying to set up Scaleway’s “S3 compatible” object storage for MIAB backups. I’ve created a bucket, and I have the endpoint and credentials. When I enter the details in MIAB’s backup config it doesn’t accept the submission, saying “Enter an S3 bucket name.”. There is no field labelled bucket name, but I assume it means the path. My bucket is called
mail-backup, which as far as I know is a valid name. What’s wrong with this?
I’ve also tried ‘backup/mail’, ‘mailbackup’ paths, but they are not accepted. It doesn’t like it appended to the endpoint either.
Try removing the hyphens? If I recall correctly there is an upstream problem with one of MIAB’s packages which doesn’t handle the allowed names properly.
It took me a few goes to get my S3 backup working but it has run properly for a couple of months now.
No, I tried that and several other variations, but nothing is accepted.
The rejection is instant, making me think it’s a client-side validation, but I’ve not managed to track down its code to find out what it wants.
For me in the S3 Path I have:
Mine rejects that too! I’ve no idea what’s going on here.
Yes, ‘path’ in this case is ‘bucket-name’
Just checking, you are setting the region to ‘Other’ ?
Have you already created a folder in your bucket to hold the backups?
That is, I don’t think MIAB will create it.
Yes, region is other because I’m using Scaleway’s service.
I have not created a folder in the bucket because I have no idea how to! There are no controls at the server end at all.
I have another (not MIAB) server that is doing backups to the same service using duplicity and it works fine, and I didn’t need to set up anything at the server end.
I assume you are triggering a backup manually to test?
Otherwise it waits to the 3am hour I believe.
For some reason, though set the UI does not show anything in the fields other than S3 path (at least on my 0.51 box). That I found confusing.
My apologies if my suggestions are too basic. It is hard to know what is going on here.
I might try Scaleway myself. I see they offer 75GB free object storage.
No, there’s no point in triggering the backup manually because I can’t even save the config, so there’s nothing to test.
Sorry not to have been of hel-
Is this problem relevant I wonder.
That’s your problem. MIAB won’t create the bucket, it needs to be pre-existing.
I found instructions to create a bucket in a Scaleway S3 instance using the command line here: How to use Object Storage with s3cmd - Scaleway
That’s not it. I found that there is in fact a way to create folders directly on Scaleway (it was right there!).
Unfortunately it makes no difference to the MAIB config: it still just says “Enter an S3 bucket name”. I checked in the web inspector and I can see that it is submitting the details via XHR, and the rejection is coming from the server, so I was wrong about it being a client-side validation. I can also see that it concatenates the endpoint and path fields to create the full URL.
Using the same base URL, bucket and path, and access credentials with s3cmd works fine.
Just to be sure it’s wasn’t a bucket-specific issue, I created a new one called “bucket”, made a folder inside it called “mail”, and entered a path of “bucket/mail” in MIAB, but it’s still unhappy, and it still works in s3cmd.
I’m suspicious of the speed of the response. When I run a command with bad credentials directly to the S3 service, the failure takes about 500ms to return. From the MIAB admin UI, I get a response in only 80ms, and that includes the additional time taken from my browser to MIAB. This makes me think either it’s miraculously fast, or it’s not actually doing an API call at all.
I found the issue!
Scaleway provides a URL for the endpoint of
https://bucketname.s3.fr-par.sc.cloud, complete with a “Copy” button, which will vary according to your data centre and bucket name. So that’s what I used.
When I used the inspector to look at the requests to MIAB, I was interested in the end of the URL, but I then noticed that the URL started with
s3://https://, which looks very wrong. So I wondered if MIAB was not expecting a URL like that but just a hostname. So I tried it without the prefix… which didn’t work either, but it showed me a different error complaining about host names. I noticed that the s3cmd config wanted a host name both with and without the bucket name, so I tried without, and that worked!
So, if your Scaleway endpoint is provided as
https://bucketname.s3.fr-par.scw.cloud, and you’re backing up to a folder called
mail within that bucket, your MIAB settings should be:
- S3 Region:
- S3 Host / Endpoint:
s3.fr-par.scw.cloud (notice it does not include the bucket name)
- S3 Path:
After all that, MIAB replied with “OK”, so I’ll wait to see if the backup goes ahead later tonight.
I also checked some of the earlier things we were suspicious about, and it works fine if your bucket name includes dashes.
This could have been avoided with some better labelling on the fields, perhaps some tooltips?
Glad your persistence paid off. Thanks for sharing the answer.
Might give Scaleway a go myself.
The backup did indeed complete successfully.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.