Glue records stops resolving after moving the box from one server to another

gaobo · January 21, 2020, 5:56pm

I was running a box on server A fine.
Then I backed it up.
Then I set up another server B.
Then I followed the instructions strictly to import everything to server B.
And I changed the glue records of my box’s domain from IP address of server A’s to server B’s.
And the glue records stops resolving…
…2 weeks passed and it still has not been resolving.

I confirmed that the port 53 of server B’s has been opened for both TCP and UDP.

I have no clue what to do next.

So I am here calling for help.

brocktice · January 21, 2020, 7:08pm

Are you OK disclosing the domain? If not, you can try:

dig +trace name-of-box

and you might find some useful info on where the missing piece is.

You can also try dig @your-registrars-nameserver NS your-domain and see what it tells you.

alento · January 21, 2020, 8:05pm

As the ‘glue records’ are not served by your MiaB server, this is not the issue … so, to best diagnose things if you’d share the hostname of your MiaB in PM, I can look at DNS and find the problem. Or feel free to visit on Slack and myself or someone can help you in real time. https://mailinabox.email/slack

gaobo · January 22, 2020, 3:02am

Domain name: GAOBO.ORG
Hostname: NS1.CASINO.GAOBO.ORG NS2.CASINO.GAOBO.ORG

I am not running a casino though.

gaobo · January 22, 2020, 3:15am

The output of the dig command has been as follows:

dig +trace casino.gaobo.org

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-9.P2.el7 <<>> +trace casino.gaobo.org
;; global options: +cmd
.			432733	IN	NS	l.root-servers.net.
.			432733	IN	NS	h.root-servers.net.
.			432733	IN	NS	g.root-servers.net.
.			432733	IN	NS	a.root-servers.net.
.			432733	IN	NS	j.root-servers.net.
.			432733	IN	NS	i.root-servers.net.
.			432733	IN	NS	f.root-servers.net.
.			432733	IN	NS	b.root-servers.net.
.			432733	IN	NS	k.root-servers.net.
.			432733	IN	NS	d.root-servers.net.
.			432733	IN	NS	m.root-servers.net.
.			432733	IN	NS	e.root-servers.net.
.			432733	IN	NS	c.root-servers.net.
;; Received 239 bytes from 169.254.169.254#53(169.254.169.254) in 1 ms

org.			172800	IN	NS	a0.org.afilias-nst.info.
org.			172800	IN	NS	a2.org.afilias-nst.info.
org.			172800	IN	NS	b0.org.afilias-nst.org.
org.			172800	IN	NS	b2.org.afilias-nst.org.
org.			172800	IN	NS	c0.org.afilias-nst.info.
org.			172800	IN	NS	d0.org.afilias-nst.org.
org.			86400	IN	DS	9795 7 1 364DFAB3DAF254CAB477B5675B10766DDAA24982
org.			86400	IN	DS	9795 7 2 3922B31B6F3A4EA92B19EB7B52120F031FD8E05FF0B03BAFCF9F891B FE7FF8E5
org.			86400	IN	RRSIG	DS 8 1 86400 20200203170000 20200121160000 33853 . AeiKRBE4E2SunytWX1FW11D2lfw9hnJSOrpaKTZEQhXmZTYvaXyNEc7Q KzftjIT7BcspKNOwsHI9VHuPeOko/BBpa+0axV3PRbFaTv+PPvs14vJn C7VFw9wbnVsAQ62YRR1a3+uH6hKXbGq+OdITUbT+T/K/0/akY9++5b5d pL3+jnQp2lV7t+H7tqRpr5voxjCOal+q5n4L59OylYhfy0CbLkIE/YJy Jby0hivJV6QVbeTSeSICj6i2tau00ETNE8Uw/kZrcmzmFU1Vl1H/eBCB WBIrwxwz/pdaq1dRxD9ksM+rRk8/Lz4+jIIrvccbJMIkZRh4wG+EStZ1 owmjIQ==
;; Received 860 bytes from 193.0.14.129#53(k.root-servers.net) in 26 ms

gaobo.org.		86400	IN	NS	ns1.casino.gaobo.org.
gaobo.org.		86400	IN	NS	ns2.casino.gaobo.org.
h9p7u7tr2u91d0v0ljs9l1gidnp90u3h.org. 86400 IN NSEC3 1 1 1 D399EAAB H9PARR669T6U8O1GSG9E1LMITK4DEM0T NS SOA RRSIG DNSKEY NSEC3PARAM
h9p7u7tr2u91d0v0ljs9l1gidnp90u3h.org. 86400 IN RRSIG NSEC3 7 2 86400 20200212031134 20200122021134 9278 org. hQKKQnt2H5VfSR2EBlMfboIYLabv8LpwSZBjvCVrsBlFH/UdWfOyAQfE cW9uGMRzfPPLygZeG0xdOu1NJj6mlN3R6ab5hefODZah3HQFQFR5IgAi 1f3vk/f6bljzPZOUV6p2g2f/gk5WZ11juDIkqVFfuWh3UOxO4qPk/II2 ALY=
2v4sr6mjukiga4n1mftvjkp7poptodku.org. 86400 IN NSEC3 1 1 1 D399EAAB 2V5FLT1P97EUA5H1V37M8T0D64NO4DB8
2v4sr6mjukiga4n1mftvjkp7poptodku.org. 86400 IN RRSIG NSEC3 7 2 86400 20200207152810 20200117142810 9278 org. HcrIO8fUCU5KE/XnPB0z83UkkazhmegcyimVyvuaJUfzzUqFSus4/hp6 86403Q59pxm4qHc/Cm6TbX9vmYQ4zxmqpXMWHrGQk675v8LzsrjqbyTK 1aqbecfySGTcI73hfqN/vZPZ+KVR7mOXKKRUZNWNdtCq4V6CyE9uUG77 IHU=
couldn't get address for 'ns1.casino.gaobo.org': not found
couldn't get address for 'ns2.casino.gaobo.org': not found
dig: couldn't get address for 'ns1.casino.gaobo.org': no more

It seems that the root cause is that the glue records, which should generate the IP addresses of ns1.casino.gaobo.org and ns2.casino.gaobo.org, has been failing to resolve. Thus, the service provider of the domain name GAOBO.ORG, is to blame. Am I right?

brocktice · January 22, 2020, 3:56am

I am finding no NS records for gaobo.org or casino.gaobo.org. In addition to the glue records, you need NS entries at your registrar pointing to those glue record names.

alento · January 22, 2020, 12:24pm

Name server delegation at the registrar is ok. Glue records are set properly. I can connect to postfix on the server just fine. I however CANNOT dig your name server.

Please post a screen cap of your status page.

gaobo · January 22, 2020, 3:59pm

That is exactly what annoyed me so far.

The status check process cannot finish normally. It ends with the dialog box as follows.

alento · January 22, 2020, 4:46pm

Then please run it manually from the terminal.

While in the /mailinabox directory type ./management/status_check.py

gaobo · January 23, 2020, 2:37am

Some relevant messages are as follows:

casino.gaobo.org
================
✖  Nameserver glue records are incorrect. The ns1.casino.gaobo.org and ns2.casino.gaobo.org nameservers must be configured at 
   your domain name registrar as having the IP address 152.67.67.163. They currently report addresses of [Not Set]/[Not Set]. It
   may take several hours for public DNS to update after a change.
✖  This domain must resolve to your box's IP address (152.67.67.163) in public DNS but it currently resolves to [Not Set]. It 
   may take several hours for public DNS to update after a change. This problem may result from other issues listed above.
✖  Your box's reverse DNS is currently [Not Set], but it should be casino.gaobo.org. Your ISP or cloud provider will have 
   instructions on setting up reverse DNS for your box.
✓  Hostmaster contact address exists as a mail alias. [hostmaster@casino.gaobo.org ↦ administrator@casino.gaobo.org]
✓  Domain's email is directed to this domain. [casino.gaobo.org has no MX record, which is ok]
✓  Postmaster contact address exists as a mail alias. [postmaster@casino.gaobo.org ↦ administrator@casino.gaobo.org]
✓  Domain is not blacklisted by dbl.spamhaus.org.
✓  TLS (SSL) certificate is signed & valid. The certificate expires in 31 days on 02/23/20.

autoconfig.casino.gaobo.org
===========================
✖  This domain should resolve to your box's IP address (A 152.67.67.163) if you would like the box to serve webmail or a website
   on this domain. The domain currently resolves to [Not Set] in public DNS. It may take several hours for public DNS to update 
   after a change. This problem may result from other issues listed here.

autodiscover.casino.gaobo.org
=============================
✖  This domain should resolve to your box's IP address (A 152.67.67.163) if you would like the box to serve webmail or a website
   on this domain. The domain currently resolves to [Not Set] in public DNS. It may take several hours for public DNS to update 
   after a change. This problem may result from other issues listed here.

gaobo.org
=========
✖  The nameservers set on this domain are incorrect. They are currently [Not Set]. Use your domain name registrar's control 
   panel to set the nameservers to ns1.casino.gaobo.org; ns2.casino.gaobo.org.
✖  This domain's DNS MX record is not set. It should be '10 casino.gaobo.org'. Mail will not be delivered to this box. It may 
   take several hours for public DNS to update after a change. This problem may result from other issues listed here.
✓  Domain is not blacklisted by dbl.spamhaus.org.
✖  This domain should resolve to your box's IP address (A 152.67.67.163) if you would like the box to serve webmail or a website
   on this domain. The domain currently resolves to [Not Set] in public DNS. It may take several hours for public DNS to update 
   after a change. This problem may result from other issues listed here.
?  This domain's DNSSEC DS record is not set. The DS record is optional. The DS record activates DNSSEC. To set a DS record, you
   must follow the instructions provided by your domain name registrar and provide to them this information:
   
   Key Tag: 61575
   Key Flags: KSK
   Algorithm: 7 / RSASHA1-NSEC3-SHA1
   Digest Type: 2 / SHA-256
   Digest: 3ad84adf8ac0e865a2258017ba6a6dc61dd42e45eb259baeb8072fdb0a0d7a9a
   Public Key: 
   
   AwEAAbFGliN3Z1fErY7SyuZ5sXz4Fb7zEvKxcrMAREcaYjvrBALX4ud0UxVJNo4741CL9OijS1Yc5BjOD9e2CRlTVxn7DXkNqF4XQeSbglhJikjyVTYvn/d6RzZZDDB0yjSG501U46lKi43gtl1gCy4USUgEE3kBmDVP0CjB8rT5YSd4r0IF7r/b0ntxwnAZiGtQWQznXvGtoSPUBC9X8gI/TylGYdCP6BIlkenRMlYbgqsM810kklfqcADQm7YagN1doYgxrie5GtMhaCxcOaVaY7fGJY/HyfNsC+tPU1hNk0QuOlJX7xH1/ez2PCsK9T9bSp6ZBVhfPBUaC/kGEmf5JIc=
   
   Bulk/Record Format:
   gaobo.org.	3600	IN	DS	61575 7 2 3ad84adf8ac0e865a2258017ba6a6dc61dd42e45eb259baeb8072fdb0a0d7a9a
   

autoconfig.gaobo.org
====================
✖  This domain should resolve to your box's IP address (A 152.67.67.163) if you would like the box to serve webmail or a website
   on this domain. The domain currently resolves to [Not Set] in public DNS. It may take several hours for public DNS to update 
   after a change. This problem may result from other issues listed here.

autodiscover.gaobo.org
======================
✖  This domain should resolve to your box's IP address (A 152.67.67.163) if you would like the box to serve webmail or a website
   on this domain. The domain currently resolves to [Not Set] in public DNS. It may take several hours for public DNS to update 
   after a change. This problem may result from other issues listed here.

alento · January 23, 2020, 5:25am

What is the output of this command:

sudo service nsd status

gaobo · January 23, 2020, 8:29am

It says (I rebooted the machine 6h ago so it has been up 6h since then):

nsd.service - Name Server Daemon
   Loaded: loaded (/lib/systemd/system/nsd.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2020-01-23 10:27:23 +08; 5h 59min ago
 Main PID: 1269 (nsd)
    Tasks: 3 (limit: 1107)
   CGroup: /system.slice/nsd.service
           ├─1269 /usr/sbin/nsd -d
           ├─1303 /usr/sbin/nsd -d
           └─1384 /usr/sbin/nsd -d

Jan 23 10:27:23 casino.gaobo.org systemd[1]: Started Name Server Daemon.

alento · January 23, 2020, 2:47pm

Everything SEEMS correct … but your name servers are not responding.

They are running. The port is not blocked.

I am almost ready to suggest spinning up a different box on another vps …

or, simpler, handle DNS externally with someone such as Cloudflare. I see that the mail server is functioning - I imagine you are receiving mails and can connect ok with a desktop/mobile client or via webmail with a hosts file entry.

gaobo · January 23, 2020, 4:57pm

Well, according to your description, switching to a different box or VPS won’t help much. And your suggestion about handling the DNS with Cloudflare is using the External DNS function of MiaB?

alento · January 23, 2020, 6:10pm

That is debatable. You are using Oracle. They are horrible.
I have seen similar situations where a user started over and had no problems whatsoever. However, in this case, I would do that with someone other than Oracle Cloud.

Yes. Sign up for a Cloudflare account. Remove the Glue records with Porkbun, change the name servers from ns1 and ns2.casino.gaobo.org to the ones assigned by CloudFlare and then copy all the dns entries…

SHIT!!! I just discovered your entire problem. Your name servers are pointed to porkbun’s name servers. Change them to the proper name servers! I checked this originally so I do not know how/when they changed …

but now they are showing the following:

OR … maybe you have changed to External DNS using Porkbun’s dns servers?

gaobo · January 24, 2020, 12:48am

I changed the NS records to Porkbun’s DNS servers just to verify that Porkbun’s system works properly. Because the day before only a couple of DNS servers worldwide can properly read that GAOBO.ORG’s NS servers are NS1.CASINO.GAOBO.ORG and NS2.CASINO.GAOBO.ORG and the rest of the world just read nothing out. After I noticed that the Porkbun’s system works properly I set the NS records back to NS1.CASINO.GAOBO.ORG and NS2.CASINO.GAOBO.ORG.

alento · January 24, 2020, 12:55am

My recommendations are the same …

First and foremost, I would use a better VPS provider. Or switch to External DNS.

Even though port 53 shows as it is not blocked, something is blocking connections.

You tested Porkbun’s DNS … they are just as good of an option for External DNS as any other.

gaobo · January 24, 2020, 2:37am

Even though port 53 shows as it is not blocked, something is blocking connections.

I totally agree with this. I am trying to figure out what is that something. Switching to another VPS is simple and direct, but that does not help understanding how the whole thing works. To be more specific, what on earth are the requirements that allow nsd service to run properly (which are the ports required, where I now opened 53 and 953, etc.) such knowledge would become sooner or later useful. What’s more if I am using External DNS it requires the nsd service to forward the requests thus it will still fail in the first place.

I am troubleshooting on the message which echoes when queried from outside the server (using @152.67.67.163 in dig):

connection timed out; no servers could be reached

And the message which echoes when queried from inside the server (using @localhost in dig, returns a SERVFAIL):

;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 23798
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: aae8180a078824ec025ba7335e2a5814a0dccfe091feb3c6 (good)
;; QUESTION SECTION:
;casino.gaobo.org.		IN	A

;; Query time: 7 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Jan 24 10:36:04 +08 2020
;; MSG SIZE  rcvd: 73

alento · January 24, 2020, 11:08am

I vote for the ISP’s incompetence. I had another poster here who was using the ‘recommended’ VPS provider from the install guide. He was having the exact same issue as you are. DNS did not respond. Ports were open, firewall on the OS and the ISP level were non existant. A week of back and forth with support accomplished nothing - so on a whim he destroyed and rebuilt the droplet.
Problem solved!

You may NEVER know what the issue is as the ISP will likely NOT ever admit that there is an issue - they will blame you.

So that is why I say to simply spin up another instance and use it. Otherwise you are going to waste your time and energy trying to understand something that the service provider doesn’t want you to understand.

In this specific case, port 53 UDP and TCP being open to the world.

No, that is how ‘Secondary DNS’ works … using ‘External DNS’ requires you to manually copy the necessary entries to the DNS provider. You seemingly are confusing the two … something easily done if you are not well versed in DNS.

Then you need to be looking at support resources for NSD as that is where the problem lies it seems. Try removing and reinstalling NSD just for the heck of it.

gaobo · January 24, 2020, 5:11pm

Thanks @alento for all the kind suggestions and patience with me. I am going to seriously take your advice. I’d try and let you know.