Hi all,
Configuring email service to work safely, effectively and reliably involves tight control over a myriad of non-intuitive DNS entries. Having run email services on other platforms I can safely what I most appreciate about MiaB is that it generates and checks that myriad of DNS entries. To effect those DNS entries the MiaB code is tightly integrated with the domain name service NSD4 which it sets up and expects to see running as authorative for the box.yourdomain.tld as well as all the domains the box runs email for. With so many DNS service providers to cater for, that strategy is by far the simplest to implement, so I fully understand why it was chosen.
That being said, MiaB does make provision for using an external DNS supplier. It works just fine but has two consequences:
-
It makes some of the system checks non-deterministic - i.e. saying “this isn’t as it’s supposed to be, but it might be OK if you using external DNS”.
-
It places a high burden on the user who has to export zone fies every time a change is made, compare that with what has been presented to the external DNS provider and apply the changes manually.
I’ve been running in that mode for a nuber of months. I even wrote a few python scripts to do manipulate the downloaded zone file or the generated .txt file for it with the changes I require for my external DNS provider.
Over this past few days I was forced by external circumstances to move my MiaB servers from ESXi to Proxmox. For my external DNS requirements I’ve been running two additional virtual machines purely as BIND9 based domain name servers backing each other up as blind masters feeding my external DNS provider which by public records (registrar) are authorative for my domains. Moving my MiaB servers also meant moving those servers.
After the first round of moving the machines was done (once I learned how and when to restore the backup) there was so much time left in the maintenance window I negotiated with my users that I felt it might be a good time to try something different. Seeing that I’ve never experienced MiaB in its intended mode where it sets up the box as a serving authoritative DNS server exposed to the public, I dedided to give that a go and see how it goes.
Naturally it was an easier setup to do as it didn’t involve external DNS servers or my BIND9 based VMs, so I was rather pleased with it from that perspective. I declared my DNS provider as secondary in the prescribed manner and was in the process of confirming that the DNS changes I made was getting through fast and reliably.
That’s when disaster struck. I subscribe to a package at my DNS provider which allows me specific limits, none of which I’ve ever been in danger of exceeding. While on my summary page at my DNS provider, I noticed that my number of records now exced my allowed limit and to my horror it was still rising.
I jumped on their support channel and we found the problem. Though I haven’t and never intend to activate DNSSEC, the DNS records being sent from MiaB’s NSD service were all signed records anyway and the DNS provider counts each of the signatures (RRSIG) as its own record. Like I said, I’ve looked into DNSSEC before, weighed the pros and cons and ultimately decided I better steer clear of it. (If you must know why, they short version is that DNSSEC can and often does fail because of expired or unavailable keys that should be reotated regularly for it to be of any real value. The counter for the failures is active monitoring. Service providers will sell you monitoring and automatically rotating keys, but at a substantial cost. One could argue that email is already bound to DNS and therefore able to tolerate temporary failures, but when your web service is also impacted it’s a completely different scenario. DNSSEC failures puts egg on web sites’ faces.)
Faced with having to upgrade my DNS package (for purposes I don’t actually use) or find a way to get rid of the extra records, I had to make a choice. I quick look at the MiaB DNS code confirmed that it’s unlikely that I’d be able to get MiaB to recognise that I don’t want DNSSEC and stop producing signed records at all in the time available to me.
So I reverted back to having my two BIND9 based DNS servers up and running but didn’t want to let go of the benefit of automatic updates from the files MiaB generates. I then wrote a fews lines of bash script using the incron daemon and scp to detect changes to the /etc/nsd/zone directory, pick out the *.txt files only, transfer those to the bind based servers and kick off a zone reload on there.
The solution I came up with in a hurry (by the time I saw the unexpectedly high record count on the DNS provider’s site I only had a few hours left before my maintenance window closed) works like clockwork. Well enough to have spurred the idea of writing up a post about it that could get tagged into the guide category in due course.
But there’s issues my solution doesn’t resolve, which makes me hesitant and eager to explore other alternative first.
- The zone files generated still lists ns1.box.mydomain.tld and ns2.box.mydomain.tld as NS records. It includes the specified secondary name servers, but won’t let go of including those.
- The secondary name servers are getting notified of zone changes. this is correct behaviour for “true secondary” servers but defeats the purpose I’m chasing. Luckily my DNS provider lets me specify which servers to accept transfers from so by excluding the actual box.mydomain.tld IPs from that list (limiting it only to the two BIND9 based DNS servers’ IPs) I was able to dodge that bullet at the expense of errors in syslog as NSD reports that its attempted zone transfers were refused.
- I found out that as it stands, MiaB runs both NSD and BIND9 already. Knowing a fair bit about configuring both by now I feel it’s entirely possible to configure them so that a separate BIND9 based DNS server isn’t needed, which would make the solution a lot more accessible to others using external DNS.
My previous request for supporting blind DNS masters in MiaB was turned down, and in effect this is heading towards a second version of that. But the emphasis and target audience has moved slightly from specifically supporting BLIND masters to improved support for External DNS with selective out-of-band automated transfer of the zone file and definitive control over what NS records gets published. That way the status checks can drop the “if you are using external DNS this might be OK” uncertainty and perform the correct checks given the known external DNS configuration.
In technical terms what I suggest we look into is to include a dns/external.yaml file in the distribution similar to the dns/custom.yaml file with a get_external_dns_config function to read its contents into memory. Functions like check_primary_hostname_dns can then check if a domain is sepecified as external and if it is, adapt its checks to confirm that that it is correct instead of assuming it should onfirm that ns1.box.mydomain.tld resolves and is set as nameserver for the domain. The idea would be for the new external DNS config to name user supplied script file which transfers the required zone files from /etc/nsd/zones to wherever it needs to go and trigger the required reload.
As I wrote the above and read code to confirm names etc. github my eyes fell on the has_dnssec checks being done in check_primary_hostname_dns function and I wonder this: Why can do_dns_update in update_dns.py not do the same check and if dense is not used, skip all the signing and then let write_nsd_conf know to specify the serveddomain.tld.txt file for the zone rather than the serveddomain.tld.txt.signed file? It wouldn’t address ALL the issues but it seems the logical thing to do. On that note, I’m not convinced that checking if the DS record is set or not is the ideal test to see if DNSSEC is enabled or not becaue it’s self-referencing in a way. If makes the assumption that DNSSEC should be activated and reports that it isn’t, but there’s not a way to say that it should not be activated. I’d prefer a check-box for a domain stating if DNSSEC is being used or not. If it’s off but the DS record is ON it can be reported as a fault, and similarly if it’s OFF but thre is no DS record that can be reported as a fault (rather than the current warning) with instructions as to how to get the DS record set.
Anyway, I’m happy and even eager to do one and only one of three things. Either:
- develop a guide based on how I implemented automated updates of external DNS from MiaB’s generated zone files,
- help code a PR which introduces explicit external DNS settings, or
- refrain from stepping on any toes with my specific requirements and suggestions.
It all depends on your feedback, so let’s hear it.
If you’ve grown up being better at expressing yourself in programming languages than English like me it’s the most natural thing in the world to take that “modify a few things” bit and do it automatically in whatever programming tool gets the job done the easiest whether it’s sed or awk or even python. I need my mind, fingers and time for other things, so I’d rather spend a little more time on the solution when I have it all within frame to put a permanent automated solution in place which I can then just monitor and largely forget about until it fails than having to keep that entire domain of intricacies in active memory all year long. I’m responsible for a very complex software solution and simply can’t spare the mind space to keep all I need to remember about email’s DNS settings in mind as well.