After fresh installs (Ubuntu 22.04.04) and restores of backups of both my MiaB servers this weekend I found that in both cases the nsd and named (bind) services are happily up and running. Though I’m pretty generally relatively clued up about DNS configurations I must admit that I never expected to see both running on the same machine so when I saw MiaB being married to nsd I went as far as seting up two additional servers hosting bind so I can do with my external DNS they way that suits me. I never even looked on the old servers to see if named was running as well.
Can someone please tell me if both nameservers running at the same time is:
a) intentional,
b) likely to persist, and
c) if a) and/or b) are true then how is the TCP & UDP traffic to port 53 on the server arranged for both packages to serve the same port?
By the NSD article on archlinux wiki it is possible but typically involves using a different port for NSD. Yet I can see that NSD on MiaB responds to port 53.
By the config it looks like bind is set up to listen only on localhost port 53 (/etc/bind/named.conf.options line 28 listen-on { 127.0.0.1; }; ) and the log file entries supports this as the only log entries are from named resolving various hostnames to ip addresses.
While I remain cautiously optimistic in the context of the diagram that BIND9 could potentially be reconfigured to perform both its current role as locally accessed recursive resolver and the purpose for which I’ve been spinning up additional VMs (actually containers at the moment) I am certain that it would be seen as too big of a change unless its done as part of a bigger project to better support external DNS configurations. such as mentioned in:
I’m sorry, but I don’t quite understand what your goal is…
NSD was added to automate and simplify the DNS configuration for the email domains hosted on Mail-in-a-Box, which is a pretty unique feature for a mail server appliance and the main reason why Mail-in-a-Box is so easy to get up and running compared to other appliances. Of course, this requires that users have the ability to manage DNS records for the domains hosted on Mail-in-a-Box if they want to use it for more than just email, which is why the Custom DNS section was implemented.
However, it seems that you want to turn MiaB into a full-fledged DNS and email hosting service provider platform, which it was never intended to be. Also, hosting DNS yourself has no advantages other than the already mentioned easier mail server configuration. In fact, rather the opposite is true for actual DNS functionality, or can a few self-hosted DNS instances e.g. provide ancyast? There are no privacy benefits either, since the DNS records are public anyway.
So if the DNS functionality provided by Mail-in-a-Box is not sufficient for you, you can simply use an external DNS provider of your choice, or if you just want to set up one or more secondary nameservers for the domains hosted on Mail-in-a-Box, you might want to take a look at my guide: Guide: How to setup NSD as a secondary nameserver for Mail-in-a-Box - #14 by miabuser
I’m sorry if that’s the impression you get, but it’s the complete opposite of what I’m after. I have a DNS provider ClouDNS and they’ve been doing a great job pretending that my domains are hosted there while in reality I keep the real master records in my own server which is closed off to everybody (DNS wise) except to ClouDNS’s own servers. It’s not a privacy thing, it’s a DDOS attack surface issue. I have no defences against any DNS based attack and don’t plan to spend any energy or resources becoming ready so I pay ClouDNS to put whatever measures into place and keep alert to every threat on my (and their own) behalf.
As I keep saying, preparing all the right DNS records for safe and effective email is a big thing and what made me choose MiaB in the first place. That MiaB chose NSD to integrate with has nothing to do with it at all. The smarts lie in knowing which DNS entries to set up for mail to flow correctly. Not everyone wants to host their own DNS servers so support for External DNS was implemented - MiaB still generates all the right DNS records, but then presents it as a listing on a web page with the option of downloading the zone file as text.
The issue I have with that is that it isn’t really a practical and reliable way to do it. All those many detailed records make it very hard and error prone to see what has changed from the last time you manually downloaded a file. If the file could be used as is, that would be not too bad, but it can’t because you’re using External DNS exactly because your DNS settings should differ ever so slightly from what MiaB prepares. Specifically in the SOA and NS records.
I am in no conceivable way saying that I want to make MiaB a better / full fledged DNS service. Not in the least. I want it to keep doing exactly what it is doing today but also make it just a little easier for those of us that want to or need to use external DNS to do so without introducing human issues. I’ve dealt with DNS for many years, against my will, so I know it is complicated topic. But the mistake many make is to conflate the server with the records. MiaB is married to NSD and that’s fine by me, but it’s not NSD that is generating the records, it’s MiaB code itself. I’ve seen the code, so I know its there. It doesn’t matter whether MiaB feeds those zone files to NSD or to something else, so I’m perfectly fine with the choice of NSD if that suits Josh for whatever reasons. In the end, it’s just a file per domain and few lines of config to set up a new domain, and the existing code is doing that well.
I’ve two objectives, both of which I’ve managed to address using a few scripts I wrote. the first is to drop the RRSIG records that’s being injected into the zones as given to NSD as signed files. I don’t want DNSSEC anywhere near me, thank you very much. That however means that if I take a feed “the DNS way” from MiaB i.e. via AXFR or IXFR then I’m going end up with all those RRSIG records and a much higher bill for hosting my zones. My second objective is to remove ns1.box.mydomain.tld and ns2.box.mydomain.tld from the list of NS records advertised for the zone. I don’t want them there because I don’t want port 53 on my servers to be open to the public and as long as those records are being advertised as name servers I’m obliged to keep those ports open.
I’m using many words to describe this because I cannot tell where your or anyone else’s level of famiiarity with DNS is. I know it doesn’t help explaining that it’s really very simple how I’m proposing we go about things, but it is. If I just said “DNSSEC and ns1/ns2.box.mydomain.tld bad, please take it away” I would get absolutely nowhere, but it really is as simple as that, plus I am saying that I’ll do it so others in the same boat as me can have the benefit as well. I have no intention of forcing this on anyone and ever less inclination to change the nature of MiaB. I understand that and why it “grabs control of DNS” and also how it goes about doing that. It’s quite fine. ut it doesn’t need to make life so difficult for someone needing or wanting to use External DNS.
Perhaps the External DNS feature was written “in concept” without a specific set of requirements from specigic users in mind. My point being that if you’re needing to use External DNS there’s some support for it but it doesn’t address the real needs of those who need to use it. Almost like “dislike our DNS approach at your own risk”. It need not be like that if we can just distinguish between form vs function. Form meaning using an integrated nameserver to directly serve all the DNS records and function meaning the compilation of the correct set of records for email to work as it should. In my book, External DNS should mean keeping the function, the actual DNS clever DNS entries, and putting those out to the public in a different way.
Am I making any sense and/or progress in explaining myself better here? I can probably guess where you’re coming from concluding that I’m trying to make MiaB into something it was never dsigned to be, but nothing can be further from the truth. It was absolutely meant to be a great email solution that takes charge also of one of the most challenging and critical parts of being a great email solution - seting up the right DNS records. It does that and I support 100% that it keeps doing that. I also understand that the choice of doing that by directly interacting with the config files and API of a specific domain server was one way to avoid a lot of complexity which really would have taken the focus away from being good at email. So I am not critisising it at all. I get it. It would be fair to say that MiaB is too much of a DNS service already. It isn’t really, or rather, it is that involved with DNS because email is very DNS-centric in the first place. But with just a little bit of modification, we can make it so much better for people who want to or need to use External DNS for whatever reasons.
Part of what I doing here is to try and get a sense of how many admins use the External DNS option, as in at all, and how many of those would appreciate the opportunity to automate the process of getting all those well-constructed DNS entries out of MiaB and into their external DNS providers’ zone files. If I’m the only user with this particular need or use-case, I’m more than happy to keep the solution I’ve already made running forever. But if there are others like me, do they not deserve the benefit of open source as well?
I encourage you to do it. Anybody who us code literate can make a New Pull Request. If it is not merged with the official updates, so be it, but eventually everybody else interested will benefit from an unmerged version.
I think its really great that your interested in improving MIAB. I get that you would be looking to see if something is welcomed before doing it and wasting time, energy, and potentially money. Having that conversation might be something best let with talking to Josh directly. Maybe Slack might be better for this?
That’s correct. bind is bound to localhost and nsd is bound to the public interface, both on the same port. The hard separation between the local-only resolving/recursive nameserver and the public non-recursive nameserver is the right architecture to avoid configuration mistakes, I think.
I probably would not accept a PR that adds a user-controllable option somewhere, but what I would accept is that the zonefile generation would silently skip DNSSEC if you manually delete $STORAGE_ROOT/dns/dnssec (and manually modify setup/dns.sh so that it doesn’t regenerate it). That way you can easily disable DNSSEC by trashing the keys.
I would accept a change that reads a key from a new dns/options.yaml, which says per zone if external DNS is used. (That is, external DNS could be activated for each domain separately.) And if so, it can drop the NS records, and the status checks could be modified. But I would really want to see this properly documented in appropriate places. In general, I should really push contributors to make sure their contributions are documented.
I am happy to reply where the conversation is taking place. I don’t read everything everywhere so you can @-mention me to get my attention if I don’t chime in (though I don’t promise I’ll always respond, depending on how busy life is and how important the topic is).
However, what I see as well is that many of the records shown on the External DNS page or in the zone files are not needed if you are using external DNS. So I guess that part could be improved. Whether the export of the records should or can be further automated, I don’t know. I think this is probably beyond the scope of a project like Mail in a Box.
However, since Josh has expressed what changes he would accept in the meantime, I guess there’s no need to continue our discussion at this point
I wouldn’t say many are not needed, but some for sure. Which ones do you consider potentially spurious?
By far the most pressing point is the SOA record which should not nominate ns1.box.yourmiabdomain…tld as authoritative but one of the most external name servers.
The second change is remove the NS records pointing at ns1.box or ns2.box.yourdomain.tld from all externally hosted domain zone files.
The third change I’d have to verify against a completely new installation is whether or not there are A records for box.firstdomain.tld in the zone file for firstdomain.tld or whether I added those as Custom DNS records. The point being that the box.firstdomain.tld zone isn’t (in my case) a zone that goes to my DNS provider but my users must be able to access box.mydomain.tld in order to get web mail and for me to do admin, so it must resolve from somewhere.
In addition to the ones you mentioned there are many records in there, that are supposed to protect the subdomains on MaiB from which you don’t send emails:
Example:
autodiscover IN TXT "v=spf1 -all"
autodiscover IN MX 0 .
Whether you want to to use them or not is up to you, but be aware, that they only really make sense if you set them up for all your (sub)domains you’re not sending mail from, also the ones not hosted on MiaB. And you have to create one TXT and one MX 0 record for each subdomain. Personally I don’t think it’s worth the effort, especially if you have many subdomains and/or if you often add/remove subdomains.
In addition to that, MaiB also has set separate_dmarc records for each of those subdomains, but unlike with SPF and MX records, you could also combine them into one by using the “sp” tag, which specifies what the policy should be used for all subdomains of a domain.
Example:
_dmarc IN TXT "v=DMARC1; p=quarantine;sp=reject"
By the way, setting dmarc policies for (sub)domains that you don’t send email only makes sense if you’ve also have set the SPF records for those (sub)domains. Otherwise, you can just set one SPF and dmarc record per (sub)domain you’re sending email from.
That speaks to the very heart of the matter for me. If you let it, MiaB will ensure that all your domain’s email related DNS entries are complete and up to code. It’s only when you’re manually having to transfer all the records it generates to an external DNS provider that you need to consider if its worth doing or not. We don’t want to make it too easy for spam providers to spin up short lived MiaB servers and integrate them into the mail ecosystem with good reputations so they can enough emails before they’re torn down again and resurrected with a new identity. It’s already a bit too easy to create a new email domain but at least they can be identified as such by the pattern MiaB uses for its SOA and NS entries. It should at least take a bit of effort to set up an “undetectable” MiaB server (at the DNS level) fronted by an exernal DNS provider. Real email domains come up and stay up for a long time. The DNS records to control their use will change continuously in the context of these modern SPF and DMARC conventions and that part should be automatic. Registering a new domain and setting it up should in my vew require care, attention and a few deliberate actions that’s harder to automate.
I guess what I mean to say is that I don’t consider these records to be spurious. I prefer them to be there consistently and reliably but I don’t want to have to do manual work on a regular basis to make sure they are. Since I also don’t want my MiaB server to be reachable on the DNS ports except from the five addresses my DNS provider requests transfers from, my approach is to let MiaB generate the records as per usual and then automate the propagation of those records to my External DNS provider out of band (i.e. specifically not using the DNS zone transfer mechanism where the MiaB controlled NSD hosted zone is the master).
Well, what you are describing is not exactly “external DNS”. It’s more like a “hidden primary” setup, which has already been discussed at length here.
Also, if you don’t want to use the DNS zone transfer mechanism, you would have to implement scripts that can interact with all the different APIs of all the well-known and lesser-known DNS providers, similar to what acme.sh does, which would be a hell of a lot of work.
The only (and critical) difference between External DNS and a hidden primary boils down to whether or not ns1.box.yourdomain.tld is designated as authoritative for the zone in the SOA record. I’m still trying to get to the bottom of all the actual use-cases I can find, but in the most general of terms based on the use-cases I am familiar with to date, the process of manually transferring the dns records to an External DNS provider’s zone files usually modifies the SOA record. When you host a domain at a DNS provider they take charge of the SOA and NS records for that domain, you don’t get to overwrite it even if you do specify SOA and NS records in the file you give them. Most I’ve see would not accept such files if they contain SOA and NS records but I imagine some could just silently ignore them. Those with API interfaces have very different ones for setting up a new zone including its glue records and those you’d use to CRUD other types of records.
Whether they’re consciously aware of it or not, anyone using actual External DNS (i.e. domains hosted at some DNS provider) ends up with the SOA and NS records their DNS provider chooses, not what MiaB’s exported zone file contains. It’s only not the case when the External DNS you use isn’t at some public DNS provider but also self-hosted somewhere external to box.yourdomain.tld. In the latter case you’d be in charge of setting up the SOA and NS records for the domains you host yourself and the choice to keep or change ns1.box.yourdomain.tld as authoritative or not is your own. You’d gain virtually nothing if you do choose to keep it as these external domains would serve as secondaries in which case you’re better off simply declaring them as aditional nameservers in Custom DNS and let the standard propagation update your servers. Like I said, the benefit would be marginal. You only get real benefit when your choices end up replacing ns1.box.yourdomain.tld as authoritative nameserver in your effective SOA. You could argue that it amounts to the same thing as a hidden primary. I never disputed that. But my initial attempt to broach the subject from that perspective was a spectacular failure. DNS can feel overwhelmingly complex to the point where most would not want to touch it for fear of everything that depends on it breaking badly. And sure, if you don’t know what you’re doing it certainly is a distinct possibility. Which is why all my efforts and proposed solutions have been trying to make a very clear distinction between the individual DNS entries and the arrangement of servers serving up those to the world.
I’m trying my best to keep MiaB in full control of the individual DNS records that needs to be present to make email work smoothly and safely. As more use cases emerge to add to my own we’ll settle on what the real commonalities are, but so far it would appear that thre are people who have to use External DNS, as in having their domains hosted somewhere else and feeding what MiaB says it needs into that. The need to use it for a variety of reasons I’m still trying to get to know well enough to make the right design decisions.
As it stands, people who use External DNS are having to manually update their domain records at their DNS service provicer. I’m aiming for a way to make that automatic, reliable and quick.
Indeed, I’ve made the same argument all along. I’ve even suggested that it’s the primary reason why Josh chose to rather set up and manage his own name service to be authoritative than having to get into that abyss of all those different api’s. Needless to say, I don’t want to get into that at all. It’s just not worth it.
Luckily for us there are ways available to us where we can automate an out-of-band domain update/transfer without having to be concerned with all the various DNS provider’s APIs. Depending on the level of buy-in we can get from Josh the options to consider might need to include something external to MiaB or could be handled entirely within MiaB. By far the least impactful way to do it, is to keep MiaB’s NDS-based DNS in place but allow an option to make that a hidden primary. Josh turned that down outright and you were very much part of that discussion. But all is not lost. We can stil make it work in other ways even if we can’t change his mind. Like I keep saying, I have already implemented it for myself without changing MiaB in any way. It’s a fast and stable solution but specific to my environment because it’s dirt cheap for me to run bind in two containers on my own infrastructure which I can transfer the zones to using scp, manipulate the text using awk, and reload the zones because those servers are the ones registered witth my DNS provider as primaries. From that point forward it’s all happening purely in the DNS domain, no APIs and other out of band nonsense.
My existing solution is unlikely to be viable to many others because of the skill, knowledge and cost involved in hiring and setting up additional virtual machines. That’s why I’ve offered to find ways to achieve the same thing in a way that works within the confines of Mail-in-a-Box so it can benefit others as well.
But perhaps my fault is that I’m not enough of a businessman. A seasoned serial-entrepreneur would see the opportunity to sell a service to MiaB users needing to feed external DNS providers with the records MiaB produces when you can’t have or don’t want MiaB’s DNS server to be open to the public. That not my usual approach to life but perhaps that’s a better approach. I’ll give it some thought.
I’m not sure if there’s much of a business in this, since for most people, email records or even other public DNS records they might have for a website or a few self-hosted services don’t tend to change very often.
Also, most companies I know, even larger ones, usually only have one public domain name that they actively use, and maybe a few dozen public DNS records, and their internal DNS is usually handled by Microsoft Active Directory and managed with tools like Infobloxs.
And of course, none of them are using Mail-in-a-Box, if they are still self-hosting their email at all.
Or to put it another way, from a Mail-in-a-Box perspective, there are two ways to handle DNS:
You can use MiaB as your primary authoritative nameserver and add one or more secondary servers if your TLD requires it and/or if you want geo-redundancy, and then manage everything through the MiaB WebUI or API.
You can use an external DNS provider (or set up your own authoritative nameservers), then manually add all the DNS records needed for email there, and use their management tools and APIs for your other DNS needs.
Of course, if you need to constantly add and remove email domains, the second option is not really feasible. But that’s a niche use case within a niche, and it almost sounds to me like you’re using MiaB to provide email services at scale, which seems beyond its intended purpose.
No, I’d rather die. I’ve come accross forum members who’re in that game despite pretences to the contrary, but I’m with you. MiaB is for people who have bigger or at least other fish to fry than worry if their email service are running fine or not.
So between option 1)
and option 2)
I choose 3), I am compelled to use an external DNS provider but want the benefit of MiaB looking after my DNS records and manage the few custom entries I need via the MiaB WebUI.
I don’t constantly add and remove email domains, and I agree that those who do are probably up to no good.
However, DNSSEC and the daily cron job to regenerate keys which results in very frequent updates may have mislead me about how often other keys generated by MiaB rotates as well. When I had MiaB feeding my external provider directly, my zone file sequence numbers updated every day. No way am I going to spend time and energy every single day figuring out what changes and which of those changes do I need to feed through to my external provider. Yet I can’t send the downloaded data through as is because they contain data that isn’t aligned with my needs.
I’ve repeatedly stated that I have no interest in turning on DNSSEC, but the encouragement by MiaB to do so is undeniable to we must assume that many will do that. For those who do, having an external DNS provider must be a genuine issue. They’d have no way to tell whether or not something changed in the records that must go to the provider because the serial changed so they’d have to download every zone every day and compare it to the previous day, pick up on any changes and go make them on the external DNS providers interface. It almost sounds to me like you’re in favour of deliberately making life difficult for anyone who dares to need or prefer external DNS.
My campaign, if you want to call it that, is to see if there are others like myself who want the best of both worlds, because it’s possible. But if you’d rather keep it the binary choice you describe we’re talking around the real problem and the debate will get nowhere.
Unless I’m completely wrong, MaiB doesn’t do key rollovers, so you should be fine.
And this is also why I’m a bit more relaxed about this hole domain/dns security thing than you are, and why I don’t, for example, set `v=spf1 -all’ records for every subdomain I use elsewhere. Nobody does that, and nobody except maybe the big guys like Cloudflare and Google get DNSSEC and DKIM, SPF, DMARC and MTA-STS right, let alone key rollovers, which would be a part of getting it right, probably the most important part.
Just recently, a 10+ year old OpenSSL bug made the news again because major email providers used that OpenSSL version to generate DKIM keys back then and have never changed the keys since.
In my opinion, these security measures that have been glued on top of email and DNS are fundamentally flawed, partly because of the technical debt and the decentralised nature of the underlying technologies, but mainly because it’s impractical to use them “right” unless you’re Google or Cloudflare.
