I need to figure out how many MiaB users actually use the External DNS facility and how their needs ties up with mine, if at all.
I both want to and have to use External DNS but MiaB’s support for it isn’t ideal for me. The workaround I’ve written for myself is too complicated for general use so I’ve offered to make whatever improvements I can based on what actual users of the feature need.
So if you either:
Have to use External DNS for any reason,
Want to use External DNS for any reason,
Already use External DNS but would like to automate the process,
Would like a way to completely opt out of using DNSSEC, or
Understand what a blind master is in DNS terms and seek a way to make MiaB a blind master to a commercial DNS hosting provider,
then please raise your hand (i.e. hit Reply and tell me) so we can start discussing what would best address your needs.
Update: 24 May 2024,
If there there’s a user out there in MiaB land that is using External DNS at the moment for watever reasons and up for a bit of fun, please contact me privately. I’m willing to set up your domains to receive your zone files in a way that suits us both and set up my server to it can be declared as master nameservers at your external DNS supplier. We’d do this setup manually this time, but if it works and there appears to be others who’d use it, I’d set up a web-based signup process whereby MiaB users can come in and do the setup any time that suits them. Ialways try to avoid automating tasks that doesn’t exist yet, so this would be a way to get familiar with what needs to happen to make my bind servers available to others in a simiilar boat than me.
Don’t use DNSSEC, might consider using this in the future, not sure what’s involved.
Would almost like to see a Export File (like MIAB does now) that could be more directly imported into cloudflare but guessing that this won’t be a popular. IMO people are just using MIAB for DNS or manually copying records…
Let’s not guess. The point of this post is to confirm or refute that opinion.
The way I have it set up at the moment is fully automated. As soon as I make any change in MiaB that impacts DNS, or MiaB decides on its own to change or update some records, my external DNS feed kicks in automatically and the changes are propagated to the external DNS provider and onto the public within minutes.
Like I suspect it would be for many, administrating an email solution is neither my day job nor my hobby. I do it because it has to be done and I don’t have anyone to delegate it to. The less time it takes out of may day to confirm that all my users’ mail is running OK the better, and when I need to add an account or domain, or make a change to a Custom DNS record for a non-static website not hosted on the box itself, then it suits me best if I can still do that in one place without having to remember several other steps to go perform manually. The exact flavour may change since not many people use MiaB as email solution for non-static web-sites that verifies users by email, but I’d be a little surprised if I’m exactly the only one who would benefit from an automated updating of External DNS servers, whichever way we end up implementing it.
Thanks @JoshData, now we’re cooking. It’s not how I’d have thought to do it but if that works for you then I’m happy to go that way. We’d thrash out the details some more, but would you agree that in principle we “define” that the presense or absense of $STORAGE_ROOT/dns/dnssec is treated as the definitive flag for whether the user has opted out of DNSSEC or not. I.e. If the directory is present, dns.sh wouldn’t create it but pupulate it. If it IS present which it is by default, then it gets populated and the DNS zones gets signed etc. Would that still be in line with you believe you can manage in terms of user education and expectations?
Naturally the appropriate documentation would be essential. That is, if we go ahead with this at all, which is still an open question to me seeing that I’ve not heard back from an overwhelming horde of people who’d like to use it. But rest assured, if I do find motive to make the changes discussed keeping you happy to accept it into the code is my only objective. I don’t have space in my life to maintain my own fork of MiaB.
@MarthinL
Please explain it to me like to a 5 year old. I am willing to use ExternalDNS but my question is why?
Is it because it has faster or more robust propagation?
Or is it more secure?
The internet is like a school full of bullies - they ignore most of the kids but once they notice you, you’re defnitely going to get attention you didn’t want. Some even look for weak kids to attack all day, every day.
My primary business is a site that will attract both good and bad people. The good people will want to use it for their own benefit but the bad people will want to make sure they stay in control.That’s much is a given, but I’m nowhere near ready to defend against the bullies on their own turf, so I’ve hired some tough guys to protect me. The bullies don’t see me running around school, they only see my bodyguards but I need a way to talk to them so they know how to answer questions on my behalf.
Some classes like my “web service” class are my I must attend myself, because that’s my turf and worth the effort and risk to protect myself when I go to those classes.
But the email class is like gym class - not safe space for me. The email space has some tricky questions where I need Mail-in-a-Box’s help to know what the right answers are.
So what I do is I let MiaB prepare all the answers in advance for me. Then I feed those through to my bodyguards so they can give those to the world out there on my behalf. The problem I’ve had though was that it took a lot of my time to take MiaB’s answers and convert them into what the bodyguards needs plus it was just too easy to make mitakes that could undo all the hard work. I made a better plan so the bodyguards can automatically get the information in the way they need it without me doing manual work, but it’s not something I’d even want to teach someone else to also do. It only works for me.
But I don’t mind sharing with others. It’s just that to make it so others can also use it, I have to do it a little differently and get MiaB people involved as well so we can make sure it works for everyone that needs to or want to have the same type of arrangement with bodyguards of their own.
So in a few years time when you’re all grown up I’ll upgrade the story and talk directly about DNS providers, Masters, Slaves, APIs, zone files, SOA records, NS records and the IXFR/AXFR protocol between DNS servers.
It’s neither more nor less secure, fast or robust. It’s more like choosing your battles when you realise you can’t protect yourself on all fronts equally well. I’ve had first hand experiences, very recently in fact, with just how quickly the baddies out there scanning the internet picked up on an oversight by my ISPs engineers which meant they could start using my computers to launch DDOS attacks for those who pay them for doing it. I told the story in another post and got flamed for it because “these things don’t happen” but they do. It’s just an example though and not why I choose to run an External DNS at all. I choose it because I know that once my web service launches I will get targeted and I cannot afford to get distracted by having to guard against DNS based attacks as well. It’s going to take all my wits to stay safe on the web front.
I hope that answers your questions. I wasn’t trying to use condescending language, as a form of malicious compliance. I was just having a bit of fun genuinely describing it as one would for a five-year-old.
People who’re “willing to use it” are not who I’m looking for or who I’d be keen to share my solution with. If you suffer no ill side effects from using MiaB’s way of serving DNS records itself, they stick with that. If you have a legitimate reason to prefer External DNS you’re most likely to already be very much aware of what it is and why you prefer it or need it. At the very least, it’s additional costs in one way or another, either for an external DNS vendor or for additional VMs hosted somewhere or both so it’s never going to be cheaper or easier.
I tried hard to avoid pushing my use case and requirements onto others. Not hard enough it seems because at least one person saw it as me trrying to convince others to use it for reasons they don’t understand. I’d be happy as a clam if it turns out I’m the only one who needs an automated Exernal DNS feed that strips DNSSEC in the process. Happy that I’ve done my open source duty by offering it to others who need it as well. I’m not looking to “convert” anyone to using it if they’re not already forced or drawn into using the current implementation of External DNS. I just want to make it easier for those who are compelled for their own set of reasons to use an External DNS to take the manual work they’re already having to do and automate it in a reliable manner.
I also use external DNS. I have to admit, the reason was entirely that copying the DNS records from the admin panel over to my host/ registrar seemed easier than finding a solution for a secondary DNS server at the time, which was just a test to begin with anyway. I was intending to get around to looking into doing it with MAIB and getting a secondary DNS solution of some sort, but obviously never did, and as nothing has gone wrong so far, I have been blissfully ignoring it ever since.
I don’t use DNSSEC. It’s not that I wanted to “opt out of it” so much, but more that my host didn’t support it, and see above about blissfully ignoring things that don’t seem to be going wrong. I also have to admit, I still don’t really understand what it’s for.
Finally, I should say, I don’t have the ability to help in anyway, but would be interested in what you come up with, if you peruse it.
Well, well, that’s a use case I haven’t considered and I want to know all about it. Here’s what I’m hearing. Your MiaB is being hosted at a company that also acts as registrar for DNS zones, probably not as a shared server but as a dedicated server which is actually a VM on their big hardware. But they also provide a firewall service which they control tightly to for safety’s sake, so even if you have port DNS ports open on your box’s firewall the outside world can’t reach it because your host/registrar blocks that traffic. In its place they give you a place where you can paste the zone records for the domain you bought from (through) them which they then load into the DNS servers they control themselves. So you could not comply with MiaB’s preference to run DNS servers exposed directly to the public even if you wanted to. Your hosting company made that impossible for you. As a “work-around” you downloaded the zone records from the External DNS page and pasted that into your host company’s interface for updating DNS records, and left it at that.
How’m I doing? Feel free to correct me where I got it wrong. I’m not interested in being right, but in knowing as much as possible about your situation because I’ve not considered it before. I’d have thought that companies who’d let you rent a place you can run your own emaill server would be in the cloud PaaS type business while the provider you seem to describe also appears to be selling email services themselves and would not be keen on customers running their own email servers inside their walled garden.
What made you choose that service hosting company/registrar and what lead to your decision to run a MiaB instance rather than rent mailboxes from them directly?
About the direction my solution is taking, it basically aims at automating that process you did one time and left it static after that so that any time a change to your DNS records is required either by something you add or a new best practice that gets taken on board, the records MiaB prepares with such care and attention gets pushed into your service providers’s DNS database (with modifications to remove the unwanted references to ns1.bos.yourdomain.tld) without you having to copy and paste. There are a number of ways to achieve this still up for debate and which one gets implemented if any will depend on us having a clear understanding of what follks like you who in essense have no choice than to use External DNS needs from it.
As for helping, don’t worry. Too many cooks will spoil the broth. Being a user that’s willing to discuss how you go about using MiaB with External DNS with a real world problem to solve and a willingness to try out what we come up with, that is a great big help already. Thank you for speaking up.
I use GoDaddy, I pay the fee for hosted DNS. I used to run my own DNS Servers, but got tired of doing it. I have a Business Internet connection with a block of 13 IP addresses. I use the GoDaddy Control Panel to enter the DNS records for each domain, and their DNS Servers send the records out to the world.
I run three VM’s of MIAB, one for each email domain, on a Dell R730xd running TrueNAS at my location. Each instance has its own Public IP and set of DNS records. I used the MIAB External DNS listing and entered all the necessary records. They have not changed since entry. I do not use DNSSEC.
DNS resolution on my network is provided by the pfSense routers. There are no custom DNS entries, it just pulls from the core DNS servers to provided local service.
Your use case is different in some ways but mostly similar to mine. The biggest difference I can see is that you’ve never needed to repeat the original process of downloading the zone file and uploading it to GoDaddy. You’ve chosen to make GoDaddy your primary / authoritative service but they are equally willing to serve as secondary / slaves.
It sounds like the domains you run are purely for email purposes with perhaps a static web-page for each, so very much in line with the general MiaB use case, except that you’ve chosen to use GoDaddy instead of letting the world get through your pfSense firewall on UDP/TCP port 53.
Since you’re not using DNSSEC either, it’s quite likely based on your description that your zone files havent changed since you installed. A way to check this is to click the Download button on the External DNS page and check the serial number of the SOA record which is based on date. If the date in the serial number still corresponds with when you initially set of the domain and copied the records across to GoDaddy, then you’ll have no benefit from what I am proposing. If the date in the serial is more recent than that, you’d likely benefit from what I’m proposing. That would be because according to MiaB’s records you zone has been modified but those changes have not been passed on to GoDaddy.
You’re welcome to let me know what you found when you did the check. I’m rather serious about wanting an accurate picture of people’s needs, practices and experiences with using an External DNS.
Thanks for your interest in trying to make improvements.
For my use cases, I use both internal and external DNS.
My main domain uses the internal DNS. I went this route to learn how MAIB implemented it, and since this is my first use on MIAB, I wanted to keep it as default as possible. But I did setup a second VPS (Oracle Free) with NSD as a secondary DNS server. I have to manually update the config for new domains in the secondary NSD. The any zone updates are automatic to the secondary.
I also have external DNS for some secondary domains that dont change much. For these I just copy/paste into Cloudflare.
In my perfect world, there would be somewhere in the DNS Settings in MIAB that I input my Cloudflare API key and it just syncs with and works directly with CF.
This would have the advantage of multiple DNS servers by default and all the other Cloudflare protections possible on the websites. (DDOS protection, caching, etc)
The Primary and Secondary Nameservers are provided by GoDaddy as part of the paid DNS Service. I don’t have to bother with Zone Files or syncing between servers. I don’t use the MIAB DNS. I used the External DNS link from the Admin Page and entered those DNS Records in the Control Panel for my DNS Service. Those records have not changed since MIAB was setup to service email for the domains. Unless I have an IP change, none of those DNS records should need any updating. If a DNS record does need changed or updated, I just enter it in the DNS Control Panel for the domain and everything else is taken care of for me.
Why would you have to do that? If the zone transfers are working properly, you don’t need to do anything on the secondary NSD. And for the domains you’re hosting at Cloudflare or any other external DNS provider, Mail-in-a-Box and your secondary NSD are completely out of the DNS game anyways.
As @Mr_Bill already said, all you have to do is enter all mail-relevant records from Mail-in-a-Box’s External DNS page into Cloudflare’s DNS control panel once, and you’re done. From then on, you’ll manage all DNS for these domains exclusively at Cloudflare.
And of course, the secondary NSD you set up on the Oracle Free tier is no longer needed for these domains either, because you will be using the Cloudflare ayncast infrastructure, which provides extremely fast worldwide DNS propagation and geo-redundancy that a single secondary can never compete with.
Sure, that would be convenient, but since it’s mainly a one-time thing to manually add all mail-related DNS records shown on MiaB’s External DNS page to an external DNS provider’s control panel, it would be hard to justify the effort required to integrate and maintain all those APIs from different DNS providers.
The only other advantage I could see in implementing this is that the APIs could also be used for zone transfers to secondary nameservers for domains hosted on MaiB, which could in theory (depending on how secure the respective API is) be somewhat more secure than using unsigned zone transfers.
That beeing said, zone transfers can be further secured by using Transaction Signature (TSIG), but MaiB doesn’t provide any GUI options to do so. Perhaps this would be a more useful feature request, and certainly easier to implement and maintain, than having to support who knows how many third-party APIs that could change at any time.
You’re pretty close. We started with a VM, but now we need a few, so it’s more economical to use a dedicated server running Proxmox.
As for the firewall blocking the ports, I don’t think that’s the case, but then I haven’t tried either. I am sure I overestimated the difficulty of setting up a secondary DNS, but at the time I understood relatively well what DNS records were (for a layman), and I realized that if I just let the mail server do it itself, if ever it went down that could be a pain because email servers are likely to retry sending emails to servers that have mx records but are currently unavailable, but generally won’t retry when there is no valid mx record to tell them where to send email to (which would happen if our server went down). That required a solution, and since I knew right away how to copy paste the records, but didn’t know immediately how I would go about finding an alternative DNS server, it just seemed easier to do it that way.
What made you choose that service hosting company/registrar and what lead to your decision to run a MiaB instance rather than rent mailboxes from them directly?
We use OVH, because they have a fairly affordable data center in Quebec and I wanted to stay in Canada (which is where we are from) just because every now and again there is a big political fuss about where companies keep Canadian’s data, so I wanted to stay within Canada just in case some government came along and made some new law which would oblige me to move or start worrying about things I didn’t want to worry about.
As for the reason, I like the idea of controlling it myself, but mostly because I thought it would be interesting/ worth learning.
I use several secondary DNS servers, because my tld demands at least two nameservers for my domain, and because it provides redundancy. I like that this is automated in Mail-in-a-Box. I also like that this works with DNSSEC without additional effort.
In the past I have also run this setup with MiaB DNS server acting as a blind master, which also worked fine. This did require some coding changes. I did this just to see if I could get it to work.
I see no reason to use External DNS. This would only mean more work for me. I now have a single source of DNS entries (my MiaB box) and automatic distribution to the world (my secondary DNS servers)
@sander-schippers To put your mind at east, changing mailserver users can only impact on the DNS records when you add one for a domain you’ve not set up to to be the mail server for before. It doesn’t seem to be your case though.
But that sounds a little odd. You don’t change users all that often but you have a template for when you add a domain? Do you add new domains often? Why? In the use-case I’m formulating I’m taking the stance that adding domains is the one activity that should require some effort (in the hope that it makes MiaB less attractive to those seeking to deply it for short lived domains with the nerfarious intent of spamming people from a source that’s long gone by the time it gets listed).
@asm0dey Thanks for the response. Why would you like to use external DNS? I cannot verify that I have the right requirements in mind until I have some insight into whether you want to or need to use external DNS, whether you’d be better served by having MiaB thinking it’s master and staying in control of the records while the zone records propagate in-band or out-of-band to external DNS, etc. Are you currenty DNSSEC enabled and/or do you hope to enable it in future? I’m very happy you responded, but I’m going to need a little more detail if you please.
@KiekerJan I get that. I recall your blind master attempt which wasn’t bad, it just didn’t or wouldn’t make it into the main branch, and who has time to keep updating an unsupported branch, right?
My view on the general External DNS use case(s) is evolving. That was the whole point of this exercise - not going off on a tangent to implement something until I have adjusted my understanding to line up all the flies I can swat with one blow.
I’m gunning for the same thing, only without a) being forced to name ns1.box.mydomain.tld in the SOA record or registering a glue record for it at my registrar, and b) being subjected to the one downside of allowing MiaB to take full control of mastering the zone which is that it would sign the entries anyway whether you have (or want) DNSSEC enabled or not.
The latter is a problem for me because it quadruples the number of records my DNS provider gets to serve and wants to charge me more for those records I have no use for. Also, when MiaB is fully in control it will habitually resign the records which is a minimum requirement if you are indeed running DNSSEC but if you don’t it causes a great many unnecessary zone updates.
Overall
My understanding of the use-case around External DNS is still growing and I will allow it to keep growing until it stabilises.
At this point though it is looking like those using External DNS at the moment do so because their MiaB servers run in hosted environments that prefer not to allow TCP/UDP port 53 traffic through to the guest VMs so the outside world cannot reach MiaB’s DNS servers.
It also seems like those running in environments requiring external DNS have such static DNS setups that they only ever download the external DNS zone file from MiaB once during initial setup and hardly ever touch it again.
Which as it stands means my needs seems unique in that because I actively develop web services whose development and deployment involve the same domains as I am seving email for, I am required to make changes to the Custom DNS records more often than most. My current solution is to have automated an out-of-band propagation mechanism that keeps MiaB thinking it’s master but by the time the DNS records reaches my External DNS provider in automated fashion, there ns1.box.mydomain.tld is no longer involved in any way. The way I’ve implemented it alo allowed me to change the source of the records I pass on to my external DNS to the domain.tld.txt file (as opposed to the domain.tld.txt.signed files which are updated daily) so not only do I not get the plethora of additional RRSig records I don’t want, my zone updates are also reduced to only happen when I make actual changes.
I’ll still be keeping this survey open another while - can’t expect MiaB users to often have reasons to visit this forum so it might take some time. But unless in that time it emerges that there are indeed others whose needs overlap with mine it seems a waste to reimplement my solution to suit the MiaB code.
You asked for it.