Mail-in-a-Box will be the only DNS server you will need to configure. Just use the guide[0] to point your Google Domain to your Box on Linode and setup rDNS on Linode.
From Linode: I think you’re combining steps where you shouldn’t:https://mailinabox.email/guide.html Note that it says “If you are using an external DNS provider, e.g. if you already have a website on your domain name, you may skip this section.” If you’re going to use Linode’s DNS service, then don’t do the nameserver bits - it’s going to make this more complicated for you. If you’re not going to use Linode’s DNS service, then you don’t need to setup anything in the Linode DNS Manager. You’d handle all the DNS records in Google’s control panel.
But Mail in a Box Status Check says Your box’s reverse DNS is currently liXXX-XXX.members.linode.com (IPv4) and [Not Set] (IPv6), but it should be box.example.com. Your ISP or cloud provider will have instructions on setting up reverse DNS for your box.
And the ISP/Cloud Provider, Linode, seems to require DNS records for reverse DNS
Following Linode - Reverse DNS on https://www.linode.com/docs/networking/dns/setting-reverse-dns
Gives me the following error message:
• No match was found for ‘mail.example.com’. Reverse DNS must have a matching forward entry that points to one of your IPs.
A matching forward entry that points to one of my IPs is, per Linode is A matching forward entry would come in the form of an A or AAAA record created in the zone file for the domain, pointing towards an IP address, i.e. an A record for domain.com pointed towards 8.8.8.8 (where domain.com is your real domain name and 8.8.8.8 is your Linode’s IP address).
Once you have created an A or AAAA record for your Linode’s IP in the Google Domains control panel (and once that change has reached the rest of the internet) you’ll be able to set Reverse DNS in the Linode’s Dashboard under the Remote Access tab.