I wander if it is a firewall problem or the redirect? But how did you originally provision the certificates at first install then?
This quote might point to a firewall problem or a false negative.
On the other hand how did you provision the certificates in the first place? Was the redirect on?
Might be a better idea to redirect via nginx: read more here: Redirect webmail.mydomain.example to box.mydomain.example/mail
An DO run:
sudo ~/mailinabox/tools/web_update
.
Make some new entries in custom dns see of they propagate. For e.g. some random TXT record.
It all looks strange to me.