I have now created a cname entry for the problematic subdomain that points to a working subdomain. With this setup, the home server can be reached. Although this cannot be considered a fix, it is at least a bypass. Nevertheless, I’d like to understand the cause of the problem.
If I understand the code in dns_update.py, the custom dns entries are stored in /home/user-data/dns/custom.yaml. After I created a new entry it shows up in the file and when I delete it via the web interface, the entry is gone. However, even after a restart, the subdomain is still resolved to the IP that I originally set. So the question is: Where are the custom dns entries stored in addition to the custom.yaml?