I’m running my MIB on an Amazon EC2 T2.micro instance.
The instance comes with 1vCPU and 1GB RAM.
The OS is Ubuntu 18.04.6 LTS, as per tutorials
The MIB is the latest build, and the OS has all updates from default channels.
My MIB setup has 2 domains, both set up on Gandi.net, just as in the tutorials.
The only ‘extra’ piece I have is an A record inside Custom DNS, pointing to a chat server I have on another machine.
Those two domains are new, purchased 2 weeks ago, and I have only 4 mailboxes for the test, with very little activity, because I’m the only one testing them.
All my MIB System Indicators on the Admin page are green, with the exception of the ‘system memory’.
Upon reboot, the System memory is green, at 27% free.
After about 4 hours, it goes down to 24%
By the next day is on 9%, eventually, going down to 2% and at some point later the MIB doesn’t allow connections to the Admin or Mail areas.
The server can be pinged, but external DNS queries start failing.
Im not being attacked on the external IP, the connections to the box can be summarized like this:
I think somehow a process is not releasing memory. It was my understanding that a 1GB RAM instance would be enough, especially if Im just testing the software…
I don’t have experience with AWS, but search for the various terms and you will find people here reporting issues and generally the resolution seems to be to have a solid understanding of how AWS works, because MiaB, by default, doesn’t seem to play well with AWS, meaning the other solution is to use a different ISP.
AWS EC2 is just a VM running Ubuntu, just as Digital Ocean, or any other cloud provider.
I clarified this here to save time because I might be asked on which hardware Im running MiaB
Yes, understood. But as @openletter stated, people have had nothing but problems running MiaB on their network. So it must be something specifically that AWS does that other providers do not do. What though, remains a mystery.
Just for the record, the MiaB issues reported on AWS are related to DNS records, VPC configurations, Lighsail settings, and general AWS firewall settings.
Most of the issues I read is because AWS limitations regarding DNS on Route 53 or because users do not know how components on AWS stack connect to one another.
I’m not using Route53 on this setup, all records are managed inside the MiaB and the glue records are on Gandi, as per tutorials.
I’m well versed in AWS concepts, VPC, DNS entries, PTR records, gateways, Elastic IPs, Security Groups, AWS firewalls, Elastic IP configurations, etc. And I use all those technologies all the time and I’m capable of troubleshooting AWS setups end to end, as I’ve using AWS in production for more than 10 years.
Now, when it comes to how MiaB is set up internally and how to troubleshoot its functionality, Im not good, as I discovered this project 2 weeks ago and I just set up my first server. I guess I might be able to figure out the issue, but Im trying to save some time, perhaps someone has the know-how already.
I enormously appreciate all the efforts from all of you thought.
Many thanks.
Reading your OP a bit more carefully … every MiaB I have running USES between 25-40% , not have that amount free.
My initial thought is that something is just not right with your vm at AWS. it is rare but it happens. If your hesitant to use a different provider, which is understandable, then my first trouble shooting would be to completely tear it all down and start over. New VM, new install, so on. For as I mentioned, the issues you are having just don’t seem to happen.
I would speculate that 1GB is the largest percentage of MiaB installs, simply because most people using the project are small users and that for some time now has been a very common small VPS size.
There are users reporting running MiaB on 512MB servers for years.
IIRC, memory issues are reported very infrequently, maybe a few times a year?
That’s a good suggestion, I will deploy a new VM at AWS with MiaB and bring a backup there and see what happens. I dont think the issue is the Elastic IP or firewall rules, so I will reattach those on the new VM.
I also have an Azure environment where I can spin VMs. I might try that afterward. That will take a little more because I would have to point the Glue records to the new Azure IP.
I will report back this weekend, but if someone suggests commands to type on my instance before I shut it down, Ill be happy to do so.
One thing I failed to say in my opening post:
When I first set up this MiaB, I had only one domain, and I didnt notice memory issues with it.
After a week, a added the second domain (basically purchased another one on Gandi, changed its DNS there to use the MiaB box, added a mailbox in MiaB and generated all the SSL certificates, etc, as per the instructions on MiaB web ‘maintenance’)
Thats when I started seeing the issues with the memory.
Sorry I didnt say that at the beginning, it might be helpful.
I’m not familiar with Azure, but these large companies seem like they want to force some secret sauce into everything. Based on original documentation and who used what back when MiaB launched, using the smaller vanilla ISPs like Linode, Vultr, and Digital Ocean (they can have severe IP address reputation issues, YMMV), etc., may be a better way to test.
If I were to guess, I would guess that the majority of MiaB instances serve only one domain. A very, very slim majority. That said however, I would not expect, nor have I seen the simple act of adding a domain cause this type of memory issue.
Now, if you had 20 email users on each domain each connecting to webmail concurrently … then maybe we’d have something there, but every indication from you that is not the case.
If you are US or Canada based and you want to try another provider, let me introduce LunaNode. I have spun up several MiaB instances with them with zero issues. I can also offer a coupon code which would make the experimentation free.
20-And-20 Affiliate Program
Our 20-And-20 affiliate program gives both you and the referred user $20 in free credit; unused credit expires after 60 days. This program is available for users in Canada and the United States only.
Tell your friends who haven’t used our platform before to sign up for an account and enter the promotion code below at Billing → Promotions.
They must verify a Canadian or U.S. credit card to validate their account. One individual may only use this promotion once, and only if they do not already have an existing active account.
Once the credit card is verified, $20 in free credit will be added to their account. You will receive $20 as well.
Both their unused free credit and your unused free credit will expire after 60 days.
For multiple referrals, you will receive $20 for each referral. This is promotional credit and cannot be exchanged for money.
I just did it.
" This promotion has been applied successfully to your account!
Then we both get the $20.
I opened the LunaNode account, even though I was not thrilled by the fact that they require a real cellphone for SMS validation (not VoIP which I prefer) and a real CC (not a burner CC service, which shields you from unexpected charges) . Even AWS and Azure let you use VoIP for SMS and burner CCs.
Thanks for the referral code.
It seems that most of the memory in my machine is used by php-fpm.
There were a number of processes created even when the box was not under load, and those were grabbing memory apparently without releasing it.
I did some reading online and found this link:
That file www.conf was located at /etc/php/7.2/fpm/pool.d/ on my box.
I played with some values, and ended up with these:
pm = ondemand
pm.max_children=50
pm.process_idle_timeout = 10s;
Rebooted the server, and waited.
I did that yesterday night, and so far the box says “System memory is 30% free” (sometimes 28% sometimes 32%) but doesnt seem to go down anymore and it has not crashed.
30% is what I start with right after reboot, so I would assume staying around that value is what would be expected.
I know, it is not recommended to manually alter those values as the MiaB upgrade process might bring them back to defaults, but since this is just a proof of concept box, it was worth a try.
Perhaps there is some benefit to changing the PHP process manager from ‘auto’ to some predetermined value, and the MiaB developers might consider adding that to the deployment script.
I will report back in a few days and see if that actually helped.
For the record, I would like to mention that I have used AWS for two years with MIAB. No problems. I initially used Lightsail but later moved to an EC2 instance just because I was using EC2 for other projects. Using an EC2 instance, AWS now permits the user to change the reverse DNS for any instance which is convenient for MIAB users.
How is your email deliverability with EC2? My emails were being received by Gmail, but on outlook.com, live.com, and msn.com were put on spam, and on O365 they were going to Quarantine. My DNS was the box itself and all my records were valid, according to MiaB dashboard. I was using an Elastic IP attached to the instance.
I moved my install to Lunanode and it became worst.
Finally, I configured the MiaB to relay messages via SendGrid, and things improved (at least on O365 recipients), but outlook.com, live and msn are all putting me in the Spam folder.
I did some research and some Spam tracker organization have IPs from SendGrid blacklisted, because some people had abused it, so I imagine my situation might be resolved if I find a better SMTP relay.
Do you use any that is good?
Ouch! I am sorry to hear that. I had usually had good luck with their IP’s, but it becomes harder and harder to maintain good reputations with everyone wanting to send spam.
But my main focus is the topic of this thread … are you experiencing issues such as you had on EC2, or have those issues been resolved?
After the changes I made on the www.conf file, the server was behaving much better and the memory didn’t go down anymore. I wouldn’t call this a ‘solution’ but a workaround. Hopefully, the devs will bring light to this, incorporate those changes on the next version, or at least allow that user to change those settings without being overwritten upon updates.
Perhaps we should ‘close’ this issue for now.
