Php-fpm %100 CPU usage

Hey everyone, I was getting %100 CPU usage on my Mail-in-a-box server a few minutes ago.
I checked htop and it seems like “php-fpm: pool www” is responsible for this and was doing this for an hour or so, I rebooted the server and everything worked fine again.

Can someone tell me why was that happening? Did I get owned?

For how long? Probably there is running some task. If it lasts less then a few minutes, this shouldn’t be a problem.

You can find more data in /var/log/nginx.error or nginx.access for the last requests.

There are also some logfiles in /var/log/php7.2-fpm.log*

For MiaB, the two most important processes running php are next cloud and the webmail client

I don’t know how long it lasted.

I checked the logs of both nginx and php7.2-fpm.log, nothing seemed to have caused this issue. I found a few log entries where someone tried invalid addressed in the website (presumably to attack it), but I don’t think that’s related.

Should I be concerned?

When running the latest versions. Not immediately. But keep an eye on it.

Can you elaborate further? What should I keep an eye on?

If it happens more and if you see a lot of strange connections (logfile), focussed on nextcloud/webmail

So ignore all the wordpress requests and other queries.

If you are on the latest version, and you use decent passwords,

If you have a small userbase, you can do some simple crosschecks.

If you grep on the nginx log file for example for the mail

grep “/mail/” /var/log/nginx/access.log | cut -f1 -d " " | sort -u

a list of different ip addresses is generated.
If you have for example 5 users and you see hundreds of IP addresses, some investigation is needed.

The same for:
grep “/cloud/” /var/log/nginx/access.log | cut -f1 -d " " | sort -u

The “other” requests are visible by gripping on the logfile:

cat /var/log/nginx/access.log | grep -v “/cloud/” | grep -v “/mail/” | less

You’ll probably see a lot of wordpress/phpmyadmin related strange queries and the queries for your websites. But that’s not causing any high load of php-fpm (except if you’re running this kind of sites under the same uid

The issue just happened again right now, what do I do?

@joshdata
Any idea of what could be the reason?

when this happens, how long does it hit 100% usage for, if it’s just a few minutes I’d say it’s normal.

Can you rebuild the server? if you believe someone else is logged into it doing stuff this is the safest course of action.

But what should I do differently this time to prevent this thing from happening in the future?

I disabled SSH password and root login, and I have changed its port.
I’m using RSA 4096 bit keys as well.

I’m thinking that Nextcloud is causing the problem here, since php-fpm seems to get installed because of Nextcloud (according to the MiaB scripts).

Is there a way to install MiaB without Nextcloud? I’m willing to rebuild the server so that’s not a problem.

IMO, Nextcloud carries more harm than good and it should be removed from MiaB since it adds extra attack surface to the server. MiaB should focus on mail and nothing else.

If @joshdata needs any help in modifying the scripts to make Nextcloud optional, I’d be happy to help.

I think it would be pretty easy to implement, the script should just ask the user if he wants to install Nextcloud, and sets an environment variable accordingly.
Every action related to Nextcloud in the scripts will check for that environment variable, and if it says “NO” (or something like that) it would just skip that action.

I will start writing a pull request for this.

Just finished making the pull request, hope it gets merged.
(This is my first pull request ever!)

2 Likes

try "strace -p "; on a high load, you will probably see what the process is handling.

try "lsof | grep "

maybe, you get more info, which is causing this.