1

I am preparing a new server (v5) to replace one that I currently have in production (v4). These machines run Ubuntu. The v5 server has PHP-FPM 8.2 and Apache 2.4.52 in mpm_event mode. I recall from setting up the v4 machine that the default apache/php conf files are not well suited to the production environment because they don't spawn enough servers/processes. I want to optimize the apache and php-fpm conf files to spawn enough processes to handle traffic, but not so many that it runs out of RAM. If my server experiences too many requests, I'd rather it dropped requests than start thrashing because it's paging RAM onto the file system.

I remember getting errors in the PHP-FPM log when there were not enough PHP processes:

[22-Jan-2019 09:15:30] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 45 total children
[22-Jan-2019 09:15:31] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 47 total children
[22-Jan-2019 09:15:32] WARNING: [pool www] server reached pm.max_children setting (48), consider raising it

I ended up making some panicked adjustments to PHP-FPM's www.conf file but I want to take a more methodical approach this time. In particular, I want to configure apache and php to spawn as many processes as reasonably possible based on the amount of RAM in the machine. The free command shows roughly 8GB on the v5 server:

$ free -m
               total        used        free      shared  buff/cache   available
Mem:            7930         274        7112           5         543        7408
Swap:              0           0           0

MySQL is running on a separate machine, so it seems reasonable to me to configure the machine to use nearly all the RAM for Apache and PHP (which is all this server really does) and leave approximately 1GB for system processes.

Q1: is it reasonable to assume I can use 6930MB of RAM for PHP and Apache processes and the remaining 1GB will be sufficient for system work?

I realize that there are a lot of variables, but I think it's reasonable to assume that I'll need roughly one PHP process for each apache process. That being the case, I can assume N = apache_proc_count = php_proc_count which yields these formulae to guide my configuration:

(mem_per_apache_proc x N) + (mem_per_php_proc x N) = (total_ram - 1GB)
(mem_per_apache_proc + mem_per_php_proc) x N = 6930MB
N = 6930 / (mem_per_apache_proc + mem_per_php_proc)

Q2: Does the preceding formula seem reasonable to decide the maximum number of apache/php processes to spawn?

Finally, I'm having difficulty determining how much RAM is required by each apache and php process. For instance, for the PHP processes, I fired up an apache bench command for my workstation to make concurrent requests to the v5 server and I tried this ps command:

$ ps -C php-fpm8.2 -o "uname,ppid,pid,%cpu,pmem,rss,command,comm"
USER        PPID     PID %CPU %MEM   RSS COMMAND                     COMMAND
root           1     832  0.0  0.4 34912 php-fpm: master process (/e php-fpm8.2
www-data     832     986  0.0  0.2 23296 php-fpm: pool www           php-fpm8.2
www-data     832     987  0.0  0.2 22468 php-fpm: pool www           php-fpm8.2

According to the ps man page:

rss RSS resident set size, the non-swapped physical memory that a task has used (in kilobytes). (alias rssize, rsz).

I note that PHP pool processes use much less RAM (only ~22.8MB) than the PHP processes on the v4 server (~75.1MB).

I also ran this pmap command on my v5 server which yields a totally different number:

$ sudo pmap -x 986 | grep total
total kB          258756   23464   15384

Those columns are, respectively, Kbytes, RSS, and Dirty. I note that the RSS value is quite close to the RSS value returned by the ps command above.

Lastly, I tried this PHP command, which reports and entirely memory usage number:

$ php -r 'sleep(60);var_dump(number_format(memory_get_peak_usage(TRUE)));'
string(9) "2,097,152"

I note that the output of 2097152 bytes is quite different than my ps command above, which reports an RSS of 32416KB for this process.

I realize that there are some elaborate issues involved here: shared memory, etc., but I don't need a precise answer for an individual process. I just need to know how much RAM, on average, these processes need. my third question is Q3: How can I get a reasonably accurate idea of how much RAM the average PHP or Apache process will consume?

Lastly, I want to make adjustments to apache conf and php-fpm conf. Here are the settings in mpm_event.conf:

    StartServers             2
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadLimit          64
    ThreadsPerChild      25
    MaxRequestWorkers     150
    MaxConnectionsPerChild   0

Q4: Should I simply set MaxRequestWorkers to the value N calculated above?, or are other changes recommended?

Here is the contents of the PHP-FPM file, www.conf:

pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3

Q5: Which of these should I change so that PHP spawns as many processes as reasonably possible?

Improve this question
10
  • your log did told you what has to be raised, it also told you how man currently was used ;)
    – djdomi
    Oct 14 at 7:29
  • @djdomi Yes, the log told me to raise it. If you read my post carefully, you'll see that I want to know how much I can raise it. Also, I suspect I'll want to raise apache configuration settings in proportion with the PHP settings I'm raising.
    – S. Imp
    Oct 14 at 14:43
  • if you use php-fpm for PHP and Apache just proxies to it with mod_proxy_fcgi, the memory footprint from Apache will be very low, consider most of the ram will have to be invested in the fpm pool
    – Daniel Ferradal
    Oct 14 at 20:11
  • 1
    ...or this....lampe2e.blogspot.com/2015/03/…
    – symcbean
    Oct 15 at 0:23
  • 1
    For your Q3, consider using PS AUX for details about every running process. Add up the ones you are interested in / divide by count for average utilization ( may be close to reality ).
    – Wilson Hauck
    Nov 4 at 21:02

1 Answer 1

Reset to default
0

The memory usage of your PHP processes depends completely on the code executed by the PHP interpreters. A complete Drupal installation with tens of plugins uses much more memory than a single PHP script.

Only practical way to determine some numbers is to run load testing and measure memory usage during testing.

Improve this answer
4
  • Yes I am aware that actual RAM usage will vary depending on the nature of the PHP script. If you have suggestions about how to 'run load testing and measure memory usage during testing' then I'd appreciate those suggestions. That is basically what I'm asking in my post.
    – S. Imp
    Oct 14 at 14:47
  • The basic principle is to simulate user behavior, for example with jmeter.apache.org . Then you can use netdata.cloud for monitoring system resources. However, the topic as such is quite large, and not addressable in Q&A format here. If you encounter any specific issues with the tools, you can ask about those as new questions.
    – Tero Kilkanen
    Oct 14 at 22:01
  • 1
    surely there is some way to get an average per-process RAM consumption estimate for per-process RAM consumption without resorting to third party tools? If you read my post carefully, I have posed five specific questions, Q1-Q5, which I have indicated in bold.
    – S. Imp
    Oct 14 at 23:47
  • For Q-3 you might want to look at detail data of information_schema.processlist.
    – Wilson Hauck
    Nov 11 at 2:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .