Recently, I’ve spent a huge amount of time working on Apache and PHP-FPM in order to allow for a threaded Apache MPM whilst still using PHP and virtual hosts (vhosts). As this article is going to be rather lengthy, I’m going to split it up into sections below. It’s my hope that after reading the article, you’ll be able to take advantage of Apache’s newer Event MPM and mod_proxy_fcgi in order to get the best performance out of your web server.

1. Definitions:

Before delving into the overarching problem, it might be best to have some definitions in place. If you’re new to the whole idea of web servers, programming languages used for web sites, and such, these definitions may help you become more acquainted with the problem that this article addresses. If you’re familiar with things like Apache, PHP, threading, preforking, and so on, feel free to skip this basic section.

• Web server – on a physical server, the web server is an application that actually hands out (or serves) web pages to clients as they request them from their browser. Apache and nginx are two popular web servers in the Linux / UNIX world.
• Apache MPM – a multi-processing module (or MPM) is the pluggable mechanism by which Apache binds to network ports on a server, processes incoming requests for web sites, and creates children to handle each request.
• Prefork – the older Apache MPM that isn’t threaded, and deals with each request by creating a completely separate Apache process. It is needed for programming languages and libraries that don’t deal well with threading (like some PHP modules).
• Event – a newer Apache MPM that is threaded, which allows Apache to handle more requests at a time by passing off some of the work to threads, so that the master process can work on other things.
• Virtual host (vhost) – a method of deploying multiple websites on the same physical server via the same web server application. Basically, you could have site1.com and site2.com both on the same server, and even using the same IP address.
• PHP – PHP is arguably one of the most common programming languages used for website creation. Many web content management systems (like WordPress, Joomla!, and MediaWiki) are written in PHP.
• mod_php – An Apache module for processing PHP. With this module, the PHP interpreter is essentially embedded within the Apache application. It is typically used with the Prefork MPM.
• mod_proxy_fcgi – An Apache module that allows for passing off operations to a FastCGI processor (PHP can be interpreted this way via PHP-FPM).
• PHP-FPM – Stands for PHP FastCGI Process Manager, and is exactly what it sounds like: a manager for PHP that’s being interpreted via FastCGI. Unlike mod_php, this means that PHP is being interpreted by FastCGI, and not directly within the Apache application.
• UNIX socket – a mechanism that allows two processes within the same UNIX/Linux operating system to communicate with one another.

Defining threads versus processes in detail is beyond the scope of this article, but basically, a thread is much smaller and less resource-intense than a process. There are many benefits to using threads instead of full processes for smaller tasks.

2. Introduction:

Okay, so now that we have a few operational definitions in place, let’s get to the underlying problem that this article addresses. When running one or more busy websites on a server, it’s desirable to reduce the amount of processing power and memory that is needed to serve those sites. Doing so will, consequently, allow the sites to be served more efficiently, rapidly, and to more simultaneous clients. One great way to do that is to switch from the Prefork MPM to the new Event MPM in Apache 2.4. However, that requires getting rid of mod_php and switching to a PHP-FPM backend proxied via mod_proxy_fcgi. All that sounds fine and dandy, except that there have been several problems with it in the past (such as .htaccess files not being honoured, effectively passing the processed PHP file back to Apache, and making the whole system work with virtual hosts [vhosts]). Later, I will show you a method that addresses these problems, but beforehand, it might help to see some diagrams that outline these two different MPMs and their respective connections to PHP process.

The de facto way to use PHP and Apache has been with the embedded PHP processor via mod_php:

Apache with the Prefork MPM and embedded mod_php
Click to enlarge

This new method involves proxying the PHP processing to PHP-FPM:

Apache with the Event MPM offloading to PHP-FPM via mod_proxy_fcgi
Click to enlarge

3. The Setup:

On the majority of my servers, I use Gentoo Linux, and as such, the exact file locations or methods for each task may be different on your server(s). Despite the distro-specific differences, though, the overall configuration should be mostly the same.

3a. OS and Linux kernel:

Before jumping into the package rebuilds needed to swap Apache to the Event MPM and to use PHP-FPM, a few OS/kernel-related items should be confirmed or modified: 1) epoll support, 2) the maximum number of open file descriptors, and 3) the number of backlogged sockets.

Epoll support
The Apache Event MPM and PHP-FPM can use several different event mechanisms, but on a Linux system, it is advisable to use epoll. Even though epoll has been the default since the 2.6 branch of the Linux kernel, you should still confirm that it is available on your system. That can be done with two commands (one for kernel support, and one for glibc support):

Kernel support for epoll (replace /usr/src/linux/.config with the actual location of your kernel config):
# grep -i epoll /usr/src/linux/.config
CONFIG_EPOLL=y

glibc support for epoll
# nm -D /lib/libc.so.6 | grep -i epoll
00000000000e7dc0 T epoll_create
00000000000e7df0 T epoll_create1
00000000000e7e20 T epoll_ctl
00000000000e7a40 T epoll_pwait
00000000000e7e50 T epoll_wait

Max open file descriptors
Another aspect you will need to tune based on your specific needs is the maximum number of open file descriptors. Basically, since there will be files opened for the Apache connections, the UNIX socket to PHP-FPM, and for the PHP-FPM processes themselves, it is a good idea to have a high maximum number of open files. You can check the current kernel-imposed limit by using the following command:

# cat /proc/sys/fs/file-max
3291373

Though that number is usually very high, you also need to check the limits based on user. For sake of ease, and seeing as servers running many vhosts may frequently add and modify user accounts, I generally set the max file descriptors pretty high for *all* users.

# grep nofile /etc/security/limits.conf
# - nofile - max number of open file descriptors
* soft nofile 16384
* hard nofile 16384


The specifics of modifying the limits.conf file are outside of the scope of this article, but the above changes will set the soft and hard limits to 16,384 open files for any user on the system.

Backlogged socket connections
The last OS configuration that you might want to initially configure is the maximum number of backlogged socket connections. This number is important because we will be proxying Apache to PHP-FPM via UNIX sockets, and this value sets the maximum number of connections per socket. It should be noted that this is the maximum number per socket, so setting this kernel parameter to some outlandish value doesn’t really make a lot of sense. As you will see later in this tutorial, I create a socket for each virtual host, so unless you plan on having more than 1024 simultaneous connections to PHP-FPM per site, this value is sufficient. You can change it permanently in /etc/sysctl.conf:

# grep somaxconn /etc/sysctl.conf
net.core.somaxconn = 1024

and remember to run sysctl -p thereafter in order to make the changes active.

3b. Package rebuilds:

Now that you’ve checked (and possibly modified) some OS-related configurations, a few packages needed to be recompiled in order to change from the Apache Prefork MPM to the Event MPM and to use PHP-FPM. Here’s a list of the changes that I needed to make:

• In /etc/portage/make.conf
• Change the MPM to read APACHE2_MPMS="event"
• Ensure you have at least the following APACHE2_MODULES enabled: cgid proxy proxy_fcgi
• Per the mod_cgid documentation you should enable mod_cgid and disable mod_cgi when using the Event MPM
• In /etc/portage/package.use
• Apache: www-servers/apache threads
• PHP must include at least: dev-lang/php cgi fpm threads
• Note that you do not need to have the ‘apache2’ USE flag enabled for PHP since you’re not using Apache’s PHP module
• However, you will still need to have the MIME types and the DirectoryIndex directives set (see subsection 3d below)
• Eselect for PHP must have: app-eselect/eselect-php fpm

Now that those changes have been made, and you’ve recompiled the appropriate packages (Apache, and PHP), it’s time to start configuring the packages before reloading them. In Gentoo, these changes won’t take effect until you restart the running instances. For me, that meant that I could compile everything ahead of time, and then make the configuration changes listed in the remaining 3.x subsections before restarting. That procedure allowed for very little downtime!

3c. Apache:

Within the configurations for the main Apache application, there are several changes that need to be made due to switching to a threaded version with the Event MPM. The first three bullet points for httpd.conf listed below relate to the proxy modules, and the fourth bullet point relates to the directives specific to the Event MPM. For more information about the MPM directives, see Apache’s documentation.

• In /etc/apache2/httpd.conf
• The proxy and proxy_fcgi modules need to be loaded:
  LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
• Comment out LoadModule cgi_module modules/mod_cgi.so so it does not get loaded
• Replace that with the one for mod_cgid: LoadModule cgid_module modules/mod_cgid.so
• Update the MPM portion for Event:
• <IfModule event.c>
StartServers 2
ServerLimit 16
MinSpareThreads 75
MaxSpareThreads 250
ThreadsPerChild 25
MaxRequestWorkers 400
MaxConnectionsPerChild 10000
</IfModule>

You will also need to make sure that Apache loads certain modules at runtime. In Gentoo, that configuration is done in /etc/conf.d/apache, and needs to contain at least the two modules listed below (note that your configuration will likely have other modules loaded as well):

• In /etc/conf.d/apache
• APACHE2_OPTS="-D PHP5 -D PROXY"

3d. Apache PHP module:

The last change that needs to be made for Apache is to the configuration file for how it handles PHP processing (in Gentoo, this configuration is found in /etc/apache2/modules.d/70_mod_php5.conf. I would suggest making a backup of the existing file, and making changes to a copy so that it can be easily reverted, if necessary.

70_mod_php5.conf BEFORE changes
<IfDefine PHP5>
  <IfModule !mod_php5.c>
    LoadModule php5_module modules/libphp5.so
  </IfModule>
  <FilesMatch "\.(php|php5|phtml)$">
    SetHandler application/x-httpd-php
  </FilesMatch>
  <FilesMatch "\.phps$">
    SetHandler application/x-httpd-php-source
  </FilesMatch>

  DirectoryIndex index.php index.phtml
</IfDefine>

70_mod_php5.conf AFTER changes
<IfDefine PHP5>
  ## Define FilesMatch in each vhost
  <IfModule mod_mime.c>
    AddHandler application/x-httpd-php .php .php5 .phtml
    AddHandler application/x-httpd-php-source .phps
  </IfModule>

  DirectoryIndex index.php index.phtml
</IfDefine>

Basically, you’re getting rid of the LoadModule reference for mod_php, and all the FileMatch directives. Now, if you’re not using virtual hosts, you can put the FilesMatch and Proxy directives from the vhosts section (3e) below in your main PHP module configuration. That being said, the point of this article is to set up everything for use with vhosts, and as such, the module should look like the example immediately above.

As mentioned in section 3b above, the reason you still need the 70_mod_php5.conf at all is so that you can definite the MIME types for PHP and the DirectoryIndex. If you would rather not have this file present, (since you’re not really using mod_php), you can just place these directives (the AddHandler and DirectoryIndex ones) in your main Apache configuration—your choice.

3e. vhosts:

For each vhost, I would recommend creating a user and corresponding group. How you manage local users and groups on your server is beyond the scope of this document. For this example, though, we’re going to have site1.com and site2.com (very creative, I know 😎 ). To keep it simple, the corresponding users will be ‘site1’ and ‘site2’, respectively. For each site, you will have a separate vhost configuration (and again, how you manage your vhosts is beyond the scope of this document). Within each vhost, the big change that you need to make is to add the FilesMatch directive. Here’s the generic template for the vhost configuration additions:

Generic template for vhost configs
<FilesMatch "\.php$">
  SetHandler "proxy:unix:///var/run/php-fpm/$pool.sock|fcgi://$pool/"
</FilesMatch>

In that template, you will replace $pool with the name of each PHP-FPM pool, which will be explained in the next subsection (3f). In my opinion, it is easiest to keep the name of the pool the same as the user assigned to each site, but you’re free to change those naming conventions so that they make sense to you. Based on my site1.com / site2.com example, the vhost configuration additions would be:

site1.com vhost config
<FilesMatch "\.php$">
  SetHandler "proxy:unix:///var/run/php-fpm/site1.sock|fcgi://site1/"
</FilesMatch>

site2.com vhost config
<FilesMatch "\.php$">
  SetHandler "proxy:unix:///var/run/php-fpm/site2.sock|fcgi://site2/"
</FilesMatch>

3f. PHP-FPM:

Since you re-compiled PHP with FPM support (in step 3b), you need to actually start the new process with /etc/init.d/php-fpm start and add it to the default runlevel so that it starts automatically when the server boots (note that this command varies based on your init system):

rc-update add php-fpm default (Gentoo with OpenRC)

Now it’s time to actually configure PHP-FPM via the php-fpm.conf file, which, in Gentoo, is located at /etc/php/fpm-php$VERSION/php-fpm.conf. There are two sections to this configuration file: 1) global directives, which apply to all PHP-FPM instances, and 2) pool directives, which can be changed for each pool (e.g. each vhost). For various non-mandatory options, please see the full list of PHP-FPM directives.

At the top of php-fpm.conf are the global directives, which are:
[global]
error_log = /var/log/php-fpm.log
events.mechanism = epoll
emergency_restart_threshold = 0

These directives should be fairly self-explanatory, but here’s a quick summary:

• [global] – The intro block stating that the following directives are global and not pool-specific
• error_log – The file to which PHP-FPM will log errors and other notices
• events.mechanism – What should PHP-FPM use to process events (relates to the epoll portion in subsection 3a)
• emergency_restart_threshold – How many PHP-FPM children must die improperly before it automatically restarts (0 means the threshold is disabled)

Thereafter are the pool directives, for which I follow this template, with one pool for each vhost:
;; $SITE.$TLD
[$USER]
listen = /var/run/php-fpm/$USER.sock
listen.owner = $USER
listen.group = apache
listen.mode = 0660
user = $USER
group = apache
pm = dynamic
pm.start_servers = 3
pm.max_children = 100
pm.min_spare_servers = 2
pm.max_spare_servers = 5
pm.max_requests = 10000
request_terminate_timeout = 300


It may look a bit intimidating at first, but only the $SITE, $TLD, and $USER variables change based on your vhosts and corresponding users. When the template is used for the site1.com / site2.com example, the pools look like:

;; site1.com
[site1]
listen = /var/run/php-fpm/site1.sock
listen.owner = site1
listen.group = apache
listen.mode = 0660
user = site1
group = apache
pm = dynamic
pm.start_servers = 3
pm.max_children = 100
pm.min_spare_servers = 2
pm.max_spare_servers = 5
pm.max_requests = 10000
request_terminate_timeout = 300

;; site2.com
[site2]
listen = /var/run/php-fpm/site2.sock
listen.owner = site2
listen.group = apache
listen.mode = 0660
user = site2
group = apache
pm = dynamic
pm.start_servers = 3
pm.max_children = 100
pm.min_spare_servers = 2
pm.max_spare_servers = 5
pm.max_requests = 10000
request_terminate_timeout = 300

Since the UNIX sockets are created in /var/run/php-fpm/, you need to make sure that that directory exists and is owned by root. The listen.owner and listen.group directives will make the actual UNIX socket for the pool owned by $USER:apache, and will make sure that the apache group can write to it (which is necessary).

The following directives are ones that you will want to change based on site traffic and server resources:

• pm.start_servers – The number of PHP-FPM children that should be spawned automatically
• pm.max_children – The maximum number of children allowed (connection limit)
• pm.min_spare_servers – The minimum number of spare idle PHP-FPM servers to have available
• pm.max_spare_servers – The maximum number of spare idle PHP-FPM servers to have available
• pm.max_requests – Maximum number of requests each child should handle before re-spawning
• pm.request_terminate_timeout – Maximum amount of time to process a request (similar to max_execution_time in php.ini

These directives should all be adjusted based on the needs of each site. For instance, a really busy site with many, many, simultaneous PHP connections may need 3 servers each with 100 children, whilst a low-traffic site may only only need 1 server with 10 children. The thing to remember is that those children are only active whilst actually processing PHP, which could be a very short amount of time (possibly measured in milliseconds). Contrapositively, setting the numbers too high for the server to handle (in terms of available memory and processor) will result in poor performance when sites are under load. As with anything, tuning the pool requirements will take time and analysis to get it right.

Remember to reload (or restart) PHP-FPM after making any changes to its configuration (global or pool-based):

/etc/init.d/php-fpm reload

4. Verification:

After you have configured Apache, your vhosts, PHP, PHP-FPM, and the other components mentioned throughout this article, you will need to restart them all (just for sake of cleanness). To verify that things are as they should be, you can simply browse to one of the sites configured in your vhosts and make sure that it functions as intended. You can also verify some of the individual components with the following commands:

Apache supports threads, and is using the Event MPM
# apache2 -V | grep 'Server MPM\|threaded'
Server MPM:     event
  threaded:     yes (fixed thread count)

Apache / vhost syntax
# apache2ctl configtest
 * Checking apache2 configuration ...     [ ok ]

5. Troubleshooting:

So you followed the directions here, and it didn’t go flawlessly?! Hark! 😮 In all seriousness, there are several potential points of failure for setting up Apache to communicate with PHP-FPM via mod_proxy_fcgi, but fortunately, there are some methods for troubleshooting (one of the beauties of Linux and other UNIX derivatives is logging).

By default, mod_proxy and mod_proxy_fcgi will report errors to the log that you have specified via the ErrorLog directive, either in your overall Apache configuration, or within each vhost. Those logs serve as a good starting point for tracking down problems. For instance, before I had appropriately set the UNIX socket permission options in php-fpm.conf, I noticed these errors:

[Wed Jul 08 00:03:26.717538 2015] [proxy:error] [pid 2582:tid 140159555143424] (13)Permission denied: AH02454: FCGI: attempt to connect to Unix domain socket /var/run/php-fpm/site1.sock (*) failed
[Wed Jul 08 00:03:26.717548 2015] [proxy_fcgi:error] [pid 2582:tid 140159555143424] [client 52350] AH01079: failed to make connection to backend: httpd-UDS

That error in bold text indicated that Apache didn’t have permission to write to the UNIX socket. Setting the permissions accordingly—see the listen.owner and listen.group directives portion of subsection 3f—fixed the problem.

Say that the errors don’t provide detailed enough information, though. You can set the LogLevel directive (again, either in your overall Apache configuration, or per vhost) to substantially increase the verbosity—to debug or even trace levels. You can even change the log level just for certain modules, so that you don’t get bogged down with information that is irrelevant to your particular error. For instance:

LogLevel warn proxy:debug proxy_fcgi:debug

will set the overall LogLevel to “warn,” but the LogLevel for mod_proxy and mod_proxy_fcgi to “debug,” in order to make them more verbose.

6. Conclusion:

If you’ve stuck with me throughout this gigantic post, you’re either really interested in systems engineering and the web stack, or you may have some borderline masochistic tendencies. 😛 I hope that you have found the article helpful in setting up PHP to proxy PHP interpretation to PHP-FPM via mod_proxy_fcgi, and that you’re able to see the benefits of this type of offloading. Not only does this method of processing PHP files free up resources so that Apache can serve more simultaneous connections with fewer resources, but it also more closely adheres to the UNIX philosophy of having applications perform only one task, and do so in the best way possible.

If you have any questions, comments, concerns, or suggestions, please feel free to leave a comment.

Cheers,
Zach