Apache MaxClients Bottleneck: Solving Server Lockup Under Maximum Concurrent Connections in MPM-Prefork


2 views

During recent load testing on my OpenVZ VPS (512MB dedicated/1GB burst RAM, no swap), I encountered a perplexing Apache behavior when hitting MaxClients limit. Here's what happened:

# Server configuration
ServerLimit 25
MaxClients 25
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxRequestsPerChild 4000

Using ab (Apache Benchmark) to stress test a WordPress installation:

ab -n 1000 -c 24 http://example.com/  # Works fine
ab -n 1000 -c 25 http://example.com/  # Causes complete lockup

The server would:

  • Accept initial connections
  • Display all 25 processes in 'W' (Waiting) state
  • Become completely unresponsive until TimeOut (45s) expired
  • Show "server reached MaxClients setting" in error logs

This behavior stems from several compounding factors:

# What 'ps aux' reveals during lockup:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
www-data  1234  0.0  4.5  24000 23000 ?        S    14:20   0:00 /usr/sbin/apache2 -k start
www-data  1235  0.0  4.5  24000 23000 ?        S    14:20   0:00 /usr/sbin/apache2 -k start
[... 25 identical processes ...]

Key observations:

  1. No connection queue: Prefork MPM lacks proper connection queueing
  2. Memory pressure: No swap space means OOM killer may intervene
  3. PHP blocking: mod_php executes synchronously

Here are concrete steps to resolve this:

# 1. Adjust Apache configuration
MaxClients 20  # Leave breathing room
ServerLimit 20

# 2. Implement KeepAlive optimization
KeepAlive On
KeepAliveTimeout 2
MaxKeepAliveRequests 100

# 3. Alternative MPM configuration
<IfModule mpm_event_module>
    StartServers 2
    MinSpareThreads 25
    MaxSpareThreads 75
    ThreadLimit 64
    ThreadsPerChild 25
    MaxRequestWorkers 150
    MaxConnectionsPerChild 0
</IfModule>

For production environments:

# Install and configure mod_cgi or php-fpm
# Example php-fpm pool configuration
[www]
user = www-data
group = www-data
listen = /run/php/php7.4-fpm.sock
pm = dynamic
pm.max_children = 20
pm.start_servers = 4
pm.min_spare_servers = 2
pm.max_spare_servers = 6

Additional recommendations:

  • Implement reverse proxy caching (Varnish/Nginx)
  • Set up monitoring with alert thresholds at 80% of MaxClients
  • Consider moving to worker/event MPM if PHP application allows

When lockup occurs, gather diagnostic data:

# Check current Apache status
apachectl fullstatus

# Check system resource usage
top -c -b -n 1 | head -20
free -m

# Check for blocked processes
strace -p $(pgrep -d, apache2) -f 2>&1 | grep -i 'blocked'

When Apache with MPM-prefork configuration hits its MaxClients limit, we're observing complete server unresponsiveness rather than graceful request queueing. This behavior manifests as:

# Typical symptoms seen in server-status
PID   State     CPU   SS  Req  Conn  Child   Slot  Client        Request
1234  W         0.3   0   1    0.0   0.23    1     192.168.1.1  GET /wp-admin HTTP/1.1
1235  W         0.2   0   1    0.0   0.22    2     192.168.1.1  GET /wp-admin HTTP/1.1
... (25 similar processes)

Under healthy conditions, Apache should:

  • Queue excess connections when MaxClients is reached
  • Maintain existing connections without freezing
  • Gradually recover as processes complete

The OpenVZ environment with 512MB guaranteed RAM creates special challenges:

# Memory calculation breakdown
Available RAM:        512MB
Apache process size:  23MB
MaxClients set to:    25 (23MB × 25 = 575MB)
Memory headroom:      -63MB (into burstable territory)

Based on production experience with similar setups, try these adjustments:

# In httpd.conf or apache2.conf
StartServers            5
MinSpareServers         5
MaxSpareServers         10
MaxClients              20  # Reduced from 25
MaxRequestsPerChild     1000

# Critical timeouts
Timeout                30  # Reduced from 45
KeepAliveTimeout       5

For memory-constrained environments:

# Switching to MPM-event with PHP-FPM (example for Debian)
sudo apt-get install apache2-mpm-event libapache2-mod-fcgid php-fpm
sudo a2dismod mpm_prefork
sudo a2enmod mpm_event proxy_fcgi

Implement this watchdog script to prevent complete lockups:

#!/bin/bash
MAX_WAIT=30
APACHE_STATUS=$(systemctl is-active apache2)

if [[ "$APACHE_STATUS" == "active" ]]; then
    LOAD=$(w | grep -o "load average: .*" | cut -d' ' -f3 | tr -d ',')
    if (( $(echo "$LOAD > 5.0" | bc -l) )); then
        echo "$(date) - High load detected, restarting Apache" >> /var/log/apache_watchdog.log
        systemctl restart apache2
    fi
fi