Debugging and Fixing PHP-CGI Memory Leak Issues in Lighttpd on Low-Memory VPS


4 views

When running PHP applications through FastCGI on lighttpd, it's not uncommon to observe gradual memory consumption increases in php-cgi processes. This becomes particularly problematic on memory-constrained VPS environments (like your 360MB setup). Let's analyze why this happens and how to properly address it.

Your setup shows several interesting characteristics:

fastcgi.server = ( ".php" =>
  ((
    "bin-path" => "/usr/bin/php-cgi",
    "socket" => "/tmp/php.socket",
    "max-procs" => 2,
    "idle-timeout" => 20,
    "bin-environment" => (
      "PHP_FCGI_CHILDREN" => "4",
      "PHP_FCGI_MAX_REQUESTS" => "1000"
    ),
    "broken-scriptfilename" => "enable" 
  ))
)

The key parameters here are:

  • max-procs: 2 - Creates 2 main PHP processes
  • PHP_FCGI_CHILDREN: 4 - Each main process spawns 4 children
  • PHP_FCGI_MAX_REQUESTS: 1000 - Each child handles 1000 requests before recycling

PHP processes naturally grow over time due to:

  1. Memory fragmentation in the PHP heap
  2. Extension memory leaks (even small ones accumulate)
  3. Opcode caches like XCache retaining compiled scripts

Your observation that processes grow even without traffic suggests either:

  • Background processes hitting PHP scripts
  • Opcode cache maintenance operations
  • Extension initialization overhead

1. Tune FastCGI Process Management

Modify your lighttpd configuration:

fastcgi.server = ( ".php" =>
  ((
    "bin-path" => "/usr/bin/php-cgi",
    "socket" => "/tmp/php.socket",
    "max-procs" => 1,  # Reduced from 2
    "idle-timeout" => 10,  # Reduced from 20
    "bin-environment" => (
      "PHP_FCGI_CHILDREN" => "3",  # Reduced from 4
      "PHP_FCGI_MAX_REQUESTS" => "500"  # Reduced from 1000
    ),
    "broken-scriptfilename" => "enable" 
  ))
)

2. Implement PHP Memory Management

Create a php.ini configuration file specifically for FastCGI:

[PHP]
memory_limit = 24M  # Reduced from 32M
max_execution_time = 30
max_input_time = 30

; Disable problematic extensions if not needed
; extension=xyz.so

; XCache specific settings
xcache.size = 8M  # Reduced from 24M
xcache.var_size = 0
xcache.gc_interval = 300

3. Implement Process Monitoring

Create a monitoring script (monitor_php.sh):

#!/bin/bash
while true; do
  DATE=$(date)
  MEM=$(ps -C php-cgi -o rss= | awk '{s+=$1}END{print s/1024}')
  echo "[$DATE] PHP-CGI Memory Usage: $MEM MB"
  if (( $(echo "$MEM > 250" | bc -l) )); then
    echo "Memory threshold exceeded - restarting lighttpd"
    /etc/init.d/lighttpd restart
  fi
  sleep 60
done

To identify memory leaks:

  1. Install php5-dev and compile PHP with --enable-debug
  2. Use valgrind to track allocations:
    valgrind --leak-check=full --show-reachable=yes /usr/bin/php-cgi -n test.php
  3. Check for extension issues by running PHP with -n (no config) and gradually enabling extensions

For memory-constrained environments, consider:

  • Switching to PHP-FPM (more efficient process manager)
  • Using mod_php if you can switch to Apache
  • Implementing a cron job to restart PHP-CGI periodically

When monitoring PHP-CGI processes in a Lighttpd/Drupal stack, I observed consistent memory growth even during idle periods. The baseline memory usage appears normal (20KB-8MB per request), but processes balloon to 28MB+ over time. This occurs despite:

  • Low traffic conditions (firewall-protected environment)
  • Proper PHP_FCGI_MAX_REQUESTS cycling (set to 1000)
  • Conservative XCache configuration (24MB total)

The system reports conflicting memory metrics:

# Process-level reporting
ps -C php-cgi -o rss= | awk '{s+=$1}END{print s/1024}'
→ 195.738MB

# System-wide reporting  
free -m
→ 127MB used (after buffers/cache)

This 68MB gap suggests either shared memory isn't being accounted for properly or kernel memory accounting differs from process-level reporting.

The FastCGI setup shows several optimization opportunities:

fastcgi.server = ( ".php" =>
  ((
    "bin-path" => "/usr/bin/php-cgi",
    "socket" => "/tmp/php.socket",
    "max-procs" => 2,               # Consider reducing
    "idle-timeout" => 20,           # Possibly too aggressive
    "bin-environment" => (
      "PHP_FCGI_CHILDREN" => "4",   # High for 360MB RAM
      "PHP_FCGI_MAX_REQUESTS" => "1000" # Could be lower
    ),
    "broken-scriptfilename" => "enable" 
  ))
)

For Drupal 7 (common in Lenny-era deployments), implement these in settings.php:

// Reduce persistent caches
$conf['cache_default_class'] = 'DrupalDatabaseCache';
$conf['cache_backends'] = array();
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';

// Limit views caching
$conf['views_skip_cache'] = TRUE;

Key directives for memory-constrained systems:

; /etc/php5/cgi/php.ini
memory_limit = 24M           ; Down from 32M
max_execution_time = 30      ; Prefer lower for FCGI
realpath_cache_size = 128k   ; Down from default 16M
opcache.enable = 0           ; When using XCache
xcache.size = 16M            ; Further reduction

Implement rolling restart via cron (every 15 minutes):

#!/bin/bash
# /etc/cron.d/phpcgi_restart
*/15 * * * * root /usr/bin/find /var/run/lighttpd/ -name "php.socket.*" -mmin +15 -exec touch {} \;

This triggers Lighttpd's socket revalidation without service interruption.

Consider PHP-FPM as more memory-stable alternative. Example config:

[www]
user = www-data
group = www-data
listen = /var/run/php5-fpm.sock
pm = dynamic
pm.max_children = 6          # Reduced from FCGI model
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.max_requests = 500        # More aggressive recycling

Use this script to track per-process memory trends:

#!/bin/bash
watch -n 60 'ps -eo pid,user,args,rss --sort -rss | grep php-cgi | 
  awk '\''{printf "%d\t%s\t%s\t%.1fMB\n",$1,$2,$3,$4/1024}'\'' |
  head -20'