Debugging and Fixing Memory Leaks in Apache/PHP Web Apps: A cURL SSL Case Study


3 views

When your EC2 instances periodically freeze with Munin graphs showing Apache processes consuming increasing memory until swap exhaustion, you're likely dealing with a classic memory leak. The key indicators:


# Sample munin memory graph pattern:
1. Steady baseline memory usage
2. Sudden upward trend in "apps" memory
3. Continuous growth until swap fills
4. System becomes unresponsive

Before diving deep, establish these monitoring basics:


# Install essential tools
sudo apt-get install strace htop

# Monitor Apache processes
watch -n 1 "ps aux | grep apache | sort -nk 4"

# Track memory allocation
sudo strace -p [PID] -tt -o trace.log -s 256 -e brk

The breakthrough came when correlating strace output with PHP execution points:


// Add debug points in PHP
error_log("MEMCHECK: " . shell_exec("ps -p " . getmypid() . " -o rss="));

// Sample strace analysis:
$ grep -A 5 "write(2, \"MEMCHECK" trace.log
$ grep -B 10 "brk(0x" trace.log | tail -20

The smoking gun appeared in the SSL handshake:


// Problematic code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://api.example.com");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch); // Leaks ~400KB per call
curl_close($ch);

// Working alternative:
curl_setopt($ch, CURLOPT_URL, "http://api.example.com"); // No leak

These config changes bought time for proper fixes:


# Apache config (prefork)

    StartServers        5
    MinSpareServers     5
    MaxSpareServers    10
    MaxClients        150
    MaxRequestsPerChild 50  # Critical for leak containment

The complete resolution involved multiple steps:


# Upgrade libcurl (Ubuntu example)
sudo apt-get install libcurl4-openssl-dev

# Verify version
php -r "echo curl_version()['version'];"

// PHP code improvement
$ch = curl_init();
if (strpos($url, 'https') === 0) {
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // Temporary workaround
}
// Always clean up
register_shutdown_function(function() use ($ch) {
    @curl_close($ch);
});

Implement these checks to catch future leaks early:


# Cron job to monitor Apache memory
*/5 * * * * /usr/bin/php /var/www/scripts/memcheck.php

# memcheck.php content:
$threshold = 500; // MB
$memory = shell_exec("ps -C apache2 -o rss= | awk '{sum+=$1} END {print sum/1024}'");
if ($memory > $threshold) {
    mail('admin@example.com', 'Memory Alert', "Apache using {$memory}MB");
}

When dealing with memory leaks in Apache/PHP environments, the first challenge is identifying the pattern. In my case, EC2 instances became unresponsive periodically, with Munin graphs showing:

  • Gradual memory consumption growth in "apps" category
  • Apache processes consuming memory without releasing it
  • Complete swap exhaustion leading to system crashes

Before jumping to conclusions about PHP or Apache being the culprit, I implemented this diagnostic strategy:

# Monitor Apache processes sorted by memory usage
watch -n 5 "ps -ylC apache2 --sort:rss | head -20"

The breakthrough came when using strace to track memory allocations at the system call level:

# Attaching to a suspicious Apache process
strace -p PID -tt -o trace.log -s 256

Key findings in the output:

  • Repeated brk() calls showing heap expansion
  • 400KB allocations occurring per request
  • Correlation with specific PHP function calls

Through systematic tracing, the leak was traced to cURL's SSL handling:

// Problematic code pattern
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://example.com");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch); // Memory leak here
curl_close($ch);

While working on a permanent fix, these measures helped stabilize the system:

# Apache configuration adjustments
<IfModule prefork.c>
  StartServers        5
  MinSpareServers     5
  MaxSpareServers     10
  MaxClients          50
  MaxRequestsPerChild 100 # Forces process recycling
</IfModule>

The complete resolution involved multiple steps:

  1. Upgrading cURL to version 7.19.5+ (fixed known SSL leaks)
  2. Implementing proper resource cleanup:
    // Safe cURL usage pattern
    $ch = curl_init();
    try {
        // ... configuration ...
        $response = curl_exec($ch);
        if ($response === false) {
            throw new RuntimeException(curl_error($ch));
        }
        return $response;
    } finally {
        curl_close($ch);
        unset($ch); // Explicit cleanup
    }
  3. Adding memory monitoring to critical operations