From your monitoring data showing 44 busy workers and 12 idle workers during peak load, we can derive several important insights:
Active Workers: 44
Idle Workers: 12
Total Workers: 56
Memory Usage: ~232MB for 40 Apache processes
Available RAM: 97GB free of 128GB
The segmentation faults (signal 11) typically indicate:
- Memory corruption in PHP extensions
- Buggy Apache modules
- Incorrect MaxClients/ServerLimit settings
For a memory-optimized prefork configuration, we need to calculate based on:
# Formula to calculate MaxClients:
MaxClients = Total Available RAM / Average Apache Process Size
# In your case with current metrics:
Average Process Size = 232MB / 40 = ~5.8MB
Theoretical MaxClients = 97GB / 5.8MB ≈ 17100
# But we need safety buffer:
Recommended MaxClients = (Total RAM - System Reserve) / Max Process Size
= (128GB - 32GB) / (8MB worst case) ≈ 12000
Your current ServerLimit (3200) is too conservative for 128GB RAM. Try this optimized configuration:
StartServers 5
MinSpareServers 10
MaxSpareServers 20
ServerLimit 8000
MaxClients 6000
MaxRequestsPerChild 10000
To diagnose and fix the segfaults:
- Enable core dumps:
ulimit -c unlimited echo "/tmp/core.%e.%p" > /proc/sys/kernel/core_pattern
- Analyze crashes:
gdb /usr/sbin/apache2 /tmp/core.apache2.13595 bt full
- Common fixes:
- Disable suspicious modules:
a2dismod php5_filter cache_disk
- Update all components:
apt-get install --only-upgrade apache2 libapache2-mod-php*
- Add stability directives:
EnableMMAP Off EnableSendfile Off
- Disable suspicious modules:
Implement this monitoring script to track Apache memory usage:
#!/bin/bash
while true; do
APACHE_MEM=$(ps -ylC apache2 | awk '{sum+=$8; count++} END {print sum/1024/1024}')
FREE_MEM=$(free -m | awk '/Mem:/ {print $4}')
echo "$(date) - Apache: ${APACHE_MEM%.*}MB | Free: ${FREE_MEM}MB"
if [ ${APACHE_MEM%.*} -gt $((FREE_MEM - 2048)) ]; then
/etc/init.d/apache2 graceful
fi
sleep 30
done
For enterprise deployments, consider these additional measures:
# In apache2.conf:
Timeout 30
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5
HostnameLookups Off
# For PHP-FPM users:
<IfModule mod_fastcgi.c>
Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi
FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi -socket /var/run/php5-fpm.sock -pass-header Authorization
</IfModule>
Remember to benchmark changes using:
ab -n 10000 -c 100 http://yourserver/test.php
siege -b -c 200 -t 2M http://yourserver/
Your server with 128GB RAM shows excellent available memory (97GB free with Apache running), but we're seeing segmentation faults during peak loads. The current prefork configuration appears excessively high given your actual worker usage (44 busy + 12 idle workers).
# Current problematic configuration
StartServers 3
MinSpareServers 3
MaxSpareServers 5
ServerLimit 3200
MaxClients 3100
MaxRequestsPerChild 0
For memory-based calculation (assuming PHP processes):
# Sample calculation script
#!/bin/bash
AVG_APACHE_MEM=$(ps -ylC apache2 | awk '{x += $8;y++} END {print x/y/1024}')
TOTAL_RAM=128000 # in MB
OTHER_SERVICES=3500 # in MB (from your metrics)
AVAILABLE_RAM=$((TOTAL_RAM - OTHER_SERVICES))
MAX_CLIENTS=$((AVAILABLE_RAM / AVG_APACHE_MEM))
echo "Suggested MaxClients: $MAX_CLIENTS"
Based on your current memory usage patterns:
StartServers 5
MinSpareServers 5
MaxSpareServers 10
ServerLimit 600
MaxClients 500
MaxRequestsPerChild 10000
The SIGSEGV errors suggest either:
- PHP extension conflicts (disable suspicious extensions)
- Memory corruption in custom modules (test with empty PHP files)
- Apache bugs (upgrade to latest stable version)
Debug steps:
# Install debug symbols and get backtrace
sudo apt-get install apache2-dbg
gdb -p $(pgrep apache2)
(gdb) bt full
Validate new settings with:
ab -n 5000 -c 100 http://yourserver/test.php
siege -b -t60S -c250 http://yourserver
Monitor with:
watch -n 1 "echo -n 'Apache Processes: '; ps -C apache2 --no-headers | wc -l"
For high-traffic PHP servers:
# In php.ini
pm = dynamic
pm.max_children = 50
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 10