When your Apache server starts accumulating CLOSE_WAIT connections like in your netstat output, it's essentially holding zombie connections that should have been terminated. These connections remain in kernel memory tables, consuming valuable resources until they eventually crash your server.
From your error logs, we can see multiple red flags:
[warn] child process 15670 still did not exit, sending a SIGTERM
[error] child process 15670 still did not exit, sending a SIGKILL
[notice] child pid 3511 exit signal Segmentation fault (11)
This indicates Apache child processes are either:
- Failing to properly terminate connections
- Crashing with segmentation faults
- Being forcibly killed after timeout periods
Add these directives to your httpd.conf:
# Reduce KeepAlive timeout
KeepAliveTimeout 2
MaxKeepAliveRequests 100
# TCP stack tuning
TimeOut 30
EnableSendfile off
# Process management
MaxRequestsPerChild 1000
GracefulShutdownTimeout 1
Add these to /etc/sysctl.conf:
# Reduce TIME_WAIT period
net.ipv4.tcp_fin_timeout = 15
# Enable socket recycling
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
# Increase available ports range
net.ipv4.ip_local_port_range = 1024 65535
Apply with sysctl -p
Create a cron job with this bash script to monitor CLOSE_WAIT states:
#!/bin/bash
THRESHOLD=50
COUNT=$(netstat -ant | grep CLOSE_WAIT | wc -l)
if [ $COUNT -gt $THRESHOLD ]; then
echo "$(date) - CLOSE_WAIT connections: $COUNT" >> /var/log/close_wait_monitor.log
# Soft restart Apache
/etc/init.d/httpd graceful
fi
For your WPMU installations:
- Disable XML-RPC if unused: add to wp-config.php:
add_filter('xmlrpc_enabled', '__return_false');
- Limit crawlers in robots.txt:
User-agent: * Crawl-delay: 10 Disallow: /wp-admin/
When running Apache on Linux servers, you might encounter a situation where netstat
shows numerous connections stuck in CLOSE_WAIT
state. This indicates that the remote peer has closed the connection (sent FIN), but your local application (Apache) hasn't closed its socket descriptor.
$ netstat -tulnp | grep CLOSE_WAIT
tcp 160 0 server:http client:51584 CLOSE_WAIT
tcp 382 0 server_ip:http crawler:58663 CLOSE_WAIT
The root causes typically fall into these categories:
- Apache child processes not properly closing connections before termination
- PHP scripts or other backend applications hanging
- Kernel parameters not optimized for web server workloads
- Client-side issues (especially with crawlers and proxies)
Error logs often show related warnings:
[warn] child process 15670 still did not exit, sending a SIGTERM
[error] child process 15670 still did not exit, sending a SIGKILL
[notice] child pid 3511 exit signal Segmentation fault (11)
First, identify and kill hanging processes:
# Find Apache processes in CLOSE_WAIT
$ ss -o state close-wait -p | grep apache
# Kill specific processes
$ kill -9 PID
Then restart Apache gracefully:
$ apachectl graceful
Modify your Apache configuration (httpd.conf
or apache2.conf
):
# Reduce KeepAliveTimeout
KeepAliveTimeout 2
MaxKeepAliveRequests 100
# Adjust mpm_prefork settings
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 1000
</IfModule>
Add these to /etc/sysctl.conf
:
# Recycle TIME_WAIT sockets faster
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
# Decrease timeout values
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 1200
# Increase available ports range
net.ipv4.ip_local_port_range = 10000 65000
Apply changes:
$ sysctl -p
Create a monitoring script (check_close_wait.sh
):
#!/bin/bash
THRESHOLD=50
COUNT=$(ss -o state close-wait | wc -l)
if [ $COUNT -gt $THRESHOLD ]; then
echo "Warning: $COUNT CLOSE_WAIT connections detected" | mail -s "Apache CLOSE_WAIT Alert" admin@example.com
systemctl restart httpd
fi
Add to cron:
*/10 * * * * /path/to/check_close_wait.sh
For persistent issues, implement connection tracking with conntrack
:
$ apt-get install conntrack
$ conntrack -L -p tcp --state CLOSE_WAIT
For WordPress Multisite (your case), add to wp-config.php
:
define('WP_HTTP_BLOCK_EXTERNAL', true);
define('WP_ACCESSIBLE_HOSTS', 'api.wordpress.org,your-essential-domains.com');