When running high-load benchmarks with Apache Benchmark (ab) against an Nginx+Apache stack, you might encounter:

apr_socket_recv: Connection reset by peer (104)

This typically occurs when the server abruptly terminates an active TCP connection. Unlike graceful connection closures (FIN packets), this is an RST (reset) packet scenario indicating a forced termination.

In your described setup with Nginx serving cached content, these are the most likely culprits:

• Nginx worker_connections limit reached
• Operating system socket exhaustion (check net.core.somaxconn)
• Keepalive misconfiguration between Nginx and Apache
• Server resource exhaustion (RAM, file descriptors)

First, check your current Nginx connection handling:

sudo netstat -antp | grep nginx | wc -l
ulimit -n
sysctl net.core.somaxconn

Compare these against your Nginx configuration:

worker_processes auto;
worker_rlimit_nofile 100000;
events {
    worker_connections 4000;
    multi_accept on;
}

For high-concurrency benchmarking scenarios:

# /etc/nginx/nginx.conf
events {
    worker_connections 8192;
    use epoll;
    accept_mutex off;
}

http {
    keepalive_timeout 30;
    keepalive_requests 1000;
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
}

And corresponding OS-level adjustments:

# /etc/sysctl.conf
net.core.somaxconn = 32768
net.ipv4.tcp_max_syn_backlog = 8192
fs.file-max = 2097152

When using ab for stress testing:

ab -k -n 10000 -c 500 http://yourserver.example.com/

Key flags:

• -k: Enable HTTP KeepAlive
• -c: Concurrency (gradually increase)
• Consider -s timeout parameter for long tests

For more reliable high-volume testing:

wrk -t12 -c400 -d30s http://yourserver.example.com/
siege -b -c 300 -t 1M http://yourserver.example.com/

These tools handle connection pooling more gracefully than ab for extreme load testing.

Run these in separate terminals during your benchmark:

watch -n 1 "netstat -ant | awk '{print \$6}' | sort | uniq -c"
vmstat 1
nginx -T | grep -i error_log
tail -f /var/log/nginx/error.log

---

The apr_socket_recv: Connection reset by peer (104) error occurs when the remote server (in this case Nginx) abruptly terminates an established TCP connection. This typically happens under heavy load conditions when:

• Server reaches maximum concurrent connections limit
• TCP buffers overflow
• Keepalive timeouts expire
• Server-side process crashes

Since you're using Nginx as reverse proxy with Apache behind, these are the most likely culprits:

# Check your current Nginx connection limits
netstat -anp | grep nginx | wc -l
ss -s | grep "Total:"

Common misconfigurations in /etc/nginx/nginx.conf:

worker_connections 1024; # Often too low for benchmarking
worker_rlimit_nofile 65535; # Missing entirely in many configs
keepalive_timeout 65; # May need adjustment

First, verify your system-wide limits:

ulimit -n  # Should show at least 65535 for serious benchmarking

Then adjust Nginx configuration:

events {
    worker_connections 65535;
    multi_accept on;
    use epoll;
}

http {
    keepalive_requests 10000; # Default is 100
    keepalive_timeout 30s;
    reset_timedout_connection on;
}

When using ab, these parameters can prevent connection resets:

ab -k -n 10000 -c 100 http://yoursite.com/ # -k enables HTTP keepalive
ab -r -n 10000 -c 50 http://yoursite.com/  # -r prevents exit on socket errors

Add these to /etc/sysctl.conf for high connection workloads:

net.ipv4.tcp_max_syn_backlog = 65536
net.core.somaxconn = 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30

Apply changes with sysctl -p

Run these in separate terminals during benchmarking:

watch -n 1 "netstat -ant | awk '{print \$6}' | sort | uniq -c"
dstat -n --tcp