When your Apache server starts logging "possible SYN flooding on port 80" messages while showing surprisingly low SYN_RECV connection counts (typically under 250), you're facing a classic TCP stack tuning challenge. This situation indicates a mismatch between your kernel's SYN queue handling and actual network conditions.
# Monitoring SYN_RECV connections
watch --interval=5 'netstat -tuna | grep "SYN_RECV" | wc -l'
# Typical output showing disconnect between logs and actual state
240 # While kernel complains about possible flooding
The Linux TCP stack involves several interacting components:
- SYN Queue (incomplete connection queue): Holds SYN packets waiting for ACKs
- Accept Queue (complete connection queue): Holds established connections waiting for accept()
- syncookies: Cryptographic workaround when SYN queue overflows
These kernel parameters directly affect SYN handling:
# /etc/sysctl.conf adjustments
net.ipv4.tcp_max_syn_backlog = 65536 # SYN queue size
net.core.somaxconn = 512 # Accept queue size
net.ipv4.tcp_syncookies = 1 # Flood protection
net.ipv4.tcp_synack_retries = 3 # SYN+ACK retries
net.ipv4.tcp_syn_retries = 3 # SYN retries
When syncookies are enabled (as they should be for production systems):
tcp_max_syn_backlog
becomes ineffective- The kernel uses cryptographic SYN cookies instead of queue space
- Warning messages appear before actual queue exhaustion
For high-traffic web servers, consider this configuration:
# Optimal settings for modern servers
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 65536
net.core.somaxconn = 1024
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_abort_on_overflow = 0
net.ipv4.tcp_tw_reuse = 1
Use these commands to identify actual constraints:
# Check socket overflow counters
ss -s | grep overflow
# Monitor SYN queue in real-time
nstat -az | grep -i syn
# Check application accept() performance
strace -p $(pgrep httpd) -e accept -c
Despite the warning messages, syncookies are doing their job when:
- Connection completion times remain stable
- No increase in failed connections
- SYN_RECV counts stay below theoretical limits
For extreme cases, consider:
# Rate limiting SYN packets
iptables -A INPUT -p tcp --syn --dport 80 -m limit --limit 1000/s -j ACCEPT
# Early SYN packet drop
echo 1 > /proc/sys/net/ipv4/tcp_abort_on_overflow
Implement these checks in your monitoring system:
# Track SYN queue usage
nstat -az TcpExtListenOverflows
# Connection establishment latency
tcprstat -p 80 -t 1 -n 0
# SYN packet rate
tcpdump -ni eth0 'tcp[tcpflags] & tcp-syn != 0' -c 1000 -w syn.pcap
When you see "possible SYN flooding" warnings in your logs despite having relatively low SYN_RECV connection counts, you're encountering a fundamental misunderstanding of how Linux handles TCP connections. The key realization here is that netstat -tuna | grep "SYN_RECV"
doesn't show the complete picture of your SYN queue state.
Several kernel parameters interact in complex ways to determine SYN queue behavior:
# Essential TCP stack parameters
net.ipv4.tcp_max_syn_backlog = 65536 # Maximum number of remembered connection requests
net.ipv4.tcp_syncookies = 1 # Enable SYN cookies protection
net.core.somaxconn = 512 # Maximum number of backlogged connections
net.ipv4.tcp_synack_retries = 3 # Number of SYN+ACK retransmits
net.ipv4.tcp_syn_retries = 3 # Number of SYN retransmits
The critical insight is that Linux maintains two separate queues for SYN packets:
- The SYN queue (incomplete connections)
- The accept queue (established connections waiting for accept())
When syncookies are enabled (net.ipv4.tcp_syncookies=1
), the SYN queue behavior changes dramatically. The system will start using cookies when either:
- The SYN queue is full (controlled by
tcp_max_syn_backlog
) - Or the accept queue is full (controlled by
somaxconn
)
Instead of just watching SYN_RECV counts, monitor both queues:
# Monitor SYN queue and accept queue sizes
watch -n1 'echo "SYN Queue: $(ss -n state syn-recv sport = :80 | wc -l)"; \
echo "Accept Queue: $(ss -n state established sport = :80 | wc -l)"'
And check your current kernel settings:
# Display current TCP settings
sysctl -a | grep -E 'tcp_max_syn_backlog|tcp_syncookies|somaxconn|tcp_synack_retries'
For web servers handling heavy traffic, consider these additional optimizations:
# Additional performance tweaks
net.ipv4.tcp_tw_reuse = 1 # Allow reuse of TIME-WAIT sockets
net.ipv4.tcp_fin_timeout = 30 # Reduce FIN timeout
net.ipv4.tcp_keepalive_time = 1200 # Start keepalives after 20 minutes
net.core.netdev_max_backlog = 5000 # Increase network interface backlog
net.ipv4.tcp_max_tw_buckets = 2000000 # Increase TIME-WAIT buckets
Despite common misconceptions, syncookies don't significantly impact performance in modern Linux kernels. They only activate when the system is under stress, making them an excellent safety net. The warning message is actually helpful - it indicates your system's protective mechanisms are working.
A better approach than disabling syncookies is to:
- Ensure your application can handle the traffic (check accept() bottlenecks)
- Monitor both SYN and accept queues
- Consider horizontal scaling if you're consistently hitting limits
Remember that these TCP stack optimizations should be tested under realistic load conditions, as the optimal settings vary based on your specific hardware and traffic patterns.