Debugging “HTTP request sent, awaiting response” Stalls on Linux Servers: Network Analysis & Solutions


3 views

html

When running network requests through wget or browsers on an ADSL-connected Linux server, you might encounter:

Connecting to example.com|xxx.xxx.xxx.122|:80... connected.
HTTP request sent, awaiting response...

The request hangs indefinitely until manual intervention (CTRL-C in terminal or browser refresh).

Begin with these diagnostic commands:

# Check TCP connections
ss -tulwnp | grep ":80"

# Verify DNS resolution timing
dig example.com +stats

# Test alternative ports
wget --debug --output-document=- http://example.com:443/

Capture network traffic during failure:

sudo tcpdump -i any -w http_debug.pcap 'host example.com and port 80'
wget http://example.com/test.html
# When hanging occurs, stop tcpdump with CTRL+C

Analyze the capture with Wireshark, looking for:

  • TCP retransmissions (Expert Info → Notes)
  • Window size changes
  • SYN/ACK delays

Adjust wget timeouts for testing:

wget --dns-timeout=10 --connect-timeout=10 \
     --read-timeout=20 --tries=3 \
     http://example.com

Check kernel network parameters:

sysctl net.ipv4.tcp_syn_retries
sysctl net.core.somaxconn

Temporary adjustments for testing:

sudo sysctl -w net.ipv4.tcp_syn_retries=3
sudo sysctl -w net.core.somaxconn=1024

MTU issues often affect ADSL connections. Test with:

ping -M do -s 1472 -c 4 example.com  # Adjust size downward until success
sudo ifconfig eth0 mtu 1492          # Temporary MTU change

For critical operations, implement retry logic:

#!/bin/bash
MAX_RETRIES=3
TIMEOUT=10
URL="http://example.com"

for i in $(seq 1 $MAX_RETRIES); do
  if wget --timeout=$TIMEOUT -qO- "$URL" > output.txt; then
    echo "Success on attempt $i"
    break
  else
    echo "Attempt $i failed, retrying..."
    sleep $((i * 2))
  fi
done

If issues persist, consider:

  • Setting TCP keepalive: sysctl -w net.ipv4.tcp_keepalive_time=60
  • Implementing a reverse proxy (nginx) with upstream timeouts
  • Using curl with happy eyeballs: curl --connect-timeout 5 --retry 2

When executing HTTP requests via wget or browsers on a Linux ADSL connection, you may encounter:

Connecting to example.com|xxx.xxx.xxx.122|:80... connected.
HTTP request sent, awaiting response...

The request appears to hang indefinitely, with temporary resolutions coming from manual intervention (CTRL-C in wget or browser refresh).

Start with these diagnostic commands:

# Check TCP connections
ss -tulnp | grep 80

# Verify routing
traceroute example.com

# Packet capture (run in background)
tcpdump -i any port 80 -w http_debug.pcap &

# Test with curl for verbose output
curl -v http://example.com --trace-time

MTU Issues:

# Check current MTU
ip link show | grep mtu

# Test optimal MTU (adjust numbers)
ping -M do -s 1472 example.com

TCP Window Scaling: Add to /etc/sysctl.conf:

net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_rmem=4096 87380 6291456
net.ipv4.tcp_wmem=4096 16384 4194304

Timeout Analysis:

# Custom wget with timeout params
wget --dns-timeout=10 --connect-timeout=10 \
--read-timeout=20 --tries=3 http://example.com

Kernel Socket Buffers:

# Monitor in real-time
watch -n 1 "cat /proc/net/sockstat"

Create a monitoring script (http_monitor.sh):

#!/bin/bash
while true; do
  TS=$(date +%s)
  if ! curl -s -o /dev/null -m 30 http://example.com; then
    echo "$TS: Connection failed" >> http_monitor.log
    ss -tanp >> http_monitor.log
    ping -c 3 example.com >> http_monitor.log
  fi
  sleep 60
done

Consider these specialized utilities:

# HTTP-specific debugging
httping -g http://example.com -t 30

# Network quality testing
nuttcp -T30 -i1 -p example.com