Debugging Nginx: Why is 127.0.0.1 Port 80 Connection Refused Despite Running Server?


4 views

When your Nginx server shows as running on port 80 but refuses connections specifically to 127.0.0.1, while working fine on other ports and IPs, we're looking at a classic networking puzzle. Let's dissect this systematically.

First, verify your Nginx is actually listening where you think it is:

sudo ss -tulpn | grep nginx
# Alternative:
sudo lsof -i :80 -P -n | grep LISTEN

When you see output like this but connections fail:

tcp    LISTEN   0  128  0.0.0.0:80   0.0.0.0:*   users:(("nginx",pid=4268,fd=6))
tcp    LISTEN   0  128    [::]:80      [::]:*      users:(("nginx",pid=4268,fd=7))

The smoking gun appears in your NAT table rules:

Chain OUTPUT (policy ACCEPT)
target     prot opt source     destination
REDIRECT   tcp  --  *         127.0.0.1    tcp dpt:80 redir ports 20559
REDIRECT   tcp  --  *         127.0.0.1    tcp dpt:443 redir ports 20558

These rules are silently redirecting your local traffic to different ports. Clear them with:

sudo iptables -t nat -D OUTPUT -d 127.0.0.1 -p tcp --dport 80 -j REDIRECT --to-port 20559
sudo iptables -t nat -D OUTPUT -d 127.0.0.1 -p tcp --dport 443 -j REDIRECT --to-port 20558

After clearing the rules, test again:

curl -v http://127.0.0.1
# Should now show proper Nginx response

telnet 127.0.0.1 80
# Should connect successfully

For comprehensive network analysis:

# Check system logs
sudo journalctl -u nginx --no-pager -n 50

# Advanced connection tracing
sudo strace -f -e trace=network -p $(pgrep nginx | head -1)

# Deep packet inspection
sudo tcpdump -i lo -nn -vv port 80 -w nginx_debug.pcap

Add these checks to your troubleshooting checklist:

# 1. Check all iptables tables
for table in filter nat mangle raw; do
    echo "=== Table $table ==="
    sudo iptables -t $table -nvL
done

# 2. Verify kernel route handling
cat /proc/sys/net/ipv4/conf/all/route_localnet
# Should be 1 for localhost routing

# 3. Test raw socket connection
sudo nc -l -p 80 -s 127.0.0.1 &  # Temporary listener
curl 127.0.0.1
kill %1

When your Nginx server shows as running on port 80 but connections to localhost fail with "Connection refused", we're dealing with one of Linux's more puzzling networking issues. The key indicators are:

# Confirmed Nginx is listening
$ sudo netstat -tulpn | grep 80
tcp     0      0 0.0.0.0:80   0.0.0.0:*    LISTEN  4268/nginx
tcp6    0      0 :::80        :::*         LISTEN  4268/nginx

# Yet connection fails
$ curl -v 127.0.0.1
* connect to 127.0.0.1 port 80 failed: Connection refused

The iptables NAT table reveals the hidden culprit:

$ sudo iptables -t nat -nvL
Chain OUTPUT (policy ACCEPT)
target     prot opt source     destination
REDIRECT   tcp  --  0.0.0.0/0  127.0.0.1   tcp dpt:80 redir ports 20559

This rule silently redirects all local port 80 traffic to port 20559 - likely from some VPN or proxy software installation.

Option 1: Remove the redirect rule

$ sudo iptables -t nat -D OUTPUT -d 127.0.0.1 -p tcp --dport 80 -j REDIRECT --to-port 20559
$ sudo iptables -t nat -D PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 20559

Option 2: Make it permanent (for systemd systems)

$ sudo systemctl stop iptables
$ sudo iptables-save | grep -v "redir ports 20559" | sudo iptables-restore
$ sudo systemctl start iptables

After fixing, verify with:

$ curl -I http://localhost
HTTP/1.1 200 OK
Server: nginx/1.18.0

$ sudo nmap -sS -p 80 127.0.0.1
PORT   STATE SERVICE
80/tcp open  http

If you need to temporarily test without modifying iptables:

# Use the IPv6 loopback
$ curl -g http://[::1]

# Bind to a different interface
$ curl http://10.0.2.15

# Use a different port in Nginx config
listen 8080;

Add these checks to your deployment scripts:

#!/bin/bash
# Verify port accessibility
if ! curl -s --connect-timeout 5 http://localhost > /dev/null; then
    echo "Checking for port redirects..."
    sudo iptables -t nat -L | grep -q "redir ports" && \
        echo "WARNING: NAT redirect rules detected" >&2
fi