LXC Container Port Forwarding: Debugging iptables DNAT Rules for External Access

When setting up port forwarding from a host (10.0.3.1) to an LXC container (10.0.3.2), the DNAT rule seems correct but connections are being refused. Here's a deep dive into what's happening:

# Current DNAT rule that isn't working as expected
iptables -t nat -A PREROUTING -p tcp --dport 7002 -j DNAT --to 10.0.3.2:7000

Before diving into solutions, let's verify some critical points:

# Check if the container is listening on port 7000
lxc exec container-name -- netstat -tulnp | grep 7000

# Verify basic connectivity from host to container
lxc exec container-name -- nc -zv 10.0.3.2 7000

# Check iptables logging (add these temporary rules)
iptables -I INPUT -p tcp --dport 7002 -j LOG --log-prefix "[IPTABLES-INPUT] "
iptables -I FORWARD -p tcp --dport 7000 -j LOG --log-prefix "[IPTABLES-FORWARD] "

1. Missing FORWARD chain rules

While your FORWARD policy is ACCEPT, restrictive rules might be blocking traffic:

# Explicitly allow forwarded traffic to container
iptables -A FORWARD -d 10.0.3.2/32 -p tcp --dport 7000 -j ACCEPT
iptables -A FORWARD -s 10.0.3.2/32 -p tcp --sport 7000 -j ACCEPT

2. Localhost access requires additional rules

DNAT in PREROUTING doesn't affect locally-generated traffic. Add OUTPUT chain NAT:

iptables -t nat -A OUTPUT -p tcp --dport 7002 -j DNAT --to 10.0.3.2:7000

3. Interface-specific binding

If your service binds to specific interfaces, check with:

lxc exec container-name -- ss -tulnp | grep 7000

Here's a tested configuration that works end-to-end:

# Flush existing rules
iptables -F
iptables -t nat -F
iptables -t mangle -F

# Default policies
iptables -P INPUT DROP
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

# Basic rules
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

# NAT rules
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o lxcbr0 -j MASQUERADE

# Port forwarding rules
iptables -t nat -A PREROUTING -p tcp --dport 7002 -j DNAT --to 10.0.3.2:7000
iptables -t nat -A OUTPUT -p tcp --dport 7002 -j DNAT --to 10.0.3.2:7000

# Forwarding rules
iptables -A FORWARD -d 10.0.3.2/32 -p tcp --dport 7000 -j ACCEPT
iptables -A FORWARD -s 10.0.3.2/32 -p tcp --sport 7000 -m state --state ESTABLISHED -j ACCEPT

Packet tracing:

# On the host:
tcpdump -i any port 7000 or port 7002 -n -v

# In the container:
lxc exec container-name -- tcpdump -i any port 7000 -n -v

Conntrack monitoring:

conntrack -E -p tcp --dport 7002

Full iptables logging:

iptables -t raw -A PREROUTING -p tcp --dport 7002 -j TRACE
iptables -t raw -A OUTPUT -p tcp --dport 7002 -j TRACE

When attempting to expose an LXC container service to external networks, the DNAT rule in your iptables configuration appears correct at first glance:

iptables -t nat -A PREROUTING -p tcp --dport 7002 -j DNAT --to 10.0.3.2:7000

However, the connection fails with "Connection refused" when testing from the host itself. This reveals several technical nuances in Linux networking that we need to address.

The key insight is that traffic originating from the host machine doesn't pass through the PREROUTING chain. The complete path looks like:

Host-originated traffic: OUTPUT → POSTROUTING
External traffic: PREROUTING → FORWARD → POSTROUTING

This explains why your current setup works for external connections but fails for local testing.

Here's the complete set of rules needed for both external and local access:

# Flush existing rules
iptables -F
iptables -F -t nat
iptables -F -t mangle
iptables -X

# Default policies
iptables -P INPUT DROP
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

# Allow SSH and ICMP
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT

# Connection tracking
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

# NAT for external interfaces
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o lxcbr0 -j MASQUERADE

# PREROUTING rule for external traffic
iptables -t nat -A PREROUTING -p tcp --dport 7002 -j DNAT --to 10.0.3.2:7000

# OUTPUT rule for local traffic
iptables -t nat -A OUTPUT -p tcp -d 10.0.3.1 --dport 7002 -j DNAT --to 10.0.3.2:7000

After applying these rules, verify with:

# Check NAT rules
iptables -t nat -L -n -v

# Test connectivity from host
telnet 127.0.0.1 7002
telnet [host_public_ip] 7002

# Check conntrack entries
conntrack -L | grep 7002

For simpler setups, consider these approaches:

# Method 1: Use REDIRECT for local traffic
iptables -t nat -A PREROUTING -p tcp --dport 7002 -j REDIRECT --to-port 7000

# Method 2: Enable route_localnet (requires careful security consideration)
echo 1 > /proc/sys/net/ipv4/conf/all/route_localnet

Remember that each solution has different security implications and use cases.

Forgetting to enable IP forwarding: echo 1 > /proc/sys/net/ipv4/ip_forward
Missing connection tracking rules in the FORWARD chain
Not considering the interface binding of the container service
Overlooking firewall rules on the container itself