When working with iptables, we're dealing with a stateful firewall that tracks the state of network connections. The four main connection states are:

• NEW - The packet starts a new connection
• ESTABLISHED - The packet belongs to an existing connection
• RELATED - The packet is related to but not part of an existing connection (like FTP data connections)
• INVALID - The packet doesn't belong to any known connection

The common practice of explicitly checking for NEW state on service ports serves several important purposes:

# Typical iptables ruleset snippet
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT

1. Security Boundary: The NEW state check creates a clear demarcation between initial connection attempts (which need authorization) and ongoing traffic (which has already been vetted).

2. Attack Surface Reduction: Without NEW checks, any ESTABLISHED connection could potentially access other services by changing ports mid-session.

Consider what happens without explicit NEW state checks:

# Vulnerable ruleset example
iptables -A INPUT -p tcp --dport 22 -j ACCEPT  # Without state check

An attacker could:

1. Establish a legitimate SSH connection (port 22)
2. Use TCP segment injection to redirect the connection to port 80
3. Potentially bypass intended restrictions

Here's a more complete example demonstrating proper state handling:

# Flush existing rules
iptables -F

# Default policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT

# Stateful rules
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT

# Service-specific NEW state rules
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --set --name SSH
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 60 --hitcount 3 --name SSH -j DROP
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT

# Additional services
iptables -A INPUT -p tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT

1. Missing ESTABLISHED/RELATED rule: This would break all ongoing connections.

2. Using ACCEPT for NEW without service restrictions: Effectively disables stateful filtering.

3. Checking NEW state on ESTABLISHED rules: Creates unnecessary overhead.

When examining iptables configurations, you'll frequently encounter rules like this:

iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT

The explicit NEW state declaration might seem redundant at first glance. After all, if a packet isn't NEW (first packet of a connection), wouldn't it be caught by the RELATED/ESTABLISHED rule anyway?

There are several technical reasons for explicitly specifying NEW state:

• Security Precision: Explicit NEW state ensures only new connection attempts are matched, preventing potential rule bypass scenarios
• Rule Order Independence: Makes the firewall behavior predictable regardless of rule ordering
• Logging Clarity: When logging new connections separately, this distinction becomes essential
• State Tracking: Helps the connection tracking system properly categorize connections

Consider this web server configuration:

# Accept established/related connections
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Explicit NEW state for HTTP/HTTPS
iptables -A INPUT -p tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT

# SSH with rate limiting for NEW connections
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --set
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 60 --hitcount 4 -j DROP
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT

Without NEW state, you might inadvertently allow invalid packets:

# Potentially problematic rule (without NEW state)
iptables -A INPUT -p tcp --dport 3306 -j ACCEPT

# This would accept:
# 1. Legitimate new MySQL connections
# 2. Invalid packets that aren't part of any tracked connection
# 3. Packets that should be handled by RELATED/ESTABLISHED rules

For enhanced security, combine NEW state with other match modules:

# Allow NEW SSH connections only from specific network
iptables -A INPUT -p tcp -s 192.168.1.0/24 --dport 22 -m conntrack --ctstate NEW -j ACCEPT

# Rate limit NEW HTTP connections per IP
iptables -A INPUT -p tcp --dport 80 -m conntrack --ctstate NEW -m hashlimit \
    --hashlimit-above 20/minute --hashlimit-burst 10 --hashlimit-mode srcip \
    --hashlimit-name http -j DROP

This approach gives you granular control while maintaining state tracking integrity.