While both --state NEW and --syn deal with TCP connection initiation, they operate at fundamentally different layers of the network stack:

# Stateful inspection (connection tracking)
iptables -A INPUT -m state --state NEW -j ACCEPT

# Packet-level SYN flag matching
iptables -A INPUT -p tcp --syn -j ACCEPT

Case 1: Retransmitted SYN packets

The state module remembers established connections, so subsequent SYN retransmissions won't match --state NEW after the initial packet. However, --syn will catch all SYN packets regardless of connection state.

# Example: SYN flood protection
iptables -A INPUT -p tcp --syn -m limit --limit 1/s --limit-burst 3 -j ACCEPT
iptables -A INPUT -p tcp --syn -j DROP

Case 2: Middlebox interference

Some network devices (like load balancers) may strip SYN flags while still initiating new connections. These would match --state NEW but not --syn.

Stateful firewall configuration:

iptables -N STATE_FILTER
iptables -A STATE_FILTER -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A STATE_FILTER -m state --state NEW -p tcp --dport 22 -j ACCEPT
iptables -A STATE_FILTER -j DROP

SYN-based rate limiting:

iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 50 -j REJECT

The connection tracking module (--state) maintains a state table in memory, which consumes resources but enables sophisticated filtering. The --syn flag check is lightweight but doesn't understand connection context.

Using --syn alone leaves you vulnerable to:

1. ACK scans (bypassing SYN checks)
2. Midstream connection injection

While --state NEW provides better protection against these advanced techniques.

While both --state NEW and --syn deal with TCP connection initiation, they operate at fundamentally different layers of the networking stack:

# State module approach
iptables -A INPUT -p tcp -m state --state NEW -j ACCEPT

# SYN flag approach 
iptables -A INPUT -p tcp --syn -j ACCEPT

The --state NEW parameter works with iptables' connection tracking system (conntrack), making it stateful. It identifies the first packet of a new connection, regardless of TCP flags. In contrast, --syn simply matches packets with the SYN flag set, operating statelessly.

• TCP Fast Open (TFO): The SYN packet may contain data, making it a NEW state packet but not necessarily just a SYN
• Connection Reuse: Some protocols reuse existing connections where NEW state doesn't trigger
• TCP Out-of-Order Packets: Retransmitted SYN packets might be treated differently

Consider a scenario with UDP traffic:

# This will match UDP "connections" (first packet)
iptables -A INPUT -p udp -m state --state NEW -j LOG --log-prefix "New UDP: "

# This won't work for UDP at all (TCP-only)
iptables -A INPUT -p udp --syn -j LOG # Invalid syntax

Another example with ICMP:

# Will match first ICMP request
iptables -A INPUT -p icmp -m state --state NEW -j ACCEPT

# SYN flag irrelevant for ICMP
iptables -A INPUT -p icmp --syn -j DROP # Doesn't make sense

The stateful approach (--state NEW) requires maintaining connection tracking tables, which consumes additional memory. The stateless --syn check is lighter but less precise for tracking actual connections.

Using --syn alone might miss:

• Malformed packets attempting connection setup
• Certain types of protocol-specific connection initiation
• Non-TCP protocols that still represent new connections