In iptables, connection tracking is fundamental to stateful firewall operations. The three main states we're examining are:

• NEW: The first packet of a connection (SYN packet in TCP)
• ESTABLISHED: Packets belonging to an existing connection
• RELATED: Packets initiating a new connection that's related to an existing one (like FTP data connections)

The typical SSH rate-limiting rules target NEW connections because:

# Track new SSH connection attempts
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set --name SSH

# Block if more than 10 attempts in 100 seconds
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 100 --hitcount 10 --name SSH -j DROP

We don't include ESTABLISHED/RELATED because:

1. We want to limit initial connection attempts, not ongoing sessions
2. Established connections don't indicate brute force attempts
3. SSH doesn't create RELATED connections like FTP does

Consider this TCP handshake example:

Client: SYN (NEW) → Server
Client: SYN-ACK (NEW reply) ← Server
Client: ACK (ESTABLISHED) → Server

The moment the third packet (ACK) arrives, the connection becomes ESTABLISHED.

Here's a more comprehensive SSH protection setup:

# Create SSH chain
iptables -N SSH_PROTECT

# First rule: Track new connections
iptables -A SSH_PROTECT -m recent --set --name SSH

# Second rule: Check for excessive attempts
iptables -A SSH_PROTECT -m recent --update --seconds 300 --hitcount 5 --name SSH -j DROP

# Third rule: Allow established connections
iptables -A SSH_PROTECT -m state --state ESTABLISHED -j ACCEPT

# Apply to INPUT chain
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -j SSH_PROTECT

RELATED comes into play with protocols that open secondary connections. For FTP:

# Allow FTP connections and related data channels
iptables -A INPUT -p tcp --dport 21 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -m state --state RELATED,ESTABLISHED -j ACCEPT

But for SSH, RELATED isn't necessary as it doesn't spawn additional connections.

• Mixing state conditions unnecessarily (like including RELATED for SSH)
• Applying rate limiting to ESTABLISHED connections
• Forgetting that UDP also has connection states (though different from TCP)

The key is matching the state conditions to your specific protocol requirements and security objectives.

In Linux firewalls, connection tracking distinguishes between three fundamental states:

NEW: First packet of a new connection (SYN packet in TCP)
ESTABLISHED: Packets belonging to an existing connection (after successful 3-way handshake)
RELATED: New connections that are logically associated with an existing one (e.g., FTP data connections)


The example SSH rate-limiting rules specifically target NEW connections because:

# This tracks only new connection attempts
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set

# This enforces rate limiting on new connections only
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 100 --hitcount 10 -j DROP
Key reasons for not including ESTABLISHED/RELATED:
1. Established connections don't need rate limiting - they've already passed initial auth

2. SSH doesn't spawn RELATED connections (unlike FTP/PASV mode)

3. Tracking all states would incorrectly count legitimate ongoing sessions
Here's how a TCP connection progresses through states:


Packet
State Transition


Client SYN
NEW


Server SYN-ACK
NEW → ESTABLISHED (server perspective)


Client ACK
ESTABLISHED (both sides)


A complete SSH firewall setup would separate state handling:
# Allow established connections (no rate limiting)
iptables -A INPUT -p tcp --dport 22 -m state --state ESTABLISHED -j ACCEPT

# Rate limit new connections
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set --name SSH
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 60 --hitcount 5 --name SSH -j DROP

# Final accept for new connections under limit
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
This structure ensures:
- Existing sessions aren't interrupted

- Brute force attacks are effectively limited

- Legitimate users get uninterrupted access after auth