When configuring firewall rules for IPSec/L2TP VPNs, we need to handle two distinct protocol suites:

# Mandatory UDP ports for all IPSec/L2TP implementations
UDP 500  - IKE (Internet Key Exchange)
UDP 4500 - NAT-Traversal (even without NAT, modern clients expect this)

For L2TP over IPSec (common in Windows native client):

# L2TP-specific ports
UDP 1701 - L2TP traffic

Many guides get confused about GRE (Protocol 47). Here's the technical reality:

# GRE protocol handling
Protocol 47 (GRE) - Required for PPTP, NOT for standard IPSec/L2TP
Protocol 50 (ESP) - Used for IPSec encapsulation
Protocol 51 (AH) - Authentication Header (less common in modern implementations)

For a non-NT environment (as in your case):

# Linux iptables example for non-NAT environment
iptables -A INPUT -p udp --dport 500 -j ACCEPT
iptables -A INPUT -p udp --dport 4500 -j ACCEPT
iptables -A INPUT -p udp --dport 1701 -j ACCEPT
iptables -A INPUT -p 50 -j ACCEPT

To test your configuration from Windows:

# PowerShell command to verify VPN connectivity
Test-NetConnection -ComputerName your.vpn.server -Port 500
Test-NetConnection -ComputerName your.vpn.server -Port 4500

If connections fail, check these diagnostic commands:

# Linux server-side debugging
sudo tcpdump -i eth0 -n udp port 500 or port 4500 or port 1701
sudo ipsec status

When configuring firewall rules for IPSec/L2TP VPN connections without NAT traversal, we need to consider three fundamental protocol groups:

// Protocol breakdown
1. ISAKMP (IKE) - UDP 500
2. IPSec ESP - Protocol 50
3. NAT-Traversal (if enabled) - UDP 4500

For basic IPSec without L2TP or NAT:

# Minimal IPSec firewall rules
iptables -A INPUT -p udp --dport 500 -j ACCEPT  # IKE
iptables -A INPUT -p 50 -j ACCEPT               # ESP
iptables -A INPUT -p udp --dport 4500 -j ACCEPT # NAT-T (optional)

When combining IPSec with L2TP (common for Windows clients):

# Full IPSec/L2TP configuration
iptables -A INPUT -p udp --dport 500 -j ACCEPT    # IKE
iptables -A INPUT -p udp --dport 1701 -j ACCEPT   # L2TP
iptables -A INPUT -p 50 -j ACCEPT                 # ESP
iptables -A INPUT -p 51 -j ACCEPT                 # AH (optional)
iptables -A INPUT -p udp --dport 4500 -j ACCEPT   # NAT-T

For Windows built-in VPN client connections:

• UDP 500 (IKE) - Mandatory for initial key exchange
• Protocol 50 (ESP) - Required for encrypted payload
• UDP 1701 (L2TP) - Layer 2 Tunnel Protocol
• UDP 4500 - Needed if behind NAT (despite your non-NAT environment)

Sample StrongSwan config showing required ports:

# /etc/ipsec.conf
conn myvpn
    keyexchange=ikev1
    authby=secret
    ike=aes256-sha1-modp1024
    esp=aes256-sha1
    left=your.server.ip
    leftprotoport=17/1701
    right=%any
    rightprotoport=17/%any
    auto=add

Verification commands:

# Check open ports
netstat -tulnp | grep -E '500|1701|4500'

# Verify ESP traffic
tcpdump -ni eth0 proto 50

# Test IKE connectivity
ike-scan your.vpn.server