Solution for strongSwan IKEv2 VPN Connection Failure Between NATed Endpoints


2 views

When establishing an IKEv2 VPN connection between two NATed endpoints, several technical hurdles emerge. The logs show the initial IKE_SA_INIT exchange succeeds (port 500), but the subsequent IKE_AUTH phase fails when switching to port 4500 for NAT-Traversal.

For dual-NAT scenarios, these elements are essential:

# Required on both endpoints
config setup
    charonstart=yes
    # Enable NAT-T globally
    uniqueids=never
    # Force port 4500 usage
    force_port_4500=yes

The server needs explicit NAT-T configuration:

conn venus
    left=%any
    leftcert=sunCert.pem
    leftsubnet=10.135.1.0/24
    leftfirewall=yes
    right=%any
    rightid="C=IL, O=KrustyKrab, CN=venus"
    # Critical NAT-T settings
    keyexchange=ikev2
    ike=aes256-sha256-modp2048!
    esp=aes256-sha256!
    rekey=no
    # NAT-T specific
    forceencaps=yes
    fragmentation=yes
    mobike=no
    auto=add

The client requires matching NAT-T parameters:

conn krustykrab
    left=%defaultroute
    leftcert=venusCert.pem
    leftsourceip=%config
    leftid="C=IL, O=KrustyKrab, CN=venus"
    right=x.x.x.x
    rightsubnet=10.135.1.0/24
    rightid="C=IL, O=KrustyKrab, CN=sun"
    # Matching crypto parameters
    keyexchange=ikev2
    ike=aes256-sha256-modp2048!
    esp=aes256-sha256!
    # NAT-T specific
    forceencaps=yes
    type=tunnel
    auto=start

Both endpoints must ensure proper firewall rules:

# For IPTables on Linux gateways
iptables -A INPUT -p udp --dport 500 -j ACCEPT
iptables -A INPUT -p udp --dport 4500 -j ACCEPT
iptables -A FORWARD --match policy --pol ipsec --dir in --proto esp -j ACCEPT
iptables -A FORWARD --match policy --pol ipsec --dir out --proto esp -j ACCEPT

When troubleshooting:

  1. Verify NAT detection works: look for "remote host is behind NAT" in logs
  2. Check encapsulation with tcpdump: tcpdump -n -i eth0 udp port 4500
  3. Test port forwarding independently using netcat
  4. Enable verbose logging: charon { filelog { /var/log/ipsec.log { default = 2 } } }

For mobile clients, consider enabling MOBIKE:

conn mobile-config
    ...
    mobike=yes
    # Requires strongSwan 5.3.2+
    rekey_time=60m
    over_time=5m

When establishing an IPsec VPN connection between two endpoints both behind NAT (Network Address Translation), we encounter several technical hurdles that standard configurations don't handle well. The primary issues stem from:

  • UDP encapsulation of ESP packets (NAT-T)
  • Port forwarding requirements (500/udp and 4500/udp)
  • NAT detection and keepalive mechanisms
  • Certificate-based authentication across NAT boundaries

For the server-side (sun) configuration, we need these essential modifications:

conn venus
    left=%any
    leftid="C=IL, O=KrustyKrab, CN=sun"
    leftcert=sunCert.pem
    leftsubnet=10.135.1.0/24
    leftfirewall=yes
    right=%any
    rightid="C=IL, O=KrustyKrab, CN=venus"
    rightcert=venusCert.pem
    keyexchange=ikev2
    ike=aes256-sha512-modp2048!
    esp=aes256-sha512!
    auto=add
    type=tunnel
    dpdaction=restart
    closeaction=restart
    fragmentation=yes
    forceencaps=yes
    rekey=no

For the client-side (venus), the corresponding changes:

conn krustykrab
    left=%defaultroute
    leftsourceip=%config
    leftid="C=IL, O=KrustyKrab, CN=venus"
    leftcert=venusCert.pem
    right=x.x.x.x
    rightsubnet=10.135.1.0/24
    rightid="C=IL, O=KrustyKrab, CN=sun"
    keyexchange=ikev2
    ike=aes256-sha512-modp2048!
    esp=aes256-sha512!
    auto=start
    type=tunnel
    dpdaction=restart
    closeaction=restart
    fragmentation=yes
    forceencaps=yes
    rekey=no

When packets aren't reaching port 4500 despite proper forwarding, consider these diagnostic steps:

# On both endpoints:
ip xfrm state
ip xfrm policy
ipsec statusall

# Check NAT-T detection:
grep "NAT-T" /var/log/syslog

# Verify firewall rules:
iptables -t nat -L -n -v
iptables -L -n -v

For debugging packet flow, use these tcpdump commands:

# Server side:
tcpdump -n -i eth0 udp port 500 or port 4500 -vv

# Client side:
tcpdump -n -i eth0 udp port 500 or port 4500 -vv

Add these to /etc/strongswan.d/charon.conf on both endpoints:

charon {
    # NAT-T specific settings
    port_nat_t = 4500
    keep_alive = 30
    send_vendor_id = yes
    ignore_acquire_ts = yes
    
    # IKEv2 optimizations
    threads = 16
    ikesa_table_size = 32
    ikesa_table_segments = 4
    init_limit_half_open = 1000

Critical iptables rules for the VPN server:

iptables -A INPUT -p udp --dport 500 -j ACCEPT
iptables -A INPUT -p udp --dport 4500 -j ACCEPT
iptables -A INPUT -p esp -j ACCEPT
iptables -A FORWARD -s 10.135.1.0/24 -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.135.1.0/24 -o eth0 -j MASQUERADE

For the client behind NAT:

iptables -A INPUT -p udp --dport 500 -j ACCEPT
iptables -A INPUT -p udp --dport 4500 -j ACCEPT
iptables -A FORWARD -m policy --dir in --pol ipsec -j ACCEPT
iptables -A FORWARD -m policy --dir out --pol ipsec -j ACCEPT

When connection attempts fail during IKE_AUTH phase:

  1. Increase logging level in /etc/strongswan.conf:
    charon {
        filelog {
            /var/log/strongswan.charon.log {
                time_format = %b %e %T
                ike_name = yes
                append = no
                default = 2
                flush_line = yes
            }
        }
    }
  2. Check for certificate validation issues with:
    openssl verify -CAfile /etc/ipsec.d/cacerts/ca.pem /etc/ipsec.d/certs/sunCert.pem