Debugging SSH Remote Port Forwarding: Why GatewayPorts Configuration Matters


2 views

When your SSH remote port forwarding (-R) appears to succeed in the debug output but fails in practice, the culprit is often server-side configuration. The debug output you saw:

debug1: remote forward success for: listen 2255, connect localhost:2255

is technically correct - the forwarding tunnel is established, but with a critical limitation: it's only bound to localhost on the remote server.

By default, SSHd binds remote forwards to 127.0.0.1 only. To make the port publicly accessible:

# On the remote server's sshd_config
GatewayPorts yes

This tells SSHd to bind to 0.0.0.0 instead. Without it, your binding will look like this:

$ netstat -tulpn | grep 2255
tcp        0      0 127.0.0.1:2255          0.0.0.0:*               LISTEN      1234/sshd: user

Before checking sshd_config, you could have:

  1. Test local binding on the remote server:
    curl http://localhost:2255
    telnet localhost 2255
  2. Check listening ports from another server:
    nmap -p 2255 your.server.ip
  3. Verify SSHd version:
    ssh -V
    sshd -V

If you can't modify sshd_config immediately, consider:

# Use socat as a relay on the remote server
socat TCP-LISTEN:2255,fork,reuseaddr TCP:localhost:2255 &

Or use an alternative forward syntax:

ssh -R *:2255:localhost:2255 user@host

Remember that opening GatewayPorts exposes the forwarded service to the network. Always combine with:

# In sshd_config
AllowTcpForwarding remote
PermitOpen host:port

For production systems, consider using SSH jump hosts or VPNs instead of exposing ports directly.

Newer OpenSSH versions (7.3+) support more granular control:

# Bind to specific interface only
GatewayPorts clientspecified

Then specify the bind address during connection:

ssh -R 1.2.3.4:2255:localhost:2255 user@host

When attempting to establish an SSH remote port forward using ssh -v -R 2255:localhost:2255 root@example.com, you might encounter silent failures where the connection appears successful but the port forwarding doesn't actually work. The debug output can be misleading:

debug1: remote forward success for: listen 2255, connect localhost:2255
debug1: All remote forwarding requests processed

In most modern SSH implementations, remote port forwarding is restricted by default to localhost (127.0.0.1) binding unless explicitly configured otherwise. This is controlled by the GatewayPorts directive in sshd_config.

Three possible GatewayPorts configurations:

# Default (most restrictive)
# GatewayPorts no

# Bind to all interfaces (least secure)
GatewayPorts yes

# Specific interface binding
GatewayPorts clientspecified

To properly diagnose this issue:

  1. Verify listening ports on the remote server:
    ssh root@example.com "netstat -tuln | grep 2255"
    # Alternative for newer systems:
    ssh root@example.com "ss -tuln | grep 2255"
    
  2. Check SSH server configuration without direct access:
    ssh -T root@example.com "sudo grep GatewayPorts /etc/ssh/sshd_config"
    
  3. Test with verbose output:
    ssh -vvv -R 2255:localhost:2255 root@example.com
    

    Look for subtle differences in the debug output when comparing with/without GatewayPorts enabled.

If you can't modify the server's sshd_config, consider these workarounds:

# Use SSH's ProxyCommand to tunnel through another connection
ssh -R 2255:localhost:2255 root@example.com \
  -o "ProxyCommand=ssh -W %h:%p jumpuser@jumpserver"

# Use autossh for persistent connections that can detect failures
autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" \
  -R 2255:localhost:2255 root@example.com

While GatewayPorts yes solves the immediate problem, it exposes the forwarded port to all network interfaces. For better security:

# Restrict to specific IP ranges
GatewayPorts clientspecified
Match Address 192.168.1.0/24
  GatewayPorts yes

Here's a complete debugging session:

# Client side (with verbose output)
$ ssh -vvv -R 2255:localhost:2255 user@remote

# On remote server while SSH is connected
$ sudo tcpdump -i any -n port 2255
$ sudo lsof -i :2255

# Check kernel routing
$ ip route get 192.168.1.100  # Replace with client IP