Debugging SSH Connection Hang After Successful Authentication on RHEL Systems


2 views

When working with RHEL 6.4 servers, you might encounter a particularly frustrating scenario where SSH connections successfully authenticate but then hang indefinitely. The client output shows normal authentication flow:

debug1: Authentication succeeded (publickey).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.

Meanwhile, the server logs show a perfectly normal session establishment:

sshd[12387]: Accepted publickey for user from 1.1.11.239 port 34135 ssh2
pam_unix(sshd:session): session opened for user user by (uid=0)

Before diving deeper, verify these basic checks:

  • Test with different authentication methods: ssh -o PreferredAuthentications=password user@host
  • Check for forced commands in authorized_keys: grep "command=" ~/.ssh/authorized_keys
  • Verify filesystem mounts: df -h and mount

Network issues can manifest as hanging connections. Try these TCP diagnostics:

# On client:
tcpdump -i eth0 host server_ip and port 22

# On server:
tcpdump -i eth0 host client_ip and port 22

Check for packet loss or retransmissions that might indicate network problems.

Investigate these critical SSH server configurations:

# Check for unusual PAM configurations:
cat /etc/pam.d/sshd

# Verify SSH server configuration:
grep -v "^#" /etc/ssh/sshd_config | grep -v "^$"

# Check for unusual shell configurations:
grep user /etc/passwd
ls -la ~user/.bashrc ~user/.bash_profile

When standard methods fail, these approaches can help:

# Run sshd in debug mode:
/usr/sbin/sshd -d -p 2222

# Check for stuck processes:
strace -p <sshd_pid>

# Verify DNS resolution:
getent hosts $(hostname)
dig +short $(hostname)

Sometimes the issue lies deeper in the system:

# Check for TCP stack issues:
sysctl -a | grep tcp

# Look for dropped packets:
netstat -s | grep -i "segments retransmitted"

# Verify system resource limits:
ulimit -a

If you need immediate access while troubleshooting:

# Force a different cipher:
ssh -c aes256-cbc user@host

# Disable pseudo-terminal allocation:
ssh -T user@host

# Use GSSAPI explicitly:
ssh -o GSSAPIAuthentication=no user@host

Remember to document all changes and revert test configurations after troubleshooting.


When working with RHEL 6.4 servers, you might encounter an SSH session that successfully authenticates but then hangs indefinitely. The debug output shows normal key exchange and authentication:

debug1: Authentication succeeded (publickey).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.

Yet the session never proceeds to the shell prompt. Let's explore this systematically.

First, confirm basic network connectivity beyond just the SSH port:

# Check for firewall rules
iptables -L -n -v
# Verify routing
traceroute 10.6.27.64
# Test TCP connection
nc -zv 10.6.27.64 22

On the server side, examine these critical SSH settings:

# Check sshd_config for these parameters:
UseDNS no
GSSAPIAuthentication no
LoginGraceTime 30

For testing, temporarily add these debug options to /etc/ssh/sshd_config:

LogLevel DEBUG3
SyslogFacility AUTHPRIV

Verify critical filesystem paths:

# Check home directory permissions
ls -ld ~user/
# Verify PAM modules
ls -l /etc/pam.d/sshd
# Check SELinux context
ls -Z /home/user/.ssh/

When standard logs don't reveal the issue, try these methods:

# Run sshd in debug mode
/usr/sbin/sshd -d -p 2222
# Then connect with:
ssh -p 2222 -vvv user@host

For stuck sessions, capture network traffic:

tcpdump -i eth0 -s 0 -w ssh.pcap port 22

Based on similar cases, these solutions often work:

# Disable GSSAPI in sshd_config
GSSAPIAuthentication no
# Or try forcing IPv4
ssh -4 user@host

For PAM-related issues:

# Check PAM stack
authconfig --test
# Temporary workaround:
ssh -o PreferredAuthentications=publickey user@host

After identifying the root cause, implement permanent fixes:

# Example: Update crypto policies
update-crypto-policies --set LEGACY
# Or modify sshd_config permanently:
echo "UseDNS no" >> /etc/ssh/sshd_config
systemctl restart sshd