When encountering the "Connection reset by peer" error during SSH authentication to a Solaris 10 server, we can observe several characteristic behaviors:

Jun  8 19:51:05 ******* sshd[26391]: [ID 800047 auth.crit] fatal: Read from socket failed: Connection reset by peer

The client-side debug output shows the connection terminates immediately after the key exchange initialization:

debug1: SSH2_MSG_KEXINIT sent
Read from socket failed: Connection reset by peer

The server logs reveal a critical clue about the authentication failure:

debug1: Failed to acquire GSS-API credentials for any mechanisms (No credentials were supplied, or the credentials were unavailable or inaccessible

Solaris 10's default SSH configuration often includes GSS-API authentication as a mandatory method, which can cause connection issues when the client can't provide proper Kerberos credentials.

First check the current SSH server configuration:

# grep GSSAPI /etc/ssh/sshd_config
GSSAPIAuthentication yes
GSSAPIKeyExchange yes
GSSAPICleanupCredentials yes

The quickest solution is to disable GSS-API authentication temporarily:

# svcadm disable ssh
# sed -i 's/^GSSAPIAuthentication yes/GSSAPIAuthentication no/' /etc/ssh/sshd_config
# sed -i 's/^GSSAPIKeyExchange yes/GSSAPIKeyExchange no/' /etc/ssh/sshd_config
# svcadm enable ssh

For environments where some clients need GSS-API while others don't, create a match block:

Match Address 192.168.1.0/24
    GSSAPIAuthentication no
    GSSAPIKeyExchange no

When troubleshooting similar issues, follow this comprehensive approach:

# Client-side debugging:
ssh -vvv user@hostname

# Server-side debugging:
/usr/lib/ssh/sshd -d -p 22

# Network-level verification:
tcpdump -i any -n port 22 and host client.ip.address

If completely disabling GSS-API isn't desirable, consider configuring alternative methods:

# /etc/ssh/sshd_config
AuthenticationMethods publickey,gssapi-with-mic
PubkeyAuthentication yes
PasswordAuthentication no

The debug output shows a potential version mismatch:

debug1: Remote protocol version 2.0, remote software version Sun_SSH_1.1
debug1: no match: Sun_SSH_1.1

Consider upgrading either the client or server SSH implementation if other solutions don't work.

---

Encountering a silent SSH failure where connections get abruptly reset during key exchange is particularly frustrating when basic connectivity checks (ping/telnet) work fine. The debug logs reveal critical clues:

debug1: Failed to acquire GSS-API credentials for any mechanisms
debug1: SSH2_MSG_KEXINIT sent
Read from socket failed: Connection reset by peer

Solaris 10's SunSSH has some unique behaviors compared to OpenSSH:

• Default GSSAPI authentication attempts even when not configured
• Stricter key exchange protocol handling
• Different cipher suite defaults

Try these client-side parameters first:

ssh -o GSSAPIAuthentication=no \
    -o KexAlgorithms=diffie-hellman-group-exchange-sha1 \
    -c aes128-ctr user@host

Edit /etc/ssh/sshd_config:

GSSAPIAuthentication no
KexAlgorithms diffie-hellman-group-exchange-sha1
Ciphers aes128-ctr,aes192-ctr,aes256-ctr
UsePAM yes

The error occurs during the cryptographic negotiation phase. Packet captures show:

1. Client sends SSH2_MSG_KEXINIT
2. Server responds with its parameters
3. Connection resets before algorithm selection completes

After making changes, verify with:

svcadm restart ssh
sshd -t # Test config syntax
netstat -an | grep 32222 # Check listening port

For persistent issues, increase logging on both ends:

# Client:
ssh -vvvv -E ssh_debug.log user@host

# Server:
/usr/lib/ssh/sshd -d -d -d -p 32222 > /var/log/sshd_debug.log

Consider these nuclear options:

• Reinstall SunSSH packages
• Compile OpenSSH from source
• Check for kernel tunables affecting TCP:

ndd /dev/tcp tcp_extra_priv_ports_add 32222