Troubleshooting SSH Daemon Startup Failures: Key Generation and Port Binding Issues


17 views

When the OpenSSH daemon fails to start, systemd typically provides detailed error messages that can help diagnose the problem. In this case, we're seeing two distinct phases of failure:

1. Initial failure due to missing host keys (status=1/FAILURE)
2. Subsequent failure due to port binding issues (status=255/n/a)

The first error indicated missing SSH host keys, which is a common issue especially on fresh installations. The solution is straightforward:

# Generate all missing host keys
sudo ssh-keygen -A

# Verify configuration
sudo sshd -t

This generates RSA, DSA, ECDSA, and ED25519 keys in /etc/ssh/. The sshd -t command tests the configuration without starting the service.

After resolving the key issue, we encounter a more complex problem:

fatal: Cannot bind any address

This suggests the daemon can't bind to its configured port (usually 22). Potential causes include:

  • Another service using port 22
  • Firewall/iptables rules blocking access
  • Incorrect ListenAddress configuration

First, check if port 22 is already in use:

sudo netstat -tulnp | grep :22
sudo lsof -i :22

If another process is using the port, either stop that service or configure SSH to use a different port in /etc/ssh/sshd_config:

# Example alternative port configuration
Port 2222
#ListenAddress 0.0.0.0
#AddressFamily any

For systems with complex networking, additional checks may be needed:

# Check SELinux status (if applicable)
sudo sestatus

# Verify systemd socket activation
sudo systemctl status sshd.socket

# Check for IP conflicts
ip addr show

After making changes, follow this sequence:

# Reload systemd to recognize changes
sudo systemctl daemon-reload

# Restart SSH service
sudo systemctl restart sshd

# Verify status
sudo systemctl status sshd

Remember that systemd has a start rate limiting mechanism. If you see "start request repeated too quickly", wait a few minutes before retrying or use:

sudo systemctl reset-failed sshd.service

When examining the systemd logs, we observe two distinct failure patterns:

1. Initial failure due to missing host keys (status=1/FAILURE)
2. Subsequent failure with "Cannot bind any address" (status=255/n/a)

The first error indicates missing SSH host keys. This can be resolved by:

# Generate all missing host keys
sudo ssh-keygen -A

# Typical output:
Generating public/private rsa1 key pair.
Your identification has been saved in /etc/ssh/ssh_host_key
[...]
Generating public/private ed25519 key pair.
Your identification has been saved in /etc/ssh/ssh_host_ed25519_key

The critical error fatal: Cannot bind any address suggests port allocation problems. Diagnostic steps:

# Check current network configuration
ip a

# Verify port availability
sudo netstat -tulnp | grep :22

# Alternative check using ss
ss -tulnp | grep ssh

Examine the SSH daemon configuration for potential conflicts:

# Check active configuration
sudo sshd -T | grep -E 'ListenAddress|Port'

# Common problematic configurations to check:
# 1. Multiple ListenAddress directives
# 2. Port conflicts (22 vs 2222)
# 3. IPv4 vs IPv6 binding issues

The start-limit error indicates rapid restart attempts. Temporary workaround:

# Reset the start limit counter
sudo systemctl reset-failed sshd.service

# Manual start with debug output
sudo /usr/sbin/sshd -d -p 2222

For persistent issues, consider strace debugging:

# Trace system calls during startup
sudo strace -f -e trace=network /usr/sbin/sshd -D -d -p 2222

# Check SELinux/AppArmor denials
sudo ausearch -m avc -ts recent
sudo dmesg | grep -i denied

As a last resort, perform a full service reset:

# Stop and clean the service
sudo systemctl stop sshd
sudo rm -fv /etc/ssh/ssh_host_*

# Regenerate keys and config
sudo ssh-keygen -A
sudo sshd -t

# Full systemd reload
sudo systemctl daemon-reload
sudo systemctl restart sshd

While troubleshooting, consider alternative access methods:

# Temporary console access
sudo systemctl start getty@tty1.service

# Web-based console (if available)
sudo systemctl start cockpit.socket