Troubleshooting SSH Daemon Startup Failures: Key Generation and Port Binding Issues


2 views

When the OpenSSH daemon fails to start, systemd typically provides detailed error messages that can help diagnose the problem. In this case, we're seeing two distinct phases of failure:

1. Initial failure due to missing host keys (status=1/FAILURE)
2. Subsequent failure due to port binding issues (status=255/n/a)

The first error indicated missing SSH host keys, which is a common issue especially on fresh installations. The solution is straightforward:

# Generate all missing host keys
sudo ssh-keygen -A

# Verify configuration
sudo sshd -t

This generates RSA, DSA, ECDSA, and ED25519 keys in /etc/ssh/. The sshd -t command tests the configuration without starting the service.

After resolving the key issue, we encounter a more complex problem:

fatal: Cannot bind any address

This suggests the daemon can't bind to its configured port (usually 22). Potential causes include:

  • Another service using port 22
  • Firewall/iptables rules blocking access
  • Incorrect ListenAddress configuration

First, check if port 22 is already in use:

sudo netstat -tulnp | grep :22
sudo lsof -i :22

If another process is using the port, either stop that service or configure SSH to use a different port in /etc/ssh/sshd_config:

# Example alternative port configuration
Port 2222
#ListenAddress 0.0.0.0
#AddressFamily any

For systems with complex networking, additional checks may be needed:

# Check SELinux status (if applicable)
sudo sestatus

# Verify systemd socket activation
sudo systemctl status sshd.socket

# Check for IP conflicts
ip addr show

After making changes, follow this sequence:

# Reload systemd to recognize changes
sudo systemctl daemon-reload

# Restart SSH service
sudo systemctl restart sshd

# Verify status
sudo systemctl status sshd

Remember that systemd has a start rate limiting mechanism. If you see "start request repeated too quickly", wait a few minutes before retrying or use:

sudo systemctl reset-failed sshd.service

When examining the systemd logs, we observe two distinct failure patterns:

1. Initial failure due to missing host keys (status=1/FAILURE)
2. Subsequent failure with "Cannot bind any address" (status=255/n/a)

The first error indicates missing SSH host keys. This can be resolved by:

# Generate all missing host keys
sudo ssh-keygen -A

# Typical output:
Generating public/private rsa1 key pair.
Your identification has been saved in /etc/ssh/ssh_host_key
[...]
Generating public/private ed25519 key pair.
Your identification has been saved in /etc/ssh/ssh_host_ed25519_key

The critical error fatal: Cannot bind any address suggests port allocation problems. Diagnostic steps:

# Check current network configuration
ip a

# Verify port availability
sudo netstat -tulnp | grep :22

# Alternative check using ss
ss -tulnp | grep ssh

Examine the SSH daemon configuration for potential conflicts:

# Check active configuration
sudo sshd -T | grep -E 'ListenAddress|Port'

# Common problematic configurations to check:
# 1. Multiple ListenAddress directives
# 2. Port conflicts (22 vs 2222)
# 3. IPv4 vs IPv6 binding issues

The start-limit error indicates rapid restart attempts. Temporary workaround:

# Reset the start limit counter
sudo systemctl reset-failed sshd.service

# Manual start with debug output
sudo /usr/sbin/sshd -d -p 2222

For persistent issues, consider strace debugging:

# Trace system calls during startup
sudo strace -f -e trace=network /usr/sbin/sshd -D -d -p 2222

# Check SELinux/AppArmor denials
sudo ausearch -m avc -ts recent
sudo dmesg | grep -i denied

As a last resort, perform a full service reset:

# Stop and clean the service
sudo systemctl stop sshd
sudo rm -fv /etc/ssh/ssh_host_*

# Regenerate keys and config
sudo ssh-keygen -A
sudo sshd -t

# Full systemd reload
sudo systemctl daemon-reload
sudo systemctl restart sshd

While troubleshooting, consider alternative access methods:

# Temporary console access
sudo systemctl start getty@tty1.service

# Web-based console (if available)
sudo systemctl start cockpit.socket