How to Configure SSH Public Key Authentication for Root User on Linux Server


3 views

Many administrators face this common scenario: You've set up SSH key authentication for regular users but need secure root access. The standard approach of simply adding keys to /root/.ssh/authorized_keys might not work due to server security configurations.

Most modern Linux distributions disable root SSH access by default in /etc/ssh/sshd_config:

# Example of default restrictive setting
PermitRootLogin prohibit-password

This means root can only login with key authentication, but we need to ensure proper key setup.

Here's the complete process to enable SSH key access for root:

1. Generate SSH Key Pair (Local Machine)

ssh-keygen -t rsa -b 4096 -f ~/.ssh/root_server_key

2. Transfer Public Key to Server

Either manually create the directory structure or use ssh-copy-id with sudo:

ssh-copy-id -i ~/.ssh/root_server_key.pub -p 30000 user@IP_ADDRESS
ssh -p 30000 user@IP_ADDRESS "sudo mkdir -p /root/.ssh"
ssh -p 30000 user@IP_ADDRESS "sudo cp ~/.ssh/authorized_keys /root/.ssh/"

3. Verify Permissions

Correct permissions are crucial for SSH to work:

ssh -p 30000 user@IP_ADDRESS "sudo chmod 700 /root/.ssh"
ssh -p 30000 user@IP_ADDRESS "sudo chmod 600 /root/.ssh/authorized_keys"
ssh -p 30000 user@IP_ADDRESS "sudo chown -R root:root /root/.ssh"

Edit /etc/ssh/sshd_config for root access:

# Enable root login with key authentication only
PermitRootLogin prohibit-password

# Optionally restrict root login to specific IPs
Match Address 192.168.1.100
    PermitRootLogin yes

Reload SSH service after changes:

sudo systemctl reload ssh

Attempt to connect using your private key:

ssh -i ~/.ssh/root_server_key -p 30000 root@IP_ADDRESS
  • Use passphrase-protected keys
  • Consider implementing two-factor authentication
  • Monitor root login attempts in /var/log/auth.log
  • Set up fail2ban for brute force protection
  • Regularly rotate SSH keys

If connection fails, check these common issues:

# Verify SELinux context (if enabled)
sudo restorecon -Rv /root/.ssh

# Check SSH server logs
journalctl -u ssh --no-pager -n 50

# Test with verbose output
ssh -vvv -i ~/.ssh/root_server_key root@IP_ADDRESS

For easier management, create an entry in ~/.ssh/config:

Host production-root
    HostName IP_ADDRESS
    Port 30000
    User root
    IdentityFile ~/.ssh/root_server_key
    IdentitiesOnly yes

Then simply connect using:

ssh production-root

When setting up a new Linux server, many administrators follow the security best practice of disabling direct root SSH login. However, there are legitimate cases where key-based root access is necessary for system maintenance. The challenge arises when you need to configure SSH keys for the root account after initial setup.

Before proceeding, ensure you have:

  • Existing non-root user SSH access (your "user@IP_ADDRESS -p 30000" access)
  • sudo privileges on the server
  • Your public SSH key ready (~/.ssh/id_rsa.pub on your local machine)

Here's how to properly set up SSH key authentication for root:

# First, access your server using your regular user account
ssh -p 30000 user@IP_ADDRESS

# Then switch to root (using sudo)
sudo su -

# Create the .ssh directory if it doesn't exist
mkdir -p /root/.ssh

# Set correct permissions
chmod 700 /root/.ssh

# Create or append to authorized_keys file
nano /root/.ssh/authorized_keys

Paste your public key (from id_rsa.pub) into the file, save and exit. Then:

# Set proper permissions for authorized_keys
chmod 600 /root/.ssh/authorized_keys

# Verify ownership
chown -R root:root /root/.ssh

Edit your SSH server configuration:

nano /etc/ssh/sshd_config

Ensure these settings are present:

PermitRootLogin prohibit-password
PubkeyAuthentication yes
PasswordAuthentication no

Restart the SSH service:

service ssh restart
# or for systemd systems:
systemctl restart sshd

From your local machine, test the connection:

ssh -p 30000 -i ~/.ssh/id_rsa root@IP_ADDRESS
  • Check /var/log/auth.log for SSH connection errors
  • Verify file permissions (root should own all files in /root/.ssh/)
  • Ensure SELinux isn't blocking access (if applicable)
  • Double-check your sshd_config syntax

For enhanced security, consider using SSH agent forwarding instead of direct root key access:

ssh -A -p 30000 user@IP_ADDRESS
sudo -i

This allows you to maintain your personal key while temporarily elevating privileges.