The ssh-keygen tool generates public keys in OpenSSH's proprietary format, while PEM (Privacy Enhanced Mail) format follows X.509 standards. These formats serve different purposes in the SSH ecosystem:

# OpenSSH format (generated by ssh-keygen)
ssh-rsa AAAAB3NzaC1yc2EAAA... user@host

# PEM format (common in web servers)
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA...
-----END PUBLIC KEY-----

The ssh-keygen -l command expects the OpenSSH format because it needs to process the key data in a specific way to generate the fingerprint. When you feed it a PEM file, it fails because:

• PEM files contain additional headers/footers
• The base64 encoding structure differs
• PEM may include certificate chains while OpenSSH expects raw public keys

To work with both formats interchangeably, you'll need conversion tools. Here are practical methods:

# Convert OpenSSH public key to PEM format
ssh-keygen -f id_rsa.pub -e -m PEM > id_rsa.pem

# Convert PEM to OpenSSH format
ssh-keygen -f id_rsa.pem -i -m PKCS8 > id_rsa_openssh.pub

# Extract fingerprint from PEM file (alternative method)
openssl rsa -in private_key.pem -pubout -outform der | openssl md5 -c

For different key formats, use these commands:

# For OpenSSH .pub files
ssh-keygen -lf id_rsa.pub

# For PEM certificates
openssl x509 -in certificate.pem -noout -fingerprint -sha256

# For raw RSA public keys
openssl rsa -in key.pem -pubin -outform der | openssl sha256 -binary | openssl base64

Here's how to handle conversions programmatically using Python's cryptography library:

from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.backends import default_backend

# Load PEM file
with open("key.pem", "rb") as key_file:
    pem_key = serialization.load_pem_public_key(
        key_file.read(),
        backend=default_backend()
    )

# Convert to OpenSSH format
openssh_key = pem_key.public_bytes(
    encoding=serialization.Encoding.OpenSSH,
    format=serialization.PublicFormat.OpenSSH
)

print(openssh_key.decode('utf-8'))

When converting between formats:

• Always verify the fingerprint matches after conversion
• Never expose private keys during conversion processes
• Use SHA-256 instead of MD5 for fingerprint generation when possible
• Be aware that some older systems might still require MD5 fingerprints

When working with SSH keys, you'll encounter two primary formats for public keys:

# ssh-keygen format (RFC 4716)
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQ... user@host

# PEM format (RFC 7468)
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArZN/ME3LC...
-----END PUBLIC KEY-----

The ssh-keygen generated .pub files use the RFC 4716 format, while PEM files follow RFC 7468. The main differences:

• Structure: The .pub format includes the key type, base64-encoded key, and optional comment
• Usage: SSH primarily uses the .pub format for authorized_keys
• Compatibility: Most SSH tools expect the RFC 4716 format

To get the fingerprint of a PEM-encoded public key:

# Convert PEM to SSH public key format
openssl rsa -in id_rsa -pubout -outform PEM | ssh-keygen -f /dev/stdin -i -m PKCS8

# Get fingerprint directly from PEM
openssl pkey -in public.pem -pubin -outform der | openssl md5 -c

Example conversion workflow:

# Generate a new key pair
openssl genrsa -out private.pem 2048
openssl rsa -in private.pem -pubout -out public.pem

# Convert PEM to SSH format
ssh-keygen -i -m PKCS8 -f public.pem > public_ssh.pub

# Verify fingerprints match
openssl pkey -in public.pem -pubin -outform der | openssl md5 -c
ssh-keygen -l -f public_ssh.pub

If you encounter "not a public key file" errors:

• Ensure your PEM file contains the public key (not private)
• Verify the PEM headers are correct (BEGIN/END PUBLIC KEY)
• Try explicitly specifying the input format: ssh-keygen -i -m PEM -f key.pem

Python example to extract fingerprints from both formats:

import hashlib
import base64
import re

def ssh_pubkey_fingerprint(pubkey):
    # For standard ssh pubkey format
    key_type, b64_data, _ = pubkey.split()
    key_bytes = base64.b64decode(b64_data)
    return hashlib.md5(key_bytes).hexdigest()

def pem_fingerprint(pem_data):
    # For PEM format
    b64_data = "".join(pem_data.splitlines()[1:-1])
    key_bytes = base64.b64decode(b64_data)
    return hashlib.md5(key_bytes).hexdigest()