When managing infrastructure at scale, SSH configuration files can quickly become monolithic beasts. A single ~/.ssh/config file containing hundreds or thousands of host definitions creates several practical problems:

• Difficulty in locating specific host configurations
• Version control conflicts when multiple team members edit the file
• Increased risk of syntax errors in large files
• Poor performance with some SSH clients when parsing huge configs

Since OpenSSH 7.3 (released in 2016), we've had the Include directive. This allows splitting configurations across multiple files while maintaining a single entry point:

# ~/.ssh/config
Include config.d/*.conf
Include config.d/production/*.conf
Include config.d/staging/*.conf

Here's how I organize my SSH configurations:

~/.ssh/
├── config              # Main file with Includes
├── config.d/
│   ├── aws.conf        # AWS hosts
│   ├── gcp.conf        # Google Cloud hosts
│   ├── internal.conf   # Corporate network
│   └── legacy/
│       ├── dc1.conf    # Old datacenter 1
│       └── dc2.conf    # Old datacenter 2
└── config.local        # Machine-specific overrides

For teams managing bastion hosts, this pattern works particularly well:

# ~/.ssh/config.d/bastion.conf
Host bastion-prod
  HostName 203.0.113.1
  User jumpuser
  IdentityFile ~/.ssh/bastion_prod_key

Host *.prod.internal
  ProxyJump bastion-prod
  User admin
  IdentityFile ~/.ssh/prod_key

For older SSH versions, you can use these approaches:

# Option 1: cat in shell rc file
alias ssh='cat ~/.ssh/config.d/* | ssh -F /dev/stdin'

# Option 2: Makefile concatenation
all:
    cat config.d/*.conf > config

• Use .conf extension for included files for clarity
• Document your include structure in the main config file
• Set strict file permissions (chmod 600) on all config files
• Consider version controlling your SSH config directory

When managing infrastructure at scale, especially with hundreds of remote hosts accessed through bastion servers, SSH configuration files can quickly become unmanageable. A single ~/.ssh/config file growing beyond 1000 lines presents several challenges:

• Difficult to navigate and maintain
• High risk of merge conflicts in version control
• Performance impact on SSH client startup
• No modularity for team collaboration

Since OpenSSH 7.3 (released 2016), there's been native support for including additional configuration files:

# ~/.ssh/config
Include config.d/*
Include config_bastions
Include config_development

This directive works similarly to how Apache or Nginx handle includes, letting you split configurations logically.

Here's how to structure a modular SSH configuration:

~/.ssh/
├── config              # Main file with includes
├── config.d/
│   ├── 00_common       # Shared settings
│   ├── 10_bastions     # Jump host configurations  
│   ├── 20_production   # Production servers
│   └── 30_development  # Dev environment
└── config_local        # Machine-specific overrides

Example include file (config.d/10_bastions):

# Bastion host configuration
Host bastion-prod
  HostName bastion.example.com
  User ec2-user
  IdentityFile ~/.ssh/prod_bastion_key
  ForwardAgent yes

Host *.prod.internal
  ProxyJump bastion-prod
  User admin
  IdentityFile ~/.ssh/prod_access_key

For systems running older OpenSSH versions, we can use these alternatives:

Option 1: Symbolic Links

cat config.d/* > ~/.ssh/config.tmp
mv ~/.ssh/config.tmp ~/.ssh/config

Option 2: Git Hooks

# .git/hooks/pre-commit
#!/bin/sh
cat config.d/* > config
git add config

For cloud environments where hosts change frequently, consider generating configurations:

#!/bin/bash
# generate_ssh_config.sh
aws ec2 describe-instances \
  --query 'Reservations[*].Instances[*].[Tags[?Key==`Name`].Value[],PublicIpAddress]' \
  --output text | \
while read name ip; do
  echo "Host $name"
  echo "  HostName $ip"
  echo "  User ec2-user"
  echo "  IdentityFile ~/.ssh/aws_key"
done > ~/.ssh/config.d/20_aws_instances

• Set strict file permissions (chmod 600 ~/.ssh/config*)
• Use HashKnownHosts to prevent information leakage
• Consider separate identity files for different environments
• Regularly audit included files for sensitive information

Modern development tools handle includes well:

• VS Code: SSH FS extension respects includes
• JetBrains IDEs: Native SSH config parser understands includes
• Ansible: ssh_connection plugin follows include directives