Best Practices for Managing Multi-Server Configs in Git: Solving Sparse Checkout & Shared Repo Challenges


2 views

Migrating server configuration management from Subversion to Git introduces unique architectural hurdles. Our legacy SVN structure looked like this:

repo/
├── www.domain.com/
│   └── etc/apache2/
├── dev.domain.com/
│   ├── etc/
│   └── opt/app1/conf/
└── staging.domain.com/

SVN's svn:externals and partial checkouts made this trivial, but Git requires different approaches.

Option 1: Symlink-Based Deployment

This maintains a canonical Git repo in a central location with symlinks to actual config locations:

# Central repo location
/var/git/configs/
├── .git
├── www.domain.com/
├── dev.domain.com/
└── staging.domain.com/

# Deployment example (run as root):
ln -s /var/git/configs/www.domain.com/etc/apache2 /etc/apache2

Pros:

  • Single source of truth
  • Preserves Git's full functionality

Cons:

  • Requires root access for symlinks
  • File-level symlinks become unwieldy

Option 2: Multi-Branch Architecture

Each environment gets its own branch with shared base configurations:

# Create branch structure
git checkout -b base_config
# Add common configurations...

git checkout -b www.domain.com
# Merge common configs
git merge base_config

# Push to remote
git push origin www.domain.com

Deploy with:

git clone --branch www.domain.com git@server:configs.git /etc

Git's sparse checkout can work with some path normalization:

# Initialize repo
git init /etc && cd /etc
git remote add origin git@server:configs.git
git config core.sparseCheckout true

# Configure desired paths
echo "www.domain.com/etc/apache2/*" >> .git/info/sparse-checkout

# Pull specific configs
git pull origin main

Sample contribution workflow:

# Clone specific config subset
git clone --filter=blob:none --no-checkout git@server:configs.git
cd configs
git sparse-checkout init --cone
git sparse-checkout set www.domain.com/etc/apache2

# Make changes
vim www.domain.com/etc/apache2/httpd.conf

# Commit and push
git commit -m "Update Apache timeout settings" --author="Admin "
git push origin HEAD

Example post-receive hook for automatic deployment:

#!/bin/bash
while read oldrev newrev refname
do
    branch=${refname#refs/heads/}
    case $branch in
        www.domain.com)
            rsync -av --delete /git/repo/www.domain.com/etc/ /etc/
            systemctl reload apache2
            ;;
        dev.domain.com)
            # Dev server deployment logic
            ;;
    esac
done

Many DevOps teams face significant challenges when migrating server configurations from SVN to Git. Our team recently navigated this transition, discovering several key differences that impact workflow:

// Legacy SVN structure example
/repo-root
├── prod-server
│   ├── etc
│   │   └── nginx
│   └── opt
├── staging-server
│   └── etc
│       └── apache2
└── dev-server
    └── home
        └── app-configs

After extensive testing, we established these non-negotiable requirements:

  • Single source of truth (monorepo approach)
  • Granular access control per environment
  • Preservation of commit authorship metadata
  • Support for partial checkouts

We evaluated multiple approaches before settling on these effective patterns:

Solution 1: Sparse Checkout with Git Worktrees

# Initialize repository
git init --bare /var/git/config-repo.git
git clone /var/git/config-repo.git
cd config-repo

# Configure sparse checkout
git config core.sparseCheckout true
echo "prod-server/etc/nginx/*" >> .git/info/sparse-checkout
echo "prod-server/etc/haproxy/*" >> .git/info/sparse-checkout

# Create worktree for production
git worktree add ../prod-configs prod-config

Solution 2: Git Submodules for Shared Configs

# Main repo structure
config-repo/
├── .gitmodules
├── base-templates/  # Shared configurations
├── prod-overrides/
└── dev-overrides/

# .gitmodules example
[submodule "base-templates"]
    path = base-templates
    url = git@github.com:org/base-configs.git
    branch = main

For systems requiring original file paths:

#!/bin/bash
# Deployment script example
REPO_ROOT="/var/git/configs"
TARGET="/etc/nginx"

# Sync and link configuration
rsync -av --delete \
    "${REPO_ROOT}/prod-server/etc/nginx/" \
    "${TARGET}/"

# Maintain original file permissions
chown -R root:root "${TARGET}"
chmod -R 644 "${TARGET}"

Key lessons from our implementation:

  • Use .gitignore aggressively for sensitive files
  • Implement pre-commit hooks for config validation
  • Leverage Git attributes for line ending normalization
  • Consider Git LFS for binary config files

Our current branching model:

git branch
  main
  * production
  staging
  development
  feature/ssl-update

This allows environment-specific changes while maintaining the ability to merge common updates across environments.