How to Set Up Real-Time File Mirroring Between Two Ubuntu Servers Using Rsync and Cron


2 views

File mirroring between Ubuntu servers typically involves synchronizing directories in either real-time or scheduled intervals. The most robust approach combines rsync for efficient differential transfers and cron for automation.

Before implementing mirroring, ensure both servers have SSH access configured:

# On primary server (Server A):
ssh-keygen -t rsa
ssh-copy-id user@secondary-server-ip

The simplest one-way sync command:

rsync -avz --delete /path/to/source/ user@secondary-server:/path/to/destination/

Flags explanation:

- -a: Archive mode (preserves permissions, ownership, timestamps)

- -v: Verbose output

- -z: Compression during transfer

- --delete: Removes files on destination that don't exist on source

For immediate synchronization upon file changes, combine rsync with inotify-tools:

sudo apt install inotify-tools

Create a monitoring script (/usr/local/bin/mirror_files.sh):

#!/bin/bash
SOURCE="/var/www/html"
DEST="user@remote-server:/var/www/html"
inotifywait -m -r -e modify,create,delete,move "$SOURCE" | while read path action file; do
    rsync -avz --delete "$SOURCE/" "$DEST/"
done

For periodic synchronization, add this to crontab (crontab -e):

# Sync every 15 minutes
*/15 * * * * rsync -avz --delete /path/to/source/ user@secondary-server:/path/to/destination/

For bidirectional synchronization:

sudo apt install unison
unison /local/path ssh://remote-server//remote/path -auto -batch -times -confirmbigdel=false

1. Use SSH keys instead of passwords

2. Restrict SSH access to specific IPs

3. Consider implementing fail2ban

4. Use read-only rsync modules when appropriate

1. Permission denied errors: Check SELinux contexts and file permissions

2. Slow transfers: Add --compress-level=3 to rsync

3. Connection drops: Use --partial and --timeout=300 flags

4. Large file sets: First run with --dry-run to estimate transfer size

Implement logging for your sync operations:

rsync -avz --delete /source/ user@remote:/dest/ >> /var/log/rsync.log 2>&1

Analyze logs with:

tail -f /var/log/rsync.log
grep -i error /var/log/rsync.log

Mirroring two Ubuntu servers for file synchronization is a common sysadmin task that can be achieved through several robust methods. The most efficient approach combines rsync for delta transfers and inotify-tools for real-time monitoring.

Before implementation, ensure both servers have:

sudo apt update
sudo apt install rsync inotify-tools openssh-server

Set up SSH key authentication between servers to avoid password prompts:

ssh-keygen -t rsa
ssh-copy-id user@secondary-server-ip

For periodic synchronization, create a cron job:

#!/bin/bash
rsync -avz --delete -e ssh /path/to/source/ user@secondary-server-ip:/path/to/destination/

Common rsync flags:

  • -a: Archive mode (preserves permissions, timestamps)
  • -v: Verbose output
  • -z: Compression during transfer
  • --delete: Remove files in destination not present in source

For instant synchronization, implement this watch script:

#!/bin/bash
SOURCE="/path/to/watch"
DEST="user@remote-server:/path/to/destination"
inotifywait -m -r -e modify,create,delete,move "$SOURCE" | while read path action file; do
    rsync -avz --delete "$SOURCE" "$DEST"
done

Explanation of inotifywait options:

  • -m: Monitor continuously
  • -r: Recursive monitoring
  • -e: Events to watch (modify,create,delete,move)

For bidirectional synchronization, install and configure Unison:

sudo apt install unison
unison /local/path ssh://remote-server//remote/path -auto -batch -confirmbigdel=false

Create a profile (~/.unison/default.prf) for persistent configuration:

root = /local/path
root = ssh://remote-server//remote/path
auto = true
batch = true

For enterprise deployments:

  • Implement lsyncd for daemonized sync operations
  • Set up proper logging with --log-file in rsync
  • Configure bandwidth throttling with --bwlimit during peak hours
  • Implement file locking mechanisms for database files

Frequent challenges and solutions:

# Permission issues:
rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r

# Connection drops:
rsync --partial --progress

# Large file sets:
rsync --max-size=1G --exclude='*.tmp'

Regularly verify sync integrity with:

rsync -n -avzi --dry-run /source/ user@remote:/destination/