Best Practices for Backing Up iSCSI SAN Storage on a Budget


2 views

When implementing an iSCSI SAN in resource-constrained environments, traditional enterprise backup solutions often become cost-prohibitive. The fundamental question revolves around creating reliable backups without expensive proprietary systems or duplicate storage arrays.

For Linux-based environments (as described in the scenario), these approaches have proven effective:

1. Host-Level Backup Strategy
Each server accessing SAN LUNs runs its own backup agent. For example:

# Example rsnapshot config for backing up Xen DomU volumes
config_version  2
snapshot_root   /backup/san/
cmd_cp          /bin/cp
cmd_rsync       /usr/bin/rsync
cmd_ssh         /usr/bin/ssh
interval        hourly  24
interval        daily   7
interval        weekly  4
backup  /mnt/san/xen_guests/  xen_backup/
backup  /var/lib/mysql/       db_backup/

2. NFS-Based SAN Backup
For the NFS-exported home directories:

# /etc/exports entry for backup access
/backup_target  192.168.1.100(rw,sync,no_subtree_check)
# rsync command for incremental backups
rsync -az --delete --numeric-ids /san/nfs_homes/ backup-server:/san_backups/nfs/

Instead of expensive tape libraries or secondary SANs:

  • Build a ZFS-based backup server with commodity hardware
  • Use cloud storage with rclone for offsite copies
  • Implement LVM snapshots for point-in-time recovery

For MySQL/MariaDB on SAN storage:

# mysqldump with SAN-aware options
mysqldump --single-transaction --master-data=2 \
--all-databases | gzip > /backup/san/db/full-$(date +%F).sql.gz

Xen-specific backup script example:

#!/bin/bash
# Backup Xen DomU configs
tar czf /backup/san/xen_configs-$(date +%s).tar.gz /etc/xen/auto/

# Snapshot running DomUs
for vm in $(xl list | awk '/^[0-9]/ {print $1}'); do
  xl save $vm /backup/san/$vm-$(date +%s).save
done

Implement these critical checks:

# Nagios check for backup completion
#!/bin/bash
if [ $(find /backup/san/ -mtime -1 | wc -l) -gt 0 ]; then
  echo "OK: Recent backups exist"
  exit 0
else
  echo "CRITICAL: No recent backups found"
  exit 2
fi

When implementing an iSCSI SAN (whether using OpenFiler, MD3000i, or MSA2000i), the distributed nature of storage creates unique backup challenges. Unlike traditional server backups where data resides locally, SAN environments separate storage from compute resources.

Option 1: Host-Level Backups
For your described environment (code repos, DBs, Xen guests, NFS homes), consider these open-source tools:


# Example rsnapshot config for backing up NFS-mounted home directories
config_version  1.2
snapshot_root   /backups/san/
cmd_cp          /bin/cp
cmd_rm          /bin/rm
cmd_rsync       /usr/bin/rsync
retain          hourly  6
retain          daily   7
retain          weekly  4
backup          /mnt/nfs/homes/    server1/homes/
backup          /var/lib/mysql/    db1/mysql/

Option 2: SAN-Native Snapshots
Many iSCSI targets support LUN snapshots. For OpenFiler:


# Create LVM snapshot (example for XEN guests volume)
lvcreate -L10G -s -n xen_backup /dev/vg_san/xen_guests
# Mount snapshot for backup
mkdir /mnt/tempbackup
mount /dev/vg_san/xen_backup /mnt/tempbackup

For budget environments, consider these storage targets:

  • ZFS-based backup server with compression
  • Used enterprise JBOD with consumer drives
  • Cloud storage backends like Backblaze B2

For MySQL/MariaDB on SAN:


# Minimal-impact backup using mysqldump with SAN snapshot
FLUSH TABLES WITH READ LOCK;
SYSTEM lvcreate -L5G -s -n db_snap /dev/vg_san/mysql_data;
UNLOCK TABLES;
# Then backup from the snapshot volume

Combine these tools for a complete solution:


#!/bin/bash
# SAN backup wrapper script
case "$1" in
  xen)
    lvcreate -s -n xen_$(date +%Y%m%d) /dev/vg_san/xen
    mount /dev/vg_san/xen_$(date +%Y%m%d) /mnt/backup
    rsync -a /mnt/backup/ /backupnas/xen/
    umount /mnt/backup
    lvremove -f /dev/vg_san/xen_$(date +%Y%m%d)
    ;;
  db)
    # DB backup logic here
    ;;
esac

Remember to test restore procedures regularly - a backup without verified restores is just expensive wishful thinking.