Efficient Network Transfer of Logical Volumes Between Linux Servers Without Intermediate Storage


3 views

When migrating KVM virtual machines between hypervisors, the conventional approach of creating intermediate disk images can become impractical with large logical volumes. The standard workflow:

# Source host
dd if=/dev/vg00/lv_vm1 of=vm1.img bs=4M
scp vm1.img user@newhost:/tmp/

# Destination host 
dd if=/tmp/vm1.img of=/dev/vg_new/lv_vm1
lvresize -L500G /dev/vg_new/lv_vm1

This method doubles storage requirements during transfer - a critical limitation when dealing with large volumes on constrained storage systems.

Several methods exist for direct LV-to-LV transfer without intermediate files:

Method 1: Combined SSH and DD Pipeline

# On destination host:
ssh root@sourcehost "dd if=/dev/vg00/lv_vm1 bs=4M" | \
dd of=/dev/vg_new/lv_vm1 bs=4M

This creates a direct pipe between the source LV and destination LV through SSH encryption.

Method 2: Using netcat for Local Network Transfers

# On destination host (receiver first):
nc -l 2222 | dd of=/dev/vg_new/lv_vm1 bs=4M

# On source host (within 60 seconds):
dd if=/dev/vg00/lv_vm1 bs=4M | nc destination_host 2222

Method 3: LVM-Specific Network Mirroring

# On destination host:
lvcreate -L500G -n lv_vm1 vg_new
lvchange --refresh vg_new/lv_vm1

# On source host:
dd if=/dev/vg00/lv_vm1 bs=4M | \
ssh root@destinationhost "dd of=/dev/vg_new/lv_vm1 bs=4M"

For production environments, consider these enhancements:

# Add compression to the pipeline:
dd if=/dev/vg00/lv_vm1 bs=4M | gzip -c | \
ssh root@destinationhost "gunzip -c | dd of=/dev/vg_new/lv_vm1 bs=4M"

# Progress monitoring with pv:
dd if=/dev/vg00/lv_vm1 bs=4M | pv -s 500G | \
ssh root@destinationhost "dd of=/dev/vg_new/lv_vm1 bs=4M"

Always verify transfers using checksums:

# Source checksum:
sha256sum /dev/vg00/lv_vm1

# Destination checksum after transfer:
ssh root@destinationhost "sha256sum /dev/vg_new/lv_vm1"

For critical systems, consider using rsync for subsequent delta transfers after the initial bulk copy.


When migrating KVM virtual machines between hypervisors using Logical Volume Manager (LVM), the traditional approach creates significant storage overhead:

# Traditional method requiring 2x storage
dd if=/dev/vg_kvm/lv_vm1 of=/mnt/temp/vm1.img bs=4M
scp /mnt/temp/vm1.img newhost:/mnt/temp/
ssh newhost "dd if=/mnt/temp/vm1.img of=/dev/vg_new/lv_vm1"

For a 500GB VM, this means temporarily needing 1TB just for the transfer process - not feasible when dealing with limited storage.

Several methods exist for direct LV-to-LV transfer over network:

Method 1: Using netcat with dd

# On destination host:
nc -l 1234 | dd of=/dev/vg_new/lv_vm1

# On source host:
dd if=/dev/vg_kvm/lv_vm1 bs=4M | nc destination_host 1234

Method 2: LVM-specific network mirroring

# First prepare the target LV
lvcreate -n lv_vm1 -L 500G vg_new

# Then use LVM mirroring over network:
lvconvert --mirrorlog networked \
          --mirrorlog core \
          --type mirror \
          -b /dev/vg_kvm/lv_vm1 \
          --destination vg_new/lv_vm1

Method 3: DRBD for block-level replication

Setup involves more configuration but provides continuous sync:

# /etc/drbd.conf configuration snippet:
resource vm1 {
  device /dev/drbd0;
  disk /dev/vg_kvm/lv_vm1;
  meta-disk internal;
  on source_host {
    address 192.168.1.100:7788;
  }
  on dest_host {
    address 192.168.1.101:7788;
  }
}

When transferring large volumes:

  • Use larger block sizes (4M-16M) with dd
  • Compress during transfer with gzip/lz4
  • Consider network bandwidth limitations
  • Monitor with iostat -x 1 during transfer

All these methods transmit raw data unencrypted. For production environments:

# Example encrypted transfer with SSH:
dd if=/dev/vg_kvm/lv_vm1 bs=4M | gzip | \
  ssh user@newhost "gzip -d | dd of=/dev/vg_new/lv_vm1"

Remember to verify checksums after transfer:

sha256sum /dev/vg_kvm/lv_vm1
sha256sum /dev/vg_new/lv_vm1