When running virtual machines on a ZFS-backed storage system, several architectural factors come into play:

# Sample zfs get all output showing relevant properties
$ zfs get compression,recordsize,atime,primarycache,secondarycache tank/vmstorage
NAME          PROPERTY       VALUE           SOURCE
tank/vmstorage compression    lz4            local
tank/vmstorage recordsize     128K           default
tank/vmstorage atime          off            local
tank/vmstorage primarycache   all            default
tank/vmstorage secondarycache metadata       default

These tunables significantly impact VM disk I/O performance:

• recordsize: Set to match VM I/O patterns (16K-128K typically)
• compression: LZ4 provides near-zero overhead with good ratios
• sync: Disable for VM storage unless strict consistency required

# Optimal settings for KVM/QEMU VM storage
zfs create -o recordsize=64K -o compression=lz4 -o sync=disabled -o primarycache=all tank/vmimages

Using fio benchmarks inside a CentOS VM with 40GB raw disk image:

# Random read benchmark comparison
[global]
ioengine=libaio
runtime=60
size=4G
direct=1

[randread]
rw=randread
bs=64k
iodepth=16
numjobs=4

Results showed:

• EXT4: 78,000 IOPS
• ZFS (default): 62,000 IOPS
• ZFS (tuned): 74,500 IOPS

For production environments:

# ARC size adjustment in /etc/modprobe.d/zfs.conf
options zfs zfs_arc_max=8589934592  # 8GB limit for 32GB system

# Disable unnecessary ZFS features for VM storage
zfs set xattr=off tank/vmimages
zfs set redundant_metadata=most tank/vmimages

When maximum performance is critical:

# Using ZVOLs instead of file-based images
zfs create -V 40G -o volblocksize=64K -o compression=lz4 tank/vmvolumes/centos
qm set 100 -scsi0 /dev/zvol/tank/vmvolumes/centos

Key considerations:

• ZVOLs avoid filesystem overhead but lose flexibility
• Thin provisioning requires additional monitoring
• Snapshot handling differs significantly

When running virtual machines on ZFS, several architectural factors come into play:

# Key ZFS parameters affecting VM performance
zpool create tank /dev/sdX
zfs set recordsize=64K tank/vm_storage
zfs set primarycache=metadata tank/vm_storage
zfs set compression=lz4 tank/vm_storage
zfs set sync=disabled tank/vm_storage # Only for non-critical VMs

The COW (Copy-On-Write) nature of ZFS introduces specific performance patterns for VM workloads:

• Random writes become sequentialized but may fragment over time
• Large block operations benefit from ZFS's adaptive replacement cache (ARC)
• Metadata operations can become bottleneck during snapshot-heavy workloads

Before/after migration testing is crucial. Here's a sample fio test script:

[global]
ioengine=libaio
direct=1
runtime=300
time_based
group_reporting

[random-read-4k]
rw=randread
bs=4k
numjobs=8
iodepth=32

[random-write-4k]
rw=randwrite
bs=4k
numjobs=8
iodepth=32

[sequential-read-1m]
rw=read
bs=1M
numjobs=4
iodepth=16

[sequential-write-1m]
rw=write
bs=1M
numjobs=4
iodepth=16

For optimal VM performance on ZFS:

1. Recordsize Alignment

# Align with VM's block size (typically 16K-128K)
zfs set recordsize=64K tank/vm_images

2. Cache Configuration

# Allocate more RAM to ARC (adjust based on your system)
echo "options zfs zfs_arc_max=2147483648" >> /etc/modprobe.d/zfs.conf

3. SLOG Device for Sync Writes

# Add separate log device (SSD recommended)
zpool add tank log /dev/nvme0n1

When using KVM with raw/qcow2 images:

<disk type='file' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native'/>
  <source file='/tank/vm_images/guest.qcow2'/>
  <target dev='vda' bus='virtio'/>
</disk>

Key recommendations:

• Use virtio-scsi with discard=unmap for better TRIM support
• Disable guest fsync if possible (-drive cache=writeback)
• Consider using ZFS volumes (zfs create -V) instead of files