Troubleshooting ZFS Pool Mount Issues: Missing Files After Cross-OS Migration


2 views

When migrating a ZFS mirrored pool from FreeBSD to Ubuntu, you might encounter mounting issues where datasets appear empty despite showing correct usage statistics. This typically manifests when examining the pool structure:

$ zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
zroot/ROOT/default   989G   790G   989G  none

ZFS handles mounts differently between FreeBSD and Linux implementations. The key observations:

  • Parent datasets (zroot/ROOT) often have mountpoint=none in FreeBSD installations
  • Child datasets inherit this property, leading to unexpected mount behavior
  • FreeBSD's boot environment management affects default mount points

Method 1: Recursive Mountpoint Correction

For a complete solution, recursively set mountpoints to match your desired structure:

$ sudo zfs set mountpoint=/mnt/zfs zroot
$ sudo zfs set mountpoint=legacy zroot/ROOT/default
$ sudo mount -t zfs zroot/ROOT/default /mnt/zfs/ROOT/default

Method 2: Targeted Dataset Mounting

For specific missing directories like /usr/local that don't appear as separate datasets:

$ sudo zfs list -o name,used,avail,refer,mountpoint -r zroot | grep local
$ sudo zfs create -o mountpoint=/mnt/zfs/usr/local zroot/usr/local

Method 3: Temporary Bind Mounts

If certain directories remain inaccessible, use bind mounts as a temporary measure:

$ sudo mkdir -p /mnt/temp_zfs
$ sudo mount -t zfs zroot/ROOT/default /mnt/temp_zfs
$ sudo mount --bind /mnt/temp_zfs/usr/local /mnt/zfs/usr/local

When mounts fail silently, check these diagnostic commands:

$ sudo zfs get all zroot/ROOT/default | grep -i mount
$ sudo zfs mount -v
$ sudo dmesg | grep zfs
$ sudo lsblk -f | grep zfs

Cross-OS migration may trigger feature flag issues. Verify and potentially downgrade features:

$ sudo zpool get all zroot | grep feature
$ sudo zpool upgrade -v zroot
$ sudo zpool set feature@async_destroy=disabled zroot

If standard methods fail, consider exporting and re-importing with explicit mountpoint settings:

$ sudo zpool export zroot
$ sudo zpool import -d /dev/disk/by-id -o altroot=/mnt zroot
$ sudo zfs set mountpoint=/mnt/zfs zroot
$ sudo zfs mount -a

When migrating ZFS pools between operating systems (especially across BSD/Linux), mount behavior can become unpredictable. Your case shows classic symptoms where dataset structures appear but critical files remain inaccessible. The key observations:

  • Datasets show correct storage allocation (989G used in zroot/ROOT/default)
  • Directory trees mount but appear empty
  • Individual datasets mount successfully when targeted specifically

Before attempting fixes, verify the complete pool structure:

sudo zpool status -v zroot
sudo zfs list -o name,mountpoint,canmount,mounted -r zroot

Pay special attention to:

sudo zfs get all zroot/ROOT/default | grep -E 'mountpoint|canmount|origin'

FreeBSD's ZFS implementation frequently uses legacy mountpoints and nested dataset structures that don't translate cleanly to Linux. In your case:

  • zroot/ROOT/default contains 989GB but refuses to mount
  • The system defaults to creating empty directories at parent mountpoints
  • Linux's ZFS implementation handles canmount=noauto differently than FreeBSD

The most reliable approach combines mounting with property adjustments:

# First unmount everything
sudo zfs umount -a

# Set explicit mountpoints for critical datasets
sudo zfs set mountpoint=/mnt/zfs/ROOT zroot/ROOT/default
sudo zfs set canmount=on zroot/ROOT/default

# Force remount
sudo zfs mount -a

# Verify with
sudo zfs list -o name,mountpoint,mounted -r zroot | grep -E 'ROOT|default'

For particularly stubborn datasets, use manual mounting with forensic options:

sudo zfs mount -o ro,noatime,forcedirectio zroot/ROOT/default /mnt/zfs/ROOT

Your missing /usr/local suggests either:

  1. A separate dataset that wasn't properly exported
  2. Files stored in the root dataset before subdatasets were created

Check for hidden data with:

sudo zfs list -t all -r zroot | grep local
sudo find /mnt/zfs/ROOT -xdev -type d -name local

If standard methods fail, consider these approaches:

# 1. Temporary mount with alternative ZFS implementation
sudo apt install zfs-fuse
zfs-fuse -m /mnt/zfs_alt

# 2. Raw data extraction (last resort)
sudo dd if=/dev/disk/by-id/[pool-disk] bs=1M | strings | grep -A10 -B10 "usr/local"

For cross-platform ZFS pools, always:

  • Set explicit mountpoints before export: zfs set mountpoint=legacy pool/dataset
  • Use consistent ZFS feature flags: zpool upgrade -v
  • Document dataset structure: zfs list -o name,mountpoint,canmount -r pool