Troubleshooting “Cannot Destroy ZFS Snapshot: Dataset Already Exists” Error on Solaris 10


2 views

When dealing with ZFS snapshots on Solaris 10, you might encounter a particularly confusing error message:

zfs destroy mysql/data/4.1.12@wibble
cannot destroy 'mysql/data/4.1.12@wibble': dataset already exists

This occurs despite the snapshot being clearly visible and accessible through the .zfs/snapshots directory. The error appears contradictory because we're trying to destroy an existing snapshot, not create one.

First, verify the snapshot's status with:

zfs list -t snapshot | grep mysql/data/4.1.12@wibble
zfs get all mysql/data/4.1.12@wibble

Check for hidden dependencies or clones that might not be immediately visible:

zfs list -o origin | grep mysql/data/4.1.12@wibble

Try these approaches in sequence:

# Attempt with -r recursive flag
zfs destroy -r mysql/data/4.1.12@wibble

# Force unmount first
zfs unmount -f mysql/data/4.1.12

# Try destroying the parent dataset (careful!)
zfs destroy -r mysql/data/4.1.12

If standard methods fail, consider these deeper diagnostics:

# Check for kernel module issues
modinfo | grep zfs

# Verify pool health
zpool status mysql

# Check for pending transactions
zpool get all mysql | grep deferred

For persistent cases, try these solutions:

# Export and reimport the pool
zpool export mysql
zpool import mysql

# As last resort, reboot the system
shutdown -y -i6 -g0

To prevent future occurrences, consider updating your Solaris 10 to the latest supported patch level and implementing regular pool scrubbing.

When scripting snapshot management, include robust error handling:

#!/bin/sh
SNAPSHOT="mysql/data/4.1.12@$(date +%Y%m%d%H%M)"

zfs snapshot "$SNAPSHOT" || {
    logger -t zfs-snap "Failed to create $SNAPSHOT"
    exit 1
}

# Cleanup with error handling
zfs destroy "$SNAPSHOT" 2>/dev/null || {
    logger -t zfs-snap "Warning: Could not destroy $SNAPSHOT"
    # Additional cleanup logic here
}

I recently encountered a bizarre ZFS snapshot issue on a Solaris 10 8/07 server (T5220 hardware). The system manages a ZFS pool named "mysql" with a filesystem at "mysql/data/4.1.12". While hourly snapshots generally work fine through a cron script, one particular snapshot became impossible to destroy.

# zfs destroy mysql/data/4.1.12@wibble
cannot destroy 'mysql/data/4.1.12@wibble': dataset already exists

Before reporting this, I verified several things:

  • The snapshot was successfully renamed from its original hourly name to "wibble"
  • The snapshot remains accessible through .zfs/snapshots directory
  • No clones depend on this snapshot (verified with zfs list -t all -o origin)
  • Standard destruction methods fail with the confusing "dataset already exists" message

After researching ZFS internals, several potential causes emerged:

  1. Metadata corruption: The ZFS internal reference count might be incorrect
  2. Pending transactions: An incomplete operation might be holding the snapshot
  3. ZFS version bug: Solaris 10's ZFS implementation had several early quirks

Here are the methods that eventually worked for me and others:

Method 1: Force Deletion with -r Flag

# zfs destroy -r mysql/data/4.1.12@wibble

Method 2: Export/Import the Pool

# zpool export mysql
# zpool import mysql
# zfs destroy mysql/data/4.1.12@wibble

Method 3: Use the -d Parameter

# zfs destroy -d mysql/data/4.1.12@wibble

To avoid similar issues in the future:

# Regular pool scrubbing
zpool scrub mysql

# Verify snapshot integrity
zfs list -t snapshot -r mysql/data/4.1.12

# Consider upgrading to a newer Solaris version
# or OpenZFS implementation

For truly stubborn cases, you might need to:

  1. Create a new pool
  2. Replicate the data (using zfs send | zfs recv)
  3. Retire the old pool

Remember that ZFS behavior can vary between implementations. The solutions here worked on Solaris 10's ZFS version, but might need adjustment for other platforms.