ZFS Tips

This assumes you are already fairly familiar with ZFS.

Mounting a pool from another system

Make sure you use the “altroot” property when importing the pool so the pool doesn’t set its own mountpoints for /etc, /usr and so on and replace those directories on the running system!

Creating a ZPool on a USB drive

If you’re wondering “why put ZFS on your USB stick instead of FAT32/Ext2/Whatever”, consider:

1) ZFS will tell you uif something gets corrupted; other FS will silently let it happen. Data corruption is particularly likely for USB drives; portable magnetic drives have to put up with lots of knocking around and vibration damage and flash drives can be more vulnerable to wearing out ofter repeated writes.

2) ZFS is one of the few POSIX-like FS which has good read-write cross platform compatibility between Linux/BSD/Mac. FAT32 has universal read-write compatibility but no POSIX features (Unix permissions, symlinks etc). All the usual Linux/BSD filesystems which have those features won’t work fully on the other OS.

3) ZFS has some useful features for setting up a drive or pool which will be used on multiple machines and administered by unprivileged users - see the properties below.

A list of all the useful properties for portable pools is below; if you just want an example that combines them all:

zpool create -o cachefile=none -o delegation=on -O atime=off -O compression=lzjb -O copies=2 -O mountpoint=legacy -O setuid=off myusbtank /dev/usbdevice

Once the pool is created, export it from the system to remove:

zpool export myusbtank

Mounting and unmounting a USB drive

Use zpool import to mount the drive; the exact command depends on how the mountpoints are set up.

1. The dataset(s) on the drive have their mountpoint(s) set to something reasonably specific and unique like /home/dylan/research-drive - they will be mounted automatically when importing the pool:

zpool import dylans-research-drive

2. The dataset(s) on the drive use mountpoints that are in the place of existing system directories (e.g. /, /home) - use the “altroot” option to set an alternative root mount point:

zpool import my-drive -o altroot=/mnt

3. The dataset(s) on the drive use mountpoint=legacy - this defers the mounting to the usual system VFS, so you must import the pool and then explicitly mount them using the system mount commands.

When setting up your own drives, I recommend using a “legacy” mountpoint if you want to explicitly specify where it is mounted each time (e.g. if you don’t know what system you will be using it on or don’t have a unique name for the root dataset).

In all cases, export the pool before removing the drive from the system:

zpool export myusbtank

Useful properties for ZPools on USB or other portable drives

Pool Properties:

cachefile=none: The cache is used for boot/system pools, not required for drives that will be added by the user after boot.
delegation=on: Allow delegation of admin to non-privileged users, based on dataset access controls. These are set on the dataset with “zfs allow …” - see zfs(1) for more details.

Dataset Properties

These are set on the root dataset with zpool -O when the pool is created, or other datasets with zfs -o:

atime=off: Unless you need to store access times for some particular reason, switch it off
canmount=noauto: Useful if you want to put a dataset on the drive which is normally not mounted.
compression=lzjb: Normally lz4 gives both faster and better compression, but the old versions of ZFS-Fuse in certain Linux distros don’t have it, so lzjb is best for a maximally compatible pool.
copies=3: For a disk which is likely vulnerable to random sector errors (i.e. virtually any portable drive) using multiple copies is useful for redundancy. Keep in mind this is not a guarantee of data recovery, the whole drive could fail, but it could save your data if you are only using 500mb of a 8gb usb stick. Note that copies can be set per-dataset, so it is possible to have a usb drive with a copies=3 dataset for your work and a copies=1 dataset for some videos you want to watch on the train.
mountpoint=legacy|[path]: Set where the dataset is mounted. If you can pick a directory ahead of time which definitely won’t clash with an existing one, use that and the dataset will be mounted in the same spot wherever you use it (e.g. mountpoint=/bobsHomeVideosDiskThree or mountpoint=/home/dleigh/uniusbstick). Otherwise, set mountpoint to “legacy” to choose the directory at mount time.
setuid=off: Nearly always desirable for security; you are unlikely to need to setuid on any non-system files.

Checksum errors on a RAIDZ Vdev but not the child devices

NAME               STATE     READ WRITE CKSUM
tank               ONLINE       0     0    36
  raidz2-0         ONLINE       0     0    72
    label/dev0     ONLINE       0     0     0
    label/dev1     ONLINE       0     0     0
    label/dev2     ONLINE       0     0     0
    label/dev3     ONLINE       0     0     0
    label/dev4     ONLINE       0     0     0

This means more than the parity level of leaf devices are returning bad data (corrupted); e.g. on a RAIDZ2 vdev at least three disks are broken. In other words, it isn’t possible to know which ones returned bad data as there isn’t enough good data to combine it with.

This also means the data cannot be reconstructed and at least some files have been lost forever (zpool status should mention “data errors” as well).

(This is your reminder that RAID is not a backup).