Skip to main content

ZFS

Decisions
  1. While encrypting the pool volume is possible, I decided to go with a sub-volume, just in case I need to add un-encrypted data later
  2. The options for encryption keys passphrase, raw key, and hex key. I'll be using a hex key for ease of backup in my password database.
  3. When creating the pool use the entire disk, not partitions where possible
  4. When creating the pool, use /dev/disk/by-id, not /dev/sda, as these can change on boot
  1. ZFS - ArchWiki
  2. PerformanceWorkload tuningTuning - OpenZFS Documentation
  3. Tips from JRS
Create the pool

Grab the disk IDs as shown below, and use those IDs to create the pool

ls -lah /dev/disk/by-id
zpool create datapool -o ashift=12 -o autoexpand=on mirror diskID1 diskID1
zfs set compression=lz4 datapool
zfs set atime=off datapool
Creating the Volume
  1. Normal Dataset
zfs create poolname/datasetname
  1. Encrypted Dataset
mkdir /etc/zfs/keys
chmod 700 /etc/zfs/keys
openssl rand -hex -out /etc/zfs/keys/datasetname 32
chmod 600 /etc/zfs/keys/datasetname
zfs create -o encryption=aes-256-gcm -o keyformat=hex -o keylocation=file:///etc/zfs/keys/datasetname poolname/datasetname
Backing up data normally
  1. Create an initial snapshot and backup
zfs snapshot -r poolname@snap1
zfs send -Rw poolname@snap1 | zfs recv -Fdu backuppool
  1. After some changes, send an incremental
zfs snapshot -r poolname@snap2
zfs send -Rw -I poolname@snap1 poolname@snap2 | zfs recv -Fdu backuppool
zfs destroy -r poolname@snap1
Importing Backup Disk without Mounts

To safely import the backup pool disk without fudging existing mounts, use:

zpool import -N backuppool
Setting Max ARC Size

To check your current arc max (in bytes):

# grep c_max /proc/spl/kstat/zfs/arcstats
c_max                           4    10485760000

To set your new arc max, update modprobe and reboot:

# cat /etc/modprobe.d/zfs.conf 
options zfs zfs_arc_max=10485760000
Programmatically Deleting Snapshots

Let's say you have a few snapshots you've grep'd for, how would you delete them?

Use zfs list with the following flags:

  • -H for scripting mode, omitting headers
  • -o name to only grab the name field

Then use xargs to pipe to zfs destroy. Let's look at an example. I want to delete old bookstack snapshots with the offline-backups tag. I can find them easily with grep, let's apply the zfs list flags from above:

~# zfs list -H -o name -rt snapshot datapool/backup/appdata/bookstack | grep offline-backup
datapool/backup/appdata/bookstack@syncoid_offline-backup_hv-02_2020-11-12:19:09:58
datapool/backup/appdata/bookstack/mariadb@syncoid_offline-backup_hv-02_2020-11-12:19:10:08
datapool/backup/appdata/bookstack/webroot@syncoid_offline-backup_hv-02_2020-11-12:19:11:29

I've verified these are indeed the snapshots I'd like to delete. Let's include xargs -n1 zfs destroy to perform the deletion:

zfs list -H -o name -rt snapshot datapool/backup/appdata/bookstack | grep offline-backup | xargs -n1 zfs destroy
Mount Encrypted ZFS Datasets on Boot

By default in Ubuntu 20.04 Encrypted datasets are not mounted on boot.

A quick fix for this is done by modifying the zfs-mount.service:

sudo systemctl edit zfs-mount.service
## Paste in the following content:
[Service]
ExecStart=/sbin/zfs mount -l -a

Do it all... with Sanoid / Syncoid!

Learning ZFS has been fun, so I've wanted to try to do everything manually, but it turns out Sanoid is the end-all-be-all of everything ZFS I need. It'll handle your backups and replication automatically.

Syncoid commands

I have some encrypted datasets, which is a relatively new feature of ZFS and requires the raw send option. To do this with syncoid:

syncoid pool/to/backup bkppool/destination --sendoptions=w
Back to top