zfs micro incompatibilities
I've been setting up ZFS replication between some servers, but between one pair, every sync was failing with:
Nov 30 11:07:11 sourcehost syncoid[890260]: Sending incremental tank/enc/var/spool@autosnap_2025-11-27_20:00:22_monthly ... autosnap_2025-11-30_11:00:39_hourly (~ 555 KB):
Nov 30 11:07:12 sourcehost syncoid[890982]: warning: cannot send 'tank/enc/var/spool@autosnap_2025-11-27_20:00:22_weekly': Invalid argument
Source machine (which was on the way to being upgraded to trixie but
hadn't been rebooted yet):
root@sourcehost:~# uname -a
Linux sourcehost 6.1.0-37-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.140-1 (2025-05-22) x86_64 GNU/Linux
root@sourcehost:~# zfs version
zfs-2.3.2-2
zfs-kmod-2.3.1-1~bpo12+1
I had assumed it was some incompatibility with the destination machine
and spent ages trying to track that down until I read the error more
carefully - cannot send, ie the problem is with the zfs tool on
sourcehost, before it even talks to the destination.
Eventually I found a
reference to a bug in
openzfs 2.3.1 that broke the user-space API, and then 2.3.2
reverted the breakage, which broke compatibility with 2.3.1 itself.
Going through PRs from between 2.3.1 and 2.3.2, it's
this, and here's the
explanation of the accidental ABI change
change
A reboot in to the new kernel with the new, same-versioned zfs
userspace and kernel module fixed it.