Merge ~xnox/ubiquity:zfs-list-cache into ubiquity:main

[Dimitri John Ledkov]

* Re-enable zfs encryption
* zsys-setup: generate correct zfs-list.cache for target (LP: #1993318)

[Nick Rosbrook (enr0n)]

I tested this by patching /usr/share/ubiquity/zsys-setup in the live environment, before starting the install. I ran the install as normal from there, and the first boot behaved normally again. Specifically, /var/lib/dpkg/status was populated correctly, apt commands worked, and the Firefox icon was present on the dock.

The output from `zfs list` and `mount` looked sane to me.

[Nick Rosbrook (enr0n)]

$ zfs list
NAME USED AVAIL REFER MOUNTPOINT
bpool 136M 1.37G 96K /boot
bpool/BOOT 135M 1.37G 96K none
bpool/BOOT/ubuntu_rvbd0s 135M 1.37G 135M /boot
rpool 5.58G 9.43G 192K /
rpool/ROOT 5.07G 9.43G 192K none
rpool/ROOT/ubuntu_rvbd0s 5.07G 9.43G 3.41G /
rpool/ROOT/ubuntu_rvbd0s/srv 192K 9.43G 192K /srv
rpool/ROOT/ubuntu_rvbd0s/usr 580K 9.43G 192K /usr
rpool/ROOT/ubuntu_rvbd0s/usr/local 388K 9.43G 388K /usr/local
rpool/ROOT/ubuntu_rvbd0s/var 1.66G 9.43G 192K /var
rpool/ROOT/ubuntu_rvbd0s/var/games 192K 9.43G 192K /var/games
rpool/ROOT/ubuntu_rvbd0s/var/lib 1.65G 9.43G 1.52G /var/lib
rpool/ROOT/ubuntu_rvbd0s/var/lib/AccountsService 208K 9.43G 208K /var/lib/AccountsService
rpool/ROOT/ubuntu_rvbd0s/var/lib/NetworkManager 224K 9.43G 224K /var/lib/NetworkManager
rpool/ROOT/ubuntu_rvbd0s/var/lib/apt 82.7M 9.43G 82.7M /var/lib/apt
rpool/ROOT/ubuntu_rvbd0s/var/lib/dpkg 50.2M 9.43G 50.2M /var/lib/dpkg
rpool/ROOT/ubuntu_rvbd0s/var/log 2.14M 9.43G 2.14M /var/log
rpool/ROOT/ubuntu_rvbd0s/var/mail 192K 9.43G 192K /var/mail
rpool/ROOT/ubuntu_rvbd0s/var/snap 2.49M 9.43G 2.49M /var/snap
rpool/ROOT/ubuntu_rvbd0s/var/spool 276K 9.43G 276K /var/spool
rpool/ROOT/ubuntu_rvbd0s/var/www 192K 9.43G 192K /var/www
rpool/USERDATA 5.19M 9.43G 192K /
rpool/USERDATA/root_5hwecp 288K 9.43G 288K /root
rpool/USERDATA/ubuntu_5hwecp 4.72M 9.43G 4.72M /home/ubuntu
rpool/keystore 518M 9.88G 63.4M -

$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=1942688k,nr_inodes=485672,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=401788k,mode=755,inode64)
/dev/mapper/keystore-rpool on /run/keystore/rpool type ext4 (rw,relatime,stripe=2)
rpool/ROOT/ubuntu_rvbd0s on / type zfs (rw,relatime,xattr,posixacl)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /...

[Didier Roche-Tolomelli (didrocks)]

This looks like the correct fix with current ZFS. Sad that we need to do this local workaround, but from what I see, you are then doing the proper cleanup and removing the prefix.

Nick's testing seems on par with our expectation. Approving.

review: Approve

diff --git a/debian/changelog b/debian/changelog
index 7c40ed1..0fb1543 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+ubiquity (22.10.11) UNRELEASED; urgency=medium
+
+  * Re-enable zfs encryption
+  * zsys-setup: generate correct zfs-list.cache for target (LP: #1993318)
+
+ -- Dimitri John Ledkov <[e-mail]>  Wed, 19 Oct 2022 15:29:28 +0100
+
 ubiquity (22.10.10) kinetic; urgency=medium
 
   * Temporarily disable zfs + encryption option (LP: #1993318)
diff --git a/scripts/zsys-setup b/scripts/zsys-setup
index 78df4f3..9298405 100755
--- a/scripts/zsys-setup
+++ b/scripts/zsys-setup
@@ -584,18 +584,6 @@ elif [ "${COMMAND}" = "finalize" ]; then
 		exit 1
 	fi
 
-	# Activate zfs generator.
-	# After enabling the generator we should run zfs set canmount=on DATASET
-	# in the chroot for one dataset of each pool to refresh the zfs cache.
-	echo "I: Activating zfs generator"
-
-	# Create zpool cache
-	zpool set cachefile= bpool
-	zpool set cachefile= rpool
-	cp /etc/zfs/zpool.cache "${TARGET}/etc/zfs/"
-	mkdir -p "${TARGET}/etc/zfs/zfs-list.cache"
-	touch "${TARGET}/etc/zfs/zfs-list.cache/bpool" "${TARGET}/etc/zfs/zfs-list.cache/rpool"
-
 	# Handle userdata
 	UUID_ORIG=$(head -100 /dev/urandom | tr -dc 'a-z0-9' |head -c6)
 	mkdir -p "${TARGET}/tmp/home"
@@ -618,6 +606,28 @@ elif [ "${COMMAND}" = "finalize" ]; then
 		chroot "${TARGET}" update-initramfs -u
 	fi
 
+	# Activate zfs generator, after all zfs commands have completed
+	echo "I: Activating zfs generator"
+
+	# Create zpool cache
+	zpool set cachefile= bpool
+	zpool set cachefile= rpool
+	cp /etc/zfs/zpool.cache "${TARGET}/etc/zfs/"
+	mkdir -p "/etc/zfs/zfs-list.cache" "${TARGET}/etc/zfs/zfs-list.cache"
+	for pool in bpool rpool; do
+		# Force cache generation
+		: >"/etc/zfs/zfs-list.cache/${pool}"
+		# Execute zfs-list-cacher with a manual fake event
+		env -i ZEVENT_POOL=${pool} ZED_ZEDLET_DIR=/etc/zfs/zed.d ZEVENT_SUBCLASS=history_event ZFS=zfs ZEVENT_HISTORY_INTERNAL_NAME=create /etc/zfs/zed.d/history_event-zfs-list-cacher.sh
+		# ZFS list doesn't honor target prefix for chroots for
+		# the mountpoint property
+		# https://github.com/openzfs/zfs/issues/1078
+		# Drop leading /target from all mountpoint fields
+		sed -E "s|\t${TARGET}/?|\t/|g" "/etc/zfs/zfs-list.cache/${pool}" > "${TARGET}/etc/zfs/zfs-list.cache/${pool}"
+		# Ensure installer system doesn't generate mount units
+		rm -f "/etc/zfs/zfs-list.cache/${pool}"
+	done
+
 	echo "I: ZFS setup complete"
 else
 	echo "E: Unknown command: $COMMAND"
diff --git a/ubiquity/plugins/ubi-partman.py b/ubiquity/plugins/ubi-partman.py
index 8059446..b9f3d9e 100644
--- a/ubiquity/plugins/ubi-partman.py
+++ b/ubiquity/plugins/ubi-partman.py
@@ -668,8 +668,7 @@ class PageGtk(PageBase):
         if not widget.get_active():
             return
 
-        # use_volume_manager = self.use_lvm.get_active() or self.use_zfs.get_active()
-        use_volume_manager = self.use_lvm.get_active()
+        use_volume_manager = self.use_lvm.get_active() or self.use_zfs.get_active()
         if not use_volume_manager:
             self.use_crypto.set_active(False)
         self.use_crypto.set_sensitive(use_volume_manager)