update-grub fails on zfs with root and boot datasets when using mawk

Thank you for your bug report, could you give some details/specific on the error and maybe add the update-grub log?

Hey, thanks for reporting this bug and help to make ubuntu better.

Can you please post the following so that we can debug what went wrong. Outputs of:
* zpool list
* zfs list
for each dataset:
* zfs get all <datasetname>

That way, we can find why your datasets and boot pools were not detected and add a test case for it.

Also, the output of update-grub would be excellent to ensure there was no failure.

zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
bpool 496M 122M 374M - 18% 24% 1.00x ONLINE -
rpool 39.2G 5.18G 34.1G - 19% 13% 1.00x ONLINE -

zfs list
NAME USED AVAIL REFER MOUNTPOINT
bpool 122M 246M 96K /
bpool/BOOT 121M 246M 96K none
bpool/BOOT/ubuntu 121M 246M 120M legacy
rpool 5.18G 32.8G 96K /
rpool/ROOT 4.90G 32.8G 96K none
rpool/ROOT/ubuntu 4.90G 32.8G 4.60G /
rpool/home 94.3M 32.8G 96K /home
rpool/home/dbergst 94.1M 32.8G 90.4M /home/dbergst
rpool/home/root 116K 32.8G 116K /root
rpool/opt 11.0M 32.8G 11.0M /opt
rpool/srv 96K 32.8G 96K /srv
rpool/tmp 292K 32.8G 160K legacy
rpool/usr 232K 32.8G 96K /usr
rpool/usr/local 136K 32.8G 136K /usr/local
rpool/var 170M 32.8G 96K /var
rpool/var/cache 146M 32.8G 146M /var/cache
rpool/var/games 96K 32.8G 96K /var/games
rpool/var/lib 288K 32.8G 96K /var/lib
rpool/var/lib/docker 96K 32.8G 96K /var/lib/docker
rpool/var/lib/nfs 96K 32.8G 96K /var/lib/nfs
rpool/var/log 23.2M 32.8G 21.8M legacy
rpool/var/mail 96K 32.8G 96K /var/mail
rpool/var/snap 96K 32.8G 96K /var/snap
rpool/var/spool 168K 32.8G 112K legacy
rpool/var/tmp 208K 32.8G 120K legacy
rpool/var/www 96K 32.8G 96K /var/www

zfs get all bpool
NAME PROPERTY VALUE SOURCE
bpool type filesystem -
bpool creation Sat Apr 27 22:20 2019 -
bpool used 122M -
bpool available 246M -
bpool referenced 96K -
bpool compressratio 1.08x -
bpool mounted no -
bpool quota none default
bpool reservation none default
bpool recordsize 128K default
bpool mountpoint / local
bpool sharenfs off default
bpool checksum on default
bpool compression lz4 local
bpool atime on default
bpool devices off local
bpool exec on default
bpool setuid on default
bpool readonly off default
bpool zoned off default
bpool snapdir hidden default
bpool aclinherit restricted default
bpool createtxg 1 -
bpool canmount off local
bpool xattr sa local
bpool copies 1 default
bpool version ...

Thanks for the results!

Can you run "zfs get all bpool/BOOT/ubuntu" please?

I'm a little bit puzzled about this line:
bpool/BOOT/ubuntu 121M 246M 120M legacy
legacy means you have a manual mountpoint set on this dataset. How do you mount it? Can you paste your /etc/fstab (I guess you put it there?)

I wonder if <pool>/BOOT/ was hardcoded in some way and it had a way to detect dataset was mounted.

zfs get all bpool/BOOT/ubuntu
NAME PROPERTY VALUE SOURCE
bpool/BOOT/ubuntu type filesystem -
bpool/BOOT/ubuntu creation Sat Apr 27 22:23 2019 -
bpool/BOOT/ubuntu used 211M -
bpool/BOOT/ubuntu available 155M -
bpool/BOOT/ubuntu referenced 116M -
bpool/BOOT/ubuntu compressratio 1.05x -
bpool/BOOT/ubuntu mounted yes -
bpool/BOOT/ubuntu quota none default
bpool/BOOT/ubuntu reservation none default
bpool/BOOT/ubuntu recordsize 128K default
bpool/BOOT/ubuntu mountpoint legacy local
bpool/BOOT/ubuntu sharenfs off default
bpool/BOOT/ubuntu checksum on default
bpool/BOOT/ubuntu compression lz4 inherited from bpool
bpool/BOOT/ubuntu atime on default
bpool/BOOT/ubuntu devices off inherited from bpool
bpool/BOOT/ubuntu exec on default
bpool/BOOT/ubuntu setuid on default
bpool/BOOT/ubuntu readonly off default
bpool/BOOT/ubuntu zoned off default
bpool/BOOT/ubuntu snapdir hidden default
bpool/BOOT/ubuntu aclinherit restricted default
bpool/BOOT/ubuntu createtxg 36 -
bpool/BOOT/ubuntu canmount noauto local
bpool/BOOT/ubuntu xattr sa inherited from bpool
bpool/BOOT/ubuntu copies 1 default
bpool/BOOT/ubuntu version 5 -
bpool/BOOT/ubuntu utf8only on -
bpool/BOOT/ubuntu normalization formD -
bpool/BOOT/ubuntu casesensitivity sensitive -
bpool/BOOT/ubuntu vscan off default
bpool/BOOT/ubuntu nbmand off default
bpool/BOOT/ubuntu sharesmb off default
bpool/BOOT/ubuntu refquota none default
bpool/BOOT/ubuntu refreservation none default
bpool/BOOT/ubuntu guid 15745059723263933600 -
bpool/BOOT/ubuntu primarycache all default
bpool/BOOT/ubuntu secondarycache all default
bpool/BOOT/ubuntu usedbysnapshots 95.8M -
bpool/BOOT/ubuntu usedbydataset 116M -
bpool/BOOT/ubuntu usedbychildren 0B -
bpool/BOOT/ubuntu usedbyrefreservation 0B -
bpool/BOOT/ubuntu logbias latency default
bpool/BOOT/ubuntu dedup off defaul...

Additional information:

cat /etc/systemd/system/zfs-import.target.wants/zfs-import-bpool.service
[Unit]
DefaultDependencies=no
Before=zfs-import-scan.service
Before=zfs-import-cache.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/sbin/zpool import -N -o cachefile=none bpool

[Install]
WantedBy=zfs-import.target

ok, found the issue. I think if you followed the zol ubuntu administration guide (https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS), which is only using a debootstrap, you are using mawk and not gawk, which is why the regexp don't match and find your kernel.

Traditional ubuntu installations have gawk by default (higher priority for the awk alternative), this is why we didn't spot it.

I'm unsure on how to fix those yet, but meanwhile, to unblock you, you can install gawk.

After installing gawk and updating the system (including grub), the issue reported with update-grub is resolved.

Thanks for confirming!

After deep analysis, the best path is to rewrite a critical part of the logic in shell it seems. At least, you are unblocked!

This is committed in the grub ubuntu packaging branch, waiting for the next release handled by the foundation team (due to a FTBFS to debug + merge)

This can be closed. The changelog has been rewritten apparently and so, didn't have the stenza on this fix, but it's in 2.04-1ubuntu2.

Reopening, this isn't in ubuntu yet.

This bug was fixed in the package grub2 - 2.04-1ubuntu3

---------------
grub2 (2.04-1ubuntu3) eoan; urgency=medium

  [ Mathieu Trudel-Lapierre ]
  * debian/patches/ubuntu-add-devicetree-command-support.patch: import patch
    into git-dpm: drop [PATCH] tag and add Patch-Name.

  [ Didier Roche ]
  * debian/patches/ubuntu-zfs-enhance-support.patch
    - Don't patch autoregenerated files.
    - rewrite generate MenuMeta implementation in shell (LP: #1834095)
      mawk doesn't support \s and other array features.
      + Change \s by their space or tab equivalent.
      + Rewrite the menumeta generation in pure shell, which is easier to
        debug, keeping globally the same algorithm
      + Support i18n in entry name generation.
      Co-authored with Jean-Baptiste.
    - Resplit all patches in debian/patches/*, so that we have upstreamable
      and non upstreamable parts separate. Also, any change in 10_linux patch
      will be reflected in 10_linux_zfs.
    - Always import pools (using force), as we don't mount them. Ensure also
      that we don't update the host cache, as we import all pools, and not
      only those attached to that system.

 -- Didier Roche <email address hidden> Mon, 29 Jul 2019 08:08:48 +0200