[0.8 regression] zfs mount -a dataset mount ordering issues

Update: I was able to reproduce it with a simpler schema (/ isn't always mounted before /var). This is to mimick the official zol guide with zfs on root: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS
$ zpool create -o ashift=12 -O atime=off -O canmount=off -O normalization=formD -O mountpoint=/ -R /mnt rpool /dev/vda2
$ zfs create rpool/ROOT -o canmount=off -o mountpoint=none
$ zfs create rpool/ROOT/ubuntu_123456 -o mountpoint=/
$ zfs create rpool/ROOT/ubuntu_123456/var

$ zpool create -o ashift=12 -O atime=off -O canmount=off -O normalization=formD -O mountpoint=/ -R /mnt rpool /dev/vda2
$ zfs create rpool/ROOT -o canmount=off -o mountpoint=none
$ zfs create rpool/ROOT/ubuntu_123456 -o mountpoint=/
$ zfs create rpool/ROOT/ubuntu_123456/var
$ zfs create rpool/ROOT/ubuntu_123456/var/lib -o canmount=off
$ zfs create rpool/var -o canmount=off
$ zfs create rpool/var/lib
$ zfs create rpool/var/games
$ zfs create rpool/ROOT/ubuntu_123456/var/lib/apt

# Zfs mount is what we expect (5 datasets mounted):
$ zfs mount
rpool/ROOT/ubuntu_123456 /mnt
rpool/ROOT/ubuntu_123456/var /mnt/var
rpool/var/games /mnt/var/games
rpool/var/lib /mnt/var/lib
rpool/ROOT/ubuntu_123456/var/lib/apt /mnt/var/lib/apt

$ zfs umount -a

# Everything unmounted as expected:
$ find /mnt/
/mnt/

# However, zfs mount -a doesn't mount everything in the correct order reliably:
$ zfs mount -a
cannot mount '/mnt': directory is not empty

# In that case, rpool/ROOT/ubuntu_123456 wasn't mounted:
$ zfs mount
rpool/ROOT/ubuntu_123456/var /mnt/var
rpool/var/lib /mnt/var/lib
rpool/var/games /mnt/var/games
rpool/ROOT/ubuntu_123456/var/lib/apt /mnt/var/lib/apt

$ find /mnt/
/mnt/
/mnt/var
/mnt/var/lib
/mnt/var/lib/apt
/mnt/var/games

$ zfs umount -a
$ find /mnt/
/mnt/

# Everything was umounted, let's try to remount all again:
$ zfs mount -a
cannot mount '/mnt/var/lib': failed to create mountpoint
$ zfs mount
rpool/ROOT/ubuntu_123456 /mnt
rpool/ROOT/ubuntu_123456/var /mnt/var
rpool/var/games /mnt/var/games
rpool/ROOT/ubuntu_123456/var/lib/apt /mnt/var/lib/apt

#This time, rpool/ROOT/ubuntu_123456 was mounted, but not rpool/var/lib (before rpool/ROOT/ubuntu_123456/var/lib/apt)

Note: the same ordering issue can happen on zfs umount -a.

Tested as well with zfs 0.7: tried to zfs mount -a && zfs umount -a in loop, no issue: all datasets are mounted in the correct order reliably.

Note that it seems to be slightly related to the version of zfs we created a pool with:
- Try zfs mount -a on zfs 0.7 with a pool/datasets created under zfs 0.8: the ordering issues still happens.
- However, the contrary isn't try: try zfs mount -a on zfs 0.8 with a pool/datasets created under zfs 0.7: there can be some ordering issues.

There is nothing specific in the journal log:
juil. 24 10:59:27 ubuntu kernel: ZFS: Loaded module v0.8.1-1ubuntu5, ZFS pool version 5000, ZFS filesystem version 5
juil. 24 10:59:39 ubuntu systemd[1]: mnt-var-lib.mount: Succeeded.
juil. 24 10:59:39 ubuntu systemd[1116]: mnt-var-lib.mount: Succeeded.
juil. 24 10:59:39 ubuntu systemd[1]: mnt-var-games.mount: Succeeded.
juil. 24 10:59:39 ubuntu systemd[1116]: mnt-var-games.mount: Succeeded.
juil. 24 10:59:42 ubuntu systemd[1116]: mnt-var-lib-apt.mount: Succeeded.
juil. 24 10:59:42 ubuntu systemd[1]: mnt-var-lib-apt.mount: Succeeded.
juil. 24 10:59:42 ubuntu systemd[1116]: mnt-var-games.mount: Succeeded.
juil. 24 10:59:42 ubuntu systemd[1]: mnt-var-games.mount: Succeeded.
juil. 24 10:59:42 ubuntu systemd[1116]: mnt-var.mount: Succeeded.
juil. 24 10:59:42 ubuntu systemd[1]: mnt-var.mount: Succeeded.
juil. 24 10:59:42 ubuntu systemd[1116]: mnt.mount: Succeeded.
juil. 24 10:59:42 ubuntu systemd[1]: mnt.mount: Succeeded.
juil. 24 10:59:45 ubuntu systemd[1116]: mnt-var-lib.mount: Succeeded.
juil. 24 10:59:45 ubuntu systemd[1]: mnt-var-lib.mount: Succeeded.
juil. 24 10:59:46 ubuntu systemd[1116]: mnt-var-games.mount: Succeeded.
juil. 24 10:59:46 ubuntu systemd[1]: mnt-var-games.mount: Succeeded.
juil. 24 11:01:06 ubuntu systemd[1116]: mnt-var-lib-apt.mount: Succeeded.
juil. 24 11:01:06 ubuntu systemd[1]: mnt-var-lib-apt.mount: Succeeded.
juil. 24 11:01:06 ubuntu systemd[1]: mnt-var-games.mount: Succeeded.
juil. 24 11:01:06 ubuntu systemd[1116]: mnt-var-games.mount: Succeeded.
juil. 24 11:01:06 ubuntu systemd[1]: mnt.mount: Succeeded.
juil. 24 11:01:06 ubuntu systemd[1116]: mnt.mount: Succeeded.
juil. 24 11:01:08 ubuntu systemd[1]: mnt-var.mount: Succeeded.
juil. 24 11:01:08 ubuntu systemd[1116]: mnt-var.mount: Succeeded.

Note that pools created with 0.8-3 from debian has the same issues.