Comment 9 for bug 1890881

I disabled/enabled arm and this confirmed it is the interpreter being used.

$ sudo update-binfmts --disable qemu-arm
# now fails missing interpreter

Eventually I found I need to unimport&import it after exchanging the binary.
Then I get it to use the new version.

Note (disable all for some ease to read)
$ for i in /usr/share/binfmts/qemu-*; do sudo update-binfmts --unimport $i; done

$ sudo update-binfmts --unimport /usr/share/binfmts/qemu-arm
$ sudo cp /usr/bin/qemu-arm-static.pkg /usr/bin/qemu-arm-static
$ sudo update-binfmts --import /usr/share/binfmts/qemu-arm
$ sudo systemd-nspawn -D /var/lib/machines/armhf-focal -M armhf-focal -b
...
works now

$ machinectl
MACHINE CLASS SERVICE OS VERSION ADDRESSES
armhf-focal container systemd-nspawn ubuntu 20.04 -

And I see it running properly with its tree using qemu static:
$ sudo machinectl status armhf-focal
armhf-focal(a008dc84011f4382b74546f73f43cbaa)
           Since: Tue 2020-08-18 12:38:02 UTC; 2min 59s ago
          Leader: 81194 (systemd)
         Service: systemd-nspawn; class container
            Root: /var/lib/machines/armhf-focal
              OS: Ubuntu 20.04 LTS
            Unit: machine-armhf\x2dfocal.scope
                  └─payload
                    ├─init.scope
                    │ └─81194 /usr/bin/qemu-arm-static /lib/systemd/systemd
                    └─system.slice
                      ├─accounts-daemon.service
                      │ └─82253 /usr/bin/qemu-arm-static /usr/lib/accountsservice/accounts-daemon
                      ├─console-getty.service
                      │ └─82301 /usr/bin/qemu-arm-static /sbin/agetty -o -p -- \u --noclear --keep-baud console 115200,38400,9600 xterm-256color
                      ├─cron.service
                      │ └─82254 /usr/bin/qemu-arm-static /usr/sbin/cron -f
                      ├─dbus.service
                      │ ├─81377 /usr/bin/qemu-arm-static /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
                      │ ├─81689 /usr/bin/qemu-arm-static /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
                      │ └─82256 /usr/bin/qemu-arm-static /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
                      ├─networkd-dispatcher.service
                      │ └─82265 /usr/bin/qemu-arm-static /usr/bin/python3 /usr/bin/networkd-dispatcher --run-startup-triggers
                      ├─rsyslog.service
                      │ └─82267 /usr/bin/qemu-arm-static /usr/sbin/rsyslogd -n -iNONE
                      ├─systemd-journald.service
                      │ └─82065 /usr/bin/qemu-arm-static /lib/systemd/systemd-journald
                      ├─systemd-logind.service
                      │ └─82271 /usr/bin/qemu-arm-static /lib/systemd/systemd-logind
                      └─systemd-resolved.service
                        └─82250 /usr/bin/qemu-arm-static /lib/systemd/systemd-resolved

There is one issue left to replace this in a bisect loop around
  cp: cannot create regular file '/usr/bin/qemu-arm-static': Text file busy
binfmt Unloads won't help, lsof doesn't list it, ...
But worst case it can be done with reboots.

Ah found it, cp refuses but rm+cp works.