Shim apparently fails to run fwupd64 (hirsute regression?)

Status changed to 'Confirmed' because the bug affects multiple users.

Probably https://github.com/rhboot/shim/pull/374/commits/87fbf42ac96f887a3620efbbabc501269dc9f918

I could try to test the suggested change...

I understand that I would need to
- generate a key pair
- roll the public key into bios
- build the shim
- sign the shim with the private key
- set a boot entry with the new shim and updater as its parameter
- try to boot via this new boot entry

Is exact procedure documented somewhere? I could not even build the shim binary when I tried..

@Eugene Hey, you can extract a signed shim from http://ppa.launchpad.net/ubuntu-uefi-team/ppa/ubuntu/dists/impish/main/signed/shim-amd64/15.4-0ubuntu3~uefi3/ once it's there.

You'll have to import the control/uefi.crt into your UEFI DB to test it.

As this is looking to be a shim bug, closing the fwupd tasks.

I tried two approaches and neither worked, unfortunately :(

I need to try Chris Coulson's patch, it certainly looks good. The current code to handle load option parsing is very confusing hacks around broken firmwares :/ But we also don't want to break more I guess.

> Chris Coulson's patch

https://github.com/rhboot/shim/pull/379

Verified the fix worked. Updated my system firmware, EC, and something else. Hooray!

Status changed to 'Confirmed' because the bug affects multiple users.

This bug was fixed in the package shim - 15.4-0ubuntu7

---------------
shim (15.4-0ubuntu7) hirsute; urgency=medium

  * Fix load option parsing, and thus fwupd execution (LP: #1929471) (PR #379)
  * Fix occasional crashes in _relocate() on arm64 (LP: #1928010) (PR #383)
  * Fix accidental deletion of RT variables (LP: #1934506) (PR #387)
  * mok: relax the maximum variable size check (LP: #1934780) (PR #369)

 -- Julian Andres Klode <email address hidden> Wed, 07 Jul 2021 10:57:35 +0200

Installed shim-signed_1.49+15.4-0ubuntu7_amd64.deb from impish on hirsute,

Verified that it works, I was able to install all pending firmware updates.

This bug was fixed in the package shim-signed - 1.49

---------------
shim-signed (1.49) impish; urgency=medium

  * Update to shim 15.4-0ubuntu7:
    - Fix load option parsing, and thus fwupd execution (LP: #1929471) (PR #379)
    - Fix occasional crashes in _relocate() on arm64 (LP: #1928010) (PR #383)
    - Fix accidental deletion of RT variables (LP: #1934506) (PR #387)
    - mok: relax the maximum variable size check (LP: #1934780) (PR #369)

 -- Julian Andres Klode <email address hidden> Thu, 15 Jul 2021 11:00:51 +0200

Hello Eugene, or anyone else affected,

Accepted shim into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Eugene, or anyone else affected,

Accepted shim into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Eugene, or anyone else affected,

Accepted shim-signed into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.40.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Eugene, or anyone else affected,

Accepted shim into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Eugene, or anyone else affected,

Accepted shim-signed into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.37~18.04.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Eugene, or anyone else affected,

Accepted shim into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim/15.4-0ubuntu7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Eugene, or anyone else affected,

Accepted shim-signed into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.33.1~16.04.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Per my understanding, for focal, we still wait for sbat-compatible fwupd landing. Without that, the new shim will make the current focal/fwupd failed in secure-boot-enabled machine.

Hello Eugene, or anyone else affected,

Accepted shim-signed into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shim-signed/1.37~18.04.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

fwupd is not being run on xenial. need to verify against new fwupd on focal; bionic fwupd SRU still unapproved.

OK so I'm actually going to mark this as verified, since I verified the load option parsing fix in 15.4-0ubuntu7 that makes it find the correct binaries, and will leave the verification of the fwupd's SBAT stuff to the fwupd SRUs.

So, the regression potential is wrong and the fallback code I was thinking of does not exist. This seems to have caused a regression in bug 1937115. This will affect some systems that contain garbage in default boot order entries that still gets parsed as a valid second stage loader.

This should only be a problem when booting installer media, and maybe when booting via fallback, though I guess fallback will fix that issue before loading grub?

This bug was fixed in the package shim-signed - 1.50

---------------
shim-signed (1.50) impish; urgency=medium

  * download-signed: Fetch signed artefacts from versioned URL instead
    of current/ symlink to work around caching (LP: #1936640)

 -- Julian Andres Klode <email address hidden> Fri, 16 Jul 2021 13:18:10 +0200

The verification of the Stable Release Update for shim has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package shim - 15.4-0ubuntu7

---------------
shim (15.4-0ubuntu7) hirsute; urgency=medium

  * Fix load option parsing, and thus fwupd execution (LP: #1929471) (PR #379)
  * Fix occasional crashes in _relocate() on arm64 (LP: #1928010) (PR #383)
  * Fix accidental deletion of RT variables (LP: #1934506) (PR #387)
  * mok: relax the maximum variable size check (LP: #1934780) (PR #369)

 -- Julian Andres Klode <email address hidden> Wed, 07 Jul 2021 10:57:35 +0200

This bug was fixed in the package shim-signed - 1.40.6

---------------
shim-signed (1.40.6) focal; urgency=medium

  * Update to shim 15.4-0ubuntu7:
    - Fix load option parsing, and thus fwupd execution (LP: #1929471) (PR #379)
    - Fix occasional crashes in _relocate() on arm64 (LP: #1928010) (PR #383)
    - Fix accidental deletion of RT variables (LP: #1934506) (PR #387)
    - mok: relax the maximum variable size check (LP: #1934780) (PR #369)
  * download-signed: Fetch signed artefacts from versioned URL instead
    of current/ symlink to work around caching (LP: #1936640)

shim-signed (1.40.5) focal; urgency=medium

  * New upstream release 15.4. LP: #1921134
  * Synchronize packaging with 1.48, summary
    - Update packaging to pull fb and mm from shim-signed package as in
      later releases, dropping the runtime dependency on shim.
    - Add download-signed script from linux-signed package
    - Include reworked Makefile from devel to better assert the integrity of
      the executables.
    - Dual-signed shim
    - Set XB-Important: yes and Protected: yes on shim-signed package
      so that it cannot be removed by accident (LP: #1898729)
  * Update to shim 15.4-0ubuntu5:
    - Stop addending vendor dbx to MokListXRT during MokListX mirroring. This
      is causing systems to run out of EFI storage space, or just hang up
      when trying to write it (LP: #1924605) (LP: #1928434)
    - Further relax the check for variable mirroring on non-secureboot systems
      avoiding boot failures on out of space conditons (pull request #372)
    - Don't unhook ExitBootServices() when EBS protection is disabled
      (LP: #1931136) (pull request #378)

 -- Julian Andres Klode <email address hidden> Fri, 16 Jul 2021 13:33:00 +0200

Is it really fixed, though? I have shim-signed 1.40.6 and fwupd still fails to do anything upon reboot.

This bug was fixed in the package shim - 15.4-0ubuntu7

---------------
shim (15.4-0ubuntu7) hirsute; urgency=medium

  * Fix load option parsing, and thus fwupd execution (LP: #1929471) (PR #379)
  * Fix occasional crashes in _relocate() on arm64 (LP: #1928010) (PR #383)
  * Fix accidental deletion of RT variables (LP: #1934506) (PR #387)
  * mok: relax the maximum variable size check (LP: #1934780) (PR #369)

 -- Julian Andres Klode <email address hidden> Wed, 07 Jul 2021 10:57:35 +0200

This bug was fixed in the package shim-signed - 1.33.1~16.04.10

---------------
shim-signed (1.33.1~16.04.10) xenial; urgency=medium

  * Update to shim 15.4-0ubuntu7:
    - Fix load option parsing, and thus fwupd execution (LP: #1929471) (PR #379)
    - Fix occasional crashes in _relocate() on arm64 (LP: #1928010) (PR #383)
    - Fix accidental deletion of RT variables (LP: #1934506) (PR #387)
    - mok: relax the maximum variable size check (LP: #1934780) (PR #369)

shim-signed (1.33.1~16.04.9) xenial; urgency=medium

  * Do not build a dual-signed shim (fixing regression from ~16.04.7), and
    disable verifying fbx64.efi and mmx64.efi certificates as xenial's
    sbverify is unable to (impish works fine)
  * Clean up debhelper log file accidentally imported into git during 16.04.7
    import.

shim-signed (1.33.1~16.04.8) xenial; urgency=medium

  * debian/*.postinst: Unconditionally call grub-install with
    --force-extra-removable, so that the \EFI\BOOT removable path as used in
    cloud images receives the updates. LP: #1930742.
  * Update to shim 15.4-0ubuntu5:
    - Stop addending vendor dbx to MokListXRT during MokListX mirroring. This
      is causing systems to run out of EFI storage space, or just hang up
      when trying to write it (LP: #1924605) (LP: #1928434)
    - Further relax the check for variable mirroring on non-secureboot systems
      avoiding boot failures on out of space conditons (pull request #372)
    - Don't unhook ExitBootServices() when EBS protection is disabled
      (LP: #1931136) (pull request #378)

shim-signed (1.33.1~16.04.7) xenial; urgency=medium

  * New upstream release 15.4. LP: #1921134
  * Update packaging to pull fb and mm from shim-signed package as in
    later releases, dropping the runtime dependency on shim.
  * Add download-signed script from linux-signed package
  * Add a versioned dependency on the mokutil that introduces --timeout, and
    call mokutil --timeout -1 so that users don't end up with broken systems
    by missing MokManager on reboot after install. LP: #1856422.
  * Add versioned dependencies on grub-efi-amd64-signed and grub2-common,
    to ensure we have SBAT-compatible grub.efi and grub 2.04-compatible
    grub-install present when we are installing new shim to the ESP.
  * Include reworked Makefile from devel to better assert the integrity of
    the executables.

 -- Julian Andres Klode <email address hidden> Fri, 16 Jul 2021 13:04:57 +0200

This bug was fixed in the package shim - 15.4-0ubuntu7

---------------
shim (15.4-0ubuntu7) hirsute; urgency=medium

  * Fix load option parsing, and thus fwupd execution (LP: #1929471) (PR #379)
  * Fix occasional crashes in _relocate() on arm64 (LP: #1928010) (PR #383)
  * Fix accidental deletion of RT variables (LP: #1934506) (PR #387)
  * mok: relax the maximum variable size check (LP: #1934780) (PR #369)

 -- Julian Andres Klode <email address hidden> Wed, 07 Jul 2021 10:57:35 +0200

This bug was fixed in the package shim-signed - 1.37~18.04.10

---------------
shim-signed (1.37~18.04.10) bionic; urgency=medium

  * Remove unnecessary efitools dependency that prevented build on arm64

shim-signed (1.37~18.04.9) bionic; urgency=medium

  * New upstream release 15.4. LP: #1921134
  * Synchronize packaging with 1.50, summary
    - Update packaging to pull fb and mm from shim-signed package as in
      later releases, dropping the runtime dependency on shim.
    - Add download-signed script from linux-signed package
    - Include reworked Makefile from devel to better assert the integrity of
      the executables.
    - Dual-signed shim
    - Set XB-Important: yes on shim-signed package so that it cannot be
      removed by accident (LP: #1898729)
    - download-signed: Fetch signed artefacts from versioned URL instead
      of current/ symlink to work around caching (LP: #1936640)
  * Update to shim 15.4-0ubuntu5:
    - Stop addending vendor dbx to MokListXRT during MokListX mirroring. This
      is causing systems to run out of EFI storage space, or just hang up
      when trying to write it (LP: #1924605) (LP: #1928434)
    - Further relax the check for variable mirroring on non-secureboot systems
      avoiding boot failures on out of space conditons (pull request #372)
    - Don't unhook ExitBootServices() when EBS protection is disabled
      (LP: #1931136) (pull request #378)
  * Update to shim 15.4-0ubuntu7:
    - Fix load option parsing, and thus fwupd execution (LP: #1929471) (PR #379)
    - Fix occasional crashes in _relocate() on arm64 (LP: #1928010) (PR #383)
    - Fix accidental deletion of RT variables (LP: #1934506) (PR #387)
    - mok: relax the maximum variable size check (LP: #1934780) (PR #369)

 -- Julian Andres Klode <email address hidden> Mon, 19 Jul 2021 17:01:19 +0200

I have a Thinkpad P14s AMD gen1, and while trying to update bios 1.35 to 1.36 (and 1.37) it would fail, and fwmgr pointed me here.
I have now been hable to upgrade the bios after disabling the order lock in the BIOS, which I haven't changed, and had upgraded the bios in the past. Might be a new flag, or something else might be different, might worth trying for the people with thinkpads having issues.

@jixbo

Thank you for your post. Yet, are you saying that somehow boot-order-lock got enabled without you enabling it? Or are you saying one should toggle that option on and off again?

I tried to cover both those bases on my ThinkPad - by doing the following. I rebooted, entered the BIOS, and checked the boot-order-lock setting. It was off. I set it on and then off again. I saved-and-exited. The I ran fwupd and tried to install the pending updates. All seemed well - but all was well up to this stage before - and I was prompted to reboot. Upon that reboot, it was the same old badness: as before, I saw some Lenovo update stuff flash up, disappear very quickly, and then I was back to the desktop and none of the three pending firmware updates had installed.

I am heartily sick of this problem: it has been going on for at least a year now; no fix is in sight; and it is not even clear whether the problem lies with Lenovo or fwupd or with Ubuntu.

I should add the following (and indeed I should have included this within my previous post).

fwupdmgr told me three devices had installable updates: Lenovo diagnostics; a Logitech wireless receiver; Intel TPM.

Using @sj's method (and obtaining the device-id via `fwupdmg get-devices`) I managed to install one of diagnostics update. Upon reboot, fwupdmgr told me only one update was outstanding: the tpm one. So: the Logitech update had disappeared; and the TPM update remained uninstalled. One can see from the issue tracker for fwup that Lenovo TPM (or 'ME') updates are notorious for not installing. See: https://github.com/fwupd/firmware-lenovo/issues?q=tpm+is%3Aissue+is%3Aopen+