Bug #1787254 “Possibly demote fwupdate to universe?” : Bugs : fwupdate package : Ubuntu

Revision history for this message

Mario Limonciello (superm1) wrote on 2018-08-15:

#1

Download full text (8.3 KiB)

Here is the IRC transcript that prompted this bug:

<superm1> hi can an AA remove and block these from syncing https://launchpad.net/ubuntu/+source/fwupdate-amd64-signed https://launchpad.net/ubuntu/+source/fwupdate-arm64-signed https://launchpad.net/ubuntu/+source/fwupdate-i386-signed ? They're causing problems with https://launchpad.net/ubuntu/+source/fwupdate-signed/1.20
<slangasek> superm1: hi! I had noticed things going on there and wanted to ask why both fwupd and fwupdate are now producing artifacts for EFI signing
<slangasek> maybe that's all resolved now and we only need to handle the -signed packages now
<superm1> slangasek, i sent some mail to sil2100 to explain it all
<superm1> but i guess that didn't float around to everyone that needs to know
<slangasek> yes, emailing an individual has that effect :)
<superm1> heh. so basically starting with fwupd 1.1.0 the library and EFI binaries were subsumed into the fwupd project
<superm1> they are installed dynamically at runtime now
<superm1> lots of various reasons for this, but the general expectation is that due to this change less bugs, better control on the process, easier to support problems
<superm1> the fwupdate project still exists and can be treated as a simple reference implementation of EFI firmware updates if you don't care about all the other stuff
<superm1> in my mind fwupdate and fwupdate-signed should move to universe in cosmic
<slangasek> superm1: http://archive.ubuntu.com/ubuntu/dists/cosmic-proposed/main/uefi/fwupdate-amd64/current/fwupx64.efi.signed http://archive.ubuntu.com/ubuntu/dists/cosmic-proposed/main/uefi/fwupd-amd64/1.1.1-1/fwupdx64.efi.signed why are there two of these?
<superm1> see above two comments
<slangasek> so
<superm1> I unseeded fwupdate for this purpose on the next meta upload: https://git.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/+git/platform/commit/?id=e44246ad2a65dacc3e816715d0df7bd86cd9a2ac
<slangasek> we shouldn't be signing two streams of EFI binaries
<slangasek> we shouldn't be signing something that's just a "reference implementation"
<cyphermox> moo?
<superm1> Moo
<slangasek> and it's regrettable if sil2100 accepted these for signing on the basis of your conversation
<slangasek> we should instead drop the signing request bits from the package
<superm1> Debian is still going to sign both of them
<slangasek> I'm not responsible for Debian's EFI keys :)
<superm1> and let folks decide if they want to use one or the other
<cyphermox> oh my
<superm1> Sledge told me that infinity and cjwatson had interest in Debian's EFI signing system as well and are going to look into it. If Ubuntu adopts something akin to it it will be more difficult to drop the signed bits only from Ubuntu for fwupdate
<slangasek> sorry, if Ubuntu adopts something akin to what?
<superm1> the signing process for EFI binaries
<slangasek> ok
<superm1> right now what happens in Debian is that template packages are created and go itno the archive as binary packages
<slangasek> so, regardless, my expectation is that the set of objects signed with the EFI key will be gated on archive admin signoff
<superm1> a signing service runs and installs these template packages and produ...

Here is the IRC transcript that prompted this bug:

<superm1> hi can an AA remove and block these from syncing https://launchpad.net/ubuntu/+source/fwupdate-amd64-signed https://launchpad.net/ubuntu/+source/fwupdate-arm64-signed https://launchpad.net/ubuntu/+source/fwupdate-i386-signed ? They're causing problems with https://launchpad.net/ubuntu/+source/fwupdate-signed/1.20
<slangasek> superm1: hi!  I had noticed things going on there and wanted to ask why both fwupd and fwupdate are now producing artifacts for EFI signing
<slangasek> maybe that's all resolved now and we only need to handle the -signed packages now
<superm1> slangasek, i sent some mail to sil2100 to explain it all 
<superm1> but i guess that didn't float around to everyone that needs to know 
<slangasek> yes, emailing an individual has that effect :)
<superm1> heh. so basically starting with fwupd 1.1.0 the library and EFI binaries were subsumed into the fwupd project
<superm1> they are installed dynamically at runtime now
<superm1> lots of various reasons for this, but the general expectation is that due to this change less bugs, better control on the process, easier to support problems
<superm1> the fwupdate project still exists and can be treated as a simple reference implementation of EFI firmware updates if you don't care about all the other stuff
<superm1> in my mind fwupdate and fwupdate-signed should move to universe in cosmic
<slangasek> superm1: http://archive.ubuntu.com/ubuntu/dists/cosmic-proposed/main/uefi/fwupdate-amd64/current/fwupx64.efi.signed http://archive.ubuntu.com/ubuntu/dists/cosmic-proposed/main/uefi/fwupd-amd64/1.1.1-1/fwupdx64.efi.signed why are there two of these?
<superm1> see above two comments
<slangasek> so
<superm1> I unseeded fwupdate for this purpose on the next meta upload: https://git.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/+git/platform/commit/?id=e44246ad2a65dacc3e816715d0df7bd86cd9a2ac
<slangasek> we shouldn't be signing two streams of EFI binaries
<slangasek> we shouldn't be signing something that's just a "reference implementation"
<cyphermox> moo?
<superm1> Moo
<slangasek> and it's regrettable if sil2100 accepted these for signing on the basis of your conversation
<slangasek> we should instead drop the signing request bits from the package
<superm1> Debian is still going to sign both of them
<slangasek> I'm not responsible for Debian's EFI keys :)
<superm1> and let folks decide if they want to use one or the other
<cyphermox> oh my
<superm1> Sledge told me that infinity and cjwatson had interest in Debian's EFI signing system as well and are going to look into it.  If Ubuntu adopts something akin to it it will be more difficult to drop the signed bits only from Ubuntu for fwupdate
<slangasek> sorry, if Ubuntu adopts something akin to what?
<superm1> the signing process for EFI binaries
<slangasek> ok
<superm1> right now what happens in Debian is that template packages are created and go itno the archive as binary packages
<slangasek> so, regardless, my expectation is that the set of objects signed with the EFI key will be gated on archive admin signoff
<superm1> a signing service runs and installs these template packages and produces new source packages "with the signed" binary
<superm1> and those source packages are uploaded to the archive
<superm1> so yes I would expect the gate to be on accepting "those" source packages
<superm1> slangasek, the other thing i'm worried about with demoting fwupdate to universe and dropping fwupdate-signed is I have no idea what that does to ubuntu core guys
<superm1> if they still use it for generating images or what
<superm1> it's in the "system-image" seed and I suspect they have an expectation that they get the EFI binary installed as part of "core" and not at runtime
<superm1> cjwatson, OK thanks, I sorta expected that
<cjwatson> There are pros and cons; it may be worth it for the sake of being in sync, and I can see the advantages
<cjwatson> Particularly reducing the number of round-trips for updates
<cjwatson> But it doesn't make it an easy change to implement :)
<superm1> yeah from a package maintainer perspective it's nice to not have to update a separate signed -source package 
<cjwatson> And it's hard to justify spending time on it when it just streamlines things a bit rather than enabling new functionality
<superm1> I think Debian is still working out kinks with their system anyway, it doesn't run automatically just yet
<cjwatson> Indeed
<cjwatson> In a Launchpad context it would involve dispatching subsidiary builds, I think
<cjwatson> Which is ... complex
<superm1> I see
<cjwatson> I kind of know what sort of general shape I'd like it to be but haven't worked out the data model
<cjwatson> What we definitely *can't* do is construct debs somewhere other than on builders
<superm1> I think even in their model the debs would be constructed on builders 
<cjwatson> Yeah, I haven't seen how the dispatch works there ...
<superm1> their model generates source packages that get uploaded
<cjwatson> Right, what I haven't worked out is how those get injected
<cjwatson> And whether there's any DB link between those and the unsigned versions
<cjwatson> I'd kind of rather construct the source packages on builders too
<cjwatson> Like recipes
<cjwatson> That means we get to use tools appropriate to the target series to build the source package, rather than having to either do it by hand or have interesting dependencies on the Ubuntu release that Launchpad happens to run on
<cjwatson> (ICBW but I suspect that Debian is taking something equivalent to the latter approach)
<cjwatson> Recipes have some pretty weird DB modelling though
<cjwatson> Anyway, that's as far as I've got
<superm1> building the source package outside of a builder is actually quite challenging from the debian side today
<superm1> if it fails to build you have no idea why
<superm1> so if you deviated from the "template" used by another signed package a little bit or have a minor typographical error you can't really debug
<superm1> so back on the initial question around fwupdate/fwupdate-signed and what to do with them.  I'll file a bug with this conversation and subscribe ubuntu-archive and whoever is the right person from ubuntu-core that can commment what ripping fwupdate and fwupdate-signed out of system-image does.  slangasek any idea who from core can speak to that?
<cjwatson> https://salsa.debian.org/ftp-team/code-signing/blob/master/secure-boot-code-sign.py is the relevant thing in Debian; definitely looks like that's just going to run with (I assume) stable's dpkg-dev
<cjwatson> which I guess isn't awful since dpkg-dev is pretty stable, but I'd hope not to have to install it on LP systems
<cjwatson> we'd need an isolated signing service in the same way
<superm1> the other dimension I didn't mention is that there may be a need for backporting either fwupdate 12 or fwupd 1.1.1 back to bionic due to https://bugs.launchpad.net/ubuntu/+source/fwupdate/+bug/1785165  I guess i'll add that to the bug for discussion too 
<ubot5> Ubuntu bug 1785165 in fwupdate-signed (Ubuntu) "firmware update on fwupdate version 10-3 not work on some AMI's firmwares" [Undecided,New]
<slangasek> superm1: I would have to check what's done with fwupdate in Ubuntu Core, but snaps are not necessarily built from the devel series and core snaps in particular are only built from LTSes.  So any change that's appropriate to make here in terms of preferred EFI implementation for Ubuntu classic should also be tracked by Ubuntu Core in the next major series, and would not affect builds of the
<slangasek> current series.
<superm1> I guess this is now a bigger problem then for them that snaps probably can't put binaries into the ESP directly 
<superm1> and they'll have to sort that in snapd by the next LTS?
<slangasek> quite possibly
<slangasek> do the two executables (fwupx64.efi, fwupdx64.efi) provide the same interface?  Do they get installed to the same well known location on the ESP?
<superm1> they purposefully use a different EFI NVRAM variable to communicate information
<superm1> and both get installed into respective binary names (fwupx64.efi or fwupdx64.efi)
<superm1> to allow both implementations to be installed side by side 
<slangasek> well, that does make a transition more complicated
<superm1> depends on which lens you look at it from.  it allows the contents of the NVRAM interface variable to change in incompatible ways in the futur

Revision history for this message

Steve Langasek (vorlon) wrote on 2018-08-15:

#2

As an archive admin I've said that we should not be signing two parallel streams of EFI binaries implementing this functionality. If fwupd is considered the preferred implementation upstream and Ubuntu is going to adopt this, then we should remove fwupdate-signed entirely from the archive and stop producing artifacts from fwupdate source package for EFI signing.

If there are design reasons why Ubuntu Core should prefer fwupx64.efi over fwupdx64.efi going forward, then we should clarify what these are and evaluate whether Ubuntu classic should follow suit. Otherwise, we should drop fwupdate-signed from the archive, adjust the fwupdate source package to not generate EFI artifacts for signing, and ensure that snapd migrates to fwupd by 20.04.

John, I suggested your name to Mario as a possible first contact for this on the Snappy side, but please escalate this as appropriate.

NB I can't see anywhere in the snapd code or in the pc gadget snap where fwupx64.efi is ever installed to the ESP, so it's entirely unclear to me how this currently works on Ubuntu Core either.

Changed in snapd (Ubuntu):
assignee:	nobody → John Lenton (chipaca)

Revision history for this message

Steve Langasek (vorlon) wrote on 2018-08-15:

#3

Based on the analysis in the previous comment, I'm going to go ahead and delete fwupdate-signed now from cosmic (and blacklist it). If at a later date it is determined that Ubuntu Core does need fwupdate in Ubuntu 20.04 we can reintroduce it, but in the meantime we shouldn't carry on producing and signing EFI executables in the archive that we don't expect to be used.

Revision history for this message

Mario Limonciello (superm1) wrote on 2018-08-15:

#4

Thanks Steve. I've made the matching change in packaging so any future fwupdate uploads to Debian that sync won't try to make signed packages again.

https://salsa.debian.org/efi-team/fwupdate/commit/104e1e86d9b318fd9831ecc16112d3d3c060426e

Changed in fwupdate (Ubuntu):
status:	New → Fix Committed

Revision history for this message

Steve Langasek (vorlon) wrote on 2018-08-15:

#5

Removing packages from cosmic:
fwupdate-signed 1.18 in cosmic
  fwupdate-amd64-signed 1.18+10-3 in cosmic amd64
  fwupdate-arm64-signed 1.18+10-3 in cosmic arm64
  fwupdate-armhf-signed 1.18+10-3 in cosmic armhf
  fwupdate-i386-signed 1.18+10-3 in cosmic i386
  fwupdate-signed 1.18+10-3 in cosmic amd64
  fwupdate-signed 1.18+10-3 in cosmic arm64
  fwupdate-signed 1.18+10-3 in cosmic armhf
  fwupdate-signed 1.18+10-3 in cosmic i386
Comment: Superseded by fwupdx64.efi from fwupd source; LP: #1787254
1 package successfully removed.
Removing packages from cosmic-proposed:
fwupdate-signed 1.20 in cosmic
  fwupdate-armhf-signed 1.20+12-3 in cosmic armhf
  fwupdate-signed 1.20+12-3 in cosmic armhf
Comment: Superseded by fwupdx64.efi from fwupd source; LP: #1787254
1 package successfully removed.

Changed in fwupdate-signed (Ubuntu):
status:	New → Fix Released

Revision history for this message

Mario Limonciello (superm1) wrote on 2018-08-16:

#6

As also found by this, the signing source packages from debian need to be blocked too. That's filed here: https://bugs.launchpad.net/ubuntu/+source/fwupdate-amd64-signed/+bug/1787315

Revision history for this message

John Lenton (chipaca) wrote on 2018-08-16:

#7

I don't know much about this, but I'm asking different people and I'll answer here as I find out.

We don't have fwupdate in ubuntu core itself:

    $ find EFI -type f
    EFI/boot/grubx64.efi
    EFI/boot/bootx64.efi
    EFI/ubuntu/grubenv
    EFI/ubuntu/grub.cfg

it is used in some _customer_ images of ubuntu core. I'll be asking field engineering about whether and why they prefer one over the other (and whether this decision affects them in any way).

Revision history for this message

Łukasz Zemczak (sil2100) wrote on 2018-09-05:

#8

I can confirm that in core18 we also don't have fwupdate. The system-image seed is used during ubuntu-base builds that are used as the base of core18, but fwupdate is not part of it - and it is not installed separately either.

Revision history for this message

Steve Langasek (vorlon) wrote on 2019-04-17:

#9

I have swapped fwupdate for fwupd in the system-image seeds so that fwupdate can be demoted to universe for disco. Since it does not appear that either fwupd or fwupdate are actually used as part of the core18 build today, we should at least be referring to the *right* unused implementation.

Override component to universe
fwupdate 12-3ubuntu2 in disco: main/misc -> universe
fwupdate 12-3ubuntu2 in disco amd64: main/admin/optional/100% -> universe
fwupdate 12-3ubuntu2 in disco arm64: main/admin/optional/100% -> universe
fwupdate 12-3ubuntu2 in disco armhf: main/admin/optional/100% -> universe
fwupdate 12-3ubuntu2 in disco i386: main/admin/optional/100% -> universe
fwupdate-amd64-signed-template 12-3ubuntu2 in disco amd64: universe/libs/optional/100% -> universe
fwupdate-arm64-signed-template 12-3ubuntu2 in disco arm64: universe/libs/optional/100% -> universe
fwupdate-armhf-signed-template 12-3ubuntu2 in disco armhf: universe/libs/optional/100% -> universe
fwupdate-i386-signed-template 12-3ubuntu2 in disco i386: universe/libs/optional/100% -> universe
libfwup-dev 12-3ubuntu2 in disco amd64: universe/libdevel/optional/100% -> universe
libfwup-dev 12-3ubuntu2 in disco arm64: universe/libdevel/optional/100% -> universe
libfwup-dev 12-3ubuntu2 in disco armhf: universe/libdevel/optional/100% -> universe
libfwup-dev 12-3ubuntu2 in disco i386: universe/libdevel/optional/100% -> universe
libfwup1 12-3ubuntu2 in disco amd64: main/libs/optional/100% -> universe
libfwup1 12-3ubuntu2 in disco arm64: main/libs/optional/100% -> universe
libfwup1 12-3ubuntu2 in disco armhf: main/libs/optional/100% -> universe
libfwup1 12-3ubuntu2 in disco i386: main/libs/optional/100% -> universe
fwupdate-amd64-signed-template 12-3ubuntu2 in disco amd64 remained the same
fwupdate-arm64-signed-template 12-3ubuntu2 in disco arm64 remained the same
fwupdate-armhf-signed-template 12-3ubuntu2 in disco armhf remained the same
fwupdate-i386-signed-template 12-3ubuntu2 in disco i386 remained the same
libfwup-dev 12-3ubuntu2 in disco amd64 remained the same
libfwup-dev 12-3ubuntu2 in disco arm64 remained the same
libfwup-dev 12-3ubuntu2 in disco armhf remained the same
libfwup-dev 12-3ubuntu2 in disco i386 remained the same
9 publications overridden; 8 publications remained the same.

And unsubscribing ubuntu-archive.

I have swapped fwupdate for fwupd in the system-image seeds so that fwupdate can be demoted to universe for disco.  Since it does not appear that either fwupd or fwupdate are actually used as part of the core18 build today, we should at least be referring to the *right* unused implementation.

Override component to universe
fwupdate 12-3ubuntu2 in disco: main/misc -> universe
fwupdate 12-3ubuntu2 in disco amd64: main/admin/optional/100% -> universe
fwupdate 12-3ubuntu2 in disco arm64: main/admin/optional/100% -> universe
fwupdate 12-3ubuntu2 in disco armhf: main/admin/optional/100% -> universe
fwupdate 12-3ubuntu2 in disco i386: main/admin/optional/100% -> universe
fwupdate-amd64-signed-template 12-3ubuntu2 in disco amd64: universe/libs/optional/100% -> universe
fwupdate-arm64-signed-template 12-3ubuntu2 in disco arm64: universe/libs/optional/100% -> universe
fwupdate-armhf-signed-template 12-3ubuntu2 in disco armhf: universe/libs/optional/100% -> universe
fwupdate-i386-signed-template 12-3ubuntu2 in disco i386: universe/libs/optional/100% -> universe
libfwup-dev 12-3ubuntu2 in disco amd64: universe/libdevel/optional/100% -> universe
libfwup-dev 12-3ubuntu2 in disco arm64: universe/libdevel/optional/100% -> universe
libfwup-dev 12-3ubuntu2 in disco armhf: universe/libdevel/optional/100% -> universe
libfwup-dev 12-3ubuntu2 in disco i386: universe/libdevel/optional/100% -> universe
libfwup1 12-3ubuntu2 in disco amd64: main/libs/optional/100% -> universe
libfwup1 12-3ubuntu2 in disco arm64: main/libs/optional/100% -> universe
libfwup1 12-3ubuntu2 in disco armhf: main/libs/optional/100% -> universe
libfwup1 12-3ubuntu2 in disco i386: main/libs/optional/100% -> universe
fwupdate-amd64-signed-template 12-3ubuntu2 in disco amd64 remained the same
fwupdate-arm64-signed-template 12-3ubuntu2 in disco arm64 remained the same
fwupdate-armhf-signed-template 12-3ubuntu2 in disco armhf remained the same
fwupdate-i386-signed-template 12-3ubuntu2 in disco i386 remained the same
libfwup-dev 12-3ubuntu2 in disco amd64 remained the same
libfwup-dev 12-3ubuntu2 in disco arm64 remained the same
libfwup-dev 12-3ubuntu2 in disco armhf remained the same
libfwup-dev 12-3ubuntu2 in disco i386 remained the same
9 publications overridden; 8 publications remained the same.

And unsubscribing ubuntu-archive.

Changed in fwupdate (Ubuntu):
status:	Fix Committed → Fix Released

Revision history for this message

Michael Vogt (mvo) wrote on 2019-10-16:

#10

I'm closing the snapd task because it looks like we don't need fwupdate for uc20. If it turns out I was wrong this needs to be reopened.

Changed in snapd:
status:	New → Invalid
Changed in snapd (Ubuntu):
status:	New → Invalid

Ubuntu
fwupdate package

Possibly demote fwupdate to universe?

Bug Description

Related branches

Other bug subscribers

Remote bug watches

	Status	Importance	Assigned to
snapd	Invalid	Undecided	Unassigned
fwupdate (Ubuntu)	Fix Released	Undecided	Unassigned
fwupdate-signed (Ubuntu)	Fix Released	Undecided	Unassigned
snapd (Ubuntu)	Invalid	Undecided	John Lenton

Ubuntufwupdate package

Possibly demote fwupdate to universe?

Bug Description

Related branches

Other bug subscribers

Remote bug watches

Ubuntu
fwupdate package