OEM kernel removed during install, metapackage marked for removal in the target system

Bug #1914091 reported by Iain Lane
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubiquity (Ubuntu)
Fix Released
Critical
Iain Lane
Focal
Fix Released
Critical
Iain Lane

Bug Description

[ Description ]

This is from a "sutton.newell-ace" system, installed from Ubuntu Focal daily 20210201.1:

laney@focal-test-nvidia-haxed:~$ sudo apt autoremove
[sudo] password for laney:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED
  libnvidia-cfg1-460 libnvidia-common-460 libnvidia-compute-460 libnvidia-decode-460 libnvidia-encode-460 libnvidia-extra-460 libnvidia-fbc1-460 libnvidia-gl-460 libnvidia-ifr1-460 libvdpau1
  libxnvctrl0 linux-modules-nvidia-460-generic-hwe-20.04-edge mesa-vdpau-drivers nvidia-compute-utils-460 nvidia-driver-460 nvidia-kernel-common-460 nvidia-kernel-source-460 nvidia-prime
  nvidia-settings nvidia-utils-460 oem-sutton.newell-ace-meta screen-resolution-extra ubuntu-oem-keyring vdpau-driver-all xserver-xorg-video-nvidia-460
0 to upgrade, 0 to newly install, 25 to remove and 0 not to upgrade.
After this operation, 491 MB disk space will be freed.
Do you want to continue? [Y/n] n
Abort.
laney@focal-test-nvidia-haxed:~$ apt policy oem-sutton.newell-ace-meta
oem-sutton.newell-ace-meta:
  Installed: 20.04~ubuntu1
  Candidate: 20.04~ubuntu1
  Version table:
 *** 20.04~ubuntu1 100
        100 /var/lib/dpkg/status
laney@focal-test-nvidia-haxed:~$ apt show oem-sutton.newell-ace-meta
Package: oem-sutton.newell-ace-meta
Version: 20.04~ubuntu1
Status: install ok installed
Priority: optional
Section: misc
Maintainer: Commercial Engineering <email address hidden>
Installed-Size: 13.3 kB
Depends: ubuntu-oem-keyring
Modaliases: meta(dmi:*bvnLENOVO:bvrN2W*)
Ubuntu-Oem-Kernel-Flavour: oem
Download-Size: unknown
APT-Manual-Installed: no
APT-Sources: /var/lib/dpkg/status
Description: hardware support for Sutton Newell Ace platform
 This is a metapackage for Sutton Newell Ace platform. It installs
 packages needed to support this hardware fully.

1) The packages are marked for autoremoval and shouldn't be.
2) The OEM kernel is not installed.

In the syslog we can see:

Feb 1 14:08:37 ubuntu /plugininstall.py: Autopurge linux-headers-5.6.0-1042-oem
Feb 1 14:08:37 ubuntu /plugininstall.py: Autopurge linux-headers-oem-20.04
Feb 1 14:08:37 ubuntu /plugininstall.py: Autopurge linux-image-5.6.0-1042-oem
Feb 1 14:08:37 ubuntu /plugininstall.py: Autopurge linux-image-oem-20.04
Feb 1 14:08:37 ubuntu /plugininstall.py: Autopurge linux-modules-5.6.0-1042-oem
Feb 1 14:08:37 ubuntu /plugininstall.py: Autopurge linux-oem-20.04
Feb 1 14:08:37 ubuntu /plugininstall.py: Autopurge linux-oem-5.6-headers-5.6.0-1042

that the kernel was autoremoved.

[ Fix ]

This is a regression from 21.04.5 / 20.04.15.5. The new code needs to use "from_user=True", so that the packages are marked as manually installed and therefore don't get autoremoved.

[ QA ]

1. Boot the installer on a system which requires an OEM metapackage and the OEM kernel. (ubuntu-drivers list-oem is nonempty; apt show THATPACKAGE shows Ubuntu-Oem-Kernel-Flavour missing or as 'oem', not 'default')
1a. Alternatively, boot to a live session and run "echo oem-sutton.newell-ace-meta | sudo tee /run/ubuntu-drivers-oem.autoinstall"
2. Install the system as normal.
3. Reboot into the target and check
  3a. That `uname -a` shows you are on the OEM kernel
  3b. That `apt autoremove` shows no packages for autoremoval

[ Where things could go wrong ]

Messing with the code which is deciding what to install in the target system is always a bit delicate. I think the most likely thing to happen is that we end up marking too much stuff as manual so it never gets autoremoved. Checking on a system installed with the proposed ubiquity shows:

laney@7390-focal-test:~$ apt list '?installed ?not(?automatic)'
Listing... Done
base-passwd/focal,now 3.5.47 amd64 [installed]
dash/focal,now 0.5.10.2-6 amd64 [installed]
diffutils/focal,now 1:3.7-3 amd64 [installed]
findutils/focal,now 4.7.0-1ubuntu1 amd64 [installed]
firefox-locale-en/focal-updates,focal-security,now 84.0.2+build1-0ubuntu0.20.04.1 amd64 [installed]
fonts-indic/focal,focal,now 2:1.3 all [installed]
grep/focal,now 3.4-1 amd64 [installed]
grub-common/focal-updates,now 2.04-1ubuntu26.8 amd64 [installed]
grub-efi-amd64-signed/focal-updates,now 1.142.10+2.04-1ubuntu26.8 amd64 [installed]
grub-gfxpayload-lists/focal,now 0.7 amd64 [installed]
grub-pc-bin/focal-updates,now 2.04-1ubuntu26.8 amd64 [installed]
grub-pc/focal-updates,now 2.04-1ubuntu26.8 amd64 [installed]
grub2-common/focal-updates,now 2.04-1ubuntu26.8 amd64 [installed]
gzip/focal,now 1.10-0ubuntu4 amd64 [installed]
hostname/focal,now 3.23 amd64 [installed]
hyphen-en-gb/focal,focal,now 1:6.4.3-1 all [installed]
hyphen-en-us/focal,focal,now 2.8.8-7 all [installed]
init/focal,now 1.57 amd64 [installed]
language-pack-en-base/focal-updates,focal-updates,now 1:20.04+20210121 all [installed]
language-pack-en/focal-updates,focal-updates,now 1:20.04+20210121 all [installed]
language-pack-gnome-en-base/focal-updates,focal-updates,now 1:20.04+20210121 all [installed]
language-pack-gnome-en/focal-updates,focal-updates,now 1:20.04+20210121 all [installed]
libdebconfclient0/focal,now 0.251ubuntu1 amd64 [installed]
libfprint-2-tod1/focal-updates,now 1:1.90.2+tod1-0ubuntu1~20.04.4 amd64 [installed]
libllvm9/focal,now 1:9.0.1-12 amd64 [installed]
libreoffice-help-common/focal-updates,focal-updates,now 1:6.4.6-0ubuntu0.20.04.1 all [installed]
libreoffice-help-en-gb/focal-updates,focal-updates,now 1:6.4.6-0ubuntu0.20.04.1 all [installed]
libreoffice-help-en-us/focal-updates,focal-updates,now 1:6.4.6-0ubuntu0.20.04.1 all [installed]
libreoffice-l10n-en-gb/focal-updates,focal-updates,now 1:6.4.6-0ubuntu0.20.04.1 all [installed]
linux-oem-20.04/focal-updates,focal-security,now 5.6.0.1042.38 amd64 [installed]
mokutil/focal,now 0.3.0+1538710437.fb6250f-1 amd64 [installed]
mythes-en-us/focal,focal,now 1:6.4.3-1 all [installed]
ncurses-base/focal,focal,now 6.2-0ubuntu2 all [installed]
ncurses-bin/focal,now 6.2-0ubuntu2 amd64 [installed]
oem-somerville-three-eyed-raven-meta/unknown,unknown 20.04ubuntu4 all [upgradable from: 20.04~ubuntu2]
oem-sutton.newell-ace-meta/unknown,unknown 20.04ubuntu7 all [upgradable from: 20.04~ubuntu1]
os-prober/focal,now 1.74ubuntu2 amd64 [installed]
shim-signed/focal-updates,now 1.40.4+15+1552672080.a4a1fbe-0ubuntu2 amd64 [installed]
thunderbird-locale-en-gb/focal-updates,focal-updates,focal-security,focal-security,now 1:68.10.0+build1-0ubuntu0.20.04.1 all [installed]
thunderbird-locale-en-us/focal-updates,focal-updates,focal-security,focal-security,now 1:68.10.0+build1-0ubuntu0.20.04.1 all [installed]
thunderbird-locale-en/focal-updates,focal-security,now 1:68.10.0+build1-0ubuntu0.20.04.1 amd64 [installed]
ubuntu-desktop-minimal/focal-updates,now 1.450.2 amd64 [installed]
ubuntu-desktop/focal-updates,now 1.450.2 amd64 [installed]
ubuntu-minimal/focal-updates,now 1.450.2 amd64 [installed]
ubuntu-standard/focal-updates,now 1.450.2 amd64 [installed]
ubuntu-wallpapers/focal,focal,now 20.04.2-0ubuntu1 all [installed]

which I think is OK, although there's scope for optimisation here with some of the libraries that are marked manual.

Related branches

Iain Lane (laney)
Changed in ubiquity (Ubuntu):
status: New → In Progress
importance: Undecided → Critical
Changed in ubiquity (Ubuntu Focal):
importance: Undecided → Critical
status: New → In Progress
Changed in ubiquity (Ubuntu):
assignee: nobody → Iain Lane (laney)
Changed in ubiquity (Ubuntu Focal):
assignee: nobody → Iain Lane (laney)
milestone: none → ubuntu-20.04.2
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Iain, or anyone else affected,

Accepted ubiquity into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubiquity/20.04.15.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubiquity (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Brian Murray (brian-murray) wrote :

I've verified the test case in the bug description.

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubiquity - 21.04.8

---------------
ubiquity (21.04.8) hirsute; urgency=medium

  * install_misc: Mark all installed packages as manually installed. When
    we're marking packages to install, mark them as `from_user=True`, which
    means manually installed. This prevents them being autoremoved either
    during installation or afterwards in the target system. Fixes removal of
    the OEM kernel resulting in unbootable certified systems. (LP: #1914091)

 -- Iain Lane <email address hidden> Mon, 01 Feb 2021 17:03:00 +0000

Changed in ubiquity (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubiquity - 20.04.15.6

---------------
ubiquity (20.04.15.6) focal; urgency=medium

  * install_misc: Mark all installed packages as manually installed. When
    we're marking packages to install, mark them as `from_user=True`, which
    means manually installed. This prevents them being autoremoved either
    during installation or afterwards in the target system. Fixes removal of
    the OEM kernel resulting in unbootable certified systems. (LP: #1914091)

 -- Iain Lane <email address hidden> Mon, 01 Feb 2021 17:14:59 +0000

Changed in ubiquity (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for ubiquity has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.