dist upgrade process passes a number along with package to apport

Bug #1884148 reported by errors.ubuntu.com bug bridge
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-release-upgrader (Ubuntu)
Fix Released
Critical
Brian Murray
Focal
Fix Released
Critical
Brian Murray

Bug Description

[Impact]
The distribution upgrade process is passing to apport packages with numbers prepended to them, this then causes apport to crash and bugs about package installation failures not to be reported.

[Test Case]
1) Install the package pure-ftpd on an 18.04 LTS system
2) Run 'do-release-upgrade -d' to upgrade to 20.04 LTS

With the version of the dist-upgrader in the release pocket you'll receive the Traceback in comment #2.

To test the dist-upgrader from -proposed you'll actually have to download and use the dist-upgrader tarball from http://archive.ubuntu.com/ubuntu/dists/focal-proposed/main/dist-upgrader-all/current/focal.tar.gz. When running that version of the dist-upgrader you should not receive a Traceback and have a pure-ftpd-common crash file in /var/crash/.

This is actually also recreatable when upgrading from 19.10 to 20.04 and that would be easier as you can use 'do-release-upgrader -p' to test the new version of the dist-upgrader.

[Regression Potential]
The code change is a just a simple check to see if the package name starts with a number which then removes the number. So the only chance of regression is the additional python code being bad which it isn't.

Original Description
--------------------
The Ubuntu Error Tracker has been receiving reports about a problem regarding apport. This problem was most recently seen with package version 2.20.11-0ubuntu40, the problem page at https://errors.ubuntu.com/problem/976abccbca7b548c9ccf155c80c8eb6f36efa7de contains more details, including versions of packages affected, stacktrace or traceback, and individual crash reports.
If you do not have access to the Ubuntu Error Tracker and are a software developer, you can request it at http://forms.canonical.com/reports/.

Revision history for this message
Brian Murray (brian-murray) wrote :

It looks to me like the hook which is supposed to report package installation failures is now not working likely because '-p' is expecting or being passed the wrong value. Here are some examples:

/usr/bin/python3 /usr/share/apport/package_hook -p 016-google-chrome-stable --tags dist-upgrade ...
/usr/bin/python3 /usr/share/apport/package_hook -p 364-openjdk-11-jdk-headless --tags dist-upgrade ...
/usr/bin/python3 /usr/share/apport/package_hook -p 687-pop-icon-theme --tags dist-upgrade ...

tags: added: rls-gg-incoming
Revision history for this message
Brian Murray (brian-murray) wrote :

Here's the traceback too:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 297, in __getitem__
    rawpkg = self._cache[key]
KeyError: '23-gimp-data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 156, in _apt_pkg
    return self._cache()[package]
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 299, in __getitem__
    raise KeyError('The cache has no package named %r' % key)
KeyError: "The cache has no package named '23-gimp-data'"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/apport/package_hook", line 48, in <module>
    pr['SourcePackage'] = apport.packaging.get_source(options.package)
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 189, in get_source
    if self._apt_pkg(package).installed:
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 158, in _apt_pkg
    raise ValueError('package %s does not exist' % package)
ValueError: package 23-gimp-data does not exist

tags: added: id-5eebdf9f5b43fc339de86acb
Revision history for this message
Brian Murray (brian-murray) wrote :

I was able to recreate this during a distribution upgrade from 18.04 LTS to 20.04 LTS with pure-ftpd installed.

Changed in apport (Ubuntu):
status: New → In Progress
importance: Undecided → Critical
assignee: nobody → Brian Murray (brian-murray)
summary: - /usr/share/apport/package_hook:ValueError:__getitem__:_apt_pkg:__getitem__:/usr/share/apport/package_hook@48:get_source:_apt_pkg
+ dist upgrade process passes a number along with package to apport
Changed in apport (Ubuntu Focal):
status: New → In Progress
importance: Undecided → Critical
assignee: nobody → Brian Murray (brian-murray)
Changed in apport (Ubuntu Focal):
milestone: none → ubuntu-20.04.1
description: updated
affects: apport (Ubuntu) → ubuntu-release-upgrader (Ubuntu)
description: updated
tags: removed: rls-gg-incoming
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:20.10.3

---------------
ubuntu-release-upgrader (1:20.10.3) groovy; urgency=medium

  [ Dimitri John Ledkov ]
  * Use ubuntu-archive-keyring to verify upgrades, irrespective of how
    apt-key is configured.

  [ Brian Murray ]
  * utils/demotions.py: also use the ubuntu-archive-keyring when determining
    packages that were demoted.

 -- Brian Murray <email address hidden> Tue, 30 Jun 2020 09:15:09 -0700

Changed in ubuntu-release-upgrader (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello errors.ubuntu.com, or anyone else affected,

Accepted ubuntu-release-upgrader into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:20.04.20 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-release-upgrader (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (ubuntu-release-upgrader/1:20.04.20)

All autopkgtests for the newly accepted ubuntu-release-upgrader (1:20.04.20) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

update-manager/1:20.04.10.1 (armhf)
update-motd/3.6-0ubuntu6 (armhf, ppc64el, amd64, arm64, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#ubuntu-release-upgrader

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Brian Murray (brian-murray) wrote :

Using the version of the dist-upgrader from focal-proposed I did not encounter a Traceback and did end up with a crash file about the installation of pure-ftpd-common. So I'm setting this to verification-done.

bdmurray@clean-bionic-amd64:/tmp/u-r-u$ ls /var/crash/
pure-ftpd-common.0.crash
bdmurray@clean-bionic-amd64:/tmp/u-r-u$ head /var/log/dist-upgrade/main.log
2020-07-02 09:55:51,371 INFO Using config files '['./DistUpgrade.cfg.bionic']'
2020-07-02 09:55:51,371 INFO uname information: 'Linux clean-bionic-amd64 4.15.0-109-generic #110-Ubuntu SMP Tue Jun 23 02:39:32 UTC 2020 x86_64'
2020-07-02 09:55:51,759 INFO apt version: '1.6.12ubuntu0.1'
2020-07-02 09:55:51,759 INFO python version: '3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0]'
2020-07-02 09:55:51,761 INFO release-upgrader version '20.04.20' started
2020-07-02 09:55:51,766 INFO locale: 'en_US' 'UTF-8'
2020-07-02 09:55:51,872 DEBUG Using 'DistUpgradeViewText' view
2020-07-02 09:55:51,912 DEBUG enable dpkg --force-overwrite
2020-07-02 09:55:51,941 DEBUG creating statefile: '/var/log/dist-upgrade/apt-clone_system_state.tar.gz'
bdmurray@clean-bionic-amd64:/tmp/u-r-u$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04 LTS"

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote :

All autopkgtests for the newly accepted ubuntu-release-upgrader (1:20.04.20) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

update-manager/1:20.04.10.1 (armhf)
update-motd/3.6-0ubuntu6 (armhf, s390x, amd64, arm64, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#ubuntu-release-upgrader

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello errors.ubuntu.com, or anyone else affected,

Accepted ubuntu-release-upgrader into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:20.04.21 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed verification-needed-focal
removed: verification-done verification-done-focal
Revision history for this message
Brian Murray (brian-murray) wrote :

The verification in comment #7 is still valid as 1:20.04.21 just changed the motd file 91-release-upgrade.

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:20.04.21

---------------
ubuntu-release-upgrader (1:20.04.21) focal; urgency=medium

  * debian/91-release-upgrade: make it work in dash. (LP: #1873545)

ubuntu-release-upgrader (1:20.04.20) focal; urgency=medium

  * debian/91-release-upgrade: exit 0 if run as a non-root user.
    (LP: #1873545)
  * Only pass the package name to apport in the event that there is a dpkg
    error during a distribution upgrade. (LP: #1884148)

 -- Brian Murray <email address hidden> Mon, 06 Jul 2020 08:42:50 -0700

Changed in ubuntu-release-upgrader (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.