Ubuntu
ubuntu-release-upgrader package

DistUpgradeController.py key 'devRelease' not set correctly

Bug #1882069 reported by Marcel Sachtleben
260
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-release-upgrader (Ubuntu)
Fix Released
High
Brian Murray
Focal
Fix Released
High
Brian Murray
Ubuntu ubuntu-20.04.1

Bug Description

Test Case
---------
1) run do-release-upgrade
2) cancel the upgrade
3) cd /tmp/ubuntu-release-upgrader-$TMPDIR
4) edit DistUpgrade/DistUpgradeCache.py line 647 or so (immediately after self._verifyChanges()) and add 'raise SystemError' this will simulate a failure to calculate the upgrade. [It's easier than installing packages which will cause the upgrade to fail.]
5) run 'sudo ./focal --frontend DistUpgradeViewGtk'

Observe a traceback which ends with "configparser.NoOptionError: No option 'devrelease' in section: 'Options'" Because no options are passed to the release upgrader devRelease is not set which causes the traceback.

With the version of the release upgrader from -proposed you will still need to edit DistUpgradeCache.py but will not encounter a Traceback.

Original Description
--------------------
I encountered this bug when execution do-release-upgrade on an Ubuntu 19.10 machine.
The release upgrade crashed ungracefully not calling abort() which should rollback all changes up to that point.
So I ended up with kind of a broken packages and configuration hell on my server machine.
I finally was able to perform the release upgrade by using apt / dpkg manually and resolving package dependencies and conflicts by hand.

The bug is on actually two places:

DistUpgradeController.py (parameter devRelease not set at all due to wrong indendation of else block)

DistUpgradeCache.py (using unsafe code within except block: accessing undefined key 'devRelease' and provoke KeyError)

In general: the idea of except or catch or whatever is not just execute different code logic in case of exception. It is about doing the most basic stuff in case anything goes wrong (which should be as exception (error) safe as possible)

Bad practice:

try
  doSomething();
catch exception
  ignoreException();
  doSomethingElse();

Good practice:

try
  doSomethingUnsafe();
catch exception e
  logAndHandleException(e);
  useAnotherTryIfAdditionalLogicIsRequired();

That kind of bad structure costed me 1 day of error analysis (I am not a python guy) and another 1 day to revert the things from the failed upgrade.

The bugs:

Since I can not find any source code git repository, the next lines target the python module which can be found in /usr/lib/python3/dist-packages/DistUpgrade (Ubuntu 19.10 / Ubuntu 20.04 LTS)
(line numbers may differ)

Major Bug:

DistUpgradeController.py:138-139 (wrong indentation of else block)

Current:
        if self.options:
            if self.options.devel_release:
                self.config.set("Options","devRelease", "True")
        else:
            self.config.set("Options","devRelease", "False")

Should be:

        if self.options:
            if self.options.devel_release:
                self.config.set("Options","devRelease", "True")
            else:
                self.config.set("Options","devRelease", "False")

Minor Bug:

DistUpgradeCache.py:651-694 (except block)

Error arised in lines 667-668

            elif self.config.get("Options", "foreignPkgs") == "False" and \
                self.config.get("Options", "devRelease") == "True":

cause "devRelease" was not set on "Options" (see Major bug)
except block should not excecute unsafe code cause its job is error handling and roll back.
please make the entire block more fail safe (also use additional "try" if neccessary)
Since it seems like that accessing keys in python is not safe (reminds me of javascript)
it should be always checked if the key does even exist before accessing it.

Thanks for taking care of that.

Edit: I found the attachment feature after writing this report so please find the DistUpgradeController.py attached containing the major bug.

Please keep me up to date
<email address hidden>

Best Regards

See original description

Related branches

lp:ubuntu-release-upgrader
Revision history for this message
Marcel Sachtleben (marcisa) wrote :
information type: Private Security → Public Security
Revision history for this message
Marcel Sachtleben (marcisa) wrote :
Download full text (10.8 KiB)

Here is the content of /var/log/dist-upgrade/main.log (file path may differ):

root@snoobeam:/# cat /var/log/dist-upgrade/main.log
2020-05-28 21:03:34,995 INFO Using config files '['./DistUpgrade.cfg']'
2020-05-28 21:03:34,995 INFO uname information: 'Linux snoobeam.com 5.4.0-33-generic #37-Ubuntu SMP Thu May 21 12:53:59 UTC 2020 x86_64'
2020-05-28 21:03:35,113 INFO apt version: '1.9.4ubuntu0.1'
2020-05-28 21:03:35,113 INFO python version: '3.7.5 (default, Apr 19 2020, 20:18:17)
[GCC 9.2.1 20191008]'
2020-05-28 21:03:35,114 INFO release-upgrader version '20.04.19' started
2020-05-28 21:03:35,129 INFO locale: 'en_US' 'UTF-8'
2020-05-28 21:03:35,202 INFO screen could not be run
2020-05-28 21:03:35,271 DEBUG Using 'DistUpgradeViewText' view
2020-05-28 21:03:35,337 DEBUG enable dpkg --force-overwrite
2020-05-28 21:03:35,436 DEBUG creating statefile: '/var/log/dist-upgrade/apt-clone_system_state.tar.gz'
2020-05-28 21:03:36,399 DEBUG lsb-release: 'eoan'
2020-05-28 21:03:36,400 DEBUG _pythonSymlinkCheck run
2020-05-28 21:03:36,402 DEBUG openCache()
2020-05-28 21:03:36,403 DEBUG quirks: running PreCacheOpen
2020-05-28 21:03:36,403 DEBUG running Quirks.PreCacheOpen
2020-05-28 21:03:36,671 DEBUG /openCache(), new cache size 10843
2020-05-28 21:03:36,672 DEBUG need_server_mode(): can not find a desktop meta package or key deps, running in server mode
2020-05-28 21:03:36,672 DEBUG checkViewDepends()
2020-05-28 21:03:36,678 DEBUG running doUpdate() (showErrors=False)
2020-05-28 21:03:37,783 DEBUG openCache()
2020-05-28 21:03:38,020 DEBUG /openCache(), new cache size 10843
2020-05-28 21:03:38,020 DEBUG doPostInitialUpdate
2020-05-28 21:03:38,020 DEBUG quirks: running focalPostInitialUpdate
2020-05-28 21:03:38,021 DEBUG running Quirks.focalPostInitialUpdate
2020-05-28 21:03:38,301 DEBUG MetaPkgs:
2020-05-28 21:03:38,692 DEBUG no PkgRecord found for 'fonts-urw-base35', skipping
2020-05-28 21:03:38,998 DEBUG no PkgRecord found for 'mariadb-client-10.1', skipping
2020-05-28 21:03:38,998 DEBUG no PkgRecord found for 'mariadb-server-10.1', skipping
2020-05-28 21:03:39,016 DEBUG no PkgRecord found for 'pgadmin4-apache2', skipping
2020-05-28 21:03:39,016 DEBUG no PkgRecord found for 'pgdg-keyring', skipping
2020-05-28 21:03:39,022 DEBUG no PkgRecord found for 'postgresql-12', skipping
2020-05-28 21:03:39,102 DEBUG Foreign:
2020-05-28 21:03:39,102 DEBUG Obsolete: apt-transport-https aptitude aptitude-common binfmt-support ca-certificates-mono certbot cli-common command-not-found-data cpp-7 cpp-8 daemon denyhosts dovecot-ldap dovecot-lmtpd dovecot-managesieved dovecot-mysql dovecot-sieve fail2ban fonts-dejavu fonts-glyphicons-halflings g++-7 galera-3 gcc-10-base gcc-7 gcc-7-base gcc-8 gcc-8-base gcc-8-multilib gir1.2-spiceclientglib-2.0 inotify-tools jenkins ldap-account-manager ldap-auth-client ldap-auth-config lib32gcc-8-dev lib32mpx2 libaopalliance-java libapache-pom-java libapache2-mod-php7.2 libapache2-mod-rpaf libapt-inst2.0 libapt-pkg5.0 libasan4 libatinject-jsr330-api-java libcdi-api-java libcilkrts5 libcommons-cli-java libcommons-io-java libcommons-lang3-java libcommons-parent-java libcrypto++-dev libcrypto++6 libcwidget3v5 libdbd-mysql-perl libdb...

Brian Murray (brian-murray)
Changed in ubuntu-release-upgrader (Ubuntu):
status: New → Fix Committed
importance: Undecided → High
assignee: nobody → Brian Murray (brian-murray)
Brian Murray (brian-murray)
Changed in ubuntu-release-upgrader (Ubuntu Focal):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Brian Murray (brian-murray)
Brian Murray (brian-murray)
description: updated
Brian Murray (brian-murray)
Changed in ubuntu-release-upgrader (Ubuntu Focal):
milestone: none → ubuntu-20.04.1
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:20.10.5

---------------
ubuntu-release-upgrader (1:20.10.5) groovy; urgency=medium

  * DistUpgrade/DistUpgradeController.py: set a default value for devRelease
    all the time. (LP: #1882069)
  * DistUpgrade/DistUpgradeQuirks.py: Update the quirk for handling the
    transition from python-minimal to python2-minimal so that it runs during a
    prepare stage and an install stage. Thanks to Lukas Märdian for the fix.
    (LP: #1875523)

 -- Brian Murray <email address hidden> Fri, 17 Jul 2020 11:49:10 -0700

Changed in ubuntu-release-upgrader (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Marcel, or anyone else affected,

Accepted ubuntu-release-upgrader into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:20.04.22 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-release-upgrader (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Marcel, or anyone else affected,

Accepted ubuntu-release-upgrader into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:20.04.23 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (ubuntu-release-upgrader/1:20.04.23)

All autopkgtests for the newly accepted ubuntu-release-upgrader (1:20.04.23) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-release-upgrader/1:20.04.23 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#ubuntu-release-upgrader

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Brian Murray (brian-murray) wrote :

I ran through the test case and with the version of the dist-upgrader from -proposed (release-upgrader version '20.04.23' started) I did not receive a traceback. Additionally, the upgrade failed to calculate which is expected as that's where the SystemError was raised.

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:20.04.23

---------------
ubuntu-release-upgrader (1:20.04.23) focal; urgency=medium

  * data/DistUpgrade.cfg: remove xscreensaver from the PostUpgradeRemove rule
    for ubuntu-desktop as it is no longer necessary. (LP: #1875107)

ubuntu-release-upgrader (1:20.04.22) focal; urgency=medium

  * DistUpgrade/DistUpgradeController.py: set a default value for devRelease
    all the time. (LP: #1882069)
  * DistUpgrade/DistUpgradeQuirks.py: Update the quirk for handling the
    transition from python-minimal to python2-minimal so that it runs during a
    prepare stage and an install stage. Thanks to Lukas Märdian for the fix.
    (LP: #1875523)

 -- Brian Murray <email address hidden> Mon, 20 Jul 2020 17:23:48 -0700

Changed in ubuntu-release-upgrader (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.