Merge ~brian-murray/ubuntu-release-upgrader:python-dbg-fix into ubuntu-release-upgrader:ubuntu/master

[Brian Murray]

The fix for bug 1875523 does not resolve failures to calculate upgrades for people who have python-dbg installed on Ubuntu 18.04 LTS. This was discovered when investigating bug 1887544 and a minimal test case can be created by just installing python-pyqt5-dbg and then trying to upgrade to Ubuntu 20.04 LTS. By replacing python-dbg with python2-dbg the upgrade can proceed.

Here's a portion of the an upgrade I did from 18.04 LTS w/ python-pyqt5-dbg installed to Ubuntu 20.04 LTS.

2020-07-31 09:45:21,140 DEBUG quirks: running PreDistUpgradeCache
2020-07-31 09:45:21,140 DEBUG running Quirks.PreDistUpgradeCache
2020-07-31 09:45:21,140 INFO checking for python-dbg
2020-07-31 09:45:21,140 INFO installing python2-dbg because python-dbg was installed
2020-07-31 09:45:21,140 DEBUG Installing 'python2-dbg' (python-dbg was installed on the system)
2020-07-31 09:45:21,282 INFO removing python-dbg because python2-dbg is being installed
2020-07-31 09:45:21,282 DEBUG Removing 'python-dbg' (python2-dbg is being installed on the system)
2020-07-31 09:45:21,282 INFO failed to remove python-dbg
2020-07-31 09:45:21,283 INFO checking for python-minimal
2020-07-31 09:45:21,283 INFO installing python-is-python2 because python-minimal was installed
2020-07-31 09:45:21,283 DEBUG Installing 'python-is-python2' (python-minimal was installed on the system)
2020-07-31 09:45:21,300 INFO removing python-minimal because python-is-python2 is being installed
2020-07-31 09:45:21,300 DEBUG Removing 'python-minimal' (python-is-python2 is being installed on the system)
2020-07-31 09:45:21,300 INFO failed to remove python-minimal

While the log file says the packages failed to remove (which I just noticed) they did end up being removed.

2020-07-31 09:45:26,664 DEBUG Remove: fwupdate-signed gnome-settings-daemon-schemas libapt-inst2.0 libapt-pkg5.0 libcupscgi1 libcupsmime1 libcupsppdc1 libebook-1.2-19 libedata-book-1.2-25 libedata-cal-1.2-28 libfwup1 libldb1 libpolkit-backend-1-0 libpython-dbg libpython-stdlib libsensors4 libsnmp30 python python-dbg python-minimal python-pyqt5 python-pyqt5-dbg python-pyqt5.qtopengl python-pyqt5.qtopengl-dbg python-pyqt5.qtsensors python-pyqt5.qtsensors-dbg python-pyqt5.qtwebchannel uno-libs3

So I think this change is okay but I'd appreciate a second opinion.

[Dimitri John Ledkov (xnox)]

python-dev => python2-dev
libpython-dev => libpython2-dev
libpython-stdlib => libpython2-stdlib
python-doc => python2-doc
libpython-dbg => libpython2-dbg

will probably need transitions too. However, i'm not sure if we should be explicitely installing the new ones, or like allow for them to be pullsed in via dependencies.

[Lukas Märdian (slyon)]

I can confirm this resolves the minimal test case described. I've added two smaller comments inline.

Should we hard-code all those transition xnox described in the upgrader, or should src:what-is-python provide proper upgrade paths for those packages? There is python-dev-is-python2 already: https://packages.ubuntu.com/focal/python-dev-is-python2

[Brian Murray (brian-murray)]

Bug 1891381 is another failure to calculate the upgrade with python-qt4-dev installed.

[Dimitri John Ledkov (xnox)]

replacements should be extended to replate:
python-doc with python2-doc
python-dbg with python2-dbg

however, the following ones should have no replacement, and simply removed:
python-dev
libpython-dev
libpython-stdlib
libpython-dbg

diff --git a/DistUpgrade/DistUpgradeQuirks.py b/DistUpgrade/DistUpgradeQuirks.py
index c414424..144335e 100644
--- a/DistUpgrade/DistUpgradeQuirks.py
+++ b/DistUpgrade/DistUpgradeQuirks.py
@@ -848,26 +848,30 @@ class DistUpgradeQuirks(object):
         other package and the python-is-python2 package is installed instead,
         if python-minimal was installed.
         """
-        old = 'python-minimal'
-        new = 'python-is-python2'
+        # python-dbg must come first for reasons unknown
+        replacements = [('python-dbg','python2-dbg'),
+                        ('python-minimal', 'python-is-python2')]
         cache = self.controller.cache
-        if old in cache and cache[old].is_installed:
-            logging.info("installing %s because %s was installed" % (new, old))
-            reason = "%s was installed on the system" % old
-            if not cache.mark_install(new, reason):
-                logging.info("failed to install %s" % new)
-            logging.info("removing %s because %s is being installed" %
-                         (old, new))
-            reason = "%s is being installed on the system" % new
-            if not cache.mark_remove(old, reason):
-                logging.info("failed to remove %s", old)
-
-            # protect our decision to remove legacy 'python' (as a dependency
-            # of python-minimal, removed above)
-            py = 'python'
-            if py in cache and cache[py].marked_delete:
-                resolver = apt.cache.ProblemResolver(cache)
-                resolver.protect(cache[py])
+        for old, new in replacements:
+            logging.info("checking for %s" % old)
+            if old in cache and cache[old].is_installed:
+                logging.info("installing %s because %s was installed" %
+                             (new, old))
+                reason = "%s was installed on the system" % old
+                if not cache.mark_install(new, reason):
+                    logging.info("failed to install %s" % new)
+                logging.info("removing %s because %s is being installed" %
+                             (old, new))
+                reason = "%s is being installed on the system" % new
+                if not cache.mark_remove(old, reason):
+                    logging.info("failed to remove %s", old)
+
+                # protect our decision to remove legacy 'python' (as a
+                # dependency of python-minimal, removed above)
+                py = 'python'
+                if py in cache and cache[py].marked_delete:
+                    resolver = apt.cache.ProblemResolver(cache)
+                    resolver.protect(cache[py])
 
     def ensure_recommends_are_installed_on_desktops(self):
         """ ensure that on a desktop install recommends are installed
diff --git a/debian/changelog b/debian/changelog
index f548e84..3d4b40d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+ubuntu-release-upgrader (1:20.10.9) UNRELEASED; urgency=medium
+
+  * DistUpgrade/DistUpgradeQuirks.py: In addition to quirking python-minimal
+    we also need to quirk python-dbg. (LP: #1887544)
+
+ -- Brian Murray <[e-mail]>  Fri, 31 Jul 2020 12:55:41 -0700
+
 ubuntu-release-upgrader (1:20.10.8) groovy; urgency=medium
 
   * DistUpgrade/DistUpgradeCache.py: Ensure that the linux metapackage is