Merge ~jibel/ubuntu-release-upgrader:fix_deb2snap_lp1856862 into ubuntu-release-upgrader:ubuntu/master

[Jean-Baptiste Lallement]

Fixes for upgrade to focal:
- Release number 19.04 -> 20.04 on UI
- Do not install snaps during upgrade if corresponding deb package is not installed
- Fixed a crash when a snap is installed but not the deb.

[Jean-Baptiste Lallement (jibel)]

I tested the following scenarios in an upgrade from 19.10 to 20.04:
- No snap installed (only core and core18)
- Snap package is preinstalled (refresh)
- Snap and deb are not installed (no-op)
- Deb is install but not snap (migration from deb to snap)

[Brian Murray (brian-murray)]

Thanks for working on this!

review: Approve

[Brian Murray (brian-murray)]

I should have tested this before merging it but I believe your changes cause the following test failure.

======================================================================
ERROR: test_prepare_snap_replacement_data (test_quirks.TestSnapQuirks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/mock/mock.py", line 1330, in patched
    return func(*args, **keywargs)
  File "/home/bdmurray/source-trees/ubuntu-release-upgrader/ubuntu-release-upgrader/tests/test_quirks.py", line 354, in test_prepare_snap_replacement_data
    q._prepare_snap_replacement_data()
  File "/home/bdmurray/source-trees/ubuntu-release-upgrader/ubuntu-release-upgrader/DistUpgrade/DistUpgradeQuirks.py", line 857, in _prepare_snap_replacement_data
    if (snap not in cache or (snap in cache and
TypeError: argument of type 'Mock' is not iterable

[Jean-Baptiste Lallement (jibel)]

I tested and the tests passed. Although I didn't try with autopkgtest. I'll do that.

[Jean-Baptiste Lallement (jibel)]

$ nosetests3 test_quirks:TestSnapQuirks -v
test_calculate_snap_size_requirements (test_quirks.TestSnapQuirks) ... ok
test_get_from_and_to_version (test_quirks.TestSnapQuirks) ... ok
test_prepare_snap_replacement_data (test_quirks.TestSnapQuirks) ... ok
test_replace_debs_with_snaps (test_quirks.TestSnapQuirks) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.031s

OK

On which release do you see this failure?

[Jean-Baptiste Lallement (jibel)]

I can reproduce the error in autopkgtest.

[Jean-Baptiste Lallement (jibel)]

This branch fixes the tests:

https://code.launchpad.net/~jibel/ubuntu-release-upgrader/+git/ubuntu-release-upgrader/+merge/377046

diff --git a/DistUpgrade/DistUpgradeCache.py b/DistUpgrade/DistUpgradeCache.py
index 5597039..0689329 100644
--- a/DistUpgrade/DistUpgradeCache.py
+++ b/DistUpgrade/DistUpgradeCache.py
@@ -939,7 +939,8 @@ class MyCache(apt.Cache):
             return False
         # ensure we honor KeepInstalledSection here as well
         for section in self.config.getlist("Distro", "KeepInstalledSection"):
-            if pkgname in self and self[pkgname].installed.section == section:
+            if (pkgname in self and self[pkgname].installed and
+                    self[pkgname].installed.section == section):
                 logging.debug("skipping '%s' (in KeepInstalledSection)" % pkgname)
                 return False
         # if we don't have the package anyway, we are fine (this can
diff --git a/DistUpgrade/DistUpgradeQuirks.py b/DistUpgrade/DistUpgradeQuirks.py
index 3ae7ea7..6c832b3 100644
--- a/DistUpgrade/DistUpgradeQuirks.py
+++ b/DistUpgrade/DistUpgradeQuirks.py
@@ -841,14 +841,26 @@ class DistUpgradeQuirks(object):
             if re.search("^installed: ", snap_info[0], re.MULTILINE):
                 logging.debug("Snap %s is installed" % snap)
                 # its not tracking the release channel so don't refresh
-                if not re.search("^tracking:.*%s" % from_channel,
+                if not re.search(r"^tracking:.*%s" % from_channel,
                                  snap_info[0], re.MULTILINE):
                     logging.debug("Snap %s is not tracking the release channel"
                                   % snap)
                     continue
                 snap_object['command'] = 'refresh'
             else:
-                match = re.search("snap-id:\s*(\w*)", snap_info[0])
+                # Do not replace packages not installed
+                # core18, gnome-3-28-1804 and gtk-common-themes do not match
+                # any deb package so never marked for installation but
+                # they'll be installed by dependency when the first gnome
+                # package is installed
+                cache = self.controller.cache
+                if (snap not in cache or (snap in cache and
+                                          not cache[snap].is_installed)):
+                    logging.debug("Deb package %s is not installed. Skipping "
+                                  "snap package installation" % snap)
+                    continue
+
+                match = re.search(r"snap-id:\s*(\w*)", snap_info[0])
                 if not match:
                     logging.debug("Could not parse snap-id for the %s snap"
                                   % snap)
diff --git a/DistUpgrade/DistUpgradeViewGtk3.py b/DistUpgrade/DistUpgradeViewGtk3.py
index 5b7fe9c..56a807b 100644
--- a/DistUpgrade/DistUpgradeViewGtk3.py
+++ b/DistUpgrade/DistUpgradeViewGtk3.py
@@ -477,7 +477,7 @@ class DistUpgradeViewGtk3(DistUpgradeView,SimpleGtkbuilderApp):
             pass
 
         title_string = self.label_title.get_label()
-        title_string = title_string.replace("%s", "19.10")
+        title_string = title_string.replace("%s", "20.04")
         self.label_title.set_label(title_string)
 
         # terminal stuff
diff --git a/DistUpgrade/DistUpgradeViewKDE.py b/DistUpgrade/DistUpgradeViewKDE.py
index 7930b63..1fd8f32 100644
--- a/DistUpgrade/DistUpgradeViewKDE.py
+++ b/DistUpgrade/DistUpgradeViewKDE.py
@@ -641,7 +641,7 @@ class DistUpgradeViewKDE(DistUpgradeView):
             name = "Kubuntu"
         title_string = self.window_main.label_title.text()
         title_string = title_string.replace("Ubuntu", name)
-        title_string = title_string.replace("%s", "19.10")
+        title_string = title_string.replace("%s", "20.04")
         self.window_main.label_title.setText(title_string)
 
         # setup terminal text in hidden by default spot
diff --git a/debian/changelog b/debian/changelog
index f2e9bb0..cf12fd9 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,13 @@
+ubuntu-release-upgrader (1:20.04.8) UNRELEASED; urgency=medium
+
+  * Do not install a snap package if the corresponding debian package is not
+    installed (LP: #1856996)
+  * Fixed crash when checking a section of a package when the package is a
+    snap (LP: #1856862)
+  * Update release to 20.04 in UI (LP: #1856948)
+
+ -- Jean-Baptiste Lallement <[e-mail]>  Thu, 19 Dec 2019 15:29:54 +0100
+
 ubuntu-release-upgrader (1:20.04.7) focal; urgency=medium
 
   * debian/release-upgrade-motd: Fix typo preventing the update of the cached info.