Merge ~chad.smith/ubuntu/+source/ubuntu-release-upgrader:uru-focal-ubuntu-advantage into ubuntu/+source/ubuntu-release-upgrader:ubuntu/focal-proposed

[Chad Smith]

Kept released version at 1:20.04.28 since 27 was released to -proposed

SRU of ESM upgrade support locks, official public PPA mirror

6 cherry picks:

git cherry-pick 6b6f79583687e56305a21f88d895925231e553b9 # ubuntu/master
git cherry-pick 3c23ab918585a9a0e72e60564ba734789f939813 # ubuntu/master
git cherry-pick 1fc540b9f5432539133db61a16af4b495be63a89 # ubuntu/master
git cherry-pick af4edfce71f3a98946fb06f823968e4aa4e5a4d7 # ubuntu/master
git cherry-pick cc15ee5e15a0b1ec20aec50d6b8dd7c03c3b90c7 # ubuntu/focal
git cherry-pick d36e5c96d514a7bb04e179b0b7ec251d2f802f27 # ubuntu/focal

Add changlog entry for 1:20.04.28

  [ Chad Smith ]
  * data/mirrors.cfg: add ubuntu advantage pro PPA url as valid mirror
    (LP: #1893717)
  * DistUpgrade/DistUpgradeController.py: release cache lock during
    runPostInstallScripts (LP: #1897778)

  [ Brian Murray ]
  * DistUpgrade/DistUpgradeQuirks.py: In addition to quirking python-minimal
    we also need to quirk python-dbg, python-doc, and python-dev.
    (LP: #1887544)

6e4e398... by Chad Smith on 2020-10-07

changelog

[Brian Murray (brian-murray)]

Because 1:20.04.27 did exist in the archive, although it was only in -proposed, it is possible that it is installed on user's systems. Subsequently, we need to use a version greater than it so those users will get the fixes we are making. (I realize that is less true because we are changing the dist-upgrader tarball but still we should use best practices here.)

review: Needs Fixing

[Brian Murray (brian-murray)]

This was merged and uploaded to the unapproved SRU queue.

review: Approve

diff --git a/DistUpgrade/DistUpgradeController.py b/DistUpgrade/DistUpgradeController.py
index 9f4e76b..18e1ac3 100644
--- a/DistUpgrade/DistUpgradeController.py
+++ b/DistUpgrade/DistUpgradeController.py
@@ -1518,7 +1518,13 @@ class DistUpgradeController(object):
         """ 
         scripts that are run in any case after the distupgrade finished 
         whether or not it was successful
+
+        Cache lock is released during script runs in the event that the
+        PostInstallScripts require apt or dpkg changes.
         """
+        if self.cache:
+            self.cache.release_lock()
+            self.cache.unlock_lists_dir()
         # now run the post-upgrade fixup scripts (if any)
         for script in self.config.getlist("Distro","PostInstallScripts"):
             if not os.path.exists(script):
@@ -1531,6 +1537,8 @@ class DistUpgradeController(object):
                 self._view.getTerminal().call([script], hidden=True)
             except Exception as e:
                 logging.error("got error from PostInstallScript %s (%s)" % (script, e))
+        if self.cache:
+            self.cache.get_lock()
 
     def abort(self):
         """ abort the upgrade, cleanup (as much as possible) """
diff --git a/DistUpgrade/DistUpgradeQuirks.py b/DistUpgrade/DistUpgradeQuirks.py
index b3fbc6c..4c3381f 100644
--- a/DistUpgrade/DistUpgradeQuirks.py
+++ b/DistUpgrade/DistUpgradeQuirks.py
@@ -898,22 +898,32 @@ class DistUpgradeQuirks(object):
         other package and the python-is-python2 package is installed instead,
         if python-minimal was installed.
         """
-        old = 'python-minimal'
-        new = 'python-is-python2'
+        # python-dbg must come first for reasons unknown
+        replacements = (('python-dbg', 'python2-dbg'),
+                        ('python-doc', 'python2-doc'),
+                        ('python-minimal', 'python-is-python2'),
+                        ('python-dev', 'python-dev-is-python2'),
+                        ('libpython-dev', None),
+                        ('libpython-stdlib', None),
+                        ('libpython-dbg', None))
         cache = self.controller.cache
-        if old in cache and cache[old].is_installed:
-            logging.info("installing %s because %s was installed" % (new, old))
-            reason = "%s was installed on the system" % old
-            if not cache.mark_install(new, reason):
-                logging.info("failed to install %s" % new)
-            logging.info("removing %s because %s is being installed" %
-                         (old, new))
-            reason = "%s is being installed on the system" % new
-            if not cache.mark_remove(old, reason):
-                logging.info("failed to remove %s", old)
-
-            # protect our decision to remove legacy 'python' (as a dependency
-            # of python-minimal, removed above)
+        for old, new in replacements:
+            logging.info("checking for %s" % old)
+            if old in cache and cache[old].is_installed:
+                if new:
+                    logging.info("installing %s because %s was installed" %
+                                 (new, old))
+                    reason = "%s was installed on the system" % old
+                    if not cache.mark_install(new, reason):
+                        logging.info("failed to install %s" % new)
+                logging.info("removing %s because %s is being installed" %
+                             (old, new))
+                reason = "%s is being installed on the system" % new
+                if not cache.mark_remove(old, reason):
+                    logging.info("failed to remove %s", old)
+
+            # protect our decision to remove legacy 'python' (as a
+            # dependency of python-minimal, removed above)
             py = 'python'
             if py in cache and cache[py].marked_delete:
                 resolver = apt.cache.ProblemResolver(cache)
diff --git a/data/DistUpgrade.cfg.bionic b/data/DistUpgrade.cfg.bionic
index aa45625..452c721 100644
--- a/data/DistUpgrade.cfg.bionic
+++ b/data/DistUpgrade.cfg.bionic
@@ -48,7 +48,7 @@ EnableApport=yes
 KeyDependencies=lightdm, unity, ubuntu-artwork, ubuntu-sounds
 # those pkgs will be marked remove right after the distUpgrade in the cache
 PostUpgradeRemove=gnome-cups-manager, powermanagement-interface, deskbar-applet, nautilus-cd-burner
-ForcedObsoletes=desktop-effects, gnome-app-install, policykit-gnome, gnome-mount, gnome-software-plugin-snap
+ForcedObsoletes=desktop-effects, gnome-app-install, policykit-gnome, gnome-mount
 
 [kubuntu-desktop]
 KeyDependencies=plasma-desktop, kubuntu-settings-desktop
diff --git a/data/mirrors.cfg b/data/mirrors.cfg
index aa935f7..6f51fd8 100644
--- a/data/mirrors.cfg
+++ b/data/mirrors.cfg
@@ -17,6 +17,8 @@ ftp://ddebs.ubuntu.com/
 #commercial (both urls are valid)
 http://archive.canonical.com
 http://archive.canonical.com/ubuntu/
+
+http://ppa.launchpad.net/ua-client/stable/ubuntu
 https://esm.ubuntu.com/ubuntu/
 https://esm.ubuntu.com/apps/ubuntu/
 https://esm.ubuntu.com/cc/ubuntu/
diff --git a/debian/changelog b/debian/changelog
index 7375b3c..ff8cc04 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,9 +1,25 @@
+ubuntu-release-upgrader (1:20.04.28) focal; urgency=medium
+
+  [ Brian Murray ]
+  * DistUpgrade/DistUpgradeQuirks.py: In addition to quirking python-minimal
+    we also need to quirk python-dbg, python-doc, and python-dev.
+    (LP: #1887544)
+
+  [ Chad Smith ]
+  * data/mirrors.cfg: add all ubuntu-advantage services as valid mirrors.
+    This includes: fips, fips-updates, esm-infra, esm-apps and cc-eal and
+    Ubuntu Pro stable public PPA. (LP: #1893717)
+  * DistUpgrade/DistUpgradeController.py: release cache lock during
+    runPostInstallScripts (LP: #1897778)
+
+ -- Chad Smith <[e-mail]>  Wed, 07 Oct 2020 10:49:37 -0600      
+
 ubuntu-release-upgrader (1:20.04.27) focal; urgency=medium
 
   * data/DistUpgrade.cfg.bionic: Add gnome-software-plugin-snap to
     ForcedObsoletes for ubuntu-desktop and ubuntukylin-desktop only. This
     allows multiple packages to be offered for removal after the upgrade
-    completes. (LP: #1894919) 
+    completes. (LP: #1894919)
 
  -- Brian Murray <[e-mail]>  Wed, 09 Sep 2020 11:27:27 -0700
 
diff --git a/utils/update_mirrors.py b/utils/update_mirrors.py
index f604ad6..05ceadb 100755
--- a/utils/update_mirrors.py
+++ b/utils/update_mirrors.py
@@ -15,8 +15,8 @@ d = feedparser.parse("https://launchpad.net/ubuntu/+archivemirrors-rss")
 #pp  = pprint.PrettyPrinter(indent=4)
 #pp.pprint(d)
 
-# the first 29 lines are permanent
-permanent_lines = 29
+# the first 31 lines are permanent
+permanent_lines = 31
 lp_mirrors = set()
 new_mirrors = set()