Merge lp:~ahasenack/landscape-client/17.03-backports-fix into lp:~landscape/landscape-client/17.03

[Andreas Hasenack]

Backport fix for #1668583 (don't report backport packages) from trunk r962.

Backport fix for #1668583 (don't report backport packages) from trunk r962.

We want 17.03 client builds to populate ppa:landscape/landscape-cloud-installer and lds-proposed.

The current LP recipe for builds out of this 17.03 branch is:

# bzr-builder format 0.3 deb-version {debupstream}-0~{revno}
lp:~landscape/landscape-client/17.03

This means that with a plain "17.03" in debian/changelog, we would get a package like this:

17.03-0~941

Which could make one think this is trunk r941 if you just look at that version without the context of which branch it was built from.

Therefore, I also changed the "debupstream" version of the package in debian/changelog so it's clear this is still client r940 from trunk *plus* the r962 fix.

In the end, the recipe will give us a build with a version like this:

17.03~bzr940-0~941

It's still not super clear where this build came from, though. We could use "17.03~release71-0~941" instead perhaps, except there is no release-71 branch for the client.

[Alberto Donato (ack)]

+1

review: Approve

[Adam Collard (adam-collard)]

review: Approve

[🤖 Landscape Builder (landscape-builder)]

The attempt to merge lp:~ahasenack/landscape-client/17.03-backports-fix into lp:~landscape/landscape-client/17.03 failed. Below is the output from the failed tests.

make: *** No rule to make target 'ci-check'. Stop.

[🤖 Landscape Builder (landscape-builder)]

No approved revision specified.

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make ci-check
Result: Fail
Revno: 944
Branch: lp:~ahasenack/landscape-client/17.03-backports-fix
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3790/

review: Needs Fixing (test results)

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 944
Branch: lp:~ahasenack/landscape-client/17.03-backports-fix
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3791/

review: Approve (test results)

=== modified file 'debian/changelog'
--- debian/changelog	2017-03-02 09:59:16 +0000
+++ debian/changelog	2017-03-27 20:35:37 +0000
@@ -1,14 +1,11 @@
-landscape-client (17.02-0ubuntu1) UNRELEASED; urgency=medium
-
-  * Bump version
-
- -- Alberto Donato <[e-mail]>  Thu, 02 Mar 2017 10:51:32 +0100
-
-landscape-client (17.01-0ubuntu1) UNRELEASED; urgency=medium
-
-  * New trunk build
-
- -- Andreas Hasenack <[e-mail]>  Fri, 05 Aug 2016 10:29:28 -0300
+landscape-client (17.03~bzr940-0ubuntu1) UNRELEASED; urgency=medium
+
+  * Client 17.03~bzr940: branched off trunk r940
+  * Don't report packages that are coming from backports, so that Landscape
+    doesn't try to upgrade to versions of packages that are in backports.
+    Backported from trunk's r962. (LP: #1668583)
+
+ -- Andreas Hasenack <[e-mail]>  Mon, 27 Mar 2017 17:17:10 -0300
 
 landscape-client (16.03-0ubuntu1) xenial; urgency=medium
 

=== modified file 'landscape/package/reporter.py'
--- landscape/package/reporter.py	2016-08-04 16:13:11 +0000
+++ landscape/package/reporter.py	2017-03-27 20:35:37 +0000
@@ -13,6 +13,7 @@
 from landscape.lib.twisted_util import gather_results, spawn_process
 from landscape.lib.fetch import fetch_async
 from landscape.lib.fs import touch_file
+from landscape.lib.lsb_release import parse_lsb_release, LSB_RELEASE_FILENAME
 from landscape.lib import bpickle
 
 from landscape.package.taskhandler import (
@@ -546,8 +547,26 @@
         current_available = set()
         current_upgrades = set()
         current_locked = set()
+        lsb = parse_lsb_release(LSB_RELEASE_FILENAME)
+        backports_archive = "{}-backports".format(lsb["code-name"])
 
         for package in self._facade.get_packages():
+            # Don't include package versions from the official backports
+            # archive. The backports archive is enabled by default since
+            # xenial with a pinning policy of 100. Ideally we would
+            # support pinning, but we don't yet. In the mean time, we
+            # ignore backports, so that packages don't get automatically
+            # upgraded to the backports version.
+            backport_origins = [
+                origin for origin in package.origins
+                if origin.archive == backports_archive]
+            if backport_origins and (
+                    len(backport_origins) == len(package.origins)):
+                # Ignore the version if it's only in the official
+                # backports archive. If it's somewhere else as well,
+                # e.g. a PPA, we assume it was added manually and the
+                # user wants to get updates from it.
+                continue
             hash = self._facade.get_package_hash(package)
             id = self._store.get_hash_id(hash)
             if id is not None:

=== modified file 'landscape/package/tests/test_reporter.py'
--- landscape/package/tests/test_reporter.py	2016-08-04 16:13:11 +0000
+++ landscape/package/tests/test_reporter.py	2017-03-27 20:35:37 +0000
@@ -3,6 +3,7 @@
 import time
 import apt_pkg
 import mock
+import shutil
 
 from twisted.internet.defer import Deferred, succeed, fail, inlineCallbacks
 from twisted.internet import reactor
@@ -10,6 +11,7 @@
 
 from landscape.lib.fs import create_file, touch_file
 from landscape.lib.fetch import FetchError
+from landscape.lib.lsb_release import parse_lsb_release, LSB_RELEASE_FILENAME
 from landscape.lib import bpickle
 from landscape.package.store import (
     PackageStore, UnknownHashIDRequest, FakePackageStore)
@@ -923,6 +925,92 @@
         result = self.reporter.detect_packages_changes()
         return result.addCallback(got_result)
 
+    def test_detect_packages_changes_with_backports(self):
+        """
+        Package versions coming from backports aren't considered to be
+        available.
+
+        This is because we don't support pinning, and the backports
+        archive is enabled by default since xenial.
+        """
+        message_store = self.broker_service.message_store
+        message_store.set_accepted_types(["packages"])
+
+        lsb = parse_lsb_release(LSB_RELEASE_FILENAME)
+        release_path = os.path.join(self.repository_dir, "Release")
+        with open(release_path, "w") as release:
+            release.write("Suite: {}-backports".format(lsb["code-name"]))
+
+        self.store.set_hash_ids({HASH1: 1, HASH2: 2, HASH3: 3})
+
+        def got_result(result):
+            self.assertMessages(message_store.get_pending_messages(), [])
+
+            self.assertEqual(sorted(self.store.get_available()), [])
+
+        result = self.reporter.detect_packages_changes()
+        return result.addCallback(got_result)
+
+    def test_detect_packages_changes_with_backports_others(self):
+        """
+        Packages coming from backport archives that aren't named like
+        the official backports archive are considered to be available.
+        """
+        message_store = self.broker_service.message_store
+        message_store.set_accepted_types(["packages"])
+
+        release_path = os.path.join(self.repository_dir, "Release")
+        with open(release_path, "w") as release:
+            release.write("Suite: my-personal-backports")
+
+        self.store.set_hash_ids({HASH1: 1, HASH2: 2, HASH3: 3})
+
+        def got_result(result):
+            self.assertMessages(message_store.get_pending_messages(),
+                                [{"type": "packages", "available": [(1, 3)]}])
+
+            self.assertEqual(sorted(self.store.get_available()), [1, 2, 3])
+
+        result = self.reporter.detect_packages_changes()
+        return result.addCallback(got_result)
+
+    def test_detect_packages_changes_with_backports_both(self):
+        """
+        If a package is both in the official backports archive and in
+        some other archive (e.g. a PPA), the package is considered to be
+        available.
+
+        The reason for this is that if you have enabled a PPA, you most
+        likely want to get updates from it.
+        """
+        message_store = self.broker_service.message_store
+        message_store.set_accepted_types(["packages"])
+
+        temp_dir = self.makeDir()
+        other_backport_dir = os.path.join(temp_dir, "my-personal-backports")
+        shutil.copytree(self.repository_dir, other_backport_dir)
+        os.remove(os.path.join(other_backport_dir, "Packages"))
+        self.facade.add_channel_deb_dir(other_backport_dir)
+
+        lsb = parse_lsb_release(LSB_RELEASE_FILENAME)
+        official_release_path = os.path.join(self.repository_dir, "Release")
+        with open(official_release_path, "w") as release:
+            release.write("Suite: {}-backports".format(lsb["code-name"]))
+        unofficial_release_path = os.path.join(other_backport_dir, "Release")
+        with open(unofficial_release_path, "w") as release:
+            release.write("Suite: my-personal-backports")
+
+        self.store.set_hash_ids({HASH1: 1, HASH2: 2, HASH3: 3})
+
+        def got_result(result):
+            self.assertMessages(message_store.get_pending_messages(),
+                                [{"type": "packages", "available": [(1, 3)]}])
+
+            self.assertEqual(sorted(self.store.get_available()), [1, 2, 3])
+
+        result = self.reporter.detect_packages_changes()
+        return result.addCallback(got_result)
+
     @inlineCallbacks
     def test_detect_packages_after_tasks(self):
         """