Merge ~xnox/germinate/+git/builtusing2:master into germinate:master

[Dimitri John Ledkov]

built-using in components mismatches try #2.

http://people.canonical.com/~xnox/germinate-output/builtusing2/components-missmatches.html

No changes to .svg

[Dimitri John Ledkov (xnox)]

Shall I upload this to the archive? But it would need deployment on snake fruit too...

[Dimitri John Ledkov (xnox)]

Note the:
golang-go-xdg: golang-go-xdg-dev
[Reverse-Depends: Rescued from golang-go-xdg (Reverse Built-Using ubuntu-push-client,golang-ubuntu-push-dev)]

in the new html.

Currently it says for this package:
golang-go-xdg: golang-go-xdg-dev
[Reverse-Depends: Rescued from golang-go-xdg]

No other changes to any other logic, just the comment that is pushed out to the .html output has more useful information.

[Colin Watson (cjwatson)]

This looks broadly OK, but I've commented on a few things I'd like to see fixed before we land this.

review: Needs Fixing

[Dimitri John Ledkov (xnox)]

Thank you for this review and comments.

Thinking about this now, I think the most user-friendly result here would be to do the following:

 - inspect built-using
 - if built-using binaries, are in build-deps
 - turn follow-build-depends back on for that build-dep

Plus, all the things you mentioned in your review comments including go and non-go test cases.

[Adam Conrad (adconrad)]

> - if built-using binaries, are in build-deps

Built-Using contains sources, not binaries. We don't require build-deps to be in main, and I don't see why binaries of Built-Using should be any more special. We should be concerned with promoting sources here, not binaries.

ie: If I build foo-main against libbaruniverse-static, Canonical is committing to support the baruniverse SOURCE that produced libbaruniverse (to make sure we get it updates with security fixes, etc), and we have to ship it alongside foo for GPL compliance reasons, but that doesn't mean we're saying we support users installing the binary package libuniverse-static.

Trying to guess at binaries seems like a pointless exercise, really, and it's entirely possible to promote a source but none of its binaries.

[Dimitri John Ledkov (xnox)]

ok. This makes more sense.

On Fri, 18 Oct 2019 at 17:09, Adam Conrad <[e-mail]> wrote:
>
> > - if built-using binaries, are in build-deps
>
> Built-Using contains sources, not binaries. We don't require build-deps to be in main, and I don't see why binaries of Built-Using should be any more special. We should be concerned with promoting sources here, not binaries.
>
> ie: If I build foo-main against libbaruniverse-static, Canonical is committing to support the baruniverse SOURCE that produced libbaruniverse (to make sure we get it updates with security fixes, etc), and we have to ship it alongside foo for GPL compliance reasons, but that doesn't mean we're saying we support users installing the binary package libuniverse-static.
>
> Trying to guess at binaries seems like a pointless exercise, really, and it's entirely possible to promote a source but none of its binaries.
> --
> https://code.launchpad.net/~xnox/germinate/+git/builtusing2/+merge/308743
> You are the owner of ~xnox/germinate/+git/builtusing2:master.

--
Regards,

Dimitri.

Unmerged commits

3a6300c... by Dimitri John Ledkov on 2016-10-15

Add built-using explanation to the Rescue excuse.

diff --git a/debian/changelog b/debian/changelog
index ab3387c..8f0b978 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,10 +1,14 @@
 germinate (2.26) UNRELEASED; urgency=medium
 
+  [ Colin Watson ]
   * Add an --always-follow-build-depends option to force germinate to follow
     Build-Depends regardless of seed feature flags.
   * Change default distribution to yakkety.
 
- -- Colin Watson <[e-mail]>  Wed, 20 Apr 2016 22:41:49 +0100
+  [ Dimitri John Ledkov ]
+  * Add built-using explanation to the Rescue excuse.
+
+ -- Dimitri John Ledkov <[e-mail]>  Mon, 17 Oct 2016 13:37:14 +0200
 
 germinate (2.25) unstable; urgency=medium
 
diff --git a/germinate/germinator.py b/germinate/germinator.py
index a86c25d..8d50608 100644
--- a/germinate/germinator.py
+++ b/germinate/germinator.py
@@ -86,7 +86,6 @@ class RecommendsReason(object):
     def __str__(self):
         return '%s (Recommends)' % self._pkg
 
-
 class DependsReason(object):
     def __init__(self, pkg):
         self._pkg = pkg
@@ -110,6 +109,14 @@ class RescueReason(object):
     def __str__(self):
         return 'Rescued from %s' % self._src
 
+class RescueBuiltUsingReason(object):
+    def __init__(self, src, reverse_bu):
+        self._src = src
+        self._reverse_bu = reverse_bu
+
+    def __str__(self):
+        return 'Rescued from %s (Reverse Built-Using %s)' % (
+            self._src, self._reverse_bu)
 
 class SeedKernelVersions(object):
     """A virtual seed entry representing lexically scoped Kernel-Version."""
@@ -579,6 +586,7 @@ class Germinator(object):
 
         self._sources[src] = {}
 
+        self._sources[src]["Reverse-Built-Using"] = set()
         self._sources[src]["Maintainer"] = \
             _ensure_unicode(section.get("Maintainer", ""))
         self._sources[src]["Version"] = ver
@@ -1480,8 +1488,7 @@ class Germinator(object):
                             _logger.error("Nothing to choose to satisfy %s",
                                           pkg)
 
-    def _remember_why(self, reasons, pkg, why, build_tree=False,
-                      recommends=False):
+    def _remember_why(self, reasons, pkg, why, build_tree=False, recommends=False):
         """Remember why this package was added to the output for this seed."""
         if pkg in reasons:
             old_why, old_build_tree, old_recommends = reasons[pkg]
@@ -1596,6 +1603,8 @@ class Germinator(object):
         for pkg_src in pkg_srcs:
             if build_tree or pkg_src in built_using:
                 seed._build_sourcepkgs.add(pkg_src)
+                if pkg_src in built_using:
+                    self._sources[pkg_src]["Reverse-Built-Using"].add(pkg)
                 if pkg_src in output._blacklist:
                     seed._blacklisted.add(pkg_src)
             else:
@@ -1667,8 +1676,13 @@ class Germinator(object):
                     seed._build_depends.add(pkg)
                 else:
                     seed._depends.add(pkg)
-                self._add_package(seed, pkg, RescueReason(src),
-                                  build_tree=build_tree)
+                # add pkg, which came from src
+                # it is rescued
+                reason = RescueReason(src)
+                if len(self._sources[src]["Reverse-Built-Using"]):
+                    reverse_bu = ', '.join(self._sources[src]["Reverse-Built-Using"])
+                    reason = RescueBuiltUsingReason(src, reverse_bu)
+                self._add_package(seed, pkg, reason, build_tree=build_tree)
 
     # Accessors.
     # ----------
diff --git a/germinate/tests/test_germinator.py b/germinate/tests/test_germinator.py
index 79956dc..306c9e8 100644
--- a/germinate/tests/test_germinator.py
+++ b/germinate/tests/test_germinator.py
@@ -161,6 +161,7 @@ class TestGerminator(TestCase):
             "Build-Depends-Indep": [],
             "Build-Depends-Arch": [],
             "Binaries": ["hello", "hello-dependency"],
+            "Reverse-Built-Using": set(),
             }, germinator._sources["hello"])
         self.assertIn("hello", germinator._packages)
         self.assertEqual({