Merge ~sespiros/ubuntu-cve-tracker/+git/ubuntu-cve-tracker:source-map-fix into ubuntu-cve-tracker:master

[Spyros Seimenis]

[Spyros Seimenis (sespiros)]

Something went wrong with the diff despite my rebase. This is an one line change in scripts/source_map.py

[Alex Murray (alexmurray)]

LGTM - although to future proof this, I wonder if we could implement some logic like rel < kinetic rather than not rel in [jammy, kinetic] ?

review: Approve

[Spyros Seimenis (sespiros)]

Thanks for taking a look. Maybe by using an enum for release information but that would require a larger change I think.

What about a new function in cve_lib that will check a release against cve_lib's subprojects dictionary for a partner component? (just pushed a change for that)

[Spyros Seimenis (sespiros)]

Re-requesting a review as I added some changes and couldn't re-request only from Alex who has already approved the previous versions of the patch.

[Alex Murray (alexmurray)]

LGTM - that is much nicer than my suggestion.

The only problem is I think python's assignment expression syntax was only introduced in 3.8 - but we still have to support python 3.6 on people.canonical.com:

ssh people.canonical.com python3 -c "if a := 1: print(a)"
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `python3 -c if a := 1: print(a)'

So this will need a slight refactoring unfortunately.

review: Needs Fixing

314a165... by Spyros Seimenis on 2023-05-24

fixup! scripts/source_map.py: Ignore partner archive for jammy and kinetic

[Spyros Seimenis (sespiros)]

Nice catch, refactored (and made it much simpler actually).

I renamed its argument to series as it's still confusing to me what's the correct naming of things and different functions use it differently in cve_lib.

[Alex Murray (alexmurray)]

Thanks - but now I hate the fact this function is named `release_has_parter()` but takes an argument called `series` - and then always adds a `ubuntu/` prefix which seems too presumptuous - sorry for the run-around - could we instead do something like:

def release_has_partner(rel):
    try:
      _, _, _, details = get_subproject_details(rel)
      return "partner" in details["components"]
    except (ValueError, KeyError):
      return False

415e770... by Spyros Seimenis on 2023-05-25

fixup! scripts/source_map.py: Ignore partner archive for jammy and kinetic

[Spyros Seimenis (sespiros)]

Thanks Alex, I hadn't noticed the get_subproject_details() API. I still find confusing the fact that I am passing something that is not technically a release and I call it release (and it's handled by that function by searching for an alias if it failed to find a key) but I just implemented your exact suggestion as it is seems more in-line with how code in cve_lib already works.

[Alex Murray (alexmurray)]

Thanks Spyros - this old code is such a mess, it would be great to refactor it all but for now keeping consistent is more important - cheers.

review: Approve

[Marc Deslauriers (mdeslaur)]

Can we just get rid of all the partner code completely? It's not actually used anymore...

[Spyros Seimenis (sespiros)]

+1 Alex. Marc: I thought about that but don't we need it around for older releases which still have partner?

[Marc Deslauriers (mdeslaur)]

There's nothing in the partner archive for focal+, for bionic and older, whatever is left in that archive is unmaintained and no longer updated as the commercial contracts have long expired. I don't believe ESM is committing to maintaining any of it.

[Marc Deslauriers (mdeslaur)]

(sorry, didn't mean to hijack this MP with my comment, ack on the MP)

[Spyros Seimenis (sespiros)]

Rebased, autosquashed and merged.

diff --git a/scripts/cve_lib.py b/scripts/cve_lib.py
index f625db5..d8256a4 100755
--- a/scripts/cve_lib.py
+++ b/scripts/cve_lib.py
@@ -2581,6 +2581,15 @@ def any_universe(map, pkg, releases, cvedata):
             return True
     return False
 
+
+def release_has_partner(rel):
+    try:
+        _,_,_,details = get_subproject_details(rel)
+        return "partner" in details["components"]
+    except (ValueError, KeyError):
+        return False
+
+
 def in_universe(map, pkg, rel, cve, cvedata):
     if pkg in map[rel] and map[rel][pkg]['section'] == 'universe':
         return True
diff --git a/scripts/source_map.py b/scripts/source_map.py
index 588a0f9..4ff605e 100755
--- a/scripts/source_map.py
+++ b/scripts/source_map.py
@@ -247,7 +247,7 @@ def _find_sources_from_apt(pockets=None, releases=None):
         for component in ['partner']:
             # partner doesn't get the devel release treatment until late in
             # the cycle
-            if '%s_%s' % (rel, component) not in saw and not (rel == cve_lib.devel_release):
+            if '%s_%s' % (rel, component) not in saw and not (rel == cve_lib.devel_release) and not cve_lib.release_has_partner(rel):
                 missing += " deb-src http://archive.canonical.com/ubuntu %s %s\n" % (rel, component)
                 errors = True
     if errors: