Merge ubuntu-cve-tracker:adding_special_ppas_flag into ubuntu-cve-tracker:master

[Leonidas S. Barbosa]

Adding --special-ppa flag to sis-generate-usn script in order to address some ppas that are special for us

e.g: canonical-chromium-browser/ubuntu/stage, is the one were Osmos builds chromium and while doing a template generate we need to check and be sure we fill all flags correctly, such as --pocket-all.

This patch adds this flag in order to check against a special ppa list and add the --pocket-all flags as it needs, also could be used in future features too.

[Seth Arnold (seth-arnold)]

I'd love a nice single list of all our special ppas somewhere, this might be a nice step towards that. Do we have ppas that we can't publicly disclose?

[Leonidas S. Barbosa (leosilvab)]

> I'd love a nice single list of all our special ppas somewhere, this might be a
> nice step towards that. Do we have ppas that we can't publicly disclose?
I don't know, to be honest I only know that one now because the last chromium-browser update I did. And AFAIK it is a public one. Not sure by example how Osmon handles embargoed issues there.
I also believe that these ppas will always depends on who is the one that is handling such packages updates. I'm not sure if these ppas changes hand to hands. Maybe we need to discuss these scenarios better to patch it better too.

[Steve Beattie (sbeattie)]

On Mon, Apr 04, 2022 at 06:27:01PM -0000, Leonidas S. Barbosa wrote:
> > I'd love a nice single list of all our special ppas somewhere, this might be a
> > nice step towards that. Do we have ppas that we can't publicly disclose?

We may at some point.

> I don't, to be honest I only know that one now because the last chromium-browser update I did.
> I also believe that these ppas will always depends on who is the one that is handling such packages updates. I'm not sure if these ppas changes hand to hands. Maybe we need to discuss these scenarios better to patch it better too.

Some others we have:

  browser/tbird ppa(s?)
  openjdk-security

I'm less familiar with the process around the first ones, but on the
second, we have started binary copying debs to the security-proposed ppa
to get autopkgtest coverage and then publishing from there. But if we
ever have a high priority issue or regression where we need to publish
quickly, we'd likely skip that step.

--
Steve Beattie
<email address hidden>

[Steve Beattie (sbeattie)]

On Mon, Apr 04, 2022 at 06:36:29PM -0000, Steve Beattie wrote:
> On Mon, Apr 04, 2022 at 06:27:01PM -0000, Leonidas S. Barbosa wrote:
> > > I'd love a nice single list of all our special ppas somewhere, this might be a
> > > nice step towards that. Do we have ppas that we can't publicly disclose?
>
> We may at some point.
>
> > I don't, to be honest I only know that one now because the last chromium-browser update I did.
> > I also believe that these ppas will always depends on who is the one that is handling such packages updates. I'm not sure if these ppas changes hand to hands. Maybe we need to discuss these scenarios better to patch it better too.
>
> Some others we have:
>
> browser/tbird ppa(s?)
> openjdk-security
>
> I'm less familiar with the process around the first ones, but on the
> second, we have started binary copying debs to the security-proposed ppa
> to get autopkgtest coverage and then publishing from there. But if we
> ever have a high priority issue or regression where we need to publish
> quickly, we'd likely skip that step.

Oh one more set of packages that don't go through our normal
processes: kernel updates, but for those I publish my USNs based off
the information that has landed in the Ubuntu archive; *occasionally*
I have prepared updates based on ppa contents for those, but mostly
as a pre-fetching stage.

--
Steve Beattie
<email address hidden>

[Leonidas S. Barbosa (leosilvab)]

...
> Oh one more set of packages that don't go through our normal
> processes: kernel updates, but for those I publish my USNs based off
> the information that has landed in the Ubuntu archive; *occasionally*
> I have prepared updates based on ppa contents for those, but mostly
> as a pre-fetching stage.
my main concern about these 'special' ppas are the fact that they can change during time (?) So pin them in a list makes it in two works to maintain. The alternative should be pass it as parameter, but we want it to be check against, In this point I believe we should have a kind of guidelines for these packages that are build out-side our sec ppas/normal-ppas/ so we could track it better. The copy binary to our ppas is a thing, but as Steve said, if in a rush, one wants to skip it.

[Eduardo Barretto (ebarretto)]

Discussed with Leo about this PR. Since it is from 2022 and we haven't hit this case anymore we will reject it, but in the future we might add such flag in a new PR.

review: Disapprove

Unmerged commits

64dd656... by Leonidas S. Barbosa on 2022-04-04

Adding special-ppa flag in order to handle ppas that are special for us and we want to adress, like ~canonical-chromium-browser

diff --git a/scripts/sis-generate-usn b/scripts/sis-generate-usn
index d81a2b2..fef6871 100755
--- a/scripts/sis-generate-usn
+++ b/scripts/sis-generate-usn
@@ -36,6 +36,7 @@ opter.add_option("--ignore-cves", metavar="CVES", help="Comma separated list of 
 opter.add_option("--embargoed", help="Include embargoed directory when looking for CVE descriptions", action='store_true')
 opter.add_option("--include-eol", help="Include EoL releases", action='store_true')
 opter.add_option("--binaries-json", help="Path to JSON mapping of binary packages to versions (can repeat, default: binaries.json)", action='append', default=[])
+opter.add_option("--special-ppa", help="Forces a check for special ppas we want to address in the template", action='store_true')
 (opt, args) = opter.parse_args()
 
 if len(args) < 2:
@@ -338,6 +339,14 @@ def is_origin_ppa_for_archive(origin):
     ]
     return origin in default_ppas
 
+# return True if ppa is a special one.
+# Some ppas are 'special' because they are used to build specific packages
+# such as chromium-browser one.
+def is_special_ppa(ppa):
+    special_ppas = [
+            "~canonical-chromium-builds/ubuntu/stage"
+    ]
+    return ppa in special_ppas
 
 # Determine where to expect the files to live (archive vs ports)
 def base_to_url(base, release, arch):
@@ -863,6 +872,8 @@ for release in releases:
                         pocket = "--pocket %s" % binaries[source][release][deb]['pocket']
                     elif is_origin_ppa_for_archive(binaries[source][release][deb]['origin']):
                         pocket = "--pocket security"
+                    elif opt.special_ppa and is_special_ppa(binaries[source][release][deb]['origin']):
+                        pocket = "--pocket security"
                     print('  usn.py --db "$DB" $USN --release %s --package %s --binary-version %s %s' % (release, deb, version, pocket))
         print()
 
@@ -914,6 +925,8 @@ for release in releases:
                         pocket = "--pocket-all %s" % binaries[source][release][deb]['pocket']
                     elif is_origin_ppa_for_archive(binaries[source][release][deb]['origin']):
                         pocket = "--pocket-all security"
+                    elif opt.special_ppa and is_special_ppa(binaries[source][release][deb]['origin']):
+                        pocket = "--pocket-all security"
                     print('  usn.py --db "$DB" $USN --release %s --package %s --all-binary-version %s --all-binary-source %s %s'
                           % (release, deb, version, source, pocket))
         print()