Merge ~alexmurray/ubuntu-cve-tracker:die-partner-die into ubuntu-cve-tracker:master

Proposed by Alex Murray
Status: Merged
Merged at revision: 058e1f911c7dd313a2f39b421ba1a5bf7c6148a7
Proposed branch: ~alexmurray/ubuntu-cve-tracker:die-partner-die
Merge into: ubuntu-cve-tracker:master
Diff against target: 1051 lines (+105/-272)
16 files modified
.launchpad.yaml (+1/-3)
README (+1/-5)
scripts/check-syntax (+1/-4)
scripts/cve-alert.sh (+1/-1)
scripts/cve_lib.py (+38/-61)
scripts/cve_packages (+3/-14)
scripts/html-report (+3/-11)
scripts/html/top/index.html (+0/-2)
scripts/html/top/priority.html (+0/-1)
scripts/html_export.py (+0/-1)
scripts/monthly-report (+0/-3)
scripts/packages-mirror (+50/-100)
scripts/report-todo-numbers (+1/-6)
scripts/report-todo-plot (+1/-1)
scripts/source_map.py (+3/-48)
scripts/ubuntu-table (+2/-11)
Reviewer Review Type Date Requested Status
Steve Beattie Approve
Marc Deslauriers Approve
Review via email: mp+444044@code.launchpad.net

Description of the change

The partner mirror hasn't been used or supported for a very long time so let's not try and pretend anymore. Remove all references to it from UCT. Hopefully this fixes CI too since it seems the partner mirror is not available on the internal archive mirror either anymore for bionic since it went into ESM.

To post a comment you must log in.
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

The only thing that sticks out is this:

+ global esm_warned

The rest LGTM. Thanks!

Revision history for this message
Alex Murray (alexmurray) wrote :

the global esm_warned was to fix pyflakes3 complaining about this variable - since I touched check-syntax pyflakes3 gets run against it in the pre-commit hook - this is needed to indicate we are using the global variable inside this function.

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

> the global esm_warned was to fix pyflakes3 complaining about this variable -
> since I touched check-syntax pyflakes3 gets run against it in the pre-commit
> hook - this is needed to indicate we are using the global variable inside this
> function.

Ah! Great :)

review: Approve
Revision history for this message
Seth Arnold (seth-arnold) wrote :

awesome :) thanks for tackling this!

Revision history for this message
Steve Beattie (sbeattie) wrote :

Thanks! I also removed the setup for partner in the ubuntu-security-tools:build-tools/build-sources-list script (though there's more cleanup that needs to be done there).

This also fixes the current lpci check-syntax breakage, so yay!

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/.launchpad.yaml b/.launchpad.yaml
index cf00cb8..b2962f9 100644
--- a/.launchpad.yaml
+++ b/.launchpad.yaml
@@ -59,9 +59,7 @@ jobs:
59 # for source_map59 # for source_map
60 echo plb_authentication=/dev/null > ~/.ubuntu-cve-tracker.conf60 echo plb_authentication=/dev/null > ~/.ubuntu-cve-tracker.conf
61 echo "packages_mirror=$HOME/mirrors/ubuntu/" >> ~/.ubuntu-cve-tracker.conf61 echo "packages_mirror=$HOME/mirrors/ubuntu/" >> ~/.ubuntu-cve-tracker.conf
62 for mirror in debian partner; do62 echo "debian_mirror=$HOME/mirrors/debian/" >> ~/.ubuntu-cve-tracker.conf
63 echo "${mirror}_mirror=$HOME/mirrors/${mirror}/" >> ~/.ubuntu-cve-tracker.conf
64 done
65 echo "Setting up packages-mirror..."63 echo "Setting up packages-mirror..."
66 # use wget for packages-mirror since rsync is not accessible in lp64 # use wget for packages-mirror since rsync is not accessible in lp
67 # builders - also invoke with -v since this seems to make downloading more65 # builders - also invoke with -v since this seems to make downloading more
diff --git a/README b/README
index fc18a69..6603d53 100644
--- a/README
+++ b/README
@@ -64,7 +64,6 @@ The following values need to be configured in ~/.ubuntu-cve-tracker.conf:
64 database (sis-generate-usn).64 database (sis-generate-usn).
65- usn_db_copy: path to copy of master USN database, used when creating65- usn_db_copy: path to copy of master USN database, used when creating
66 a template for a USN where "N" != 1 (sis-generate-usn).66 a template for a USN where "N" != 1 (sis-generate-usn).
67- partner_mirror: same as packages_mirror, but for the partner repo.
68- debian_mirror: same as packages_mirror, but for Debian testing repo.67- debian_mirror: same as packages_mirror, but for Debian testing repo.
69- mitre_loc: where to download the MITRE CVE database68- mitre_loc: where to download the MITRE CVE database
70- nvd_loc: where to download the NVD database69- nvd_loc: where to download the NVD database
@@ -441,9 +440,6 @@ Show all active CVEs for supported (main) packages:
441Number of active CVEs that are in main:440Number of active CVEs that are in main:
442./scripts/ubuntu-table --supported 2>/dev/null | grep SUPPORTED | wc -l441./scripts/ubuntu-table --supported 2>/dev/null | grep SUPPORTED | wc -l
443442
444Show all active CVEs for partner packages:
445./scripts/ubuntu-table --supported 2>/dev/null | grep PARTNER
446
447Show all out-of-sync CVEs for the devel release:443Show all out-of-sync CVEs for the devel release:
448./scripts/ubuntu-table --supported | grep 'out of sync' | awk '{if ($8 == "needed*" || $8 == "needs-triage*") { print $0 }}'444./scripts/ubuntu-table --supported | grep 'out of sync' | awk '{if ($8 == "needed*" || $8 == "needs-triage*") { print $0 }}'
449445
@@ -477,7 +473,7 @@ See all bugs with priority of medium or higher:
477See a listing of packages with number of CVEs attached to them, weighted by473See a listing of packages with number of CVEs attached to them, weighted by
478CVE priority:474CVE priority:
479./scripts/cve_packages475./scripts/cve_packages
480./scripts/cve_packages -m (just supported (main, restricted and partner))476./scripts/cve_packages -m (just supported (main and restricted))
481./scripts/cve_packages -u (just unsupported (universe and multiverse))477./scripts/cve_packages -u (just unsupported (universe and multiverse))
482./scripts/cve_packages -t (just totals)478./scripts/cve_packages -t (just totals)
483./scripts/cve_packages -a (with assignees)479./scripts/cve_packages -a (with assignees)
diff --git a/scripts/check-syntax b/scripts/check-syntax
index 92c3606..9d701e3 100755
--- a/scripts/check-syntax
+++ b/scripts/check-syntax
@@ -589,10 +589,6 @@ def check_cve(cve):
589 # Skip DNE's589 # Skip DNE's
590 if state == "DNE":590 if state == "DNE":
591 if pkg in source[rel]:591 if pkg in source[rel]:
592 # TODO: remove this when partner archive is back in sync
593 # sigh, partner archive mirrors are out of date since removing sun-java6
594 if pkg == "sun-java6" and source[rel][pkg]["section"] == "partner":
595 continue
596 print(592 print(
597 "%s: %d: package '%s' DOES exist in '%s'"593 "%s: %d: package '%s' DOES exist in '%s'"
598 % (filename, linenum, pkg, rel),594 % (filename, linenum, pkg, rel),
@@ -685,6 +681,7 @@ def check_cve(cve):
685 released = details[1]681 released = details[1]
686 # XXX 'version' existence check should not be needed682 # XXX 'version' existence check should not be needed
687 if "version" not in source[rel][pkg]:683 if "version" not in source[rel][pkg]:
684 global esm_warned
688 if not esm_warned:685 if not esm_warned:
689 print(686 print(
690 "%s: %d: unable to lookup source version for %s in %s"687 "%s: %d: unable to lookup source version for %s in %s"
diff --git a/scripts/cve-alert.sh b/scripts/cve-alert.sh
index 417bc4c..84d761e 100755
--- a/scripts/cve-alert.sh
+++ b/scripts/cve-alert.sh
@@ -9,7 +9,7 @@ set -e
9alert_threshold=109alert_threshold=10
10alert_high_threshold=010alert_high_threshold=0
11alert_community_threshold=011alert_community_threshold=0
12alert_ignore="[[:blank:]]linux:|linux-lts-|linux-hwe|linux-ti-|linux-raspi|linux-snapdragon|linux-azure|linux-aws|linux-bluefield|linux-dell|linux-gcp|linux-gke|linux-ibm|linux-intel|linux-iot|linux-lowlatency|linux-kvm|linux-oem|linux-oracle|linux-riscv|linux-xilinx-zynqmp|qtwebkit-source|qtwebkit-opensource-src|webkit:|webkitgtk|oxide-qt|PARTNER"12alert_ignore="[[:blank:]]linux:|linux-lts-|linux-hwe|linux-ti-|linux-raspi|linux-snapdragon|linux-azure|linux-aws|linux-bluefield|linux-dell|linux-gcp|linux-gke|linux-ibm|linux-intel|linux-iot|linux-lowlatency|linux-kvm|linux-oem|linux-oracle|linux-riscv|linux-xilinx-zynqmp|qtwebkit-source|qtwebkit-opensource-src|webkit:|webkitgtk|oxide-qt"
13alert_high_ignore="linux-hwe-edge:|linux-gcp-edge:|linux-azure-edge:"13alert_high_ignore="linux-hwe-edge:|linux-gcp-edge:|linux-azure-edge:"
14# important universe packages that we'd like to have up to date14# important universe packages that we'd like to have up to date
15alert_community="adobe-flashplugin|chromium-browser|docker\.io|flashplugin-nonfree"15alert_community="adobe-flashplugin|chromium-browser|docker\.io|flashplugin-nonfree"
diff --git a/scripts/cve_lib.py b/scripts/cve_lib.py
index 00d6075..4f5f7b8 100755
--- a/scripts/cve_lib.py
+++ b/scripts/cve_lib.py
@@ -277,7 +277,7 @@ subprojects = {
277 },277 },
278 "ubuntu/warty": {278 "ubuntu/warty": {
279 "eol": True,279 "eol": True,
280 "components": ["main", "restricted", "universe", "multiverse", "partner"],280 "components": ["main", "restricted", "universe", "multiverse"],
281 "name": "Ubuntu 4.10",281 "name": "Ubuntu 4.10",
282 "version": 4.10,282 "version": 4.10,
283 "codename": "Warty Warthog",283 "codename": "Warty Warthog",
@@ -287,7 +287,7 @@ subprojects = {
287 },287 },
288 "ubuntu/hoary": {288 "ubuntu/hoary": {
289 "eol": True,289 "eol": True,
290 "components": ["main", "restricted", "universe", "multiverse", "partner"],290 "components": ["main", "restricted", "universe", "multiverse"],
291 "name": "Ubuntu 5.04",291 "name": "Ubuntu 5.04",
292 "version": 5.04,292 "version": 5.04,
293 "codename": "Hoary Hedgehog",293 "codename": "Hoary Hedgehog",
@@ -297,7 +297,7 @@ subprojects = {
297 },297 },
298 "ubuntu/breezy": {298 "ubuntu/breezy": {
299 "eol": True,299 "eol": True,
300 "components": ["main", "restricted", "universe", "multiverse", "partner"],300 "components": ["main", "restricted", "universe", "multiverse"],
301 "name": "Ubuntu 5.10",301 "name": "Ubuntu 5.10",
302 "version": 5.10,302 "version": 5.10,
303 "codename": "Breezy Badger",303 "codename": "Breezy Badger",
@@ -307,7 +307,7 @@ subprojects = {
307 },307 },
308 "ubuntu/dapper": {308 "ubuntu/dapper": {
309 "eol": True,309 "eol": True,
310 "components": ["main", "restricted", "universe", "multiverse", "partner"],310 "components": ["main", "restricted", "universe", "multiverse"],
311 "name": "Ubuntu 6.06 LTS",311 "name": "Ubuntu 6.06 LTS",
312 "version": 6.06,312 "version": 6.06,
313 "codename": "Dapper Drake",313 "codename": "Dapper Drake",
@@ -317,7 +317,7 @@ subprojects = {
317 },317 },
318 "ubuntu/edgy": {318 "ubuntu/edgy": {
319 "eol": True,319 "eol": True,
320 "components": ["main", "restricted", "universe", "multiverse", "partner"],320 "components": ["main", "restricted", "universe", "multiverse"],
321 "name": "Ubuntu 6.10",321 "name": "Ubuntu 6.10",
322 "version": 6.10,322 "version": 6.10,
323 "codename": "Edgy Eft",323 "codename": "Edgy Eft",
@@ -327,7 +327,7 @@ subprojects = {
327 },327 },
328 "ubuntu/feisty": {328 "ubuntu/feisty": {
329 "eol": True,329 "eol": True,
330 "components": ["main", "restricted", "universe", "multiverse", "partner"],330 "components": ["main", "restricted", "universe", "multiverse"],
331 "name": "Ubuntu 7.04",331 "name": "Ubuntu 7.04",
332 "version": 7.04,332 "version": 7.04,
333 "codename": "Feisty Fawn",333 "codename": "Feisty Fawn",
@@ -337,7 +337,7 @@ subprojects = {
337 },337 },
338 "ubuntu/gutsy": {338 "ubuntu/gutsy": {
339 "eol": True,339 "eol": True,
340 "components": ["main", "restricted", "universe", "multiverse", "partner"],340 "components": ["main", "restricted", "universe", "multiverse"],
341 "name": "Ubuntu 7.10",341 "name": "Ubuntu 7.10",
342 "version": 7.10,342 "version": 7.10,
343 "codename": "Gutsy Gibbon",343 "codename": "Gutsy Gibbon",
@@ -347,7 +347,7 @@ subprojects = {
347 },347 },
348 "ubuntu/hardy": {348 "ubuntu/hardy": {
349 "eol": True,349 "eol": True,
350 "components": ["main", "restricted", "universe", "multiverse", "partner"],350 "components": ["main", "restricted", "universe", "multiverse"],
351 "name": "Ubuntu 8.04 LTS",351 "name": "Ubuntu 8.04 LTS",
352 "version": 8.04,352 "version": 8.04,
353 "codename": "Hardy Heron",353 "codename": "Hardy Heron",
@@ -357,7 +357,7 @@ subprojects = {
357 },357 },
358 "ubuntu/intrepid": {358 "ubuntu/intrepid": {
359 "eol": True,359 "eol": True,
360 "components": ["main", "restricted", "universe", "multiverse", "partner"],360 "components": ["main", "restricted", "universe", "multiverse"],
361 "name": "Ubuntu 8.10",361 "name": "Ubuntu 8.10",
362 "version": 8.10,362 "version": 8.10,
363 "codename": "Intrepid Ibex",363 "codename": "Intrepid Ibex",
@@ -367,7 +367,7 @@ subprojects = {
367 },367 },
368 "ubuntu/jaunty": {368 "ubuntu/jaunty": {
369 "eol": True,369 "eol": True,
370 "components": ["main", "restricted", "universe", "multiverse", "partner"],370 "components": ["main", "restricted", "universe", "multiverse"],
371 "name": "Ubuntu 9.04",371 "name": "Ubuntu 9.04",
372 "version": 9.04,372 "version": 9.04,
373 "codename": "Jaunty Jackalope",373 "codename": "Jaunty Jackalope",
@@ -377,7 +377,7 @@ subprojects = {
377 },377 },
378 "ubuntu/karmic": {378 "ubuntu/karmic": {
379 "eol": True,379 "eol": True,
380 "components": ["main", "restricted", "universe", "multiverse", "partner"],380 "components": ["main", "restricted", "universe", "multiverse"],
381 "name": "Ubuntu 9.10",381 "name": "Ubuntu 9.10",
382 "version": 9.10,382 "version": 9.10,
383 "codename": "Karmic Koala",383 "codename": "Karmic Koala",
@@ -387,7 +387,7 @@ subprojects = {
387 },387 },
388 "ubuntu/lucid": {388 "ubuntu/lucid": {
389 "eol": True,389 "eol": True,
390 "components": ["main", "restricted", "universe", "multiverse", "partner"],390 "components": ["main", "restricted", "universe", "multiverse"],
391 "name": "Ubuntu 10.04 LTS",391 "name": "Ubuntu 10.04 LTS",
392 "version": 10.04,392 "version": 10.04,
393 "codename": "Lucid Lynx",393 "codename": "Lucid Lynx",
@@ -397,7 +397,7 @@ subprojects = {
397 },397 },
398 "ubuntu/maverick": {398 "ubuntu/maverick": {
399 "eol": True,399 "eol": True,
400 "components": ["main", "restricted", "universe", "multiverse", "partner"],400 "components": ["main", "restricted", "universe", "multiverse"],
401 "name": "Ubuntu 10.10",401 "name": "Ubuntu 10.10",
402 "version": 10.10,402 "version": 10.10,
403 "codename": "Maverick Meerkat",403 "codename": "Maverick Meerkat",
@@ -407,7 +407,7 @@ subprojects = {
407 },407 },
408 "ubuntu/natty": {408 "ubuntu/natty": {
409 "eol": True,409 "eol": True,
410 "components": ["main", "restricted", "universe", "multiverse", "partner"],410 "components": ["main", "restricted", "universe", "multiverse"],
411 "name": "Ubuntu 11.04",411 "name": "Ubuntu 11.04",
412 "version": 11.04,412 "version": 11.04,
413 "codename": "Natty Narwhal",413 "codename": "Natty Narwhal",
@@ -417,7 +417,7 @@ subprojects = {
417 },417 },
418 "ubuntu/oneiric": {418 "ubuntu/oneiric": {
419 "eol": True,419 "eol": True,
420 "components": ["main", "restricted", "universe", "multiverse", "partner"],420 "components": ["main", "restricted", "universe", "multiverse"],
421 "name": "Ubuntu 11.10",421 "name": "Ubuntu 11.10",
422 "version": 11.10,422 "version": 11.10,
423 "codename": "Oneiric Ocelot",423 "codename": "Oneiric Ocelot",
@@ -427,7 +427,7 @@ subprojects = {
427 },427 },
428 "ubuntu/precise": {428 "ubuntu/precise": {
429 "eol": True,429 "eol": True,
430 "components": ["main", "restricted", "universe", "multiverse", "partner"],430 "components": ["main", "restricted", "universe", "multiverse"],
431 "name": "Ubuntu 12.04 LTS",431 "name": "Ubuntu 12.04 LTS",
432 "version": 12.04,432 "version": 12.04,
433 "codename": "Precise Pangolin",433 "codename": "Precise Pangolin",
@@ -437,7 +437,7 @@ subprojects = {
437 },437 },
438 "ubuntu/quantal": {438 "ubuntu/quantal": {
439 "eol": True,439 "eol": True,
440 "components": ["main", "restricted", "universe", "multiverse", "partner"],440 "components": ["main", "restricted", "universe", "multiverse"],
441 "name": "Ubuntu 12.10",441 "name": "Ubuntu 12.10",
442 "version": 12.10,442 "version": 12.10,
443 "codename": "Quantal Quetzal",443 "codename": "Quantal Quetzal",
@@ -447,7 +447,7 @@ subprojects = {
447 },447 },
448 "ubuntu/raring": {448 "ubuntu/raring": {
449 "eol": True,449 "eol": True,
450 "components": ["main", "restricted", "universe", "multiverse", "partner"],450 "components": ["main", "restricted", "universe", "multiverse"],
451 "name": "Ubuntu 13.04",451 "name": "Ubuntu 13.04",
452 "version": 13.04,452 "version": 13.04,
453 "codename": "Raring Ringtail",453 "codename": "Raring Ringtail",
@@ -457,7 +457,7 @@ subprojects = {
457 },457 },
458 "ubuntu/saucy": {458 "ubuntu/saucy": {
459 "eol": True,459 "eol": True,
460 "components": ["main", "restricted", "universe", "multiverse", "partner"],460 "components": ["main", "restricted", "universe", "multiverse"],
461 "name": "Ubuntu 13.10",461 "name": "Ubuntu 13.10",
462 "version": 13.10,462 "version": 13.10,
463 "codename": "Saucy Salamander",463 "codename": "Saucy Salamander",
@@ -467,7 +467,7 @@ subprojects = {
467 },467 },
468 "ubuntu/trusty": {468 "ubuntu/trusty": {
469 "eol": True,469 "eol": True,
470 "components": ["main", "restricted", "universe", "multiverse", "partner"],470 "components": ["main", "restricted", "universe", "multiverse"],
471 "name": "Ubuntu 14.04 LTS",471 "name": "Ubuntu 14.04 LTS",
472 "version": 14.04,472 "version": 14.04,
473 "codename": "Trusty Tahr",473 "codename": "Trusty Tahr",
@@ -477,7 +477,7 @@ subprojects = {
477 },477 },
478 "ubuntu/utopic": {478 "ubuntu/utopic": {
479 "eol": True,479 "eol": True,
480 "components": ["main", "restricted", "universe", "multiverse", "partner"],480 "components": ["main", "restricted", "universe", "multiverse"],
481 "name": "Ubuntu 14.10",481 "name": "Ubuntu 14.10",
482 "version": 14.10,482 "version": 14.10,
483 "codename": "Utopic Unicorn",483 "codename": "Utopic Unicorn",
@@ -487,7 +487,7 @@ subprojects = {
487 },487 },
488 "ubuntu/vivid": {488 "ubuntu/vivid": {
489 "eol": True,489 "eol": True,
490 "components": ["main", "restricted", "universe", "multiverse", "partner"],490 "components": ["main", "restricted", "universe", "multiverse"],
491 "name": "Ubuntu 15.04",491 "name": "Ubuntu 15.04",
492 "version": 15.04,492 "version": 15.04,
493 "codename": "Vivid Vervet",493 "codename": "Vivid Vervet",
@@ -497,7 +497,7 @@ subprojects = {
497 },497 },
498 "ubuntu/wily": {498 "ubuntu/wily": {
499 "eol": True,499 "eol": True,
500 "components": ["main", "restricted", "universe", "multiverse", "partner"],500 "components": ["main", "restricted", "universe", "multiverse"],
501 "name": "Ubuntu 15.10",501 "name": "Ubuntu 15.10",
502 "version": 15.10,502 "version": 15.10,
503 "codename": "Wily Werewolf",503 "codename": "Wily Werewolf",
@@ -507,7 +507,7 @@ subprojects = {
507 },507 },
508 "ubuntu/xenial": {508 "ubuntu/xenial": {
509 "eol": True,509 "eol": True,
510 "components": ["main", "restricted", "universe", "multiverse", "partner"],510 "components": ["main", "restricted", "universe", "multiverse"],
511 "name": "Ubuntu 16.04 LTS",511 "name": "Ubuntu 16.04 LTS",
512 "version": 16.04,512 "version": 16.04,
513 "codename": "Xenial Xerus",513 "codename": "Xenial Xerus",
@@ -517,7 +517,7 @@ subprojects = {
517 },517 },
518 "ubuntu/yakkety": {518 "ubuntu/yakkety": {
519 "eol": True,519 "eol": True,
520 "components": ["main", "restricted", "universe", "multiverse", "partner"],520 "components": ["main", "restricted", "universe", "multiverse"],
521 "name": "Ubuntu 16.10",521 "name": "Ubuntu 16.10",
522 "version": 16.10,522 "version": 16.10,
523 "codename": "Yakkety Yak",523 "codename": "Yakkety Yak",
@@ -527,7 +527,7 @@ subprojects = {
527 },527 },
528 "ubuntu/zesty": {528 "ubuntu/zesty": {
529 "eol": True,529 "eol": True,
530 "components": ["main", "restricted", "universe", "multiverse", "partner"],530 "components": ["main", "restricted", "universe", "multiverse"],
531 "name": "Ubuntu 17.04",531 "name": "Ubuntu 17.04",
532 "version": 17.04,532 "version": 17.04,
533 "codename": "Zesty Zapus",533 "codename": "Zesty Zapus",
@@ -537,7 +537,7 @@ subprojects = {
537 },537 },
538 "ubuntu/artful": {538 "ubuntu/artful": {
539 "eol": True,539 "eol": True,
540 "components": ["main", "restricted", "universe", "multiverse", "partner"],540 "components": ["main", "restricted", "universe", "multiverse"],
541 "name": "Ubuntu 17.10",541 "name": "Ubuntu 17.10",
542 "version": 17.10,542 "version": 17.10,
543 "codename": "Artful Aardvark",543 "codename": "Artful Aardvark",
@@ -548,7 +548,7 @@ subprojects = {
548 "ubuntu/bionic": {548 "ubuntu/bionic": {
549 "eol": True,549 "eol": True,
550 "oval": True,550 "oval": True,
551 "components": ["main", "restricted", "universe", "multiverse", "partner"],551 "components": ["main", "restricted", "universe", "multiverse"],
552 "name": "Ubuntu 18.04 LTS",552 "name": "Ubuntu 18.04 LTS",
553 "version": 18.04,553 "version": 18.04,
554 "codename": "Bionic Beaver",554 "codename": "Bionic Beaver",
@@ -558,7 +558,7 @@ subprojects = {
558 },558 },
559 "ubuntu/cosmic": {559 "ubuntu/cosmic": {
560 "eol": True,560 "eol": True,
561 "components": ["main", "restricted", "universe", "multiverse", "partner"],561 "components": ["main", "restricted", "universe", "multiverse"],
562 "name": "Ubuntu 18.10",562 "name": "Ubuntu 18.10",
563 "version": 18.10,563 "version": 18.10,
564 "codename": "Cosmic Cuttlefish",564 "codename": "Cosmic Cuttlefish",
@@ -568,7 +568,7 @@ subprojects = {
568 },568 },
569 "ubuntu/disco": {569 "ubuntu/disco": {
570 "eol": True,570 "eol": True,
571 "components": ["main", "restricted", "universe", "multiverse", "partner"],571 "components": ["main", "restricted", "universe", "multiverse"],
572 "name": "Ubuntu 19.04",572 "name": "Ubuntu 19.04",
573 "version": 19.04,573 "version": 19.04,
574 "codename": "Disco Dingo",574 "codename": "Disco Dingo",
@@ -578,7 +578,7 @@ subprojects = {
578 },578 },
579 "ubuntu/eoan": {579 "ubuntu/eoan": {
580 "eol": True,580 "eol": True,
581 "components": ["main", "restricted", "universe", "multiverse", "partner"],581 "components": ["main", "restricted", "universe", "multiverse"],
582 "name": "Ubuntu 19.10",582 "name": "Ubuntu 19.10",
583 "version": 19.10,583 "version": 19.10,
584 "codename": "Eoan Ermine",584 "codename": "Eoan Ermine",
@@ -589,7 +589,7 @@ subprojects = {
589 "ubuntu/focal": {589 "ubuntu/focal": {
590 "eol": False,590 "eol": False,
591 "oval": True,591 "oval": True,
592 "components": ["main", "restricted", "universe", "multiverse", "partner"],592 "components": ["main", "restricted", "universe", "multiverse"],
593 "name": "Ubuntu 20.04 LTS",593 "name": "Ubuntu 20.04 LTS",
594 "version": 20.04,594 "version": 20.04,
595 "codename": "Focal Fossa",595 "codename": "Focal Fossa",
@@ -599,7 +599,7 @@ subprojects = {
599 },599 },
600 "ubuntu/groovy": {600 "ubuntu/groovy": {
601 "eol": True,601 "eol": True,
602 "components": ["main", "restricted", "universe", "multiverse", "partner"],602 "components": ["main", "restricted", "universe", "multiverse"],
603 "name": "Ubuntu 20.10",603 "name": "Ubuntu 20.10",
604 "version": 20.10,604 "version": 20.10,
605 "codename": "Groovy Gorilla",605 "codename": "Groovy Gorilla",
@@ -609,7 +609,7 @@ subprojects = {
609 },609 },
610 "ubuntu/hirsute": {610 "ubuntu/hirsute": {
611 "eol": True,611 "eol": True,
612 "components": ["main", "restricted", "universe", "multiverse", "partner"],612 "components": ["main", "restricted", "universe", "multiverse"],
613 "name": "Ubuntu 21.04",613 "name": "Ubuntu 21.04",
614 "version": 21.04,614 "version": 21.04,
615 "codename": "Hirsute Hippo",615 "codename": "Hirsute Hippo",
@@ -619,7 +619,7 @@ subprojects = {
619 },619 },
620 "ubuntu/impish": {620 "ubuntu/impish": {
621 "eol": True,621 "eol": True,
622 "components": ["main", "restricted", "universe", "multiverse", "partner"],622 "components": ["main", "restricted", "universe", "multiverse"],
623 "name": "Ubuntu 21.10",623 "name": "Ubuntu 21.10",
624 "version": 21.10,624 "version": 21.10,
625 "codename": "Impish Indri",625 "codename": "Impish Indri",
@@ -630,7 +630,7 @@ subprojects = {
630 "ubuntu/jammy": {630 "ubuntu/jammy": {
631 "eol": False,631 "eol": False,
632 "oval": True,632 "oval": True,
633 "components": ["main", "restricted", "universe", "multiverse", "partner"],633 "components": ["main", "restricted", "universe", "multiverse"],
634 "name": "Ubuntu 22.04 LTS",634 "name": "Ubuntu 22.04 LTS",
635 "version": 22.04,635 "version": 22.04,
636 "codename": "Jammy Jellyfish",636 "codename": "Jammy Jellyfish",
@@ -2572,21 +2572,8 @@ def any_supported(map, pkg, releases, cvedata):
2572 return False2572 return False
25732573
25742574
2575def is_partner(map, pkg, rel):
2576 if rel in map and pkg in map[rel] and map[rel][pkg]['section'] == 'partner':
2577 return True
2578 return False
2579
2580
2581def any_partner(map, pkg, releases):
2582 for rel in releases:
2583 if is_partner(map, pkg, rel):
2584 return True
2585 return False
2586
2587
2588def is_universe(map, pkg, rel, cvedata):2575def is_universe(map, pkg, rel, cvedata):
2589 if is_supported(map, pkg, rel, cvedata) or is_partner(map, pkg, rel):2576 if is_supported(map, pkg, rel, cvedata):
2590 return False2577 return False
2591 return True2578 return True
25922579
@@ -2598,14 +2585,6 @@ def any_universe(map, pkg, releases, cvedata):
2598 return False2585 return False
25992586
26002587
2601def release_has_partner(rel):
2602 try:
2603 _,_,_,details = get_subproject_details(rel)
2604 return "partner" in details["components"]
2605 except (ValueError, KeyError):
2606 return False
2607
2608
2609def in_universe(map, pkg, rel, cve, cvedata):2588def in_universe(map, pkg, rel, cve, cvedata):
2610 if pkg in map[rel] and map[rel][pkg]['section'] == 'universe':2589 if pkg in map[rel] and map[rel][pkg]['section'] == 'universe':
2611 return True2590 return True
@@ -3008,7 +2987,7 @@ def git_delete_branch(branch, debug=False):
3008# cve_lib.check_mirror_timestamp(config)2987# cve_lib.check_mirror_timestamp(config)
3009# cve_lib.check_mirror_timestamp(config, mirror='packages_mirror')2988# cve_lib.check_mirror_timestamp(config, mirror='packages_mirror')
3010def check_mirror_timestamp(config, mirror=None):2989def check_mirror_timestamp(config, mirror=None):
3011 mirrors = ['packages_mirror', 'partner_mirror']2990 mirrors = ['packages_mirror']
3012 if mirror is not None:2991 if mirror is not None:
3013 mirrors = [mirror]2992 mirrors = [mirror]
3014 for m in mirrors:2993 for m in mirrors:
@@ -3017,8 +2996,6 @@ def check_mirror_timestamp(config, mirror=None):
3017 a = config[m]2996 a = config[m]
30182997
3019 secs = 864002998 secs = 86400
3020 if m == 'partner_mirror':
3021 secs = secs * 7
30222999
3023 if os.path.exists(a + ".timestamp") and time.mktime(time.localtime()) - os.stat(a + ".timestamp").st_mtime > secs:3000 if os.path.exists(a + ".timestamp") and time.mktime(time.localtime()) - os.stat(a + ".timestamp").st_mtime > secs:
3024 print("WARNING: '%s' is %1.1f days older than %1.1f day(s). Please run '$UCT/scripts/packages-mirror -t'." %3001 print("WARNING: '%s' is %1.1f days older than %1.1f day(s). Please run '$UCT/scripts/packages-mirror -t'." %
diff --git a/scripts/cve_packages b/scripts/cve_packages
index d301464..da038c4 100755
--- a/scripts/cve_packages
+++ b/scripts/cve_packages
@@ -86,7 +86,6 @@ deferred_packages = dict()
86popularity = dict()86popularity = dict()
87pockets = { 'supported' : dict(),87pockets = { 'supported' : dict(),
88 'universe' : dict(),88 'universe' : dict(),
89 'partner' : dict(),
90 'only_in_supported_overlay' : dict() }89 'only_in_supported_overlay' : dict() }
9190
92points = { 'negligible' : 0,91points = { 'negligible' : 0,
@@ -175,7 +174,6 @@ for cve in sorted(cves):
175 continue174 continue
176175
177 supported = False176 supported = False
178 partner = False
179 universe = False177 universe = False
180 found = False178 found = False
181 overlay = False179 overlay = False
@@ -219,18 +217,16 @@ for cve in sorted(cves):
219 found = True217 found = True
220 if cve_lib.is_supported(map, namemap[pkg][r], r, cveinfo[cve]):218 if cve_lib.is_supported(map, namemap[pkg][r], r, cveinfo[cve]):
221 supported = True219 supported = True
222 if cve_lib.is_partner(map, namemap[pkg][r], r):
223 partner = True
224 if cve_lib.is_universe(map, namemap[pkg][r], r, cveinfo[cve]):220 if cve_lib.is_universe(map, namemap[pkg][r], r, cveinfo[cve]):
225 universe = True221 universe = True
226222
227 if not found:223 if not found:
228 continue224 continue
229225
230 if opt.only_supported and not supported and not partner:226 if opt.only_supported and not supported:
231 continue227 continue
232228
233 if opt.not_supported and (supported or partner):229 if opt.not_supported and supported:
234 continue230 continue
235231
236 if supported:232 if supported:
@@ -245,11 +241,6 @@ for cve in sorted(cves):
245 else:241 else:
246 pockets['only_in_supported_overlay'][pkg] = 1242 pockets['only_in_supported_overlay'][pkg] = 1
247243
248 if partner:
249 if pkg in pockets['partner']:
250 pockets['partner'][pkg] += 1
251 else:
252 pockets['partner'][pkg] = 1
253 if universe:244 if universe:
254 if pkg in pockets['universe']:245 if pkg in pockets['universe']:
255 pockets['universe'][pkg] += 1246 pockets['universe'][pkg] += 1
@@ -409,9 +400,7 @@ else:
409 sys.stdout.write(", %s %s" % (str(priorities[p][pkg]), p))400 sys.stdout.write(", %s %s" % (str(priorities[p][pkg]), p))
410401
411 extra_info = []402 extra_info = []
412 if pkg in pockets['partner']:403 if pkg in pockets['supported']:
413 extra_info.append('PARTNER')
414 elif pkg in pockets['supported']:
415 if pkg in pockets['only_in_supported_overlay']:404 if pkg in pockets['only_in_supported_overlay']:
416 extra_info.append('SUPPORTED-ONLY-OVERLAY')405 extra_info.append('SUPPORTED-ONLY-OVERLAY')
417 else:406 else:
diff --git a/scripts/html-report b/scripts/html-report
index 63a15a5..70ea75c 100755
--- a/scripts/html-report
+++ b/scripts/html-report
@@ -80,14 +80,11 @@ header() {
80 show="$2"80 show="$2"
81 main_active=""81 main_active=""
82 universe_active=""82 universe_active=""
83 partner_active=""
8483
85 if [ "$show" = "main" ]; then84 if [ "$show" = "main" ]; then
86 main_active="active ";85 main_active="active ";
87 elif [ "$show" = "universe" ]; then86 elif [ "$show" = "universe" ]; then
88 universe_active="active ";87 universe_active="active ";
89 elif [ "$show" = "partner" ]; then
90 partner_active="active ";
91 fi88 fi
9289
93 cat > "$outfile" <<EOM90 cat > "$outfile" <<EOM
@@ -129,7 +126,6 @@ header() {
129 <li class="nav-item"><a class="nav-link" href=".">Home</a></li>126 <li class="nav-item"><a class="nav-link" href=".">Home</a></li>
130 <li class="nav-item"><a class="${main_active}nav-link" href="main.html">Main</a></li>127 <li class="nav-item"><a class="${main_active}nav-link" href="main.html">Main</a></li>
131 <li class="nav-item"><a class="${universe_active}nav-link" href="universe.html">Universe</a></li>128 <li class="nav-item"><a class="${universe_active}nav-link" href="universe.html">Universe</a></li>
132 <li class="nav-item"><a class="${partner_active}nav-link" href="partner.html">Partner</a></li>
133 </ul>129 </ul>
134 </div><!--/.nav-collapse -->130 </div><!--/.nav-collapse -->
135</nav>131</nav>
@@ -152,7 +148,6 @@ This report is divided into the following sections:</p>
152<ul class="style-none text-left">148<ul class="style-none text-left">
153 <li><a href="main${suffix}.html">Main</a> (supported by Canonical Ltd)</li>149 <li><a href="main${suffix}.html">Main</a> (supported by Canonical Ltd)</li>
154 <li><a href="universe${suffix}.html">Universe</a> (supported by the Ubuntu community)</li>150 <li><a href="universe${suffix}.html">Universe</a> (supported by the Ubuntu community)</li>
155 <li><a href="partner${suffix}.html">Partner</a> (supported by upstream vendor)</li>
156</ul>151</ul>
157152
158<p class="text-left">See <a href="priority.html">Priority Color Key</a> for information about the significance of the colors on this page.</p>153<p class="text-left">See <a href="priority.html">Priority Color Key</a> for information about the significance of the colors on this page.</p>
@@ -210,8 +205,8 @@ EOM
210}205}
211206
212OUTPUT=$($ubuntu_table_exe --html --supported --no-retired $table_args)207OUTPUT=$($ubuntu_table_exe --html --supported --no-retired $table_args)
213#for show in main universe partner index208#for show in main universe index
214for show in main universe partner209for show in main universe
215do210do
216 outfile="${tmpdir}/${show}${suffix}.html"211 outfile="${tmpdir}/${show}${suffix}.html"
217 header "$outfile" "$show"212 header "$outfile" "$show"
@@ -224,11 +219,8 @@ do
224 echo "<p class='note text-right'>* supported by Canonical Ltd</p>" >> "$outfile"219 echo "<p class='note text-right'>* supported by Canonical Ltd</p>" >> "$outfile"
225 elif [ "$show" = "universe" ]; then220 elif [ "$show" = "universe" ]; then
226 echo "<h3 class=\"text-center\">Universe</h3>" >> "$outfile"221 echo "<h3 class=\"text-center\">Universe</h3>" >> "$outfile"
227 echo "$OUTPUT" | egrep '(<table id="cves".*>|</table>|<thead.*>|</thead>|<tbody>|<tbody>|<th>|UNIVERSE)' | egrep -v 'PARTNER' | egrep -v "$embargoed" | sed -r 's#<td>[[:space:]]+(<p>[[:space:]]*[a-zA-Z0-9\-]*[[:space:]]*</p>[[:space:]]+)+</td>[[:space:]]+</tr>$#</tr>#' | sed -r 's#<th>Notes</th></tr>$#</tr>#' >> "$outfile"222 echo "$OUTPUT" | egrep '(<table id="cves".*>|</table>|<thead.*>|</thead>|<tbody>|<tbody>|<th>|UNIVERSE)' | egrep -v "$embargoed" | sed -r 's#<td>[[:space:]]+(<p>[[:space:]]*[a-zA-Z0-9\-]*[[:space:]]*</p>[[:space:]]+)+</td>[[:space:]]+</tr>$#</tr>#' | sed -r 's#<th>Notes</th></tr>$#</tr>#' >> "$outfile"
228 echo "<p class='note text-right'>* supported by Canonical Ltd</p>" >> "$outfile"223 echo "<p class='note text-right'>* supported by Canonical Ltd</p>" >> "$outfile"
229 elif [ "$show" = "partner" ]; then
230 echo "<h3 class=\"text-center\">Partner</h3>" >> "$outfile"
231 echo "$OUTPUT" | egrep '(<table id="cves".*>|</table>|<thead.*>|</thead>|<tbody>|<tbody>|<th>|PARTNER)' | egrep -v "$embargoed"| sed -r 's#<td>[[:space:]]+(<p>[[:space:]]*[a-zA-Z0-9\-]*[[:space:]]*</p>[[:space:]]+)+</td>[[:space:]]+</tr>$#</tr>#' | sed -r 's#<th>Notes</th></tr>$#</tr>#' >> "$outfile"
232 fi224 fi
233225
234 footer "$outfile"226 footer "$outfile"
diff --git a/scripts/html/top/index.html b/scripts/html/top/index.html
index 34047fd..e68dea1 100644
--- a/scripts/html/top/index.html
+++ b/scripts/html/top/index.html
@@ -53,7 +53,6 @@
53 <li class="nav-item"><a class="active nav-link" href=".">Home</a></li>53 <li class="nav-item"><a class="active nav-link" href=".">Home</a></li>
54 <li class="nav-item"><a class="nav-link" href="main.html">Main</a></li>54 <li class="nav-item"><a class="nav-link" href="main.html">Main</a></li>
55 <li class="nav-item"><a class="nav-link" href="universe.html">Universe</a></li>55 <li class="nav-item"><a class="nav-link" href="universe.html">Universe</a></li>
56 <li class="nav-item"><a class="nav-link" href="partner.html">Partner</a></li>
57 </ul>56 </ul>
58 </div><!--/.nav-collapse -->57 </div><!--/.nav-collapse -->
59</nav>58</nav>
@@ -66,7 +65,6 @@
6665
67<div>66<div>
68 <p />Open <a href="main.html">CVEs in Main</a> archive (and <a href="main-released.html">without devel release</a>)67 <p />Open <a href="main.html">CVEs in Main</a> archive (and <a href="main-released.html">without devel release</a>)
69 <p />Open <a href="partner.html">CVEs in Partner</a> archive (and <a href="partner-released.html">without devel release</a>)
70 <p />Open <a href="universe.html">CVEs in Universe</a> archive (and <a href="universe-released.html">without devel release</a>)68 <p />Open <a href="universe.html">CVEs in Universe</a> archive (and <a href="universe-released.html">without devel release</a>)
71 <p />&nbsp;69 <p />&nbsp;
72 <p />Search for a specific CVE in Ubuntu (CVE-YYYY-NNNN): <form method="GET"><input name="cve" type="text" /></form>70 <p />Search for a specific CVE in Ubuntu (CVE-YYYY-NNNN): <form method="GET"><input name="cve" type="text" /></form>
diff --git a/scripts/html/top/priority.html b/scripts/html/top/priority.html
index d339e9f..4fa80b7 100644
--- a/scripts/html/top/priority.html
+++ b/scripts/html/top/priority.html
@@ -33,7 +33,6 @@
33 <li class="nav-item"><a class="nav-link" href=".">Home</a></li>33 <li class="nav-item"><a class="nav-link" href=".">Home</a></li>
34 <li class="nav-item"><a class="nav-link" href="main.html">Main</a></li>34 <li class="nav-item"><a class="nav-link" href="main.html">Main</a></li>
35 <li class="nav-item"><a class="nav-link" href="universe.html">Universe</a></li>35 <li class="nav-item"><a class="nav-link" href="universe.html">Universe</a></li>
36 <li class="nav-item"><a class="nav-link" href="partner.html">Partner</a></li>
37 </ul>36 </ul>
38 </div><!--/.nav-collapse -->37 </div><!--/.nav-collapse -->
39</nav>38</nav>
diff --git a/scripts/html_export.py b/scripts/html_export.py
index c009a3a..1dc8e29 100755
--- a/scripts/html_export.py
+++ b/scripts/html_export.py
@@ -67,7 +67,6 @@ def html_header(title, description, outfd):
67 <li class="nav-item"><a class="nav-link" href="..">Home</a></li>67 <li class="nav-item"><a class="nav-link" href="..">Home</a></li>
68 <li class="nav-item"><a class="nav-link" href="../main.html">Main</a></li>68 <li class="nav-item"><a class="nav-link" href="../main.html">Main</a></li>
69 <li class="nav-item"><a class="nav-link" href="../universe.html">Universe</a></li>69 <li class="nav-item"><a class="nav-link" href="../universe.html">Universe</a></li>
70 <li class="nav-item"><a class="nav-link" href="../partner.html">Partner</a></li>
71 </ul>70 </ul>
72 </div><!--/.nav-collapse -->71 </div><!--/.nav-collapse -->
73</nav>72</nav>
diff --git a/scripts/monthly-report b/scripts/monthly-report
index 37757d6..939d483 100755
--- a/scripts/monthly-report
+++ b/scripts/monthly-report
@@ -19,9 +19,6 @@
19# For all the supported packages in Ubuntu, there are 67 medium-priority19# For all the supported packages in Ubuntu, there are 67 medium-priority
20# issues and 206 low issues that need to be fixed in 142 packages.20# issues and 206 low issues that need to be fixed in 142 packages.
21#21#
22# For all partner packages in Ubuntu, there is 1 medium-priority issue
23# that needs to be fixed in 1 package.
24#
25# For all community-supported packages in Ubuntu, there are 7 high-priority22# For all community-supported packages in Ubuntu, there are 7 high-priority
26# issues, 721 medium-priority, and 1005 low-priority issues that need to be23# issues, 721 medium-priority, and 1005 low-priority issues that need to be
27# fixed in 686 packages.24# fixed in 686 packages.
diff --git a/scripts/packages-mirror b/scripts/packages-mirror
index 71413a6..f053ff0 100755
--- a/scripts/packages-mirror
+++ b/scripts/packages-mirror
@@ -26,7 +26,6 @@ Usage: packages_mirror [OPTIONS]
26 -V even more verbose output26 -V even more verbose output
27 -r RELEASE only update Ubuntu release27 -r RELEASE only update Ubuntu release
28 -A pull only release pocket for the devel release28 -A pull only release pocket for the devel release
29 -p pull only partner
30 -u pull only ubuntu archive29 -u pull only ubuntu archive
31EOM30EOM
32}31}
@@ -42,7 +41,6 @@ verbosity_args="-q"
42only_release=41only_release=
43devel_quick_pockets=42devel_quick_pockets=
44very_verbose=""43very_verbose=""
45only_partner="no"
46only_ubuntu="no"44only_ubuntu="no"
4745
48while getopts "AwhftvVpur:" opt46while getopts "AwhftvVpur:" opt
@@ -56,7 +54,6 @@ do
56 very_verbose="yes";;54 very_verbose="yes";;
57 A) devel_quick_pockets="yes";;55 A) devel_quick_pockets="yes";;
58 r) only_release="$OPTARG";;56 r) only_release="$OPTARG";;
59 p) only_partner="yes";;
60 u) only_ubuntu="yes";;57 u) only_ubuntu="yes";;
61 h) help ; exit 0;;58 h) help ; exit 0;;
62 ?) help;;59 ?) help;;
@@ -73,7 +70,7 @@ debian=${debian:-ftp.debian.org}
73# shellcheck disable=SC109170# shellcheck disable=SC1091
74. "$HOME"/.ubuntu-cve-tracker.conf71. "$HOME"/.ubuntu-cve-tracker.conf
7572
76for var in packages_mirror debian_mirror partner_mirror; do73for var in packages_mirror debian_mirror; do
77 if [ -z "${!var}" ]; then74 if [ -z "${!var}" ]; then
78 echo "'$var' not defined in ~/.ubuntu-cve-tracker.conf" >&275 echo "'$var' not defined in ~/.ubuntu-cve-tracker.conf" >&2
79 exit 176 exit 1
@@ -87,23 +84,6 @@ mkdir -p "$outPath"
87debianPath=$debian_mirror84debianPath=$debian_mirror
88mkdir -p "$debianPath"85mkdir -p "$debianPath"
8986
90# shellcheck disable=SC2154
91partnerPath=$partner_mirror
92mkdir -p "$partnerPath"
93
94partner_dir_excludes=$(PYTHONPATH="${PYTHONPATH:+$PYTHONPATH:}$(dirname "$0")" python3 -c '#
95import cve_lib, sys
96s = ""
97for r in cve_lib.eol_releases + ["breezy", "warty", "hoary"]:
98 if r in ["dapper", "breezy", "warty", "hoary"]: # http://archive.canonical.com/dists/ does not have these any more
99 continue
100 # Dont exclude ESM releases
101 if r in ["trusty", "xenial"]:
102 continue
103 s += "dists/%s*/," % r
104sys.stdout.write(s.rstrip(","))
105')
106
107function compressed_ext()87function compressed_ext()
108{88{
109 rel="$1"89 rel="$1"
@@ -286,56 +266,54 @@ function do_use_timestamp() {
286 return 1266 return 1
287}267}
288268
289if [ "$only_partner" = "no" ]; then269timestamp="${outPath}.timestamp"
290 timestamp="${outPath}.timestamp"270if [ ! -e "${outPath}/dists" ] || do_use_timestamp "$timestamp" ; then
291 if [ ! -e "${outPath}/dists" ] || do_use_timestamp "$timestamp" ; then271 pull=$(mktemp -t packages-XXXXXX)
292 pull=$(mktemp -t packages-XXXXXX)272 # shellcheck disable=SC2064
293 # shellcheck disable=SC2064273 trap "rm -f $pull" EXIT HUP INT QUIT TERM
294 trap "rm -f $pull" EXIT HUP INT QUIT TERM
295274
296 # Sync Packages for non-ports supported arches275 # Sync Packages for non-ports supported arches
297 if [ "$verbosity_args" != "-q" ]; then276 if [ "$verbosity_args" != "-q" ]; then
298 echo "Generating arch lists"277 echo "Generating arch lists"
299 fi278 fi
300 # shellcheck disable=SC2129279 # shellcheck disable=SC2129
301 gen_packages trusty "amd64 i386" >> "$pull"280 gen_packages trusty "amd64 i386" >> "$pull"
302 gen_packages xenial "amd64 i386" >> "$pull"281 gen_packages xenial "amd64 i386" >> "$pull"
303 gen_packages bionic "amd64 i386" >> "$pull"282 gen_packages bionic "amd64 i386" >> "$pull"
304 gen_packages focal "amd64 i386" >> "$pull"283 gen_packages focal "amd64 i386" >> "$pull"
305 gen_packages jammy "amd64 i386" >> "$pull"284 gen_packages jammy "amd64 i386" >> "$pull"
306 gen_packages kinetic "amd64 i386" >> "$pull"285 gen_packages kinetic "amd64 i386" >> "$pull"
307 gen_packages lunar "amd64 i386" >> "$pull"286 gen_packages lunar "amd64 i386" >> "$pull"
308 gen_packages mantic "amd64 i386" >> "$pull"287 gen_packages mantic "amd64 i386" >> "$pull"
309288
310 # don't bother to pull duplicates289 # don't bother to pull duplicates
311 sort -u "$pull" > "$pull.sorted" && mv "$pull.sorted" "$pull"; rm -f "$pull.sorted"290 sort -u "$pull" > "$pull.sorted" && mv "$pull.sorted" "$pull"; rm -f "$pull.sorted"
312 pull_packages "$pull" "$server/ubuntu" || echo "FAIL: supported architectures" >&2291 pull_packages "$pull" "$server/ubuntu" || echo "FAIL: supported architectures" >&2
313 cat /dev/null > "$pull"292 cat /dev/null > "$pull"
314293
315 # Sync Packages for ports arches294 # Sync Packages for ports arches
316 if [ "$verbosity_args" != "-q" ]; then295 if [ "$verbosity_args" != "-q" ]; then
317 echo "Generating ports arch lists"296 echo "Generating ports arch lists"
318 fi297 fi
319 # shellcheck disable=SC2129298 # shellcheck disable=SC2129
320 gen_packages trusty "powerpc ppc64el armhf arm64" >> "$pull"299 gen_packages trusty "powerpc ppc64el armhf arm64" >> "$pull"
321 gen_packages xenial "powerpc ppc64el armhf arm64 s390x" >> "$pull"300 gen_packages xenial "powerpc ppc64el armhf arm64 s390x" >> "$pull"
322 gen_packages bionic "ppc64el armhf arm64 s390x" >> "$pull"301 gen_packages bionic "ppc64el armhf arm64 s390x" >> "$pull"
323 gen_packages focal "ppc64el armhf arm64 s390x riscv64" >> "$pull"302 gen_packages focal "ppc64el armhf arm64 s390x riscv64" >> "$pull"
324 gen_packages jammy "ppc64el armhf arm64 s390x riscv64" >> "$pull"303 gen_packages jammy "ppc64el armhf arm64 s390x riscv64" >> "$pull"
325 gen_packages kinetic "ppc64el armhf arm64 s390x riscv64" >> "$pull"304 gen_packages kinetic "ppc64el armhf arm64 s390x riscv64" >> "$pull"
326 gen_packages lunar "ppc64el armhf arm64 s390x riscv64" >> "$pull"305 gen_packages lunar "ppc64el armhf arm64 s390x riscv64" >> "$pull"
327 gen_packages mantic "ppc64el armhf arm64 s390x riscv64" >> "$pull"306 gen_packages mantic "ppc64el armhf arm64 s390x riscv64" >> "$pull"
328307
329 # don't bother to pull duplicates308 # don't bother to pull duplicates
330 sort -u "$pull" > "$pull.sorted" && mv "$pull.sorted" "$pull"; rm -f "$pull.sorted"309 sort -u "$pull" > "$pull.sorted" && mv "$pull.sorted" "$pull"; rm -f "$pull.sorted"
331 pull_packages "$pull" "$ports/ubuntu-ports" || echo "FAIL: ports architectures" >&2310 pull_packages "$pull" "$ports/ubuntu-ports" || echo "FAIL: ports architectures" >&2
332 cat /dev/null > "$pull"311 cat /dev/null > "$pull"
333312
334 if [ "$use_timestamp" = "yes" ]; then313 if [ "$use_timestamp" = "yes" ]; then
335 touch "$timestamp"314 touch "$timestamp"
336 else315 else
337 rm -f "$timestamp"316 rm -f "$timestamp"
338 fi
339 fi317 fi
340fi318fi
341319
@@ -348,7 +326,7 @@ if [ "$only_ubuntu" = "no" ]; then
348 fi326 fi
349327
350 # TODO: add Debian support for -r328 # TODO: add Debian support for -r
351 if [ -z "$only_release" ] && [ "$only_partner" = "no" ]; then329 if [ -z "$only_release" ]; then
352 # Sync Sources from Debian testing330 # Sync Sources from Debian testing
353 timestamp="${debianPath}.timestamp"331 timestamp="${debianPath}.timestamp"
354 if [ ! -e "${debianPath}/dists" ] || do_use_timestamp "$timestamp" 6 ; then332 if [ ! -e "${debianPath}/dists" ] || do_use_timestamp "$timestamp" 6 ; then
@@ -368,37 +346,9 @@ if [ "$only_ubuntu" = "no" ]; then
368 fi346 fi
369 fi347 fi
370 fi348 fi
371
372 if [ -z "$only_release" ] || [ "$only_partner" = "yes" ]; then
373 # Sync Canonical Partner Repo
374 timestamp="${partnerPath}.timestamp"
375 if [ ! -e "${partnerPath}/dists" ] || do_use_timestamp "$timestamp" 6 ; then
376 cd "$partnerPath"
377 misc_partner_excludes="robots.txt,misc/,project/,icons/,ls-lR.gz"
378 wget $verbosity_args -N -R '*=*' -R 'Contents*' -X "${misc_partner_excludes},${partner_dir_excludes}" -np -r http://archive.canonical.com/dists/
379
380 ln -sf archive.canonical.com/dists dists
381 # remove dangling symlink
382 rm -f archive.canonical.com/dists/dists || true
383
384 # remove dists.[0-9] files
385 rm -f archive.canonical.com/dists.[0-9]*
386
387 if [ "$use_timestamp" = "yes" ]; then
388 touch "$timestamp"
389 else
390 rm -f "$timestamp"
391 fi
392 has_sources=$(find archive.canonical.com/dists -name Sources$)
393 if [ -n "$has_sources" ]; then
394 echo ""
395 echo "WARNING: $partnerPath has Sources files! These should be removed now that -partner uses Sources.gz"
396 fi
397 fi
398 fi
399fi349fi
400350
401# generate source package lists for umt grep351# generate source package lists for umt grep
402for path in "$outPath" "$debianPath" "$partnerPath"; do352for path in "$outPath" "$debianPath"; do
403 find "$path" -name Sources.gz -exec zgrep '^Package: .*' {} \; | cut -c10- | sort -u > "$path/sources"353 find "$path" -name Sources.gz -exec zgrep '^Package: .*' {} \; | cut -c10- | sort -u > "$path/sources"
404done354done
diff --git a/scripts/report-todo-numbers b/scripts/report-todo-numbers
index 32b7d1f..502b5b1 100755
--- a/scripts/report-todo-numbers
+++ b/scripts/report-todo-numbers
@@ -45,7 +45,7 @@ only_packages = []
45if opt.only_packages:45if opt.only_packages:
46 only_packages = re.split(',', opt.only_packages)46 only_packages = re.split(',', opt.only_packages)
4747
48for category in ['supported','partner','universe']:48for category in ['supported','universe']:
49 item[category] = dict()49 item[category] = dict()
50 srcpkg_item[category] = dict()50 srcpkg_item[category] = dict()
5151
@@ -63,9 +63,6 @@ for category in ['supported','partner','universe']:
63 if 'SUPPORTED' in line:63 if 'SUPPORTED' in line:
64 if category != 'supported':64 if category != 'supported':
65 continue65 continue
66 elif 'PARTNER' in line:
67 if category != 'partner':
68 continue
69 elif category != 'universe':66 elif category != 'universe':
70 continue67 continue
7168
@@ -131,8 +128,6 @@ for category in ['supported','partner','universe']:
131 else:128 else:
132 if category == 'supported':129 if category == 'supported':
133 print " * For all the supported packages in Ubuntu, there",130 print " * For all the supported packages in Ubuntu, there",
134 elif category == 'partner':
135 print " * For all partner packages in Ubuntu, there",
136 elif category == 'universe':131 elif category == 'universe':
137 print " * For all community-supported packages in Ubuntu, there",132 print " * For all community-supported packages in Ubuntu, there",
138 if opt.show_unique_sources:133 if opt.show_unique_sources:
diff --git a/scripts/report-todo-plot b/scripts/report-todo-plot
index e5faa4a..d59014f 100755
--- a/scripts/report-todo-plot
+++ b/scripts/report-todo-plot
@@ -13,7 +13,7 @@ TABLE=$(./scripts/ubuntu-table -s "$@")
13for category in ${CATEGORY:-SUPPORTED}13for category in ${CATEGORY:-SUPPORTED}
14do14do
15 if [ "x$category" = "xuniverse" ]; then15 if [ "x$category" = "xuniverse" ]; then
16 filter="-v '(SUPPORTED|PARTNER)'"16 filter="-v 'SUPPORTED'"
17 else17 else
18 filter=$category18 filter=$category
19 fi19 fi
diff --git a/scripts/source_map.py b/scripts/source_map.py
index 4ff605e..05e258b 100755
--- a/scripts/source_map.py
+++ b/scripts/source_map.py
@@ -48,10 +48,9 @@ def read_config_file(config_file):
4848
49def _find_sources(pockets=None, releases=None, skip_eol_releases=True, arch='amd64'):49def _find_sources(pockets=None, releases=None, skip_eol_releases=True, arch='amd64'):
50 config = read_config_file(os.path.expanduser("~/.ubuntu-cve-tracker.conf"))50 config = read_config_file(os.path.expanduser("~/.ubuntu-cve-tracker.conf"))
51 if 'packages_mirror' in config and 'partner_mirror' in config:51 if 'packages_mirror' in config:
52 cve_lib.check_mirror_timestamp(config)52 cve_lib.check_mirror_timestamp(config)
53 return _find_from_mirror(config['packages_mirror'],53 return _find_from_mirror(config['packages_mirror'],
54 config['partner_mirror'],
55 'sources',54 'sources',
56 pockets=pockets,55 pockets=pockets,
57 releases=releases,56 releases=releases,
@@ -63,10 +62,9 @@ def _find_sources(pockets=None, releases=None, skip_eol_releases=True, arch='amd
6362
64def _find_packages(pockets=None, releases=None, skip_eol_releases=True, arch='amd64'):63def _find_packages(pockets=None, releases=None, skip_eol_releases=True, arch='amd64'):
65 config = read_config_file(os.path.expanduser("~/.ubuntu-cve-tracker.conf"))64 config = read_config_file(os.path.expanduser("~/.ubuntu-cve-tracker.conf"))
66 if 'packages_mirror' in config and 'partner_mirror' in config:65 if 'packages_mirror' in config:
67 cve_lib.check_mirror_timestamp(config)66 cve_lib.check_mirror_timestamp(config)
68 return _find_from_mirror(config['packages_mirror'],67 return _find_from_mirror(config['packages_mirror'],
69 config['partner_mirror'],
70 'packages',68 'packages',
71 pockets=pockets,69 pockets=pockets,
72 releases=releases,70 releases=releases,
@@ -102,7 +100,7 @@ def load_debian(basedir, data_type='sources'):
102 return debian_sources100 return debian_sources
103101
104102
105def _find_from_mirror(ubuntu, canonical, data_type, arch='amd64', pockets=None, releases=None, skip_eol_releases=True):103def _find_from_mirror(ubuntu, data_type, arch='amd64', pockets=None, releases=None, skip_eol_releases=True):
106 if data_type not in ['sources', 'packages']:104 if data_type not in ['sources', 'packages']:
107 raise ValueError("'data_type' should be either 'sources' or 'packages'")105 raise ValueError("'data_type' should be either 'sources' or 'packages'")
108106
@@ -138,9 +136,6 @@ def _find_from_mirror(ubuntu, canonical, data_type, arch='amd64', pockets=None,
138 # free136 # free
139 for pocket in pockets:137 for pocket in pockets:
140 for section in sections:138 for section in sections:
141 if section == 'partner':
142 # handled separately below
143 continue
144 if data_type == 'sources':139 if data_type == 'sources':
145 fn = os.path.join(ubuntu, 'dists', series + pocket, section,140 fn = os.path.join(ubuntu, 'dists', series + pocket, section,
146 'source', 'Sources')141 'source', 'Sources')
@@ -162,40 +157,6 @@ def _find_from_mirror(ubuntu, canonical, data_type, arch='amd64', pockets=None,
162 name = ''157 name = ''
163158
164 collection += [(fn, rel, name, section)]159 collection += [(fn, rel, name, section)]
165 # partner
166 pocket = ''
167 section = 'partner'
168
169 if data_type == 'sources':
170 fn = os.path.join(canonical, 'dists', series + pocket, section,
171 'source', 'Sources')
172 else:
173 fn = os.path.join(canonical, 'dists', series + pocket, section,
174 'binary-%s' % arch, 'Packages')
175
176 found, fn_path = _find_path_with_ext(fn)
177 if not found:
178 # Only warn about missing partner for devel release
179 # FIXME: 2020-04-29: disabled partner warning for dev for now
180 # FIXME: 2021-04-24: disabling for both devel and hirsute;
181 # looks like mirroring on people.c.c stopped?
182 if rel == cve_lib.devel_release or rel in ['hirsute', 'impish', 'jammy', 'kinetic', 'lunar']:
183 prefreeze = os.path.join(canonical, 'dists', '%s-series' % rel[0])
184 prefreeze
185 #if not os.path.isdir(prefreeze):
186 # print(sys.stderr, "WARNING: missing partner mirror element: %s" % (fn), file=sys.stderr)
187 # else:
188 # print("WARNING: found prefreeze element: %s" % (prefreeze), file=sys.stderr)
189 else:
190 missing += " %s\n" % (fn)
191 errors = True
192 continue
193 else:
194 fn = fn_path
195
196 if section not in sections:
197 continue
198 collection += [(fn, rel, pocket, section)]
199160
200 if errors:161 if errors:
201 raise NameError("Missing mirror elements:\n" + missing)162 raise NameError("Missing mirror elements:\n" + missing)
@@ -244,12 +205,6 @@ def _find_sources_from_apt(pockets=None, releases=None):
244 not (rel == cve_lib.devel_release and pocket in ['-updates', '-security']):205 not (rel == cve_lib.devel_release and pocket in ['-updates', '-security']):
245 missing += " deb-src http://archive.ubuntu.com/ubuntu %s%s %s\n" % (rel, pocket, ' '.join(cve_lib.components))206 missing += " deb-src http://archive.ubuntu.com/ubuntu %s%s %s\n" % (rel, pocket, ' '.join(cve_lib.components))
246 errors = True207 errors = True
247 for component in ['partner']:
248 # partner doesn't get the devel release treatment until late in
249 # the cycle
250 if '%s_%s' % (rel, component) not in saw and not (rel == cve_lib.devel_release) and not cve_lib.release_has_partner(rel):
251 missing += " deb-src http://archive.canonical.com/ubuntu %s %s\n" % (rel, component)
252 errors = True
253 if errors:208 if errors:
254 raise NameError("Missing /etc/apt/sources.list lines:\n%s" % (missing))209 raise NameError("Missing /etc/apt/sources.list lines:\n%s" % (missing))
255210
diff --git a/scripts/ubuntu-table b/scripts/ubuntu-table
index eeb633c..b1651c0 100755
--- a/scripts/ubuntu-table
+++ b/scripts/ubuntu-table
@@ -232,7 +232,6 @@ for cve in sorted(cves):
232 action_needed = False # A package in any release needs updating232 action_needed = False # A package in any release needs updating
233 released = False # A package in any release has been fixed233 released = False # A package in any release has been fixed
234 supported = False # A package in any release is supported and needs to be fixed234 supported = False # A package in any release is supported and needs to be fixed
235 partner = False # A package in any release is partner-supplied
236 universe = False # A package in any release is in universe and needs to be fixed235 universe = False # A package in any release is in universe and needs to be fixed
237236
238 mark = dict()237 mark = dict()
@@ -272,10 +271,6 @@ for cve in sorted(cves):
272 if pkg_rel_supported:271 if pkg_rel_supported:
273 supported = True272 supported = True
274273
275 # A partner issue?
276 if cve_lib.is_partner(map, pkg, rel):
277 partner = True
278
279 if opt.debug:274 if opt.debug:
280 print("%s: %s: %s (%s, released:%d)" % (cve, pkg, table[cve][pkg][rel], rel, released), file=sys.stderr)275 print("%s: %s: %s (%s, released:%d)" % (cve, pkg, table[cve][pkg][rel], rel, released), file=sys.stderr)
281276
@@ -317,9 +312,7 @@ for cve in sorted(cves):
317 print('<td style="font-size: small;"><a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=%s">Mitre</a> <a href="https://launchpad.net/bugs/cve/%s">LP</a> <a href="http://security-tracker.debian.org/tracker/%s">Debian</a></td>' % (quote(cve), quote(cve), quote(cve)), end=' ')312 print('<td style="font-size: small;"><a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=%s">Mitre</a> <a href="https://launchpad.net/bugs/cve/%s">LP</a> <a href="http://security-tracker.debian.org/tracker/%s">Debian</a></td>' % (quote(cve), quote(cve), quote(cve)), end=' ')
318 print('<td>', end=' ')313 print('<td>', end=' ')
319 if opt.supported:314 if opt.supported:
320 if partner:315 if supported:
321 print('<p>PARTNER</p>', end=' ')
322 elif supported:
323 print('<p>SUPPORTED</p>', end=' ')316 print('<p>SUPPORTED</p>', end=' ')
324 if universe:317 if universe:
325 print('<p>UNIVERSE</p>', end=' ')318 print('<p>UNIVERSE</p>', end=' ')
@@ -348,9 +341,7 @@ for cve in sorted(cves):
348 if action_needed and released:341 if action_needed and released:
349 print('[out of sync]', end=' ')342 print('[out of sync]', end=' ')
350 if opt.supported:343 if opt.supported:
351 if partner:344 if supported:
352 print('PARTNER', end=' ')
353 elif supported:
354 print('SUPPORTED', end=' ')345 print('SUPPORTED', end=' ')
355 if universe:346 if universe:
356 print('UNIVERSE', end=' ')347 print('UNIVERSE', end=' ')

Subscribers

People subscribed via source and target branches