Merge ~sbeattie/ubuntu-cve-tracker/+git/ubuntu-cve-tracker:publish_cve-add_deletion_option into ubuntu-cve-tracker:master

[Steve Beattie]

publish-cves-to-website-api.py: add capability to delete cves from website

Add command line option to allow deleting one or more CVEs. This
changes the behavior of the script subtly: it now does not require
any (CVE) paths to be passed as arguments; if none are given, no CVEs
will be updated.

Some example invocations:

Delete CVE-2023-42572:
  $ scripts/publish-cves-to-website-api.py --delete CVE-2023-42572

Delete CVE-2023-42572 and CVE-2024-42572, exiting early if the first CVE
is not present on the web endpoint:
  $ scripts/publish-cves-to-website-api.py --stop --delete CVE-2023-42572 --delete CVE-2024-42572

Delete CVE-2023-42572 and update active/CVE-2023-42752, exiting early if
CVE-2023-42572 is not present on the web endpoint; i.e. CVE-2023-42752
will not get updated:
  $ scripts/publish-cves-to-website-api.py --stop --delete CVE-2023-42572 active/CVE-2023-42752

Note that the backend cronjob that runs this script has not been
modified to take advantage of this option; as of now, deleting a CVE
from the website requires a manual invocation of the script. (And
while the json endpoint will be updated immediately, it's not clear
how long the squid proxy will hold the cached version, alas :/ ).

Refactoring out the CVE_regex also fixes a subtle bug; if there was
a CVE with an suffix ID greater than 7 digits, the script would have
skipped updating it.

Thanks to me propogating a typo in a kernel changelog, I needed to delete a CVE from the website API. Unfortunately, our tools are not smart enough to do that when they see a CVE file go missing from the git repo. This merge request adds the capability to the publish-cves-to-website-api.py to manually delete a CVE from the website with the added --delete option (can be applied multiple times to delete multiple CVEs).

[Alex Murray (alexmurray)]

LGTM!

review: Approve

diff --git a/scripts/publish-cves-to-website-api.py b/scripts/publish-cves-to-website-api.py
index 64ce483..3e9f494 100755
--- a/scripts/publish-cves-to-website-api.py
+++ b/scripts/publish-cves-to-website-api.py
@@ -17,6 +17,7 @@ from macaroonbakery import httpbakery
 #import http
 #http.client.HTTPConnection.debuglevel = 1
 
+CVE_regex_pattern = "CVE-\\d{4}-\\d{4,}$"
 
 IGNORE_CACHE = os.path.expanduser("~/.publish-cves-ignore-cache")
 
@@ -81,6 +82,34 @@ def get_devel_codename(cve_releases):
     return cve_devel_release
 
 
+def delete_single_cve(args, cve):
+    http_method = "DELETE"
+    endpoint = f"{args.endpoint}cves/{cve}.json"
+    if args.dry_run:
+        print(f"DEBUG: would send delete http request for: {endpoint}")
+    else:
+        if args.verbose:
+            print(f"INFO: sending delete http request for: {endpoint}")
+
+        response = authentication(method=http_method, url=endpoint, payload=None)
+        print(response, response.text[0:60])
+        if args.stop and not re.match(r"^<Response \[2..\]>$", str(response)):
+            sys.exit(1)
+
+
+def do_delete_cves(args, cve_list):
+    CVE_regex = re.compile(f"^{CVE_regex_pattern}$")
+
+    for cve in cve_list:
+        if not re.match(CVE_regex, cve):
+            print(
+                f"WARNING: skipping deletion of non-CVE '{cve}'",
+                file=sys.stderr
+            )
+            continue
+        delete_single_cve(args, cve)
+
+
 def post_single_cve(cve_filename):
     # Upload active and ignored (in Ubuntu)
     cve_data = cve_lib.load_cve(cve_filename)
@@ -254,18 +283,28 @@ def main(argv=None):
         help="API endpoint url.",
     )
     parser.add_argument(
+        "--delete",
+        action="append",
+        type=str,
+        help="CVE(s) to delete from the web endpoint",
+    )
+    parser.add_argument(
         "file_path",
         action="store",
         type=str,
-        nargs="+",
+        nargs="*",
         help="[Required] The path of the CVE file(s) or folder(s)",
     )
+
     args = parser.parse_args(argv)
 
+    if args.delete is not None:
+        do_delete_cves(args, args.delete)
+
     ## if args:
     ## headers = {"Content-type": "application/json"}
     cves = []
-    CVE_filename_regex = re.compile(".*/?CVE-\\d{4}-\\d{4,7}$" if args.filename_check else ".*")
+    CVE_filename_regex = re.compile(f".*/?{CVE_regex_pattern}$" if args.filename_check else ".*")
     NFU_filename_regex = re.compile(".*/not-for-us.txt$")
     ignore_paths = ['experimental', 'subprojects', 'scripts']
     cache_not_for_us_cve_ids = list()