Merge ~sbeattie/ubuntu-cve-tracker/+git/ubuntu-cve-tracker:fix_publish_usn_removal_operations into ubuntu-cve-tracker:master

[Steve Beattie]

publish-usn-to-website (+ api script): fix removal of usns from the website

As Rodrigo recently experienced, the script and process for removing a USN from the website had not been exercised very well. Wehn removing a USN from the API, the api script was not honoring a prefix if it was passed to it, so fix that, and also report more of the response message from the web api endpoint.

The wrapper publish-usn-to-website script assumed and checked that the USN db would have information that it could export and publish to the website via the api script, which is a mistaken assumption when we are removing a USN, as it should be purged from the USN db first, and indeed, the api script does not use anything beyond the USN ID for deletion, so fix the script to correctly invoke the api script in the removal case, and also adjust the consistency checks to handle both the removal and non-removal cases.kj

[Rodrigo Figueiredo Zaiden (rodrigo-zaiden)]

LGTM. Thanks!
As I've exercised this lately, I did a few debugging
tests with what I had in mind and it all worked fine.

review: Approve

diff --git a/scripts/publish-usn-to-website b/scripts/publish-usn-to-website
index 5c57a64..ba660fe 100755
--- a/scripts/publish-usn-to-website
+++ b/scripts/publish-usn-to-website
@@ -69,18 +69,25 @@ push_to_website_api() {
 
 	FILES_TO_CLEANUP+=($TMP_JSON)
 
-	# export the single USN from the pickle to a yaml file
-	if ! "$usn_tool"/usn.py --db "$database" --export-json "$usn" >"$TMP_JSON" ; then
-		perr "Failed to export $usn from pickle to yaml"
-		exit 1
-	fi
+	if [ "$action" == remove ] ; then
+		# remove the single USN via the website API
+		if ! "$UCT"/scripts/publish-usn-to-website-api.py --action "$action" --usn "$usn"  --prefix "USN-" --stop --no-upsert; then
+			perr "Failed to publish $usn pickle to website API"
+			exit 1
+		fi
+	else
+		# export the single USN from the pickle to a yaml file
+		if ! "$usn_tool"/usn.py --db "$database" --export-json "$usn" >"$TMP_JSON" ; then
+			perr "Failed to export $usn from pickle to yaml"
+			exit 1
+		fi
 
-	# post the single USN json file to the website API
-	if ! "$UCT"/scripts/publish-usn-to-website-api.py --action "$action" --json "$TMP_JSON" --prefix "USN-" --stop --no-upsert; then
-		perr "Failed to publish $usn pickle to website API"
-		exit 1
+		# post the single USN json file to the website API
+		if ! "$UCT"/scripts/publish-usn-to-website-api.py --action "$action" --json "$TMP_JSON" --prefix "USN-" --stop --no-upsert; then
+			perr "Failed to publish $usn pickle to website API"
+			exit 1
+		fi
 	fi
-
 }
 
 
@@ -134,9 +141,13 @@ fi
 
 
 for usn in $usns; do
-	if ! is_usn_in_db "$database" "$usn"; then
+	if ! [ "$action" == "remove" ] && ! is_usn_in_db "$database" "$usn"; then
 		perr "Ensure that USN-$usn is in $database and try again"
 		exit 1
+	elif [ "$action" == "remove" ] && is_usn_in_db "$database" "$usn"; then
+		perr "Found USN-$usn in ${database}"
+		perr "When removing USNs, ensure that USN-$usn is not in $database and try again"
+		exit 1
 	fi
 
 	push_to_website_api "$action" "$database" "$usn"
diff --git a/scripts/publish-usn-to-website-api.py b/scripts/publish-usn-to-website-api.py
index 082a671..e947a54 100755
--- a/scripts/publish-usn-to-website-api.py
+++ b/scripts/publish-usn-to-website-api.py
@@ -195,12 +195,16 @@ if args.startwith and not re.match(r"\d{1,5}-\d{1,2}", args.startwith):
 if args.action == "remove" and args.usns:
     http_method = "DELETE"
     for usn in args.usns:
-        endpoint = f"{args.endpoint}/{usn}.json"
+        if args.prefix:
+            prefix = args.prefix
+        else:
+            prefix = ""
+        endpoint = f"{args.endpoint}/{prefix}{usn}.json"
         if args.debug:
             print(f"DEBUG: would send delete http request for: {endpoint}")
         else:
             response = authentication(method=http_method, url=endpoint, payload=None)
-            print(response, response.text[0:60])
+            print(response, response.text[0:256])
             if args.stop and not re.match(r"^<Response \[2..\]>$", str(response)):
                 sys.exit(1)
     sys.exit(0)