Merge ~sbeattie/ubuntu-cve-tracker/+git/ubuntu-cve-tracker:check-cves_ignore_cache into ubuntu-cve-tracker:master

[Steve Beattie]

[WIP] check-cves: add a cache of 5 last ignored reasons

Often times when performing triage I bounce between CVEs of a few
different products from the same vendor that end up being ignored. It's
useful to have roughly a cache of five or so entries, plus the debian
reason if available plus the heuristic guess.

Implement a small cache of previously used reasons for ignoring a cve.
It keeps the ordering the same as before, with the cache entries added
last, orted in most recently used order.

Using this results in things looking like:

   Debian CVE Tracker: NOT-FOR-US: Node sds

  A]dd (or R]epeat), I]gnore forever, S]kip for now, or Q]uit? [ignore]
  Reason to be ignored?
     a) Node sds
     b) sds from
     c) BlogEngine.NET
     d) InHand Networks InRouter302
     e) OpenClinica
     f) Intel(R) NUCs
     g) Check Point Enterprise Endpoint

Signed-off-by: Steve Beattie <email address hidden>
TODO: unit tests. Sorry.

[Paulo Flabiano Smorigo (pfsmorigo)]

Need unit tests, otherwise LGTM :P

[Seth Arnold (seth-arnold)]

Nice :)

review: Approve

[Alex Murray (alexmurray)]

Would it be possible to also add this in the "experimental" mode as well?

[Steve Beattie (sbeattie)]

On Wed, May 18, 2022 at 01:11:48AM -0000, Alex Murray wrote:
> Would it be possible to also add this in the "experimental" mode as well?

I'm open to the idea, but I'm not sure how. In interactive mode,
whatever reason you give to ignore a cve is added to the cache (limited
to 5 entries), and the next time you attempt to ignore a CVE, it shows
you the last five you entered.

Since the experimental mode is non-interactive and thus all proposed
interactions are pre-computed, I'm not sure how to do something similar.

Now, there are some ways we could improve both interactive and
non-interactive, given that we have a gigantic saved history of entries
from people in the not-for-us.txt file.

For interactive mode, I'd like to make full use of readline's
capabilities with possibly two history files[1], one for ignored reasons,
and one for packages entered, so that similar to bash and other
shells, you could search through history for an appropriate entry
(and also to make interactive editing better).

There's probably more processing we could do for the non-interactive
version.

[1] unfortunately, libreadline only supports one global history per
    process, so when switching between adding a CVE and ignoring a CVE
    or vice versa, we'd need to flush the history and reload the saved
    history from the other save file.

--
Steve Beattie
<email address hidden>

[Alex Murray (alexmurray)]

Bah of course, this doesn't really make sense in the non-interactive case - sorry I wasn't thinking this through properly. That is a good idea re combing not-for-us.txt but that sounds like something for a future piece of work.

review: Approve

[Eduardo Barretto (ebarretto)]

This was merged back in fbb11d1762fcbfd1d2114b2d745736d7108eb165

diff --git a/scripts/check-cves b/scripts/check-cves
index 7f061f2..975b1cd 100755
--- a/scripts/check-cves
+++ b/scripts/check-cves
@@ -8,7 +8,7 @@
 # Author: Jamie Strandboge <[e-mail]>
 # Author: Marc Deslauriers <[e-mail]>
 # Author: Steve Beattie <[e-mail]>
-# Copyright (C) 2005-2021 Canonical Ltd.
+# Copyright (C) 2005-2022 Canonical Ltd.
 #
 # This script is distributed under the terms and conditions of the GNU General
 # Public License, Version 2 or later. See http://www.gnu.org/copyleft/gpl.html
@@ -38,6 +38,7 @@ import progressbar
 
 import cve_lib
 import source_map
+from check_cves.ignored_cache import IgnoredCache
 
 # load settings, if any
 cve_lib.read_config()
@@ -550,6 +551,7 @@ def cvss_source_from_filename(name):
             return source
     return 'unknown'
 
+
 class CVEHandler(xml.sax.handler.ContentHandler):
     """SAX handler for processing mitre's CVE database XML."""
 
@@ -574,7 +576,7 @@ class CVEHandler(xml.sax.handler.ContentHandler):
         self.cve_seen = set()
         self.cve_list = []
         self.cve_data = dict()
-        self.saved_ignore_reason = ""
+        self.saved_ignore_cache = IgnoredCache()
         self.saved_package = ""
         self.saved_cve = ""
         self.debian = None
@@ -1127,11 +1129,11 @@ class CVEHandler(xml.sax.handler.ContentHandler):
                 prompts = []
 
                 # Show debian reason first, then automatically determined
-                # reason, then saved reason. This makes more sense
+                # reason, then saved reasons. This makes more sense
                 # than sorting the reasons and is more predictable
-                for choice in [reason, 
-                               self.get_ignore_suggestion(self.cve_data[cve]['desc']),
-                               self.saved_ignore_reason]:
+                choices = [reason, self.get_ignore_suggestion(self.cve_data[cve]['desc'])]
+                choices.extend(self.saved_ignore_cache.get())
+                for choice in choices:
                     if choice != "" and choice not in prompts:
                         prompts.append(choice)
 
@@ -1153,7 +1155,7 @@ class CVEHandler(xml.sax.handler.ContentHandler):
                 elif len(info) < 3:  # Fat fingers protection
                     print('\nError: Reason must be at least 3 characters long!\n')
                     info = ""
-            self.saved_ignore_reason = info
+            self.saved_ignore_cache.insert(info)
             self.ignore_cve(cve, info)
 
         elif info.startswith('s'):
diff --git a/scripts/check_cves/ignored_cache.py b/scripts/check_cves/ignored_cache.py
 new file mode 100644
index 0000000..6130994
--- /dev/null
+++ b/scripts/check_cves/ignored_cache.py
@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#
+#  Copyright (C) 2022 Canonical Ltd.
+#
+#  Implements a simple cache for recent reasons for ignoring CVE
+#  entries, as used by check-cves
+
+
+class IgnoredCache():
+    # Keeps a list of ignored entries with a maximum length
+
+    _ignored = list()
+
+    def __init__(self, initial_list=None, max_len=5):
+        if initial_list:
+            self._ignored = initial_list
+        self.max_len = max_len
+        self._prune_cache()
+
+    # ensure list is less than or equal to the maximum length, and prune
+    # if not
+    def _prune_cache(self):
+        if len(self._ignored) > self.max_len:
+            self._ignored = self._ignored[:self.max_len]
+
+    def insert(self, reason):
+        # if the reason already exists, remove it so it can be
+        # reinserted at the head of the list
+        if reason in self._ignored:
+            self._ignored.remove(reason)
+
+        self._ignored.insert(0, reason)
+        self._prune_cache()
+
+    def get(self):
+        return self._ignored