Merge ~alexmurray/ubuntu-cve-tracker:import-rhel8oval into ubuntu-cve-tracker:master

[Alex Murray]

See https://trello.com/c/tMAhQivD/2007-add-support-for-importing-cves-from-red-hats-unpatched-oval

[Alex Murray (alexmurray)]

Hmm launchpad seems to have messed up in generation of the diff for review - this is only the following 2 commits:

https://git.launchpad.net/~alexmurray/ubuntu-cve-tracker/commit/?id=1d2f6cb86412081333d6c55fc4ba8ba4ca6e9242

https://git.launchpad.net/~alexmurray/ubuntu-cve-tracker/commit/?id=5d3252d6bb6631d0c391720e5b3aa2b19a483378

[Steve Beattie (sbeattie)]

Looks good to me, merged. I fixed in a follow up commit the xml tag used to identify reference URLs; the relevant XML is:

  <reference ref_id="CVE-2019-10218" ref_url="https://access.redhat.com/security/cve/CVE-2019-10218" source="CVE"/>

Thanks!

review: Approve

[Alex Murray (alexmurray)]

Ah, thanks Steve.

diff --git a/active/CVE-2019-13616 b/active/CVE-2019-13616
index e3fa4d2..44f1cea 100644
--- a/active/CVE-2019-13616
+++ b/active/CVE-2019-13616
@@ -55,7 +55,7 @@ xenial_libsdl2-image: needed
 bionic_libsdl2-image: needed
 disco_libsdl2-image: ignored (reached end-of-life)
 eoan_libsdl2-image: needed
-devel_libsdl2-image: needed
+devel_libsdl2-image: not-affected (2.0.5+dfsg1-2)
 
 Patches_sdl-image1.2:
  upstream: https://hg.libsdl.org/SDL_image/rev/a59bfe382008
diff --git a/active/CVE-2019-20433 b/active/CVE-2019-20433
index 5d7bbcf..e9fa25e 100644
--- a/active/CVE-2019-20433
+++ b/active/CVE-2019-20433
@@ -10,6 +10,8 @@ Description:
  variable.
 Ubuntu-Description:
 Notes:
+ leosilva> the fix proposed can potentially break applications
+ leosilva> that depends on it.
 Mitigation:
 Bugs:
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=935128
diff --git a/active/CVE-2020-5209 b/active/CVE-2020-5209
index c3c32b9..a9f4a71 100644
--- a/active/CVE-2020-5209
+++ b/active/CVE-2020-5209
@@ -12,15 +12,16 @@ Description:
  options. Users should upgrade to NetHack 3.6.5.
 Ubuntu-Description:
 Notes:
+ msalvatore> Nethack is installed sgid games, but not suid or sgid root.
 Mitigation:
 Bugs:
-Priority: untriaged
+Priority: low
 Discovered-by:
 Assigned-to:
 
 
 Patches_nethack:
-upstream_nethack: needs-triage
+upstream_nethack: released (2.6.5)
 precise/esm_nethack: DNE
 trusty_nethack: ignored (out of standard support)
 trusty/esm_nethack: DNE
diff --git a/active/CVE-2020-5210 b/active/CVE-2020-5210
index 3f7cd5d..638c80a 100644
--- a/active/CVE-2020-5210
+++ b/active/CVE-2020-5210
@@ -12,15 +12,16 @@ Description:
  influence command line options. Users should upgrade to NetHack 3.6.5.
 Ubuntu-Description:
 Notes:
+ msalvatore> Nethack is installed sgid games, but not suid or sgid root.
 Mitigation:
 Bugs:
-Priority: untriaged
+Priority: low
 Discovered-by:
 Assigned-to:
 
 
 Patches_nethack:
-upstream_nethack: needs-triage
+upstream_nethack: released (2.6.5)
 precise/esm_nethack: DNE
 trusty_nethack: ignored (out of standard support)
 trusty/esm_nethack: DNE
diff --git a/active/CVE-2020-5211 b/active/CVE-2020-5211
index 9f3a8aa..3b099c2 100644
--- a/active/CVE-2020-5211
+++ b/active/CVE-2020-5211
@@ -12,19 +12,20 @@ Description:
  Users should upgrade to NetHack 3.6.5.
 Ubuntu-Description:
 Notes:
+ msalvatore> Nethack is installed sgid games, but not suid or sgid root.
 Mitigation:
 Bugs:
-Priority: untriaged
+Priority: low
 Discovered-by:
 Assigned-to:
 
 
 Patches_nethack:
-upstream_nethack: needs-triage
+upstream_nethack: released (2.6.5)
 precise/esm_nethack: DNE
 trusty_nethack: ignored (out of standard support)
 trusty/esm_nethack: DNE
-xenial_nethack: needs-triage
-bionic_nethack: needs-triage
-eoan_nethack: needs-triage
-devel_nethack: needs-triage
+xenial_nethack: needed
+bionic_nethack: needed
+eoan_nethack: needed
+devel_nethack: needed
diff --git a/active/CVE-2020-5212 b/active/CVE-2020-5212
index 5323e39..0565cbe 100644
--- a/active/CVE-2020-5212
+++ b/active/CVE-2020-5212
@@ -12,15 +12,16 @@ Description:
  NetHack 3.6.5.
 Ubuntu-Description:
 Notes:
+ msalvatore> Nethack is installed sgid games, but not suid or sgid root.
 Mitigation:
 Bugs:
-Priority: untriaged
+Priority: low
 Discovered-by:
 Assigned-to:
 
 
 Patches_nethack:
-upstream_nethack: needs-triage
+upstream_nethack: released (2.6.5)
 precise/esm_nethack: DNE
 trusty_nethack: ignored (out of standard support)
 trusty/esm_nethack: DNE
diff --git a/active/CVE-2020-5213 b/active/CVE-2020-5213
index 0679c33..74b6bea 100644
--- a/active/CVE-2020-5213
+++ b/active/CVE-2020-5213
@@ -12,15 +12,16 @@ Description:
  3.6.5.
 Ubuntu-Description:
 Notes:
+ msalvatore> Nethack is installed sgid games, but not suid or sgid root.
 Mitigation:
 Bugs:
-Priority: untriaged
+Priority: low
 Discovered-by:
 Assigned-to:
 
 
 Patches_nethack:
-upstream_nethack: needs-triage
+upstream_nethack: released (2.6.5)
 precise/esm_nethack: DNE
 trusty_nethack: ignored (out of standard support)
 trusty/esm_nethack: DNE
diff --git a/active/CVE-2020-5214 b/active/CVE-2020-5214
index e0fa344..3b82e91 100644
--- a/active/CVE-2020-5214
+++ b/active/CVE-2020-5214
@@ -12,15 +12,16 @@ Description:
  3.6.5.
 Ubuntu-Description:
 Notes:
+ msalvatore> Nethack is installed sgid games, but not suid or sgid root.
 Mitigation:
 Bugs:
-Priority: untriaged
+Priority: low
 Discovered-by:
 Assigned-to:
 
 
 Patches_nethack:
-upstream_nethack: needs-triage
+upstream_nethack: released (2.6.5)
 precise/esm_nethack: DNE
 trusty_nethack: ignored (out of standard support)
 trusty/esm_nethack: DNE
diff --git a/active/CVE-2020-7969 b/active/CVE-2020-7969
index faf087e..d9108b9 100644
--- a/active/CVE-2020-7969
+++ b/active/CVE-2020-7969
@@ -19,7 +19,7 @@ upstream_gitlab: not-affected (debian: Only affects Gitlab EE 8.0 and later)
 precise/esm_gitlab: DNE
 trusty_gitlab: ignored (out of standard support)
 trusty/esm_gitlab: DNE
-xenial_gitlab: needs-triage
+xenial_gitlab: not-affected (code not present)
 bionic_gitlab: DNE
 eoan_gitlab: DNE
 devel_gitlab: DNE
diff --git a/active/CVE-2020-8002 b/active/CVE-2020-8002
index 3b3baea..68b1499 100644
--- a/active/CVE-2020-8002
+++ b/active/CVE-2020-8002
@@ -22,11 +22,11 @@ Assigned-to:
 
 
 Patches_virglrenderer:
-upstream_virglrenderer: needs-triage
+upstream_virglrenderer: needed
 precise/esm_virglrenderer: DNE
 trusty_virglrenderer: ignored (out of standard support)
 trusty/esm_virglrenderer: DNE
 xenial_virglrenderer: DNE
-bionic_virglrenderer: needs-triage
-eoan_virglrenderer: needs-triage
-devel_virglrenderer: needs-triage
+bionic_virglrenderer: not-affected (code not present)
+eoan_virglrenderer: needed
+devel_virglrenderer: needed
diff --git a/active/CVE-2020-8003 b/active/CVE-2020-8003
index 257c734..8a39f3d 100644
--- a/active/CVE-2020-8003
+++ b/active/CVE-2020-8003
@@ -23,11 +23,11 @@ Assigned-to:
 
 
 Patches_virglrenderer:
-upstream_virglrenderer: needs-triage
+upstream_virglrenderer: needed
 precise/esm_virglrenderer: DNE
 trusty_virglrenderer: ignored (out of standard support)
 trusty/esm_virglrenderer: DNE
 xenial_virglrenderer: DNE
-bionic_virglrenderer: needs-triage
-eoan_virglrenderer: needs-triage
-devel_virglrenderer: needs-triage
+bionic_virglrenderer: needed
+eoan_virglrenderer: needed
+devel_virglrenderer: needed
diff --git a/scripts/check-cves b/scripts/check-cves
index cbf1631..7c9b47e 100755
--- a/scripts/check-cves
+++ b/scripts/check-cves
@@ -50,6 +50,7 @@ parser.add_option("-R", "--refresh", help="Refresh CVE descriptions", action="st
 parser.add_option("", "--test", help="Run regression tests", action="store_true")
 parser.add_option("--untriaged", help="Process untriaged CVEs from output of locate_cves.py", metavar="FILE")
 parser.add_option("--mbox", help="Process untriaged CVEs from mbox file", metavar="FILE")
+parser.add_option("--rhel8oval", help="Process untriaged RHEL8 CVEs", metavar="FILE")
 parser.add_option("--import-missing-debian", help="Process missing Debian CVEs", action="store_true")
 parser.add_option("--debug", help="Report debugging information", action="store_true")
 (opt, args) = parser.parse_args()
@@ -382,6 +383,93 @@ def import_debian(handler):
     return tmpname
 
 
+class RHEL8OVALHandler(xml.sax.handler.ContentHandler):
+    """SAX handler for processing rhel8 OVAL XML."""
+
+    def __init__(self, ignore=[]):
+        # For per-hit processing
+        self._curr_vuln = None
+        self._curr_cve = None
+        self._curr_pkgs = []
+        self._curr_url = []
+        self._curr_source = None
+
+        self._curr_chars_collect = False
+        self._curr_chars = ""
+
+        self._timestamp = None
+        self._cves = dict()
+
+    def startElement(self, name, attrs):
+        if name == 'oval:timestamp':
+            if opt.verbose:
+                print("Parsing RHEL8 OVAL schema", file=sys.stderr)
+            self._curr_chars_collect = True
+            self._curr_chars = ""
+        if name == "definition" and attrs['class'] == 'vulnerability':
+            self._curr_vuln = attrs['id']
+            self._curr_desc = None
+            self._curr_cve = None
+            self._curr_url = None
+        if name == "title":
+            self._curr_chars_collect = True
+            self._curr_chars = ""
+        if name == "reference":
+            self._curr_cve = attrs['ref_id']
+            if 'url' in attrs:
+                self._curr_url = attrs['url']
+
+    def characters(self, content):
+        if self._curr_chars_collect:
+            self._curr_chars += content
+
+    def endElement(self, name):
+        self._curr_chars_collect = False
+        if name == 'oval:timestamp':
+            self._timestamp = datetime.datetime.strptime(self._curr_chars, "%Y-%m-%dT%H:%M:%S")
+        if name == "title":
+            title = self._curr_chars
+            # rhel oval titles are of form "CVE-XXXX-XXXX Name: Description
+            # here (priority)" - we want to keep "Name: Description here"
+            self._curr_desc = ' '.join(title.split(' ')[1:-1])
+        if name == "definition":
+            self._cves.setdefault(self._curr_cve, dict())
+            self._cves[self._curr_cve].setdefault('desc', self._curr_desc)
+            self._cves[self._curr_cve].setdefault('refs', [self._curr_url])
+            self._cves[self._curr_cve].setdefault('date', self._timestamp)
+
+    def cves(self):
+        return self._cves
+
+
+def read_rhel8oval_file(f):
+    '''Read in rhel8 oval
+       This is sneaky because we read in the oval and then output a fake JSON
+       file for processing.
+    '''
+    if not os.path.isfile(f):
+        print("'%s' not a file" % f, file=sys.stderr)
+        sys.exit(1)
+
+    name = os.path.abspath(f + ".json")
+    if os.path.exists(name):
+        print("'%s' already exists" % name, file=sys.stderr)
+        sys.exit(1)
+
+    parser = xml.sax.make_parser()
+    handler = RHEL8OVALHandler()
+    parser.setContentHandler(handler)
+    parser.parse(f)
+
+    cves = handler.cves()
+    nvd = convert_to_nvd(cves, lambda cve: cves[cve]['desc'])
+    tmp = tempfile.NamedTemporaryFile(mode='w', prefix='rhel8oval-import_', suffix='.json', delete=False)
+    tmpname = tmp.name
+    tmp.file.write(json.dumps(nvd))
+    tmp.close()
+
+    return tmpname
+
 def read_locate_cves_output(f):
     '''Read in output of UCT/scripts/locate_cves.py
        This is sneaky because we read in the output and then output a fake JSON
@@ -1367,6 +1455,11 @@ if opt.mbox:
     untriaged_json = read_mbox_file(opt.mbox)
     args.append(untriaged_json)
 
+rhel8oval_import_json = ""
+if opt.rhel8oval:
+    untriaged_json = read_rhel8oval_file(opt.rhel8oval)
+    args.append(untriaged_json)
+
 debian_import_json = ""
 if opt.import_missing_debian and handler.debian is not None:
     debian_import_json = import_debian(handler)
diff --git a/scripts/process_cves b/scripts/process_cves
index 752e255..3152b0f 100755
--- a/scripts/process_cves
+++ b/scripts/process_cves
@@ -126,7 +126,7 @@ process_missing_debian() {
     ./scripts/check-cves --import-missing-debian
 }
 
-download_missing_redhat() {
+download_missing_rhsa() {
     tmpdir="$1"
     archive="$2.txt"
     fn="$tmpdir/$archive"
@@ -148,12 +148,30 @@ download_missing_redhat() {
     ./scripts/locate_cves.py "$fn" >> "$tmpdir/redhat.mbox"
 }
 
+download_rhel8oval() {
+    tmpdir="$1"
+    archive="$2"
+    fn="$tmpdir/$archive"
+    url="https://www.redhat.com/security/data/oval/v2/RHEL8/$archive.bz2"
+
+    echo "Downloading $url..."
+    # curl and wget fail with redhat's web proxy
+    w3m -dump_source "$url" > "$fn.bz2"
+    echo "Unzipping $fn.bz2..."
+    bzip2 -d "$fn.bz2"
+}
+
 process_missing_redhat() {
     tmpdir=$(mktemp -d)
-    download_missing_redhat "$tmpdir" "$(date +%Y-%B)"
-    download_missing_redhat "$tmpdir" "$(date --date="$(date +%Y-%m-15) -1 month" +%Y-%B)"
+    download_missing_rhsa "$tmpdir" "$(date +%Y-%B)"
+    download_missing_rhsa "$tmpdir" "$(date --date="$(date +%Y-%m-15) -1 month" +%Y-%B)"
     ./scripts/check-cves --untriaged "$tmpdir/redhat.mbox"
     rm -rf "$tmpdir"
+
+    tmpdir=$(mktemp -d)
+    download_rhel8oval "$tmpdir" "rhel-8-including-unpatched.oval.xml"
+    ./scripts/check-cves --rhel8oval "$tmpdir/rhel-8-including-unpatched.oval.xml"
+    rm -rf "$tmpdir"
 }
 
 check_syntax() {