Merge ~litios/ubuntu-cve-tracker:oval-cve-tag into ubuntu-cve-tracker:master

[David Fernandez Gonzalez]

This change will add a new tag: <cve>

This tag will be used to represent the cve in the advisory, as this is especially needed for OVAL for packages. This cve tag tries to follow Red Hat approach:

<cve cvss3="5.4/CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:L/A:N" cwe="CWE-285" href="https://access.redhat.com/security/cve/CVE-2019-0816" impact="moderate" public="20190305">CVE-2019-0816</cve>

Some comments:

* Severity is extracted from Ubuntu's priority.
* Severity and public date are omitted in CVE format as they are already on the advisory.
* USNs are also added as a comma-separated list of elements.
* The most recent CVSS vector and score are added.

Examples:

- OVAL PKG advisory:

<rights>Copyright (C) 2023 Canonical Ltd.</rights>
<component>main</component>
<current_version>2.4.52-1ubuntu4.4</current_version>
<cve href="https://ubuntu.com/security/CVE-2023-25690" severity="medium" public="20230307" cvss_score="9.8" cvss_vector="CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" usns="942-1,942-2">CVE-2023-25690</cve>
<cve href="https://ubuntu.com/security/CVE-2023-27522" severity="medium" public="20230307" cvss_score="7.5" cvss_vector="CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:H/A:N" usns="942-1">CVE-2023-27522</cve>

- OVAL CVE advisory:

<severity>Medium</severity>
<rights>Copyright (C) 2023 Canonical Ltd.</rights>
<public_date>2023-01-11</public_date>
<public_date_at_usn>2023-01-11</public_date_at_usn>
<cve href="https://ubuntu.com/security/CVE-2023-0210" cvss_score="7.5" cvss_vector="CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" usns="915-1,982-1,987-1,004-1">CVE-2023-0210</cve>

32f0063... by David Fernandez Gonzalez on 2023-04-14

oval_lib: add cve tag to usns

[Eduardo Barretto (ebarretto)]

overall lgtm, but I do have a doubt if we decided to not have severity in the <cve> tag for CVE-based OVAL and USN-based OVAL. Do you remember?

review: Approve

[David Fernandez Gonzalez (litios)]

Yeah, for CVE-based OVAL we decided to not add it as it was redundant.

> we already have severity on the top, I wonder if we keep severity from us, and in the cve tag whatever is from cvss

For USN-based OVAL we didn't discuss it but the severity we are using is computed from all the CVEs to represent the general severity. I think it would be useful to have the severities for each of the CVEs on the USN.

349332b... by David Fernandez Gonzalez on 2023-04-25

oval_lib: refactor cve tag generation to unify across formats

[David Fernandez Gonzalez (litios)]

As discussed with Eduardo, it's best to keep a stable format for all OVAL formats. The format follows the structure as described in the PKG format in the description:

<cve href="https://ubuntu.com/security/CVE-2022-21427" severity="medium" public="20220419" cvss_score="4.9" cvss_vector="CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:N/I:N/A:H" usns="400-1,400-2,739-1">CVE-2022-21427</cve>

[Eduardo Barretto (ebarretto)]

lgtm, thanks for addressing the last changes!

review: Approve

diff --git a/scripts/generate-oval b/scripts/generate-oval
index f703733..d168786 100755
--- a/scripts/generate-oval
+++ b/scripts/generate-oval
@@ -234,7 +234,6 @@ def parse_cve_file(filepath, cache, pkg_filter=None):
     cve_header_data['References'].insert(0, get_cve_url(filepath))
     cve_header_data['Bugs'] = [bug.strip() for bug in cve_header_data['Bugs'].split('\n') if len(bug.strip()) > 0]
     cve_header_data['Notes'] = ' '.join(user + '> ' + note.replace('\n', ' ') for user, note in cve_header_data['Notes'])
-    cve_header_data['CVSS'] = ' '.join(cvss['source'] + ': ' + cvss['vector'] for cvss in cve_header_data['CVSS'])
 
     packages = {}
     for pkg in data['pkgs']:
diff --git a/scripts/oval_lib.py b/scripts/oval_lib.py
index d8365c2..a08edab 100644
--- a/scripts/oval_lib.py
+++ b/scripts/oval_lib.py
@@ -120,6 +120,26 @@ def debug(message):
     if debug_level > 0:
         sys.stdout.write('\rDEBUG: {0}\n'.format(message))
 
+def generate_cve_tag(cve):
+    cve_ref = '<cve href="https://ubuntu.com/security/{0}" severity="{1}" public="{2}"'.format(cve['Candidate'], cve['Priority'], cve['PublicDate'].split(' ')[0].replace('-', ''))
+
+    if cve['CVSS']:
+        cve_ref += ' cvss_score="{0}" cvss_vector="{1}"'.format(cve['CVSS'][0]['baseScore'], cve['CVSS'][0]['vector'])
+
+    cve_ref_usns = False
+    for ref in cve['References']:
+        if 'https://ubuntu.com/security/notices/USN' in ref:
+            if not cve_ref_usns:
+                cve_ref_usns = True
+                cve_ref += ' usns="'
+            cve_ref += '{0},'.format(ref[41:])
+
+    if cve_ref_usns:
+        cve_ref = '{0}"'.format(cve_ref[:-1])
+
+    cve_ref += '>{0}</cve>'.format(cve['Candidate'])
+    return cve_ref
+
 class OvalGenerator:
     supported_oval_elements = ('definition', 'test', 'object', 'state', 'variable')
     generator_version = '1.1'
@@ -424,6 +444,13 @@ class CVE:
     def __init__(self, number, info, pkgs=[]) -> None:
         self.number = number
         self.description = info['Description']
+        self.severity = info['Priority']
+        self.public_date = info['PublicDate']
+        self.cvss = info['CVSS']
+        self.usns = []
+        for url in info['References'].split('\n'):
+            if 'https://ubuntu.com/security/notices/USN-' in url:
+                self.usns.append(url[41:])
         self.pkg_rel_entries = {}
         self.pkgs = pkgs
 
@@ -501,6 +528,10 @@ class OvalGeneratorPkg(OvalGenerator):
         component = etree.SubElement(advisory, "component")
         version = etree.SubElement(advisory, "current_version")
 
+        for cve in package.cves:
+            cve_obj = self._generate_cve_object(cve)
+            advisory.append(cve_obj)
+
         rights.text = f"Copyright (C) {datetime.now().year} Canonical Ltd."
         component.text = package.section
         version.text = package.version
@@ -558,6 +589,22 @@ class OvalGeneratorPkg(OvalGenerator):
         definition.append(criteria)
         return definition
 
+    def _generate_cve_object(self, cve: CVE) -> etree.Element:
+        cve_tag = etree.Element("cve", 
+            attrib={
+                'href' : f"https://ubuntu.com/security/{cve.number}",
+                'severity': cve.severity,
+                'public': cve.public_date.split(' ')[0].replace('-', '')
+            })
+        
+        cve_tag.text = cve.number
+        if cve.cvss:
+            cve_tag.set('cvss_score', cve.cvss[0]['baseScore'])
+            cve_tag.set('cvss_vector', cve.cvss[0]['vector'])
+            if cve.usns:
+                cve_tag.set('usns', ','.join(cve.usns))
+
+        return cve_tag
     def _generate_var_object(self, comment, id, binaries) -> etree.Element:
         var = etree.Element("constant_variable",
             attrib={
@@ -1297,8 +1344,9 @@ class OvalGeneratorCVE:
                     continue
                 mapping['cve_title'] = escape(cve_title)
                 mapping['references'] = '\n                    <reference source="CVE" ref_id="{0}" ref_url="{1}" />'.format(mapping['cve_title'], escape(ref))
-            else:
-                advisory.append('<ref>{0}</ref>'.format(escape(ref)))
+
+        cve_ref = generate_cve_tag(header)
+        advisory.append(cve_ref)
         mapping['advisory_elements'] = '\n                        '.join(advisory)
 
         if self.oval_format == 'dpkg':
@@ -1993,10 +2041,10 @@ class OvalGeneratorUSN():
     def create_cves_references(self, cves):
         references = ""
         for cve in cves:
+            cve_ref = generate_cve_tag(cve)
             references += \
-                    """                <reference source="CVE" ref_url="{}" ref_id="{}"/>
-""".format(cve['CVE_URL'], cve['Candidate'])
-
+                """{0}
+                    """.format(cve_ref)
         return references.strip()
 
     def get_usn_severity(self, cves):
@@ -2075,11 +2123,11 @@ class OvalGeneratorUSN():
                     <platform>{platform}</platform>
                 </affected>
                 <reference source="USN" ref_url="{usn_url}" ref_id="{usn_id}"/>
-                {cves_references}
                 <description>{description}</description>
                 <advisory from="[e-mail]">
                     <severity>{severity}</severity>
                     <issued date="{usn_timestamp}"/>
+                    {cves_references}
                     {bug_references}
                 </advisory>
             </metadata>
@@ -2342,23 +2390,14 @@ class OvalGeneratorUSN():
 
         public_date = cve_object['PublicDate']
         priority = cve_object['Priority']
-
+        references = cve_object['References']
         # TODO: deal with multiple CVSS?
-        cvss = None
-        baseScore = None
-        baseSeverity = None
-        if cve_object['CVSS']:
-            cvss = cve_object['CVSS'][0]['vector']
-            baseScore = cve_object['CVSS'][0]['baseScore']
-            baseSeverity = cve_object['CVSS'][0]['baseSeverity'].title()
-
         cve_info = {
                 'Candidate': cve,
                 'PublicDate': public_date,
                 'Priority': priority,
-                'CVSS': cvss,
-                'CVSS_SEVERITY_LEVEL': baseSeverity,
-                'CVSS_SCORE': baseScore,
+                'CVSS': cve_object['CVSS'],
+                'References': references.split('\n'),
                 'CVE_URL': self.cve_base_url.format(cve),
                 'MITRE_URL': self.mitre_base_url.format(cve)
                 }