Merge ~alexmurray/ubuntu-cve-tracker:check-syntax-support-for-kernel-patches into ubuntu-cve-tracker:master

[Alex Murray]

[Seth Arnold (seth-arnold)]

A few random comments throughout; I realize some are against the previous version, but it's better late than never...

[Alex Murray (alexmurray)]

This will likely be easier to review by looking at the individual commits.

[Alex Murray (alexmurray)]

Thanks Seth - I've added some extra changes for the regex precompilation etc, and tried to address your other comments too.

[Marc Deslauriers (mdeslaur)]

The commits look good! Now that I think about it, I agree that putting it in check-syntax and only applying it if there is no break-fix is the right approach.

The only thing that may need changing in the future is:

+ assert commit_hash is not None
+

I'd probably just "return []" there so that a bad upstream patch for some reason wouldn't break our triage, but I think that's unlikely to ever happen.

review: Approve

[Alex Murray (alexmurray)]

Thanks Marc - good point, I made it print an error as well in that case.

diff --git a/active/CVE-2021-46904 b/active/CVE-2021-46904
index f19d00f..72c1b40 100644
--- a/active/CVE-2021-46904
+++ b/active/CVE-2021-46904
@@ -15,6 +15,7 @@ Assigned-to:
 CVSS:
 
 Patches_linux:
+ break-fix: 72dc1c096c7051a48ab1dbb12f71976656b55eb5 8a12f8836145ffe37e9c8733dce18c22fb668b66
 upstream_linux: needs-triage
 trusty_linux: ignored (end of standard support)
 trusty/esm_linux: needs-triage
diff --git a/active/CVE-2021-46905 b/active/CVE-2021-46905
index c1b64e9..c422718 100644
--- a/active/CVE-2021-46905
+++ b/active/CVE-2021-46905
@@ -15,6 +15,7 @@ Assigned-to:
 CVSS:
 
 Patches_linux:
+ break-fix: 8a12f8836145ffe37e9c8733dce18c22fb668b66 2ad5692db72874f02b9ad551d26345437ea4f7f3
 upstream_linux: not-affected (debian: No Debian released version vulnerable)
 trusty_linux: ignored (end of standard support)
 trusty/esm_linux: needs-triage
diff --git a/active/CVE-2022-48626 b/active/CVE-2022-48626
index dc2d16c..76d45c6 100644
--- a/active/CVE-2022-48626
+++ b/active/CVE-2022-48626
@@ -15,6 +15,7 @@ Assigned-to:
 CVSS:
 
 Patches_linux:
+ break-fix: - bd2db32e7c3e35bd4d9b8bbff689434a50893546
 upstream_linux: needs-triage
 trusty_linux: ignored (end of standard support)
 trusty/esm_linux: needs-triage
diff --git a/active/CVE-2023-52469 b/active/CVE-2023-52469
index 5d4154b..f4d8a33 100644
--- a/active/CVE-2023-52469
+++ b/active/CVE-2023-52469
@@ -17,6 +17,7 @@ Assigned-to:
 CVSS:
 
 Patches_linux:
+ break-fix: a2e73f56fa6282481927ec43aa9362c03c2e2104 28dd788382c43b330480f57cd34cde0840896743
 upstream_linux: needs-triage
 trusty_linux: ignored (end of standard support)
 trusty/esm_linux: needs-triage
diff --git a/active/CVE-2024-26602 b/active/CVE-2024-26602
index b7e01d9..630af59 100644
--- a/active/CVE-2024-26602
+++ b/active/CVE-2024-26602
@@ -15,6 +15,10 @@ Assigned-to:
 CVSS:
 
 Patches_linux:
+<<<<<<< active/CVE-2024-26602
+=======
+ break-fix: 22e4ebb975822833b083533035233d128b30e98f 944d5fe50f3f03daacfea16300e656a1691c4a23
+>>>>>>> active/CVE-2024-26602
  break-fix: c5f58bd58f432be5d92df33c5458e0bcbee3aadf 944d5fe50f3f03daacfea16300e656a1691c4a23
 upstream_linux: needs-triage
 trusty_linux: ignored (end of standard support)
diff --git a/ignored/CVE-2019-12379 b/ignored/CVE-2019-12379
index a4f7043..8dd5d99 100644
--- a/ignored/CVE-2019-12379
+++ b/ignored/CVE-2019-12379
@@ -21,6 +21,7 @@ CVSS:
  nvd: CVSS:3.0/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H [5.5 MEDIUM]
 
 Patches_linux:
+ break-fix: - 84ecc2f6eb1cb12e6d44818f94fa49b50f06e6ac
 upstream_linux: ignored (not an issue)
 precise/esm_linux: ignored (end of life, was needs-triage)
 trusty_linux: ignored (end of standard support)
diff --git a/ignored/CVE-2019-12454 b/ignored/CVE-2019-12454
index b77df14..ec5441e 100644
--- a/ignored/CVE-2019-12454
+++ b/ignored/CVE-2019-12454
@@ -24,6 +24,7 @@ CVSS:
  nvd: CVSS:3.0/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H [7.8 HIGH]
 
 Patches_linux:
+ break-fix: - a54988113985ca22e414e132054f234fc8a92604
 upstream_linux: not-affected (debian: Vulnerable code not present, introduced in 5.1-rc1)
 precise/esm_linux: ignored (end of life, was needs-triage)
 trusty_linux: ignored (end of standard support)
diff --git a/ignored/CVE-2022-3642 b/ignored/CVE-2022-3642
index 5d2904a..f3cf007 100644
--- a/ignored/CVE-2022-3642
+++ b/ignored/CVE-2022-3642
@@ -19,6 +19,7 @@ CVSS:
  nvd: CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N [5.5 MEDIUM]
 
 Patches_linux:
+ break-fix: c888183b21f36a247bb166ca9365705611bea847 80e5acb6dd72b25a6e6527443b9e9c1c3a7bcef6
 upstream_linux: not-affected (debian: only wireless-next)
 esm-infra/xenial_linux: needs-triage
 trusty_linux: ignored (end of standard support)
diff --git a/ignored/CVE-2023-35825 b/ignored/CVE-2023-35825
index ff67bb5..6e42c4e 100644
--- a/ignored/CVE-2023-35825
+++ b/ignored/CVE-2023-35825
@@ -23,6 +23,7 @@ Assigned-to:
 CVSS:
 
 Patches_linux:
+ break-fix: - 63264422785021704c39b38f65a78ab9e4a186d7
 upstream_linux: released (6.3.7-1)
 trusty_linux: ignored (end of standard support)
 trusty/esm_linux: needs-triage
diff --git a/retired/CVE-2008-1375 b/retired/CVE-2008-1375
index cfacb91..87c677d 100644
--- a/retired/CVE-2008-1375
+++ b/retired/CVE-2008-1375
@@ -46,6 +46,7 @@ hardy_linux-source-2.6.22: DNE
 devel_linux-source-2.6.22: DNE
 
 Patches_linux:
+ break-fix: - 214b7049a7929f03bbd2786aaef04b8b79db34e2
 upstream_linux: pending (2.6.26-rc1)
 dapper_linux: DNE
 feisty_linux: DNE
diff --git a/retired/CVE-2019-12455 b/retired/CVE-2019-12455
index 61a8e7e..397354f 100644
--- a/retired/CVE-2019-12455
+++ b/retired/CVE-2019-12455
@@ -2,7 +2,7 @@ Candidate: CVE-2019-12455
 PublicDate: 2019-05-30 04:29:00 UTC
 References:
  https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12455
- https://git.kernel.org/pub/scm/linux/kernel/git/sunxi/linux.git/commit/?h=sunxi/clk-for-5.3&id=fcdf445ff42f036d22178b49cf64e92d527c1330
+ https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fcdf445ff42f036d22178b49cf64e92d527c1330
  https://www.mail-archive.com/[e-mail]/msg2010240.html
 Description:
  ** DISPUTED ** An issue was discovered in sunxi_divs_clk_setup in
@@ -28,6 +28,7 @@ CVSS:
  nvd: CVSS:3.0/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H [5.5 MEDIUM]
 
 Patches_linux:
+ break-fix: - fcdf445ff42f036d22178b49cf64e92d527c1330
 upstream_linux: needed
 precise/esm_linux: ignored (end of life, was needs-triage)
 trusty_linux: ignored (end of standard support)
diff --git a/retired/CVE-2021-3542 b/retired/CVE-2021-3542
index d24b901..f916d9b 100644
--- a/retired/CVE-2021-3542
+++ b/retired/CVE-2021-3542
@@ -23,6 +23,7 @@ Assigned-to:
 CVSS:
 
 Patches_linux:
+ break-fix: - 35d2969ea3c7d32aee78066b1f3cf61a0d935a4e
 upstream_linux: not-affected
 trusty_linux: ignored (end of standard support)
 trusty/esm_linux: not-affected
diff --git a/scripts/active_edit b/scripts/active_edit
index dadaf35..d71f0b3 100755
--- a/scripts/active_edit
+++ b/scripts/active_edit
@@ -14,7 +14,6 @@ import os
 import pathlib
 import re
 import sys
-import urllib.request
 
 import cve_lib
 import source_map
@@ -72,99 +71,6 @@ def release_wants_dne(release):
     _, product, _, _ = cve_lib.get_subproject_details(release)
     return product != None and product == cve_lib.PRODUCT_UBUNTU
 
-def fetch_kernel_fixes(url):
-    '''Downloads a kernel commit and returns a list of break-fixes'''
-    commit_hash = None
-    fixes = []
-
-    # Strip off comment at the end
-    if ' ' in url:
-        url = url.split(' ')[0]
-
-    # Short URL, turn it into long one
-    if url.startswith('https://git.kernel.org/linus/'):
-        url = url.replace('https://git.kernel.org/linus/',
-                          'https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=')
-    if url.startswith('https://git.kernel.org/stable/c/'):
-        url = url.replace('https://git.kernel.org/stable/c/',
-                          'https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=')
-
-    # Get the raw patch
-    url = url.replace('/commit/', '/patch/')
-
-    with urllib.request.urlopen(url) as response:
-       patch = response.read().decode('utf-8')
-
-    for line in patch.split("\n"):
-        if re.match("commit [0-9a-f]{40} upstream.", line):
-            # This is an LTS backport, skip it
-            return []
-        if re.match("\[ Upstream commit [0-9a-f]{40} \]", line):
-            # This is an LTS backport, skip it
-            return []
-        if not commit_hash and line.startswith("From "):
-            commit_hash = line.split(' ')[1]
-            continue
-        elif line.startswith("Fixes: "):
-            fix_hash = line.split(' ')[1]
-            fixes.append([fix_hash, commit_hash])
-
-    # If we didn't find a Fixes tag, just use -
-    if fixes == []:
-        fixes.append(['-', commit_hash])
-
-    return fixes
-
-def in_break_fixes(commit, break_fixes):
-    '''See if a commit is in the hash_list'''
-    # properly handle comparing short and long hashes
-    for [break_hash,fix_hash] in break_fixes:
-        if commit.startswith(break_hash):
-            return True
-        if break_hash.startswith(commit):
-            return True
-    return False
-
-def get_long_kernel_hash(short_hash):
-    '''Attempts to get a long kernel hash'''
-
-    url = 'https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/patch/?id=' + short_hash
-    with urllib.request.urlopen(url) as response:
-       patch = response.read().decode('utf-8')
-
-    for line in patch.split("\n"):
-        if line.startswith("From "):
-            commit_hash = line.split(' ')[1]
-            if commit_hash.startswith(short_hash):
-                return commit_hash
-
-    return short_hash
-
-
-def validate_kernel_fixes(break_fixes):
-    '''Validate list of break-fixes'''
-
-    if break_fixes == []:
-        return []
-
-    # Make sure a breaks URL wasn't listed in the URLs by mistake
-    validated = []
-    for [break_hash,fix_hash] in break_fixes:
-        # Don't check this for now, it can result in false positives
-        #if not in_break_fixes(fix_hash, break_fixes):
-        if True:
-            if break_hash != '-' and len(break_hash) < 40:
-                break_hash = get_long_kernel_hash(break_hash)
-            # Make sure it's not a dupe
-            dupe = False
-            for [v_break_hash,v_fix_hash] in validated:
-                if break_hash == v_break_hash and fix_hash == v_fix_hash:
-                    dupe = True
-            if not dupe:
-                validated.append([break_hash, fix_hash])
-
-    return validated
-
 def _add_pkg(p, fp, fixed, parent, embargoed, break_fixes):
     print('', file=fp)
     print('Patches_%s:' % p, file=fp)
@@ -291,8 +197,8 @@ def create_or_update_cve(cve, packages, priority=None, bug_urls=None,
                 url.startswith('https://git.kernel.org/stable/c/') or
                 url.startswith('https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=') or
                 url.startswith('https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=')):
-                break_fixes += fetch_kernel_fixes(url)
-        break_fixes = validate_kernel_fixes(break_fixes)
+                break_fixes += cve_lib.fetch_kernel_fixes(url)
+        break_fixes = cve_lib.validate_kernel_fixes(break_fixes)
 
     # collect notes from pkg_db and add any extra pkgs from pkg_db as well
     notes = []
diff --git a/scripts/check-syntax b/scripts/check-syntax
index e5c909f..f89a0e4 100755
--- a/scripts/check-syntax
+++ b/scripts/check-syntax
@@ -475,6 +475,19 @@ def fixup_entry_wrong(filename, pkg, rel):
 
     cve_lib.update_state(filename, pkg, rel, status, None)
 
+def fixup_entry_missing_break_fix(filename, pkg, ref):
+    urls = cve_lib.fetch_kernel_fixes(ref)
+    urls = cve_lib.validate_kernel_fixes(urls)
+
+    for url in urls:
+        # convert to the break-fix format as a string
+        url = " ".join(url)
+        if opt.dry_run:
+            print("Dry-Run: adding break-fix %s, %s, to %s" % (filename, pkg, url))
+            return
+
+        cve_lib.add_patch(filename, pkg, url, "break-fix")
+
 def get_cve_path(cve, rel):
 
     cve = os.path.basename(cve)
@@ -1132,6 +1145,25 @@ def check_cve(cve):
                     )
                     cve_okay = False
 
+    # if there is a reference URL to a kernel commit then check there is a
+    # break-fix entry against the linux package
+    if "References" in data:
+        for ref in data["References"].split("\n"):
+            if "git.kernel.org" in ref:
+                # the CVE needs to already be triaged against linux and hence
+                # have a patches entry for it
+                if "linux" in data["patches"] and len(data["patches"]["linux"]) == 0:
+                    filename = srcmap["References"][0] if "References" in srcmap else cvepath
+                    linenum = srcmap["References"][1] if "References" in srcmap else 1
+                    print(
+                        "%s: %d: missing break-fix entry for kernel commit"
+                        % (filename, linenum),
+                        file=sys.stderr,
+                    )
+                    cve_okay = False
+                    if opt.autofix:
+                        fixup_entry_missing_break_fix(filename, "linux", ref)
+
     for entry in data["CVSS"]:
         srcname = entry['source']
         filename = srcmap["CVSS"][srcname][0]
diff --git a/scripts/cve_lib.py b/scripts/cve_lib.py
index c97cae6..416d5f7 100755
--- a/scripts/cve_lib.py
+++ b/scripts/cve_lib.py
@@ -22,6 +22,8 @@ import time
 import cache_urllib
 import json
 import yaml
+import urllib.error
+import urllib.request
 
 from functools import reduce
 
@@ -3456,3 +3458,113 @@ def wrap_text(text, width=75):
     Wrap text to width chars wide.
     """
     return wordwrap(text, width).replace(' \n', '\n')
+
+def fetch_kernel_fixes(url):
+    '''Downloads a kernel commit and returns a list of break-fixes'''
+    commit_hash = None
+    fixes = []
+
+    # Strip off comment at the end
+    if ' ' in url:
+        url = url.split(' ')[0]
+
+    # Short URL, turn it into long one
+    if url.startswith('https://git.kernel.org/linus/'):
+        url = url.replace('https://git.kernel.org/linus/',
+                          'https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=')
+    if url.startswith('https://git.kernel.org/stable/c/'):
+        url = url.replace('https://git.kernel.org/stable/c/',
+                          'https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=')
+    # old URL style - replace to be more modern
+    if url.startswith('http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h='):
+        url = url.replace('http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=',
+                          'https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=')
+
+    # Get the raw patch
+    url = url.replace('/commit/', '/patch/')
+
+    try:
+        with urllib.request.urlopen(url) as response:
+            patch = response.read().decode('utf-8')
+    except urllib.error.HTTPError as e:
+        print("WARNING: Failed to fetch patch URL %s: %s" % (url, str(e)), file=sys.stderr)
+        return fixes
+
+    backport_re = re.compile(r"(commit [0-9a-f]{40} upstream.|\[ Upstream commit [0-9a-f]{40} \])")
+    for line in patch.split("\n"):
+        # stop early if we have reached the main patch body
+        if line.startswith("---"):
+            break
+        if backport_re.match(line):
+            # This is an LTS backport, skip it
+            return []
+        if not commit_hash and line.startswith("From "):
+            commit_hash = line.split(' ')[1]
+            continue
+        elif line.startswith("Fixes: "):
+            fix_hash = line.split(' ')[1]
+            fixes.append([fix_hash, commit_hash])
+
+    if commit_hash is None:
+        print("Failed to get commit hash from %s" % url, file=sys.stderr)
+        return []
+
+    # If we didn't find a Fixes tag, just use -
+    if fixes == []:
+        fixes.append(['-', commit_hash])
+
+    return fixes
+
+def in_break_fixes(commit, break_fixes):
+    '''See if a commit is in the hash_list'''
+    # properly handle comparing short and long hashes
+    for [break_hash,fix_hash] in break_fixes:
+        if commit.startswith(break_hash):
+            return True
+        if break_hash.startswith(commit):
+            return True
+    return False
+
+def get_long_kernel_hash(short_hash):
+    '''Attempts to get a long kernel hash'''
+
+    url = 'https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/patch/?id=' + short_hash
+    with urllib.request.urlopen(url) as response:
+       patch = response.read().decode('utf-8')
+
+    for line in patch.split("\n"):
+        if line.startswith("From "):
+            commit_hash = line.split(' ')[1]
+            if commit_hash.startswith(short_hash):
+                return commit_hash
+
+    return short_hash
+
+
+def validate_kernel_fixes(break_fixes):
+    '''Validate list of break-fixes'''
+
+    if break_fixes == []:
+        return []
+
+    # Make sure a breaks URL wasn't listed in the URLs by mistake
+    validated = []
+    for [break_hash,fix_hash] in break_fixes:
+        # Don't check this for now, it can result in false positives
+        #if not in_break_fixes(fix_hash, break_fixes):
+        if True:
+            if break_hash is None or fix_hash is None:
+                continue
+            if break_hash != '-' and len(break_hash) < 40:
+                break_hash = get_long_kernel_hash(break_hash)
+            # Make sure it's not a dupe
+            dupe = False
+            for [v_break_hash,v_fix_hash] in validated:
+                if break_hash == v_break_hash and fix_hash == v_fix_hash:
+                    dupe = True
+            if not dupe:
+                validated.append([break_hash, fix_hash])
+
+    return validated
+
+