Merge ~rodrigo-zaiden/ubuntu-cve-tracker:kernel_cna_cves_usn into ubuntu-cve-tracker:master

Proposed by Rodrigo Figueiredo Zaiden
Status: Needs review
Proposed branch: ~rodrigo-zaiden/ubuntu-cve-tracker:kernel_cna_cves_usn
Merge into: ubuntu-cve-tracker:master
Diff against target: 405 lines (+249/-12)
8 files modified
README.linux (+7/-3)
meta_lists/kernel_paths_overrides.json (+40/-0)
scripts/cve_lib.py (+4/-6)
scripts/kernel_lib.py (+100/-1)
scripts/prepare-kernel-usn.py (+8/-1)
scripts/pull-usn-desc.py (+8/-0)
scripts/sis-generate-usn (+52/-0)
scripts/test_kernel_lib.py (+30/-1)
Reviewer Review Type Date Requested Status
Alex Murray Approve
Steve Beattie Pending
Review via email: mp+462266@code.launchpad.net

Commit message

scripts: kernel: add automation to generate kernel USN

 adding a new parameter: '--kernel-cna-cves' that, when used, will automatically
 find which CVEs where published by the Kernel CNA and will generate a standard
 paragraph for the USN based on the affected subsystems in the Linux kernel based
 on the fix commits for each CVE.

Description of the change

When the Kernel CNA started to publish Linux Kernel CVEs, we noticed that a
big amount of new CVEs are coming each day. With that, it is much harder to
keep track of the USN descriptions as we were used to.

So, the idea with this series of commits, is that, when we want to generate
a Kernel USN with the script 'prepare-kenel-usn.py' we can now use the
argument '--kernel-cna-cves' that will go over the CVEs that would be included
in the Kernel USN and will check which were published by the Kernel CNA and will
add a standard paragraph will the affected subsystems.
It sounds like:

   Several security issues were discovered in the Linux kernel.
   An attacker could possibly use these to compromise the system.
   This update corrects flaws in the following subsystems:
      - Architecture specifics;
      - Block layer;
      - ACPI drivers;
      ....

A few more detailed information on how it works in this proposal:
- It starts with passing '--kernel-cna-cves' to 'prepare-kenel-usn.py', like:
    ./scripts/prepare-kernel-usn.py -d -n -p Proposed --kernel-cna-cves jammy jammy/linux-oem-6.1: 6.1.0-1035.3

- It them invokes the 'sis-generate-usn' script passing along the argument
  --kernel-cna-cves, that will do the most importants checks:
  1) it check which CVEs that were found being fixed in this kernel release
     were published by the Kernel CNA by trying to verify if the CVE file exists
     in the https://git.kernel.org/pub/scm/linux/security/vulns.git/ repository.
     It checks locally if the repository is cloned (more information on that
     was added to README.linux) and if it does not find it locally it checks
     on the repository online: the local repository could be outdated, or it
     really does not exist. This is implemented in cve_lib.py in the method
     is_kernel_cna_cve().

  2) it them checks which are the fix commits for the CVEs, implemented in
     cve_lib.py in method get_kernel_break_fix()

  3) it finally look for the affected subsystems based on the fix commit on a
     kernel repository local tree. If the repo is not found, it returns an error.
     This is in cve_lib.py in method get_affected_subsystems().

  With the affected subsystems in hand, it generated the paragraph with the
  template above. In get_affected_subsystems() there is a mapping for the
  subsystem paths based on the kernel hierarchy that translates the directory
  into human readable/understandable topics.

A way to run this for current kernels is:
./scripts/prepare-kernel-usn.py -d -n -p Security --ignore-released-cves-in-changelog --kernel-cna-cves jammy jammy/linux-oem-6.1: 6.1.0-1035.35 --add CVE-2023-46838 --add CVE-2023-50431 --add CVE-2023-52436 --add CVE-2023-52438 --add CVE-2023-52439 --add CVE-2023-52443 --add CVE-2023-52444 --add CVE-2023-52445 --add CVE-2023-52447 --add CVE-2023-52448 --add CVE-2023-52449 --add CVE-2023-52451 --add CVE-2023-52454 --add CVE-2023-52456 --add CVE-2023-52457 --add CVE-2023-52458 --add CVE-2023-52462 --add CVE-2023-52463 --add CVE-2023-52464 --add CVE-2023-52467 --add CVE-2023-52469 --add CVE-2023-52470 --add CVE-2023-5633 --add CVE-2023-6610 --add CVE-2024-0340 --add CVE-2024-1085 --add CVE-2024-1086 --add CVE-2024-23849 --add CVE-2024-24860 --add CVE-2024-26581 --add CVE-2024-26588 --add CVE-2024-26589 --add CVE-2024-26591 --add CVE-2024-26592 --add CVE-2024-26594 --add CVE-2024-26597 --add CVE-2024-26598 --add CVE-2024-26599 --add CVE-2024-26600 --add CVE-2024-26601 --bypass-usn-check

or, check the pastebins to see how the script is outputing in stdout and in the USN bash for the above command:
prepare-kernel-usn log: https://pastebin.canonical.com/p/WxS4TqdXQx/
USN bash script: https://pastebin.canonical.com/p/5dhcf83xCx/

To post a comment you must log in.
Revision history for this message
Alex Murray (alexmurray) wrote :

This sounds like a great quality-of-life improvement for doing kernel USNs - I wonder though if instead of hard-coding the mapping of subsystem paths to descriptions, whether you could use the contents of MAINTAINERS (via ./scripts/get_maintainer.pl --subsystem -f $path) and then fallback to this if needed?

2d50af1... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: make use of read_uct_config

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

35a5c02... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: improve get_kernel_break_fix

 break-fix commits can be in a variety of formats that were not being
 considered, besides not filtering out '-' that is used when there is
 no specific commit for either the break or fix commit.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Revision history for this message
Rodrigo Figueiredo Zaiden (rodrigo-zaiden) wrote :

Hi Alex,

> This sounds like a great quality-of-life improvement for doing kernel USNs - I
> wonder though if instead of hard-coding the mapping of subsystem paths to
> descriptions, whether you could use the contents of MAINTAINERS (via
> ./scripts/get_maintainer.pl --subsystem -f $path) and then fallback to this if
> needed?

I really like your idea, I think that we could get ride of the hard-coding style
and move to a better way. I've tried a few things outside of this tree in order
to find a way of using the get_maintainer.pl that I'd like to ask your review,
if possible, please.

The suggestion is in this standalone python file that can be ran outside of
the tree without any inputs:
https://pastebin.canonical.com/p/jyKc8mVw3q/

The output for it is at:
https://pastebin.canonical.com/p/jBc5vVvQQX/

Mainly, I'm running the get_maintainer as:
 ./scripts/get_maintainer.pl --subsystem --sections -f $file_path
and filtering out every line that is not a subsystem

What is not really nice:
- It is not yet ordered by kernel tree hierarchy;
- We have a few "subsystems" that are not really useful like "BPF", "NETWORKING"
  that are part of the output because I'm getting everything from the
  'get_maintainer' besides "THE REST"
- It is a bit slow;

Do you like this output style or/and have other suggestions, please?
Really appreciate it.
Thank you very much!

fa7076b... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: move linux_kernel_path verification

 the linux_kernel_path config verification was inside the interation
 when I was trying to implement a second stage verification that would
 check for the commit online, but as this approach was dropped we can
 check it once before running through the commits.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

b265fe6... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: handle git command errors

 make use of the cmd() method defined in cve_lib and add handle for
 possible git errors.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

bb96afc... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Revision history for this message
Mark Esler (eslerm) wrote :

This looks great.

The points you raised about an ordered hierarchy and useful subsystem would be nice. Maybe those could be solved together?

Personally, I'm trying to move code out of cve_lib.py. Perhaps that's not called for here or others on the team want to extend cve_lib. A kernel_cve.py or similarly named script might be an easier way to maintain helper functions for parsing kernel cve data.

Revision history for this message
Steve Beattie (sbeattie) wrote :

Apologies, I haven't look at the details of the merge proposal very closely.

Using get_maintainer.pl would be ideal, and perhaps you'd wnat to use something like following to simplify parsing the output:

  ./scripts/get_maintainer.pl --separator ':' --subsystem [...]

so that the subsystems are all on one line, and then pulling out the first entry, to get the most specific subsystem.

My concern with this approach is that it might turn out to be like how we end up writing descriptions for source pacakges in our USNs via the package_info_overrides.json file, because the description summaryies in the debian pacakges are not always ina state usable for general public consumption. My experiments with some sample files are okay, but there are things like:

$ ./scripts/get_maintainer.pl --separator ':' --subsystem drivers/md/dm-ioctl.c | tail -1
DEVICE-MAPPER (LVM):THE REST

and

$ ./scripts/get_maintainer.pl --separator \; --subsystem -f drivers/net/dsa/vitesse-vsc73xx-core.c | tail -1
NETWORKING [DSA];NETWORKING DRIVERS;THE REST

so at a minimum, we might want an override mechanism anyway.

Revision history for this message
Steve Beattie (sbeattie) wrote :

(sorry, in my examples, I switched up the separators due to experimenting with different ones in my shell history.)

629b8cc... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: make load json file overrides more generic

 with the idea of using the method with other json files, this commit
 renames 'load_package_info_overrides' to 'load_json_file_overrides'

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

a375e3f... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: move hardcode mapping to json file

 instead of using a hardcode mapping, it seems better to use a json
 file with the overrides, like we do for package names and titles,
 seems easier to view and cleaner to maintain.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

a461e5f... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: trust that linux_cna_tracker is updated

 for the sake of performance, instead of falling back to a online search
 every time a CVE is not found in the kernel cna repo tracker, only
 fallback if the repository itself is not found.
 we will trust that the local repository is updated.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Revision history for this message
Rodrigo Figueiredo Zaiden (rodrigo-zaiden) wrote :

Hi Mark,
> This looks great.
>
> The points you raised about an ordered hierarchy and useful subsystem would be
> nice. Maybe those could be solved together?
>

I'm having a few issues with the get_maintainer approach that I'll share and
propose a solution in another comment.

> Personally, I'm trying to move code out of cve_lib.py. Perhaps that's not
> called for here or others on the team want to extend cve_lib. A kernel_cve.py
> or similarly named script might be an easier way to maintain helper functions
> for parsing kernel cve data.

This is actually a good idea, I started working with 'kernel_lib.py' but decided
to move to 'cve_lib.py' as 'kernel_lib.py' is not really for CVEs but more for
kernel versions. but I didn't think of using a new 'kernel_cve.py' where we
could keep kernel CVE related matters. I'll just finish this round of updates
and will try this suggestion, thanks for it.

Revision history for this message
Rodrigo Figueiredo Zaiden (rodrigo-zaiden) wrote :

General Notes:
I'm personally inclined to vote for us to move out of the usage of
get_maintainer.pl to generate the list of affected subsystem for two reasons:

1) performance: this would be the main reason. unless, we can find a different
approach on how to run the 'get_maintainer.pl' script, it is proving to be
too much slow to get the subsystem information for each file on each commit.
I have examples of time constrains below.

2) quality of text: don't really know how to name this, but I'm not in favor of
having something like "EDAC-CAVIUM THUNDERX" and "EDAC-CORE" (as the result of
get_maintainer.pl for "drivers/edac/thunderx_edac.c") where we could instead
have "EDAC drivers" (for "drivers/edac/").

So, in the end, I personally don't think maintaining the hardcode bits is that
big of an issue. Considering that, from now on the idea is that the Kernel CNA
will be assigning most of the kernel CVEs, we would be moving from writing USN
descriptions to writing hierarch descriptions...

One thing that I'm proposing with the next commits is to move from a hardcode
mapping to a json meta file as a result of an input from Steve's comment
earlier.

Testing:
I've put together a better scenario for testing both approaches:

# move to my branch:
 $ git remote add rodrigo-zaiden git+ssh://<email address hidden>/~rodrigo-zaiden/ubuntu-cve-tracker
 $ git checkout rodrigo-zaiden/kernel_cna_cves_usn

# standalone script using get_maintainer.pl with:
 ./scripts/get_maintainer.pl --separator ':' --subsystem -f <file path>
https://pastebin.canonical.com/p/sGTN76wDN3/

 $ time python3 test-kernel_cna.py
 ...
  - ARM/FREESCALE IMX / MXC ARM ARCHITECTURE;
  - ARM64 PORT (AARCH64 ARCHITECTURE);
  - BLOCK LAYER;
  - BLUETOOTH SUBSYSTEM;
  - BPF JIT for LOONGARCH;
  - BPF [CORE];
  - BPF [GENERAL] (Safe Dynamic Programs and Tools);
 ...
 real 0m36,722s
 user 0m33,750s
 sys 0m2,888s

full results: https://pastebin.canonical.com/p/bprVtW7gb5/

# standalone script using json meta file:
https://pastebin.canonical.com/p/KFQRFpNdXG/

 $ time python3 test_kernel-cna-overrides.py
 ...
   - EDAC drivers;
   - GPU drivers;
   - Media drivers;
   - Multifunction device drivers;
   - MTD block device drivers;
 ...
 real 0m0,963s
 user 0m0,849s
 sys 0m0,113s

full results: https://pastebin.canonical.com/p/BV2qYc66tr/

Revision history for this message
Alex Murray (alexmurray) wrote :

Thanks for investigating the get_maintainer.pl approach - it's a shame it was so unwieldy in practice and IMO day-to-day usability trumps maintainership - ie. so the json overrides approach sounds like the better option. LGTM!

review: Approve
3a5fdde... by Rodrigo Figueiredo Zaiden

meta_lists/kernel_paths_overrides.json: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

831742f... by Rodrigo Figueiredo Zaiden

scripts/kernel: move kernel-cna-cves methods to kernel_lib

 after a reconsideration, it would be better to start moving the kernel
 related methods to kernel_lib instead of over populating cve_lib.
 there is a few other methods related to kernel in cve_lib that should
 be moved later, for now, moving only the ones related to the new
 argument, --kernel-cna-cves.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

5d86515... by Rodrigo Figueiredo Zaiden

scripts/test_kernel_lib.py: add tests for kernel cna cves methods

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

2a1444a... by Rodrigo Figueiredo Zaiden

scripts/kernel_lib.py: add directory fallback

 while running the tests, noticed that the subsequent tests after
 the kernel ones were failing because the directory was being moved
 elsewhere. added a chdir to go back to the previous and added a test
 for the failure code path.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

8031869... by Rodrigo Figueiredo Zaiden

scripts/test_kernel_lib.py: add condition to skip test

 if there is not configuration for the linux kernel tree, we should
 skip testing. that would be the case mainly for the servers, or,
 having the linux kernel tree would be the case only for local runs.
 not sure if that is something we could mock, don't really think so.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Unmerged commits

8031869... by Rodrigo Figueiredo Zaiden

scripts/test_kernel_lib.py: add condition to skip test

 if there is not configuration for the linux kernel tree, we should
 skip testing. that would be the case mainly for the servers, or,
 having the linux kernel tree would be the case only for local runs.
 not sure if that is something we could mock, don't really think so.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Succeeded
[SUCCEEDED] unit-tests:0 (build)
[SUCCEEDED] check-cves:0 (build)
12 of 2 results
2a1444a... by Rodrigo Figueiredo Zaiden

scripts/kernel_lib.py: add directory fallback

 while running the tests, noticed that the subsequent tests after
 the kernel ones were failing because the directory was being moved
 elsewhere. added a chdir to go back to the previous and added a test
 for the failure code path.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

5d86515... by Rodrigo Figueiredo Zaiden

scripts/test_kernel_lib.py: add tests for kernel cna cves methods

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

831742f... by Rodrigo Figueiredo Zaiden

scripts/kernel: move kernel-cna-cves methods to kernel_lib

 after a reconsideration, it would be better to start moving the kernel
 related methods to kernel_lib instead of over populating cve_lib.
 there is a few other methods related to kernel in cve_lib that should
 be moved later, for now, moving only the ones related to the new
 argument, --kernel-cna-cves.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

3a5fdde... by Rodrigo Figueiredo Zaiden

meta_lists/kernel_paths_overrides.json: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

a461e5f... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: trust that linux_cna_tracker is updated

 for the sake of performance, instead of falling back to a online search
 every time a CVE is not found in the kernel cna repo tracker, only
 fallback if the repository itself is not found.
 we will trust that the local repository is updated.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

a375e3f... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: move hardcode mapping to json file

 instead of using a hardcode mapping, it seems better to use a json
 file with the overrides, like we do for package names and titles,
 seems easier to view and cleaner to maintain.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

629b8cc... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: make load json file overrides more generic

 with the idea of using the method with other json files, this commit
 renames 'load_package_info_overrides' to 'load_json_file_overrides'

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

bb96afc... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

b265fe6... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: handle git command errors

 make use of the cmd() method defined in cve_lib and add handle for
 possible git errors.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/README.linux b/README.linux
2index 48dae64..ca1ec75 100644
3--- a/README.linux
4+++ b/README.linux
5@@ -58,12 +58,16 @@ common ones are the
6 - [Debian's kernel security tracker][debian-kernel]
7 - Nicholas Leudtke's [linux github tracker][linux-github]
8
9-The latter two can be configured as references in
10+The three of them can be configured as references in
11 `~/.ubuntu-cve-tracker.conf` for use with tools, via the
12-`debian_kernel_cve_tracker` and `linux_kernel_cve_tracker` variables
13-respectively. An example:
14+`linux_cna_tracker`, `debian_kernel_cve_tracker` and
15+`linux_kernel_cve_tracker` variables respectively. An example:
16
17 ```
18+# path to a copy of the upstream kernel CNA tracker from
19+# https://git.kernel.org/pub/scm/linux/security/vulns.git/
20+linux_cna_tracker='/path/to/git/cve_trackers/linux-cna/'
21+
22 # path to a copy of the linux kernel cve tracker from
23 # https://github.com/nluedtke/linux_kernel_cves.git
24 linux_kernel_cve_tracker='/path/to/git/cve_trackers/linux_kernel_cves/'
25diff --git a/meta_lists/kernel_paths_overrides.json b/meta_lists/kernel_paths_overrides.json
26new file mode 100644
27index 0000000..86fde02
28--- /dev/null
29+++ b/meta_lists/kernel_paths_overrides.json
30@@ -0,0 +1,40 @@
31+{
32+ "arch/": "Architecture specifics",
33+ "block/": "Block layer",
34+ "crypto/": "Cryptographic API",
35+ "drivers/acpi": "ACPI drivers",
36+ "drivers/android/": "Android drivers",
37+ "drivers/edac/": "EDAC drivers",
38+ "drivers/gpu/": "GPU drivers",
39+ "drivers/i2c/": "I2C subsystem",
40+ "drivers/infiniband/": "InfiniBand drivers",
41+ "drivers/media/": "Media drivers",
42+ "drivers/mfd/": "Multifunction device drivers",
43+ "drivers/mtd/": "MTD block device drivers",
44+ "drivers/net/": "Network drivers",
45+ "drivers/nvme/": "NVME drivers",
46+ "drivers/phy/": "PHY drivers",
47+ "drivers/pwm/": "PWM drivers",
48+ "drivers/scsi/": "SCSI drivers",
49+ "drivers/spmi/": "SPMI drivers",
50+ "drivers/tty/": "TTY drivers",
51+ "drivers/uio/": "Userspace I/O drivers",
52+ "fs/ceph/": "Ceph distributed file system",
53+ "fs/efivarfs/": "EFI Variable file system",
54+ "fs/ext4/": "Ext4 file system",
55+ "fs/f2fs/": "F2FS file system",
56+ "fs/gfs2/": "GFS2 file system",
57+ "fs/jfs/": "JFS file system",
58+ "fs/smb/": "SMB network file system",
59+ "kernel/bpf/": "BPF subsystem",
60+ "kernel/": "Core kernel",
61+ "net/core/": "Networking core",
62+ "net/ipv6/": "IPv6 Networking",
63+ "net/ipv4/": "IPv4 Networking",
64+ "net/llc/": "Logical Link Layer",
65+ "net/netfilter/": "Netfilter",
66+ "net/openvswitch/": "Open vSwitch",
67+ "net/sched/": "Network Traffic Control",
68+ "net/unix/": "Unix domain sockets",
69+ "security/apparmor/": "AppArmor security module"
70+}
71diff --git a/scripts/cve_lib.py b/scripts/cve_lib.py
72index 2ea85bb..199dc26 100755
73--- a/scripts/cve_lib.py
74+++ b/scripts/cve_lib.py
75@@ -2771,13 +2771,13 @@ def load_boilerplates():
76 data[alias]["aliases"] = sorted(list(aliases[alias]))
77 return data
78
79-# for sanity, try to keep these in alphabetical order in the json file
80-def load_package_info_overrides():
81- with open(os.path.join(meta_dir, "package_info_overrides.json"), "r") as fp:
82+def load_json_file_overrides(json_file):
83+ with open(os.path.join(meta_dir, json_file), "r") as fp:
84 data = json.load(fp)
85 return data
86
87-package_info_overrides = load_package_info_overrides()
88+# for sanity, try to keep these in alphabetical order in the json file
89+package_info_overrides = load_json_file_overrides("package_info_overrides.json")
90
91 def load_package_db():
92 pkg_db = load_boilerplates()
93@@ -3705,5 +3705,3 @@ def validate_kernel_fixes(break_fixes):
94 validated.append([break_hash, fix_hash])
95
96 return validated
97-
98-
99diff --git a/scripts/kernel_lib.py b/scripts/kernel_lib.py
100index 3fa7c0c..c963a2a 100755
101--- a/scripts/kernel_lib.py
102+++ b/scripts/kernel_lib.py
103@@ -14,9 +14,14 @@
104 from __future__ import print_function
105
106 import sys
107+import os
108+import urllib.error
109+import urllib.request
110+from uct.config import read_uct_config
111
112 # XXX kernel_srcs probably belongs here, too
113-from cve_lib import (kernel_srcs, get_esm_name, is_active_esm_release)
114+from cve_lib import (kernel_srcs, get_esm_name, is_active_esm_release, \
115+ load_cve, find_cve, load_json_file_overrides, cmd)
116
117
118 # search for the kernel SRU cycle for a kernel in the format
119@@ -1517,3 +1522,97 @@ def kernel_package_version(version):
120 # XXX may need to add an offset exception to the kernel table
121 def kernel_meta_abi(version, offset=3):
122 return int(version.split('.').pop(offset))
123+
124+
125+def get_kernel_break_fix(cve):
126+ '''Return the break-fix commits of a giver kernel CVE'''
127+
128+ break_commits = []
129+ fix_commits = []
130+
131+ cve_data = load_cve(find_cve(cve))
132+ if cve_data and cve_data['patches']['linux']:
133+ for (patch_type, break_fix) in cve_data['patches']['linux']:
134+ if patch_type == 'break-fix':
135+ break_commit, fix_commit = break_fix.split()[:2]
136+
137+ for entry in break_commit.split('|'):
138+ if entry != '-':
139+ break_commits.append(entry)
140+ for entry in fix_commit.split('|'):
141+ if entry != '-':
142+ fix_commits.append(entry)
143+
144+ return break_commits, fix_commits
145+
146+
147+def is_kernel_cna_cve(cve):
148+ '''Check if a given CVE was published by the Kernel CNA'''
149+
150+ year = cve.split('-')[1]
151+
152+ config = read_uct_config()
153+ if 'linux_cna_tracker' in config:
154+ if os.path.exists(os.path.join(config['linux_cna_tracker'], "cve", "published", year, cve)):
155+ return True
156+ return False
157+
158+ print("WARNING: Couldn't find linux_cna_tracker repo, will try to verify it online." , file=sys.stderr)
159+
160+ url = ("https://git.kernel.org/pub/scm/linux/security/vulns.git/tree/cve/published/%s/%s" % (year, cve))
161+
162+ try:
163+ with urllib.request.urlopen(url) as response:
164+ if response.getcode() == 200:
165+ return True
166+ else:
167+ print("WARNING: Failed to get OK response from URL %s: %s" % (url, response.getcode()), file=sys.stderr)
168+ return False
169+ except urllib.error.HTTPError as e:
170+ print("WARNING: Failed to fetch patch URL %s: %s" % (url, str(e)), file=sys.stderr)
171+ return False
172+
173+
174+def get_affected_subsystems(commits):
175+ '''Return the kernel subsystems affected by a list of commits'''
176+
177+ config = read_uct_config()
178+ if 'linux_kernel_path' not in config:
179+ raise ValueError("linux_kernel_path not configured, please read README.linux")
180+
181+ subsystems = {}
182+ other_files = []
183+ kernel_paths_overrides = load_json_file_overrides("kernel_paths_overrides.json")
184+
185+ pwd = os.getcwd()
186+ os.chdir(config['linux_kernel_path'])
187+
188+ for _commits in commits:
189+ for commit in _commits:
190+
191+ rc, output = cmd(['git', 'show', '--name-only', '--pretty=format:', commit])
192+ if rc !=0:
193+ os.chdir(pwd)
194+ raise ValueError('failed to get file names from commit %s:\n%s' % (commit, output))
195+
196+ files_changed = output.splitlines()
197+ for file_path in files_changed:
198+ matched_subsystem = False
199+ for kernel_path in kernel_paths_overrides:
200+ if kernel_path in file_path:
201+ if kernel_paths_overrides[kernel_path] not in subsystems:
202+ subsystems[kernel_paths_overrides[kernel_path]] = set()
203+ subsystems[kernel_paths_overrides[kernel_path]].add(file_path)
204+ matched_subsystem = True
205+ break
206+ if not matched_subsystem:
207+ if file_path not in other_files:
208+ other_files.append(file_path)
209+
210+ os.chdir(pwd)
211+
212+ # sort alphabetically by kernel hierarchy tree
213+ sorted_subsystems = {key: sorted(value) for key, value in sorted(subsystems.items(), key=lambda item: sorted(item[1]))}
214+ other_files.sort()
215+
216+ return sorted_subsystems, other_files
217diff --git a/scripts/prepare-kernel-usn.py b/scripts/prepare-kernel-usn.py
218index ee11cb9..7ec14fd 100755
219--- a/scripts/prepare-kernel-usn.py
220+++ b/scripts/prepare-kernel-usn.py
221@@ -176,6 +176,7 @@ parser.add_argument('--use-changes', action='store', default=None, help='Use pre
222 parser.add_argument('--keep-changes', action='store_true', default=False, help='Keep changes files downloaded by sis-changes')
223 parser.add_argument("--ignore-released-cves-in-changelog", action='store_true', help="Filter out CVEs already marked as released")
224 parser.add_argument("--esm-ppa", action='store', help="Add kernels from ESM PPA if any, can be used when merging ESM with active kernels (sets --include-eol)")
225+parser.add_argument("--kernel-cna-cves", action='store_true', help="Check for Kernel CNA CVEs and add a paragraph for that")
226 parser.add_argument('release', action='store', nargs=1, help='Primary release name (e.g. xenial)')
227 parser.add_argument('kernel', action=KernelVersionAction, nargs='+', help='Kernel source package name and versions; e.g. "linux 4.4.0-42.62. Source package can be a release/name pair"')
228 args = parser.parse_args()
229@@ -214,7 +215,7 @@ for release in cve_lib.release_sort(kernels.keys()):
230
231 if not args.embargoed:
232 rc = display_pending_cves(release, kernel, last_usn_version, version, args.add)
233- if not rc == 0 and not args.force:
234+ if not rc == 0 and not (args.force or args.kernel_cna_cves):
235 print("Incomplete descriptions detected, please fix.")
236 exit(1)
237
238@@ -321,9 +322,15 @@ try:
239 if args.ignore_released_cves_in_changelog:
240 cmd += ['--ignore-released-cves-in-changelog']
241
242+ if args.kernel_cna_cves:
243+ cmd += ['--kernel-cna-cves']
244+
245 if args.embargoed:
246 cmd += ['--embargoed']
247
248+ if args.debug:
249+ cmd += ['--debug']
250+
251 cmd += ['--filter-bins', generate_usn_regex, usn]
252 for release in kernels:
253 cmd += ['--binaries-json', '%s-binaries.json' % release]
254diff --git a/scripts/pull-usn-desc.py b/scripts/pull-usn-desc.py
255index 6bb6ce1..71354f9 100755
256--- a/scripts/pull-usn-desc.py
257+++ b/scripts/pull-usn-desc.py
258@@ -42,6 +42,7 @@ opter.add_option("--releases", help="List of releases CVEs affect to be filter u
259 opter.add_option("--this-only-affected", help="Makes this only affected feature optional", action='store_true')
260 opter.add_option("--src", help="The package source to be check through if --this-only-affected is used", action='append', default=[])
261 opter.add_option("--embargoed", help="Use the embargoed tree to look for desctiptions in addition", action='store_true')
262+opter.add_option("--kernel-cna-cves", help="List of CVEs from the Kernel CNA", default=None)
263 (opt, args) = opter.parse_args()
264
265 cves = dict()
266@@ -106,6 +107,13 @@ for cve in opt.cve + args:
267 chunks = cves[cve]
268
269 desc = chunks['Ubuntu-Description'].strip()
270+
271+ # we don't want to include the empty description CVEs that are part of
272+ # the list from the Kernel CNA but we still may want to follow the
273+ # normal process for those that have a description.
274+ if len(desc) == 0 and cve in opt.kernel_cna_cves:
275+ continue
276+
277 if len(desc) == 0:
278 rc = 1
279 disc = chunks.get('Discovered-by', '').strip()
280diff --git a/scripts/sis-generate-usn b/scripts/sis-generate-usn
281index 9ae7a8b..a5c7b26 100755
282--- a/scripts/sis-generate-usn
283+++ b/scripts/sis-generate-usn
284@@ -22,7 +22,11 @@ import re
285 import subprocess
286 import sys
287 import tempfile
288+import textwrap
289 import cve_lib
290+from kernel_lib import (is_kernel_cna_cve,
291+ get_kernel_break_fix,
292+ get_affected_subsystems)
293 from uct.config import read_uct_config
294
295 opter = optparse.OptionParser()
296@@ -39,6 +43,7 @@ opter.add_option("--embargoed", help="Include embargoed directory when looking f
297 opter.add_option("--include-eol", help="Include EoL releases", action='store_true')
298 opter.add_option("--binaries-json", help="Path to JSON mapping of binary packages to versions (can repeat, default: binaries.json)", action='append', default=[])
299 opter.add_option("--this-only-affected", help="Makes this only affected feature optional", action='store_true')
300+opter.add_option("--kernel-cna-cves", help="Check for Kernel CNA CVEs and add a paragraph for that", action='store_true')
301 (opt, args) = opter.parse_args()
302
303 if len(args) < 2:
304@@ -752,7 +757,54 @@ if len(CVEs):
305
306 if opt.embargoed:
307 pull_usn_cmd.append('--embargoed')
308+
309+ kernel_cna_cves = []
310+ if opt.kernel_cna_cves:
311+ for cve in CVEs:
312+ if is_kernel_cna_cve(cve):
313+ kernel_cna_cves.append(cve)
314+ pull_usn_cmd.append('--kernel-cna-cves=%s' % kernel_cna_cves)
315+
316 print(subprocess.Popen(pull_usn_cmd + sorted(CVEs), stdout=subprocess.PIPE).communicate()[0].decode("utf-8"))
317+
318+ if opt.kernel_cna_cves:
319+ all_fix_commits = []
320+ for cve in kernel_cna_cves:
321+ break_commits, fix_commits = get_kernel_break_fix(cve)
322+ all_fix_commits.append(fix_commits)
323+
324+ affected_subsystems, other_files = get_affected_subsystems(all_fix_commits)
325+
326+ print("XXX-CHECK-XXX")
327+ if affected_subsystems:
328+ print("Several security issues were discovered in the Linux kernel.")
329+ print("An attacker could possibly use these to compromise the system.")
330+ print("This update corrects flaws in the following subsystems:")
331+ if opt.debug:
332+ debug("Subsystems affected in Kernel CNA CVEs:")
333+ for subsystem, files in affected_subsystems.items():
334+ print(f" - {subsystem};")
335+ if opt.debug:
336+ debug(f" - {subsystem};")
337+ for file_path in files:
338+ debug(f" - {file_path}")
339+ else:
340+ print("No affected subsystems found.")
341+ print(textwrap.fill("(%s)" % (", ".join(kernel_cna_cves)), 75))
342+ print("XXX-CHECK-XXX")
343+
344+ if other_files:
345+ print("XXX-CHECK-XXX")
346+ print("Other Files (Not Belonging to Any Subsystem):")
347+ if opt.debug:
348+ debug("Other Files (Not Belonging to Any Subsystem):")
349+ for file_path in other_files:
350+ print(f"- {file_path}")
351+ if opt.debug:
352+ debug(f"- {file_path}")
353+ print("XXX-CHECK-XXX")
354+
355+
356 if not opt.kernel_mode:
357 print('XXX IF COMPILER PROTECTED XXX')
358 print('The default compiler options for affected releases should reduce the')
359diff --git a/scripts/test_kernel_lib.py b/scripts/test_kernel_lib.py
360index 1bb35c5..405300a 100755
361--- a/scripts/test_kernel_lib.py
362+++ b/scripts/test_kernel_lib.py
363@@ -17,8 +17,11 @@ from kernel_lib import (
364 kernel_package_version,
365 convert_name_to_meta,
366 convert_name_to_signed,
367+ is_kernel_cna_cve,
368+ get_affected_subsystems,
369+ get_kernel_break_fix
370 )
371-
372+from uct.config import read_uct_config
373
374 class TestKernelMabiCalc:
375 def test_basic_version(self):
376@@ -72,3 +75,29 @@ class TestKernelComputeSignedName:
377 def test_bad_kernel_name(self):
378 with pytest.raises(ValueError):
379 convert_name_to_signed("not-linux")
380+
381+
382+class TestKernelCNACVEs:
383+ def test_is_kernel_cna_cve(self):
384+ assert is_kernel_cna_cve('CVE-2024-26581')
385+ assert is_kernel_cna_cve('CVE-2024-24860') is False
386+
387+ def test_get_affected_subsystems(self):
388+ break_commits, fix_commits = get_kernel_break_fix('CVE-2023-52447')
389+ assert fix_commits == ['876673364161da50eed6b472d746ef88242b2368', '876673364161da50eed6b472d746ef88242b2368']
390+ assert break_commits == ['bba1dc0b55ac462d24ed1228ad49800c238cd6d7', '638e4b825d523bed7a55e776c153049fb7716466']
391+
392+ expected_subsystems = {'BPF subsystem': ['kernel/bpf/map_in_map.c', 'kernel/bpf/syscall.c']}
393+ expected_other_files = ['include/linux/bpf.h']
394+
395+ config = read_uct_config()
396+ if 'linux_kernel_path' not in config:
397+ pytest.skip("linux_kernel_path not configured, skipping!")
398+
399+ subsystems, other_files = get_affected_subsystems([fix_commits])
400+ assert subsystems == expected_subsystems
401+ assert other_files == expected_other_files
402+
403+ fail_fix_commit = ['deadb33f']
404+ with pytest.raises(ValueError):
405+ get_affected_subsystems([fail_fix_commit])

Subscribers

People subscribed via source and target branches