Ubuntu CVE Tracker

Merge ~rodrigo-zaiden/ubuntu-cve-tracker:kernel_cna_cves_usn into ubuntu-cve-tracker:master

  1. Git
  2. lp:~rodrigo-zaiden/ubuntu-cve-tracker
  3. kernel_cna_cves_usn
  4. Merge into master
Proposed by Rodrigo Figueiredo Zaiden
Status: Needs review
Proposed branch: ~rodrigo-zaiden/ubuntu-cve-tracker:kernel_cna_cves_usn
Merge into: ubuntu-cve-tracker:master
Diff against target: 458 lines (+302/-12)
8 files modified
README.linux (+7/-3)
meta_lists/kernel_paths_overrides.json (+93/-0)
scripts/cve_lib.py (+4/-6)
scripts/kernel_lib.py (+100/-1)
scripts/prepare-kernel-usn.py (+8/-1)
scripts/pull-usn-desc.py (+8/-0)
scripts/sis-generate-usn (+52/-0)
scripts/test_kernel_lib.py (+30/-1)
Reviewer Review Type Date Requested Status
Alex Murray Approve
Steve Beattie Pending
Review via email: mp+462266@code.launchpad.net

Commit message

scripts: kernel: add automation to generate kernel USN

 adding a new parameter: '--kernel-cna-cves' that, when used, will automatically
 find which CVEs where published by the Kernel CNA and will generate a standard
 paragraph for the USN based on the affected subsystems in the Linux kernel based
 on the fix commits for each CVE.

Description of the change

When the Kernel CNA started to publish Linux Kernel CVEs, we noticed that a
big amount of new CVEs are coming each day. With that, it is much harder to
keep track of the USN descriptions as we were used to.

So, the idea with this series of commits, is that, when we want to generate
a Kernel USN with the script 'prepare-kenel-usn.py' we can now use the
argument '--kernel-cna-cves' that will go over the CVEs that would be included
in the Kernel USN and will check which were published by the Kernel CNA and will
add a standard paragraph will the affected subsystems.
It sounds like:

   Several security issues were discovered in the Linux kernel.
   An attacker could possibly use these to compromise the system.
   This update corrects flaws in the following subsystems:
      - Architecture specifics;
      - Block layer;
      - ACPI drivers;
      ....

A few more detailed information on how it works in this proposal:
- It starts with passing '--kernel-cna-cves' to 'prepare-kenel-usn.py', like:
    ./scripts/prepare-kernel-usn.py -d -n -p Proposed --kernel-cna-cves jammy jammy/linux-oem-6.1: 6.1.0-1035.3

- It them invokes the 'sis-generate-usn' script passing along the argument
  --kernel-cna-cves, that will do the most importants checks:
  1) it check which CVEs that were found being fixed in this kernel release
     were published by the Kernel CNA by trying to verify if the CVE file exists
     in the https://git.kernel.org/pub/scm/linux/security/vulns.git/ repository.
     It checks locally if the repository is cloned (more information on that
     was added to README.linux) and if it does not find it locally it checks
     on the repository online: the local repository could be outdated, or it
     really does not exist. This is implemented in cve_lib.py in the method
     is_kernel_cna_cve().

  2) it them checks which are the fix commits for the CVEs, implemented in
     cve_lib.py in method get_kernel_break_fix()

  3) it finally look for the affected subsystems based on the fix commit on a
     kernel repository local tree. If the repo is not found, it returns an error.
     This is in cve_lib.py in method get_affected_subsystems().

  With the affected subsystems in hand, it generated the paragraph with the
  template above. In get_affected_subsystems() there is a mapping for the
  subsystem paths based on the kernel hierarchy that translates the directory
  into human readable/understandable topics.

A way to run this for current kernels is:
./scripts/prepare-kernel-usn.py -d -n -p Security --ignore-released-cves-in-changelog --kernel-cna-cves jammy jammy/linux-oem-6.1: 6.1.0-1035.35 --add CVE-2023-46838 --add CVE-2023-50431 --add CVE-2023-52436 --add CVE-2023-52438 --add CVE-2023-52439 --add CVE-2023-52443 --add CVE-2023-52444 --add CVE-2023-52445 --add CVE-2023-52447 --add CVE-2023-52448 --add CVE-2023-52449 --add CVE-2023-52451 --add CVE-2023-52454 --add CVE-2023-52456 --add CVE-2023-52457 --add CVE-2023-52458 --add CVE-2023-52462 --add CVE-2023-52463 --add CVE-2023-52464 --add CVE-2023-52467 --add CVE-2023-52469 --add CVE-2023-52470 --add CVE-2023-5633 --add CVE-2023-6610 --add CVE-2024-0340 --add CVE-2024-1085 --add CVE-2024-1086 --add CVE-2024-23849 --add CVE-2024-24860 --add CVE-2024-26581 --add CVE-2024-26588 --add CVE-2024-26589 --add CVE-2024-26591 --add CVE-2024-26592 --add CVE-2024-26594 --add CVE-2024-26597 --add CVE-2024-26598 --add CVE-2024-26599 --add CVE-2024-26600 --add CVE-2024-26601 --bypass-usn-check

or, check the pastebins to see how the script is outputing in stdout and in the USN bash for the above command:
prepare-kernel-usn log: https://pastebin.canonical.com/p/WxS4TqdXQx/
USN bash script: https://pastebin.canonical.com/p/5dhcf83xCx/

To post a comment you must log in.
Revision history for this message
Alex Murray (alexmurray) wrote :

This sounds like a great quality-of-life improvement for doing kernel USNs - I wonder though if instead of hard-coding the mapping of subsystem paths to descriptions, whether you could use the contents of MAINTAINERS (via ./scripts/get_maintainer.pl --subsystem -f $path) and then fallback to this if needed?

Reply
~rodrigo-zaiden/ubuntu-cve-tracker:kernel_cna_cves_usn updated
fd9210f... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: make use of read_uct_config

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

b0d5678... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: improve get_kernel_break_fix

 break-fix commits can be in a variety of formats that were not being
 considered, besides not filtering out '-' that is used when there is
 no specific commit for either the break or fix commit.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Revision history for this message
Rodrigo Figueiredo Zaiden (rodrigo-zaiden) wrote :

Hi Alex,

> This sounds like a great quality-of-life improvement for doing kernel USNs - I
> wonder though if instead of hard-coding the mapping of subsystem paths to
> descriptions, whether you could use the contents of MAINTAINERS (via
> ./scripts/get_maintainer.pl --subsystem -f $path) and then fallback to this if
> needed?

I really like your idea, I think that we could get ride of the hard-coding style
and move to a better way. I've tried a few things outside of this tree in order
to find a way of using the get_maintainer.pl that I'd like to ask your review,
if possible, please.

The suggestion is in this standalone python file that can be ran outside of
the tree without any inputs:
https://pastebin.canonical.com/p/jyKc8mVw3q/

The output for it is at:
https://pastebin.canonical.com/p/jBc5vVvQQX/

Mainly, I'm running the get_maintainer as:
 ./scripts/get_maintainer.pl --subsystem --sections -f $file_path
and filtering out every line that is not a subsystem

What is not really nice:
- It is not yet ordered by kernel tree hierarchy;
- We have a few "subsystems" that are not really useful like "BPF", "NETWORKING"
  that are part of the output because I'm getting everything from the
  'get_maintainer' besides "THE REST"
- It is a bit slow;

Do you like this output style or/and have other suggestions, please?
Really appreciate it.
Thank you very much!

Reply
~rodrigo-zaiden/ubuntu-cve-tracker:kernel_cna_cves_usn updated
216433d... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: move linux_kernel_path verification

 the linux_kernel_path config verification was inside the interation
 when I was trying to implement a second stage verification that would
 check for the commit online, but as this approach was dropped we can
 check it once before running through the commits.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

09fb096... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: handle git command errors

 make use of the cmd() method defined in cve_lib and add handle for
 possible git errors.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

978a7de... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Revision history for this message
Mark Esler (eslerm) wrote :

This looks great.

The points you raised about an ordered hierarchy and useful subsystem would be nice. Maybe those could be solved together?

Personally, I'm trying to move code out of cve_lib.py. Perhaps that's not called for here or others on the team want to extend cve_lib. A kernel_cve.py or similarly named script might be an easier way to maintain helper functions for parsing kernel cve data.

Reply
Revision history for this message
Steve Beattie (sbeattie) wrote :

Apologies, I haven't look at the details of the merge proposal very closely.

Using get_maintainer.pl would be ideal, and perhaps you'd wnat to use something like following to simplify parsing the output:

  ./scripts/get_maintainer.pl --separator ':' --subsystem [...]

so that the subsystems are all on one line, and then pulling out the first entry, to get the most specific subsystem.

My concern with this approach is that it might turn out to be like how we end up writing descriptions for source pacakges in our USNs via the package_info_overrides.json file, because the description summaryies in the debian pacakges are not always ina state usable for general public consumption. My experiments with some sample files are okay, but there are things like:

$ ./scripts/get_maintainer.pl --separator ':' --subsystem drivers/md/dm-ioctl.c | tail -1
DEVICE-MAPPER (LVM):THE REST

and

$ ./scripts/get_maintainer.pl --separator \; --subsystem -f drivers/net/dsa/vitesse-vsc73xx-core.c | tail -1
NETWORKING [DSA];NETWORKING DRIVERS;THE REST

so at a minimum, we might want an override mechanism anyway.

Reply
Revision history for this message
Steve Beattie (sbeattie) wrote :

(sorry, in my examples, I switched up the separators due to experimenting with different ones in my shell history.)

Reply
~rodrigo-zaiden/ubuntu-cve-tracker:kernel_cna_cves_usn updated
a70f0a6... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: make load json file overrides more generic

 with the idea of using the method with other json files, this commit
 renames 'load_package_info_overrides' to 'load_json_file_overrides'

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

39b463c... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: move hardcode mapping to json file

 instead of using a hardcode mapping, it seems better to use a json
 file with the overrides, like we do for package names and titles,
 seems easier to view and cleaner to maintain.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

56c577d... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: trust that linux_cna_tracker is updated

 for the sake of performance, instead of falling back to a online search
 every time a CVE is not found in the kernel cna repo tracker, only
 fallback if the repository itself is not found.
 we will trust that the local repository is updated.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Revision history for this message
Rodrigo Figueiredo Zaiden (rodrigo-zaiden) wrote :

Hi Mark,
> This looks great.
>
> The points you raised about an ordered hierarchy and useful subsystem would be
> nice. Maybe those could be solved together?
>

I'm having a few issues with the get_maintainer approach that I'll share and
propose a solution in another comment.

> Personally, I'm trying to move code out of cve_lib.py. Perhaps that's not
> called for here or others on the team want to extend cve_lib. A kernel_cve.py
> or similarly named script might be an easier way to maintain helper functions
> for parsing kernel cve data.

This is actually a good idea, I started working with 'kernel_lib.py' but decided
to move to 'cve_lib.py' as 'kernel_lib.py' is not really for CVEs but more for
kernel versions. but I didn't think of using a new 'kernel_cve.py' where we
could keep kernel CVE related matters. I'll just finish this round of updates
and will try this suggestion, thanks for it.

Reply
Revision history for this message
Rodrigo Figueiredo Zaiden (rodrigo-zaiden) wrote :

General Notes:
I'm personally inclined to vote for us to move out of the usage of
get_maintainer.pl to generate the list of affected subsystem for two reasons:

1) performance: this would be the main reason. unless, we can find a different
approach on how to run the 'get_maintainer.pl' script, it is proving to be
too much slow to get the subsystem information for each file on each commit.
I have examples of time constrains below.

2) quality of text: don't really know how to name this, but I'm not in favor of
having something like "EDAC-CAVIUM THUNDERX" and "EDAC-CORE" (as the result of
get_maintainer.pl for "drivers/edac/thunderx_edac.c") where we could instead
have "EDAC drivers" (for "drivers/edac/").

So, in the end, I personally don't think maintaining the hardcode bits is that
big of an issue. Considering that, from now on the idea is that the Kernel CNA
will be assigning most of the kernel CVEs, we would be moving from writing USN
descriptions to writing hierarch descriptions...

One thing that I'm proposing with the next commits is to move from a hardcode
mapping to a json meta file as a result of an input from Steve's comment
earlier.

Testing:
I've put together a better scenario for testing both approaches:

# move to my branch:
 $ git remote add rodrigo-zaiden git+ssh://<email address hidden>/~rodrigo-zaiden/ubuntu-cve-tracker
 $ git checkout rodrigo-zaiden/kernel_cna_cves_usn

# standalone script using get_maintainer.pl with:
 ./scripts/get_maintainer.pl --separator ':' --subsystem -f <file path>
https://pastebin.canonical.com/p/sGTN76wDN3/

 $ time python3 test-kernel_cna.py
 ...
  - ARM/FREESCALE IMX / MXC ARM ARCHITECTURE;
  - ARM64 PORT (AARCH64 ARCHITECTURE);
  - BLOCK LAYER;
  - BLUETOOTH SUBSYSTEM;
  - BPF JIT for LOONGARCH;
  - BPF [CORE];
  - BPF [GENERAL] (Safe Dynamic Programs and Tools);
 ...
 real 0m36,722s
 user 0m33,750s
 sys 0m2,888s

full results: https://pastebin.canonical.com/p/bprVtW7gb5/

# standalone script using json meta file:
https://pastebin.canonical.com/p/KFQRFpNdXG/

 $ time python3 test_kernel-cna-overrides.py
 ...
   - EDAC drivers;
   - GPU drivers;
   - Media drivers;
   - Multifunction device drivers;
   - MTD block device drivers;
 ...
 real 0m0,963s
 user 0m0,849s
 sys 0m0,113s

full results: https://pastebin.canonical.com/p/BV2qYc66tr/

Reply
Revision history for this message
Alex Murray (alexmurray) wrote :

Thanks for investigating the get_maintainer.pl approach - it's a shame it was so unwieldy in practice and IMO day-to-day usability trumps maintainership - ie. so the json overrides approach sounds like the better option. LGTM!

review: Approve
Reply
~rodrigo-zaiden/ubuntu-cve-tracker:kernel_cna_cves_usn updated
f16b003... by Rodrigo Figueiredo Zaiden

meta_lists/kernel_paths_overrides.json: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

c48b7bc... by Rodrigo Figueiredo Zaiden

scripts/kernel: move kernel-cna-cves methods to kernel_lib

 after a reconsideration, it would be better to start moving the kernel
 related methods to kernel_lib instead of over populating cve_lib.
 there is a few other methods related to kernel in cve_lib that should
 be moved later, for now, moving only the ones related to the new
 argument, --kernel-cna-cves.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

0667c23... by Rodrigo Figueiredo Zaiden

scripts/test_kernel_lib.py: add tests for kernel cna cves methods

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

e72bfae... by Rodrigo Figueiredo Zaiden

scripts/kernel_lib.py: add directory fallback

 while running the tests, noticed that the subsequent tests after
 the kernel ones were failing because the directory was being moved
 elsewhere. added a chdir to go back to the previous and added a test
 for the failure code path.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

ebd52c4... by Rodrigo Figueiredo Zaiden

scripts/test_kernel_lib.py: add condition to skip test

 if there is not configuration for the linux kernel tree, we should
 skip testing. that would be the case mainly for the servers, or,
 having the linux kernel tree would be the case only for local runs.
 not sure if that is something we could mock, don't really think so.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

744be15... by Rodrigo Figueiredo Zaiden

meta_lists/kernel_paths_overrides.json: expand arch/

 instead of using the more generic arch/, expand the tree for the next
 level.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

6c3a721... by Rodrigo Figueiredo Zaiden

meta_lists/kernel_paths_overrides.json: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Unmerged commits

6c3a721... by Rodrigo Figueiredo Zaiden

meta_lists/kernel_paths_overrides.json: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Failed
[SUCCEEDED] unit-tests:0 (build)
[FAILED] check-cves:0 (build)
[WAITING] check-cve-website-state:0 (build)
1 → 3 of 3 results First • Previous • Next • Last
744be15... by Rodrigo Figueiredo Zaiden

meta_lists/kernel_paths_overrides.json: expand arch/

 instead of using the more generic arch/, expand the tree for the next
 level.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

ebd52c4... by Rodrigo Figueiredo Zaiden

scripts/test_kernel_lib.py: add condition to skip test

 if there is not configuration for the linux kernel tree, we should
 skip testing. that would be the case mainly for the servers, or,
 having the linux kernel tree would be the case only for local runs.
 not sure if that is something we could mock, don't really think so.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

e72bfae... by Rodrigo Figueiredo Zaiden

scripts/kernel_lib.py: add directory fallback

 while running the tests, noticed that the subsequent tests after
 the kernel ones were failing because the directory was being moved
 elsewhere. added a chdir to go back to the previous and added a test
 for the failure code path.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

0667c23... by Rodrigo Figueiredo Zaiden

scripts/test_kernel_lib.py: add tests for kernel cna cves methods

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

c48b7bc... by Rodrigo Figueiredo Zaiden

scripts/kernel: move kernel-cna-cves methods to kernel_lib

 after a reconsideration, it would be better to start moving the kernel
 related methods to kernel_lib instead of over populating cve_lib.
 there is a few other methods related to kernel in cve_lib that should
 be moved later, for now, moving only the ones related to the new
 argument, --kernel-cna-cves.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

f16b003... by Rodrigo Figueiredo Zaiden

meta_lists/kernel_paths_overrides.json: add more subsystems

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

56c577d... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: trust that linux_cna_tracker is updated

 for the sake of performance, instead of falling back to a online search
 every time a CVE is not found in the kernel cna repo tracker, only
 fallback if the repository itself is not found.
 we will trust that the local repository is updated.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

39b463c... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: move hardcode mapping to json file

 instead of using a hardcode mapping, it seems better to use a json
 file with the overrides, like we do for package names and titles,
 seems easier to view and cleaner to maintain.

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

a70f0a6... by Rodrigo Figueiredo Zaiden

scripts/cve_lib.py: make load json file overrides more generic

 with the idea of using the method with other json files, this commit
 renames 'load_package_info_overrides' to 'load_json_file_overrides'

Signed-off-by: Rodrigo Figueiredo Zaiden <email address hidden>

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/README.linux b/README.linux
index 48dae64..ca1ec75 100644
--- a/README.linux
+++ b/README.linux
@@ -58,12 +58,16 @@ common ones are the
58- [Debian's kernel security tracker][debian-kernel]58- [Debian's kernel security tracker][debian-kernel]
59- Nicholas Leudtke's [linux github tracker][linux-github]59- Nicholas Leudtke's [linux github tracker][linux-github]
6060
61The latter two can be configured as references in61The three of them can be configured as references in
62`~/.ubuntu-cve-tracker.conf` for use with tools, via the62`~/.ubuntu-cve-tracker.conf` for use with tools, via the
63`debian_kernel_cve_tracker` and `linux_kernel_cve_tracker` variables63`linux_cna_tracker`, `debian_kernel_cve_tracker` and
64respectively. An example:64`linux_kernel_cve_tracker` variables respectively. An example:
6565
66```66```
67# path to a copy of the upstream kernel CNA tracker from
68# https://git.kernel.org/pub/scm/linux/security/vulns.git/
69linux_cna_tracker='/path/to/git/cve_trackers/linux-cna/'
70
67# path to a copy of the linux kernel cve tracker from71# path to a copy of the linux kernel cve tracker from
68# https://github.com/nluedtke/linux_kernel_cves.git72# https://github.com/nluedtke/linux_kernel_cves.git
69linux_kernel_cve_tracker='/path/to/git/cve_trackers/linux_kernel_cves/'73linux_kernel_cve_tracker='/path/to/git/cve_trackers/linux_kernel_cves/'
diff --git a/meta_lists/kernel_paths_overrides.json b/meta_lists/kernel_paths_overrides.json
70new file mode 10064474new file mode 100644
index 0000000..ab189fa
--- /dev/null
+++ b/meta_lists/kernel_paths_overrides.json
@@ -0,0 +1,93 @@
1{
2 "arch/arm/": "ARM32 architecture",
3 "arch/arm64/": "ARM64 architecture",
4 "arch/powerpc/": "PowerPC architecture",
5 "arch/riscv/": "RISC-V architecture",
6 "arch/s390/": "S390 architecture",
7 "arch/x86/": "x86 architecture",
8 "block/": "Block layer subsystem",
9 "crypto/": "Cryptographic API",
10 "drivers/acpi": "ACPI drivers",
11 "drivers/android/": "Android drivers",
12 "drivers/base/power/": "Power management core",
13 "drivers/base/": "Drivers core",
14 "drivers/bus/": "Bus devices",
15 "drivers/char/hw_random/": "Hardware random number generator core",
16 "drivers/char/tpm/": "TPM device driver",
17 "drivers/devfreq/": "Device frequency",
18 "drivers/dma/": "DMA engine subsystem",
19 "drivers/edac/": "EDAC drivers",
20 "drivers/firmware/arm_scmi/": "ARM SCMI message protocol",
21 "drivers/gpu/": "GPU drivers",
22 "drivers/hid": "HID subsystem",
23 "drivers/hwmon/": "Hardware monitoring drivers",
24 "drivers/i2c/": "I2C subsystem",
25 "drivers/iio/adc/": "IIO ADC drivers",
26 "drivers/iio/magnetometer/": "IIO Magnetometer sensors drivers",
27 "drivers/iio/": "IIO subsystem",
28 "drivers/infiniband/": "InfiniBand drivers",
29 "drivers/iommu/": "IOMMU subsystem",
30 "drivers/media/": "Media drivers",
31 "drivers/mfd/": "Multifunction device drivers",
32 "drivers/mtd/": "MTD block device drivers",
33 "drivers/net/": "Network drivers",
34 "drivers/nvme/": "NVME drivers",
35 "drivers/pci/switch/": "PCI driver for MicroSemi Switchtec",
36 "drivers/pci/":"PCI subsystem",
37 "drivers/phy/": "PHY drivers",
38 "drivers/platform/x86/": "x86 platform drivers",
39 "drivers/power/supply/": "Power supply drivers",
40 "drivers/power/reset/": "System reset/shutdown drivers",
41 "drivers/pwm/": "PWM drivers",
42 "drivers/scsi/": "SCSI drivers",
43 "drivers/soc/qcom/": "QCOM SoC drivers",
44 "drivers/spmi/": "SPMI drivers",
45 "drivers/thermal/": "Thermal drivers",
46 "drivers/tty/": "TTY drivers",
47 "drivers/uio/": "Userspace I/O drivers",
48 "drivers/usb/dwc3/": "DesignWare USB3 driver",
49 "drivers/vfio/": "VFIO drivers",
50 "fs/btrfs/": "BTRFS file system",
51 "fs/ceph/": "Ceph distributed file system",
52 "fs/efivarfs/": "EFI Variable file system",
53 "fs/erofs/": "EROFS file system",
54 "fs/ext4/": "Ext4 file system",
55 "fs/f2fs/": "F2FS file system",
56 "fs/gfs2/": "GFS2 file system",
57 "fs/netfs/": "Network file systems library",
58 "fs/nfsd/": "Network file system server daemon",
59 "fs/nilfs2/": "NILFS2 file system",
60 "fs/ntfs3/": "NTFS3 file system",
61 "fs/jfs/": "JFS file system",
62 "fs/pstore/": "Pstore file system",
63 "fs/reiserfs/": "ReiserFS file system",
64 "fs/smb/": "SMB network file system",
65 "include/linux/bio.h": "Block layer subsystem",
66 "include/linux/bpf.h": "BPF subsystem",
67 "include/linux/mmzone.h": "Memory management",
68 "kernel/bpf/": "BPF subsystem",
69 "kernel/": "Core kernel",
70 "mm/": "Memory management",
71 "net/bluetooth/": "Bluetooth subsystem",
72 "net/can/": "CAN network layer",
73 "net/core/": "Networking core",
74 "net/hsr": "HSR network protocol",
75 "net/ipv6/": "IPv6 networking",
76 "net/ipv4/": "IPv4 networking",
77 "net/llc/": "Logical Link layer",
78 "net/netfilter/": "Netfilter",
79 "net/netlink/": "Netlink",
80 "net/nfc/": "NFC subsystem",
81 "net/mac80211/": "MAC80211 subsystem",
82 "net/mptcp/": "Multipath TCP",
83 "net/openvswitch/": "Open vSwitch",
84 "net/sched/": "Network traffic control",
85 "net/smc/": "SMC sockets",
86 "net/sunrpc/": "Sun RPC protocol",
87 "net/tipc/": "TIPC protocol",
88 "net/tls": "TLS protocol",
89 "net/unix/": "Unix domain sockets",
90 "security/apparmor/": "AppArmor security module",
91 "security/tomoyo/": "Tomoyo security module",
92 "sound/soc/codecs/rt": "Realtek audio codecs"
93}
diff --git a/scripts/cve_lib.py b/scripts/cve_lib.py
index f8cff79..e87096f 100755
--- a/scripts/cve_lib.py
+++ b/scripts/cve_lib.py
@@ -2805,13 +2805,13 @@ def load_boilerplates():
2805 data[alias]["aliases"] = sorted(list(aliases[alias]))2805 data[alias]["aliases"] = sorted(list(aliases[alias]))
2806 return data2806 return data
28072807
2808# for sanity, try to keep these in alphabetical order in the json file2808def load_json_file_overrides(json_file):
2809def load_package_info_overrides():2809 with open(os.path.join(meta_dir, json_file), "r") as fp:
2810 with open(os.path.join(meta_dir, "package_info_overrides.json"), "r") as fp:
2811 data = json.load(fp)2810 data = json.load(fp)
2812 return data2811 return data
28132812
2814package_info_overrides = load_package_info_overrides()2813# for sanity, try to keep these in alphabetical order in the json file
2814package_info_overrides = load_json_file_overrides("package_info_overrides.json")
28152815
2816def load_package_db():2816def load_package_db():
2817 pkg_db = load_boilerplates()2817 pkg_db = load_boilerplates()
@@ -3739,5 +3739,3 @@ def validate_kernel_fixes(break_fixes):
3739 validated.append([break_hash, fix_hash])3739 validated.append([break_hash, fix_hash])
37403740
3741 return validated3741 return validated
3742
3743
diff --git a/scripts/kernel_lib.py b/scripts/kernel_lib.py
index 60ec87c..5ab497a 100755
--- a/scripts/kernel_lib.py
+++ b/scripts/kernel_lib.py
@@ -14,9 +14,14 @@
14from __future__ import print_function14from __future__ import print_function
1515
16import sys16import sys
17import os
18import urllib.error
19import urllib.request
20from uct.config import read_uct_config
1721
18# XXX kernel_srcs probably belongs here, too22# XXX kernel_srcs probably belongs here, too
19from cve_lib import (kernel_srcs, get_esm_name, is_active_esm_release)23from cve_lib import (kernel_srcs, get_esm_name, is_active_esm_release, \
24 load_cve, find_cve, load_json_file_overrides, cmd)
2025
2126
22# search for the kernel SRU cycle for a kernel in the format27# search for the kernel SRU cycle for a kernel in the format
@@ -1542,3 +1547,97 @@ def kernel_package_version(version):
1542# XXX may need to add an offset exception to the kernel table1547# XXX may need to add an offset exception to the kernel table
1543def kernel_meta_abi(version, offset=3):1548def kernel_meta_abi(version, offset=3):
1544 return int(version.split('.').pop(offset))1549 return int(version.split('.').pop(offset))
1550
1551
1552def get_kernel_break_fix(cve):
1553 '''Return the break-fix commits of a giver kernel CVE'''
1554
1555 break_commits = []
1556 fix_commits = []
1557
1558 cve_data = load_cve(find_cve(cve))
1559 if cve_data and cve_data['patches']['linux']:
1560 for (patch_type, break_fix) in cve_data['patches']['linux']:
1561 if patch_type == 'break-fix':
1562 break_commit, fix_commit = break_fix.split()[:2]
1563
1564 for entry in break_commit.split('|'):
1565 if entry != '-':
1566 break_commits.append(entry)
1567 for entry in fix_commit.split('|'):
1568 if entry != '-':
1569 fix_commits.append(entry)
1570
1571 return break_commits, fix_commits
1572
1573
1574def is_kernel_cna_cve(cve):
1575 '''Check if a given CVE was published by the Kernel CNA'''
1576
1577 year = cve.split('-')[1]
1578
1579 config = read_uct_config()
1580 if 'linux_cna_tracker' in config:
1581 if os.path.exists(os.path.join(config['linux_cna_tracker'], "cve", "published", year, cve)):
1582 return True
1583 return False
1584
1585 print("WARNING: Couldn't find linux_cna_tracker repo, will try to verify it online." , file=sys.stderr)
1586
1587 url = ("https://git.kernel.org/pub/scm/linux/security/vulns.git/tree/cve/published/%s/%s" % (year, cve))
1588
1589 try:
1590 with urllib.request.urlopen(url) as response:
1591 if response.getcode() == 200:
1592 return True
1593 else:
1594 print("WARNING: Failed to get OK response from URL %s: %s" % (url, response.getcode()), file=sys.stderr)
1595 return False
1596 except urllib.error.HTTPError as e:
1597 print("WARNING: Failed to fetch patch URL %s: %s" % (url, str(e)), file=sys.stderr)
1598 return False
1599
1600
1601def get_affected_subsystems(commits):
1602 '''Return the kernel subsystems affected by a list of commits'''
1603
1604 config = read_uct_config()
1605 if 'linux_kernel_path' not in config:
1606 raise ValueError("linux_kernel_path not configured, please read README.linux")
1607
1608 subsystems = {}
1609 other_files = []
1610 kernel_paths_overrides = load_json_file_overrides("kernel_paths_overrides.json")
1611
1612 pwd = os.getcwd()
1613 os.chdir(config['linux_kernel_path'])
1614
1615 for _commits in commits:
1616 for commit in _commits:
1617
1618 rc, output = cmd(['git', 'show', '--name-only', '--pretty=format:', commit])
1619 if rc !=0:
1620 os.chdir(pwd)
1621 raise ValueError('failed to get file names from commit %s:\n%s' % (commit, output))
1622
1623 files_changed = output.splitlines()
1624 for file_path in files_changed:
1625 matched_subsystem = False
1626 for kernel_path in kernel_paths_overrides:
1627 if kernel_path in file_path:
1628 if kernel_paths_overrides[kernel_path] not in subsystems:
1629 subsystems[kernel_paths_overrides[kernel_path]] = set()
1630 subsystems[kernel_paths_overrides[kernel_path]].add(file_path)
1631 matched_subsystem = True
1632 break
1633 if not matched_subsystem:
1634 if file_path not in other_files:
1635 other_files.append(file_path)
1636
1637 os.chdir(pwd)
1638
1639 # sort alphabetically by kernel hierarchy tree
1640 sorted_subsystems = {key: sorted(value) for key, value in sorted(subsystems.items(), key=lambda item: sorted(item[1]))}
1641 other_files.sort()
1642
1643 return sorted_subsystems, other_files
diff --git a/scripts/prepare-kernel-usn.py b/scripts/prepare-kernel-usn.py
index ee11cb9..7ec14fd 100755
--- a/scripts/prepare-kernel-usn.py
+++ b/scripts/prepare-kernel-usn.py
@@ -176,6 +176,7 @@ parser.add_argument('--use-changes', action='store', default=None, help='Use pre
176parser.add_argument('--keep-changes', action='store_true', default=False, help='Keep changes files downloaded by sis-changes')176parser.add_argument('--keep-changes', action='store_true', default=False, help='Keep changes files downloaded by sis-changes')
177parser.add_argument("--ignore-released-cves-in-changelog", action='store_true', help="Filter out CVEs already marked as released")177parser.add_argument("--ignore-released-cves-in-changelog", action='store_true', help="Filter out CVEs already marked as released")
178parser.add_argument("--esm-ppa", action='store', help="Add kernels from ESM PPA if any, can be used when merging ESM with active kernels (sets --include-eol)")178parser.add_argument("--esm-ppa", action='store', help="Add kernels from ESM PPA if any, can be used when merging ESM with active kernels (sets --include-eol)")
179parser.add_argument("--kernel-cna-cves", action='store_true', help="Check for Kernel CNA CVEs and add a paragraph for that")
179parser.add_argument('release', action='store', nargs=1, help='Primary release name (e.g. xenial)')180parser.add_argument('release', action='store', nargs=1, help='Primary release name (e.g. xenial)')
180parser.add_argument('kernel', action=KernelVersionAction, nargs='+', help='Kernel source package name and versions; e.g. "linux 4.4.0-42.62. Source package can be a release/name pair"')181parser.add_argument('kernel', action=KernelVersionAction, nargs='+', help='Kernel source package name and versions; e.g. "linux 4.4.0-42.62. Source package can be a release/name pair"')
181args = parser.parse_args()182args = parser.parse_args()
@@ -214,7 +215,7 @@ for release in cve_lib.release_sort(kernels.keys()):
214215
215 if not args.embargoed:216 if not args.embargoed:
216 rc = display_pending_cves(release, kernel, last_usn_version, version, args.add)217 rc = display_pending_cves(release, kernel, last_usn_version, version, args.add)
217 if not rc == 0 and not args.force:218 if not rc == 0 and not (args.force or args.kernel_cna_cves):
218 print("Incomplete descriptions detected, please fix.")219 print("Incomplete descriptions detected, please fix.")
219 exit(1)220 exit(1)
220221
@@ -321,9 +322,15 @@ try:
321 if args.ignore_released_cves_in_changelog:322 if args.ignore_released_cves_in_changelog:
322 cmd += ['--ignore-released-cves-in-changelog']323 cmd += ['--ignore-released-cves-in-changelog']
323324
325 if args.kernel_cna_cves:
326 cmd += ['--kernel-cna-cves']
327
324 if args.embargoed:328 if args.embargoed:
325 cmd += ['--embargoed']329 cmd += ['--embargoed']
326330
331 if args.debug:
332 cmd += ['--debug']
333
327 cmd += ['--filter-bins', generate_usn_regex, usn]334 cmd += ['--filter-bins', generate_usn_regex, usn]
328 for release in kernels:335 for release in kernels:
329 cmd += ['--binaries-json', '%s-binaries.json' % release]336 cmd += ['--binaries-json', '%s-binaries.json' % release]
diff --git a/scripts/pull-usn-desc.py b/scripts/pull-usn-desc.py
index 6bb6ce1..71354f9 100755
--- a/scripts/pull-usn-desc.py
+++ b/scripts/pull-usn-desc.py
@@ -42,6 +42,7 @@ opter.add_option("--releases", help="List of releases CVEs affect to be filter u
42opter.add_option("--this-only-affected", help="Makes this only affected feature optional", action='store_true')42opter.add_option("--this-only-affected", help="Makes this only affected feature optional", action='store_true')
43opter.add_option("--src", help="The package source to be check through if --this-only-affected is used", action='append', default=[])43opter.add_option("--src", help="The package source to be check through if --this-only-affected is used", action='append', default=[])
44opter.add_option("--embargoed", help="Use the embargoed tree to look for desctiptions in addition", action='store_true')44opter.add_option("--embargoed", help="Use the embargoed tree to look for desctiptions in addition", action='store_true')
45opter.add_option("--kernel-cna-cves", help="List of CVEs from the Kernel CNA", default=None)
45(opt, args) = opter.parse_args()46(opt, args) = opter.parse_args()
4647
47cves = dict()48cves = dict()
@@ -106,6 +107,13 @@ for cve in opt.cve + args:
106 chunks = cves[cve]107 chunks = cves[cve]
107108
108 desc = chunks['Ubuntu-Description'].strip()109 desc = chunks['Ubuntu-Description'].strip()
110
111 # we don't want to include the empty description CVEs that are part of
112 # the list from the Kernel CNA but we still may want to follow the
113 # normal process for those that have a description.
114 if len(desc) == 0 and cve in opt.kernel_cna_cves:
115 continue
116
109 if len(desc) == 0:117 if len(desc) == 0:
110 rc = 1118 rc = 1
111 disc = chunks.get('Discovered-by', '').strip()119 disc = chunks.get('Discovered-by', '').strip()
diff --git a/scripts/sis-generate-usn b/scripts/sis-generate-usn
index 9ae7a8b..a5c7b26 100755
--- a/scripts/sis-generate-usn
+++ b/scripts/sis-generate-usn
@@ -22,7 +22,11 @@ import re
22import subprocess22import subprocess
23import sys23import sys
24import tempfile24import tempfile
25import textwrap
25import cve_lib26import cve_lib
27from kernel_lib import (is_kernel_cna_cve,
28 get_kernel_break_fix,
29 get_affected_subsystems)
26from uct.config import read_uct_config30from uct.config import read_uct_config
2731
28opter = optparse.OptionParser()32opter = optparse.OptionParser()
@@ -39,6 +43,7 @@ opter.add_option("--embargoed", help="Include embargoed directory when looking f
39opter.add_option("--include-eol", help="Include EoL releases", action='store_true')43opter.add_option("--include-eol", help="Include EoL releases", action='store_true')
40opter.add_option("--binaries-json", help="Path to JSON mapping of binary packages to versions (can repeat, default: binaries.json)", action='append', default=[])44opter.add_option("--binaries-json", help="Path to JSON mapping of binary packages to versions (can repeat, default: binaries.json)", action='append', default=[])
41opter.add_option("--this-only-affected", help="Makes this only affected feature optional", action='store_true')45opter.add_option("--this-only-affected", help="Makes this only affected feature optional", action='store_true')
46opter.add_option("--kernel-cna-cves", help="Check for Kernel CNA CVEs and add a paragraph for that", action='store_true')
42(opt, args) = opter.parse_args()47(opt, args) = opter.parse_args()
4348
44if len(args) < 2:49if len(args) < 2:
@@ -752,7 +757,54 @@ if len(CVEs):
752757
753 if opt.embargoed:758 if opt.embargoed:
754 pull_usn_cmd.append('--embargoed')759 pull_usn_cmd.append('--embargoed')
760
761 kernel_cna_cves = []
762 if opt.kernel_cna_cves:
763 for cve in CVEs:
764 if is_kernel_cna_cve(cve):
765 kernel_cna_cves.append(cve)
766 pull_usn_cmd.append('--kernel-cna-cves=%s' % kernel_cna_cves)
767
755 print(subprocess.Popen(pull_usn_cmd + sorted(CVEs), stdout=subprocess.PIPE).communicate()[0].decode("utf-8"))768 print(subprocess.Popen(pull_usn_cmd + sorted(CVEs), stdout=subprocess.PIPE).communicate()[0].decode("utf-8"))
769
770 if opt.kernel_cna_cves:
771 all_fix_commits = []
772 for cve in kernel_cna_cves:
773 break_commits, fix_commits = get_kernel_break_fix(cve)
774 all_fix_commits.append(fix_commits)
775
776 affected_subsystems, other_files = get_affected_subsystems(all_fix_commits)
777
778 print("XXX-CHECK-XXX")
779 if affected_subsystems:
780 print("Several security issues were discovered in the Linux kernel.")
781 print("An attacker could possibly use these to compromise the system.")
782 print("This update corrects flaws in the following subsystems:")
783 if opt.debug:
784 debug("Subsystems affected in Kernel CNA CVEs:")
785 for subsystem, files in affected_subsystems.items():
786 print(f" - {subsystem};")
787 if opt.debug:
788 debug(f" - {subsystem};")
789 for file_path in files:
790 debug(f" - {file_path}")
791 else:
792 print("No affected subsystems found.")
793 print(textwrap.fill("(%s)" % (", ".join(kernel_cna_cves)), 75))
794 print("XXX-CHECK-XXX")
795
796 if other_files:
797 print("XXX-CHECK-XXX")
798 print("Other Files (Not Belonging to Any Subsystem):")
799 if opt.debug:
800 debug("Other Files (Not Belonging to Any Subsystem):")
801 for file_path in other_files:
802 print(f"- {file_path}")
803 if opt.debug:
804 debug(f"- {file_path}")
805 print("XXX-CHECK-XXX")
806
807
756if not opt.kernel_mode:808if not opt.kernel_mode:
757 print('XXX IF COMPILER PROTECTED XXX')809 print('XXX IF COMPILER PROTECTED XXX')
758 print('The default compiler options for affected releases should reduce the')810 print('The default compiler options for affected releases should reduce the')
diff --git a/scripts/test_kernel_lib.py b/scripts/test_kernel_lib.py
index 1bb35c5..405300a 100755
--- a/scripts/test_kernel_lib.py
+++ b/scripts/test_kernel_lib.py
@@ -17,8 +17,11 @@ from kernel_lib import (
17 kernel_package_version,17 kernel_package_version,
18 convert_name_to_meta,18 convert_name_to_meta,
19 convert_name_to_signed,19 convert_name_to_signed,
20 is_kernel_cna_cve,
21 get_affected_subsystems,
22 get_kernel_break_fix
20)23)
2124from uct.config import read_uct_config
2225
23class TestKernelMabiCalc:26class TestKernelMabiCalc:
24 def test_basic_version(self):27 def test_basic_version(self):
@@ -72,3 +75,29 @@ class TestKernelComputeSignedName:
72 def test_bad_kernel_name(self):75 def test_bad_kernel_name(self):
73 with pytest.raises(ValueError):76 with pytest.raises(ValueError):
74 convert_name_to_signed("not-linux")77 convert_name_to_signed("not-linux")
78
79
80class TestKernelCNACVEs:
81 def test_is_kernel_cna_cve(self):
82 assert is_kernel_cna_cve('CVE-2024-26581')
83 assert is_kernel_cna_cve('CVE-2024-24860') is False
84
85 def test_get_affected_subsystems(self):
86 break_commits, fix_commits = get_kernel_break_fix('CVE-2023-52447')
87 assert fix_commits == ['876673364161da50eed6b472d746ef88242b2368', '876673364161da50eed6b472d746ef88242b2368']
88 assert break_commits == ['bba1dc0b55ac462d24ed1228ad49800c238cd6d7', '638e4b825d523bed7a55e776c153049fb7716466']
89
90 expected_subsystems = {'BPF subsystem': ['kernel/bpf/map_in_map.c', 'kernel/bpf/syscall.c']}
91 expected_other_files = ['include/linux/bpf.h']
92
93 config = read_uct_config()
94 if 'linux_kernel_path' not in config:
95 pytest.skip("linux_kernel_path not configured, skipping!")
96
97 subsystems, other_files = get_affected_subsystems([fix_commits])
98 assert subsystems == expected_subsystems
99 assert other_files == expected_other_files
100
101 fail_fix_commit = ['deadb33f']
102 with pytest.raises(ValueError):
103 get_affected_subsystems([fail_fix_commit])

Subscribers

People subscribed via source and target branches