Launchpad itself

Merge ~jugmac00/launchpad:allow_limiting_uct_imports into launchpad:master

  1. Git
  2. lp:~jugmac00/launchpad
  3. allow_limiting_uct_imports
  4. Merge into master
Proposed by Jürgen Gmach
Status: Merged
Approved by: Jürgen Gmach
Approved revision: 9ba7fdfdea57839dc0b4fdbd4f5e3c2524a11356
Merge reported by: Otto Co-Pilot
Merged at revision: not available
Proposed branch: ~jugmac00/launchpad:allow_limiting_uct_imports
Merge into: launchpad:master
Diff against target: 346 lines (+256/-53)
5 files modified
lib/lp/bugs/scripts/tests/sampledata/CVE-2007-0255 (+61/-0)
lib/lp/bugs/scripts/tests/sampledata/CVE-2022-3219 (+43/-0)
lib/lp/bugs/scripts/tests/test_uctimport.py (+88/-0)
lib/lp/bugs/scripts/uctimport.py (+63/-0)
scripts/uct-import.py (+1/-53)
Reviewer Review Type Date Requested Status
Colin Watson (community) Approve
Review via email: mp+436146@code.launchpad.net

Commit message

Enable filtering for UCTImports

To post a comment you must log in.
Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Enable filtering for UCTImports

Revision history for this message
Colin Watson (cjwatson) :
review: Approve
Revision history for this message
Colin Watson (cjwatson) wrote :

Could you fix the commit message of this MP, since that will be used by the merge bot when landing this?

Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Thanks for the review!

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/lib/lp/bugs/scripts/tests/sampledata/CVE-2007-0255 b/lib/lp/bugs/scripts/tests/sampledata/CVE-2007-0255
0new file mode 1006440new file mode 100644
index 0000000..db2403d
--- /dev/null
+++ b/lib/lp/bugs/scripts/tests/sampledata/CVE-2007-0255
@@ -0,0 +1,61 @@
1PublicDate: 2007-01-16 23:28:00 UTC
2Candidate: CVE-2007-0255
3References:
4 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-0255
5 http://xine.sourceforge.net/security
6Description:
7 XINE 0.99.4 allows user-assisted remote attackers to cause a denial of
8 service (application crash) and possibly execute arbitrary code via a
9 certain M3U file that contains a long #EXTINF line and contains format
10 string specifiers in an invalid udp:// URI, possibly a variant of
11 CVE-2007-0017.
12Ubuntu-Description:
13Notes:
14 sbeattie> issue is unlisted on xine upstream website
15Priority: medium
16Bugs:
17Discovered-by:
18Assigned-to:
19CVSS:
20
21Patches_xine-ui:
22upstream_xine-ui: needs-triage
23dapper_xine-ui: ignored (reached end-of-life)
24edgy_xine-ui: needed (reached end-of-life)
25feisty_xine-ui: needed (reached end-of-life)
26gutsy_xine-ui: needed (reached end-of-life)
27hardy_xine-ui: ignored (reached end-of-life)
28intrepid_xine-ui: needed (reached end-of-life)
29jaunty_xine-ui: ignored (reached end-of-life)
30karmic_xine-ui: ignored (reached end-of-life)
31lucid_xine-ui: ignored (reached end-of-life)
32maverick_xine-ui: ignored (reached end-of-life)
33natty_xine-ui: ignored (reached end-of-life)
34oneiric_xine-ui: ignored (reached end-of-life)
35precise_xine-ui: ignored (reached end-of-life)
36precise/esm_xine-ui: DNE (precise was needed)
37quantal_xine-ui: ignored (reached end-of-life)
38raring_xine-ui: ignored (reached end-of-life)
39saucy_xine-ui: ignored (reached end-of-life)
40trusty_xine-ui: ignored (reached end-of-life)
41trusty/esm_xine-ui: DNE (trusty was needed)
42utopic_xine-ui: ignored (reached end-of-life)
43vivid_xine-ui: ignored (reached end-of-life)
44vivid/stable-phone-overlay_xine-ui: DNE
45vivid/ubuntu-core_xine-ui: DNE
46wily_xine-ui: ignored (reached end-of-life)
47xenial_xine-ui: ignored (end of standard support, was needed)
48yakkety_xine-ui: ignored (reached end-of-life)
49zesty_xine-ui: ignored (reached end-of-life)
50artful_xine-ui: ignored (reached end-of-life)
51bionic_xine-ui: needed
52cosmic_xine-ui: ignored (reached end-of-life)
53disco_xine-ui: ignored (reached end-of-life)
54eoan_xine-ui: ignored (reached end-of-life)
55focal_xine-ui: needed
56groovy_xine-ui: ignored (reached end-of-life)
57hirsute_xine-ui: ignored (reached end-of-life)
58impish_xine-ui: ignored (reached end-of-life)
59jammy_xine-ui: needed
60kinetic_xine-ui: needed
61devel_xine-ui: needed
0\ No newline at end of file62\ No newline at end of file
diff --git a/lib/lp/bugs/scripts/tests/sampledata/CVE-2022-3219 b/lib/lp/bugs/scripts/tests/sampledata/CVE-2022-3219
1new file mode 10064463new file mode 100644
index 0000000..14aaa73
--- /dev/null
+++ b/lib/lp/bugs/scripts/tests/sampledata/CVE-2022-3219
@@ -0,0 +1,43 @@
1Candidate: CVE-2022-3219
2PublicDate: 2022-09-28
3References:
4 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-3219
5 https://access.redhat.com/security/cve/CVE-2022-3219
6 https://marc.info/?l=oss-security&m=165696590211434&w=4
7Description:
8 gnupg: denial of service issue (resource consumption) using compressed
9 packets
10Ubuntu-Description:
11Notes:
12 mdeslaur> per the upstream gnupg bug, the change will not be applied
13 mdeslaur> as of 2022-09-28, proposed patch has not been accepted by
14 mdeslaur> upstream developers
15Mitigation:
16Bugs:
17 https://dev.gnupg.org/T5993
18Priority: low
19Discovered-by:
20Assigned-to:
21CVSS:
22
23Patches_gnupg:
24upstream_gnupg: needs-triage
25esm-infra/xenial_gnupg: deferred (2022-09-28)
26trusty_gnupg: ignored (out of standard support)
27xenial_gnupg: ignored (out of standard support)
28bionic_gnupg: DNE
29focal_gnupg: DNE
30jammy_gnupg: DNE
31trusty/esm_gnupg: deferred (2022-09-28)
32
33Patches_gnupg2:
34 other: https://dev.gnupg.org/D556
35upstream_gnupg2: needs-triage
36esm-infra/xenial_gnupg2: deferred (2022-09-28)
37trusty_gnupg2: ignored (out of standard support)
38xenial_gnupg2: ignored (end of standard support)
39bionic_gnupg2: deferred (2022-09-28)
40focal_gnupg2: deferred (2022-09-28)
41jammy_gnupg2: deferred (2022-09-28)
42kinetic_gnupg2: deferred (2022-09-28)
43devel_gnupg2: deferred (2022-09-28)
0\ No newline at end of file44\ No newline at end of file
diff --git a/lib/lp/bugs/scripts/tests/test_uctimport.py b/lib/lp/bugs/scripts/tests/test_uctimport.py
1new file mode 10064445new file mode 100644
index 0000000..2e36bb5
--- /dev/null
+++ b/lib/lp/bugs/scripts/tests/test_uctimport.py
@@ -0,0 +1,88 @@
1from pathlib import Path
2
3from lp.testing import TestCase
4from lp.testing.layers import LaunchpadZopelessLayer
5from lp.testing.script import run_script
6
7
8class TestUCTImportScript(TestCase):
9 """Test the TestUCTImportScript class."""
10
11 layer = LaunchpadZopelessLayer
12
13 def test_no_path_given(self):
14 """TestUCTImportScript errors when no path given"""
15 exit_code, out, err = run_script(
16 script="scripts/uct-import.py",
17 args=[],
18 )
19 self.assertEqual(2, exit_code)
20 self.assertEqual("", out)
21 self.assertEqual(
22 "Usage: uct-import.py [options] PATH\n\nuct-import.py: "
23 "error: Please specify a path to import\n",
24 err,
25 )
26
27 def test_load_from_file(self):
28 load_from = Path(__file__).parent / "sampledata" / "CVE-2022-23222"
29 exit_code, out, err = run_script(
30 script="scripts/uct-import.py",
31 args=[str(load_from)],
32 )
33 self.assertEqual(0, exit_code)
34 self.assertEqual("", out)
35 self.assertIn("CVE-2022-23222 was imported successfully", err)
36
37 def test_load_from_directory(self):
38 load_from = Path(__file__).parent / "sampledata"
39 exit_code, out, err = run_script(
40 script="scripts/uct-import.py",
41 args=[str(load_from)],
42 )
43 self.assertEqual(0, exit_code)
44 self.assertEqual("", out)
45 self.assertIn("CVE-2007-0255 was imported successfully", err)
46 self.assertIn("CVE-2022-3219 was imported successfully", err)
47 self.assertIn("CVE-2022-23222 was imported successfully", err)
48
49 def test_dry_run_does_not_crash(self):
50 load_from = Path(__file__).parent / "sampledata" / "CVE-2022-23222"
51 exit_code, out, err = run_script(
52 script="scripts/uct-import.py",
53 args=[str(load_from), "--dry-run"],
54 )
55 self.assertEqual(0, exit_code)
56 self.assertEqual("", out)
57 self.assertRegex(err, r"^INFO Importing.*CVE-2022-23222.*")
58
59 def test_filter_cve(self):
60 load_from = Path(__file__).parent / "sampledata"
61 exit_code, out, err = run_script(
62 script="scripts/uct-import.py",
63 args=[str(load_from), "--filter", "2007*"],
64 )
65 self.assertEqual(0, exit_code)
66 self.assertEqual("", out)
67 self.assertNotIn("CVE-2022-23222 was imported successfully", err)
68 self.assertIn("CVE-2007-0255 was imported successfully", err)
69
70 exit_code, out, err = run_script(
71 script="scripts/uct-import.py",
72 args=[str(load_from), "--filter", "2022*"],
73 )
74 self.assertEqual(0, exit_code)
75 self.assertEqual("", out)
76 self.assertIn("CVE-2022-23222 was imported successfully", err)
77 self.assertIn("CVE-2022-3219 was imported successfully", err)
78 self.assertNotIn("CVE-2007-0255 was imported successfully", err)
79
80 exit_code, out, err = run_script(
81 script="scripts/uct-import.py",
82 args=[str(load_from), "--filter", "20[02][27]*"],
83 )
84 self.assertEqual(0, exit_code)
85 self.assertEqual("", out)
86 self.assertIn("CVE-2022-23222 was imported successfully", err)
87 self.assertIn("CVE-2022-3219 was imported successfully", err)
88 self.assertIn("CVE-2007-0255 was imported successfully", err)
diff --git a/lib/lp/bugs/scripts/uctimport.py b/lib/lp/bugs/scripts/uctimport.py
0new file mode 10064489new file mode 100644
index 0000000..aed11c3
--- /dev/null
+++ b/lib/lp/bugs/scripts/uctimport.py
@@ -0,0 +1,63 @@
1import logging
2from pathlib import Path
3
4from lp.app.validators.cve import CVEREF_PATTERN
5from lp.bugs.scripts.uct import UCTImporter
6from lp.services.scripts.base import LaunchpadScript
7
8logger = logging.getLogger(__name__)
9
10
11class UCTImportScript(LaunchpadScript):
12 """CLI for UCTImport
13
14 Command line options:
15 The filter option takes a glob-style pattern.
16 Example: `2007*` filters all CVEs from the year 2007.
17 """
18
19 usage = "usage: %prog [options] PATH"
20 description = (
21 "Import bugs into Launchpad from CVE entries in ubuntu-cve-tracker. "
22 "PATH is either path to a CVE file, or path to a directory "
23 "containing the CVE files."
24 )
25 loglevel = logging.INFO
26
27 def add_my_options(self):
28 self.parser.add_option(
29 "--dry-run",
30 action="store_true",
31 dest="dry_run",
32 default=False,
33 help="Don't commit changes to the DB.",
34 )
35 self.parser.add_option(
36 "--filter",
37 action="store",
38 dest="filter",
39 default="*",
40 help="Apply given glob-style pattern to filter CVEs.",
41 )
42
43 def main(self):
44 if len(self.args) != 1:
45 self.parser.error("Please specify a path to import")
46 path = Path(self.args[0])
47 if path.is_dir():
48 logger.info(
49 "Importing CVE files from directory: %s", path.resolve()
50 )
51 cve_paths = sorted(
52 p
53 for p in path.rglob("CVE-%s" % self.options.filter)
54 if p.is_file() and CVEREF_PATTERN.match(p.name)
55 )
56 if not cve_paths:
57 logger.warning("Could not find CVE files in %s", path)
58 return
59 else:
60 cve_paths = [path]
61 importer = UCTImporter(dry_run=self.options.dry_run)
62 for cve_path in cve_paths:
63 importer.import_cve_from_file(cve_path)
diff --git a/scripts/uct-import.py b/scripts/uct-import.py
index 489d6ea..9ade412 100755
--- a/scripts/uct-import.py
+++ b/scripts/uct-import.py
@@ -4,59 +4,7 @@
4# GNU Affero General Public License version 3 (see the file LICENSE).4# GNU Affero General Public License version 3 (see the file LICENSE).
5import _pythonpath # noqa: F4015import _pythonpath # noqa: F401
66
7import logging7from lp.bugs.scripts.uctimport import UCTImportScript
8from pathlib import Path
9
10from lp.app.validators.cve import CVEREF_PATTERN
11from lp.bugs.scripts.uct import UCTImporter
12from lp.services.scripts.base import LaunchpadScript
13
14logger = logging.getLogger(__name__)
15
16
17class UCTImportScript(LaunchpadScript):
18
19 usage = "usage: %prog [options] PATH"
20 description = (
21 "Import bugs into Launchpad from CVE entries in ubuntu-cve-tracker. "
22 "PATH is either path to a CVE file, or path to a directory "
23 "containing the CVE files"
24 )
25 loglevel = logging.INFO
26
27 def add_my_options(self):
28 self.parser.add_option(
29 "--dry-run",
30 action="store_true",
31 dest="dry_run",
32 default=False,
33 help="Don't commit changes to the DB.",
34 )
35
36 def main(self):
37 if len(self.args) != 1:
38 self.parser.error("Please specify a path to import")
39
40 path = Path(self.args[0])
41 if path.is_dir():
42 logger.info(
43 "Importing CVE files from directory: %s", path.resolve()
44 )
45 cve_paths = sorted(
46 p
47 for p in path.rglob("CVE-*")
48 if p.is_file() and CVEREF_PATTERN.match(p.name)
49 )
50 if not cve_paths:
51 logger.warning("Could not find CVE files in %s", path)
52 return
53 else:
54 cve_paths = [path]
55
56 importer = UCTImporter(dry_run=self.options.dry_run)
57 for cve_path in cve_paths:
58 importer.import_cve_from_file(cve_path)
59
608
61if __name__ == "__main__":9if __name__ == "__main__":
62 script = UCTImportScript("lp.services.scripts.uctimport")10 script = UCTImportScript("lp.services.scripts.uctimport")

Subscribers

People subscribed via source and target branches

to status/vote changes: