Ubuntu CVE Tracker

Merge ~mdeslaur/ubuntu-cve-tracker:more-source-map-cache into ubuntu-cve-tracker:master

  1. Git
  2. lp:~mdeslaur/ubuntu-cve-tracker
  3. more-source-map-cache
  4. Merge into master
Proposed by Marc Deslauriers
Status: Merged
Merge reported by: Marc Deslauriers
Merged at revision: 5661c00df492483a5448327823c31f40c44bf77a
Proposed branch: ~mdeslaur/ubuntu-cve-tracker:more-source-map-cache
Merge into: ubuntu-cve-tracker:master
Diff against target: 71 lines (+22/-3)
2 files modified
scripts/gen-source-map-cache (+13/-0)
scripts/source_map.py (+9/-3)
Reviewer Review Type Date Requested Status
Alex Murray Approve
Review via email: mp+462252@code.launchpad.net

Commit message

commit 5661c00df492483a5448327823c31f40c44bf77a
Author: Marc Deslauriers <email address hidden>
Date: Tue Mar 12 12:43:20 2024 -0400

    source_map.py: use detailed cache if available

commit 3335f8ca63d56e8ea81a15f61a217103ec573fb1
Author: Marc Deslauriers <email address hidden>
Date: Tue Mar 12 12:30:11 2024 -0400

    gen-source-map-cache: create cache for detailed source map

Description of the change

This merge request will add a cache for the detailed source map, this should speed up most of the other tools in UCT.

To post a comment you must log in.
Revision history for this message
Alex Murray (alexmurray) wrote :

LGTM!

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/scripts/gen-source-map-cache b/scripts/gen-source-map-cache
2index 1c84cc6..cb86360 100755
3--- a/scripts/gen-source-map-cache
4+++ b/scripts/gen-source-map-cache
5@@ -32,19 +32,29 @@ def main():
6 sys.exit(1)
7
8 mirror = config['packages_mirror']
9+
10+ # This is used by check-syntax and active_edit
11 sources_pickle = os.path.join(mirror, 'sources_map.pickle')
12+
13+ # This is used by active_edit
14 sources_pickle_dev_proposed = os.path.join(mirror, 'sources_map_dev_proposed.pickle')
15
16+ # This is used by most of the other tools
17+ sources_pickle_detailed = os.path.join(mirror, 'sources_map_detailed.pickle')
18+
19 # Remove these first so that source_map.load doesn't load them!
20 if os.path.exists(sources_pickle):
21 os.unlink(sources_pickle)
22 if os.path.exists(sources_pickle_dev_proposed):
23 os.unlink(sources_pickle_dev_proposed)
24+ if os.path.exists(sources_pickle_detailed):
25+ os.unlink(sources_pickle_detailed)
26
27 sources = source_map.load(detailed=False)
28 sources_dev_proposed = source_map.load(pockets=["-proposed"],
29 releases=[cve_lib.devel_release],
30 detailed=False)
31+ sources_detailed = source_map.load(detailed=True)
32
33 # Use protocol=3 to remain compatible with focal
34 with open(sources_pickle, 'wb') as handle:
35@@ -53,5 +63,8 @@ def main():
36 with open(sources_pickle_dev_proposed, 'wb') as handle:
37 pickle.dump(sources_dev_proposed, handle, protocol=3)
38
39+ with open(sources_pickle_detailed, 'wb') as handle:
40+ pickle.dump(sources_detailed, handle, protocol=3)
41+
42 if __name__ == '__main__':
43 main()
44diff --git a/scripts/source_map.py b/scripts/source_map.py
45index c5ec9ad..e353fa8 100755
46--- a/scripts/source_map.py
47+++ b/scripts/source_map.py
48@@ -222,14 +222,20 @@ def load(data_type='sources', pockets=None, releases=None, skip_eol_releases=Tru
49 if 'packages_mirror' in config:
50 mirror = config['packages_mirror']
51
52- if data_type == 'sources' and skip_eol_releases == True and arch == "amd64" and detailed == False:
53- if pockets == None and releases == None:
54+ if data_type == 'sources' and skip_eol_releases == True and arch == "amd64":
55+ if detailed == False and pockets == None and releases == None:
56 sources_pickle = os.path.join(mirror, 'sources_map.pickle')
57 if os.path.exists(sources_pickle):
58 with open(sources_pickle, 'rb') as handle:
59 map = pickle.load(handle)
60 return map
61- elif pockets == ["-proposed"] and releases == [cve_lib.devel_release]:
62+ elif detailed == True and pockets == None and releases == None:
63+ sources_pickle = os.path.join(mirror, 'sources_map_detailed.pickle')
64+ if os.path.exists(sources_pickle):
65+ with open(sources_pickle, 'rb') as handle:
66+ map = pickle.load(handle)
67+ return map
68+ elif detailed == False and pockets == ["-proposed"] and releases == [cve_lib.devel_release]:
69 sources_pickle_dev_proposed = os.path.join(mirror, 'sources_map_dev_proposed.pickle')
70 if os.path.exists(sources_pickle_dev_proposed):
71 with open(sources_pickle_dev_proposed, 'rb') as handle:

Subscribers

People subscribed via source and target branches