Merge lp:~sergiodj/lp-ftbfs-report/fix-internal-pkgset-team-links into lp:lp-ftbfs-report
- fix-internal-pkgset-team-links
- Merge into trunk
Status: | Superseded |
---|---|
Proposed branch: | lp:~sergiodj/lp-ftbfs-report/fix-internal-pkgset-team-links |
Merge into: | lp:lp-ftbfs-report |
Diff against target: |
460 lines (+182/-37) (has conflicts) 3 files modified
source/build_status.html (+30/-10) source/build_status.py (+148/-27) source/style.css (+4/-0) Text conflict in source/build_status.html Text conflict in source/build_status.py |
To merge this branch: | bzr merge lp:~sergiodj/lp-ftbfs-report/fix-internal-pkgset-team-links |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
William Grant | Pending | ||
Review via email:
|
Commit message
Description of the change
When the packageset and the team names are the same, the internal links for each of these will end up also being the same. Currently, because the packageset section comes first in the page, that means that the team internal link will point to it.
A simple fix is to use distinct suffixes to compose the internal link names, which help differentiate them.
Unmerged revisions
- 65. By Sergio Durigan Junior
-
Use suffixes for pkgset and team internal links
When a packageset has the same name of a team, the internal links will
point to the same place, which currently is the packageset section
(because it comes first in the page). We can solve that by using
distinct suffixes when creating the links. - 64. By William Grant
-
Exclude copy archive failures that also failed in the primary archive.
- 63. By William Grant
-
Update source link.
- 62. By William Grant
-
Include only packages in main in team sections.
- 61. By William Grant
-
Generate per team lists.
- 60. By William Grant
-
Port to modern python-apt.
- 59. By William Grant
-
Add --release-only which skips packages that only fail outside the release pocket. Useful for port bootstrapping.
- 58. By William Grant
-
Add space before pocket name.
- 57. By William Grant
-
Fix spacing with more architectures.
- 56. By William Grant
-
Don't always use the current series when comparing test rebuilds to the primary archive. We rebuild old series too.
Preview Diff
1 | === modified file 'source/build_status.html' | |||
2 | --- source/build_status.html 2013-05-28 21:39:21 +0000 | |||
3 | +++ source/build_status.html 2021-08-13 16:24:09 +0000 | |||
4 | @@ -4,7 +4,7 @@ | |||
5 | 4 | {#- Macro definitions -#} | 4 | {#- Macro definitions -#} |
6 | 5 | {% macro legend_row(row_class, field, field_desc) -%} | 5 | {% macro legend_row(row_class, field, field_desc) -%} |
7 | 6 | <tr class="{{ row_class }}" style="text-align: center;"> | 6 | <tr class="{{ row_class }}" style="text-align: center;"> |
9 | 7 | <td class="{{ field }}">arch ({{ field[0] }})</td> | 7 | <td class="{{ field }}">arch ({{ abbrs[field] }})</td> |
10 | 8 | <td style="text-align: left;">{{ field_desc }}</td> | 8 | <td style="text-align: left;">{{ field_desc }}</td> |
11 | 9 | {% for arch in arch_list -%} | 9 | {% for arch in arch_list -%} |
12 | 10 | {% if stats[field][arch].cnt -%} | 10 | {% if stats[field][arch].cnt -%} |
13 | @@ -26,9 +26,9 @@ | |||
14 | 26 | <col width="25%" /> | 26 | <col width="25%" /> |
15 | 27 | <col width="24%" /> | 27 | <col width="24%" /> |
16 | 28 | <col width="2%" /> | 28 | <col width="2%" /> |
20 | 29 | <col width="11%" span="{{ arch_list|count }}" /> | 29 | <col width="8%" span="{{ arch_list|count }}" /> |
21 | 30 | <col width="5%" /> | 30 | <col width="6%" /> |
22 | 31 | <col width="5%" /> | 31 | <col width="6%" /> |
23 | 32 | </colgroup> | 32 | </colgroup> |
24 | 33 | <thead> | 33 | <thead> |
25 | 34 | <tr> | 34 | <tr> |
26 | @@ -79,7 +79,7 @@ | |||
27 | 79 | {% if arch in ver.logs -%} | 79 | {% if arch in ver.logs -%} |
28 | 80 | <td class="{{ ver.logs[arch].buildstate }}" | 80 | <td class="{{ ver.logs[arch].buildstate }}" |
29 | 81 | onmouseover="Tip('{{ ver.logs[arch].tooltip|e }}')" onmouseout="UnTip()"> | 81 | onmouseover="Tip('{{ ver.logs[arch].tooltip|e }}')" onmouseout="UnTip()"> |
31 | 82 | <a href="{{ ver.logs[arch].url }}">{{ arch }} ({{ ver.logs[arch].buildstate[0] }})</a> | 82 | <a href="{{ ver.logs[arch].url }}">{{ arch }} ({{ abbrs[ver.logs[arch].buildstate] }})</a> |
32 | 83 | → | 83 | → |
33 | 84 | <a href="{{ ver.logs[arch].log }}">Log</a> | 84 | <a href="{{ ver.logs[arch].log }}">Log</a> |
34 | 85 | </td> | 85 | </td> |
35 | @@ -130,16 +130,25 @@ | |||
36 | 130 | 130 | ||
37 | 131 | <p>Jump to packageset: | 131 | <p>Jump to packageset: |
38 | 132 | {% for (ps, lst) in packagesets.items()|sort if lst|count > 0 -%} | 132 | {% for (ps, lst) in packagesets.items()|sort if lst|count > 0 -%} |
42 | 133 | <a href="#{{ ps }}">{{ ps }}</a> ({{ lst|count }}) | 133 | <a href="#{{ ps }}-pkgset">{{ ps }}</a> ({{ lst|count }}) |
43 | 134 | {% endfor -%} | 134 | {% endfor -%} |
44 | 135 | </p> | 135 | </p> |
45 | 136 | <p>Jump to team: | ||
46 | 137 | {% for (team, lst) in teams.items()|sort if lst|count > 0 -%} | ||
47 | 138 | <a href="#{{ team }}-team">{{ team }}</a> ({{ lst|count }}) | ||
48 | 139 | {% endfor -%} | ||
49 | 140 | </p> | ||
50 | 141 | |||
51 | 142 | {% if notice %} | ||
52 | 143 | {{ notice }} | ||
53 | 144 | {% endif %} | ||
54 | 136 | 145 | ||
55 | 137 | <h2>Legend and statistics for the displayed data</h2> | 146 | <h2>Legend and statistics for the displayed data</h2> |
56 | 138 | <table class="grid" width="95%"> | 147 | <table class="grid" width="95%"> |
57 | 139 | <colgroup> | 148 | <colgroup> |
58 | 140 | <col width="6%" /> | 149 | <col width="6%" /> |
59 | 141 | <col width="45%" /> | 150 | <col width="45%" /> |
61 | 142 | <col width="11%" span="{{ arch_list|count }}" /> | 151 | <col width="8%" span="{{ arch_list|count }}" /> |
62 | 143 | </colgroup> | 152 | </colgroup> |
63 | 144 | <thead> | 153 | <thead> |
64 | 145 | <tr> | 154 | <tr> |
65 | @@ -159,6 +168,7 @@ | |||
66 | 159 | </thead> | 168 | </thead> |
67 | 160 | <tbody> | 169 | <tbody> |
68 | 161 | {{ legend_row('even', 'FAILEDTOBUILD', 'Package failed to build') }} | 170 | {{ legend_row('even', 'FAILEDTOBUILD', 'Package failed to build') }} |
69 | 171 | {{ legend_row('odd', 'CANCELLED', 'Cancelled build') }} | ||
70 | 162 | {{ legend_row('odd', 'MANUALDEPWAIT', 'Package is waiting on another package') }} | 172 | {{ legend_row('odd', 'MANUALDEPWAIT', 'Package is waiting on another package') }} |
71 | 163 | {{ legend_row('even', 'CHROOTWAIT', 'Failure in the chroot') }} | 173 | {{ legend_row('even', 'CHROOTWAIT', 'Failure in the chroot') }} |
72 | 164 | {{ legend_row('odd', 'UPLOADFAIL', 'Failed to upload') }} | 174 | {{ legend_row('odd', 'UPLOADFAIL', 'Failed to upload') }} |
73 | @@ -179,10 +189,16 @@ | |||
74 | 179 | 189 | ||
75 | 180 | <h2>Packagesets</h2> | 190 | <h2>Packagesets</h2> |
76 | 181 | {% for (ps, lst) in packagesets.items()|sort if lst|count > 0 -%} | 191 | {% for (ps, lst) in packagesets.items()|sort if lst|count > 0 -%} |
78 | 182 | <h3 id="{{ ps }}">{{ ps }}: {{ lst|count }} packages (<a href="#top">top</a>)</h3> | 192 | <h3 id="{{ ps }}-pkgset">{{ ps }}: {{ lst|count }} packages (<a href="#top">top</a>)</h3> |
79 | 183 | {{ table(lst, ps, 'Also belongs to another packageset?') }} | 193 | {{ table(lst, ps, 'Also belongs to another packageset?') }} |
80 | 184 | {% endfor -%} | 194 | {% endfor -%} |
81 | 185 | 195 | ||
82 | 196 | <h2>Teams (main only)</h2> | ||
83 | 197 | {% for (team, lst) in teams.items()|sort if lst|count > 0 -%} | ||
84 | 198 | <h3 id="{{ team }}-team">{{ team }}: {{ lst|count }} packages (<a href="#top">top</a>)</h3> | ||
85 | 199 | {{ table(lst, team, 'Also belongs to another team?') }} | ||
86 | 200 | {% endfor -%} | ||
87 | 201 | |||
88 | 186 | {% if main_archive %} | 202 | {% if main_archive %} |
89 | 187 | <h2 id="main_superseded">main (superseded): {{ main_superseded|count }} packages (<a href="#top">top</a>)</h2> | 203 | <h2 id="main_superseded">main (superseded): {{ main_superseded|count }} packages (<a href="#top">top</a>)</h2> |
90 | 188 | {{ table(main_superseded) }} | 204 | {{ table(main_superseded) }} |
91 | @@ -217,6 +233,7 @@ | |||
92 | 217 | alt="Valid XHTML 1.1" height="31" width="88" | 233 | alt="Valid XHTML 1.1" height="31" width="88" |
93 | 218 | style="border-width:0px" /></a> | 234 | style="border-width:0px" /></a> |
94 | 219 | </p> | 235 | </p> |
95 | 236 | <<<<<<< TREE | ||
96 | 220 | <p style="font-size:smaller">Source: <a href="https://code.launchpad.net/lp-ftbfs-report">lp:lp-ftbfs-report</a></p> | 237 | <p style="font-size:smaller">Source: <a href="https://code.launchpad.net/lp-ftbfs-report">lp:lp-ftbfs-report</a></p> |
97 | 221 | <script type="text/javascript" src="source/wz_tooltip.js"></script> | 238 | <script type="text/javascript" src="source/wz_tooltip.js"></script> |
98 | 222 | <script type="text/javascript" src="source/jquery.min.js"></script> | 239 | <script type="text/javascript" src="source/jquery.min.js"></script> |
99 | @@ -357,5 +374,8 @@ | |||
100 | 357 | /* Build failures raw data */ | 374 | /* Build failures raw data */ |
101 | 358 | var json_data={{ historical_json }}; | 375 | var json_data={{ historical_json }}; |
102 | 359 | </script> | 376 | </script> |
103 | 377 | ======= | ||
104 | 378 | <p style="font-size:smaller">Source: <a href="https://code.launchpad.net/~wgrant/lp-ftbfs-report/production">lp:~wgrant/lp-ftbfs-report/production</a></p> | ||
105 | 379 | >>>>>>> MERGE-SOURCE | ||
106 | 360 | </body> | 380 | </body> |
107 | 361 | </html> | 381 | </html> |
108 | 362 | 382 | ||
109 | === modified file 'source/build_status.py' | |||
110 | --- source/build_status.py 2012-04-14 17:11:03 +0000 | |||
111 | +++ source/build_status.py 2021-08-13 16:24:09 +0000 | |||
112 | @@ -20,6 +20,8 @@ | |||
113 | 20 | #import httplib2 | 20 | #import httplib2 |
114 | 21 | #httplib2.debuglevel = 1 | 21 | #httplib2.debuglevel = 1 |
115 | 22 | 22 | ||
116 | 23 | import os | ||
117 | 24 | import requests | ||
118 | 23 | import sys | 25 | import sys |
119 | 24 | import time | 26 | import time |
120 | 25 | import apt_pkg | 27 | import apt_pkg |
121 | @@ -30,12 +32,13 @@ | |||
122 | 30 | from launchpadlib.errors import HTTPError | 32 | from launchpadlib.errors import HTTPError |
123 | 31 | from launchpadlib.launchpad import Launchpad | 33 | from launchpadlib.launchpad import Launchpad |
124 | 32 | from operator import (attrgetter, methodcaller) | 34 | from operator import (attrgetter, methodcaller) |
125 | 35 | from optparse import OptionParser | ||
126 | 33 | 36 | ||
127 | 34 | lp_service = 'production' | 37 | lp_service = 'production' |
128 | 35 | api_version = 'devel' | 38 | api_version = 'devel' |
129 | 36 | default_arch_list = [] | 39 | default_arch_list = [] |
130 | 37 | find_tagged_bugs = 'ftbfs' | 40 | find_tagged_bugs = 'ftbfs' |
132 | 38 | apt_pkg.InitSystem() | 41 | apt_pkg.init_system() |
133 | 39 | 42 | ||
134 | 40 | class PersonTeam(object): | 43 | class PersonTeam(object): |
135 | 41 | _cache = dict() | 44 | _cache = dict() |
136 | @@ -77,7 +80,7 @@ | |||
137 | 77 | class VersionList(list): | 80 | class VersionList(list): |
138 | 78 | def append(self, item): | 81 | def append(self, item): |
139 | 79 | super(SourcePackage.VersionList, self).append(item) | 82 | super(SourcePackage.VersionList, self).append(item) |
141 | 80 | self.sort(key = attrgetter('version'), cmp = apt_pkg.VersionCompare) | 83 | self.sort(key = attrgetter('version'), cmp = apt_pkg.version_compare) |
142 | 81 | 84 | ||
143 | 82 | def __new__(cls, spph): | 85 | def __new__(cls, spph): |
144 | 83 | try: | 86 | try: |
145 | @@ -99,6 +102,10 @@ | |||
146 | 99 | for ps in srcpkg.packagesets: | 102 | for ps in srcpkg.packagesets: |
147 | 100 | packagesets_ftbfs[ps].append(srcpkg) | 103 | packagesets_ftbfs[ps].append(srcpkg) |
148 | 101 | 104 | ||
149 | 105 | srcpkg.teams = set([team for (team, srcpkglist) in teams.items() if spph.source_package_name in srcpkglist and spph.component_name == "main"]) | ||
150 | 106 | for team in srcpkg.teams: | ||
151 | 107 | teams_ftbfs[team].append(srcpkg) | ||
152 | 108 | |||
153 | 102 | # add to cache | 109 | # add to cache |
154 | 103 | cls._cache[spph.source_package_name] = srcpkg | 110 | cls._cache[spph.source_package_name] = srcpkg |
155 | 104 | 111 | ||
156 | @@ -136,6 +143,33 @@ | |||
157 | 136 | else: | 143 | else: |
158 | 137 | return list(self.packagesets.difference((name,))) | 144 | return list(self.packagesets.difference((name,))) |
159 | 138 | 145 | ||
160 | 146 | class MainArchiveBuilds(object): | ||
161 | 147 | _cache = dict() | ||
162 | 148 | |||
163 | 149 | def __new__(cls, main_archive, source, version): | ||
164 | 150 | try: | ||
165 | 151 | return cls._cache["%s,%s" % (source, version)] | ||
166 | 152 | except KeyError: | ||
167 | 153 | bfm = super(MainArchiveBuilds, cls).__new__(cls) | ||
168 | 154 | results = {} | ||
169 | 155 | sourcepubs = main_archive.getPublishedSources( | ||
170 | 156 | exact_match=True, source_name=source, version=version) | ||
171 | 157 | for pub in sourcepubs: | ||
172 | 158 | for build in pub.getBuilds(): | ||
173 | 159 | # assumes sourcepubs are sorted latest release to oldest, | ||
174 | 160 | # so first record wins | ||
175 | 161 | if build.arch_tag not in results: | ||
176 | 162 | results[build.arch_tag] = build.buildstate | ||
177 | 163 | bfm.results = results | ||
178 | 164 | # add to cache | ||
179 | 165 | cls._cache["%s,%s" % (source, version)] = bfm | ||
180 | 166 | |||
181 | 167 | return bfm | ||
182 | 168 | |||
183 | 169 | @classmethod | ||
184 | 170 | def clear(cls): | ||
185 | 171 | cls._cache.clear() | ||
186 | 172 | |||
187 | 139 | class SPPH(object): | 173 | class SPPH(object): |
188 | 140 | _cache = dict() # dict with all SPPH objects | 174 | _cache = dict() # dict with all SPPH objects |
189 | 141 | 175 | ||
190 | @@ -172,6 +206,7 @@ | |||
191 | 172 | 'Dependency wait': 'MANUALDEPWAIT', | 206 | 'Dependency wait': 'MANUALDEPWAIT', |
192 | 173 | 'Chroot problem': 'CHROOTWAIT', | 207 | 'Chroot problem': 'CHROOTWAIT', |
193 | 174 | 'Failed to upload': 'UPLOADFAIL', | 208 | 'Failed to upload': 'UPLOADFAIL', |
194 | 209 | 'Cancelled build': 'CANCELLED', | ||
195 | 175 | } | 210 | } |
196 | 176 | self.buildstate = buildstates[build.buildstate] | 211 | self.buildstate = buildstates[build.buildstate] |
197 | 177 | self.url = build.web_link | 212 | self.url = build.web_link |
198 | @@ -203,7 +238,7 @@ | |||
199 | 203 | return u'Changed-By: %s' % (self.changed_by) | 238 | return u'Changed-By: %s' % (self.changed_by) |
200 | 204 | 239 | ||
201 | 205 | 240 | ||
203 | 206 | def fetch_pkg_list(archive, series, state, last_published, arch_list=default_arch_list, main_archive=None, main_series=None): | 241 | def fetch_pkg_list(archive, series, state, last_published, arch_list=default_arch_list, main_archive=None, main_series=None, release_only=False): |
204 | 207 | print "Processing '%s'" % state | 242 | print "Processing '%s'" % state |
205 | 208 | 243 | ||
206 | 209 | cur_last_published = None | 244 | cur_last_published = None |
207 | @@ -250,30 +285,59 @@ | |||
208 | 250 | version=spph._lp.source_package_version, | 285 | version=spph._lp.source_package_version, |
209 | 251 | status='Published') | 286 | status='Published') |
210 | 252 | spph.current = len(main_publications[:1]) > 0 | 287 | spph.current = len(main_publications[:1]) > 0 |
211 | 288 | elif release_only: | ||
212 | 289 | release_publications = archive.getPublishedSources( | ||
213 | 290 | distro_series=series, | ||
214 | 291 | pocket='Release', | ||
215 | 292 | exact_match=True, | ||
216 | 293 | source_name=spph._lp.source_package_name, | ||
217 | 294 | version=spph._lp.source_package_version, | ||
218 | 295 | status='Published') | ||
219 | 296 | spph.current = len(release_publications[:1]) > 0 | ||
220 | 297 | if not spph.current: | ||
221 | 298 | release_publications = archive.getPublishedSources( | ||
222 | 299 | distro_series=series, | ||
223 | 300 | pocket='Release', | ||
224 | 301 | exact_match=True, | ||
225 | 302 | source_name=spph._lp.source_package_name, | ||
226 | 303 | version=spph._lp.source_package_version, | ||
227 | 304 | status='Pending') | ||
228 | 305 | spph.current = len(release_publications[:1]) > 0 | ||
229 | 253 | else: | 306 | else: |
230 | 254 | spph.current = True | 307 | spph.current = True |
231 | 255 | 308 | ||
232 | 256 | if not spph.current: | 309 | if not spph.current: |
233 | 257 | print " superseded" | 310 | print " superseded" |
234 | 311 | |||
235 | 312 | if main_archive: | ||
236 | 313 | # If this build failure is not a regression versus the | ||
237 | 314 | # main archive, do not report it. | ||
238 | 315 | main_builds = MainArchiveBuilds(main_archive, | ||
239 | 316 | spph._lp.source_package_name, | ||
240 | 317 | spph._lp.source_package_version) | ||
241 | 318 | try: | ||
242 | 319 | if main_builds.results[arch] != 'Successfully built': | ||
243 | 320 | print " Skipping %s" % build.title | ||
244 | 321 | continue | ||
245 | 322 | except KeyError: | ||
246 | 323 | pass | ||
247 | 324 | |||
248 | 258 | SPPH(csp_link).addBuildLog(build) | 325 | SPPH(csp_link).addBuildLog(build) |
249 | 259 | 326 | ||
250 | 260 | return cur_last_published | 327 | return cur_last_published |
251 | 261 | 328 | ||
252 | 262 | 329 | ||
259 | 263 | def generate_page(archive, series, archs_by_archive, main_archive, template = 'build_status.html', arch_list = default_arch_list): | 330 | def generate_page(name, archive, series, archs_by_archive, main_archive, template = 'build_status.html', arch_list = default_arch_list, notice=None, release_only=False): |
254 | 264 | try: | ||
255 | 265 | out = open('../%s-%s.html' % (archive.name, series.name), 'w') | ||
256 | 266 | except IOError: | ||
257 | 267 | return | ||
258 | 268 | |||
260 | 269 | # sort the package lists | 331 | # sort the package lists |
261 | 270 | filter_ftbfs = lambda pkglist, current: filter(methodcaller('isFTBFS', arch_list, current), sorted(pkglist)) | 332 | filter_ftbfs = lambda pkglist, current: filter(methodcaller('isFTBFS', arch_list, current), sorted(pkglist)) |
262 | 271 | data = {} | 333 | data = {} |
263 | 272 | for comp in ('main', 'restricted', 'universe', 'multiverse'): | 334 | for comp in ('main', 'restricted', 'universe', 'multiverse'): |
264 | 273 | data[comp] = filter_ftbfs(components[comp], True) | 335 | data[comp] = filter_ftbfs(components[comp], True) |
266 | 274 | data['%s_superseded' % comp] = filter_ftbfs(components[comp], False) | 336 | data['%s_superseded' % comp] = filter_ftbfs(components[comp], False) if not release_only else [] |
267 | 275 | for pkgset, pkglist in packagesets_ftbfs.items(): | 337 | for pkgset, pkglist in packagesets_ftbfs.items(): |
268 | 276 | packagesets_ftbfs[pkgset] = filter_ftbfs(pkglist, True) | 338 | packagesets_ftbfs[pkgset] = filter_ftbfs(pkglist, True) |
269 | 339 | for team, pkglist in teams_ftbfs.items(): | ||
270 | 340 | teams_ftbfs[team] = filter_ftbfs(pkglist, True) | ||
271 | 277 | 341 | ||
272 | 278 | # container object to hold the counts and the tooltip | 342 | # container object to hold the counts and the tooltip |
273 | 279 | class StatData(object): | 343 | class StatData(object): |
274 | @@ -284,7 +348,7 @@ | |||
275 | 284 | 348 | ||
276 | 285 | # compute some statistics (number of packages for each build failure type) | 349 | # compute some statistics (number of packages for each build failure type) |
277 | 286 | stats = {} | 350 | stats = {} |
279 | 287 | for state in ('FAILEDTOBUILD', 'MANUALDEPWAIT', 'CHROOTWAIT', 'UPLOADFAIL'): | 351 | for state in ('FAILEDTOBUILD', 'MANUALDEPWAIT', 'CHROOTWAIT', 'UPLOADFAIL', 'CANCELLED'): |
280 | 288 | stats[state] = {} | 352 | stats[state] = {} |
281 | 289 | for arch in arch_list: | 353 | for arch in arch_list: |
282 | 290 | tooltip = [] | 354 | tooltip = [] |
283 | @@ -313,31 +377,47 @@ | |||
284 | 313 | data['archs_by_archive'] = archs_by_archive | 377 | data['archs_by_archive'] = archs_by_archive |
285 | 314 | data['lastupdate'] = time.strftime('%F %T %z') | 378 | data['lastupdate'] = time.strftime('%F %T %z') |
286 | 315 | data['packagesets'] = packagesets_ftbfs | 379 | data['packagesets'] = packagesets_ftbfs |
287 | 380 | <<<<<<< TREE | ||
288 | 316 | data['historical_json'] = generate_historical_json(archive, series) | 381 | data['historical_json'] = generate_historical_json(archive, series) |
289 | 382 | ======= | ||
290 | 383 | data['teams'] = teams_ftbfs | ||
291 | 384 | data['notice'] = notice | ||
292 | 385 | data['abbrs'] = { | ||
293 | 386 | 'FAILEDTOBUILD': 'F', | ||
294 | 387 | 'CANCELLED': 'X', | ||
295 | 388 | 'MANUALDEPWAIT': 'M', | ||
296 | 389 | 'CHROOTWAIT': 'C', | ||
297 | 390 | 'UPLOADFAIL': 'U', | ||
298 | 391 | } | ||
299 | 392 | >>>>>>> MERGE-SOURCE | ||
300 | 317 | 393 | ||
301 | 318 | env = Environment(loader=FileSystemLoader('.')) | 394 | env = Environment(loader=FileSystemLoader('.')) |
302 | 319 | template = env.get_template('build_status.html') | 395 | template = env.get_template('build_status.html') |
303 | 320 | stream = template.render(**data) | 396 | stream = template.render(**data) |
304 | 397 | |||
305 | 398 | fn = '../%s.html' % name | ||
306 | 399 | out = open('%s.new' % fn, 'w') | ||
307 | 321 | out.write(stream.encode('utf-8')) | 400 | out.write(stream.encode('utf-8')) |
308 | 322 | out.close() | 401 | out.close() |
309 | 402 | os.rename('%s.new' % fn, fn) | ||
310 | 323 | 403 | ||
313 | 324 | def generate_csvfile(archive, series, arch_list = default_arch_list): | 404 | def generate_csvfile(name, arch_list = default_arch_list): |
314 | 325 | csvout = open('../%s-%s.csv' % (archive.name, series.name), 'w') | 405 | csvout = open('../%s.csv' % name, 'w') |
315 | 326 | linetemplate = '%(name)s,%(link)s,%(explain)s\n' | 406 | linetemplate = '%(name)s,%(link)s,%(explain)s\n' |
316 | 327 | for comp in components.values(): | 407 | for comp in components.values(): |
317 | 328 | for pkg in comp: | 408 | for pkg in comp: |
318 | 329 | for ver in pkg.versions: | 409 | for ver in pkg.versions: |
320 | 330 | for state in ('FAILEDTOBUILD', 'MANUALDEPWAIT', 'CHROOTWAIT', 'UPLOADFAIL'): | 410 | for state in ('FAILEDTOBUILD', 'MANUALDEPWAIT', 'CHROOTWAIT', 'UPLOADFAIL', 'CANCELLED'): |
321 | 331 | archs = [ arch for (arch, log) in ver.logs.items() if log.buildstate == state ] | 411 | archs = [ arch for (arch, log) in ver.logs.items() if log.buildstate == state ] |
322 | 332 | if archs: | 412 | if archs: |
323 | 333 | log = ver.logs[archs[0]].log | 413 | log = ver.logs[archs[0]].log |
324 | 334 | csvout.write(linetemplate % {'name': pkg.name, 'link': log, | 414 | csvout.write(linetemplate % {'name': pkg.name, 'link': log, |
325 | 335 | 'explain':"[%s] %s" %(', '.join(archs), state)}) | 415 | 'explain':"[%s] %s" %(', '.join(archs), state)}) |
326 | 336 | 416 | ||
328 | 337 | def load_timestamps(archive, series): | 417 | def load_timestamps(name): |
329 | 338 | '''Load the saved timestamps about the last still published FTBFS build record.''' | 418 | '''Load the saved timestamps about the last still published FTBFS build record.''' |
330 | 339 | try: | 419 | try: |
332 | 340 | timestamp_file = file('%s-%s.json' % (archive.name, series.name), 'r') | 420 | timestamp_file = file('%s.json' % name, 'r') |
333 | 341 | tmp = json.load(timestamp_file) | 421 | tmp = json.load(timestamp_file) |
334 | 342 | timestamps = {} | 422 | timestamps = {} |
335 | 343 | for state, timestamp in tmp.items(): | 423 | for state, timestamp in tmp.items(): |
336 | @@ -352,11 +432,12 @@ | |||
337 | 352 | 'Dependency wait': None, | 432 | 'Dependency wait': None, |
338 | 353 | 'Chroot problem': None, | 433 | 'Chroot problem': None, |
339 | 354 | 'Failed to upload': None, | 434 | 'Failed to upload': None, |
340 | 435 | 'Cancelled build': None, | ||
341 | 355 | } | 436 | } |
342 | 356 | 437 | ||
344 | 357 | def save_timestamps(archive, series, timestamps): | 438 | def save_timestamps(name, timestamps): |
345 | 358 | '''Save the timestamps of the last still published FTBFS build record into a JSON file.''' | 439 | '''Save the timestamps of the last still published FTBFS build record into a JSON file.''' |
347 | 359 | timestamp_file = file('%s-%s.json' % (archive.name, series.name), 'w') | 440 | timestamp_file = file('%s.json' % name, 'w') |
348 | 360 | tmp = {} | 441 | tmp = {} |
349 | 361 | for state, timestamp in timestamps.items(): | 442 | for state, timestamp in timestamps.items(): |
350 | 362 | if timestamp is not None: | 443 | if timestamp is not None: |
351 | @@ -439,6 +520,7 @@ | |||
352 | 439 | launchpad = Launchpad.login_anonymously('qa-ftbfs', lp_service, version=api_version) | 520 | launchpad = Launchpad.login_anonymously('qa-ftbfs', lp_service, version=api_version) |
353 | 440 | 521 | ||
354 | 441 | ubuntu = launchpad.distributions['ubuntu'] | 522 | ubuntu = launchpad.distributions['ubuntu'] |
355 | 523 | <<<<<<< TREE | ||
356 | 442 | assert len(sys.argv) >= 4 | 524 | assert len(sys.argv) >= 4 |
357 | 443 | 525 | ||
358 | 444 | try: | 526 | try: |
359 | @@ -451,15 +533,44 @@ | |||
360 | 451 | except HTTPError: | 533 | except HTTPError: |
361 | 452 | print 'Error: %s is not a valid series.' % sys.argv[2] | 534 | print 'Error: %s is not a valid series.' % sys.argv[2] |
362 | 453 | sys.exit(1) | 535 | sys.exit(1) |
363 | 536 | ======= | ||
364 | 537 | |||
365 | 538 | usage = "usage: %prog [options] <archive> <series> <arch> [<arch> ...]" | ||
366 | 539 | parser = OptionParser(usage=usage) | ||
367 | 540 | parser.add_option( | ||
368 | 541 | "-f", "--filename", dest="name", | ||
369 | 542 | help="File name prefix for the result.") | ||
370 | 543 | parser.add_option( | ||
371 | 544 | "-n", "--notice", dest="notice_file", | ||
372 | 545 | help="HTML notice file to include in the page header.") | ||
373 | 546 | parser.add_option( | ||
374 | 547 | "--release-only", dest="release_only", action="store_true", | ||
375 | 548 | help="Only include sources currently published in the release pocket.") | ||
376 | 549 | (options, args) = parser.parse_args() | ||
377 | 550 | if len(args) < 3: | ||
378 | 551 | parser.error("Need at least 4 arguments.") | ||
379 | 552 | |||
380 | 553 | try: | ||
381 | 554 | archive = ubuntu.getArchive(name=args[0]) | ||
382 | 555 | except HTTPError: | ||
383 | 556 | print 'Error: %s is not a valid archive.' % args[0] | ||
384 | 557 | try: | ||
385 | 558 | series = ubuntu.getSeries(name_or_version=args[1]) | ||
386 | 559 | except HTTPError: | ||
387 | 560 | print 'Error: %s is not a valid series.' % args[1] | ||
388 | 561 | |||
389 | 562 | if options.name is None: | ||
390 | 563 | options.name = '%s-%s' % (archive.name, series.name) | ||
391 | 564 | >>>>>>> MERGE-SOURCE | ||
392 | 454 | 565 | ||
393 | 455 | if archive.name != 'primary': | 566 | if archive.name != 'primary': |
394 | 456 | main_archive = ubuntu.main_archive | 567 | main_archive = ubuntu.main_archive |
396 | 457 | main_series = ubuntu.current_series | 568 | main_series = series |
397 | 458 | else: | 569 | else: |
398 | 459 | main_archive = main_series = None | 570 | main_archive = main_series = None |
399 | 460 | 571 | ||
400 | 461 | archs_by_archive = dict(main=[], ports=[]) | 572 | archs_by_archive = dict(main=[], ports=[]) |
402 | 462 | for arch in sys.argv[3:]: | 573 | for arch in args[2:]: |
403 | 463 | das = series.getDistroArchSeries(archtag=arch) | 574 | das = series.getDistroArchSeries(archtag=arch) |
404 | 464 | archs_by_archive[das.official and 'main' or 'ports'].append(arch) | 575 | archs_by_archive[das.official and 'main' or 'ports'].append(arch) |
405 | 465 | default_arch_list.extend(archs_by_archive['main']) | 576 | default_arch_list.extend(archs_by_archive['main']) |
406 | @@ -472,7 +583,7 @@ | |||
407 | 472 | PersonTeam.clear() | 583 | PersonTeam.clear() |
408 | 473 | SourcePackage.clear() | 584 | SourcePackage.clear() |
409 | 474 | SPPH.clear() | 585 | SPPH.clear() |
411 | 475 | last_published = load_timestamps(archive, series) | 586 | last_published = load_timestamps(options.name) |
412 | 476 | 587 | ||
413 | 477 | # list of SourcePackages for each component | 588 | # list of SourcePackages for each component |
414 | 478 | components = { | 589 | components = { |
415 | @@ -491,14 +602,24 @@ | |||
416 | 491 | packagesets[ps.name] = ps.getSourcesIncluded(direct_inclusion=False) | 602 | packagesets[ps.name] = ps.getSourcesIncluded(direct_inclusion=False) |
417 | 492 | packagesets_ftbfs[ps.name] = [] # empty list to add FTBFS for each package set later | 603 | packagesets_ftbfs[ps.name] = [] # empty list to add FTBFS for each package set later |
418 | 493 | 604 | ||
423 | 494 | for state in ('Failed to build', 'Dependency wait', 'Chroot problem', 'Failed to upload'): | 605 | teams = requests.get('https://people.canonical.com/~ubuntu-archive/package-team-mapping.json').json() |
424 | 495 | last_published[state] = fetch_pkg_list(archive, series, state, last_published[state], default_arch_list, main_archive, main_series) | 606 | |
425 | 496 | 607 | # Per team list of FTBFS | |
426 | 497 | save_timestamps(archive, series, last_published) | 608 | teams_ftbfs = {team: [] for team in teams} |
427 | 609 | |||
428 | 610 | for state in ('Failed to build', 'Dependency wait', 'Chroot problem', 'Failed to upload', 'Cancelled build'): | ||
429 | 611 | last_published[state] = fetch_pkg_list(archive, series, state, last_published[state], default_arch_list, main_archive, main_series, options.release_only) | ||
430 | 612 | |||
431 | 613 | save_timestamps(options.name, last_published) | ||
432 | 614 | |||
433 | 615 | if options.notice_file: | ||
434 | 616 | notice = open(options.notice_file).read() | ||
435 | 617 | else: | ||
436 | 618 | notice = None | ||
437 | 498 | 619 | ||
438 | 499 | print "Updating Historical database..." | 620 | print "Updating Historical database..." |
439 | 500 | store_historical(archive, series) | 621 | store_historical(archive, series) |
440 | 501 | print "Generating HTML page..." | 622 | print "Generating HTML page..." |
442 | 502 | generate_page(archive, series, archs_by_archive, main_archive) | 623 | generate_page(options.name, archive, series, archs_by_archive, main_archive, notice=notice, release_only=options.release_only) |
443 | 503 | print "Generating CSV file..." | 624 | print "Generating CSV file..." |
445 | 504 | generate_csvfile(archive, series) | 625 | generate_csvfile(options.name) |
446 | 505 | 626 | ||
447 | === modified file 'source/style.css' | |||
448 | --- source/style.css 2011-10-16 14:22:34 +0000 | |||
449 | +++ source/style.css 2021-08-13 16:24:09 +0000 | |||
450 | @@ -52,6 +52,10 @@ | |||
451 | 52 | background-color:#cc0000; | 52 | background-color:#cc0000; |
452 | 53 | } | 53 | } |
453 | 54 | 54 | ||
454 | 55 | .CANCELLED { | ||
455 | 56 | background-color:#aa0000; | ||
456 | 57 | } | ||
457 | 58 | |||
458 | 55 | .MANUALDEPWAIT { | 59 | .MANUALDEPWAIT { |
459 | 56 | background:#ff9900; | 60 | background:#ff9900; |
460 | 57 | } | 61 | } |