Merge lp:~toykeeper/qakit/add-pt-to-html into lp:qakit
- add-pt-to-html
- Merge into trunk
Status: | Needs review |
---|---|
Proposed branch: | lp:~toykeeper/qakit/add-pt-to-html |
Merge into: | lp:qakit |
Diff against target: |
812 lines (+777/-0) 5 files modified
qakit/practitest/all-suites.sh (+9/-0) qakit/practitest/practitest.py (+126/-0) qakit/practitest/pt-export.py (+495/-0) qakit/practitest/scripts.js (+97/-0) qakit/practitest/style.css (+50/-0) |
To merge this branch: | bzr merge lp:~toykeeper/qakit/add-pt-to-html |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Canonical Platform QA Team | Pending | ||
Review via email: mp+259446@code.launchpad.net |
Commit message
Added a PractiTest suite exporter for easy viewing of cases.
Description of the change
Added a PractiTest suite exporter for easy viewing of cases.
- 12. By Selene ToyKeeper
-
Made details much wider (is its own table row, made some code more complex),
made entire row clickable to expand/collapse details,
made search filter each word individually,
added expand/collapse all links. - 13. By Selene ToyKeeper
-
Added -r / --regenerate CLI option.
PT can only export every 2 hours, so it helps to be able to control
whether the export is re-requested or not. - 14. By Selene ToyKeeper
-
Added support for new test suite schema. Choose a schema based on project ID.
Added a count of matching tests in current view. - 15. By Selene ToyKeeper
-
Fixed pep8 lint complaints.
(except for the ones relating to vim's indent settings
and commented-out lines of code) - 16. By Selene ToyKeeper
-
Small bug fix in filter function; could match strings which spanned multiple html tags.
Allowed linewrap in "domain" column.
Minor code cleaning. - 17. By Selene ToyKeeper
-
Added a script to export all current PT projects.
- 18. By Selene ToyKeeper
-
Ensure row colors will still alternate after a search.
Allan LeSage (allanlesage) wrote : | # |
This is big! I'm looking through but will need a more thorough review than I'm able to give presently.
Selene ToyKeeper (toykeeper) wrote : | # |
It might help to first merge bzr+ssh:
Max Brustkern (nuclearbob) wrote : | # |
I'm not sure how to do it, but if you set that branch as a prerequisite, it'll only show the new diff.
Brendan Donegan (brendan-donegan) wrote : | # |
You need to resubmit the MP and can specify the prerequisite there
On Fri, May 22, 2015 at 10:17 PM, Max Brustkern <<email address hidden>
> wrote:
> I'm not sure how to do it, but if you set that branch as a prerequisite,
> it'll only show the new diff.
> --
> https:/
> Your team Canonical Platform QA Team is requested to review the proposed
> merge of lp:~toykeeper/qakit/add-pt-to-html into lp:qakit.
>
- 19. By Selene ToyKeeper
-
Added ability to filter during conversion, and to dump external IDs.
Made it easier to filter client-side html by automated status. - 20. By Selene ToyKeeper
-
Renamed pt-to-html to pt-export.
- 21. By Selene ToyKeeper
-
merged updates from trunk
Unmerged revisions
- 21. By Selene ToyKeeper
-
merged updates from trunk
- 20. By Selene ToyKeeper
-
Renamed pt-to-html to pt-export.
- 19. By Selene ToyKeeper
-
Added ability to filter during conversion, and to dump external IDs.
Made it easier to filter client-side html by automated status. - 18. By Selene ToyKeeper
-
Ensure row colors will still alternate after a search.
- 17. By Selene ToyKeeper
-
Added a script to export all current PT projects.
- 16. By Selene ToyKeeper
-
Small bug fix in filter function; could match strings which spanned multiple html tags.
Allowed linewrap in "domain" column.
Minor code cleaning. - 15. By Selene ToyKeeper
-
Fixed pep8 lint complaints.
(except for the ones relating to vim's indent settings
and commented-out lines of code) - 14. By Selene ToyKeeper
-
Added support for new test suite schema. Choose a schema based on project ID.
Added a count of matching tests in current view. - 13. By Selene ToyKeeper
-
Added -r / --regenerate CLI option.
PT can only export every 2 hours, so it helps to be able to control
whether the export is re-requested or not. - 12. By Selene ToyKeeper
-
Made details much wider (is its own table row, made some code more complex),
made entire row clickable to expand/collapse details,
made search filter each word individually,
added expand/collapse all links.
Preview Diff
1 | === added file 'qakit/practitest/all-suites.sh' | |||
2 | --- qakit/practitest/all-suites.sh 1970-01-01 00:00:00 +0000 | |||
3 | +++ qakit/practitest/all-suites.sh 2015-08-10 20:32:55 +0000 | |||
4 | @@ -0,0 +1,9 @@ | |||
5 | 1 | #!/bin/sh | ||
6 | 2 | |||
7 | 3 | # LANG is required to avoid python unicode issues | ||
8 | 4 | export LANG=en_US.UTF-8 | ||
9 | 5 | export PYTHONPATH=../.. | ||
10 | 6 | for suite in 1294 1548 ; do | ||
11 | 7 | perl -i.old -pe 's/^(PRACTITEST_PROJECT_ID) = (\d+)/$1 = '$suite'/;' config.ini | ||
12 | 8 | ./pt-export.py $* -o /tmp/$suite.html | ||
13 | 9 | done | ||
14 | 0 | 10 | ||
15 | === modified file 'qakit/practitest/practitest.py' | |||
16 | --- qakit/practitest/practitest.py 2015-06-30 14:44:59 +0000 | |||
17 | +++ qakit/practitest/practitest.py 2015-08-10 20:32:55 +0000 | |||
18 | @@ -17,7 +17,11 @@ | |||
19 | 17 | 17 | ||
20 | 18 | import json | 18 | import json |
21 | 19 | import logging | 19 | import logging |
22 | 20 | import os | ||
23 | 20 | import requests | 21 | import requests |
24 | 22 | import subprocess | ||
25 | 23 | import time | ||
26 | 24 | import xlrd | ||
27 | 21 | 25 | ||
28 | 22 | from pprint import pformat | 26 | from pprint import pformat |
29 | 23 | 27 | ||
30 | @@ -324,3 +328,125 @@ | |||
31 | 324 | logging.debug(pformat(result)) | 328 | logging.debug(pformat(result)) |
32 | 325 | logging.info('Updating issue pt:{} / lp:{}'.format(ptid, lpid)) | 329 | logging.info('Updating issue pt:{} / lp:{}'.format(ptid, lpid)) |
33 | 326 | return self._put(url, data=result) | 330 | return self._put(url, data=result) |
34 | 331 | |||
35 | 332 | def request_export(self, entity='Step'): | ||
36 | 333 | """Ask the server to queue an export for all data of one type. | ||
37 | 334 | |||
38 | 335 | Will create a list of Issues, Tests , TestSets, Requirements or Steps, | ||
39 | 336 | in a CSV format, to be able to export these entities to a regular | ||
40 | 337 | spreadsheet. The method will put the export in the queue, and usually | ||
41 | 338 | within couple of minutes it's ready to download. | ||
42 | 339 | |||
43 | 340 | Parameters: | ||
44 | 341 | :param entity: one of the following: 'Issue', 'Test', 'TestSet', | ||
45 | 342 | 'Requirement', 'Step' | ||
46 | 343 | |||
47 | 344 | Returns the following: | ||
48 | 345 | id: the id of the export - a reference to receive the output, | ||
49 | 346 | hours_gap: the # of hours until the user can re-create the same report | ||
50 | 347 | hours_to_delete: the # of hours until the system will auto-delete this | ||
51 | 348 | file | ||
52 | 349 | """ | ||
53 | 350 | #print('request_export(%s)' % (entity)) | ||
54 | 351 | url = 'https://prod.practitest.com/api/exporter.json' | ||
55 | 352 | data = dict(project_id=self.project_id, entity=entity) | ||
56 | 353 | val = self._post(url, data).json() | ||
57 | 354 | if 'error' in val: | ||
58 | 355 | raise ValueError(val['error']) | ||
59 | 356 | return val | ||
60 | 357 | |||
61 | 358 | def poll_export(self, id_): | ||
62 | 359 | """Will check the status of a requested export. | ||
63 | 360 | Parameters: | ||
64 | 361 | :param id_: from the request_export method above | ||
65 | 362 | |||
66 | 363 | Returns: | ||
67 | 364 | id: the id of this export, | ||
68 | 365 | updated_at: the time it was updated | ||
69 | 366 | status: the of this export (In Queue, working, completed), | ||
70 | 367 | in_progress: true if it's not completed, | ||
71 | 368 | url: the location to take the file from (via wget or curl) | ||
72 | 369 | """ | ||
73 | 370 | print('poll_export(%s)' % (id_)) | ||
74 | 371 | url = 'https://prod.practitest.com/api/exporter/%s.json' % (id_) | ||
75 | 372 | val = self._get(url).json() | ||
76 | 373 | return val | ||
77 | 374 | |||
78 | 375 | def get_completed_export(self, url): | ||
79 | 376 | """Download a completed export file and return its local filesystem path. | ||
80 | 377 | """ | ||
81 | 378 | save_path = '/tmp/export.%s' % (time.strftime('%Y-%m-%d_%H:%M:%S')) | ||
82 | 379 | #print('get_completed_export(%s) -> %s' % (url, save_path)) | ||
83 | 380 | cmd = ('wget', '-o', '/dev/null', '-O', save_path, url) | ||
84 | 381 | subprocess.check_call(cmd) | ||
85 | 382 | # PT sends a .xslx in a .zip file :( | ||
86 | 383 | unzipped_dir = '%s.d' % (save_path) | ||
87 | 384 | if not os.path.exists(unzipped_dir): os.makedirs(unzipped_dir) | ||
88 | 385 | prev_dir = os.getcwd() | ||
89 | 386 | os.chdir(unzipped_dir) | ||
90 | 387 | cmd = ('unzip', save_path) | ||
91 | 388 | subprocess.check_call(cmd) | ||
92 | 389 | os.chdir(prev_dir) | ||
93 | 390 | # find the unzipped .xlsx file | ||
94 | 391 | matches = [x for x in os.listdir(unzipped_dir) if x.endswith('.xlsx')] | ||
95 | 392 | if len(matches) != 1: | ||
96 | 393 | raise ValueError('PT export file had unexpected contents.') | ||
97 | 394 | xlsx_path = os.path.join(unzipped_dir, matches[0]) | ||
98 | 395 | #print('get_completed_export() done') | ||
99 | 396 | return xlsx_path | ||
100 | 397 | |||
101 | 398 | def load_csv_export(self, path): | ||
102 | 399 | """Load a .csv export file saved by get_completed_export(). | ||
103 | 400 | """ | ||
104 | 401 | #print('load_csv_export(%s)' % (path)) | ||
105 | 402 | wb = xlrd.open_workbook(path) | ||
106 | 403 | names = wb.sheet_names() | ||
107 | 404 | if len(names) != 1: | ||
108 | 405 | raise ValueError('XLSX sheet layout not as expected') | ||
109 | 406 | sheet = wb.sheet_by_index(0) | ||
110 | 407 | return sheet | ||
111 | 408 | |||
112 | 409 | def request_and_wait_for_exports(self, entities=None, ids=None): | ||
113 | 410 | #print('request_and_wait_for_exports(): %s' % (', '.join(entities))) | ||
114 | 411 | if not entities: entities=[] | ||
115 | 412 | if not ids: ids=[] | ||
116 | 413 | |||
117 | 414 | requested = [] | ||
118 | 415 | for i, entity in enumerate(entities): | ||
119 | 416 | if len(ids) > i: # already downloaded, ID known | ||
120 | 417 | continue | ||
121 | 418 | d = dict(done=False, entity=entity) | ||
122 | 419 | j = self.request_export(entity) | ||
123 | 420 | print(j) | ||
124 | 421 | for f in ('id', 'hours_gap', 'hours_to_delete'): | ||
125 | 422 | d[f] = j[f] | ||
126 | 423 | #print('request_export(%s): %s=%s' % (entity, f, d[f])) | ||
127 | 424 | requested.append(d) | ||
128 | 425 | |||
129 | 426 | for i, id_ in enumerate(ids): | ||
130 | 427 | if len(entities) > i: # already downloaded, ID known | ||
131 | 428 | entity = entities[i] | ||
132 | 429 | else: | ||
133 | 430 | entity = 'unknown' | ||
134 | 431 | d = dict(done=False, id=id_, entity=entity) | ||
135 | 432 | requested.append(d) | ||
136 | 433 | |||
137 | 434 | while True: | ||
138 | 435 | for d in requested: | ||
139 | 436 | if not d['done']: | ||
140 | 437 | j = self.poll_export(d['id']) | ||
141 | 438 | for f in ('updated_at', 'status', 'in_progress', 'url'): | ||
142 | 439 | d[f] = j[f] | ||
143 | 440 | print('poll_export(%s): %s=%s' % (d['id'], f, d[f])) | ||
144 | 441 | print('request_and_wait_for_export(%s): id=%s, status=%s' | ||
145 | 442 | % (d['entity'], d['id'], d['status'])) | ||
146 | 443 | if not d['in_progress']: | ||
147 | 444 | d['saved_path'] = self.get_completed_export(d['url']) | ||
148 | 445 | d['data'] = self.load_csv_export(d['saved_path']) | ||
149 | 446 | d['done'] = True | ||
150 | 447 | |||
151 | 448 | done = all([d['done'] for d in requested]) | ||
152 | 449 | if done: break | ||
153 | 450 | time.sleep(5) | ||
154 | 451 | |||
155 | 452 | return requested | ||
156 | 327 | 453 | ||
157 | === added file 'qakit/practitest/pt-export.py' | |||
158 | --- qakit/practitest/pt-export.py 1970-01-01 00:00:00 +0000 | |||
159 | +++ qakit/practitest/pt-export.py 2015-08-10 20:32:55 +0000 | |||
160 | @@ -0,0 +1,495 @@ | |||
161 | 1 | #!/usr/bin/env python3 | ||
162 | 2 | |||
163 | 3 | # Export PractiTest data to HTML for viewing without an account | ||
164 | 4 | |||
165 | 5 | # Copyright (C) 2015 Canonical | ||
166 | 6 | # | ||
167 | 7 | # Author: Selene Scriven | ||
168 | 8 | # | ||
169 | 9 | # This program is free software: you can redistribute it and/or modify | ||
170 | 10 | # it under the terms of the GNU General Public License as published by | ||
171 | 11 | # the Free Software Foundation, either version 3 of the License, or | ||
172 | 12 | # (at your option) any later version. | ||
173 | 13 | # | ||
174 | 14 | # This program is distributed in the hope that it will be useful, | ||
175 | 15 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
176 | 16 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
177 | 17 | # GNU General Public License for more details. | ||
178 | 18 | # | ||
179 | 19 | # You should have received a copy of the GNU General Public License | ||
180 | 20 | # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
181 | 21 | |||
182 | 22 | import configparser | ||
183 | 23 | import os | ||
184 | 24 | import sys | ||
185 | 25 | |||
186 | 26 | import practitest | ||
187 | 27 | |||
188 | 28 | #from pprint import pprint | ||
189 | 29 | |||
190 | 30 | cfg = None | ||
191 | 31 | |||
192 | 32 | |||
193 | 33 | def main(args): | ||
194 | 34 | """pt-export: Export PractiTest data to plain HTML. | ||
195 | 35 | Usage: pt-export [options] [filters] | ||
196 | 36 | -h --help Show this help message and exit | ||
197 | 37 | -c --cfg PATH Read config from PATH (default: config.ini) | ||
198 | 38 | -o --out PATH Write output to PATH | ||
199 | 39 | -r --regenerate Ask PT for a new export instead of previous one. | ||
200 | 40 | --dump-extid Export external IDs in a tab-separated format. | ||
201 | 41 | |||
202 | 42 | Filters can be any of the following: (values are not case sensitive) | ||
203 | 43 | Column=value Require test column to have an exact value. | ||
204 | 44 | Column like value Require value to be somewhere in test column. | ||
205 | 45 | any=value Require at least one column to have an exact value. | ||
206 | 46 | any like value Require at least one column to contain value. | ||
207 | 47 | value Same as 'any like value'. | ||
208 | 48 | """ | ||
209 | 49 | global cfg | ||
210 | 50 | cfg_path = 'config.ini' | ||
211 | 51 | out_path = '' | ||
212 | 52 | out_fmt = 'html' | ||
213 | 53 | regenerate = False | ||
214 | 54 | filters = [] | ||
215 | 55 | |||
216 | 56 | # Very simple CLI, nothing fancy needed | ||
217 | 57 | i = 0 | ||
218 | 58 | while i < len(args): | ||
219 | 59 | a = args[i] | ||
220 | 60 | if a in ('-h', '--help'): | ||
221 | 61 | return usage() | ||
222 | 62 | elif a in ('-c', '--cfg', '--config'): | ||
223 | 63 | i += 1 | ||
224 | 64 | cfg_path = args[i] | ||
225 | 65 | elif a in ('-o', '--out'): | ||
226 | 66 | i += 1 | ||
227 | 67 | out_path = args[i] | ||
228 | 68 | elif a in ('-r', '--regenerate'): | ||
229 | 69 | regenerate = True | ||
230 | 70 | elif a in ('--dump-extid',): | ||
231 | 71 | out_fmt = 'extid' | ||
232 | 72 | elif a in ('-u', '--force-unicode'): | ||
233 | 73 | # TODO: Find a way to force UTF-8 for all I/O instead of ascii | ||
234 | 74 | #os.environ['LANG'] = 'en_US.UTF-8' | ||
235 | 75 | # Instead, run the script like this: | ||
236 | 76 | # LANG=en_US.UTF-8 ./pt-export.py | ||
237 | 77 | pass | ||
238 | 78 | else: | ||
239 | 79 | #return usage() | ||
240 | 80 | filters.append(a) | ||
241 | 81 | i += 1 | ||
242 | 82 | |||
243 | 83 | cfg = read_config(cfg_path) | ||
244 | 84 | if cfg['practitest_project_id'] not in ('1294', '1548'): | ||
245 | 85 | raise ValueError('Unknown project ID: %s' | ||
246 | 86 | % (cfg['practitest_project_id'])) | ||
247 | 87 | PT = practitest.PractitestSession( | ||
248 | 88 | cfg['practitest_project_id'], | ||
249 | 89 | cfg['practitest_api_key'], | ||
250 | 90 | cfg['practitest_api_secret_key'], | ||
251 | 91 | cfg['practitest_user_email']) | ||
252 | 92 | |||
253 | 93 | export_file(PT, out_path, regenerate, out_fmt, filters) | ||
254 | 94 | |||
255 | 95 | |||
256 | 96 | def usage(): | ||
257 | 97 | print(main.__doc__) | ||
258 | 98 | |||
259 | 99 | |||
260 | 100 | def export_file(PT, out_path, regenerate, out_fmt, filters): | ||
261 | 101 | global cfg | ||
262 | 102 | |||
263 | 103 | print('Downloading tests...') | ||
264 | 104 | if regenerate: | ||
265 | 105 | exports = PT.request_and_wait_for_exports(entities=['Test']) | ||
266 | 106 | else: | ||
267 | 107 | if cfg['practitest_project_id'] == '1294': | ||
268 | 108 | exports = PT.request_and_wait_for_exports(entities=['Test'], | ||
269 | 109 | ids=[41]) | ||
270 | 110 | elif cfg['practitest_project_id'] == '1548': | ||
271 | 111 | exports = PT.request_and_wait_for_exports(entities=['Test'], | ||
272 | 112 | ids=[330]) | ||
273 | 113 | |||
274 | 114 | # So, PT includes steps in 'Test' export, even though steps aren't | ||
275 | 115 | # in the Test API... not necessary to load 'Step' export too. | ||
276 | 116 | tests = exports[0]['data'] | ||
277 | 117 | |||
278 | 118 | print('Parsing tests...') | ||
279 | 119 | tests = interpret_test_sheet(tests) | ||
280 | 120 | |||
281 | 121 | print('Filtering tests...') | ||
282 | 122 | tests = filter_tests(tests, filters) | ||
283 | 123 | |||
284 | 124 | print('Formatting tests...') | ||
285 | 125 | if out_fmt == 'html': | ||
286 | 126 | lines = exported_tests_to_html(tests) | ||
287 | 127 | elif out_fmt == 'extid': | ||
288 | 128 | lines = dump_extid(tests) | ||
289 | 129 | else: | ||
290 | 130 | raise ValueError('Unknown output format: %s' % out_fmt) | ||
291 | 131 | |||
292 | 132 | print('Saving...') | ||
293 | 133 | if out_path: | ||
294 | 134 | open(out_path, 'w').writelines(lines) | ||
295 | 135 | |||
296 | 136 | |||
297 | 137 | def filter_tests(tests, filters): | ||
298 | 138 | """Interpret a list of filters, apply them to tests, return all matches. | ||
299 | 139 | Return all tests by default. | ||
300 | 140 | Filters are a list in any of the following formats: | ||
301 | 141 | Column=value | ||
302 | 142 | Column like value | ||
303 | 143 | value | ||
304 | 144 | The 'Column' part may be 'any' to match all columns. | ||
305 | 145 | The plain 'value' format translates to 'any like value'. | ||
306 | 146 | """ | ||
307 | 147 | |||
308 | 148 | if not filters: return tests | ||
309 | 149 | |||
310 | 150 | # First, interpret the filters | ||
311 | 151 | parsed_filters = [] | ||
312 | 152 | for f in filters: | ||
313 | 153 | if '=' in f: | ||
314 | 154 | left, right = f.split('=') | ||
315 | 155 | right = right.lower() | ||
316 | 156 | parsed_filters.append(('=', left, right)) | ||
317 | 157 | elif ' like ' in f: | ||
318 | 158 | left, right = f.split(' like ') | ||
319 | 159 | right = right.lower() | ||
320 | 160 | parsed_filters.append(('like', left, right)) | ||
321 | 161 | else: | ||
322 | 162 | parsed_filters.append(('like', 'any', f)) | ||
323 | 163 | |||
324 | 164 | # Apply the filters | ||
325 | 165 | filtered = [] | ||
326 | 166 | for test in tests: | ||
327 | 167 | match = True | ||
328 | 168 | |||
329 | 169 | for type_, key, needle in parsed_filters: | ||
330 | 170 | values = [] | ||
331 | 171 | if key == 'any': | ||
332 | 172 | for key in test: | ||
333 | 173 | values.append(str(test[key]).lower()) | ||
334 | 174 | else: | ||
335 | 175 | if key not in test: | ||
336 | 176 | match = False | ||
337 | 177 | continue | ||
338 | 178 | val = str(test[key]).lower() | ||
339 | 179 | values = [val] | ||
340 | 180 | |||
341 | 181 | found = False | ||
342 | 182 | for val in values: | ||
343 | 183 | if type_ == '=': | ||
344 | 184 | if val == needle: | ||
345 | 185 | found = True | ||
346 | 186 | elif type_ == 'like': | ||
347 | 187 | if needle in val: | ||
348 | 188 | found = True | ||
349 | 189 | if not found: | ||
350 | 190 | match = False | ||
351 | 191 | |||
352 | 192 | if match: | ||
353 | 193 | filtered.append(test) | ||
354 | 194 | |||
355 | 195 | return filtered | ||
356 | 196 | |||
357 | 197 | |||
358 | 198 | def dump_extid(tests): | ||
359 | 199 | """Make a report of test ID, external test ID, and test name | ||
360 | 200 | (in a tab-separated format, nulls will be represented as '-') | ||
361 | 201 | """ | ||
362 | 202 | lines = [] | ||
363 | 203 | columns = ('id', 'External ID', 'Name') | ||
364 | 204 | for t in tests: | ||
365 | 205 | fields = [] | ||
366 | 206 | for c in columns: | ||
367 | 207 | v = str(t[c]) | ||
368 | 208 | if not v: v = '-' | ||
369 | 209 | fields.append(v) | ||
370 | 210 | line = '\t'.join(fields) + '\n' | ||
371 | 211 | lines.append(line) | ||
372 | 212 | |||
373 | 213 | return lines | ||
374 | 214 | |||
375 | 215 | |||
376 | 216 | def interpret_test_sheet(tests): | ||
377 | 217 | """Convert the PT xlsx export into a usable object tree. | ||
378 | 218 | The input file is an awkward cross between a database and a spreadsheet, | ||
379 | 219 | with related tables (Steps) broken out into an embedded sheet rectangle | ||
380 | 220 | for each parent object... so it's a little awkward. | ||
381 | 221 | """ | ||
382 | 222 | |||
383 | 223 | # pull column names from sheet | ||
384 | 224 | # save it into a format of test_columns[title] = (index, conversion func) | ||
385 | 225 | test_columns = dict() | ||
386 | 226 | columns = tests.row_values(0) | ||
387 | 227 | for i, c in enumerate(columns): | ||
388 | 228 | #print('col %i: %s' % (i, c)) | ||
389 | 229 | test_columns[c] = [i, str] | ||
390 | 230 | |||
391 | 231 | def int_or_none(x): | ||
392 | 232 | if x == '': | ||
393 | 233 | return None | ||
394 | 234 | else: | ||
395 | 235 | return int(x) | ||
396 | 236 | |||
397 | 237 | # convert non-string columns | ||
398 | 238 | test_columns['id'][1] = int_or_none | ||
399 | 239 | test_columns['Step position'][1] = int_or_none | ||
400 | 240 | |||
401 | 241 | def tests_with_steps(tests): | ||
402 | 242 | merged = [] | ||
403 | 243 | for i in range(1, tests.nrows): | ||
404 | 244 | row = tests.row_values(i) | ||
405 | 245 | rowlen = tests.row_len(i) | ||
406 | 246 | |||
407 | 247 | # is this is a continuation of the previous row? | ||
408 | 248 | if row[0]: | ||
409 | 249 | t = dict() | ||
410 | 250 | merged.append(t) | ||
411 | 251 | t['steps'] = [] # start with no test steps | ||
412 | 252 | child = False | ||
413 | 253 | else: | ||
414 | 254 | t = merged[-1] | ||
415 | 255 | child = True | ||
416 | 256 | |||
417 | 257 | step = dict() | ||
418 | 258 | for name, (col, func) in test_columns.items(): | ||
419 | 259 | # skip empty child cells | ||
420 | 260 | if child and not row[col]: | ||
421 | 261 | continue | ||
422 | 262 | try: | ||
423 | 263 | val = func(row[col]) | ||
424 | 264 | except Exception as e: | ||
425 | 265 | print('Warn: row %i conversion error: %s' % (i, str(e))) | ||
426 | 266 | val = row[col] | ||
427 | 267 | if name.startswith('Step '): | ||
428 | 268 | step[name] = val | ||
429 | 269 | else: | ||
430 | 270 | if child: | ||
431 | 271 | t[name] = t[name] + '\n' + val | ||
432 | 272 | else: | ||
433 | 273 | t[name] = val | ||
434 | 274 | if ('Step position' in step) \ | ||
435 | 275 | and (step['Step position'] is not None): | ||
436 | 276 | t['steps'].append(step) | ||
437 | 277 | |||
438 | 278 | #print('%s: %s (%i steps)' % (t['id'], t['Name'], len(t['steps']))) | ||
439 | 279 | #pprint(t['steps']) | ||
440 | 280 | |||
441 | 281 | return merged | ||
442 | 282 | |||
443 | 283 | all_tests = tests_with_steps(tests) | ||
444 | 284 | |||
445 | 285 | return all_tests | ||
446 | 286 | |||
447 | 287 | |||
448 | 288 | def exported_tests_to_html(tests): | ||
449 | 289 | |||
450 | 290 | def cr_to_br(t): | ||
451 | 291 | return t.replace('\n', '<br />') | ||
452 | 292 | |||
453 | 293 | def nobr(t): | ||
454 | 294 | return str(t).replace(' ', ' ') | ||
455 | 295 | |||
456 | 296 | def step_field(step, key): | ||
457 | 297 | if key in step: | ||
458 | 298 | val = step[key] | ||
459 | 299 | if isinstance(val, str): | ||
460 | 300 | val = cr_to_br(val) | ||
461 | 301 | return val | ||
462 | 302 | else: | ||
463 | 303 | return '' | ||
464 | 304 | |||
465 | 305 | def fmt_steps(steps, test, tr_class): | ||
466 | 306 | fmt_steps.counter += 1 | ||
467 | 307 | lines = [] | ||
468 | 308 | if len(steps) > 0: | ||
469 | 309 | label = '%i steps' % (len(steps)) | ||
470 | 310 | else: | ||
471 | 311 | label = '...' | ||
472 | 312 | lines.append('%s <br />' % (label)) | ||
473 | 313 | hide_id = 'steps-%s' % (fmt_steps.counter) | ||
474 | 314 | #lines.append('''<span class="clickable" ' | ||
475 | 315 | # 'onclick="toggle_display('%s')">%s</span><br />''' | ||
476 | 316 | # % (hide_id, label)) | ||
477 | 317 | lines.append('</td></tr>') | ||
478 | 318 | lines.append('<tr class="%s child"><td></td><td></td><td colspan="%s">' | ||
479 | 319 | % (tr_class, len(columns) - 2)) | ||
480 | 320 | |||
481 | 321 | lines.append('<table class="steps" id="%s"><tr><td>' % (hide_id)) | ||
482 | 322 | |||
483 | 323 | for extra in ['External ID', 'Description', 'Pre-requisites and setup', | ||
484 | 324 | 'Whiteboard', 'Comments']: | ||
485 | 325 | val = get_field(test, extra) | ||
486 | 326 | if val and (val != 'N/A'): | ||
487 | 327 | lines.append('<b>%s</b>:<br />%s<br />' | ||
488 | 328 | % (extra, cr_to_br(val))) | ||
489 | 329 | |||
490 | 330 | if len(steps) > 0: | ||
491 | 331 | lines.append('<b>Steps</b>:<br />') | ||
492 | 332 | lines.append('<table>') | ||
493 | 333 | lines.append('<tr>') | ||
494 | 334 | for header in ('#', '', 'desc', 'result'): | ||
495 | 335 | lines.append('<th align="left">%s</th>' % (header)) | ||
496 | 336 | lines.append('</tr>') | ||
497 | 337 | i = 0 | ||
498 | 338 | for step in steps: | ||
499 | 339 | i += 1 | ||
500 | 340 | lines.append('<tr class="step-%s">' % (i % 2)) | ||
501 | 341 | for field in ('Step position', 'Step name', | ||
502 | 342 | 'Step description', 'Step expected_results'): | ||
503 | 343 | lines.append('<td>%s </td>' % step_field(step, field)) | ||
504 | 344 | lines.append('</tr>') | ||
505 | 345 | lines.append('</table>') | ||
506 | 346 | |||
507 | 347 | lines.append('</td></tr></table>') | ||
508 | 348 | return '\n'.join(lines) | ||
509 | 349 | fmt_steps.counter = 0 | ||
510 | 350 | |||
511 | 351 | def fmt_automated(x): | ||
512 | 352 | if x.lower() == 'yes': return 'automated' | ||
513 | 353 | else: return 'manual' | ||
514 | 354 | |||
515 | 355 | if cfg['practitest_project_id'] == '1294': | ||
516 | 356 | columns = [ | ||
517 | 357 | ('id', 'ID', str, False), | ||
518 | 358 | ('Test Level', 'Suite', str, False), | ||
519 | 359 | ('Subsystem', 'Category', nobr, False), | ||
520 | 360 | ('Component', 'Subcat', str, False), | ||
521 | 361 | ('Name', 'Title', str, False), | ||
522 | 362 | ('Status', 'Status', str, False), | ||
523 | 363 | ('Tags', 'Tags', str, False), | ||
524 | 364 | ('Devices', 'Devices', str, False), | ||
525 | 365 | ('Release', 'Release', str, False), | ||
526 | 366 | ('Automated', 'Automated', fmt_automated, False), | ||
527 | 367 | ('steps', 'Steps', fmt_steps, True), | ||
528 | 368 | ] | ||
529 | 369 | elif cfg['practitest_project_id'] == '1548': | ||
530 | 370 | columns = [ | ||
531 | 371 | ('id', 'ID', str, False), | ||
532 | 372 | ('Test Level', 'Suite', str, False), | ||
533 | 373 | ('Image Part', 'Part', str, False), | ||
534 | 374 | ('Test Domain', 'Domain', str, False), | ||
535 | 375 | ('Application', 'App', str, False), | ||
536 | 376 | ('Name', 'Title', str, False), | ||
537 | 377 | ('Status', 'Status', str, False), | ||
538 | 378 | ('Tags', 'Tags', str, False), | ||
539 | 379 | ('Devices', 'Devices', str, False), | ||
540 | 380 | ('Release', 'Release', str, False), | ||
541 | 381 | ('Automated', 'Automated', fmt_automated, False), | ||
542 | 382 | ('steps', 'Steps', fmt_steps, True), | ||
543 | 383 | ] | ||
544 | 384 | |||
545 | 385 | # Sort by various fields... | ||
546 | 386 | if cfg['practitest_project_id'] == '1294': | ||
547 | 387 | foo = [( | ||
548 | 388 | t['Test Level'] or '', | ||
549 | 389 | t['Subsystem'] or '', | ||
550 | 390 | t['Component'] or '', | ||
551 | 391 | t['id'], t) for t in tests] | ||
552 | 392 | elif cfg['practitest_project_id'] == '1548': | ||
553 | 393 | foo = [( | ||
554 | 394 | t['Test Level'] or '', | ||
555 | 395 | t['Test Domain'] or '', | ||
556 | 396 | t['Image Part'] or '', | ||
557 | 397 | t['Application'] or '', | ||
558 | 398 | t['id'], t) for t in tests] | ||
559 | 399 | foo.sort() | ||
560 | 400 | tests = [x[-1] for x in foo] | ||
561 | 401 | |||
562 | 402 | lines = [] | ||
563 | 403 | lines.append('<html>') | ||
564 | 404 | lines.append('<head>') | ||
565 | 405 | lines.append('<title>All Tests</title>') | ||
566 | 406 | lines.append('<style><!--') | ||
567 | 407 | lines.append(open('style.css').read()) | ||
568 | 408 | lines.append('--></style>') | ||
569 | 409 | lines.append('<script>') | ||
570 | 410 | lines.append(open('scripts.js').read()) | ||
571 | 411 | lines.append('</script>') | ||
572 | 412 | lines.append('</head>') | ||
573 | 413 | lines.append('<body>') | ||
574 | 414 | |||
575 | 415 | lines.append('Filter:') | ||
576 | 416 | lines.append('<input type="search" class="light-table-filter" ' | ||
577 | 417 | 'name="filter-string" data-table="test-table" ' | ||
578 | 418 | 'placeholder="search string"></input>') | ||
579 | 419 | |||
580 | 420 | lines.append(' - ') | ||
581 | 421 | lines.append('<span id="numtests">%i</span> tests' % (len(tests))) | ||
582 | 422 | lines.append(' - ') | ||
583 | 423 | lines.append('<span class="clickable" onclick="toggle_all_steps(1)">' | ||
584 | 424 | 'Expand All</span>') | ||
585 | 425 | lines.append(' - ') | ||
586 | 426 | lines.append('<span class="clickable" onclick="toggle_all_steps(0)">' | ||
587 | 427 | 'Collapse All</span>') | ||
588 | 428 | |||
589 | 429 | lines.append('<hr />') | ||
590 | 430 | |||
591 | 431 | lines.append('<table id="all" class="test-table table">') | ||
592 | 432 | |||
593 | 433 | def table_header(): | ||
594 | 434 | lines.append('<thead><tr>') | ||
595 | 435 | for field, title, func, full in columns: | ||
596 | 436 | lines.append('<th align="left">%s</th>' % (title)) | ||
597 | 437 | lines.append('</tr></thead>') | ||
598 | 438 | |||
599 | 439 | def none_to_empty(v): | ||
600 | 440 | if v is None: | ||
601 | 441 | return '' | ||
602 | 442 | return v | ||
603 | 443 | |||
604 | 444 | def get_field(test, key): | ||
605 | 445 | if key in test: | ||
606 | 446 | return none_to_empty(test[key]) | ||
607 | 447 | return None | ||
608 | 448 | |||
609 | 449 | table_header() | ||
610 | 450 | lines.append('<tbody>') | ||
611 | 451 | i = 0 | ||
612 | 452 | for test in tests: | ||
613 | 453 | i += 1 | ||
614 | 454 | #if (i%25) == 1: # repeat header every 25 tests | ||
615 | 455 | # table_header() | ||
616 | 456 | tr_class = 'tests-%s' % (i % 2) | ||
617 | 457 | hide_id = 'steps-%s' % (fmt_steps.counter + 1) | ||
618 | 458 | lines.append('''<tr class="%s" onclick="toggle_display('%s')">''' | ||
619 | 459 | % (tr_class, hide_id)) | ||
620 | 460 | for field, title, func, full in columns: | ||
621 | 461 | val = get_field(test, field) | ||
622 | 462 | if full: | ||
623 | 463 | val = func(val, test, tr_class) | ||
624 | 464 | else: | ||
625 | 465 | val = func(val) | ||
626 | 466 | # the extra space here prevents a bug where | ||
627 | 467 | # a search for "emul" would match "Device</td><td>Multimedia" | ||
628 | 468 | lines.append('<td>{} </td>'.format(val)) | ||
629 | 469 | lines.append('</tr>') | ||
630 | 470 | |||
631 | 471 | lines.append('</tbody>') | ||
632 | 472 | lines.append('</table>') | ||
633 | 473 | |||
634 | 474 | lines.append('</body>') | ||
635 | 475 | lines.append('</html>') | ||
636 | 476 | |||
637 | 477 | return lines | ||
638 | 478 | |||
639 | 479 | |||
640 | 480 | def read_config(path, section='DEFAULT'): | ||
641 | 481 | """Get values from config.ini as a dict (env vars override file)""" | ||
642 | 482 | # Read config as dict | ||
643 | 483 | cfg = configparser.ConfigParser() | ||
644 | 484 | cfg.read(path) | ||
645 | 485 | cfg = dict(cfg[section]) | ||
646 | 486 | # let env vars override config file | ||
647 | 487 | for k in cfg: | ||
648 | 488 | if k.upper() in os.environ: | ||
649 | 489 | cfg[k] = os.environ[k.upper()] | ||
650 | 490 | return cfg | ||
651 | 491 | |||
652 | 492 | |||
653 | 493 | if __name__ == "__main__": | ||
654 | 494 | import sys | ||
655 | 495 | main(sys.argv[1:]) | ||
656 | 0 | 496 | ||
657 | === added file 'qakit/practitest/scripts.js' | |||
658 | --- qakit/practitest/scripts.js 1970-01-01 00:00:00 +0000 | |||
659 | +++ qakit/practitest/scripts.js 2015-08-10 20:32:55 +0000 | |||
660 | @@ -0,0 +1,97 @@ | |||
661 | 1 | function show(target) { | ||
662 | 2 | document.getElementById(target).style.display = 'block'; | ||
663 | 3 | } | ||
664 | 4 | |||
665 | 5 | function hide(target) { | ||
666 | 6 | document.getElementById(target).style.display = 'none'; | ||
667 | 7 | } | ||
668 | 8 | |||
669 | 9 | function toggle_display(target) { | ||
670 | 10 | if (document.getElementById(target).style.display == 'block') { | ||
671 | 11 | hide(target); | ||
672 | 12 | } else { | ||
673 | 13 | show(target); | ||
674 | 14 | } | ||
675 | 15 | } | ||
676 | 16 | |||
677 | 17 | function toggle_all_steps(expand) { | ||
678 | 18 | for (i=1; ; i++) { | ||
679 | 19 | target = "steps-" + i; | ||
680 | 20 | if (document.getElementById(target)) { | ||
681 | 21 | if (expand == 1) { | ||
682 | 22 | show(target); | ||
683 | 23 | } else if (expand == 0) { | ||
684 | 24 | hide(target); | ||
685 | 25 | } else if (expand == -1) { | ||
686 | 26 | toggle_display(target); | ||
687 | 27 | } | ||
688 | 28 | } else { | ||
689 | 29 | break; | ||
690 | 30 | } | ||
691 | 31 | } | ||
692 | 32 | } | ||
693 | 33 | |||
694 | 34 | (function(document) { | ||
695 | 35 | 'use strict'; | ||
696 | 36 | |||
697 | 37 | var LightTableFilter = (function(Arr) { | ||
698 | 38 | |||
699 | 39 | var _input; | ||
700 | 40 | var matches = 0; | ||
701 | 41 | |||
702 | 42 | function _onInputEvent(e) { | ||
703 | 43 | _input = e.target; | ||
704 | 44 | matches = 0; | ||
705 | 45 | var tables = document.getElementsByClassName(_input.getAttribute('data-table')); | ||
706 | 46 | Arr.forEach.call(tables, function(table) { | ||
707 | 47 | Arr.forEach.call(table.tBodies, function(tbody) { | ||
708 | 48 | // rows come in pairs; main row and collapse-able details | ||
709 | 49 | var row; | ||
710 | 50 | for(row=0; row<tbody.rows.length; row+=2) { | ||
711 | 51 | _filter(tbody.rows[row],tbody.rows[row+1]); | ||
712 | 52 | } | ||
713 | 53 | }); | ||
714 | 54 | }); | ||
715 | 55 | document.getElementById("numtests").innerHTML = matches; | ||
716 | 56 | } | ||
717 | 57 | |||
718 | 58 | function _filter(row1,row2) { | ||
719 | 59 | var text = row1.textContent.toLowerCase() | ||
720 | 60 | + row2.textContent.toLowerCase(); | ||
721 | 61 | // search for each word separately; require all words to match | ||
722 | 62 | var vals = _input.value.toLowerCase().split(' '); | ||
723 | 63 | var show = 1; | ||
724 | 64 | var i, val; | ||
725 | 65 | for(i=0; i<vals.length; i++) { | ||
726 | 66 | val = vals[i]; | ||
727 | 67 | if(text.indexOf(val) < 0) { show = 0; } | ||
728 | 68 | } | ||
729 | 69 | // count how many items will be visible | ||
730 | 70 | if (show) { matches ++; } | ||
731 | 71 | // hide non-matching rows | ||
732 | 72 | row1.style.display = show ? 'table-row' : 'none'; | ||
733 | 73 | row2.style.display = row1.style.display; | ||
734 | 74 | // preserve alternating row colors | ||
735 | 75 | var newclass = "tests-" + (matches%2); | ||
736 | 76 | row1.setAttribute('class', newclass); | ||
737 | 77 | row2.setAttribute('class', newclass); | ||
738 | 78 | } | ||
739 | 79 | |||
740 | 80 | return { | ||
741 | 81 | init: function() { | ||
742 | 82 | var inputs = document.getElementsByClassName('light-table-filter'); | ||
743 | 83 | Arr.forEach.call(inputs, function(input) { | ||
744 | 84 | input.oninput = _onInputEvent; | ||
745 | 85 | }); | ||
746 | 86 | } | ||
747 | 87 | }; | ||
748 | 88 | })(Array.prototype); | ||
749 | 89 | |||
750 | 90 | document.addEventListener('readystatechange', function() { | ||
751 | 91 | if (document.readyState === 'complete') { | ||
752 | 92 | LightTableFilter.init(); | ||
753 | 93 | } | ||
754 | 94 | }); | ||
755 | 95 | |||
756 | 96 | })(document); | ||
757 | 97 | |||
758 | 0 | 98 | ||
759 | === added file 'qakit/practitest/style.css' | |||
760 | --- qakit/practitest/style.css 1970-01-01 00:00:00 +0000 | |||
761 | +++ qakit/practitest/style.css 2015-08-10 20:32:55 +0000 | |||
762 | @@ -0,0 +1,50 @@ | |||
763 | 1 | table { | ||
764 | 2 | border: none; | ||
765 | 3 | border-collapse: collapse; | ||
766 | 4 | } | ||
767 | 5 | |||
768 | 6 | #all { | ||
769 | 7 | } | ||
770 | 8 | |||
771 | 9 | #all th { | ||
772 | 10 | background-color: #ffeedd; | ||
773 | 11 | } | ||
774 | 12 | |||
775 | 13 | #all td { | ||
776 | 14 | vertical-align: top; | ||
777 | 15 | padding-right: 10px; | ||
778 | 16 | } | ||
779 | 17 | |||
780 | 18 | .tests-0 td { | ||
781 | 19 | background-color: #eeeeee; | ||
782 | 20 | } | ||
783 | 21 | |||
784 | 22 | .tests-1 td { | ||
785 | 23 | background-color: #ffffff; | ||
786 | 24 | } | ||
787 | 25 | |||
788 | 26 | .steps { | ||
789 | 27 | display: none; | ||
790 | 28 | } | ||
791 | 29 | |||
792 | 30 | .steps th { | ||
793 | 31 | background-color: #ffffff; | ||
794 | 32 | } | ||
795 | 33 | |||
796 | 34 | .steps, .steps td { | ||
797 | 35 | border: none; | ||
798 | 36 | border-bottom: none; | ||
799 | 37 | } | ||
800 | 38 | |||
801 | 39 | .steps .step-0 td { | ||
802 | 40 | background-color: #dddddd; | ||
803 | 41 | } | ||
804 | 42 | |||
805 | 43 | .steps .step-1 td { | ||
806 | 44 | background-color: #eeeeee; | ||
807 | 45 | } | ||
808 | 46 | |||
809 | 47 | .clickable { | ||
810 | 48 | text-decoration: underline; | ||
811 | 49 | color: #4444cc; | ||
812 | 50 | } |
Probably not ready for actual merging yet, but it does function.
Mostly needs cleanup.