qakit

Merge lp:~canonical-platform-qa/qakit/add-vis into lp:qakit

  1. add-vis
  2. Merge into trunk
Proposed by Allan LeSage
Status: Merged
Approved by: Allan LeSage
Approved revision: 117
Merged at revision: 127
Proposed branch: lp:~canonical-platform-qa/qakit/add-vis
Merge into: lp:qakit
Diff against target: 1346 lines (+1273/-0)
14 files modified
qakit/vis/dashboard/main.js (+55/-0)
qakit/vis/dashboard/ust_rc-proposed_arale_regression.html (+48/-0)
qakit/vis/dashboard/ust_rc-proposed_arale_sanity.html (+48/-0)
qakit/vis/dashboard/ust_rc-proposed_krillin_regression.html (+48/-0)
qakit/vis/dashboard/ust_rc-proposed_krillin_sanity.html (+48/-0)
qakit/vis/dashboard/ust_rc_arale_regression.html (+48/-0)
qakit/vis/dashboard/ust_rc_arale_sanity.html (+48/-0)
qakit/vis/dashboard/ust_rc_krillin_regression.html (+48/-0)
qakit/vis/dashboard/ust_rc_krillin_sanity.html (+48/-0)
qakit/vis/jenkins_results.py (+70/-0)
qakit/vis/retrieve_jenkins_results.py (+229/-0)
qakit/vis/tests/test_retrieve_jenkins_results.py (+47/-0)
qakit/vis/tests/test_visualize.py (+194/-0)
qakit/vis/visualize.py (+294/-0)
To merge this branch: bzr merge lp:~canonical-platform-qa/qakit/add-vis
Reviewer Review Type Date Requested Status
Canonical Platform QA Team Pending
Review via email: mp+299606@code.launchpad.net

Commit message

Add a vis tool to visualize test suite performance.

Description of the change

Add a 'vis' tool for visualizing test suite status.

Presently we're using a few html files to present this datatables/bootstrap version, could be nicer with tabs but we'll save that for a later version.

To post a comment you must log in.
Revision history for this message
Sergio Cazzolato (sergio-j-cazzolato) wrote :

I made a high level review, I left some comments inline.

lp:~canonical-platform-qa/qakit/add-vis updated
113. By Allan LeSage

Move dashboard-type files to a dashboard dir.

114. By Allan LeSage

Context manager for json writer.

115. By Allan LeSage

Use statistics to compute mean.

116. By Allan LeSage

Move to dao, fix tests.

117. By Allan LeSage

Typo!

Revision history for this message
Allan LeSage (allanlesage) wrote :

Made all the requested changes, thanks cachio for a further review!

Revision history for this message
Sergio Cazzolato (sergio-j-cazzolato) wrote :

Organized now! go ahead with this change

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== added directory 'qakit/vis'
=== added directory 'qakit/vis/dashboard'
=== added file 'qakit/vis/dashboard/main.js'
--- qakit/vis/dashboard/main.js 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/main.js 2016-07-13 18:36:50 +0000
@@ -0,0 +1,55 @@
1
2function createdCellFtn(cell, cellData, rowData, rowIndex, colIndex) {
3 if (cellData.skipped == true) {
4 // skipped tests get dashed borders
5 $(cell).css('border-style', 'dashed');
6 $(cell).css('border-width', 'medium');
7 } else {
8 // tests get colors
9 $(cell).css('background-color', cellData.color);
10 };
11 if (colIndex > 2) {
12 $(cell).html("<a href=" + cellData.link + ">&nbsp;</a>");
13 } else if (colIndex == 2) {
14 // no links in the trend column pls
15 $(cell).html("");
16 };
17}
18
19function visTable(data, selector) {
20 var tableData = data.data.map(function(obj) {
21 var row = [ obj.section, obj.test_name ];
22 row.push(obj.trend);
23 obj.builds.forEach(function(build) {
24 row.push(build);
25 });
26 return row;
27 });
28 var tableColumns = data.header.map(function(obj) {
29 return {
30 title: obj,
31 createdCell: createdCellFtn,
32 orderable: false,
33 searchable: false,
34 // hide Section column
35 visible: (obj == 'Section' ? false : true),
36 };
37 });
38 $(selector).dataTable({
39 data: tableData,
40 columns: tableColumns,
41 searching: false,
42 order: [[0, 'dsc']],
43 bDestroy: true,
44 lengthChange: false,
45 lengthMenu: [-1],
46 });
47}
48
49function draw(filename) {
50 $.getJSON(filename, function(data) {
51 visTable(data, "#visTable");
52 }).fail(function(data, status) {
53 console.log("Faled to load vis JSON.");
54 });
55}
056
=== added file 'qakit/vis/dashboard/ust_rc-proposed_arale_regression.html'
--- qakit/vis/dashboard/ust_rc-proposed_arale_regression.html 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/ust_rc-proposed_arale_regression.html 2016-07-13 18:36:50 +0000
@@ -0,0 +1,48 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7
8 <title>Platform QA - VIS</title>
9
10 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css">
11 <link href="http://fonts.googleapis.com/css?family=Ubuntu" rel="stylesheet" type="text/css">
12 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
13 <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.css"/>
14 <script src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
15 <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js"></script>
16 <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
17 <script type="text/javascript" src="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.js"></script>
18 <script src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
19 <script src="main.js"></script>
20 <style>
21 .ar {
22 text-align: right;
23 }
24 .vertical{
25 writing-mode:tb-rl;
26 white-space:nowrap;
27 display:block;
28 bottom:0;
29 height:20px;
30 }
31 td a {
32 display:block;
33 width:100%;
34 text-decoration:none;
35 }
36 </style>
37</head>
38<body>
39 <div class="col-md-12">
40 <table id="visTable" class="table table-striped table-bordered"></table>
41 </div>
42 <script>
43 window.onload = function() {
44 draw("ust_rc-proposed_arale_regression.json");
45 }
46 </script>
47</body>
48</html>
049
=== added file 'qakit/vis/dashboard/ust_rc-proposed_arale_sanity.html'
--- qakit/vis/dashboard/ust_rc-proposed_arale_sanity.html 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/ust_rc-proposed_arale_sanity.html 2016-07-13 18:36:50 +0000
@@ -0,0 +1,48 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7
8 <title>Platform QA - VIS</title>
9
10 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css">
11 <link href="http://fonts.googleapis.com/css?family=Ubuntu" rel="stylesheet" type="text/css">
12 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
13 <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.css"/>
14 <script src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
15 <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js"></script>
16 <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
17 <script type="text/javascript" src="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.js"></script>
18 <script src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
19 <script src="main.js"></script>
20 <style>
21 .ar {
22 text-align: right;
23 }
24 .vertical{
25 writing-mode:tb-rl;
26 white-space:nowrap;
27 display:block;
28 bottom:0;
29 height:20px;
30 }
31 td a {
32 display:block;
33 width:100%;
34 text-decoration:none;
35 }
36 </style>
37</head>
38<body>
39 <div class="col-md-12">
40 <table id="visTable" class="table table-striped table-bordered"></table>
41 </div>
42 <script>
43 window.onload = function() {
44 draw("ust_rc-proposed_arale_sanity.json");
45 }
46 </script>
47</body>
48</html>
049
=== added file 'qakit/vis/dashboard/ust_rc-proposed_krillin_regression.html'
--- qakit/vis/dashboard/ust_rc-proposed_krillin_regression.html 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/ust_rc-proposed_krillin_regression.html 2016-07-13 18:36:50 +0000
@@ -0,0 +1,48 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7
8 <title>Platform QA - VIS</title>
9
10 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css">
11 <link href="http://fonts.googleapis.com/css?family=Ubuntu" rel="stylesheet" type="text/css">
12 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
13 <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.css"/>
14 <script src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
15 <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js"></script>
16 <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
17 <script type="text/javascript" src="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.js"></script>
18 <script src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
19 <script src="main.js"></script>
20 <style>
21 .ar {
22 text-align: right;
23 }
24 .vertical{
25 writing-mode:tb-rl;
26 white-space:nowrap;
27 display:block;
28 bottom:0;
29 height:20px;
30 }
31 td a {
32 display:block;
33 width:100%;
34 text-decoration:none;
35 }
36 </style>
37</head>
38<body>
39 <div class="col-md-12">
40 <table id="visTable" class="table table-striped table-bordered"></table>
41 </div>
42 <script>
43 window.onload = function() {
44 draw("ust_rc-proposed_krillin_regression.json");
45 }
46 </script>
47</body>
48</html>
049
=== added file 'qakit/vis/dashboard/ust_rc-proposed_krillin_sanity.html'
--- qakit/vis/dashboard/ust_rc-proposed_krillin_sanity.html 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/ust_rc-proposed_krillin_sanity.html 2016-07-13 18:36:50 +0000
@@ -0,0 +1,48 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7
8 <title>Platform QA - VIS</title>
9
10 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css">
11 <link href="http://fonts.googleapis.com/css?family=Ubuntu" rel="stylesheet" type="text/css">
12 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
13 <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.css"/>
14 <script src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
15 <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js"></script>
16 <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
17 <script type="text/javascript" src="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.js"></script>
18 <script src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
19 <script src="main.js"></script>
20 <style>
21 .ar {
22 text-align: right;
23 }
24 .vertical{
25 writing-mode:tb-rl;
26 white-space:nowrap;
27 display:block;
28 bottom:0;
29 height:20px;
30 }
31 td a {
32 display:block;
33 width:100%;
34 text-decoration:none;
35 }
36 </style>
37</head>
38<body>
39 <div class="col-md-12">
40 <table id="visTable" class="table table-striped table-bordered"></table>
41 </div>
42 <script>
43 window.onload = function() {
44 draw("ust_rc-proposed_krillin_sanity.json");
45 }
46 </script>
47</body>
48</html>
049
=== added file 'qakit/vis/dashboard/ust_rc_arale_regression.html'
--- qakit/vis/dashboard/ust_rc_arale_regression.html 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/ust_rc_arale_regression.html 2016-07-13 18:36:50 +0000
@@ -0,0 +1,48 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7
8 <title>Platform QA - VIS</title>
9
10 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css">
11 <link href="http://fonts.googleapis.com/css?family=Ubuntu" rel="stylesheet" type="text/css">
12 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
13 <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.css"/>
14 <script src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
15 <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js"></script>
16 <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
17 <script type="text/javascript" src="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.js"></script>
18 <script src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
19 <script src="main.js"></script>
20 <style>
21 .ar {
22 text-align: right;
23 }
24 .vertical{
25 writing-mode:tb-rl;
26 white-space:nowrap;
27 display:block;
28 bottom:0;
29 height:20px;
30 }
31 td a {
32 display:block;
33 width:100%;
34 text-decoration:none;
35 }
36 </style>
37</head>
38<body>
39 <div class="col-md-12">
40 <table id="visTable" class="table table-striped table-bordered"></table>
41 </div>
42 <script>
43 window.onload = function() {
44 draw("ust_rc_arale_regression.json");
45 }
46 </script>
47</body>
48</html>
049
=== added file 'qakit/vis/dashboard/ust_rc_arale_sanity.html'
--- qakit/vis/dashboard/ust_rc_arale_sanity.html 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/ust_rc_arale_sanity.html 2016-07-13 18:36:50 +0000
@@ -0,0 +1,48 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7
8 <title>Platform QA - VIS</title>
9
10 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css">
11 <link href="http://fonts.googleapis.com/css?family=Ubuntu" rel="stylesheet" type="text/css">
12 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
13 <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.css"/>
14 <script src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
15 <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js"></script>
16 <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
17 <script type="text/javascript" src="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.js"></script>
18 <script src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
19 <script src="main.js"></script>
20 <style>
21 .ar {
22 text-align: right;
23 }
24 .vertical{
25 writing-mode:tb-rl;
26 white-space:nowrap;
27 display:block;
28 bottom:0;
29 height:20px;
30 }
31 td a {
32 display:block;
33 width:100%;
34 text-decoration:none;
35 }
36 </style>
37</head>
38<body>
39 <div class="col-md-12">
40 <table id="visTable" class="table table-striped table-bordered"></table>
41 </div>
42 <script>
43 window.onload = function() {
44 draw("ust_rc_arale_sanity.json");
45 }
46 </script>
47</body>
48</html>
049
=== added file 'qakit/vis/dashboard/ust_rc_krillin_regression.html'
--- qakit/vis/dashboard/ust_rc_krillin_regression.html 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/ust_rc_krillin_regression.html 2016-07-13 18:36:50 +0000
@@ -0,0 +1,48 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7
8 <title>Platform QA - VIS</title>
9
10 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css">
11 <link href="http://fonts.googleapis.com/css?family=Ubuntu" rel="stylesheet" type="text/css">
12 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
13 <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.css"/>
14 <script src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
15 <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js"></script>
16 <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
17 <script type="text/javascript" src="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.js"></script>
18 <script src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
19 <script src="main.js"></script>
20 <style>
21 .ar {
22 text-align: right;
23 }
24 .vertical{
25 writing-mode:tb-rl;
26 white-space:nowrap;
27 display:block;
28 bottom:0;
29 height:20px;
30 }
31 td a {
32 display:block;
33 width:100%;
34 text-decoration:none;
35 }
36 </style>
37</head>
38<body>
39 <div class="col-md-12">
40 <table id="visTable" class="table table-striped table-bordered"></table>
41 </div>
42 <script>
43 window.onload = function() {
44 draw("ust_rc_krillin_regression.json");
45 }
46 </script>
47</body>
48</html>
049
=== added file 'qakit/vis/dashboard/ust_rc_krillin_sanity.html'
--- qakit/vis/dashboard/ust_rc_krillin_sanity.html 1970-01-01 00:00:00 +0000
+++ qakit/vis/dashboard/ust_rc_krillin_sanity.html 2016-07-13 18:36:50 +0000
@@ -0,0 +1,48 @@
1<!DOCTYPE html>
2<html>
3<head>
4 <meta charset="utf-8">
5 <meta http-equiv="X-UA-Compatible" content="IE=edge">
6 <meta name="viewport" content="width=device-width, initial-scale=1">
7
8 <title>Platform QA - VIS</title>
9
10 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css">
11 <link href="http://fonts.googleapis.com/css?family=Ubuntu" rel="stylesheet" type="text/css">
12 <link rel="stylesheet" href="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css">
13 <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.css"/>
14 <script src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
15 <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js"></script>
16 <script src="http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
17 <script type="text/javascript" src="https://cdn.datatables.net/s/bs/dt-1.10.10,b-1.1.0,fh-3.1.0,r-2.0.0/datatables.min.js"></script>
18 <script src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
19 <script src="main.js"></script>
20 <style>
21 .ar {
22 text-align: right;
23 }
24 .vertical{
25 writing-mode:tb-rl;
26 white-space:nowrap;
27 display:block;
28 bottom:0;
29 height:20px;
30 }
31 td a {
32 display:block;
33 width:100%;
34 text-decoration:none;
35 }
36 </style>
37</head>
38<body>
39 <div class="col-md-12">
40 <table id="visTable" class="table table-striped table-bordered"></table>
41 </div>
42 <script>
43 window.onload = function() {
44 draw("ust_rc_krillin_sanity.json");
45 }
46 </script>
47</body>
48</html>
049
=== added file 'qakit/vis/jenkins_results.py'
--- qakit/vis/jenkins_results.py 1970-01-01 00:00:00 +0000
+++ qakit/vis/jenkins_results.py 2016-07-13 18:36:50 +0000
@@ -0,0 +1,70 @@
1#!/usr/bin/env python3
2# UEQA Vis
3# Copyright (C) 2016 Canonical
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License as published by
7# the Free Software Foundation, either version 3 of the License, or
8# (at your option) any later version.
9#
10# This program is distributed in the hope that it will be useful,
11# but WITHOUT ANY WARRANTY; without even the implied warranty of
12# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13# GNU General Public License for more details.
14#
15# You should have received a copy of the GNU General Public License
16# along with this program. If not, see <http://www.gnu.org/licenses/>.
17
18
19import pymongo
20
21
22class JenkinsResults:
23
24 def __init__(self, db):
25 self.db = db
26
27 def get_build_test_results(self, job_name, build_number):
28 """Query db for test results.
29
30 NOTE that we don't return skipped tests.
31
32 """
33 return self.db.tests.find(
34 {'job_name': job_name,
35 'build_number': build_number,
36 'status': {'$ne': 'SKIPPED'}}
37 )
38
39 def get_test_result(self, job_name, build_number, test_name):
40 """Query db for a single test result."""
41 return self.db.tests.find_one(
42 {'test_name': test_name,
43 'job_name': job_name,
44 'build_number': build_number})
45
46 def get_trailing_test_results(self, job_name, test_name, limit=10):
47 """Return last 'limit' test_name results."""
48 return self.db.tests.find(
49 {'test_name': test_name,
50 'job_name': job_name,
51 'status': {'$ne': 'SKIPPED'}}).sort(
52 'build_number', pymongo.DESCENDING).limit(limit)
53
54 def get_test_names(self, job_name):
55 """Get all test names associated with a Jenkins job from db."""
56 return self.db.tests.find(
57 {'job_name': job_name}).distinct('test_name')
58
59 def get_build_numbers(self, job_name):
60 """Get all build numbers associated with a Jenkins job from db."""
61 return sorted(
62 self.db.builds.find({'job_name': job_name}).distinct('number'),
63 reverse=True)
64
65 def get_build(self, job_name, build_number):
66 """Return a specific build."""
67 return self.db.builds.find_one(
68 {'job_name': job_name,
69 'number': build_number}
70 )
071
=== added file 'qakit/vis/retrieve_jenkins_results.py'
--- qakit/vis/retrieve_jenkins_results.py 1970-01-01 00:00:00 +0000
+++ qakit/vis/retrieve_jenkins_results.py 2016-07-13 18:36:50 +0000
@@ -0,0 +1,229 @@
1#!/usr/bin/python3
2
3# QA KPI Utilities
4# Copyright (C) 2015-2016 Canonical
5#
6# This program is free software: you can redistribute it and/or modify
7# it under the terms of the GNU General Public License as published by
8# the Free Software Foundation, either version 3 of the License, or
9# (at your option) any later version.
10#
11# This program is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU General Public License
17# along with this program. If not, see <http://www.gnu.org/licenses/>.
18
19import argparse
20import logging
21import sys
22
23import bson
24import jenkinsapi
25import pymongo
26
27logger = logging.getLogger(__name__)
28handler = logging.StreamHandler()
29handler.setLevel(logging.DEBUG)
30logger.addHandler(handler)
31logger.setLevel(logging.DEBUG)
32
33
34def filter_job_names(job_names, pattern):
35 return list(filter(lambda x: pattern in x, job_names))
36
37
38class JenkinsMongoSession():
39
40 def __init__(self, db, baseurl, username=None, password=None):
41 super().__init__()
42 self.db = db
43 self.jenkins = jenkinsapi.jenkins.Jenkins(
44 baseurl, username, password)
45
46 def get_job(self, job_name):
47 return self.jenkins[job_name]
48
49 def retrieve_build_resultset(self, db_build_id):
50 """Retrieve the resultset of a known build in our db."""
51 db_build = self.db.builds.find_one(
52 {'_id': bson.ObjectId(db_build_id)})
53 job_name = db_build['job_name']
54 jenkins_job = self.get_job(job_name)
55 build_number = db_build['number']
56 jenkins_build = jenkins_job.get_build(build_number)
57 try:
58 resultset = jenkins_build.get_resultset()
59 except jenkinsapi.custom_exceptions.NoResults:
60 return {}
61 test_results = []
62 for test_name in resultset.keys():
63 result = resultset[test_name]
64 test_result = {
65 'test_name': test_name,
66 'status': result.status,
67 'build_number': build_number,
68 'build_timestamp': db_build['timestamp'],
69 'build_name': db_build['description'],
70 'job_name': job_name,
71 }
72 test_results.append(test_result)
73 return test_results
74
75 def retrieve_and_insert_build_resultset(self, db_build_id):
76 """Retrieve test results from Jenkins, inserting into our db."""
77 build_resultset = self.retrieve_build_resultset(db_build_id)
78 return self.db.tests.insert(build_resultset)
79
80 def retrieve_build(self, job_name, build_number):
81 """Retrieve build from Jenkins."""
82 logger.info(
83 'Retrieving {} build #{}.'.format(job_name, build_number)
84 )
85 job = self.get_job(job_name)
86 build = job.get_build(build_number)
87 data = build.get_data(build.python_api_url(build.baseurl))
88 actions = build.get_actions()
89 total = failed = skipped = 0
90 try:
91 total = actions['totalCount']
92 failed = actions['failCount']
93 skipped = actions['skipCount']
94 except KeyError:
95 # we don't have a real result here but we'll still save
96 pass
97 return dict({
98 'job_name': job_name,
99 'total': total,
100 'failed': failed,
101 'skipped': skipped},
102 **data)
103
104 def retrieve_and_insert_build(self, job_name, build_number):
105 """Retrieve build from Jenkins and deposit in db."""
106 build_results = self.retrieve_build(job_name, build_number)
107 return self.db.builds.insert(build_results)
108
109 def get_db_build_ids_for_job(self, job_name):
110 """Get a list of known build ids for the given job for comparison."""
111 return [
112 build['number'] for build in self.db.builds.find(
113 {'job_name': job_name},
114 {'number': 1, '_id': 0})
115 ]
116
117 def get_missing_builds(self, job_name):
118 """Return a list of build ids in Jenkins we're missing in db."""
119 job = self.get_job(job_name)
120 jenkins_build_ids = set(job.get_build_ids())
121 db_build_ids = set(self.get_db_build_ids_for_job(job_name))
122 missing_builds = [
123 build_id for build_id in
124 # take the set-difference
125 list(jenkins_build_ids.difference(db_build_ids))
126 # filter out running builds
127 if not job.get_build(build_id).is_running()]
128 logger.info(
129 'For job {}, missing builds {}.'.format(job_name, missing_builds))
130 return missing_builds
131
132 def retrieve_data_for_job(self, job_name):
133 """Retrieve builds and results from Jenkins, inserting into our db."""
134 logger.debug('Retrieving data for job {}.'.format(job_name))
135 build_numbers = self.get_missing_builds(job_name)
136 for build_number in reversed(build_numbers):
137 db_build_id = self.retrieve_and_insert_build(
138 job_name, build_number)
139 self.retrieve_and_insert_build_resultset(db_build_id)
140
141 def retrieve_jenkins_results(self, job_names=None, pattern=None):
142 """Retrieve builds and results either by name or matching a pattern."""
143 if pattern is not None:
144 all_job_names = self.jenkins.get_jobs_list()
145 job_names = filter_job_names(all_job_names, pattern)
146 for job_name in job_names:
147 self.retrieve_data_for_job(job_name)
148
149
150def _parse_arguments():
151 parser = argparse.ArgumentParser(
152 "Retrieve Jenkins build results, storing in a mongo db.")
153 parser.add_argument(
154 '-v',
155 '--verbose',
156 help='Verbose logging.',
157 default=False,
158 action='store_true',
159 )
160 parser.add_argument(
161 '--vis-db',
162 help='MongoDB collection in which to store data.',
163 type=str,
164 default='vis',
165 required=False,
166 )
167 parser.add_argument(
168 '--jenkins-url',
169 help='Jenkins URL from which to collect data.',
170 type=str,
171 required=True,
172 )
173 parser.add_argument(
174 '--jenkins-user',
175 help='Jenkins username.',
176 type=str,
177 required=False,
178 )
179 parser.add_argument(
180 '--jenkins-password',
181 help='Jenkins API token for Jenkins user.',
182 type=str,
183 required=False,
184 )
185 parser.add_argument(
186 '--mongo-host',
187 help='MongoDB host IP.',
188 type=str,
189 default='127.0.0.1',
190 required=False,
191 )
192 parser.add_argument(
193 nargs='*',
194 help='Job names from which to collect data.',
195 type=str,
196 dest='job_names',
197 )
198 parser.add_argument(
199 '-p',
200 '--pattern',
201 help='Job name search pattern.',
202 type=str,
203 )
204 args = parser.parse_args()
205 if args.pattern and args.job_names:
206 print("Please specify either job names or pattern.")
207 parser.print_help()
208 sys.exit(1)
209 return args
210
211
212def main():
213 args = _parse_arguments()
214 if args.verbose:
215 logger.setLevel(logging.DEBUG)
216 conn = pymongo.MongoClient(host=args.mongo_host)
217 db = getattr(conn, args.vis_db)
218 session = JenkinsMongoSession(
219 db,
220 args.jenkins_url,
221 args.jenkins_user,
222 args.jenkins_password,
223 )
224 session.retrieve_jenkins_results(
225 args.job_names, args.pattern)
226
227
228if __name__ == '__main__':
229 sys.exit(main())
0230
=== added directory 'qakit/vis/tests'
=== added file 'qakit/vis/tests/__init__.py'
=== added file 'qakit/vis/tests/test_retrieve_jenkins_results.py'
--- qakit/vis/tests/test_retrieve_jenkins_results.py 1970-01-01 00:00:00 +0000
+++ qakit/vis/tests/test_retrieve_jenkins_results.py 2016-07-13 18:36:50 +0000
@@ -0,0 +1,47 @@
1#!/usr/bin/env python3
2# UEQA VIS
3# Copyright (C) 2016 Canonical
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License as published by
7# the Free Software Foundation, either version 3 of the License, or
8# (at your option) any later version.
9#
10# This program is distributed in the hope that it will be useful,
11# but WITHOUT ANY WARRANTY; without even the implied warranty of
12# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13# GNU General Public License for more details.
14#
15# You should have received a copy of the GNU General Public License
16# along with this program. If not, see <http://www.gnu.org/licenses/>.
17
18import unittest
19import unittest.mock as mock
20
21import qakit.vis.retrieve_jenkins_results as retrieve
22
23
24JOB_NAMES = [
25 'ust_rc-proposed_krillin_sanity',
26 'ust_rc-proposed_krillin_regression',
27 'ust_rc-proposed_arale_sanity',
28 'foo',
29]
30
31
32class RegexTestCase(unittest.TestCase):
33
34 def test_vanilla(self):
35 result = retrieve.filter_job_names(JOB_NAMES, 'ust_')
36 self.assertEqual(
37 ['ust_rc-proposed_krillin_sanity',
38 'ust_rc-proposed_krillin_regression',
39 'ust_rc-proposed_arale_sanity'],
40 result)
41
42 def test_just_krillin(self):
43 result = retrieve.filter_job_names(JOB_NAMES, 'krillin')
44 self.assertEqual(
45 ['ust_rc-proposed_krillin_sanity',
46 'ust_rc-proposed_krillin_regression'],
47 result)
048
=== added file 'qakit/vis/tests/test_visualize.py'
--- qakit/vis/tests/test_visualize.py 1970-01-01 00:00:00 +0000
+++ qakit/vis/tests/test_visualize.py 2016-07-13 18:36:50 +0000
@@ -0,0 +1,194 @@
1#!/usr/bin/env python3
2# UEQA VIS
3# Copyright (C) 2014-2016 Canonical
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License as published by
7# the Free Software Foundation, either version 3 of the License, or
8# (at your option) any later version.
9#
10# This program is distributed in the hope that it will be useful,
11# but WITHOUT ANY WARRANTY; without even the implied warranty of
12# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13# GNU General Public License for more details.
14#
15# You should have received a copy of the GNU General Public License
16# along with this program. If not, see <http://www.gnu.org/licenses/>.
17
18import unittest
19import unittest.mock as mock
20
21import qakit.vis.visualize as visualize
22
23
24class StatusToFloatTestCase(unittest.TestCase):
25
26 def test_passed(self):
27 result = visualize.status_to_float('PASSED')
28 self.assertEqual(1.0, result)
29
30 def test_failed(self):
31 result = visualize.status_to_float('FAILED')
32 self.assertEqual(0.0, result)
33
34 def test_fixed(self):
35 result = visualize.status_to_float('FIXED')
36 self.assertEqual(1.0, result)
37
38 def test_regression(self):
39 result = visualize.status_to_float('REGRESSION')
40 self.assertEqual(0.0, result)
41
42 def test_gibberish(self):
43 result = visualize.status_to_float('GIBBERISH')
44 self.assertEqual(0.0, result)
45
46
47class ComputeColorTestCase(unittest.TestCase):
48
49 def test_compute_color_pass(self):
50 self.assertEqual('rgb(0, 255, 0)',
51 visualize.compute_color(1.0))
52
53 def test_compute_color_fail(self):
54 self.assertEqual('rgb(255, 0, 0)',
55 visualize.compute_color(0.0))
56
57 def test_compute_color_none(self):
58 self.assertEqual('rgb(255, 255, 255)',
59 visualize.compute_color(None))
60
61
62class DbVisTestCase(unittest.TestCase):
63
64 def setUp(self):
65 super().setUp()
66 self.db = mock.Mock(
67 vis=mock.Mock())
68
69
70@unittest.skip('Temporarily deprecated.')
71class GetPassRateTestCase(unittest.TestCase):
72
73 def test_pass_rate(self):
74 pass_rate = 0.9
75 pass_rate_dict = {'pass_rate': pass_rate}
76 db = mock.Mock(
77 dashboard_tests_pass_rates=mock.Mock(
78 find_one=mock.Mock(
79 return_value=pass_rate_dict)))
80 self.assertEqual(
81 pass_rate,
82 visualize.get_pass_rate(
83 'fake_suite_name',
84 'fake_build_name',
85 db))
86
87 def test_pass_rate_no_results(self):
88 db = mock.Mock(
89 dashboard_tests_pass_rates=mock.Mock(
90 find_one=mock.Mock(
91 side_effect=TypeError)))
92 self.assertEqual(
93 None,
94 visualize.get_pass_rate(
95 'fake_suite_name',
96 'fake_build_name',
97 db))
98
99
100class GetUrlPathFromTestNameTestCase(unittest.TestCase):
101
102 def test_vanilla(self):
103 result = visualize.get_url_path_from_test_name(
104 'ubuntu_system_tests.tests.test_calls.CallsTestCase.'
105 'test_receive_incoming_call_from_contact')
106 self.assertEqual(
107 'ubuntu_system_tests.tests.test_calls/CallsTestCase/'
108 'test_receive_incoming_call_from_contact',
109 result)
110
111 def test_more_nesting(self):
112 result = visualize.get_url_path_from_test_name(
113 'ubuntu_system_tests.tests.webapps.test_here.HereTestCase.'
114 'test_retrieve_location')
115 self.assertEqual(
116 'ubuntu_system_tests.tests.webapps.test_here/HereTestCase/'
117 'test_retrieve_location',
118 result)
119
120 def test_odd_test_name_directs_to_build_test_result(self):
121 result = visualize.get_url_path_from_test_name(
122 'subunit.parser')
123 self.assertEqual('', result)
124
125
126class ComputePassRateFromListOfResultsTestCase(unittest.TestCase):
127
128 def test_vanilla(self):
129 result = visualize.compute_pass_rate_from_statuses(
130 ['PASSED', 'PASSED'])
131 self.assertEqual(1.0, result)
132
133 def test_half_passed(self):
134 result = visualize.compute_pass_rate_from_statuses(
135 ['PASSED', 'FAILED'])
136 self.assertEqual(0.5, result)
137
138
139class SplitTestNameTestCase(unittest.TestCase):
140
141 def test_vanilla(self):
142 result = visualize.split_test_name(
143 'foo.bar.FooTestCase.test_something')
144 self.assertEqual(
145 ('foo.bar', 'FooTestCase.test_something'),
146 result,
147 )
148
149 def test_deeper_nesting(self):
150 result = visualize.split_test_name(
151 'foo.bar.baz.FooTestCase.test_something')
152 self.assertEqual(
153 ('foo.bar.baz', 'FooTestCase.test_something'),
154 result,
155 )
156
157 def test_gibberish(self):
158 result = visualize.split_test_name(
159 'gibberish')
160 self.assertEqual(
161 ('gibberish', 'gibberish'),
162 result,
163 )
164
165 def test_gibberish_with_dots(self):
166 result = visualize.split_test_name(
167 'gibberish.test_something')
168 self.assertEqual(
169 ('gibberish.test_something', 'gibberish.test_something'),
170 result,
171 )
172
173
174class GetColorFromStatus(unittest.TestCase):
175
176 def test_skipped(self):
177 result = visualize.get_color_from_status('SKIPPED')
178 self.assertEqual('white', result)
179
180 def test_passed(self):
181 result = visualize.get_color_from_status('PASSED')
182 self.assertEqual('lime', result)
183
184 def test_failed(self):
185 result = visualize.get_color_from_status('FAILED')
186 self.assertEqual('red', result)
187
188 def test_regression(self):
189 result = visualize.get_color_from_status('REGRESSION')
190 self.assertEqual('red', result)
191
192 def test_fixed(self):
193 result = visualize.get_color_from_status('FIXED')
194 self.assertEqual('lime', result)
0195
=== added file 'qakit/vis/visualize.py'
--- qakit/vis/visualize.py 1970-01-01 00:00:00 +0000
+++ qakit/vis/visualize.py 2016-07-13 18:36:50 +0000
@@ -0,0 +1,294 @@
1#!/usr/bin/env python3
2# UEQA Vis
3# Copyright (C) 2014-2016 Canonical
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License as published by
7# the Free Software Foundation, either version 3 of the License, or
8# (at your option) any later version.
9#
10# This program is distributed in the hope that it will be useful,
11# but WITHOUT ANY WARRANTY; without even the implied warranty of
12# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13# GNU General Public License for more details.
14#
15# You should have received a copy of the GNU General Public License
16# along with this program. If not, see <http://www.gnu.org/licenses/>.
17
18import argparse
19import json
20import math
21import statistics
22import sys
23
24import pymongo
25
26import jenkins_results
27
28
29def get_color_from_status(status):
30 """Return the color for the given cell.
31
32 PASSED (or FIXED) = green,
33 FAILED (or REGRESSED) = red,
34 SKIPPED = white
35
36 """
37 red = 'red'
38 white = 'white'
39 # lime happens to help with trend color-calculations
40 green = 'lime'
41 return {
42 'SKIPPED': white,
43 'PASSED': green,
44 'FIXED': green,
45 'REGRESSION': red,
46 'FAILED': red,
47 }[status]
48
49
50def compute_color(pass_rate):
51 """Return a red-green color representing the given pass rate.
52
53 Use a sqrt curve to highlight the difference between 0.0 and
54 0.01: invert the value, apply sqrt, then invert the value
55 again.
56
57 """
58 if pass_rate is None:
59 # white indicates "no result"
60 return 'rgb(255, 255, 255)'
61 p = abs(
62 1.0 - math.sqrt(
63 abs(1.0 - float(pass_rate))
64 )
65 ) * 255.0
66 return 'rgb({}, {}, 0)'.format(
67 int(255 - p),
68 int(p),
69 )
70
71
72def status_to_float(status):
73 """Convert JUnit test status to float for averaging."""
74 if status == 'PASSED' or status == 'FIXED':
75 return 1.0
76 else:
77 return 0.0
78
79
80def compute_pass_rate_from_statuses(results):
81 """Compute pass rate from a list of JUnit test statuses."""
82 float_results = [status_to_float(result) for result in results]
83 try:
84 return statistics.mean(float_results)
85 except statistics.StatisticsError:
86 return None
87
88
89def compute_color_from_statuses(statuses):
90 """Compute a red-green color from a list of JUnit test statuses."""
91 return compute_color(compute_pass_rate_from_statuses(statuses))
92
93
94def get_build_pass_rate(dao, job_name, build_number):
95 """Calculate pass rate for a specified build in the given db."""
96 results = dao.get_build_test_results(job_name, build_number)
97 try:
98 statuses = [result['status'] for result in results]
99 return compute_pass_rate_from_statuses(statuses)
100 except TypeError:
101 # no result in db
102 return None
103
104
105def get_url_path_from_test_name(test_name):
106 """Return url path of Jenkins report for test.
107
108 Jenkins displays the result (incl. e.g. traceback) for a given
109 test at specific URL path under the build URL.
110
111 """
112 split_test_name = test_name.split('.')
113 if len(split_test_name) < 5:
114 return ''
115 try:
116 url_path = '/'.join(
117 ['.'.join(
118 split_test_name[:-2]),
119 split_test_name[-2],
120 split_test_name[-1]])
121 except IndexError:
122 url_path = ''
123 return url_path
124
125
126def get_test_link_for_build(build_url, test_name):
127 """Return a link to the test under the given Jenkins build."""
128 link = get_url_path_from_test_name(test_name)
129 return build_url + 'testReport/' + link
130
131
132def get_cell_data(dao, job_name, build_number, test_name):
133 """Return a dict representing the given test for rendering."""
134 cell = {'test_name': test_name,
135 'job_name': job_name,
136 'build_number': build_number}
137 test_result = dao.get_test_result(**cell)
138 build = dao.get_build(job_name, build_number)
139 if not test_result:
140 return dict({'status': None, 'color': None, 'link': None}, **cell)
141 cell['status'] = test_result['status']
142 try:
143 cell['color'] = get_color_from_status(test_result['status'])
144 except TypeError:
145 # no run for test, default to white
146 cell['color'] = 'rgb{255, 255, 255}'
147 cell['skipped'] = test_result['status'] == 'SKIPPED'
148 cell['link'] = get_test_link_for_build(build['url'], test_name)
149 return cell
150
151
152def get_trend_cell(dao, job_name, test_name, limit=10):
153 """Return a dict representing the pass rate trend."""
154 cell = {'test_name': test_name,
155 'job_name': job_name}
156 test_results = dao.get_trailing_test_results(**cell)
157 cell['color'] = compute_color_from_statuses(
158 [test_result['status'] for test_result in test_results]
159 )
160 return cell
161
162
163def split_test_name(test_name):
164 """Return a tuple of (section, test_name) for given test name.
165
166 Typically tests are named as
167
168 foo.bar.baz.TestCaseClass.test_something
169
170 so return a "section" as 'foo.bar.baz', test_name as
171 TestCaseClass.test_something.
172
173 Non-conforming test names (e.g. 'gibberish') are granted their own
174 section names: ('gibberish', 'gibberish').
175
176 """
177 split_test_name = test_name.split('.')
178 if len(split_test_name) <= 2:
179 return (test_name, test_name)
180 test_name = '.'.join(split_test_name[-2:])
181 section_name = '.'.join(split_test_name[:-2])
182 return (section_name, test_name)
183
184
185def get_test_row(dao, job_name, test_name, build_numbers):
186 """Return a dict of test results for rendering."""
187 test_name_split = split_test_name(test_name)
188 row = {
189 'section': test_name_split[0],
190 'test_name': test_name_split[1],
191 }
192 row['trend'] = get_trend_cell(dao, job_name, test_name)
193 builds = []
194 for build_number in build_numbers:
195 builds.append(get_cell_data(
196 dao, job_name, build_number, test_name))
197 row['builds'] = builds
198 return row
199
200
201def get_report(
202 dao,
203 job_name):
204 """Generate a JSON representation from db data."""
205 build_numbers = dao.get_build_numbers(job_name)
206 pass_rates = []
207 for build_number in build_numbers:
208 pass_rates.append(get_build_pass_rate(dao, job_name, build_number))
209
210 report = {}
211 report['header'] = ['Section', 'Test Name', 'Trend']
212
213 # TODO: this may belong in the DataTables js, easier to compose here
214 def get_pass_rate_build_number_header(pass_rate, build_number):
215 try:
216 # express pass_rate as a percentage
217 return "{} ({:2.1f}%)".format(build_number, pass_rate*100)
218 except TypeError:
219 # when we encounter a None:
220 return "{}".format(build_number)
221
222 report['header'].extend([
223 get_pass_rate_build_number_header(pass_rate, build_number)
224 for build_number, pass_rate in
225 zip(build_numbers, pass_rates)
226 ])
227
228 test_names = dao.get_test_names(job_name)
229 data = []
230 for test_name in test_names:
231 data.append(
232 get_test_row(
233 dao,
234 job_name,
235 test_name,
236 build_numbers,
237 )
238 )
239 report['data'] = data
240 return report
241
242
243def write_json_report(
244 dao,
245 job_name,
246 output_filepath=None):
247 """Write a JSON report of test performance to the given filepath."""
248 json_report = get_report(dao, job_name)
249 with open(output_filepath, 'w') as f:
250 f.write(json.dumps(json_report, indent=4))
251
252
253def _parse_arguments():
254 parser = argparse.ArgumentParser(
255 "Write a JSON report of test performance.")
256 parser.add_argument(
257 help='Jenkins job to visualize.',
258 type=str,
259 dest='jenkins_job')
260 parser.add_argument(
261 '-o',
262 help="Output filepath for JSON report.",
263 type=str,
264 dest='output_filepath',
265 required=True)
266 parser.add_argument(
267 '--mongo-host',
268 help='MongoDB host IP.',
269 type=str,
270 default='127.0.0.1',
271 required=False)
272 parser.add_argument(
273 '--vis-db',
274 help='MongoDB database from which to generate report.',
275 type=str,
276 default='vis',
277 required=False)
278 return parser.parse_args()
279
280
281def main():
282 args = _parse_arguments()
283 conn = pymongo.MongoClient(host=args.mongo_host)
284 db = getattr(conn, args.vis_db)
285 dao = jenkins_results.JenkinsResults(db)
286 return write_json_report(
287 dao,
288 args.jenkins_job,
289 args.output_filepath,
290 )
291
292
293if __name__ == '__main__':
294 sys.exit(main())

Subscribers

People subscribed via source and target branches

to all changes: