Merge lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread into lp:ubuntu-system-tests

[Brendan Donegan]

A simple script for parsing a subunit file and uploading the results to a Google spreadsheet

A simple script for parsing a subunit file and uploading the results to a Google spreadsheet

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:112
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/224/
Executed test runs:
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-amd64-ci/225
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-armhf-ci/224
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-i386-ci/224

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/224/rebuild

review: Approve (continuous-integration)

[Federico Gimenez (fgimenez)]

Looks very good in general, some comments inline.

Cheers!

[lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread]

113. By Brendan Donegan on 2015-05-12

Complete refactoring

114. By Brendan Donegan on 2015-05-12

Fix flake8 errors

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:114
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/225/
Executed test runs:
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-amd64-ci/226
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-armhf-ci/225
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-i386-ci/225

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/225/rebuild

review: Approve (continuous-integration)

[Leo Arias (elopio)]

Is there a way to test this?
We could put mocks in the gspread namespace, use a fake server, or maybe there's a staging google docs.
I wouldn't like to merge it before at least discussion testing.

Also, I think the readme should mention what this script is for, and the requirements to run it. Or maybe it could be in the help for the command.

I find it useful when the script ends in .py.

[lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread]

115. By Brendan Donegan on 2015-05-14

Move the script to under ubuntu_sanity_tests/scripts and add a section to the README

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:115
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/226/
Executed test runs:
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-amd64-ci/227
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-armhf-ci/226
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-i386-ci/226

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/226/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread]

116. By Brendan Donegan on 2015-05-14

Add a licence to upload_to_gspread.py

117. By Brendan Donegan on 2015-05-14

Add a simple unit test for the subunit parsing

[Brendan Donegan (brendan-donegan)]

I added a test for the most complicated part of the script that isn't mainly handled by something else - that is the subunit file parsing. I've not looked into using a staging server and I'm not sure mocking will be very useful here. I can still try something if you feel strongly about it.

[Leo Arias (elopio)]

Hey, did you forget to upload the test? I can see only the subunit file.
As gspread takes care of all the remote processing, maybe you are right, we won't gain much testing that part. And if the spreadsheet is going away as Julien suggested in the meeting yesterday, we shouldn't worry that much about it.
About the subunit file, it would be nice to make it as small as possible, so I think you can remove all the debug finger messages.

[lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread]

118. By Brendan Donegan on 2015-05-14

Finish README update

119. By Brendan Donegan on 2015-05-14

Remove test and subunit file for upload_to_gspread script

120. By Brendan Donegan on 2015-05-14

add link on how to create credentials

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:119
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/227/
Executed test runs:
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-amd64-ci/228
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-armhf-ci/227
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-i386-ci/227

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/227/rebuild

review: Approve (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:120
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/228/
Executed test runs:
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-amd64-ci/229
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-armhf-ci/228
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-i386-ci/228

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/228/rebuild

review: Approve (continuous-integration)

[Christopher Lee (veebers)]

Looks good to me, one inline question which could be a bug. Otherwise this is an approve from me.

review: Needs Information

[lp:~canonical-platform-qa/ubuntu-system-tests/upload_results_gspread]

121. By Brendan Donegan on 2015-05-15

Mention the sheet parameter in the README

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:121
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/229/
Executed test runs:
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-amd64-ci/230
    FAILURE: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-armhf-ci/229/console
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-vivid-i386-ci/229

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-sanity-tests-ci/229/rebuild

review: Needs Fixing (continuous-integration)

[Leo Arias (elopio)]

Things to add to the README:

As veebers mentioned, the sheet name argument.
I got stuck for a long time because I didn't got service API credentials, I first tried with the other two kinds of credentials. It would be good to mention this. And also that the drive api has to be enabled.
Then I got stuck because I didn't add the email address of the credentials to the spreadsheed with edit permissions.
The spreadsheet and the device sheet must exist.

When I finally got it working without errors like this:
$ ubuntu_sanity_tests/scripts/upload_to_gspread.py /tmp/test-002d7944d73f.json "Sanity Test Results" test 12 /tmp/output/artifacts/test-results.subunit

The test sheet is empty. Am I doing something wrong?

Some inline things to fix.

review: Needs Fixing

[Brendan Donegan (brendan-donegan)]

As discussed on IRC maybe we shouldn't merge this? We can always just use the script directly in jenkins if we need it temporarily. This is a short term thing and gspread is terribly akward to work with so let's not merge something that isn't going to be used for very long. We can always reconsider if things change.

review: Disapprove

Unmerged revisions

121. By Brendan Donegan on 2015-05-15

Mention the sheet parameter in the README

120. By Brendan Donegan on 2015-05-14

add link on how to create credentials

119. By Brendan Donegan on 2015-05-14

Remove test and subunit file for upload_to_gspread script

118. By Brendan Donegan on 2015-05-14

Finish README update

117. By Brendan Donegan on 2015-05-14

Add a simple unit test for the subunit parsing

116. By Brendan Donegan on 2015-05-14

Add a licence to upload_to_gspread.py

115. By Brendan Donegan on 2015-05-14

Move the script to under ubuntu_sanity_tests/scripts and add a section to the README

114. By Brendan Donegan on 2015-05-12

Fix flake8 errors

113. By Brendan Donegan on 2015-05-12

Complete refactoring

112. By Brendan Donegan on 2015-05-11

Add script to upload to gspread

=== modified file 'README.rst'
--- README.rst	2015-04-30 11:11:03 +0000
+++ README.rst	2015-05-15 06:28:08 +0000
@@ -235,3 +235,26 @@
 
 If you don't pass any test as argument it will run all the tests in the sanity
 suite.
+
+Upload the results to a Google sheet
+------------------------------------
+
+Results from the sanity tests can be uploaded to a Google sheet using the
+command:
+
+::
+
+  $ upload_to_gspread <secrets file> <sheet> <device> <build> <results subunit>
+
+The secrets file is a JSON file containing secrets necessary for authentication.
+See the `Google OAuth2 documentation <https://developers.google.com/api-client-library/python/auth/installed-app#creatingcred>`_
+for information on how to obtain it. The sheet is the name of the spreadsheet
+to upload to, the device is used as the name of the tab
+in the spreadsheet and the build is what the row is prefixed with.
+The results subunit file contains the test results to be uploaded.
+
+It depends on the gspread and oauth2 modules which can be installed with:
+
+::
+
+  $ pip install gspread oauth2client

=== added directory 'scripts'
=== added directory 'ubuntu_sanity_tests/scripts'
=== added file 'ubuntu_sanity_tests/scripts/upload_to_gspread.py'
--- ubuntu_sanity_tests/scripts/upload_to_gspread.py	1970-01-01 00:00:00 +0000
+++ ubuntu_sanity_tests/scripts/upload_to_gspread.py	2015-05-15 06:28:08 +0000
@@ -0,0 +1,91 @@
+#!/usr/bin/python3
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+#
+# This file is part of the Ubuntu Sanity Tests project
+# Copyright (C) 2015 Canonical
+#
+# This program is free software: you can redistribute it and/or
+# modify it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+import os
+
+import gspread
+
+from argparse import ArgumentParser
+from oauth2client.client import GoogleCredentials
+from subunit.v2 import ByteStreamToStreamResult
+from testtools import StreamResult
+
+
+class TestStreamResult(StreamResult):
+
+    results = {}
+
+    def status(self, test_id=None, test_status=None, test_tags=None,
+               runnable=True, file_name=None, file_bytes=None, eof=False,
+               mime_type=None, route_code=None, timestamp=None):
+        if test_status and test_status != 'inprogress':
+            self.results[test_id] = test_status
+
+
+def parse_subunit(results):
+    test_results = TestStreamResult()
+    with open(results, 'rb') as subunit_file:
+        converter = ByteStreamToStreamResult(source=subunit_file)
+        test_results.startTestRun()
+        converter.run(test_results)
+        test_results.stopTestRun()
+    return test_results.results
+
+
+def authorize(secrets):
+    credentials = get_credentials(secrets)
+    return gspread.authorize(credentials)
+
+
+def get_credentials(secrets):
+    os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = os.path.expanduser(secrets)
+    app_default = GoogleCredentials.get_application_default()
+    return app_default.create_scoped(['https://spreadsheets.google.com/feeds'])
+
+
+def upload_results(gc, sheet, device, build, results):
+    results_doc = gc.open(sheet)
+    device_sheet = results_doc.worksheet(device)
+    results_row = [build]
+    for result in sorted(results.keys()):
+        results_row.append(results[result])
+    device_sheet.append_row(results_row)
+
+
+def _parse_arguments(argv=None):
+    parser = ArgumentParser('Upload results to a Google Spreadsheet')
+    parser.add_argument('secrets', help='Secrets json file')
+    parser.add_argument('sheet', help='The name of the spreadsheet')
+    parser.add_argument('device', help='The name of the device')
+    parser.add_argument('build', help='The build number')
+    parser.add_argument('subunit_file',
+                        help='The subunit file with the results')
+    return parser.parse_args()
+
+
+def main():
+    args = _parse_arguments()
+    results = parse_subunit(args.subunit_file)
+    gc = authorize(args.secrets)
+    upload_results(gc, args.sheet, args.device, args.build, results)
+
+
+if __name__ == "__main__":
+    main()