Merge lp:~sbaldassin/qa-jenkins-jobs/run_tests into lp:qa-jenkins-jobs

[Santiago Baldassin]

Adding a script to select the tests to be run from the commit message

This mp adds a python script to get the list of tests to be run from the commit message in the merge proposal

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:165
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/23/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2949
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2949/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2949
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2949/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/466/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/23/rebuild

review: Approve (continuous-integration)

[Max Brustkern (nuclearbob)]

I'd like the file to be executable. When I run it like this:
python3 get_tests_to_run.py -mpu "https://code.launchpad.net/~canonical-platform-qa/ubuntu-system-tests/dynamic-dependencies"
I get this:
Traceback (most recent call last):
  File "get_tests_to_run.py", line 30, in <module>
    main(args.merge_proposal_url)
  File "get_tests_to_run.py", line 20, in main
    mp = json.loads(requests.get(branch.landing_targets_collection_link).text)
AttributeError: 'NoneType' object has no attribute 'landing_targets_collection_link'

review: Needs Fixing

[Santiago Baldassin (sbaldassin)]

Thanks for the feedback Max. There's a problem with the line above, you should use the lp url instead of the http url. python3 get_tests_to_run.py -mpu "lp:~canonical-platform-qa/ubuntu-system-tests/dynamic-dependencies"

I'll make a couple of changes to enforce the use of the lp url. Moreover the above line will through an error message because there are no tests included in the commit message so I'll improve that as well

[lp:~sbaldassin/qa-jenkins-jobs/run_tests]

166. By Santiago Baldassin on 2016-07-25

Addressing comments from the reviews

[Santiago Baldassin (sbaldassin)]

I've updated the script and tested it with the following command line: TESTS_TO_RUN=$(python3 get_tests_to_run.py -mpu "lp:~canonical-platform-qa/ubuntu-system-tests/restart_unity8" 2>&1).
That's the command that we should include in the jenkins step, that way, TESTS_TO_RUN should be available within the jenkins job context

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:166
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/24/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2954
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2954/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2954
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2954/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/471/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/24/rebuild

review: Approve (continuous-integration)

[Max Brustkern (nuclearbob)]

I'm getting commas in the list. Will ubuntu-system-tests accept that? I always send a list of tests separated by spaces with no commas.

Also, I'd like to see the script be executable and inside the ubuntu-system-tests directory.

review: Needs Information

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:165
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci-test/1/
Executed test runs:
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2979/console
    FAILURE: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2979/console
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/492/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/ubuntu-system-tests-ci-test/1/rebuild

review: Needs Fixing (continuous-integration)

[Allan LeSage (allanlesage)]

Ignore above failure :) .

[Allan LeSage (allanlesage)]

Sorry for the delay in testing this, you can see it working here: https://code.launchpad.net/~canonical-platform-qa/ubuntu-system-tests/mp-test-do-not-merge/+merge/301542 .

One note, that I think we should be getting the MP not via the branch but via the MP itself, although I can't find any way to do this cleanly via the API (https://launchpad.net/+apidoc/1.0.html) :/ . It's pretty rare that we're proposing to merge against more than one though so ok for now.

I agree with Max about relocating the script to /scripts/ubuntu-system-tests .

Also let's do a flake8 check on this just to be extra-picky.

review: Needs Fixing

[lp:~sbaldassin/qa-jenkins-jobs/run_tests]

167. By Santiago Baldassin on 2016-08-01

Addressing comments from the reviews

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:167
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/27/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2992
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2992/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2992
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2992/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/505/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/27/rebuild

review: Approve (continuous-integration)

[lp:~sbaldassin/qa-jenkins-jobs/run_tests]

168. By Santiago Baldassin on 2016-08-01

Adding an option to get the tests to run from the http url

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:168
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/29/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2996
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/2996/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2996
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/2996/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/508/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/29/rebuild

review: Approve (continuous-integration)

[Allan LeSage (allanlesage)]

Hey do we need that get_test_to_run.sh script, sbalda? (Asking b/c I'm skeptical :) .)

Otherwise this runs well and thanks for the MP option.

review: Needs Fixing

[lp:~sbaldassin/qa-jenkins-jobs/run_tests]

169. By Santiago Baldassin on 2016-08-02

addressing comments from the reviews

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:169
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/30/
Executed test runs:
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3005
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-amd64-package/3005/artifact/work/output/*zip*/output.zip
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3005
        deb: https://platform-qa-jenkins.ubuntu.com/job/build-wily-i386-package/3005/artifact/work/output/*zip*/output.zip
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/517/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/qa-jenkins-jobs-ci/30/rebuild

review: Approve (continuous-integration)

[Allan LeSage (allanlesage)]

review: Approve

=== added file 'scripts/ubuntu-system-tests/get_tests_to_run.py'
--- scripts/ubuntu-system-tests/get_tests_to_run.py	1970-01-01 00:00:00 +0000
+++ scripts/ubuntu-system-tests/get_tests_to_run.py	2016-08-02 16:23:52 +0000
@@ -0,0 +1,46 @@
+#!/usr/bin/python
+import argparse
+import requests
+import sys
+from launchpadlib.launchpad import Launchpad
+
+MP_URL_PREFIX = 'https://code.launchpad.net/'
+LAUNCHPAD_API_PREFIX = 'https://api.launchpad.net/1.0/'
+
+
+def main(mp_url):
+    if mp_url.startswith('lp'):
+        # Login to launchpad anonymously
+        launchpad = Launchpad.login_anonymously('ci job', 'production')
+
+        # Get the branch
+        branch = launchpad.branches.getByUrl(url=mp_url)
+
+        if branch is not None:
+            # Get the merge proposal data
+            mp = requests.get(
+                branch.landing_targets_collection_link).json()['entries'][0]
+        else:
+            raise ValueError("The url provided did not returned any "
+                             "data from launchpad")
+    elif mp_url.startswith('http'):
+        url = mp_url.replace(MP_URL_PREFIX, LAUNCHPAD_API_PREFIX)
+        mp = requests.get(url).json()
+    else:
+        raise ValueError("Please provide a valid url")
+
+    commit_message = mp['commit_message']
+    try:
+        tests = commit_message.split('@run_tests:')[1:][0].strip()
+    except IndexError:
+        tests = []
+    sys.stdout.write(str(tests))
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(
+        description='This script will get the list of tests '
+                    'to be run from the mp commit message')
+    parser.add_argument('-mpu', '--merge_proposal_url',
+                        help='Merge proposal url', required=True)
+    args = parser.parse_args()
+    main(args.merge_proposal_url)