Merge lp:~canonical-platform-qa/qakit/trigger_jenkins_builds into lp:qakit

[Santiago Baldassin]

Ubuntu system tests worker

This mp includes a worker that could be used by Bileto to trigger the ubuntu system tests. There are only two public apis that should be used and that can be invoked directly with python or via an amqp message. Those apis are

run_ust(ppa)
get_results(ppa)

Every single request in Bileto gets it's own ppa with the source packages involved in the change.

The first api will execute the following steps:
        1. Get the source packages from launchpad that are
        included in the ppa under test
        2. Use the packages versions to create a request id that will be use to track
        the test results for the source packages. The idea is to run the tests for every
        version of the source packages in the ppa
        3. Get the practitest filter id associated to the source packages in the ppa
        4. Get the list of test cases to be run from practitest
        5. Trigger a jenkins build that will:
            a. Flash the device
            b. Update the device with the ppa under test
            c. Setup the device for testing
            d. Run the tests

The second api will retrive the test results for a given ppa

[Sergio Cazzolato (sergio-j-cazzolato)]

Some minor comments inline. In parallel you could start testing it integrated with jenkins to see how it works.

Please make sure in the final solution the jenkins jobs invoked have timeouts set.

review: Needs Fixing

[Santiago Baldassin (sbaldassin)]

Thanks for your review Sergio. I've addressed all your comments. By the way, I'm already testing this using real Jenkins job. This is the job that I'm using: https://platform-qa-jenkins.ubuntu.com/job/test_alesage_ust_rc-proposed_arale_add_ppa/

[lp:~canonical-platform-qa/qakit/trigger_jenkins_builds]

154. By Santiago Baldassin on 2016-09-06

Addressing comments from the reviews

[Richard Huddie (rhuddie)]

Some minor comments and questions below.

review: Needs Fixing

[lp:~canonical-platform-qa/qakit/trigger_jenkins_builds]

155. By Santiago Baldassin on 2016-09-15

Updating the worker and the domains

156. By Santiago Baldassin on 2016-09-15

Addressing comments from the reviews

157. By Santiago Baldassin on 2016-09-15

Merge from trunk

[Sergio Cazzolato (sergio-j-cazzolato)]

Approved, code lgtm

review: Approve

Unmerged revisions

157. By Santiago Baldassin on 2016-09-15

Merge from trunk

156. By Santiago Baldassin on 2016-09-15

Addressing comments from the reviews

155. By Santiago Baldassin on 2016-09-15

Updating the worker and the domains

154. By Santiago Baldassin on 2016-09-06

Addressing comments from the reviews

153. By Santiago Baldassin on 2016-09-03

Ubuntu system tests worker

=== added directory 'qakit/config'
=== added file 'qakit/config/__init__.py'
--- qakit/config/__init__.py	1970-01-01 00:00:00 +0000
+++ qakit/config/__init__.py	2016-09-15 18:52:25 +0000
@@ -0,0 +1,14 @@
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'qakit/config/config.py'
--- qakit/config/config.py	1970-01-01 00:00:00 +0000
+++ qakit/config/config.py	2016-09-15 18:52:25 +0000
@@ -0,0 +1,26 @@
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+from configparser import ConfigParser
+
+
+class Config:
+
+    def __init__(self):
+        self.parser = ConfigParser()
+        conf_file = os.getenv('QAKIT_CONFIG',
+                              os.path.expanduser('~/.config/qakit.ini'))
+        self.parser.read(conf_file)

=== added directory 'qakit/launchpad'
=== added file 'qakit/launchpad/__init__.py'
--- qakit/launchpad/__init__.py	1970-01-01 00:00:00 +0000
+++ qakit/launchpad/__init__.py	2016-09-15 18:52:25 +0000
@@ -0,0 +1,14 @@
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'qakit/launchpad/ppa.py'
--- qakit/launchpad/ppa.py	1970-01-01 00:00:00 +0000
+++ qakit/launchpad/ppa.py	2016-09-15 18:52:25 +0000
@@ -0,0 +1,49 @@
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+import hashlib
+
+from launchpadlib.launchpad import Launchpad
+
+
+class Ppa:
+
+    def __init__(self, name, team):
+        self.name = name
+        self.team = team
+
+    def get_source_packages(self, status='Published'):
+        """
+        This method returns from Launchpad, the list of source packages
+        included in the ppa. By default, only Published source packages
+        are returned
+        :param status the source package status to filter the packages
+        in the ppa
+        :return a list of source packages
+        """
+        launchpad = Launchpad.login_anonymously('ci job', 'production')
+        team = launchpad.people.findTeam(text=self.team)[0]
+        archive = team.getPPAByName(name=self.name)
+        return archive.getPublishedSources(status=status)
+
+    def calculate_hash(self):
+        """
+        This method get the list of published source packages in the ppa
+        and use their version to calculate a hash which uniquely identify
+        the current set of published packages in the ppa
+        :return: the hash that identifies the list of packages in the ppa
+        """
+        src_pkgs = self.get_source_packages()
+        versions = ''.join([package.source_package_version for package in src_pkgs])
+        return hashlib.md5(versions.encode('utf-8')).hexdigest()

=== added directory 'qakit/practitest/data'
=== added file 'qakit/practitest/data/__init__.py'
--- qakit/practitest/data/__init__.py	1970-01-01 00:00:00 +0000
+++ qakit/practitest/data/__init__.py	2016-09-15 18:52:25 +0000
@@ -0,0 +1,14 @@
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'qakit/practitest/data/domains.json'
--- qakit/practitest/data/domains.json	1970-01-01 00:00:00 +0000
+++ qakit/practitest/data/domains.json	2016-09-15 18:52:25 +0000
@@ -0,0 +1,8 @@
+{
+    "unity-scope-click": ["126288"],
+    "pay-service": ["126288"],
+    "account-plugins": ["126288"],
+    "signon-plugin-oauth2": ["126288"],
+    "messaging-app": ["126288"],
+    "goget-ubuntu-touch": ["126288"]
+}

=== modified file 'qakit/practitest/practitest.py'
--- qakit/practitest/practitest.py	2015-11-30 18:24:48 +0000
+++ qakit/practitest/practitest.py	2016-09-15 18:52:25 +0000
@@ -18,6 +18,8 @@
 import datetime
 import json
 import logging
+import os
+
 import requests
 
 from pprint import pformat
@@ -384,3 +386,20 @@
         logging.debug(pformat(result))
         logging.info('Updating issue pt:{} / lp:{}'.format(ptid, lpid))
         return self._put(url, data=result)
+
+    @staticmethod
+    def get_filters_for_src_pkgs(src_pkgs):
+        """
+        This method return the list of practitest filter ids associated
+        to the src pacckages in the params
+        :param src_pkgs: list of source packages
+        :return: list of filters
+        """
+        practitest_dir = os.path.dirname(__file__)
+        domains_file = os.path.join(practitest_dir, 'data', 'domains.json')
+        with open(domains_file) as domains:
+            domains_json = json.loads(domains.read())
+        filters = [domains_json[package] for package in src_pkgs]
+
+        # Flat the lists and remove duplicates
+        return list(set([item for sublist in filters for item in sublist]))

=== added directory 'qakit/ust'
=== added file 'qakit/ust/__init__.py'
--- qakit/ust/__init__.py	1970-01-01 00:00:00 +0000
+++ qakit/ust/__init__.py	2016-09-15 18:52:25 +0000
@@ -0,0 +1,26 @@
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+import logging
+import sys
+
+root = logging.getLogger()
+root.setLevel(logging.DEBUG)
+
+ch = logging.StreamHandler(sys.stdout)
+ch.setLevel(logging.INFO)
+formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+ch.setFormatter(formatter)
+root.addHandler(ch)
+

=== added file 'qakit/ust/worker.py'
--- qakit/ust/worker.py	1970-01-01 00:00:00 +0000
+++ qakit/ust/worker.py	2016-09-15 18:52:25 +0000
@@ -0,0 +1,180 @@
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+import json
+import logging
+
+from jenkins import Jenkins
+from pika import BlockingConnection
+from pika import ConnectionParameters
+
+from qakit.config.config import Config
+from qakit.launchpad.ppa import Ppa
+from qakit.practitest.practitest import PractitestSession
+
+
+class Worker:
+
+    bileto_results = {"PASSED": "Approved", "FAILURE": "Failed", "QUEUED": "Queued"}
+
+    def __init__(self):
+        self.conf = Config()
+        self.jenkins = Jenkins(self.conf.parser.get('jenkins', 'server'),
+                               self.conf.parser.get('jenkins', 'username'),
+                               self.conf.parser.get('jenkins', 'password'))
+
+    def run_ust(self, ppa):
+        """"
+        This method will execute the following steps:
+        1. Get the source packages from launchpad that are
+        included in the ppa under test
+        2. Use the packages versions to create a request id that will be use to track
+        the test results for the source packages. The idea is to run the tests for every
+        version of the source packages in the ppa
+        3. Get the practitest filter id associated to the source packages in the ppa
+        4. Get the list of test cases to be run from practitest
+        5. Trigger a jenkins build that will:
+            a. Flash the device
+            b. Update the device with the ppa under test
+            c. Setup the device for testing
+            d. Run the tests
+
+        :param ppa is the ppa associated to the request in bileto and it is
+        expected to be in the format team/ppa_name.
+        For example: ci-train-ppa-service/landing-059
+
+        This method has two entry points, it can be called by importing the
+        qakit module and creating a Worker instance like this
+            Worker().run_ust("ci-train-ppa-service/landing-059")
+        And there's also an amqp entry point. The start method will start an amqp
+        client which will be subscribed to /ubuntu-system-test, whenever a message
+        is published in such a topic, it will be process by the
+        process_ust_request method which will call this one to run the tests
+        """
+        team, ppa_name = ppa.split('/')
+        launchpad_ppa = Ppa(team=team, name=ppa_name)
+        src_pkgs = launchpad_ppa.get_source_packages()
+        packages = list(
+            set([package.source_package_name for package in src_pkgs]))
+        request_id = launchpad_ppa.calculate_hash()
+
+        pt = PractitestSession(self.conf.parser.get('practitest', 'project_id'),
+                               self.conf.parser.get('practitest', 'api_key'),
+                               self.conf.parser.get('practitest', 'api_secret_key'))
+        filters = pt.get_filters_for_src_pkgs(packages)
+        tests = pt.get_tests(filters)
+
+        try:
+            tests_ids = ' '.join([test['___f_10589']['value'] for test in tests])
+        except KeyError:
+            tests_ids = ' '
+
+        self.jenkins.build_job(self.conf.parser.get('jenkins', 'job_name'),
+                               {"PPA": ppa, "TESTS_TO_RUN": tests_ids, "REQUEST_ID": request_id},
+                               {'token': self.conf.parser.get('jenkins', 'token')})
+
+    def get_result(self, ppa):
+        """
+        The tests results are retrieved from jenkins and the ppa is used to get the
+        appropriate tests results. As when the tests were triggered, the list of
+        source packages belonging to the ppa are retrieved from launchpad and their
+        versions is used to get the request id associated to those group of packages
+
+        We'll first iterate on the builds returned by the get_job_info and we'll return
+        the results associated to the build which request id matches the request id calculated
+        by this method.
+
+        If the correct build is not found in the get_job_info results, then we'll check the
+        get_queue_info and if the build is not found there, then an error is returned
+        """
+        team, ppa_name = ppa.split('/')
+        launchpad_ppa = Ppa(team=team, name=ppa_name)
+        request_id = launchpad_ppa.calculate_hash()
+
+        builds = self.jenkins.get_job_info(
+            self.conf.parser.get('jenkins', 'job_name'), depth=2)['builds']
+        build = self._get_build_by_request_id(builds, request_id)
+
+        if build:
+            build_info = self.jenkins.get_build_info(
+                self.conf.parser.get('jenkins', 'job_name'), build['number'])
+            return UstResult(self.bileto_results[build_info['result']], build_info['url'])
+        else:
+            builds = self.jenkins.get_queue_info()
+            build = self._get_build_by_request_id(builds, request_id)
+            if build:
+                return UstResult("QUEUED", "")
+            else:
+                return None
+
+    @staticmethod
+    def _get_build_by_request_id(builds, request_id):
+        """
+        This method iterates over a list of buils and return the one which
+        request id in the parameters matches the request_id
+        :param builds: List of builds
+        :param request_id: request id to filter the builds
+        :return: build
+        """
+        for build in builds:
+            parameters = list(filter(
+                lambda action: action.get('parameters', None),
+                build['actions']))[0]['parameters']
+            try:
+                req_id = list(filter(
+                    lambda kv: kv['name'] == 'REQUEST_ID', parameters))[0]['value']
+                if request_id == req_id:
+                    return build
+            except IndexError:
+                logging.info("Build does not have a request id set")
+        return None
+
+    def start(self):
+        """
+        This method would start an amqp client and subscribe to the queue
+        set in the configuration file
+        """
+        try:
+            logging.info('Connecting to AMQP server: {}'.format(
+                self.conf.parser.get('amqp', 'host')))
+            connection = BlockingConnection(ConnectionParameters(
+                host=self.conf.parser.get('amqp', 'host')))
+            channel = connection.channel()
+            queue = self.conf.parser.get('amqp', 'queue')
+
+            logging.info('Subscribing to queue: {}'.format(
+                self.conf.parser.get('amqp', 'queue')))
+            channel.queue_declare(queue, durable=True, auto_delete=False)
+            channel.basic_consume(
+                self.process_ust_request, queue=queue, no_ack=True)
+            channel.start_consuming()
+        except KeyboardInterrupt:
+            channel.stop_consuming()
+            connection.close()
+
+    def process_ust_request(self, ch, method, properties, body):
+        """Callback for ubuntu system test request"""
+        logging.info('Message received {}'.format(body))
+
+        ppa = json.loads(body.decode('utf-8'))['ppa']
+        self.run_ust(ppa)
+
+
+class UstResult:
+    def __init__(self, status, build_url):
+        self.status = status
+        self.build_url = build_url
+
+if __name__ == '__main__':
+    Worker().start()

=== added file 'start_worker.py'
--- start_worker.py	1970-01-01 00:00:00 +0000
+++ start_worker.py	2016-09-15 18:52:25 +0000
@@ -0,0 +1,31 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import sys
+
+from qakit.ust.worker import Worker
+
+
+def main():
+    worker = Worker()
+    return worker.start()
+
+if __name__ == '__main__':
+    sys.exit(main())