Merge lp:~ralsina/tanuki-agent/orgless-e2e-snap into lp:tanuki-agent

[Roberto Alsina]

Orgless e2e-snap script

Orgless e2e-snap script

[Celso Providelo (cprov)]

ROberto,

Minor rename suggested inline, looks good to land anyway.

Thank you.

review: Approve

[Roberto Alsina (ralsina)]

approving for the class rename

review: Approve

[Tanuki Bot (tanuki-bot)]

The attempt to merge lp:~ralsina/tanuki-agent/orgless-e2e-snap into lp:tanuki-agent failed. Below is the output from the failed tests.

rm -rf env logs
rm -f version_info.py
find . -name '*.pyc' -delete
find . -name '*.~*' -delete
virtualenv -p python3 env
Using base prefix '/usr'
New python executable in env/bin/python3
Also creating executable in env/bin/python
Installing setuptools, pip...done.
Running virtualenv with interpreter /usr/bin/python3
bzr version-info -r "-1" --format=custom --template="revno = '{revno}'\n" > "version_info.py" || echo "revno = 'UNKNOWN'" > "version_info.py"
rm -rf /tmp/pip-cache
bzr branch lp:~tanuki/tanuki-agent/pip-cache /tmp/pip-cache
env/bin/pip install --no-index --find-links=/tmp/pip-cache -r requirements.txt
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking docopt==0.6.2 (from -r requirements.txt (line 1))
Downloading/unpacking requests==2.7.0 (from -r requirements.txt (line 2))
Downloading/unpacking requests-oauthlib==0.5.0 (from -r requirements.txt (line 3))
Downloading/unpacking oauthlib==1.0.3 (from -r requirements.txt (line 4))
Installing collected packages: docopt, requests, requests-oauthlib, oauthlib
Successfully installed docopt requests requests-oauthlib oauthlib
Cleaning up...
env/bin/pip install --no-index --find-links=/tmp/pip-cache -r test_requirements.txt
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking flake8==2.4.1 (from -r test_requirements.txt (line 1))
Downloading/unpacking pytest==2.7.2 (from -r test_requirements.txt (line 2))
Downloading/unpacking pytest-capturelog==0.7 (from -r test_requirements.txt (line 3))
Downloading/unpacking Markdown==2.6.2 (from -r test_requirements.txt (line 4))
Downloading/unpacking pep8>=1.5.7,!=1.6.0,!=1.6.1,!=1.6.2 (from flake8==2.4.1->-r test_requirements.txt (line 1))
Downloading/unpacking mccabe>=0.2.1,<0.4 (from flake8==2.4.1->-r test_requirements.txt (line 1))
Downloading/unpacking pyflakes>=0.8.1,<0.9 (from flake8==2.4.1->-r test_requirements.txt (line 1))
Downloading/unpacking py>=1.4.29 (from pytest==2.7.2->-r test_requirements.txt (line 2))
Installing collected packages: flake8, pytest, pytest-capturelog, Markdown, pep8, mccabe, pyflakes, py
Successfully installed flake8 pytest pytest-capturelog Markdown pep8 mccabe pyflakes py
Cleaning up...
rm -rf logs
env/bin/py.test agent.py test_agent.py
============================= test session starts ==============================
platform linux -- Python 3.4.3 -- py-1.4.30 -- pytest-2.7.2
rootdir: /tmp/tarmac/branch.43DxWp, inifile:
plugins: capturelog
collected 48 items

test_agent.py ................................................

========================== 48 passed in 0.34 seconds ===========================
env/bin/flake8 *.py scripts/*.py --max-line-length=99
scripts/e2e-snap.py:82:32: E222 multiple spaces after operator
scripts/e2e-snap.py:102:25: E122 continuation line missing indentation or outdented
scripts/e2e-snap.py:103:25: E122 continuation line missing indentation or outdented
scripts/e2e-snap.py:104:25: E122 continuation line missing indentation or outdented
scripts/e2e-snap.py:105:25: E122 continuation line missing indentation or outdente...

[lp:~ralsina/tanuki-agent/orgless-e2e-snap]

183. By Roberto Alsina on 2015-12-03

lint

=== added file 'scripts/e2e-snap.py'
--- scripts/e2e-snap.py	1970-01-01 00:00:00 +0000
+++ scripts/e2e-snap.py	2015-12-03 13:01:36 +0000
@@ -0,0 +1,152 @@
+#!/usr/bin/env python3
+
+# © Canonical 2015
+# Licensed under AGPLv3
+
+"""
+Tanuki End to End Test
+
+Usage:
+  e2e-image.py [-i] <configfile>
+  e2e-image.py --default-config
+
+Options:
+  -h --help             Show this screen
+  --default-config      Show default configuration
+  --server              MessageHub URL override
+  --specmgr             SpecManager URL override
+  -i                    Interactive
+"""
+
+import docopt
+import time
+
+from e2e import E2e, logger, GRY
+
+
+class E2eUpdate(E2e):
+
+    def run(self):
+        self.annotate('Starting.')
+        logger.info('Server : %s', self.server)
+        self.cleanup_products()
+        self.annotate('As Developer or QA, Create Product via Spec Manager')
+        self.create_product()
+        self.submit_gm()
+        self.annotate('As Developer or QA, Create Test Spec via Spec Manager')
+        self.create_spec()
+        self.prompt(
+            'As Lab Owner, Run agent first to register with Message Hub')
+        self.run_agent(reg_only=True)
+        self.pause()
+        self.submit_snap()
+        self.annotate('Sleeping 1s to wait for propagation to hub', color=GRY)
+        time.sleep(1)
+        self.annotate('As Lab Owner, Running agent')
+        self.annotate(
+            'Should pick up at least 1 Test Opportunity,'
+            ' provision and run tests', color=GRY)
+        self.annotate('Note this can also run continously in daemon mode',
+                      color=GRY)
+        self.pause()
+        self.run_agent()
+        self.prompt('Completed agent run')
+        self.annotate('As Developer or QA, Checking result progress via Result Enumerator')
+        self.check_progress()
+        self.annotate('As Developer or QA, Checking final results via Result Enumerator')
+        results = self.check_results()
+        if self.exit_code == 0:  # No need to verify it anything failed
+            self.verify_results(results)
+        self.cleanup_products()
+
+    def verify_results(self, results):
+        for r in results['test_results']:
+            if r['test_opportunity_id'] in self.testopp_ids:
+                # Pop random data
+                for k1, k2 in [
+                        ['test_spec', 'created_at'],
+                        ['product', 'created_at'],
+                        ['agent', 'timing']]:
+                    try:
+                        r[k1].pop(k2)
+                    except KeyError:
+                        logger.error('Missing key in result: r[{}][{}]'.format(k1, k2))
+                        self.exit_code = 1
+
+                for k in "image_unique_id", "base_image_id":
+                    i = r.pop(k)
+                    if i not in self.image_unique_ids:
+                        logger.error("%s is %s, not in %r", k, i, self.image_unique_ids)
+                r["update_snaps"] = sorted(r["update_snaps"])
+
+                expected_data = {
+                    "product_id": self.product_id,
+                    "platform": "amd64",
+                    "base_channel": "stable",
+                    "base_image_reference": None,
+                    "agent_id": "e2e-id",
+                    "update_snaps": sorted([
+                        ["webdm.canonical", 1],
+                        ["ubuntu-core.canonical", 1],
+                        ["test-snap.%s" % self.namespace, 2],
+                        ["pi2.canonical", 1]
+                    ]),
+                    "spec_name": self.spec_name,
+                    "image_reference": None,
+                    "product": {
+                        "active": True,
+                        "primary_snap_name": "test-snap.%s" % self.namespace,
+                        "name": "test-product.%s" % self.namespace,
+                        "release": "rolling-core",
+                        "snaps": [
+                            "ubuntu-core.canonical",
+                            "pi2.canonical",
+                            "webdm.canonical",
+                            "test-snap.%s" % self.namespace
+                        ],
+                        "id": self.product_id
+                    },
+                    "agent": {
+                        "id": "e2e-id"
+                    },
+                    "test_payload": "{'foo': 'bar'}",
+                    "test_status": "PASSED",
+                    "result_payload": {
+                        "foo": "bar"
+                    },
+                    "base_snaps": [
+                        "ubuntu-core.canonical",
+                        "pi2.canonical",
+                        "webdm.canonical",
+                        "test-snap.%s" % self.namespace
+                    ],
+                    "primary_snap_name": "test-snap.%s" % self.namespace,
+                    "type": "update",
+                    "test_spec": {
+                        "active": True,
+                        "product_id": self.product_id,
+                        "channel_combos": [{"update": "edge", "base": "stable"}],
+                        "name": self.spec_name,
+                        "test_payload": "{'foo': 'bar'}",
+                        "platform": "amd64",
+                        "id": self.spec_id
+                    },
+                    "product_name": "test-product.%s" % self.namespace,
+                    "update_channel": "edge"
+                    }
+
+                for k, v in expected_data.items():
+                    try:
+                        if r[k] != v:
+                            self.exit_code = 1
+                            logger.error('expected result[{}] == {},'
+                                         ' got {} instead'.format(k, v, r[k]))
+                    except KeyError:
+                        logger.error('Missing key in result: {}'.format(k))
+                        self.exit_code = 1
+
+if __name__ == '__main__':
+    arguments = docopt.docopt(__doc__, version='Tanuki E2e Proto 1')
+    e2e = E2eUpdate(arguments)
+    e2e.run()
+    exit(e2e.exit_code)

=== modified file 'scripts/e2e.py'
--- scripts/e2e.py	2015-12-02 19:55:42 +0000
+++ scripts/e2e.py	2015-12-03 13:01:36 +0000
@@ -409,27 +409,30 @@
         self.annotate('Deleted Manifest:')
         logger.info(res.json())
 
-    def submit_snap_revision(self):
+    def submit_snap(self):
         self.annotate(
             'As Developer hacking code, Submit a new revision '
             'of a snap to Spec Manager')
         d = {
-            'name': 'snap-b',
-            'revision': 4
+            'channel': 'edge',
+            'name': 'test-snap.%s' % self.namespace,
+            'sequence': 2,
         }
+
+        snap_url = urllib.parse.urljoin(
+            self.specmgr_server,
+            '/'.join(['products', self.product_id, 'snaps']))
         logger.info(
             'POST %s:%s',
-            self.snap_revision_url,
-            json.dumps(
-                d,
-                indent=2))
+            snap_url,
+            json.dumps(d, indent=2))
         self.pause()
 
-        res = retry_it(self.session.post, self.snap_revision_url, json=d, timeout=self.timeout)
+        res = retry_it(self.session.post, snap_url, json=d, timeout=self.timeout)
         res.raise_for_status()
         if res.status_code not in (200, 201, 202) or res.json() is None:
             logger.error(
-                'Error submitting snap revision: %r, %r',
+                'Error submitting snap: %r, %r',
                 res.status_code,
                 res.text)
             raise Exception