Merge lp:~cjohnston/ubuntu-ci-services-itself/TS-integration-tests into lp:ubuntu-ci-services-itself

[Chris Johnston]

Test that after the TS is deployed, we can create a ticket, then that the TS displays that ticket via API.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:238
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/193/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/193/rebuild

review: Approve (continuous-integration)

[Evan (ev)]

Hm, why wasn't this done on top of Amulet like the other integration tests? We won't know if something is broken in the link between the ticket system, gunicorn, and the Apache frontend. I ran into just such an issue before we put Apache in front of gunicorn where gunicorn's braindead URL parsing broke "http://ticket_system//api/v1/..."

review: Needs Information

[Evan (ev)]

D'oh. I need more coffee. I now see that you're just using virtualenv for the CLI.

[Vincent Ladeuil (vila)]

61 +
62 + def tearDown(self):
63 + delete_virtual_env(self.temp_dir)

This miss the upcall to tearDown but you completely avoid tearDown itself by doing:

   self.addCleanup(delete_virtual_env, self.temp_dir)

in setUp()

Since we already have a venv set to run the unit tests (and plan to also use it to run the other integration tests), I would rather make it a requirement that these tests should run under the venv than creating a new one each time. This is a costly operation which also have known failures which in this case will fail the test for the wrong reason.

65 + def get_server_status_and_content(self, url, service='ts-django'):

86 + resp, content = self.get_server_status_and_content(url)

98 + resp, content = self.get_server_status_and_content(
99 + url, service='webui-apache')

It would be more consistent to have no default value in get_server_status_and_content (unless there is a good reason for that which in this case is worth documenting).

88 + self.assertEqual(resp['status'], '200')
89 + self.assertEqual(content['title'], 'New feature')
90 + self.assertEqual(content['current_workflow_step'], 'Queued')

All other integration tests use expected, actual, let's not add more inconsistency, thanks ;)

101 + self.assertEqual(resp['status'], '200')
102 + self.assertIn('<a href="/" class="first ">Ticket List</a>', content)

especially with assertIn also using expected, actual

[Andy Doan (doanac)]

25 +def delete_virtual_env(temp_dir):
26 + remove = ('rm -rf %s' % temp_dir)
27 + subprocess.check_call([remove], shell=True)

Is there a reason shutil.rmtree isn't being used here instead?

//massive bikeshedding, but a style suggestion for the future:
55 + args = ('%s develop' % os.path.join(
56 + os.path.dirname(__file__), '../../ci-utils/setup.py'))

could be:

    p = os.path.join(os.path.dirname(__file__), '../../ci-utils/setup.py'))
    args = '%s develop' % p

and then you don't get weird multi-line parentheses to match with your eyes.

92 + def test_ticket_in_webui_ticket_list(self):

This method only works when "test_create_ticket_via_cli" is called first. Its probably smarter if you move the ticket creation logic from "test_create_ticket_via_cli" into your setUp method and then just let your 2 test methods assert that both the ticket-system and webui are displaying what we expect.

[Evan (ev)]

Other than the issues already raised, this looks good. +1

review: Approve

[Chris Johnston (cjohnston)]

On Thu, Feb 20, 2014 at 11:31 AM, Andy Doan <email address hidden>wrote:

> 25 +def delete_virtual_env(temp_dir):
> 26 + remove = ('rm -rf %s' % temp_dir)
> 27 + subprocess.check_call([remove], shell=True)
>
> Is there a reason shutil.rmtree isn't being used here instead?
>

r241

> //massive bikeshedding, but a style suggestion for the future:
> 55 + args = ('%s develop' % os.path.join(
> 56 + os.path.dirname(__file__), '../../ci-utils/setup.py'))
>
> could be:
>
> p = os.path.join(os.path.dirname(__file__), '../../ci-utils/setup.py'))
> args = '%s develop' % p
>
> and then you don't get weird multi-line parentheses to match with your
> eyes.
>

r239

>
> 92 + def test_ticket_in_webui_ticket_list(self):
>
> This method only works when "test_create_ticket_via_cli" is called first.
> Its probably smarter if you move the ticket creation logic from
> "test_create_ticket_via_cli" into your setUp method and then just let your
> 2 test methods assert that both the ticket-system and webui are displaying
> what we expect.
>

r240

[lp:~cjohnston/ubuntu-ci-services-itself/TS-integration-tests]

239. By Chris Johnston on 2014-02-20

Fix formatting for args strings

240. By Chris Johnston on 2014-02-20

Move create ticket into setUp

241. By Chris Johnston on 2014-02-20

use shutil

[Chris Johnston (cjohnston)]

review: Needs Resubmitting

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:241
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/200/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/200/rebuild

review: Needs Fixing (continuous-integration)

[Andy Doan (doanac)]

solves my complaints

review: Approve

[lp:~cjohnston/ubuntu-ci-services-itself/TS-integration-tests]

242. By Chris Johnston on 2014-02-28

Switch to using self.addCleanup

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:242
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/259/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/259/rebuild

review: Approve (continuous-integration)

=== added directory 'tests/integration'
=== added file 'tests/integration/test.py'
--- tests/integration/test.py	1970-01-01 00:00:00 +0000
+++ tests/integration/test.py	2014-02-28 20:55:47 +0000
@@ -0,0 +1,98 @@
+#!/usr/bin/env python
+
+import os
+import json
+import tempfile
+import subprocess
+import unittest
+import shutil
+
+import httplib2
+
+from tests.deployers import DEPLOYER_DIR, DeployerTest
+
+
+def create_virtual_env():
+    temp_dir = tempfile.mkdtemp()
+    subprocess.check_call(['virtualenv', temp_dir])
+    return temp_dir
+
+
+def delete_virtual_env(temp_dir):
+    shutil.rmtree(temp_dir)
+
+
+def run_virtual_env(temp_dir, args, cwd=None):
+    if args is 'create':
+        cwd = os.path.join(os.path.dirname(__file__), '../../cli')
+        changes = os.path.abspath(os.path.join(
+            os.path.dirname(__file__),
+            '../../cli/tests/data/foobar_0.1-1_source.changes'))
+        args = ('python ubuntu-ci create_ticket -t "New feature" -b 1234 -o '
+                '[e-mail] -d "This is a cool new feature" -r baz '
+                '-s %s' % changes)
+    args = '. %s ; %s' % (os.path.join(temp_dir, 'bin/activate'),
+                          args)
+    if cwd:
+        subprocess.check_call([args], cwd=cwd, shell=True)
+    else:
+        subprocess.check_call(args, shell=True)
+
+
+class IntegrationTest(DeployerTest):
+    """Integration tests for ticket-system service run on a juju deployment"""
+
+    deployer_cfg = os.path.join(DEPLOYER_DIR, 'ticket-system.yaml')
+
+    def setUp(self):
+        super(IntegrationTest, self).setUp()
+        self.temp_dir = create_virtual_env()
+        p = os.path.join(os.path.dirname(__file__), '../../ci-utils/setup.py')
+        args = '%s develop' % p
+        run_virtual_env(self.temp_dir, args)
+        p = os.path.join(os.path.dirname(__file__), '../../cli/setup.py')
+        args = '%s develop' % p
+        run_virtual_env(self.temp_dir, args)
+        config_url = 'http://{}:{}'
+        config_url = config_url.format(*self.get_ip_and_port('ts-django'))
+        config = os.path.join(os.environ["HOME"], '.ubuntu-ci')
+        with open(config, 'a') as fp:
+            fp.write('ci_url: "{}"\n'.format(config_url))
+
+        run_virtual_env(self.temp_dir, args='create')
+        self.addCleanup(delete_virtual_env, self.temp_dir)
+
+    def get_server_status_and_content(self, url, service='ts-django'):
+        final_url = url.format(*self.get_ip_and_port(service))
+
+        client = httplib2.Http()
+        resp, content = client.request(final_url, 'GET')
+        return resp, content
+
+    def test_create_ticket_via_cli(self):
+        """
+        This tests that the CLI can create a ticket and that the ticket shows
+        up with the proper status when viewed via the API.
+        """
+
+        url = 'http://{}:{}/api/v1/ticket/1/'
+        resp, content = self.get_server_status_and_content(url)
+        content = json.loads(content)
+        self.assertEqual(resp['status'], '200')
+        self.assertEqual(content['title'], 'New feature')
+        self.assertEqual(content['current_workflow_step'], 'Queued')
+
+    def test_ticket_in_webui_ticket_list(self):
+        """
+        This tests that the WebUI main page shows the ticket that we created
+        via the CLI
+        """
+        url = 'http://{}:{}/'
+        resp, content = self.get_server_status_and_content(
+            url, service='webui-apache')
+
+        self.assertEqual(resp['status'], '200')
+        self.assertIn('<a href="/" class="first ">Ticket List</a>', content)
+
+if __name__ == "__main__":
+    unittest.main()