Merge lp:~benji/lpsetup/bug-1020734-first-integration-test into lp:lpsetup

[Benji York]

Add a first integration test.

This is a first cut at an integration test for lpsetup that uses juju to provision a clean EC2 instance and then uses that instance to run a test.

This first cut only tests init-host because I couldn't get init-repo to work yet. That is either a bug in init-repo or a deficiency in the test setup, either way I wanted to get this test landed and we can address testing init-repo separately.

See the README for instructions on running the test.

[Gary Poster (gary)]

Yay! Thank you! The following are random thoughts while looking at the
code. We can address all of them later, I think (post-landing) but I'd
like to address them one way or another. Maybe #1 should be addressed now.

1) I'm not sure how we can run this in tarmac as is; it will need to
transfer the branch as currently tested over to the other machine, and
not a particular branch. Maybe juju scp -r [the current code tree, as
calculated in the test by os.path fun with __file__] would be better for
transferring the code? Once we have that ironed out, and this branch
landed, I'd really like a new card that hooks up the integration test on
tarmac to be created and started ASAP.

2) I understand why python-shelltoolbox is needed but not why apache is
needed. Shouldn't lpsetup get apache?

3) I'm not sure why you have to set up bzr whoami, and I think lpsetup
is supposed to be able to do that. The test should probably take a
non-root user as argument, though. We can do that as a next step, though.

4) We can refine the time.sleep(30) later.

5) The lxc story will need to have lp-lxc-ip installed. I'd suggest
making the setup run python setup.py install, and then have the test run
the lp-setup from the environment.

6) The lxc story also will need to install lpsetup in the lxc container.
 Normally that is done from the PPA, but we can't do that here, because
that would not test the branch that we currently are running. Since the
home directories are shared in the host and the directory, I think we
could have an option to initlxc that passes a directory, and means
"instead of getting the PPA in the container, use python setup.py
install [this directory]".

#6 runs counter to what I wrote in my previous email about this, but
there are many upsides to your alternative approach and I think we ought
to stick with it.

Thanks again!

[Gary Poster (gary)]

review: Approve

[Benji York (benji)]

> 1) I'm not sure how we can run this in tarmac as is; it will need to
> transfer the branch as currently tested over to the other machine, and
> not a particular branch. Maybe juju scp -r [the current code tree, as
> calculated in the test by os.path fun with __file__] would be better for
> transferring the code?

Good idea. Done.

> Once we have that ironed out, and this branch
> landed, I'd really like a new card that hooks up the integration test on
> tarmac to be created and started ASAP.

I have created a card ("Tarmac should run the integration tests"), but
because of the SSH-needs-desperately-to-ask-yes-or-no-questions issue we
talked about earlier, it may be tricky.

> 2) I understand why python-shelltoolbox is needed but not why apache is
> needed. Shouldn't lpsetup get apache?

It should, and now it does (apache2.2-common, that is).

> 3) I'm not sure why you have to set up bzr whoami, and I think lpsetup
> is supposed to be able to do that. The test should probably take a
> non-root user as argument, though. We can do that as a next step, though.

This was so the test takes the most common path through lpsetup. As we
discussed, I moved the whomai setup into the test function, which makes
more sense.

> 4) We can refine the time.sleep(30) later.

I guess we're... <puts on sunglasses> hitting the snooze bar.

> 5) The lxc story will need to have lp-lxc-ip installed. I'd suggest
> making the setup run python setup.py install, and then have the test run
> the lp-setup from the environment.
>
> 6) The lxc story also will need to install lpsetup in the lxc container.
> Normally that is done from the PPA, but we can't do that here, because
> that would not test the branch that we currently are running. Since the
> home directories are shared in the host and the directory, I think we
> could have an option to initlxc that passes a directory, and means
> "instead of getting the PPA in the container, use python setup.py
> install [this directory]".
>
> #6 runs counter to what I wrote in my previous email about this, but
> there are many upsides to your alternative approach and I think we ought
> to stick with it.

The above has been moved to a new "Make LXC integration test" card.

[Launchpad QA Bot (lpqabot)]

The attempt to merge lp:~benji/lpsetup/bug-1020734-first-integration-test into lp:lpsetup failed. Below is the output from the failed tests.

./lpsetup/tests/integration/non-lxc.py
      11: 'STDOUT' imported but unused
      18: E302 expected 2 blank lines, found 1
      24: E302 expected 2 blank lines, found 1
./lpsetup/subcommands/initrepo.py
      18: 'LP_SSH_REPO' imported but unused

[lp:~benji/lpsetup/bug-1020734-first-integration-test]

53. By Benji York on 2012-07-10

fix lint

[Gary Poster (gary)]

Benji and I have verified that juju ssh can take the arguments we need, so we can address the tarmac story in the future without too much trouble, it seems. Example:

$ juju ssh -t -t -o 'StrictHostKeyChecking=no' -o 'UserKnownHostsFile=/dev/null' 2

=== modified file 'README.rst'
--- README.rst	2012-07-03 18:02:41 +0000
+++ README.rst	2012-07-10 16:18:18 +0000
@@ -44,6 +44,19 @@
     nosetests
 
 
+Integration tests
+~~~~~~~~~~~~~~~~~
+
+Integration tests can be found in lpsetup/tests/integration.  They
+require the "ubuntu" juju charm which, at the time of this writing, is
+not available in the charm store.  Check it out into the local juju
+charm repository from lp:~charmers/charms/precise/ubuntu/trunk.
+
+The integration tests use a juju environment named "lpsetup-testing" so
+you must create an appropriately configured juju environment with that
+name before running the tests.
+
+
 Linting
 ~~~~~~~
 

=== modified file 'lpsetup/handlers.py'
--- lpsetup/handlers.py	2012-07-06 19:18:58 +0000
+++ lpsetup/handlers.py	2012-07-10 16:18:18 +0000
@@ -255,7 +255,7 @@
 def handle_source(namespace):
     """Handle the --source and --use-http options."""
     # If a source has been provided, then there is nothing else to do.
-    if namespace.source:
+    if namespace.source is not None:
         return
     if namespace.use_http:
         namespace.source = LP_HTTP_REPO

=== modified file 'lpsetup/settings.py'
--- lpsetup/settings.py	2012-07-09 14:40:11 +0000
+++ lpsetup/settings.py	2012-07-10 16:18:18 +0000
@@ -12,7 +12,8 @@
     'ppa:launchpad/ppa',
     'ppa:bzr/ppa',
     )
-BASE_PACKAGES = ['ssh', 'bzr']
+# a2enmod requires apache2.2-common
+BASE_PACKAGES = ['ssh', 'bzr', 'apache2.2-common']
 CHECKOUT_DIR = '~/launchpad/lp-branches'
 DEPENDENCIES_DIR = '~/launchpad/lp-sourcedeps'
 HOSTS_CONTENT = (

=== modified file 'lpsetup/subcommands/inithost.py'
--- lpsetup/subcommands/inithost.py	2012-07-09 11:36:53 +0000
+++ lpsetup/subcommands/inithost.py	2012-07-10 16:18:18 +0000
@@ -201,9 +201,10 @@
 
         # Create Apache document roots, to avoid warnings.
         mkdirs(*LP_APACHE_ROOTS)
-        # Set up Apache modules.
-        for module in LP_APACHE_MODULES.split():
-            call('a2enmod', module)
+
+    # Set up Apache modules.
+    for module in LP_APACHE_MODULES.split():
+        call('a2enmod', module)
 
     # Set up container hosts file.
     lines = [get_file_header()]

=== modified file 'lpsetup/subcommands/initrepo.py'
--- lpsetup/subcommands/initrepo.py	2012-07-05 18:38:51 +0000
+++ lpsetup/subcommands/initrepo.py	2012-07-10 16:18:18 +0000
@@ -20,7 +20,6 @@
     LP_BRANCH_NAME,
     LP_BZR_LOCATIONS,
     LP_CHECKOUT_NAME,
-    LP_SSH_REPO,
     )
 from lpsetup.utils import (
     call,
@@ -85,7 +84,7 @@
     def add_arguments(self, parser):
         super(SubCommand, self).add_arguments(parser)
         parser.add_argument(
-            '--source', default=LP_SSH_REPO,
+            '--source', default=None,
             help='Location from which to retrieve Launchpad source.  Default '
                 'is lp:launchpad (if --use-http is false), otherwise '
                 'http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel.')

=== added directory 'lpsetup/tests/integration'
=== added file 'lpsetup/tests/integration/non-lxc.py'
--- lpsetup/tests/integration/non-lxc.py	1970-01-01 00:00:00 +0000
+++ lpsetup/tests/integration/non-lxc.py	2012-07-10 16:18:18 +0000
@@ -0,0 +1,105 @@
+#!/usr/bin/env python
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""A simple, end-to-end integration test."""
+
+import os
+import sys
+import time
+
+from subprocess import (
+    check_call,
+    Popen,
+    PIPE,
+    )
+
+
+def banner(s):
+    width = 70
+    print '*' * width
+    print '* ' + s.ljust(width - 4) + ' *'
+    print '*' * width
+
+
+def on_remote(args):
+    if type(args) == str:
+        args = args.split()
+
+    check_call('juju ssh 1 -e lpsetup-testing'.split() + list(args))
+
+
+def check_environment():
+    """Be sure the test environment doesn't exist."""
+    banner('Checking test environment.')
+    # We want to be really sure we do not clobber an already-existing juju
+    # environment.  Therefore we make sure one doesn't exist before
+    # bootstrapping.
+    code = os.system('juju status -e lpsetup-testing')
+    if code == 0:
+        # The "juju status" should have failed.
+        raise RuntimeError('A juju environment unexpectedly exists.')
+
+
+def set_up():
+    """Set up a juju-managed instance to run the tests on."""
+    banner('Setting up the test environment.')
+    check_call('juju bootstrap -e lpsetup-testing'.split())
+    # XXX The "ubuntu" charm is broken, so it has to be loaded from the local
+    # repository.  Get it from lp:~charmers/charms/precise/ubuntu/trunk.
+    check_call('juju deploy local:ubuntu --repository ~/juju-charms'
+        ' -e lpsetup-testing --constraints instance-type=m1.large'.split())
+
+    start_time = time.time()
+    while time.time() - start_time < 600:
+        process = Popen('juju status -e lpsetup-testing'.split(), stdout=PIPE)
+        stdout, stderr = process.communicate()
+        if 'instance-state: running' in stdout:
+            break
+    else:
+        raise RuntimeError('starting the instance took too long')
+
+    # Even though the instance-state is "running", it's still not ready, so
+    # wait a little while before continuing.
+    time.sleep(30)
+
+    on_remote('sudo apt-add-repository --yes ppa:yellow/ppa')
+    on_remote('sudo apt-get update')
+    on_remote('sudo apt-get install python-shelltoolbox')
+    check_call('juju scp -e lpsetup-testing -r . 1:lpsetup'.split())
+
+
+def do_test():
+    """Run an end-to-end integration tests of the non-LXC lpsetup story."""
+    banner('Running (non-LXC) integration test.')
+    # Since the most common scenario is to have bzr whoami setup, we do that
+    # instead of providing the arguments directly to lpsetup.
+    on_remote(('bzr', 'whoami', '"Not A Real Person <[e-mail]>"'))
+    on_remote('cd lpsetup; ./lp-setup init-host'.split())
+
+
+def tear_down():
+    banner('Cleaning up.')
+    code = os.system('juju destroy-environment -e lpsetup-testing')
+    if code != 0:
+        raise RuntimeError('Destroying the test juju environment failed.')
+
+
+def main():
+    check_environment()
+    try:
+        set_up()
+        try:
+            do_test()
+        except:
+            banner('Test failed.  Sorry.')
+            return 1
+        else:
+            banner('Test succeeded.')
+            return 0
+    finally:
+        tear_down()
+
+
+if __name__ == '__main__':
+    sys.exit(main())