Merge lp:~cprov/uci-engine/test-image-store into lp:uci-engine

[Celso Providelo]

Adding an integration test for probing credential capabilities for IBS.

Adding an integration test for probing credential capabilities for IBS.

IBS requires credentials able to upload/register new images to the available Glance service. It requires pyOpenSSL dependencies (added to .deps in a separated branch).

This is a preparation step for splitting credentials for Airline & Testbed.

Also adds a minor fix to the test_runner integration test (needs exposed rabbit).

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:634
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/974/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/974/rebuild

review: Needs Fixing (continuous-integration)

[Celso Providelo (cprov)]

Only fails because the lack of PIP dependencies being added by https://code.launchpad.net/~cprov/uci-engine/pyopenssl-for-glanceclient

[Andy Doan (doanac)]

just a nitpick and a question

[Celso Providelo (cprov)]

Thanks for the review, comments replied inline.

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:635
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/991/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/991/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:636
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/993/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/993/rebuild

review: Needs Fixing (continuous-integration)

[Celso Providelo (cprov)]

Francis,

Can we have libffi-dev installed in s-jenkins env for uci-engine ? it's needed for installing pyOpenSSL inside our venv and probe glance capabilities in our integration-tests.

[Andy Doan (doanac)]

+1, thanks,

review: Approve

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:636
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/998/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/998/rebuild

review: Needs Fixing (continuous-integration)

[Francis Ginther (fginther)]

> Francis,
>
> Can we have libffi-dev installed in s-jenkins env for uci-engine ? it's needed
> for installing pyOpenSSL inside our venv and probe glance capabilities in our
> integration-tests.

I've updated the chroot and it's now running.

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:636
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/999/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/999/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:636
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/1002/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/1002/rebuild

review: Approve (continuous-integration)

[Ubuntu CI Bot (uci-bot)]

The attempt to merge lp:~cprov/uci-engine/test-image-store into lp:uci-engine failed. Below is the output from the failed tests.

dpkg is /usr/bin/dpkg
Required packages are missing. Please ensure that the missing packages are installed.
Run: sudo apt-get install build-essential python-pip python-virtualenv python-setuptools python-dev make python-apt-dev libffi-dev

[Francis Ginther (fginther)]

Dependency added to tarmac host. Lets try this again.

=== modified file 'ci-utils/ci_utils/image_store/__init__.py'
--- ci-utils/ci_utils/image_store/__init__.py	2014-07-01 20:49:09 +0000
+++ ci-utils/ci_utils/image_store/__init__.py	2014-07-02 04:26:47 +0000
@@ -58,16 +58,28 @@
                 password=config['auth_password'],
                 tenant_name=config['auth_tenant_name']
             )
-        except (KeyError, ksclient.exceptions.Unauthorized,
-                ksclient.exceptions.AuthorizationFailure):
+        except KeyError as exc:
+            raise ImageStoreException(
+                "Missing authentication info: {}".format(exc.message))
+        except (ksclient.exceptions.Unauthorized,
+                ksclient.exceptions.AuthorizationFailure) as exc:
             logging.exception('Unable to connect keystone')
-            raise self.ImageStoreException(
-                "Missing or invalid authentication info."
-            )
+            raise ImageStoreException(
+                "Unable to contact keystone in {} ({})".format(
+                    config['auth_url'], exc.message))
 
     def image_create(self, filename, image, is_public=False,
                      disk_format='qcow2', container_format='bare'):
         with open(image) as imgdata:
-            self.glance.images.create(
-                name=filename, is_public=is_public, disk_format=disk_format,
-                container_format=container_format, data=imgdata)
+            try:
+                image = self.glance.images.create(
+                    name=filename,
+                    is_public=is_public,
+                    disk_format=disk_format,
+                    container_format=container_format,
+                    data=imgdata)
+            except glanceclient.exc.HTTPException as exc:
+                raise ImageStoreException(
+                    "Unable to create an image in {} ({})".format(
+                        self.glance.endpoint, str(exc)))
+        return image

=== modified file 'image-builder/setup.py'
--- image-builder/setup.py	2014-06-18 04:04:04 +0000
+++ image-builder/setup.py	2014-07-02 04:26:47 +0000
@@ -18,6 +18,7 @@
 import sys
 
 requires = [
+    'pyOpenSSL==0.14',
     'PyYAML==3.10',
     'amqplib==1.0.0',
     'launchpadlib==1.10.2',

=== modified file 'setup.sh'
--- setup.sh	2014-06-24 09:18:12 +0000
+++ setup.sh	2014-07-02 04:26:47 +0000
@@ -20,7 +20,7 @@
 # For OSX, see http://kaspermunck.github.io/2014/03/fixing-clang-error/
 export ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future
 
-pkgs='build-essential python-pip python-virtualenv python-setuptools python-dev make python-apt-dev'
+pkgs='build-essential python-pip python-virtualenv python-setuptools python-dev make python-apt-dev libffi-dev'
 if type dpkg 2>/dev/null && ! dpkg -s $pkgs 2>/dev/null >/dev/null ; then
     echo "Required packages are missing. Please ensure that the missing packages are installed."
     echo "Run: sudo apt-get install $pkgs"

=== added file 'tests/test_image_store.py'
--- tests/test_image_store.py	1970-01-01 00:00:00 +0000
+++ tests/test_image_store.py	2014-07-02 04:26:47 +0000
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+# Ubuntu Continuous Integration Engine
+# Copyright 2014 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""Check ImageStore features in an existing deployment."""
+import os
+import tempfile
+import unittest
+
+from ci_utils import unit_config
+from ci_utils.image_store import ImageStore
+
+
+class ImageStoreTest(unittest.TestCase):
+
+    def test_image_creation(self):
+        # `ImageStore.image_create` is capable of uploading an arbitrary
+        # image to the available Glance server. This is how the Image-Builder
+        # publishes a testing image to the Test-Runner.
+        _fd, image_path = tempfile.mkstemp()
+        self.addCleanup(os.remove, image_path)
+        open(image_path, 'w').write('I can haz image ...')
+        config = unit_config.get_auth_config()
+
+        glance = ImageStore(config)
+        image = glance.image_create('ci-airline-testing.img', image_path)
+        self.assertEquals('active', image.status)
+
+
+if __name__ == "__main__":
+    unittest.main()

=== modified file 'tests/test_test_runner.py'
--- tests/test_test_runner.py	2014-06-25 09:48:37 +0000
+++ tests/test_test_runner.py	2014-07-02 04:26:47 +0000
@@ -61,6 +61,9 @@
         units =  status['services']['ci-airline-rabbit']['units']
         self.unit = 'ci-airline-rabbit/0'
         self.public_ip = units[self.unit]['public-address']
+        # Expose rabbit service for this test.
+        deployers.juju_expose(RABBIT_SERVICE)
+        self.addCleanup(deployers.juju_expose, RABBIT_SERVICE, False)
 
         class AmqpConfig(object):