Merge lp:~1chb1n/charms/trusty/swift-storage/amulet-update-1508 into lp:~openstack-charmers-archive/charms/trusty/swift-storage/next

[Ryan Beisner]

Update amulet tests, enable T-K, V-K, prep for T-L and W-L.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8487 swift-storage-next for 1chb1n mp268788
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8487/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #7878 swift-storage-next for 1chb1n mp268788
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7878/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5947 swift-storage-next for 1chb1n mp268788
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5947/

[lp:~1chb1n/charms/trusty/swift-storage/amulet-update-1508]

81. By Ryan Beisner on 2015-08-22

update makefile, correct import order

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8544 swift-storage-next for 1chb1n mp268788
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8544/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #7932 swift-storage-next for 1chb1n mp268788
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7932/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5956 swift-storage-next for 1chb1n mp268788
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/5956/

[Adam Collard (adam-collard)]

Broadly looks good, thanks for the work to freshen this up! I have a few comments inline.

review: Needs Information

[Ryan Beisner (1chb1n)]

Thank you for the review, replies in line. 1 modification will be pushed momentarily re: dependency.

[lp:~1chb1n/charms/trusty/swift-storage/amulet-update-1508]

82. By Ryan Beisner on 2015-08-24

adjust tests.yaml for superfluous python-amulet

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #8649 swift-storage-next for 1chb1n mp268788
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/8649/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #7986 swift-storage-next for 1chb1n mp268788
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/7986/

[Adam Collard (adam-collard)]

Looks good, thanks for explaining.

review: Approve

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6014 swift-storage-next for 1chb1n mp268788
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/6014/

[Corey Bryant (corey.bryant)]

review: Approve

=== modified file 'Makefile'
--- Makefile	2015-08-11 08:17:24 +0000
+++ Makefile	2015-08-24 13:33:42 +0000
@@ -2,20 +2,17 @@
 PYTHON := /usr/bin/env python
 
 lint:
-	@flake8 lib
-	@flake8 hooks
-	@flake8 unit_tests tests
+	@flake8 --exclude hooks/charmhelpers,tests/charmhelpers \
+        actions lib hooks unit_tests tests
 	@charm proof
 
-unit_test:
+test:
+	@# Bundletester expects unit tests here.
 	@echo Starting unit tests...
 	@$(PYTHON) /usr/bin/nosetests -v --nologcapture --with-coverage unit_tests
 
-test:
+functional_test:
 	@echo Starting Amulet tests...
-	# coreycb note: The -v should only be temporary until Amulet sends
-	# raise_status() messages to stderr:
-	#   https://bugs.launchpad.net/amulet/+bug/1320357
 	@juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700
 
 bin/charm_helpers_sync.py:
@@ -31,4 +28,4 @@
 	bzr push lp:charms/swift-storage
 	bzr push lp:charms/trusty/swift-storage
 
-.PHONY: lint unit_test test sync publish
+.PHONY: lint test functional_test sync publish

=== modified file 'metadata.yaml'
--- metadata.yaml	2014-10-30 05:52:15 +0000
+++ metadata.yaml	2015-08-24 13:33:42 +0000
@@ -3,7 +3,8 @@
 summary: "Swift storage node"
 description: |
   Contains swift-object, swift-container and swift-account servers.
-categories:
+tags:
+    - openstack
     - file-servers
 provides:
   nrpe-external-master:

=== modified file 'tests/00-setup'
--- tests/00-setup	2015-08-11 13:58:24 +0000
+++ tests/00-setup	2015-08-24 13:33:42 +0000
@@ -4,9 +4,13 @@
 
 sudo add-apt-repository --yes ppa:juju/stable
 sudo apt-get update --yes
-sudo apt-get install --yes python-amulet \
-                           python-swiftclient \
+sudo apt-get install --yes amulet \
+                           python-cinderclient \
+                           python-distro-info \
                            python-glanceclient \
                            python-heatclient \
                            python-keystoneclient \
-                           python-novaclient
+                           python-neutronclient \
+                           python-novaclient \
+                           python-pika \
+                           python-swiftclient

=== modified file 'tests/010-basic-precise-essex' (properties changed: +x to -x)
=== modified file 'tests/017-basic-trusty-kilo' (properties changed: -x to +x)
=== modified file 'tests/019-basic-vivid-kilo' (properties changed: -x to +x)
=== added file 'tests/020-basic-trusty-liberty'
--- tests/020-basic-trusty-liberty	1970-01-01 00:00:00 +0000
+++ tests/020-basic-trusty-liberty	2015-08-24 13:33:42 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic swift-storage deployment on trusty-liberty."""
+
+from basic_deployment import SwiftStorageBasicDeployment
+
+if __name__ == '__main__':
+    deployment = SwiftStorageBasicDeployment(series='trusty',
+                                        openstack='cloud:trusty-liberty',
+                                        source='cloud:trusty-updates/liberty')
+    deployment.run_tests()

=== added file 'tests/021-basic-wily-liberty'
--- tests/021-basic-wily-liberty	1970-01-01 00:00:00 +0000
+++ tests/021-basic-wily-liberty	2015-08-24 13:33:42 +0000
@@ -0,0 +1,9 @@
+#!/usr/bin/python
+
+"""Amulet tests on a basic swift-storage deployment on wily-liberty."""
+
+from basic_deployment import SwiftStorageBasicDeployment
+
+if __name__ == '__main__':
+    deployment = SwiftStorageBasicDeployment(series='wily')
+    deployment.run_tests()

=== modified file 'tests/README'
--- tests/README	2014-09-29 21:14:05 +0000
+++ tests/README	2015-08-24 13:33:42 +0000
@@ -1,6 +1,16 @@
 This directory provides Amulet tests that focus on verification of swift-storage
 deployments.
 
+test_* methods are called in lexical sort order, although each individual test
+should be idempotent, and expected to pass regardless of run order.
+
+Test name convention to ensure desired test order:
+    1xx service and endpoint checks
+    2xx relation checks
+    3xx config checks
+    4xx functional checks
+    9xx restarts and other final checks
+
 In order to run tests, you'll need charm-tools installed (in addition to
 juju, of course):
     sudo add-apt-repository ppa:juju/stable

=== modified file 'tests/basic_deployment.py'
--- tests/basic_deployment.py	2015-08-17 13:44:42 +0000
+++ tests/basic_deployment.py	2015-08-24 13:33:42 +0000
@@ -1,3 +1,5 @@
+import time
+
 import amulet
 import swiftclient
 
@@ -78,6 +80,14 @@
         self.swift_proxy_sentry = self.d.sentry.unit['swift-proxy/0']
         self.swift_storage_sentry = self.d.sentry.unit['swift-storage/0']
 
+        u.log.debug('openstack release val: {}'.format(
+            self._get_openstack_release()))
+        u.log.debug('openstack release str: {}'.format(
+            self._get_openstack_release_string()))
+
+        # Let things settle a bit before moving forward
+        time.sleep(30)
+
         # Authenticate admin with keystone
         self.keystone = u.authenticate_keystone_admin(self.keystone_sentry,
                                                       user='admin',
@@ -119,9 +129,10 @@
                                          password='password',
                                          tenant=self.demo_tenant)
 
-    def test_services(self):
+    def test_100_services(self):
         """Verify the expected services are running on the corresponding
            service units."""
+        u.log.debug('Checking system services...')
         swift_storage_services = ['swift-account',
                                   'swift-account-auditor',
                                   'swift-account-reaper',
@@ -139,8 +150,8 @@
         service_names = {
             self.mysql_sentry: ['mysql'],
             self.keystone_sentry: ['keystone'],
-            self.glance_sentry: [
-                'glance-registry', 'glance-api'],
+            self.glance_sentry: ['glance-registry',
+                                 'glance-api'],
             self.swift_proxy_sentry: ['swift-proxy'],
             self.swift_storage_sentry: swift_storage_services
         }
@@ -149,8 +160,9 @@
         if ret:
             amulet.raise_status(amulet.FAIL, msg=ret)
 
-    def test_users(self):
+    def test_102_users(self):
         """Verify all existing roles."""
+        u.log.debug('Checking keystone users...')
         user1 = {'name': 'demoUser',
                  'enabled': True,
                  'tenantId': u.not_null,
@@ -178,19 +190,15 @@
         if ret:
             amulet.raise_status(amulet.FAIL, msg=ret)
 
-    def test_service_catalog(self):
+    def test_104_keystone_service_catalog(self):
         """Verify that the service catalog endpoint data is valid."""
-        endpoint_vol = {'adminURL': u.valid_url,
-                        'region': 'RegionOne',
-                        'publicURL': u.valid_url,
-                        'internalURL': u.valid_url}
+        u.log.debug('Checking keystone service catalog...')
         endpoint_id = {'adminURL': u.valid_url,
                        'region': 'RegionOne',
                        'publicURL': u.valid_url,
-                       'internalURL': u.valid_url}
-        if self._get_openstack_release() >= self.precise_folsom:
-            endpoint_vol['id'] = u.not_null
-            endpoint_id['id'] = u.not_null
+                       'internalURL': u.valid_url,
+                       'id': u.not_null}
+
         expected = {'image': [endpoint_id], 'object-store': [endpoint_id],
                     'identity': [endpoint_id]}
         actual = self.keystone_demo.service_catalog.get_endpoints()
@@ -199,8 +207,9 @@
         if ret:
             amulet.raise_status(amulet.FAIL, msg=ret)
 
-    def test_openstack_object_store_endpoint(self):
+    def test_106_swift_object_store_endpoint(self):
         """Verify the swift object-store endpoint data."""
+        u.log.debug('Checking keystone endpoint for swift object store...')
         endpoints = self.keystone.endpoints.list()
         admin_port = internal_port = public_port = '8080'
         expected = {'id': u.not_null,
@@ -216,9 +225,10 @@
             message = 'object-store endpoint: {}'.format(ret)
             amulet.raise_status(amulet.FAIL, msg=message)
 
-    def test_swift_storage_swift_storage_relation(self):
+    def test_200_swift_storage_swift_storage_relation(self):
         """Verify the swift-storage to swift-proxy swift-storage relation
            data."""
+        u.log.debug('Checking swift:swift-proxy swift-storage relation...')
         unit = self.swift_storage_sentry
         relation = ['swift-storage', 'swift-proxy:swift-storage']
         expected = {
@@ -235,9 +245,10 @@
             message = u.relation_error('swift-storage swift-storage', ret)
             amulet.raise_status(amulet.FAIL, msg=message)
 
-    def test_swift_proxy_swift_storage_relation(self):
+    def test_202_swift_proxy_swift_storage_relation(self):
         """Verify the swift-proxy to swift-storage swift-storage relation
            data."""
+        u.log.debug('Checking swift-proxy:swift swift-storage relation...')
         unit = self.swift_proxy_sentry
         relation = ['swift-storage', 'swift-storage:swift-storage']
         expected = {
@@ -252,56 +263,10 @@
             message = u.relation_error('swift-proxy swift-storage', ret)
             amulet.raise_status(amulet.FAIL, msg=message)
 
-    def test_z_restart_on_config_change(self):
-        """Verify that the specified services are restarted when the config
-           is changed.
-
-           Note(coreycb): The method name with the _z_ is a little odd
-           but it forces the test to run last.  It just makes things
-           easier because restarting services requires re-authorization.
-           """
-        # NOTE(coreycb): Skipping failing test on until resolved. This test
-        #                fails because the config file's last mod time is
-        #                slightly after the process' last mod time.
-        if self._get_openstack_release() >= self.precise_essex:
-            u.log.error("Skipping failing test until resolved")
-            return
-
-        services = {'swift-account-server': 'account-server.conf',
-                    'swift-account-auditor': 'account-server.conf',
-                    'swift-account-reaper': 'account-server.conf',
-                    'swift-account-replicator': 'account-server.conf',
-                    'swift-container-server': 'container-server.conf',
-                    'swift-container-auditor': 'container-server.conf',
-                    'swift-container-replicator': 'container-server.conf',
-                    'swift-container-updater': 'container-server.conf',
-                    'swift-object-server': 'object-server.conf',
-                    'swift-object-auditor': 'object-server.conf',
-                    'swift-object-replicator': 'object-server.conf',
-                    'swift-object-updater': 'object-server.conf'}
-        if self._get_openstack_release() >= self.precise_icehouse:
-            services['swift-container-sync'] = 'container-server.conf'
-
-        self.d.configure('swift-storage',
-                         {'object-server-threads-per-disk': '2'})
-
-        time = 20
-        for s, conf in services.iteritems():
-            config = '/etc/swift/{}'.format(conf)
-            if not u.service_restarted(self.swift_storage_sentry, s, config,
-                                       pgrep_full=True, sleep_time=time):
-                self.d.configure('swift-storage',
-                                 {'object-server-threads-per-disk': '4'})
-                msg = "service {} didn't restart after config change".format(s)
-                amulet.raise_status(amulet.FAIL, msg=msg)
-            time = 0
-
-        self.d.configure('swift-storage',
-                         {'object-server-threads-per-disk': '4'})
-
-    def test_swift_config(self):
+    def test_300_swift_config(self):
         """Verify the data in the swift-hash section of the swift config
            file."""
+        u.log.debug('Checking swift config...')
         unit = self.swift_storage_sentry
         conf = '/etc/swift/swift.conf'
         swift_proxy_relation = self.swift_proxy_sentry.relation(
@@ -315,8 +280,9 @@
             message = "swift config error: {}".format(ret)
             amulet.raise_status(amulet.FAIL, msg=message)
 
-    def test_account_server_config(self):
+    def test_302_account_server_config(self):
         """Verify the data in the account server config file."""
+        u.log.debug('Checking swift account-server config...')
         unit = self.swift_storage_sentry
         conf = '/etc/swift/account-server.conf'
         expected = {
@@ -343,8 +309,9 @@
                 message = "account server config error: {}".format(ret)
                 amulet.raise_status(amulet.FAIL, msg=message)
 
-    def test_container_server_config(self):
+    def test_304_container_server_config(self):
         """Verify the data in the container server config file."""
+        u.log.debug('Checking swift container-server config...')
         unit = self.swift_storage_sentry
         conf = '/etc/swift/container-server.conf'
         expected = {
@@ -372,8 +339,9 @@
                 message = "container server config error: {}".format(ret)
                 amulet.raise_status(amulet.FAIL, msg=message)
 
-    def test_object_server_config(self):
+    def test_306_object_server_config(self):
         """Verify the data in the object server config file."""
+        u.log.debug('Checking swift object-server config...')
         unit = self.swift_storage_sentry
         conf = '/etc/swift/object-server.conf'
         expected = {
@@ -404,35 +372,17 @@
                 message = "object server config error: {}".format(ret)
                 amulet.raise_status(amulet.FAIL, msg=message)
 
-    def test_image_create(self):
+    def test_400_swift_backed_image_create(self):
         """Create an instance in glance, which is backed by swift, and validate
-           that some of the metadata for the image match in glance and swift.
-        """
-        # NOTE(coreycb): Skipping failing test on folsom until resolved. On
-        #                folsom only, uploading an image to glance gets 400 Bad
-        #                Request - Error uploading image: (error): [Errno 111]
-        #                ECONNREFUSED (HTTP 400)
-        if self._get_openstack_release() == self.precise_folsom:
-            u.log.error("Skipping failing test until resolved")
-            return
-
-        # Create glance image
-        image = u.create_cirros_image(self.glance, "cirros-image")
-        if not image:
-            amulet.raise_status(amulet.FAIL, msg="Image create failed")
-
-        # Validate that cirros image exists in glance and get its checksum/size
-        images = list(self.glance.images.list())
-        if len(images) != 1:
-            msg = "Expected 1 glance image, found {}".format(len(images))
-            amulet.raise_status(amulet.FAIL, msg=msg)
-
-        if images[0].name != 'cirros-image':
-            message = "cirros image does not exist"
-            amulet.raise_status(amulet.FAIL, msg=message)
-
-        glance_image_md5 = image.checksum
-        glance_image_size = image.size
+        that some of the metadata for the image match in glance and swift."""
+        u.log.debug('Checking swift objects and containers with a '
+                    'swift-backed glance image...')
+
+        # Create swift-backed glance image
+        img_new = u.create_cirros_image(self.glance, "cirros-image-1")
+        img_id = img_new.id
+        img_md5 = img_new.checksum
+        img_size = img_new.size
 
         # Validate that swift object's checksum/size match that from glance
         headers, containers = self.swift.get_account()
@@ -451,18 +401,67 @@
         swift_object_size = objects[0].get('bytes')
         swift_object_md5 = objects[0].get('hash')
 
-        if glance_image_size != swift_object_size:
+        if img_size != swift_object_size:
             msg = "Glance image size {} != swift object size {}".format(
-                glance_image_size, swift_object_size)
+                img_size, swift_object_size)
             amulet.raise_status(amulet.FAIL, msg=msg)
 
-        if glance_image_md5 != swift_object_md5:
+        if img_md5 != swift_object_md5:
             msg = "Glance image hash {} != swift object hash {}".format(
-                glance_image_md5, swift_object_md5)
+                img_md5, swift_object_md5)
             amulet.raise_status(amulet.FAIL, msg=msg)
 
         # Cleanup
-        u.delete_image(self.glance, image)
+        u.delete_resource(self.glance.images, img_id, msg="glance image")
+        u.log.info('OK')
+
+    def test_900_restart_on_config_change(self):
+        """Verify that the specified services are restarted when the config
+           is changed."""
+        u.log.info('Checking that conf files and system services respond '
+                   'to a charm config change...')
+        sentry = self.swift_storage_sentry
+        juju_service = 'swift-storage'
+
+        # Expected default and alternate values
+        set_default = {'object-server-threads-per-disk': '4'}
+        set_alternate = {'object-server-threads-per-disk': '2'}
+
+        # Config file affected by juju set config change, and
+        # services which are expected to restart upon config change
+        services = {'swift-account-server': 'account-server.conf',
+                    'swift-account-auditor': 'account-server.conf',
+                    'swift-account-reaper': 'account-server.conf',
+                    'swift-account-replicator': 'account-server.conf',
+                    'swift-container-server': 'container-server.conf',
+                    'swift-container-auditor': 'container-server.conf',
+                    'swift-container-replicator': 'container-server.conf',
+                    'swift-container-updater': 'container-server.conf',
+                    'swift-object-server': 'object-server.conf',
+                    'swift-object-auditor': 'object-server.conf',
+                    'swift-object-replicator': 'object-server.conf',
+                    'swift-object-updater': 'object-server.conf',
+                    'swift-container-sync': 'container-server.conf'}
+
+        # Make config change, check for service restarts
+        u.log.debug('Making config change on {}...'.format(juju_service))
+        mtime = u.get_sentry_time(sentry)
+        self.d.configure(juju_service, set_alternate)
+
+        sleep_time = 40
+        for s, conf_file in services.iteritems():
+            u.log.debug("Checking that service restarted: {}".format(s))
+            conf_file_abs = '/etc/swift/{}'.format(conf_file)
+            if not u.validate_service_config_changed(sentry, mtime, s,
+                                                     conf_file_abs,
+                                                     sleep_time=sleep_time,
+                                                     pgrep_full=True):
+                self.d.configure(juju_service, set_default)
+                msg = "service {} didn't restart after config change".format(s)
+                amulet.raise_status(amulet.FAIL, msg=msg)
+            sleep_time = 0
+
+        self.d.configure(juju_service, set_default)
 
     def _assert_services(self, should_run):
         swift_storage_services = ['swift-account-auditor',
@@ -506,12 +505,8 @@
 
         self._assert_services(should_run=True)
 
-    def test_z_actions(self):
-        """Pause and then resume swift-storage.
-
-           Note(sparkiegeek): The method name with the _z_ is a little odd
-           but it forces the test to run last.  It just makes things
-           easier because restarting services requires re-authorization.
-        """
+    def test_910_pause_resume_actions(self):
+        """Pause and then resume swift-storage."""
+        u.log.debug('Checking pause/resume actions...')
         self._test_pause()
         self._test_resume()

=== added file 'tests/tests.yaml'
--- tests/tests.yaml	1970-01-01 00:00:00 +0000
+++ tests/tests.yaml	2015-08-24 13:33:42 +0000
@@ -0,0 +1,19 @@
+bootstrap: true
+reset: true
+virtualenv: true
+makefile:
+    - lint
+    - test
+sources:
+    - ppa:juju/stable
+packages:
+    - amulet
+    - python-cinderclient
+    - python-distro-info
+    - python-glanceclient
+    - python-heatclient
+    - python-keystoneclient
+    - python-neutronclient
+    - python-novaclient
+    - python-pika
+    - python-swiftclient