Merge lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support into lp:~chad.smith/charms/precise/block-storage-broker/trunk

[Chad Smith]

Add EC2 support to block-storage-broker.

Also consolidate tests and functions into a single StorageServiceUtil class that takes a provider "nova" or "ec2" to enable different volume creation, labeling and attachment actions.

The basic changes in the consolidated StorageServiceUtil class are the introduction of provider-specific (_ec2 vs. _nova) methods for the commands or library specifics that need to be called to actually describe_volumes, describe_instances, create, attach and detach volumes using nova or ec2 tooling. The common methods contain the same overall logic that was in the original nova_util to discover exist, avoid volume duplication and detach only that the actual work being done is found in the _ec2/_nova methods. This branch introduces all the new _ec2_* methods which use euca python libraries to walk over volume and instance information. These _ec2_* methods hadn't been previously tested as most of out testing was originally using nova on canonistack. As you can see by the postgres-storage-bundle.cfg file below, this branch depends on a small extension to the storage subordinate charm at for EC2 metadata support present at lp:~chad.smith/charms/precise/storage/storage-ec2-support

Sample deployment bundle file:
csmith@fringe:~/src/charms/landscape-charm/config$ cat postgres-storage-ec2.cfg common:
    services:
        postgresql:
            branch: lp:~chad.smith/charms/precise/postgresql/postgresql-using-storage-subordinate
            constraints: mem=2048
            options:
                extra-packages: python-apt postgresql-contrib postgresql-9.1-debversion
                max_connections: 500
        storage:
            branch: lp:~chad.smith/charms/precise/storage/storage-ec2-support
            options:
                provider: block-storage-broker
                volume_size: 9
        block-storage-broker-ec2:
            branch: lp:~chad.smith/charms/precise/block-storage-broker-ec2/trunk
            options:
                provider: ec2
                key: YOUR_EC2_ACCESS_KEY
                endpoint: YOUR_EC2_URL
                secret: YOUR_EC2_SECRET_KEY

doit:
    inherits: common
    series: precise
    relations:
        - [postgresql, storage]
        - [storage, block-storage-broker-ec2]

$ juju-deployer -c postgresql-storage-ec2.cfg doit

[David Britton (dpb)]

[0] Test failures for me. I suspect some kind of isolation. Playing with the -j# on trial varies the test that fails, sometimes making it pass. csmith notified in IRC

[1] Can we remove CHARM_DIR=`pwd` thing from the makefile?

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support]

75. By Chad Smith on 2014-03-13

update make test to run trial -j2 to exacerbate threaded workers unit test issues

76. By Chad Smith on 2014-03-13

our import of StorageServiceUtil should be global instead of performed withing individual hooks

77. By Chad Smith on 2014-03-13

when mocking avoid mocker.replace of global imports using simple strings like 'util.StorageServiceUtil.attach_volume'. Instead use mocker.patch of our local imported StorageServiceUtil object and patch the attach_volume() method. This local patching seems to avoid collisions multiple processes run unit tests simultaneously

78. By Chad Smith on 2014-03-13

remove CHARM_DIR environment variable declaration from trial call in Makfile. Didn't realize I already worked this into the unit tests

79. By Chad Smith on 2014-03-14

- use of awk in subprocess command parsing
- add additional validation of euca-create-volume output to ensure our response type is a VOLUME response
- don't use global import self.mocker.replace("module.path.name") instead patch our local imported objects for load_environment to avoid unit test collisions with multiple jobs

80. By Chad Smith on 2014-03-19

Move imports for euca2ools commands into class provider initialization. update unit tests to avoid global mocker of euca2ools imports. this collides when unit tests are run in parallel

[David Britton (dpb)]

[2] per discussions in IRC, trusty gets this failure. follow-on work in a separate branch, I guess. :(

test_util.TestEC2Util.test_wb_ec2_describe_volumes_without_attached_instances
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "/usr/lib/python2.7/dist-packages/mocker.py", line 146, in test_method_wrapper
    result = test_method()
  File "/home/dpb/src/canonical/juju/charms/blockstoragebroker/bsb-ec2-support/hooks/test_util.py", line 1332, in test_wb_ec2_describe_volumes_with_attac
hed_instances
    self.storage._ec2_describe_volumes(), expected)
  File "/home/dpb/src/canonical/juju/charms/blockstoragebroker/bsb-ec2-support/hooks/util.py", line 250, in _ec2_describe_volumes
    command = describevolumes.DescribeVolumes()
  File "/usr/lib/python2.7/dist-packages/euca2ools/commands/euca/__init__.py", line 213, in __init__
    AWSQueryRequest.__init__(self, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requestbuilder/request.py", line 88, in __init__
    BaseCommand.__init__(self, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requestbuilder/command.py", line 98, in __init__
    self._post_init()
  File "/usr/lib/python2.7/dist-packages/requestbuilder/request.py", line 94, in _post_init
    BaseCommand._post_init(self)
  File "/usr/lib/python2.7/dist-packages/requestbuilder/command.py", line 107, in _post_init
    self.configure()
  File "/usr/lib/python2.7/dist-packages/requestbuilder/request.py", line 107, in configure
    self.service.configure()
  File "/usr/lib/python2.7/dist-packages/euca2ools/commands/euca/__init__.py", line 199, in configure
    self.validate_config()
  File "/usr/lib/python2.7/dist-packages/requestbuilder/service.py", line 154, in validate_config
    raise ServiceInitError(msg)
requestbuilder.exceptions.ServiceInitError: No ec2 endpoint to connect to was given. ec2 endpoints may be specified in a config file with "ec2-url".

test_util.TestEC2Util.test_wb_ec2_describe_volumes_with_attached_instances
-------------------------------------------------------------------------------
Ran 77 tests in 0.205s

FAILED (errors=4, successes=73)
make: *** [test] Error 1

[David Britton (dpb)]

[3]: in StrogaeServiceUtil.__init__, the environment needs to be loaded
before:

    self.ec2_instances_command = getinstances.DescribeInstances()
    self.ec2_volumes_command = getvolumes.DescribeVolumes()

If not, it leads to:

2014-03-19 18:36:43 INFO juju juju-log.go:66 block-storage-broker-ec2/0 block-storage:2: ERROR: Couldn't contact EC2 using euca-describe-volumes

--
David Britton <email address hidden>

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support]

81. By Chad Smith on 2014-03-19

make test should only attempt trial against py files, not pyc

82. By Chad Smith on 2014-03-19

don't make direct calls in __init__ to DescribeVolumes() and DescribeInstances() this needs to take place after StorageServiceUtil.load_environment(). Update unit tests with MockEucaInstance to better mock results from underlying euca2ools modules

[Fernando Correa Neto (fcorrea)]

Hey Chad, just a few points.

After looking the entire branch, I guess maybe we could have proposed this in a different way.

1 - Single branch that introduce the EC2 changes into the old util_nova.py so that we could see the diff better. Or maybe add a separate util_ec2.py which would be even cleaner. Same for tests
2 - Single branch that merged the two into a new module
3 - Single branch that introduced the mappings etc.

But it's only possible to see it when we reach this point :)

The points:

[1]
+ self.environment_map = ENVIRONMENT_MAP[provider]
+ self.commands = PROVIDER_COMMANDS[provider]
+ self.required_config_options = REQUIRED_CONFIG_OPTIONS[provider]

I'm wondering if we could make it a little bit more safer on grabbing values from those mappings.
Maybe we could have a helper function like get_value_from_mapping(mapping, value) that would return the value if it succeeds or log something useful so the user can see it in the logs. Maybe "ERROR: ce2 provider is not supported".
I'm not sure if we are doing it someplace else, if we are, disregard.

[2]
+ def validate_credentials(self):
+ """Attempt to contact nova volume service or exit(1)"""

Since we are accessing self.commands and it could be either nova or ec2, I think it should also mention ec2, or even not mention either, since it's generic.

[3]
+ def get_volume_id(self, volume_designation=None, instance_id=None):
+ """Return the ec2 volume id associated with this unit
+
+ Optionally, C{volume_designation} can be either a volume-id or
+ volume-display-name and the matching C{volume-id} will be returned.
+ If no matching volume is return C{None}.
+ """

Same as [2]

[4]
+ def _ec2_describe_volumes(self, volume_id=None):
+ import euca2ools.commands.euca.describevolumes as describevolumes

Missing docstring for that one

[5]
I see we have PROVIDER_COMMANDS and we also have provider specific methods. I guess that we could maybe lose the mapping and align with what you've done for the other methods as well. Meaning, add _ec2_validate, _ec2_detattach, _nova_validate, _nova_detattach methods. They would be testable like the others in the end and we'd be green on the coverage side of things. Feel free to refactor in a separate branch.

I'm going for a real test now and will check it later.

Thanks!

[Fernando Correa Neto (fcorrea)]

Hey Chad, got a failure while deploying:

fcorrea@beast:~/src/landscape/landscape-charm/config: juju-deployer -c postgres-storage-ec2.cfg doit -e fcorrea-amazon
2014-03-20 17:38:53 Starting deployment of doit
2014-03-20 17:39:13 Deploying services...
2014-03-20 17:39:16 Deploying service block-storage-broker-ec2 using local:precise/block-storage-broker
2014-03-20 17:39:32 Deploying service postgresql using local:precise/postgresql
2014-03-20 17:39:42 Deploying service storage using local:precise/storage
2014-03-20 17:40:06 Config specifies num units for subordinate: storage
2014-03-20 17:43:33 Adding relations...
2014-03-20 17:43:34 Adding relation postgresql <-> storage
Traceback (most recent call last):
  File "/usr/bin/juju-deployer", line 9, in <module>
    load_entry_point('juju-deployer==0.3.4', 'console_scripts', 'juju-deployer')()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 127, in main
    run()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 225, in run
    importer.Importer(env, deployment, options).run()
  File "/usr/lib/python2.7/dist-packages/deployer/action/importer.py", line 198, in run
    rels_created = self.add_relations()
  File "/usr/lib/python2.7/dist-packages/deployer/action/importer.py", line 128, in add_relations
    self.env.add_relation(end_a, end_b)
  File "/usr/lib/python2.7/dist-packages/deployer/env/go.py", line 57, in add_relation
    return self.client.add_relation(endpoint_a, endpoint_b)
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 561, in add_relation
    'Endpoints': [endpoint_a, endpoint_b]
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 152, in _rpc
    raise EnvError(result)
jujuclient.EnvError: <Env Error - Details:
 { u'Error': u'no relations found', u'RequestId': 1, u'Response': { }}
 >

I've tested juju-deployer by deploying the landscape-charm and it was fine.

This is what I have in my postgres-storage-ec2.cfg

common:
    services:
        postgresql:
            branch: lp:~chad.smith/charms/precise/postgresql/postgresql-using-storage-subordinate
            constraints: mem=2048
            options:
                extra-packages: python-apt postgresql-contrib postgresql-9.1-debversion
                max_connections: 500
        storage:
            branch: lp:~chad.smith/charms/precise/storage/storage-ec2-support
            options:
                provider: block-storage-broker
                volume_size: 9
        block-storage-broker-ec2:
            branch: lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support
            options:
                provider: ec2
                key: <the same key I use to run euca-commands>
                endpoint: https://ec2.us-east-1.amazonaws.com
                secret: <the same secret I use to run euca-commands>

doit:
    inherits: common
    series: precise
    relations:
        - [postgresql, storage]
        - [storage, block-storage-broker-ec2]

Please, lemme know if I can provide you with any extra info.

[Chad Smith (chad.smith)]

Thanks for the reviews guys:

dpb[0 & 1]:
- test isolation and makefile fixes have been made to ensure -j3 is run during unit tests. had to drop global mocker.replace("path.to.module") and instead mock.patch local methods/functions to avoid collisions with running in multiple threads

[2] trusty failures:
- they are resolved in a followon MP: at https://code.launchpad.net/~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support/+merge/211963

dpb[3] in StrogaeServiceUtil.__init__, the environment needs to be loaded
before:

    self.ec2_instances_command = getinstances.DescribeInstances()
    self.ec2_volumes_command = getvolumes.DescribeVolumes()

  - Resolved in this branch to actually call DescribeVolumes() and DescribeInstances() during _ec2_describe_volumes and _ec2_describe_instances respectively so that we ensure load_environment is called 1st. importing these classes directly from euca2ools is a no-no and broke us on trusty. The trusty support branch mentioned about drops these imports completely in favor of parsing the euca commandline tool output.

[Chad Smith (chad.smith)]

> Hey Chad, got a failure while deploying:
>
> fcorrea@beast:~/src/landscape/landscape-charm/config: juju-deployer -c
> postgres-storage-ec2.cfg doit -e fcorrea-amazon
> 2014-03-20 17:38:53 Starting deployment of doit

That seems like a juju service issue more than something related to the branches you are deploying but two thoughts:
- rm -rf your precise subdir under whereever your are running juju-deployer (otherwise it uses a local cache instead of checking out the branches your deployer file mentioned)
- the storage charm branch has already been merged into dpb's trunk so you could use
> branch: lp:~chad.smith/charms/precise/storage/storage-ec2-support

              branch: lp:~davidpbritton/charms/precise/storage/trunk

But this 2nd case shouldn't be functionally different currently than what your original deployer bundle represents

[Fernando Correa Neto (fcorrea)]

Hey Chad

> > Hey Chad, got a failure while deploying:
> >
> > fcorrea@beast:~/src/landscape/landscape-charm/config: juju-deployer -c
> > postgres-storage-ec2.cfg doit -e fcorrea-amazon
> > 2014-03-20 17:38:53 Starting deployment of doit
>
>
>
> That seems like a juju service issue more than something related to the
> branches you are deploying but two thoughts:
> - rm -rf your precise subdir under whereever your are running juju-deployer
> (otherwise it uses a local cache instead of checking out the branches your
> deployer file mentioned)

Yep. That was it. Since I was at the same path as you, meaning, landscape-charm/config, it had a precise cache in there that was causing the issue.
All good now. I'll just wait a bit more on the other comments I've made.

Thank you!

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support]

83. By Chad Smith on 2014-03-20

add error checking on configured provider type. complete w/ hook failure

84. By Chad Smith on 2014-03-20

resolve fcorrea review comments 1-4. Add validation of charm config.provider

85. By Chad Smith on 2014-03-20

lint

[Chad Smith (chad.smith)]

Thanks for digging through this Fernando
[0] I guess maybe we could have proposed this in a different way
>
> 1 - Single branch that introduce the EC2 changes into the old util_nova.py so
> that we could see the diff better.

Agreed, this got way too large for a simple review. I should have decomposed this branch into at least 2 parts when it took folks greater than a week to grab the review in the 1st place and a couple weeks to get through it. Those were indicators that maybe this could be made easier.

>
> [1]
> + self.environment_map = ENVIRONMENT_MAP[provider]
> + self.commands = PROVIDER_COMMANDS[provider]
> + self.required_config_options = REQUIRED_CONFIG_OPTIONS[provider]

Added a validation in class __init__ and log a hook error exit(1) if provider is unsupported. Thanks
> [2]
> + def validate_credentials(self):
> + """Attempt to contact nova volume service or exit(1)"""
>
> Since we are accessing self.commands and it could be either nova or ec2, I
> think it should also mention ec2, or even not mention either, since it's
> generic.
>
> [3]
> + def get_volume_id(self, volume_designation=None, instance_id=None):
> + """Return the ec2 volume id associated with this unit

[2-3] Done and Done
> +
> [4]
> + def _ec2_describe_volumes(self, volume_id=None):
> + import euca2ools.commands.euca.describevolumes as describevolumes
Added docstring

> [5]
> I see we have PROVIDER_COMMANDS and we also have provider specific methods. I
> guess that we could maybe lose the mapping and align with what you've done for
> the other methods as well. Meaning, add _ec2_validate, _ec2_detattach,
> _nova_validate, _nova_detattach methods. They would be testable like the
> others in the end and we'd be green on the coverage side of things. Feel free
> to refactor in a separate branch.

The reason I left PROVIDER_COMMANDS in place is because there was no difference in how the command was run or how the output was handled so this avoided additional methods. The original reason I broke our the _ec2_* versus _nova_* methods was only in cases where the implementation of the methods differed significantly either due to library or command differences. We do get coverage on both of these commands because we testing them in TestEC2Util versus TestNovaUtil classes. But, if you think we should do this for conformity's sake, I agree with you that it should probably be a separate branch.

review: Approve

[Chad Smith (chad.smith)]

review: Abstain

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support]

86. By Chad Smith on 2014-03-21

add required copyright file

87. By Chad Smith on 2014-03-21

add icon.svg

[David Britton (dpb)]

[4]: If you specify your api endpoint as a valid region, but not the one
you are currently in, you can get some confusing error messages.
Noteably, something like:

   'availability_zone'

Then an error exit. It would be nice if it were more clear about what
was failing. (this is not a blocker for this branch, maybe a follow-on
MP).

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support]

88. By Chad Smith on 2014-03-21

During volume creation, if we can't find instance information for the relation-provided-instance-id, then some *guy* configured his block-storage-broker to contact a different region than the region in which he deployed. Log an appropriate error for dpb and exit(1)

89. By Chad Smith on 2014-03-21

don't use awk. use python to parse lsof output. Add unit test for lsof subprocess command failure

90. By Chad Smith on 2014-03-21

revert rev 89

91. By Chad Smith on 2014-03-21

persist the requested volume_label instead of the requesting instance-id in block_storage_relation_changed, use that label during volume_detach()

92. By Chad Smith on 2014-03-21

add simple generate_volume_label to return the volume label string if none was provided by the related charm. ensure util unit tests are passing volume_label into get_volume_id instead of instance_id

93. By Chad Smith on 2014-03-21

lint

[Chad Smith (chad.smith)]

>
> [4]: If you specify your api endpoint as a valid region, but not the one
> you are currently in, you can get some confusing error messages.
> Noteably, something like:
>
> 'availability_zone'
>
> Then an error exit. It would be nice if it were more clear about what
> was failing. (this is not a blocker for this branch, maybe a follow-on
> MP).

[4] for this branch I've added a quick check if BSB is unable to find instance information for instance to which it is related. If no instance data is found, we know our charm config must be talking to a different region than the one our instances are running in so we exit(1) and log the following:

Could not create volume for instance i-77459754. No instance details discovered by euca-describe-instances. Maybe the charm configured endpoint https://ec2.us-west-1.amazonaws.com/ is not valid for this region.

[David Britton (dpb)]

[5]: if you deploy with EBS backed instances (which juju seems to do now), you will get two attachments to the system, and the remove-hook will fail (bsb). We should be smarter about only removing the volume that we attach.

[David Britton (dpb)]

[6]: Please add just a note to the README that volumes will be re-used
if they exist and are in the same AZ. Also would be a good place to
mention that this behavior is slightly erratic due to lp:1183831

--
David Britton <email address hidden>

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support]

94. By Chad Smith on 2014-03-21

drop icon.svg in favor of using stock fileserver icon

[David Britton (dpb)]

[7]: I hit a error on service destroy in the storage charm. Verify
simply it didn't wait long enough. I came on 5-10 minutes after it was
finished, ran resolved --retry and everything was fine.

Can we multiply what we have now x5 or x10? Storage is by nature very
slow at this kind of stuff.

--
David Britton <email address hidden>

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support]

95. By Chad Smith on 2014-03-21

touchup README to describe reattch volume behavior and known issues with mounting volumes in separate regions

[Chad Smith (chad.smith)]

[5-6] handled and pushed
[7] is due to data-relation-departed hook on the subordinate firing before the principal's data-relation-departed.. as a result, the principal hasn't stopped the postgresql service yet. the storage subordinate charm attempts to umount and gets an error because postgresql is still using the volume. there should be 20 logs in the juju log saying
WARNING: umount /srv/data failed. Retrying. Device in use by (postgresql).

We may have to create a branch on storage subordinate charm to sort out this departed ordering dependency w/ the principal.

[Fernando Correa Neto (fcorrea)]

Thanks for addressing all the comments, Chad. Code looks good.

Will watch for the final testing results.

+1!

review: Approve

[David Britton (dpb)]

After all these fixes, looks good. +1

review: Approve

=== modified file 'Makefile'
--- Makefile	2014-02-05 16:57:45 +0000
+++ Makefile	2014-03-21 22:07:32 +0000
@@ -5,7 +5,7 @@
 	find . -name *.pyc | xargs -r rm
 	find . -name _trial_temp | xargs -r rm -r
 test:
-	cd hooks; CHARM_DIR=${CHARM_DIR} trial test_*
+	cd hooks; trial -j3 test_*py
 
 lint:
 	@flake8 --exclude hooks/charmhelpers hooks
 
=== modified file 'README.md'
--- README.md	2014-02-15 16:13:21 +0000
+++ README.md	2014-03-21 22:07:32 +0000
@@ -15,12 +15,12 @@
       volume-label or volume-id via relation-set calls.
 
       When creating a new volume, this charm's default_volume_size configuration
-      setting will be used if no size is provided via the relation and
-      a volume label such as "<your_juju_unit_name> unit volume" will be used
-      if no volume-label is provided via the relation data.
-      When reattaching an existing volume to an instance, the relation data for
-      volume-id is used if set, and as a fallback option, any volumes matching
-      volume-label will be attached to the instance.
+      setting will be used if no size is provided via the relation. A
+      volume label of the format "<your_juju_unit_name> unit volume" will be
+      used if no volume-label is provided via the relation data.
+      When reattaching an existing volumes to an instance, the relation data
+      volume-id will be used if set, and as a fallback option, any volume
+      matching the relation volume-label will be attached to the instance.
       
       When the volume is attached, the block-storage-broker charm will publish
       block-device-path via the relation data to announce the
@@ -105,6 +105,16 @@
     service my_service start
 
 
+Known Issues
+----
+Since juju may not set target availability zones when deploying units per
+feature bug lp:1183831, block-storage-broker charm will avoid trying to attach
+volumes that exist in a different availability zone than the instance which
+is requesting the volume. Instead of trying to copy volumes from other zones
+into the existing instance's zone, block-storage-broker will create a new
+volume and mount that to the instance. This way the admin can manually copy
+needed files from other region volumes.
+
 TODO
 ----
 
 
=== modified file 'config.yaml'
--- config.yaml	2014-01-28 00:01:57 +0000
+++ config.yaml	2014-03-21 22:07:32 +0000
@@ -1,4 +1,10 @@
 options:
+    provider:
+        type: string
+        description: |
+            The storage provider service, either "nova" (openstack) or
+            "ec2" (aws)
+        default: "nova"
     key:
         type: string
         description: The provider specific api credential key
 
=== added file 'copyright'
--- copyright	1970-01-01 00:00:00 +0000
+++ copyright	2014-03-21 22:07:32 +0000
@@ -0,0 +1,17 @@
+Format: http://dep.debian.net/deps/dep5/
+
+Files: *
+Copyright: Copyright 2011, Canonical Ltd., All Rights Reserved.
+License: GPL-3
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+ .
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
+ .
+ You should have received a copy of the GNU General Public License
+ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2014-02-11 22:45:24 +0000
+++ hooks/hooks.py	2014-03-21 22:07:32 +0000
@@ -7,6 +7,7 @@
 import json
 import os
 import sys
+from util import StorageServiceUtil, generate_volume_label
 
 hooks = hookenv.Hooks()
 
@@ -62,9 +63,9 @@
 @hooks.hook()
 def config_changed():
     """Update changed endpoints and credentials"""
-    import nova_util
+    storage_util = StorageServiceUtil(hookenv.config("provider"))
     missing_options = []
-    for config_option in REQUIRED_CONFIGURATION_OPTIONS:
+    for config_option in storage_util.required_config_options:
         if not hookenv.config(config_option):
             missing_options.append(config_option)
     if missing_options:
@@ -72,67 +73,68 @@
             "WARNING: Incomplete charm config. Missing values for: %s" %
             ",".join(missing_options), hookenv.WARNING)
         sys.exit(0)
-    nova_util.load_environment()
+    storage_util.load_environment()
 
 
 @hooks.hook()
 def install():
     """Install required packages if not present"""
     from charmhelpers import fetch
-    required_packages = ["python-novaclient"]
-
-    fetch.add_source("cloud-archive:havana")
+    provider = hookenv.config("provider")
+    if provider == "nova":
+        required_packages = ["python-novaclient"]
+        fetch.add_source("cloud-archive:havana")
+    elif provider == "ec2":
+        required_packages = ["euca2ools"]
     fetch.apt_update(fatal=True)
     fetch.apt_install(required_packages, fatal=True)
 
 
 @hooks.hook("block-storage-relation-departed")
 def block_storage_relation_departed():
-    """Detach a nova volume from the related instance when relation departs"""
-    import nova_util
+    """Detach a volume from its related instance when relation departs"""
+    storage_util = StorageServiceUtil(hookenv.config("provider"))
+
     # XXX juju bug:1279018 for departed-hooks to see relation-data
-    instance_id = _get_persistent_data(
+    volume_label = _get_persistent_data(
         key=hookenv.remote_unit(), remove_values=True)
-    if not instance_id:
+    if not volume_label:
         hookenv.log(
-            "Cannot detach nova volume from instance without instance-id",
+            "Cannot detach volume from instance without volume_label",
             ERROR)
         sys.exit(1)
-    nova_util.detach_nova_volume(instance_id)
+    storage_util.detach_volume(volume_label)
 
 
 @hooks.hook("block-storage-relation-changed")
 def block_storage_relation_changed():
-    """Attach a nova device to the C{instance-id} requested by the relation
+    """Attach a volume to the C{instance-id} requested by the relation
 
     Optionally the relation can specify:
         - C{volume-label} the label to set on the created volume
         - C{volume-id} to attach an existing volume-id or volume-name
         - C{size} to request a specific volume size
     """
-    import nova_util
+    storage_util = StorageServiceUtil(hookenv.config("provider"))
     instance_id = hookenv.relation_get('instance-id')
     volume_label = hookenv.relation_get('volume-label')
     if not instance_id:
         hookenv.log("Waiting for relation to define instance-id", INFO)
         return
-    _persist_data(hookenv.remote_unit(), instance_id)
     volume_id = hookenv.relation_get('volume-id')  # optional
     size = hookenv.relation_get('size')            # optional
-    device_path = nova_util.attach_nova_volume(
+    device_path = storage_util.attach_volume(
         instance_id=instance_id, volume_id=volume_id, size=size,
         volume_label=volume_label)
+    remote_unit = hookenv.remote_unit()
+    if not volume_label:
+        volume_label = generate_volume_label(remote_unit)
+    _persist_data(remote_unit, volume_label)
 
     # Volume is attached, send the path back to the remote storage unit
     hookenv.relation_set(relation_settings={"block-device-path": device_path})
 
 
-###############################################################################
-# Global variables
-###############################################################################
-REQUIRED_CONFIGURATION_OPTIONS = [
-    "endpoint", "region", "tenant", "key", "secret"]
-
 if __name__ == '__main__':
     hook_name = os.path.basename(sys.argv[0])
     hookenv.log("Running {} hook".format(hook_name))
 
=== removed file 'hooks/nova_util.py'
--- hooks/nova_util.py	2014-02-11 19:58:21 +0000
+++ hooks/nova_util.py	1970-01-01 00:00:00 +0000
@@ -1,227 +0,0 @@
-"""Common python utilities for the nova provider"""
-
-from charmhelpers.core import hookenv
-import subprocess
-import os
-import sys
-from time import sleep
-from hooks import REQUIRED_CONFIGURATION_OPTIONS
-
-METADATA_URL = "http://169.254.169.254/openstack/2012-08-10/meta_data.json"
-NOVA_ENVIRONMENT_MAP = {
-    "endpoint": "OS_AUTH_URL", "region": "OS_REGION_NAME",
-    "tenant": "OS_TENANT_NAME", "key": "OS_USERNAME", "secret": "OS_PASSWORD"}
-
-
-def load_environment():
-    """
-    Source our credentials from the configuration definitions into our
-    operating environment
-    """
-    config_data = hookenv.config()
-    for option in REQUIRED_CONFIGURATION_OPTIONS:
-        environment_variable = NOVA_ENVIRONMENT_MAP[option]
-        os.environ[environment_variable] = config_data[option].strip()
-    validate_credentials()
-
-
-def validate_credentials():
-    """Attempt to contact nova volume service or exit(1)"""
-    try:
-        subprocess.check_call("nova list", shell=True)
-    except subprocess.CalledProcessError, e:
-        hookenv.log(
-            "ERROR: Charm configured credentials can't access endpoint. %s" %
-            str(e),
-            hookenv.ERROR)
-        sys.exit(1)
-    hookenv.log(
-        "Validated charm configuration credentials have access to block "
-        "storage service")
-
-
-def get_volume_attachments(volume_id):
-    """Return a C{list} of volume attachments if present"""
-    from ast import literal_eval
-    try:
-        output = subprocess.check_output(
-            "nova volume-show %s | grep attachments | awk -F '|' '{print $3}'"
-            % volume_id, shell=True)
-        attachments = literal_eval(output.strip())
-    except subprocess.CalledProcessError:
-        return []
-    return attachments
-
-
-def volume_exists(volume_id):
-    """Returns C{True} when C{volume_id} already exists"""
-    try:
-        subprocess.check_call(
-            "nova volume-show %s" % volume_id, shell=True)
-    except subprocess.CalledProcessError:
-        return False
-    return True
-
-
-def get_volume_id(volume_designation=None, instance_id=None):
-    """Return the nova volume id associated with this unit
-
-    Optionally, C{volume_designation} can be either a volume-id or
-    volume-display-name and the matching C{volume-id} will be returned.
-    If no matching volume is found, return C{None}.
-    """
-    token = volume_designation
-    if token:
-        # Get volume by name or volume-id
-        # nova volume-show will error if multiple matches are found
-        command = (
-            "nova volume-show '%s' | grep ' id ' | awk '{ print $4 }'" % token)
-    elif instance_id:
-        command = "nova volume-list | grep %s | awk '{print $2}'" % instance_id
-    else:
-        # Find volume by unit name
-        token = hookenv.remote_unit()
-        command = "nova volume-list | grep %s | awk '{print $2}'" % token
-
-    try:
-        output = subprocess.check_output(command, shell=True)
-    except subprocess.CalledProcessError, e:
-        hookenv.log(
-            "ERROR: Couldn't find nova volume id for %s. %s" % (token, str(e)),
-            hookenv.ERROR)
-        sys.exit(1)
-
-    lines = output.strip().split("\n")
-    if len(lines) > 1:
-        hookenv.log(
-            "Error: Multiple nova volumes labeled as associated with "
-            "%s. Cannot get_volume_id." % token, hookenv.ERROR)
-        sys.exit(1)
-    if lines[0]:
-        return lines[0]
-    return None
-
-
-def get_volume_status(volume_designation=None):
-    """Return the status of a nova volume
-    If C{volume_designation} is specified, return the status of that volume,
-    otherwise use L{get_volume_id} to grab the volume currently related to
-    this unit. If no volume is discoverable, return C{None}.
-    """
-    if volume_designation is None:
-        volume_designation = get_volume_id()
-        if volume_designation is None:
-            hookenv.log(
-                "WARNING: Can't find volume_id to get status.",
-                hookenv.WARNING)
-            return None
-    try:
-        output = subprocess.check_output(
-            "nova volume-show '%s' | grep ' status ' | awk '{ print $4 }'"
-            % volume_designation, shell=True)
-    except subprocess.CalledProcessError, e:
-        hookenv.log(
-            "Error: nova couldn't get status of volume %s. %s" %
-            (volume_designation, str(e)), hookenv.ERROR)
-        return None
-    return output.strip()
-
-
-def attach_nova_volume(instance_id, volume_id=None, size=None,
-                       volume_label=None):
-    """
-    Run nova commands to create and attach a volume to the remote unit if none
-    exists. Attempt to attach and validate the attached volume 10 times. If
-    unable to resolve the attach issues, exit in error and log the issue.
-
-    Log errors if the nova volume is in an unsupported state, and if C{in-use}
-    report it is already attached.
-    Return the device-path of the attached volume to the caller.
-    """
-    load_environment()   # Will fail if proper environment is not set up
-    remote_unit = hookenv.remote_unit()
-    if volume_label is None:
-        volume_label = "%s unit volume" % remote_unit
-    if volume_id:
-        if not volume_exists(volume_id):
-            hookenv.log(
-                "Requested volume-id (%s) does not exist. Unable to associate "
-                "storage with %s" % (volume_id, remote_unit),
-                hookenv.ERROR)
-            sys.exit(1)
-
-        # Validate that current volume status is supported
-        status = get_volume_status(volume_id)
-        if status in ["in-use", "attaching"]:
-            hookenv.log("Volume %s already attached. Done" % volume_id)
-            attachment = get_volume_attachments(volume_id)[0]
-            return attachment["device"]  # The device path on the instance
-        if status != "available":
-            hookenv.log(
-                "Cannot attach nova volume. Volume has unsupported status: %s"
-                % status)
-            sys.exit(1)
-    else:
-        # No volume_id, create a new volume if one isn't already created for
-        # this JUJU_REMOTE_UNIT
-        volume_id = get_volume_id(volume_label)
-        if not volume_id:
-            if not size:
-                size = hookenv.config("default_volume_size")
-            hookenv.log(
-                "Creating a %sGig volume named (%s) for instance %s" %
-                (size, volume_label, instance_id))
-            subprocess.check_call(
-                "nova volume-create --display-name '%s' %s" %
-                (volume_label, size), shell=True)
-            # Get new volume_id search for remote_unit in volume label
-            volume_id = get_volume_id(volume_label)
-
-    device = None
-    hookenv.log("Attaching %s (%s)" % (volume_label, volume_id))
-    for x in range(10):
-        status = get_volume_status(volume_id)
-        if status == "in-use":
-            attachment = get_volume_attachments(volume_id)[0]
-            return attachment["device"]  # The device path on the instance
-        if status == "available":
-            device = subprocess.check_output(
-                "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" %
-                (instance_id, volume_id), shell=True)
-            break
-        else:
-            sleep(5)
-
-    if not device:
-        hookenv.log(
-            "ERROR: Unable to discover device attached by nova volume-attach",
-            hookenv.ERROR)
-        sys.exit(1)
-    return device.strip()
-
-
-def detach_nova_volume(instance_id):
-    """Use nova commands to detach a volume from remote unit if present"""
-    load_environment()   # Will fail if proper environment is not set up
-    volume_id = get_volume_id(instance_id=instance_id)
-    if volume_id:
-        status = get_volume_status(volume_id)
-    else:
-        hookenv.log("Cannot find volume name to detach, done")
-        return
-
-    if status == "available":
-        hookenv.log("Volume (%s) already detached. Done" % volume_id)
-        return
-
-    hookenv.log(
-        "Detaching volume (%s) from instance %s" % (volume_id, instance_id))
-    try:
-        subprocess.check_call(
-            "nova volume-detach %s %s" % (instance_id, volume_id), shell=True)
-    except subprocess.CalledProcessError, e:
-        hookenv.log(
-            "ERROR: Couldn't detach nova volume %s. %s" % (volume_id, str(e)),
-            hookenv.ERROR)
-        sys.exit(1)
-    return
 
=== modified file 'hooks/test_hooks.py'
--- hooks/test_hooks.py	2014-02-11 22:45:24 +0000
+++ hooks/test_hooks.py	2014-03-21 22:07:32 +0000
@@ -4,15 +4,17 @@
 import mocker
 import os
 from testing import TestHookenv
+from util import StorageServiceUtil
 
 
 class TestHooks(mocker.MockerTestCase):
 
     def setUp(self):
+        super(TestHooks, self).setUp()
         self.maxDiff = None
         hooks.hookenv = TestHookenv(
             {"key": "myusername", "tenant": "myusername_project",
-             "secret": "password", "region": "region1",
+             "secret": "password", "region": "region1", "provider": "nova",
              "endpoint": "https://keystone_url:443/v2.0/",
              "default_volume_size": 11})
 
@@ -157,10 +159,9 @@
         self.addCleanup(
             setattr, hooks.hookenv, "_config", hooks.hookenv._config)
         hooks.hookenv._config = (
-            ("key", "myusername"), ("tenant", ""),
+            ("key", "myusername"), ("tenant", ""), ("provider", "nova"),
             ("secret", "password"), ("region", ""),
             ("endpoint", "https://keystone_url:443/v2.0/"))
-        self.mocker.replay()
 
         result = self.assertRaises(SystemExit, hooks.config_changed)
         self.assertEqual(result.code, 0)
@@ -176,8 +177,8 @@
         configured credentials when all mandatory configuration options are
         set.
         """
-        load_environment = self.mocker.replace("nova_util.load_environment")
-        load_environment()
+        self.storage_util = self.mocker.patch(StorageServiceUtil)
+        self.storage_util.load_environment()
         self.mocker.replay()
         hooks.config_changed()
 
@@ -214,7 +215,7 @@
 
     def test_block_storage_relation_changed_with_instance_id(self):
         """
-        L{block_storage_relation_changed} calls L{attach_nova_volume} when
+        L{block_storage_relation_changed} calls L{attach_volume} when
         C{instance-id} is available in the relation data. To report
         a successful device attach, it sets the relation data
         C{block-device-path} to the attached volume's device path from nova.
@@ -228,9 +229,9 @@
         hooks.hookenv._incoming_relation_data = (("instance-id", "i-123"),)
 
         persist = self.mocker.replace(hooks._persist_data)
-        persist("storage/0", "i-123")
-        nova_attach = self.mocker.replace("nova_util.attach_nova_volume")
-        nova_attach(
+        persist("storage/0", "storage/0 unit volume")
+        self.storage_util = self.mocker.patch(StorageServiceUtil)
+        self.storage_util.attach_volume(
             instance_id="i-123", volume_id=None, size=None, volume_label=None)
         self.mocker.result(device_path)   # The attached device path from nova
         self.mocker.replay()
@@ -247,7 +248,7 @@
     def test_block_storage_relation_changed_with_instance_id_volume_id(self):
         """
         When C{volume-id} and C{instance-id} are both present in the relation
-        data, they will both be passed to L{attach_nova_volume}. To report a
+        data, they will both be passed to L{attach_volume}. To report a
         successful device attach, it sets the relation data
         C{block-device-path} to the attached volume's device path from nova.
         """
@@ -262,9 +263,9 @@
             ("instance-id", "i-123"), ("volume-id", volume_id))
 
         persist = self.mocker.replace(hooks._persist_data)
-        persist("storage/0", "i-123")
-        nova_attach = self.mocker.replace("nova_util.attach_nova_volume")
-        nova_attach(
+        persist("storage/0", "storage/0 unit volume")
+        self.storage_util = self.mocker.patch(StorageServiceUtil)
+        self.storage_util.attach_volume(
             instance_id="i-123", volume_id=volume_id, size=None,
             volume_label=None)
         self.mocker.result(device_path)   # The attached device path from nova
@@ -282,7 +283,7 @@
     def test_block_storage_relation_changed_with_instance_id_size(self):
         """
         When C{size} and C{instance-id} are both present in the relation data,
-        they will be passed to L{attach_nova_volume}. To report a successful
+        they will be passed to L{attach_volume}. To report a successful
         device attach, it sets the relation data C{block-device-path} to the
         attached volume's device path from nova.
         """
@@ -296,9 +297,9 @@
         hooks.hookenv._incoming_relation_data = (
             ("instance-id", "i-123"), ("size", size))
         persist = self.mocker.replace(hooks._persist_data)
-        persist("storage/0", "i-123")
-        nova_attach = self.mocker.replace("nova_util.attach_nova_volume")
-        nova_attach(
+        persist("storage/0", "storage/0 unit volume")
+        self.storage_util = self.mocker.patch(StorageServiceUtil)
+        self.storage_util.attach_volume(
             instance_id="i-123", volume_id=None, size=size, volume_label=None)
         self.mocker.result(device_path)   # The attached device path from nova
         self.mocker.replay()
@@ -325,7 +326,7 @@
         result = self.assertRaises(
             SystemExit, hooks.block_storage_relation_departed)
         self.assertEqual(result.code, 1)
-        message = "Cannot detach nova volume from instance without instance-id"
+        message = "Cannot detach volume from instance without volume_label"
         self.assertIn(
             message, hooks.hookenv._log_ERROR, "Not logged- %s" % message)
 
@@ -343,8 +344,8 @@
         with open(persist_path, "w") as outfile:
             outfile.write(unicode(json.dumps(data, ensure_ascii=False)))
 
-        nova_detach = self.mocker.replace("nova_util.detach_nova_volume")
-        nova_detach("i-123")
+        self.storage_util = self.mocker.patch(StorageServiceUtil)
+        self.storage_util.detach_volume("i-123")
         self.mocker.replay()
 
         hooks.block_storage_relation_departed()
 
=== removed file 'hooks/test_nova_util.py'
--- hooks/test_nova_util.py	2014-02-11 22:07:32 +0000
+++ hooks/test_nova_util.py	1970-01-01 00:00:00 +0000
@@ -1,578 +0,0 @@
-import nova_util as util
-import mocker
-import os
-import subprocess
-from testing import TestHookenv
-
-
-class TestNovaUtil(mocker.MockerTestCase):
-
-    def setUp(self):
-        self.maxDiff = None
-        util.hookenv = TestHookenv(
-            {"key": "myusername", "tenant": "myusername_project",
-             "secret": "password", "region": "region1",
-             "endpoint": "https://keystone_url:443/v2.0/",
-             "default_volume_size": 11})
-        util.log = util.hookenv.log
-
-    def test_load_environment_with_nova_variables(self):
-        """
-        L{load_environment} will setup script environment variables for nova
-        by mapping configuration values provided to openstack OS_* environment
-        variables and then call L{validate_credentials} to assert
-        that environment variables provided give access to the service.
-        """
-        self.addCleanup(setattr, util.os, "environ", util.os.environ)
-        util.os.environ = {}
-        credentials = self.mocker.replace(util.validate_credentials)
-        credentials()
-        self.mocker.replay()
-
-        util.load_environment()
-        expected = {
-            "OS_AUTH_URL": "https://keystone_url:443/v2.0/",
-            "OS_PASSWORD": "password",
-            "OS_REGION_NAME": "region1",
-            "OS_TENANT_NAME": "myusername_project",
-            "OS_USERNAME": "myusername"
-        }
-        self.assertEqual(util.os.environ, expected)
-
-    def test_load_environment_error_missing_config_options(self):
-        """
-        L{load_environment} will exit in failure and log a message if any
-        required configuration option is not set.
-        """
-        self.addCleanup(setattr, util.os, "environ", util.os.environ)
-        credentials = self.mocker.replace(util.validate_credentials)
-        credentials()
-        self.mocker.throw(SystemExit)
-        self.mocker.replay()
-
-        self.assertRaises(SystemExit, util.load_environment)
-
-    def test_validate_credentials_failure(self):
-        """
-        L{validate_credentials} will attempt a simple nova command to ensure
-        the environment is properly configured to access the nova service.
-        Upon failure to contact the nova service, L{validate_credentials} will
-        exit in error and log a message.
-        """
-        command = "nova list"
-        nova_cmd = self.mocker.replace(subprocess.check_call)
-        nova_cmd(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
-
-        result = self.assertRaises(SystemExit, util.validate_credentials)
-        self.assertEqual(result.code, 1)
-        message = (
-            "ERROR: Charm configured credentials can't access endpoint. "
-            "Command '%s' returned non-zero exit status 1" % command)
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
-    def test_validate_credentials(self):
-        """
-        L{validate_credentials} will succeed when a simple nova command
-        succeeds due to a properly configured environment based on the charm
-        configuration options.
-        """
-        command = "nova list"
-        nova_cmd = self.mocker.replace(subprocess.check_call)
-        nova_cmd(command, shell=True)
-        self.mocker.replay()
-
-        util.validate_credentials()
-        message = (
-            "Validated charm configuration credentials have access to "
-            "block storage service"
-        )
-        self.assertIn(
-            message, util.hookenv._log_INFO, "Not logged- %s" % message)
-
-    def test_get_volume_attachments_present(self):
-        """
-        L{get_volume_attachments} returns a C{list} of available volume
-        attachments for the given C{volume_id}.
-        """
-        volume_id = "123-123-123"
-        command = (
-            "nova volume-show %s | grep attachments | awk -F '|' '{print $3}'"
-            % volume_id)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.result(
-            "[{u'device': u'/dev/vdc', u'server_id': u'blah', "
-            "u'id': u'i-123123', u'volume_id': u'%s'}]" % volume_id)
-        self.mocker.replay()
-
-        expected = [{
-            "device": "/dev/vdc", "server_id": "blah", "id": "i-123123",
-            "volume_id": volume_id}]
-
-        self.assertEqual(util.get_volume_attachments(volume_id), expected)
-
-    def test_get_volume_attachments_no_attachments_present(self):
-        """
-        L{get_volume_attachments} returns an empty C{list} if no available
-        volume attachments are reported for the given C{volume_id}.
-        """
-        volume_id = "123-123-123"
-        command = (
-            "nova volume-show %s | grep attachments | awk -F '|' '{print $3}'"
-            % volume_id)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.result("[]")
-        self.mocker.replay()
-
-        self.assertEqual(util.get_volume_attachments(volume_id), [])
-
-    def test_get_volume_attachments_no_volume_present(self):
-        """
-        L{get_volume_attachments} returns an empty C{list} if no available
-        volume is discovered for the given C{volume_id}.
-        """
-        volume_id = "123-123-123"
-        command = (
-            "nova volume-show %s | grep attachments | awk -F '|' '{print $3}'"
-            % volume_id)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
-
-        self.assertEqual(util.get_volume_attachments(volume_id), [])
-
-    def test_volume_exists_true(self):
-        """
-        L{volume_exists} returns C{True} when C{volume_id} is seen by the nova
-        client command C{nova volume-show}.
-        """
-        volume_id = "123134124-1241412-1242141"
-        command = "nova volume-show %s" % volume_id
-        nova_cmd = self.mocker.replace(subprocess.call)
-        nova_cmd(command, shell=True)
-        self.mocker.result(0)
-        self.mocker.replay()
-        self.assertTrue(util.volume_exists(volume_id))
-
-    def test_volume_exists_false(self):
-        """
-        L{volume_exists} returns C{False} when C{volume_id} is not seen by the
-        nova client command C{nova volume-show}.
-        """
-        volume_id = "123134124-1241412-1242141"
-        command = "nova volume-show %s" % volume_id
-        nova_cmd = self.mocker.replace(subprocess.call)
-        nova_cmd(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, "Volume not here"))
-        self.mocker.replay()
-
-        self.assertFalse(util.volume_exists(volume_id))
-
-    def test_get_volume_id_by_volume_name(self):
-        """
-        L{get_volume_id} provided with a existing C{volume_name} returns the
-        corresponding nova volume id.
-        """
-        volume_name = "my-volume"
-        volume_id = "12312412-412312\n"
-        command = (
-            "nova volume-show '%s' | grep ' id ' | awk '{ print $4 }'" %
-            volume_name)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.result(volume_id)
-        self.mocker.replay()
-        self.assertEqual(util.get_volume_id(volume_name), volume_id.strip())
-
-    def test_get_volume_id_command_error(self):
-        """
-        L{get_volume_id} handles any nova command error by reporting the error
-        and exiting the hook.
-        """
-        volume_name = "my-volume"
-        command = (
-            "nova volume-show '%s' | grep ' id ' | awk '{ print $4 }'" %
-            volume_name)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
-
-        result = self.assertRaises(SystemExit, util.get_volume_id, volume_name)
-        self.assertEqual(result.code, 1)
-        message = (
-            "ERROR: Couldn't find nova volume id for %s. Command '%s' "
-            "returned non-zero exit status 1" % (volume_name, command))
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
-    def test_get_volume_id_without_volume_name(self):
-        """
-        L{get_volume_id} without a provided C{volume_name} will discover the
-        nova volume id by searching nova volume-list for volumes labelled with
-        the os.environ[JUJU_REMOTE_UNIT].
-        """
-        unit_name = "postgresql/0"
-        self.addCleanup(
-            setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
-        volume_id = "123134124-1241412-1242141\n"
-        command = (
-            "nova volume-list | grep %s | awk '{print $2}'" % unit_name)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.result(volume_id)
-        self.mocker.replay()
-
-        self.assertEqual(util.get_volume_id(), volume_id.strip())
-
-    def test_get_volume_id_without_volume_name_no_matching_volume(self):
-        """
-        L{get_volume_id} without a provided C{volume_name} will return C{None}
-        when it cannot find a matching volume label from nova volume-list for
-        the os.environ[JUJU_REMOTE_UNIT].
-        """
-        unit_name = "postgresql/0"
-        self.addCleanup(
-            setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
-        command = (
-            "nova volume-list | grep %s | awk '{print $2}'" % unit_name)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.result("\n")   # Empty result string from awk
-        self.mocker.replay()
-
-        self.assertIsNone(util.get_volume_id())
-
-    def test_get_volume_id_without_volume_name_multiple_matching_volumes(self):
-        """
-        L{get_volume_id} does not support multiple volumes associated with the
-        the instance represented by os.environ[JUJU_REMOTE_UNIT]. When
-        C{volume_name} is not specified and nova volume-list returns multiple
-        results the function exits with an error.
-        """
-        unit_name = "postgresql/0"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
-        command = (
-            "nova volume-list | grep %s | awk '{print $2}'" % unit_name)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.result("123-123-123\n456-456-456\n")   # Two results
-        self.mocker.replay()
-
-        result = self.assertRaises(SystemExit, util.get_volume_id)
-        self.assertEqual(result.code, 1)
-        message = (
-            "Error: Multiple nova volumes labeled as associated with "
-            "%s. Cannot get_volume_id." % unit_name)
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
-    def test_get_volume_status_by_known_volume_id(self):
-        """
-        L{get_volume_status} returns the status of a volume matching
-        C{volume_id} by using the nova client commands.
-        """
-        volume_id = "123134124-1241412-1242141"
-        command = (
-            "nova volume-show '%s' | grep ' status ' | awk '{ print $4 }'" %
-            volume_id)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.result("available\n")
-        self.mocker.replay()
-        self.assertEqual(util.get_volume_status(volume_id), "available")
-
-    def test_get_volume_status_by_invalid_volume_id(self):
-        """
-        L{get_volume_status} returns the status of a volume matching
-        C{volume_id} by using the nova client commands.
-        """
-        volume_id = "123134124-1241412-1242141"
-        command = (
-            "nova volume-show '%s' | grep ' status ' | awk '{ print $4 }'" %
-            volume_id)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
-        self.assertIsNone(util.get_volume_status(volume_id))
-        message = (
-            "Error: nova couldn't get status of volume %s. "
-            "Command '%s' returned non-zero exit status 1" %
-            (volume_id, command))
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
-    def test_get_volume_status_when_get_volume_id_none(self):
-        """
-        L{get_volume_status} logs a warning and returns C{None} when
-        C{volume_id} is not specified and L{get_volume_id} returns C{None}.
-        """
-        get_vol_id = self.mocker.replace(util.get_volume_id)
-        get_vol_id()
-        self.mocker.result(None)
-        self.mocker.replay()
-
-        self.assertIsNone(util.get_volume_status())
-        message = "WARNING: Can't find volume_id to get status."
-        self.assertIn(
-            message, util.hookenv._log_WARNING, "Not logged- %s" % message)
-
-    def test_get_volume_status_when_get_volume_id_discovers(self):
-        """
-        When C{volume_id} is not specified, L{get_volume_status} obtains the
-        volume id from L{get_volume_id} gets the status using nova commands.
-        """
-        volume_id = "123-123-123"
-        get_vol_id = self.mocker.replace(util.get_volume_id)
-        get_vol_id()
-        self.mocker.result(volume_id)
-        command = (
-            "nova volume-show '%s' | grep ' status ' | awk '{ print $4 }'" %
-            volume_id)
-        nova_cmd = self.mocker.replace(subprocess.check_output)
-        nova_cmd(command, shell=True)
-        self.mocker.result("in-use\n")
-        self.mocker.replay()
-
-        self.assertEqual(util.get_volume_status(), "in-use")
-
-    def test_attach_nova_volume_failure_when_volume_id_does_not_exist(self):
-        """
-        When L{attach_nova_volume} is provided a C{volume_id} that doesn't
-        exist it logs and error and exits.
-        """
-        unit_name = "postgresql/0"
-        instance_id = "i-123123"
-        volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
-
-        load_environment = self.mocker.replace(util.load_environment)
-        load_environment()
-        volume_exists = self.mocker.replace(util.volume_exists)
-        volume_exists(volume_id)
-        self.mocker.result(False)
-        self.mocker.replay()
-
-        result = self.assertRaises(
-            SystemExit, util.attach_nova_volume, instance_id, volume_id)
-        self.assertEqual(result.code, 1)
-        message = (
-            "Requested volume-id (%s) does not exist. "
-            "Unable to associate storage with %s" % (volume_id, unit_name))
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
-    def test_attach_nova_volume_when_volume_id_already_attached(self):
-        """
-        When L{attach_nova_volume} is provided a C{volume_id} that already
-        has the state C{in-use} it logs that the volume is already attached
-        and returns.
-        """
-        unit_name = "postgresql/0"
-        instance_id = "i-123123"
-        volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
-
-        load_environment = self.mocker.replace(util.load_environment)
-        load_environment()
-        volume_exists = self.mocker.replace(util.volume_exists)
-        volume_exists(volume_id)
-        self.mocker.result(True)
-        get_vol_status = self.mocker.replace(util.get_volume_status)
-        get_vol_status(volume_id)
-        self.mocker.result("in-use")
-        get_attachments = self.mocker.replace(util.get_volume_attachments)
-        get_attachments(volume_id)
-        self.mocker.result([{"device": "/dev/vdc"}])
-        self.mocker.replay()
-
-        self.assertEqual(
-            util.attach_nova_volume(instance_id, volume_id), "/dev/vdc")
-
-        message = "Volume %s already attached. Done" % volume_id
-        self.assertIn(
-            message, util.hookenv._log_INFO, "Not logged- %s" % message)
-
-    def test_attach_nova_volume_failure_when_volume_unsupported_status(self):
-        """
-        When L{attach_nova_volume} is provided a C{volume_id} that has an
-        unsupported status. It logs the error and exits.
-        """
-        unit_name = "postgresql/0"
-        instance_id = "i-123123"
-        volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
-
-        load_environment = self.mocker.replace(util.load_environment)
-        load_environment()
-        volume_exists = self.mocker.replace(util.volume_exists)
-        volume_exists(volume_id)
-        self.mocker.result(True)
-        get_vol_status = self.mocker.replace(util.get_volume_status)
-        get_vol_status(volume_id)
-        self.mocker.result("deleting")
-        self.mocker.replay()
-
-        result = self.assertRaises(
-            SystemExit, util.attach_nova_volume, instance_id, volume_id)
-        self.assertEqual(result.code, 1)
-        message = ("Cannot attach nova volume. "
-                   "Volume has unsupported status: deleting")
-        self.assertIn(
-            message, util.hookenv._log_INFO, "Not logged- %s" % message)
-
-    def test_attach_nova_volume_creates_with_config_size(self):
-        """
-        When C{volume_id} is C{None}, L{attach_nova_volume} will create a new
-        nova volume with the configured C{default_volume_size} when the volume
-        doesn't exist and C{size} is not provided.
-        """
-        unit_name = "postgresql/0"
-        instance_id = "i-123123"
-        volume_id = "123-123-123"
-        volume_label = "%s unit volume" % unit_name
-        default_volume_size = util.hookenv.config("default_volume_size")
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
-
-        load_environment = self.mocker.replace(util.load_environment)
-        load_environment()
-        get_vol_id = self.mocker.replace(util.get_volume_id)
-        get_vol_id(volume_label)
-        self.mocker.result(None)
-        command = (
-            "nova volume-create --display-name '%s' %s" %
-            (volume_label, default_volume_size))
-        nova_cmd = self.mocker.replace(subprocess.check_call)
-        nova_cmd(command, shell=True)
-        get_vol_id(volume_label)
-        self.mocker.result(volume_id)   # Found the volume now
-        get_vol_status = self.mocker.replace(util.get_volume_status)
-        get_vol_status(volume_id)
-        self.mocker.result("available")
-        command = (
-            "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" %
-            (instance_id, volume_id))
-        attach_cmd = self.mocker.replace(subprocess.check_output)
-        attach_cmd(command, shell=True)
-        self.mocker.result("/dev/vdc\n")
-        self.mocker.replay()
-
-        self.assertEqual(util.attach_nova_volume(instance_id), "/dev/vdc")
-        messages = [
-            "Creating a %sGig volume named (%s) for instance %s" %
-            (default_volume_size, volume_label, instance_id),
-            "Attaching %s (%s)" % (volume_label, volume_id)]
-        for message in messages:
-            self.assertIn(
-                message, util.hookenv._log_INFO, "Not logged- %s" % message)
-
-    def test_detach_nova_volume_no_volume_found(self):
-        """
-        When L{get_volume_id} is unable to find an attached volume and returns
-        C{None}, L{detach_volume} will log a message and perform no work.
-        """
-        instance_id = "i-123123"
-        load_environment = self.mocker.replace(util.load_environment)
-        load_environment()
-        get_vol_id = self.mocker.replace(util.get_volume_id)
-        get_vol_id(instance_id=instance_id)
-        self.mocker.result(None)
-        self.mocker.replay()
-
-        util.detach_nova_volume(instance_id)
-        message = "Cannot find volume name to detach, done"
-        self.assertIn(
-            message, util.hookenv._log_INFO, "Not logged- %s" % message)
-
-    def test_detach_nova_volume_volume_already_detached(self):
-        """
-        When L{get_volume_id} finds a volume that is already C{available} it
-        logs that the volume is already detached and does no work.
-        """
-        instance_id = "i-123123"
-        volume_id = "123-123-123"
-        load_environment = self.mocker.replace(util.load_environment)
-        load_environment()
-        get_vol_id = self.mocker.replace(util.get_volume_id)
-        get_vol_id(instance_id=instance_id)
-        self.mocker.result(volume_id)
-        get_vol_status = self.mocker.replace(util.get_volume_status)
-        get_vol_status(volume_id)
-        self.mocker.result("available")
-        self.mocker.replay()
-
-        util.detach_nova_volume(instance_id)  # pass in our instance_id
-        message = "Volume (%s) already detached. Done" % volume_id
-        self.assertIn(
-            message, util.hookenv._log_INFO, "Not logged- %s" % message)
-
-    def test_detach_nova_volume_command_error(self):
-        """
-        When the nova volume-detach command fails, L{detach_nova_volume} will
-        log a message and exit in error.
-        """
-        volume_id = "123-123-123"
-        instance_id = "i-123123"
-        load_environment = self.mocker.replace(util.load_environment)
-        load_environment()
-        get_vol_id = self.mocker.replace(util.get_volume_id)
-        get_vol_id(instance_id=instance_id)
-        self.mocker.result(volume_id)
-        get_vol_status = self.mocker.replace(util.get_volume_status)
-        get_vol_status(volume_id)
-        self.mocker.result("in-use")
-        command = "nova volume-detach %s %s" % (instance_id, volume_id)
-        nova_cmd = self.mocker.replace(subprocess.check_call)
-        nova_cmd(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
-
-        result = self.assertRaises(
-            SystemExit, util.detach_nova_volume, instance_id)
-        self.assertEqual(result.code, 1)
-        message = (
-            "ERROR: Couldn't detach nova volume %s. Command '%s' "
-            "returned non-zero exit status 1" % (volume_id, command))
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
-    def test_detach_nova_volume(self):
-        """
-        When L{get_volume_id} finds a volume associated with this instance
-        which has a volume state not equal to C{available}, it detaches that
-        volume using nova commands.
-        """
-        volume_id = "123-123-123"
-        instance_id = "i-123123"
-        load_environment = self.mocker.replace(util.load_environment)
-        load_environment()
-        get_vol_id = self.mocker.replace(util.get_volume_id)
-        get_vol_id(instance_id=instance_id)
-        self.mocker.result(volume_id)
-        get_vol_status = self.mocker.replace(util.get_volume_status)
-        get_vol_status(volume_id)
-        self.mocker.result("in-use")
-        command = "nova volume-detach %s %s" % (instance_id, volume_id)
-        nova_cmd = self.mocker.replace(subprocess.check_call)
-        nova_cmd(command, shell=True)
-        self.mocker.replay()
-
-        util.detach_nova_volume(instance_id)
-        message = (
-            "Detaching volume (%s) from instance %s" %
-            (volume_id, instance_id))
-        self.assertIn(
-            message, util.hookenv._log_INFO, "Not logged- %s" % message)
 
=== added file 'hooks/test_util.py'
--- hooks/test_util.py	1970-01-01 00:00:00 +0000
+++ hooks/test_util.py	2014-03-21 22:07:32 +0000
@@ -0,0 +1,1730 @@
+import util
+from util import StorageServiceUtil, ENVIRONMENT_MAP, generate_volume_label
+import mocker
+import os
+import subprocess
+from testing import TestHookenv
+
+
+class TestNovaUtil(mocker.MockerTestCase):
+
+    def setUp(self):
+        super(TestNovaUtil, self).setUp()
+        self.maxDiff = None
+        util.hookenv = TestHookenv(
+            {"key": "myusername", "tenant": "myusername_project",
+             "secret": "password", "region": "region1",
+             "endpoint": "https://keystone_url:443/v2.0/",
+             "default_volume_size": 11})
+        util.log = util.hookenv.log
+        self.storage = StorageServiceUtil("nova")
+
+    def test_invalid_provier_config(self):
+        """When an invalid provider config is set and error is reported."""
+        result = self.assertRaises(SystemExit, StorageServiceUtil, "ce2")
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Invalid charm configuration setting for provider. "
+            "'ce2' must be one of: %s" % ", ".join(ENVIRONMENT_MAP.keys()))
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_generate_volume_label(self):
+        """
+        L{generate_volume_label} return a string based on the C{remove_unit}
+        """
+        self.assertEqual("blah unit volume", generate_volume_label("blah"))
+
+    def test_load_environment_with_nova_variables(self):
+        """
+        L{load_environment} will setup script environment variables for nova
+        by mapping configuration values provided to openstack OS_* environment
+        variables and then call L{validate_credentials} to assert
+        that environment variables provided give access to the service.
+        """
+        self.addCleanup(setattr, util.os, "environ", util.os.environ)
+        util.os.environ = {}
+
+        def mock_validate():
+            pass
+        self.storage.validate_credentials = mock_validate
+
+        self.storage.load_environment()
+        expected = {
+            "OS_AUTH_URL": "https://keystone_url:443/v2.0/",
+            "OS_PASSWORD": "password",
+            "OS_REGION_NAME": "region1",
+            "OS_TENANT_NAME": "myusername_project",
+            "OS_USERNAME": "myusername"
+        }
+        self.assertEqual(util.os.environ, expected)
+
+    def test_load_environment_error_missing_config_options(self):
+        """
+        L{load_environment} will exit in failure and log a message if any
+        required configuration option is not set.
+        """
+        self.addCleanup(setattr, util.os, "environ", util.os.environ)
+
+        def mock_validate():
+            raise SystemExit("something invalid")
+        self.storage.validate_credentials = mock_validate
+
+        self.assertRaises(SystemExit, self.storage.load_environment)
+
+    def test_validate_credentials_failure(self):
+        """
+        L{validate_credentials} will attempt a simple nova command to ensure
+        the environment is properly configured to access the nova service.
+        Upon failure to contact the nova service, L{validate_credentials} will
+        exit in error and log a message.
+        """
+        command = "nova list"
+        nova_cmd = self.mocker.replace(subprocess.check_call)
+        nova_cmd(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage.validate_credentials)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Charm configured credentials can't access endpoint. "
+            "Command '%s' returned non-zero exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_validate_credentials(self):
+        """
+        L{validate_credentials} will succeed when a simple nova command
+        succeeds due to a properly configured environment based on the charm
+        configuration options.
+        """
+        command = "nova list"
+        nova_cmd = self.mocker.replace(subprocess.check_call)
+        nova_cmd(command, shell=True)
+        self.mocker.replay()
+
+        self.storage.validate_credentials()
+        message = (
+            "Validated charm configuration credentials have access to "
+            "block storage service"
+        )
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_wb_nova_volume_show_with_attachments(self):
+        """
+        L{_nova_volume_show} returns a C{dict} of volume information for
+        the given C{volume_id}.
+        """
+        volume_id = "123-123-123"
+        command = "nova volume-show '%s'" % volume_id
+        output = (
+            "+-------------------------------------+\n"
+            "| Property | Value |\n"
+            "+---------------------------------------------------------+\n"
+            "| status              | in-use                            |\n"
+            "| display_name        | my-volume                         |\n"
+            "| attachments         | [{u'device': u'/dev/vdc', "
+            "u'server_id': u'blah', u'id': u'%s', u'volume_id': u'%s'}] |\n"
+            "| availability_zone   | nova                              |\n"
+            "| bootable            | false                             |\n"
+            "| created_at          | 2014-02-12T21:02:29.000000        |\n"
+            "| display_description | None                              |\n"
+            "| volume_type         | None                              |\n"
+            "| snapshot_id         | None                              |\n"
+            "| source_volid        | None                              |\n"
+            "| size                | 9                                 |\n"
+            "| id                  | %s                                |\n"
+            "| metadata            | {}                                |\n"
+            "+---------------------+-----------------------------------+\n" %
+            (volume_id, volume_id, volume_id))
+        nova_cmd = self.mocker.replace(subprocess.check_output)
+        nova_cmd(command, shell=True)
+        self.mocker.result(output)
+        self.mocker.replay()
+
+        expected = {
+            "device": "/dev/vdc", "instance_id": "blah", "id": volume_id,
+            "size": "9", "volume_label": "my-volume",
+            "snapshot_id": "None", "availability_zone": "nova",
+            "status": "in-use", "tags": {"volume_label": "my-volume"}}
+
+        self.assertEqual(
+            self.storage._nova_volume_show(volume_id), expected)
+
+    def test_wb_nova_volume_show_without_attachments(self):
+        """
+        L{_nova_volume_show} returns a C{dict} of volume information for
+        the given C{volume_id}. When no attachments are present, C{device} and
+        C{instance_id} will be an empty C{str}.
+        """
+        volume_id = "123-123-123"
+        command = "nova volume-show '%s'" % volume_id
+        output = (
+            "+-------------------------------------+\n"
+            "| Property | Value |\n"
+            "+---------------------------------------------------------+\n"
+            "| status              | in-use                            |\n"
+            "| display_name        | my-volume                         |\n"
+            "| attachments         | []                                |\n"
+            "| availability_zone   | nova                              |\n"
+            "| bootable            | false                             |\n"
+            "| created_at          | 2014-02-12T21:02:29.000000        |\n"
+            "| display_description | None                              |\n"
+            "| volume_type         | None                              |\n"
+            "| snapshot_id         | None                              |\n"
+            "| source_volid        | None                              |\n"
+            "| size                | 9                                 |\n"
+            "| id                  | %s                                |\n"
+            "| metadata            | {}                                |\n"
+            "+---------------------+-----------------------------------+\n" %
+            (volume_id))
+        nova_cmd = self.mocker.replace(subprocess.check_output)
+        nova_cmd(command, shell=True)
+        self.mocker.result(output)
+        self.mocker.replay()
+
+        expected = {
+            "device": "", "instance_id": "", "id": volume_id,
+            "size": "9", "volume_label": "my-volume",
+            "snapshot_id": "None", "availability_zone": "nova",
+            "status": "in-use", "tags": {"volume_label": "my-volume"}}
+
+        self.assertEqual(
+            self.storage._nova_volume_show(volume_id), expected)
+
+    def test_wb_nova_volume_show_no_volume_present(self):
+        """
+        L{_nova_volume_show} exits when no matching volume is present due to
+        receiving a C{CalledProcessError} and logs the error when unable to
+        execute the C{nova volume-show} command.
+        """
+        volume_id = "123-123-123"
+        command = "nova volume-show '%s'" % volume_id
+        nova_cmd = self.mocker.replace(subprocess.check_output)
+        nova_cmd(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._nova_volume_show, volume_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Failed to get nova volume info. Command '%s' returned "
+            "non-zero exit status 1" % (command))
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_get_volume_id_by_volume_name(self):
+        """
+        L{get_volume_id} provided with a existing C{volume_name} returns the
+        corresponding nova volume id.
+        """
+        volume_name = "my-volume"
+        volume_id = "12312412-412312"
+
+        def mock_describe():
+            return {volume_id: {"tags": {"volume_name": volume_name}},
+                    "456456-456456": {"tags": {"volume_name": "blah"}}}
+        self.storage.describe_volumes = mock_describe
+
+        self.assertEqual(self.storage.get_volume_id(volume_name), volume_id)
+
+    def test_get_volume_id_without_volume_name(self):
+        """
+        L{get_volume_id} without a provided C{volume_name} will discover the
+        nova volume id by searching L{describe_volumes} for volumes labelled
+        with the os.environ[JUJU_REMOTE_UNIT].
+        """
+        unit_name = "postgresql/0"
+        self.addCleanup(
+            setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        volume_id = "123134124-1241412-1242141"
+
+        def mock_describe():
+            return {volume_id:
+                    {"tags": {"volume_name": "postgresql/0 unit volume"}},
+                    "456456-456456": {"tags": {"volume_name": "blah"}}}
+        self.storage.describe_volumes = mock_describe
+
+        self.assertEqual(self.storage.get_volume_id(), volume_id)
+
+    def test_get_volume_id_without_volume_name_no_matching_volume(self):
+        """
+        L{get_volume_id} without a provided C{volume_name} will return C{None}
+        when it cannot find a matching volume label from L{describe_volumes}
+        for the os.environ[JUJU_REMOTE_UNIT].
+        """
+        unit_name = "postgresql/0"
+        self.addCleanup(
+            setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        def mock_describe(val):
+            self.assertIsNone(val)
+            return {"123123-123123":
+                    {"tags": {"volume_name": "postgresql/1 unit volume"}},
+                    "456456-456456": {"tags": {"volume_name": "blah"}}}
+        self.storage._nova_describe_volumes = mock_describe
+
+        self.assertIsNone(self.storage.get_volume_id())
+
+    def test_get_volume_id_without_volume_name_multiple_matching_volumes(self):
+        """
+        L{get_volume_id} does not support multiple volumes associated with the
+        the instance represented by os.environ[JUJU_REMOTE_UNIT]. When
+        C{volume_name} is not specified and L{describe_volumes} returns
+        multiple results the function exits with an error.
+        """
+        unit_name = "postgresql/0"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        def mock_describe():
+            return {"123123-123123":
+                    {"tags": {"volume_name": "postgresql/0 unit volume"}},
+                    "456456-456456":
+                    {"tags": {"volume_name": "unit postgresql/0 volume2"}}}
+        self.storage.describe_volumes = mock_describe
+
+        result = self.assertRaises(SystemExit, self.storage.get_volume_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "Error: Multiple volumes are associated with %s. "
+            "Cannot get_volume_id." % unit_name)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_attach_volume_failure_when_volume_id_does_not_exist(self):
+        """
+        When L{attach_volume} is provided a C{volume_id} that doesn't
+        exist, it logs an error and exits.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.storage.load_environment = lambda: None
+        self.storage.describe_volumes = lambda volume_id: {}
+
+        result = self.assertRaises(
+            SystemExit, self.storage.attach_volume, instance_id=instance_id,
+            volume_id=volume_id)
+        self.assertEqual(result.code, 1)
+        message = ("Requested volume-id (%s) does not exist. Unable to "
+                   "associate storage with %s" % (volume_id, unit_name))
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_attach_volume_without_volume_label(self):
+        """
+        L{attach_volume} without a provided C{volume_label} or C{volume_id}
+        will discover the nova volume id by searching L{describe_volumes}
+        for volumes with a label based on the os.environ[JUJU_REMOTE_UNIT].
+        """
+        unit_name = "postgresql/0"
+        volume_id = "123-123-123"
+        instance_id = "i-123123123"
+        volume_label = "%s unit volume" % unit_name
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "device": "/dev/vdc"}
+        self.storage.describe_volumes = mock_describe_volumes
+
+        self.assertEqual(self.storage.attach_volume(instance_id), "/dev/vdc")
+        message = (
+            "Attaching %s (%s)" % (volume_label, volume_id))
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_attach_volume_when_volume_id_already_attached(self):
+        """
+        When L{attach_volume} is provided a C{volume_id} that already
+        has the state C{in-use} it logs that the volume is already attached
+        and returns.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.storage.load_environment = lambda: None
+
+        def mock_describe(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "device": "/dev/vdc"}
+        self.storage.describe_volumes = mock_describe
+
+        self.assertEqual(
+            self.storage.attach_volume(instance_id, volume_id), "/dev/vdc")
+
+        message = "Volume %s already attached. Done" % volume_id
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_attach_volume_when_volume_id_attaching_retry(self):
+        """
+        When L{attach_volume} is provided a C{volume_id} that has the status
+        C{in-use} it logs, sleeps and retries until the volume is C{in-use}.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.describe_count = 0
+
+        self.storage.load_environment = lambda: None
+
+        sleep = self.mocker.replace("util.sleep")
+        sleep(5)
+        self.mocker.replay()
+
+        def mock_describe(my_id):
+            self.assertEqual(my_id, volume_id)
+            if self.describe_count == 0:
+                self.describe_count += 1
+                return {"status": "attaching"}
+            else:
+                self.describe_count += 1
+                return {"status": "in-use", "device": "/dev/vdc"}
+        self.storage.describe_volumes = mock_describe
+
+        self.assertEqual(
+            self.storage.attach_volume(instance_id, volume_id), "/dev/vdc")
+
+        messages = ["Volume %s already attached. Done" % volume_id,
+                    "Volume %s still attaching. Waiting." % volume_id]
+        for message in messages:
+            self.assertIn(
+                message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_attach_volume_failure_with_volume_unsupported_status(self):
+        """
+        When L{attach_volume} is provided a C{volume_id} that has an
+        unsupported status. It logs the error and exits.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.storage.load_environment = lambda: None
+
+        def mock_describe(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "deleting", "device": "/dev/vdc"}
+        self.storage.describe_volumes = mock_describe
+
+        result = self.assertRaises(
+            SystemExit, self.storage.attach_volume, instance_id, volume_id)
+        self.assertEqual(result.code, 1)
+        message = ("Cannot attach volume. "
+                   "Volume has unsupported status: deleting")
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_attach_volume_creates_with_config_size(self):
+        """
+        When C{volume_id} is C{None}, L{attach_volume} will create a new
+        volume with the configured C{default_volume_size} when the volume
+        doesn't exist and C{size} is not provided.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        volume_label = "%s unit volume" % unit_name
+        default_volume_size = util.hookenv.config("default_volume_size")
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.storage.load_environment = lambda: None
+        self.storage.get_volume_id = lambda _: None
+
+        def mock_describe(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "device": "/dev/vdc"}
+        self.storage.describe_volumes = mock_describe
+
+        def mock_nova_create(size, label, instance):
+            self.assertEqual(size, default_volume_size)
+            self.assertEqual(label, volume_label)
+            self.assertEqual(instance, instance_id)
+            return volume_id
+        self.storage._nova_create_volume = mock_nova_create
+
+        self.assertEqual(self.storage.attach_volume(instance_id), "/dev/vdc")
+        message = "Attaching %s (%s)" % (volume_label, volume_id)
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_wb_nova_describe_volumes_command_error(self):
+        """
+        L{_nova_describe_volumes} will exit in error when the
+        C{nova volume-list} command fails.
+        """
+        command = "nova volume-list"
+        nova_list = self.mocker.replace(subprocess.check_output)
+        nova_list(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._nova_describe_volumes)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Command '%s' returned non-zero exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_nova_describe_volumes_without_attached_instances(self):
+        """
+        L{_nova_describe_volumes} parses the command to {nova volume-list} to
+        create a C{dict} of volume information. When no C{instance_id}s are
+        present the volumes are not attached so L{nova_volume_show} is not
+        called to provide attachment details.
+        """
+        command = "nova volume-list"
+        output = (
+            "+-------------------------------------+\n"
+            "| ID | Status | Display Name | Size | Volume Type| Attached to|\n"
+            "+--------------------------------------------------------+\n"
+            "| 123-123-123 | available | None           | 10 | None | |\n"
+            "| 456-456-456 | available | my volume name | 8  | None | |\n"
+            "+---------------------+----------------------------------+\n")
+
+        nova_list = self.mocker.replace(subprocess.check_output)
+        nova_list(command, shell=True)
+        self.mocker.result(output)
+        self.mocker.replay()
+
+        def mock_nova_show(my_id):
+            raise Exception("_nova_volume_show should not be called")
+        self.storage._nova_volume_show = mock_nova_show
+
+        expected = {"123-123-123": {"id": "123-123-123", "status": "available",
+                                    "volume_label": "", "size": "10",
+                                    "instance_id": "",
+                                    "tags": {"volume_name": ""}},
+                    "456-456-456": {"id": "456-456-456", "status": "available",
+                                    "volume_label": "my volume name",
+                                    "size": "8", "instance_id": "",
+                                    "tags": {"volume_name": "my volume name"}}}
+        self.assertEqual(self.storage._nova_describe_volumes(), expected)
+
+    def test_wb_nova_describe_volumes_matches_volume_id_supplied(self):
+        """
+        L{_nova_describe_volumes} parses the command to {nova volume-list} to
+        create a C{dict} of volume information. When C{volume_id} is provided
+        return a C{dict} for the matched volume.
+        """
+        command = "nova volume-list"
+        volume_id = "123-123-123"
+        output = (
+            "+-------------------------------------+\n"
+            "| ID | Status | Display Name | Size | Volume Type| Attached to|\n"
+            "+--------------------------------------------------------+\n"
+            "| %s          | available | None           | 10 | None | |\n"
+            "| 456-456-456 | available | my volume name | 8  | None | |\n"
+            "+---------------------+----------------------------------+\n" %
+            volume_id)
+
+        nova_list = self.mocker.replace(subprocess.check_output)
+        nova_list(command, shell=True)
+        self.mocker.result(output)
+        self.mocker.replay()
+
+        expected = {"id": "123-123-123", "status": "available",
+                    "volume_label": "", "size": "10",
+                    "instance_id": "", "tags": {"volume_name": ""}}
+        self.assertEqual(
+            self.storage._nova_describe_volumes(volume_id), expected)
+
+    def test_wb_nova_describe_volumes_unmatched_volume_id_supplied(self):
+        """
+        L{_nova_describe_volumes} parses the command to {nova volume-list} to
+        create a C{dict} of volume information. Return and empty C{dict} when
+        to volume matches the provided C{volume_id}.
+        """
+        command = "nova volume-list"
+        volume_id = "123-123-123"
+        output = (
+            "+-------------------------------------+\n"
+            "| ID | Status | Display Name | Size | Volume Type| Attached to|\n"
+            "+--------------------------------------------------------+\n"
+            "| 789-789-789 | available | None           | 10 | None | |\n"
+            "| 456-456-456 | available | my volume name | 8  | None | |\n"
+            "+---------------------+----------------------------------+\n")
+
+        nova_list = self.mocker.replace(subprocess.check_output)
+        nova_list(command, shell=True)
+        self.mocker.result(output)
+        self.mocker.replay()
+
+        self.assertEqual(
+            self.storage._nova_describe_volumes(volume_id), {})
+
+    def test_wb_nova_describe_volumes_with_attached_instances(self):
+        """
+        L{_nova_describe_volumes} parses the command to {nova volume-list} to
+        create a C{dict} of volume information. When C{instance_id}s are
+        present the volumes are attached and L{nova_volume_show} will be called
+        to provide attachment details.
+        """
+        command = "nova volume-list"
+        attached_volume_id = "456-456-456"
+        output = (
+            "+-------------------------------------+\n"
+            "| ID | Status | Display Name | Size | Volume Type| Attached to|\n"
+            "+------------------------------------------------------------+\n"
+            "| 123-123-123 | available | None    | 10 | None |          | |\n"
+            "| %s          | in-use    | my name | 8  | None | i-789789 | |\n"
+            "+---------------------+--------------------------------------+\n"
+            % attached_volume_id)
+
+        nova_list = self.mocker.replace(subprocess.check_output)
+        nova_list(command, shell=True)
+        self.mocker.result(output)
+        self.mocker.replay()
+
+        def mock_nova_show(my_id):
+            self.assertEqual(my_id, attached_volume_id)
+            return {
+                "volume_label": "my name", "tags": {"volume_name": "my name"},
+                "instance_id": "i-789789", "device": "/dev/vdx",
+                "id": attached_volume_id, "status": "in-use",
+                "availability_zone": "nova", "size": "8",
+                "snapshot_id": "blah"}
+        self.storage._nova_volume_show = mock_nova_show
+
+        expected = {"123-123-123": {"id": "123-123-123", "status": "available",
+                                    "volume_label": "", "size": "10",
+                                    "instance_id": "",
+                                    "tags": {"volume_name": ""}},
+                    "456-456-456": {"id": "456-456-456", "status": "in-use",
+                                    "volume_label": "my name",
+                                    "size": "8", "instance_id": "i-789789",
+                                    "snapshot_id": "blah",
+                                    "device": "/dev/vdx",
+                                    "availability_zone": "nova",
+                                    "tags": {"volume_name": "my name"}}}
+        self.assertEqual(self.storage._nova_describe_volumes(), expected)
+
+    def test_wb_nova_create_volume(self):
+        """
+        L{_nova_create_volume} uses the command {nova volume-create} and
+        logs its action.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+        command = (
+            "nova volume-create --display-name '%s' %s" % (volume_label, size))
+
+        create = self.mocker.replace(subprocess.check_call)
+        create(command, shell=True)
+        self.mocker.replay()
+
+        self.storage.get_volume_id = lambda label: volume_id
+
+        self.assertEqual(
+            self.storage._nova_create_volume(size, volume_label, instance_id),
+            volume_id)
+        message = (
+            "Creating a %sGig volume named (%s) for instance %s" %
+            (size, volume_label, instance_id))
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_wb_nova_create_volume_error_volume_not_created(self):
+        """
+        L{_nova_create_volume} will log an error and exit when unable to find
+        the volume it just created.
+        """
+        instance_id = "i-123123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+        command = (
+            "nova volume-create --display-name '%s' %s" % (volume_label, size))
+
+        create = self.mocker.replace(subprocess.check_call)
+        create(command, shell=True)
+        self.mocker.replay()
+        self.storage.get_volume_id = lambda label: None
+
+        result = self.assertRaises(
+            SystemExit, self.storage._nova_create_volume, size, volume_label,
+            instance_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Couldn't find newly created nova volume '%s'." %
+            volume_label)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_nova_create_volume_error_command_failed(self):
+        """
+        L{_nova_create_volume} will log an error and exit when
+        C{nova create-volume} command fails.
+        """
+        instance_id = "i-123123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+        command = (
+            "nova volume-create --display-name '%s' %s" % (volume_label, size))
+
+        create = self.mocker.replace(subprocess.check_call)
+        create(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._nova_create_volume, size, volume_label,
+            instance_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Command '%s' returned non-zero exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_nova_attach_volume(self):
+        """
+        L{_nova_attach_volume} uses the command to {nova volume-attach} and
+        returns the attached volume path.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        command = (
+            "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" %
+            (instance_id, volume_id))
+
+        attach = self.mocker.replace(subprocess.check_output)
+        attach(command, shell=True)
+        self.mocker.result("/dev/vdz\n")
+        self.mocker.replay()
+
+        self.assertEqual(
+            self.storage._nova_attach_volume(instance_id, volume_id),
+            "/dev/vdz")
+
+    def test_wb_nova_attach_volume_no_device_path(self):
+        """
+        L{_nova_attach_volume} uses the command to {nova volume-attach} and
+        returns an empty C{str} if the attached volume path was not discovered.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        command = (
+            "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" %
+            (instance_id, volume_id))
+
+        attach = self.mocker.replace(subprocess.check_output)
+        attach(command, shell=True)
+        self.mocker.result("\n")
+        self.mocker.replay()
+
+        self.assertEqual(
+            self.storage._nova_attach_volume(instance_id, volume_id),
+            "")
+
+    def test_wb_nova_attach_volume_command_error(self):
+        """
+        L{_nova_attach_volume} will exit in error when the
+        C{nova volume-attach} command fails.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        command = (
+            "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" %
+            (instance_id, volume_id))
+        attach = self.mocker.replace(subprocess.check_output)
+        attach(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._nova_attach_volume, instance_id,
+            volume_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Command '%s' returned non-zero exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_detach_volume_no_volume_found(self):
+        """
+        When L{get_volume_id} is unable to find an attached volume and returns
+        C{None}, L{detach_volume} will log a message and perform no work.
+        """
+        volume_label = "postgresql/0 unit volume"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, "postgresql/0 unit volume")
+            return None
+        self.storage.get_volume_id = mock_get_volume_id
+
+        self.storage.detach_volume(volume_label)
+        message = "Cannot find volume name to detach, done"
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_detach_volume_volume_already_detached(self):
+        """
+        When L{get_volume_id} finds a volume that is already C{available} it
+        logs that the volume is already detached and does no work.
+        """
+        volume_label = "mycharm/1 unit volume"
+        volume_id = "123-123-123"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "available"}
+        self.storage.describe_volumes = mock_describe_volumes
+
+        self.storage.detach_volume(volume_label)  # pass in our volume_label
+        message = "Volume (%s) already detached. Done" % volume_id
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_detach_volume_command_error(self):
+        """
+        When the C{nova volume-detach} command fails, L{detach_volume} will
+        log a message and exit in error.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        volume_label = "mycharm/1 unit volume"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "instance_id": instance_id}
+        self.storage.describe_volumes = mock_describe_volumes
+
+        command = "nova volume-detach %s %s" % (instance_id, volume_id)
+        nova_cmd = self.mocker.replace(subprocess.check_call)
+        nova_cmd(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage.detach_volume, volume_label)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Couldn't detach volume. Command '%s' "
+            "returned non-zero exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_detach_volume(self):
+        """
+        When L{get_volume_id} finds a volume associated with this instance
+        which has a volume state not equal to C{available}, it detaches that
+        volume using nova commands.
+        """
+        volume_id = "123-123-123"
+        instance_id = "i-123123"
+        volume_label = "postgresql/0 unit volume"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "instance_id": instance_id}
+        self.storage.describe_volumes = mock_describe_volumes
+
+        command = "nova volume-detach %s %s" % (instance_id, volume_id)
+        nova_cmd = self.mocker.replace(subprocess.check_call)
+        nova_cmd(command, shell=True)
+        self.mocker.replay()
+
+        self.storage.detach_volume(volume_label)
+        message = (
+            "Detaching volume (%s) from instance %s" %
+            (volume_id, instance_id))
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+
+class MockEucaCommand(object):
+    def __init__(self, result):
+        self.result = result
+
+    def main(self):
+        return self.result
+
+
+class MockEucaReservation(object):
+    def __init__(self, instances):
+        self.instances = instances
+        self.id = 1
+
+
+class MockEucaInstance(object):
+    def __init__(self, instance_id=None, ip_address=None, image_id=None,
+                 instance_type=None, kernel=None, private_dns_name=None,
+                 public_dns_name=None, state=None, tags=[],
+                 availability_zone=None):
+        self.id = instance_id
+        self.ip_address = ip_address
+        self.image_id = image_id
+        self.instance_type = instance_type
+        self.kernel = kernel
+        self.private_dns_name = private_dns_name
+        self.public_dns_name = public_dns_name
+        self.state = state
+        self.tags = tags
+        self.placement = availability_zone
+
+
+class MockAttachData(object):
+    def __init__(self, device, instance_id):
+        self.device = device
+        self.instance_id = instance_id
+
+
+class MockVolume(object):
+    def __init__(self, vol_id, device, instance_id, zone, size, status,
+                 snapshot_id, tags):
+        self.id = vol_id
+        self.attach_data = MockAttachData(device, instance_id)
+        self.zone = zone
+        self.size = size
+        self.status = status
+        self.snapshot_id = snapshot_id
+        self.tags = tags
+
+
+class TestEC2Util(mocker.MockerTestCase):
+
+    def setUp(self):
+        super(TestEC2Util, self).setUp()
+        self.maxDiff = None
+        util.hookenv = TestHookenv(
+            {"key": "ec2key", "secret": "ec2password",
+             "endpoint": "https://ec2-region-url:443/v2.0/",
+             "default_volume_size": 11})
+        util.log = util.hookenv.log
+        self.storage = StorageServiceUtil("ec2")
+
+    def test_load_environment_with_ec2_variables(self):
+        """
+        L{load_environment} will setup script environment variables for ec2
+        by mapping configuration values provided to openstack OS_* environment
+        variables and then call L{validate_credentials} to assert
+        that environment variables provided give access to the service.
+        """
+        self.addCleanup(setattr, util.os, "environ", util.os.environ)
+        util.os.environ = {}
+
+        def mock_validate():
+            pass
+        self.storage.validate_credentials = mock_validate
+
+        self.storage.load_environment()
+        expected = {
+            "EC2_ACCESS_KEY": "ec2key",
+            "EC2_SECRET_KEY": "ec2password",
+            "EC2_URL": "https://ec2-region-url:443/v2.0/"
+        }
+        self.assertEqual(util.os.environ, expected)
+
+    def test_load_environment_error_missing_config_options(self):
+        """
+        L{load_environment} will exit in failure and log a message if any
+        required configuration option is not set.
+        """
+        self.addCleanup(setattr, util.os, "environ", util.os.environ)
+
+        def mock_validate():
+            raise SystemExit("something invalid")
+        self.storage.validate_credentials = mock_validate
+
+        self.assertRaises(SystemExit, self.storage.load_environment)
+
+    def test_validate_credentials_failure(self):
+        """
+        L{validate_credentials} will attempt a simple euca command to ensure
+        the environment is properly configured to access the nova service.
+        Upon failure to contact the nova service, L{validate_credentials} will
+        exit in error and log a message.
+        """
+        command = "euca-describe-instances"
+        nova_cmd = self.mocker.replace(subprocess.check_call)
+        nova_cmd(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage.validate_credentials)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Charm configured credentials can't access endpoint. "
+            "Command '%s' returned non-zero exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_validate_credentials(self):
+        """
+        L{validate_credentials} will succeed when a simple euca command
+        succeeds due to a properly configured environment based on the charm
+        configuration options.
+        """
+        command = "euca-describe-instances"
+        nova_cmd = self.mocker.replace(subprocess.check_call)
+        nova_cmd(command, shell=True)
+        self.mocker.replay()
+
+        self.storage.validate_credentials()
+        message = (
+            "Validated charm configuration credentials have access to "
+            "block storage service"
+        )
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_get_volume_id_by_volume_name(self):
+        """
+        L{get_volume_id} provided with a existing C{volume_name} returns the
+        corresponding ec2 volume id from L{_ec2_describe_volumes}.
+        """
+        volume_name = "my-volume"
+        volume_id = "12312412-412312"
+
+        def mock_describe(val):
+            self.assertIsNone(val)
+            return {volume_id: {"tags": {"volume_name": volume_name}},
+                    "456456-456456": {"tags": {"volume_name": "blah"}}}
+        self.storage._ec2_describe_volumes = mock_describe
+
+        self.assertEqual(self.storage.get_volume_id(volume_name), volume_id)
+
+    def test_get_volume_id_without_volume_name(self):
+        """
+        L{get_volume_id} without a provided C{volume_name} will discover the
+        nova volume id by searching L{_ec2_describe_volumes} for volumes
+        labelled with the os.environ[JUJU_REMOTE_UNIT].
+        """
+        unit_name = "postgresql/0"
+        self.addCleanup(
+            setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        volume_id = "123134124-1241412-1242141"
+
+        def mock_describe(val):
+            self.assertIsNone(val)
+            return {volume_id:
+                    {"tags": {"volume_name": "postgresql/0 unit volume"}},
+                    "456456-456456": {"tags": {"volume_name": "blah"}}}
+        self.storage._ec2_describe_volumes = mock_describe
+
+        self.assertEqual(self.storage.get_volume_id(), volume_id)
+
+    def test_get_volume_id_without_volume_name_no_matching_volume(self):
+        """
+        L{get_volume_id} without a provided C{volume_name} will return C{None}
+        when it cannot find a matching volume label from
+        L{_ec2_describe_volumes} for the os.environ[JUJU_REMOTE_UNIT].
+        """
+        unit_name = "postgresql/0"
+        self.addCleanup(
+            setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        def mock_describe(val):
+            self.assertIsNone(val)
+            return {"123123-123123":
+                    {"tags": {"volume_name": "postgresql/1 unit volume"}},
+                    "456456-456456": {"tags": {"volume_name": "blah"}}}
+        self.storage._ec2_describe_volumes = mock_describe
+
+        self.assertIsNone(self.storage.get_volume_id())
+
+    def test_get_volume_id_without_volume_name_multiple_matching_volumes(self):
+        """
+        L{get_volume_id} does not support multiple volumes associated with the
+        the instance represented by os.environ[JUJU_REMOTE_UNIT]. When
+        C{volume_name} is not specified and L{_ec2_describe_volumes} returns
+        multiple results the function exits with an error.
+        """
+        unit_name = "postgresql/0"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        def mock_describe(val):
+            self.assertIsNone(val)
+            return {"123123-123123":
+                    {"tags": {"volume_name": "postgresql/0 unit volume"}},
+                    "456456-456456":
+                    {"tags": {"volume_name": "unit postgresql/0 volume2"}}}
+        self.storage._ec2_describe_volumes = mock_describe
+
+        result = self.assertRaises(SystemExit, self.storage.get_volume_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "Error: Multiple volumes are associated with %s. "
+            "Cannot get_volume_id." % unit_name)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_attach_volume_failure_when_volume_id_does_not_exist(self):
+        """
+        When L{attach_volume} is provided a C{volume_id} that doesn't
+        exist, it logs an error and exits.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.storage.load_environment = lambda: None
+        self.storage._ec2_describe_volumes = lambda volume_id: {}
+
+        result = self.assertRaises(
+            SystemExit, self.storage.attach_volume, instance_id=instance_id,
+            volume_id=volume_id)
+        self.assertEqual(result.code, 1)
+        message = ("Requested volume-id (%s) does not exist. Unable to "
+                   "associate storage with %s" % (volume_id, unit_name))
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_attach_volume_without_volume_label(self):
+        """
+        L{attach_volume} without a provided C{volume_label} or C{volume_id}
+        will discover the nova volume id by searching L{_ec2_describe_volumes}
+        for volumes with a label based on the os.environ[JUJU_REMOTE_UNIT].
+        """
+        unit_name = "postgresql/0"
+        volume_id = "123-123-123"
+        instance_id = "i-123123123"
+        volume_label = "%s unit volume" % unit_name
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "device": "/dev/vdc"}
+        self.storage._ec2_describe_volumes = mock_describe_volumes
+
+        self.assertEqual(self.storage.attach_volume(instance_id), "/dev/vdc")
+        message = (
+            "Attaching %s (%s)" % (volume_label, volume_id))
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_attach_volume_when_volume_id_already_attached(self):
+        """
+        When L{attach_volume} is provided a C{volume_id} that already
+        has the state C{in-use} it logs that the volume is already attached
+        and returns.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.storage.load_environment = lambda: None
+
+        def mock_describe(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "device": "/dev/vdc"}
+        self.storage._ec2_describe_volumes = mock_describe
+
+        self.assertEqual(
+            self.storage.attach_volume(instance_id, volume_id), "/dev/vdc")
+
+        message = "Volume %s already attached. Done" % volume_id
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_attach_volume_failure_with_volume_unsupported_status(self):
+        """
+        When L{attach_volume} is provided a C{volume_id} that has an
+        unsupported status. It logs the error and exits.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.storage.load_environment = lambda: None
+
+        def mock_describe(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "deleting", "device": "/dev/vdc"}
+        self.storage._ec2_describe_volumes = mock_describe
+
+        result = self.assertRaises(
+            SystemExit, self.storage.attach_volume, instance_id, volume_id)
+        self.assertEqual(result.code, 1)
+        message = ("Cannot attach volume. "
+                   "Volume has unsupported status: deleting")
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_attach_volume_creates_with_config_size(self):
+        """
+        When C{volume_id} is C{None}, L{attach_volume} will create a new
+        volume with the configured C{default_volume_size} when the volume
+        doesn't exist and C{size} is not provided.
+        """
+        unit_name = "postgresql/0"
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        volume_label = "%s unit volume" % unit_name
+        default_volume_size = util.hookenv.config("default_volume_size")
+        self.addCleanup(setattr, os, "environ", os.environ)
+        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+
+        self.storage.load_environment = lambda: None
+        self.storage.get_volume_id = lambda _: None
+
+        def mock_describe(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "device": "/dev/vdc"}
+        self.storage._ec2_describe_volumes = mock_describe
+
+        def mock_ec2_create(size, label, instance):
+            self.assertEqual(size, default_volume_size)
+            self.assertEqual(label, volume_label)
+            self.assertEqual(instance, instance_id)
+            return volume_id
+        self.storage._ec2_create_volume = mock_ec2_create
+
+        self.assertEqual(self.storage.attach_volume(instance_id), "/dev/vdc")
+        message = "Attaching %s (%s)" % (volume_label, volume_id)
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_wb_ec2_describe_volumes_command_error(self):
+        """
+        L{_ec2_describe_volumes} will exit in error when the euca2ools
+        C{DescribeVolumes} command fails.
+        """
+        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
+        euca_command()
+        self.mocker.throw(SystemExit(1))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_describe_volumes)
+        self.assertEqual(result.code, 1)
+        message = "ERROR: Couldn't contact EC2 using euca-describe-volumes"
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_describe_volumes_without_attached_instances(self):
+        """
+        L{_ec2_describe_volumes} parses the results of euca2ools
+        C{DescribeVolumes} to create a C{dict} of volume information. When no
+        C{instance_id}s are present the volumes are not attached so no
+        C{device} or C{instance_id} information will be present.
+        """
+        volume1 = MockVolume(
+            "123-123-123", device="/dev/notshown", instance_id="notseen",
+            zone="ec2-az1", size="10", status="available",
+            snapshot_id="some-shot", tags={})
+        volume2 = MockVolume(
+            "456-456-456", device="/dev/notshown", instance_id="notseen",
+            zone="ec2-az2", size="8", status="available",
+            snapshot_id="some-shot", tags={"volume_name": "my volume name"})
+        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([volume1, volume2]))
+        self.mocker.replay()
+
+        expected = {"123-123-123": {"id": "123-123-123", "status": "available",
+                                    "device": "",
+                                    "availability_zone": "ec2-az1",
+                                    "volume_label": "", "size": "10",
+                                    "instance_id": "",
+                                    "snapshot_id": "some-shot",
+                                    "tags": {"volume_name": ""}},
+                    "456-456-456": {"id": "456-456-456", "status": "available",
+                                    "device": "",
+                                    "availability_zone": "ec2-az2",
+                                    "volume_label": "my volume name",
+                                    "size": "8", "instance_id": "",
+                                    "snapshot_id": "some-shot",
+                                    "tags": {"volume_name": "my volume name"}}}
+        self.assertEqual(self.storage._ec2_describe_volumes(), expected)
+
+    def test_wb_ec2_describe_volumes_matches_volume_id_supplied(self):
+        """
+        L{_ec2_describe_volumes} parses the results of euca2ools
+        C{DescribeVolumes} to create a C{dict} of volume information.
+        When C{volume_id} is provided return a C{dict} for the matched volume.
+        """
+        volume_id = "123-123-123"
+        volume1 = MockVolume(
+            volume_id, device="/dev/notshown", instance_id="notseen",
+            zone="ec2-az1", size="10", status="available",
+            snapshot_id="some-shot", tags={})
+        volume2 = MockVolume(
+            "456-456-456", device="/dev/notshown", instance_id="notseen",
+            zone="ec2-az2", size="8", status="available",
+            snapshot_id="some-shot", tags={"volume_name": "my volume name"})
+        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([volume1, volume2]))
+        self.mocker.replay()
+
+        expected = {
+            "id": volume_id, "status": "available", "device": "",
+            "availability_zone": "ec2-az1", "volume_label": "", "size": "10",
+            "instance_id": "", "snapshot_id": "some-shot",
+            "tags": {"volume_name": ""}}
+        self.assertEqual(
+            self.storage._ec2_describe_volumes(volume_id), expected)
+
+    def test_wb_ec2_describe_volumes_unmatched_volume_id_supplied(self):
+        """
+        L{_ec2_describe_volumes} parses the results of euca2ools
+        C{DescribeVolumes} to create a C{dict} of volume information.
+        When C{volume_id} is provided and unmatched, return an empty C{dict}.
+        """
+        unmatched_volume_id = "456-456-456"
+        volume1 = MockVolume(
+            "123-123-123", device="/dev/notshown", instance_id="notseen",
+            zone="ec2-az1", size="10", status="available",
+            snapshot_id="some-shot", tags={})
+        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([volume1]))
+        self.mocker.replay()
+
+        self.assertEqual(
+            self.storage._ec2_describe_volumes(unmatched_volume_id), {})
+
+    def test_wb_ec2_describe_volumes_with_attached_instances(self):
+        """
+        L{_ec2_describe_volumes} parses the results of euca2ools
+        C{DescribeVolumes} to create a C{dict} of volume information. If
+        C{status} is C{in-use}, both C{device} and C{instance_id} will be
+        returned in the C{dict}.
+        """
+        volume1 = MockVolume(
+            "123-123-123", device="/dev/notshown", instance_id="notseen",
+            zone="ec2-az1", size="10", status="available",
+            snapshot_id="some-shot", tags={})
+        volume2 = MockVolume(
+            "456-456-456", device="/dev/xvdc", instance_id="i-456456",
+            zone="ec2-az2", size="8", status="in-use",
+            snapshot_id="some-shot", tags={"volume_name": "my volume name"})
+        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([volume1, volume2]))
+        self.mocker.replay()
+
+        expected = {"123-123-123": {"id": "123-123-123", "status": "available",
+                                    "device": "",
+                                    "availability_zone": "ec2-az1",
+                                    "volume_label": "", "size": "10",
+                                    "instance_id": "",
+                                    "snapshot_id": "some-shot",
+                                    "tags": {"volume_name": ""}},
+                    "456-456-456": {"id": "456-456-456", "status": "in-use",
+                                    "device": "/dev/xvdc",
+                                    "availability_zone": "ec2-az2",
+                                    "volume_label": "my volume name",
+                                    "size": "8", "instance_id": "i-456456",
+                                    "snapshot_id": "some-shot",
+                                    "tags": {"volume_name": "my volume name"}}}
+        self.assertEqual(
+            self.storage._ec2_describe_volumes(), expected)
+
+    def test_wb_ec2_create_volume(self):
+        """
+        L{_ec2_create_volume} uses the command C{euca-create-volume} to create
+        a volume. It determines the availability zone for the volume by
+        querying L{_ec2_describe_instances} on the provided C{instance_id}
+        to ensure it matches the same availablity zone. It will then call
+        L{_ec2_create_tag} to setup the C{volume_name} tag for the volume.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+        zone = "ec2-az3"
+        command = "euca-create-volume -z %s -s %s" % (zone, size)
+
+        reservation = MockEucaReservation(
+            [MockEucaInstance(
+                instance_id=instance_id, availability_zone=zone)])
+        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([reservation]))
+        create = self.mocker.replace(subprocess.check_output)
+        create(command, shell=True)
+        self.mocker.result(
+            "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id)
+        self.mocker.replay()
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"id": volume_id}
+        self.storage._ec2_describe_volumes = mock_describe_volumes
+
+        def mock_create_tag(my_id, key, value):
+            self.assertEqual(my_id, volume_id)
+            self.assertEqual(key, "volume_name")
+            self.assertEqual(value, volume_label)
+        self.storage._ec2_create_tag = mock_create_tag
+
+        self.assertEqual(
+            self.storage._ec2_create_volume(size, volume_label, instance_id),
+            volume_id)
+        message = (
+            "Creating a %sGig volume named (%s) for instance %s" %
+            (size, volume_label, instance_id))
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_wb_ec2_create_volume_error_different_region_configured(self):
+        """
+        When L{_ec2_create_volume} received an C{instance_id} which is not
+        present in the current output of C{ec2_describe_instances} it is likely
+        that the region for which the charm was configured is not the region in
+        which the block-storage-broker is deployed. Log an error suggesting
+        the misconfiguration of the charm in this case.
+        """
+        instance_id = "i-123123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+
+        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([]))  # Empty results from euca
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_create_volume, size, volume_label,
+            instance_id)
+        self.assertEqual(result.code, 1)
+        config = util.hookenv.config_get()
+        message = (
+            "ERROR: Could not create volume for instance %s. No instance "
+            "details discovered by euca-describe-instances. Maybe the "
+            "charm configured endpoint %s is not valid for this region." %
+            (instance_id, config["endpoint"]))
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_create_volume_error_invalid_response_type(self):
+        """
+        L{_ec2_create_volume} will log an error and exit when it receives an
+        unparseable response type from the C{euca-create-volume} command.
+        """
+        instance_id = "i-123123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+        zone = "ec2-az3"
+        command = "euca-create-volume -z %s -s %s" % (zone, size)
+
+        reservation = MockEucaReservation(
+            [MockEucaInstance(
+                instance_id=instance_id, availability_zone=zone)])
+        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([reservation]))
+        create = self.mocker.replace(subprocess.check_output)
+        create(command, shell=True)
+        self.mocker.result("INSTANCE invalid-instance-type-response\n")
+        self.mocker.replay()
+
+        def mock_describe_volumes(my_id):
+            raise Exception("_ec2_describe_volumes should not be called")
+        self.storage._ec2_describe_volumes = mock_describe_volumes
+
+        def mock_create_tag(my_id, key, value):
+            raise Exception("_ec2_create_tag should not be called")
+        self.storage._ec2_create_tag = mock_create_tag
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_create_volume, size, volume_label,
+            instance_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Didn't get VOLUME response from euca-create-volume. "
+            "Response: INSTANCE invalid-instance-type-response\n")
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_create_volume_error_new_volume_not_found(self):
+        """
+        L{_ec2_create_volume} will log an error and exit when it cannot find
+        details of the newly created C{volume_id} through a subsequent call to
+        L{_ec2_descibe_volumes}.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+        zone = "ec2-az3"
+        command = "euca-create-volume -z %s -s %s" % (zone, size)
+
+        reservation = MockEucaReservation(
+            [MockEucaInstance(
+                instance_id=instance_id, availability_zone=zone)])
+        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([reservation]))
+        create = self.mocker.replace(subprocess.check_output)
+        create(command, shell=True)
+        self.mocker.result(
+            "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id)
+        self.mocker.replay()
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {}  # No details found for this volume
+        self.storage._ec2_describe_volumes = mock_describe_volumes
+
+        def mock_create_tag(my_id, key, value):
+            raise Exception("_ec2_create_tag should not be called")
+        self.storage._ec2_create_tag = mock_create_tag
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_create_volume, size, volume_label,
+            instance_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Unable to find volume '%s'" % volume_id)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_create_volume_error_command_failed(self):
+        """
+        L{_ec2_create_volume} will log an error and exit when the
+        C{euca-create-volume} fails.
+        """
+        instance_id = "i-123123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+        zone = "ec2-az3"
+        command = "euca-create-volume -z %s -s %s" % (zone, size)
+
+        reservation = MockEucaReservation(
+            [MockEucaInstance(
+                instance_id=instance_id, availability_zone=zone)])
+        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
+        euca_command()
+        self.mocker.result(MockEucaCommand([reservation]))
+        create = self.mocker.replace(subprocess.check_output)
+        create(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        def mock_exception(my_id):
+            raise Exception("These methods should not be called")
+        self.storage._ec2_describe_volumes = mock_exception
+        self.storage._ec2_create_tag = mock_exception
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_create_volume, size, volume_label,
+            instance_id)
+        self.assertEqual(result.code, 1)
+
+        message = (
+            "ERROR: Command '%s' returned non-zero exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_attach_volume(self):
+        """
+        L{_ec2_attach_volume} uses the command C{euca-attach-volume} and
+        returns the attached volume path.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        device = "/dev/xvdc"
+        command = (
+            "euca-attach-volume -i %s -d %s %s" %
+            (instance_id, device, volume_id))
+
+        attach = self.mocker.replace(subprocess.check_call)
+        attach(command, shell=True)
+        self.mocker.replay()
+
+        self.assertEqual(
+            self.storage._ec2_attach_volume(instance_id, volume_id),
+            device)
+
+    def test_wb_ec2_attach_volume_command_failed(self):
+        """
+        L{_ec2_attach_volume} exits in error when C{euca-attach-volume} fails.
+        """
+        instance_id = "i-123123"
+        volume_id = "123-123-123"
+        device = "/dev/xvdc"
+        command = (
+            "euca-attach-volume -i %s -d %s %s" %
+            (instance_id, device, volume_id))
+
+        attach = self.mocker.replace(subprocess.check_call)
+        attach(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_attach_volume, instance_id,
+            volume_id)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Command '%s' returned non-zero exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_detach_volume_no_volume_found(self):
+        """
+        When L{get_volume_id} is unable to find an attached volume and returns
+        C{None}, L{detach_volume} will log a message and perform no work.
+        """
+        volume_label = "postgresql/0 unit volume"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return None
+        self.storage.get_volume_id = mock_get_volume_id
+
+        self.storage.detach_volume(volume_label)
+        message = "Cannot find volume name to detach, done"
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_detach_volume_volume_already_detached(self):
+        """
+        When L{get_volume_id} finds a volume that is already C{available} it
+        logs that the volume is already detached and does no work.
+        """
+        volume_label = "postgresql/0 unit volume"
+        volume_id = "123-123-123"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "available"}
+        self.storage.describe_volumes = mock_describe_volumes
+
+        self.storage.detach_volume(volume_label)
+        message = "Volume (%s) already detached. Done" % volume_id
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_detach_volume_command_error(self):
+        """
+        When the C{euca-detach-volume} command fails, L{detach_volume} will
+        log a message and exit in error.
+        """
+        volume_label = "postgresql/0 unit volume"
+        volume_id = "123-123-123"
+        instance_id = "i-123123"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "instance_id": instance_id}
+        self.storage.describe_volumes = mock_describe_volumes
+
+        command = "euca-detach-volume -i %s %s" % (instance_id, volume_id)
+        ec2_cmd = self.mocker.replace(subprocess.check_call)
+        ec2_cmd(command, shell=True)
+        self.mocker.throw(subprocess.CalledProcessError(1, command))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage.detach_volume, volume_label)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Couldn't detach volume. Command '%s' returned non-zero "
+            "exit status 1" % command)
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_detach_volume(self):
+        """
+        When L{get_volume_id} finds a volume associated with this instance
+        which has a volume state not equal to C{available}, it detaches that
+        volume using euca2ools commands.
+        """
+        volume_label = "postgresql/0 unit volume"
+        volume_id = "123-123-123"
+        instance_id = "i-123123"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "instance_id": instance_id}
+        self.storage.describe_volumes = mock_describe_volumes
+
+        command = "euca-detach-volume -i %s %s" % (instance_id, volume_id)
+        ec2_cmd = self.mocker.replace(subprocess.check_call)
+        ec2_cmd(command, shell=True)
+        self.mocker.replay()
+
+        self.storage.detach_volume(volume_label)
+        message = (
+            "Detaching volume (%s) from instance %s" %
+            (volume_id, instance_id))
+        self.assertIn(
+            message, util.hookenv._log_INFO, "Not logged- %s" % message)
 
=== added file 'hooks/util.py'
--- hooks/util.py	1970-01-01 00:00:00 +0000
+++ hooks/util.py	2014-03-21 22:07:32 +0000
@@ -0,0 +1,472 @@
+"""Common python utilities for the ec2 provider"""
+
+from charmhelpers.core import hookenv
+import subprocess
+import os
+import sys
+from time import sleep
+
+ENVIRONMENT_MAP = {
+    "ec2": {"endpoint": "EC2_URL", "key": "EC2_ACCESS_KEY",
+            "secret": "EC2_SECRET_KEY"},
+    "nova": {"endpoint": "OS_AUTH_URL", "region": "OS_REGION_NAME",
+             "tenant": "OS_TENANT_NAME", "key": "OS_USERNAME",
+             "secret": "OS_PASSWORD"}}
+
+REQUIRED_CONFIG_OPTIONS = {
+    "ec2": ["endpoint", "key", "secret"],
+    "nova": ["endpoint", "region", "tenant", "key", "secret"]}
+
+PROVIDER_COMMANDS = {
+    "ec2": {"validate": "euca-describe-instances",
+            "detach": "euca-detach-volume -i %s %s"},
+    "nova": {"validate": "nova list",
+             "detach": "nova volume-detach %s %s"}}
+
+
+class StorageServiceUtil(object):
+    """Interact with an underlying cloud storage provider.
+    Create, attach, label and detach storage volumes using EC2 or nova APIs.
+    """
+    provider = None
+    environment_map = None
+    required_config_options = None
+    commands = None
+
+    def __init__(self, provider):
+        self.provider = provider
+        if provider not in ENVIRONMENT_MAP:
+            hookenv.log(
+                "ERROR: Invalid charm configuration setting for provider. "
+                "'%s' must be one of: %s" %
+                (provider, ", ".join(ENVIRONMENT_MAP.keys())),
+                hookenv.ERROR)
+            sys.exit(1)
+        self.environment_map = ENVIRONMENT_MAP[provider]
+        self.commands = PROVIDER_COMMANDS[provider]
+        self.required_config_options = REQUIRED_CONFIG_OPTIONS[provider]
+        if provider == "ec2":
+            import euca2ools.commands.euca.describevolumes as getvolumes
+            import euca2ools.commands.euca.describeinstances as getinstances
+            self.ec2_volume_class = getvolumes.DescribeVolumes
+            self.ec2_instance_class = getinstances.DescribeInstances
+
+    def load_environment(self):
+        """
+        Source our credentials from the configuration definitions into our
+        operating environment
+        """
+        config_data = hookenv.config()
+        for option in self.required_config_options:
+            environment_variable = self.environment_map[option]
+            os.environ[environment_variable] = config_data[option].strip()
+        self.validate_credentials()
+
+    def validate_credentials(self):
+        """
+        Attempt to contact the respective ec2 or nova volume service or exit(1)
+        """
+        try:
+            subprocess.check_call(self.commands["validate"], shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log(
+                "ERROR: Charm configured credentials can't access endpoint. "
+                "%s" % str(e),
+                hookenv.ERROR)
+            sys.exit(1)
+        hookenv.log(
+            "Validated charm configuration credentials have access to block "
+            "storage service")
+
+    def describe_volumes(self, volume_id=None):
+        method = getattr(self, "_%s_describe_volumes" % self.provider)
+        return method(volume_id)
+
+    def get_volume_id(self, volume_designation=None):
+        """Return the ec2 or nova volume id associated with this unit
+
+        Optionally, C{volume_designation} can be either a volume-id or
+        volume-display-name and the matching C{volume-id} will be returned.
+        If no matching volume is return C{None}.
+        """
+        matches = []
+        volumes = self.describe_volumes()
+        if volume_designation:
+            token = volume_designation
+        else:
+            # Try to find volume label containing remote_unit name
+            token = hookenv.remote_unit()
+        for volume_id in volumes.keys():
+            volume = volumes[volume_id]
+            # Get volume by name or volume-id
+            volume_name = volume["tags"].get("volume_name", "")
+            if token == volume_id:
+                matches.append(volume_id)
+            elif token in volume_name:
+                matches.append(volume_id)
+        if len(matches) > 1:
+            hookenv.log(
+                "Error: Multiple volumes are associated with "
+                "%s. Cannot get_volume_id." % token, hookenv.ERROR)
+            sys.exit(1)
+        elif matches:
+            return matches[0]
+        return None
+
+    def attach_volume(self, instance_id, volume_id=None, size=None,
+                      volume_label=None):
+        """
+        Create and attach a volume to the remote unit if none exists.
+
+        Attempt to attach and validate the attached volume 10
+        times. If unable to resolve the attach issues, exit in error and log
+        the error.
+        Log errors if the volume is in an unsupported state, and if C{in-use}
+        report it is already attached.
+
+        Return the device-path of the attached volume to the caller.
+        """
+        self.load_environment()   # Will fail if invalid environment
+        remote_unit = hookenv.remote_unit()
+        if volume_label is None:
+            volume_label = generate_volume_label(remote_unit)
+        if volume_id:
+            volume = self.describe_volumes(volume_id)
+            if not volume:
+                hookenv.log(
+                    "Requested volume-id (%s) does not exist. Unable to "
+                    "associate storage with %s" % (volume_id, remote_unit),
+                    hookenv.ERROR)
+                sys.exit(1)
+
+            # Validate that current volume status is supported
+            while volume["status"] == "attaching":
+                hookenv.log("Volume %s still attaching. Waiting." % volume_id)
+                sleep(5)
+                volume = self.describe_volumes(volume_id)
+
+            if volume["status"] == "in-use":
+                hookenv.log("Volume %s already attached. Done" % volume_id)
+                return volume["device"]  # The device path on the instance
+            if volume["status"] != "available":
+                hookenv.log(
+                    "Cannot attach volume. Volume has unsupported status: "
+                    "%s" % volume["status"], hookenv.ERROR)
+                sys.exit(1)
+        else:
+            # No volume_id, create a new volume if one isn't already created
+            # for the principal of this JUJU_REMOTE_UNIT
+            volume_id = self.get_volume_id(volume_label)
+            if not volume_id:
+                create = getattr(self, "_%s_create_volume" % self.provider)
+                if not size:
+                    size = hookenv.config("default_volume_size")
+                volume_id = create(size, volume_label, instance_id)
+
+        device = None
+        hookenv.log("Attaching %s (%s)" % (volume_label, volume_id))
+        for x in range(10):
+            volume = self.describe_volumes(volume_id)
+            if volume["status"] == "in-use":
+                return volume["device"]  # The device path on the instance
+            if volume["status"] == "available":
+                attach = getattr(self, "_%s_attach_volume" % self.provider)
+                device = attach(instance_id, volume_id)
+                break
+            else:
+                sleep(5)
+        if not device:
+            hookenv.log(
+                "ERROR: Unable to discover device attached by "
+                "euca-attach-volume",
+                hookenv.ERROR)
+            sys.exit(1)
+        return device
+
+    def detach_volume(self, volume_label):
+        """Detach a volume from remote unit if present"""
+        self.load_environment()   # Will fail if invalid environment
+        volume_id = self.get_volume_id(volume_label)
+
+        if volume_id:
+            volume = self.describe_volumes(volume_id)
+        else:
+            hookenv.log("Cannot find volume name to detach, done")
+            return
+
+        if volume["status"] == "available":
+            hookenv.log("Volume (%s) already detached. Done" % volume_id)
+            return
+
+        hookenv.log(
+            "Detaching volume (%s) from instance %s" %
+            (volume_id, volume["instance_id"]))
+        try:
+            subprocess.check_call(
+                self.commands["detach"] % (volume["instance_id"], volume_id),
+                shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log(
+                "ERROR: Couldn't detach volume. %s" % str(e), hookenv.ERROR)
+            sys.exit(1)
+        return
+
+    # EC2-specific methods
+    def _ec2_create_tag(self, volume_id, tag_name, tag_value=None):
+        """Attach a tag and optional C{tag_value} to the given C{volume_id}"""
+        tag_string = tag_name
+        if tag_value:
+            tag_string += "=%s" % tag_value
+        command = 'euca-create-tags %s --tag "%s"' % (volume_id, tag_string)
+
+        try:
+            subprocess.check_call(command, shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log(
+                "ERROR: Couldn't add tags to the resource. %s" % str(e),
+                hookenv.ERROR)
+            sys.exit(1)
+        hookenv.log("Tagged (%s) to %s." % (tag_string, volume_id))
+
+    def _ec2_describe_instances(self, instance_id=None):
+        """
+        Use euca2ools libraries to describe instances and return a C{dict}
+        """
+        result = {}
+        try:
+            command = self.ec2_instance_class()
+            reservations = command.main()
+        except SystemExit:
+            hookenv.log(
+                "ERROR: Couldn't contact EC2 using euca-describe-instances",
+                hookenv.ERROR)
+            sys.exit(1)
+        for reservation in reservations:
+            for inst in reservation.instances:
+                result[inst.id] = {
+                    "ip-address": inst.ip_address, "image-id": inst.image_id,
+                    "instance-type": inst.image_id, "kernel": inst.kernel,
+                    "private-dns-name": inst.private_dns_name,
+                    "public-dns-name": inst.public_dns_name,
+                    "reservation-id": reservation.id,
+                    "state": inst.state, "tags": inst.tags,
+                    "availability_zone": inst.placement}
+        if instance_id:
+            if instance_id in result:
+                return result[instance_id]
+            return {}
+        return result
+
+    def _ec2_describe_volumes(self, volume_id=None):
+        """
+        Use euca2ools libraries to describe volumes and return a C{dict}
+        """
+        result = {}
+        try:
+            command = self.ec2_volume_class()
+            volumes = command.main()
+        except SystemExit:
+            hookenv.log(
+                "ERROR: Couldn't contact EC2 using euca-describe-volumes",
+                hookenv.ERROR)
+            sys.exit(1)
+        for volume in volumes:
+            result[volume.id] = {
+                "device": "",
+                "instance_id": "",
+                "size": volume.size,
+                "snapshot_id": volume.snapshot_id,
+                "status": volume.status,
+                "tags": volume.tags,
+                "id": volume.id,
+                "availability_zone": volume.zone}
+            if "volume_name" in volume.tags:
+                result[volume.id]["volume_label"] = volume.tags["volume_name"]
+            else:
+                result[volume.id]["tags"]["volume_name"] = ""
+                result[volume.id]["volume_label"] = ""
+            if volume.status == "in-use":
+                result[volume.id]["instance_id"] = (
+                    volume.attach_data.instance_id)
+                result[volume.id]["device"] = volume.attach_data.device
+        if volume_id:
+            if volume_id in result:
+                return result[volume_id]
+            return {}
+        return result
+
+    def _ec2_create_volume(self, size, volume_label, instance_id):
+        """Create an EC2 volume with a specific C{size} and C{volume_label}"""
+        # Volumes need to be in the same zone as the instance
+        hookenv.log(
+            "Creating a %sGig volume named (%s) for instance %s" %
+            (size, volume_label, instance_id))
+        instance = self._ec2_describe_instances(instance_id)
+        if not instance:
+            config_data = hookenv.config()
+            hookenv.log(
+                "ERROR: Could not create volume for instance %s. No instance "
+                "details discovered by euca-describe-instances. Maybe the "
+                "charm configured endpoint %s is not valid for this region." %
+                (instance_id, config_data["endpoint"]), hookenv.ERROR)
+            sys.exit(1)
+
+        try:
+            output = subprocess.check_output(
+                "euca-create-volume -z %s -s %s" %
+                (instance["availability_zone"], size), shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
+            sys.exit(1)
+
+        response_type, volume_id = output.split()[:2]
+        if response_type != "VOLUME":
+            hookenv.log(
+                "ERROR: Didn't get VOLUME response from euca-create-volume. "
+                "Response: %s" % output, hookenv.ERROR)
+            sys.exit(1)
+        volume = self.describe_volumes(volume_id.strip())
+        if not volume:
+            hookenv.log(
+                "ERROR: Unable to find volume '%s'" % volume_id.strip(),
+                hookenv.ERROR)
+            sys.exit(1)
+        volume_id = volume["id"]
+        self._ec2_create_tag(volume_id, "volume_name", volume_label)
+        return volume_id
+
+    def _ec2_attach_volume(self, instance_id, volume_id):
+        """
+        Attach an EC2 C{volume_id} to the provided C{instance_id} and return
+        the device path.
+        """
+        device = "/dev/xvdc"
+        try:
+            subprocess.check_call(
+                "euca-attach-volume -i %s -d %s %s" %
+                (instance_id, device, volume_id), shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
+            sys.exit(1)
+        return device
+
+    # Nova-specific methods
+    def _nova_volume_show(self, volume_id):
+        """
+        Read detailed information about a C{volume_id} from nova-volume-show
+        and return a C{dict} of data we are interested in.
+        """
+        from ast import literal_eval
+        result = {"tags": {}, "instance_id": "", "device": ""}
+        command = "nova volume-show '%s'" % volume_id
+        try:
+            output = subprocess.check_output(command, shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log(
+                "ERROR: Failed to get nova volume info. %s" % str(e),
+                hookenv.ERROR)
+            sys.exit(1)
+        for line in output.split("\n"):
+            if not line.strip():  # Skip empty lines
+                continue
+            if "+----" in line or "Property" in line:
+                continue
+            (_, key, value, _) = line.split("|")
+            key = key.strip()
+            value = value.strip()
+            if key in ["availability_zone", "size", "id", "snapshot_id",
+                       "status"]:
+                result[key] = value
+            if key == "display_name":   # added for compatibility with ec2
+                result["volume_label"] = value
+                result["tags"]["volume_label"] = value
+            if key == "attachments":
+                attachments = literal_eval(value)
+                if attachments:
+                    for key, value in attachments[0].items():
+                        if key in ["device"]:
+                            result[key] = value
+                        if key == "server_id":
+                            result["instance_id"] = value
+        return result
+
+    def _nova_describe_volumes(self, volume_id=None):
+        """Create a C{dict} describing all nova volumes"""
+        result = {}
+        command = "nova volume-list"
+        try:
+            output = subprocess.check_output(command, shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
+            sys.exit(1)
+        for line in output.split("\n"):
+            if not line.strip():  # Skip empty lines
+                continue
+            if "+----" in line or "ID" in line:
+                continue
+            values = line.split("|")
+            if volume_id and values[1].strip() != volume_id:
+                continue
+            volume = values[1].strip()
+            volume_label = values[3].strip()
+            if volume_label == "None":
+                volume_label = ""
+            instance_id = values[6].strip()
+            if instance_id == "None":
+                instance_id = ""
+            result[volume] = {
+                "id": volume,
+                "tags": {"volume_name": volume_label},
+                "status": values[2].strip(),
+                "volume_label": volume_label,
+                "size": values[4].strip(),
+                "instance_id": instance_id}
+            if instance_id:
+                result[volume].update(self._nova_volume_show(volume))
+        if volume_id:
+            if volume_id in result:
+                return result[volume_id]
+            return {}
+        return result
+
+    def _nova_attach_volume(self, instance_id, volume_id):
+        """
+        Attach a Nova C{volume_id} to the provided C{instance_id} and return
+        the device path.
+        """
+        try:
+            device = subprocess.check_output(
+                "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" %
+                (instance_id, volume_id), shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
+            sys.exit(1)
+        if device.strip():
+            return device.strip()
+        return ""
+
+    def _nova_create_volume(self, size, volume_label, instance_id):
+        """Create an Nova volume with a specific C{size} and C{volume_label}"""
+        hookenv.log(
+            "Creating a %sGig volume named (%s) for instance %s" %
+            (size, volume_label, instance_id))
+        try:
+            subprocess.check_call(
+                "nova volume-create --display-name '%s' %s" %
+                (volume_label, size), shell=True)
+        except subprocess.CalledProcessError, e:
+            hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
+            sys.exit(1)
+
+        volume_id = self.get_volume_id(volume_label)
+        if not volume_id:
+            hookenv.log(
+                "ERROR: Couldn't find newly created nova volume '%s'." %
+                volume_label, hookenv.ERROR)
+            sys.exit(1)
+        return volume_id
+
+
+def generate_volume_label(remote_unit):
+    """Create a volume label for the requesting remote unit"""
+    return "%s unit volume" % remote_unit