Merge lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-boto into lp:charms/block-storage-broker

[Chad Smith]

This branch moves block-storage-broker's EC2-specific methods from euca2ools to AWS' python SDK (python-boto).

The reason for this change is to avoid significant incompatibilities in euca2ools libraries that have been introduced across euca2ools major releases. By consuming python-boto instead of internal euca2ools libraries, we access a more stable API supported by Amazon that will remain more stable across releases than internal euca libs. As written this code currently also has been tested on trusty and will be used as well as the trusty release of this charm.

This can be quickly tested on AWS either using precise or trusty by changing the postgresql-storage-bundle.cfg:
  - change all mention of precise to trusty
  - change postgresql-9.1-debversion to postgresql-9.3.debversion

Test procedure is something like the following:

1. Create postgresql-storage-bundle.cfg as exemplified below, replacing the EC2_* values with valid credentials and endpoints.

2. juju-bootstrap -e your-ec2-environment
# to deploy block-storage-broker, storage subordinate and postgresql and create and attach a volume.
3. juju-deployer -c postgresql-storage-bundle.cfg doit-no-volume

----- postgresql-storage-bundle.cfg -----
common:
    services:
        postgresql:
            branch: lp:~charmers/charms/precise/postgresql/trunk
            constraints: mem=2048
            options:
                extra-packages: python-apt postgresql-contrib postgresql-9.1-debversion
                max_connections: 500
        block-storage-broker:
            branch: lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-boto
            options:
                provider: ec2
                key: <your EC2_ACCESS_KEY>
                endpoint: <your EC2_URL>
                secret: <your EC2_SECRET_KEY>

doit-no-volume:
    inherits: common
    series: precise
    services:
        storage:
            branch: lp:~charmers/charms/precise/storage/trunk
            options:
                provider: block-storage-broker
                volume_size: 9
    relations:
        - [postgresql, storage]
        - [storage, block-storage-broker]

doit-with-volume-map:
    inherits: common
    series: precise
    services:
        storage:
            branch: lp:~charmers/charms/precise/storage/trunk
            options:
                provider: block-storage-broker
                volume_size: 9
                volume_map: "{postgresql/0: YOUR-EXISTING-EUCA-VOLUME-ID}"

    relations:
        - [postgresql, storage]
        - [storage, block-storage-broker]

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-boto]

60. By Chad Smith on 2014-08-19

mocker assert cloud-archive:havana is not added on trusty or later

61. By Chad Smith on 2014-08-19

fcorrea review comment. docstring update

[Fernando Correa Neto (fcorrea)]

Hey Chad, overall it looks great.

Just a few points inline.

review: Needs Fixing

[Chad Smith (chad.smith)]

Good deal Fernando thanks. I'll fix those comments right here with the exception of the isolation of ec2 from nova comment. We can handle that as a separate bug.

[Dean Henrichsmeyer (dean)]

Yeah, let's not do the re-factoring for now. I'd prefer to focus on product priorities and circle back to this another time.

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-boto]

62. By Chad Smith on 2014-08-22

add apt_install and add_source params to install hook for testing

63. By Chad Smith on 2014-08-22

pull ec2_url regex parsing and connect_to_region calls out of try/except block to simplify error handling

64. By Chad Smith on 2014-08-22

unit test updates to use add_source apt_update testing params. add unit test for installing python-boto for ec2 provider provider

65. By Chad Smith on 2014-08-22

unit test rename for whitebox testing

[Chad Smith (chad.smith)]

Fernando, I addressed your review comments on this branch with the exception of the re-factor. If you can create a bug/card for that we can work it when we have time after audit/history tasks

[Fernando Correa Neto (fcorrea)]

+1, Chad.
I filed a bug about the refactoring. Also, if you want to address the tests cleanup in a separate branch, I'm fine as it's really a cleanup.

review: Approve

[lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-boto]

66. By Chad Smith on 2014-08-22

consolidation of environment setup in a class method _set_environment_vars

[David Britton (dpb)]

Hi Chad -- Thanks for submitting this, I tested successfully on both precise and trusty, appreciate your attention to detail on it. Since this test already has unit tests will submit to both precise and trusty.

review: Approve

=== modified file 'hooks/hooks.py'
--- hooks/hooks.py	2014-07-18 00:58:58 +0000
+++ hooks/hooks.py	2014-08-22 13:42:09 +0000
@@ -1,6 +1,7 @@
 #!/usr/bin/env python
 # vim: et ai ts=4 sw=4:
 
+from charmhelpers import fetch
 from charmhelpers.core import hookenv
 from charmhelpers.core.hookenv import ERROR, INFO
 
@@ -77,18 +78,23 @@
 
 
 @hooks.hook()
-def install():
-    """Install required packages if not present"""
-    from charmhelpers import fetch
+def install(apt_install=None, add_source=None):
+    """Install required packages if not present."""
+
+    if apt_install is None:  # for testing purposes
+        apt_install = fetch.apt_install
+    if add_source is None:  # for testing purposes
+        add_source = fetch.add_source
+
     provider = hookenv.config("provider")
     if provider == "nova":
         required_packages = ["python-novaclient"]
         if int(get_running_series()['release'].split(".")[0]) < 14:
-            fetch.add_source("cloud-archive:havana")
+            add_source("cloud-archive:havana")
     elif provider == "ec2":
-        required_packages = ["euca2ools"]
+        required_packages = ["python-boto"]
     fetch.apt_update(fatal=True)
-    fetch.apt_install(required_packages, fatal=True)
+    apt_install(required_packages, fatal=True)
 
 
 @hooks.hook("block-storage-relation-departed")
 
=== modified file 'hooks/test_hooks.py'
--- hooks/test_hooks.py	2014-07-18 04:13:23 +0000
+++ hooks/test_hooks.py	2014-08-22 13:42:09 +0000
@@ -182,24 +182,27 @@
         self.mocker.replay()
         hooks.config_changed()
 
-    def test_install_installs_novaclient(self):
+    def test_install_installs_novaclient_without_cloud_archive(self):
         """
-        L{install} will call C{fetch.add_source} to add a cloud repository and
-        install the C{python-novaclient} package.
+        On releases C{trusty} and later L{install} will install the
+        python-novaclient package without installing the cloud-archive
+        repository.
         """
         get_running_series = self.mocker.replace(hooks.get_running_series)
         get_running_series()
-        self.mocker.result({'release': '19.04'})  # Not precise
+        self.mocker.result({'release': '14.04'})  # Trusty series
         add_source = self.mocker.replace(fetch.add_source)
-        # We are testing that add_source is not called.
-        # add_source("cloud-archive:havana")
+        add_source("cloud-archive:havana")
+        self.mocker.count(0)  # Test we never called add_source
         apt_update = self.mocker.replace(fetch.apt_update)
         apt_update(fatal=True)
-        apt_install = self.mocker.replace(fetch.apt_install)
-        apt_install(["python-novaclient"], fatal=True)
         self.mocker.replay()
 
-        hooks.install()
+        def apt_install(packages, fatal):
+            self.assertEqual(["python-novaclient"], packages)
+            self.assertTrue(fatal)
+
+        hooks.install(apt_install=apt_install, add_source=add_source)
 
     def test_precise_install_adds_apt_source_and_installs_novaclient(self):
         """
@@ -209,15 +212,40 @@
         get_running_series = self.mocker.replace(hooks.get_running_series)
         get_running_series()
         self.mocker.result({'release': '12.04'})  # precise
-        add_source = self.mocker.replace(fetch.add_source)
-        add_source("cloud-archive:havana")  # precise needs havana
-        apt_update = self.mocker.replace(fetch.apt_update)
-        apt_update(fatal=True)
-        apt_install = self.mocker.replace(fetch.apt_install)
-        apt_install(["python-novaclient"], fatal=True)
-        self.mocker.replay()
-
-        hooks.install()
+        apt_update = self.mocker.replace(fetch.apt_update)
+        apt_update(fatal=True)
+        self.mocker.replay()
+
+        def add_source(source):
+            self.assertEqual("cloud-archive:havana", source)
+
+        def apt_install(packages, fatal):
+            self.assertEqual(["python-novaclient"], packages)
+            self.assertTrue(fatal)
+
+        hooks.install(apt_install=apt_install, add_source=add_source)
+
+    def test_ec2_provider_install_installs_ec2_dependencies(self):
+        """
+        When the provider is configured as C{ec2}, L{install} will install
+        the C{python-boto} package.
+        """
+        self.addCleanup(
+            setattr, hooks.hookenv, "_config", hooks.hookenv._config)
+        hooks.hookenv._config = (
+            ("key", ""), ("tenant", ""), ("provider", "ec2"),
+            ("secret", ""), ("region", ""),
+            ("endpoint", ""))
+
+        apt_update = self.mocker.replace(fetch.apt_update)
+        apt_update(fatal=True)
+        self.mocker.replay()
+
+        def apt_install(packages, fatal):
+            self.assertEqual(["python-boto"], packages)
+            self.assertTrue(fatal)
+
+        hooks.install(apt_install=apt_install)
 
     def test_block_storage_relation_changed_waits_without_instance_id(self):
         """
 
=== modified file 'hooks/test_util.py'
--- hooks/test_util.py	2014-03-21 17:42:00 +0000
+++ hooks/test_util.py	2014-08-22 13:42:09 +0000
@@ -1,7 +1,8 @@
 import util
 from util import StorageServiceUtil, ENVIRONMENT_MAP, generate_volume_label
 import mocker
-import os
+from boto.exception import NoAuthHandlerFound, EC2ResponseError
+from socket import gaierror
 import subprocess
 from testing import TestHookenv
 
@@ -19,6 +20,10 @@
         util.log = util.hookenv.log
         self.storage = StorageServiceUtil("nova")
 
+    def _set_environment_vars(self, environ={}):
+        self.addCleanup(setattr, util.os, "environ", util.os.environ)
+        util.os.environ = environ
+
     def test_invalid_provier_config(self):
         """When an invalid provider config is set and error is reported."""
         result = self.assertRaises(SystemExit, StorageServiceUtil, "ce2")
@@ -42,8 +47,7 @@
         variables and then call L{validate_credentials} to assert
         that environment variables provided give access to the service.
         """
-        self.addCleanup(setattr, util.os, "environ", util.os.environ)
-        util.os.environ = {}
+        self._set_environment_vars({})
 
         def mock_validate():
             pass
@@ -64,7 +68,7 @@
         L{load_environment} will exit in failure and log a message if any
         required configuration option is not set.
         """
-        self.addCleanup(setattr, util.os, "environ", util.os.environ)
+        self._set_environment_vars({})
 
         def mock_validate():
             raise SystemExit("something invalid")
@@ -89,7 +93,7 @@
             SystemExit, self.storage.validate_credentials)
         self.assertEqual(result.code, 1)
         message = (
-            "ERROR: Charm configured credentials can't access endpoint. "
+            "ERROR: Charm configured credentials can't access nova endpoint. "
             "Command '%s' returned non-zero exit status 1" % command)
         self.assertIn(
             message, util.hookenv._log_ERROR, "Not logged- %s" % message)
@@ -239,9 +243,7 @@
         with the os.environ[JUJU_REMOTE_UNIT].
         """
         unit_name = "postgresql/0"
-        self.addCleanup(
-            setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
         volume_id = "123134124-1241412-1242141"
 
         def mock_describe():
@@ -259,9 +261,7 @@
         for the os.environ[JUJU_REMOTE_UNIT].
         """
         unit_name = "postgresql/0"
-        self.addCleanup(
-            setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         def mock_describe(val):
             self.assertIsNone(val)
@@ -280,8 +280,7 @@
         multiple results the function exits with an error.
         """
         unit_name = "postgresql/0"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         def mock_describe():
             return {"123123-123123":
@@ -306,8 +305,7 @@
         unit_name = "postgresql/0"
         instance_id = "i-123123"
         volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.storage.load_environment = lambda: None
         self.storage.describe_volumes = lambda volume_id: {}
@@ -331,8 +329,7 @@
         volume_id = "123-123-123"
         instance_id = "i-123123123"
         volume_label = "%s unit volume" % unit_name
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
         self.storage.load_environment = lambda: None
 
         def mock_get_volume_id(label):
@@ -360,8 +357,7 @@
         unit_name = "postgresql/0"
         instance_id = "i-123123"
         volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.storage.load_environment = lambda: None
 
@@ -385,14 +381,13 @@
         unit_name = "postgresql/0"
         instance_id = "i-123123"
         volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.describe_count = 0
 
         self.storage.load_environment = lambda: None
 
-        sleep = self.mocker.replace("util.sleep")
+        sleep = self.mocker.replace("time.sleep")
         sleep(5)
         self.mocker.replay()
 
@@ -423,8 +418,7 @@
         unit_name = "postgresql/0"
         instance_id = "i-123123"
         volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.storage.load_environment = lambda: None
 
@@ -452,8 +446,7 @@
         volume_id = "123-123-123"
         volume_label = "%s unit volume" % unit_name
         default_volume_size = util.hookenv.config("default_volume_size")
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.storage.load_environment = lambda: None
         self.storage.get_volume_id = lambda _: None
@@ -880,6 +873,19 @@
             message, util.hookenv._log_INFO, "Not logged- %s" % message)
 
 
+class MockBotoEC2Connection(object):
+    """Mock all calls to python-boto's EC2Connection class."""
+    def __init__(self, get_volumes=None, get_instances=None,
+                 create_volume=None, create_tags=None, attach_volume=None,
+                 detach_volume=None):
+        self.get_all_volumes = get_volumes
+        self.get_all_instances = get_instances
+        self.create_volume = create_volume
+        self.attach_volume = attach_volume
+        self.detach_volume = detach_volume
+        self.create_tags = create_tags
+
+
 class MockEucaCommand(object):
     def __init__(self, result):
         self.result = result
@@ -918,8 +924,8 @@
 
 
 class MockVolume(object):
-    def __init__(self, vol_id, device, instance_id, zone, size, status,
-                 snapshot_id, tags):
+    def __init__(self, vol_id, device=None, instance_id=None, zone=None,
+                 size=None, status=None, snapshot_id=None, tags=None):
         self.id = vol_id
         self.attach_data = MockAttachData(device, instance_id)
         self.zone = zone
@@ -936,11 +942,31 @@
         self.maxDiff = None
         util.hookenv = TestHookenv(
             {"key": "ec2key", "secret": "ec2password",
-             "endpoint": "https://ec2-region-url:443/v2.0/",
+             "endpoint": "https://ec2-region-1.com",
              "default_volume_size": 11})
+        util.EC2_BOTO_CONFIG_FILE = self.makeFile()
         util.log = util.hookenv.log
         self.storage = StorageServiceUtil("ec2")
 
+    def _set_environment_vars(self, environ={}):
+        self.addCleanup(setattr, util.os, "environ", util.os.environ)
+        util.os.environ = environ
+
+    def test_wb_setup_boto_config(self):
+        """
+        L{_setup_boto_config} writes C{EC2_BOTO_CONFIG_FILE} with the provided
+        C{key} and C{secret} parameters.
+        """
+        key = "my_ec2_key"
+        secret = "my_secret_access_key"
+        expected = ["[Credentials]",
+                    "aws_access_key_id = %s" % key,
+                    "aws_secret_access_key = %s" % secret]
+        self.storage._setup_boto_config(key=key, secret=secret)
+        with open(util.EC2_BOTO_CONFIG_FILE) as boto_config:
+            lines = boto_config.read().splitlines()
+        self.assertEqual(lines, expected)
+
     def test_load_environment_with_ec2_variables(self):
         """
         L{load_environment} will setup script environment variables for ec2
@@ -948,8 +974,7 @@
         variables and then call L{validate_credentials} to assert
         that environment variables provided give access to the service.
         """
-        self.addCleanup(setattr, util.os, "environ", util.os.environ)
-        util.os.environ = {}
+        self._set_environment_vars({})
 
         def mock_validate():
             pass
@@ -959,7 +984,7 @@
         expected = {
             "EC2_ACCESS_KEY": "ec2key",
             "EC2_SECRET_KEY": "ec2password",
-            "EC2_URL": "https://ec2-region-url:443/v2.0/"
+            "EC2_URL": "https://ec2-region-1.com"
         }
         self.assertEqual(util.os.environ, expected)
 
@@ -968,45 +993,28 @@
         L{load_environment} will exit in failure and log a message if any
         required configuration option is not set.
         """
-        self.addCleanup(setattr, util.os, "environ", util.os.environ)
-
         def mock_validate():
             raise SystemExit("something invalid")
         self.storage.validate_credentials = mock_validate
 
         self.assertRaises(SystemExit, self.storage.load_environment)
 
-    def test_validate_credentials_failure(self):
-        """
-        L{validate_credentials} will attempt a simple euca command to ensure
-        the environment is properly configured to access the nova service.
-        Upon failure to contact the nova service, L{validate_credentials} will
-        exit in error and log a message.
-        """
-        command = "euca-describe-instances"
-        nova_cmd = self.mocker.replace(subprocess.check_call)
-        nova_cmd(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
-
-        result = self.assertRaises(
-            SystemExit, self.storage.validate_credentials)
-        self.assertEqual(result.code, 1)
-        message = (
-            "ERROR: Charm configured credentials can't access endpoint. "
-            "Command '%s' returned non-zero exit status 1" % command)
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
     def test_validate_credentials(self):
         """
-        L{validate_credentials} will succeed when a simple euca command
-        succeeds due to a properly configured environment based on the charm
-        configuration options.
+        L{validate_credentials} will succeed when a boto's L{get_all_volumes}
+        succeeds using C{EC2_URL} environment variable from charm configuration
+        options.
         """
-        command = "euca-describe-instances"
-        nova_cmd = self.mocker.replace(subprocess.check_call)
-        nova_cmd(command, shell=True)
+        ec2_url = "https://ec2.us-west-1.amazonaws.com"
+        self._set_environment_vars({"EC2_URL": ec2_url})
+        connect = self.mocker.replace("boto.ec2.connect_to_region")
+        connect("us-west-1")
+
+        def get_volumes():
+            return []
+
+        self.mocker.result(MockBotoEC2Connection(
+            get_volumes=get_volumes))
         self.mocker.replay()
 
         self.storage.validate_credentials()
@@ -1040,9 +1048,7 @@
         labelled with the os.environ[JUJU_REMOTE_UNIT].
         """
         unit_name = "postgresql/0"
-        self.addCleanup(
-            setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
         volume_id = "123134124-1241412-1242141"
 
         def mock_describe(val):
@@ -1061,9 +1067,7 @@
         L{_ec2_describe_volumes} for the os.environ[JUJU_REMOTE_UNIT].
         """
         unit_name = "postgresql/0"
-        self.addCleanup(
-            setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         def mock_describe(val):
             self.assertIsNone(val)
@@ -1082,8 +1086,7 @@
         multiple results the function exits with an error.
         """
         unit_name = "postgresql/0"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         def mock_describe(val):
             self.assertIsNone(val)
@@ -1109,8 +1112,7 @@
         unit_name = "postgresql/0"
         instance_id = "i-123123"
         volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.storage.load_environment = lambda: None
         self.storage._ec2_describe_volumes = lambda volume_id: {}
@@ -1134,8 +1136,7 @@
         volume_id = "123-123-123"
         instance_id = "i-123123123"
         volume_label = "%s unit volume" % unit_name
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
         self.storage.load_environment = lambda: None
 
         def mock_get_volume_id(label):
@@ -1163,8 +1164,7 @@
         unit_name = "postgresql/0"
         instance_id = "i-123123"
         volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.storage.load_environment = lambda: None
 
@@ -1188,8 +1188,7 @@
         unit_name = "postgresql/0"
         instance_id = "i-123123"
         volume_id = "123-123-123"
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.storage.load_environment = lambda: None
 
@@ -1217,8 +1216,7 @@
         volume_id = "123-123-123"
         volume_label = "%s unit volume" % unit_name
         default_volume_size = util.hookenv.config("default_volume_size")
-        self.addCleanup(setattr, os, "environ", os.environ)
-        os.environ = {"JUJU_REMOTE_UNIT": unit_name}
+        self._set_environment_vars({"JUJU_REMOTE_UNIT": unit_name})
 
         self.storage.load_environment = lambda: None
         self.storage.get_volume_id = lambda _: None
@@ -1240,26 +1238,116 @@
         self.assertIn(
             message, util.hookenv._log_INFO, "Not logged- %s" % message)
 
-    def test_wb_ec2_describe_volumes_command_error(self):
-        """
-        L{_ec2_describe_volumes} will exit in error when the euca2ools
-        C{DescribeVolumes} command fails.
-        """
-        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
-        euca_command()
-        self.mocker.throw(SystemExit(1))
-        self.mocker.replay()
-
-        result = self.assertRaises(
-            SystemExit, self.storage._ec2_describe_volumes)
-        self.assertEqual(result.code, 1)
-        message = "ERROR: Couldn't contact EC2 using euca-describe-volumes"
+    def test_wb_ec2_validate_credentials_invalid_ec2_url_environment_var(self):
+        """
+        L{_ec2_validate_credentials} will exit in error when the C{EC2_URL}
+        environment variable is invalid.
+        """
+        self._set_environment_vars({"EC2_URL": "not-a-valid-ec2-url"})
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_validate_credentials)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Couldn't get region from EC2_URL environment variable: "
+            "not-a-valid-ec2-url.")
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_validate_credentials_absent_ec2_url_environment_var(self):
+        """
+        L{_ec2_validate_credentials} will exit in error when the C{EC2_URL}
+        environment variable is not present.
+        """
+        self._set_environment_vars({})
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_validate_credentials)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Couldn't get region from EC2_URL environment variable: "
+            "NOT SET.")
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_validate_credentials_unavailable_credentials(self):
+        """
+        L{_ec2_validate_credentials} will exit in error when the EC2
+        credentials are not present in /etc/boto.cfg.
+        """
+        self._set_environment_vars(
+            {"EC2_URL": "https://ec2.us-west-1.amazonaws.com"})
+        connect = self.mocker.replace("boto.ec2.connect_to_region")
+        connect("us-west-1")
+
+        def get_volumes_error():
+            raise NoAuthHandlerFound("No handler was ready to auth.")
+
+        self.mocker.result(
+            MockBotoEC2Connection(get_volumes=get_volumes_error))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_validate_credentials)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: EC2 credentials not found in /etc/boto.cfg. Cannot "
+            "authenticate.")
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_validate_credentials_invalid_credentials(self):
+        """
+        L{_ec2_validate_credentials} will exit in error when the EC2
+        credentials are not valid for the C{EC2_URL} provided.
+        """
+        self._set_environment_vars(
+            {"EC2_URL": "https://ec2.us-west-1.amazonaws.com"})
+        connect = self.mocker.replace("boto.ec2.connect_to_region")
+        connect("us-west-1")
+
+        def get_volumes_error():
+            raise EC2ResponseError(401, "Unauthorized")
+
+        self.mocker.result(
+            MockBotoEC2Connection(get_volumes=get_volumes_error))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_validate_credentials)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Invalid EC2 credentials in /etc/boto.cfg. Unauthorized.")
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
+
+    def test_wb_ec2_validate_credentials_timeout(self):
+        """
+        L{_ec2_validate_credentials} will exit in error when a connection
+        timeout occurs against the region endpoint.
+        """
+        ec2_url = "https://ec2.us-west-1.amazonaws.com"
+        self._set_environment_vars({"EC2_URL": ec2_url})
+        connect = self.mocker.replace("boto.ec2.connect_to_region")
+        connect("us-west-1")
+
+        def get_volumes_error():
+            raise gaierror("Temporary failure in name resolution")
+
+        self.mocker.result(
+            MockBotoEC2Connection(get_volumes=get_volumes_error))
+        self.mocker.replay()
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_validate_credentials)
+        self.assertEqual(result.code, 1)
+        message = "ERROR: Connectivity error accessing %s" % ec2_url
         self.assertIn(
             message, util.hookenv._log_ERROR, "Not logged- %s" % message)
 
     def test_wb_ec2_describe_volumes_without_attached_instances(self):
         """
-        L{_ec2_describe_volumes} parses the results of euca2ools
+        L{_ec2_describe_volumes} parses the results of boto.ec2.get_all_volumes
         C{DescribeVolumes} to create a C{dict} of volume information. When no
         C{instance_id}s are present the volumes are not attached so no
         C{device} or C{instance_id} information will be present.
@@ -1272,10 +1360,11 @@
             "456-456-456", device="/dev/notshown", instance_id="notseen",
             zone="ec2-az2", size="8", status="available",
             snapshot_id="some-shot", tags={"volume_name": "my volume name"})
-        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([volume1, volume2]))
-        self.mocker.replay()
+
+        def get_volumes():
+            return [volume1, volume2]
+
+        self.storage.ec2_conn = MockBotoEC2Connection(get_volumes=get_volumes)
 
         expected = {"123-123-123": {"id": "123-123-123", "status": "available",
                                     "device": "",
@@ -1295,9 +1384,8 @@
 
     def test_wb_ec2_describe_volumes_matches_volume_id_supplied(self):
         """
-        L{_ec2_describe_volumes} parses the results of euca2ools
-        C{DescribeVolumes} to create a C{dict} of volume information.
-        When C{volume_id} is provided return a C{dict} for the matched volume.
+        L{_ec2_describe_volumes} matches the provided C{volume_id} to the
+        results of boto's L{get_all_volumes}.
         """
         volume_id = "123-123-123"
         volume1 = MockVolume(
@@ -1308,10 +1396,11 @@
             "456-456-456", device="/dev/notshown", instance_id="notseen",
             zone="ec2-az2", size="8", status="available",
             snapshot_id="some-shot", tags={"volume_name": "my volume name"})
-        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([volume1, volume2]))
-        self.mocker.replay()
+
+        def get_volumes():
+            return [volume1, volume2]
+
+        self.storage.ec2_conn = MockBotoEC2Connection(get_volumes=get_volumes)
 
         expected = {
             "id": volume_id, "status": "available", "device": "",
@@ -1323,29 +1412,28 @@
 
     def test_wb_ec2_describe_volumes_unmatched_volume_id_supplied(self):
         """
-        L{_ec2_describe_volumes} parses the results of euca2ools
-        C{DescribeVolumes} to create a C{dict} of volume information.
-        When C{volume_id} is provided and unmatched, return an empty C{dict}.
+        L{_ec2_describe_volumes} returns an empty C{dict} when it does not
+        match the provided C{volume_id} to any volumes reported by boto's
+        L{get_all_volumes} method.
         """
         unmatched_volume_id = "456-456-456"
         volume1 = MockVolume(
             "123-123-123", device="/dev/notshown", instance_id="notseen",
             zone="ec2-az1", size="10", status="available",
             snapshot_id="some-shot", tags={})
-        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([volume1]))
-        self.mocker.replay()
+
+        def get_volumes():
+            return [volume1]
+
+        self.storage.ec2_conn = MockBotoEC2Connection(get_volumes=get_volumes)
 
         self.assertEqual(
             self.storage._ec2_describe_volumes(unmatched_volume_id), {})
 
     def test_wb_ec2_describe_volumes_with_attached_instances(self):
         """
-        L{_ec2_describe_volumes} parses the results of euca2ools
-        C{DescribeVolumes} to create a C{dict} of volume information. If
-        C{status} is C{in-use}, both C{device} and C{instance_id} will be
-        returned in the C{dict}.
+        L{_ec2_describe_volumes} will report attached instance_id information
+        when boto's L{get_all_volumes} lists instance information for a volume.
         """
         volume1 = MockVolume(
             "123-123-123", device="/dev/notshown", instance_id="notseen",
@@ -1355,10 +1443,11 @@
             "456-456-456", device="/dev/xvdc", instance_id="i-456456",
             zone="ec2-az2", size="8", status="in-use",
             snapshot_id="some-shot", tags={"volume_name": "my volume name"})
-        euca_command = self.mocker.replace(self.storage.ec2_volume_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([volume1, volume2]))
-        self.mocker.replay()
+
+        def get_volumes():
+            return [volume1, volume2]
+
+        self.storage.ec2_conn = MockBotoEC2Connection(get_volumes=get_volumes)
 
         expected = {"123-123-123": {"id": "123-123-123", "status": "available",
                                     "device": "",
@@ -1379,41 +1468,38 @@
 
     def test_wb_ec2_create_volume(self):
         """
-        L{_ec2_create_volume} uses the command C{euca-create-volume} to create
+        L{_ec2_create_volume} calls boto's L{create_volume} to create
         a volume. It determines the availability zone for the volume by
-        querying L{_ec2_describe_instances} on the provided C{instance_id}
-        to ensure it matches the same availablity zone. It will then call
-        L{_ec2_create_tag} to setup the C{volume_name} tag for the volume.
+        querying L{_ec2_describe_instances} for the provided C{instance_id}
+        to ensure the volume is created in the same availablity zone.
+        L{_ec2_create_volume} will also call L{_ec2_create_tag} to setup the
+        C{volume_name} tag for the volume.
         """
         instance_id = "i-123123"
         volume_id = "123-123-123"
         volume_label = "postgresql/0 unit volume"
         size = 10
         zone = "ec2-az3"
-        command = "euca-create-volume -z %s -s %s" % (zone, size)
 
         reservation = MockEucaReservation(
             [MockEucaInstance(
                 instance_id=instance_id, availability_zone=zone)])
-        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([reservation]))
-        create = self.mocker.replace(subprocess.check_output)
-        create(command, shell=True)
-        self.mocker.result(
-            "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id)
-        self.mocker.replay()
-
-        def mock_describe_volumes(my_id):
-            self.assertEqual(my_id, volume_id)
-            return {"id": volume_id}
-        self.storage._ec2_describe_volumes = mock_describe_volumes
-
-        def mock_create_tag(my_id, key, value):
-            self.assertEqual(my_id, volume_id)
-            self.assertEqual(key, "volume_name")
-            self.assertEqual(value, volume_label)
-        self.storage._ec2_create_tag = mock_create_tag
+
+        def get_instances():
+            return [reservation]
+
+        def create_volume(size, zone):
+            self.assertEqual(zone, "ec2-az3")
+            self.assertEqual(size, 10)
+            return MockVolume(vol_id=volume_id)
+
+        def create_tags(resource_ids, tags):
+            self.assertEqual(resource_ids, [volume_id])
+            self.assertEqual({"volume_name": volume_label}, tags)
+
+        self.storage.ec2_conn = MockBotoEC2Connection(
+            get_instances=get_instances, create_volume=create_volume,
+            create_tags=create_tags)
 
         self.assertEqual(
             self.storage._ec2_create_volume(size, volume_label, instance_id),
@@ -1436,10 +1522,11 @@
         volume_label = "postgresql/0 unit volume"
         size = 10
 
-        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([]))  # Empty results from euca
-        self.mocker.replay()
+        def get_instances():
+            return []
+
+        self.storage.ec2_conn = MockBotoEC2Connection(
+            get_instances=get_instances)
 
         result = self.assertRaises(
             SystemExit, self.storage._ec2_create_volume, size, volume_label,
@@ -1448,174 +1535,85 @@
         config = util.hookenv.config_get()
         message = (
             "ERROR: Could not create volume for instance %s. No instance "
-            "details discovered by euca-describe-instances. Maybe the "
+            "details discovered by boto.ec2.get_all_instances. Maybe the "
             "charm configured endpoint %s is not valid for this region." %
             (instance_id, config["endpoint"]))
         self.assertIn(
             message, util.hookenv._log_ERROR, "Not logged- %s" % message)
 
-    def test_wb_ec2_create_volume_error_invalid_response_type(self):
+    def test_wb_ec2_create_volume_error(self):
         """
         L{_ec2_create_volume} will log an error and exit when it receives an
-        unparseable response type from the C{euca-create-volume} command.
-        """
-        instance_id = "i-123123"
-        volume_label = "postgresql/0 unit volume"
-        size = 10
-        zone = "ec2-az3"
-        command = "euca-create-volume -z %s -s %s" % (zone, size)
-
-        reservation = MockEucaReservation(
-            [MockEucaInstance(
-                instance_id=instance_id, availability_zone=zone)])
-        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([reservation]))
-        create = self.mocker.replace(subprocess.check_output)
-        create(command, shell=True)
-        self.mocker.result("INSTANCE invalid-instance-type-response\n")
-        self.mocker.replay()
-
-        def mock_describe_volumes(my_id):
-            raise Exception("_ec2_describe_volumes should not be called")
-        self.storage._ec2_describe_volumes = mock_describe_volumes
-
-        def mock_create_tag(my_id, key, value):
-            raise Exception("_ec2_create_tag should not be called")
-        self.storage._ec2_create_tag = mock_create_tag
-
-        result = self.assertRaises(
-            SystemExit, self.storage._ec2_create_volume, size, volume_label,
-            instance_id)
-        self.assertEqual(result.code, 1)
-        message = (
-            "ERROR: Didn't get VOLUME response from euca-create-volume. "
-            "Response: INSTANCE invalid-instance-type-response\n")
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
-    def test_wb_ec2_create_volume_error_new_volume_not_found(self):
-        """
-        L{_ec2_create_volume} will log an error and exit when it cannot find
-        details of the newly created C{volume_id} through a subsequent call to
-        L{_ec2_descibe_volumes}.
-        """
-        instance_id = "i-123123"
-        volume_id = "123-123-123"
-        volume_label = "postgresql/0 unit volume"
-        size = 10
-        zone = "ec2-az3"
-        command = "euca-create-volume -z %s -s %s" % (zone, size)
-
-        reservation = MockEucaReservation(
-            [MockEucaInstance(
-                instance_id=instance_id, availability_zone=zone)])
-        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([reservation]))
-        create = self.mocker.replace(subprocess.check_output)
-        create(command, shell=True)
-        self.mocker.result(
-            "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id)
-        self.mocker.replay()
-
-        def mock_describe_volumes(my_id):
-            self.assertEqual(my_id, volume_id)
-            return {}  # No details found for this volume
-        self.storage._ec2_describe_volumes = mock_describe_volumes
-
-        def mock_create_tag(my_id, key, value):
-            raise Exception("_ec2_create_tag should not be called")
-        self.storage._ec2_create_tag = mock_create_tag
-
-        result = self.assertRaises(
-            SystemExit, self.storage._ec2_create_volume, size, volume_label,
-            instance_id)
-        self.assertEqual(result.code, 1)
-        message = (
-            "ERROR: Unable to find volume '%s'" % volume_id)
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
-    def test_wb_ec2_create_volume_error_command_failed(self):
-        """
-        L{_ec2_create_volume} will log an error and exit when the
-        C{euca-create-volume} fails.
-        """
-        instance_id = "i-123123"
-        volume_label = "postgresql/0 unit volume"
-        size = 10
-        zone = "ec2-az3"
-        command = "euca-create-volume -z %s -s %s" % (zone, size)
-
-        reservation = MockEucaReservation(
-            [MockEucaInstance(
-                instance_id=instance_id, availability_zone=zone)])
-        euca_command = self.mocker.replace(self.storage.ec2_instance_class)
-        euca_command()
-        self.mocker.result(MockEucaCommand([reservation]))
-        create = self.mocker.replace(subprocess.check_output)
-        create(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
-
-        def mock_exception(my_id):
-            raise Exception("These methods should not be called")
-        self.storage._ec2_describe_volumes = mock_exception
-        self.storage._ec2_create_tag = mock_exception
-
-        result = self.assertRaises(
-            SystemExit, self.storage._ec2_create_volume, size, volume_label,
-            instance_id)
-        self.assertEqual(result.code, 1)
-
-        message = (
-            "ERROR: Command '%s' returned non-zero exit status 1" % command)
+        exception from boto's C{create_volume} command.
+        """
+        instance_id = "i-123123"
+        volume_label = "postgresql/0 unit volume"
+        size = 10
+        zone = "ec2-az3"
+
+        reservation = MockEucaReservation(
+            [MockEucaInstance(
+                instance_id=instance_id, availability_zone=zone)])
+
+        def get_instances():
+            return [reservation]
+
+        def create_volume(size, zone):
+            self.assertEqual(zone, "ec2-az3")
+            self.assertEqual(size, 10)
+            raise EC2ResponseError(401, "Unauthorized")
+
+        self.storage.ec2_conn = MockBotoEC2Connection(
+            get_instances=get_instances, create_volume=create_volume)
+
+        result = self.assertRaises(
+            SystemExit, self.storage._ec2_create_volume, size, volume_label,
+            instance_id)
+        self.assertEqual(result.code, 1)
+        message = "ERROR: EC2ResponseError: 401 Unauthorized\n"
         self.assertIn(
             message, util.hookenv._log_ERROR, "Not logged- %s" % message)
 
     def test_wb_ec2_attach_volume(self):
         """
-        L{_ec2_attach_volume} uses the command C{euca-attach-volume} and
-        returns the attached volume path.
+        L{_ec2_attach_volume} uses the EC2 boto's C{attach-volume} and
+        returns the attached device path.
         """
         instance_id = "i-123123"
         volume_id = "123-123-123"
         device = "/dev/xvdc"
-        command = (
-            "euca-attach-volume -i %s -d %s %s" %
-            (instance_id, device, volume_id))
-
-        attach = self.mocker.replace(subprocess.check_call)
-        attach(command, shell=True)
-        self.mocker.replay()
+
+        def attach_volume(volume_id, instance_id, device):
+            self.assertEqual(instance_id, "i-123123")
+            self.assertEqual(volume_id, "123-123-123")
+            self.assertEqual(device, "/dev/xvdc")
+
+        self.storage.ec2_conn = MockBotoEC2Connection(
+            attach_volume=attach_volume)
 
         self.assertEqual(
             self.storage._ec2_attach_volume(instance_id, volume_id),
             device)
 
-    def test_wb_ec2_attach_volume_command_failed(self):
+    def test_wb_ec2_attach_volume_failed(self):
         """
-        L{_ec2_attach_volume} exits in error when C{euca-attach-volume} fails.
+        L{_ec2_attach_volume} exits in error when boto's C{attach-volume}
+        fails.
         """
         instance_id = "i-123123"
         volume_id = "123-123-123"
-        device = "/dev/xvdc"
-        command = (
-            "euca-attach-volume -i %s -d %s %s" %
-            (instance_id, device, volume_id))
-
-        attach = self.mocker.replace(subprocess.check_call)
-        attach(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
+
+        def attach_volume(volume_id, instance_id, device):
+            raise EC2ResponseError(401, "Unauthorized")
+
+        self.storage.ec2_conn = MockBotoEC2Connection(
+            attach_volume=attach_volume)
 
         result = self.assertRaises(
             SystemExit, self.storage._ec2_attach_volume, instance_id,
             volume_id)
         self.assertEqual(result.code, 1)
-        message = (
-            "ERROR: Command '%s' returned non-zero exit status 1" % command)
+        message = "ERROR: EC2ResponseError: 401 Unauthorized\n"
         self.assertIn(
             message, util.hookenv._log_ERROR, "Not logged- %s" % message)
 
@@ -1661,46 +1659,11 @@
         self.assertIn(
             message, util.hookenv._log_INFO, "Not logged- %s" % message)
 
-    def test_detach_volume_command_error(self):
-        """
-        When the C{euca-detach-volume} command fails, L{detach_volume} will
-        log a message and exit in error.
-        """
-        volume_label = "postgresql/0 unit volume"
-        volume_id = "123-123-123"
-        instance_id = "i-123123"
-        self.storage.load_environment = lambda: None
-
-        def mock_get_volume_id(label):
-            self.assertEqual(label, volume_label)
-            return volume_id
-        self.storage.get_volume_id = mock_get_volume_id
-
-        def mock_describe_volumes(my_id):
-            self.assertEqual(my_id, volume_id)
-            return {"status": "in-use", "instance_id": instance_id}
-        self.storage.describe_volumes = mock_describe_volumes
-
-        command = "euca-detach-volume -i %s %s" % (instance_id, volume_id)
-        ec2_cmd = self.mocker.replace(subprocess.check_call)
-        ec2_cmd(command, shell=True)
-        self.mocker.throw(subprocess.CalledProcessError(1, command))
-        self.mocker.replay()
-
-        result = self.assertRaises(
-            SystemExit, self.storage.detach_volume, volume_label)
-        self.assertEqual(result.code, 1)
-        message = (
-            "ERROR: Couldn't detach volume. Command '%s' returned non-zero "
-            "exit status 1" % command)
-        self.assertIn(
-            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
-
     def test_detach_volume(self):
         """
         When L{get_volume_id} finds a volume associated with this instance
         which has a volume state not equal to C{available}, it detaches that
-        volume using euca2ools commands.
+        volume using boto's L{detach_volume}.
         """
         volume_label = "postgresql/0 unit volume"
         volume_id = "123-123-123"
@@ -1717,10 +1680,12 @@
             return {"status": "in-use", "instance_id": instance_id}
         self.storage.describe_volumes = mock_describe_volumes
 
-        command = "euca-detach-volume -i %s %s" % (instance_id, volume_id)
-        ec2_cmd = self.mocker.replace(subprocess.check_call)
-        ec2_cmd(command, shell=True)
-        self.mocker.replay()
+        def detach_volume(instance_id, volume_id):
+            self.assertEqual(instance_id, "i-123123")
+            self.assertEqual(volume_id, "123-123-123")
+
+        self.storage.ec2_conn = MockBotoEC2Connection(
+            detach_volume=detach_volume)
 
         self.storage.detach_volume(volume_label)
         message = (
@@ -1728,3 +1693,38 @@
             (volume_id, instance_id))
         self.assertIn(
             message, util.hookenv._log_INFO, "Not logged- %s" % message)
+
+    def test_detach_volume_error(self):
+        """
+        An error is logged and we exit(1) when boto's L{detach_volume} raises
+        an error.
+        """
+        volume_label = "postgresql/0 unit volume"
+        volume_id = "123-123-123"
+        instance_id = "i-123123"
+        self.storage.load_environment = lambda: None
+
+        def mock_get_volume_id(label):
+            self.assertEqual(label, volume_label)
+            return volume_id
+        self.storage.get_volume_id = mock_get_volume_id
+
+        def mock_describe_volumes(my_id):
+            self.assertEqual(my_id, volume_id)
+            return {"status": "in-use", "instance_id": instance_id}
+        self.storage.describe_volumes = mock_describe_volumes
+
+        def detach_volume_error(instance_id, volume_id):
+            raise EC2ResponseError(401, "Unauthorized")
+
+        self.storage.ec2_conn = MockBotoEC2Connection(
+            detach_volume=detach_volume_error)
+
+        result = self.assertRaises(
+            SystemExit, self.storage.detach_volume, volume_label)
+        self.assertEqual(result.code, 1)
+        message = (
+            "ERROR: Couldn't detach volume. EC2ResponseError: "
+            "401 Unauthorized\n")
+        self.assertIn(
+            message, util.hookenv._log_ERROR, "Not logged- %s" % message)
 
=== modified file 'hooks/util.py'
--- hooks/util.py	2014-07-18 00:58:58 +0000
+++ hooks/util.py	2014-08-22 13:42:09 +0000
@@ -3,6 +3,7 @@
 from charmhelpers.core import hookenv
 import subprocess
 import os
+import re
 import sys
 from time import sleep
 
@@ -17,11 +18,8 @@
     "ec2": ["endpoint", "key", "secret"],
     "nova": ["endpoint", "region", "tenant", "key", "secret"]}
 
-PROVIDER_COMMANDS = {
-    "ec2": {"validate": "euca-describe-instances",
-            "detach": "euca-detach-volume -i %s %s"},
-    "nova": {"validate": "nova list",
-             "detach": "nova volume-detach %s %s"}}
+# Use python-boto for AWS describe-volumes describe-instances
+EC2_BOTO_CONFIG_FILE = "/etc/boto.cfg"
 
 
 class StorageServiceUtil(object):
@@ -31,7 +29,6 @@
     provider = None
     environment_map = None
     required_config_options = None
-    commands = None
 
     def __init__(self, provider):
         self.provider = provider
@@ -43,13 +40,8 @@
                 hookenv.ERROR)
             sys.exit(1)
         self.environment_map = ENVIRONMENT_MAP[provider]
-        self.commands = PROVIDER_COMMANDS[provider]
         self.required_config_options = REQUIRED_CONFIG_OPTIONS[provider]
-        if provider == "ec2":
-            import euca2ools.commands.euca.describevolumes as getvolumes
-            import euca2ools.commands.euca.describeinstances as getinstances
-            self.ec2_volume_class = getvolumes.DescribeVolumes
-            self.ec2_instance_class = getinstances.DescribeInstances
+        self.ec2_conn = None
 
     def load_environment(self):
         """
@@ -60,18 +52,26 @@
         for option in self.required_config_options:
             environment_variable = self.environment_map[option]
             os.environ[environment_variable] = config_data[option].strip()
+        if self.provider == "ec2":
+            self._setup_boto_config(
+                os.environ.get("EC2_ACCESS_KEY", None),
+                os.environ.get("EC2_SECRET_KEY", None))
         self.validate_credentials()
 
     def validate_credentials(self):
+        method = getattr(self, "_%s_validate_credentials" % self.provider)
+        return method()
+
+    def _nova_validate_credentials(self):
         """
-        Attempt to contact the respective ec2 or nova volume service or exit(1)
+        Attempt to contact nova volume service or exit(1)
         """
         try:
-            subprocess.check_call(self.commands["validate"], shell=True)
+            subprocess.check_call("nova list", shell=True)
         except subprocess.CalledProcessError, e:
             hookenv.log(
-                "ERROR: Charm configured credentials can't access endpoint. "
-                "%s" % str(e),
+                "ERROR: Charm configured credentials can't access nova "
+                "endpoint. %s" % str(e),
                 hookenv.ERROR)
             sys.exit(1)
         hookenv.log(
@@ -177,8 +177,8 @@
                 sleep(5)
         if not device:
             hookenv.log(
-                "ERROR: Unable to discover device attached by "
-                "euca-attach-volume",
+                "ERROR: Unable to discover device attached using "
+                "python boto.ec2.attach_volume.",
                 hookenv.ERROR)
             sys.exit(1)
         return device
@@ -201,9 +201,15 @@
         hookenv.log(
             "Detaching volume (%s) from instance %s" %
             (volume_id, volume["instance_id"]))
+
+        method = getattr(self, "_%s_detach_volume" % self.provider)
+        return method(volume_id=volume_id, instance_id=volume["instance_id"])
+
+    def _nova_detach_volume(self, volume_id, instance_id):
+        """Detach specified C{volume_id} from the nova C{instance_id}."""
         try:
             subprocess.check_call(
-                self.commands["detach"] % (volume["instance_id"], volume_id),
+                "nova volume-detach %s %s" % (instance_id, volume_id),
                 shell=True)
         except subprocess.CalledProcessError, e:
             hookenv.log(
@@ -212,35 +218,72 @@
         return
 
     # EC2-specific methods
-    def _ec2_create_tag(self, volume_id, tag_name, tag_value=None):
+    def _setup_boto_config(self, key, secret):
+        """Write EC2 credentials to C{EC2_BOTO_CONFIG_FILE}."""
+        with open(EC2_BOTO_CONFIG_FILE, "w") as config_file:
+            config_file.write(
+                "[Credentials]\naws_access_key_id = %s\n"
+                "aws_secret_access_key = %s\n" % (key, secret))
+
+    def _ec2_validate_credentials(self):
+        """Attempt to contact EC2 storage service or exit(1)."""
+        from boto.exception import NoAuthHandlerFound, EC2ResponseError
+        import boto.ec2
+        from socket import gaierror
+        message = None
+        if self.provider == "ec2" and not self.ec2_conn:
+            # parse region out of EC2_URL environment variable
+            ec2_url = os.environ.get("EC2_URL", "NOT SET")
+            matches = re.search("\S{2}-\S+-\d", ec2_url)
+            if not matches:
+                message = (
+                    "ERROR: Couldn't get region from EC2_URL environment "
+                    "variable: %s." % ec2_url)
+                hookenv.log(message, hookenv.ERROR)
+                sys.exit(1)
+            ec2_region = matches.group(0)
+
+            # Create connection object, doesn't actually contact AWS
+            self.ec2_conn = boto.ec2.connect_to_region(ec2_region)
+            try:
+                # Test credentials against AWS with a simple command
+                self.ec2_conn.get_all_volumes()
+            except EC2ResponseError:
+                message = (
+                    "ERROR: Invalid EC2 credentials in /etc/boto.cfg. "
+                    "Unauthorized.")
+            except NoAuthHandlerFound:
+                message = (
+                    "ERROR: EC2 credentials not found in /etc/boto.cfg. "
+                    "Cannot authenticate.")
+            except gaierror:
+                message = "ERROR: Connectivity error accessing %s" % ec2_url
+            if message:
+                hookenv.log(message, hookenv.ERROR)
+                sys.exit(1)
+            hookenv.log(
+                "Validated charm configuration credentials have access to "
+                "block storage service")
+
+    def _ec2_create_tag(self, volume_id, tag_name, tag_value=""):
         """Attach a tag and optional C{tag_value} to the given C{volume_id}"""
         tag_string = tag_name
         if tag_value:
             tag_string += "=%s" % tag_value
-        command = 'euca-create-tags %s --tag "%s"' % (volume_id, tag_string)
-
         try:
-            subprocess.check_call(command, shell=True)
-        except subprocess.CalledProcessError, e:
-            hookenv.log(
-                "ERROR: Couldn't add tags to the resource. %s" % str(e),
-                hookenv.ERROR)
+            self.ec2_conn.create_tags(
+                resource_ids=[volume_id], tags={tag_name: tag_value})
+        except Exception, e:
+            hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
             sys.exit(1)
         hookenv.log("Tagged (%s) to %s." % (tag_string, volume_id))
 
     def _ec2_describe_instances(self, instance_id=None):
         """
-        Use euca2ools libraries to describe instances and return a C{dict}
+        Use AWS Dev API (boto) to describe instances and return a C{dict}
         """
         result = {}
-        try:
-            command = self.ec2_instance_class()
-            reservations = command.main()
-        except SystemExit:
-            hookenv.log(
-                "ERROR: Couldn't contact EC2 using euca-describe-instances",
-                hookenv.ERROR)
-            sys.exit(1)
+        reservations = self.ec2_conn.get_all_instances()
         for reservation in reservations:
             for inst in reservation.instances:
                 result[inst.id] = {
@@ -259,17 +302,10 @@
 
     def _ec2_describe_volumes(self, volume_id=None):
         """
-        Use euca2ools libraries to describe volumes and return a C{dict}
+        Use AWS Developer API (boto) to describe volumes and return a C{dict}
         """
         result = {}
-        try:
-            command = self.ec2_volume_class()
-            volumes = command.main()
-        except SystemExit:
-            hookenv.log(
-                "ERROR: Couldn't contact EC2 using euca-describe-volumes",
-                hookenv.ERROR)
-            sys.exit(1)
+        volumes = self.ec2_conn.get_all_volumes()
         for volume in volumes:
             result[volume.id] = {
                 "device": "",
@@ -306,34 +342,19 @@
             config_data = hookenv.config()
             hookenv.log(
                 "ERROR: Could not create volume for instance %s. No instance "
-                "details discovered by euca-describe-instances. Maybe the "
+                "details discovered by boto.ec2.get_all_instances. Maybe the "
                 "charm configured endpoint %s is not valid for this region." %
                 (instance_id, config_data["endpoint"]), hookenv.ERROR)
             sys.exit(1)
 
         try:
-            output = subprocess.check_output(
-                "euca-create-volume -z %s -s %s" %
-                (instance["availability_zone"], size), shell=True)
-        except subprocess.CalledProcessError, e:
+            volume = self.ec2_conn.create_volume(
+                size=size, zone=instance["availability_zone"])
+        except Exception, e:
             hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
             sys.exit(1)
-
-        response_type, volume_id = output.split()[:2]
-        if response_type != "VOLUME":
-            hookenv.log(
-                "ERROR: Didn't get VOLUME response from euca-create-volume. "
-                "Response: %s" % output, hookenv.ERROR)
-            sys.exit(1)
-        volume = self.describe_volumes(volume_id.strip())
-        if not volume:
-            hookenv.log(
-                "ERROR: Unable to find volume '%s'" % volume_id.strip(),
-                hookenv.ERROR)
-            sys.exit(1)
-        volume_id = volume["id"]
-        self._ec2_create_tag(volume_id, "volume_name", volume_label)
-        return volume_id
+        self._ec2_create_tag(volume.id, "volume_name", volume_label)
+        return volume.id
 
     def _ec2_attach_volume(self, instance_id, volume_id):
         """
@@ -342,14 +363,25 @@
         """
         device = "/dev/xvdc"
         try:
-            subprocess.check_call(
-                "euca-attach-volume -i %s -d %s %s" %
-                (instance_id, device, volume_id), shell=True)
-        except subprocess.CalledProcessError, e:
+            self.ec2_conn.attach_volume(
+                volume_id=volume_id, instance_id=instance_id, device=device)
+        except Exception, e:
             hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
             sys.exit(1)
         return device
 
+    def _ec2_detach_volume(self, instance_id, volume_id):
+        """
+        Detach an EC2 C{volume_id} from the provided C{instance_id}.
+        """
+        try:
+            self.ec2_conn.detach_volume(
+                volume_id=volume_id, instance_id=instance_id)
+        except Exception, e:
+            hookenv.log(
+                "ERROR: Couldn't detach volume. %s" % str(e), hookenv.ERROR)
+            sys.exit(1)
+
     # Nova-specific methods
     def _nova_volume_show(self, volume_id):
         """