Merge lp:~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support into lp:~chad.smith/charms/precise/block-storage-broker/trunk

Proposed by Chad Smith on 2014-03-20
Status: Rejected
Rejected by: Chad Smith on 2014-08-18
Proposed branch: lp:~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support
Merge into: lp:~chad.smith/charms/precise/block-storage-broker/trunk
Prerequisite: lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support
Diff against target: 1107 lines (+602/-255)
2 files modified
hooks/test_util.py (+394/-143)
hooks/util.py (+208/-112)
To merge this branch: bzr merge lp:~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support
Reviewer Review Type Date Requested Status
Chad Smith Disapprove on 2014-08-18
Alberto Donato (community) 2014-03-20 Needs Fixing on 2014-03-27
Review via email: mp+211963@code.launchpad.net

Description of the change

This branch does a couple things:
 1. stops the block-storage-broker from importing euca2ools.commands.euca.describevolumes and euca2ools.commands.euca.describeinstances directly to avoid being affected by internal class/module changes on next major euca release. Instead block-storage-broker sets up a couple of simple parse_ec2* functions to help parsing the output of euca-describe-instances, euca-describe-volumes and euca-describe-tags commands. The euca response types supported by the parse_ec2* functions are INSTANCE, ATTACHMENT, VOLUME and TAG.

 2. To handle the fact that euca-2.X doesn't report volume TAG responses in the euca-describe-volumes command output, _ec2_describe_volumes also calls _ec2_describe_tags to augment any volume tag data (which we use to label volumes in EC2).

 3. In using euca-describe-tags to augment existing volume data I found that tags can be orphaned once a volume is removed by an admin via euca-delete-volume, if that is the case we will ignore tags for volumes that don't exist and log a message

 4. In unit tests drop all MockEuca* classes as they are no longer needed because we mock the subprocess.check_output results for the euca-* commands we run.

By parsing the output from the euca-* commands, we shield ourselves from the internal changes in class structure definition in future euca releases. This branch adds very basic validation of euca-describe-instance output. We can add more validation in separate branches as I didn't want this MP to grow too large.

To post a comment you must log in.
87. By Chad Smith on 2014-03-21

merge fixes from bsb-ec2-support and resolve conflicts + lint

88. By Chad Smith on 2014-03-21

revert Makefile change

89. By Chad Smith on 2014-03-21

Ignore any orphaned TAG lines from euca-describe-tags that don't map to current existing volume-ids

Chad Smith (chad.smith) wrote :

Here's a deployer bundle to deploy.
rm -rf ./trusty wherever you are about to juju-deployer from to ensure you don't use cached charm checkouts

cat block-storage-trusty.yaml

common:
    services:
        storage:
            branch: lp:~chad.smith/charms/precise/storage/storage-volume-label-availability-zone
            options:
                provider: block-storage-broker
                volume_size: 9
        block-storage-broker-ec2:
            branch: lp:~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support
            options:
                provider: ec2
                key: <YOUR_EC2_ACCESS_KEY>
                endpoint: <YOUR_EC2_URL>
                secret: <YOUR_EC2_SECRET_KEY>
        postgresql:
            branch: lp:~chad.smith/charms/precise/postgresql/postgresql-using-storage-subordinate
            constraints: mem=2048
            options:
                extra-packages: python-apt postgresql-contrib postgresql-9.3-debversion
                max_connections: 500

doit:
    inherits: common
    series: trusty
    relations:
        - [postgresql, storage]
        - [storage, block-storage-broker-ec2]

juju-deployer -c block-storage-trusty.yaml doit

Chad Smith (chad.smith) wrote :

The bundle above works from the w/ juju 1.17.6 be wary of 1.17. < 6 as it suffers from bug 1285901

Alberto Donato (ack) wrote :

As discussed on IRC, I think the code could be refactored to reduce duplication, for example as in https://pastebin.canonical.com/107195/.
Tests could then mock _run_command rather than subprocess.check_output.

It should be even possible to perform the line.split("\n") in the _run_command method, returning a list of lists (so the parse_* methods don't need to perform it), I'm not sure if it fits all the use cases, though.

#1:
+ data["device"] = ""
+ data["volume_label"] = ""
+ data["instance_id"] = ""
+ data["tags"] = {"volume_name": ""}

These can be set with a single data.update()

#2:
+ volume_data.update({"instance_id": data["instance_id"]})
+ volume_data.update({"device": data["device"]})

Same as #1

#3:
+ for tag_volume_id in volume_tag_data.keys():
+ additional_tags = volume_tag_data[tag_volume_id]["tags"]
+ if tag_volume_id not in result:
+ hookenv.log(
+ "Ignoring tags for volume-id %s that doesn't exist" %
+ tag_volume_id)
             else:

The first line inside the loop can be moved inside the else clause.
Also there's an extra indentation space on the hookenv.log() call

#4:
I think you can drop the checks on the response_type in the parse_* methods, since it's already checked by call sites (related tests can be dropped then).

review: Needs Fixing
90. By Chad Smith on 2014-08-08

pull in ack's _run_command patch for simplification

Chad Smith (chad.smith) wrote :

Rejecting this branch in favor of dropping euca2ools and using Amazon's python SDK: python-boto as boto is more stable across Ubuntu releases than trying to parse and unify the output of euca2ools commandline utilities.

review: Disapprove

Unmerged revisions

90. By Chad Smith on 2014-08-08

pull in ack's _run_command patch for simplification

89. By Chad Smith on 2014-03-21

Ignore any orphaned TAG lines from euca-describe-tags that don't map to current existing volume-ids

88. By Chad Smith on 2014-03-21

revert Makefile change

87. By Chad Smith on 2014-03-21

merge fixes from bsb-ec2-support and resolve conflicts + lint

86. By Chad Smith on 2014-03-20

now that _ec2_describe_volumes calls euca-describe-tags, we need to mock that in unit tests. Add unit tests for parse_ec2_*_reponse functions

85. By Chad Smith on 2014-03-20

docstring updates

84. By Chad Smith on 2014-03-20

perform validation of INSTANCE reponse_type before attmpting to parse the value in the line. lint fixes in unit tests

83. By Chad Smith on 2014-03-20

parse euca-describe-instances and euca-describe-volumes output instead of tying into euca2ools.commands.euca.describeinstances directly

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'hooks/test_util.py'
2--- hooks/test_util.py 2014-03-21 17:42:00 +0000
3+++ hooks/test_util.py 2014-08-08 02:34:47 +0000
4@@ -1,5 +1,9 @@
5 import util
6-from util import StorageServiceUtil, ENVIRONMENT_MAP, generate_volume_label
7+from util import (
8+ StorageServiceUtil, EUCA2_INSTANCE_RESPONSE_LENGTH_V3,
9+ EUCA2_INSTANCE_RESPONSE_LENGTH_V2, parse_ec2_tag_response,
10+ parse_ec2_volume_response, parse_ec2_instance_response,
11+ parse_ec2_attachment_response, ENVIRONMENT_MAP, generate_volume_label)
12 import mocker
13 import os
14 import subprocess
15@@ -9,7 +13,6 @@
16 class TestNovaUtil(mocker.MockerTestCase):
17
18 def setUp(self):
19- super(TestNovaUtil, self).setUp()
20 self.maxDiff = None
21 util.hookenv = TestHookenv(
22 {"key": "myusername", "tenant": "myusername_project",
23@@ -880,59 +883,9 @@
24 message, util.hookenv._log_INFO, "Not logged- %s" % message)
25
26
27-class MockEucaCommand(object):
28- def __init__(self, result):
29- self.result = result
30-
31- def main(self):
32- return self.result
33-
34-
35-class MockEucaReservation(object):
36- def __init__(self, instances):
37- self.instances = instances
38- self.id = 1
39-
40-
41-class MockEucaInstance(object):
42- def __init__(self, instance_id=None, ip_address=None, image_id=None,
43- instance_type=None, kernel=None, private_dns_name=None,
44- public_dns_name=None, state=None, tags=[],
45- availability_zone=None):
46- self.id = instance_id
47- self.ip_address = ip_address
48- self.image_id = image_id
49- self.instance_type = instance_type
50- self.kernel = kernel
51- self.private_dns_name = private_dns_name
52- self.public_dns_name = public_dns_name
53- self.state = state
54- self.tags = tags
55- self.placement = availability_zone
56-
57-
58-class MockAttachData(object):
59- def __init__(self, device, instance_id):
60- self.device = device
61- self.instance_id = instance_id
62-
63-
64-class MockVolume(object):
65- def __init__(self, vol_id, device, instance_id, zone, size, status,
66- snapshot_id, tags):
67- self.id = vol_id
68- self.attach_data = MockAttachData(device, instance_id)
69- self.zone = zone
70- self.size = size
71- self.status = status
72- self.snapshot_id = snapshot_id
73- self.tags = tags
74-
75-
76 class TestEC2Util(mocker.MockerTestCase):
77
78 def setUp(self):
79- super(TestEC2Util, self).setUp()
80 self.maxDiff = None
81 util.hookenv = TestHookenv(
82 {"key": "ec2key", "secret": "ec2password",
83@@ -1005,8 +958,8 @@
84 configuration options.
85 """
86 command = "euca-describe-instances"
87- nova_cmd = self.mocker.replace(subprocess.check_call)
88- nova_cmd(command, shell=True)
89+ euca_cmd = self.mocker.replace(subprocess.check_call)
90+ euca_cmd(command, shell=True)
91 self.mocker.replay()
92
93 self.storage.validate_credentials()
94@@ -1242,39 +1195,47 @@
95
96 def test_wb_ec2_describe_volumes_command_error(self):
97 """
98- L{_ec2_describe_volumes} will exit in error when the euca2ools
99- C{DescribeVolumes} command fails.
100+ L{_ec2_describe_volumes} will exit in error when the
101+ C{euca-describe-volumes} command fails.
102 """
103- euca_command = self.mocker.replace(self.storage.ec2_volume_class)
104- euca_command()
105- self.mocker.throw(SystemExit(1))
106+ command = "euca-describe-volumes"
107+ euca_volumes = self.mocker.replace(subprocess.check_output)
108+ euca_volumes(command, shell=True)
109+ self.mocker.throw(subprocess.CalledProcessError(1, command))
110 self.mocker.replay()
111
112 result = self.assertRaises(
113 SystemExit, self.storage._ec2_describe_volumes)
114 self.assertEqual(result.code, 1)
115- message = "ERROR: Couldn't contact EC2 using euca-describe-volumes"
116+ message = (
117+ "ERROR: Command '%s' returned non-zero exit status 1" % command)
118 self.assertIn(
119 message, util.hookenv._log_ERROR, "Not logged- %s" % message)
120
121 def test_wb_ec2_describe_volumes_without_attached_instances(self):
122 """
123- L{_ec2_describe_volumes} parses the results of euca2ools
124- C{DescribeVolumes} to create a C{dict} of volume information. When no
125- C{instance_id}s are present the volumes are not attached so no
126- C{device} or C{instance_id} information will be present.
127+ L{_ec2_describe_volumes} parses the output from the
128+ C{euca-describe-volumes} command and returns a C{dict} of volume
129+ information. When no C{ATTACHMENT} response types are present the
130+ volumes are not attached so no C{device} or C{instance_id} information
131+ will be present. Any C{TAG} response types will be added to the
132+ associated volume.
133 """
134- volume1 = MockVolume(
135- "123-123-123", device="/dev/notshown", instance_id="notseen",
136- zone="ec2-az1", size="10", status="available",
137- snapshot_id="some-shot", tags={})
138- volume2 = MockVolume(
139- "456-456-456", device="/dev/notshown", instance_id="notseen",
140- zone="ec2-az2", size="8", status="available",
141- snapshot_id="some-shot", tags={"volume_name": "my volume name"})
142- euca_command = self.mocker.replace(self.storage.ec2_volume_class)
143- euca_command()
144- self.mocker.result(MockEucaCommand([volume1, volume2]))
145+ volume_output = (
146+ "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t"
147+ "2014-03-19T22:00:02.580Z\nVOLUME\t456-456-456\t 8\tsome-shot\t"
148+ "ec2-az2\tavailable\t2014-03-19T22:00:02.580Z\n"
149+ "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name")
150+ volume_command = "euca-describe-volumes"
151+ tag_output = (
152+ "TAG\t456-456-456\tvolume\tother_key\tother value")
153+ tag_command = "euca-describe-tags"
154+
155+ euca_command = self.mocker.replace(subprocess.check_output)
156+ euca_command(volume_command, shell=True)
157+ self.mocker.result(volume_output)
158+ euca_command(tag_command, shell=True)
159+ self.mocker.result(tag_output)
160 self.mocker.replay()
161
162 expected = {"123-123-123": {"id": "123-123-123", "status": "available",
163@@ -1290,27 +1251,30 @@
164 "volume_label": "my volume name",
165 "size": "8", "instance_id": "",
166 "snapshot_id": "some-shot",
167- "tags": {"volume_name": "my volume name"}}}
168+ "tags": {"volume_name": "my volume name",
169+ "other_key": "other value"}}}
170 self.assertEqual(self.storage._ec2_describe_volumes(), expected)
171
172 def test_wb_ec2_describe_volumes_matches_volume_id_supplied(self):
173 """
174- L{_ec2_describe_volumes} parses the results of euca2ools
175- C{DescribeVolumes} to create a C{dict} of volume information.
176- When C{volume_id} is provided return a C{dict} for the matched volume.
177+ L{_ec2_describe_volumes} parses the output of the
178+ C{euca-describe-volumes} command and returns a C{dict} of volume
179+ information. When C{volume_id} is provided, return a C{dict} for the
180+ matched volume.
181 """
182 volume_id = "123-123-123"
183- volume1 = MockVolume(
184- volume_id, device="/dev/notshown", instance_id="notseen",
185- zone="ec2-az1", size="10", status="available",
186- snapshot_id="some-shot", tags={})
187- volume2 = MockVolume(
188- "456-456-456", device="/dev/notshown", instance_id="notseen",
189- zone="ec2-az2", size="8", status="available",
190- snapshot_id="some-shot", tags={"volume_name": "my volume name"})
191- euca_command = self.mocker.replace(self.storage.ec2_volume_class)
192- euca_command()
193- self.mocker.result(MockEucaCommand([volume1, volume2]))
194+
195+ output = (
196+ "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t"
197+ "2014-03-19T22:00:02.580Z\nVOLUME\t456-456-456\t 8\tsome-shot\t"
198+ "ec2-az2\tavailable\t2014-03-19T22:00:02.580Z\n")
199+ command = "euca-describe-volumes"
200+
201+ euca_command = self.mocker.replace(subprocess.check_output)
202+ euca_command(command, shell=True)
203+ self.mocker.result(output)
204+ euca_command("euca-describe-tags", shell=True)
205+ self.mocker.result("\n")
206 self.mocker.replay()
207
208 expected = {
209@@ -1323,18 +1287,22 @@
210
211 def test_wb_ec2_describe_volumes_unmatched_volume_id_supplied(self):
212 """
213- L{_ec2_describe_volumes} parses the results of euca2ools
214- C{DescribeVolumes} to create a C{dict} of volume information.
215- When C{volume_id} is provided and unmatched, return an empty C{dict}.
216+ L{_ec2_describe_volumes} parses the output of the
217+ C{euca-describe-volumes} command and returns a C{dict} of volume
218+ information. When C{volume_id} is provided and unmatched, return an
219+ empty C{dict}.
220 """
221 unmatched_volume_id = "456-456-456"
222- volume1 = MockVolume(
223- "123-123-123", device="/dev/notshown", instance_id="notseen",
224- zone="ec2-az1", size="10", status="available",
225- snapshot_id="some-shot", tags={})
226- euca_command = self.mocker.replace(self.storage.ec2_volume_class)
227- euca_command()
228- self.mocker.result(MockEucaCommand([volume1]))
229+ output = (
230+ "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t"
231+ "2014-03-19T22:00:02.580Z\n")
232+ command = "euca-describe-volumes"
233+
234+ euca_command = self.mocker.replace(subprocess.check_output)
235+ euca_command(command, shell=True)
236+ self.mocker.result(output)
237+ euca_command("euca-describe-tags", shell=True)
238+ self.mocker.result("\n")
239 self.mocker.replay()
240
241 self.assertEqual(
242@@ -1342,22 +1310,28 @@
243
244 def test_wb_ec2_describe_volumes_with_attached_instances(self):
245 """
246- L{_ec2_describe_volumes} parses the results of euca2ools
247- C{DescribeVolumes} to create a C{dict} of volume information. If
248- C{status} is C{in-use}, both C{device} and C{instance_id} will be
249- returned in the C{dict}.
250+ L{_ec2_describe_volumes} parses the output of the
251+ C{euca-describe-volumes} command and returns a C{dict} of volume
252+ information. When C{status} is C{in-use}, both C{device} and
253+ C{instance_id} will be returned in the C{dict}.
254 """
255- volume1 = MockVolume(
256- "123-123-123", device="/dev/notshown", instance_id="notseen",
257- zone="ec2-az1", size="10", status="available",
258- snapshot_id="some-shot", tags={})
259- volume2 = MockVolume(
260- "456-456-456", device="/dev/xvdc", instance_id="i-456456",
261- zone="ec2-az2", size="8", status="in-use",
262- snapshot_id="some-shot", tags={"volume_name": "my volume name"})
263- euca_command = self.mocker.replace(self.storage.ec2_volume_class)
264- euca_command()
265- self.mocker.result(MockEucaCommand([volume1, volume2]))
266+ output = (
267+ "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t"
268+ "2014-03-19T22:00:02.580Z\nVOLUME\t456-456-456\t 8\tsome-shot\t"
269+ "ec2-az2\tin-use\t2014-03-19T22:00:02.580Z\nATTACHMENT\t"
270+ "456-456-456\ti-456456\t/dev/xvdc\tattached\t"
271+ "2014-03-19T22:00:02.000Z\nTAG\tvolume\t456-456-456\tvolume_name\t"
272+ "my volume name\n")
273+ command = "euca-describe-volumes"
274+ tag_output = (
275+ "TAG\t456-456-456\tvolume\tvolume_name\tmy volume name")
276+ tag_command = "euca-describe-tags"
277+
278+ euca_command = self.mocker.replace(subprocess.check_output)
279+ euca_command(command, shell=True)
280+ self.mocker.result(output)
281+ euca_command(tag_command, shell=True)
282+ self.mocker.result(tag_output)
283 self.mocker.replay()
284
285 expected = {"123-123-123": {"id": "123-123-123", "status": "available",
286@@ -1377,6 +1351,47 @@
287 self.assertEqual(
288 self.storage._ec2_describe_volumes(), expected)
289
290+ def test_wb_ec2_describe_volumes_orphaned_tags(self):
291+ """
292+ L{_ec2_describe_volumes} parses the output of the
293+ C{euca-describe-volumes} and augments that data with volume-related
294+ tags from C{euca-describe-tags}. When C{euca-describe-tags} returns
295+ information about orphaned tags for volumes that no longer exist,
296+ L{_ec2_describe_volumes} will log this information and move on.
297+ """
298+ output = (
299+ "VOLUME\t456-456-456\t 8\tsome-shot\t"
300+ "ec2-az2\tin-use\t2014-03-19T22:00:02.580Z\nATTACHMENT\t"
301+ "456-456-456\ti-456456\t/dev/xvdc\tattached\t"
302+ "2014-03-19T22:00:02.000Z\nTAG\tvolume\t456-456-456\tvolume_name\t"
303+ "my volume name\n")
304+ command = "euca-describe-volumes"
305+ tag_output = (
306+ "TAG\tvolume-not-here\tvolume\tvolume_name\tmy volume name\n"
307+ "TAG\t456-456-456\tvolume\tvolume_name\tmy volume name")
308+ tag_command = "euca-describe-tags"
309+
310+ euca_command = self.mocker.replace(subprocess.check_output)
311+ euca_command(command, shell=True)
312+ self.mocker.result(output)
313+ euca_command(tag_command, shell=True)
314+ self.mocker.result(tag_output)
315+ self.mocker.replay()
316+
317+ expected = {"456-456-456": {"id": "456-456-456", "status": "in-use",
318+ "device": "/dev/xvdc",
319+ "availability_zone": "ec2-az2",
320+ "volume_label": "my volume name",
321+ "size": "8", "instance_id": "i-456456",
322+ "snapshot_id": "some-shot",
323+ "tags": {"volume_name": "my volume name"}}}
324+ self.assertEqual(
325+ self.storage._ec2_describe_volumes(), expected)
326+ message = (
327+ "Ignoring tags for volume-id volume-not-here that doesn't exist")
328+ self.assertIn(
329+ message, util.hookenv._log_INFO, "Not logged- %s" % message)
330+
331 def test_wb_ec2_create_volume(self):
332 """
333 L{_ec2_create_volume} uses the command C{euca-create-volume} to create
334@@ -1392,18 +1407,17 @@
335 zone = "ec2-az3"
336 command = "euca-create-volume -z %s -s %s" % (zone, size)
337
338- reservation = MockEucaReservation(
339- [MockEucaInstance(
340- instance_id=instance_id, availability_zone=zone)])
341- euca_command = self.mocker.replace(self.storage.ec2_instance_class)
342- euca_command()
343- self.mocker.result(MockEucaCommand([reservation]))
344 create = self.mocker.replace(subprocess.check_output)
345 create(command, shell=True)
346 self.mocker.result(
347 "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id)
348 self.mocker.replay()
349
350+ def mock_describe_instances(my_id):
351+ self.assertEqual(my_id, instance_id)
352+ return {"availability_zone": zone}
353+ self.storage._ec2_describe_instances = mock_describe_instances
354+
355 def mock_describe_volumes(my_id):
356 self.assertEqual(my_id, volume_id)
357 return {"id": volume_id}
358@@ -1436,10 +1450,10 @@
359 volume_label = "postgresql/0 unit volume"
360 size = 10
361
362- euca_command = self.mocker.replace(self.storage.ec2_instance_class)
363- euca_command()
364- self.mocker.result(MockEucaCommand([])) # Empty results from euca
365- self.mocker.replay()
366+ def mock_describe_instances(my_id):
367+ self.assertEqual(my_id, instance_id)
368+ return {} # No results found
369+ self.storage._ec2_describe_instances = mock_describe_instances
370
371 result = self.assertRaises(
372 SystemExit, self.storage._ec2_create_volume, size, volume_label,
373@@ -1465,17 +1479,16 @@
374 zone = "ec2-az3"
375 command = "euca-create-volume -z %s -s %s" % (zone, size)
376
377- reservation = MockEucaReservation(
378- [MockEucaInstance(
379- instance_id=instance_id, availability_zone=zone)])
380- euca_command = self.mocker.replace(self.storage.ec2_instance_class)
381- euca_command()
382- self.mocker.result(MockEucaCommand([reservation]))
383 create = self.mocker.replace(subprocess.check_output)
384 create(command, shell=True)
385 self.mocker.result("INSTANCE invalid-instance-type-response\n")
386 self.mocker.replay()
387
388+ def mock_describe_instances(my_id):
389+ self.assertEqual(my_id, instance_id)
390+ return {"availability_zone": zone}
391+ self.storage._ec2_describe_instances = mock_describe_instances
392+
393 def mock_describe_volumes(my_id):
394 raise Exception("_ec2_describe_volumes should not be called")
395 self.storage._ec2_describe_volumes = mock_describe_volumes
396@@ -1507,18 +1520,17 @@
397 zone = "ec2-az3"
398 command = "euca-create-volume -z %s -s %s" % (zone, size)
399
400- reservation = MockEucaReservation(
401- [MockEucaInstance(
402- instance_id=instance_id, availability_zone=zone)])
403- euca_command = self.mocker.replace(self.storage.ec2_instance_class)
404- euca_command()
405- self.mocker.result(MockEucaCommand([reservation]))
406 create = self.mocker.replace(subprocess.check_output)
407 create(command, shell=True)
408 self.mocker.result(
409 "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id)
410 self.mocker.replay()
411
412+ def mock_describe_instances(my_id):
413+ self.assertEqual(my_id, instance_id)
414+ return {"availability_zone": zone}
415+ self.storage._ec2_describe_instances = mock_describe_instances
416+
417 def mock_describe_volumes(my_id):
418 self.assertEqual(my_id, volume_id)
419 return {} # No details found for this volume
420@@ -1548,17 +1560,16 @@
421 zone = "ec2-az3"
422 command = "euca-create-volume -z %s -s %s" % (zone, size)
423
424- reservation = MockEucaReservation(
425- [MockEucaInstance(
426- instance_id=instance_id, availability_zone=zone)])
427- euca_command = self.mocker.replace(self.storage.ec2_instance_class)
428- euca_command()
429- self.mocker.result(MockEucaCommand([reservation]))
430 create = self.mocker.replace(subprocess.check_output)
431 create(command, shell=True)
432 self.mocker.throw(subprocess.CalledProcessError(1, command))
433 self.mocker.replay()
434
435+ def mock_describe_instances(my_id):
436+ self.assertEqual(my_id, instance_id)
437+ return {"availability_zone": zone}
438+ self.storage._ec2_describe_instances = mock_describe_instances
439+
440 def mock_exception(my_id):
441 raise Exception("These methods should not be called")
442 self.storage._ec2_describe_volumes = mock_exception
443@@ -1728,3 +1739,243 @@
444 (volume_id, instance_id))
445 self.assertIn(
446 message, util.hookenv._log_INFO, "Not logged- %s" % message)
447+
448+ def test_wb_ec2_describe_instances_euca_v2(self):
449+ """
450+ L{_ec2_describe_instances} parses the output of the command
451+ C{euca-describe-instances} for euca2ools version 2.X and returns a
452+ C{dict} of instance information.
453+ """
454+ command = "euca-describe-instances"
455+ output = (
456+ "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t"
457+ "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t"
458+ "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t"
459+ "monitoring-disabled\t54.81.8.9\t10.138.61.2\t\t\tebs\t\t\t\t\t"
460+ "paravirtual\t\t\t\t")
461+
462+ ec2_list = self.mocker.replace(subprocess.check_output)
463+ ec2_list(command, shell=True)
464+ self.mocker.result(output)
465+ self.mocker.replay()
466+
467+ expected = {
468+ "i-aaa": {
469+ "ip-address": "54.81.8.9", "private-ip-address": "10.138.61.2",
470+ "kernel": "aki-aaa", "instance-type": "m1.small",
471+ "state": "running", "public-dns-name": "aaa.amazonaws.com",
472+ "private-dns-name": "aaa.ec2.internal", "instance_id": "i-aaa",
473+ "image-id": "ami-aaa", "availability_zone": "us-east-1a"}}
474+
475+ self.assertEqual(
476+ EUCA2_INSTANCE_RESPONSE_LENGTH_V2, len(output.split("\t")))
477+
478+ self.assertEqual(
479+ self.storage._ec2_describe_instances(), expected)
480+
481+ def test_wb_ec2_describe_instances_euca_invalid_output_version(self):
482+ """
483+ L{_ec2_describe_instances} parses the output of the command
484+ C{euca-describe-instances} and if it sees an output length that is
485+ unsupported, an error is logged.
486+ """
487+ command = "euca-describe-instances"
488+ output = (
489+ "INSTANCE\ti-aaa\tami-0b9c9f62\tblah.amazonaws.com\t"
490+ "blah.ec2.internal\trunning\t\t0\t")
491+
492+ ec2_list = self.mocker.replace(subprocess.check_output)
493+ ec2_list(command, shell=True)
494+ self.mocker.result(output)
495+ self.mocker.replay()
496+
497+ self.assertNotIn(
498+ len(output.split("\t")),
499+ [EUCA2_INSTANCE_RESPONSE_LENGTH_V3,
500+ EUCA2_INSTANCE_RESPONSE_LENGTH_V3])
501+
502+ result = self.assertRaises(
503+ SystemExit, self.storage._ec2_describe_instances)
504+ self.assertEqual(result.code, 1)
505+ message = "ERROR: Unsupported euca INSTANCE output format version"
506+ self.assertIn(
507+ message, util.hookenv._log_ERROR, "Not logged- %s" % message)
508+
509+ def test_wb_ec2_describe_instances_specific_instance_id(self):
510+ """
511+ L{_ec2_describe_instances} parses the output of the command
512+ C{euca-describe-instances} for euca2ools version 3.X and returns a
513+ C{dict} of instance information. When C{instance_id} is provided,
514+ results are filtered to only the matching C{instance_id}.
515+ """
516+ command = "euca-describe-instances"
517+ output = (
518+ "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t"
519+ "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t"
520+ "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t"
521+ "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs"
522+ "\t\t\t\t\tparavirtual\txen\t"
523+ "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3"
524+ "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t\n"
525+ "INSTANCE\ti-bbb\tami-bbb\tbbb.amazonaws.com\t"
526+ "bbb.ec2.internal\trunning\t\t0\t\tm1.small\t"
527+ "2014-03-19T17:52:02.000Z\tus-east-1a\taki-bbb\t\t\t"
528+ "monitoring-disabled\t54.81.8.2\t10.138.61.2\t\t\tebs"
529+ "\t\t\t\t\tparavirtual\txen\t"
530+ "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3"
531+ "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t")
532+
533+ ec2_list = self.mocker.replace(subprocess.check_output)
534+ ec2_list(command, shell=True)
535+ self.mocker.result(output)
536+ self.mocker.replay()
537+
538+ expected = {
539+ "ip-address": "54.81.8.1", "private-ip-address": "10.138.61.1",
540+ "kernel": "aki-aaa", "instance-type": "m1.small",
541+ "state": "running", "public-dns-name": "aaa.amazonaws.com",
542+ "private-dns-name": "aaa.ec2.internal", "instance_id": "i-aaa",
543+ "image-id": "ami-aaa", "availability_zone": "us-east-1a"}
544+
545+ self.assertEqual(
546+ EUCA2_INSTANCE_RESPONSE_LENGTH_V3,
547+ len(output.split("\n")[0].split("\t")))
548+
549+ self.assertEqual(
550+ self.storage._ec2_describe_instances("i-aaa"), expected)
551+
552+ def test_parse_ec2_tag_response_not_tag_response(self):
553+ """
554+ L{parse_ec2_tag_response} returns an empty C{dict} when the
555+ line does not represent a C{TAG} response type.
556+ """
557+ non_tag_output_lines = (
558+ "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t"
559+ "2014-03-19T22:00:02.580Z\n"
560+ "ATTACHMENT\t123-123-123\ti-123123\t/dev/xvdc\tattached\t"
561+ "2014-03-19T22:00:02.000Z\n"
562+ "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t"
563+ "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t"
564+ "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t"
565+ "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs"
566+ "\t\t\t\t\tparavirtual\txen\t"
567+ "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3"
568+ "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t")
569+ for line in non_tag_output_lines.split("\n"):
570+ self.assertEqual(parse_ec2_tag_response(line), {})
571+
572+ def test_parse_ec2_attachment_response_not_attachment_response(self):
573+ """
574+ L{parse_ec2_attachment_response} returns an empty C{dict} when the
575+ line does not represent a C{ATTACHMENT} response type.
576+ """
577+ non_attachment_output_lines = (
578+ "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name\n"
579+ "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t"
580+ "2014-03-19T22:00:02.580Z\n"
581+ "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t"
582+ "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t"
583+ "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t"
584+ "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs"
585+ "\t\t\t\t\tparavirtual\txen\t"
586+ "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3"
587+ "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t")
588+ for line in non_attachment_output_lines.split("\n"):
589+ self.assertEqual(parse_ec2_attachment_response(line), {})
590+
591+ def test_parse_ec2_volume_response_not_volume_response(self):
592+ """
593+ L{parse_ec2_volume_response} returns an empty C{dict} when the
594+ line does not represent a C{VOLUME} response type.
595+ """
596+ non_volume_output_lines = (
597+ "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name\n"
598+ "ATTACHMENT\t123-123-123\ti-123123\t/dev/xvdc\tattached\t"
599+ "2014-03-19T22:00:02.000Z\n"
600+ "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t"
601+ "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t"
602+ "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t"
603+ "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs"
604+ "\t\t\t\t\tparavirtual\txen\t"
605+ "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3"
606+ "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t")
607+ for line in non_volume_output_lines.split("\n"):
608+ self.assertEqual(parse_ec2_volume_response(line), {})
609+
610+ def test_parse_ec2_instance_response_not_instance_response(self):
611+ """
612+ L{parse_ec2_instance_response} returns an empty C{dict} when the
613+ line does not represent a C{INSTANCE} response type.
614+ """
615+ non_instance_output_lines = (
616+ "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name\n"
617+ "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t"
618+ "2014-03-19T22:00:02.580Z\n"
619+ "ATTACHMENT\t123-123-123\ti-123123\t/dev/xvdc\tattached\t"
620+ "2014-03-19T22:00:02.000Z\n")
621+ for line in non_instance_output_lines.split("\n"):
622+ self.assertEqual(parse_ec2_instance_response(line), {})
623+
624+ def test_parse_ec2_instance_response(self):
625+ """
626+ L{parse_ec2_instance_response} returns a C{dict} of the parsed
627+ C{INSTANCE} response type.
628+ """
629+ output = (
630+ "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t"
631+ "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t"
632+ "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t"
633+ "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs"
634+ "\t\t\t\t\tparavirtual\txen\t"
635+ "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3"
636+ "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t")
637+
638+ expected = {
639+ "ip-address": "54.81.8.1", "private-ip-address": "10.138.61.1",
640+ "image-id": "ami-aaa", "instance-type": "m1.small",
641+ "kernel": "aki-aaa", "private-dns-name": "aaa.ec2.internal",
642+ "public-dns-name": "aaa.amazonaws.com", "state": "running",
643+ "availability_zone": "us-east-1a", "instance_id": "i-aaa"}
644+
645+ self.assertEqual(parse_ec2_instance_response(output), expected)
646+
647+ def test_parse_ec2_tag_response(self):
648+ """
649+ L{parse_ec2_tag_response} returns a C{dict} of the parsed C{TAG}
650+ response type.
651+ """
652+ output = "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name"
653+
654+ expected = {
655+ "tag_type": "volume", "id": "456-456-456", "key": "volume_name",
656+ "value": "my volume name"}
657+ self.assertEqual(parse_ec2_tag_response(output), expected)
658+
659+ def test_parse_ec2_attachment_response(self):
660+ """
661+ L{parse_ec2_attachment_response} returns a C{dict} of the parsed
662+ C{ATTACHMENT} response type.
663+ """
664+ output = (
665+ "ATTACHMENT\t123-123-123\ti-123123\t/dev/xvdc\tattached\t"
666+ "2014-03-19T22:00:02.000Z\n")
667+
668+ expected = {
669+ "volume_id": "123-123-123", "instance_id": "i-123123",
670+ "device": "/dev/xvdc", "attach_status": "attached"}
671+ self.assertEqual(parse_ec2_attachment_response(output), expected)
672+
673+ def test_parse_ec2_volume_response(self):
674+ """
675+ L{parse_ec2_volume_response} returns a C{dict} of the parsed
676+ C{VOLUME} response type.
677+ """
678+ output = (
679+ "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t"
680+ "2014-03-19T22:00:02.580Z\n")
681+
682+ expected = {
683+ "id": "123-123-123", "size": "10",
684+ "snapshot_id": "some-shot", "availability_zone": "ec2-az1",
685+ "status": "available"}
686+ self.assertEqual(parse_ec2_volume_response(output), expected)
687
688=== modified file 'hooks/util.py'
689--- hooks/util.py 2014-03-21 17:39:48 +0000
690+++ hooks/util.py 2014-08-08 02:34:47 +0000
691@@ -23,6 +23,9 @@
692 "nova": {"validate": "nova list",
693 "detach": "nova volume-detach %s %s"}}
694
695+EUCA2_INSTANCE_RESPONSE_LENGTH_V2 = 30
696+EUCA2_INSTANCE_RESPONSE_LENGTH_V3 = 32
697+
698
699 class StorageServiceUtil(object):
700 """Interact with an underlying cloud storage provider.
701@@ -45,11 +48,6 @@
702 self.environment_map = ENVIRONMENT_MAP[provider]
703 self.commands = PROVIDER_COMMANDS[provider]
704 self.required_config_options = REQUIRED_CONFIG_OPTIONS[provider]
705- if provider == "ec2":
706- import euca2ools.commands.euca.describevolumes as getvolumes
707- import euca2ools.commands.euca.describeinstances as getinstances
708- self.ec2_volume_class = getvolumes.DescribeVolumes
709- self.ec2_instance_class = getinstances.DescribeInstances
710
711 def load_environment(self):
712 """
713@@ -212,83 +210,112 @@
714 return
715
716 # EC2-specific methods
717+ def _run_command(self, command):
718+ try:
719+ output = subprocess.check_call(command, shell=True)
720+ except subprocess.CalledProcessError, e:
721+ hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
722+ sys.exit(1)
723+
724+ # Split the output by lines
725+ lines = []
726+ if output:
727+ for line in output.split("\n"):
728+ line = line.strip()
729+ if line:
730+ lines.append(line)
731+ return lines
732+
733 def _ec2_create_tag(self, volume_id, tag_name, tag_value=None):
734 """Attach a tag and optional C{tag_value} to the given C{volume_id}"""
735 tag_string = tag_name
736 if tag_value:
737 tag_string += "=%s" % tag_value
738- command = 'euca-create-tags %s --tag "%s"' % (volume_id, tag_string)
739+ command = "euca-create-tags %s --tag \"%s\"" % (volume_id, tag_string)
740
741- try:
742- subprocess.check_call(command, shell=True)
743- except subprocess.CalledProcessError, e:
744- hookenv.log(
745- "ERROR: Couldn't add tags to the resource. %s" % str(e),
746- hookenv.ERROR)
747- sys.exit(1)
748+ self._run_command(command)
749 hookenv.log("Tagged (%s) to %s." % (tag_string, volume_id))
750
751 def _ec2_describe_instances(self, instance_id=None):
752 """
753- Use euca2ools libraries to describe instances and return a C{dict}
754+ Use euca2ools command output to describe instances and return a C{dict}
755 """
756 result = {}
757- try:
758- command = self.ec2_instance_class()
759- reservations = command.main()
760- except SystemExit:
761- hookenv.log(
762- "ERROR: Couldn't contact EC2 using euca-describe-instances",
763- hookenv.ERROR)
764- sys.exit(1)
765- for reservation in reservations:
766- for inst in reservation.instances:
767- result[inst.id] = {
768- "ip-address": inst.ip_address, "image-id": inst.image_id,
769- "instance-type": inst.image_id, "kernel": inst.kernel,
770- "private-dns-name": inst.private_dns_name,
771- "public-dns-name": inst.public_dns_name,
772- "reservation-id": reservation.id,
773- "state": inst.state, "tags": inst.tags,
774- "availability_zone": inst.placement}
775+
776+ lines = self._run_command("euca-describe-instances")
777+ for line in lines:
778+ response_type = line.split("\t")[0]
779+ if response_type != "INSTANCE":
780+ continue
781+ instance_data = parse_ec2_instance_response(line)
782+ result[instance_data["instance_id"]] = instance_data
783+
784 if instance_id:
785 if instance_id in result:
786 return result[instance_id]
787 return {}
788 return result
789
790+ def _ec2_describe_volume_tags(self):
791+ """
792+ Parse output of C{euca-describe-tags} returning a C{dict} of volume ids
793+ and any associated tags.
794+ """
795+ result = {}
796+
797+ lines = self._run_command("euca-describe-tags")
798+ for line in lines:
799+ response_type = line.split("\t")[0]
800+ if response_type != "TAG":
801+ continue
802+ data = parse_ec2_tag_response(line)
803+ if not result.get(data["id"]):
804+ result[data["id"]] = {"tags": {}}
805+ result[data["id"]]["tags"].update({data["key"]: data["value"]})
806+ return result
807+
808 def _ec2_describe_volumes(self, volume_id=None):
809 """
810- Use euca2ools libraries to describe volumes and return a C{dict}
811+ Parse output of C{euca-describe-volumes} returning a C{dict} of volume
812+ ids all related volume information.
813 """
814 result = {}
815- try:
816- command = self.ec2_volume_class()
817- volumes = command.main()
818- except SystemExit:
819- hookenv.log(
820- "ERROR: Couldn't contact EC2 using euca-describe-volumes",
821- hookenv.ERROR)
822- sys.exit(1)
823- for volume in volumes:
824- result[volume.id] = {
825- "device": "",
826- "instance_id": "",
827- "size": volume.size,
828- "snapshot_id": volume.snapshot_id,
829- "status": volume.status,
830- "tags": volume.tags,
831- "id": volume.id,
832- "availability_zone": volume.zone}
833- if "volume_name" in volume.tags:
834- result[volume.id]["volume_label"] = volume.tags["volume_name"]
835- else:
836- result[volume.id]["tags"]["volume_name"] = ""
837- result[volume.id]["volume_label"] = ""
838- if volume.status == "in-use":
839- result[volume.id]["instance_id"] = (
840- volume.attach_data.instance_id)
841- result[volume.id]["device"] = volume.attach_data.device
842+
843+ lines = self._run_command("euca-describe-volumes")
844+ for line in lines:
845+ response_type = line.split("\t")[0]
846+ if response_type == "VOLUME":
847+ data = parse_ec2_volume_response(line)
848+ data["device"] = ""
849+ data["volume_label"] = ""
850+ data["instance_id"] = ""
851+ data["tags"] = {"volume_name": ""}
852+ result[data["id"]] = data
853+ elif response_type == "TAG":
854+ data = parse_ec2_tag_response(line)
855+ tags = result[data["id"]]["tags"]
856+ tags.update({data["key"]: data["value"]})
857+ volume_name = tags.get("volume_name")
858+ if volume_name:
859+ result[data["id"]]["volume_label"] = volume_name
860+ elif response_type == "ATTACHMENT":
861+ data = parse_ec2_attachment_response(line)
862+ volume_data = result[data["volume_id"]]
863+ volume_data.update({"instance_id": data["instance_id"]})
864+ volume_data.update({"device": data["device"]})
865+ else:
866+ continue
867+
868+ volume_tag_data = self._ec2_describe_volume_tags()
869+ for tag_volume_id in volume_tag_data.keys():
870+ additional_tags = volume_tag_data[tag_volume_id]["tags"]
871+ if tag_volume_id not in result:
872+ hookenv.log(
873+ "Ignoring tags for volume-id %s that doesn't exist" %
874+ tag_volume_id)
875+ else:
876+ result[tag_volume_id]["tags"].update(additional_tags)
877+
878 if volume_id:
879 if volume_id in result:
880 return result[volume_id]
881@@ -311,19 +338,15 @@
882 (instance_id, config_data["endpoint"]), hookenv.ERROR)
883 sys.exit(1)
884
885- try:
886- output = subprocess.check_output(
887- "euca-create-volume -z %s -s %s" %
888- (instance["availability_zone"], size), shell=True)
889- except subprocess.CalledProcessError, e:
890- hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
891- sys.exit(1)
892+ command = "euca-create-volume -z %s -s %s" % (
893+ instance["availability_zone"], size)
894
895- response_type, volume_id = output.split()[:2]
896+ lines = self._run_command(command)
897+ response_type, volume_id = lines[:2]
898 if response_type != "VOLUME":
899 hookenv.log(
900 "ERROR: Didn't get VOLUME response from euca-create-volume. "
901- "Response: %s" % output, hookenv.ERROR)
902+ "Response: %s" % "".join(lines), hookenv.ERROR)
903 sys.exit(1)
904 volume = self.describe_volumes(volume_id.strip())
905 if not volume:
906@@ -341,13 +364,10 @@
907 the device path.
908 """
909 device = "/dev/xvdc"
910- try:
911- subprocess.check_call(
912- "euca-attach-volume -i %s -d %s %s" %
913- (instance_id, device, volume_id), shell=True)
914- except subprocess.CalledProcessError, e:
915- hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
916- sys.exit(1)
917+ command = "euca-attach-volume -i %s -d %s %s" % (
918+ instance_id, device, volume_id)
919+
920+ self._run_command(command)
921 return device
922
923 # Nova-specific methods
924@@ -358,17 +378,9 @@
925 """
926 from ast import literal_eval
927 result = {"tags": {}, "instance_id": "", "device": ""}
928- command = "nova volume-show '%s'" % volume_id
929- try:
930- output = subprocess.check_output(command, shell=True)
931- except subprocess.CalledProcessError, e:
932- hookenv.log(
933- "ERROR: Failed to get nova volume info. %s" % str(e),
934- hookenv.ERROR)
935- sys.exit(1)
936- for line in output.split("\n"):
937- if not line.strip(): # Skip empty lines
938- continue
939+
940+ lines = self._run_command("nova volume-show '%s'" % volume_id)
941+ for line in lines:
942 if "+----" in line or "Property" in line:
943 continue
944 (_, key, value, _) = line.split("|")
945@@ -393,15 +405,9 @@
946 def _nova_describe_volumes(self, volume_id=None):
947 """Create a C{dict} describing all nova volumes"""
948 result = {}
949- command = "nova volume-list"
950- try:
951- output = subprocess.check_output(command, shell=True)
952- except subprocess.CalledProcessError, e:
953- hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
954- sys.exit(1)
955- for line in output.split("\n"):
956- if not line.strip(): # Skip empty lines
957- continue
958+
959+ lines = self._run_command("nova volume-list")
960+ for line in lines:
961 if "+----" in line or "ID" in line:
962 continue
963 values = line.split("|")
964@@ -434,15 +440,12 @@
965 Attach a Nova C{volume_id} to the provided C{instance_id} and return
966 the device path.
967 """
968- try:
969- device = subprocess.check_output(
970- "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" %
971- (instance_id, volume_id), shell=True)
972- except subprocess.CalledProcessError, e:
973- hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
974- sys.exit(1)
975- if device.strip():
976- return device.strip()
977+ command = (
978+ "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" %
979+ (instance_id, volume_id))
980+ lines = self._run_ec2_command(command)
981+ if lines:
982+ return lines[0]
983 return ""
984
985 def _nova_create_volume(self, size, volume_label, instance_id):
986@@ -450,14 +453,11 @@
987 hookenv.log(
988 "Creating a %sGig volume named (%s) for instance %s" %
989 (size, volume_label, instance_id))
990- try:
991- subprocess.check_call(
992- "nova volume-create --display-name '%s' %s" %
993- (volume_label, size), shell=True)
994- except subprocess.CalledProcessError, e:
995- hookenv.log("ERROR: %s" % str(e), hookenv.ERROR)
996- sys.exit(1)
997+ command = (
998+ "nova volume-create --display-name '%s' %s" %
999+ (volume_label, size))
1000
1001+ self._run_command(command)
1002 volume_id = self.get_volume_id(volume_label)
1003 if not volume_id:
1004 hookenv.log(
1005@@ -467,6 +467,102 @@
1006 return volume_id
1007
1008
1009+def parse_ec2_tag_response(line):
1010+ """
1011+ Parses a line of output of response type TAG from euca commands and
1012+ returns a C{dict} of the values. If not a TAG response, an empty C{dict} is
1013+ returned.
1014+ """
1015+ values = line.split("\t")
1016+ response_type = values[0]
1017+ if response_type != "TAG":
1018+ return {}
1019+
1020+ # There are two versions of the TAG response
1021+ if values[1] == "volume":
1022+ tag_type = values[1]
1023+ object_id = values[2]
1024+ else:
1025+ tag_type = values[2]
1026+ object_id = values[1]
1027+ if tag_type != "volume":
1028+ hookenv.log(
1029+ "Ignoring non-volume TAG response: %s." % line, hookenv.WARNING)
1030+ return {}
1031+ return {
1032+ "tag_type": tag_type, "id": object_id, "key": values[3],
1033+ "value": values[4]}
1034+
1035+
1036+def parse_ec2_attachment_response(line):
1037+ """
1038+ Parses a line of output of response type ATTACHMENT from euca commands and
1039+ returns a C{dict} of the values. If not a ATTACHMENT response, an empty
1040+ C{dict} is returned.
1041+ """
1042+ values = line.split("\t")
1043+ response_type = values[0]
1044+ if response_type != "ATTACHMENT":
1045+ return {}
1046+
1047+ return {
1048+ "volume_id": values[1], "instance_id": values[2], "device": values[3],
1049+ "attach_status": values[4]}
1050+
1051+
1052+def parse_ec2_volume_response(line):
1053+ """
1054+ Parses a line of output of response type VOLUME from euca commands and
1055+ returns a C{dict} of the values. If not a VOLUME response, an empty C{dict}
1056+ is returned.
1057+ """
1058+ values = line.split("\t")
1059+ response_type = values[0]
1060+ if response_type != "VOLUME":
1061+ return {}
1062+
1063+ return {
1064+ "id": values[1], "size": values[2].strip(),
1065+ "snapshot_id": values[3], "availability_zone": values[4],
1066+ "status": values[5]}
1067+
1068+
1069+def parse_ec2_instance_response(line):
1070+ """
1071+ Parses a line of output of response type INSTANCE from euca commands and
1072+ returns a C{dict} of the values. If not an INSTANCE response, and empty
1073+ C{dict} is returned.
1074+ """
1075+ values = line.split("\t")
1076+ response_type = values[0]
1077+ if response_type != "INSTANCE":
1078+ return {}
1079+
1080+ if len(values) not in [EUCA2_INSTANCE_RESPONSE_LENGTH_V2,
1081+ EUCA2_INSTANCE_RESPONSE_LENGTH_V3]:
1082+ hookenv.log(
1083+ "ERROR: Unsupported euca INSTANCE output format version",
1084+ hookenv.ERROR)
1085+ sys.exit(1)
1086+
1087+ (ec2_instance_id, image_id) = values[1:3]
1088+ public_dns = values[3]
1089+ private_dns = values[4]
1090+ state = values[5]
1091+ instance_type = values[9]
1092+ availability_zone = values[11]
1093+ kernel = values[12]
1094+ public_ip = values[16]
1095+ private_ip = values[17]
1096+
1097+ return {
1098+ "ip-address": public_ip, "private-ip-address": private_ip,
1099+ "image-id": image_id, "instance-type": instance_type,
1100+ "kernel": kernel, "private-dns-name": private_dns,
1101+ "public-dns-name": public_dns, "state": state,
1102+ "availability_zone": availability_zone, "instance_id": ec2_instance_id}
1103+
1104+
1105 def generate_volume_label(remote_unit):
1106 """Create a volume label for the requesting remote unit"""
1107 return "%s unit volume" % remote_unit

Subscribers

People subscribed via source and target branches

to all changes: