Merge lp:~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support into lp:~chad.smith/charms/precise/block-storage-broker/trunk
- Precise Pangolin (12.04)
- bsb-trusty-support
- Merge into trunk
Status: | Rejected |
---|---|
Rejected by: | Chad Smith |
Proposed branch: | lp:~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support |
Merge into: | lp:~chad.smith/charms/precise/block-storage-broker/trunk |
Prerequisite: | lp:~chad.smith/charms/precise/block-storage-broker/bsb-ec2-support |
Diff against target: |
1107 lines (+602/-255) 2 files modified
hooks/test_util.py (+394/-143) hooks/util.py (+208/-112) |
To merge this branch: | bzr merge lp:~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Chad Smith | Disapprove | ||
Alberto Donato (community) | Needs Fixing | ||
Review via email: mp+211963@code.launchpad.net |
Commit message
Description of the change
This branch does a couple things:
1. stops the block-storage-
2. To handle the fact that euca-2.X doesn't report volume TAG responses in the euca-describe-
3. In using euca-describe-tags to augment existing volume data I found that tags can be orphaned once a volume is removed by an admin via euca-delete-volume, if that is the case we will ignore tags for volumes that don't exist and log a message
4. In unit tests drop all MockEuca* classes as they are no longer needed because we mock the subprocess.
By parsing the output from the euca-* commands, we shield ourselves from the internal changes in class structure definition in future euca releases. This branch adds very basic validation of euca-describe-
- 87. By Chad Smith
-
merge fixes from bsb-ec2-support and resolve conflicts + lint
- 88. By Chad Smith
-
revert Makefile change
- 89. By Chad Smith
-
Ignore any orphaned TAG lines from euca-describe-tags that don't map to current existing volume-ids
Chad Smith (chad.smith) wrote : | # |
Chad Smith (chad.smith) wrote : | # |
The bundle above works from the w/ juju 1.17.6 be wary of 1.17. < 6 as it suffers from bug 1285901
Alberto Donato (ack) wrote : | # |
As discussed on IRC, I think the code could be refactored to reduce duplication, for example as in https:/
Tests could then mock _run_command rather than subprocess.
It should be even possible to perform the line.split("\n") in the _run_command method, returning a list of lists (so the parse_* methods don't need to perform it), I'm not sure if it fits all the use cases, though.
#1:
+ data["device"] = ""
+ data["volume_
+ data["instance_id"] = ""
+ data["tags"] = {"volume_name": ""}
These can be set with a single data.update()
#2:
+ volume_
+ volume_
Same as #1
#3:
+ for tag_volume_id in volume_
+ additional_tags = volume_
+ if tag_volume_id not in result:
+ hookenv.log(
+ "Ignoring tags for volume-id %s that doesn't exist" %
+ tag_volume_id)
else:
The first line inside the loop can be moved inside the else clause.
Also there's an extra indentation space on the hookenv.log() call
#4:
I think you can drop the checks on the response_type in the parse_* methods, since it's already checked by call sites (related tests can be dropped then).
- 90. By Chad Smith
-
pull in ack's _run_command patch for simplification
Chad Smith (chad.smith) wrote : | # |
Rejecting this branch in favor of dropping euca2ools and using Amazon's python SDK: python-boto as boto is more stable across Ubuntu releases than trying to parse and unify the output of euca2ools commandline utilities.
Unmerged revisions
- 90. By Chad Smith
-
pull in ack's _run_command patch for simplification
- 89. By Chad Smith
-
Ignore any orphaned TAG lines from euca-describe-tags that don't map to current existing volume-ids
- 88. By Chad Smith
-
revert Makefile change
- 87. By Chad Smith
-
merge fixes from bsb-ec2-support and resolve conflicts + lint
- 86. By Chad Smith
-
now that _ec2_describe_
volumes calls euca-describe-tags, we need to mock that in unit tests. Add unit tests for parse_ec2_*_reponse functions - 85. By Chad Smith
-
docstring updates
- 84. By Chad Smith
-
perform validation of INSTANCE reponse_type before attmpting to parse the value in the line. lint fixes in unit tests
- 83. By Chad Smith
-
parse euca-describe-
instances and euca-describe- volumes output instead of tying into euca2ools. commands. euca.describein stances directly
Preview Diff
1 | === modified file 'hooks/test_util.py' | |||
2 | --- hooks/test_util.py 2014-03-21 17:42:00 +0000 | |||
3 | +++ hooks/test_util.py 2014-08-08 02:34:47 +0000 | |||
4 | @@ -1,5 +1,9 @@ | |||
5 | 1 | import util | 1 | import util |
7 | 2 | from util import StorageServiceUtil, ENVIRONMENT_MAP, generate_volume_label | 2 | from util import ( |
8 | 3 | StorageServiceUtil, EUCA2_INSTANCE_RESPONSE_LENGTH_V3, | ||
9 | 4 | EUCA2_INSTANCE_RESPONSE_LENGTH_V2, parse_ec2_tag_response, | ||
10 | 5 | parse_ec2_volume_response, parse_ec2_instance_response, | ||
11 | 6 | parse_ec2_attachment_response, ENVIRONMENT_MAP, generate_volume_label) | ||
12 | 3 | import mocker | 7 | import mocker |
13 | 4 | import os | 8 | import os |
14 | 5 | import subprocess | 9 | import subprocess |
15 | @@ -9,7 +13,6 @@ | |||
16 | 9 | class TestNovaUtil(mocker.MockerTestCase): | 13 | class TestNovaUtil(mocker.MockerTestCase): |
17 | 10 | 14 | ||
18 | 11 | def setUp(self): | 15 | def setUp(self): |
19 | 12 | super(TestNovaUtil, self).setUp() | ||
20 | 13 | self.maxDiff = None | 16 | self.maxDiff = None |
21 | 14 | util.hookenv = TestHookenv( | 17 | util.hookenv = TestHookenv( |
22 | 15 | {"key": "myusername", "tenant": "myusername_project", | 18 | {"key": "myusername", "tenant": "myusername_project", |
23 | @@ -880,59 +883,9 @@ | |||
24 | 880 | message, util.hookenv._log_INFO, "Not logged- %s" % message) | 883 | message, util.hookenv._log_INFO, "Not logged- %s" % message) |
25 | 881 | 884 | ||
26 | 882 | 885 | ||
27 | 883 | class MockEucaCommand(object): | ||
28 | 884 | def __init__(self, result): | ||
29 | 885 | self.result = result | ||
30 | 886 | |||
31 | 887 | def main(self): | ||
32 | 888 | return self.result | ||
33 | 889 | |||
34 | 890 | |||
35 | 891 | class MockEucaReservation(object): | ||
36 | 892 | def __init__(self, instances): | ||
37 | 893 | self.instances = instances | ||
38 | 894 | self.id = 1 | ||
39 | 895 | |||
40 | 896 | |||
41 | 897 | class MockEucaInstance(object): | ||
42 | 898 | def __init__(self, instance_id=None, ip_address=None, image_id=None, | ||
43 | 899 | instance_type=None, kernel=None, private_dns_name=None, | ||
44 | 900 | public_dns_name=None, state=None, tags=[], | ||
45 | 901 | availability_zone=None): | ||
46 | 902 | self.id = instance_id | ||
47 | 903 | self.ip_address = ip_address | ||
48 | 904 | self.image_id = image_id | ||
49 | 905 | self.instance_type = instance_type | ||
50 | 906 | self.kernel = kernel | ||
51 | 907 | self.private_dns_name = private_dns_name | ||
52 | 908 | self.public_dns_name = public_dns_name | ||
53 | 909 | self.state = state | ||
54 | 910 | self.tags = tags | ||
55 | 911 | self.placement = availability_zone | ||
56 | 912 | |||
57 | 913 | |||
58 | 914 | class MockAttachData(object): | ||
59 | 915 | def __init__(self, device, instance_id): | ||
60 | 916 | self.device = device | ||
61 | 917 | self.instance_id = instance_id | ||
62 | 918 | |||
63 | 919 | |||
64 | 920 | class MockVolume(object): | ||
65 | 921 | def __init__(self, vol_id, device, instance_id, zone, size, status, | ||
66 | 922 | snapshot_id, tags): | ||
67 | 923 | self.id = vol_id | ||
68 | 924 | self.attach_data = MockAttachData(device, instance_id) | ||
69 | 925 | self.zone = zone | ||
70 | 926 | self.size = size | ||
71 | 927 | self.status = status | ||
72 | 928 | self.snapshot_id = snapshot_id | ||
73 | 929 | self.tags = tags | ||
74 | 930 | |||
75 | 931 | |||
76 | 932 | class TestEC2Util(mocker.MockerTestCase): | 886 | class TestEC2Util(mocker.MockerTestCase): |
77 | 933 | 887 | ||
78 | 934 | def setUp(self): | 888 | def setUp(self): |
79 | 935 | super(TestEC2Util, self).setUp() | ||
80 | 936 | self.maxDiff = None | 889 | self.maxDiff = None |
81 | 937 | util.hookenv = TestHookenv( | 890 | util.hookenv = TestHookenv( |
82 | 938 | {"key": "ec2key", "secret": "ec2password", | 891 | {"key": "ec2key", "secret": "ec2password", |
83 | @@ -1005,8 +958,8 @@ | |||
84 | 1005 | configuration options. | 958 | configuration options. |
85 | 1006 | """ | 959 | """ |
86 | 1007 | command = "euca-describe-instances" | 960 | command = "euca-describe-instances" |
89 | 1008 | nova_cmd = self.mocker.replace(subprocess.check_call) | 961 | euca_cmd = self.mocker.replace(subprocess.check_call) |
90 | 1009 | nova_cmd(command, shell=True) | 962 | euca_cmd(command, shell=True) |
91 | 1010 | self.mocker.replay() | 963 | self.mocker.replay() |
92 | 1011 | 964 | ||
93 | 1012 | self.storage.validate_credentials() | 965 | self.storage.validate_credentials() |
94 | @@ -1242,39 +1195,47 @@ | |||
95 | 1242 | 1195 | ||
96 | 1243 | def test_wb_ec2_describe_volumes_command_error(self): | 1196 | def test_wb_ec2_describe_volumes_command_error(self): |
97 | 1244 | """ | 1197 | """ |
100 | 1245 | L{_ec2_describe_volumes} will exit in error when the euca2ools | 1198 | L{_ec2_describe_volumes} will exit in error when the |
101 | 1246 | C{DescribeVolumes} command fails. | 1199 | C{euca-describe-volumes} command fails. |
102 | 1247 | """ | 1200 | """ |
106 | 1248 | euca_command = self.mocker.replace(self.storage.ec2_volume_class) | 1201 | command = "euca-describe-volumes" |
107 | 1249 | euca_command() | 1202 | euca_volumes = self.mocker.replace(subprocess.check_output) |
108 | 1250 | self.mocker.throw(SystemExit(1)) | 1203 | euca_volumes(command, shell=True) |
109 | 1204 | self.mocker.throw(subprocess.CalledProcessError(1, command)) | ||
110 | 1251 | self.mocker.replay() | 1205 | self.mocker.replay() |
111 | 1252 | 1206 | ||
112 | 1253 | result = self.assertRaises( | 1207 | result = self.assertRaises( |
113 | 1254 | SystemExit, self.storage._ec2_describe_volumes) | 1208 | SystemExit, self.storage._ec2_describe_volumes) |
114 | 1255 | self.assertEqual(result.code, 1) | 1209 | self.assertEqual(result.code, 1) |
116 | 1256 | message = "ERROR: Couldn't contact EC2 using euca-describe-volumes" | 1210 | message = ( |
117 | 1211 | "ERROR: Command '%s' returned non-zero exit status 1" % command) | ||
118 | 1257 | self.assertIn( | 1212 | self.assertIn( |
119 | 1258 | message, util.hookenv._log_ERROR, "Not logged- %s" % message) | 1213 | message, util.hookenv._log_ERROR, "Not logged- %s" % message) |
120 | 1259 | 1214 | ||
121 | 1260 | def test_wb_ec2_describe_volumes_without_attached_instances(self): | 1215 | def test_wb_ec2_describe_volumes_without_attached_instances(self): |
122 | 1261 | """ | 1216 | """ |
127 | 1262 | L{_ec2_describe_volumes} parses the results of euca2ools | 1217 | L{_ec2_describe_volumes} parses the output from the |
128 | 1263 | C{DescribeVolumes} to create a C{dict} of volume information. When no | 1218 | C{euca-describe-volumes} command and returns a C{dict} of volume |
129 | 1264 | C{instance_id}s are present the volumes are not attached so no | 1219 | information. When no C{ATTACHMENT} response types are present the |
130 | 1265 | C{device} or C{instance_id} information will be present. | 1220 | volumes are not attached so no C{device} or C{instance_id} information |
131 | 1221 | will be present. Any C{TAG} response types will be added to the | ||
132 | 1222 | associated volume. | ||
133 | 1266 | """ | 1223 | """ |
145 | 1267 | volume1 = MockVolume( | 1224 | volume_output = ( |
146 | 1268 | "123-123-123", device="/dev/notshown", instance_id="notseen", | 1225 | "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t" |
147 | 1269 | zone="ec2-az1", size="10", status="available", | 1226 | "2014-03-19T22:00:02.580Z\nVOLUME\t456-456-456\t 8\tsome-shot\t" |
148 | 1270 | snapshot_id="some-shot", tags={}) | 1227 | "ec2-az2\tavailable\t2014-03-19T22:00:02.580Z\n" |
149 | 1271 | volume2 = MockVolume( | 1228 | "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name") |
150 | 1272 | "456-456-456", device="/dev/notshown", instance_id="notseen", | 1229 | volume_command = "euca-describe-volumes" |
151 | 1273 | zone="ec2-az2", size="8", status="available", | 1230 | tag_output = ( |
152 | 1274 | snapshot_id="some-shot", tags={"volume_name": "my volume name"}) | 1231 | "TAG\t456-456-456\tvolume\tother_key\tother value") |
153 | 1275 | euca_command = self.mocker.replace(self.storage.ec2_volume_class) | 1232 | tag_command = "euca-describe-tags" |
154 | 1276 | euca_command() | 1233 | |
155 | 1277 | self.mocker.result(MockEucaCommand([volume1, volume2])) | 1234 | euca_command = self.mocker.replace(subprocess.check_output) |
156 | 1235 | euca_command(volume_command, shell=True) | ||
157 | 1236 | self.mocker.result(volume_output) | ||
158 | 1237 | euca_command(tag_command, shell=True) | ||
159 | 1238 | self.mocker.result(tag_output) | ||
160 | 1278 | self.mocker.replay() | 1239 | self.mocker.replay() |
161 | 1279 | 1240 | ||
162 | 1280 | expected = {"123-123-123": {"id": "123-123-123", "status": "available", | 1241 | expected = {"123-123-123": {"id": "123-123-123", "status": "available", |
163 | @@ -1290,27 +1251,30 @@ | |||
164 | 1290 | "volume_label": "my volume name", | 1251 | "volume_label": "my volume name", |
165 | 1291 | "size": "8", "instance_id": "", | 1252 | "size": "8", "instance_id": "", |
166 | 1292 | "snapshot_id": "some-shot", | 1253 | "snapshot_id": "some-shot", |
168 | 1293 | "tags": {"volume_name": "my volume name"}}} | 1254 | "tags": {"volume_name": "my volume name", |
169 | 1255 | "other_key": "other value"}}} | ||
170 | 1294 | self.assertEqual(self.storage._ec2_describe_volumes(), expected) | 1256 | self.assertEqual(self.storage._ec2_describe_volumes(), expected) |
171 | 1295 | 1257 | ||
172 | 1296 | def test_wb_ec2_describe_volumes_matches_volume_id_supplied(self): | 1258 | def test_wb_ec2_describe_volumes_matches_volume_id_supplied(self): |
173 | 1297 | """ | 1259 | """ |
177 | 1298 | L{_ec2_describe_volumes} parses the results of euca2ools | 1260 | L{_ec2_describe_volumes} parses the output of the |
178 | 1299 | C{DescribeVolumes} to create a C{dict} of volume information. | 1261 | C{euca-describe-volumes} command and returns a C{dict} of volume |
179 | 1300 | When C{volume_id} is provided return a C{dict} for the matched volume. | 1262 | information. When C{volume_id} is provided, return a C{dict} for the |
180 | 1263 | matched volume. | ||
181 | 1301 | """ | 1264 | """ |
182 | 1302 | volume_id = "123-123-123" | 1265 | volume_id = "123-123-123" |
194 | 1303 | volume1 = MockVolume( | 1266 | |
195 | 1304 | volume_id, device="/dev/notshown", instance_id="notseen", | 1267 | output = ( |
196 | 1305 | zone="ec2-az1", size="10", status="available", | 1268 | "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t" |
197 | 1306 | snapshot_id="some-shot", tags={}) | 1269 | "2014-03-19T22:00:02.580Z\nVOLUME\t456-456-456\t 8\tsome-shot\t" |
198 | 1307 | volume2 = MockVolume( | 1270 | "ec2-az2\tavailable\t2014-03-19T22:00:02.580Z\n") |
199 | 1308 | "456-456-456", device="/dev/notshown", instance_id="notseen", | 1271 | command = "euca-describe-volumes" |
200 | 1309 | zone="ec2-az2", size="8", status="available", | 1272 | |
201 | 1310 | snapshot_id="some-shot", tags={"volume_name": "my volume name"}) | 1273 | euca_command = self.mocker.replace(subprocess.check_output) |
202 | 1311 | euca_command = self.mocker.replace(self.storage.ec2_volume_class) | 1274 | euca_command(command, shell=True) |
203 | 1312 | euca_command() | 1275 | self.mocker.result(output) |
204 | 1313 | self.mocker.result(MockEucaCommand([volume1, volume2])) | 1276 | euca_command("euca-describe-tags", shell=True) |
205 | 1277 | self.mocker.result("\n") | ||
206 | 1314 | self.mocker.replay() | 1278 | self.mocker.replay() |
207 | 1315 | 1279 | ||
208 | 1316 | expected = { | 1280 | expected = { |
209 | @@ -1323,18 +1287,22 @@ | |||
210 | 1323 | 1287 | ||
211 | 1324 | def test_wb_ec2_describe_volumes_unmatched_volume_id_supplied(self): | 1288 | def test_wb_ec2_describe_volumes_unmatched_volume_id_supplied(self): |
212 | 1325 | """ | 1289 | """ |
216 | 1326 | L{_ec2_describe_volumes} parses the results of euca2ools | 1290 | L{_ec2_describe_volumes} parses the output of the |
217 | 1327 | C{DescribeVolumes} to create a C{dict} of volume information. | 1291 | C{euca-describe-volumes} command and returns a C{dict} of volume |
218 | 1328 | When C{volume_id} is provided and unmatched, return an empty C{dict}. | 1292 | information. When C{volume_id} is provided and unmatched, return an |
219 | 1293 | empty C{dict}. | ||
220 | 1329 | """ | 1294 | """ |
221 | 1330 | unmatched_volume_id = "456-456-456" | 1295 | unmatched_volume_id = "456-456-456" |
229 | 1331 | volume1 = MockVolume( | 1296 | output = ( |
230 | 1332 | "123-123-123", device="/dev/notshown", instance_id="notseen", | 1297 | "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t" |
231 | 1333 | zone="ec2-az1", size="10", status="available", | 1298 | "2014-03-19T22:00:02.580Z\n") |
232 | 1334 | snapshot_id="some-shot", tags={}) | 1299 | command = "euca-describe-volumes" |
233 | 1335 | euca_command = self.mocker.replace(self.storage.ec2_volume_class) | 1300 | |
234 | 1336 | euca_command() | 1301 | euca_command = self.mocker.replace(subprocess.check_output) |
235 | 1337 | self.mocker.result(MockEucaCommand([volume1])) | 1302 | euca_command(command, shell=True) |
236 | 1303 | self.mocker.result(output) | ||
237 | 1304 | euca_command("euca-describe-tags", shell=True) | ||
238 | 1305 | self.mocker.result("\n") | ||
239 | 1338 | self.mocker.replay() | 1306 | self.mocker.replay() |
240 | 1339 | 1307 | ||
241 | 1340 | self.assertEqual( | 1308 | self.assertEqual( |
242 | @@ -1342,22 +1310,28 @@ | |||
243 | 1342 | 1310 | ||
244 | 1343 | def test_wb_ec2_describe_volumes_with_attached_instances(self): | 1311 | def test_wb_ec2_describe_volumes_with_attached_instances(self): |
245 | 1344 | """ | 1312 | """ |
250 | 1345 | L{_ec2_describe_volumes} parses the results of euca2ools | 1313 | L{_ec2_describe_volumes} parses the output of the |
251 | 1346 | C{DescribeVolumes} to create a C{dict} of volume information. If | 1314 | C{euca-describe-volumes} command and returns a C{dict} of volume |
252 | 1347 | C{status} is C{in-use}, both C{device} and C{instance_id} will be | 1315 | information. When C{status} is C{in-use}, both C{device} and |
253 | 1348 | returned in the C{dict}. | 1316 | C{instance_id} will be returned in the C{dict}. |
254 | 1349 | """ | 1317 | """ |
266 | 1350 | volume1 = MockVolume( | 1318 | output = ( |
267 | 1351 | "123-123-123", device="/dev/notshown", instance_id="notseen", | 1319 | "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t" |
268 | 1352 | zone="ec2-az1", size="10", status="available", | 1320 | "2014-03-19T22:00:02.580Z\nVOLUME\t456-456-456\t 8\tsome-shot\t" |
269 | 1353 | snapshot_id="some-shot", tags={}) | 1321 | "ec2-az2\tin-use\t2014-03-19T22:00:02.580Z\nATTACHMENT\t" |
270 | 1354 | volume2 = MockVolume( | 1322 | "456-456-456\ti-456456\t/dev/xvdc\tattached\t" |
271 | 1355 | "456-456-456", device="/dev/xvdc", instance_id="i-456456", | 1323 | "2014-03-19T22:00:02.000Z\nTAG\tvolume\t456-456-456\tvolume_name\t" |
272 | 1356 | zone="ec2-az2", size="8", status="in-use", | 1324 | "my volume name\n") |
273 | 1357 | snapshot_id="some-shot", tags={"volume_name": "my volume name"}) | 1325 | command = "euca-describe-volumes" |
274 | 1358 | euca_command = self.mocker.replace(self.storage.ec2_volume_class) | 1326 | tag_output = ( |
275 | 1359 | euca_command() | 1327 | "TAG\t456-456-456\tvolume\tvolume_name\tmy volume name") |
276 | 1360 | self.mocker.result(MockEucaCommand([volume1, volume2])) | 1328 | tag_command = "euca-describe-tags" |
277 | 1329 | |||
278 | 1330 | euca_command = self.mocker.replace(subprocess.check_output) | ||
279 | 1331 | euca_command(command, shell=True) | ||
280 | 1332 | self.mocker.result(output) | ||
281 | 1333 | euca_command(tag_command, shell=True) | ||
282 | 1334 | self.mocker.result(tag_output) | ||
283 | 1361 | self.mocker.replay() | 1335 | self.mocker.replay() |
284 | 1362 | 1336 | ||
285 | 1363 | expected = {"123-123-123": {"id": "123-123-123", "status": "available", | 1337 | expected = {"123-123-123": {"id": "123-123-123", "status": "available", |
286 | @@ -1377,6 +1351,47 @@ | |||
287 | 1377 | self.assertEqual( | 1351 | self.assertEqual( |
288 | 1378 | self.storage._ec2_describe_volumes(), expected) | 1352 | self.storage._ec2_describe_volumes(), expected) |
289 | 1379 | 1353 | ||
290 | 1354 | def test_wb_ec2_describe_volumes_orphaned_tags(self): | ||
291 | 1355 | """ | ||
292 | 1356 | L{_ec2_describe_volumes} parses the output of the | ||
293 | 1357 | C{euca-describe-volumes} and augments that data with volume-related | ||
294 | 1358 | tags from C{euca-describe-tags}. When C{euca-describe-tags} returns | ||
295 | 1359 | information about orphaned tags for volumes that no longer exist, | ||
296 | 1360 | L{_ec2_describe_volumes} will log this information and move on. | ||
297 | 1361 | """ | ||
298 | 1362 | output = ( | ||
299 | 1363 | "VOLUME\t456-456-456\t 8\tsome-shot\t" | ||
300 | 1364 | "ec2-az2\tin-use\t2014-03-19T22:00:02.580Z\nATTACHMENT\t" | ||
301 | 1365 | "456-456-456\ti-456456\t/dev/xvdc\tattached\t" | ||
302 | 1366 | "2014-03-19T22:00:02.000Z\nTAG\tvolume\t456-456-456\tvolume_name\t" | ||
303 | 1367 | "my volume name\n") | ||
304 | 1368 | command = "euca-describe-volumes" | ||
305 | 1369 | tag_output = ( | ||
306 | 1370 | "TAG\tvolume-not-here\tvolume\tvolume_name\tmy volume name\n" | ||
307 | 1371 | "TAG\t456-456-456\tvolume\tvolume_name\tmy volume name") | ||
308 | 1372 | tag_command = "euca-describe-tags" | ||
309 | 1373 | |||
310 | 1374 | euca_command = self.mocker.replace(subprocess.check_output) | ||
311 | 1375 | euca_command(command, shell=True) | ||
312 | 1376 | self.mocker.result(output) | ||
313 | 1377 | euca_command(tag_command, shell=True) | ||
314 | 1378 | self.mocker.result(tag_output) | ||
315 | 1379 | self.mocker.replay() | ||
316 | 1380 | |||
317 | 1381 | expected = {"456-456-456": {"id": "456-456-456", "status": "in-use", | ||
318 | 1382 | "device": "/dev/xvdc", | ||
319 | 1383 | "availability_zone": "ec2-az2", | ||
320 | 1384 | "volume_label": "my volume name", | ||
321 | 1385 | "size": "8", "instance_id": "i-456456", | ||
322 | 1386 | "snapshot_id": "some-shot", | ||
323 | 1387 | "tags": {"volume_name": "my volume name"}}} | ||
324 | 1388 | self.assertEqual( | ||
325 | 1389 | self.storage._ec2_describe_volumes(), expected) | ||
326 | 1390 | message = ( | ||
327 | 1391 | "Ignoring tags for volume-id volume-not-here that doesn't exist") | ||
328 | 1392 | self.assertIn( | ||
329 | 1393 | message, util.hookenv._log_INFO, "Not logged- %s" % message) | ||
330 | 1394 | |||
331 | 1380 | def test_wb_ec2_create_volume(self): | 1395 | def test_wb_ec2_create_volume(self): |
332 | 1381 | """ | 1396 | """ |
333 | 1382 | L{_ec2_create_volume} uses the command C{euca-create-volume} to create | 1397 | L{_ec2_create_volume} uses the command C{euca-create-volume} to create |
334 | @@ -1392,18 +1407,17 @@ | |||
335 | 1392 | zone = "ec2-az3" | 1407 | zone = "ec2-az3" |
336 | 1393 | command = "euca-create-volume -z %s -s %s" % (zone, size) | 1408 | command = "euca-create-volume -z %s -s %s" % (zone, size) |
337 | 1394 | 1409 | ||
338 | 1395 | reservation = MockEucaReservation( | ||
339 | 1396 | [MockEucaInstance( | ||
340 | 1397 | instance_id=instance_id, availability_zone=zone)]) | ||
341 | 1398 | euca_command = self.mocker.replace(self.storage.ec2_instance_class) | ||
342 | 1399 | euca_command() | ||
343 | 1400 | self.mocker.result(MockEucaCommand([reservation])) | ||
344 | 1401 | create = self.mocker.replace(subprocess.check_output) | 1410 | create = self.mocker.replace(subprocess.check_output) |
345 | 1402 | create(command, shell=True) | 1411 | create(command, shell=True) |
346 | 1403 | self.mocker.result( | 1412 | self.mocker.result( |
347 | 1404 | "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id) | 1413 | "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id) |
348 | 1405 | self.mocker.replay() | 1414 | self.mocker.replay() |
349 | 1406 | 1415 | ||
350 | 1416 | def mock_describe_instances(my_id): | ||
351 | 1417 | self.assertEqual(my_id, instance_id) | ||
352 | 1418 | return {"availability_zone": zone} | ||
353 | 1419 | self.storage._ec2_describe_instances = mock_describe_instances | ||
354 | 1420 | |||
355 | 1407 | def mock_describe_volumes(my_id): | 1421 | def mock_describe_volumes(my_id): |
356 | 1408 | self.assertEqual(my_id, volume_id) | 1422 | self.assertEqual(my_id, volume_id) |
357 | 1409 | return {"id": volume_id} | 1423 | return {"id": volume_id} |
358 | @@ -1436,10 +1450,10 @@ | |||
359 | 1436 | volume_label = "postgresql/0 unit volume" | 1450 | volume_label = "postgresql/0 unit volume" |
360 | 1437 | size = 10 | 1451 | size = 10 |
361 | 1438 | 1452 | ||
366 | 1439 | euca_command = self.mocker.replace(self.storage.ec2_instance_class) | 1453 | def mock_describe_instances(my_id): |
367 | 1440 | euca_command() | 1454 | self.assertEqual(my_id, instance_id) |
368 | 1441 | self.mocker.result(MockEucaCommand([])) # Empty results from euca | 1455 | return {} # No results found |
369 | 1442 | self.mocker.replay() | 1456 | self.storage._ec2_describe_instances = mock_describe_instances |
370 | 1443 | 1457 | ||
371 | 1444 | result = self.assertRaises( | 1458 | result = self.assertRaises( |
372 | 1445 | SystemExit, self.storage._ec2_create_volume, size, volume_label, | 1459 | SystemExit, self.storage._ec2_create_volume, size, volume_label, |
373 | @@ -1465,17 +1479,16 @@ | |||
374 | 1465 | zone = "ec2-az3" | 1479 | zone = "ec2-az3" |
375 | 1466 | command = "euca-create-volume -z %s -s %s" % (zone, size) | 1480 | command = "euca-create-volume -z %s -s %s" % (zone, size) |
376 | 1467 | 1481 | ||
377 | 1468 | reservation = MockEucaReservation( | ||
378 | 1469 | [MockEucaInstance( | ||
379 | 1470 | instance_id=instance_id, availability_zone=zone)]) | ||
380 | 1471 | euca_command = self.mocker.replace(self.storage.ec2_instance_class) | ||
381 | 1472 | euca_command() | ||
382 | 1473 | self.mocker.result(MockEucaCommand([reservation])) | ||
383 | 1474 | create = self.mocker.replace(subprocess.check_output) | 1482 | create = self.mocker.replace(subprocess.check_output) |
384 | 1475 | create(command, shell=True) | 1483 | create(command, shell=True) |
385 | 1476 | self.mocker.result("INSTANCE invalid-instance-type-response\n") | 1484 | self.mocker.result("INSTANCE invalid-instance-type-response\n") |
386 | 1477 | self.mocker.replay() | 1485 | self.mocker.replay() |
387 | 1478 | 1486 | ||
388 | 1487 | def mock_describe_instances(my_id): | ||
389 | 1488 | self.assertEqual(my_id, instance_id) | ||
390 | 1489 | return {"availability_zone": zone} | ||
391 | 1490 | self.storage._ec2_describe_instances = mock_describe_instances | ||
392 | 1491 | |||
393 | 1479 | def mock_describe_volumes(my_id): | 1492 | def mock_describe_volumes(my_id): |
394 | 1480 | raise Exception("_ec2_describe_volumes should not be called") | 1493 | raise Exception("_ec2_describe_volumes should not be called") |
395 | 1481 | self.storage._ec2_describe_volumes = mock_describe_volumes | 1494 | self.storage._ec2_describe_volumes = mock_describe_volumes |
396 | @@ -1507,18 +1520,17 @@ | |||
397 | 1507 | zone = "ec2-az3" | 1520 | zone = "ec2-az3" |
398 | 1508 | command = "euca-create-volume -z %s -s %s" % (zone, size) | 1521 | command = "euca-create-volume -z %s -s %s" % (zone, size) |
399 | 1509 | 1522 | ||
400 | 1510 | reservation = MockEucaReservation( | ||
401 | 1511 | [MockEucaInstance( | ||
402 | 1512 | instance_id=instance_id, availability_zone=zone)]) | ||
403 | 1513 | euca_command = self.mocker.replace(self.storage.ec2_instance_class) | ||
404 | 1514 | euca_command() | ||
405 | 1515 | self.mocker.result(MockEucaCommand([reservation])) | ||
406 | 1516 | create = self.mocker.replace(subprocess.check_output) | 1523 | create = self.mocker.replace(subprocess.check_output) |
407 | 1517 | create(command, shell=True) | 1524 | create(command, shell=True) |
408 | 1518 | self.mocker.result( | 1525 | self.mocker.result( |
409 | 1519 | "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id) | 1526 | "VOLUME %s 9 nova creating 2014-03-14T17:26:20\n" % volume_id) |
410 | 1520 | self.mocker.replay() | 1527 | self.mocker.replay() |
411 | 1521 | 1528 | ||
412 | 1529 | def mock_describe_instances(my_id): | ||
413 | 1530 | self.assertEqual(my_id, instance_id) | ||
414 | 1531 | return {"availability_zone": zone} | ||
415 | 1532 | self.storage._ec2_describe_instances = mock_describe_instances | ||
416 | 1533 | |||
417 | 1522 | def mock_describe_volumes(my_id): | 1534 | def mock_describe_volumes(my_id): |
418 | 1523 | self.assertEqual(my_id, volume_id) | 1535 | self.assertEqual(my_id, volume_id) |
419 | 1524 | return {} # No details found for this volume | 1536 | return {} # No details found for this volume |
420 | @@ -1548,17 +1560,16 @@ | |||
421 | 1548 | zone = "ec2-az3" | 1560 | zone = "ec2-az3" |
422 | 1549 | command = "euca-create-volume -z %s -s %s" % (zone, size) | 1561 | command = "euca-create-volume -z %s -s %s" % (zone, size) |
423 | 1550 | 1562 | ||
424 | 1551 | reservation = MockEucaReservation( | ||
425 | 1552 | [MockEucaInstance( | ||
426 | 1553 | instance_id=instance_id, availability_zone=zone)]) | ||
427 | 1554 | euca_command = self.mocker.replace(self.storage.ec2_instance_class) | ||
428 | 1555 | euca_command() | ||
429 | 1556 | self.mocker.result(MockEucaCommand([reservation])) | ||
430 | 1557 | create = self.mocker.replace(subprocess.check_output) | 1563 | create = self.mocker.replace(subprocess.check_output) |
431 | 1558 | create(command, shell=True) | 1564 | create(command, shell=True) |
432 | 1559 | self.mocker.throw(subprocess.CalledProcessError(1, command)) | 1565 | self.mocker.throw(subprocess.CalledProcessError(1, command)) |
433 | 1560 | self.mocker.replay() | 1566 | self.mocker.replay() |
434 | 1561 | 1567 | ||
435 | 1568 | def mock_describe_instances(my_id): | ||
436 | 1569 | self.assertEqual(my_id, instance_id) | ||
437 | 1570 | return {"availability_zone": zone} | ||
438 | 1571 | self.storage._ec2_describe_instances = mock_describe_instances | ||
439 | 1572 | |||
440 | 1562 | def mock_exception(my_id): | 1573 | def mock_exception(my_id): |
441 | 1563 | raise Exception("These methods should not be called") | 1574 | raise Exception("These methods should not be called") |
442 | 1564 | self.storage._ec2_describe_volumes = mock_exception | 1575 | self.storage._ec2_describe_volumes = mock_exception |
443 | @@ -1728,3 +1739,243 @@ | |||
444 | 1728 | (volume_id, instance_id)) | 1739 | (volume_id, instance_id)) |
445 | 1729 | self.assertIn( | 1740 | self.assertIn( |
446 | 1730 | message, util.hookenv._log_INFO, "Not logged- %s" % message) | 1741 | message, util.hookenv._log_INFO, "Not logged- %s" % message) |
447 | 1742 | |||
448 | 1743 | def test_wb_ec2_describe_instances_euca_v2(self): | ||
449 | 1744 | """ | ||
450 | 1745 | L{_ec2_describe_instances} parses the output of the command | ||
451 | 1746 | C{euca-describe-instances} for euca2ools version 2.X and returns a | ||
452 | 1747 | C{dict} of instance information. | ||
453 | 1748 | """ | ||
454 | 1749 | command = "euca-describe-instances" | ||
455 | 1750 | output = ( | ||
456 | 1751 | "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t" | ||
457 | 1752 | "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t" | ||
458 | 1753 | "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t" | ||
459 | 1754 | "monitoring-disabled\t54.81.8.9\t10.138.61.2\t\t\tebs\t\t\t\t\t" | ||
460 | 1755 | "paravirtual\t\t\t\t") | ||
461 | 1756 | |||
462 | 1757 | ec2_list = self.mocker.replace(subprocess.check_output) | ||
463 | 1758 | ec2_list(command, shell=True) | ||
464 | 1759 | self.mocker.result(output) | ||
465 | 1760 | self.mocker.replay() | ||
466 | 1761 | |||
467 | 1762 | expected = { | ||
468 | 1763 | "i-aaa": { | ||
469 | 1764 | "ip-address": "54.81.8.9", "private-ip-address": "10.138.61.2", | ||
470 | 1765 | "kernel": "aki-aaa", "instance-type": "m1.small", | ||
471 | 1766 | "state": "running", "public-dns-name": "aaa.amazonaws.com", | ||
472 | 1767 | "private-dns-name": "aaa.ec2.internal", "instance_id": "i-aaa", | ||
473 | 1768 | "image-id": "ami-aaa", "availability_zone": "us-east-1a"}} | ||
474 | 1769 | |||
475 | 1770 | self.assertEqual( | ||
476 | 1771 | EUCA2_INSTANCE_RESPONSE_LENGTH_V2, len(output.split("\t"))) | ||
477 | 1772 | |||
478 | 1773 | self.assertEqual( | ||
479 | 1774 | self.storage._ec2_describe_instances(), expected) | ||
480 | 1775 | |||
481 | 1776 | def test_wb_ec2_describe_instances_euca_invalid_output_version(self): | ||
482 | 1777 | """ | ||
483 | 1778 | L{_ec2_describe_instances} parses the output of the command | ||
484 | 1779 | C{euca-describe-instances} and if it sees an output length that is | ||
485 | 1780 | unsupported, an error is logged. | ||
486 | 1781 | """ | ||
487 | 1782 | command = "euca-describe-instances" | ||
488 | 1783 | output = ( | ||
489 | 1784 | "INSTANCE\ti-aaa\tami-0b9c9f62\tblah.amazonaws.com\t" | ||
490 | 1785 | "blah.ec2.internal\trunning\t\t0\t") | ||
491 | 1786 | |||
492 | 1787 | ec2_list = self.mocker.replace(subprocess.check_output) | ||
493 | 1788 | ec2_list(command, shell=True) | ||
494 | 1789 | self.mocker.result(output) | ||
495 | 1790 | self.mocker.replay() | ||
496 | 1791 | |||
497 | 1792 | self.assertNotIn( | ||
498 | 1793 | len(output.split("\t")), | ||
499 | 1794 | [EUCA2_INSTANCE_RESPONSE_LENGTH_V3, | ||
500 | 1795 | EUCA2_INSTANCE_RESPONSE_LENGTH_V3]) | ||
501 | 1796 | |||
502 | 1797 | result = self.assertRaises( | ||
503 | 1798 | SystemExit, self.storage._ec2_describe_instances) | ||
504 | 1799 | self.assertEqual(result.code, 1) | ||
505 | 1800 | message = "ERROR: Unsupported euca INSTANCE output format version" | ||
506 | 1801 | self.assertIn( | ||
507 | 1802 | message, util.hookenv._log_ERROR, "Not logged- %s" % message) | ||
508 | 1803 | |||
509 | 1804 | def test_wb_ec2_describe_instances_specific_instance_id(self): | ||
510 | 1805 | """ | ||
511 | 1806 | L{_ec2_describe_instances} parses the output of the command | ||
512 | 1807 | C{euca-describe-instances} for euca2ools version 3.X and returns a | ||
513 | 1808 | C{dict} of instance information. When C{instance_id} is provided, | ||
514 | 1809 | results are filtered to only the matching C{instance_id}. | ||
515 | 1810 | """ | ||
516 | 1811 | command = "euca-describe-instances" | ||
517 | 1812 | output = ( | ||
518 | 1813 | "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t" | ||
519 | 1814 | "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t" | ||
520 | 1815 | "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t" | ||
521 | 1816 | "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs" | ||
522 | 1817 | "\t\t\t\t\tparavirtual\txen\t" | ||
523 | 1818 | "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3" | ||
524 | 1819 | "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t\n" | ||
525 | 1820 | "INSTANCE\ti-bbb\tami-bbb\tbbb.amazonaws.com\t" | ||
526 | 1821 | "bbb.ec2.internal\trunning\t\t0\t\tm1.small\t" | ||
527 | 1822 | "2014-03-19T17:52:02.000Z\tus-east-1a\taki-bbb\t\t\t" | ||
528 | 1823 | "monitoring-disabled\t54.81.8.2\t10.138.61.2\t\t\tebs" | ||
529 | 1824 | "\t\t\t\t\tparavirtual\txen\t" | ||
530 | 1825 | "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3" | ||
531 | 1826 | "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t") | ||
532 | 1827 | |||
533 | 1828 | ec2_list = self.mocker.replace(subprocess.check_output) | ||
534 | 1829 | ec2_list(command, shell=True) | ||
535 | 1830 | self.mocker.result(output) | ||
536 | 1831 | self.mocker.replay() | ||
537 | 1832 | |||
538 | 1833 | expected = { | ||
539 | 1834 | "ip-address": "54.81.8.1", "private-ip-address": "10.138.61.1", | ||
540 | 1835 | "kernel": "aki-aaa", "instance-type": "m1.small", | ||
541 | 1836 | "state": "running", "public-dns-name": "aaa.amazonaws.com", | ||
542 | 1837 | "private-dns-name": "aaa.ec2.internal", "instance_id": "i-aaa", | ||
543 | 1838 | "image-id": "ami-aaa", "availability_zone": "us-east-1a"} | ||
544 | 1839 | |||
545 | 1840 | self.assertEqual( | ||
546 | 1841 | EUCA2_INSTANCE_RESPONSE_LENGTH_V3, | ||
547 | 1842 | len(output.split("\n")[0].split("\t"))) | ||
548 | 1843 | |||
549 | 1844 | self.assertEqual( | ||
550 | 1845 | self.storage._ec2_describe_instances("i-aaa"), expected) | ||
551 | 1846 | |||
552 | 1847 | def test_parse_ec2_tag_response_not_tag_response(self): | ||
553 | 1848 | """ | ||
554 | 1849 | L{parse_ec2_tag_response} returns an empty C{dict} when the | ||
555 | 1850 | line does not represent a C{TAG} response type. | ||
556 | 1851 | """ | ||
557 | 1852 | non_tag_output_lines = ( | ||
558 | 1853 | "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t" | ||
559 | 1854 | "2014-03-19T22:00:02.580Z\n" | ||
560 | 1855 | "ATTACHMENT\t123-123-123\ti-123123\t/dev/xvdc\tattached\t" | ||
561 | 1856 | "2014-03-19T22:00:02.000Z\n" | ||
562 | 1857 | "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t" | ||
563 | 1858 | "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t" | ||
564 | 1859 | "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t" | ||
565 | 1860 | "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs" | ||
566 | 1861 | "\t\t\t\t\tparavirtual\txen\t" | ||
567 | 1862 | "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3" | ||
568 | 1863 | "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t") | ||
569 | 1864 | for line in non_tag_output_lines.split("\n"): | ||
570 | 1865 | self.assertEqual(parse_ec2_tag_response(line), {}) | ||
571 | 1866 | |||
572 | 1867 | def test_parse_ec2_attachment_response_not_attachment_response(self): | ||
573 | 1868 | """ | ||
574 | 1869 | L{parse_ec2_attachment_response} returns an empty C{dict} when the | ||
575 | 1870 | line does not represent a C{ATTACHMENT} response type. | ||
576 | 1871 | """ | ||
577 | 1872 | non_attachment_output_lines = ( | ||
578 | 1873 | "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name\n" | ||
579 | 1874 | "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t" | ||
580 | 1875 | "2014-03-19T22:00:02.580Z\n" | ||
581 | 1876 | "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t" | ||
582 | 1877 | "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t" | ||
583 | 1878 | "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t" | ||
584 | 1879 | "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs" | ||
585 | 1880 | "\t\t\t\t\tparavirtual\txen\t" | ||
586 | 1881 | "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3" | ||
587 | 1882 | "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t") | ||
588 | 1883 | for line in non_attachment_output_lines.split("\n"): | ||
589 | 1884 | self.assertEqual(parse_ec2_attachment_response(line), {}) | ||
590 | 1885 | |||
591 | 1886 | def test_parse_ec2_volume_response_not_volume_response(self): | ||
592 | 1887 | """ | ||
593 | 1888 | L{parse_ec2_volume_response} returns an empty C{dict} when the | ||
594 | 1889 | line does not represent a C{VOLUME} response type. | ||
595 | 1890 | """ | ||
596 | 1891 | non_volume_output_lines = ( | ||
597 | 1892 | "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name\n" | ||
598 | 1893 | "ATTACHMENT\t123-123-123\ti-123123\t/dev/xvdc\tattached\t" | ||
599 | 1894 | "2014-03-19T22:00:02.000Z\n" | ||
600 | 1895 | "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t" | ||
601 | 1896 | "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t" | ||
602 | 1897 | "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t" | ||
603 | 1898 | "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs" | ||
604 | 1899 | "\t\t\t\t\tparavirtual\txen\t" | ||
605 | 1900 | "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3" | ||
606 | 1901 | "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t") | ||
607 | 1902 | for line in non_volume_output_lines.split("\n"): | ||
608 | 1903 | self.assertEqual(parse_ec2_volume_response(line), {}) | ||
609 | 1904 | |||
610 | 1905 | def test_parse_ec2_instance_response_not_instance_response(self): | ||
611 | 1906 | """ | ||
612 | 1907 | L{parse_ec2_instance_response} returns an empty C{dict} when the | ||
613 | 1908 | line does not represent a C{INSTANCE} response type. | ||
614 | 1909 | """ | ||
615 | 1910 | non_instance_output_lines = ( | ||
616 | 1911 | "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name\n" | ||
617 | 1912 | "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t" | ||
618 | 1913 | "2014-03-19T22:00:02.580Z\n" | ||
619 | 1914 | "ATTACHMENT\t123-123-123\ti-123123\t/dev/xvdc\tattached\t" | ||
620 | 1915 | "2014-03-19T22:00:02.000Z\n") | ||
621 | 1916 | for line in non_instance_output_lines.split("\n"): | ||
622 | 1917 | self.assertEqual(parse_ec2_instance_response(line), {}) | ||
623 | 1918 | |||
624 | 1919 | def test_parse_ec2_instance_response(self): | ||
625 | 1920 | """ | ||
626 | 1921 | L{parse_ec2_instance_response} returns a C{dict} of the parsed | ||
627 | 1922 | C{INSTANCE} response type. | ||
628 | 1923 | """ | ||
629 | 1924 | output = ( | ||
630 | 1925 | "INSTANCE\ti-aaa\tami-aaa\taaa.amazonaws.com\t" | ||
631 | 1926 | "aaa.ec2.internal\trunning\t\t0\t\tm1.small\t" | ||
632 | 1927 | "2014-03-19T17:52:02.000Z\tus-east-1a\taki-aaa\t\t\t" | ||
633 | 1928 | "monitoring-disabled\t54.81.8.1\t10.138.61.1\t\t\tebs" | ||
634 | 1929 | "\t\t\t\t\tparavirtual\txen\t" | ||
635 | 1930 | "5926eb12c796888691d9fa135f6e6f8ada297a9b1ecbea0c12282522400636f3" | ||
636 | 1931 | "\tsg-55e2e33e,sg-29e2e342\tdefault\tfalse\t") | ||
637 | 1932 | |||
638 | 1933 | expected = { | ||
639 | 1934 | "ip-address": "54.81.8.1", "private-ip-address": "10.138.61.1", | ||
640 | 1935 | "image-id": "ami-aaa", "instance-type": "m1.small", | ||
641 | 1936 | "kernel": "aki-aaa", "private-dns-name": "aaa.ec2.internal", | ||
642 | 1937 | "public-dns-name": "aaa.amazonaws.com", "state": "running", | ||
643 | 1938 | "availability_zone": "us-east-1a", "instance_id": "i-aaa"} | ||
644 | 1939 | |||
645 | 1940 | self.assertEqual(parse_ec2_instance_response(output), expected) | ||
646 | 1941 | |||
647 | 1942 | def test_parse_ec2_tag_response(self): | ||
648 | 1943 | """ | ||
649 | 1944 | L{parse_ec2_tag_response} returns a C{dict} of the parsed C{TAG} | ||
650 | 1945 | response type. | ||
651 | 1946 | """ | ||
652 | 1947 | output = "TAG\tvolume\t456-456-456\tvolume_name\tmy volume name" | ||
653 | 1948 | |||
654 | 1949 | expected = { | ||
655 | 1950 | "tag_type": "volume", "id": "456-456-456", "key": "volume_name", | ||
656 | 1951 | "value": "my volume name"} | ||
657 | 1952 | self.assertEqual(parse_ec2_tag_response(output), expected) | ||
658 | 1953 | |||
659 | 1954 | def test_parse_ec2_attachment_response(self): | ||
660 | 1955 | """ | ||
661 | 1956 | L{parse_ec2_attachment_response} returns a C{dict} of the parsed | ||
662 | 1957 | C{ATTACHMENT} response type. | ||
663 | 1958 | """ | ||
664 | 1959 | output = ( | ||
665 | 1960 | "ATTACHMENT\t123-123-123\ti-123123\t/dev/xvdc\tattached\t" | ||
666 | 1961 | "2014-03-19T22:00:02.000Z\n") | ||
667 | 1962 | |||
668 | 1963 | expected = { | ||
669 | 1964 | "volume_id": "123-123-123", "instance_id": "i-123123", | ||
670 | 1965 | "device": "/dev/xvdc", "attach_status": "attached"} | ||
671 | 1966 | self.assertEqual(parse_ec2_attachment_response(output), expected) | ||
672 | 1967 | |||
673 | 1968 | def test_parse_ec2_volume_response(self): | ||
674 | 1969 | """ | ||
675 | 1970 | L{parse_ec2_volume_response} returns a C{dict} of the parsed | ||
676 | 1971 | C{VOLUME} response type. | ||
677 | 1972 | """ | ||
678 | 1973 | output = ( | ||
679 | 1974 | "VOLUME\t123-123-123\t 10\tsome-shot\tec2-az1\tavailable\t" | ||
680 | 1975 | "2014-03-19T22:00:02.580Z\n") | ||
681 | 1976 | |||
682 | 1977 | expected = { | ||
683 | 1978 | "id": "123-123-123", "size": "10", | ||
684 | 1979 | "snapshot_id": "some-shot", "availability_zone": "ec2-az1", | ||
685 | 1980 | "status": "available"} | ||
686 | 1981 | self.assertEqual(parse_ec2_volume_response(output), expected) | ||
687 | 1731 | 1982 | ||
688 | === modified file 'hooks/util.py' | |||
689 | --- hooks/util.py 2014-03-21 17:39:48 +0000 | |||
690 | +++ hooks/util.py 2014-08-08 02:34:47 +0000 | |||
691 | @@ -23,6 +23,9 @@ | |||
692 | 23 | "nova": {"validate": "nova list", | 23 | "nova": {"validate": "nova list", |
693 | 24 | "detach": "nova volume-detach %s %s"}} | 24 | "detach": "nova volume-detach %s %s"}} |
694 | 25 | 25 | ||
695 | 26 | EUCA2_INSTANCE_RESPONSE_LENGTH_V2 = 30 | ||
696 | 27 | EUCA2_INSTANCE_RESPONSE_LENGTH_V3 = 32 | ||
697 | 28 | |||
698 | 26 | 29 | ||
699 | 27 | class StorageServiceUtil(object): | 30 | class StorageServiceUtil(object): |
700 | 28 | """Interact with an underlying cloud storage provider. | 31 | """Interact with an underlying cloud storage provider. |
701 | @@ -45,11 +48,6 @@ | |||
702 | 45 | self.environment_map = ENVIRONMENT_MAP[provider] | 48 | self.environment_map = ENVIRONMENT_MAP[provider] |
703 | 46 | self.commands = PROVIDER_COMMANDS[provider] | 49 | self.commands = PROVIDER_COMMANDS[provider] |
704 | 47 | self.required_config_options = REQUIRED_CONFIG_OPTIONS[provider] | 50 | self.required_config_options = REQUIRED_CONFIG_OPTIONS[provider] |
705 | 48 | if provider == "ec2": | ||
706 | 49 | import euca2ools.commands.euca.describevolumes as getvolumes | ||
707 | 50 | import euca2ools.commands.euca.describeinstances as getinstances | ||
708 | 51 | self.ec2_volume_class = getvolumes.DescribeVolumes | ||
709 | 52 | self.ec2_instance_class = getinstances.DescribeInstances | ||
710 | 53 | 51 | ||
711 | 54 | def load_environment(self): | 52 | def load_environment(self): |
712 | 55 | """ | 53 | """ |
713 | @@ -212,83 +210,112 @@ | |||
714 | 212 | return | 210 | return |
715 | 213 | 211 | ||
716 | 214 | # EC2-specific methods | 212 | # EC2-specific methods |
717 | 213 | def _run_command(self, command): | ||
718 | 214 | try: | ||
719 | 215 | output = subprocess.check_call(command, shell=True) | ||
720 | 216 | except subprocess.CalledProcessError, e: | ||
721 | 217 | hookenv.log("ERROR: %s" % str(e), hookenv.ERROR) | ||
722 | 218 | sys.exit(1) | ||
723 | 219 | |||
724 | 220 | # Split the output by lines | ||
725 | 221 | lines = [] | ||
726 | 222 | if output: | ||
727 | 223 | for line in output.split("\n"): | ||
728 | 224 | line = line.strip() | ||
729 | 225 | if line: | ||
730 | 226 | lines.append(line) | ||
731 | 227 | return lines | ||
732 | 228 | |||
733 | 215 | def _ec2_create_tag(self, volume_id, tag_name, tag_value=None): | 229 | def _ec2_create_tag(self, volume_id, tag_name, tag_value=None): |
734 | 216 | """Attach a tag and optional C{tag_value} to the given C{volume_id}""" | 230 | """Attach a tag and optional C{tag_value} to the given C{volume_id}""" |
735 | 217 | tag_string = tag_name | 231 | tag_string = tag_name |
736 | 218 | if tag_value: | 232 | if tag_value: |
737 | 219 | tag_string += "=%s" % tag_value | 233 | tag_string += "=%s" % tag_value |
739 | 220 | command = 'euca-create-tags %s --tag "%s"' % (volume_id, tag_string) | 234 | command = "euca-create-tags %s --tag \"%s\"" % (volume_id, tag_string) |
740 | 221 | 235 | ||
748 | 222 | try: | 236 | self._run_command(command) |
742 | 223 | subprocess.check_call(command, shell=True) | ||
743 | 224 | except subprocess.CalledProcessError, e: | ||
744 | 225 | hookenv.log( | ||
745 | 226 | "ERROR: Couldn't add tags to the resource. %s" % str(e), | ||
746 | 227 | hookenv.ERROR) | ||
747 | 228 | sys.exit(1) | ||
749 | 229 | hookenv.log("Tagged (%s) to %s." % (tag_string, volume_id)) | 237 | hookenv.log("Tagged (%s) to %s." % (tag_string, volume_id)) |
750 | 230 | 238 | ||
751 | 231 | def _ec2_describe_instances(self, instance_id=None): | 239 | def _ec2_describe_instances(self, instance_id=None): |
752 | 232 | """ | 240 | """ |
754 | 233 | Use euca2ools libraries to describe instances and return a C{dict} | 241 | Use euca2ools command output to describe instances and return a C{dict} |
755 | 234 | """ | 242 | """ |
756 | 235 | result = {} | 243 | result = {} |
775 | 236 | try: | 244 | |
776 | 237 | command = self.ec2_instance_class() | 245 | lines = self._run_command("euca-describe-instances") |
777 | 238 | reservations = command.main() | 246 | for line in lines: |
778 | 239 | except SystemExit: | 247 | response_type = line.split("\t")[0] |
779 | 240 | hookenv.log( | 248 | if response_type != "INSTANCE": |
780 | 241 | "ERROR: Couldn't contact EC2 using euca-describe-instances", | 249 | continue |
781 | 242 | hookenv.ERROR) | 250 | instance_data = parse_ec2_instance_response(line) |
782 | 243 | sys.exit(1) | 251 | result[instance_data["instance_id"]] = instance_data |
783 | 244 | for reservation in reservations: | 252 | |
766 | 245 | for inst in reservation.instances: | ||
767 | 246 | result[inst.id] = { | ||
768 | 247 | "ip-address": inst.ip_address, "image-id": inst.image_id, | ||
769 | 248 | "instance-type": inst.image_id, "kernel": inst.kernel, | ||
770 | 249 | "private-dns-name": inst.private_dns_name, | ||
771 | 250 | "public-dns-name": inst.public_dns_name, | ||
772 | 251 | "reservation-id": reservation.id, | ||
773 | 252 | "state": inst.state, "tags": inst.tags, | ||
774 | 253 | "availability_zone": inst.placement} | ||
784 | 254 | if instance_id: | 253 | if instance_id: |
785 | 255 | if instance_id in result: | 254 | if instance_id in result: |
786 | 256 | return result[instance_id] | 255 | return result[instance_id] |
787 | 257 | return {} | 256 | return {} |
788 | 258 | return result | 257 | return result |
789 | 259 | 258 | ||
790 | 259 | def _ec2_describe_volume_tags(self): | ||
791 | 260 | """ | ||
792 | 261 | Parse output of C{euca-describe-tags} returning a C{dict} of volume ids | ||
793 | 262 | and any associated tags. | ||
794 | 263 | """ | ||
795 | 264 | result = {} | ||
796 | 265 | |||
797 | 266 | lines = self._run_command("euca-describe-tags") | ||
798 | 267 | for line in lines: | ||
799 | 268 | response_type = line.split("\t")[0] | ||
800 | 269 | if response_type != "TAG": | ||
801 | 270 | continue | ||
802 | 271 | data = parse_ec2_tag_response(line) | ||
803 | 272 | if not result.get(data["id"]): | ||
804 | 273 | result[data["id"]] = {"tags": {}} | ||
805 | 274 | result[data["id"]]["tags"].update({data["key"]: data["value"]}) | ||
806 | 275 | return result | ||
807 | 276 | |||
808 | 260 | def _ec2_describe_volumes(self, volume_id=None): | 277 | def _ec2_describe_volumes(self, volume_id=None): |
809 | 261 | """ | 278 | """ |
811 | 262 | Use euca2ools libraries to describe volumes and return a C{dict} | 279 | Parse output of C{euca-describe-volumes} returning a C{dict} of volume |
812 | 280 | ids all related volume information. | ||
813 | 263 | """ | 281 | """ |
814 | 264 | result = {} | 282 | result = {} |
842 | 265 | try: | 283 | |
843 | 266 | command = self.ec2_volume_class() | 284 | lines = self._run_command("euca-describe-volumes") |
844 | 267 | volumes = command.main() | 285 | for line in lines: |
845 | 268 | except SystemExit: | 286 | response_type = line.split("\t")[0] |
846 | 269 | hookenv.log( | 287 | if response_type == "VOLUME": |
847 | 270 | "ERROR: Couldn't contact EC2 using euca-describe-volumes", | 288 | data = parse_ec2_volume_response(line) |
848 | 271 | hookenv.ERROR) | 289 | data["device"] = "" |
849 | 272 | sys.exit(1) | 290 | data["volume_label"] = "" |
850 | 273 | for volume in volumes: | 291 | data["instance_id"] = "" |
851 | 274 | result[volume.id] = { | 292 | data["tags"] = {"volume_name": ""} |
852 | 275 | "device": "", | 293 | result[data["id"]] = data |
853 | 276 | "instance_id": "", | 294 | elif response_type == "TAG": |
854 | 277 | "size": volume.size, | 295 | data = parse_ec2_tag_response(line) |
855 | 278 | "snapshot_id": volume.snapshot_id, | 296 | tags = result[data["id"]]["tags"] |
856 | 279 | "status": volume.status, | 297 | tags.update({data["key"]: data["value"]}) |
857 | 280 | "tags": volume.tags, | 298 | volume_name = tags.get("volume_name") |
858 | 281 | "id": volume.id, | 299 | if volume_name: |
859 | 282 | "availability_zone": volume.zone} | 300 | result[data["id"]]["volume_label"] = volume_name |
860 | 283 | if "volume_name" in volume.tags: | 301 | elif response_type == "ATTACHMENT": |
861 | 284 | result[volume.id]["volume_label"] = volume.tags["volume_name"] | 302 | data = parse_ec2_attachment_response(line) |
862 | 285 | else: | 303 | volume_data = result[data["volume_id"]] |
863 | 286 | result[volume.id]["tags"]["volume_name"] = "" | 304 | volume_data.update({"instance_id": data["instance_id"]}) |
864 | 287 | result[volume.id]["volume_label"] = "" | 305 | volume_data.update({"device": data["device"]}) |
865 | 288 | if volume.status == "in-use": | 306 | else: |
866 | 289 | result[volume.id]["instance_id"] = ( | 307 | continue |
867 | 290 | volume.attach_data.instance_id) | 308 | |
868 | 291 | result[volume.id]["device"] = volume.attach_data.device | 309 | volume_tag_data = self._ec2_describe_volume_tags() |
869 | 310 | for tag_volume_id in volume_tag_data.keys(): | ||
870 | 311 | additional_tags = volume_tag_data[tag_volume_id]["tags"] | ||
871 | 312 | if tag_volume_id not in result: | ||
872 | 313 | hookenv.log( | ||
873 | 314 | "Ignoring tags for volume-id %s that doesn't exist" % | ||
874 | 315 | tag_volume_id) | ||
875 | 316 | else: | ||
876 | 317 | result[tag_volume_id]["tags"].update(additional_tags) | ||
877 | 318 | |||
878 | 292 | if volume_id: | 319 | if volume_id: |
879 | 293 | if volume_id in result: | 320 | if volume_id in result: |
880 | 294 | return result[volume_id] | 321 | return result[volume_id] |
881 | @@ -311,19 +338,15 @@ | |||
882 | 311 | (instance_id, config_data["endpoint"]), hookenv.ERROR) | 338 | (instance_id, config_data["endpoint"]), hookenv.ERROR) |
883 | 312 | sys.exit(1) | 339 | sys.exit(1) |
884 | 313 | 340 | ||
892 | 314 | try: | 341 | command = "euca-create-volume -z %s -s %s" % ( |
893 | 315 | output = subprocess.check_output( | 342 | instance["availability_zone"], size) |
887 | 316 | "euca-create-volume -z %s -s %s" % | ||
888 | 317 | (instance["availability_zone"], size), shell=True) | ||
889 | 318 | except subprocess.CalledProcessError, e: | ||
890 | 319 | hookenv.log("ERROR: %s" % str(e), hookenv.ERROR) | ||
891 | 320 | sys.exit(1) | ||
894 | 321 | 343 | ||
896 | 322 | response_type, volume_id = output.split()[:2] | 344 | lines = self._run_command(command) |
897 | 345 | response_type, volume_id = lines[:2] | ||
898 | 323 | if response_type != "VOLUME": | 346 | if response_type != "VOLUME": |
899 | 324 | hookenv.log( | 347 | hookenv.log( |
900 | 325 | "ERROR: Didn't get VOLUME response from euca-create-volume. " | 348 | "ERROR: Didn't get VOLUME response from euca-create-volume. " |
902 | 326 | "Response: %s" % output, hookenv.ERROR) | 349 | "Response: %s" % "".join(lines), hookenv.ERROR) |
903 | 327 | sys.exit(1) | 350 | sys.exit(1) |
904 | 328 | volume = self.describe_volumes(volume_id.strip()) | 351 | volume = self.describe_volumes(volume_id.strip()) |
905 | 329 | if not volume: | 352 | if not volume: |
906 | @@ -341,13 +364,10 @@ | |||
907 | 341 | the device path. | 364 | the device path. |
908 | 342 | """ | 365 | """ |
909 | 343 | device = "/dev/xvdc" | 366 | device = "/dev/xvdc" |
917 | 344 | try: | 367 | command = "euca-attach-volume -i %s -d %s %s" % ( |
918 | 345 | subprocess.check_call( | 368 | instance_id, device, volume_id) |
919 | 346 | "euca-attach-volume -i %s -d %s %s" % | 369 | |
920 | 347 | (instance_id, device, volume_id), shell=True) | 370 | self._run_command(command) |
914 | 348 | except subprocess.CalledProcessError, e: | ||
915 | 349 | hookenv.log("ERROR: %s" % str(e), hookenv.ERROR) | ||
916 | 350 | sys.exit(1) | ||
921 | 351 | return device | 371 | return device |
922 | 352 | 372 | ||
923 | 353 | # Nova-specific methods | 373 | # Nova-specific methods |
924 | @@ -358,17 +378,9 @@ | |||
925 | 358 | """ | 378 | """ |
926 | 359 | from ast import literal_eval | 379 | from ast import literal_eval |
927 | 360 | result = {"tags": {}, "instance_id": "", "device": ""} | 380 | result = {"tags": {}, "instance_id": "", "device": ""} |
939 | 361 | command = "nova volume-show '%s'" % volume_id | 381 | |
940 | 362 | try: | 382 | lines = self._run_command("nova volume-show '%s'" % volume_id) |
941 | 363 | output = subprocess.check_output(command, shell=True) | 383 | for line in lines: |
931 | 364 | except subprocess.CalledProcessError, e: | ||
932 | 365 | hookenv.log( | ||
933 | 366 | "ERROR: Failed to get nova volume info. %s" % str(e), | ||
934 | 367 | hookenv.ERROR) | ||
935 | 368 | sys.exit(1) | ||
936 | 369 | for line in output.split("\n"): | ||
937 | 370 | if not line.strip(): # Skip empty lines | ||
938 | 371 | continue | ||
942 | 372 | if "+----" in line or "Property" in line: | 384 | if "+----" in line or "Property" in line: |
943 | 373 | continue | 385 | continue |
944 | 374 | (_, key, value, _) = line.split("|") | 386 | (_, key, value, _) = line.split("|") |
945 | @@ -393,15 +405,9 @@ | |||
946 | 393 | def _nova_describe_volumes(self, volume_id=None): | 405 | def _nova_describe_volumes(self, volume_id=None): |
947 | 394 | """Create a C{dict} describing all nova volumes""" | 406 | """Create a C{dict} describing all nova volumes""" |
948 | 395 | result = {} | 407 | result = {} |
958 | 396 | command = "nova volume-list" | 408 | |
959 | 397 | try: | 409 | lines = self._run_command("nova volume-list") |
960 | 398 | output = subprocess.check_output(command, shell=True) | 410 | for line in lines: |
952 | 399 | except subprocess.CalledProcessError, e: | ||
953 | 400 | hookenv.log("ERROR: %s" % str(e), hookenv.ERROR) | ||
954 | 401 | sys.exit(1) | ||
955 | 402 | for line in output.split("\n"): | ||
956 | 403 | if not line.strip(): # Skip empty lines | ||
957 | 404 | continue | ||
961 | 405 | if "+----" in line or "ID" in line: | 411 | if "+----" in line or "ID" in line: |
962 | 406 | continue | 412 | continue |
963 | 407 | values = line.split("|") | 413 | values = line.split("|") |
964 | @@ -434,15 +440,12 @@ | |||
965 | 434 | Attach a Nova C{volume_id} to the provided C{instance_id} and return | 440 | Attach a Nova C{volume_id} to the provided C{instance_id} and return |
966 | 435 | the device path. | 441 | the device path. |
967 | 436 | """ | 442 | """ |
977 | 437 | try: | 443 | command = ( |
978 | 438 | device = subprocess.check_output( | 444 | "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" % |
979 | 439 | "nova volume-attach %s %s auto | egrep -o \"/dev/vd[b-z]\"" % | 445 | (instance_id, volume_id)) |
980 | 440 | (instance_id, volume_id), shell=True) | 446 | lines = self._run_ec2_command(command) |
981 | 441 | except subprocess.CalledProcessError, e: | 447 | if lines: |
982 | 442 | hookenv.log("ERROR: %s" % str(e), hookenv.ERROR) | 448 | return lines[0] |
974 | 443 | sys.exit(1) | ||
975 | 444 | if device.strip(): | ||
976 | 445 | return device.strip() | ||
983 | 446 | return "" | 449 | return "" |
984 | 447 | 450 | ||
985 | 448 | def _nova_create_volume(self, size, volume_label, instance_id): | 451 | def _nova_create_volume(self, size, volume_label, instance_id): |
986 | @@ -450,14 +453,11 @@ | |||
987 | 450 | hookenv.log( | 453 | hookenv.log( |
988 | 451 | "Creating a %sGig volume named (%s) for instance %s" % | 454 | "Creating a %sGig volume named (%s) for instance %s" % |
989 | 452 | (size, volume_label, instance_id)) | 455 | (size, volume_label, instance_id)) |
997 | 453 | try: | 456 | command = ( |
998 | 454 | subprocess.check_call( | 457 | "nova volume-create --display-name '%s' %s" % |
999 | 455 | "nova volume-create --display-name '%s' %s" % | 458 | (volume_label, size)) |
993 | 456 | (volume_label, size), shell=True) | ||
994 | 457 | except subprocess.CalledProcessError, e: | ||
995 | 458 | hookenv.log("ERROR: %s" % str(e), hookenv.ERROR) | ||
996 | 459 | sys.exit(1) | ||
1000 | 460 | 459 | ||
1001 | 460 | self._run_command(command) | ||
1002 | 461 | volume_id = self.get_volume_id(volume_label) | 461 | volume_id = self.get_volume_id(volume_label) |
1003 | 462 | if not volume_id: | 462 | if not volume_id: |
1004 | 463 | hookenv.log( | 463 | hookenv.log( |
1005 | @@ -467,6 +467,102 @@ | |||
1006 | 467 | return volume_id | 467 | return volume_id |
1007 | 468 | 468 | ||
1008 | 469 | 469 | ||
1009 | 470 | def parse_ec2_tag_response(line): | ||
1010 | 471 | """ | ||
1011 | 472 | Parses a line of output of response type TAG from euca commands and | ||
1012 | 473 | returns a C{dict} of the values. If not a TAG response, an empty C{dict} is | ||
1013 | 474 | returned. | ||
1014 | 475 | """ | ||
1015 | 476 | values = line.split("\t") | ||
1016 | 477 | response_type = values[0] | ||
1017 | 478 | if response_type != "TAG": | ||
1018 | 479 | return {} | ||
1019 | 480 | |||
1020 | 481 | # There are two versions of the TAG response | ||
1021 | 482 | if values[1] == "volume": | ||
1022 | 483 | tag_type = values[1] | ||
1023 | 484 | object_id = values[2] | ||
1024 | 485 | else: | ||
1025 | 486 | tag_type = values[2] | ||
1026 | 487 | object_id = values[1] | ||
1027 | 488 | if tag_type != "volume": | ||
1028 | 489 | hookenv.log( | ||
1029 | 490 | "Ignoring non-volume TAG response: %s." % line, hookenv.WARNING) | ||
1030 | 491 | return {} | ||
1031 | 492 | return { | ||
1032 | 493 | "tag_type": tag_type, "id": object_id, "key": values[3], | ||
1033 | 494 | "value": values[4]} | ||
1034 | 495 | |||
1035 | 496 | |||
1036 | 497 | def parse_ec2_attachment_response(line): | ||
1037 | 498 | """ | ||
1038 | 499 | Parses a line of output of response type ATTACHMENT from euca commands and | ||
1039 | 500 | returns a C{dict} of the values. If not a ATTACHMENT response, an empty | ||
1040 | 501 | C{dict} is returned. | ||
1041 | 502 | """ | ||
1042 | 503 | values = line.split("\t") | ||
1043 | 504 | response_type = values[0] | ||
1044 | 505 | if response_type != "ATTACHMENT": | ||
1045 | 506 | return {} | ||
1046 | 507 | |||
1047 | 508 | return { | ||
1048 | 509 | "volume_id": values[1], "instance_id": values[2], "device": values[3], | ||
1049 | 510 | "attach_status": values[4]} | ||
1050 | 511 | |||
1051 | 512 | |||
1052 | 513 | def parse_ec2_volume_response(line): | ||
1053 | 514 | """ | ||
1054 | 515 | Parses a line of output of response type VOLUME from euca commands and | ||
1055 | 516 | returns a C{dict} of the values. If not a VOLUME response, an empty C{dict} | ||
1056 | 517 | is returned. | ||
1057 | 518 | """ | ||
1058 | 519 | values = line.split("\t") | ||
1059 | 520 | response_type = values[0] | ||
1060 | 521 | if response_type != "VOLUME": | ||
1061 | 522 | return {} | ||
1062 | 523 | |||
1063 | 524 | return { | ||
1064 | 525 | "id": values[1], "size": values[2].strip(), | ||
1065 | 526 | "snapshot_id": values[3], "availability_zone": values[4], | ||
1066 | 527 | "status": values[5]} | ||
1067 | 528 | |||
1068 | 529 | |||
1069 | 530 | def parse_ec2_instance_response(line): | ||
1070 | 531 | """ | ||
1071 | 532 | Parses a line of output of response type INSTANCE from euca commands and | ||
1072 | 533 | returns a C{dict} of the values. If not an INSTANCE response, and empty | ||
1073 | 534 | C{dict} is returned. | ||
1074 | 535 | """ | ||
1075 | 536 | values = line.split("\t") | ||
1076 | 537 | response_type = values[0] | ||
1077 | 538 | if response_type != "INSTANCE": | ||
1078 | 539 | return {} | ||
1079 | 540 | |||
1080 | 541 | if len(values) not in [EUCA2_INSTANCE_RESPONSE_LENGTH_V2, | ||
1081 | 542 | EUCA2_INSTANCE_RESPONSE_LENGTH_V3]: | ||
1082 | 543 | hookenv.log( | ||
1083 | 544 | "ERROR: Unsupported euca INSTANCE output format version", | ||
1084 | 545 | hookenv.ERROR) | ||
1085 | 546 | sys.exit(1) | ||
1086 | 547 | |||
1087 | 548 | (ec2_instance_id, image_id) = values[1:3] | ||
1088 | 549 | public_dns = values[3] | ||
1089 | 550 | private_dns = values[4] | ||
1090 | 551 | state = values[5] | ||
1091 | 552 | instance_type = values[9] | ||
1092 | 553 | availability_zone = values[11] | ||
1093 | 554 | kernel = values[12] | ||
1094 | 555 | public_ip = values[16] | ||
1095 | 556 | private_ip = values[17] | ||
1096 | 557 | |||
1097 | 558 | return { | ||
1098 | 559 | "ip-address": public_ip, "private-ip-address": private_ip, | ||
1099 | 560 | "image-id": image_id, "instance-type": instance_type, | ||
1100 | 561 | "kernel": kernel, "private-dns-name": private_dns, | ||
1101 | 562 | "public-dns-name": public_dns, "state": state, | ||
1102 | 563 | "availability_zone": availability_zone, "instance_id": ec2_instance_id} | ||
1103 | 564 | |||
1104 | 565 | |||
1105 | 470 | def generate_volume_label(remote_unit): | 566 | def generate_volume_label(remote_unit): |
1106 | 471 | """Create a volume label for the requesting remote unit""" | 567 | """Create a volume label for the requesting remote unit""" |
1107 | 472 | return "%s unit volume" % remote_unit | 568 | return "%s unit volume" % remote_unit |
Here's a deployer bundle to deploy.
rm -rf ./trusty wherever you are about to juju-deployer from to ensure you don't use cached charm checkouts
cat block-storage- trusty. yaml
common:
options:
provider: block-storage- broker
volume_ size: 9
block- storage- broker- ec2:
options:
provider: ec2
key: <YOUR_EC2_ ACCESS_ KEY>
endpoint: <YOUR_EC2_URL>
secret: <YOUR_EC2_ SECRET_ KEY>
constraint s: mem=2048
options:
extra- packages: python-apt postgresql-contrib postgresql- 9.3-debversion
max_connectio ns: 500
services:
storage:
branch: lp:~chad.smith/charms/precise/storage/storage-volume-label-availability-zone
branch: lp:~chad.smith/charms/precise/block-storage-broker/bsb-trusty-support
postgresql:
branch: lp:~chad.smith/charms/precise/postgresql/postgresql-using-storage-subordinate
doit: broker- ec2]
inherits: common
series: trusty
relations:
- [postgresql, storage]
- [storage, block-storage-
juju-deployer -c block-storage- trusty. yaml doit