Merge lp:~yamahata/nova/boot-from-volume-2 into lp:~hudson-openstack/nova/trunk
- boot-from-volume-2
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Brian Lamar |
Approved revision: | 1334 |
Merged at revision: | 1400 |
Proposed branch: | lp:~yamahata/nova/boot-from-volume-2 |
Merge into: | lp:~hudson-openstack/nova/trunk |
Diff against target: |
1798 lines (+981/-161) 21 files modified
nova/api/ec2/cloud.py (+153/-27) nova/api/ec2/ec2utils.py (+0/-29) nova/block_device.py (+71/-0) nova/compute/api.py (+56/-17) nova/compute/manager.py (+26/-8) nova/db/sqlalchemy/api.py (+18/-1) nova/image/glance.py (+46/-0) nova/tests/image/test_glance.py (+36/-0) nova/tests/test_api.py (+6/-3) nova/tests/test_block_device.py (+87/-0) nova/tests/test_cloud.py (+153/-4) nova/tests/test_compute.py (+28/-12) nova/tests/test_libvirt.py (+37/-0) nova/tests/test_metadata.py (+1/-0) nova/tests/test_virt.py (+83/-0) nova/virt/driver.py (+29/-2) nova/virt/fake.py (+2/-2) nova/virt/hyperv.py (+2/-2) nova/virt/libvirt.xml.template (+23/-10) nova/virt/libvirt/connection.py (+122/-42) nova/virt/xenapi_conn.py (+2/-2) |
To merge this branch: | bzr merge lp:~yamahata/nova/boot-from-volume-2 |
Related bugs: | |
Related blueprints: |
Boot From Volume
(High)
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Brian Lamar (community) | Approve | ||
Devin Carlen (community) | Approve | ||
Vish Ishaya (community) | Approve | ||
Soren Hansen (community) | Needs Information | ||
Review via email: mp+68496@code.launchpad.net |
Commit message
Description of the change
With this branch, boot-from-volume can be marked as completed in some sense.
The remaining is minor if any and will be addressed as bug fixes.
With this branch the following is enabled.
- describe instance attribute
- get_metadata for euca-bundle-vol
- root/ephemeral/swap device support
Devin Carlen (devcamcar) wrote : | # |
+1 to Brian's comments. There is insufficient separation of concerns here.
Brian Lamar (blamar) wrote : | # |
Moving to WIP due to 2 "Needs Fixings".
Isaku Yamahata (yamahata) wrote : | # |
On Wed, Jul 20, 2011 at 04:48:39PM -0000, Brian Lamar wrote:
> Review: Needs Fixing
> 401 +from nova.api.ec2 import ec2utils
> 475 +from nova.api.ec2 import ec2utils
>
> The database layer and the compute manager must not depend on something from the API layer. There are a couple degrees of separation missing here. This needs to be rethought, in my opinion.
thank you for review. I addressed it and added unit tests.
So the branch is ready for merge.
--
yamahata
Brian Lamar (blamar) wrote : | # |
341 +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>
After asking in the last meeting, I believe code should be copyright OpenStack, LLC.
365 + # NOTE(yamahata): see image_service.
366 + for bdm in properties.
367 + if bdm['virtual'] == 'root':
368 + root_device_name = bdm['device']
This logic seems a little off, I'm not certain why you'd loop here unless there could be multiple mappings with 'virtual' == 'root'. Maybe it is a little late to bring this up, and let me know if it is, but is 'virtual' unique? It would seem that having mappings look like:
{
"ami": {"device": "/dev/sda"},
"root": {"device": "sda"},
"ephemeral0": {"device": "sdb"},
}
instead of:
[
{'virtual': 'ami', 'device': '/dev/sda'},
{'virtual': 'root', 'device': 'sda'},
{'virtual': 'ephemeral0', 'device': 'sdb'},
]
might make the logic a bit cleaner in a lot of areas? This might not work but I thought I'd suggest it. For example, the block of code referenced earlier would become:
root_device_name = mappings.
I'm a little confused that mappings sometimes are a list and other times are a dictionary. Which one is correct?
Can you give more information or point me to documentation on what "ephemeral" means in this context? Seemingly "ephemeral" devices won't persist on reboot, or something to that effect?
Vish Ishaya (vishvananda) wrote : | # |
On Jul 25, 2011, at 10:41 AM, Brian Lamar wrote:
> Review: Needs Fixing
> 341 +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>
>
> After asking in the last meeting, I believe code should be copyright OpenStack, LLC.
I'm not sure what the confusion is here, but we DO NOT require copyright assignment. People are free to add their own copyrights to code that they right. The CLA is what allows us to license the the code through the apache license.
Vish
Vish Ishaya (vishvananda) wrote : | # |
s/right/write
Brian Lamar (blamar) wrote : | # |
Thanks Vish, I spoke up at the last meeting but I guess it was missed or I mis-asked the question. Good to have clarification and I'll stop asking for this.
>
> On Jul 25, 2011, at 10:41 AM, Brian Lamar wrote:
>
> > Review: Needs Fixing
> > 341 +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>
> >
> > After asking in the last meeting, I believe code should be copyright
> OpenStack, LLC.
>
> I'm not sure what the confusion is here, but we DO NOT require copyright
> assignment. People are free to add their own copyrights to code that they
> right. The CLA is what allows us to license the the code through the apache
> license.
>
> Vish
Vish Ishaya (vishvananda) wrote : | # |
> Can you give more information or point me to documentation on what "ephemeral"
> means in this context? Seemingly "ephemeral" devices won't persist on reboot,
> or something to that effect?
Ephemeral is the aws name for local storage that the vm gets that isn't part of the image. In the libvirt driver we attach a second unformatted drive as the "ephemeral" space.
Isaku Yamahata (yamahata) wrote : | # |
On Mon, Jul 25, 2011 at 05:41:30PM -0000, Brian Lamar wrote:
> Review: Needs Fixing
> 341 +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>
>
> After asking in the last meeting, I believe code should be copyright OpenStack, LLC.
>
> 365 + # NOTE(yamahata): see image_service.
> 366 + for bdm in properties.
> 367 + if bdm['virtual'] == 'root':
> 368 + root_device_name = bdm['device']
>
> This logic seems a little off, I'm not certain why you'd loop here unless there could be multiple mappings with 'virtual' == 'root'. Maybe it is a little late to bring this up, and let me know if it is, but is 'virtual' unique? It would seem that having mappings look like:
'virtual' is unique.
> {
> "ami": {"device": "/dev/sda"},
> "root": {"device": "sda"},
> "ephemeral0": {"device": "sdb"},
> }
>
> instead of:
>
> [
> {'virtual': 'ami', 'device': '/dev/sda'},
> {'virtual': 'root', 'device': 'sda'},
> {'virtual': 'ephemeral0', 'device': 'sdb'},
> ]
>
> might make the logic a bit cleaner in a lot of areas? This might not work but I thought I'd suggest it. For example, the block of code referenced earlier would become:
>
> root_device_name = mappings.
>
> I'm a little confused that mappings sometimes are a list and other times are a dictionary. Which one is correct?
A list as the internal representation. It's because they correspond to RDB rows.
Thus internally they are handled as lists consistently.(At least my intension is so)
The conversion from/to lists to/from dicts occurs when parsing API requests/
formatting API results if necessary.
Does it clarify?
Hmm, nova.api.
should be _format_
> Can you give more information or point me to documentation on what "ephemeral" means in this context? Seemingly "ephemeral" devices won't persist on reboot, or something to that effect?
Vish kindly answered this.
--
yamahata
- 1331. By Isaku Yamahata
-
api/ec2: rename CloudController
._get_instance_ mapping into _format_ instance_ mapping This patch renames nova.api.
ec2.cloud. CouldController ._get_instance_ mapping
to _format_instance_ mapping in order to make it clear that the method is
for API formatting, not for internal use.
Isaku Yamahata (yamahata) wrote : | # |
> Hmm, nova.api.
> should be _format_
Done.
- 1332. By Isaku Yamahata
-
merged with nova trunk
Vish Ishaya (vishvananda) wrote : | # |
This is all looking very good. Just a question. Will this cause problems for the other hypervisors? Do we need to add support in xen and vmware to create swap partitions based on block_device_
Isaku Yamahata (yamahata) wrote : | # |
On Tue, Aug 02, 2011 at 10:49:23PM -0000, Vish Ishaya wrote:
> Review: Needs Information
> This is all looking very good. Just a question. Will this cause problems for the other hypervisors?
No (except bugs). As long as the newly introduced features aren't used,
they should work as before.
Even if such features are used, e.g. block device mapping is specified,
they will be simply ignored.
> Do we need to add support in xen and vmware to create swap partitions based on block_device_
Although I'm not familiar with xen/vmware related code, I'm quite happy to help
those who implement it.
Does any user want it? Is there anyone who will implement it?
We will see the answers soon or after Diablo release.
--
yamahata
Soren Hansen (soren) wrote : | # |
Can you explain why you pulled the disk_prefix variable into python code instead of leaving it in the template. It was intentinoally put in the template so that if someone wanted to e.g. not use virtio disks with kvm, they didn't have to patch any code, they could just make a new template, setting the disk_prefix to something else, and point nova at the new template. That seems more awkward now.
Isaku Yamahata (yamahata) wrote : | # |
On Thu, Aug 04, 2011 at 11:11:26AM -0000, Soren Hansen wrote:
> Review: Needs Information
> Can you explain why you pulled the disk_prefix variable into python code instead of leaving it in the template. It was intentinoally put in the template so that if someone wanted to e.g. not use virtio disks with kvm, they didn't have to patch any code, they could just make a new template, setting the disk_prefix to something else, and point nova at the new template. That seems more awkward now.
It's because ec2 api exposes disk prefix to the users.
For exmample,
euca-run-instances ...
-b /dev/vda=nodevice
-b /dev/sdb=
-b /dev/hdc=ephemeral0
Thus the user can override the predefined devices (e.g. disk prefix + 'a')
or add new disks.
(Using inconsistent disk prefix would be insane. But this shows
the point well, I think. This is just for the explanation)
So the code needs to check if the device specified by the user
overrides the predefined device or not.
The cheetah template syntax isn't expressive enough to check it.
If disk prefix is so important, how about introducing another flag
like libvirt_
--
yamahata
- 1333. By Isaku Yamahata
-
merged with nova trunk.
- 1334. By Isaku Yamahata
-
fix mismerge
Vish Ishaya (vishvananda) wrote : | # |
My concerns were addressed
Brian Lamar (blamar) : | # |
Preview Diff
1 | === modified file 'nova/api/ec2/cloud.py' |
2 | --- nova/api/ec2/cloud.py 2011-07-26 20:58:02 +0000 |
3 | +++ nova/api/ec2/cloud.py 2011-08-05 06:31:22 +0000 |
4 | @@ -30,6 +30,7 @@ |
5 | import time |
6 | import shutil |
7 | |
8 | +from nova import block_device |
9 | from nova import compute |
10 | from nova import context |
11 | |
12 | @@ -78,6 +79,10 @@ |
13 | |
14 | # TODO(yamahata): hypervisor dependent default device name |
15 | _DEFAULT_ROOT_DEVICE_NAME = '/dev/sda1' |
16 | +_DEFAULT_MAPPINGS = {'ami': 'sda1', |
17 | + 'ephemeral0': 'sda2', |
18 | + 'root': _DEFAULT_ROOT_DEVICE_NAME, |
19 | + 'swap': 'sda3'} |
20 | |
21 | |
22 | def _parse_block_device_mapping(bdm): |
23 | @@ -105,7 +110,7 @@ |
24 | |
25 | |
26 | def _properties_get_mappings(properties): |
27 | - return ec2utils.mappings_prepend_dev(properties.get('mappings', [])) |
28 | + return block_device.mappings_prepend_dev(properties.get('mappings', [])) |
29 | |
30 | |
31 | def _format_block_device_mapping(bdm): |
32 | @@ -144,8 +149,7 @@ |
33 | """Format multiple BlockDeviceMappingItemType""" |
34 | mappings = [{'virtualName': m['virtual'], 'deviceName': m['device']} |
35 | for m in _properties_get_mappings(properties) |
36 | - if (m['virtual'] == 'swap' or |
37 | - m['virtual'].startswith('ephemeral'))] |
38 | + if block_device.is_swap_or_ephemeral(m['virtual'])] |
39 | |
40 | block_device_mapping = [_format_block_device_mapping(bdm) for bdm in |
41 | properties.get('block_device_mapping', [])] |
42 | @@ -233,6 +237,30 @@ |
43 | state = 'available' |
44 | return image['properties'].get('image_state', state) |
45 | |
46 | + def _format_instance_mapping(self, ctxt, instance_ref): |
47 | + root_device_name = instance_ref['root_device_name'] |
48 | + if root_device_name is None: |
49 | + return _DEFAULT_MAPPINGS |
50 | + |
51 | + mappings = {} |
52 | + mappings['ami'] = block_device.strip_dev(root_device_name) |
53 | + mappings['root'] = root_device_name |
54 | + |
55 | + # 'ephemeralN' and 'swap' |
56 | + for bdm in db.block_device_mapping_get_all_by_instance( |
57 | + ctxt, instance_ref['id']): |
58 | + if (bdm['volume_id'] or bdm['snapshot_id'] or bdm['no_device']): |
59 | + continue |
60 | + |
61 | + virtual_name = bdm['virtual_name'] |
62 | + if not virtual_name: |
63 | + continue |
64 | + |
65 | + if block_device.is_swap_or_ephemeral(virtual_name): |
66 | + mappings[virtual_name] = bdm['device_name'] |
67 | + |
68 | + return mappings |
69 | + |
70 | def get_metadata(self, address): |
71 | ctxt = context.get_admin_context() |
72 | instance_ref = self.compute_api.get_all(ctxt, fixed_ip=address) |
73 | @@ -259,18 +287,14 @@ |
74 | security_groups = db.security_group_get_by_instance(ctxt, |
75 | instance_ref['id']) |
76 | security_groups = [x['name'] for x in security_groups] |
77 | + mappings = self._format_instance_mapping(ctxt, instance_ref) |
78 | data = { |
79 | - 'user-data': base64.b64decode(instance_ref['user_data']), |
80 | + 'user-data': self._format_user_data(instance_ref), |
81 | 'meta-data': { |
82 | 'ami-id': image_ec2_id, |
83 | 'ami-launch-index': instance_ref['launch_index'], |
84 | 'ami-manifest-path': 'FIXME', |
85 | - 'block-device-mapping': { |
86 | - # TODO(vish): replace with real data |
87 | - 'ami': 'sda1', |
88 | - 'ephemeral0': 'sda2', |
89 | - 'root': _DEFAULT_ROOT_DEVICE_NAME, |
90 | - 'swap': 'sda3'}, |
91 | + 'block-device-mapping': mappings, |
92 | 'hostname': hostname, |
93 | 'instance-action': 'none', |
94 | 'instance-id': ec2_id, |
95 | @@ -948,13 +972,102 @@ |
96 | 'status': volume['attach_status'], |
97 | 'volumeId': ec2utils.id_to_ec2_vol_id(volume_id)} |
98 | |
99 | - def _convert_to_set(self, lst, label): |
100 | + @staticmethod |
101 | + def _convert_to_set(lst, label): |
102 | if lst is None or lst == []: |
103 | return None |
104 | if not isinstance(lst, list): |
105 | lst = [lst] |
106 | return [{label: x} for x in lst] |
107 | |
108 | + def _format_kernel_id(self, instance_ref, result, key): |
109 | + kernel_id = instance_ref['kernel_id'] |
110 | + if kernel_id is None: |
111 | + return |
112 | + result[key] = self.image_ec2_id(instance_ref['kernel_id'], 'aki') |
113 | + |
114 | + def _format_ramdisk_id(self, instance_ref, result, key): |
115 | + ramdisk_id = instance_ref['ramdisk_id'] |
116 | + if ramdisk_id is None: |
117 | + return |
118 | + result[key] = self.image_ec2_id(instance_ref['ramdisk_id'], 'ari') |
119 | + |
120 | + @staticmethod |
121 | + def _format_user_data(instance_ref): |
122 | + return base64.b64decode(instance_ref['user_data']) |
123 | + |
124 | + def describe_instance_attribute(self, context, instance_id, attribute, |
125 | + **kwargs): |
126 | + def _unsupported_attribute(instance, result): |
127 | + raise exception.ApiError(_('attribute not supported: %s') % |
128 | + attribute) |
129 | + |
130 | + def _format_attr_block_device_mapping(instance, result): |
131 | + tmp = {} |
132 | + self._format_instance_root_device_name(instance, tmp) |
133 | + self._format_instance_bdm(context, instance_id, |
134 | + tmp['rootDeviceName'], result) |
135 | + |
136 | + def _format_attr_disable_api_termination(instance, result): |
137 | + _unsupported_attribute(instance, result) |
138 | + |
139 | + def _format_attr_group_set(instance, result): |
140 | + CloudController._format_group_set(instance, result) |
141 | + |
142 | + def _format_attr_instance_initiated_shutdown_behavior(instance, |
143 | + result): |
144 | + state_description = instance['state_description'] |
145 | + state_to_value = {'stopping': 'stop', |
146 | + 'stopped': 'stop', |
147 | + 'terminating': 'terminate'} |
148 | + value = state_to_value.get(state_description) |
149 | + if value: |
150 | + result['instanceInitiatedShutdownBehavior'] = value |
151 | + |
152 | + def _format_attr_instance_type(instance, result): |
153 | + self._format_instance_type(instance, result) |
154 | + |
155 | + def _format_attr_kernel(instance, result): |
156 | + self._format_kernel_id(instance, result, 'kernel') |
157 | + |
158 | + def _format_attr_ramdisk(instance, result): |
159 | + self._format_ramdisk_id(instance, result, 'ramdisk') |
160 | + |
161 | + def _format_attr_root_device_name(instance, result): |
162 | + self._format_instance_root_device_name(instance, result) |
163 | + |
164 | + def _format_attr_source_dest_check(instance, result): |
165 | + _unsupported_attribute(instance, result) |
166 | + |
167 | + def _format_attr_user_data(instance, result): |
168 | + result['userData'] = self._format_user_data(instance) |
169 | + |
170 | + attribute_formatter = { |
171 | + 'blockDeviceMapping': _format_attr_block_device_mapping, |
172 | + 'disableApiTermination': _format_attr_disable_api_termination, |
173 | + 'groupSet': _format_attr_group_set, |
174 | + 'instanceInitiatedShutdownBehavior': |
175 | + _format_attr_instance_initiated_shutdown_behavior, |
176 | + 'instanceType': _format_attr_instance_type, |
177 | + 'kernel': _format_attr_kernel, |
178 | + 'ramdisk': _format_attr_ramdisk, |
179 | + 'rootDeviceName': _format_attr_root_device_name, |
180 | + 'sourceDestCheck': _format_attr_source_dest_check, |
181 | + 'userData': _format_attr_user_data, |
182 | + } |
183 | + |
184 | + fn = attribute_formatter.get(attribute) |
185 | + if fn is None: |
186 | + raise exception.ApiError( |
187 | + _('attribute not supported: %s') % attribute) |
188 | + |
189 | + ec2_instance_id = instance_id |
190 | + instance_id = ec2utils.ec2_id_to_id(ec2_instance_id) |
191 | + instance = self.compute_api.get(context, instance_id) |
192 | + result = {'instance_id': ec2_instance_id} |
193 | + fn(instance, result) |
194 | + return result |
195 | + |
196 | def describe_instances(self, context, **kwargs): |
197 | return self._format_describe_instances(context, **kwargs) |
198 | |
199 | @@ -1001,6 +1114,27 @@ |
200 | result['blockDeviceMapping'] = mapping |
201 | result['rootDeviceType'] = root_device_type |
202 | |
203 | + @staticmethod |
204 | + def _format_instance_root_device_name(instance, result): |
205 | + result['rootDeviceName'] = (instance.get('root_device_name') or |
206 | + _DEFAULT_ROOT_DEVICE_NAME) |
207 | + |
208 | + @staticmethod |
209 | + def _format_instance_type(instance, result): |
210 | + if instance['instance_type']: |
211 | + result['instanceType'] = instance['instance_type'].get('name') |
212 | + else: |
213 | + result['instanceType'] = None |
214 | + |
215 | + @staticmethod |
216 | + def _format_group_set(instance, result): |
217 | + security_group_names = [] |
218 | + if instance.get('security_groups'): |
219 | + for security_group in instance['security_groups']: |
220 | + security_group_names.append(security_group['name']) |
221 | + result['groupSet'] = CloudController._convert_to_set( |
222 | + security_group_names, 'groupId') |
223 | + |
224 | def _format_instances(self, context, instance_id=None, **kwargs): |
225 | # TODO(termie): this method is poorly named as its name does not imply |
226 | # that it will be making a variety of database calls |
227 | @@ -1026,6 +1160,8 @@ |
228 | ec2_id = ec2utils.id_to_ec2_id(instance_id) |
229 | i['instanceId'] = ec2_id |
230 | i['imageId'] = self.image_ec2_id(instance['image_ref']) |
231 | + self._format_kernel_id(instance, i, 'kernelId') |
232 | + self._format_ramdisk_id(instance, i, 'ramdiskId') |
233 | i['instanceState'] = { |
234 | 'code': instance['state'], |
235 | 'name': instance['state_description']} |
236 | @@ -1054,16 +1190,12 @@ |
237 | instance['project_id'], |
238 | instance['host']) |
239 | i['productCodesSet'] = self._convert_to_set([], 'product_codes') |
240 | - if instance['instance_type']: |
241 | - i['instanceType'] = instance['instance_type'].get('name') |
242 | - else: |
243 | - i['instanceType'] = None |
244 | + self._format_instance_type(instance, i) |
245 | i['launchTime'] = instance['created_at'] |
246 | i['amiLaunchIndex'] = instance['launch_index'] |
247 | i['displayName'] = instance['display_name'] |
248 | i['displayDescription'] = instance['display_description'] |
249 | - i['rootDeviceName'] = (instance.get('root_device_name') or |
250 | - _DEFAULT_ROOT_DEVICE_NAME) |
251 | + self._format_instance_root_device_name(instance, i) |
252 | self._format_instance_bdm(context, instance_id, |
253 | i['rootDeviceName'], i) |
254 | host = instance['host'] |
255 | @@ -1073,12 +1205,7 @@ |
256 | r = {} |
257 | r['reservationId'] = instance['reservation_id'] |
258 | r['ownerId'] = instance['project_id'] |
259 | - security_group_names = [] |
260 | - if instance.get('security_groups'): |
261 | - for security_group in instance['security_groups']: |
262 | - security_group_names.append(security_group['name']) |
263 | - r['groupSet'] = self._convert_to_set(security_group_names, |
264 | - 'groupId') |
265 | + self._format_group_set(instance, r) |
266 | r['instancesSet'] = [] |
267 | reservations[instance['reservation_id']] = r |
268 | reservations[instance['reservation_id']]['instancesSet'].append(i) |
269 | @@ -1314,7 +1441,7 @@ |
270 | i['architecture'] = image['properties'].get('architecture') |
271 | |
272 | properties = image['properties'] |
273 | - root_device_name = ec2utils.properties_root_device_name(properties) |
274 | + root_device_name = block_device.properties_root_device_name(properties) |
275 | root_device_type = 'instance-store' |
276 | for bdm in properties.get('block_device_mapping', []): |
277 | if (bdm.get('device_name') == root_device_name and |
278 | @@ -1387,7 +1514,7 @@ |
279 | |
280 | def _root_device_name_attribute(image, result): |
281 | result['rootDeviceName'] = \ |
282 | - ec2utils.properties_root_device_name(image['properties']) |
283 | + block_device.properties_root_device_name(image['properties']) |
284 | if result['rootDeviceName'] is None: |
285 | result['rootDeviceName'] = _DEFAULT_ROOT_DEVICE_NAME |
286 | |
287 | @@ -1520,8 +1647,7 @@ |
288 | if virtual_name in ('ami', 'root'): |
289 | continue |
290 | |
291 | - assert (virtual_name == 'swap' or |
292 | - virtual_name.startswith('ephemeral')) |
293 | + assert block_device.is_swap_or_ephemeral(virtual_name) |
294 | device_name = m['device'] |
295 | if device_name in [b['device_name'] for b in mapping |
296 | if not b.get('no_device', False)]: |
297 | |
298 | === modified file 'nova/api/ec2/ec2utils.py' |
299 | --- nova/api/ec2/ec2utils.py 2011-06-27 11:20:42 +0000 |
300 | +++ nova/api/ec2/ec2utils.py 2011-08-05 06:31:22 +0000 |
301 | @@ -135,32 +135,3 @@ |
302 | args[key] = value |
303 | |
304 | return args |
305 | - |
306 | - |
307 | -def properties_root_device_name(properties): |
308 | - """get root device name from image meta data. |
309 | - If it isn't specified, return None. |
310 | - """ |
311 | - root_device_name = None |
312 | - |
313 | - # NOTE(yamahata): see image_service.s3.s3create() |
314 | - for bdm in properties.get('mappings', []): |
315 | - if bdm['virtual'] == 'root': |
316 | - root_device_name = bdm['device'] |
317 | - |
318 | - # NOTE(yamahata): register_image's command line can override |
319 | - # <machine>.manifest.xml |
320 | - if 'root_device_name' in properties: |
321 | - root_device_name = properties['root_device_name'] |
322 | - |
323 | - return root_device_name |
324 | - |
325 | - |
326 | -def mappings_prepend_dev(mappings): |
327 | - """Prepend '/dev/' to 'device' entry of swap/ephemeral virtual type""" |
328 | - for m in mappings: |
329 | - virtual = m['virtual'] |
330 | - if ((virtual == 'swap' or virtual.startswith('ephemeral')) and |
331 | - (not m['device'].startswith('/'))): |
332 | - m['device'] = '/dev/' + m['device'] |
333 | - return mappings |
334 | |
335 | === added file 'nova/block_device.py' |
336 | --- nova/block_device.py 1970-01-01 00:00:00 +0000 |
337 | +++ nova/block_device.py 2011-08-05 06:31:22 +0000 |
338 | @@ -0,0 +1,71 @@ |
339 | +# vim: tabstop=4 shiftwidth=4 softtabstop=4 |
340 | + |
341 | +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp> |
342 | +# All Rights Reserved. |
343 | +# |
344 | +# Licensed under the Apache License, Version 2.0 (the "License"); you may |
345 | +# not use this file except in compliance with the License. You may obtain |
346 | +# a copy of the License at |
347 | +# |
348 | +# http://www.apache.org/licenses/LICENSE-2.0 |
349 | +# |
350 | +# Unless required by applicable law or agreed to in writing, software |
351 | +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT |
352 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
353 | +# License for the specific language governing permissions and limitations |
354 | +# under the License. |
355 | + |
356 | +import re |
357 | + |
358 | + |
359 | +def properties_root_device_name(properties): |
360 | + """get root device name from image meta data. |
361 | + If it isn't specified, return None. |
362 | + """ |
363 | + root_device_name = None |
364 | + |
365 | + # NOTE(yamahata): see image_service.s3.s3create() |
366 | + for bdm in properties.get('mappings', []): |
367 | + if bdm['virtual'] == 'root': |
368 | + root_device_name = bdm['device'] |
369 | + |
370 | + # NOTE(yamahata): register_image's command line can override |
371 | + # <machine>.manifest.xml |
372 | + if 'root_device_name' in properties: |
373 | + root_device_name = properties['root_device_name'] |
374 | + |
375 | + return root_device_name |
376 | + |
377 | + |
378 | +_ephemeral = re.compile('^ephemeral(\d|[1-9]\d+)$') |
379 | + |
380 | + |
381 | +def is_ephemeral(device_name): |
382 | + return _ephemeral.match(device_name) |
383 | + |
384 | + |
385 | +def ephemeral_num(ephemeral_name): |
386 | + assert is_ephemeral(ephemeral_name) |
387 | + return int(_ephemeral.sub('\\1', ephemeral_name)) |
388 | + |
389 | + |
390 | +def is_swap_or_ephemeral(device_name): |
391 | + return device_name == 'swap' or is_ephemeral(device_name) |
392 | + |
393 | + |
394 | +def mappings_prepend_dev(mappings): |
395 | + """Prepend '/dev/' to 'device' entry of swap/ephemeral virtual type""" |
396 | + for m in mappings: |
397 | + virtual = m['virtual'] |
398 | + if (is_swap_or_ephemeral(virtual) and |
399 | + (not m['device'].startswith('/'))): |
400 | + m['device'] = '/dev/' + m['device'] |
401 | + return mappings |
402 | + |
403 | + |
404 | +_dev = re.compile('^/dev/') |
405 | + |
406 | + |
407 | +def strip_dev(device_name): |
408 | + """remove leading '/dev/'""" |
409 | + return _dev.sub('', device_name) |
410 | |
411 | === modified file 'nova/compute/api.py' |
412 | --- nova/compute/api.py 2011-08-04 16:26:14 +0000 |
413 | +++ nova/compute/api.py 2011-08-05 06:31:22 +0000 |
414 | @@ -22,6 +22,7 @@ |
415 | import re |
416 | import time |
417 | |
418 | +from nova import block_device |
419 | from nova import db |
420 | from nova import exception |
421 | from nova import flags |
422 | @@ -32,7 +33,6 @@ |
423 | from nova import rpc |
424 | from nova import utils |
425 | from nova import volume |
426 | -from nova.api.ec2 import ec2utils |
427 | from nova.compute import instance_types |
428 | from nova.compute import power_state |
429 | from nova.compute.utils import terminate_volumes |
430 | @@ -218,7 +218,7 @@ |
431 | if reservation_id is None: |
432 | reservation_id = utils.generate_uid('r') |
433 | |
434 | - root_device_name = ec2utils.properties_root_device_name( |
435 | + root_device_name = block_device.properties_root_device_name( |
436 | image['properties']) |
437 | |
438 | base_options = { |
439 | @@ -250,34 +250,64 @@ |
440 | |
441 | return (num_instances, base_options, image) |
442 | |
443 | - def _update_image_block_device_mapping(self, elevated_context, instance_id, |
444 | + @staticmethod |
445 | + def _ephemeral_size(instance_type, ephemeral_name): |
446 | + num = block_device.ephemeral_num(ephemeral_name) |
447 | + |
448 | + # TODO(yamahata): ephemeralN where N > 0 |
449 | + # Only ephemeral0 is allowed for now because InstanceTypes |
450 | + # table only allows single local disk, local_gb. |
451 | + # In order to enhance it, we need to add a new columns to |
452 | + # instance_types table. |
453 | + if num > 0: |
454 | + return 0 |
455 | + |
456 | + return instance_type.get('local_gb') |
457 | + |
458 | + def _update_image_block_device_mapping(self, elevated_context, |
459 | + instance_type, instance_id, |
460 | mappings): |
461 | """tell vm driver to create ephemeral/swap device at boot time by |
462 | updating BlockDeviceMapping |
463 | """ |
464 | - for bdm in ec2utils.mappings_prepend_dev(mappings): |
465 | + instance_type = (instance_type or |
466 | + instance_types.get_default_instance_type()) |
467 | + |
468 | + for bdm in block_device.mappings_prepend_dev(mappings): |
469 | LOG.debug(_("bdm %s"), bdm) |
470 | |
471 | virtual_name = bdm['virtual'] |
472 | if virtual_name == 'ami' or virtual_name == 'root': |
473 | continue |
474 | |
475 | - assert (virtual_name == 'swap' or |
476 | - virtual_name.startswith('ephemeral')) |
477 | + if not block_device.is_swap_or_ephemeral(virtual_name): |
478 | + continue |
479 | + |
480 | + size = 0 |
481 | + if virtual_name == 'swap': |
482 | + size = instance_type.get('swap', 0) |
483 | + elif block_device.is_ephemeral(virtual_name): |
484 | + size = self._ephemeral_size(instance_type, virtual_name) |
485 | + |
486 | + if size == 0: |
487 | + continue |
488 | + |
489 | values = { |
490 | 'instance_id': instance_id, |
491 | 'device_name': bdm['device'], |
492 | - 'virtual_name': virtual_name, } |
493 | + 'virtual_name': virtual_name, |
494 | + 'volume_size': size} |
495 | self.db.block_device_mapping_update_or_create(elevated_context, |
496 | values) |
497 | |
498 | - def _update_block_device_mapping(self, elevated_context, instance_id, |
499 | + def _update_block_device_mapping(self, elevated_context, |
500 | + instance_type, instance_id, |
501 | block_device_mapping): |
502 | """tell vm driver to attach volume at boot time by updating |
503 | BlockDeviceMapping |
504 | """ |
505 | + LOG.debug(_("block_device_mapping %s"), block_device_mapping) |
506 | for bdm in block_device_mapping: |
507 | - LOG.debug(_('bdm %s'), bdm) |
508 | assert 'device_name' in bdm |
509 | |
510 | values = {'instance_id': instance_id} |
511 | @@ -286,10 +316,18 @@ |
512 | 'no_device'): |
513 | values[key] = bdm.get(key) |
514 | |
515 | + virtual_name = bdm.get('virtual_name') |
516 | + if (virtual_name is not None and |
517 | + block_device.is_ephemeral(virtual_name)): |
518 | + size = self._ephemeral_size(instance_type, virtual_name) |
519 | + if size == 0: |
520 | + continue |
521 | + values['volume_size'] = size |
522 | + |
523 | # NOTE(yamahata): NoDevice eliminates devices defined in image |
524 | # files by command line option. |
525 | # (--block-device-mapping) |
526 | - if bdm.get('virtual_name') == 'NoDevice': |
527 | + if virtual_name == 'NoDevice': |
528 | values['no_device'] = True |
529 | for k in ('delete_on_termination', 'volume_id', |
530 | 'snapshot_id', 'volume_id', 'volume_size', |
531 | @@ -299,8 +337,8 @@ |
532 | self.db.block_device_mapping_update_or_create(elevated_context, |
533 | values) |
534 | |
535 | - def create_db_entry_for_new_instance(self, context, image, base_options, |
536 | - security_group, block_device_mapping, num=1): |
537 | + def create_db_entry_for_new_instance(self, context, instance_type, image, |
538 | + base_options, security_group, block_device_mapping, num=1): |
539 | """Create an entry in the DB for this new instance, |
540 | including any related table updates (such as security group, |
541 | etc). |
542 | @@ -333,12 +371,12 @@ |
543 | security_group_id) |
544 | |
545 | # BlockDeviceMapping table |
546 | - self._update_image_block_device_mapping(elevated, instance_id, |
547 | - image['properties'].get('mappings', [])) |
548 | - self._update_block_device_mapping(elevated, instance_id, |
549 | + self._update_image_block_device_mapping(elevated, instance_type, |
550 | + instance_id, image['properties'].get('mappings', [])) |
551 | + self._update_block_device_mapping(elevated, instance_type, instance_id, |
552 | image['properties'].get('block_device_mapping', [])) |
553 | # override via command line option |
554 | - self._update_block_device_mapping(elevated, instance_id, |
555 | + self._update_block_device_mapping(elevated, instance_type, instance_id, |
556 | block_device_mapping) |
557 | |
558 | # Set sane defaults if not specified |
559 | @@ -453,7 +491,8 @@ |
560 | instances = [] |
561 | LOG.debug(_("Going to run %s instances..."), num_instances) |
562 | for num in range(num_instances): |
563 | - instance = self.create_db_entry_for_new_instance(context, image, |
564 | + instance = self.create_db_entry_for_new_instance(context, |
565 | + instance_type, image, |
566 | base_options, security_group, |
567 | block_device_mapping, num=num) |
568 | instances.append(instance) |
569 | |
570 | === modified file 'nova/compute/manager.py' |
571 | --- nova/compute/manager.py 2011-08-04 21:32:56 +0000 |
572 | +++ nova/compute/manager.py 2011-08-05 06:31:22 +0000 |
573 | @@ -45,6 +45,7 @@ |
574 | from eventlet import greenthread |
575 | |
576 | import nova.context |
577 | +from nova import block_device |
578 | from nova import exception |
579 | from nova import flags |
580 | import nova.image |
581 | @@ -260,6 +261,8 @@ |
582 | |
583 | volume_api = volume.API() |
584 | block_device_mapping = [] |
585 | + swap = None |
586 | + ephemerals = [] |
587 | for bdm in self.db.block_device_mapping_get_all_by_instance( |
588 | context, instance_id): |
589 | LOG.debug(_("setting up bdm %s"), bdm) |
590 | @@ -267,11 +270,18 @@ |
591 | if bdm['no_device']: |
592 | continue |
593 | if bdm['virtual_name']: |
594 | - # TODO(yamahata): |
595 | - # block devices for swap and ephemeralN will be |
596 | - # created by virt driver locally in compute node. |
597 | - assert (bdm['virtual_name'] == 'swap' or |
598 | - bdm['virtual_name'].startswith('ephemeral')) |
599 | + virtual_name = bdm['virtual_name'] |
600 | + device_name = bdm['device_name'] |
601 | + assert block_device.is_swap_or_ephemeral(virtual_name) |
602 | + if virtual_name == 'swap': |
603 | + swap = {'device_name': device_name, |
604 | + 'swap_size': bdm['volume_size']} |
605 | + elif block_device.is_ephemeral(virtual_name): |
606 | + eph = {'num': block_device.ephemeral_num(virtual_name), |
607 | + 'virtual_name': virtual_name, |
608 | + 'device_name': device_name, |
609 | + 'size': bdm['volume_size']} |
610 | + ephemerals.append(eph) |
611 | continue |
612 | |
613 | if ((bdm['snapshot_id'] is not None) and |
614 | @@ -307,7 +317,7 @@ |
615 | 'mount_device': |
616 | bdm['device_name']}) |
617 | |
618 | - return block_device_mapping |
619 | + return (swap, ephemerals, block_device_mapping) |
620 | |
621 | def _run_instance(self, context, instance_id, **kwargs): |
622 | """Launch a new instance with specified options.""" |
623 | @@ -348,13 +358,21 @@ |
624 | # all vif creation and network injection, maybe this is correct |
625 | network_info = [] |
626 | |
627 | - bd_mapping = self._setup_block_device_mapping(context, instance_id) |
628 | + (swap, ephemerals, |
629 | + block_device_mapping) = self._setup_block_device_mapping( |
630 | + context, instance_id) |
631 | + block_device_info = { |
632 | + 'root_device_name': instance['root_device_name'], |
633 | + 'swap': swap, |
634 | + 'ephemerals': ephemerals, |
635 | + 'block_device_mapping': block_device_mapping} |
636 | |
637 | # TODO(vish) check to make sure the availability zone matches |
638 | self._update_state(context, instance_id, power_state.BUILDING) |
639 | |
640 | try: |
641 | - self.driver.spawn(context, instance, network_info, bd_mapping) |
642 | + self.driver.spawn(context, instance, |
643 | + network_info, block_device_info) |
644 | except Exception as ex: # pylint: disable=W0702 |
645 | msg = _("Instance '%(instance_id)s' failed to spawn. Is " |
646 | "virtualization enabled in the BIOS? Details: " |
647 | |
648 | === modified file 'nova/db/sqlalchemy/api.py' |
649 | --- nova/db/sqlalchemy/api.py 2011-08-03 23:17:08 +0000 |
650 | +++ nova/db/sqlalchemy/api.py 2011-08-05 06:31:22 +0000 |
651 | @@ -20,6 +20,7 @@ |
652 | """ |
653 | import warnings |
654 | |
655 | +from nova import block_device |
656 | from nova import db |
657 | from nova import exception |
658 | from nova import flags |
659 | @@ -1681,7 +1682,9 @@ |
660 | session = get_session() |
661 | result = session.query(models.Network).\ |
662 | filter(or_(models.Network.cidr == cidr, |
663 | - models.Network.cidr_v6 == cidr)).first() |
664 | + models.Network.cidr_v6 == cidr)).\ |
665 | + filter_by(deleted=False).\ |
666 | + first() |
667 | |
668 | if not result: |
669 | raise exception.NetworkNotFoundForCidr(cidr=cidr) |
670 | @@ -2265,6 +2268,20 @@ |
671 | else: |
672 | result.update(values) |
673 | |
674 | + # NOTE(yamahata): same virtual device name can be specified multiple |
675 | + # times. So delete the existing ones. |
676 | + virtual_name = values['virtual_name'] |
677 | + if (virtual_name is not None and |
678 | + block_device.is_swap_or_ephemeral(virtual_name)): |
679 | + session.query(models.BlockDeviceMapping).\ |
680 | + filter_by(instance_id=values['instance_id']).\ |
681 | + filter_by(virtual_name=virtual_name).\ |
682 | + filter(models.BlockDeviceMapping.device_name != |
683 | + values['device_name']).\ |
684 | + update({'deleted': True, |
685 | + 'deleted_at': utils.utcnow(), |
686 | + 'updated_at': literal_column('updated_at')}) |
687 | + |
688 | |
689 | @require_context |
690 | def block_device_mapping_get_all_by_instance(context, instance_id): |
691 | |
692 | === modified file 'nova/image/glance.py' |
693 | --- nova/image/glance.py 2011-08-01 18:59:29 +0000 |
694 | +++ nova/image/glance.py 2011-08-05 06:31:22 +0000 |
695 | @@ -19,7 +19,9 @@ |
696 | |
697 | from __future__ import absolute_import |
698 | |
699 | +import copy |
700 | import datetime |
701 | +import json |
702 | import random |
703 | |
704 | from glance.common import exception as glance_exception |
705 | @@ -194,6 +196,7 @@ |
706 | self._set_client_context(context) |
707 | # NOTE(vish): show is to check if image is available |
708 | self.show(context, image_id) |
709 | + image_meta = _convert_to_string(image_meta) |
710 | try: |
711 | image_meta = self.client.update_image(image_id, image_meta, data) |
712 | except glance_exception.NotFound: |
713 | @@ -222,11 +225,19 @@ |
714 | pass |
715 | |
716 | @classmethod |
717 | + def _translate_to_service(cls, image_meta): |
718 | + image_meta = super(GlanceImageService, |
719 | + cls)._translate_to_service(image_meta) |
720 | + image_meta = _convert_to_string(image_meta) |
721 | + return image_meta |
722 | + |
723 | + @classmethod |
724 | def _translate_to_base(cls, image_meta): |
725 | """Override translation to handle conversion to datetime objects.""" |
726 | image_meta = service.BaseImageService._propertify_metadata( |
727 | image_meta, cls.SERVICE_IMAGE_ATTRS) |
728 | image_meta = _convert_timestamps_to_datetimes(image_meta) |
729 | + image_meta = _convert_from_string(image_meta) |
730 | return image_meta |
731 | |
732 | |
733 | @@ -252,3 +263,38 @@ |
734 | |
735 | raise ValueError(_('%(timestamp)s does not follow any of the ' |
736 | 'signatures: %(ISO_FORMATS)s') % locals()) |
737 | + |
738 | + |
739 | +# TODO(yamahata): use block-device-mapping extension to glance |
740 | +def _json_loads(properties, attr): |
741 | + prop = properties[attr] |
742 | + if isinstance(prop, basestring): |
743 | + properties[attr] = json.loads(prop) |
744 | + |
745 | + |
746 | +def _json_dumps(properties, attr): |
747 | + prop = properties[attr] |
748 | + if not isinstance(prop, basestring): |
749 | + properties[attr] = json.dumps(prop) |
750 | + |
751 | + |
752 | +_CONVERT_PROPS = ('block_device_mapping', 'mappings') |
753 | + |
754 | + |
755 | +def _convert(method, metadata): |
756 | + metadata = copy.deepcopy(metadata) # don't touch original metadata |
757 | + properties = metadata.get('properties') |
758 | + if properties: |
759 | + for attr in _CONVERT_PROPS: |
760 | + if attr in properties: |
761 | + method(properties, attr) |
762 | + |
763 | + return metadata |
764 | + |
765 | + |
766 | +def _convert_from_string(metadata): |
767 | + return _convert(_json_loads, metadata) |
768 | + |
769 | + |
770 | +def _convert_to_string(metadata): |
771 | + return _convert(_json_dumps, metadata) |
772 | |
773 | === modified file 'nova/tests/image/test_glance.py' |
774 | --- nova/tests/image/test_glance.py 2011-07-25 22:38:59 +0000 |
775 | +++ nova/tests/image/test_glance.py 2011-08-05 06:31:22 +0000 |
776 | @@ -235,3 +235,39 @@ |
777 | 'updated_at': None, |
778 | 'deleted_at': None} |
779 | return fixture |
780 | + |
781 | + |
782 | +class TestGlanceSerializer(unittest.TestCase): |
783 | + def test_serialize(self): |
784 | + metadata = {'name': 'image1', |
785 | + 'is_public': True, |
786 | + 'foo': 'bar', |
787 | + 'properties': { |
788 | + 'prop1': 'propvalue1', |
789 | + 'mappings': [ |
790 | + {'virtual': 'aaa', |
791 | + 'device': 'bbb'}, |
792 | + {'virtual': 'xxx', |
793 | + 'device': 'yyy'}], |
794 | + 'block_device_mapping': [ |
795 | + {'virtual_device': 'fake', |
796 | + 'device_name': '/dev/fake'}, |
797 | + {'virtual_device': 'ephemeral0', |
798 | + 'device_name': '/dev/fake0'}]}} |
799 | + |
800 | + converted_expected = { |
801 | + 'name': 'image1', |
802 | + 'is_public': True, |
803 | + 'foo': 'bar', |
804 | + 'properties': { |
805 | + 'prop1': 'propvalue1', |
806 | + 'mappings': |
807 | + '[{"device": "bbb", "virtual": "aaa"}, ' |
808 | + '{"device": "yyy", "virtual": "xxx"}]', |
809 | + 'block_device_mapping': |
810 | + '[{"virtual_device": "fake", "device_name": "/dev/fake"}, ' |
811 | + '{"virtual_device": "ephemeral0", ' |
812 | + '"device_name": "/dev/fake0"}]'}} |
813 | + converted = glance._convert_to_string(metadata) |
814 | + self.assertEqual(converted, converted_expected) |
815 | + self.assertEqual(glance._convert_from_string(converted), metadata) |
816 | |
817 | === modified file 'nova/tests/test_api.py' |
818 | --- nova/tests/test_api.py 2011-07-28 01:17:21 +0000 |
819 | +++ nova/tests/test_api.py 2011-08-05 06:31:22 +0000 |
820 | @@ -27,6 +27,7 @@ |
821 | import StringIO |
822 | import webob |
823 | |
824 | +from nova import block_device |
825 | from nova import context |
826 | from nova import exception |
827 | from nova import test |
828 | @@ -147,10 +148,12 @@ |
829 | properties0 = {'mappings': mappings} |
830 | properties1 = {'root_device_name': '/dev/sdb', 'mappings': mappings} |
831 | |
832 | - root_device_name = ec2utils.properties_root_device_name(properties0) |
833 | + root_device_name = block_device.properties_root_device_name( |
834 | + properties0) |
835 | self.assertEqual(root_device_name, '/dev/sda1') |
836 | |
837 | - root_device_name = ec2utils.properties_root_device_name(properties1) |
838 | + root_device_name = block_device.properties_root_device_name( |
839 | + properties1) |
840 | self.assertEqual(root_device_name, '/dev/sdb') |
841 | |
842 | def test_mapping_prepend_dev(self): |
843 | @@ -184,7 +187,7 @@ |
844 | 'device': '/dev/sdc1'}, |
845 | {'virtual': 'ephemeral1', |
846 | 'device': '/dev/sdc1'}] |
847 | - self.assertDictListMatch(ec2utils.mappings_prepend_dev(mappings), |
848 | + self.assertDictListMatch(block_device.mappings_prepend_dev(mappings), |
849 | expected_result) |
850 | |
851 | |
852 | |
853 | === added file 'nova/tests/test_block_device.py' |
854 | --- nova/tests/test_block_device.py 1970-01-01 00:00:00 +0000 |
855 | +++ nova/tests/test_block_device.py 2011-08-05 06:31:22 +0000 |
856 | @@ -0,0 +1,87 @@ |
857 | +# vim: tabstop=4 shiftwidth=4 softtabstop=4 |
858 | + |
859 | +# Copyright 2011 Isaku Yamahata |
860 | +# All Rights Reserved. |
861 | +# |
862 | +# Licensed under the Apache License, Version 2.0 (the "License"); you may |
863 | +# not use this file except in compliance with the License. You may obtain |
864 | +# a copy of the License at |
865 | +# |
866 | +# http://www.apache.org/licenses/LICENSE-2.0 |
867 | +# |
868 | +# Unless required by applicable law or agreed to in writing, software |
869 | +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT |
870 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
871 | +# License for the specific language governing permissions and limitations |
872 | +# under the License. |
873 | + |
874 | +""" |
875 | +Tests for Block Device utility functions. |
876 | +""" |
877 | + |
878 | +from nova import block_device |
879 | +from nova import test |
880 | + |
881 | + |
882 | +class BlockDeviceTestCase(test.TestCase): |
883 | + def test_properties(self): |
884 | + root_device0 = '/dev/sda' |
885 | + root_device1 = '/dev/sdb' |
886 | + mappings = [{'virtual': 'root', |
887 | + 'device': root_device0}] |
888 | + |
889 | + properties0 = {'mappings': mappings} |
890 | + properties1 = {'mappings': mappings, |
891 | + 'root_device_name': root_device1} |
892 | + |
893 | + self.assertEqual(block_device.properties_root_device_name({}), None) |
894 | + self.assertEqual( |
895 | + block_device.properties_root_device_name(properties0), |
896 | + root_device0) |
897 | + self.assertEqual( |
898 | + block_device.properties_root_device_name(properties1), |
899 | + root_device1) |
900 | + |
901 | + def test_ephemeral(self): |
902 | + self.assertFalse(block_device.is_ephemeral('ephemeral')) |
903 | + self.assertTrue(block_device.is_ephemeral('ephemeral0')) |
904 | + self.assertTrue(block_device.is_ephemeral('ephemeral1')) |
905 | + self.assertTrue(block_device.is_ephemeral('ephemeral11')) |
906 | + self.assertFalse(block_device.is_ephemeral('root')) |
907 | + self.assertFalse(block_device.is_ephemeral('swap')) |
908 | + self.assertFalse(block_device.is_ephemeral('/dev/sda1')) |
909 | + |
910 | + self.assertEqual(block_device.ephemeral_num('ephemeral0'), 0) |
911 | + self.assertEqual(block_device.ephemeral_num('ephemeral1'), 1) |
912 | + self.assertEqual(block_device.ephemeral_num('ephemeral11'), 11) |
913 | + |
914 | + self.assertFalse(block_device.is_swap_or_ephemeral('ephemeral')) |
915 | + self.assertTrue(block_device.is_swap_or_ephemeral('ephemeral0')) |
916 | + self.assertTrue(block_device.is_swap_or_ephemeral('ephemeral1')) |
917 | + self.assertTrue(block_device.is_swap_or_ephemeral('swap')) |
918 | + self.assertFalse(block_device.is_swap_or_ephemeral('root')) |
919 | + self.assertFalse(block_device.is_swap_or_ephemeral('/dev/sda1')) |
920 | + |
921 | + def test_mappings_prepend_dev(self): |
922 | + mapping = [ |
923 | + {'virtual': 'ami', 'device': '/dev/sda'}, |
924 | + {'virtual': 'root', 'device': 'sda'}, |
925 | + {'virtual': 'ephemeral0', 'device': 'sdb'}, |
926 | + {'virtual': 'swap', 'device': 'sdc'}, |
927 | + {'virtual': 'ephemeral1', 'device': 'sdd'}, |
928 | + {'virtual': 'ephemeral2', 'device': 'sde'}] |
929 | + |
930 | + expected = [ |
931 | + {'virtual': 'ami', 'device': '/dev/sda'}, |
932 | + {'virtual': 'root', 'device': 'sda'}, |
933 | + {'virtual': 'ephemeral0', 'device': '/dev/sdb'}, |
934 | + {'virtual': 'swap', 'device': '/dev/sdc'}, |
935 | + {'virtual': 'ephemeral1', 'device': '/dev/sdd'}, |
936 | + {'virtual': 'ephemeral2', 'device': '/dev/sde'}] |
937 | + |
938 | + prepended = block_device.mappings_prepend_dev(mapping) |
939 | + self.assertEqual(prepended.sort(), expected.sort()) |
940 | + |
941 | + def test_strip_dev(self): |
942 | + self.assertEqual(block_device.strip_dev('/dev/sda'), 'sda') |
943 | + self.assertEqual(block_device.strip_dev('sda'), 'sda') |
944 | |
945 | === modified file 'nova/tests/test_cloud.py' |
946 | --- nova/tests/test_cloud.py 2011-08-03 21:13:37 +0000 |
947 | +++ nova/tests/test_cloud.py 2011-08-05 06:31:22 +0000 |
948 | @@ -17,6 +17,8 @@ |
949 | # under the License. |
950 | import mox |
951 | |
952 | +import functools |
953 | + |
954 | from base64 import b64decode |
955 | from M2Crypto import BIO |
956 | from M2Crypto import RSA |
957 | @@ -892,13 +894,16 @@ |
958 | def test_modify_image_attribute(self): |
959 | modify_image_attribute = self.cloud.modify_image_attribute |
960 | |
961 | + fake_metadata = {'id': 1, 'container_format': 'ami', |
962 | + 'properties': {'kernel_id': 1, 'ramdisk_id': 1, |
963 | + 'type': 'machine'}, 'is_public': False} |
964 | + |
965 | def fake_show(meh, context, id): |
966 | - return {'id': 1, 'container_format': 'ami', |
967 | - 'properties': {'kernel_id': 1, 'ramdisk_id': 1, |
968 | - 'type': 'machine'}, 'is_public': False} |
969 | + return fake_metadata |
970 | |
971 | def fake_update(meh, context, image_id, metadata, data=None): |
972 | - return metadata |
973 | + fake_metadata.update(metadata) |
974 | + return fake_metadata |
975 | |
976 | self.stubs.Set(fake._FakeImageService, 'show', fake_show) |
977 | self.stubs.Set(fake._FakeImageService, 'show_by_name', fake_show) |
978 | @@ -1464,3 +1469,147 @@ |
979 | # TODO(yamahata): clean up snapshot created by CreateImage. |
980 | |
981 | self._restart_compute_service() |
982 | + |
983 | + @staticmethod |
984 | + def _fake_bdm_get(ctxt, id): |
985 | + return [{'volume_id': 87654321, |
986 | + 'snapshot_id': None, |
987 | + 'no_device': None, |
988 | + 'virtual_name': None, |
989 | + 'delete_on_termination': True, |
990 | + 'device_name': '/dev/sdh'}, |
991 | + {'volume_id': None, |
992 | + 'snapshot_id': 98765432, |
993 | + 'no_device': None, |
994 | + 'virtual_name': None, |
995 | + 'delete_on_termination': True, |
996 | + 'device_name': '/dev/sdi'}, |
997 | + {'volume_id': None, |
998 | + 'snapshot_id': None, |
999 | + 'no_device': True, |
1000 | + 'virtual_name': None, |
1001 | + 'delete_on_termination': None, |
1002 | + 'device_name': None}, |
1003 | + {'volume_id': None, |
1004 | + 'snapshot_id': None, |
1005 | + 'no_device': None, |
1006 | + 'virtual_name': 'ephemeral0', |
1007 | + 'delete_on_termination': None, |
1008 | + 'device_name': '/dev/sdb'}, |
1009 | + {'volume_id': None, |
1010 | + 'snapshot_id': None, |
1011 | + 'no_device': None, |
1012 | + 'virtual_name': 'swap', |
1013 | + 'delete_on_termination': None, |
1014 | + 'device_name': '/dev/sdc'}, |
1015 | + {'volume_id': None, |
1016 | + 'snapshot_id': None, |
1017 | + 'no_device': None, |
1018 | + 'virtual_name': 'ephemeral1', |
1019 | + 'delete_on_termination': None, |
1020 | + 'device_name': '/dev/sdd'}, |
1021 | + {'volume_id': None, |
1022 | + 'snapshot_id': None, |
1023 | + 'no_device': None, |
1024 | + 'virtual_name': 'ephemeral2', |
1025 | + 'delete_on_termination': None, |
1026 | + 'device_name': '/dev/sd3'}, |
1027 | + ] |
1028 | + |
1029 | + def test_get_instance_mapping(self): |
1030 | + """Make sure that _get_instance_mapping works""" |
1031 | + ctxt = None |
1032 | + instance_ref0 = {'id': 0, |
1033 | + 'root_device_name': None} |
1034 | + instance_ref1 = {'id': 0, |
1035 | + 'root_device_name': '/dev/sda1'} |
1036 | + |
1037 | + self.stubs.Set(db, 'block_device_mapping_get_all_by_instance', |
1038 | + self._fake_bdm_get) |
1039 | + |
1040 | + expected = {'ami': 'sda1', |
1041 | + 'root': '/dev/sda1', |
1042 | + 'ephemeral0': '/dev/sdb', |
1043 | + 'swap': '/dev/sdc', |
1044 | + 'ephemeral1': '/dev/sdd', |
1045 | + 'ephemeral2': '/dev/sd3'} |
1046 | + |
1047 | + self.assertEqual(self.cloud._format_instance_mapping(ctxt, |
1048 | + instance_ref0), |
1049 | + cloud._DEFAULT_MAPPINGS) |
1050 | + self.assertEqual(self.cloud._format_instance_mapping(ctxt, |
1051 | + instance_ref1), |
1052 | + expected) |
1053 | + |
1054 | + def test_describe_instance_attribute(self): |
1055 | + """Make sure that describe_instance_attribute works""" |
1056 | + self.stubs.Set(db, 'block_device_mapping_get_all_by_instance', |
1057 | + self._fake_bdm_get) |
1058 | + |
1059 | + def fake_get(ctxt, instance_id): |
1060 | + return { |
1061 | + 'id': 0, |
1062 | + 'root_device_name': '/dev/sdh', |
1063 | + 'security_groups': [{'name': 'fake0'}, {'name': 'fake1'}], |
1064 | + 'state_description': 'stopping', |
1065 | + 'instance_type': {'name': 'fake_type'}, |
1066 | + 'kernel_id': 1, |
1067 | + 'ramdisk_id': 2, |
1068 | + 'user_data': 'fake-user data', |
1069 | + } |
1070 | + self.stubs.Set(self.cloud.compute_api, 'get', fake_get) |
1071 | + |
1072 | + def fake_volume_get(ctxt, volume_id, session=None): |
1073 | + if volume_id == 87654321: |
1074 | + return {'id': volume_id, |
1075 | + 'attach_time': '13:56:24', |
1076 | + 'status': 'in-use'} |
1077 | + raise exception.VolumeNotFound(volume_id=volume_id) |
1078 | + self.stubs.Set(db.api, 'volume_get', fake_volume_get) |
1079 | + |
1080 | + get_attribute = functools.partial( |
1081 | + self.cloud.describe_instance_attribute, |
1082 | + self.context, 'i-12345678') |
1083 | + |
1084 | + bdm = get_attribute('blockDeviceMapping') |
1085 | + bdm['blockDeviceMapping'].sort() |
1086 | + |
1087 | + expected_bdm = {'instance_id': 'i-12345678', |
1088 | + 'rootDeviceType': 'ebs', |
1089 | + 'blockDeviceMapping': [ |
1090 | + {'deviceName': '/dev/sdh', |
1091 | + 'ebs': {'status': 'in-use', |
1092 | + 'deleteOnTermination': True, |
1093 | + 'volumeId': 87654321, |
1094 | + 'attachTime': '13:56:24'}}]} |
1095 | + expected_bdm['blockDeviceMapping'].sort() |
1096 | + self.assertEqual(bdm, expected_bdm) |
1097 | + # NOTE(yamahata): this isn't supported |
1098 | + # get_attribute('disableApiTermination') |
1099 | + groupSet = get_attribute('groupSet') |
1100 | + groupSet['groupSet'].sort() |
1101 | + expected_groupSet = {'instance_id': 'i-12345678', |
1102 | + 'groupSet': [{'groupId': 'fake0'}, |
1103 | + {'groupId': 'fake1'}]} |
1104 | + expected_groupSet['groupSet'].sort() |
1105 | + self.assertEqual(groupSet, expected_groupSet) |
1106 | + self.assertEqual(get_attribute('instanceInitiatedShutdownBehavior'), |
1107 | + {'instance_id': 'i-12345678', |
1108 | + 'instanceInitiatedShutdownBehavior': 'stop'}) |
1109 | + self.assertEqual(get_attribute('instanceType'), |
1110 | + {'instance_id': 'i-12345678', |
1111 | + 'instanceType': 'fake_type'}) |
1112 | + self.assertEqual(get_attribute('kernel'), |
1113 | + {'instance_id': 'i-12345678', |
1114 | + 'kernel': 'aki-00000001'}) |
1115 | + self.assertEqual(get_attribute('ramdisk'), |
1116 | + {'instance_id': 'i-12345678', |
1117 | + 'ramdisk': 'ari-00000002'}) |
1118 | + self.assertEqual(get_attribute('rootDeviceName'), |
1119 | + {'instance_id': 'i-12345678', |
1120 | + 'rootDeviceName': '/dev/sdh'}) |
1121 | + # NOTE(yamahata): this isn't supported |
1122 | + # get_attribute('sourceDestCheck') |
1123 | + self.assertEqual(get_attribute('userData'), |
1124 | + {'instance_id': 'i-12345678', |
1125 | + 'userData': '}\xa9\x1e\xba\xc7\xabu\xabZ'}) |
1126 | |
1127 | === modified file 'nova/tests/test_compute.py' |
1128 | --- nova/tests/test_compute.py 2011-08-03 11:31:10 +0000 |
1129 | +++ nova/tests/test_compute.py 2011-08-05 06:31:22 +0000 |
1130 | @@ -877,15 +877,17 @@ |
1131 | return bdm |
1132 | |
1133 | def test_update_block_device_mapping(self): |
1134 | + swap_size = 1 |
1135 | + instance_type = {'swap': swap_size} |
1136 | instance_id = self._create_instance() |
1137 | mappings = [ |
1138 | {'virtual': 'ami', 'device': 'sda1'}, |
1139 | {'virtual': 'root', 'device': '/dev/sda1'}, |
1140 | |
1141 | + {'virtual': 'swap', 'device': 'sdb4'}, |
1142 | + {'virtual': 'swap', 'device': 'sdb3'}, |
1143 | + {'virtual': 'swap', 'device': 'sdb2'}, |
1144 | {'virtual': 'swap', 'device': 'sdb1'}, |
1145 | - {'virtual': 'swap', 'device': 'sdb2'}, |
1146 | - {'virtual': 'swap', 'device': 'sdb3'}, |
1147 | - {'virtual': 'swap', 'device': 'sdb4'}, |
1148 | |
1149 | {'virtual': 'ephemeral0', 'device': 'sdc1'}, |
1150 | {'virtual': 'ephemeral1', 'device': 'sdc2'}, |
1151 | @@ -927,32 +929,36 @@ |
1152 | 'no_device': True}] |
1153 | |
1154 | self.compute_api._update_image_block_device_mapping( |
1155 | - self.context, instance_id, mappings) |
1156 | + self.context, instance_type, instance_id, mappings) |
1157 | |
1158 | bdms = [self._parse_db_block_device_mapping(bdm_ref) |
1159 | for bdm_ref in db.block_device_mapping_get_all_by_instance( |
1160 | self.context, instance_id)] |
1161 | expected_result = [ |
1162 | - {'virtual_name': 'swap', 'device_name': '/dev/sdb1'}, |
1163 | - {'virtual_name': 'swap', 'device_name': '/dev/sdb2'}, |
1164 | - {'virtual_name': 'swap', 'device_name': '/dev/sdb3'}, |
1165 | - {'virtual_name': 'swap', 'device_name': '/dev/sdb4'}, |
1166 | + {'virtual_name': 'swap', 'device_name': '/dev/sdb1', |
1167 | + 'volume_size': swap_size}, |
1168 | {'virtual_name': 'ephemeral0', 'device_name': '/dev/sdc1'}, |
1169 | - {'virtual_name': 'ephemeral1', 'device_name': '/dev/sdc2'}, |
1170 | - {'virtual_name': 'ephemeral2', 'device_name': '/dev/sdc3'}] |
1171 | + |
1172 | + # NOTE(yamahata): ATM only ephemeral0 is supported. |
1173 | + # they're ignored for now |
1174 | + #{'virtual_name': 'ephemeral1', 'device_name': '/dev/sdc2'}, |
1175 | + #{'virtual_name': 'ephemeral2', 'device_name': '/dev/sdc3'} |
1176 | + ] |
1177 | bdms.sort() |
1178 | expected_result.sort() |
1179 | self.assertDictListMatch(bdms, expected_result) |
1180 | |
1181 | self.compute_api._update_block_device_mapping( |
1182 | - self.context, instance_id, block_device_mapping) |
1183 | + self.context, instance_types.get_default_instance_type(), |
1184 | + instance_id, block_device_mapping) |
1185 | bdms = [self._parse_db_block_device_mapping(bdm_ref) |
1186 | for bdm_ref in db.block_device_mapping_get_all_by_instance( |
1187 | self.context, instance_id)] |
1188 | expected_result = [ |
1189 | {'snapshot_id': 0x12345678, 'device_name': '/dev/sda1'}, |
1190 | |
1191 | - {'virtual_name': 'swap', 'device_name': '/dev/sdb1'}, |
1192 | + {'virtual_name': 'swap', 'device_name': '/dev/sdb1', |
1193 | + 'volume_size': swap_size}, |
1194 | {'snapshot_id': 0x23456789, 'device_name': '/dev/sdb2'}, |
1195 | {'snapshot_id': 0x3456789A, 'device_name': '/dev/sdb3'}, |
1196 | {'no_device': True, 'device_name': '/dev/sdb4'}, |
1197 | @@ -974,3 +980,13 @@ |
1198 | self.context, instance_id): |
1199 | db.block_device_mapping_destroy(self.context, bdm['id']) |
1200 | self.compute.terminate_instance(self.context, instance_id) |
1201 | + |
1202 | + def test_ephemeral_size(self): |
1203 | + local_size = 2 |
1204 | + inst_type = {'local_gb': local_size} |
1205 | + self.assertEqual(self.compute_api._ephemeral_size(inst_type, |
1206 | + 'ephemeral0'), |
1207 | + local_size) |
1208 | + self.assertEqual(self.compute_api._ephemeral_size(inst_type, |
1209 | + 'ephemeral1'), |
1210 | + 0) |
1211 | |
1212 | === modified file 'nova/tests/test_libvirt.py' |
1213 | --- nova/tests/test_libvirt.py 2011-08-03 19:22:58 +0000 |
1214 | +++ nova/tests/test_libvirt.py 2011-08-05 06:31:22 +0000 |
1215 | @@ -169,6 +169,7 @@ |
1216 | 'project_id': 'fake', |
1217 | 'bridge': 'br101', |
1218 | 'image_ref': '123456', |
1219 | + 'local_gb': 20, |
1220 | 'instance_type_id': '5'} # m1.small |
1221 | |
1222 | def lazy_load_library_exists(self): |
1223 | @@ -744,6 +745,42 @@ |
1224 | ip = conn.get_host_ip_addr() |
1225 | self.assertEquals(ip, FLAGS.my_ip) |
1226 | |
1227 | + def test_volume_in_mapping(self): |
1228 | + conn = connection.LibvirtConnection(False) |
1229 | + swap = {'device_name': '/dev/sdb', |
1230 | + 'swap_size': 1} |
1231 | + ephemerals = [{'num': 0, |
1232 | + 'virtual_name': 'ephemeral0', |
1233 | + 'device_name': '/dev/sdc1', |
1234 | + 'size': 1}, |
1235 | + {'num': 2, |
1236 | + 'virtual_name': 'ephemeral2', |
1237 | + 'device_name': '/dev/sdd', |
1238 | + 'size': 1}] |
1239 | + block_device_mapping = [{'mount_device': '/dev/sde', |
1240 | + 'device_path': 'fake_device'}, |
1241 | + {'mount_device': '/dev/sdf', |
1242 | + 'device_path': 'fake_device'}] |
1243 | + block_device_info = { |
1244 | + 'root_device_name': '/dev/sda', |
1245 | + 'swap': swap, |
1246 | + 'ephemerals': ephemerals, |
1247 | + 'block_device_mapping': block_device_mapping} |
1248 | + |
1249 | + def _assert_volume_in_mapping(device_name, true_or_false): |
1250 | + self.assertEquals(conn._volume_in_mapping(device_name, |
1251 | + block_device_info), |
1252 | + true_or_false) |
1253 | + |
1254 | + _assert_volume_in_mapping('sda', False) |
1255 | + _assert_volume_in_mapping('sdb', True) |
1256 | + _assert_volume_in_mapping('sdc1', True) |
1257 | + _assert_volume_in_mapping('sdd', True) |
1258 | + _assert_volume_in_mapping('sde', True) |
1259 | + _assert_volume_in_mapping('sdf', True) |
1260 | + _assert_volume_in_mapping('sdg', False) |
1261 | + _assert_volume_in_mapping('sdh1', False) |
1262 | + |
1263 | |
1264 | class NWFilterFakes: |
1265 | def __init__(self): |
1266 | |
1267 | === modified file 'nova/tests/test_metadata.py' |
1268 | --- nova/tests/test_metadata.py 2011-06-27 13:58:17 +0000 |
1269 | +++ nova/tests/test_metadata.py 2011-08-05 06:31:22 +0000 |
1270 | @@ -43,6 +43,7 @@ |
1271 | 'reservation_id': 'r-xxxxxxxx', |
1272 | 'user_data': '', |
1273 | 'image_ref': 7, |
1274 | + 'root_device_name': '/dev/sda1', |
1275 | 'hostname': 'test'}) |
1276 | |
1277 | def instance_get(*args, **kwargs): |
1278 | |
1279 | === added file 'nova/tests/test_virt.py' |
1280 | --- nova/tests/test_virt.py 1970-01-01 00:00:00 +0000 |
1281 | +++ nova/tests/test_virt.py 2011-08-05 06:31:22 +0000 |
1282 | @@ -0,0 +1,83 @@ |
1283 | +# vim: tabstop=4 shiftwidth=4 softtabstop=4 |
1284 | + |
1285 | +# Copyright 2011 Isaku Yamahata |
1286 | +# All Rights Reserved. |
1287 | +# |
1288 | +# Licensed under the Apache License, Version 2.0 (the "License"); you may |
1289 | +# not use this file except in compliance with the License. You may obtain |
1290 | +# a copy of the License at |
1291 | +# |
1292 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1293 | +# |
1294 | +# Unless required by applicable law or agreed to in writing, software |
1295 | +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT |
1296 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
1297 | +# License for the specific language governing permissions and limitations |
1298 | +# under the License. |
1299 | + |
1300 | +from nova import flags |
1301 | +from nova import test |
1302 | +from nova.virt import driver |
1303 | + |
1304 | +FLAGS = flags.FLAGS |
1305 | + |
1306 | + |
1307 | +class TestVirtDriver(test.TestCase): |
1308 | + def test_block_device(self): |
1309 | + swap = {'device_name': '/dev/sdb', |
1310 | + 'swap_size': 1} |
1311 | + ephemerals = [{'num': 0, |
1312 | + 'virtual_name': 'ephemeral0', |
1313 | + 'device_name': '/dev/sdc1', |
1314 | + 'size': 1}] |
1315 | + block_device_mapping = [{'mount_device': '/dev/sde', |
1316 | + 'device_path': 'fake_device'}] |
1317 | + block_device_info = { |
1318 | + 'root_device_name': '/dev/sda', |
1319 | + 'swap': swap, |
1320 | + 'ephemerals': ephemerals, |
1321 | + 'block_device_mapping': block_device_mapping} |
1322 | + |
1323 | + empty_block_device_info = {} |
1324 | + |
1325 | + self.assertEqual( |
1326 | + driver.block_device_info_get_root(block_device_info), '/dev/sda') |
1327 | + self.assertEqual( |
1328 | + driver.block_device_info_get_root(empty_block_device_info), None) |
1329 | + self.assertEqual( |
1330 | + driver.block_device_info_get_root(None), None) |
1331 | + |
1332 | + self.assertEqual( |
1333 | + driver.block_device_info_get_swap(block_device_info), swap) |
1334 | + self.assertEqual(driver.block_device_info_get_swap( |
1335 | + empty_block_device_info)['device_name'], None) |
1336 | + self.assertEqual(driver.block_device_info_get_swap( |
1337 | + empty_block_device_info)['swap_size'], 0) |
1338 | + self.assertEqual( |
1339 | + driver.block_device_info_get_swap({'swap': None})['device_name'], |
1340 | + None) |
1341 | + self.assertEqual( |
1342 | + driver.block_device_info_get_swap({'swap': None})['swap_size'], |
1343 | + 0) |
1344 | + self.assertEqual( |
1345 | + driver.block_device_info_get_swap(None)['device_name'], None) |
1346 | + self.assertEqual( |
1347 | + driver.block_device_info_get_swap(None)['swap_size'], 0) |
1348 | + |
1349 | + self.assertEqual( |
1350 | + driver.block_device_info_get_ephemerals(block_device_info), |
1351 | + ephemerals) |
1352 | + self.assertEqual( |
1353 | + driver.block_device_info_get_ephemerals(empty_block_device_info), |
1354 | + []) |
1355 | + self.assertEqual( |
1356 | + driver.block_device_info_get_ephemerals(None), |
1357 | + []) |
1358 | + |
1359 | + def test_swap_is_usable(self): |
1360 | + self.assertFalse(driver.swap_is_usable(None)) |
1361 | + self.assertFalse(driver.swap_is_usable({'device_name': None})) |
1362 | + self.assertFalse(driver.swap_is_usable({'device_name': '/dev/sdb', |
1363 | + 'swap_size': 0})) |
1364 | + self.assertTrue(driver.swap_is_usable({'device_name': '/dev/sdb', |
1365 | + 'swap_size': 1})) |
1366 | |
1367 | === modified file 'nova/virt/driver.py' |
1368 | --- nova/virt/driver.py 2011-07-29 21:58:27 +0000 |
1369 | +++ nova/virt/driver.py 2011-08-05 06:31:22 +0000 |
1370 | @@ -32,6 +32,33 @@ |
1371 | self.state = state |
1372 | |
1373 | |
1374 | +def block_device_info_get_root(block_device_info): |
1375 | + block_device_info = block_device_info or {} |
1376 | + return block_device_info.get('root_device_name') |
1377 | + |
1378 | + |
1379 | +def block_device_info_get_swap(block_device_info): |
1380 | + block_device_info = block_device_info or {} |
1381 | + return block_device_info.get('swap') or {'device_name': None, |
1382 | + 'swap_size': 0} |
1383 | + |
1384 | + |
1385 | +def swap_is_usable(swap): |
1386 | + return swap and swap['device_name'] and swap['swap_size'] > 0 |
1387 | + |
1388 | + |
1389 | +def block_device_info_get_ephemerals(block_device_info): |
1390 | + block_device_info = block_device_info or {} |
1391 | + ephemerals = block_device_info.get('ephemerals') or [] |
1392 | + return ephemerals |
1393 | + |
1394 | + |
1395 | +def block_device_info_get_mapping(block_device_info): |
1396 | + block_device_info = block_device_info or {} |
1397 | + block_device_mapping = block_device_info.get('block_device_mapping') or [] |
1398 | + return block_device_mapping |
1399 | + |
1400 | + |
1401 | class ComputeDriver(object): |
1402 | """Base class for compute drivers. |
1403 | |
1404 | @@ -65,8 +92,8 @@ |
1405 | # TODO(Vek): Need to pass context in for access to auth_token |
1406 | raise NotImplementedError() |
1407 | |
1408 | - def spawn(self, context, instance, network_info, |
1409 | - block_device_mapping=None): |
1410 | + def spawn(self, context, instance, |
1411 | + network_info=None, block_device_info=None): |
1412 | """Launch a VM for the specified instance""" |
1413 | raise NotImplementedError() |
1414 | |
1415 | |
1416 | === modified file 'nova/virt/fake.py' |
1417 | --- nova/virt/fake.py 2011-08-02 21:45:57 +0000 |
1418 | +++ nova/virt/fake.py 2011-08-05 06:31:22 +0000 |
1419 | @@ -129,8 +129,8 @@ |
1420 | info_list.append(self._map_to_instance_info(instance)) |
1421 | return info_list |
1422 | |
1423 | - def spawn(self, context, instance, network_info, |
1424 | - block_device_mapping=None): |
1425 | + def spawn(self, context, instance, |
1426 | + network_info=None, block_device_info=None): |
1427 | """ |
1428 | Create a new instance/VM/domain on the virtualization platform. |
1429 | |
1430 | |
1431 | === modified file 'nova/virt/hyperv.py' |
1432 | --- nova/virt/hyperv.py 2011-07-29 21:58:27 +0000 |
1433 | +++ nova/virt/hyperv.py 2011-08-05 06:31:22 +0000 |
1434 | @@ -138,8 +138,8 @@ |
1435 | |
1436 | return instance_infos |
1437 | |
1438 | - def spawn(self, context, instance, network_info, |
1439 | - block_device_mapping=None): |
1440 | + def spawn(self, context, instance, |
1441 | + network_info=None, block_device_info=None): |
1442 | """ Create a new VM and start it.""" |
1443 | vm = self._lookup(instance.name) |
1444 | if vm is not None: |
1445 | |
1446 | === modified file 'nova/virt/libvirt.xml.template' |
1447 | --- nova/virt/libvirt.xml.template 2011-07-26 00:41:55 +0000 |
1448 | +++ nova/virt/libvirt.xml.template 2011-08-05 06:31:22 +0000 |
1449 | @@ -3,24 +3,22 @@ |
1450 | <memory>${memory_kb}</memory> |
1451 | <os> |
1452 | #if $type == 'lxc' |
1453 | - #set $disk_prefix = '' |
1454 | #set $disk_bus = '' |
1455 | <type>exe</type> |
1456 | <init>/sbin/init</init> |
1457 | #else if $type == 'uml' |
1458 | - #set $disk_prefix = 'ubd' |
1459 | #set $disk_bus = 'uml' |
1460 | <type>uml</type> |
1461 | <kernel>/usr/bin/linux</kernel> |
1462 | - <root>/dev/ubda</root> |
1463 | + #set $root_device_name = $getVar('root_device_name', '/dev/ubda') |
1464 | + <root>${root_device_name}</root> |
1465 | #else |
1466 | #if $type == 'xen' |
1467 | - #set $disk_prefix = 'sd' |
1468 | #set $disk_bus = 'scsi' |
1469 | <type>linux</type> |
1470 | - <root>/dev/xvda</root> |
1471 | + #set $root_device_name = $getVar('root_device_name', '/dev/xvda') |
1472 | + <root>${root_device_name}</root> |
1473 | #else |
1474 | - #set $disk_prefix = 'vd' |
1475 | #set $disk_bus = 'virtio' |
1476 | <type>hvm</type> |
1477 | #end if |
1478 | @@ -33,7 +31,8 @@ |
1479 | #if $type == 'xen' |
1480 | <cmdline>ro</cmdline> |
1481 | #else |
1482 | - <cmdline>root=/dev/vda console=ttyS0</cmdline> |
1483 | + #set $root_device_name = $getVar('root_device_name', '/dev/vda') |
1484 | + <cmdline>root=${root_device_name} console=ttyS0</cmdline> |
1485 | #end if |
1486 | #if $getVar('ramdisk', None) |
1487 | <initrd>${ramdisk}</initrd> |
1488 | @@ -71,16 +70,30 @@ |
1489 | <disk type='file'> |
1490 | <driver type='${driver_type}'/> |
1491 | <source file='${basepath}/disk'/> |
1492 | - <target dev='${disk_prefix}a' bus='${disk_bus}'/> |
1493 | + <target dev='${root_device}' bus='${disk_bus}'/> |
1494 | </disk> |
1495 | #end if |
1496 | - #if $getVar('local', False) |
1497 | + #if $getVar('local_device', False) |
1498 | <disk type='file'> |
1499 | <driver type='${driver_type}'/> |
1500 | <source file='${basepath}/disk.local'/> |
1501 | - <target dev='${disk_prefix}b' bus='${disk_bus}'/> |
1502 | + <target dev='${local_device}' bus='${disk_bus}'/> |
1503 | </disk> |
1504 | #end if |
1505 | + #for $eph in $ephemerals |
1506 | + <disk type='block'> |
1507 | + <driver type='${driver_type}'/> |
1508 | + <source dev='${basepath}/${eph.device_path}'/> |
1509 | + <target dev='${eph.device}' bus='${disk_bus}'/> |
1510 | + </disk> |
1511 | + #end for |
1512 | + #if $getVar('swap_device', False) |
1513 | + <disk type='file'> |
1514 | + <driver type='${driver_type}'/> |
1515 | + <source file='${basepath}/disk.swap'/> |
1516 | + <target dev='${swap_device}' bus='${disk_bus}'/> |
1517 | + </disk> |
1518 | + #end if |
1519 | #for $vol in $volumes |
1520 | <disk type='${vol.type}'> |
1521 | <driver type='raw'/> |
1522 | |
1523 | === modified file 'nova/virt/libvirt/connection.py' |
1524 | --- nova/virt/libvirt/connection.py 2011-08-05 02:22:13 +0000 |
1525 | +++ nova/virt/libvirt/connection.py 2011-08-05 06:31:22 +0000 |
1526 | @@ -54,6 +54,7 @@ |
1527 | from eventlet import greenthread |
1528 | from eventlet import tpool |
1529 | |
1530 | +from nova import block_device |
1531 | from nova import context as nova_context |
1532 | from nova import db |
1533 | from nova import exception |
1534 | @@ -151,8 +152,8 @@ |
1535 | Template = t.Template |
1536 | |
1537 | |
1538 | -def _strip_dev(mount_path): |
1539 | - return re.sub(r'^/dev/', '', mount_path) |
1540 | +def _get_eph_disk(ephemeral): |
1541 | + return 'disk.eph' + str(ephemeral['num']) |
1542 | |
1543 | |
1544 | class LibvirtConnection(driver.ComputeDriver): |
1545 | @@ -570,15 +571,14 @@ |
1546 | # NOTE(ilyaalekseyev): Implementation like in multinics |
1547 | # for xenapi(tr3buchet) |
1548 | @exception.wrap_exception() |
1549 | - def spawn(self, context, instance, network_info, |
1550 | - block_device_mapping=None): |
1551 | + def spawn(self, context, instance, |
1552 | + network_info=None, block_device_info=None): |
1553 | xml = self.to_xml(instance, False, network_info=network_info, |
1554 | - block_device_mapping=block_device_mapping) |
1555 | - block_device_mapping = block_device_mapping or [] |
1556 | + block_device_info=block_device_info) |
1557 | self.firewall_driver.setup_basic_filtering(instance, network_info) |
1558 | self.firewall_driver.prepare_instance_filter(instance, network_info) |
1559 | self._create_image(context, instance, xml, network_info=network_info, |
1560 | - block_device_mapping=block_device_mapping) |
1561 | + block_device_info=block_device_info) |
1562 | domain = self._create_new_domain(xml) |
1563 | LOG.debug(_("instance %s: is running"), instance['name']) |
1564 | self.firewall_driver.apply_instance_filter(instance) |
1565 | @@ -755,11 +755,14 @@ |
1566 | utils.execute('truncate', target, '-s', "%dG" % local_gb) |
1567 | # TODO(vish): should we format disk by default? |
1568 | |
1569 | + def _create_swap(self, target, swap_gb): |
1570 | + """Create a swap file of specified size""" |
1571 | + self._create_local(target, swap_gb) |
1572 | + utils.execute('mkswap', target) |
1573 | + |
1574 | def _create_image(self, context, inst, libvirt_xml, suffix='', |
1575 | disk_images=None, network_info=None, |
1576 | - block_device_mapping=None): |
1577 | - block_device_mapping = block_device_mapping or [] |
1578 | - |
1579 | + block_device_info=None): |
1580 | if not suffix: |
1581 | suffix = '' |
1582 | |
1583 | @@ -818,8 +821,8 @@ |
1584 | size = None |
1585 | root_fname += "_sm" |
1586 | |
1587 | - if not self._volume_in_mapping(self.root_mount_device, |
1588 | - block_device_mapping): |
1589 | + if not self._volume_in_mapping(self.default_root_device, |
1590 | + block_device_info): |
1591 | self._cache_image(fn=self._fetch_image, |
1592 | context=context, |
1593 | target=basepath('disk'), |
1594 | @@ -830,13 +833,38 @@ |
1595 | project_id=inst['project_id'], |
1596 | size=size) |
1597 | |
1598 | - if inst_type['local_gb'] and not self._volume_in_mapping( |
1599 | - self.local_mount_device, block_device_mapping): |
1600 | + local_gb = inst['local_gb'] |
1601 | + if local_gb and not self._volume_in_mapping( |
1602 | + self.default_local_device, block_device_info): |
1603 | self._cache_image(fn=self._create_local, |
1604 | target=basepath('disk.local'), |
1605 | - fname="local_%s" % inst_type['local_gb'], |
1606 | - cow=FLAGS.use_cow_images, |
1607 | - local_gb=inst_type['local_gb']) |
1608 | + fname="local_%s" % local_gb, |
1609 | + cow=FLAGS.use_cow_images, |
1610 | + local_gb=local_gb) |
1611 | + |
1612 | + for eph in driver.block_device_info_get_ephemerals(block_device_info): |
1613 | + self._cache_image(fn=self._create_local, |
1614 | + target=basepath(_get_eph_disk(eph)), |
1615 | + fname="local_%s" % eph['size'], |
1616 | + cow=FLAGS.use_cow_images, |
1617 | + local_gb=eph['size']) |
1618 | + |
1619 | + swap_gb = 0 |
1620 | + |
1621 | + swap = driver.block_device_info_get_swap(block_device_info) |
1622 | + if driver.swap_is_usable(swap): |
1623 | + swap_gb = swap['swap_size'] |
1624 | + elif (inst_type['swap'] > 0 and |
1625 | + not self._volume_in_mapping(self.default_swap_device, |
1626 | + block_device_info)): |
1627 | + swap_gb = inst_type['swap'] |
1628 | + |
1629 | + if swap_gb > 0: |
1630 | + self._cache_image(fn=self._create_swap, |
1631 | + target=basepath('disk.swap'), |
1632 | + fname="swap_%s" % swap_gb, |
1633 | + cow=FLAGS.use_cow_images, |
1634 | + swap_gb=swap_gb) |
1635 | |
1636 | # For now, we assume that if we're not using a kernel, we're using a |
1637 | # partitioned disk image where the target partition is the first |
1638 | @@ -917,16 +945,35 @@ |
1639 | if FLAGS.libvirt_type == 'uml': |
1640 | utils.execute('sudo', 'chown', 'root', basepath('disk')) |
1641 | |
1642 | - root_mount_device = 'vda' # FIXME for now. it's hard coded. |
1643 | - local_mount_device = 'vdb' # FIXME for now. it's hard coded. |
1644 | - |
1645 | - def _volume_in_mapping(self, mount_device, block_device_mapping): |
1646 | - mount_device_ = _strip_dev(mount_device) |
1647 | - for vol in block_device_mapping: |
1648 | - vol_mount_device = _strip_dev(vol['mount_device']) |
1649 | - if vol_mount_device == mount_device_: |
1650 | - return True |
1651 | - return False |
1652 | + if FLAGS.libvirt_type == 'uml': |
1653 | + _disk_prefix = 'ubd' |
1654 | + elif FLAGS.libvirt_type == 'xen': |
1655 | + _disk_prefix = 'sd' |
1656 | + elif FLAGS.libvirt_type == 'lxc': |
1657 | + _disk_prefix = '' |
1658 | + else: |
1659 | + _disk_prefix = 'vd' |
1660 | + |
1661 | + default_root_device = _disk_prefix + 'a' |
1662 | + default_local_device = _disk_prefix + 'b' |
1663 | + default_swap_device = _disk_prefix + 'c' |
1664 | + |
1665 | + def _volume_in_mapping(self, mount_device, block_device_info): |
1666 | + block_device_list = [block_device.strip_dev(vol['mount_device']) |
1667 | + for vol in |
1668 | + driver.block_device_info_get_mapping( |
1669 | + block_device_info)] |
1670 | + swap = driver.block_device_info_get_swap(block_device_info) |
1671 | + if driver.swap_is_usable(swap): |
1672 | + block_device_list.append( |
1673 | + block_device.strip_dev(swap['device_name'])) |
1674 | + block_device_list += [block_device.strip_dev(ephemeral['device_name']) |
1675 | + for ephemeral in |
1676 | + driver.block_device_info_get_ephemerals( |
1677 | + block_device_info)] |
1678 | + |
1679 | + LOG.debug(_("block_device_list %s"), block_device_list) |
1680 | + return block_device.strip_dev(mount_device) in block_device_list |
1681 | |
1682 | def _get_volume_device_info(self, device_path): |
1683 | if device_path.startswith('/dev/'): |
1684 | @@ -938,8 +985,9 @@ |
1685 | raise exception.InvalidDevicePath(path=device_path) |
1686 | |
1687 | def _prepare_xml_info(self, instance, rescue=False, network_info=None, |
1688 | - block_device_mapping=None): |
1689 | - block_device_mapping = block_device_mapping or [] |
1690 | + block_device_info=None): |
1691 | + block_device_mapping = driver.block_device_info_get_mapping( |
1692 | + block_device_info) |
1693 | # TODO(adiantum) remove network_info creation code |
1694 | # when multinics will be completed |
1695 | if not network_info: |
1696 | @@ -958,17 +1006,27 @@ |
1697 | driver_type = 'raw' |
1698 | |
1699 | for vol in block_device_mapping: |
1700 | - vol['mount_device'] = _strip_dev(vol['mount_device']) |
1701 | + vol['mount_device'] = block_device.strip_dev(vol['mount_device']) |
1702 | (vol['type'], vol['protocol'], vol['name']) = \ |
1703 | self._get_volume_device_info(vol['device_path']) |
1704 | |
1705 | - ebs_root = self._volume_in_mapping(self.root_mount_device, |
1706 | - block_device_mapping) |
1707 | - if self._volume_in_mapping(self.local_mount_device, |
1708 | - block_device_mapping): |
1709 | - local_gb = False |
1710 | - else: |
1711 | - local_gb = inst_type['local_gb'] |
1712 | + ebs_root = self._volume_in_mapping(self.default_root_device, |
1713 | + block_device_info) |
1714 | + |
1715 | + local_device = False |
1716 | + if not (self._volume_in_mapping(self.default_local_device, |
1717 | + block_device_info) or |
1718 | + 0 in [eph['num'] for eph in |
1719 | + driver.block_device_info_get_ephemerals( |
1720 | + block_device_info)]): |
1721 | + if instance['local_gb'] > 0: |
1722 | + local_device = self.default_local_device |
1723 | + |
1724 | + ephemerals = [] |
1725 | + for eph in driver.block_device_info_get_ephemerals(block_device_info): |
1726 | + ephemerals.append({'device_path': _get_eph_disk(eph), |
1727 | + 'device': block_device.strip_dev( |
1728 | + eph['device_name'])}) |
1729 | |
1730 | xml_info = {'type': FLAGS.libvirt_type, |
1731 | 'name': instance['name'], |
1732 | @@ -977,12 +1035,35 @@ |
1733 | 'memory_kb': inst_type['memory_mb'] * 1024, |
1734 | 'vcpus': inst_type['vcpus'], |
1735 | 'rescue': rescue, |
1736 | - 'local': local_gb, |
1737 | + 'disk_prefix': self._disk_prefix, |
1738 | 'driver_type': driver_type, |
1739 | 'vif_type': FLAGS.libvirt_vif_type, |
1740 | 'nics': nics, |
1741 | 'ebs_root': ebs_root, |
1742 | - 'volumes': block_device_mapping} |
1743 | + 'local_device': local_device, |
1744 | + 'volumes': block_device_mapping, |
1745 | + 'ephemerals': ephemerals} |
1746 | + |
1747 | + root_device_name = driver.block_device_info_get_root(block_device_info) |
1748 | + if root_device_name: |
1749 | + xml_info['root_device'] = block_device.strip_dev(root_device_name) |
1750 | + xml_info['root_device_name'] = root_device_name |
1751 | + else: |
1752 | + # NOTE(yamahata): |
1753 | + # for nova.api.ec2.cloud.CloudController.get_metadata() |
1754 | + xml_info['root_device'] = self.default_root_device |
1755 | + db.instance_update( |
1756 | + nova_context.get_admin_context(), instance['id'], |
1757 | + {'root_device_name': '/dev/' + self.default_root_device}) |
1758 | + |
1759 | + swap = driver.block_device_info_get_swap(block_device_info) |
1760 | + if driver.swap_is_usable(swap): |
1761 | + xml_info['swap_device'] = block_device.strip_dev( |
1762 | + swap['device_name']) |
1763 | + elif (inst_type['swap'] > 0 and |
1764 | + not self._volume_in_mapping(self.default_swap_device, |
1765 | + block_device_info)): |
1766 | + xml_info['swap_device'] = self.default_swap_device |
1767 | |
1768 | if FLAGS.vnc_enabled and FLAGS.libvirt_type not in ('lxc', 'uml'): |
1769 | xml_info['vncserver_host'] = FLAGS.vncserver_host |
1770 | @@ -998,12 +1079,11 @@ |
1771 | return xml_info |
1772 | |
1773 | def to_xml(self, instance, rescue=False, network_info=None, |
1774 | - block_device_mapping=None): |
1775 | - block_device_mapping = block_device_mapping or [] |
1776 | + block_device_info=None): |
1777 | # TODO(termie): cache? |
1778 | LOG.debug(_('instance %s: starting toXML method'), instance['name']) |
1779 | xml_info = self._prepare_xml_info(instance, rescue, network_info, |
1780 | - block_device_mapping) |
1781 | + block_device_info) |
1782 | xml = str(Template(self.libvirt_xml, searchList=[xml_info])) |
1783 | LOG.debug(_('instance %s: finished toXML method'), instance['name']) |
1784 | return xml |
1785 | |
1786 | === modified file 'nova/virt/xenapi_conn.py' |
1787 | --- nova/virt/xenapi_conn.py 2011-08-05 02:22:13 +0000 |
1788 | +++ nova/virt/xenapi_conn.py 2011-08-05 06:31:22 +0000 |
1789 | @@ -184,8 +184,8 @@ |
1790 | def list_instances_detail(self): |
1791 | return self._vmops.list_instances_detail() |
1792 | |
1793 | - def spawn(self, context, instance, network_info, |
1794 | - block_device_mapping=None): |
1795 | + def spawn(self, context, instance, |
1796 | + network_info=None, block_device_info=None): |
1797 | """Create VM instance""" |
1798 | self._vmops.spawn(context, instance, network_info) |
1799 |
401 +from nova.api.ec2 import ec2utils
475 +from nova.api.ec2 import ec2utils
The database layer and the compute manager must not depend on something from the API layer. There are a couple degrees of separation missing here. This needs to be rethought, in my opinion.