Merge lp:~yamahata/nova/boot-from-volume-2 into lp:~hudson-openstack/nova/trunk

Proposed by Isaku Yamahata
Status: Merged
Approved by: Brian Lamar
Approved revision: 1334
Merged at revision: 1400
Proposed branch: lp:~yamahata/nova/boot-from-volume-2
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 1798 lines (+981/-161)
21 files modified
nova/api/ec2/cloud.py (+153/-27)
nova/api/ec2/ec2utils.py (+0/-29)
nova/block_device.py (+71/-0)
nova/compute/api.py (+56/-17)
nova/compute/manager.py (+26/-8)
nova/db/sqlalchemy/api.py (+18/-1)
nova/image/glance.py (+46/-0)
nova/tests/image/test_glance.py (+36/-0)
nova/tests/test_api.py (+6/-3)
nova/tests/test_block_device.py (+87/-0)
nova/tests/test_cloud.py (+153/-4)
nova/tests/test_compute.py (+28/-12)
nova/tests/test_libvirt.py (+37/-0)
nova/tests/test_metadata.py (+1/-0)
nova/tests/test_virt.py (+83/-0)
nova/virt/driver.py (+29/-2)
nova/virt/fake.py (+2/-2)
nova/virt/hyperv.py (+2/-2)
nova/virt/libvirt.xml.template (+23/-10)
nova/virt/libvirt/connection.py (+122/-42)
nova/virt/xenapi_conn.py (+2/-2)
To merge this branch: bzr merge lp:~yamahata/nova/boot-from-volume-2
Reviewer Review Type Date Requested Status
Brian Lamar (community) Approve
Devin Carlen (community) Approve
Vish Ishaya (community) Approve
Soren Hansen (community) Needs Information
Review via email: mp+68496@code.launchpad.net

Description of the change

With this branch, boot-from-volume can be marked as completed in some sense.
The remaining is minor if any and will be addressed as bug fixes.

With this branch the following is enabled.
- describe instance attribute
- get_metadata for euca-bundle-vol
- root/ephemeral/swap device support

To post a comment you must log in.
Revision history for this message
Brian Lamar (blamar) wrote :

401 +from nova.api.ec2 import ec2utils
475 +from nova.api.ec2 import ec2utils

The database layer and the compute manager must not depend on something from the API layer. There are a couple degrees of separation missing here. This needs to be rethought, in my opinion.

review: Needs Fixing
Revision history for this message
Devin Carlen (devcamcar) wrote :

+1 to Brian's comments. There is insufficient separation of concerns here.

review: Needs Fixing
Revision history for this message
Brian Lamar (blamar) wrote :

Moving to WIP due to 2 "Needs Fixings".

Revision history for this message
Isaku Yamahata (yamahata) wrote :

On Wed, Jul 20, 2011 at 04:48:39PM -0000, Brian Lamar wrote:
> Review: Needs Fixing
> 401 +from nova.api.ec2 import ec2utils
> 475 +from nova.api.ec2 import ec2utils
>
> The database layer and the compute manager must not depend on something from the API layer. There are a couple degrees of separation missing here. This needs to be rethought, in my opinion.

thank you for review. I addressed it and added unit tests.
So the branch is ready for merge.

--
yamahata

Revision history for this message
Brian Lamar (blamar) wrote :

341 +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>

After asking in the last meeting, I believe code should be copyright OpenStack, LLC.

365 + # NOTE(yamahata): see image_service.s3.s3create()
366 + for bdm in properties.get('mappings', []):
367 + if bdm['virtual'] == 'root':
368 + root_device_name = bdm['device']

This logic seems a little off, I'm not certain why you'd loop here unless there could be multiple mappings with 'virtual' == 'root'. Maybe it is a little late to bring this up, and let me know if it is, but is 'virtual' unique? It would seem that having mappings look like:

{
    "ami": {"device": "/dev/sda"},
    "root": {"device": "sda"},
    "ephemeral0": {"device": "sdb"},
}

instead of:

[
    {'virtual': 'ami', 'device': '/dev/sda'},
    {'virtual': 'root', 'device': 'sda'},
    {'virtual': 'ephemeral0', 'device': 'sdb'},
]

might make the logic a bit cleaner in a lot of areas? This might not work but I thought I'd suggest it. For example, the block of code referenced earlier would become:

root_device_name = mappings.get("root", None)

I'm a little confused that mappings sometimes are a list and other times are a dictionary. Which one is correct?

Can you give more information or point me to documentation on what "ephemeral" means in this context? Seemingly "ephemeral" devices won't persist on reboot, or something to that effect?

review: Needs Fixing
Revision history for this message
Vish Ishaya (vishvananda) wrote :

On Jul 25, 2011, at 10:41 AM, Brian Lamar wrote:

> Review: Needs Fixing
> 341 +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>
>
> After asking in the last meeting, I believe code should be copyright OpenStack, LLC.

I'm not sure what the confusion is here, but we DO NOT require copyright assignment. People are free to add their own copyrights to code that they right. The CLA is what allows us to license the the code through the apache license.

Vish

Revision history for this message
Vish Ishaya (vishvananda) wrote :

s/right/write

Revision history for this message
Brian Lamar (blamar) wrote :

Thanks Vish, I spoke up at the last meeting but I guess it was missed or I mis-asked the question. Good to have clarification and I'll stop asking for this.

>
> On Jul 25, 2011, at 10:41 AM, Brian Lamar wrote:
>
> > Review: Needs Fixing
> > 341 +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>
> >
> > After asking in the last meeting, I believe code should be copyright
> OpenStack, LLC.
>
> I'm not sure what the confusion is here, but we DO NOT require copyright
> assignment. People are free to add their own copyrights to code that they
> right. The CLA is what allows us to license the the code through the apache
> license.
>
> Vish

Revision history for this message
Vish Ishaya (vishvananda) wrote :

> Can you give more information or point me to documentation on what "ephemeral"
> means in this context? Seemingly "ephemeral" devices won't persist on reboot,
> or something to that effect?

Ephemeral is the aws name for local storage that the vm gets that isn't part of the image. In the libvirt driver we attach a second unformatted drive as the "ephemeral" space.

Revision history for this message
Isaku Yamahata (yamahata) wrote :

On Mon, Jul 25, 2011 at 05:41:30PM -0000, Brian Lamar wrote:
> Review: Needs Fixing
> 341 +# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>
>
> After asking in the last meeting, I believe code should be copyright OpenStack, LLC.
>
> 365 + # NOTE(yamahata): see image_service.s3.s3create()
> 366 + for bdm in properties.get('mappings', []):
> 367 + if bdm['virtual'] == 'root':
> 368 + root_device_name = bdm['device']
>
> This logic seems a little off, I'm not certain why you'd loop here unless there could be multiple mappings with 'virtual' == 'root'. Maybe it is a little late to bring this up, and let me know if it is, but is 'virtual' unique? It would seem that having mappings look like:

'virtual' is unique.

> {
> "ami": {"device": "/dev/sda"},
> "root": {"device": "sda"},
> "ephemeral0": {"device": "sdb"},
> }
>
> instead of:
>
> [
> {'virtual': 'ami', 'device': '/dev/sda'},
> {'virtual': 'root', 'device': 'sda'},
> {'virtual': 'ephemeral0', 'device': 'sdb'},
> ]
>
> might make the logic a bit cleaner in a lot of areas? This might not work but I thought I'd suggest it. For example, the block of code referenced earlier would become:
>
> root_device_name = mappings.get("root", None)
>
> I'm a little confused that mappings sometimes are a list and other times are a dictionary. Which one is correct?

A list as the internal representation. It's because they correspond to RDB rows.
Thus internally they are handled as lists consistently.(At least my intension is so)
The conversion from/to lists to/from dicts occurs when parsing API requests/
formatting API results if necessary.

Does it clarify?

Hmm, nova.api.ec2.cloud.CloudController._get_instance_mapping()
should be _format_instance_mapping(). I'll fix it.

> Can you give more information or point me to documentation on what "ephemeral" means in this context? Seemingly "ephemeral" devices won't persist on reboot, or something to that effect?

Vish kindly answered this.

--
yamahata

lp:~yamahata/nova/boot-from-volume-2 updated
1331. By Isaku Yamahata

api/ec2: rename CloudController._get_instance_mapping into _format_instance_mapping

This patch renames nova.api.ec2.cloud.CouldController._get_instance_mapping
to _format_instance_mapping in order to make it clear that the method is
for API formatting, not for internal use.

Revision history for this message
Isaku Yamahata (yamahata) wrote :

> Hmm, nova.api.ec2.cloud.CloudController._get_instance_mapping()
> should be _format_instance_mapping(). I'll fix it.

Done.

lp:~yamahata/nova/boot-from-volume-2 updated
1332. By Isaku Yamahata

merged with nova trunk

Revision history for this message
Vish Ishaya (vishvananda) wrote :

This is all looking very good. Just a question. Will this cause problems for the other hypervisors? Do we need to add support in xen and vmware to create swap partitions based on block_device_mapping?

review: Needs Information
Revision history for this message
Isaku Yamahata (yamahata) wrote :

On Tue, Aug 02, 2011 at 10:49:23PM -0000, Vish Ishaya wrote:
> Review: Needs Information
> This is all looking very good. Just a question. Will this cause problems for the other hypervisors?

No (except bugs). As long as the newly introduced features aren't used,
they should work as before.
Even if such features are used, e.g. block device mapping is specified,
they will be simply ignored.

> Do we need to add support in xen and vmware to create swap partitions based on block_device_mapping?

Although I'm not familiar with xen/vmware related code, I'm quite happy to help
those who implement it.

Does any user want it? Is there anyone who will implement it?
We will see the answers soon or after Diablo release.
--
yamahata

Revision history for this message
Soren Hansen (soren) wrote :

Can you explain why you pulled the disk_prefix variable into python code instead of leaving it in the template. It was intentinoally put in the template so that if someone wanted to e.g. not use virtio disks with kvm, they didn't have to patch any code, they could just make a new template, setting the disk_prefix to something else, and point nova at the new template. That seems more awkward now.

review: Needs Information
Revision history for this message
Isaku Yamahata (yamahata) wrote :

On Thu, Aug 04, 2011 at 11:11:26AM -0000, Soren Hansen wrote:
> Review: Needs Information
> Can you explain why you pulled the disk_prefix variable into python code instead of leaving it in the template. It was intentinoally put in the template so that if someone wanted to e.g. not use virtio disks with kvm, they didn't have to patch any code, they could just make a new template, setting the disk_prefix to something else, and point nova at the new template. That seems more awkward now.

It's because ec2 api exposes disk prefix to the users.
For exmample,
euca-run-instances ...
  -b /dev/vda=nodevice
  -b /dev/sdb=snap-00000003
  -b /dev/hdc=ephemeral0

Thus the user can override the predefined devices (e.g. disk prefix + 'a')
or add new disks.
(Using inconsistent disk prefix would be insane. But this shows
the point well, I think. This is just for the explanation)

So the code needs to check if the device specified by the user
overrides the predefined device or not.
The cheetah template syntax isn't expressive enough to check it.

If disk prefix is so important, how about introducing another flag
like libvirt_disk_prefix?
--
yamahata

lp:~yamahata/nova/boot-from-volume-2 updated
1333. By Isaku Yamahata

merged with nova trunk.

1334. By Isaku Yamahata

fix mismerge

Revision history for this message
Vish Ishaya (vishvananda) wrote :

My concerns were addressed

review: Approve
Revision history for this message
Devin Carlen (devcamcar) wrote :

mine as well!

review: Approve
Revision history for this message
Brian Lamar (blamar) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'nova/api/ec2/cloud.py'
2--- nova/api/ec2/cloud.py 2011-07-26 20:58:02 +0000
3+++ nova/api/ec2/cloud.py 2011-08-05 06:31:22 +0000
4@@ -30,6 +30,7 @@
5 import time
6 import shutil
7
8+from nova import block_device
9 from nova import compute
10 from nova import context
11
12@@ -78,6 +79,10 @@
13
14 # TODO(yamahata): hypervisor dependent default device name
15 _DEFAULT_ROOT_DEVICE_NAME = '/dev/sda1'
16+_DEFAULT_MAPPINGS = {'ami': 'sda1',
17+ 'ephemeral0': 'sda2',
18+ 'root': _DEFAULT_ROOT_DEVICE_NAME,
19+ 'swap': 'sda3'}
20
21
22 def _parse_block_device_mapping(bdm):
23@@ -105,7 +110,7 @@
24
25
26 def _properties_get_mappings(properties):
27- return ec2utils.mappings_prepend_dev(properties.get('mappings', []))
28+ return block_device.mappings_prepend_dev(properties.get('mappings', []))
29
30
31 def _format_block_device_mapping(bdm):
32@@ -144,8 +149,7 @@
33 """Format multiple BlockDeviceMappingItemType"""
34 mappings = [{'virtualName': m['virtual'], 'deviceName': m['device']}
35 for m in _properties_get_mappings(properties)
36- if (m['virtual'] == 'swap' or
37- m['virtual'].startswith('ephemeral'))]
38+ if block_device.is_swap_or_ephemeral(m['virtual'])]
39
40 block_device_mapping = [_format_block_device_mapping(bdm) for bdm in
41 properties.get('block_device_mapping', [])]
42@@ -233,6 +237,30 @@
43 state = 'available'
44 return image['properties'].get('image_state', state)
45
46+ def _format_instance_mapping(self, ctxt, instance_ref):
47+ root_device_name = instance_ref['root_device_name']
48+ if root_device_name is None:
49+ return _DEFAULT_MAPPINGS
50+
51+ mappings = {}
52+ mappings['ami'] = block_device.strip_dev(root_device_name)
53+ mappings['root'] = root_device_name
54+
55+ # 'ephemeralN' and 'swap'
56+ for bdm in db.block_device_mapping_get_all_by_instance(
57+ ctxt, instance_ref['id']):
58+ if (bdm['volume_id'] or bdm['snapshot_id'] or bdm['no_device']):
59+ continue
60+
61+ virtual_name = bdm['virtual_name']
62+ if not virtual_name:
63+ continue
64+
65+ if block_device.is_swap_or_ephemeral(virtual_name):
66+ mappings[virtual_name] = bdm['device_name']
67+
68+ return mappings
69+
70 def get_metadata(self, address):
71 ctxt = context.get_admin_context()
72 instance_ref = self.compute_api.get_all(ctxt, fixed_ip=address)
73@@ -259,18 +287,14 @@
74 security_groups = db.security_group_get_by_instance(ctxt,
75 instance_ref['id'])
76 security_groups = [x['name'] for x in security_groups]
77+ mappings = self._format_instance_mapping(ctxt, instance_ref)
78 data = {
79- 'user-data': base64.b64decode(instance_ref['user_data']),
80+ 'user-data': self._format_user_data(instance_ref),
81 'meta-data': {
82 'ami-id': image_ec2_id,
83 'ami-launch-index': instance_ref['launch_index'],
84 'ami-manifest-path': 'FIXME',
85- 'block-device-mapping': {
86- # TODO(vish): replace with real data
87- 'ami': 'sda1',
88- 'ephemeral0': 'sda2',
89- 'root': _DEFAULT_ROOT_DEVICE_NAME,
90- 'swap': 'sda3'},
91+ 'block-device-mapping': mappings,
92 'hostname': hostname,
93 'instance-action': 'none',
94 'instance-id': ec2_id,
95@@ -948,13 +972,102 @@
96 'status': volume['attach_status'],
97 'volumeId': ec2utils.id_to_ec2_vol_id(volume_id)}
98
99- def _convert_to_set(self, lst, label):
100+ @staticmethod
101+ def _convert_to_set(lst, label):
102 if lst is None or lst == []:
103 return None
104 if not isinstance(lst, list):
105 lst = [lst]
106 return [{label: x} for x in lst]
107
108+ def _format_kernel_id(self, instance_ref, result, key):
109+ kernel_id = instance_ref['kernel_id']
110+ if kernel_id is None:
111+ return
112+ result[key] = self.image_ec2_id(instance_ref['kernel_id'], 'aki')
113+
114+ def _format_ramdisk_id(self, instance_ref, result, key):
115+ ramdisk_id = instance_ref['ramdisk_id']
116+ if ramdisk_id is None:
117+ return
118+ result[key] = self.image_ec2_id(instance_ref['ramdisk_id'], 'ari')
119+
120+ @staticmethod
121+ def _format_user_data(instance_ref):
122+ return base64.b64decode(instance_ref['user_data'])
123+
124+ def describe_instance_attribute(self, context, instance_id, attribute,
125+ **kwargs):
126+ def _unsupported_attribute(instance, result):
127+ raise exception.ApiError(_('attribute not supported: %s') %
128+ attribute)
129+
130+ def _format_attr_block_device_mapping(instance, result):
131+ tmp = {}
132+ self._format_instance_root_device_name(instance, tmp)
133+ self._format_instance_bdm(context, instance_id,
134+ tmp['rootDeviceName'], result)
135+
136+ def _format_attr_disable_api_termination(instance, result):
137+ _unsupported_attribute(instance, result)
138+
139+ def _format_attr_group_set(instance, result):
140+ CloudController._format_group_set(instance, result)
141+
142+ def _format_attr_instance_initiated_shutdown_behavior(instance,
143+ result):
144+ state_description = instance['state_description']
145+ state_to_value = {'stopping': 'stop',
146+ 'stopped': 'stop',
147+ 'terminating': 'terminate'}
148+ value = state_to_value.get(state_description)
149+ if value:
150+ result['instanceInitiatedShutdownBehavior'] = value
151+
152+ def _format_attr_instance_type(instance, result):
153+ self._format_instance_type(instance, result)
154+
155+ def _format_attr_kernel(instance, result):
156+ self._format_kernel_id(instance, result, 'kernel')
157+
158+ def _format_attr_ramdisk(instance, result):
159+ self._format_ramdisk_id(instance, result, 'ramdisk')
160+
161+ def _format_attr_root_device_name(instance, result):
162+ self._format_instance_root_device_name(instance, result)
163+
164+ def _format_attr_source_dest_check(instance, result):
165+ _unsupported_attribute(instance, result)
166+
167+ def _format_attr_user_data(instance, result):
168+ result['userData'] = self._format_user_data(instance)
169+
170+ attribute_formatter = {
171+ 'blockDeviceMapping': _format_attr_block_device_mapping,
172+ 'disableApiTermination': _format_attr_disable_api_termination,
173+ 'groupSet': _format_attr_group_set,
174+ 'instanceInitiatedShutdownBehavior':
175+ _format_attr_instance_initiated_shutdown_behavior,
176+ 'instanceType': _format_attr_instance_type,
177+ 'kernel': _format_attr_kernel,
178+ 'ramdisk': _format_attr_ramdisk,
179+ 'rootDeviceName': _format_attr_root_device_name,
180+ 'sourceDestCheck': _format_attr_source_dest_check,
181+ 'userData': _format_attr_user_data,
182+ }
183+
184+ fn = attribute_formatter.get(attribute)
185+ if fn is None:
186+ raise exception.ApiError(
187+ _('attribute not supported: %s') % attribute)
188+
189+ ec2_instance_id = instance_id
190+ instance_id = ec2utils.ec2_id_to_id(ec2_instance_id)
191+ instance = self.compute_api.get(context, instance_id)
192+ result = {'instance_id': ec2_instance_id}
193+ fn(instance, result)
194+ return result
195+
196 def describe_instances(self, context, **kwargs):
197 return self._format_describe_instances(context, **kwargs)
198
199@@ -1001,6 +1114,27 @@
200 result['blockDeviceMapping'] = mapping
201 result['rootDeviceType'] = root_device_type
202
203+ @staticmethod
204+ def _format_instance_root_device_name(instance, result):
205+ result['rootDeviceName'] = (instance.get('root_device_name') or
206+ _DEFAULT_ROOT_DEVICE_NAME)
207+
208+ @staticmethod
209+ def _format_instance_type(instance, result):
210+ if instance['instance_type']:
211+ result['instanceType'] = instance['instance_type'].get('name')
212+ else:
213+ result['instanceType'] = None
214+
215+ @staticmethod
216+ def _format_group_set(instance, result):
217+ security_group_names = []
218+ if instance.get('security_groups'):
219+ for security_group in instance['security_groups']:
220+ security_group_names.append(security_group['name'])
221+ result['groupSet'] = CloudController._convert_to_set(
222+ security_group_names, 'groupId')
223+
224 def _format_instances(self, context, instance_id=None, **kwargs):
225 # TODO(termie): this method is poorly named as its name does not imply
226 # that it will be making a variety of database calls
227@@ -1026,6 +1160,8 @@
228 ec2_id = ec2utils.id_to_ec2_id(instance_id)
229 i['instanceId'] = ec2_id
230 i['imageId'] = self.image_ec2_id(instance['image_ref'])
231+ self._format_kernel_id(instance, i, 'kernelId')
232+ self._format_ramdisk_id(instance, i, 'ramdiskId')
233 i['instanceState'] = {
234 'code': instance['state'],
235 'name': instance['state_description']}
236@@ -1054,16 +1190,12 @@
237 instance['project_id'],
238 instance['host'])
239 i['productCodesSet'] = self._convert_to_set([], 'product_codes')
240- if instance['instance_type']:
241- i['instanceType'] = instance['instance_type'].get('name')
242- else:
243- i['instanceType'] = None
244+ self._format_instance_type(instance, i)
245 i['launchTime'] = instance['created_at']
246 i['amiLaunchIndex'] = instance['launch_index']
247 i['displayName'] = instance['display_name']
248 i['displayDescription'] = instance['display_description']
249- i['rootDeviceName'] = (instance.get('root_device_name') or
250- _DEFAULT_ROOT_DEVICE_NAME)
251+ self._format_instance_root_device_name(instance, i)
252 self._format_instance_bdm(context, instance_id,
253 i['rootDeviceName'], i)
254 host = instance['host']
255@@ -1073,12 +1205,7 @@
256 r = {}
257 r['reservationId'] = instance['reservation_id']
258 r['ownerId'] = instance['project_id']
259- security_group_names = []
260- if instance.get('security_groups'):
261- for security_group in instance['security_groups']:
262- security_group_names.append(security_group['name'])
263- r['groupSet'] = self._convert_to_set(security_group_names,
264- 'groupId')
265+ self._format_group_set(instance, r)
266 r['instancesSet'] = []
267 reservations[instance['reservation_id']] = r
268 reservations[instance['reservation_id']]['instancesSet'].append(i)
269@@ -1314,7 +1441,7 @@
270 i['architecture'] = image['properties'].get('architecture')
271
272 properties = image['properties']
273- root_device_name = ec2utils.properties_root_device_name(properties)
274+ root_device_name = block_device.properties_root_device_name(properties)
275 root_device_type = 'instance-store'
276 for bdm in properties.get('block_device_mapping', []):
277 if (bdm.get('device_name') == root_device_name and
278@@ -1387,7 +1514,7 @@
279
280 def _root_device_name_attribute(image, result):
281 result['rootDeviceName'] = \
282- ec2utils.properties_root_device_name(image['properties'])
283+ block_device.properties_root_device_name(image['properties'])
284 if result['rootDeviceName'] is None:
285 result['rootDeviceName'] = _DEFAULT_ROOT_DEVICE_NAME
286
287@@ -1520,8 +1647,7 @@
288 if virtual_name in ('ami', 'root'):
289 continue
290
291- assert (virtual_name == 'swap' or
292- virtual_name.startswith('ephemeral'))
293+ assert block_device.is_swap_or_ephemeral(virtual_name)
294 device_name = m['device']
295 if device_name in [b['device_name'] for b in mapping
296 if not b.get('no_device', False)]:
297
298=== modified file 'nova/api/ec2/ec2utils.py'
299--- nova/api/ec2/ec2utils.py 2011-06-27 11:20:42 +0000
300+++ nova/api/ec2/ec2utils.py 2011-08-05 06:31:22 +0000
301@@ -135,32 +135,3 @@
302 args[key] = value
303
304 return args
305-
306-
307-def properties_root_device_name(properties):
308- """get root device name from image meta data.
309- If it isn't specified, return None.
310- """
311- root_device_name = None
312-
313- # NOTE(yamahata): see image_service.s3.s3create()
314- for bdm in properties.get('mappings', []):
315- if bdm['virtual'] == 'root':
316- root_device_name = bdm['device']
317-
318- # NOTE(yamahata): register_image's command line can override
319- # <machine>.manifest.xml
320- if 'root_device_name' in properties:
321- root_device_name = properties['root_device_name']
322-
323- return root_device_name
324-
325-
326-def mappings_prepend_dev(mappings):
327- """Prepend '/dev/' to 'device' entry of swap/ephemeral virtual type"""
328- for m in mappings:
329- virtual = m['virtual']
330- if ((virtual == 'swap' or virtual.startswith('ephemeral')) and
331- (not m['device'].startswith('/'))):
332- m['device'] = '/dev/' + m['device']
333- return mappings
334
335=== added file 'nova/block_device.py'
336--- nova/block_device.py 1970-01-01 00:00:00 +0000
337+++ nova/block_device.py 2011-08-05 06:31:22 +0000
338@@ -0,0 +1,71 @@
339+# vim: tabstop=4 shiftwidth=4 softtabstop=4
340+
341+# Copyright 2011 Isaku Yamahata <yamahata@valinux co jp>
342+# All Rights Reserved.
343+#
344+# Licensed under the Apache License, Version 2.0 (the "License"); you may
345+# not use this file except in compliance with the License. You may obtain
346+# a copy of the License at
347+#
348+# http://www.apache.org/licenses/LICENSE-2.0
349+#
350+# Unless required by applicable law or agreed to in writing, software
351+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
352+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
353+# License for the specific language governing permissions and limitations
354+# under the License.
355+
356+import re
357+
358+
359+def properties_root_device_name(properties):
360+ """get root device name from image meta data.
361+ If it isn't specified, return None.
362+ """
363+ root_device_name = None
364+
365+ # NOTE(yamahata): see image_service.s3.s3create()
366+ for bdm in properties.get('mappings', []):
367+ if bdm['virtual'] == 'root':
368+ root_device_name = bdm['device']
369+
370+ # NOTE(yamahata): register_image's command line can override
371+ # <machine>.manifest.xml
372+ if 'root_device_name' in properties:
373+ root_device_name = properties['root_device_name']
374+
375+ return root_device_name
376+
377+
378+_ephemeral = re.compile('^ephemeral(\d|[1-9]\d+)$')
379+
380+
381+def is_ephemeral(device_name):
382+ return _ephemeral.match(device_name)
383+
384+
385+def ephemeral_num(ephemeral_name):
386+ assert is_ephemeral(ephemeral_name)
387+ return int(_ephemeral.sub('\\1', ephemeral_name))
388+
389+
390+def is_swap_or_ephemeral(device_name):
391+ return device_name == 'swap' or is_ephemeral(device_name)
392+
393+
394+def mappings_prepend_dev(mappings):
395+ """Prepend '/dev/' to 'device' entry of swap/ephemeral virtual type"""
396+ for m in mappings:
397+ virtual = m['virtual']
398+ if (is_swap_or_ephemeral(virtual) and
399+ (not m['device'].startswith('/'))):
400+ m['device'] = '/dev/' + m['device']
401+ return mappings
402+
403+
404+_dev = re.compile('^/dev/')
405+
406+
407+def strip_dev(device_name):
408+ """remove leading '/dev/'"""
409+ return _dev.sub('', device_name)
410
411=== modified file 'nova/compute/api.py'
412--- nova/compute/api.py 2011-08-04 16:26:14 +0000
413+++ nova/compute/api.py 2011-08-05 06:31:22 +0000
414@@ -22,6 +22,7 @@
415 import re
416 import time
417
418+from nova import block_device
419 from nova import db
420 from nova import exception
421 from nova import flags
422@@ -32,7 +33,6 @@
423 from nova import rpc
424 from nova import utils
425 from nova import volume
426-from nova.api.ec2 import ec2utils
427 from nova.compute import instance_types
428 from nova.compute import power_state
429 from nova.compute.utils import terminate_volumes
430@@ -218,7 +218,7 @@
431 if reservation_id is None:
432 reservation_id = utils.generate_uid('r')
433
434- root_device_name = ec2utils.properties_root_device_name(
435+ root_device_name = block_device.properties_root_device_name(
436 image['properties'])
437
438 base_options = {
439@@ -250,34 +250,64 @@
440
441 return (num_instances, base_options, image)
442
443- def _update_image_block_device_mapping(self, elevated_context, instance_id,
444+ @staticmethod
445+ def _ephemeral_size(instance_type, ephemeral_name):
446+ num = block_device.ephemeral_num(ephemeral_name)
447+
448+ # TODO(yamahata): ephemeralN where N > 0
449+ # Only ephemeral0 is allowed for now because InstanceTypes
450+ # table only allows single local disk, local_gb.
451+ # In order to enhance it, we need to add a new columns to
452+ # instance_types table.
453+ if num > 0:
454+ return 0
455+
456+ return instance_type.get('local_gb')
457+
458+ def _update_image_block_device_mapping(self, elevated_context,
459+ instance_type, instance_id,
460 mappings):
461 """tell vm driver to create ephemeral/swap device at boot time by
462 updating BlockDeviceMapping
463 """
464- for bdm in ec2utils.mappings_prepend_dev(mappings):
465+ instance_type = (instance_type or
466+ instance_types.get_default_instance_type())
467+
468+ for bdm in block_device.mappings_prepend_dev(mappings):
469 LOG.debug(_("bdm %s"), bdm)
470
471 virtual_name = bdm['virtual']
472 if virtual_name == 'ami' or virtual_name == 'root':
473 continue
474
475- assert (virtual_name == 'swap' or
476- virtual_name.startswith('ephemeral'))
477+ if not block_device.is_swap_or_ephemeral(virtual_name):
478+ continue
479+
480+ size = 0
481+ if virtual_name == 'swap':
482+ size = instance_type.get('swap', 0)
483+ elif block_device.is_ephemeral(virtual_name):
484+ size = self._ephemeral_size(instance_type, virtual_name)
485+
486+ if size == 0:
487+ continue
488+
489 values = {
490 'instance_id': instance_id,
491 'device_name': bdm['device'],
492- 'virtual_name': virtual_name, }
493+ 'virtual_name': virtual_name,
494+ 'volume_size': size}
495 self.db.block_device_mapping_update_or_create(elevated_context,
496 values)
497
498- def _update_block_device_mapping(self, elevated_context, instance_id,
499+ def _update_block_device_mapping(self, elevated_context,
500+ instance_type, instance_id,
501 block_device_mapping):
502 """tell vm driver to attach volume at boot time by updating
503 BlockDeviceMapping
504 """
505+ LOG.debug(_("block_device_mapping %s"), block_device_mapping)
506 for bdm in block_device_mapping:
507- LOG.debug(_('bdm %s'), bdm)
508 assert 'device_name' in bdm
509
510 values = {'instance_id': instance_id}
511@@ -286,10 +316,18 @@
512 'no_device'):
513 values[key] = bdm.get(key)
514
515+ virtual_name = bdm.get('virtual_name')
516+ if (virtual_name is not None and
517+ block_device.is_ephemeral(virtual_name)):
518+ size = self._ephemeral_size(instance_type, virtual_name)
519+ if size == 0:
520+ continue
521+ values['volume_size'] = size
522+
523 # NOTE(yamahata): NoDevice eliminates devices defined in image
524 # files by command line option.
525 # (--block-device-mapping)
526- if bdm.get('virtual_name') == 'NoDevice':
527+ if virtual_name == 'NoDevice':
528 values['no_device'] = True
529 for k in ('delete_on_termination', 'volume_id',
530 'snapshot_id', 'volume_id', 'volume_size',
531@@ -299,8 +337,8 @@
532 self.db.block_device_mapping_update_or_create(elevated_context,
533 values)
534
535- def create_db_entry_for_new_instance(self, context, image, base_options,
536- security_group, block_device_mapping, num=1):
537+ def create_db_entry_for_new_instance(self, context, instance_type, image,
538+ base_options, security_group, block_device_mapping, num=1):
539 """Create an entry in the DB for this new instance,
540 including any related table updates (such as security group,
541 etc).
542@@ -333,12 +371,12 @@
543 security_group_id)
544
545 # BlockDeviceMapping table
546- self._update_image_block_device_mapping(elevated, instance_id,
547- image['properties'].get('mappings', []))
548- self._update_block_device_mapping(elevated, instance_id,
549+ self._update_image_block_device_mapping(elevated, instance_type,
550+ instance_id, image['properties'].get('mappings', []))
551+ self._update_block_device_mapping(elevated, instance_type, instance_id,
552 image['properties'].get('block_device_mapping', []))
553 # override via command line option
554- self._update_block_device_mapping(elevated, instance_id,
555+ self._update_block_device_mapping(elevated, instance_type, instance_id,
556 block_device_mapping)
557
558 # Set sane defaults if not specified
559@@ -453,7 +491,8 @@
560 instances = []
561 LOG.debug(_("Going to run %s instances..."), num_instances)
562 for num in range(num_instances):
563- instance = self.create_db_entry_for_new_instance(context, image,
564+ instance = self.create_db_entry_for_new_instance(context,
565+ instance_type, image,
566 base_options, security_group,
567 block_device_mapping, num=num)
568 instances.append(instance)
569
570=== modified file 'nova/compute/manager.py'
571--- nova/compute/manager.py 2011-08-04 21:32:56 +0000
572+++ nova/compute/manager.py 2011-08-05 06:31:22 +0000
573@@ -45,6 +45,7 @@
574 from eventlet import greenthread
575
576 import nova.context
577+from nova import block_device
578 from nova import exception
579 from nova import flags
580 import nova.image
581@@ -260,6 +261,8 @@
582
583 volume_api = volume.API()
584 block_device_mapping = []
585+ swap = None
586+ ephemerals = []
587 for bdm in self.db.block_device_mapping_get_all_by_instance(
588 context, instance_id):
589 LOG.debug(_("setting up bdm %s"), bdm)
590@@ -267,11 +270,18 @@
591 if bdm['no_device']:
592 continue
593 if bdm['virtual_name']:
594- # TODO(yamahata):
595- # block devices for swap and ephemeralN will be
596- # created by virt driver locally in compute node.
597- assert (bdm['virtual_name'] == 'swap' or
598- bdm['virtual_name'].startswith('ephemeral'))
599+ virtual_name = bdm['virtual_name']
600+ device_name = bdm['device_name']
601+ assert block_device.is_swap_or_ephemeral(virtual_name)
602+ if virtual_name == 'swap':
603+ swap = {'device_name': device_name,
604+ 'swap_size': bdm['volume_size']}
605+ elif block_device.is_ephemeral(virtual_name):
606+ eph = {'num': block_device.ephemeral_num(virtual_name),
607+ 'virtual_name': virtual_name,
608+ 'device_name': device_name,
609+ 'size': bdm['volume_size']}
610+ ephemerals.append(eph)
611 continue
612
613 if ((bdm['snapshot_id'] is not None) and
614@@ -307,7 +317,7 @@
615 'mount_device':
616 bdm['device_name']})
617
618- return block_device_mapping
619+ return (swap, ephemerals, block_device_mapping)
620
621 def _run_instance(self, context, instance_id, **kwargs):
622 """Launch a new instance with specified options."""
623@@ -348,13 +358,21 @@
624 # all vif creation and network injection, maybe this is correct
625 network_info = []
626
627- bd_mapping = self._setup_block_device_mapping(context, instance_id)
628+ (swap, ephemerals,
629+ block_device_mapping) = self._setup_block_device_mapping(
630+ context, instance_id)
631+ block_device_info = {
632+ 'root_device_name': instance['root_device_name'],
633+ 'swap': swap,
634+ 'ephemerals': ephemerals,
635+ 'block_device_mapping': block_device_mapping}
636
637 # TODO(vish) check to make sure the availability zone matches
638 self._update_state(context, instance_id, power_state.BUILDING)
639
640 try:
641- self.driver.spawn(context, instance, network_info, bd_mapping)
642+ self.driver.spawn(context, instance,
643+ network_info, block_device_info)
644 except Exception as ex: # pylint: disable=W0702
645 msg = _("Instance '%(instance_id)s' failed to spawn. Is "
646 "virtualization enabled in the BIOS? Details: "
647
648=== modified file 'nova/db/sqlalchemy/api.py'
649--- nova/db/sqlalchemy/api.py 2011-08-03 23:17:08 +0000
650+++ nova/db/sqlalchemy/api.py 2011-08-05 06:31:22 +0000
651@@ -20,6 +20,7 @@
652 """
653 import warnings
654
655+from nova import block_device
656 from nova import db
657 from nova import exception
658 from nova import flags
659@@ -1681,7 +1682,9 @@
660 session = get_session()
661 result = session.query(models.Network).\
662 filter(or_(models.Network.cidr == cidr,
663- models.Network.cidr_v6 == cidr)).first()
664+ models.Network.cidr_v6 == cidr)).\
665+ filter_by(deleted=False).\
666+ first()
667
668 if not result:
669 raise exception.NetworkNotFoundForCidr(cidr=cidr)
670@@ -2265,6 +2268,20 @@
671 else:
672 result.update(values)
673
674+ # NOTE(yamahata): same virtual device name can be specified multiple
675+ # times. So delete the existing ones.
676+ virtual_name = values['virtual_name']
677+ if (virtual_name is not None and
678+ block_device.is_swap_or_ephemeral(virtual_name)):
679+ session.query(models.BlockDeviceMapping).\
680+ filter_by(instance_id=values['instance_id']).\
681+ filter_by(virtual_name=virtual_name).\
682+ filter(models.BlockDeviceMapping.device_name !=
683+ values['device_name']).\
684+ update({'deleted': True,
685+ 'deleted_at': utils.utcnow(),
686+ 'updated_at': literal_column('updated_at')})
687+
688
689 @require_context
690 def block_device_mapping_get_all_by_instance(context, instance_id):
691
692=== modified file 'nova/image/glance.py'
693--- nova/image/glance.py 2011-08-01 18:59:29 +0000
694+++ nova/image/glance.py 2011-08-05 06:31:22 +0000
695@@ -19,7 +19,9 @@
696
697 from __future__ import absolute_import
698
699+import copy
700 import datetime
701+import json
702 import random
703
704 from glance.common import exception as glance_exception
705@@ -194,6 +196,7 @@
706 self._set_client_context(context)
707 # NOTE(vish): show is to check if image is available
708 self.show(context, image_id)
709+ image_meta = _convert_to_string(image_meta)
710 try:
711 image_meta = self.client.update_image(image_id, image_meta, data)
712 except glance_exception.NotFound:
713@@ -222,11 +225,19 @@
714 pass
715
716 @classmethod
717+ def _translate_to_service(cls, image_meta):
718+ image_meta = super(GlanceImageService,
719+ cls)._translate_to_service(image_meta)
720+ image_meta = _convert_to_string(image_meta)
721+ return image_meta
722+
723+ @classmethod
724 def _translate_to_base(cls, image_meta):
725 """Override translation to handle conversion to datetime objects."""
726 image_meta = service.BaseImageService._propertify_metadata(
727 image_meta, cls.SERVICE_IMAGE_ATTRS)
728 image_meta = _convert_timestamps_to_datetimes(image_meta)
729+ image_meta = _convert_from_string(image_meta)
730 return image_meta
731
732
733@@ -252,3 +263,38 @@
734
735 raise ValueError(_('%(timestamp)s does not follow any of the '
736 'signatures: %(ISO_FORMATS)s') % locals())
737+
738+
739+# TODO(yamahata): use block-device-mapping extension to glance
740+def _json_loads(properties, attr):
741+ prop = properties[attr]
742+ if isinstance(prop, basestring):
743+ properties[attr] = json.loads(prop)
744+
745+
746+def _json_dumps(properties, attr):
747+ prop = properties[attr]
748+ if not isinstance(prop, basestring):
749+ properties[attr] = json.dumps(prop)
750+
751+
752+_CONVERT_PROPS = ('block_device_mapping', 'mappings')
753+
754+
755+def _convert(method, metadata):
756+ metadata = copy.deepcopy(metadata) # don't touch original metadata
757+ properties = metadata.get('properties')
758+ if properties:
759+ for attr in _CONVERT_PROPS:
760+ if attr in properties:
761+ method(properties, attr)
762+
763+ return metadata
764+
765+
766+def _convert_from_string(metadata):
767+ return _convert(_json_loads, metadata)
768+
769+
770+def _convert_to_string(metadata):
771+ return _convert(_json_dumps, metadata)
772
773=== modified file 'nova/tests/image/test_glance.py'
774--- nova/tests/image/test_glance.py 2011-07-25 22:38:59 +0000
775+++ nova/tests/image/test_glance.py 2011-08-05 06:31:22 +0000
776@@ -235,3 +235,39 @@
777 'updated_at': None,
778 'deleted_at': None}
779 return fixture
780+
781+
782+class TestGlanceSerializer(unittest.TestCase):
783+ def test_serialize(self):
784+ metadata = {'name': 'image1',
785+ 'is_public': True,
786+ 'foo': 'bar',
787+ 'properties': {
788+ 'prop1': 'propvalue1',
789+ 'mappings': [
790+ {'virtual': 'aaa',
791+ 'device': 'bbb'},
792+ {'virtual': 'xxx',
793+ 'device': 'yyy'}],
794+ 'block_device_mapping': [
795+ {'virtual_device': 'fake',
796+ 'device_name': '/dev/fake'},
797+ {'virtual_device': 'ephemeral0',
798+ 'device_name': '/dev/fake0'}]}}
799+
800+ converted_expected = {
801+ 'name': 'image1',
802+ 'is_public': True,
803+ 'foo': 'bar',
804+ 'properties': {
805+ 'prop1': 'propvalue1',
806+ 'mappings':
807+ '[{"device": "bbb", "virtual": "aaa"}, '
808+ '{"device": "yyy", "virtual": "xxx"}]',
809+ 'block_device_mapping':
810+ '[{"virtual_device": "fake", "device_name": "/dev/fake"}, '
811+ '{"virtual_device": "ephemeral0", '
812+ '"device_name": "/dev/fake0"}]'}}
813+ converted = glance._convert_to_string(metadata)
814+ self.assertEqual(converted, converted_expected)
815+ self.assertEqual(glance._convert_from_string(converted), metadata)
816
817=== modified file 'nova/tests/test_api.py'
818--- nova/tests/test_api.py 2011-07-28 01:17:21 +0000
819+++ nova/tests/test_api.py 2011-08-05 06:31:22 +0000
820@@ -27,6 +27,7 @@
821 import StringIO
822 import webob
823
824+from nova import block_device
825 from nova import context
826 from nova import exception
827 from nova import test
828@@ -147,10 +148,12 @@
829 properties0 = {'mappings': mappings}
830 properties1 = {'root_device_name': '/dev/sdb', 'mappings': mappings}
831
832- root_device_name = ec2utils.properties_root_device_name(properties0)
833+ root_device_name = block_device.properties_root_device_name(
834+ properties0)
835 self.assertEqual(root_device_name, '/dev/sda1')
836
837- root_device_name = ec2utils.properties_root_device_name(properties1)
838+ root_device_name = block_device.properties_root_device_name(
839+ properties1)
840 self.assertEqual(root_device_name, '/dev/sdb')
841
842 def test_mapping_prepend_dev(self):
843@@ -184,7 +187,7 @@
844 'device': '/dev/sdc1'},
845 {'virtual': 'ephemeral1',
846 'device': '/dev/sdc1'}]
847- self.assertDictListMatch(ec2utils.mappings_prepend_dev(mappings),
848+ self.assertDictListMatch(block_device.mappings_prepend_dev(mappings),
849 expected_result)
850
851
852
853=== added file 'nova/tests/test_block_device.py'
854--- nova/tests/test_block_device.py 1970-01-01 00:00:00 +0000
855+++ nova/tests/test_block_device.py 2011-08-05 06:31:22 +0000
856@@ -0,0 +1,87 @@
857+# vim: tabstop=4 shiftwidth=4 softtabstop=4
858+
859+# Copyright 2011 Isaku Yamahata
860+# All Rights Reserved.
861+#
862+# Licensed under the Apache License, Version 2.0 (the "License"); you may
863+# not use this file except in compliance with the License. You may obtain
864+# a copy of the License at
865+#
866+# http://www.apache.org/licenses/LICENSE-2.0
867+#
868+# Unless required by applicable law or agreed to in writing, software
869+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
870+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
871+# License for the specific language governing permissions and limitations
872+# under the License.
873+
874+"""
875+Tests for Block Device utility functions.
876+"""
877+
878+from nova import block_device
879+from nova import test
880+
881+
882+class BlockDeviceTestCase(test.TestCase):
883+ def test_properties(self):
884+ root_device0 = '/dev/sda'
885+ root_device1 = '/dev/sdb'
886+ mappings = [{'virtual': 'root',
887+ 'device': root_device0}]
888+
889+ properties0 = {'mappings': mappings}
890+ properties1 = {'mappings': mappings,
891+ 'root_device_name': root_device1}
892+
893+ self.assertEqual(block_device.properties_root_device_name({}), None)
894+ self.assertEqual(
895+ block_device.properties_root_device_name(properties0),
896+ root_device0)
897+ self.assertEqual(
898+ block_device.properties_root_device_name(properties1),
899+ root_device1)
900+
901+ def test_ephemeral(self):
902+ self.assertFalse(block_device.is_ephemeral('ephemeral'))
903+ self.assertTrue(block_device.is_ephemeral('ephemeral0'))
904+ self.assertTrue(block_device.is_ephemeral('ephemeral1'))
905+ self.assertTrue(block_device.is_ephemeral('ephemeral11'))
906+ self.assertFalse(block_device.is_ephemeral('root'))
907+ self.assertFalse(block_device.is_ephemeral('swap'))
908+ self.assertFalse(block_device.is_ephemeral('/dev/sda1'))
909+
910+ self.assertEqual(block_device.ephemeral_num('ephemeral0'), 0)
911+ self.assertEqual(block_device.ephemeral_num('ephemeral1'), 1)
912+ self.assertEqual(block_device.ephemeral_num('ephemeral11'), 11)
913+
914+ self.assertFalse(block_device.is_swap_or_ephemeral('ephemeral'))
915+ self.assertTrue(block_device.is_swap_or_ephemeral('ephemeral0'))
916+ self.assertTrue(block_device.is_swap_or_ephemeral('ephemeral1'))
917+ self.assertTrue(block_device.is_swap_or_ephemeral('swap'))
918+ self.assertFalse(block_device.is_swap_or_ephemeral('root'))
919+ self.assertFalse(block_device.is_swap_or_ephemeral('/dev/sda1'))
920+
921+ def test_mappings_prepend_dev(self):
922+ mapping = [
923+ {'virtual': 'ami', 'device': '/dev/sda'},
924+ {'virtual': 'root', 'device': 'sda'},
925+ {'virtual': 'ephemeral0', 'device': 'sdb'},
926+ {'virtual': 'swap', 'device': 'sdc'},
927+ {'virtual': 'ephemeral1', 'device': 'sdd'},
928+ {'virtual': 'ephemeral2', 'device': 'sde'}]
929+
930+ expected = [
931+ {'virtual': 'ami', 'device': '/dev/sda'},
932+ {'virtual': 'root', 'device': 'sda'},
933+ {'virtual': 'ephemeral0', 'device': '/dev/sdb'},
934+ {'virtual': 'swap', 'device': '/dev/sdc'},
935+ {'virtual': 'ephemeral1', 'device': '/dev/sdd'},
936+ {'virtual': 'ephemeral2', 'device': '/dev/sde'}]
937+
938+ prepended = block_device.mappings_prepend_dev(mapping)
939+ self.assertEqual(prepended.sort(), expected.sort())
940+
941+ def test_strip_dev(self):
942+ self.assertEqual(block_device.strip_dev('/dev/sda'), 'sda')
943+ self.assertEqual(block_device.strip_dev('sda'), 'sda')
944
945=== modified file 'nova/tests/test_cloud.py'
946--- nova/tests/test_cloud.py 2011-08-03 21:13:37 +0000
947+++ nova/tests/test_cloud.py 2011-08-05 06:31:22 +0000
948@@ -17,6 +17,8 @@
949 # under the License.
950 import mox
951
952+import functools
953+
954 from base64 import b64decode
955 from M2Crypto import BIO
956 from M2Crypto import RSA
957@@ -892,13 +894,16 @@
958 def test_modify_image_attribute(self):
959 modify_image_attribute = self.cloud.modify_image_attribute
960
961+ fake_metadata = {'id': 1, 'container_format': 'ami',
962+ 'properties': {'kernel_id': 1, 'ramdisk_id': 1,
963+ 'type': 'machine'}, 'is_public': False}
964+
965 def fake_show(meh, context, id):
966- return {'id': 1, 'container_format': 'ami',
967- 'properties': {'kernel_id': 1, 'ramdisk_id': 1,
968- 'type': 'machine'}, 'is_public': False}
969+ return fake_metadata
970
971 def fake_update(meh, context, image_id, metadata, data=None):
972- return metadata
973+ fake_metadata.update(metadata)
974+ return fake_metadata
975
976 self.stubs.Set(fake._FakeImageService, 'show', fake_show)
977 self.stubs.Set(fake._FakeImageService, 'show_by_name', fake_show)
978@@ -1464,3 +1469,147 @@
979 # TODO(yamahata): clean up snapshot created by CreateImage.
980
981 self._restart_compute_service()
982+
983+ @staticmethod
984+ def _fake_bdm_get(ctxt, id):
985+ return [{'volume_id': 87654321,
986+ 'snapshot_id': None,
987+ 'no_device': None,
988+ 'virtual_name': None,
989+ 'delete_on_termination': True,
990+ 'device_name': '/dev/sdh'},
991+ {'volume_id': None,
992+ 'snapshot_id': 98765432,
993+ 'no_device': None,
994+ 'virtual_name': None,
995+ 'delete_on_termination': True,
996+ 'device_name': '/dev/sdi'},
997+ {'volume_id': None,
998+ 'snapshot_id': None,
999+ 'no_device': True,
1000+ 'virtual_name': None,
1001+ 'delete_on_termination': None,
1002+ 'device_name': None},
1003+ {'volume_id': None,
1004+ 'snapshot_id': None,
1005+ 'no_device': None,
1006+ 'virtual_name': 'ephemeral0',
1007+ 'delete_on_termination': None,
1008+ 'device_name': '/dev/sdb'},
1009+ {'volume_id': None,
1010+ 'snapshot_id': None,
1011+ 'no_device': None,
1012+ 'virtual_name': 'swap',
1013+ 'delete_on_termination': None,
1014+ 'device_name': '/dev/sdc'},
1015+ {'volume_id': None,
1016+ 'snapshot_id': None,
1017+ 'no_device': None,
1018+ 'virtual_name': 'ephemeral1',
1019+ 'delete_on_termination': None,
1020+ 'device_name': '/dev/sdd'},
1021+ {'volume_id': None,
1022+ 'snapshot_id': None,
1023+ 'no_device': None,
1024+ 'virtual_name': 'ephemeral2',
1025+ 'delete_on_termination': None,
1026+ 'device_name': '/dev/sd3'},
1027+ ]
1028+
1029+ def test_get_instance_mapping(self):
1030+ """Make sure that _get_instance_mapping works"""
1031+ ctxt = None
1032+ instance_ref0 = {'id': 0,
1033+ 'root_device_name': None}
1034+ instance_ref1 = {'id': 0,
1035+ 'root_device_name': '/dev/sda1'}
1036+
1037+ self.stubs.Set(db, 'block_device_mapping_get_all_by_instance',
1038+ self._fake_bdm_get)
1039+
1040+ expected = {'ami': 'sda1',
1041+ 'root': '/dev/sda1',
1042+ 'ephemeral0': '/dev/sdb',
1043+ 'swap': '/dev/sdc',
1044+ 'ephemeral1': '/dev/sdd',
1045+ 'ephemeral2': '/dev/sd3'}
1046+
1047+ self.assertEqual(self.cloud._format_instance_mapping(ctxt,
1048+ instance_ref0),
1049+ cloud._DEFAULT_MAPPINGS)
1050+ self.assertEqual(self.cloud._format_instance_mapping(ctxt,
1051+ instance_ref1),
1052+ expected)
1053+
1054+ def test_describe_instance_attribute(self):
1055+ """Make sure that describe_instance_attribute works"""
1056+ self.stubs.Set(db, 'block_device_mapping_get_all_by_instance',
1057+ self._fake_bdm_get)
1058+
1059+ def fake_get(ctxt, instance_id):
1060+ return {
1061+ 'id': 0,
1062+ 'root_device_name': '/dev/sdh',
1063+ 'security_groups': [{'name': 'fake0'}, {'name': 'fake1'}],
1064+ 'state_description': 'stopping',
1065+ 'instance_type': {'name': 'fake_type'},
1066+ 'kernel_id': 1,
1067+ 'ramdisk_id': 2,
1068+ 'user_data': 'fake-user data',
1069+ }
1070+ self.stubs.Set(self.cloud.compute_api, 'get', fake_get)
1071+
1072+ def fake_volume_get(ctxt, volume_id, session=None):
1073+ if volume_id == 87654321:
1074+ return {'id': volume_id,
1075+ 'attach_time': '13:56:24',
1076+ 'status': 'in-use'}
1077+ raise exception.VolumeNotFound(volume_id=volume_id)
1078+ self.stubs.Set(db.api, 'volume_get', fake_volume_get)
1079+
1080+ get_attribute = functools.partial(
1081+ self.cloud.describe_instance_attribute,
1082+ self.context, 'i-12345678')
1083+
1084+ bdm = get_attribute('blockDeviceMapping')
1085+ bdm['blockDeviceMapping'].sort()
1086+
1087+ expected_bdm = {'instance_id': 'i-12345678',
1088+ 'rootDeviceType': 'ebs',
1089+ 'blockDeviceMapping': [
1090+ {'deviceName': '/dev/sdh',
1091+ 'ebs': {'status': 'in-use',
1092+ 'deleteOnTermination': True,
1093+ 'volumeId': 87654321,
1094+ 'attachTime': '13:56:24'}}]}
1095+ expected_bdm['blockDeviceMapping'].sort()
1096+ self.assertEqual(bdm, expected_bdm)
1097+ # NOTE(yamahata): this isn't supported
1098+ # get_attribute('disableApiTermination')
1099+ groupSet = get_attribute('groupSet')
1100+ groupSet['groupSet'].sort()
1101+ expected_groupSet = {'instance_id': 'i-12345678',
1102+ 'groupSet': [{'groupId': 'fake0'},
1103+ {'groupId': 'fake1'}]}
1104+ expected_groupSet['groupSet'].sort()
1105+ self.assertEqual(groupSet, expected_groupSet)
1106+ self.assertEqual(get_attribute('instanceInitiatedShutdownBehavior'),
1107+ {'instance_id': 'i-12345678',
1108+ 'instanceInitiatedShutdownBehavior': 'stop'})
1109+ self.assertEqual(get_attribute('instanceType'),
1110+ {'instance_id': 'i-12345678',
1111+ 'instanceType': 'fake_type'})
1112+ self.assertEqual(get_attribute('kernel'),
1113+ {'instance_id': 'i-12345678',
1114+ 'kernel': 'aki-00000001'})
1115+ self.assertEqual(get_attribute('ramdisk'),
1116+ {'instance_id': 'i-12345678',
1117+ 'ramdisk': 'ari-00000002'})
1118+ self.assertEqual(get_attribute('rootDeviceName'),
1119+ {'instance_id': 'i-12345678',
1120+ 'rootDeviceName': '/dev/sdh'})
1121+ # NOTE(yamahata): this isn't supported
1122+ # get_attribute('sourceDestCheck')
1123+ self.assertEqual(get_attribute('userData'),
1124+ {'instance_id': 'i-12345678',
1125+ 'userData': '}\xa9\x1e\xba\xc7\xabu\xabZ'})
1126
1127=== modified file 'nova/tests/test_compute.py'
1128--- nova/tests/test_compute.py 2011-08-03 11:31:10 +0000
1129+++ nova/tests/test_compute.py 2011-08-05 06:31:22 +0000
1130@@ -877,15 +877,17 @@
1131 return bdm
1132
1133 def test_update_block_device_mapping(self):
1134+ swap_size = 1
1135+ instance_type = {'swap': swap_size}
1136 instance_id = self._create_instance()
1137 mappings = [
1138 {'virtual': 'ami', 'device': 'sda1'},
1139 {'virtual': 'root', 'device': '/dev/sda1'},
1140
1141+ {'virtual': 'swap', 'device': 'sdb4'},
1142+ {'virtual': 'swap', 'device': 'sdb3'},
1143+ {'virtual': 'swap', 'device': 'sdb2'},
1144 {'virtual': 'swap', 'device': 'sdb1'},
1145- {'virtual': 'swap', 'device': 'sdb2'},
1146- {'virtual': 'swap', 'device': 'sdb3'},
1147- {'virtual': 'swap', 'device': 'sdb4'},
1148
1149 {'virtual': 'ephemeral0', 'device': 'sdc1'},
1150 {'virtual': 'ephemeral1', 'device': 'sdc2'},
1151@@ -927,32 +929,36 @@
1152 'no_device': True}]
1153
1154 self.compute_api._update_image_block_device_mapping(
1155- self.context, instance_id, mappings)
1156+ self.context, instance_type, instance_id, mappings)
1157
1158 bdms = [self._parse_db_block_device_mapping(bdm_ref)
1159 for bdm_ref in db.block_device_mapping_get_all_by_instance(
1160 self.context, instance_id)]
1161 expected_result = [
1162- {'virtual_name': 'swap', 'device_name': '/dev/sdb1'},
1163- {'virtual_name': 'swap', 'device_name': '/dev/sdb2'},
1164- {'virtual_name': 'swap', 'device_name': '/dev/sdb3'},
1165- {'virtual_name': 'swap', 'device_name': '/dev/sdb4'},
1166+ {'virtual_name': 'swap', 'device_name': '/dev/sdb1',
1167+ 'volume_size': swap_size},
1168 {'virtual_name': 'ephemeral0', 'device_name': '/dev/sdc1'},
1169- {'virtual_name': 'ephemeral1', 'device_name': '/dev/sdc2'},
1170- {'virtual_name': 'ephemeral2', 'device_name': '/dev/sdc3'}]
1171+
1172+ # NOTE(yamahata): ATM only ephemeral0 is supported.
1173+ # they're ignored for now
1174+ #{'virtual_name': 'ephemeral1', 'device_name': '/dev/sdc2'},
1175+ #{'virtual_name': 'ephemeral2', 'device_name': '/dev/sdc3'}
1176+ ]
1177 bdms.sort()
1178 expected_result.sort()
1179 self.assertDictListMatch(bdms, expected_result)
1180
1181 self.compute_api._update_block_device_mapping(
1182- self.context, instance_id, block_device_mapping)
1183+ self.context, instance_types.get_default_instance_type(),
1184+ instance_id, block_device_mapping)
1185 bdms = [self._parse_db_block_device_mapping(bdm_ref)
1186 for bdm_ref in db.block_device_mapping_get_all_by_instance(
1187 self.context, instance_id)]
1188 expected_result = [
1189 {'snapshot_id': 0x12345678, 'device_name': '/dev/sda1'},
1190
1191- {'virtual_name': 'swap', 'device_name': '/dev/sdb1'},
1192+ {'virtual_name': 'swap', 'device_name': '/dev/sdb1',
1193+ 'volume_size': swap_size},
1194 {'snapshot_id': 0x23456789, 'device_name': '/dev/sdb2'},
1195 {'snapshot_id': 0x3456789A, 'device_name': '/dev/sdb3'},
1196 {'no_device': True, 'device_name': '/dev/sdb4'},
1197@@ -974,3 +980,13 @@
1198 self.context, instance_id):
1199 db.block_device_mapping_destroy(self.context, bdm['id'])
1200 self.compute.terminate_instance(self.context, instance_id)
1201+
1202+ def test_ephemeral_size(self):
1203+ local_size = 2
1204+ inst_type = {'local_gb': local_size}
1205+ self.assertEqual(self.compute_api._ephemeral_size(inst_type,
1206+ 'ephemeral0'),
1207+ local_size)
1208+ self.assertEqual(self.compute_api._ephemeral_size(inst_type,
1209+ 'ephemeral1'),
1210+ 0)
1211
1212=== modified file 'nova/tests/test_libvirt.py'
1213--- nova/tests/test_libvirt.py 2011-08-03 19:22:58 +0000
1214+++ nova/tests/test_libvirt.py 2011-08-05 06:31:22 +0000
1215@@ -169,6 +169,7 @@
1216 'project_id': 'fake',
1217 'bridge': 'br101',
1218 'image_ref': '123456',
1219+ 'local_gb': 20,
1220 'instance_type_id': '5'} # m1.small
1221
1222 def lazy_load_library_exists(self):
1223@@ -744,6 +745,42 @@
1224 ip = conn.get_host_ip_addr()
1225 self.assertEquals(ip, FLAGS.my_ip)
1226
1227+ def test_volume_in_mapping(self):
1228+ conn = connection.LibvirtConnection(False)
1229+ swap = {'device_name': '/dev/sdb',
1230+ 'swap_size': 1}
1231+ ephemerals = [{'num': 0,
1232+ 'virtual_name': 'ephemeral0',
1233+ 'device_name': '/dev/sdc1',
1234+ 'size': 1},
1235+ {'num': 2,
1236+ 'virtual_name': 'ephemeral2',
1237+ 'device_name': '/dev/sdd',
1238+ 'size': 1}]
1239+ block_device_mapping = [{'mount_device': '/dev/sde',
1240+ 'device_path': 'fake_device'},
1241+ {'mount_device': '/dev/sdf',
1242+ 'device_path': 'fake_device'}]
1243+ block_device_info = {
1244+ 'root_device_name': '/dev/sda',
1245+ 'swap': swap,
1246+ 'ephemerals': ephemerals,
1247+ 'block_device_mapping': block_device_mapping}
1248+
1249+ def _assert_volume_in_mapping(device_name, true_or_false):
1250+ self.assertEquals(conn._volume_in_mapping(device_name,
1251+ block_device_info),
1252+ true_or_false)
1253+
1254+ _assert_volume_in_mapping('sda', False)
1255+ _assert_volume_in_mapping('sdb', True)
1256+ _assert_volume_in_mapping('sdc1', True)
1257+ _assert_volume_in_mapping('sdd', True)
1258+ _assert_volume_in_mapping('sde', True)
1259+ _assert_volume_in_mapping('sdf', True)
1260+ _assert_volume_in_mapping('sdg', False)
1261+ _assert_volume_in_mapping('sdh1', False)
1262+
1263
1264 class NWFilterFakes:
1265 def __init__(self):
1266
1267=== modified file 'nova/tests/test_metadata.py'
1268--- nova/tests/test_metadata.py 2011-06-27 13:58:17 +0000
1269+++ nova/tests/test_metadata.py 2011-08-05 06:31:22 +0000
1270@@ -43,6 +43,7 @@
1271 'reservation_id': 'r-xxxxxxxx',
1272 'user_data': '',
1273 'image_ref': 7,
1274+ 'root_device_name': '/dev/sda1',
1275 'hostname': 'test'})
1276
1277 def instance_get(*args, **kwargs):
1278
1279=== added file 'nova/tests/test_virt.py'
1280--- nova/tests/test_virt.py 1970-01-01 00:00:00 +0000
1281+++ nova/tests/test_virt.py 2011-08-05 06:31:22 +0000
1282@@ -0,0 +1,83 @@
1283+# vim: tabstop=4 shiftwidth=4 softtabstop=4
1284+
1285+# Copyright 2011 Isaku Yamahata
1286+# All Rights Reserved.
1287+#
1288+# Licensed under the Apache License, Version 2.0 (the "License"); you may
1289+# not use this file except in compliance with the License. You may obtain
1290+# a copy of the License at
1291+#
1292+# http://www.apache.org/licenses/LICENSE-2.0
1293+#
1294+# Unless required by applicable law or agreed to in writing, software
1295+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
1296+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
1297+# License for the specific language governing permissions and limitations
1298+# under the License.
1299+
1300+from nova import flags
1301+from nova import test
1302+from nova.virt import driver
1303+
1304+FLAGS = flags.FLAGS
1305+
1306+
1307+class TestVirtDriver(test.TestCase):
1308+ def test_block_device(self):
1309+ swap = {'device_name': '/dev/sdb',
1310+ 'swap_size': 1}
1311+ ephemerals = [{'num': 0,
1312+ 'virtual_name': 'ephemeral0',
1313+ 'device_name': '/dev/sdc1',
1314+ 'size': 1}]
1315+ block_device_mapping = [{'mount_device': '/dev/sde',
1316+ 'device_path': 'fake_device'}]
1317+ block_device_info = {
1318+ 'root_device_name': '/dev/sda',
1319+ 'swap': swap,
1320+ 'ephemerals': ephemerals,
1321+ 'block_device_mapping': block_device_mapping}
1322+
1323+ empty_block_device_info = {}
1324+
1325+ self.assertEqual(
1326+ driver.block_device_info_get_root(block_device_info), '/dev/sda')
1327+ self.assertEqual(
1328+ driver.block_device_info_get_root(empty_block_device_info), None)
1329+ self.assertEqual(
1330+ driver.block_device_info_get_root(None), None)
1331+
1332+ self.assertEqual(
1333+ driver.block_device_info_get_swap(block_device_info), swap)
1334+ self.assertEqual(driver.block_device_info_get_swap(
1335+ empty_block_device_info)['device_name'], None)
1336+ self.assertEqual(driver.block_device_info_get_swap(
1337+ empty_block_device_info)['swap_size'], 0)
1338+ self.assertEqual(
1339+ driver.block_device_info_get_swap({'swap': None})['device_name'],
1340+ None)
1341+ self.assertEqual(
1342+ driver.block_device_info_get_swap({'swap': None})['swap_size'],
1343+ 0)
1344+ self.assertEqual(
1345+ driver.block_device_info_get_swap(None)['device_name'], None)
1346+ self.assertEqual(
1347+ driver.block_device_info_get_swap(None)['swap_size'], 0)
1348+
1349+ self.assertEqual(
1350+ driver.block_device_info_get_ephemerals(block_device_info),
1351+ ephemerals)
1352+ self.assertEqual(
1353+ driver.block_device_info_get_ephemerals(empty_block_device_info),
1354+ [])
1355+ self.assertEqual(
1356+ driver.block_device_info_get_ephemerals(None),
1357+ [])
1358+
1359+ def test_swap_is_usable(self):
1360+ self.assertFalse(driver.swap_is_usable(None))
1361+ self.assertFalse(driver.swap_is_usable({'device_name': None}))
1362+ self.assertFalse(driver.swap_is_usable({'device_name': '/dev/sdb',
1363+ 'swap_size': 0}))
1364+ self.assertTrue(driver.swap_is_usable({'device_name': '/dev/sdb',
1365+ 'swap_size': 1}))
1366
1367=== modified file 'nova/virt/driver.py'
1368--- nova/virt/driver.py 2011-07-29 21:58:27 +0000
1369+++ nova/virt/driver.py 2011-08-05 06:31:22 +0000
1370@@ -32,6 +32,33 @@
1371 self.state = state
1372
1373
1374+def block_device_info_get_root(block_device_info):
1375+ block_device_info = block_device_info or {}
1376+ return block_device_info.get('root_device_name')
1377+
1378+
1379+def block_device_info_get_swap(block_device_info):
1380+ block_device_info = block_device_info or {}
1381+ return block_device_info.get('swap') or {'device_name': None,
1382+ 'swap_size': 0}
1383+
1384+
1385+def swap_is_usable(swap):
1386+ return swap and swap['device_name'] and swap['swap_size'] > 0
1387+
1388+
1389+def block_device_info_get_ephemerals(block_device_info):
1390+ block_device_info = block_device_info or {}
1391+ ephemerals = block_device_info.get('ephemerals') or []
1392+ return ephemerals
1393+
1394+
1395+def block_device_info_get_mapping(block_device_info):
1396+ block_device_info = block_device_info or {}
1397+ block_device_mapping = block_device_info.get('block_device_mapping') or []
1398+ return block_device_mapping
1399+
1400+
1401 class ComputeDriver(object):
1402 """Base class for compute drivers.
1403
1404@@ -65,8 +92,8 @@
1405 # TODO(Vek): Need to pass context in for access to auth_token
1406 raise NotImplementedError()
1407
1408- def spawn(self, context, instance, network_info,
1409- block_device_mapping=None):
1410+ def spawn(self, context, instance,
1411+ network_info=None, block_device_info=None):
1412 """Launch a VM for the specified instance"""
1413 raise NotImplementedError()
1414
1415
1416=== modified file 'nova/virt/fake.py'
1417--- nova/virt/fake.py 2011-08-02 21:45:57 +0000
1418+++ nova/virt/fake.py 2011-08-05 06:31:22 +0000
1419@@ -129,8 +129,8 @@
1420 info_list.append(self._map_to_instance_info(instance))
1421 return info_list
1422
1423- def spawn(self, context, instance, network_info,
1424- block_device_mapping=None):
1425+ def spawn(self, context, instance,
1426+ network_info=None, block_device_info=None):
1427 """
1428 Create a new instance/VM/domain on the virtualization platform.
1429
1430
1431=== modified file 'nova/virt/hyperv.py'
1432--- nova/virt/hyperv.py 2011-07-29 21:58:27 +0000
1433+++ nova/virt/hyperv.py 2011-08-05 06:31:22 +0000
1434@@ -138,8 +138,8 @@
1435
1436 return instance_infos
1437
1438- def spawn(self, context, instance, network_info,
1439- block_device_mapping=None):
1440+ def spawn(self, context, instance,
1441+ network_info=None, block_device_info=None):
1442 """ Create a new VM and start it."""
1443 vm = self._lookup(instance.name)
1444 if vm is not None:
1445
1446=== modified file 'nova/virt/libvirt.xml.template'
1447--- nova/virt/libvirt.xml.template 2011-07-26 00:41:55 +0000
1448+++ nova/virt/libvirt.xml.template 2011-08-05 06:31:22 +0000
1449@@ -3,24 +3,22 @@
1450 <memory>${memory_kb}</memory>
1451 <os>
1452 #if $type == 'lxc'
1453- #set $disk_prefix = ''
1454 #set $disk_bus = ''
1455 <type>exe</type>
1456 <init>/sbin/init</init>
1457 #else if $type == 'uml'
1458- #set $disk_prefix = 'ubd'
1459 #set $disk_bus = 'uml'
1460 <type>uml</type>
1461 <kernel>/usr/bin/linux</kernel>
1462- <root>/dev/ubda</root>
1463+ #set $root_device_name = $getVar('root_device_name', '/dev/ubda')
1464+ <root>${root_device_name}</root>
1465 #else
1466 #if $type == 'xen'
1467- #set $disk_prefix = 'sd'
1468 #set $disk_bus = 'scsi'
1469 <type>linux</type>
1470- <root>/dev/xvda</root>
1471+ #set $root_device_name = $getVar('root_device_name', '/dev/xvda')
1472+ <root>${root_device_name}</root>
1473 #else
1474- #set $disk_prefix = 'vd'
1475 #set $disk_bus = 'virtio'
1476 <type>hvm</type>
1477 #end if
1478@@ -33,7 +31,8 @@
1479 #if $type == 'xen'
1480 <cmdline>ro</cmdline>
1481 #else
1482- <cmdline>root=/dev/vda console=ttyS0</cmdline>
1483+ #set $root_device_name = $getVar('root_device_name', '/dev/vda')
1484+ <cmdline>root=${root_device_name} console=ttyS0</cmdline>
1485 #end if
1486 #if $getVar('ramdisk', None)
1487 <initrd>${ramdisk}</initrd>
1488@@ -71,16 +70,30 @@
1489 <disk type='file'>
1490 <driver type='${driver_type}'/>
1491 <source file='${basepath}/disk'/>
1492- <target dev='${disk_prefix}a' bus='${disk_bus}'/>
1493+ <target dev='${root_device}' bus='${disk_bus}'/>
1494 </disk>
1495 #end if
1496- #if $getVar('local', False)
1497+ #if $getVar('local_device', False)
1498 <disk type='file'>
1499 <driver type='${driver_type}'/>
1500 <source file='${basepath}/disk.local'/>
1501- <target dev='${disk_prefix}b' bus='${disk_bus}'/>
1502+ <target dev='${local_device}' bus='${disk_bus}'/>
1503 </disk>
1504 #end if
1505+ #for $eph in $ephemerals
1506+ <disk type='block'>
1507+ <driver type='${driver_type}'/>
1508+ <source dev='${basepath}/${eph.device_path}'/>
1509+ <target dev='${eph.device}' bus='${disk_bus}'/>
1510+ </disk>
1511+ #end for
1512+ #if $getVar('swap_device', False)
1513+ <disk type='file'>
1514+ <driver type='${driver_type}'/>
1515+ <source file='${basepath}/disk.swap'/>
1516+ <target dev='${swap_device}' bus='${disk_bus}'/>
1517+ </disk>
1518+ #end if
1519 #for $vol in $volumes
1520 <disk type='${vol.type}'>
1521 <driver type='raw'/>
1522
1523=== modified file 'nova/virt/libvirt/connection.py'
1524--- nova/virt/libvirt/connection.py 2011-08-05 02:22:13 +0000
1525+++ nova/virt/libvirt/connection.py 2011-08-05 06:31:22 +0000
1526@@ -54,6 +54,7 @@
1527 from eventlet import greenthread
1528 from eventlet import tpool
1529
1530+from nova import block_device
1531 from nova import context as nova_context
1532 from nova import db
1533 from nova import exception
1534@@ -151,8 +152,8 @@
1535 Template = t.Template
1536
1537
1538-def _strip_dev(mount_path):
1539- return re.sub(r'^/dev/', '', mount_path)
1540+def _get_eph_disk(ephemeral):
1541+ return 'disk.eph' + str(ephemeral['num'])
1542
1543
1544 class LibvirtConnection(driver.ComputeDriver):
1545@@ -570,15 +571,14 @@
1546 # NOTE(ilyaalekseyev): Implementation like in multinics
1547 # for xenapi(tr3buchet)
1548 @exception.wrap_exception()
1549- def spawn(self, context, instance, network_info,
1550- block_device_mapping=None):
1551+ def spawn(self, context, instance,
1552+ network_info=None, block_device_info=None):
1553 xml = self.to_xml(instance, False, network_info=network_info,
1554- block_device_mapping=block_device_mapping)
1555- block_device_mapping = block_device_mapping or []
1556+ block_device_info=block_device_info)
1557 self.firewall_driver.setup_basic_filtering(instance, network_info)
1558 self.firewall_driver.prepare_instance_filter(instance, network_info)
1559 self._create_image(context, instance, xml, network_info=network_info,
1560- block_device_mapping=block_device_mapping)
1561+ block_device_info=block_device_info)
1562 domain = self._create_new_domain(xml)
1563 LOG.debug(_("instance %s: is running"), instance['name'])
1564 self.firewall_driver.apply_instance_filter(instance)
1565@@ -755,11 +755,14 @@
1566 utils.execute('truncate', target, '-s', "%dG" % local_gb)
1567 # TODO(vish): should we format disk by default?
1568
1569+ def _create_swap(self, target, swap_gb):
1570+ """Create a swap file of specified size"""
1571+ self._create_local(target, swap_gb)
1572+ utils.execute('mkswap', target)
1573+
1574 def _create_image(self, context, inst, libvirt_xml, suffix='',
1575 disk_images=None, network_info=None,
1576- block_device_mapping=None):
1577- block_device_mapping = block_device_mapping or []
1578-
1579+ block_device_info=None):
1580 if not suffix:
1581 suffix = ''
1582
1583@@ -818,8 +821,8 @@
1584 size = None
1585 root_fname += "_sm"
1586
1587- if not self._volume_in_mapping(self.root_mount_device,
1588- block_device_mapping):
1589+ if not self._volume_in_mapping(self.default_root_device,
1590+ block_device_info):
1591 self._cache_image(fn=self._fetch_image,
1592 context=context,
1593 target=basepath('disk'),
1594@@ -830,13 +833,38 @@
1595 project_id=inst['project_id'],
1596 size=size)
1597
1598- if inst_type['local_gb'] and not self._volume_in_mapping(
1599- self.local_mount_device, block_device_mapping):
1600+ local_gb = inst['local_gb']
1601+ if local_gb and not self._volume_in_mapping(
1602+ self.default_local_device, block_device_info):
1603 self._cache_image(fn=self._create_local,
1604 target=basepath('disk.local'),
1605- fname="local_%s" % inst_type['local_gb'],
1606- cow=FLAGS.use_cow_images,
1607- local_gb=inst_type['local_gb'])
1608+ fname="local_%s" % local_gb,
1609+ cow=FLAGS.use_cow_images,
1610+ local_gb=local_gb)
1611+
1612+ for eph in driver.block_device_info_get_ephemerals(block_device_info):
1613+ self._cache_image(fn=self._create_local,
1614+ target=basepath(_get_eph_disk(eph)),
1615+ fname="local_%s" % eph['size'],
1616+ cow=FLAGS.use_cow_images,
1617+ local_gb=eph['size'])
1618+
1619+ swap_gb = 0
1620+
1621+ swap = driver.block_device_info_get_swap(block_device_info)
1622+ if driver.swap_is_usable(swap):
1623+ swap_gb = swap['swap_size']
1624+ elif (inst_type['swap'] > 0 and
1625+ not self._volume_in_mapping(self.default_swap_device,
1626+ block_device_info)):
1627+ swap_gb = inst_type['swap']
1628+
1629+ if swap_gb > 0:
1630+ self._cache_image(fn=self._create_swap,
1631+ target=basepath('disk.swap'),
1632+ fname="swap_%s" % swap_gb,
1633+ cow=FLAGS.use_cow_images,
1634+ swap_gb=swap_gb)
1635
1636 # For now, we assume that if we're not using a kernel, we're using a
1637 # partitioned disk image where the target partition is the first
1638@@ -917,16 +945,35 @@
1639 if FLAGS.libvirt_type == 'uml':
1640 utils.execute('sudo', 'chown', 'root', basepath('disk'))
1641
1642- root_mount_device = 'vda' # FIXME for now. it's hard coded.
1643- local_mount_device = 'vdb' # FIXME for now. it's hard coded.
1644-
1645- def _volume_in_mapping(self, mount_device, block_device_mapping):
1646- mount_device_ = _strip_dev(mount_device)
1647- for vol in block_device_mapping:
1648- vol_mount_device = _strip_dev(vol['mount_device'])
1649- if vol_mount_device == mount_device_:
1650- return True
1651- return False
1652+ if FLAGS.libvirt_type == 'uml':
1653+ _disk_prefix = 'ubd'
1654+ elif FLAGS.libvirt_type == 'xen':
1655+ _disk_prefix = 'sd'
1656+ elif FLAGS.libvirt_type == 'lxc':
1657+ _disk_prefix = ''
1658+ else:
1659+ _disk_prefix = 'vd'
1660+
1661+ default_root_device = _disk_prefix + 'a'
1662+ default_local_device = _disk_prefix + 'b'
1663+ default_swap_device = _disk_prefix + 'c'
1664+
1665+ def _volume_in_mapping(self, mount_device, block_device_info):
1666+ block_device_list = [block_device.strip_dev(vol['mount_device'])
1667+ for vol in
1668+ driver.block_device_info_get_mapping(
1669+ block_device_info)]
1670+ swap = driver.block_device_info_get_swap(block_device_info)
1671+ if driver.swap_is_usable(swap):
1672+ block_device_list.append(
1673+ block_device.strip_dev(swap['device_name']))
1674+ block_device_list += [block_device.strip_dev(ephemeral['device_name'])
1675+ for ephemeral in
1676+ driver.block_device_info_get_ephemerals(
1677+ block_device_info)]
1678+
1679+ LOG.debug(_("block_device_list %s"), block_device_list)
1680+ return block_device.strip_dev(mount_device) in block_device_list
1681
1682 def _get_volume_device_info(self, device_path):
1683 if device_path.startswith('/dev/'):
1684@@ -938,8 +985,9 @@
1685 raise exception.InvalidDevicePath(path=device_path)
1686
1687 def _prepare_xml_info(self, instance, rescue=False, network_info=None,
1688- block_device_mapping=None):
1689- block_device_mapping = block_device_mapping or []
1690+ block_device_info=None):
1691+ block_device_mapping = driver.block_device_info_get_mapping(
1692+ block_device_info)
1693 # TODO(adiantum) remove network_info creation code
1694 # when multinics will be completed
1695 if not network_info:
1696@@ -958,17 +1006,27 @@
1697 driver_type = 'raw'
1698
1699 for vol in block_device_mapping:
1700- vol['mount_device'] = _strip_dev(vol['mount_device'])
1701+ vol['mount_device'] = block_device.strip_dev(vol['mount_device'])
1702 (vol['type'], vol['protocol'], vol['name']) = \
1703 self._get_volume_device_info(vol['device_path'])
1704
1705- ebs_root = self._volume_in_mapping(self.root_mount_device,
1706- block_device_mapping)
1707- if self._volume_in_mapping(self.local_mount_device,
1708- block_device_mapping):
1709- local_gb = False
1710- else:
1711- local_gb = inst_type['local_gb']
1712+ ebs_root = self._volume_in_mapping(self.default_root_device,
1713+ block_device_info)
1714+
1715+ local_device = False
1716+ if not (self._volume_in_mapping(self.default_local_device,
1717+ block_device_info) or
1718+ 0 in [eph['num'] for eph in
1719+ driver.block_device_info_get_ephemerals(
1720+ block_device_info)]):
1721+ if instance['local_gb'] > 0:
1722+ local_device = self.default_local_device
1723+
1724+ ephemerals = []
1725+ for eph in driver.block_device_info_get_ephemerals(block_device_info):
1726+ ephemerals.append({'device_path': _get_eph_disk(eph),
1727+ 'device': block_device.strip_dev(
1728+ eph['device_name'])})
1729
1730 xml_info = {'type': FLAGS.libvirt_type,
1731 'name': instance['name'],
1732@@ -977,12 +1035,35 @@
1733 'memory_kb': inst_type['memory_mb'] * 1024,
1734 'vcpus': inst_type['vcpus'],
1735 'rescue': rescue,
1736- 'local': local_gb,
1737+ 'disk_prefix': self._disk_prefix,
1738 'driver_type': driver_type,
1739 'vif_type': FLAGS.libvirt_vif_type,
1740 'nics': nics,
1741 'ebs_root': ebs_root,
1742- 'volumes': block_device_mapping}
1743+ 'local_device': local_device,
1744+ 'volumes': block_device_mapping,
1745+ 'ephemerals': ephemerals}
1746+
1747+ root_device_name = driver.block_device_info_get_root(block_device_info)
1748+ if root_device_name:
1749+ xml_info['root_device'] = block_device.strip_dev(root_device_name)
1750+ xml_info['root_device_name'] = root_device_name
1751+ else:
1752+ # NOTE(yamahata):
1753+ # for nova.api.ec2.cloud.CloudController.get_metadata()
1754+ xml_info['root_device'] = self.default_root_device
1755+ db.instance_update(
1756+ nova_context.get_admin_context(), instance['id'],
1757+ {'root_device_name': '/dev/' + self.default_root_device})
1758+
1759+ swap = driver.block_device_info_get_swap(block_device_info)
1760+ if driver.swap_is_usable(swap):
1761+ xml_info['swap_device'] = block_device.strip_dev(
1762+ swap['device_name'])
1763+ elif (inst_type['swap'] > 0 and
1764+ not self._volume_in_mapping(self.default_swap_device,
1765+ block_device_info)):
1766+ xml_info['swap_device'] = self.default_swap_device
1767
1768 if FLAGS.vnc_enabled and FLAGS.libvirt_type not in ('lxc', 'uml'):
1769 xml_info['vncserver_host'] = FLAGS.vncserver_host
1770@@ -998,12 +1079,11 @@
1771 return xml_info
1772
1773 def to_xml(self, instance, rescue=False, network_info=None,
1774- block_device_mapping=None):
1775- block_device_mapping = block_device_mapping or []
1776+ block_device_info=None):
1777 # TODO(termie): cache?
1778 LOG.debug(_('instance %s: starting toXML method'), instance['name'])
1779 xml_info = self._prepare_xml_info(instance, rescue, network_info,
1780- block_device_mapping)
1781+ block_device_info)
1782 xml = str(Template(self.libvirt_xml, searchList=[xml_info]))
1783 LOG.debug(_('instance %s: finished toXML method'), instance['name'])
1784 return xml
1785
1786=== modified file 'nova/virt/xenapi_conn.py'
1787--- nova/virt/xenapi_conn.py 2011-08-05 02:22:13 +0000
1788+++ nova/virt/xenapi_conn.py 2011-08-05 06:31:22 +0000
1789@@ -184,8 +184,8 @@
1790 def list_instances_detail(self):
1791 return self._vmops.list_instances_detail()
1792
1793- def spawn(self, context, instance, network_info,
1794- block_device_mapping=None):
1795+ def spawn(self, context, instance,
1796+ network_info=None, block_device_info=None):
1797 """Create VM instance"""
1798 self._vmops.spawn(context, instance, network_info)
1799