Merge lp:~chiradeep/nova/msft-hyper-v-support into lp:~hudson-openstack/nova/trunk
- msft-hyper-v-support
- Merge into trunk
Status: | Superseded |
---|---|
Proposed branch: | lp:~chiradeep/nova/msft-hyper-v-support |
Merge into: | lp:~hudson-openstack/nova/trunk |
Diff against target: |
619 lines (+512/-26) 5 files modified
nova/compute/manager.py (+2/-0) nova/twistd.py (+16/-15) nova/virt/connection.py (+3/-0) nova/virt/hyperv.py (+453/-0) nova/virt/images.py (+38/-11) |
To merge this branch: | bzr merge lp:~chiradeep/nova/msft-hyper-v-support |
Related bugs: | |
Related blueprints: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jay Pipes (community) | Needs Fixing | ||
Review via email: mp+38036@code.launchpad.net |
This proposal has been superseded by a proposal from 2010-10-13.
Commit message
Introduces basic support for spawning, rebooting and destroying vms when using Microsoft Hyper-V as the hypervisor
Description of the change
Introduces basic support for spawning, rebooting and destroying vms when using Microsoft Hyper-V as the hypervisor.
Images need to be in VHD format. Note that although Hyper-V doesn't accept kernel and ramdisk
separate from the image, the nova objectstore api still expects an image to have an associated aki and ari. You can use dummy aki and ari images -- the hyper-v driver won't use them or try to download them.
Requires Python's WMI module.
Jay Pipes (jaypipes) wrote : | # |
Also, I forgot to mention that we'd really need some Hyper-V specific test cases when Monty and others get around to getting our testing hardware and platform up-to-speed with Windows stuff...
Chiradeep Vittal (chiradeep) wrote : | # |
Thanks Jay. I'll put together something along the lines of the XenAPI driver.
The WMI calls are well documented on MSDN. Also, the Python WMI library generates the docstring for any WMI function that lists the parameters and return tuple.
Not sure that I want to regurgitate *all* those details in this doc, but I'll see if I can highlight some of the exemplary calls.
Some more comments below, preceded by >>>>
_______
From: <email address hidden> [<email address hidden>] On Behalf Of Jay Pipes [<email address hidden>]
Sent: Monday, October 11, 2010 10:11 AM
To: <email address hidden>
Subject: Re: [Merge] lp:~chiradeep/nova/msft-hyper-v-support into lp:nova
Review: Needs Fixing
Hi Chiradeep!
Thanks for the contribution and a start of Windows support in OpenStack! :)
Here are my review notes for you.
1) Documentation is sparse :)
It would be excellent to get some overall design and module documentation for your work. :) This will help those coders in the future who are maintaining and enhancing the hyper-V work. Please see Ewan Mellor's excellent documentation in the xenapi virtualization layer for an example of the kind of documentation that is helpful for maintainers.
This documentation is, IMHO, pretty critical considering the sorry state of python-wmi documentation regarding virtual environments.
Additional code comments would be very useful in places such as the following:
181 + (job, ret_val) = vs_man_
182 + [], None, vs_gs_data.
What does vs_gs_data.
>>>> This is simply how WMI works. When you want to pass in object references, you generate an XML representation using GetText. I think that whoever maintains this
needs a bookmark to the MSDN libraries and the PowerShell libraries. Equally someone maintaining the XenApi driver or libvirt driver needs to *get* the object model and the conventions.
2) Please use Nova's exceptions, which are tested for in the test suites and programs, instead of generic Exception:
402 + raise Exception('instance not present %s' % instance_id)
Should be replaced with:
from nova import exception
...
raise exception.
same for here:
360 + if vm is None:
361 + raise Exception('instance not present %s' % instance.name)
>>>> sure.
3) Your rewrite of the _fetch_s3_image() function is good, however, it removes the efficiency of using a shared process pool in Linux/BSD environments in favour of cross-platform compatibility. It would be nice to keep the existing code and use your rewrite in an if sys.platform.
>>>> sure
============
And here are some style-related things to fix up...
1) Please make sure imports are ordered according to the instructions in the HACKING file. In particular, please make sure system lib imports are first, in alphabetical order, followed by a newline, then imports of 3rd-party libraries ...
Jay Pipes (jaypipes) wrote : | # |
> Thanks Jay. I'll put together something along the lines of the XenAPI driver.
> The WMI calls are well documented on MSDN. Also, the Python WMI library
> generates the docstring for any WMI function that lists the parameters and
> return tuple.
> Not sure that I want to regurgitate *all* those details in this doc, but I'll
> see if I can highlight some of the exemplary calls.
Well, just do your best to provide a general overview of how the WMI calls are made, what's expected back from them, etc. :)
> Additional code comments would be very useful in places such as the following:
>
> 181 + (job, ret_val) = vs_man_
> 182 + [], None, vs_gs_data.
>
> What does vs_gs_data.
> sv_man_
> things, maintaining developers have zero chance of successfully fixing bugs
> that may occur :)
>
> >>>> This is simply how WMI works. When you want to pass in object references,
> you generate an XML representation using GetText. I think that whoever
> maintains this
> needs a bookmark to the MSDN libraries and the PowerShell libraries. Equally
> someone maintaining the XenApi driver or libvirt driver needs to *get* the
> object model and the conventions.
I will hold off judgment about the WMI and PowerShell APIs...
If you could just put the above note in your general documentation, and even provide a couple of those links for MSDN/PowerShell docs in there, that would be great! :)
Cheers!
jay
- 340. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Add design doc, docstrings, document hyper-v wmi, python wmi usage. Adhere to pep-8 more closely
- 341. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Fix typo, fix import
- 342. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Remove extraneous newlines
- 343. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Newlines again, reorder imports
- 344. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Added a unit test but not integrated it
- 345. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
review comments
- 346. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
fix indent
- 347. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
revert to generic exceptions
- 348. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
remove nonexistent exception
- 349. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Merged from trunk and fixed merge issues.
Also fixed pep8 issues - 350. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
remove some logging
- 351. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Merge from trunk: process replaced with util
- 352. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Merge from trunk: process replaced with util
- 353. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Merge from trunk again -- get rid of twistd dependencies
- 354. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
i18n logging and exception strings
- 355. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Merge from trunk again
- 356. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Revert some unneeded formatting since twistd is no longer used
- 357. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
Redis dependency no longer needed
- 358. By Chiradeep Vittal <chiradeep@chiradeep-lt2>
-
need one more newline
- 359. By Chiradeep Vittal
-
Make test case work again
- 360. By Chiradeep Vittal
-
Add to Authors and mailmap
Unmerged revisions
Preview Diff
1 | === modified file 'nova/compute/manager.py' |
2 | --- nova/compute/manager.py 2010-10-12 20:18:29 +0000 |
3 | +++ nova/compute/manager.py 2010-10-13 07:08:43 +0000 |
4 | @@ -39,6 +39,8 @@ |
5 | 'where instances are stored on disk') |
6 | flags.DEFINE_string('compute_driver', 'nova.virt.connection.get_connection', |
7 | 'Driver to use for volume creation') |
8 | +flags.DEFINE_string('images_path', utils.abspath('../images'), |
9 | + 'path to decrypted local images if not using s3') |
10 | |
11 | |
12 | class ComputeManager(manager.Manager): |
13 | |
14 | === modified file 'nova/twistd.py' |
15 | --- nova/twistd.py 2010-08-17 22:05:06 +0000 |
16 | +++ nova/twistd.py 2010-10-13 07:08:43 +0000 |
17 | @@ -224,21 +224,22 @@ |
18 | logging.getLogger('amqplib').setLevel(logging.WARN) |
19 | FLAGS.python = filename |
20 | FLAGS.no_save = True |
21 | - if not FLAGS.pidfile: |
22 | - FLAGS.pidfile = '%s.pid' % name |
23 | - elif FLAGS.pidfile.endswith('twistd.pid'): |
24 | - FLAGS.pidfile = FLAGS.pidfile.replace('twistd.pid', '%s.pid' % name) |
25 | - # NOTE(vish): if we're running nodaemon, redirect the log to stdout |
26 | - if FLAGS.nodaemon and not FLAGS.logfile: |
27 | - FLAGS.logfile = "-" |
28 | - if not FLAGS.logfile: |
29 | - FLAGS.logfile = '%s.log' % name |
30 | - elif FLAGS.logfile.endswith('twistd.log'): |
31 | - FLAGS.logfile = FLAGS.logfile.replace('twistd.log', '%s.log' % name) |
32 | - if not FLAGS.prefix: |
33 | - FLAGS.prefix = name |
34 | - elif FLAGS.prefix.endswith('twisted'): |
35 | - FLAGS.prefix = FLAGS.prefix.replace('twisted', name) |
36 | + if sys.platform != 'win32': |
37 | + if not FLAGS.pidfile: |
38 | + FLAGS.pidfile = '%s.pid' % name |
39 | + elif FLAGS.pidfile.endswith('twistd.pid'): |
40 | + FLAGS.pidfile = FLAGS.pidfile.replace('twistd.pid', '%s.pid' % name) |
41 | + # NOTE(vish): if we're running nodaemon, redirect the log to stdout |
42 | + if FLAGS.nodaemon and not FLAGS.logfile: |
43 | + FLAGS.logfile = "-" |
44 | + if not FLAGS.logfile: |
45 | + FLAGS.logfile = '%s.log' % name |
46 | + elif FLAGS.logfile.endswith('twistd.log'): |
47 | + FLAGS.logfile = FLAGS.logfile.replace('twistd.log', '%s.log' % name) |
48 | + if not FLAGS.prefix: |
49 | + FLAGS.prefix = name |
50 | + elif FLAGS.prefix.endswith('twisted'): |
51 | + FLAGS.prefix = FLAGS.prefix.replace('twisted', name) |
52 | |
53 | action = 'start' |
54 | if len(argv) > 1: |
55 | |
56 | === modified file 'nova/virt/connection.py' |
57 | --- nova/virt/connection.py 2010-08-30 13:19:14 +0000 |
58 | +++ nova/virt/connection.py 2010-10-13 07:08:43 +0000 |
59 | @@ -26,6 +26,7 @@ |
60 | from nova.virt import fake |
61 | from nova.virt import libvirt_conn |
62 | from nova.virt import xenapi |
63 | +from nova.virt import hyperv |
64 | |
65 | |
66 | FLAGS = flags.FLAGS |
67 | @@ -49,6 +50,8 @@ |
68 | conn = libvirt_conn.get_connection(read_only) |
69 | elif t == 'xenapi': |
70 | conn = xenapi.get_connection(read_only) |
71 | + elif t == 'hyperv': |
72 | + conn = hyperv.get_connection(read_only) |
73 | else: |
74 | raise Exception('Unknown connection type "%s"' % t) |
75 | |
76 | |
77 | === added file 'nova/virt/hyperv.py' |
78 | --- nova/virt/hyperv.py 1970-01-01 00:00:00 +0000 |
79 | +++ nova/virt/hyperv.py 2010-10-13 07:08:43 +0000 |
80 | @@ -0,0 +1,453 @@ |
81 | +# vim: tabstop=4 shiftwidth=4 softtabstop=4 |
82 | + |
83 | +# Copyright (c) 2010 Cloud.com, Inc |
84 | +# |
85 | +# Licensed under the Apache License, Version 2.0 (the "License"); you may |
86 | +# not use this file except in compliance with the License. You may obtain |
87 | +# a copy of the License at |
88 | +# |
89 | +# http://www.apache.org/licenses/LICENSE-2.0 |
90 | +# |
91 | +# Unless required by applicable law or agreed to in writing, software |
92 | +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT |
93 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
94 | +# License for the specific language governing permissions and limitations |
95 | +# under the License. |
96 | + |
97 | +""" |
98 | +A connection to Hyper-V . |
99 | +Uses Windows Management Instrumentation (WMI) calls to interact with Hyper-V |
100 | +Hyper-V WMI usage: |
101 | + http://msdn.microsoft.com/en-us/library/cc723875%28v=VS.85%29.aspx |
102 | +The Hyper-V object model briefly: |
103 | + The physical computer and its hosted virtual machines are each represented |
104 | + by the Msvm_ComputerSystem class. |
105 | + |
106 | + Each virtual machine is associated with a |
107 | + Msvm_VirtualSystemGlobalSettingData (vs_gs_data) instance and one or more |
108 | + Msvm_VirtualSystemSettingData (vmsetting) instances. For each vmsetting |
109 | + there is a series of Msvm_ResourceAllocationSettingData (rasd) objects. |
110 | + The rasd objects describe the settings for each device in a virtual machine. |
111 | + Together, the vs_gs_data, vmsettings and rasds describe the configuration |
112 | + of the virtual machine. |
113 | + |
114 | + Creating new resources such as disks and nics involves cloning a default |
115 | + rasd object and appropriately modifying the clone and calling the |
116 | + AddVirtualSystemResources WMI method |
117 | + Changing resources such as memory uses the ModifyVirtualSystemResources |
118 | + WMI method |
119 | + |
120 | +Using the Python WMI library: |
121 | + Tutorial: |
122 | + http://timgolden.me.uk/python/wmi/tutorial.html |
123 | + Hyper-V WMI objects can be retrieved simply by using the class name |
124 | + of the WMI object and optionally specifying a column to filter the |
125 | + result set. More complex filters can be formed using WQL (sql-like) |
126 | + queries. |
127 | + The parameters and return tuples of WMI method calls can gleaned by |
128 | + examining the doc string. For example: |
129 | + >>> vs_man_svc.ModifyVirtualSystemResources.__doc__ |
130 | + ModifyVirtualSystemResources (ComputerSystem, ResourceSettingData[]) |
131 | + => (Job, ReturnValue)' |
132 | + When passing setting data (ResourceSettingData) to the WMI method, |
133 | + an XML representation of the data is passed in using the GetText_(1) method. |
134 | + Available methods on a service can be determined using method.keys(): |
135 | + >>> vs_man_svc.methods.keys() |
136 | + vmsettings and rasds for a vm can be retrieved using the 'associators' |
137 | + method with the appropriate return class. |
138 | + Long running WMI commands generally return a Job (an instance of |
139 | + Msvm_ConcreteJob) whose state can be polled to determine when it finishes |
140 | + |
141 | +""" |
142 | + |
143 | +import os |
144 | +import logging |
145 | +import time |
146 | + |
147 | +from twisted.internet import defer |
148 | +import wmi |
149 | + |
150 | +from nova import exception |
151 | +from nova import flags |
152 | +from nova.auth.manager import AuthManager |
153 | +from nova.compute import power_state |
154 | +from nova.virt import images |
155 | + |
156 | + |
157 | +FLAGS = flags.FLAGS |
158 | + |
159 | + |
160 | +HYPERV_POWER_STATE = { |
161 | + 3 : power_state.SHUTDOWN, |
162 | + 2 : power_state.RUNNING, |
163 | + 32768 : power_state.PAUSED, |
164 | +} |
165 | + |
166 | + |
167 | +REQ_POWER_STATE = { |
168 | + 'Enabled' : 2, |
169 | + 'Disabled': 3, |
170 | + 'Reboot' : 10, |
171 | + 'Reset' : 11, |
172 | + 'Paused' : 32768, |
173 | + 'Suspended': 32769 |
174 | +} |
175 | + |
176 | + |
177 | +WMI_JOB_STATUS_STARTED = 4096 |
178 | +WMI_JOB_STATE_RUNNING = 4 |
179 | +WMI_JOB_STATE_COMPLETED = 7 |
180 | + |
181 | + |
182 | +def get_connection(_): |
183 | + return HyperVConnection() |
184 | + |
185 | + |
186 | +class HyperVConnection(object): |
187 | + def __init__(self): |
188 | + self._conn = wmi.WMI(moniker = '//./root/virtualization') |
189 | + self._cim_conn = wmi.WMI(moniker = '//./root/cimv2') |
190 | + |
191 | + def list_instances(self): |
192 | + """ Return the names of all the instances known to Hyper-V. """ |
193 | + vms = [v.ElementName \ |
194 | + for v in self._conn.Msvm_ComputerSystem(['ElementName'])] |
195 | + return vms |
196 | + |
197 | + @defer.inlineCallbacks |
198 | + def spawn(self, instance): |
199 | + """ Create a new VM and start it.""" |
200 | + vm = yield self._lookup(instance.name) |
201 | + if vm is not None: |
202 | + raise exception.Duplicate('Attempted to create duplicate name %s' % |
203 | + instance.name) |
204 | + |
205 | + user = AuthManager().get_user(instance['user_id']) |
206 | + project = AuthManager().get_project(instance['project_id']) |
207 | + #Fetch the file, assume it is a VHD file. |
208 | + vhdfile = os.path.join(FLAGS.instances_path, instance['str_id'])+".vhd" |
209 | + yield images.fetch(instance['image_id'], vhdfile, user, project) |
210 | + |
211 | + try: |
212 | + yield self._create_vm(instance) |
213 | + |
214 | + yield self._create_disk(instance['name'], vhdfile) |
215 | + yield self._create_nic(instance['name'], instance['mac_address']) |
216 | + |
217 | + logging.debug ('Starting VM %s ', instance.name) |
218 | + yield self._set_vm_state(instance['name'], 'Enabled') |
219 | + logging.info('Started VM %s ', instance.name) |
220 | + except Exception as exn: |
221 | + logging.error('spawn vm failed: %s', exn) |
222 | + self.destroy(instance) |
223 | + |
224 | + def _create_vm(self, instance): |
225 | + """Create a VM but don't start it. """ |
226 | + vs_man_svc = self._conn.Msvm_VirtualSystemManagementService()[0] |
227 | + |
228 | + vs_gs_data = self._conn.Msvm_VirtualSystemGlobalSettingData.new() |
229 | + vs_gs_data.ElementName = instance['name'] |
230 | + (job, ret_val) = vs_man_svc.DefineVirtualSystem( |
231 | + [], None, vs_gs_data.GetText_(1))[1:] |
232 | + if ret_val == WMI_JOB_STATUS_STARTED: |
233 | + success = self._check_job_status(job) |
234 | + else: |
235 | + success = (ret_val == 0) |
236 | + |
237 | + if not success: |
238 | + raise Exception('Failed to create VM %s', instance.name) |
239 | + |
240 | + logging.debug('Created VM %s...', instance.name) |
241 | + vm = self._conn.Msvm_ComputerSystem (ElementName=instance.name)[0] |
242 | + |
243 | + vmsettings = vm.associators(wmi_result_class= |
244 | + 'Msvm_VirtualSystemSettingData') |
245 | + vmsetting = [s for s in vmsettings |
246 | + if s.SettingType == 3][0] #avoid snapshots |
247 | + memsetting = vmsetting.associators(wmi_result_class= |
248 | + 'Msvm_MemorySettingData')[0] |
249 | + #No Dynamic Memory, so reservation, limit and quantity are identical. |
250 | + mem = long(str(instance['memory_mb'])) |
251 | + memsetting.VirtualQuantity = mem |
252 | + memsetting.Reservation = mem |
253 | + memsetting.Limit = mem |
254 | + |
255 | + (job, ret_val) = vs_man_svc.ModifyVirtualSystemResources( |
256 | + vm.path_(), [memsetting.GetText_(1)]) |
257 | + |
258 | + logging.debug('Set memory for vm %s...', instance.name) |
259 | + procsetting = vmsetting.associators(wmi_result_class= |
260 | + 'Msvm_ProcessorSettingData')[0] |
261 | + vcpus = long(instance['vcpus']) |
262 | + procsetting.VirtualQuantity = vcpus |
263 | + procsetting.Reservation = vcpus |
264 | + procsetting.Limit = vcpus |
265 | + |
266 | + (job, ret_val) = vs_man_svc.ModifyVirtualSystemResources( |
267 | + vm.path_(), [procsetting.GetText_(1)]) |
268 | + |
269 | + logging.debug('Set vcpus for vm %s...', instance.name) |
270 | + |
271 | + def _create_disk(self, vm_name, vhdfile): |
272 | + """Create a disk and attach it to the vm""" |
273 | + logging.debug("Creating disk for %s by attaching disk file %s", \ |
274 | + vm_name, vhdfile) |
275 | + #Find the IDE controller for the vm. |
276 | + vms = self._conn.MSVM_ComputerSystem (ElementName=vm_name) |
277 | + vm = vms[0] |
278 | + vmsettings = vm.associators( |
279 | + wmi_result_class='Msvm_VirtualSystemSettingData') |
280 | + rasds = vmsettings[0].associators( |
281 | + wmi_result_class='MSVM_ResourceAllocationSettingData') |
282 | + ctrller = [r for r in rasds |
283 | + if r.ResourceSubType == 'Microsoft Emulated IDE Controller'\ |
284 | + and r.Address == "0" ] |
285 | + #Find the default disk drive object for the vm and clone it. |
286 | + diskdflt = self._conn.query( |
287 | + "SELECT * FROM Msvm_ResourceAllocationSettingData \ |
288 | + WHERE ResourceSubType LIKE 'Microsoft Synthetic Disk Drive'\ |
289 | + AND InstanceID LIKE '%Default%'")[0] |
290 | + diskdrive = self._clone_wmi_obj( |
291 | + 'Msvm_ResourceAllocationSettingData', diskdflt) |
292 | + #Set the IDE ctrller as parent. |
293 | + diskdrive.Parent = ctrller[0].path_() |
294 | + diskdrive.Address = 0 |
295 | + #Add the cloned disk drive object to the vm. |
296 | + new_resources = self._add_virt_resource(diskdrive, vm) |
297 | + |
298 | + if new_resources is None: |
299 | + raise Exception('Failed to add diskdrive to VM %s', vm_name) |
300 | + |
301 | + diskdrive_path = new_resources[0] |
302 | + logging.debug("New disk drive path is " + diskdrive_path) |
303 | + #Find the default VHD disk object. |
304 | + vhddefault = self._conn.query( |
305 | + "SELECT * FROM Msvm_ResourceAllocationSettingData \ |
306 | + WHERE ResourceSubType LIKE 'Microsoft Virtual Hard Disk' AND \ |
307 | + InstanceID LIKE '%Default%' ")[0] |
308 | + |
309 | + #Clone the default and point it to the image file. |
310 | + vhddisk = self._clone_wmi_obj( |
311 | + 'Msvm_ResourceAllocationSettingData', vhddefault) |
312 | + #Set the new drive as the parent. |
313 | + vhddisk.Parent = diskdrive_path |
314 | + vhddisk.Connection = [vhdfile] |
315 | + |
316 | + #Add the new vhd object as a virtual hard disk to the vm. |
317 | + new_resources = self._add_virt_resource(vhddisk, vm) |
318 | + if new_resources is None: |
319 | + raise Exception('Failed to add vhd file to VM %s', vm_name) |
320 | + logging.info("Created disk for %s ", vm_name) |
321 | + |
322 | + def _create_nic(self, vm_name, mac): |
323 | + """Create a (emulated) nic and attach it to the vm""" |
324 | + logging.debug("Creating nic for %s ", vm_name) |
325 | + #Find the vswitch that is connected to the physical nic. |
326 | + vms = self._conn.Msvm_ComputerSystem (ElementName=vm_name) |
327 | + extswitch = self._find_external_network() |
328 | + vm = vms[0] |
329 | + switch_svc = self._conn.Msvm_VirtualSwitchManagementService ()[0] |
330 | + #Find the default nic and clone it to create a new nic for the vm. |
331 | + #Use Msvm_SyntheticEthernetPortSettingData for Windows VMs or Linux with |
332 | + #Linux Integration Components installed. |
333 | + emulatednics_data = self._conn.Msvm_EmulatedEthernetPortSettingData() |
334 | + default_nic_data = [n for n in emulatednics_data |
335 | + if n.InstanceID.rfind('Default') >0 ] |
336 | + new_nic_data = self._clone_wmi_obj( |
337 | + 'Msvm_EmulatedEthernetPortSettingData', |
338 | + default_nic_data[0]) |
339 | + |
340 | + #Create a port on the vswitch. |
341 | + (created_sw, ret_val) = switch_svc.CreateSwitchPort(vm_name, vm_name, |
342 | + "", extswitch.path_()) |
343 | + if ret_val != 0: |
344 | + logging.debug("Failed to create a new port on the external network") |
345 | + return |
346 | + logging.debug("Created switch port %s on switch %s", |
347 | + vm_name, extswitch.path_()) |
348 | + #Connect the new nic to the new port. |
349 | + new_nic_data.Connection = [created_sw] |
350 | + new_nic_data.ElementName = vm_name + ' nic' |
351 | + new_nic_data.Address = ''.join(mac.split(':')) |
352 | + new_nic_data.StaticMacAddress = 'TRUE' |
353 | + #Add the new nic to the vm. |
354 | + new_resources = self._add_virt_resource(new_nic_data, vm) |
355 | + if new_resources is None: |
356 | + raise Exception('Failed to add nic to VM %s', vm_name) |
357 | + logging.info("Created nic for %s ", vm_name) |
358 | + |
359 | + def _add_virt_resource(self, res_setting_data, target_vm): |
360 | + """Add a new resource (disk/nic) to the VM""" |
361 | + vs_man_svc = self._conn.Msvm_VirtualSystemManagementService()[0] |
362 | + (job, new_resources, return_val) = vs_man_svc.\ |
363 | + AddVirtualSystemResources([res_setting_data.GetText_(1)], |
364 | + target_vm.path_()) |
365 | + success = True |
366 | + if return_val == WMI_JOB_STATUS_STARTED: |
367 | + success = self._check_job_status(job) |
368 | + else: |
369 | + success = (return_val == 0) |
370 | + if success: |
371 | + return new_resources |
372 | + else: |
373 | + return None |
374 | + |
375 | + #TODO: use the reactor to poll instead of sleep |
376 | + def _check_job_status(self, jobpath): |
377 | + """Poll WMI job state for completion""" |
378 | + inst_id = jobpath.split(':')[1].split('=')[1].strip('\"') |
379 | + jobs = self._conn.Msvm_ConcreteJob(InstanceID=inst_id) |
380 | + if len(jobs) == 0: |
381 | + return False |
382 | + job = jobs[0] |
383 | + while job.JobState == WMI_JOB_STATE_RUNNING: |
384 | + time.sleep(0.1) |
385 | + job = self._conn.Msvm_ConcreteJob(InstanceID=inst_id)[0] |
386 | + |
387 | + if job.JobState != WMI_JOB_STATE_COMPLETED: |
388 | + logging.debug("WMI job failed: " + job.ErrorSummaryDescription) |
389 | + return False |
390 | + |
391 | + logging.debug("WMI job succeeded: " + job.Description + ",Elapsed = " \ |
392 | + + job.ElapsedTime) |
393 | + |
394 | + return True |
395 | + |
396 | + def _find_external_network(self): |
397 | + """Find the vswitch that is connected to the physical nic. |
398 | + Assumes only one physical nic on the host |
399 | + """ |
400 | + #If there are no physical nics connected to networks, return. |
401 | + bound = self._conn.Msvm_ExternalEthernetPort(IsBound='TRUE') |
402 | + if len(bound) == 0: |
403 | + return None |
404 | + |
405 | + return self._conn.Msvm_ExternalEthernetPort(IsBound='TRUE')[0]\ |
406 | + .associators(wmi_result_class='Msvm_SwitchLANEndpoint')[0]\ |
407 | + .associators(wmi_result_class='Msvm_SwitchPort')[0]\ |
408 | + .associators(wmi_result_class='Msvm_VirtualSwitch')[0] |
409 | + |
410 | + def _clone_wmi_obj(self, wmi_class, wmi_obj): |
411 | + """Clone a WMI object""" |
412 | + cl = self._conn.__getattr__(wmi_class) #get the class |
413 | + newinst = cl.new() |
414 | + #Copy the properties from the original. |
415 | + for prop in wmi_obj._properties: |
416 | + newinst.Properties_.Item(prop).Value =\ |
417 | + wmi_obj.Properties_.Item(prop).Value |
418 | + return newinst |
419 | + |
420 | + @defer.inlineCallbacks |
421 | + def reboot(self, instance): |
422 | + """Reboot the specified instance.""" |
423 | + vm = yield self._lookup(instance.name) |
424 | + if vm is None: |
425 | + raise exception.NotFound('instance not present %s' % instance.name) |
426 | + self._set_vm_state(instance.name, 'Reboot') |
427 | + |
428 | + @defer.inlineCallbacks |
429 | + def destroy(self, instance): |
430 | + """Destroy the VM. Also destroy the associated VHD disk files""" |
431 | + logging.debug("Got request to destroy vm %s", instance.name) |
432 | + vm = yield self._lookup(instance.name) |
433 | + if vm is None: |
434 | + defer.returnValue(None) |
435 | + vm = self._conn.Msvm_ComputerSystem (ElementName=instance.name)[0] |
436 | + vs_man_svc = self._conn.Msvm_VirtualSystemManagementService()[0] |
437 | + #Stop the VM first. |
438 | + self._set_vm_state(instance.name, 'Disabled') |
439 | + vmsettings = vm.associators(wmi_result_class= |
440 | + 'Msvm_VirtualSystemSettingData') |
441 | + rasds = vmsettings[0].associators(wmi_result_class= |
442 | + 'MSVM_ResourceAllocationSettingData') |
443 | + disks = [r for r in rasds \ |
444 | + if r.ResourceSubType == 'Microsoft Virtual Hard Disk' ] |
445 | + diskfiles = [] |
446 | + #Collect disk file information before destroying the VM. |
447 | + for disk in disks: |
448 | + diskfiles.extend([c for c in disk.Connection]) |
449 | + #Nuke the VM. Does not destroy disks. |
450 | + (job, ret_val) = vs_man_svc.DestroyVirtualSystem(vm.path_()) |
451 | + if ret_val == WMI_JOB_STATUS_STARTED: |
452 | + success = self._check_job_status(job) |
453 | + elif ret_val == 0: |
454 | + success = True |
455 | + if not success: |
456 | + raise Exception('Failed to destroy vm %s' % instance.name) |
457 | + #Delete associated vhd disk files. |
458 | + for disk in diskfiles: |
459 | + vhdfile = self._cim_conn.CIM_DataFile(Name=disk) |
460 | + for vf in vhdfile: |
461 | + vf.Delete() |
462 | + logging.debug("Deleted disk %s vm %s", vhdfile, instance.name) |
463 | + |
464 | + def get_info(self, instance_id): |
465 | + """Get information about the VM""" |
466 | + vm = self._lookup(instance_id) |
467 | + if vm is None: |
468 | + raise exception.NotFound('instance not present %s' % instance_id) |
469 | + vm = self._conn.Msvm_ComputerSystem(ElementName=instance_id)[0] |
470 | + vs_man_svc = self._conn.Msvm_VirtualSystemManagementService()[0] |
471 | + vmsettings = vm.associators(wmi_result_class= |
472 | + 'Msvm_VirtualSystemSettingData') |
473 | + settings_paths = [ v.path_() for v in vmsettings] |
474 | + #See http://msdn.microsoft.com/en-us/library/cc160706%28VS.85%29.aspx |
475 | + summary_info = vs_man_svc.GetSummaryInformation( |
476 | + [4,100,103,105], settings_paths)[1] |
477 | + info = summary_info[0] |
478 | + logging.debug("Got Info for vm %s: state=%s, mem=%s, num_cpu=%s, \ |
479 | + cpu_time=%s", instance_id, |
480 | + str(HYPERV_POWER_STATE[info.EnabledState]), |
481 | + str(info.MemoryUsage), |
482 | + str(info.NumberOfProcessors), |
483 | + str(info.UpTime)) |
484 | + |
485 | + return {'state': HYPERV_POWER_STATE[info.EnabledState], |
486 | + 'max_mem': info.MemoryUsage, |
487 | + 'mem': info.MemoryUsage, |
488 | + 'num_cpu': info.NumberOfProcessors, |
489 | + 'cpu_time': info.UpTime} |
490 | + |
491 | + def _lookup(self, i): |
492 | + vms = self._conn.Msvm_ComputerSystem (ElementName=i) |
493 | + n = len(vms) |
494 | + if n == 0: |
495 | + return None |
496 | + elif n > 1: |
497 | + raise Exception('duplicate name found: %s' % i) |
498 | + else: |
499 | + return vms[0].ElementName |
500 | + |
501 | + def _set_vm_state(self, vm_name, req_state): |
502 | + """Set the desired state of the VM""" |
503 | + vms = self._conn.Msvm_ComputerSystem (ElementName=vm_name) |
504 | + if len(vms) == 0: |
505 | + return False |
506 | + status = vms[0].RequestStateChange(REQ_POWER_STATE[req_state]) |
507 | + job = status[0] |
508 | + return_val = status[1] |
509 | + if return_val == WMI_JOB_STATUS_STARTED: |
510 | + success = self._check_job_status(job) |
511 | + elif return_val == 0: |
512 | + success = True |
513 | + if success: |
514 | + logging.info("Successfully changed vm state of %s to %s", |
515 | + vm_name, req_state) |
516 | + return True |
517 | + else: |
518 | + logging.debug("Failed to change vm state of %s to %s", |
519 | + vm_name, req_state) |
520 | + return False |
521 | + |
522 | + def attach_volume(self, instance_name, device_path, mountpoint): |
523 | + vm = self._lookup(instance_name) |
524 | + if vm is None: |
525 | + raise exception.NotFound('Cannot attach volume to missing %s vm' % |
526 | + instance_name) |
527 | + |
528 | + def detach_volume(self, instance_name, mountpoint): |
529 | + vm = self._lookup(instance_name) |
530 | + if vm is None: |
531 | + raise exception.NotFound('Cannot detach volume from missing %s ' % |
532 | + instance_name) |
533 | + |
534 | |
535 | === modified file 'nova/virt/images.py' |
536 | --- nova/virt/images.py 2010-10-07 14:03:43 +0000 |
537 | +++ nova/virt/images.py 2010-10-13 07:08:43 +0000 |
538 | @@ -21,15 +21,19 @@ |
539 | Handling of VM disk images. |
540 | """ |
541 | |
542 | +import logging |
543 | +import os |
544 | import os.path |
545 | +import shutil |
546 | +import sys |
547 | import time |
548 | +import urllib2 |
549 | import urlparse |
550 | |
551 | from nova import flags |
552 | from nova import process |
553 | from nova.auth import manager |
554 | from nova.auth import signer |
555 | -from nova.objectstore import image |
556 | |
557 | |
558 | FLAGS = flags.FLAGS |
559 | @@ -45,8 +49,27 @@ |
560 | return f(image, path, user, project) |
561 | |
562 | |
563 | +def _fetch_image_no_curl(url, path, headers): |
564 | + request = urllib2.Request(url) |
565 | + for (k, v) in headers.iteritems(): |
566 | + request.add_header(k, v) |
567 | + |
568 | + def urlretrieve(urlfile, fpath): |
569 | + chunk = 1*1024*1024 |
570 | + f = open(fpath, "wb") |
571 | + while 1: |
572 | + data = urlfile.read(chunk) |
573 | + if not data: |
574 | + break |
575 | + f.write(data) |
576 | + |
577 | + urlopened = urllib2.urlopen(request) |
578 | + urlretrieve(urlopened, path) |
579 | + logging.debug("Finished retreving %s -- placed in %s", url, path) |
580 | + |
581 | def _fetch_s3_image(image, path, user, project): |
582 | url = image_url(image) |
583 | + logging.debug("About to retrieve %s and place it in %s", url, path) |
584 | |
585 | # This should probably move somewhere else, like e.g. a download_as |
586 | # method on User objects and at the same time get rewritten to use |
587 | @@ -61,18 +84,22 @@ |
588 | url_path) |
589 | headers['Authorization'] = 'AWS %s:%s' % (access, signature) |
590 | |
591 | - cmd = ['/usr/bin/curl', '--fail', '--silent', url] |
592 | - for (k,v) in headers.iteritems(): |
593 | - cmd += ['-H', '%s: %s' % (k,v)] |
594 | - |
595 | - cmd += ['-o', path] |
596 | - return process.SharedPool().execute(executable=cmd[0], args=cmd[1:]) |
597 | - |
598 | + if sys.platform.startswith('win'): |
599 | + return _fetch_image_no_curl(url, path, headers) |
600 | + else: |
601 | + cmd = ['/usr/bin/curl', '--fail', '--silent', url] |
602 | + for (k,v) in headers.iteritems(): |
603 | + cmd += ['-H', '%s: %s' % (k,v)] |
604 | + cmd += ['-o', path] |
605 | + return process.SharedPool().execute(executable=cmd[0], args=cmd[1:]) |
606 | |
607 | def _fetch_local_image(image, path, user, project): |
608 | - source = _image_path('%s/image' % image) |
609 | - return process.simple_execute('cp %s %s' % (source, path)) |
610 | - |
611 | + source = _image_path(os.path.join(image,'image')) |
612 | + logging.debug("About to copy %s to %s", source, path) |
613 | + if sys.platform.startswith('win'): |
614 | + return shutil.copy(source, path) |
615 | + else: |
616 | + return process.simple_execute('cp %s %s' % (source, path)) |
617 | |
618 | def _image_path(path): |
619 | return os.path.join(FLAGS.images_path, path) |
Hi Chiradeep!
Thanks for the contribution and a start of Windows support in OpenStack! :)
Here are my review notes for you.
1) Documentation is sparse :)
It would be excellent to get some overall design and module documentation for your work. :) This will help those coders in the future who are maintaining and enhancing the hyper-V work. Please see Ewan Mellor's excellent documentation in the xenapi virtualization layer for an example of the kind of documentation that is helpful for maintainers.
This documentation is, IMHO, pretty critical considering the sorry state of python-wmi documentation regarding virtual environments.
Additional code comments would be very useful in places such as the following:
181 + (job, ret_val) = vs_man_ svc.DefineVirtu alSystem( GetText_ (1))[1: ]
182 + [], None, vs_gs_data.
What does vs_gs_data. GetText( 1) return? What does sv_man_ svc.DefineVirtu alSystem( ) return? Without understanding these types of things, maintaining developers have zero chance of successfully fixing bugs that may occur :)
2) Please use Nova's exceptions, which are tested for in the test suites and programs, instead of generic Exception:
402 + raise Exception('instance not present %s' % instance_id)
Should be replaced with:
from nova import exception NotFound( 'instance not found %s' % instance_id)
...
raise exception.
same for here:
360 + if vm is None:
361 + raise Exception('instance not present %s' % instance.name)
3) Your rewrite of the _fetch_s3_image() function is good, however, it removes the efficiency of using a shared process pool in Linux/BSD environments in favour of cross-platform compatibility. It would be nice to keep the existing code and use your rewrite in an if sys.platform. startswith( 'win'): block.
============
And here are some style-related things to fix up...
1) Please make sure imports are ordered according to the instructions in the HACKING file. In particular, please make sure system lib imports are first, in alphabetical order, followed by a newline, then imports of 3rd-party libraries (like wmi) in alphabetical order, followed by a newline and imports of nova modules.
2) Something is amiss here:
118 +HYPERV_POWER_STATE = { SHUTDOWN, RUNNING,
119 + 3 : power_state.
120 + 2 : power_state.
121 + 32768 : power_state.PAUSED,
122 + 32768: power_state.PAUSED, # TODO
123 + 3 : power_state.CRASHED
124 +}
The indexes 3 and 32768 are doubled up.
3) Define constants for "magic numbers"
183 + if (ret_val == 4096 ): #WMI job started
Feel free to either create a constant WMI_JOB_STARTED and use that instead of the number 4096 and putting a comment at the end of the line. Self-documenting constants++ :)
Same goes for these locations:
323 + while job.JobState == 4: #job started
327 + if (job.JobState != 7): #job success
385 + if (ret_val == 4096 ): #WMI job started
442 + if (return_val == 4096 ): #WMI job started
4) This looks hacky :)
212 + vcpus = long(str( instance[ 'vcpus' ]))
I recommend just:
vcpus = long(instance[ 'vcpus' ])
5) Please remove extraneous commented-out debugging stuff:
213 + #vcpus = 1
6) Python != C ;)
Please remove outer extraneous parentheses:
286 + if (ret_va...