Merge lp:~sandy-walsh/nova/api-parity into lp:~hudson-openstack/nova/trunk

Proposed by Sandy Walsh
Status: Merged
Approved by: Matt Dietz
Approved revision: 533
Merged at revision: 545
Proposed branch: lp:~sandy-walsh/nova/api-parity
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 469 lines (+125/-20)
15 files modified
krm_mapping.json.sample (+3/-0)
nova/api/openstack/__init__.py (+8/-3)
nova/api/openstack/backup_schedules.py (+4/-2)
nova/api/openstack/common.py (+24/-0)
nova/api/openstack/images.py (+18/-1)
nova/api/openstack/servers.py (+27/-2)
nova/api/openstack/shared_ip_groups.py (+6/-4)
nova/auth/manager.py (+9/-5)
nova/auth/novarc.template (+4/-0)
nova/compute/api.py (+4/-1)
nova/flags.py (+2/-0)
nova/tests/api/openstack/test_images.py (+2/-0)
nova/tests/api/openstack/test_servers.py (+10/-0)
nova/tests/api/openstack/test_shared_ip_groups.py (+1/-1)
nova/virt/xenapi/vm_utils.py (+3/-1)
To merge this branch: bzr merge lp:~sandy-walsh/nova/api-parity
Reviewer Review Type Date Requested Status
Cory Wright (community) Approve
Matt Dietz (community) Approve
Trey Morris (community) Approve
Eric Day (community) Needs Information
Review via email: mp+45371@code.launchpad.net

Description of the change

Read Full Spec for implementation details and notes on how to boot an instance using OS API.
http://etherpad.openstack.org/B2RK0q1CYj

Look at these notes for known issues:
http://etherpad.openstack.org/BXOU0TTj9M

To post a comment you must log in.
Revision history for this message
Cory Wright (corywright) wrote :

Looks like you have a few conflicts in this commit.

review: Needs Fixing
Revision history for this message
Sandy Walsh (sandy-walsh) wrote :

Conflicts resolved, tests run. Thx dubs!

Revision history for this message
Trey Morris (tr3buchet) wrote :

I like it! exporting the cloudservers environment variables was a great idea.

question about lines 190-191, I thought the current api didn't have a paused state, wouldn't returning paused cause trouble? Pause isn't implemented in the current api so i chose to set it to "error" because if an instance somehow got "paused" something weird would have happened.

review: Approve
Revision history for this message
Sandy Walsh (sandy-walsh) wrote :

Trey I went back and forth over that decision for a while. It just looked so weird seeing "error" when it was paused. I figured since other clients are unable to issue a "pause" command, they should never see it.

But I see your point and can live with it either way.

Revision history for this message
Eric Day (eday) wrote :

367/368: is this needed now that instance is returned as a dict?

Should sharedipgroups.py be renamed to shared_ip_groups.py for consistency?

I don't like the 'grab all and find w/ hash' on image ID as I mentioned in IRC, but hopefully this will get better with Glance. :)

review: Needs Information
Revision history for this message
Sandy Walsh (sandy-walsh) wrote :

good point ... I will rename.

I've got a tweet out looking for a two-way function. Good idea.

Revision history for this message
Sandy Walsh (sandy-walsh) wrote :

Renamed, remerged with trunk and fixed a bug in the display_name patch test.

Revision history for this message
Sandy Walsh (sandy-walsh) wrote :

Hmm, this branch is busted now after the trunk merge until the instance_id thing is straightened out.

Revision history for this message
Sandy Walsh (sandy-walsh) wrote :

Branch working again.

Eric, as your question about "not in", I'm not getting a dict back, but a sqlalchemy object (verified).

Revision history for this message
Trey Morris (tr3buchet) wrote :

I like your point of view on pause/error sandy. Let's definitely go with pause.

review: Approve
Revision history for this message
Matt Dietz (cerberus) wrote :

284-285: We should be consistent with the /controller/detail calls. If you look at openstack/__init__.py detail is a sub-resource of server, with an implemented verb in the controller itself.

review: Needs Fixing
lp:~sandy-walsh/nova/api-parity updated
531. By Sandy Walsh

another merge with trunk to remedy instance_id issues

532. By Sandy Walsh

Changed shared_ip_group detail routing

533. By Sandy Walsh

Changed shared_ip_group detail routing

Revision history for this message
Sandy Walsh (sandy-walsh) wrote :

> 284-285: We should be consistent with the /controller/detail calls. If you
> look at openstack/__init__.py detail is a sub-resource of server, with an
> implemented verb in the controller itself.

All done sir. Good suggestion!

Revision history for this message
Matt Dietz (cerberus) wrote :

\o/

review: Approve
Revision history for this message
Cory Wright (corywright) wrote :

Looks good.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'krm_mapping.json.sample'
2--- krm_mapping.json.sample 1970-01-01 00:00:00 +0000
3+++ krm_mapping.json.sample 2011-01-11 19:42:14 +0000
4@@ -0,0 +1,3 @@
5+{
6+ "machine" : ["kernel", "ramdisk"]
7+}
8
9=== modified file 'nova/api/openstack/__init__.py'
10--- nova/api/openstack/__init__.py 2011-01-10 17:37:06 +0000
11+++ nova/api/openstack/__init__.py 2011-01-11 19:42:14 +0000
12@@ -35,7 +35,7 @@
13 from nova.api.openstack import flavors
14 from nova.api.openstack import images
15 from nova.api.openstack import servers
16-from nova.api.openstack import sharedipgroups
17+from nova.api.openstack import shared_ip_groups
18
19
20 LOG = logging.getLogger('nova.api.openstack')
21@@ -48,6 +48,10 @@
22 'nova.api.openstack.ratelimiting.RateLimitingMiddleware',
23 'Default ratelimiting implementation for the Openstack API')
24
25+flags.DEFINE_string('os_krm_mapping_file',
26+ 'krm_mapping.json',
27+ 'Location of OpenStack Flavor/OS:EC2 Kernel/Ramdisk/Machine JSON file.')
28+
29 flags.DEFINE_bool('allow_admin_api',
30 False,
31 'When True, this API service will accept admin operations.')
32@@ -110,8 +114,9 @@
33 collection={'detail': 'GET'})
34 mapper.resource("flavor", "flavors", controller=flavors.Controller(),
35 collection={'detail': 'GET'})
36- mapper.resource("sharedipgroup", "sharedipgroups",
37- controller=sharedipgroups.Controller())
38+ mapper.resource("shared_ip_group", "shared_ip_groups",
39+ collection={'detail': 'GET'},
40+ controller=shared_ip_groups.Controller())
41
42 super(APIRouter, self).__init__(mapper)
43
44
45=== modified file 'nova/api/openstack/backup_schedules.py'
46--- nova/api/openstack/backup_schedules.py 2010-12-28 21:48:48 +0000
47+++ nova/api/openstack/backup_schedules.py 2011-01-11 19:42:14 +0000
48@@ -15,7 +15,9 @@
49 # License for the specific language governing permissions and limitations
50 # under the License.
51
52+import logging
53 import time
54+
55 from webob import exc
56
57 from nova import wsgi
58@@ -46,8 +48,8 @@
59 def create(self, req, server_id):
60 """ No actual update method required, since the existing API allows
61 both create and update through a POST """
62- return faults.Fault(exc.HTTPNotFound())
63+ return faults.Fault(exc.HTTPNotImplemented())
64
65 def delete(self, req, server_id, id):
66 """ Deletes an existing backup schedule """
67- return faults.Fault(exc.HTTPNotFound())
68+ return faults.Fault(exc.HTTPNotImplemented())
69
70=== modified file 'nova/api/openstack/common.py'
71--- nova/api/openstack/common.py 2010-12-23 19:17:53 +0000
72+++ nova/api/openstack/common.py 2011-01-11 19:42:14 +0000
73@@ -15,6 +15,8 @@
74 # License for the specific language governing permissions and limitations
75 # under the License.
76
77+from nova import exception
78+
79
80 def limited(items, req):
81 """Return a slice of items according to requested offset and limit.
82@@ -34,3 +36,25 @@
83 limit = min(1000, limit)
84 range_end = offset + limit
85 return items[offset:range_end]
86+
87+
88+def get_image_id_from_image_hash(image_service, context, image_hash):
89+ """Given an Image ID Hash, return an objectstore Image ID.
90+
91+ image_service - reference to objectstore compatible image service.
92+ context - security context for image service requests.
93+ image_hash - hash of the image ID.
94+ """
95+
96+ # FIX(sandy): This is terribly inefficient. It pulls all images
97+ # from objectstore in order to find the match. ObjectStore
98+ # should have a numeric counterpart to the string ID.
99+ try:
100+ items = image_service.detail(context)
101+ except NotImplementedError:
102+ items = image_service.index(context)
103+ for image in items:
104+ image_id = image['imageId']
105+ if abs(hash(image_id)) == int(image_hash):
106+ return image_id
107+ raise exception.NotFound(image_hash)
108
109=== modified file 'nova/api/openstack/images.py'
110--- nova/api/openstack/images.py 2010-12-31 03:55:00 +0000
111+++ nova/api/openstack/images.py 2011-01-11 19:42:14 +0000
112@@ -15,6 +15,8 @@
113 # License for the specific language governing permissions and limitations
114 # under the License.
115
116+import logging
117+
118 from webob import exc
119
120 from nova import compute
121@@ -26,6 +28,7 @@
122 from nova.api.openstack import faults
123 import nova.image.service
124
125+
126 FLAGS = flags.FLAGS
127
128
129@@ -88,6 +91,12 @@
130 return dict((k, v) for k, v in item.iteritems() if k in keys)
131
132
133+def _convert_image_id_to_hash(image):
134+ image_id = abs(hash(image['imageId']))
135+ image['imageId'] = image_id
136+ image['id'] = image_id
137+
138+
139 class Controller(wsgi.Controller):
140
141 _serialization_metadata = {
142@@ -112,6 +121,9 @@
143 items = self._service.detail(req.environ['nova.context'])
144 except NotImplementedError:
145 items = self._service.index(req.environ['nova.context'])
146+ for image in items:
147+ _convert_image_id_to_hash(image)
148+
149 items = common.limited(items, req)
150 items = [_translate_keys(item) for item in items]
151 items = [_translate_status(item) for item in items]
152@@ -119,7 +131,12 @@
153
154 def show(self, req, id):
155 """Return data about the given image id"""
156- return dict(image=self._service.show(req.environ['nova.context'], id))
157+ image_id = common.get_image_id_from_image_hash(self._service,
158+ req.environ['nova.context'], id)
159+
160+ image = self._service.show(req.environ['nova.context'], image_id)
161+ _convert_image_id_to_hash(image)
162+ return dict(image=image)
163
164 def delete(self, req, id):
165 # Only public images are supported for now.
166
167=== modified file 'nova/api/openstack/servers.py'
168--- nova/api/openstack/servers.py 2011-01-07 14:46:17 +0000
169+++ nova/api/openstack/servers.py 2011-01-11 19:42:14 +0000
170@@ -15,14 +15,17 @@
171 # License for the specific language governing permissions and limitations
172 # under the License.
173
174+import json
175 import traceback
176
177 from webob import exc
178
179 from nova import compute
180 from nova import exception
181+from nova import flags
182 from nova import log as logging
183 from nova import wsgi
184+from nova import utils
185 from nova.api.openstack import common
186 from nova.api.openstack import faults
187 from nova.auth import manager as auth_manager
188@@ -35,6 +38,9 @@
189 LOG.setLevel(logging.DEBUG)
190
191
192+FLAGS = flags.FLAGS
193+
194+
195 def _translate_detail_keys(inst):
196 """ Coerces into dictionary format, mapping everything to Rackspace-like
197 attributes for return"""
198@@ -44,7 +50,7 @@
199 power_state.RUNNING: 'active',
200 power_state.BLOCKED: 'active',
201 power_state.SUSPENDED: 'suspended',
202- power_state.PAUSED: 'error',
203+ power_state.PAUSED: 'paused',
204 power_state.SHUTDOWN: 'active',
205 power_state.SHUTOFF: 'active',
206 power_state.CRASHED: 'error'}
207@@ -81,6 +87,7 @@
208
209 def __init__(self):
210 self.compute_api = compute.API()
211+ self._image_service = utils.import_object(FLAGS.image_service)
212 super(Controller, self).__init__()
213
214 def index(self, req):
215@@ -117,6 +124,18 @@
216 return faults.Fault(exc.HTTPNotFound())
217 return exc.HTTPAccepted()
218
219+ def _get_kernel_ramdisk_from_image(self, image_id):
220+ mapping_filename = FLAGS.os_krm_mapping_file
221+
222+ with open(mapping_filename) as f:
223+ mapping = json.load(f)
224+ if image_id in mapping:
225+ return mapping[image_id]
226+
227+ raise exception.NotFound(
228+ _("No entry for image '%s' in mapping file '%s'") %
229+ (image_id, mapping_filename))
230+
231 def create(self, req):
232 """ Creates a new server for a given user """
233 env = self._deserialize(req.body, req)
234@@ -125,10 +144,15 @@
235
236 key_pair = auth_manager.AuthManager.get_key_pairs(
237 req.environ['nova.context'])[0]
238+ image_id = common.get_image_id_from_image_hash(self._image_service,
239+ req.environ['nova.context'], env['server']['imageId'])
240+ kernel_id, ramdisk_id = self._get_kernel_ramdisk_from_image(image_id)
241 instances = self.compute_api.create(
242 req.environ['nova.context'],
243 instance_types.get_by_flavor_id(env['server']['flavorId']),
244- env['server']['imageId'],
245+ image_id,
246+ kernel_id=kernel_id,
247+ ramdisk_id=ramdisk_id,
248 display_name=env['server']['name'],
249 display_description=env['server']['name'],
250 key_name=key_pair['name'],
251@@ -158,6 +182,7 @@
252 """ Multi-purpose method used to reboot, rebuild, and
253 resize a server """
254 input_dict = self._deserialize(req.body, req)
255+ #TODO(sandy): rebuild/resize not supported.
256 try:
257 reboot_type = input_dict['reboot']['type']
258 except Exception:
259
260=== renamed file 'nova/api/openstack/sharedipgroups.py' => 'nova/api/openstack/shared_ip_groups.py'
261--- nova/api/openstack/sharedipgroups.py 2010-12-28 21:48:48 +0000
262+++ nova/api/openstack/shared_ip_groups.py 2011-01-11 19:42:14 +0000
263@@ -15,6 +15,8 @@
264 # License for the specific language governing permissions and limitations
265 # under the License.
266
267+import logging
268+
269 from webob import exc
270
271 from nova import wsgi
272@@ -29,7 +31,7 @@
273 def _translate_detail_keys(inst):
274 """ Coerces a shared IP group instance into proper dictionary format with
275 correctly mapped attributes """
276- return dict(sharedIpGroup=inst)
277+ return dict(sharedIpGroups=inst)
278
279
280 class Controller(wsgi.Controller):
281@@ -54,12 +56,12 @@
282
283 def delete(self, req, id):
284 """ Deletes a Shared IP Group """
285- raise faults.Fault(exc.HTTPNotFound())
286+ raise faults.Fault(exc.HTTPNotImplemented())
287
288- def detail(self, req, id):
289+ def detail(self, req):
290 """ Returns a complete list of Shared IP Groups """
291 return _translate_detail_keys({})
292
293 def create(self, req):
294 """ Creates a new Shared IP group """
295- raise faults.Fault(exc.HTTPNotFound())
296+ raise faults.Fault(exc.HTTPNotImplemented())
297
298=== modified file 'nova/auth/manager.py'
299--- nova/auth/manager.py 2011-01-04 05:26:41 +0000
300+++ nova/auth/manager.py 2011-01-11 19:42:14 +0000
301@@ -684,8 +684,7 @@
302 else:
303 regions = {'nova': FLAGS.cc_host}
304 for region, host in regions.iteritems():
305- rc = self.__generate_rc(user.access,
306- user.secret,
307+ rc = self.__generate_rc(user,
308 pid,
309 use_dmz,
310 host)
311@@ -725,7 +724,7 @@
312 return self.__generate_rc(user.access, user.secret, pid, use_dmz)
313
314 @staticmethod
315- def __generate_rc(access, secret, pid, use_dmz=True, host=None):
316+ def __generate_rc(user, pid, use_dmz=True, host=None):
317 """Generate rc file for user"""
318 if use_dmz:
319 cc_host = FLAGS.cc_dmz
320@@ -738,14 +737,19 @@
321 s3_host = host
322 cc_host = host
323 rc = open(FLAGS.credentials_template).read()
324- rc = rc % {'access': access,
325+ rc = rc % {'access': user.access,
326 'project': pid,
327- 'secret': secret,
328+ 'secret': user.secret,
329 'ec2': '%s://%s:%s%s' % (FLAGS.ec2_prefix,
330 cc_host,
331 FLAGS.cc_port,
332 FLAGS.ec2_suffix),
333 's3': 'http://%s:%s' % (s3_host, FLAGS.s3_port),
334+ 'os': '%s://%s:%s%s' % (FLAGS.os_prefix,
335+ cc_host,
336+ FLAGS.cc_port,
337+ FLAGS.os_suffix),
338+ 'user': user.name,
339 'nova': FLAGS.ca_file,
340 'cert': FLAGS.credential_cert_file,
341 'key': FLAGS.credential_key_file}
342
343=== modified file 'nova/auth/novarc.template'
344--- nova/auth/novarc.template 2010-07-15 15:52:11 +0000
345+++ nova/auth/novarc.template 2011-01-11 19:42:14 +0000
346@@ -10,3 +10,7 @@
347 export EUCALYPTUS_CERT=${NOVA_CERT} # euca-bundle-image seems to require this set
348 alias ec2-bundle-image="ec2-bundle-image --cert ${EC2_CERT} --privatekey ${EC2_PRIVATE_KEY} --user 42 --ec2cert ${NOVA_CERT}"
349 alias ec2-upload-bundle="ec2-upload-bundle -a ${EC2_ACCESS_KEY} -s ${EC2_SECRET_KEY} --url ${S3_URL} --ec2cert ${NOVA_CERT}"
350+export CLOUD_SERVERS_API_KEY="%(access)s"
351+export CLOUD_SERVERS_USERNAME="%(user)s"
352+export CLOUD_SERVERS_URL="%(os)s"
353+
354
355=== modified file 'nova/compute/api.py'
356--- nova/compute/api.py 2011-01-08 14:35:50 +0000
357+++ nova/compute/api.py 2011-01-11 19:42:14 +0000
358@@ -108,6 +108,8 @@
359 ramdisk_id = None
360 LOG.debug(_("Creating a raw instance"))
361 # Make sure we have access to kernel and ramdisk (if not raw)
362+ logging.debug("Using Kernel=%s, Ramdisk=%s" %
363+ (kernel_id, ramdisk_id))
364 if kernel_id:
365 self.image_service.show(context, kernel_id)
366 if ramdisk_id:
367@@ -171,7 +173,8 @@
368
369 # Set sane defaults if not specified
370 updates = dict(hostname=generate_hostname(instance_id))
371- if 'display_name' not in instance:
372+ if (not hasattr(instance, 'display_name')) or \
373+ instance.display_name == None:
374 updates['display_name'] = "Server %s" % instance_id
375
376 instance = self.update(context, instance_id, **updates)
377
378=== modified file 'nova/flags.py'
379--- nova/flags.py 2011-01-11 12:24:58 +0000
380+++ nova/flags.py 2011-01-11 19:42:14 +0000
381@@ -248,10 +248,12 @@
382 DEFINE_integer('rabbit_max_retries', 12, 'rabbit connection attempts')
383 DEFINE_string('control_exchange', 'nova', 'the main exchange to connect to')
384 DEFINE_string('ec2_prefix', 'http', 'prefix for ec2')
385+DEFINE_string('os_prefix', 'http', 'prefix for openstack')
386 DEFINE_string('cc_host', '$my_ip', 'ip of api server')
387 DEFINE_string('cc_dmz', '$my_ip', 'internal ip of api server')
388 DEFINE_integer('cc_port', 8773, 'cloud controller port')
389 DEFINE_string('ec2_suffix', '/services/Cloud', 'suffix for ec2')
390+DEFINE_string('os_suffix', '/v1.0/', 'suffix for openstack')
391
392 DEFINE_string('default_project', 'openstack', 'default project for openstack')
393 DEFINE_string('default_image', 'ami-11111',
394
395=== modified file 'nova/tests/api/openstack/test_images.py'
396--- nova/tests/api/openstack/test_images.py 2011-01-04 05:23:35 +0000
397+++ nova/tests/api/openstack/test_images.py 2011-01-11 19:42:14 +0000
398@@ -172,6 +172,7 @@
399
400 IMAGE_FIXTURES = [
401 {'id': '23g2ogk23k4hhkk4k42l',
402+ 'imageId': '23g2ogk23k4hhkk4k42l',
403 'name': 'public image #1',
404 'created_at': str(datetime.datetime.utcnow()),
405 'updated_at': str(datetime.datetime.utcnow()),
406@@ -181,6 +182,7 @@
407 'status': 'available',
408 'image_type': 'kernel'},
409 {'id': 'slkduhfas73kkaskgdas',
410+ 'imageId': 'slkduhfas73kkaskgdas',
411 'name': 'public image #2',
412 'created_at': str(datetime.datetime.utcnow()),
413 'updated_at': str(datetime.datetime.utcnow()),
414
415=== modified file 'nova/tests/api/openstack/test_servers.py'
416--- nova/tests/api/openstack/test_servers.py 2011-01-06 22:35:48 +0000
417+++ nova/tests/api/openstack/test_servers.py 2011-01-11 19:42:14 +0000
418@@ -133,6 +133,12 @@
419 def queue_get_for(context, *args):
420 return 'network_topic'
421
422+ def kernel_ramdisk_mapping(*args, **kwargs):
423+ return (1, 1)
424+
425+ def image_id_from_hash(*args, **kwargs):
426+ return 2
427+
428 self.stubs.Set(nova.db.api, 'project_get_network', project_get_network)
429 self.stubs.Set(nova.db.api, 'instance_create', instance_create)
430 self.stubs.Set(nova.rpc, 'cast', fake_method)
431@@ -142,6 +148,10 @@
432 self.stubs.Set(nova.db.api, 'queue_get_for', queue_get_for)
433 self.stubs.Set(nova.network.manager.VlanManager, 'allocate_fixed_ip',
434 fake_method)
435+ self.stubs.Set(nova.api.openstack.servers.Controller,
436+ "_get_kernel_ramdisk_from_image", kernel_ramdisk_mapping)
437+ self.stubs.Set(nova.api.openstack.common,
438+ "get_image_id_from_image_hash", image_id_from_hash)
439
440 body = dict(server=dict(
441 name='server_test', imageId=2, flavorId=2, metadata={},
442
443=== renamed file 'nova/tests/api/openstack/test_sharedipgroups.py' => 'nova/tests/api/openstack/test_shared_ip_groups.py'
444--- nova/tests/api/openstack/test_sharedipgroups.py 2010-10-08 20:39:00 +0000
445+++ nova/tests/api/openstack/test_shared_ip_groups.py 2011-01-11 19:42:14 +0000
446@@ -19,7 +19,7 @@
447
448 import stubout
449
450-from nova.api.openstack import sharedipgroups
451+from nova.api.openstack import shared_ip_groups
452
453
454 class SharedIpGroupsTest(unittest.TestCase):
455
456=== modified file 'nova/virt/xenapi/vm_utils.py'
457--- nova/virt/xenapi/vm_utils.py 2011-01-04 05:23:35 +0000
458+++ nova/virt/xenapi/vm_utils.py 2011-01-11 19:42:14 +0000
459@@ -357,7 +357,9 @@
460 if i >= 3 and i <= 11:
461 ref = node.childNodes
462 # Name and Value
463- diags[ref[0].firstChild.data] = ref[6].firstChild.data
464+ if len(ref) > 6:
465+ diags[ref[0].firstChild.data] = \
466+ ref[6].firstChild.data
467 return diags
468 except cls.XenAPI.Failure as e:
469 return {"Unable to retrieve diagnostics": e}