Merge lp:~rackspace-titan/glance/wsgi-refactor into lp:~hudson-openstack/glance/trunk

Proposed by Brian Waldon
Status: Merged
Approved by: Rick Harris
Approved revision: 148
Merged at revision: 140
Proposed branch: lp:~rackspace-titan/glance/wsgi-refactor
Merge into: lp:~hudson-openstack/glance/trunk
Diff against target: 1199 lines (+558/-226)
10 files modified
glance/api/v1/__init__.py (+4/-4)
glance/api/v1/images.py (+154/-86)
glance/common/exception.py (+26/-0)
glance/common/wsgi.py (+133/-101)
glance/registry/client.py (+12/-2)
glance/registry/server.py (+20/-14)
glance/utils.py (+0/-18)
tests/functional/test_curl_api.py (+1/-1)
tests/unit/test_api.py (+26/-0)
tests/unit/test_wsgi.py (+182/-0)
To merge this branch: bzr merge lp:~rackspace-titan/glance/wsgi-refactor
Reviewer Review Type Date Requested Status
Rick Harris (community) Approve
Brian Lamar (community) Approve
Jay Pipes (community) Approve
Review via email: mp+64311@code.launchpad.net

Description of the change

- refactoring wsgi code to divide deserialization, controller, serialization among different objects
- Resource object acts as coordinator

- tests are coming, this is for review purposes

To post a comment you must log in.
Revision history for this message
Brian Waldon (bcwaldon) wrote :

I can't figure out why tests/functional/test_curl_api.py:TestCurlApi.test_traceback_not_consumed is failing. It is "turned off" in my merge prop but I would like to fix it before I send it in.

Revision history for this message
Jay Pipes (jaypipes) wrote :

Hey Brian,

The reason that test is failing is because of this change:

148 - image_id = image_meta['id']
149 - content_type = req.headers.get('content-type', 'notset')
150 - if content_type != 'application/octet-stream':
151 - self._safe_kill(req, image_id)
152 - msg = ("Content-Type must be application/octet-stream")
153 + try:
154 + req.get_content_type('application/octet-stream')
155 + except exception.InvalidContentType:
156 + msg = "Content-Type must be application/octet-stream"

The issue is that the image is registered and set to a 'queued' status before this code is run. The call to _safe_kill() is what (correctly) sets the image status to 'killed' and returns the error message. You need to add the call to _safe_kill() back to the new except InvalidContentType block.

Hope that helps,

jay

Revision history for this message
Brian Waldon (bcwaldon) wrote :

Fixed the test case.

Revision history for this message
Jay Pipes (jaypipes) wrote :
Download full text (6.6 KiB)

Hey again Brian, here's a full review...

Overall, excellent and clean work. As mentioned on IRC, in the future, try to submit style/formatting-only stuff in a separate patch.

Design Discussion
-----------------

The one big design/architecture thing that I'd like to suggest a change for is making the responsibility of the Serializer subclasses very specific to transforming the dicts that may compose the body of a Response object. As they currently stand, the Serializer subclasses look to be doing more work that may more appropriately belong in the Controller.

I'll work through an example.

Consider the ImageSerializer class (shortened for brevity):

357 +class ImageSerializer(wsgi.JSONResponseSerializer):
...
360 + def _inject_location_header(self, response, image_meta):
361 + location = self._get_image_location(image_meta)
362 + response.headers.add('Location', location)
363 +
364 + def _inject_checksum_header(self, response, image_meta):
365 + response.headers.add('ETag', image_meta['checksum'])
366 +
367 + def _inject_image_meta_headers(self, response, image_meta):
...
379 + headers = utils.image_meta_to_http_headers(image_meta)
380 +
381 + for k, v in headers.items():
382 + response.headers.add(k, v)
383 +
384 + def _get_image_location(self, image_meta):
...
386 + return "/v1/images/%s" % image_meta['id']
...
396 + def show(self, result):
397 + image_meta = result['image_meta']
398 +
399 + response = webob.Response(app_iter=result['image_iterator'])
400 + # Using app_iter blanks content-length, so we set it here...
401 + response.headers.add('Content-Length', image_meta['size'])
402 + response.headers.add('Content-Type', 'application/octet-stream')
403 +
404 + self._inject_image_meta_headers(response, image_meta)
405 + self._inject_location_header(response, image_meta)
406 + self._inject_checksum_header(response, image_meta)
...
419 + def create(self, result):
420 + image_meta = result['image_meta']
421 + response = webob.Response()
422 + response.status = httplib.CREATED
423 + response.headers.add('Content-Type', 'application/json')
424 + response.body = self.to_json(dict(image=image_meta))
425 + self._inject_location_header(response, image_meta)
426 + self._inject_checksum_header(response, image_meta)
427 + return response

I think that I would prefer that the ImageSerializer class NOT be responsible for:

a) creating a webob.Response object
b) modifying any headers of the Response object other than Content-Type

I think that all the Serializer subclasses should be doing is transforming the Response object's body attribute to the necessary serialized format, and setting the Content-Type header appropriately.

All the other activities that are happening in ImageSerializer I believe should be in the Controller object, since the Controller is responsible for setting various headers (ETag, Location, etc) and for creating the Response object itself...

So, I think I'd prefer the ImageSerializer to look like this:

class ImageSerializer(wsgi.JSONResponseSerializer):
    def _serialize(self, response):
        response.content_type = 'application/json'
        meta = response....

Read more...

review: Needs Fixing
Revision history for this message
Mark Washenberger (markwash) wrote :

>
> I think that I would prefer that the ImageSerializer class NOT be responsible
> for:
>
> a) creating a webob.Response object
> b) modifying any headers of the Response object other than Content-Type
>
> I think that all the Serializer subclasses should be doing is transforming the
> Response object's body attribute to the necessary serialized format, and
> setting the Content-Type header appropriately.
>
> All the other activities that are happening in ImageSerializer I believe
> should be in the Controller object, since the Controller is responsible for
> setting various headers (ETag, Location, etc) and for creating the Response
> object itself...
>

I think it is fitting for the Serializer to be responsible for creating the Webob Response. It seems like part of the idea here is to take some of the wsgi- and http api- specific responsibilities away from the controller so that it can focus on its responsibilities rather than its context. In my mind I see the ImageSerializer as responsible for translating image responses (which are really domain-model objects) into something that a wsgi server can present.

>
> Also, have you thought about how you will deal with multiple serializers? How
> do you plan to support a resource that can be translated into both JSON and
> XML?

I believe Waldon has given this a good bit of thought, but I actually encouraged him to leave out code going towards this direction. Let's XP it up: do only the minimum required for the features needed today. I believe the design to incorporate xml will be cleaner if we do it when we are actually trying to support it.

Cheers :-)

Revision history for this message
Jay Pipes (jaypipes) wrote :

> >
> > I think that I would prefer that the ImageSerializer class NOT be
> responsible
> > for:
> >
> > a) creating a webob.Response object
> > b) modifying any headers of the Response object other than Content-Type
> >
> > I think that all the Serializer subclasses should be doing is transforming
> the
> > Response object's body attribute to the necessary serialized format, and
> > setting the Content-Type header appropriately.
> >
> > All the other activities that are happening in ImageSerializer I believe
> > should be in the Controller object, since the Controller is responsible for
> > setting various headers (ETag, Location, etc) and for creating the Response
> > object itself...
> >
>
> I think it is fitting for the Serializer to be responsible for creating the
> Webob Response. It seems like part of the idea here is to take some of the
> wsgi- and http api- specific responsibilities away from the controller so that
> it can focus on its responsibilities rather than its context. In my mind I see
> the ImageSerializer as responsible for translating image responses (which are
> really domain-model objects) into something that a wsgi server can present.

I strongly disagree :) Serializers should serialize. Nothing more. Controllers *control* things. They have the requisite information to set non-content-type headers, to create the appropriate Response or HTTPException objects. Moving that code into the Serializer class just doesn't make much sense to me...

> > Also, have you thought about how you will deal with multiple serializers?
> How
> > do you plan to support a resource that can be translated into both JSON and
> > XML?
>
> I believe Waldon has given this a good bit of thought, but I actually
> encouraged him to leave out code going towards this direction. Let's XP it up:
> do only the minimum required for the features needed today. I believe the
> design to incorporate xml will be cleaner if we do it when we are actually
> trying to support it.

Sure, no probs. I was just wondering if Brian had considered it :)

-jay

Revision history for this message
Mark Washenberger (markwash) wrote :

> I strongly disagree :) Serializers should serialize. Nothing more. Controllers
> *control* things. They have the requisite information to set non-content-type
> headers, to create the appropriate Response or HTTPException objects. Moving
> that code into the Serializer class just doesn't make much sense to me...
>

It seems that there are two issues: proper naming, and a proper division of responsibility.

Perhaps Serializer is not the right name for this set of functionality. But I hope you will agree that "Controller" is a bit too broad of a term to fit into a system with a good division of responsibility. I am certainly open to renaming both. But maybe we ought not rename Controller, so that we remain consistent with other such units across Openstack. However, our naming mistake doesn't require us to make a similar mistake in the division of responsibility just to conform.

Although the naming is extremely important, if we set it aside I believe that leaving it to the "postprocessor" of the controller's output (currently the serializer) to create the WebOb response is a powerful approach--it lets the Controller focus on the problem domain rather than on wsgi details.

I hope some of that makes sense! :-)

Revision history for this message
Jay Pipes (jaypipes) wrote :

> Perhaps Serializer is not the right name for this set of functionality. But I
> hope you will agree that "Controller" is a bit too broad of a term to fit into
> a system with a good division of responsibility.

Not really... it's called MVC for a reason... The Controller deals with logic and business rules, and that's what *most* of the stuff that was placed into the ImageSerializer was doing. And I think that is incorrect. Logic should stay in the Controller. The Controller should create and populate a Response object and return it. Middleware (both paste-constructed and manually created as in the case of serializers) should modify the Response object. That's it. No logic.

> I am certainly open to
> renaming both. But maybe we ought not rename Controller, so that we remain
> consistent with other such units across Openstack.

I wasn't suggesting we rename Controller :) Like I said, it's an integral part of the MVC pattern.

> However, our naming mistake
> doesn't require us to make a similar mistake in the division of responsibility
> just to conform.

I don't understand this, sorry. I think I've explained why a piece of middleware should not be constructing the Response object. That's the responsibility of the center-most object in the chain: the Controller.

> Although the naming is extremely important, if we set it aside I believe that
> leaving it to the "postprocessor" of the controller's output (currently the
> serializer) to create the WebOb response is a powerful approach--it lets the
> Controller focus on the problem domain rather than on wsgi details.

Postprocessing what? Oh, that's right... postprocessing the *Response* object :) Postprocessing implies that you are processing a previously-constructed object, no? ;)

> I hope some of that makes sense! :-)

Not really, but I still dig you man :)

-jay

Revision history for this message
Brian Waldon (bcwaldon) wrote :
Download full text (4.0 KiB)

> I think that I would prefer that the ImageSerializer class NOT be responsible
> for:
>
> a) creating a webob.Response object

I moved this into wsgi.Resource

> b) modifying any headers of the Response object other than Content-Type

I'm going to take some time to figure this out. I'll have another commit for you later today.

> Tiny Nits/Fixes
> ---------------
>
> Here are some little nits to correct before an approve.
>
> 1) docstring stuff
>
> 445 + """Base Nova Exception
> 446 +
> 447 + To correctly use this class, inherit from it and define
> 448 + a 'message' property. That message will get printf'd
> 449 + with the keyword arguments provided to the constructor.
> 450 +
> 451 + """
>
> Please use this style instead:
>
> """
> Base Nova Exception
>
> To correctly use this class, inherit from it and define
> a 'message' property. That message will get printf'd
> with the keyword arguments provided to the constructor.
> """
>
> No need for extra newline at the end, and please put the short description on
> a new line after the opening """
>
> Some of the new docstrings are correct, others aren't... please do a quick
> run-through to check they are all consistent like the above style.
>
> Also, we're Glance, not Nova ;P

Copy/paste error. Sorry for that. I fixed the problems I introduced. There are a bunch of docstrings that don't follow your suggestion in registry/server.py (probably elsewhere as well) that I did not fix because I did not introduce them in this merge prop. Maybe a style cleanup is in order?

> 2)
>
> 475 import datetime
> 476 -
> 477 import eventlet
>
> eventlet is a 3rd party library. We separate stdlib modules from 3rd party
> modules with a single newline.

Ah, I thought it was just separation of non-glance and glance modules. Fixed that.

> 3) few little typos
>
> 67 + :retval similiar to 'show' method but without image_data
>
> s/similiar/similar

Fixed. I do that all the time :(

> 4)
>
> 133 + if 'location' in image_meta and image_meta['location'] is not None:
> 264 + if 'location' in image_meta and image_meta['location'] is not None:
>
> can be shortened to:
>
> if image_meta.get('location'):

Fixed. I took it a step farther and stored location in a variable before the conditional. We needed to use it again immediately anyways, so I removed the duplication of reading from the image_meta dict.

> 5)
>
> 342 + def create(self, request):
> 343 + result = {}
> 344 + result['image_meta'] = utils.get_image_meta_from_headers(request)
> 345 + data = request.body if self.has_body(request) else None
> 346 + result['image_data'] = data
> 347 + return result
> 348 +
> 349 + def update(self, request):
> 350 + result = {}
> 351 + result['image_meta'] = utils.get_image_meta_from_headers(request)
> 352 + data = request.body if self.has_body(request) else None
> 353 + result['image_data'] = data
> 354 + return result
>
> Little bit of DRY cleanup needed there... something like this would do:

Good catch. I didn't even notice that.

> 6) the wsgi.Resource constructor
>
> 665 + def __init__(self, des...

Read more...

Revision history for this message
Brian Waldon (bcwaldon) wrote :

Looking for suggestions on separating out header "serialization" code.

Revision history for this message
Jay Pipes (jaypipes) wrote :

Let's work on cleaning up the header serialization code in a later patch. Approved from me.

review: Approve
Revision history for this message
Rick Harris (rconradharris) wrote :

Overall looks good. Some thoughts:

> 425 + self._inject_location_header(response, image_meta)
> 426 + self._inject_checksum_header(response, image_meta)
> 427 + return response

Super nit-picky here, but might be worth DRY'ing up some of this code since
it's repeated quite a bit.

> 582 + return (request.content_length and request.content_length > 0) \
> 583 + or 'transfer-encoding' in request.headers

This statement was a little hard to read at-first. Once I understood it, it
was clear enough. That said, given that we're, for the most part, optimizing
for readability, maybe it would be clearer as:

  if 'transfer-encoding' in request.headers:
    return True
  elif request.content_length is None:
    raise Exception("Content-Length was not specified and we're not "
                    "Transfer-Encoding")

  return request.content_length > 0:

> 683 + if type(action_result) is dict:

Duck-typing might be preferable here. Do you know specifically what dict
methods the serializer is using; if you're not sure, you could always check
for presence of `keys()`

  if hasattr(action_result, 'keys'):
    blah

(This is a really small nit, but if we ever need to do something tricky by
subclassing `dict`, this would keep the code working)

> + """Find action-spefic method on self and call it."""
s/spefic/specific/

> 695 + except Exception:
> 696 + method = getattr(obj, 'default')

Should this handler be this broad? Would AttributeError work here?

If that's the case, we could do:

  method = getattr(obj, action, 'default')

Revision history for this message
Brian Waldon (bcwaldon) wrote :

> Overall looks good. Some thoughts:
>
> > 425 + self._inject_location_header(response, image_meta)
> > 426 + self._inject_checksum_header(response, image_meta)
> > 427 + return response
>
> Super nit-picky here, but might be worth DRY'ing up some of this code since
> it's repeated quite a bit.

I would factor this out a bit more if it were clear exactly how to do it. Now, I could make a method called inject_location_and_checksum_headers, but I don't see much value in that. I already factored out quite a bit with regards to these two functions. If you still feel strongly about this, I'll do it for you.

> > 582 + return (request.content_length and request.content_length >
> 0) \
> > 583 + or 'transfer-encoding' in request.headers
>
> This statement was a little hard to read at-first. Once I understood it, it
> was clear enough. That said, given that we're, for the most part, optimizing
> for readability, maybe it would be clearer as:
>
>
> if 'transfer-encoding' in request.headers:
> return True
> elif request.content_length is None:
> raise Exception("Content-Length was not specified and we're not "
> "Transfer-Encoding")
>
> return request.content_length > 0:

I cleaned it up with a longer if/elif block, while keeping the same logic. This is a boolean check, so I didn't want to throw an exception.

> > 683 + if type(action_result) is dict:
>
> Duck-typing might be preferable here. Do you know specifically what dict
> methods the serializer is using; if you're not sure, you could always check
> for presence of `keys()`
>
> if hasattr(action_result, 'keys'):
> blah
>
> (This is a really small nit, but if we ever need to do something tricky by
> subclassing `dict`, this would keep the code working)

How about I just assume I can serialize it, and if there is an exception I will just return the original results? I checked in a possible implementation of this. Tell me what you think.

> > + """Find action-spefic method on self and call it."""
> s/spefic/specific/

Good catch. Fixed.

> > 695 + except Exception:
> > 696 + method = getattr(obj, 'default')
>
> Should this handler be this broad? Would AttributeError work here?
>
> If that's the case, we could do:
>
> method = getattr(obj, action, 'default')

I added AttributeError to the except statement. I can't use 'default' as a third argument there because it expects a callable. I would have to pass in obj.default, and I can't assume that exists.

I also caught some issues in the wsgi.Resource docstring and cleaned that up.

147. By Brian Waldon

refactoring from Rick's comments

Revision history for this message
Brian Lamar (blamar) wrote :

164: HTTPBadRequest needs explanation=msg to actually display the passed-in message.

680: Not sure if it will hinder readability, but this blank line seems out of place.

716 + try:
717 + del args['controller']
718 + except KeyError:
719 + pass

I've been playing with this replacement logic/syntax:

args.pop("controller", None)

It's not as explicit, which I don't like, but it gets the job done and isn't too cryptic.

Starting at 1064, should those each be it's own test?

I like your style/spacing! :)

review: Needs Fixing
Revision history for this message
Brian Waldon (bcwaldon) wrote :

> 164: HTTPBadRequest needs explanation=msg to actually display the passed-in
> message.

Fixed.

> 680: Not sure if it will hinder readability, but this blank line seems out of
> place.

Fixed.

> 716 + try:
> 717 + del args['controller']
> 718 + except KeyError:
> 719 + pass
>
> I've been playing with this replacement logic/syntax:
>
> args.pop("controller", None)
>
> It's not as explicit, which I don't like, but it gets the job done and isn't
> too cryptic.

I could go either way, and originally I did have it this way. When I merge prop'd this code in Nova, I was asked to change it to the try/except style. I hope it's okay to leave it.

> Starting at 1064, should those each be it's own test?

Yes, sir.

> I like your style/spacing! :)

I like your style, too.

148. By Brian Waldon

refactoring for Brian

Revision history for this message
Brian Lamar (blamar) :
review: Approve
Revision history for this message
Rick Harris (rconradharris) wrote :

Looks good, nice job Brian.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'glance/api/v1/__init__.py'
--- glance/api/v1/__init__.py 2011-05-11 23:03:51 +0000
+++ glance/api/v1/__init__.py 2011-06-17 20:46:38 +0000
@@ -32,11 +32,11 @@
32 def __init__(self, options):32 def __init__(self, options):
33 self.options = options33 self.options = options
34 mapper = routes.Mapper()34 mapper = routes.Mapper()
35 controller = images.Controller(options)35 resource = images.create_resource(options)
36 mapper.resource("image", "images", controller=controller,36 mapper.resource("image", "images", controller=resource,
37 collection={'detail': 'GET'})37 collection={'detail': 'GET'})
38 mapper.connect("/", controller=controller, action="index")38 mapper.connect("/", controller=resource, action="index")
39 mapper.connect("/images/{id}", controller=controller, action="meta",39 mapper.connect("/images/{id}", controller=resource, action="meta",
40 conditions=dict(method=["HEAD"]))40 conditions=dict(method=["HEAD"]))
41 super(API, self).__init__(mapper)41 super(API, self).__init__(mapper)
4242
4343
=== modified file 'glance/api/v1/images.py'
--- glance/api/v1/images.py 2011-05-17 13:34:42 +0000
+++ glance/api/v1/images.py 2011-06-17 20:46:38 +0000
@@ -24,7 +24,7 @@
24import logging24import logging
25import sys25import sys
2626
27from webob import Response27import webob
28from webob.exc import (HTTPNotFound,28from webob.exc import (HTTPNotFound,
29 HTTPConflict,29 HTTPConflict,
30 HTTPBadRequest)30 HTTPBadRequest)
@@ -46,7 +46,7 @@
46 'size_min', 'size_max']46 'size_min', 'size_max']
4747
4848
49class Controller(wsgi.Controller):49class Controller(object):
5050
51 """51 """
52 WSGI controller for images resource in Glance v1 API52 WSGI controller for images resource in Glance v1 API
@@ -80,7 +80,7 @@
80 * checksum -- MD5 checksum of the image data80 * checksum -- MD5 checksum of the image data
81 * size -- Size of image data in bytes81 * size -- Size of image data in bytes
8282
83 :param request: The WSGI/Webob Request object83 :param req: The WSGI/Webob Request object
84 :retval The response body is a mapping of the following form::84 :retval The response body is a mapping of the following form::
8585
86 {'images': [86 {'images': [
@@ -107,7 +107,7 @@
107 """107 """
108 Returns detailed information for all public, available images108 Returns detailed information for all public, available images
109109
110 :param request: The WSGI/Webob Request object110 :param req: The WSGI/Webob Request object
111 :retval The response body is a mapping of the following form::111 :retval The response body is a mapping of the following form::
112112
113 {'images': [113 {'images': [
@@ -155,29 +155,22 @@
155 Returns metadata about an image in the HTTP headers of the155 Returns metadata about an image in the HTTP headers of the
156 response object156 response object
157157
158 :param request: The WSGI/Webob Request object158 :param req: The WSGI/Webob Request object
159 :param id: The opaque image identifier159 :param id: The opaque image identifier
160 :retval similar to 'show' method but without image_data
160161
161 :raises HTTPNotFound if image metadata is not available to user162 :raises HTTPNotFound if image metadata is not available to user
162 """163 """
163 image = self.get_image_meta_or_404(req, id)164 return {
164165 'image_meta': self.get_image_meta_or_404(req, id),
165 res = Response(request=req)166 }
166 utils.inject_image_meta_into_headers(res, image)
167 res.headers.add('Location', "/v1/images/%s" % id)
168 res.headers.add('ETag', image['checksum'])
169
170 return req.get_response(res)
171167
172 def show(self, req, id):168 def show(self, req, id):
173 """169 """
174 Returns an iterator as a Response object that170 Returns an iterator that can be used to retrieve an image's
175 can be used to retrieve an image's data. The171 data along with the image metadata.
176 content-type of the response is the content-type
177 of the image, or application/octet-stream if none
178 is known or found.
179172
180 :param request: The WSGI/Webob Request object173 :param req: The WSGI/Webob Request object
181 :param id: The opaque image identifier174 :param id: The opaque image identifier
182175
183 :raises HTTPNotFound if image is not available to user176 :raises HTTPNotFound if image is not available to user
@@ -192,31 +185,26 @@
192 for chunk in chunks:185 for chunk in chunks:
193 yield chunk186 yield chunk
194187
195 res = Response(app_iter=image_iterator(),188 return {
196 content_type="application/octet-stream")189 'image_iterator': image_iterator(),
197 # Using app_iter blanks content-length, so we set it here...190 'image_meta': image,
198 res.headers.add('Content-Length', image['size'])191 }
199 utils.inject_image_meta_into_headers(res, image)
200 res.headers.add('Location', "/v1/images/%s" % id)
201 res.headers.add('ETag', image['checksum'])
202 return req.get_response(res)
203192
204 def _reserve(self, req):193 def _reserve(self, req, image_meta):
205 """194 """
206 Adds the image metadata to the registry and assigns195 Adds the image metadata to the registry and assigns
207 an image identifier if one is not supplied in the request196 an image identifier if one is not supplied in the request
208 headers. Sets the image's status to `queued`197 headers. Sets the image's status to `queued`.
209198
210 :param request: The WSGI/Webob Request object199 :param req: The WSGI/Webob Request object
211 :param id: The opaque image identifier200 :param id: The opaque image identifier
212201
213 :raises HTTPConflict if image already exists202 :raises HTTPConflict if image already exists
214 :raises HTTPBadRequest if image metadata is not valid203 :raises HTTPBadRequest if image metadata is not valid
215 """204 """
216 image_meta = utils.get_image_meta_from_headers(req)205 location = image_meta.get('location')
217206 if location:
218 if 'location' in image_meta:207 store = get_store_from_location(location)
219 store = get_store_from_location(image_meta['location'])
220 # check the store exists before we hit the registry, but we208 # check the store exists before we hit the registry, but we
221 # don't actually care what it is at this point209 # don't actually care what it is at this point
222 self.get_store_or_400(req, store)210 self.get_store_or_400(req, store)
@@ -250,28 +238,27 @@
250 will attempt to use that store, if not, Glance will use the238 will attempt to use that store, if not, Glance will use the
251 store set by the flag `default_store`.239 store set by the flag `default_store`.
252240
253 :param request: The WSGI/Webob Request object241 :param req: The WSGI/Webob Request object
254 :param image_meta: Mapping of metadata about image242 :param image_meta: Mapping of metadata about image
255243
256 :raises HTTPConflict if image already exists244 :raises HTTPConflict if image already exists
257 :retval The location where the image was stored245 :retval The location where the image was stored
258 """246 """
259 image_id = image_meta['id']247 try:
260 content_type = req.headers.get('content-type', 'notset')248 req.get_content_type('application/octet-stream')
261 if content_type != 'application/octet-stream':249 except exception.InvalidContentType:
262 self._safe_kill(req, image_id)250 self._safe_kill(req, image_meta['id'])
263 msg = ("Content-Type must be application/octet-stream")251 msg = "Content-Type must be application/octet-stream"
264 logger.error(msg)252 logger.error(msg)
265 raise HTTPBadRequest(msg, content_type="text/plain",253 raise HTTPBadRequest(explanation=msg)
266 request=req)
267254
268 store_name = req.headers.get(255 store_name = req.headers.get('x-image-meta-store',
269 'x-image-meta-store', self.options['default_store'])256 self.options['default_store'])
270257
271 store = self.get_store_or_400(req, store_name)258 store = self.get_store_or_400(req, store_name)
272259
273 logger.debug("Setting image %s to status 'saving'"260 image_id = image_meta['id']
274 % image_id)261 logger.debug("Setting image %s to status 'saving'" % image_id)
275 registry.update_image_metadata(self.options, image_id,262 registry.update_image_metadata(self.options, image_id,
276 {'status': 'saving'})263 {'status': 'saving'})
277 try:264 try:
@@ -304,11 +291,13 @@
304 'size': size})291 'size': size})
305292
306 return location293 return location
294
307 except exception.Duplicate, e:295 except exception.Duplicate, e:
308 msg = ("Attempt to upload duplicate image: %s") % str(e)296 msg = ("Attempt to upload duplicate image: %s") % str(e)
309 logger.error(msg)297 logger.error(msg)
310 self._safe_kill(req, image_id)298 self._safe_kill(req, image_id)
311 raise HTTPConflict(msg, request=req)299 raise HTTPConflict(msg, request=req)
300
312 except Exception, e:301 except Exception, e:
313 msg = ("Error uploading image: %s") % str(e)302 msg = ("Error uploading image: %s") % str(e)
314 logger.error(msg)303 logger.error(msg)
@@ -320,8 +309,8 @@
320 Sets the image status to `active` and the image's location309 Sets the image status to `active` and the image's location
321 attribute.310 attribute.
322311
323 :param request: The WSGI/Webob Request object312 :param req: The WSGI/Webob Request object
324 :param image_meta: Mapping of metadata about image313 :param image_id: Opaque image identifier
325 :param location: Location of where Glance stored this image314 :param location: Location of where Glance stored this image
326 """315 """
327 image_meta = {}316 image_meta = {}
@@ -333,9 +322,9 @@
333322
334 def _kill(self, req, image_id):323 def _kill(self, req, image_id):
335 """324 """
336 Marks the image status to `killed`325 Marks the image status to `killed`.
337326
338 :param request: The WSGI/Webob Request object327 :param req: The WSGI/Webob Request object
339 :param image_id: Opaque image identifier328 :param image_id: Opaque image identifier
340 """329 """
341 registry.update_image_metadata(self.options,330 registry.update_image_metadata(self.options,
@@ -349,7 +338,7 @@
349 Since _kill is meant to be called from exceptions handlers, it should338 Since _kill is meant to be called from exceptions handlers, it should
350 not raise itself, rather it should just log its error.339 not raise itself, rather it should just log its error.
351340
352 :param request: The WSGI/Webob Request object341 :param req: The WSGI/Webob Request object
353 :param image_id: Opaque image identifier342 :param image_id: Opaque image identifier
354 """343 """
355 try:344 try:
@@ -364,7 +353,7 @@
364 and activates the image in the registry after a successful353 and activates the image in the registry after a successful
365 upload.354 upload.
366355
367 :param request: The WSGI/Webob Request object356 :param req: The WSGI/Webob Request object
368 :param image_meta: Mapping of metadata about image357 :param image_meta: Mapping of metadata about image
369358
370 :retval Mapping of updated image data359 :retval Mapping of updated image data
@@ -373,7 +362,7 @@
373 location = self._upload(req, image_meta)362 location = self._upload(req, image_meta)
374 return self._activate(req, image_id, location)363 return self._activate(req, image_id, location)
375364
376 def create(self, req):365 def create(self, req, image_meta, image_data):
377 """366 """
378 Adds a new image to Glance. Three scenarios exist when creating an367 Adds a new image to Glance. Three scenarios exist when creating an
379 image:368 image:
@@ -399,32 +388,27 @@
399 containing metadata about the image is returned, including its388 containing metadata about the image is returned, including its
400 opaque identifier.389 opaque identifier.
401390
402 :param request: The WSGI/Webob Request object391 :param req: The WSGI/Webob Request object
392 :param image_meta: Mapping of metadata about image
393 :param image_data: Actual image data that is to be stored
403394
404 :raises HTTPBadRequest if no x-image-meta-location is missing395 :raises HTTPBadRequest if x-image-meta-location is missing
405 and the request body is not application/octet-stream396 and the request body is not application/octet-stream
406 image data.397 image data.
407 """398 """
408 image_meta = self._reserve(req)399 image_meta = self._reserve(req, image_meta)
409 image_id = image_meta['id']400 image_id = image_meta['id']
410401
411 if utils.has_body(req):402 if image_data is not None:
412 image_meta = self._upload_and_activate(req, image_meta)403 image_meta = self._upload_and_activate(req, image_meta)
413 else:404 else:
414 if 'x-image-meta-location' in req.headers:405 location = image_meta.get('location')
415 location = req.headers['x-image-meta-location']406 if location:
416 image_meta = self._activate(req, image_id, location)407 image_meta = self._activate(req, image_id, location)
417408
418 # APP states we should return a Location: header with the edit409 return {'image_meta': image_meta}
419 # URI of the resource newly-created.410
420 res = Response(request=req, body=json.dumps(dict(image=image_meta)),411 def update(self, req, id, image_meta, image_data):
421 status=httplib.CREATED, content_type="text/plain")
422 res.headers.add('Location', "/v1/images/%s" % image_id)
423 res.headers.add('ETag', image_meta['checksum'])
424
425 return req.get_response(res)
426
427 def update(self, req, id):
428 """412 """
429 Updates an existing image with the registry.413 Updates an existing image with the registry.
430414
@@ -433,29 +417,17 @@
433417
434 :retval Returns the updated image information as a mapping418 :retval Returns the updated image information as a mapping
435 """419 """
436 has_body = utils.has_body(req)
437
438 orig_image_meta = self.get_image_meta_or_404(req, id)420 orig_image_meta = self.get_image_meta_or_404(req, id)
439 orig_status = orig_image_meta['status']421 orig_status = orig_image_meta['status']
440422
441 if has_body and orig_status != 'queued':423 if image_data is not None and orig_status != 'queued':
442 raise HTTPConflict("Cannot upload to an unqueued image")424 raise HTTPConflict("Cannot upload to an unqueued image")
443425
444 new_image_meta = utils.get_image_meta_from_headers(req)
445 try:426 try:
446 image_meta = registry.update_image_metadata(self.options,427 image_meta = registry.update_image_metadata(self.options, id,
447 id,428 image_meta, True)
448 new_image_meta,429 if image_data is not None:
449 True)
450 if has_body:
451 image_meta = self._upload_and_activate(req, image_meta)430 image_meta = self._upload_and_activate(req, image_meta)
452
453 res = Response(request=req,
454 body=json.dumps(dict(image=image_meta)),
455 content_type="text/plain")
456 res.headers.add('Location', "/images/%s" % id)
457 res.headers.add('ETag', image_meta['checksum'])
458 return res
459 except exception.Invalid, e:431 except exception.Invalid, e:
460 msg = ("Failed to update image metadata. Got error: %(e)s"432 msg = ("Failed to update image metadata. Got error: %(e)s"
461 % locals())433 % locals())
@@ -463,11 +435,13 @@
463 logger.error(line)435 logger.error(line)
464 raise HTTPBadRequest(msg, request=req, content_type="text/plain")436 raise HTTPBadRequest(msg, request=req, content_type="text/plain")
465437
438 return {'image_meta': image_meta}
439
466 def delete(self, req, id):440 def delete(self, req, id):
467 """441 """
468 Deletes the image and all its chunks from the Glance442 Deletes the image and all its chunks from the Glance
469443
470 :param request: The WSGI/Webob Request object444 :param req: The WSGI/Webob Request object
471 :param id: The opaque image identifier445 :param id: The opaque image identifier
472446
473 :raises HttpBadRequest if image registry is invalid447 :raises HttpBadRequest if image registry is invalid
@@ -527,3 +501,97 @@
527 logger.error(msg)501 logger.error(msg)
528 raise HTTPBadRequest(msg, request=request,502 raise HTTPBadRequest(msg, request=request,
529 content_type='text/plain')503 content_type='text/plain')
504
505
506class ImageDeserializer(wsgi.JSONRequestDeserializer):
507 """Handles deserialization of specific controller method requests."""
508
509 def _deserialize(self, request):
510 result = {}
511 result['image_meta'] = utils.get_image_meta_from_headers(request)
512 data = request.body if self.has_body(request) else None
513 result['image_data'] = data
514 return result
515
516 def create(self, request):
517 return self._deserialize(request)
518
519 def update(self, request):
520 return self._deserialize(request)
521
522
523class ImageSerializer(wsgi.JSONResponseSerializer):
524 """Handles serialization of specific controller method responses."""
525
526 def _inject_location_header(self, response, image_meta):
527 location = self._get_image_location(image_meta)
528 response.headers.add('Location', location)
529
530 def _inject_checksum_header(self, response, image_meta):
531 response.headers.add('ETag', image_meta['checksum'])
532
533 def _inject_image_meta_headers(self, response, image_meta):
534 """
535 Given a response and mapping of image metadata, injects
536 the Response with a set of HTTP headers for the image
537 metadata. Each main image metadata field is injected
538 as a HTTP header with key 'x-image-meta-<FIELD>' except
539 for the properties field, which is further broken out
540 into a set of 'x-image-meta-property-<KEY>' headers
541
542 :param response: The Webob Response object
543 :param image_meta: Mapping of image metadata
544 """
545 headers = utils.image_meta_to_http_headers(image_meta)
546
547 for k, v in headers.items():
548 response.headers.add(k, v)
549
550 def _get_image_location(self, image_meta):
551 """Build a relative url to reach the image defined by image_meta."""
552 return "/v1/images/%s" % image_meta['id']
553
554 def meta(self, response, result):
555 image_meta = result['image_meta']
556 self._inject_image_meta_headers(response, image_meta)
557 self._inject_location_header(response, image_meta)
558 self._inject_checksum_header(response, image_meta)
559 return response
560
561 def show(self, response, result):
562 image_meta = result['image_meta']
563
564 response.app_iter = result['image_iterator']
565 # Using app_iter blanks content-length, so we set it here...
566 response.headers.add('Content-Length', image_meta['size'])
567 response.headers.add('Content-Type', 'application/octet-stream')
568
569 self._inject_image_meta_headers(response, image_meta)
570 self._inject_location_header(response, image_meta)
571 self._inject_checksum_header(response, image_meta)
572
573 return response
574
575 def update(self, response, result):
576 image_meta = result['image_meta']
577 response.body = self.to_json(dict(image=image_meta))
578 response.headers.add('Content-Type', 'application/json')
579 self._inject_location_header(response, image_meta)
580 self._inject_checksum_header(response, image_meta)
581 return response
582
583 def create(self, response, result):
584 image_meta = result['image_meta']
585 response.status = httplib.CREATED
586 response.headers.add('Content-Type', 'application/json')
587 response.body = self.to_json(dict(image=image_meta))
588 self._inject_location_header(response, image_meta)
589 self._inject_checksum_header(response, image_meta)
590 return response
591
592
593def create_resource(options):
594 """Images resource factory method"""
595 deserializer = ImageDeserializer()
596 serializer = ImageSerializer()
597 return wsgi.Resource(Controller(options), deserializer, serializer)
530598
=== modified file 'glance/common/exception.py'
--- glance/common/exception.py 2011-02-02 01:43:16 +0000
+++ glance/common/exception.py 2011-06-17 20:46:38 +0000
@@ -96,3 +96,29 @@
96 raise96 raise
97 _wrap.func_name = f.func_name97 _wrap.func_name = f.func_name
98 return _wrap98 return _wrap
99
100
101class GlanceException(Exception):
102 """
103 Base Glance Exception
104
105 To correctly use this class, inherit from it and define
106 a 'message' property. That message will get printf'd
107 with the keyword arguments provided to the constructor.
108 """
109 message = "An unknown exception occurred"
110
111 def __init__(self, **kwargs):
112 try:
113 self._error_string = self.message % kwargs
114
115 except Exception:
116 # at least get the core message out if something happened
117 self._error_string = self.message
118
119 def __str__(self):
120 return self._error_string
121
122
123class InvalidContentType(GlanceException):
124 message = "Invalid content type %(content_type)s"
99125
=== modified file 'glance/common/wsgi.py'
--- glance/common/wsgi.py 2011-05-18 00:11:46 +0000
+++ glance/common/wsgi.py 2011-06-17 20:46:38 +0000
@@ -34,6 +34,8 @@
34import webob.dec34import webob.dec
35import webob.exc35import webob.exc
3636
37from glance.common import exception
38
3739
38class WritableLogger(object):40class WritableLogger(object):
39 """A thin wrapper that responds to `write` and logs."""41 """A thin wrapper that responds to `write` and logs."""
@@ -205,73 +207,55 @@
205 return app207 return app
206208
207209
208class Controller(object):210class Request(webob.Request):
209 """211 """Add some Openstack API-specific logic to the base webob.Request."""
210 WSGI app that reads routing information supplied by RoutesMiddleware212
211 and calls the requested action method upon itself. All action methods213 def best_match_content_type(self):
212 must, in addition to their normal parameters, accept a 'req' argument214 """Determine the requested response content-type."""
213 which is the incoming webob.Request. They raise a webob.exc exception,215 supported = ('application/json',)
214 or return a dict which will be serialized by requested content type.216 bm = self.accept.best_match(supported)
215 """217 return bm or 'application/json'
216218
217 @webob.dec.wsgify219 def get_content_type(self, allowed_content_types):
218 def __call__(self, req):220 """Determine content type of the request body."""
219 """221 if not "Content-Type" in self.headers:
220 Call the method specified in req.environ by RoutesMiddleware.222 raise exception.InvalidContentType(content_type=None)
221 """223
222 arg_dict = req.environ['wsgiorg.routing_args'][1]224 content_type = self.content_type
223 action = arg_dict['action']225
224 method = getattr(self, action)226 if content_type not in allowed_content_types:
225 del arg_dict['controller']227 raise exception.InvalidContentType(content_type=content_type)
226 del arg_dict['action']228 else:
227 arg_dict['req'] = req229 return content_type
228 result = method(**arg_dict)230
229 if type(result) is dict:231
230 return self._serialize(result, req)232class JSONRequestDeserializer(object):
231 else:233 def has_body(self, request):
232 return result234 """
233235 Returns whether a Webob.Request object will possess an entity body.
234 def _serialize(self, data, request):236
235 """237 :param request: Webob.Request object
236 Serialize the given dict to the response type requested in request.238 """
237 Uses self._serialization_metadata if it exists, which is a dict mapping239 if 'transfer-encoding' in request.headers:
238 MIME types to information needed to serialize to that type.240 return True
239 """241 elif request.content_length > 0:
240 _metadata = getattr(type(self), "_serialization_metadata", {})242 return True
241 serializer = Serializer(request.environ, _metadata)243
242 return serializer.to_content_type(data)244 return False
243245
244246 def from_json(self, datastring):
245class Serializer(object):247 return json.loads(datastring)
246 """248
247 Serializes a dictionary to a Content Type specified by a WSGI environment.249 def default(self, request):
248 """250 if self.has_body(request):
249251 return {'body': self.from_json(request.body)}
250 def __init__(self, environ, metadata=None):252 else:
251 """253 return {}
252 Create a serializer based on the given WSGI environment.254
253 'metadata' is an optional dict mapping MIME types to information255
254 needed to serialize a dictionary to that type.256class JSONResponseSerializer(object):
255 """257
256 self.environ = environ258 def to_json(self, data):
257 self.metadata = metadata or {}
258 self._methods = {
259 'application/json': self._to_json,
260 'application/xml': self._to_xml}
261
262 def to_content_type(self, data):
263 """
264 Serialize a dictionary into a string. The format of the string
265 will be decided based on the Content Type requested in self.environ:
266 by Accept: header, or by URL suffix.
267 """
268 # FIXME(sirp): for now, supporting json only
269 #mimetype = 'application/xml'
270 mimetype = 'application/json'
271 # TODO(gundlach): determine mimetype from request
272 return self._methods.get(mimetype, repr)(data)
273
274 def _to_json(self, data):
275 def sanitizer(obj):259 def sanitizer(obj):
276 if isinstance(obj, datetime.datetime):260 if isinstance(obj, datetime.datetime):
277 return obj.isoformat()261 return obj.isoformat()
@@ -279,37 +263,85 @@
279263
280 return json.dumps(data, default=sanitizer)264 return json.dumps(data, default=sanitizer)
281265
282 def _to_xml(self, data):266 def default(self, response, result):
283 metadata = self.metadata.get('application/xml', {})267 response.headers.add('Content-Type', 'application/json')
284 # We expect data to contain a single key which is the XML root.268 response.body = self.to_json(result)
285 root_key = data.keys()[0]269
286 from xml.dom import minidom270
287 doc = minidom.Document()271class Resource(object):
288 node = self._to_xml_node(doc, metadata, root_key, data[root_key])272 """
289 return node.toprettyxml(indent=' ')273 WSGI app that handles (de)serialization and controller dispatch.
290274
291 def _to_xml_node(self, doc, metadata, nodename, data):275 Reads routing information supplied by RoutesMiddleware and calls
292 """Recursive method to convert data members to XML nodes."""276 the requested action method upon its deserializer, controller,
293 result = doc.createElement(nodename)277 and serializer. Those three objects may implement any of the basic
294 if type(data) is list:278 controller action methods (create, update, show, index, delete)
295 singular = metadata.get('plurals', {}).get(nodename, None)279 along with any that may be specified in the api router. A 'default'
296 if singular is None:280 method may also be implemented to be used in place of any
297 if nodename.endswith('s'):281 non-implemented actions. Deserializer methods must accept a request
298 singular = nodename[:-1]282 argument and return a dictionary. Controller methods must accept a
299 else:283 request argument. Additionally, they must also accept keyword
300 singular = 'item'284 arguments that represent the keys returned by the Deserializer. They
301 for item in data:285 may raise a webob.exc exception or return a dict, which will be
302 node = self._to_xml_node(doc, metadata, singular, item)286 serialized by requested content type.
303 result.appendChild(node)287 """
304 elif type(data) is dict:288 def __init__(self, controller, deserializer, serializer):
305 attrs = metadata.get('attributes', {}).get(nodename, {})289 """
306 for k, v in data.items():290 :param controller: object that implement methods created by routes lib
307 if k in attrs:291 :param deserializer: object that supports webob request deserialization
308 result.setAttribute(k, str(v))292 through controller-like actions
309 else:293 :param serializer: object that supports webob response serialization
310 node = self._to_xml_node(doc, metadata, k, v)294 through controller-like actions
311 result.appendChild(node)295 """
312 else: # atom296 self.controller = controller
313 node = doc.createTextNode(str(data))297 self.serializer = serializer
314 result.appendChild(node)298 self.deserializer = deserializer
315 return result299
300 @webob.dec.wsgify(RequestClass=Request)
301 def __call__(self, request):
302 """WSGI method that controls (de)serialization and method dispatch."""
303 action_args = self.get_action_args(request.environ)
304 action = action_args.pop('action', None)
305
306 deserialized_request = self.dispatch(self.deserializer,
307 action, request)
308 action_args.update(deserialized_request)
309
310 action_result = self.dispatch(self.controller, action,
311 request, **action_args)
312 try:
313 response = webob.Response()
314 self.dispatch(self.serializer, action, response, action_result)
315 return response
316
317 # return unserializable result (typically a webob exc)
318 except Exception:
319 return action_result
320
321 def dispatch(self, obj, action, *args, **kwargs):
322 """Find action-specific method on self and call it."""
323 try:
324 method = getattr(obj, action)
325 except AttributeError:
326 method = getattr(obj, 'default')
327
328 return method(*args, **kwargs)
329
330 def get_action_args(self, request_environment):
331 """Parse dictionary created by routes library."""
332 try:
333 args = request_environment['wsgiorg.routing_args'][1].copy()
334 except Exception:
335 return {}
336
337 try:
338 del args['controller']
339 except KeyError:
340 pass
341
342 try:
343 del args['format']
344 except KeyError:
345 pass
346
347 return args
316348
=== modified file 'glance/registry/client.py'
--- glance/registry/client.py 2011-05-27 19:48:51 +0000
+++ glance/registry/client.py 2011-06-17 20:46:38 +0000
@@ -94,10 +94,16 @@
94 """94 """
95 Tells registry about an image's metadata95 Tells registry about an image's metadata
96 """96 """
97 headers = {
98 'Content-Type': 'application/json',
99 }
100
97 if 'image' not in image_metadata.keys():101 if 'image' not in image_metadata.keys():
98 image_metadata = dict(image=image_metadata)102 image_metadata = dict(image=image_metadata)
103
99 body = json.dumps(image_metadata)104 body = json.dumps(image_metadata)
100 res = self.do_request("POST", "/images", body)105
106 res = self.do_request("POST", "/images", body, headers=headers)
101 # Registry returns a JSONified dict(image=image_info)107 # Registry returns a JSONified dict(image=image_info)
102 data = json.loads(res.read())108 data = json.loads(res.read())
103 return data['image']109 return data['image']
@@ -111,9 +117,13 @@
111117
112 body = json.dumps(image_metadata)118 body = json.dumps(image_metadata)
113119
114 headers = {}120 headers = {
121 'Content-Type': 'application/json',
122 }
123
115 if purge_props:124 if purge_props:
116 headers["X-Glance-Registry-Purge-Props"] = "true"125 headers["X-Glance-Registry-Purge-Props"] = "true"
126
117 res = self.do_request("PUT", "/images/%s" % image_id, body, headers)127 res = self.do_request("PUT", "/images/%s" % image_id, body, headers)
118 data = json.loads(res.read())128 data = json.loads(res.read())
119 image = data['image']129 image = data['image']
120130
=== modified file 'glance/registry/server.py'
--- glance/registry/server.py 2011-05-18 00:19:44 +0000
+++ glance/registry/server.py 2011-06-17 20:46:38 +0000
@@ -42,7 +42,7 @@
42MAX_ITEM_LIMIT = 2542MAX_ITEM_LIMIT = 25
4343
4444
45class Controller(wsgi.Controller):45class Controller(object):
46 """Controller for the reference implementation registry server"""46 """Controller for the reference implementation registry server"""
4747
48 def __init__(self, options):48 def __init__(self, options):
@@ -167,7 +167,7 @@
167 """167 """
168 Deletes an existing image with the registry.168 Deletes an existing image with the registry.
169169
170 :param req: Request body. Ignored.170 :param req: wsgi Request object
171 :param id: The opaque internal identifier for the image171 :param id: The opaque internal identifier for the image
172172
173 :retval Returns 200 if delete was successful, a fault if not.173 :retval Returns 200 if delete was successful, a fault if not.
@@ -179,19 +179,19 @@
179 except exception.NotFound:179 except exception.NotFound:
180 return exc.HTTPNotFound()180 return exc.HTTPNotFound()
181181
182 def create(self, req):182 def create(self, req, body):
183 """183 """
184 Registers a new image with the registry.184 Registers a new image with the registry.
185185
186 :param req: Request body. A JSON-ified dict of information about186 :param req: wsgi Request object
187 the image.187 :param body: Dictionary of information about the image
188188
189 :retval Returns the newly-created image information as a mapping,189 :retval Returns the newly-created image information as a mapping,
190 which will include the newly-created image's internal id190 which will include the newly-created image's internal id
191 in the 'id' field191 in the 'id' field
192192
193 """193 """
194 image_data = json.loads(req.body)['image']194 image_data = body['image']
195195
196 # Ensure the image has a status set196 # Ensure the image has a status set
197 image_data.setdefault('status', 'active')197 image_data.setdefault('status', 'active')
@@ -209,18 +209,17 @@
209 logger.error(msg)209 logger.error(msg)
210 return exc.HTTPBadRequest(msg)210 return exc.HTTPBadRequest(msg)
211211
212 def update(self, req, id):212 def update(self, req, id, body):
213 """Updates an existing image with the registry.213 """Updates an existing image with the registry.
214214
215 :param req: Request body. A JSON-ified dict of information about215 :param req: wsgi Request object
216 the image. This will replace the information in the216 :param body: Dictionary of information about the image
217 registry about this image
218 :param id: The opaque internal identifier for the image217 :param id: The opaque internal identifier for the image
219218
220 :retval Returns the updated image information as a mapping,219 :retval Returns the updated image information as a mapping,
221220
222 """221 """
223 image_data = json.loads(req.body)['image']222 image_data = body['image']
224223
225 purge_props = req.headers.get("X-Glance-Registry-Purge-Props", "false")224 purge_props = req.headers.get("X-Glance-Registry-Purge-Props", "false")
226 context = None225 context = None
@@ -244,15 +243,22 @@
244 content_type='text/plain')243 content_type='text/plain')
245244
246245
246def create_resource(options):
247 """Images resource factory method."""
248 deserializer = wsgi.JSONRequestDeserializer()
249 serializer = wsgi.JSONResponseSerializer()
250 return wsgi.Resource(Controller(options), deserializer, serializer)
251
252
247class API(wsgi.Router):253class API(wsgi.Router):
248 """WSGI entry point for all Registry requests."""254 """WSGI entry point for all Registry requests."""
249255
250 def __init__(self, options):256 def __init__(self, options):
251 mapper = routes.Mapper()257 mapper = routes.Mapper()
252 controller = Controller(options)258 resource = create_resource(options)
253 mapper.resource("image", "images", controller=controller,259 mapper.resource("image", "images", controller=resource,
254 collection={'detail': 'GET'})260 collection={'detail': 'GET'})
255 mapper.connect("/", controller=controller, action="index")261 mapper.connect("/", controller=resource, action="index")
256 super(API, self).__init__(mapper)262 super(API, self).__init__(mapper)
257263
258264
259265
=== modified file 'glance/utils.py'
--- glance/utils.py 2011-04-12 21:45:16 +0000
+++ glance/utils.py 2011-06-17 20:46:38 +0000
@@ -43,24 +43,6 @@
43 return headers43 return headers
4444
4545
46def inject_image_meta_into_headers(response, image_meta):
47 """
48 Given a response and mapping of image metadata, injects
49 the Response with a set of HTTP headers for the image
50 metadata. Each main image metadata field is injected
51 as a HTTP header with key 'x-image-meta-<FIELD>' except
52 for the properties field, which is further broken out
53 into a set of 'x-image-meta-property-<KEY>' headers
54
55 :param response: The Webob Response object
56 :param image_meta: Mapping of image metadata
57 """
58 headers = image_meta_to_http_headers(image_meta)
59
60 for k, v in headers.items():
61 response.headers.add(k, v)
62
63
64def get_image_meta_from_headers(response):46def get_image_meta_from_headers(response):
65 """47 """
66 Processes HTTP headers from a supplied response that48 Processes HTTP headers from a supplied response that
6749
=== modified file 'tests/functional/test_curl_api.py'
--- tests/functional/test_curl_api.py 2011-05-27 19:48:51 +0000
+++ tests/functional/test_curl_api.py 2011-06-17 20:46:38 +0000
@@ -773,7 +773,7 @@
773 with tempfile.NamedTemporaryFile() as test_data_file:773 with tempfile.NamedTemporaryFile() as test_data_file:
774 test_data_file.write("XXX")774 test_data_file.write("XXX")
775 test_data_file.flush()775 test_data_file.flush()
776 cmd = ("curl -i -X POST --upload-file %s "776 cmd = ("curl -i -X POST --upload-file %s -H 'Expect: ' "
777 "http://0.0.0.0:%d/v1/images") % (test_data_file.name,777 "http://0.0.0.0:%d/v1/images") % (test_data_file.name,
778 api_port)778 api_port)
779779
780780
=== modified file 'tests/unit/test_api.py'
--- tests/unit/test_api.py 2011-05-31 19:32:17 +0000
+++ tests/unit/test_api.py 2011-06-17 20:46:38 +0000
@@ -678,6 +678,7 @@
678 req = webob.Request.blank('/images')678 req = webob.Request.blank('/images')
679679
680 req.method = 'POST'680 req.method = 'POST'
681 req.content_type = 'application/json'
681 req.body = json.dumps(dict(image=fixture))682 req.body = json.dumps(dict(image=fixture))
682683
683 res = req.get_response(self.api)684 res = req.get_response(self.api)
@@ -706,6 +707,7 @@
706 req = webob.Request.blank('/images')707 req = webob.Request.blank('/images')
707708
708 req.method = 'POST'709 req.method = 'POST'
710 req.content_type = 'application/json'
709 req.body = json.dumps(dict(image=fixture))711 req.body = json.dumps(dict(image=fixture))
710712
711 res = req.get_response(self.api)713 res = req.get_response(self.api)
@@ -723,6 +725,7 @@
723 req = webob.Request.blank('/images')725 req = webob.Request.blank('/images')
724726
725 req.method = 'POST'727 req.method = 'POST'
728 req.content_type = 'application/json'
726 req.body = json.dumps(dict(image=fixture))729 req.body = json.dumps(dict(image=fixture))
727730
728 res = req.get_response(self.api)731 res = req.get_response(self.api)
@@ -739,6 +742,7 @@
739 req = webob.Request.blank('/images')742 req = webob.Request.blank('/images')
740743
741 req.method = 'POST'744 req.method = 'POST'
745 req.content_type = 'application/json'
742 req.body = json.dumps(dict(image=fixture))746 req.body = json.dumps(dict(image=fixture))
743747
744 res = req.get_response(self.api)748 res = req.get_response(self.api)
@@ -758,6 +762,7 @@
758 req = webob.Request.blank('/images')762 req = webob.Request.blank('/images')
759763
760 req.method = 'POST'764 req.method = 'POST'
765 req.content_type = 'application/json'
761 req.body = json.dumps(dict(image=fixture))766 req.body = json.dumps(dict(image=fixture))
762767
763 res = req.get_response(self.api)768 res = req.get_response(self.api)
@@ -772,6 +777,7 @@
772 req = webob.Request.blank('/images/2')777 req = webob.Request.blank('/images/2')
773778
774 req.method = 'PUT'779 req.method = 'PUT'
780 req.content_type = 'application/json'
775 req.body = json.dumps(dict(image=fixture))781 req.body = json.dumps(dict(image=fixture))
776782
777 res = req.get_response(self.api)783 res = req.get_response(self.api)
@@ -791,6 +797,7 @@
791 req = webob.Request.blank('/images/3')797 req = webob.Request.blank('/images/3')
792798
793 req.method = 'PUT'799 req.method = 'PUT'
800 req.content_type = 'application/json'
794 req.body = json.dumps(dict(image=fixture))801 req.body = json.dumps(dict(image=fixture))
795802
796 res = req.get_response(self.api)803 res = req.get_response(self.api)
@@ -804,6 +811,7 @@
804 req = webob.Request.blank('/images/2')811 req = webob.Request.blank('/images/2')
805812
806 req.method = 'PUT'813 req.method = 'PUT'
814 req.content_type = 'application/json'
807 req.body = json.dumps(dict(image=fixture))815 req.body = json.dumps(dict(image=fixture))
808816
809 res = req.get_response(self.api)817 res = req.get_response(self.api)
@@ -817,6 +825,7 @@
817 req = webob.Request.blank('/images/2')825 req = webob.Request.blank('/images/2')
818826
819 req.method = 'PUT'827 req.method = 'PUT'
828 req.content_type = 'application/json'
820 req.body = json.dumps(dict(image=fixture))829 req.body = json.dumps(dict(image=fixture))
821830
822 res = req.get_response(self.api)831 res = req.get_response(self.api)
@@ -830,6 +839,7 @@
830 req = webob.Request.blank('/images/2')839 req = webob.Request.blank('/images/2')
831840
832 req.method = 'PUT'841 req.method = 'PUT'
842 req.content_type = 'application/json'
833 req.body = json.dumps(dict(image=fixture))843 req.body = json.dumps(dict(image=fixture))
834844
835 res = req.get_response(self.api)845 res = req.get_response(self.api)
@@ -844,6 +854,7 @@
844 req = webob.Request.blank('/images/2') # Image 2 has disk format 'vhd'854 req = webob.Request.blank('/images/2') # Image 2 has disk format 'vhd'
845855
846 req.method = 'PUT'856 req.method = 'PUT'
857 req.content_type = 'application/json'
847 req.body = json.dumps(dict(image=fixture))858 req.body = json.dumps(dict(image=fixture))
848859
849 res = req.get_response(self.api)860 res = req.get_response(self.api)
@@ -972,6 +983,21 @@
972 res_body = json.loads(res.body)['image']983 res_body = json.loads(res.body)['image']
973 self.assertEquals('queued', res_body['status'])984 self.assertEquals('queued', res_body['status'])
974985
986 def test_add_image_no_location_no_content_type(self):
987 """Tests creates a queued image for no body and no loc header"""
988 fixture_headers = {'x-image-meta-store': 'file',
989 'x-image-meta-disk-format': 'vhd',
990 'x-image-meta-container-format': 'ovf',
991 'x-image-meta-name': 'fake image #3'}
992
993 req = webob.Request.blank("/images")
994 req.body = "chunk00000remainder"
995 req.method = 'POST'
996 for k, v in fixture_headers.iteritems():
997 req.headers[k] = v
998 res = req.get_response(self.api)
999 self.assertEquals(res.status_int, 400)
1000
975 def test_add_image_bad_store(self):1001 def test_add_image_bad_store(self):
976 """Tests raises BadRequest for invalid store header"""1002 """Tests raises BadRequest for invalid store header"""
977 fixture_headers = {'x-image-meta-store': 'bad',1003 fixture_headers = {'x-image-meta-store': 'bad',
9781004
=== added file 'tests/unit/test_wsgi.py'
--- tests/unit/test_wsgi.py 1970-01-01 00:00:00 +0000
+++ tests/unit/test_wsgi.py 2011-06-17 20:46:38 +0000
@@ -0,0 +1,182 @@
1# vim: tabstop=4 shiftwidth=4 softtabstop=4
2
3# Copyright 2010-2011 OpenStack, LLC
4# All Rights Reserved.
5#
6# Licensed under the Apache License, Version 2.0 (the "License"); you may
7# not use this file except in compliance with the License. You may obtain
8# a copy of the License at
9#
10# http://www.apache.org/licenses/LICENSE-2.0
11#
12# Unless required by applicable law or agreed to in writing, software
13# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
14# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
15# License for the specific language governing permissions and limitations
16# under the License.
17
18import unittest
19import webob
20
21from glance.common import wsgi
22from glance.common import exception
23
24
25class RequestTest(unittest.TestCase):
26 def test_content_type_missing(self):
27 request = wsgi.Request.blank('/tests/123')
28 request.body = "<body />"
29 self.assertRaises(exception.InvalidContentType,
30 request.get_content_type, ('application/xml'))
31
32 def test_content_type_unsupported(self):
33 request = wsgi.Request.blank('/tests/123')
34 request.headers["Content-Type"] = "text/html"
35 request.body = "asdf<br />"
36 self.assertRaises(exception.InvalidContentType,
37 request.get_content_type, ('application/xml'))
38
39 def test_content_type_with_charset(self):
40 request = wsgi.Request.blank('/tests/123')
41 request.headers["Content-Type"] = "application/json; charset=UTF-8"
42 result = request.get_content_type(('application/json'))
43 self.assertEqual(result, "application/json")
44
45 def test_content_type_from_accept_xml(self):
46 request = wsgi.Request.blank('/tests/123')
47 request.headers["Accept"] = "application/xml"
48 result = request.best_match_content_type()
49 self.assertEqual(result, "application/json")
50
51 def test_content_type_from_accept_json(self):
52 request = wsgi.Request.blank('/tests/123')
53 request.headers["Accept"] = "application/json"
54 result = request.best_match_content_type()
55 self.assertEqual(result, "application/json")
56
57 def test_content_type_from_accept_xml_json(self):
58 request = wsgi.Request.blank('/tests/123')
59 request.headers["Accept"] = "application/xml, application/json"
60 result = request.best_match_content_type()
61 self.assertEqual(result, "application/json")
62
63 def test_content_type_from_accept_json_xml_quality(self):
64 request = wsgi.Request.blank('/tests/123')
65 request.headers["Accept"] = \
66 "application/json; q=0.3, application/xml; q=0.9"
67 result = request.best_match_content_type()
68 self.assertEqual(result, "application/json")
69
70 def test_content_type_accept_default(self):
71 request = wsgi.Request.blank('/tests/123.unsupported')
72 request.headers["Accept"] = "application/unsupported1"
73 result = request.best_match_content_type()
74 self.assertEqual(result, "application/json")
75
76
77class ResourceTest(unittest.TestCase):
78 def test_get_action_args(self):
79 env = {
80 'wsgiorg.routing_args': [
81 None,
82 {
83 'controller': None,
84 'format': None,
85 'action': 'update',
86 'id': 12,
87 },
88 ],
89 }
90
91 expected = {'action': 'update', 'id': 12}
92 actual = wsgi.Resource(None, None, None).get_action_args(env)
93
94 self.assertEqual(actual, expected)
95
96 def test_dispatch(self):
97 class Controller(object):
98 def index(self, shirt, pants=None):
99 return (shirt, pants)
100
101 resource = wsgi.Resource(None, None, None)
102 actual = resource.dispatch(Controller(), 'index', 'on', pants='off')
103 expected = ('on', 'off')
104 self.assertEqual(actual, expected)
105
106 def test_dispatch_default(self):
107 class Controller(object):
108 def default(self, shirt, pants=None):
109 return (shirt, pants)
110
111 resource = wsgi.Resource(None, None, None)
112 actual = resource.dispatch(Controller(), 'index', 'on', pants='off')
113 expected = ('on', 'off')
114 self.assertEqual(actual, expected)
115
116 def test_dispatch_no_default(self):
117 class Controller(object):
118 def show(self, shirt, pants=None):
119 return (shirt, pants)
120
121 resource = wsgi.Resource(None, None, None)
122 self.assertRaises(AttributeError, resource.dispatch, Controller(),
123 'index', 'on', pants='off')
124
125
126class JSONResponseSerializerTest(unittest.TestCase):
127 def test_to_json(self):
128 fixture = {"key": "value"}
129 expected = '{"key": "value"}'
130 actual = wsgi.JSONResponseSerializer().to_json(fixture)
131 self.assertEqual(actual, expected)
132
133 def test_default(self):
134 fixture = {"key": "value"}
135 response = webob.Response()
136 wsgi.JSONResponseSerializer().default(response, fixture)
137 self.assertEqual(response.status_int, 200)
138 self.assertEqual(response.content_type, 'application/json')
139 self.assertEqual(response.body, '{"key": "value"}')
140
141
142class JSONRequestDeserializerTest(unittest.TestCase):
143 def test_has_body_no_content_length(self):
144 request = wsgi.Request.blank('/')
145 request.body = 'asdf'
146 request.headers.pop('Content-Length')
147 self.assertFalse(wsgi.JSONRequestDeserializer().has_body(request))
148
149 def test_has_body_zero_content_length(self):
150 request = wsgi.Request.blank('/')
151 request.body = 'asdf'
152 request.headers['Content-Length'] = 0
153 self.assertFalse(wsgi.JSONRequestDeserializer().has_body(request))
154
155 def test_has_body_has_content_length(self):
156 request = wsgi.Request.blank('/')
157 request.body = 'asdf'
158 self.assertTrue('Content-Length' in request.headers)
159 self.assertTrue(wsgi.JSONRequestDeserializer().has_body(request))
160
161 def test_no_body_no_content_length(self):
162 request = wsgi.Request.blank('/')
163 self.assertFalse(wsgi.JSONRequestDeserializer().has_body(request))
164
165 def test_from_json(self):
166 fixture = '{"key": "value"}'
167 expected = {"key": "value"}
168 actual = wsgi.JSONRequestDeserializer().from_json(fixture)
169 self.assertEqual(actual, expected)
170
171 def test_default_no_body(self):
172 request = wsgi.Request.blank('/')
173 actual = wsgi.JSONRequestDeserializer().default(request)
174 expected = {}
175 self.assertEqual(actual, expected)
176
177 def test_default_with_body(self):
178 request = wsgi.Request.blank('/')
179 request.body = '{"key": "value"}'
180 actual = wsgi.JSONRequestDeserializer().default(request)
181 expected = {"body": {"key": "value"}}
182 self.assertEqual(actual, expected)

Subscribers

People subscribed via source and target branches