Merge lp:~jaypipes/glance/clients into lp:~hudson-openstack/glance/trunk
- clients
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Christopher MacGown |
Approved revision: | 20 |
Merged at revision: | 19 |
Proposed branch: | lp:~jaypipes/glance/clients |
Merge into: | lp:~hudson-openstack/glance/trunk |
Diff against target: |
866 lines (+580/-48) 10 files modified
glance/client.py (+223/-0) glance/common/exception.py (+0/-2) glance/parallax/controllers.py (+20/-8) glance/parallax/db/api.py (+3/-4) glance/parallax/db/sqlalchemy/api.py (+8/-9) glance/parallax/db/sqlalchemy/models.py (+7/-8) tests/stubs.py (+71/-3) tests/unit/test_clients.py (+233/-0) tests/unit/test_parallax_api.py (+8/-5) tests/unit/test_parallax_models.py (+7/-9) |
To merge this branch: | bzr merge lp:~jaypipes/glance/clients |
Related bugs: | |
Related blueprints: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Christopher MacGown (community) | Approve | ||
Rick Harris (community) | Approve | ||
Michael Gundlach | Pending | ||
Glance Core security contacts | Pending | ||
Review via email: mp+41498@code.launchpad.net |
Commit message
Adds client classes for Parallax and Teller and fixes some issues where our controller was not returning proper HTTP response codes on errors...
Description of the change
Adds client classes for Parallax and Teller and fixes some issues where our controller was not returning proper HTTP response codes on errors...
Rick Harris (rconradharris) wrote : | # |
Jay Pipes (jaypipes) wrote : | # |
On Mon, Nov 22, 2010 at 4:04 PM, Rick Harris <email address hidden> wrote:
> Overall lgtm. Have a few points I'd like to open up for discussion:
>
> 1. The catching of KeyError on line 77 seems too broad; it should just wrap the dictionary lookup, not the lookup and the ping. If the connection object (or more likely a mocked version of the connection object) erroneously raised KeyError, the re-raised exception would be misleading.
Yep, I'll fix that up.
> 2. Pinging the endpoint from within the __init__ feels messy: the usual errors associated with network traffic are unexpected when all you're trying to do is construct and initialize an object. I would feel better if this test were in a separate connect() method. Another option would be to provide a kwarg test_connection
Hmm, yeah, so I toyed around with all three of those options while
developing this, actually :)
I'll rework and propose something along the lines of option #3 since
that indeed is how most network libraries behave.
> 3. Very minor but the pattern
> 137 + c = self.connection
> 138 + c.request("GET", "images")
> 139 + res = c.getresponse()
> is repeated in a bunch of places.
>
> Perhaps a do_request method would DRY up the code a bit.
>
> def do_request(method, path, headers=None, data=None):
> """Something like this"""
> conn = self.connection
> conn.request(
> # could check for non-200 status code and raise exceptions here as well.
> return conn.getresponse()
Yup. Will do.
> 4. At some point we should clarify terminology: what exactly is meant by "image metadata"? I think in difference parts of the code and design docs we're using it to mean different things. In one case (I think how it's being used here), it's used to refer to the image registry information (name, backend_uri, created_at, updated_at) as opposed to "image data" which is used to refer to the actual image files.
> The other use of "image metadata" is within the parallax db.model where I added the table "image_metadatum". This is a table used to store arbitrary key/value pairs associated with an image. This makes it image metadata metadata, I guess.
>
> Perhaps we should rename the latter "image_properties", or maybe the former should be called "image_descriptor". Whatever the names, we should just clarify this soon.
I will clarify in the docs. Metadata is ALL the information *about*
the image. I will rename the ImageMetadatum table to ImageTag, and
change the attributes to "tags" instead of "metadata" which will match
the EC2 API as well...
Come to think of it, the ImageMetadatum table is not even used or
tested in the code... I'll fix that :)
Jay Pipes (jaypipes) wrote : | # |
I like "properties" better than "tags"...
Rick Harris (rconradharris) wrote : | # |
Looks great!
Preview Diff
1 | === added file 'glance/client.py' |
2 | --- glance/client.py 1970-01-01 00:00:00 +0000 |
3 | +++ glance/client.py 2010-11-23 18:13:46 +0000 |
4 | @@ -0,0 +1,223 @@ |
5 | +# vim: tabstop=4 shiftwidth=4 softtabstop=4 |
6 | + |
7 | +# Copyright 2010 OpenStack, LLC |
8 | +# All Rights Reserved. |
9 | +# |
10 | +# Licensed under the Apache License, Version 2.0 (the "License"); you may |
11 | +# not use this file except in compliance with the License. You may obtain |
12 | +# a copy of the License at |
13 | +# |
14 | +# http://www.apache.org/licenses/LICENSE-2.0 |
15 | +# |
16 | +# Unless required by applicable law or agreed to in writing, software |
17 | +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT |
18 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
19 | +# License for the specific language governing permissions and limitations |
20 | +# under the License. |
21 | + |
22 | +""" |
23 | +Client classes for callers of a Glance system |
24 | +""" |
25 | + |
26 | +import httplib |
27 | +import json |
28 | +import logging |
29 | +import urlparse |
30 | +import socket |
31 | +import sys |
32 | + |
33 | +from glance.common import exception |
34 | + |
35 | +#TODO(jaypipes) Allow a logger param for client classes |
36 | +#TODO(jaypipes) Raise proper errors or OpenStack API faults |
37 | + |
38 | + |
39 | +class UnsupportedProtocolError(Exception): |
40 | + """ |
41 | + Error resulting from a client connecting to a server |
42 | + on an unsupported protocol |
43 | + """ |
44 | + pass |
45 | + |
46 | + |
47 | +class ClientConnectionError(Exception): |
48 | + """Error resulting from a client connecting to a server""" |
49 | + pass |
50 | + |
51 | + |
52 | +class BadInputError(Exception): |
53 | + """Error resulting from a client sending bad input to a server""" |
54 | + pass |
55 | + |
56 | + |
57 | +class BaseClient(object): |
58 | + |
59 | + """A base client class""" |
60 | + |
61 | + DEFAULT_ADDRESS = 'http://127.0.0.1' |
62 | + DEFAULT_PORT = 9090 |
63 | + |
64 | + def __init__(self, **kwargs): |
65 | + """ |
66 | + Creates a new client to some service. All args are keyword |
67 | + arguments. |
68 | + |
69 | + :param address: The address where service resides (defaults to |
70 | + http://127.0.0.1) |
71 | + :param port: The port where service resides (defaults to 9090) |
72 | + """ |
73 | + self.address = kwargs.get('address', self.DEFAULT_ADDRESS) |
74 | + self.port = kwargs.get('port', self.DEFAULT_PORT) |
75 | + url = urlparse.urlparse(self.address) |
76 | + self.netloc = url.netloc |
77 | + self.protocol = url.scheme |
78 | + self.connection = None |
79 | + |
80 | + def do_request(self, method, action, body=None): |
81 | + """ |
82 | + Connects to the server and issues a request. Handles converting |
83 | + any returned HTTP error status codes to OpenStack/Glance exceptions |
84 | + and closing the server connection. Returns the result data, or |
85 | + raises an appropriate exception. |
86 | + |
87 | + :param method: HTTP method ("GET", "POST", "PUT", etc...) |
88 | + :param action: part of URL after root netloc |
89 | + :param headers: mapping of headers to send |
90 | + :param data: string of data to send, or None (default) |
91 | + """ |
92 | + try: |
93 | + connection_type = {'http': httplib.HTTPConnection, |
94 | + 'https': httplib.HTTPSConnection}\ |
95 | + [self.protocol] |
96 | + except KeyError: |
97 | + raise UnsupportedProtocolError("Unsupported protocol %s. Unable " |
98 | + " to connect to server." |
99 | + % self.protocol) |
100 | + try: |
101 | + c = connection_type(self.netloc, self.port) |
102 | + c.request(method, action, body) |
103 | + res = c.getresponse() |
104 | + status_code = self.get_status_code(res) |
105 | + if status_code == httplib.OK: |
106 | + return res |
107 | + elif status_code == httplib.UNAUTHORIZED: |
108 | + raise exception.NotAuthorized |
109 | + elif status_code == httplib.FORBIDDEN: |
110 | + raise exception.NotAuthorized |
111 | + elif status_code == httplib.NOT_FOUND: |
112 | + raise exception.NotFound |
113 | + elif status_code == httplib.CONFLICT: |
114 | + raise exception.Duplicate |
115 | + elif status_code == httplib.BAD_REQUEST: |
116 | + raise BadInputError |
117 | + else: |
118 | + raise Exception("Unknown error occurred! %d" % status_code) |
119 | + |
120 | + except (socket.error, IOError), e: |
121 | + raise ClientConnectionError("Unable to connect to " |
122 | + "server. Got error: %s" % e) |
123 | + finally: |
124 | + c.close() |
125 | + |
126 | + def get_status_code(self, response): |
127 | + """ |
128 | + Returns the integer status code from the response, which |
129 | + can be either a Webob.Response (used in testing) or httplib.Response |
130 | + """ |
131 | + if hasattr(response, 'status_int'): |
132 | + return response.status_int |
133 | + else: |
134 | + return response.status |
135 | + |
136 | + |
137 | +class TellerClient(BaseClient): |
138 | + |
139 | + """A client for the Teller image caching service""" |
140 | + |
141 | + DEFAULT_ADDRESS = 'http://127.0.0.1' |
142 | + DEFAULT_PORT = 9191 |
143 | + |
144 | + def __init__(self, **kwargs): |
145 | + """ |
146 | + Creates a new client to a Teller service. All args are keyword |
147 | + arguments. |
148 | + |
149 | + :param address: The address where Teller resides (defaults to |
150 | + http://127.0.0.1) |
151 | + :param port: The port where Teller resides (defaults to 9191) |
152 | + """ |
153 | + super(TellerClient, self).__init__(**kwargs) |
154 | + |
155 | + |
156 | +class ParallaxClient(BaseClient): |
157 | + |
158 | + """A client for the Parallax image metadata service""" |
159 | + |
160 | + DEFAULT_ADDRESS = 'http://127.0.0.1' |
161 | + DEFAULT_PORT = 9292 |
162 | + |
163 | + def __init__(self, **kwargs): |
164 | + """ |
165 | + Creates a new client to a Parallax service. All args are keyword |
166 | + arguments. |
167 | + |
168 | + :param address: The address where Parallax resides (defaults to |
169 | + http://127.0.0.1) |
170 | + :param port: The port where Parallax resides (defaults to 9292) |
171 | + """ |
172 | + super(ParallaxClient, self).__init__(**kwargs) |
173 | + |
174 | + def get_images(self): |
175 | + """ |
176 | + Returns a list of image id/name mappings from Parallax |
177 | + """ |
178 | + res = self.do_request("GET", "images") |
179 | + data = json.loads(res.read())['images'] |
180 | + return data |
181 | + |
182 | + def get_images_detailed(self): |
183 | + """ |
184 | + Returns a list of detailed image data mappings from Parallax |
185 | + """ |
186 | + res = self.do_request("GET", "images/detail") |
187 | + data = json.loads(res.read())['images'] |
188 | + return data |
189 | + |
190 | + def get_image(self, image_id): |
191 | + """ |
192 | + Returns a mapping of image metadata from Parallax |
193 | + |
194 | + :raises exception.NotFound if image is not in registry |
195 | + """ |
196 | + res = self.do_request("GET", "images/%s" % image_id) |
197 | + data = json.loads(res.read())['image'] |
198 | + return data |
199 | + |
200 | + def add_image(self, image_metadata): |
201 | + """ |
202 | + Tells parallax about an image's metadata |
203 | + """ |
204 | + if 'image' not in image_metadata.keys(): |
205 | + image_metadata = dict(image=image_metadata) |
206 | + body = json.dumps(image_metadata) |
207 | + res = self.do_request("POST", "images", body) |
208 | + # Parallax returns a JSONified dict(image=image_info) |
209 | + data = json.loads(res.read()) |
210 | + return data['image']['id'] |
211 | + |
212 | + def update_image(self, image_id, image_metadata): |
213 | + """ |
214 | + Updates Parallax's information about an image |
215 | + """ |
216 | + if 'image' not in image_metadata.keys(): |
217 | + image_metadata = dict(image=image_metadata) |
218 | + body = json.dumps(image_metadata) |
219 | + self.do_request("PUT", "images/%s" % image_id, body) |
220 | + return True |
221 | + |
222 | + def delete_image(self, image_id): |
223 | + """ |
224 | + Deletes Parallax's information about an image |
225 | + """ |
226 | + self.do_request("DELETE", "images/%s" % image_id) |
227 | + return True |
228 | |
229 | === modified file 'glance/common/exception.py' |
230 | --- glance/common/exception.py 2010-09-28 16:59:01 +0000 |
231 | +++ glance/common/exception.py 2010-11-23 18:13:46 +0000 |
232 | @@ -83,5 +83,3 @@ |
233 | raise |
234 | _wrap.func_name = f.func_name |
235 | return _wrap |
236 | - |
237 | - |
238 | |
239 | === modified file 'glance/parallax/controllers.py' |
240 | --- glance/parallax/controllers.py 2010-10-17 16:43:19 +0000 |
241 | +++ glance/parallax/controllers.py 2010-11-23 18:13:46 +0000 |
242 | @@ -83,7 +83,10 @@ |
243 | |
244 | """ |
245 | context = None |
246 | - updated_image = db.image_destroy(context, id) |
247 | + try: |
248 | + db.image_destroy(context, id) |
249 | + except exception.NotFound: |
250 | + return exc.HTTPNotFound() |
251 | |
252 | def create(self, req): |
253 | """Registers a new image with the registry. |
254 | @@ -102,8 +105,13 @@ |
255 | image_data.setdefault('status', 'available') |
256 | |
257 | context = None |
258 | - new_image = db.image_create(context, image_data) |
259 | - return dict(image=new_image) |
260 | + try: |
261 | + new_image = db.image_create(context, image_data) |
262 | + return dict(image=new_image) |
263 | + except exception.Duplicate: |
264 | + return exc.HTTPConflict() |
265 | + except exception.Invalid: |
266 | + return exc.HTTPBadRequest() |
267 | |
268 | def update(self, req, id): |
269 | """Updates an existing image with the registry. |
270 | @@ -119,8 +127,11 @@ |
271 | image_data = json.loads(req.body)['image'] |
272 | |
273 | context = None |
274 | - updated_image = db.image_update(context, id, image_data) |
275 | - return dict(image=updated_image) |
276 | + try: |
277 | + updated_image = db.image_update(context, id, image_data) |
278 | + return dict(image=updated_image) |
279 | + except exception.NotFound: |
280 | + return exc.HTTPNotFound() |
281 | |
282 | @staticmethod |
283 | def _make_image_dict(image): |
284 | @@ -137,13 +148,14 @@ |
285 | # TODO(sirp): should this be a dict, or a list of dicts? |
286 | # A plain dict is more convenient, but list of dicts would provide |
287 | # access to created_at, etc |
288 | - metadata = dict((m['key'], m['value']) for m in image['metadata'] |
289 | - if not m['deleted']) |
290 | + properties = dict((p['key'], p['value']) |
291 | + for p in image['properties'] |
292 | + if not p['deleted']) |
293 | |
294 | image_dict = _fetch_attrs(image, db.IMAGE_ATTRS) |
295 | |
296 | image_dict['files'] = files |
297 | - image_dict['metadata'] = metadata |
298 | + image_dict['properties'] = properties |
299 | return image_dict |
300 | |
301 | |
302 | |
303 | === modified file 'glance/parallax/db/api.py' |
304 | --- glance/parallax/db/api.py 2010-10-01 20:27:48 +0000 |
305 | +++ glance/parallax/db/api.py 2010-11-23 18:13:46 +0000 |
306 | @@ -86,7 +86,6 @@ |
307 | ################### |
308 | |
309 | |
310 | -def image_metadatum_create(context, values): |
311 | - """Create an image metadatum from the values dictionary.""" |
312 | - return IMPL.image_metadatum_create(context, values) |
313 | - |
314 | +def image_property_create(context, values): |
315 | + """Create an image property from the values dictionary.""" |
316 | + return IMPL.image_property_create(context, values) |
317 | |
318 | === modified file 'glance/parallax/db/sqlalchemy/api.py' |
319 | --- glance/parallax/db/sqlalchemy/api.py 2010-10-14 17:16:09 +0000 |
320 | +++ glance/parallax/db/sqlalchemy/api.py 2010-11-23 18:13:46 +0000 |
321 | @@ -70,7 +70,7 @@ |
322 | try: |
323 | return session.query(models.Image |
324 | ).options(joinedload(models.Image.files) |
325 | - ).options(joinedload(models.Image.metadata) |
326 | + ).options(joinedload(models.Image.properties) |
327 | ).filter_by(deleted=_deleted(context) |
328 | ).filter_by(id=image_id |
329 | ).one() |
330 | @@ -83,7 +83,7 @@ |
331 | session = get_session() |
332 | return session.query(models.Image |
333 | ).options(joinedload(models.Image.files) |
334 | - ).options(joinedload(models.Image.metadata) |
335 | + ).options(joinedload(models.Image.properties) |
336 | ).filter_by(deleted=_deleted(context) |
337 | ).all() |
338 | |
339 | @@ -92,7 +92,7 @@ |
340 | session = get_session() |
341 | return session.query(models.Image |
342 | ).options(joinedload(models.Image.files) |
343 | - ).options(joinedload(models.Image.metadata) |
344 | + ).options(joinedload(models.Image.properties) |
345 | ).filter_by(deleted=_deleted(context) |
346 | ).filter_by(is_public=public |
347 | ).all() |
348 | @@ -124,10 +124,9 @@ |
349 | ################### |
350 | |
351 | |
352 | -def image_metadatum_create(_context, values): |
353 | - image_metadatum_ref = models.ImageMetadatum() |
354 | +def image_property_create(_context, values): |
355 | + image_properties_ref = models.ImageProperty() |
356 | for (key, value) in values.iteritems(): |
357 | - image_metadatum_ref[key] = value |
358 | - image_metadatum_ref.save() |
359 | - return image_metadatum_ref |
360 | - |
361 | + image_properties_ref[key] = value |
362 | + image_properties_ref.save() |
363 | + return image_properties_ref |
364 | |
365 | === modified file 'glance/parallax/db/sqlalchemy/models.py' |
366 | --- glance/parallax/db/sqlalchemy/models.py 2010-10-15 20:03:13 +0000 |
367 | +++ glance/parallax/db/sqlalchemy/models.py 2010-11-23 18:13:46 +0000 |
368 | @@ -149,7 +149,7 @@ |
369 | raise exception.Invalid("Invalid status '%s' for image." % status) |
370 | return status |
371 | |
372 | - # TODO(sirp): should these be stored as metadata? |
373 | + # TODO(sirp): should these be stored as properties? |
374 | #user_id = Column(String(255)) |
375 | #project_id = Column(String(255)) |
376 | #arch = Column(String(255)) |
377 | @@ -179,22 +179,22 @@ |
378 | size = Column(Integer) |
379 | |
380 | |
381 | -class ImageMetadatum(BASE, ModelBase): |
382 | - """Represents an image metadata in the datastore""" |
383 | - __tablename__ = 'image_metadata' |
384 | - __prefix__ = 'img-meta' |
385 | +class ImageProperty(BASE, ModelBase): |
386 | + """Represents an image properties in the datastore""" |
387 | + __tablename__ = 'image_properties' |
388 | + __prefix__ = 'img-prop' |
389 | __table_args__ = (UniqueConstraint('image_id', 'key'), {}) |
390 | |
391 | id = Column(Integer, primary_key=True) |
392 | image_id = Column(Integer, ForeignKey('images.id'), nullable=False) |
393 | - image = relationship(Image, backref=backref('metadata')) |
394 | + image = relationship(Image, backref=backref('properties')) |
395 | |
396 | key = Column(String(255), index=True) |
397 | value = Column(Text) |
398 | |
399 | |
400 | def register_models(): |
401 | - """Register Models and create metadata""" |
402 | + """Register Models and create properties""" |
403 | engine = get_engine() |
404 | BASE.metadata.create_all(engine) |
405 | |
406 | @@ -203,4 +203,3 @@ |
407 | """Unregister Models, useful clearing out data before testing""" |
408 | engine = get_engine() |
409 | BASE.metadata.drop_all(engine) |
410 | - |
411 | |
412 | === modified file 'tests/stubs.py' |
413 | --- tests/stubs.py 2010-10-15 20:00:47 +0000 |
414 | +++ tests/stubs.py 2010-11-23 18:13:46 +0000 |
415 | @@ -23,11 +23,14 @@ |
416 | import sys |
417 | |
418 | import stubout |
419 | +import webob |
420 | |
421 | from glance.common import exception |
422 | +from glance.parallax import controllers as parallax_controllers |
423 | import glance.teller.backends.swift |
424 | import glance.parallax.db.sqlalchemy.api |
425 | |
426 | + |
427 | def stub_out_http_backend(stubs): |
428 | """Stubs out the httplib.HTTPRequest.getresponse to return |
429 | faked-out data instead of grabbing actual contents of a resource |
430 | @@ -153,6 +156,53 @@ |
431 | fake_parallax_registry.lookup) |
432 | |
433 | |
434 | +def stub_out_parallax_server(stubs): |
435 | + """ |
436 | + Mocks httplib calls to 127.0.0.1:9292 for testing so |
437 | + that a real Parallax server does not need to be up and |
438 | + running |
439 | + """ |
440 | + |
441 | + def fake_http_connection_constructor(address, port): |
442 | + """ |
443 | + Returns either a faked connection or a real |
444 | + one depending on if the connection is to a parallax |
445 | + server or not... |
446 | + """ |
447 | + return FakeParallaxConnection() |
448 | + |
449 | + |
450 | + class FakeParallaxConnection(object): |
451 | + |
452 | + def __init__(self): |
453 | + pass |
454 | + |
455 | + def connect(self): |
456 | + return True |
457 | + |
458 | + def close(self): |
459 | + return True |
460 | + |
461 | + def request(self, method, url, body=None): |
462 | + self.req = webob.Request.blank("/" + url.lstrip("/")) |
463 | + self.req.method = method |
464 | + if body: |
465 | + self.req.body = body |
466 | + |
467 | + def getresponse(self): |
468 | + res = self.req.get_response(parallax_controllers.API()) |
469 | + |
470 | + # httplib.Response has a read() method...fake it out |
471 | + def fake_reader(): |
472 | + return res.body |
473 | + |
474 | + setattr(res, 'read', fake_reader) |
475 | + return res |
476 | + |
477 | + stubs.Set(httplib, 'HTTPConnection', |
478 | + fake_http_connection_constructor) |
479 | + |
480 | + |
481 | def stub_out_parallax_db_image_api(stubs): |
482 | """Stubs out the database set/fetch API calls for Parallax |
483 | so the calls are routed to an in-memory dict. This helps us |
484 | @@ -176,7 +226,7 @@ |
485 | 'deleted_at': None, |
486 | 'deleted': False, |
487 | 'files': [], |
488 | - 'metadata': []}, |
489 | + 'properties': []}, |
490 | {'id': 2, |
491 | 'name': 'fake image #2', |
492 | 'status': 'available', |
493 | @@ -187,7 +237,7 @@ |
494 | 'deleted_at': None, |
495 | 'deleted': False, |
496 | 'files': [], |
497 | - 'metadata': []}] |
498 | + 'properties': []}] |
499 | |
500 | VALID_STATUSES = ('available', 'disabled', 'pending') |
501 | |
502 | @@ -196,7 +246,12 @@ |
503 | self.next_id = 3 |
504 | |
505 | def image_create(self, _context, values): |
506 | - values['id'] = self.next_id |
507 | + |
508 | + values['id'] = values.get('id', self.next_id) |
509 | + |
510 | + if values['id'] in [image['id'] for image in self.images]: |
511 | + raise exception.Duplicate("Duplicate image id: %s" % |
512 | + values['id']) |
513 | |
514 | if 'status' not in values.keys(): |
515 | values['status'] = 'available' |
516 | @@ -204,6 +259,19 @@ |
517 | if not values['status'] in self.VALID_STATUSES: |
518 | raise exception.Invalid("Invalid status '%s' for image" % |
519 | values['status']) |
520 | + |
521 | + values['deleted'] = False |
522 | + values['files'] = values.get('files', []) |
523 | + values['properties'] = values.get('properties', []) |
524 | + values['created_at'] = datetime.datetime.utcnow() |
525 | + values['updated_at'] = datetime.datetime.utcnow() |
526 | + values['deleted_at'] = None |
527 | + |
528 | + for p in values['properties']: |
529 | + p['deleted'] = False |
530 | + p['created_at'] = datetime.datetime.utcnow() |
531 | + p['updated_at'] = datetime.datetime.utcnow() |
532 | + p['deleted_at'] = None |
533 | |
534 | self.next_id += 1 |
535 | self.images.append(values) |
536 | |
537 | === added file 'tests/unit/test_clients.py' |
538 | --- tests/unit/test_clients.py 1970-01-01 00:00:00 +0000 |
539 | +++ tests/unit/test_clients.py 2010-11-23 18:13:46 +0000 |
540 | @@ -0,0 +1,233 @@ |
541 | +# vim: tabstop=4 shiftwidth=4 softtabstop=4 |
542 | + |
543 | +# Copyright 2010 OpenStack, LLC |
544 | +# All Rights Reserved. |
545 | +# |
546 | +# Licensed under the Apache License, Version 2.0 (the "License"); you may |
547 | +# not use this file except in compliance with the License. You may obtain |
548 | +# a copy of the License at |
549 | +# |
550 | +# http://www.apache.org/licenses/LICENSE-2.0 |
551 | +# |
552 | +# Unless required by applicable law or agreed to in writing, software |
553 | +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT |
554 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
555 | +# License for the specific language governing permissions and limitations |
556 | +# under the License. |
557 | + |
558 | +import json |
559 | +import stubout |
560 | +import unittest |
561 | + |
562 | +import webob |
563 | + |
564 | +from glance import client |
565 | +from glance.common import exception |
566 | +from tests import stubs |
567 | + |
568 | + |
569 | +class TestBadClients(unittest.TestCase): |
570 | + |
571 | + """Test exceptions raised for bad clients""" |
572 | + |
573 | + def test_bad_protocol(self): |
574 | + """Test unsupported protocol raised""" |
575 | + c = client.ParallaxClient(address="hdsa://127.012..1./") |
576 | + self.assertRaises(client.UnsupportedProtocolError, |
577 | + c.get_image, |
578 | + 1) |
579 | + |
580 | + def test_bad_address(self): |
581 | + """Test unsupported protocol raised""" |
582 | + c = client.ParallaxClient(address="http://127.999.1.1/") |
583 | + self.assertRaises(client.ClientConnectionError, |
584 | + c.get_image, |
585 | + 1) |
586 | + |
587 | + |
588 | +class TestParallaxClient(unittest.TestCase): |
589 | + |
590 | + """Test proper actions made for both valid and invalid requests""" |
591 | + |
592 | + def setUp(self): |
593 | + """Establish a clean test environment""" |
594 | + self.stubs = stubout.StubOutForTesting() |
595 | + stubs.stub_out_parallax_db_image_api(self.stubs) |
596 | + stubs.stub_out_parallax_server(self.stubs) |
597 | + self.client = client.ParallaxClient() |
598 | + |
599 | + def tearDown(self): |
600 | + """Clear the test environment""" |
601 | + self.stubs.UnsetAll() |
602 | + |
603 | + def test_get_image_index(self): |
604 | + """Test correct set of public image returned""" |
605 | + fixture = {'id': 2, |
606 | + 'name': 'fake image #2'} |
607 | + images = self.client.get_images() |
608 | + self.assertEquals(len(images), 1) |
609 | + |
610 | + for k,v in fixture.iteritems(): |
611 | + self.assertEquals(v, images[0][k]) |
612 | + |
613 | + def test_get_image_details(self): |
614 | + """Tests that the detailed info about public images returned""" |
615 | + fixture = {'id': 2, |
616 | + 'name': 'fake image #2', |
617 | + 'is_public': True, |
618 | + 'image_type': 'kernel', |
619 | + 'status': 'available' |
620 | + } |
621 | + |
622 | + images = self.client.get_images_detailed() |
623 | + self.assertEquals(len(images), 1) |
624 | + |
625 | + for k,v in fixture.iteritems(): |
626 | + self.assertEquals(v, images[0][k]) |
627 | + |
628 | + def test_get_image_metadata(self): |
629 | + """Tests that the detailed info about an image returned""" |
630 | + fixture = {'id': 2, |
631 | + 'name': 'fake image #2', |
632 | + 'is_public': True, |
633 | + 'image_type': 'kernel', |
634 | + 'status': 'available' |
635 | + } |
636 | + |
637 | + data = self.client.get_image(2) |
638 | + |
639 | + for k,v in fixture.iteritems(): |
640 | + self.assertEquals(v, data[k]) |
641 | + |
642 | + def test_get_image_non_existing(self): |
643 | + """Tests that NotFound is raised when getting a non-existing image""" |
644 | + |
645 | + self.assertRaises(exception.NotFound, |
646 | + self.client.get_image, |
647 | + 42) |
648 | + |
649 | + def test_add_image_metadata_basic(self): |
650 | + """Tests that we can add image metadata and returns the new id""" |
651 | + fixture = {'name': 'fake public image', |
652 | + 'is_public': True, |
653 | + 'image_type': 'kernel' |
654 | + } |
655 | + |
656 | + new_id = self.client.add_image(fixture) |
657 | + |
658 | + # Test ID auto-assigned properly |
659 | + self.assertEquals(3, new_id) |
660 | + |
661 | + # Test all other attributes set |
662 | + data = self.client.get_image(3) |
663 | + |
664 | + for k,v in fixture.iteritems(): |
665 | + self.assertEquals(v, data[k]) |
666 | + |
667 | + # Test status was updated properly |
668 | + self.assertTrue('status' in data.keys()) |
669 | + self.assertEquals('available', data['status']) |
670 | + |
671 | + def test_add_image_metadata_with_properties(self): |
672 | + """Tests that we can add image metadata with properties""" |
673 | + fixture = {'name': 'fake public image', |
674 | + 'is_public': True, |
675 | + 'image_type': 'kernel', |
676 | + 'properties': [{'key':'disco', |
677 | + 'value': 'baby'}] |
678 | + } |
679 | + expected = {'name': 'fake public image', |
680 | + 'is_public': True, |
681 | + 'image_type': 'kernel', |
682 | + 'properties': {'disco': 'baby'} |
683 | + } |
684 | + |
685 | + new_id = self.client.add_image(fixture) |
686 | + |
687 | + # Test ID auto-assigned properly |
688 | + self.assertEquals(3, new_id) |
689 | + |
690 | + # Test all other attributes set |
691 | + data = self.client.get_image(3) |
692 | + |
693 | + for k,v in expected.iteritems(): |
694 | + self.assertEquals(v, data[k]) |
695 | + |
696 | + # Test status was updated properly |
697 | + self.assertTrue('status' in data.keys()) |
698 | + self.assertEquals('available', data['status']) |
699 | + |
700 | + def test_add_image_already_exists(self): |
701 | + """Tests proper exception is raised if image with ID already exists""" |
702 | + fixture = {'id': 2, |
703 | + 'name': 'fake public image', |
704 | + 'is_public': True, |
705 | + 'image_type': 'kernel', |
706 | + 'status': 'bad status' |
707 | + } |
708 | + |
709 | + self.assertRaises(exception.Duplicate, |
710 | + self.client.add_image, |
711 | + fixture) |
712 | + |
713 | + def test_add_image_with_bad_status(self): |
714 | + """Tests proper exception is raised if a bad status is set""" |
715 | + fixture = {'id': 3, |
716 | + 'name': 'fake public image', |
717 | + 'is_public': True, |
718 | + 'image_type': 'kernel', |
719 | + 'status': 'bad status' |
720 | + } |
721 | + |
722 | + self.assertRaises(client.BadInputError, |
723 | + self.client.add_image, |
724 | + fixture) |
725 | + |
726 | + def test_update_image(self): |
727 | + """Tests that the /images PUT parallax API updates the image""" |
728 | + fixture = {'name': 'fake public image #2', |
729 | + 'image_type': 'ramdisk' |
730 | + } |
731 | + |
732 | + self.assertTrue(self.client.update_image(2, fixture)) |
733 | + |
734 | + # Test all other attributes set |
735 | + data = self.client.get_image(2) |
736 | + |
737 | + for k,v in fixture.iteritems(): |
738 | + self.assertEquals(v, data[k]) |
739 | + |
740 | + def test_update_image_not_existing(self): |
741 | + """Tests non existing image update doesn't work""" |
742 | + fixture = {'id': 3, |
743 | + 'name': 'fake public image', |
744 | + 'is_public': True, |
745 | + 'image_type': 'kernel', |
746 | + 'status': 'bad status' |
747 | + } |
748 | + |
749 | + self.assertRaises(exception.NotFound, |
750 | + self.client.update_image, |
751 | + 3, |
752 | + fixture) |
753 | + |
754 | + def test_delete_image(self): |
755 | + """Tests that image metadata is deleted properly""" |
756 | + |
757 | + # Grab the original number of images |
758 | + orig_num_images = len(self.client.get_images()) |
759 | + |
760 | + # Delete image #2 |
761 | + self.assertTrue(self.client.delete_image(2)) |
762 | + |
763 | + # Verify one less image |
764 | + new_num_images = len(self.client.get_images()) |
765 | + |
766 | + self.assertEquals(new_num_images, orig_num_images - 1) |
767 | + |
768 | + def test_delete_image_not_existing(self): |
769 | + """Tests cannot delete non-existing image""" |
770 | + |
771 | + self.assertRaises(exception.NotFound, |
772 | + self.client.delete_image, |
773 | + 3) |
774 | |
775 | === modified file 'tests/unit/test_parallax_api.py' |
776 | --- tests/unit/test_parallax_api.py 2010-10-17 16:31:04 +0000 |
777 | +++ tests/unit/test_parallax_api.py 2010-11-23 18:13:46 +0000 |
778 | @@ -22,7 +22,6 @@ |
779 | |
780 | from glance.common import exception |
781 | from glance.parallax import controllers |
782 | -from glance.parallax import db |
783 | from tests import stubs |
784 | |
785 | |
786 | @@ -30,7 +29,6 @@ |
787 | def setUp(self): |
788 | """Establish a clean test environment""" |
789 | self.stubs = stubout.StubOutForTesting() |
790 | - self.image_controller = controllers.ImageController() |
791 | stubs.stub_out_parallax_db_image_api(self.stubs) |
792 | |
793 | def tearDown(self): |
794 | @@ -139,7 +137,8 @@ |
795 | # TODO(jaypipes): Port Nova's Fault infrastructure |
796 | # over to Glance to support exception catching into |
797 | # standard HTTP errors. |
798 | - self.assertRaises(exception.Invalid, req.get_response, controllers.API()) |
799 | + res = req.get_response(controllers.API()) |
800 | + self.assertEquals(res.status_int, webob.exc.HTTPBadRequest.code) |
801 | |
802 | def test_update_image(self): |
803 | """Tests that the /images PUT parallax API updates the image""" |
804 | @@ -179,7 +178,9 @@ |
805 | # TODO(jaypipes): Port Nova's Fault infrastructure |
806 | # over to Glance to support exception catching into |
807 | # standard HTTP errors. |
808 | - self.assertRaises(exception.NotFound, req.get_response, controllers.API()) |
809 | + res = req.get_response(controllers.API()) |
810 | + self.assertEquals(res.status_int, |
811 | + webob.exc.HTTPNotFound.code) |
812 | |
813 | def test_delete_image(self): |
814 | """Tests that the /images DELETE parallax API deletes the image""" |
815 | @@ -221,4 +222,6 @@ |
816 | # TODO(jaypipes): Port Nova's Fault infrastructure |
817 | # over to Glance to support exception catching into |
818 | # standard HTTP errors. |
819 | - self.assertRaises(exception.NotFound, req.get_response, controllers.API()) |
820 | + res = req.get_response(controllers.API()) |
821 | + self.assertEquals(res.status_int, |
822 | + webob.exc.HTTPNotFound.code) |
823 | |
824 | === modified file 'tests/unit/test_parallax_models.py' |
825 | --- tests/unit/test_parallax_models.py 2010-10-15 05:35:15 +0000 |
826 | +++ tests/unit/test_parallax_models.py 2010-11-23 18:13:46 +0000 |
827 | @@ -40,20 +40,20 @@ |
828 | same key |
829 | |
830 | """ |
831 | - self._make_metadatum(self.image, key="spam", value="eggs") |
832 | + self._make_property(self.image, key="spam", value="eggs") |
833 | |
834 | second_image = self._make_image(id=3, name='fake image #3') |
835 | - self._make_metadatum(second_image, key="spam", value="eggs") |
836 | + self._make_property(second_image, key="spam", value="eggs") |
837 | |
838 | def test_metadata_key_constraint_bad(self): |
839 | """The same image cannot have two distinct pieces of metadata with the |
840 | same key. |
841 | |
842 | """ |
843 | - self._make_metadatum(self.image, key="spam", value="eggs") |
844 | + self._make_property(self.image, key="spam", value="eggs") |
845 | |
846 | self.assertRaises(sa_exc.IntegrityError, |
847 | - self._make_metadatum, self.image, key="spam", value="eggs") |
848 | + self._make_property, self.image, key="spam", value="eggs") |
849 | |
850 | def _make_image(self, id, name): |
851 | """Convenience method to create an image with a given name and id""" |
852 | @@ -67,11 +67,9 @@ |
853 | image = db.api.image_create(context, fixture) |
854 | return image |
855 | |
856 | - def _make_metadatum(self, image, key, value): |
857 | + def _make_property(self, image, key, value): |
858 | """Convenience method to create metadata attached to an image""" |
859 | metadata = {'image_id': image['id'], 'key': key, 'value': value} |
860 | context = None |
861 | - metadatum = db.api.image_metadatum_create(context, metadata) |
862 | - return metadatum |
863 | - |
864 | - |
865 | + property = db.api.image_property_create(context, metadata) |
866 | + return property |
Overall lgtm. Have a few points I'd like to open up for discussion:
1. The catching of KeyError on line 77 seems too broad; it should just wrap the dictionary lookup, not the lookup and the ping. If the connection object (or more likely a mocked version of the connection object) erroneously raised KeyError, the re-raised exception would be misleading.
2. Pinging the endpoint from within the __init__ feels messy: the usual errors associated with network traffic are unexpected when all you're trying to do is construct and initialize an object. I would feel better if this test were in a separate connect() method. Another option would be to provide a kwarg test_connection =False on __init__ which requires the caller to acknowledge that their __init__ invocation is causing network traffic. A third option would be to just omit it and require the consumer to handle the destination being down just as they would for any other network library.
3. Very minor but the pattern _type(self. netloc, self.port)
137 + c = self.connection
138 + c.request("GET", "images")
139 + res = c.getresponse()
is repeated in a bunch of places.
Perhaps a do_request method would DRY up the code a bit.
def do_request(method, path, headers=None, data=None): _type(self. netloc, self.port) request( method, path, headers=header or {}, data=data or None)
"""Something like this"""
conn = self.connection
conn.
# could check for non-200 status code and raise exceptions here as well.
return conn.getresponse()
4. At some point we should clarify terminology: what exactly is meant by "image metadata"? I think in difference parts of the code and design docs we're using it to mean different things. In one case (I think how it's being used here), it's used to refer to the image registry information (name, backend_uri, created_at, updated_at) as opposed to "image data" which is used to refer to the actual image files.
The other use of "image metadata" is within the parallax db.model where I added the table "image_metadatum". This is a table used to store arbitrary key/value pairs associated with an image. This makes it image metadata metadata, I guess.
Perhaps we should rename the latter "image_properties", or maybe the former should be called "image_descriptor". Whatever the names, we should just clarify this soon.
Anyway these are all minor points, interested to hear your thoughts.