Merge lp:~jaypipes/glance/bug704854 into lp:~glance-coresec/glance/cactus-trunk

Proposed by Jay Pipes
Status: Merged
Approved by: Jay Pipes
Approved revision: 82
Merged at revision: 75
Proposed branch: lp:~jaypipes/glance/bug704854
Merge into: lp:~glance-coresec/glance/cactus-trunk
Prerequisite: lp:~jaypipes/glance/add-pidfile-option
Diff against target: 619 lines (+310/-57)
14 files modified
MANIFEST.in (+1/-0)
bin/glance-api (+7/-3)
bin/glance-combined (+7/-3)
bin/glance-control (+8/-0)
bin/glance-manage (+7/-3)
bin/glance-registry (+7/-3)
bin/glance-upload (+5/-3)
glance/client.py (+1/-1)
glance/registry/db/api.py (+30/-0)
glance/registry/db/models.py (+0/-13)
glance/registry/server.py (+14/-4)
glance/server.py (+43/-22)
tests/unit/test_clients.py (+2/-2)
tests/unit/test_misc.py (+178/-0)
To merge this branch: bzr merge lp:~jaypipes/glance/bug704854
Reviewer Review Type Date Requested Status
Jay Pipes (community) Approve
Devin Carlen (community) Approve
Rick Harris (community) Approve
Soren Hansen Pending
Review via email: mp+50773@code.launchpad.net

Description of the change

This adds a test case for LP Bug 704854 -- Exception
raised by Registry server gets eaten by API server.

The test involves spinning up an API server and a registry
server, firing cURL and glance-upload against the API
server with invalid requests, and verifying that appropriate
exception messages are contained in the response.

I should probably rebase this commit considering all the previous
commits weren't actually addressing the issue. The fact that I
had glance-api and glance-registry installed on my local machine
was causing the test runs to improperly return a passing result.

It just so happens that the glance-api and glance-registry that
I had installed to my /usr/local/bin were the very same programs
that were from a previous branch I had locally where I fixed the
root cause of this issue, which was that the sqlalchemy @validates
decorator does NOT fire for *new* objects, only existing ones, which
resulted in image_create() improperly storing NULL data in type, name,
and other non-nullable fields in the database. This then set off
a domino effect which caused the next call from
glance.server._upload_and_activate() to die a horrible death due to
the @validates decorator then firing on the already-created Image
object. This horrible death was improperly being raised from the
glance.client as a BadRequest instead of exception.Invalid, which
caused the API server to ignore the text in the actual Invalid
exception coming from the registry server.

In short, this patch finally fixes the root of the problem by
placing a validate_image() function guard which throws exception.Invalid
for any invalid data coming into the _image_update() method in the db
API. It also adds a bunch of logging statements and ensures that
exceptions throughout the call stack between the API server to the
glance.registry.Client to the Registry server are properly handled and
that the text of those exceptions isn't thrown away willy-nilly.

To post a comment you must log in.
Revision history for this message
Rick Harris (rconradharris) wrote :

Jay-

Sorry I'm late on reviewing this, overall this looks really good. Surprised by the decorator only validating on update (major WTF).

Femto-nits:

> 170 + type = values.get('type', None)

We probably shouldn't be using `type` as a variable name (since it's a builtin). `type_` seems ok, `image_type` seems better. (Was burned by this recently :-)

> 320 + logger.debug("Uploading image data for image %(image_id)s "
> 321 + "to %(store_name)s store"
> 322 + % {'image_id': image_id,
> 323 + 'store_name': image_store})

Certainly not a problem, but is there a reason you didn't use the ` % locals()` idiom here?

+ with tempfile.NamedTemporaryFile() as f:

Rather than just `f`, I think `temp_file` or `tmp` would be a little clearer and help with grep-ability.

(Should we eliminate single-letter variable names entirely since they are usually a bad idea? Should we special case the counting variables `i` and `j` since they are *soo* common/idiomatic?)

review: Needs Fixing
Revision history for this message
Jay Pipes (jaypipes) wrote :

> Jay-
>
> Sorry I'm late on reviewing this, overall this looks really good. Surprised by
> the decorator only validating on update (major WTF).
>
> Femto-nits:
>
> > 170 + type = values.get('type', None)
>
> We probably shouldn't be using `type` as a variable name (since it's a
> builtin). `type_` seems ok, `image_type` seems better. (Was burned by this
> recently :-)

ya.

> > 320 + logger.debug("Uploading image data for image %(image_id)s
> "
> > 321 + "to %(store_name)s store"
> > 322 + % {'image_id': image_id,
> > 323 + 'store_name': image_store})
>
> Certainly not a problem, but is there a reason you didn't use the ` %
> locals()` idiom here?

nope, will fix. :)

> + with tempfile.NamedTemporaryFile() as f:

Copied from Swift ;) I'll rename to temp_file.

> Rather than just `f`, I think `temp_file` or `tmp` would be a little clearer
> and help with grep-ability.
>
> (Should we eliminate single-letter variable names entirely since they are
> usually a bad idea? Should we special case the counting variables `i` and `j`
> since they are *soo* common/idiomatic?)

Revision history for this message
Jay Pipes (jaypipes) wrote :

fixes made. pls re-review.

Revision history for this message
Rick Harris (rconradharris) wrote :

Looks good, thanks for the fixes!

review: Approve
Revision history for this message
Devin Carlen (devcamcar) wrote :

lgtm!

review: Approve
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :
Download full text (4.9 KiB)

The attempt to merge lp:~jaypipes/glance/bug704854 into lp:glance failed. Below is the output from the failed tests.

running test
running egg_info
creating glance.egg-info
writing glance.egg-info/PKG-INFO
writing top-level names to glance.egg-info/top_level.txt
writing dependency_links to glance.egg-info/dependency_links.txt
writing manifest file 'glance.egg-info/SOURCES.txt'
reading manifest file 'glance.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'LICENSE'
warning: no files found matching 'ChangeLog'
warning: no files found matching 'tests/test_data.py'
writing manifest file 'glance.egg-info/SOURCES.txt'
running build_ext

Tests raises BadRequest for invalid store header ... ok
Tests raises BadRequest for invalid store header ... ok
Tests creates a queued image for no body and no loc header ... ok
test_delete_image (tests.unit.test_api.TestGlanceAPI) ... ok
test_delete_non_exists_image (tests.unit.test_api.TestGlanceAPI) ... ok
Test for HEAD /images/<ID> ... ok
test_show_image_basic (tests.unit.test_api.TestGlanceAPI) ... ok
test_show_non_exists_image (tests.unit.test_api.TestGlanceAPI) ... ok
Tests that the /images POST registry API creates the image ... ok
Tests proper exception is raised if a bad status is set ... ok
Tests that the /images DELETE registry API deletes the image ... ok
Tests proper exception is raised if attempt to delete non-existing ... ok
Tests that the /images/detail registry API returns ... ok
Tests that the /images registry API returns list of ... ok
Tests that the root registry API returns "index", ... ok
Tests that the /images PUT registry API updates the image ... ok
Tests proper exception is raised if attempt to update non-existing ... ok
Test ClientConnectionError raised ... ok
Tests proper exception is raised if image with ID already exists ... ok
Tests that we can add image metadata and returns the new id ... ok
Tests a bad status is set to a proper one by server ... ok
Tests BadRequest raised when supplying bad store name in meta ... ok
Tests can add image by passing image data as file ... ok
Tests can add image by passing image data as string ... ok
Tests add image by passing image data as string w/ no size attr ... ok
Tests that we can add image metadata with properties ... ok
Tests client returns image as queued ... ok
Tests that image metadata is deleted properly ... ok
Tests cannot delete non-existing image ... ok
Test a simple file backend retrieval works as expected ... ok
Tests that the detailed info about public images returned ... ok
Test correct set of public image returned ... ok
Tests that the detailed info about an image returned ... ok
Tests that NotFound is raised when getting a non-existing image ... ok
Test retrieval of a non-existing image returns a 404 ... ok
Tests that the /images PUT registry API updates the image ... ok
Tests non existing image update doesn't work ... ok
Tests proper exception is raised if image with ID already exists ... ok
Tests that we can add image metadata and returns the new id ... ok
Tests proper exception is raised if a bad status is set ... ok
Tests that we can add image metadata with properties ... ok
Tests t...

Read more...

Revision history for this message
Jay Pipes (jaypipes) wrote :

I'm getting SERIOUSLY annoyed with not being able to reproduce shit that Hudson keeps barfing on. Grr.

Revision history for this message
Jay Pipes (jaypipes) wrote :

approving debugging statements in test...

review: Approve
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :
Download full text (5.2 KiB)

The attempt to merge lp:~jaypipes/glance/bug704854 into lp:glance failed. Below is the output from the failed tests.

running test
running egg_info
creating glance.egg-info
writing glance.egg-info/PKG-INFO
writing top-level names to glance.egg-info/top_level.txt
writing dependency_links to glance.egg-info/dependency_links.txt
writing manifest file 'glance.egg-info/SOURCES.txt'
reading manifest file 'glance.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'LICENSE'
warning: no files found matching 'ChangeLog'
warning: no files found matching 'tests/test_data.py'
writing manifest file 'glance.egg-info/SOURCES.txt'
running build_ext

Tests raises BadRequest for invalid store header ... ok
Tests raises BadRequest for invalid store header ... ok
Tests creates a queued image for no body and no loc header ... ok
test_delete_image (tests.unit.test_api.TestGlanceAPI) ... ok
test_delete_non_exists_image (tests.unit.test_api.TestGlanceAPI) ... ok
Test for HEAD /images/<ID> ... ok
test_show_image_basic (tests.unit.test_api.TestGlanceAPI) ... ok
test_show_non_exists_image (tests.unit.test_api.TestGlanceAPI) ... ok
Tests that the /images POST registry API creates the image ... ok
Tests proper exception is raised if a bad status is set ... ok
Tests that the /images DELETE registry API deletes the image ... ok
Tests proper exception is raised if attempt to delete non-existing ... ok
Tests that the /images/detail registry API returns ... ok
Tests that the /images registry API returns list of ... ok
Tests that the root registry API returns "index", ... ok
Tests that the /images PUT registry API updates the image ... ok
Tests proper exception is raised if attempt to update non-existing ... ok
Test ClientConnectionError raised ... ok
Tests proper exception is raised if image with ID already exists ... ok
Tests that we can add image metadata and returns the new id ... ok
Tests a bad status is set to a proper one by server ... ok
Tests BadRequest raised when supplying bad store name in meta ... ok
Tests can add image by passing image data as file ... ok
Tests can add image by passing image data as string ... ok
Tests add image by passing image data as string w/ no size attr ... ok
Tests that we can add image metadata with properties ... ok
Tests client returns image as queued ... ok
Tests that image metadata is deleted properly ... ok
Tests cannot delete non-existing image ... ok
Test a simple file backend retrieval works as expected ... ok
Tests that the detailed info about public images returned ... ok
Test correct set of public image returned ... ok
Tests that the detailed info about an image returned ... ok
Tests that NotFound is raised when getting a non-existing image ... ok
Test retrieval of a non-existing image returns a 404 ... ok
Tests that the /images PUT registry API updates the image ... ok
Tests non existing image update doesn't work ... ok
Tests proper exception is raised if image with ID already exists ... ok
Tests that we can add image metadata and returns the new id ... ok
Tests proper exception is raised if a bad status is set ... ok
Tests that we can add image metadata with properties ... ok
Tests t...

Read more...

Revision history for this message
Vish Ishaya (vishvananda) wrote :
Download full text (5.6 KiB)

argparse isn't in python 2.6, no?
On Feb 23, 2011, at 12:48 PM, OpenStack Hudson wrote:

> The attempt to merge lp:~jaypipes/glance/bug704854 into lp:glance failed. Below is the output from the failed tests.
>
> running test
> running egg_info
> creating glance.egg-info
> writing glance.egg-info/PKG-INFO
> writing top-level names to glance.egg-info/top_level.txt
> writing dependency_links to glance.egg-info/dependency_links.txt
> writing manifest file 'glance.egg-info/SOURCES.txt'
> reading manifest file 'glance.egg-info/SOURCES.txt'
> reading manifest template 'MANIFEST.in'
> warning: no files found matching 'LICENSE'
> warning: no files found matching 'ChangeLog'
> warning: no files found matching 'tests/test_data.py'
> writing manifest file 'glance.egg-info/SOURCES.txt'
> running build_ext
>
> Tests raises BadRequest for invalid store header ... ok
> Tests raises BadRequest for invalid store header ... ok
> Tests creates a queued image for no body and no loc header ... ok
> test_delete_image (tests.unit.test_api.TestGlanceAPI) ... ok
> test_delete_non_exists_image (tests.unit.test_api.TestGlanceAPI) ... ok
> Test for HEAD /images/<ID> ... ok
> test_show_image_basic (tests.unit.test_api.TestGlanceAPI) ... ok
> test_show_non_exists_image (tests.unit.test_api.TestGlanceAPI) ... ok
> Tests that the /images POST registry API creates the image ... ok
> Tests proper exception is raised if a bad status is set ... ok
> Tests that the /images DELETE registry API deletes the image ... ok
> Tests proper exception is raised if attempt to delete non-existing ... ok
> Tests that the /images/detail registry API returns ... ok
> Tests that the /images registry API returns list of ... ok
> Tests that the root registry API returns "index", ... ok
> Tests that the /images PUT registry API updates the image ... ok
> Tests proper exception is raised if attempt to update non-existing ... ok
> Test ClientConnectionError raised ... ok
> Tests proper exception is raised if image with ID already exists ... ok
> Tests that we can add image metadata and returns the new id ... ok
> Tests a bad status is set to a proper one by server ... ok
> Tests BadRequest raised when supplying bad store name in meta ... ok
> Tests can add image by passing image data as file ... ok
> Tests can add image by passing image data as string ... ok
> Tests add image by passing image data as string w/ no size attr ... ok
> Tests that we can add image metadata with properties ... ok
> Tests client returns image as queued ... ok
> Tests that image metadata is deleted properly ... ok
> Tests cannot delete non-existing image ... ok
> Test a simple file backend retrieval works as expected ... ok
> Tests that the detailed info about public images returned ... ok
> Test correct set of public image returned ... ok
> Tests that the detailed info about an image returned ... ok
> Tests that NotFound is raised when getting a non-existing image ... ok
> Test retrieval of a non-existing image returns a 404 ... ok
> Tests that the /images PUT registry API updates the image ... ok
> Tests non existing image update doesn't work ... ok
> Tests proper exception is raised if image with ID already exists ... o...

Read more...

Revision history for this message
Jay Pipes (jaypipes) wrote :
Download full text (5.9 KiB)

No it's not.

On Wed, Feb 23, 2011 at 3:58 PM, Vish Ishaya <email address hidden> wrote:
> argparse isn't in python 2.6, no?
> On Feb 23, 2011, at 12:48 PM, OpenStack Hudson wrote:
>
>> The attempt to merge lp:~jaypipes/glance/bug704854 into lp:glance failed. Below is the output from the failed tests.
>>
>> running test
>> running egg_info
>> creating glance.egg-info
>> writing glance.egg-info/PKG-INFO
>> writing top-level names to glance.egg-info/top_level.txt
>> writing dependency_links to glance.egg-info/dependency_links.txt
>> writing manifest file 'glance.egg-info/SOURCES.txt'
>> reading manifest file 'glance.egg-info/SOURCES.txt'
>> reading manifest template 'MANIFEST.in'
>> warning: no files found matching 'LICENSE'
>> warning: no files found matching 'ChangeLog'
>> warning: no files found matching 'tests/test_data.py'
>> writing manifest file 'glance.egg-info/SOURCES.txt'
>> running build_ext
>>
>> Tests raises BadRequest for invalid store header ... ok
>> Tests raises BadRequest for invalid store header ... ok
>> Tests creates a queued image for no body and no loc header ... ok
>> test_delete_image (tests.unit.test_api.TestGlanceAPI) ... ok
>> test_delete_non_exists_image (tests.unit.test_api.TestGlanceAPI) ... ok
>> Test for HEAD /images/<ID> ... ok
>> test_show_image_basic (tests.unit.test_api.TestGlanceAPI) ... ok
>> test_show_non_exists_image (tests.unit.test_api.TestGlanceAPI) ... ok
>> Tests that the /images POST registry API creates the image ... ok
>> Tests proper exception is raised if a bad status is set ... ok
>> Tests that the /images DELETE registry API deletes the image ... ok
>> Tests proper exception is raised if attempt to delete non-existing ... ok
>> Tests that the /images/detail registry API returns ... ok
>> Tests that the /images registry API returns list of ... ok
>> Tests that the root registry API returns "index", ... ok
>> Tests that the /images PUT registry API updates the image ... ok
>> Tests proper exception is raised if attempt to update non-existing ... ok
>> Test ClientConnectionError raised ... ok
>> Tests proper exception is raised if image with ID already exists ... ok
>> Tests that we can add image metadata and returns the new id ... ok
>> Tests a bad status is set to a proper one by server ... ok
>> Tests BadRequest raised when supplying bad store name in meta ... ok
>> Tests can add image by passing image data as file ... ok
>> Tests can add image by passing image data as string ... ok
>> Tests add image by passing image data as string w/ no size attr ... ok
>> Tests that we can add image metadata with properties ... ok
>> Tests client returns image as queued ... ok
>> Tests that image metadata is deleted properly ... ok
>> Tests cannot delete non-existing image ... ok
>> Test a simple file backend retrieval works as expected ... ok
>> Tests that the detailed info about public images returned ... ok
>> Test correct set of public image returned ... ok
>> Tests that the detailed info about an image returned ... ok
>> Tests that NotFound is raised when getting a non-existing image ... ok
>> Test retrieval of a non-existing image returns a 404 ... ok
>> Tests that the /images PUT registry API updat...

Read more...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'MANIFEST.in'
--- MANIFEST.in 2011-02-02 01:43:16 +0000
+++ MANIFEST.in 2011-02-23 20:37:56 +0000
@@ -8,4 +8,5 @@
8include run_tests.py8include run_tests.py
9include glance/registry/db/migrate_repo/migrate.cfg9include glance/registry/db/migrate_repo/migrate.cfg
10graft doc10graft doc
11graft etc
11graft tools12graft tools
1213
=== modified file 'bin/glance-api'
--- bin/glance-api 2011-02-09 20:24:59 +0000
+++ bin/glance-api 2011-02-23 20:37:56 +0000
@@ -26,9 +26,13 @@
26import os26import os
27import sys27import sys
2828
29ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))29# If ../glance/__init__.py exists, add ../ to Python search path, so that
3030# it will override what happens to be installed in /usr/(local/)lib/python...
31sys.path.append(ROOT_DIR)31possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
32 os.pardir,
33 os.pardir))
34if os.path.exists(os.path.join(possible_topdir, 'glance', '__init__.py')):
35 sys.path.insert(0, possible_topdir)
3236
33from glance import version37from glance import version
34from glance.common import config38from glance.common import config
3539
=== modified file 'bin/glance-combined'
--- bin/glance-combined 2011-02-09 20:24:59 +0000
+++ bin/glance-combined 2011-02-23 20:37:56 +0000
@@ -28,9 +28,13 @@
28import os28import os
29import sys29import sys
3030
31ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))31# If ../glance/__init__.py exists, add ../ to Python search path, so that
3232# it will override what happens to be installed in /usr/(local/)lib/python...
33sys.path.append(ROOT_DIR)33possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
34 os.pardir,
35 os.pardir))
36if os.path.exists(os.path.join(possible_topdir, 'glance', '__init__.py')):
37 sys.path.insert(0, possible_topdir)
3438
35from glance import version39from glance import version
36from glance.common import config40from glance.common import config
3741
=== modified file 'bin/glance-control'
--- bin/glance-control 2011-02-16 07:43:11 +0000
+++ bin/glance-control 2011-02-23 20:37:56 +0000
@@ -31,6 +31,14 @@
31import sys31import sys
32import time32import time
3333
34# If ../glance/__init__.py exists, add ../ to Python search path, so that
35# it will override what happens to be installed in /usr/(local/)lib/python...
36possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
37 os.pardir,
38 os.pardir))
39if os.path.exists(os.path.join(possible_topdir, 'glance', '__init__.py')):
40 sys.path.insert(0, possible_topdir)
41
34from glance import version42from glance import version
35from glance.common import config43from glance.common import config
3644
3745
=== modified file 'bin/glance-manage'
--- bin/glance-manage 2011-02-02 18:47:08 +0000
+++ bin/glance-manage 2011-02-23 20:37:56 +0000
@@ -30,9 +30,13 @@
30import os30import os
31import sys31import sys
3232
33ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))33# If ../glance/__init__.py exists, add ../ to Python search path, so that
3434# it will override what happens to be installed in /usr/(local/)lib/python...
35sys.path.append(ROOT_DIR)35possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
36 os.pardir,
37 os.pardir))
38if os.path.exists(os.path.join(possible_topdir, 'glance', '__init__.py')):
39 sys.path.insert(0, possible_topdir)
3640
37from glance import version as glance_version41from glance import version as glance_version
38from glance.common import config42from glance.common import config
3943
=== modified file 'bin/glance-registry'
--- bin/glance-registry 2011-02-09 20:24:59 +0000
+++ bin/glance-registry 2011-02-23 20:37:56 +0000
@@ -26,9 +26,13 @@
26import os26import os
27import sys27import sys
2828
29ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))29# If ../glance/__init__.py exists, add ../ to Python search path, so that
3030# it will override what happens to be installed in /usr/(local/)lib/python...
31sys.path.append(ROOT_DIR)31possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
32 os.pardir,
33 os.pardir))
34if os.path.exists(os.path.join(possible_topdir, 'glance', '__init__.py')):
35 sys.path.insert(0, possible_topdir)
3236
33from glance import version37from glance import version
34from glance.common import config38from glance.common import config
3539
=== modified file 'bin/glance-upload'
--- bin/glance-upload 2011-02-02 01:43:16 +0000
+++ bin/glance-upload 2011-02-23 20:37:56 +0000
@@ -53,10 +53,12 @@
53 parser.add_argument('filename', help='file to upload into Glance')53 parser.add_argument('filename', help='file to upload into Glance')
54 parser.add_argument('name', help='name of image')54 parser.add_argument('name', help='name of image')
55 parser.add_argument('--host', metavar='HOST', default='127.0.0.1',55 parser.add_argument('--host', metavar='HOST', default='127.0.0.1',
56 help='Location of Glance Server (default: 127.0.0.1)')56 help='Location of Glance Server (default: %default)')
57 parser.add_argument('--port', metavar='PORT', type=int, default=9292,
58 help='Port of Glance Server (default: %default)')
57 parser.add_argument('--type', metavar='TYPE', default='raw',59 parser.add_argument('--type', metavar='TYPE', default='raw',
58 help='Type of Image [kernel, ramdisk, machine, raw] '60 help='Type of Image [kernel, ramdisk, machine, raw] '
59 '(default: raw)')61 '(default: %default)')
60 parser.add_argument('--kernel', metavar='KERNEL',62 parser.add_argument('--kernel', metavar='KERNEL',
61 help='ID of kernel associated with this machine image')63 help='ID of kernel associated with this machine image')
62 parser.add_argument('--ramdisk', metavar='RAMDISK',64 parser.add_argument('--ramdisk', metavar='RAMDISK',
@@ -77,7 +79,7 @@
77 else:79 else:
78 die("kernel and ramdisk required for machine image")80 die("kernel and ramdisk required for machine image")
7981
80 client = Client(args.host, 9292)82 client = Client(args.host, args.port)
81 with open(args.filename) as f:83 with open(args.filename) as f:
82 new_meta = client.add_image(meta, f)84 new_meta = client.add_image(meta, f)
8385
8486
=== modified file 'glance/client.py'
--- glance/client.py 2011-02-05 02:32:31 +0000
+++ glance/client.py 2011-02-23 20:37:56 +0000
@@ -153,7 +153,7 @@
153 elif status_code == httplib.CONFLICT:153 elif status_code == httplib.CONFLICT:
154 raise exception.Duplicate(res.read())154 raise exception.Duplicate(res.read())
155 elif status_code == httplib.BAD_REQUEST:155 elif status_code == httplib.BAD_REQUEST:
156 raise exception.BadInputError(res.read())156 raise exception.Invalid(res.read())
157 else:157 else:
158 raise Exception("Unknown error occurred! %s" % res.__dict__)158 raise Exception("Unknown error occurred! %s" % res.__dict__)
159159
160160
=== modified file 'glance/registry/db/api.py'
--- glance/registry/db/api.py 2011-02-11 00:12:51 +0000
+++ glance/registry/db/api.py 2011-02-23 20:37:56 +0000
@@ -144,6 +144,30 @@
144 del values[attr]144 del values[attr]
145145
146146
147def validate_image(values):
148 """
149 Validates the incoming data and raises a Invalid exception
150 if anything is out of order.
151
152 :param values: Mapping of image metadata to check
153 """
154
155 image_type = values.get('type', None)
156 if not image_type:
157 msg = "Image type is required."
158 raise exception.Invalid(msg)
159 if image_type not in ('machine', 'kernel', 'ramdisk', 'raw', 'vhd'):
160 msg = "Invalid image type '%s' for image." % image_type
161 raise exception.Invalid(msg)
162 status = values.get('status', None)
163 if not status:
164 msg = "Image status is required."
165 raise exception.Invalid(msg)
166 if status not in ('active', 'queued', 'killed', 'saving'):
167 msg = "Invalid image status '%s' for image." % status
168 raise exception.Invalid(msg)
169
170
147def _image_update(context, values, image_id):171def _image_update(context, values, image_id):
148 """Used internally by image_create and image_update172 """Used internally by image_create and image_update
149173
@@ -151,6 +175,12 @@
151 :param values: A dict of attributes to set175 :param values: A dict of attributes to set
152 :param image_id: If None, create the image, otherwise, find and update it176 :param image_id: If None, create the image, otherwise, find and update it
153 """177 """
178
179 # Validate the attributes before we go any further. From my investigation,
180 # the @validates decorator does not validate on new records, only on
181 # existing records, which is, well, idiotic.
182 validate_image(values)
183
154 session = get_session()184 session = get_session()
155 with session.begin():185 with session.begin():
156 _drop_protected_attrs(models.Image, values)186 _drop_protected_attrs(models.Image, values)
157187
=== modified file 'glance/registry/db/models.py'
--- glance/registry/db/models.py 2011-02-17 18:34:28 +0000
+++ glance/registry/db/models.py 2011-02-23 20:37:56 +0000
@@ -101,19 +101,6 @@
101 is_public = Column(Boolean, nullable=False, default=False)101 is_public = Column(Boolean, nullable=False, default=False)
102 location = Column(Text)102 location = Column(Text)
103103
104 @validates('type')
105 def validate_type(self, key, type):
106 if not type in ('machine', 'kernel', 'ramdisk', 'raw', 'vhd'):
107 raise exception.Invalid(
108 "Invalid image type '%s' for image." % type)
109 return type
110
111 @validates('status')
112 def validate_status(self, key, status):
113 if not status in ('active', 'queued', 'killed', 'saving'):
114 raise exception.Invalid("Invalid status '%s' for image." % status)
115 return status
116
117104
118class ImageProperty(BASE, ModelBase):105class ImageProperty(BASE, ModelBase):
119 """Represents an image properties in the datastore"""106 """Represents an image properties in the datastore"""
120107
=== modified file 'glance/registry/server.py'
--- glance/registry/server.py 2011-02-03 15:23:48 +0000
+++ glance/registry/server.py 2011-02-23 20:37:56 +0000
@@ -19,6 +19,7 @@
19"""19"""
2020
21import json21import json
22import logging
2223
23import routes24import routes
24from webob import exc25from webob import exc
@@ -28,6 +29,9 @@
28from glance.registry.db import api as db_api29from glance.registry.db import api as db_api
2930
3031
32logger = logging.getLogger('glance.registry.server')
33
34
31class Controller(wsgi.Controller):35class Controller(wsgi.Controller):
32 """Controller for the reference implementation registry server"""36 """Controller for the reference implementation registry server"""
3337
@@ -118,9 +122,13 @@
118 image_data = db_api.image_create(context, image_data)122 image_data = db_api.image_create(context, image_data)
119 return dict(image=make_image_dict(image_data))123 return dict(image=make_image_dict(image_data))
120 except exception.Duplicate:124 except exception.Duplicate:
121 return exc.HTTPConflict()125 msg = ("Image with identifier %s already exists!" % id)
122 except exception.Invalid:126 logger.error(msg)
123 return exc.HTTPBadRequest()127 return exc.HTTPConflict(msg)
128 except exception.Invalid, e:
129 msg = ("Failed to add image metadata. Got error: %(e)s" % locals())
130 logger.error(msg)
131 return exc.HTTPBadRequest(msg)
124132
125 def update(self, req, id):133 def update(self, req, id):
126 """Updates an existing image with the registry.134 """Updates an existing image with the registry.
@@ -140,7 +148,9 @@
140 updated_image = db_api.image_update(context, id, image_data)148 updated_image = db_api.image_update(context, id, image_data)
141 return dict(image=make_image_dict(updated_image))149 return dict(image=make_image_dict(updated_image))
142 except exception.NotFound:150 except exception.NotFound:
143 return exc.HTTPNotFound()151 raise exc.HTTPNotFound(body='Image not found',
152 request=req,
153 content_type='text/plain')
144154
145155
146class API(wsgi.Router):156class API(wsgi.Router):
147157
=== modified file 'glance/server.py'
--- glance/server.py 2011-02-16 10:01:21 +0000
+++ glance/server.py 2011-02-23 20:37:56 +0000
@@ -49,6 +49,9 @@
49from glance import utils49from glance import utils
5050
5151
52logger = logging.getLogger('glance.server')
53
54
52class Controller(wsgi.Controller):55class Controller(wsgi.Controller):
5356
54 """57 """
@@ -188,10 +191,13 @@
188 except exception.Duplicate:191 except exception.Duplicate:
189 msg = "An image with identifier %s already exists"\192 msg = "An image with identifier %s already exists"\
190 % image_meta['id']193 % image_meta['id']
191 logging.error(msg)194 logger.error(msg)
192 raise HTTPConflict(msg, request=req)195 raise HTTPConflict(msg, request=req, content_type="text/plain")
193 except exception.Invalid:196 except exception.Invalid, e:
194 raise HTTPBadRequest()197 msg = ("Failed to reserve image. Got error: %(e)s" % locals())
198 for line in msg.split('\n'):
199 logger.error(line)
200 raise HTTPBadRequest(msg, request=req, content_type="text/plain")
195201
196 def _upload(self, req, image_meta):202 def _upload(self, req, image_meta):
197 """203 """
@@ -211,17 +217,22 @@
211 raise HTTPBadRequest(217 raise HTTPBadRequest(
212 "Content-Type must be application/octet-stream")218 "Content-Type must be application/octet-stream")
213219
214 image_store = req.headers.get(220 store_name = req.headers.get(
215 'x-image-meta-store', self.options['default_store'])221 'x-image-meta-store', self.options['default_store'])
216222
217 store = self.get_store_or_400(req, image_store)223 store = self.get_store_or_400(req, store_name)
218224
219 image_meta['status'] = 'saving'225 image_meta['status'] = 'saving'
226 image_id = image_meta['id']
227 logger.debug("Updating image metadata for image %s"
228 % image_id)
220 registry.update_image_metadata(self.options,229 registry.update_image_metadata(self.options,
221 image_meta['id'],230 image_meta['id'],
222 image_meta)231 image_meta)
223232
224 try:233 try:
234 logger.debug("Uploading image data for image %(image_id)s "
235 "to %(store_name)s store" % locals())
225 location, size = store.add(image_meta['id'],236 location, size = store.add(image_meta['id'],
226 req.body_file,237 req.body_file,
227 self.options)238 self.options)
@@ -230,12 +241,14 @@
230 # the new size of the image241 # the new size of the image
231 if image_meta.get('size', 0) != size:242 if image_meta.get('size', 0) != size:
232 image_meta['size'] = size243 image_meta['size'] = size
244 logger.debug("Updating image metadata for image %s"
245 % image_id)
233 registry.update_image_metadata(self.options,246 registry.update_image_metadata(self.options,
234 image_meta['id'],247 image_meta['id'],
235 image_meta)248 image_meta)
236 return location249 return location
237 except exception.Duplicate, e:250 except exception.Duplicate, e:
238 logging.error("Error adding image to store: %s", str(e))251 logger.error("Error adding image to store: %s", str(e))
239 raise HTTPConflict(str(e), request=req)252 raise HTTPConflict(str(e), request=req)
240253
241 def _activate(self, req, image_meta, location):254 def _activate(self, req, image_meta, location):
@@ -277,7 +290,7 @@
277 try:290 try:
278 self._kill(req, image_meta)291 self._kill(req, image_meta)
279 except Exception, e:292 except Exception, e:
280 logging.error("Unable to kill image %s: %s",293 logger.error("Unable to kill image %s: %s",
281 image_meta['id'], repr(e))294 image_meta['id'], repr(e))
282295
283 def _upload_and_activate(self, req, image_meta):296 def _upload_and_activate(self, req, image_meta):
@@ -367,14 +380,21 @@
367 raise HTTPConflict("Cannot upload to an unqueued image")380 raise HTTPConflict("Cannot upload to an unqueued image")
368381
369 new_image_meta = utils.get_image_meta_from_headers(req)382 new_image_meta = utils.get_image_meta_from_headers(req)
370 image_meta = registry.update_image_metadata(self.options,383 try:
371 id,384 image_meta = registry.update_image_metadata(self.options,
372 new_image_meta)385 id,
373386 new_image_meta)
374 if has_body:387
375 self._upload_and_activate(req, image_meta)388 if has_body:
376389 self._upload_and_activate(req, image_meta)
377 return dict(image=image_meta)390
391 return dict(image=image_meta)
392 except exception.Invalid, e:
393 msg = ("Failed to update image metadata. Got error: %(e)s"
394 % locals())
395 for line in msg.split('\n'):
396 logger.error(line)
397 raise HTTPBadRequest(msg, request=req, content_type="text/plain")
378398
379 def delete(self, req, id):399 def delete(self, req, id):
380 """400 """
@@ -407,8 +427,9 @@
407 try:427 try:
408 return registry.get_image_metadata(self.options, id)428 return registry.get_image_metadata(self.options, id)
409 except exception.NotFound:429 except exception.NotFound:
410 raise HTTPNotFound(body='Image not found',430 msg = "Image with identifier %s not found" % id
411 request=request,431 logger.debug(msg)
432 raise HTTPNotFound(msg, request=request,
412 content_type='text/plain')433 content_type='text/plain')
413434
414 def get_store_or_400(self, request, store_name):435 def get_store_or_400(self, request, store_name):
@@ -424,10 +445,10 @@
424 try:445 try:
425 return get_backend_class(store_name)446 return get_backend_class(store_name)
426 except UnsupportedBackend:447 except UnsupportedBackend:
427 raise HTTPBadRequest(body='Requested store %s not available '448 msg = ("Requested store %s not available on this Glance server"
428 'for storage on this Glance node'449 % store_name)
429 % store_name,450 logger.error(msg)
430 request=request,451 raise HTTPBadRequest(msg, request=request,
431 content_type='text/plain')452 content_type='text/plain')
432453
433454
434455
=== modified file 'tests/unit/test_clients.py'
--- tests/unit/test_clients.py 2011-01-28 21:54:34 +0000
+++ tests/unit/test_clients.py 2011-02-23 20:37:56 +0000
@@ -204,7 +204,7 @@
204 'location': "file:///tmp/glance-tests/2",204 'location': "file:///tmp/glance-tests/2",
205 }205 }
206206
207 self.assertRaises(exception.BadInputError,207 self.assertRaises(exception.Invalid,
208 self.client.add_image,208 self.client.add_image,
209 fixture)209 fixture)
210210
@@ -567,7 +567,7 @@
567567
568 image_data_fixture = r"chunk00000remainder"568 image_data_fixture = r"chunk00000remainder"
569569
570 self.assertRaises(exception.BadInputError,570 self.assertRaises(exception.Invalid,
571 self.client.add_image,571 self.client.add_image,
572 fixture,572 fixture,
573 image_data_fixture)573 image_data_fixture)
574574
=== added file 'tests/unit/test_misc.py'
--- tests/unit/test_misc.py 1970-01-01 00:00:00 +0000
+++ tests/unit/test_misc.py 2011-02-23 20:37:56 +0000
@@ -0,0 +1,178 @@
1# vim: tabstop=4 shiftwidth=4 softtabstop=4
2
3# Copyright 2011 OpenStack, LLC
4# All Rights Reserved.
5#
6# Licensed under the Apache License, Version 2.0 (the "License"); you may
7# not use this file except in compliance with the License. You may obtain
8# a copy of the License at
9#
10# http://www.apache.org/licenses/LICENSE-2.0
11#
12# Unless required by applicable law or agreed to in writing, software
13# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
14# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
15# License for the specific language governing permissions and limitations
16# under the License.
17
18import os
19import shutil
20import signal
21import subprocess
22import tempfile
23import time
24import unittest
25
26
27def execute(cmd):
28 env = os.environ.copy()
29 # Make sure that we use the programs in the
30 # current source directory's bin/ directory.
31 env['PATH'] = os.path.join(os.getcwd(), 'bin') + ':' + env['PATH']
32 process = subprocess.Popen(cmd,
33 shell=True,
34 stdin=subprocess.PIPE,
35 stdout=subprocess.PIPE,
36 stderr=subprocess.PIPE,
37 env=env)
38 result = process.communicate()
39 (out, err) = result
40 exitcode = process.returncode
41 if process.returncode != 0:
42 msg = "Command %(cmd)s did not succeed. Returned an exit "\
43 "code of %(exitcode)d."\
44 "\n\nSTDOUT: %(out)s"\
45 "\n\nSTDERR: %(err)s" % locals()
46 raise RuntimeError(msg)
47 return exitcode, out, err
48
49
50class TestMiscellaneous(unittest.TestCase):
51
52 """Some random tests for various bugs and stuff"""
53
54 def tearDown(self):
55 self._cleanup_test_servers()
56
57 def _cleanup_test_servers(self):
58 # Clean up any leftover test servers...
59 pid_files = ('glance-api.pid', 'glance-registry.pid')
60 for pid_file in pid_files:
61 if os.path.exists(pid_file):
62 pid = int(open(pid_file).read().strip())
63 try:
64 os.killpg(pid, signal.SIGTERM)
65 except:
66 pass # Ignore if the process group is dead
67 os.unlink(pid_file)
68
69 def test_exception_not_eaten_from_registry_to_api(self):
70 """
71 A test for LP bug #704854 -- Exception thrown by registry
72 server is consumed by API server.
73
74 We start both servers daemonized.
75
76 We then use curl to try adding an image that does not
77 meet validation requirements on the registry server and test
78 that the error returned from the API server to curl is appropriate
79
80 We also fire the glance-upload tool against the API server
81 and verify that glance-upload doesn't eat the exception either...
82 """
83
84 self._cleanup_test_servers()
85
86 # Port numbers hopefully not used by anything...
87 api_port = 32001
88 reg_port = 32000
89 image_dir = "/tmp/test.images.%d" % api_port
90 if os.path.exists(image_dir):
91 shutil.rmtree(image_dir)
92
93 # A config file to use just for this test...we don't want
94 # to trample on currently-running Glance servers, now do we?
95 with tempfile.NamedTemporaryFile() as conf_file:
96 conf_contents = """[DEFAULT]
97verbose = True
98debug = True
99
100[app:glance-api]
101paste.app_factory = glance.server:app_factory
102filesystem_store_datadir=%(image_dir)s
103default_store = file
104bind_host = 0.0.0.0
105bind_port = %(api_port)s
106registry_host = 0.0.0.0
107registry_port = %(reg_port)s
108
109[app:glance-registry]
110paste.app_factory = glance.registry.server:app_factory
111bind_host = 0.0.0.0
112bind_port = %(reg_port)s
113sql_connection = sqlite://
114sql_idle_timeout = 3600
115""" % locals()
116 conf_file.write(conf_contents)
117 conf_file.flush()
118 conf_file_name = conf_file.name
119
120 venv = ""
121 if 'VIRTUAL_ENV' in os.environ:
122 venv = "tools/with_venv.sh "
123
124 # Start up the API and default registry server
125 cmd = venv + "./bin/glance-control api start "\
126 "%s --pid-file=glance-api.pid" % conf_file_name
127 exitcode, out, err = execute(cmd)
128
129 self.assertEquals(0, exitcode)
130 self.assertTrue("Starting glance-api with" in out)
131
132 cmd = venv + "./bin/glance-control registry start "\
133 "%s --pid-file=glance-registry.pid" % conf_file_name
134 exitcode, out, err = execute(cmd)
135
136 self.assertEquals(0, exitcode)
137 self.assertTrue("Starting glance-registry with" in out)
138
139 time.sleep(2) # Gotta give some time for spin up...
140
141 cmd = "curl -g http://0.0.0.0:%d/images" % api_port
142
143 exitcode, out, err = execute(cmd)
144
145 self.assertEquals(0, exitcode)
146 self.assertEquals('{"images": []}', out.strip())
147
148 cmd = "curl -X POST -H 'Content-Type: application/octet-stream' "\
149 "-dinvalid http://0.0.0.0:%d/images" % api_port
150 ignored, out, err = execute(cmd)
151
152 self.assertTrue('Image type is required' in out,
153 "Could not find 'Image type is required' "
154 "in output: %s" % out)
155
156 cmd = "./bin/glance-upload --port=%(api_port)d "\
157 "--type=invalid %(conf_file_name)s 'my image'" % locals()
158
159 # Normally, we would simply self.assertRaises() here, but
160 # we also want to check that the Invalid image type is in
161 # the exception text...
162 hit_exception = False
163 try:
164 ignored, out, err = execute(cmd)
165 except RuntimeError, e:
166 hit_exception = True
167 self.assertTrue('Invalid image type' in str(e),
168 "Could not find 'Invalid image type' in "
169 "result from glance-upload:\n%(e)s" % locals())
170 self.assertTrue(hit_exception)
171
172 # Spin down the API and default registry server
173 cmd = "./bin/glance-control api stop "\
174 "%s --pid-file=glance-api.pid" % conf_file_name
175 ignored, out, err = execute(cmd)
176 cmd = "./bin/glance-control registry stop "\
177 "%s --pid-file=glance-registry.pid" % conf_file_name
178 ignored, out, err = execute(cmd)

Subscribers

People subscribed via source and target branches