Merge lp:~rconradharris/glance/lp615675 into lp:~hudson-openstack/glance/trunk

Proposed by Rick Harris
Status: Merged
Approved by: Christopher MacGown
Approved revision: 12
Merged at revision: 8
Proposed branch: lp:~rconradharris/glance/lp615675
Merge into: lp:~hudson-openstack/glance/trunk
Diff against target: 1755 lines (+1655/-16)
16 files modified
bin/parallax-server.py (+51/-0)
glance/common/db/__init__.py (+23/-0)
glance/common/db/api.py (+88/-0)
glance/common/db/sqlalchemy/__init__.py (+24/-0)
glance/common/db/sqlalchemy/api.py (+127/-0)
glance/common/db/sqlalchemy/models.py (+191/-0)
glance/common/db/sqlalchemy/session.py (+42/-0)
glance/common/exception.py (+87/-0)
glance/common/flags.py (+175/-0)
glance/common/server.py (+144/-0)
glance/common/utils.py (+204/-0)
glance/common/wsgi.py (+298/-0)
glance/parallax/__init__.py (+21/-16)
glance/parallax/api/__init__.py (+49/-0)
glance/parallax/api/images.py (+82/-0)
tests/test_data.py (+49/-0)
To merge this branch: bzr merge lp:~rconradharris/glance/lp615675
Reviewer Review Type Date Requested Status
Christopher MacGown (community) Approve
Review via email: mp+37024@code.launchpad.net

Description of the change

This patch:
  * pulls in a number of useful libraries from Nova under the common/ path (we can factor those out to a shared library in Bexar-release)
  * Defines the models in common.db.sqlalchemy.models.py (this should be factored out into the parallax package soon)
  * Adds the parallax api-server under /bin (if PyPI was used to pull python-daemon and python-lockfile, you may need to apply a patch I have against it)

To post a comment you must log in.
Revision history for this message
Christopher MacGown (0x44) wrote :

lgtm

review: Approve
Revision history for this message
Christopher MacGown (0x44) wrote :

Actually, hold that. In the below, any instance of chunk_dicts and chunk should be replaced by file, since they're the files that make up an image. Teller uses the term 'chunk' to mean an arbitrary sized piece of the read file buffer yielded to the requester.

82 + chunk_dicts = []
1683 + for chunk in image.image_chunks:
1684 + chunk_dict = dict(location=chunk.location, size=chunk.size)
1685 + chunk_dicts.append(chunk_dict)
1686 +
1687 + metadata_dicts = []
1688 + for metadatum in image.image_metadata:
1689 + metadatum_dict = dict(key=metadatum.key, value=metadatum.value)
1690 + metadata_dicts.append(metadatum_dict)
1691 +
1692 + image_dict = dict(id=image.id, name=image.name, state=image.state,
1693 + public=image.public, chunks=chunk_dicts,

review: Needs Fixing
Revision history for this message
Jay Pipes (jaypipes) wrote :

Looks good! We can refactor the common stuff into openstack-common after Austin, of course.

The only suggestion I have is to combine the glance.parallax.api.API and glance.parallax.api.APIRouter class into a single class. wsgi.Router *is* middleware (since its constructor sets routes.RoutesMiddleware as the application to return, and its __call__ method returns that application, so unless I'm mistaken, the following code should work fine:

class API(wsgi.Router):
    """
    Routes requests on the Parallax to the appropriate controller
    and method.
    """

    def __init__(self):
        mapper = routes.Mapper()
        mapper.resource("image", "images", controller=images.Controller(),
            collection={'detail': 'GET'})
        super(APIRouter, self).__init__(mapper)

Cheers!
jay

Revision history for this message
Jay Pipes (jaypipes) wrote :

s/super(APIRouter/super(API/

in the code above.

lp:~rconradharris/glance/lp615675 updated
12. By Rick Harris

ImageChunk -> ImageFile, merging APIRouter into API for now

Revision history for this message
Christopher MacGown (0x44) wrote :

Awesome.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'bin/parallax-server.py'
2--- bin/parallax-server.py 1970-01-01 00:00:00 +0000
3+++ bin/parallax-server.py 2010-09-29 20:56:47 +0000
4@@ -0,0 +1,51 @@
5+#!/usr/bin/env python
6+# pylint: disable-msg=C0103
7+# vim: tabstop=4 shiftwidth=4 softtabstop=4
8+
9+# Copyright 2010 United States Government as represented by the
10+# Administrator of the National Aeronautics and Space Administration.
11+# All Rights Reserved.
12+#
13+# Licensed under the Apache License, Version 2.0 (the "License");
14+# you may not use this file except in compliance with the License.
15+# You may obtain a copy of the License at
16+#
17+# http://www.apache.org/licenses/LICENSE-2.0
18+#
19+# Unless required by applicable law or agreed to in writing, software
20+# distributed under the License is distributed on an "AS IS" BASIS,
21+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+# See the License for the specific language governing permissions and
23+# limitations under the License.
24+"""
25+Parallax API daemon.
26+"""
27+
28+import os
29+import sys
30+
31+# If ../parallax/__init__.py exists, add ../ to Python search path, so that
32+# it will override what happens to be installed in /usr/(local/)lib/python...
33+possible_topdir = os.path.normpath(os.path.join(os.path.abspath(sys.argv[0]),
34+ os.pardir,
35+ os.pardir))
36+if os.path.exists(os.path.join(possible_topdir, 'parallax', '__init__.py')):
37+ sys.path.insert(0, possible_topdir)
38+
39+from glance.common import flags
40+from glance.common import utils
41+from glance.common import server
42+from glance.common import wsgi
43+from glance.parallax import api
44+
45+
46+FLAGS = flags.FLAGS
47+# TODO(sirp): ensure no conflicts in port selection
48+flags.DEFINE_integer('parallax_port', 9191, 'Parallax port')
49+
50+def main(_args):
51+ wsgi.run_server(api.API(), FLAGS.parallax_port)
52+
53+if __name__ == '__main__':
54+ utils.default_flagfile()
55+ server.serve('parallax-server', main)
56
57=== added directory 'glance/common'
58=== added file 'glance/common/__init__.py'
59=== added directory 'glance/common/db'
60=== added file 'glance/common/db/__init__.py'
61--- glance/common/db/__init__.py 1970-01-01 00:00:00 +0000
62+++ glance/common/db/__init__.py 2010-09-29 20:56:47 +0000
63@@ -0,0 +1,23 @@
64+# vim: tabstop=4 shiftwidth=4 softtabstop=4
65+# vim: tabstop=4 shiftwidth=4 softtabstop=4
66+
67+# Copyright 2010 United States Government as represented by the
68+# Administrator of the National Aeronautics and Space Administration.
69+# All Rights Reserved.
70+#
71+# Licensed under the Apache License, Version 2.0 (the "License"); you may
72+# not use this file except in compliance with the License. You may obtain
73+# a copy of the License at
74+#
75+# http://www.apache.org/licenses/LICENSE-2.0
76+#
77+# Unless required by applicable law or agreed to in writing, software
78+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
79+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
80+# License for the specific language governing permissions and limitations
81+# under the License.
82+"""
83+DB abstraction for Nova and Glance
84+"""
85+
86+from glance.common.db.api import *
87
88=== added file 'glance/common/db/api.py'
89--- glance/common/db/api.py 1970-01-01 00:00:00 +0000
90+++ glance/common/db/api.py 2010-09-29 20:56:47 +0000
91@@ -0,0 +1,88 @@
92+# vim: tabstop=4 shiftwidth=4 softtabstop=4
93+
94+# Copyright 2010 United States Government as represented by the
95+# Administrator of the National Aeronautics and Space Administration.
96+# All Rights Reserved.
97+#
98+# Licensed under the Apache License, Version 2.0 (the "License"); you may
99+# not use this file except in compliance with the License. You may obtain
100+# a copy of the License at
101+#
102+# http://www.apache.org/licenses/LICENSE-2.0
103+#
104+# Unless required by applicable law or agreed to in writing, software
105+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
106+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
107+# License for the specific language governing permissions and limitations
108+# under the License.
109+"""
110+Defines interface for DB access
111+"""
112+
113+from glance.common import exception
114+from glance.common import flags
115+from glance.common import utils
116+
117+
118+FLAGS = flags.FLAGS
119+flags.DEFINE_string('db_backend', 'sqlalchemy',
120+ 'The backend to use for db')
121+
122+
123+IMPL = utils.LazyPluggable(FLAGS['db_backend'],
124+ sqlalchemy='glance.common.db.sqlalchemy.api')
125+
126+
127+###################
128+
129+
130+def image_create(context, values):
131+ """Create an image from the values dictionary."""
132+ return IMPL.image_create(context, values)
133+
134+
135+
136+def image_destroy(context, image_id):
137+ """Destroy the image or raise if it does not exist."""
138+ return IMPL.image_destroy(context, image_id)
139+
140+
141+def image_get(context, image_id):
142+ """Get an image or raise if it does not exist."""
143+ return IMPL.image_get(context, image_id)
144+
145+
146+def image_get_all(context):
147+ """Get all images."""
148+ return IMPL.image_get_all(context)
149+
150+
151+def image_get_by_str(context, str_id):
152+ """Get an image by string id."""
153+ return IMPL.image_get_by_str(context, str_id)
154+
155+
156+def image_update(context, image_id, values):
157+ """Set the given properties on an image and update it.
158+
159+ Raises NotFound if image does not exist.
160+
161+ """
162+ return IMPL.image_update(context, image_id, values)
163+
164+
165+###################
166+
167+
168+def image_file_create(context, values):
169+ """Create an image file from the values dictionary."""
170+ return IMPL.image_file_create(context, values)
171+
172+
173+###################
174+
175+
176+def image_metadatum_create(context, values):
177+ """Create an image metadatum from the values dictionary."""
178+ return IMPL.image_metadatum_create(context, values)
179+
180
181=== added directory 'glance/common/db/sqlalchemy'
182=== added file 'glance/common/db/sqlalchemy/__init__.py'
183--- glance/common/db/sqlalchemy/__init__.py 1970-01-01 00:00:00 +0000
184+++ glance/common/db/sqlalchemy/__init__.py 2010-09-29 20:56:47 +0000
185@@ -0,0 +1,24 @@
186+# vim: tabstop=4 shiftwidth=4 softtabstop=4
187+
188+# Copyright 2010 United States Government as represented by the
189+# Administrator of the National Aeronautics and Space Administration.
190+# All Rights Reserved.
191+#
192+# Licensed under the Apache License, Version 2.0 (the "License"); you may
193+# not use this file except in compliance with the License. You may obtain
194+# a copy of the License at
195+#
196+# http://www.apache.org/licenses/LICENSE-2.0
197+#
198+# Unless required by applicable law or agreed to in writing, software
199+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
200+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
201+# License for the specific language governing permissions and limitations
202+# under the License.
203+
204+"""
205+SQLAlchemy database backend
206+"""
207+from glance.common.db.sqlalchemy import models
208+
209+models.register_models()
210
211=== added file 'glance/common/db/sqlalchemy/api.py'
212--- glance/common/db/sqlalchemy/api.py 1970-01-01 00:00:00 +0000
213+++ glance/common/db/sqlalchemy/api.py 2010-09-29 20:56:47 +0000
214@@ -0,0 +1,127 @@
215+# vim: tabstop=4 shiftwidth=4 softtabstop=4
216+
217+# Copyright 2010 United States Government as represented by the
218+# Administrator of the National Aeronautics and Space Administration.
219+# All Rights Reserved.
220+#
221+# Licensed under the Apache License, Version 2.0 (the "License"); you may
222+# not use this file except in compliance with the License. You may obtain
223+# a copy of the License at
224+#
225+# http://www.apache.org/licenses/LICENSE-2.0
226+#
227+# Unless required by applicable law or agreed to in writing, software
228+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
229+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
230+# License for the specific language governing permissions and limitations
231+# under the License.
232+"""
233+Implementation of SQLAlchemy backend
234+"""
235+
236+import sys
237+from glance.common import db
238+from glance.common import exception
239+from glance.common import flags
240+from glance.common.db.sqlalchemy import models
241+from glance.common.db.sqlalchemy.session import get_session
242+from sqlalchemy.orm import exc
243+
244+#from sqlalchemy.orm import joinedload_all
245+# TODO(sirp): add back eager loading
246+from sqlalchemy.orm import joinedload
247+from sqlalchemy.sql import func
248+
249+FLAGS = flags.FLAGS
250+
251+
252+# NOTE(vish): disabling docstring pylint because the docstrings are
253+# in the interface definition
254+# pylint: disable-msg=C0111
255+def _deleted(context):
256+ """Calculates whether to include deleted objects based on context.
257+
258+ Currently just looks for a flag called deleted in the context dict.
259+ """
260+ if not hasattr(context, 'get'):
261+ return False
262+ return context.get('deleted', False)
263+
264+
265+###################
266+
267+
268+def image_create(_context, values):
269+ image_ref = models.Image()
270+ for (key, value) in values.iteritems():
271+ image_ref[key] = value
272+ image_ref.save()
273+ return image_ref
274+
275+
276+def image_destroy(_context, image_id):
277+ session = get_session()
278+ with session.begin():
279+ image_ref = models.Image.find(image_id, session=session)
280+ image_ref.delete(session=session)
281+
282+
283+def image_get(context, image_id):
284+ session = get_session()
285+ try:
286+ return session.query(models.Image
287+ ).options(joinedload(models.Image.files)
288+ ).options(joinedload(models.Image.metadata)
289+ ).filter_by(deleted=_deleted(context)
290+ ).filter_by(id=image_id
291+ ).one()
292+ except exc.NoResultFound:
293+ new_exc = exception.NotFound("No model for id %s" % image_id)
294+ raise new_exc.__class__, new_exc, sys.exc_info()[2]
295+ return models.Image.find(image_id, deleted=_deleted(context))
296+
297+
298+def image_get_all(context):
299+ session = get_session()
300+ # TODO(sirp): add back eager loading
301+ return session.query(models.Image
302+ #).options(joinedload(models.Image.files)
303+ #).options(joinedload(models.Image.metadata)
304+ ).filter_by(deleted=_deleted(context)
305+ ).all()
306+
307+
308+def image_get_by_str(context, str_id):
309+ return models.Image.find_by_str(str_id, deleted=_deleted(context))
310+
311+
312+def image_update(_context, image_id, values):
313+ session = get_session()
314+ with session.begin():
315+ image_ref = models.Image.find(image_id, session=session)
316+ for (key, value) in values.iteritems():
317+ image_ref[key] = value
318+ image_ref.save(session=session)
319+
320+
321+###################
322+
323+
324+def image_file_create(_context, values):
325+ image_file_ref = models.ImageFile()
326+ for (key, value) in values.iteritems():
327+ image_file_ref[key] = value
328+ image_file_ref.save()
329+ return image_file_ref
330+
331+
332+###################
333+
334+
335+def image_metadatum_create(_context, values):
336+ image_metadatum_ref = models.ImageMetadatum()
337+ for (key, value) in values.iteritems():
338+ image_metadatum_ref[key] = value
339+ image_metadatum_ref.save()
340+ return image_metadatum_ref
341+
342
343=== added file 'glance/common/db/sqlalchemy/models.py'
344--- glance/common/db/sqlalchemy/models.py 1970-01-01 00:00:00 +0000
345+++ glance/common/db/sqlalchemy/models.py 2010-09-29 20:56:47 +0000
346@@ -0,0 +1,191 @@
347+# vim: tabstop=4 shiftwidth=4 softtabstop=4
348+
349+# Copyright 2010 United States Government as represented by the
350+# Administrator of the National Aeronautics and Space Administration.
351+# All Rights Reserved.
352+#
353+# Licensed under the Apache License, Version 2.0 (the "License"); you may
354+# not use this file except in compliance with the License. You may obtain
355+# a copy of the License at
356+#
357+# http://www.apache.org/licenses/LICENSE-2.0
358+#
359+# Unless required by applicable law or agreed to in writing, software
360+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
361+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
362+# License for the specific language governing permissions and limitations
363+# under the License.
364+
365+"""
366+SQLAlchemy models for glance data
367+"""
368+
369+import sys
370+import datetime
371+
372+# TODO(vish): clean up these imports
373+from sqlalchemy.orm import relationship, backref, exc, object_mapper
374+from sqlalchemy import Column, Integer, String
375+from sqlalchemy import ForeignKey, DateTime, Boolean, Text
376+from sqlalchemy.ext.declarative import declarative_base
377+
378+from glance.common.db.sqlalchemy.session import get_session
379+
380+# FIXME(sirp): confirm this is not needed
381+#from common import auth
382+from glance.common import exception
383+from glance.common import flags
384+
385+FLAGS = flags.FLAGS
386+
387+BASE = declarative_base()
388+
389+#TODO(sirp): ModelBase should be moved out so Glance and Nova can share it
390+class ModelBase(object):
391+ """Base class for Nova and Glance Models"""
392+ __table_args__ = {'mysql_engine': 'InnoDB'}
393+ __table_initialized__ = False
394+ __prefix__ = 'none'
395+ created_at = Column(DateTime, default=datetime.datetime.utcnow)
396+ updated_at = Column(DateTime, onupdate=datetime.datetime.utcnow)
397+ deleted_at = Column(DateTime)
398+ deleted = Column(Boolean, default=False)
399+
400+ @classmethod
401+ def all(cls, session=None, deleted=False):
402+ """Get all objects of this type"""
403+ if not session:
404+ session = get_session()
405+ return session.query(cls
406+ ).filter_by(deleted=deleted
407+ ).all()
408+
409+ @classmethod
410+ def count(cls, session=None, deleted=False):
411+ """Count objects of this type"""
412+ if not session:
413+ session = get_session()
414+ return session.query(cls
415+ ).filter_by(deleted=deleted
416+ ).count()
417+
418+ @classmethod
419+ def find(cls, obj_id, session=None, deleted=False):
420+ """Find object by id"""
421+ if not session:
422+ session = get_session()
423+ try:
424+ return session.query(cls
425+ ).filter_by(id=obj_id
426+ ).filter_by(deleted=deleted
427+ ).one()
428+ except exc.NoResultFound:
429+ new_exc = exception.NotFound("No model for id %s" % obj_id)
430+ raise new_exc.__class__, new_exc, sys.exc_info()[2]
431+
432+ @classmethod
433+ def find_by_str(cls, str_id, session=None, deleted=False):
434+ """Find object by str_id"""
435+ int_id = int(str_id.rpartition('-')[2])
436+ return cls.find(int_id, session=session, deleted=deleted)
437+
438+ @property
439+ def str_id(self):
440+ """Get string id of object (generally prefix + '-' + id)"""
441+ return "%s-%s" % (self.__prefix__, self.id)
442+
443+ def save(self, session=None):
444+ """Save this object"""
445+ if not session:
446+ session = get_session()
447+ session.add(self)
448+ session.flush()
449+
450+ def delete(self, session=None):
451+ """Delete this object"""
452+ self.deleted = True
453+ self.deleted_at = datetime.datetime.utcnow()
454+ self.save(session=session)
455+
456+ def __setitem__(self, key, value):
457+ setattr(self, key, value)
458+
459+ def __getitem__(self, key):
460+ return getattr(self, key)
461+
462+ def __iter__(self):
463+ self._i = iter(object_mapper(self).columns)
464+ return self
465+
466+ def next(self):
467+ n = self._i.next().name
468+ return n, getattr(self, n)
469+
470+class Image(BASE, ModelBase):
471+ """Represents an image in the datastore"""
472+ __tablename__ = 'images'
473+ __prefix__ = 'img'
474+
475+ id = Column(Integer, primary_key=True)
476+ name = Column(String(255))
477+ image_type = Column(String(255))
478+ state = Column(String(255))
479+ public = Column(Boolean, default=False)
480+
481+ #@validates('image_type')
482+ #def validate_image_type(self, key, image_type):
483+ # assert(image_type in ('machine', 'kernel', 'ramdisk', 'raw'))
484+ #
485+ #@validates('state')
486+ #def validate_state(self, key, state):
487+ # assert(state in ('available', 'pending', 'disabled'))
488+ #
489+ # TODO(sirp): should these be stored as metadata?
490+ #user_id = Column(String(255))
491+ #project_id = Column(String(255))
492+ #arch = Column(String(255))
493+ #default_kernel_id = Column(String(255))
494+ #default_ramdisk_id = Column(String(255))
495+ #
496+ #@validates('default_kernel_id')
497+ #def validate_kernel_id(self, key, val):
498+ # if val != 'machine':
499+ # assert(val is None)
500+ #
501+ #@validates('default_ramdisk_id')
502+ #def validate_ramdisk_id(self, key, val):
503+ # if val != 'machine':
504+ # assert(val is None)
505+
506+
507+class ImageFile(BASE, ModelBase):
508+ """Represents an image file in the datastore"""
509+ __tablename__ = 'image_files'
510+ __prefix__ = 'img-file'
511+ id = Column(Integer, primary_key=True)
512+ image_id = Column(Integer, ForeignKey('images.id'), nullable=False)
513+ image = relationship(Image, backref=backref('files'))
514+
515+ location = Column(String(255))
516+ size = Column(Integer)
517+
518+
519+class ImageMetadatum(BASE, ModelBase):
520+ """Represents an image metadata in the datastore"""
521+ __tablename__ = 'image_metadata'
522+ __prefix__ = 'mdata'
523+ id = Column(Integer, primary_key=True)
524+ image_id = Column(Integer, ForeignKey('images.id'), nullable=False)
525+ image = relationship(Image, backref=backref('metadata'))
526+
527+ key = Column(String(255), index=True, unique=True)
528+ value = Column(Text)
529+
530+
531+def register_models():
532+ """Register Models and create metadata"""
533+ from sqlalchemy import create_engine
534+ models = (Image, ImageFile, ImageMetadatum)
535+ engine = create_engine(FLAGS.sql_connection, echo=False)
536+ for model in models:
537+ model.metadata.create_all(engine)
538
539=== added file 'glance/common/db/sqlalchemy/session.py'
540--- glance/common/db/sqlalchemy/session.py 1970-01-01 00:00:00 +0000
541+++ glance/common/db/sqlalchemy/session.py 2010-09-29 20:56:47 +0000
542@@ -0,0 +1,42 @@
543+# vim: tabstop=4 shiftwidth=4 softtabstop=4
544+
545+# Copyright 2010 United States Government as represented by the
546+# Administrator of the National Aeronautics and Space Administration.
547+# All Rights Reserved.
548+#
549+# Licensed under the Apache License, Version 2.0 (the "License"); you may
550+# not use this file except in compliance with the License. You may obtain
551+# a copy of the License at
552+#
553+# http://www.apache.org/licenses/LICENSE-2.0
554+#
555+# Unless required by applicable law or agreed to in writing, software
556+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
557+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
558+# License for the specific language governing permissions and limitations
559+# under the License.
560+"""
561+Session Handling for SQLAlchemy backend
562+"""
563+
564+from sqlalchemy import create_engine
565+from sqlalchemy.orm import sessionmaker
566+
567+from glance.common import flags
568+
569+FLAGS = flags.FLAGS
570+
571+_ENGINE = None
572+_MAKER = None
573+
574+def get_session(autocommit=True, expire_on_commit=False):
575+ """Helper method to grab session"""
576+ global _ENGINE
577+ global _MAKER
578+ if not _MAKER:
579+ if not _ENGINE:
580+ _ENGINE = create_engine(FLAGS.sql_connection, echo=False)
581+ _MAKER = sessionmaker(bind=_ENGINE,
582+ autocommit=autocommit,
583+ expire_on_commit=expire_on_commit)
584+ return _MAKER()
585
586=== added file 'glance/common/exception.py'
587--- glance/common/exception.py 1970-01-01 00:00:00 +0000
588+++ glance/common/exception.py 2010-09-29 20:56:47 +0000
589@@ -0,0 +1,87 @@
590+# vim: tabstop=4 shiftwidth=4 softtabstop=4
591+
592+# Copyright 2010 United States Government as represented by the
593+# Administrator of the National Aeronautics and Space Administration.
594+# All Rights Reserved.
595+#
596+# Licensed under the Apache License, Version 2.0 (the "License"); you may
597+# not use this file except in compliance with the License. You may obtain
598+# a copy of the License at
599+#
600+# http://www.apache.org/licenses/LICENSE-2.0
601+#
602+# Unless required by applicable law or agreed to in writing, software
603+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
604+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
605+# License for the specific language governing permissions and limitations
606+# under the License.
607+
608+"""
609+Nova base exception handling, including decorator for re-raising
610+Nova-type exceptions. SHOULD include dedicated exception logging.
611+"""
612+
613+import logging
614+import sys
615+import traceback
616+
617+
618+class ProcessExecutionError(IOError):
619+ def __init__(self, stdout=None, stderr=None, exit_code=None, cmd=None,
620+ description=None):
621+ if description is None:
622+ description = "Unexpected error while running command."
623+ if exit_code is None:
624+ exit_code = '-'
625+ message = "%s\nCommand: %s\nExit code: %s\nStdout: %r\nStderr: %r" % (
626+ description, cmd, exit_code, stdout, stderr)
627+ IOError.__init__(self, message)
628+
629+
630+class Error(Exception):
631+ def __init__(self, message=None):
632+ super(Error, self).__init__(message)
633+
634+
635+class ApiError(Error):
636+ def __init__(self, message='Unknown', code='Unknown'):
637+ self.message = message
638+ self.code = code
639+ super(ApiError, self).__init__('%s: %s'% (code, message))
640+
641+
642+class NotFound(Error):
643+ pass
644+
645+
646+class Duplicate(Error):
647+ pass
648+
649+
650+class NotAuthorized(Error):
651+ pass
652+
653+
654+class NotEmpty(Error):
655+ pass
656+
657+
658+class Invalid(Error):
659+ pass
660+
661+
662+def wrap_exception(f):
663+ def _wrap(*args, **kw):
664+ try:
665+ return f(*args, **kw)
666+ except Exception, e:
667+ if not isinstance(e, Error):
668+ #exc_type, exc_value, exc_traceback = sys.exc_info()
669+ logging.exception('Uncaught exception')
670+ #logging.error(traceback.extract_stack(exc_traceback))
671+ raise Error(str(e))
672+ raise
673+ _wrap.func_name = f.func_name
674+ return _wrap
675+
676+
677
678=== added file 'glance/common/flags.py'
679--- glance/common/flags.py 1970-01-01 00:00:00 +0000
680+++ glance/common/flags.py 2010-09-29 20:56:47 +0000
681@@ -0,0 +1,175 @@
682+# vim: tabstop=4 shiftwidth=4 softtabstop=4
683+
684+# Copyright 2010 United States Government as represented by the
685+# Administrator of the National Aeronautics and Space Administration.
686+# All Rights Reserved.
687+#
688+# Licensed under the Apache License, Version 2.0 (the "License"); you may
689+# not use this file except in compliance with the License. You may obtain
690+# a copy of the License at
691+#
692+# http://www.apache.org/licenses/LICENSE-2.0
693+#
694+# Unless required by applicable law or agreed to in writing, software
695+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
696+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
697+# License for the specific language governing permissions and limitations
698+# under the License.
699+
700+"""
701+Package-level global flags are defined here, the rest are defined
702+where they're used.
703+"""
704+
705+import getopt
706+import os
707+import socket
708+import sys
709+
710+import gflags
711+
712+
713+class FlagValues(gflags.FlagValues):
714+ """Extension of gflags.FlagValues that allows undefined and runtime flags.
715+
716+ Unknown flags will be ignored when parsing the command line, but the
717+ command line will be kept so that it can be replayed if new flags are
718+ defined after the initial parsing.
719+
720+ """
721+
722+ def __init__(self):
723+ gflags.FlagValues.__init__(self)
724+ self.__dict__['__dirty'] = []
725+ self.__dict__['__was_already_parsed'] = False
726+ self.__dict__['__stored_argv'] = []
727+
728+ def __call__(self, argv):
729+ # We're doing some hacky stuff here so that we don't have to copy
730+ # out all the code of the original verbatim and then tweak a few lines.
731+ # We're hijacking the output of getopt so we can still return the
732+ # leftover args at the end
733+ sneaky_unparsed_args = {"value": None}
734+ original_argv = list(argv)
735+
736+ if self.IsGnuGetOpt():
737+ orig_getopt = getattr(getopt, 'gnu_getopt')
738+ orig_name = 'gnu_getopt'
739+ else:
740+ orig_getopt = getattr(getopt, 'getopt')
741+ orig_name = 'getopt'
742+
743+ def _sneaky(*args, **kw):
744+ optlist, unparsed_args = orig_getopt(*args, **kw)
745+ sneaky_unparsed_args['value'] = unparsed_args
746+ return optlist, unparsed_args
747+
748+ try:
749+ setattr(getopt, orig_name, _sneaky)
750+ args = gflags.FlagValues.__call__(self, argv)
751+ except gflags.UnrecognizedFlagError:
752+ # Undefined args were found, for now we don't care so just
753+ # act like everything went well
754+ # (these three lines are copied pretty much verbatim from the end
755+ # of the __call__ function we are wrapping)
756+ unparsed_args = sneaky_unparsed_args['value']
757+ if unparsed_args:
758+ if self.IsGnuGetOpt():
759+ args = argv[:1] + unparsed_args
760+ else:
761+ args = argv[:1] + original_argv[-len(unparsed_args):]
762+ else:
763+ args = argv[:1]
764+ finally:
765+ setattr(getopt, orig_name, orig_getopt)
766+
767+ # Store the arguments for later, we'll need them for new flags
768+ # added at runtime
769+ self.__dict__['__stored_argv'] = original_argv
770+ self.__dict__['__was_already_parsed'] = True
771+ self.ClearDirty()
772+ return args
773+
774+ def SetDirty(self, name):
775+ """Mark a flag as dirty so that accessing it will case a reparse."""
776+ self.__dict__['__dirty'].append(name)
777+
778+ def IsDirty(self, name):
779+ return name in self.__dict__['__dirty']
780+
781+ def ClearDirty(self):
782+ self.__dict__['__is_dirty'] = []
783+
784+ def WasAlreadyParsed(self):
785+ return self.__dict__['__was_already_parsed']
786+
787+ def ParseNewFlags(self):
788+ if '__stored_argv' not in self.__dict__:
789+ return
790+ new_flags = FlagValues()
791+ for k in self.__dict__['__dirty']:
792+ new_flags[k] = gflags.FlagValues.__getitem__(self, k)
793+
794+ new_flags(self.__dict__['__stored_argv'])
795+ for k in self.__dict__['__dirty']:
796+ setattr(self, k, getattr(new_flags, k))
797+ self.ClearDirty()
798+
799+ def __setitem__(self, name, flag):
800+ gflags.FlagValues.__setitem__(self, name, flag)
801+ if self.WasAlreadyParsed():
802+ self.SetDirty(name)
803+
804+ def __getitem__(self, name):
805+ if self.IsDirty(name):
806+ self.ParseNewFlags()
807+ return gflags.FlagValues.__getitem__(self, name)
808+
809+ def __getattr__(self, name):
810+ if self.IsDirty(name):
811+ self.ParseNewFlags()
812+ return gflags.FlagValues.__getattr__(self, name)
813+
814+
815+FLAGS = FlagValues()
816+
817+
818+def _wrapper(func):
819+ def _wrapped(*args, **kw):
820+ kw.setdefault('flag_values', FLAGS)
821+ func(*args, **kw)
822+ _wrapped.func_name = func.func_name
823+ return _wrapped
824+
825+
826+DEFINE = _wrapper(gflags.DEFINE)
827+DEFINE_string = _wrapper(gflags.DEFINE_string)
828+DEFINE_integer = _wrapper(gflags.DEFINE_integer)
829+DEFINE_bool = _wrapper(gflags.DEFINE_bool)
830+DEFINE_boolean = _wrapper(gflags.DEFINE_boolean)
831+DEFINE_float = _wrapper(gflags.DEFINE_float)
832+DEFINE_enum = _wrapper(gflags.DEFINE_enum)
833+DEFINE_list = _wrapper(gflags.DEFINE_list)
834+DEFINE_spaceseplist = _wrapper(gflags.DEFINE_spaceseplist)
835+DEFINE_multistring = _wrapper(gflags.DEFINE_multistring)
836+DEFINE_multi_int = _wrapper(gflags.DEFINE_multi_int)
837+
838+
839+def DECLARE(name, module_string, flag_values=FLAGS):
840+ if module_string not in sys.modules:
841+ __import__(module_string, globals(), locals())
842+ if name not in flag_values:
843+ raise gflags.UnrecognizedFlag(
844+ "%s not defined by %s" % (name, module_string))
845+
846+
847+# __GLOBAL FLAGS ONLY__
848+# Define any app-specific flags in their own files, docs at:
849+# http://code.google.com/p/python-gflags/source/browse/trunk/gflags.py#39
850+
851+# TODO(sirp): move this out to an application specific setting when we create
852+# Nova/Glance common library
853+DEFINE_string('sql_connection',
854+ 'sqlite:///%s/glance.sqlite' % os.path.abspath("./"),
855+ 'connection string for sql database')
856+DEFINE_bool('verbose', False, 'show debug output')
857
858=== added file 'glance/common/server.py'
859--- glance/common/server.py 1970-01-01 00:00:00 +0000
860+++ glance/common/server.py 2010-09-29 20:56:47 +0000
861@@ -0,0 +1,144 @@
862+# vim: tabstop=4 shiftwidth=4 softtabstop=4
863+
864+# Copyright 2010 United States Government as represented by the
865+# Administrator of the National Aeronautics and Space Administration.
866+# All Rights Reserved.
867+#
868+# Licensed under the Apache License, Version 2.0 (the "License"); you may
869+# not use this file except in compliance with the License. You may obtain
870+# a copy of the License at
871+#
872+# http://www.apache.org/licenses/LICENSE-2.0
873+#
874+# Unless required by applicable law or agreed to in writing, software
875+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
876+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
877+# License for the specific language governing permissions and limitations
878+# under the License.
879+
880+"""
881+Base functionality for nova daemons - gradually being replaced with twistd.py.
882+"""
883+
884+import daemon
885+from daemon import pidlockfile
886+import logging
887+import logging.handlers
888+import os
889+import signal
890+import sys
891+import time
892+
893+from glance.common import flags
894+
895+
896+FLAGS = flags.FLAGS
897+flags.DEFINE_bool('daemonize', False, 'daemonize this process')
898+# NOTE(termie): right now I am defaulting to using syslog when we daemonize
899+# it may be better to do something else -shrug-
900+# NOTE(Devin): I think we should let each process have its own log file
901+# and put it in /var/logs/nova/(appname).log
902+# This makes debugging much easier and cuts down on sys log
903+# clutter.
904+flags.DEFINE_bool('use_syslog', True, 'output to syslog when daemonizing')
905+flags.DEFINE_string('logfile', None, 'log file to output to')
906+flags.DEFINE_string('pidfile', None, 'pid file to output to')
907+flags.DEFINE_string('working_directory', './', 'working directory...')
908+flags.DEFINE_integer('uid', os.getuid(), 'uid under which to run')
909+flags.DEFINE_integer('gid', os.getgid(), 'gid under which to run')
910+
911+
912+def stop(pidfile):
913+ """
914+ Stop the daemon
915+ """
916+ # Get the pid from the pidfile
917+ try:
918+ pid = int(open(pidfile,'r').read().strip())
919+ except IOError:
920+ message = "pidfile %s does not exist. Daemon not running?\n"
921+ sys.stderr.write(message % pidfile)
922+ return # not an error in a restart
923+
924+ # Try killing the daemon process
925+ try:
926+ while 1:
927+ os.kill(pid, signal.SIGTERM)
928+ time.sleep(0.1)
929+ except OSError, err:
930+ err = str(err)
931+ if err.find("No such process") > 0:
932+ if os.path.exists(pidfile):
933+ os.remove(pidfile)
934+ else:
935+ print str(err)
936+ sys.exit(1)
937+
938+
939+def serve(name, main):
940+ """Controller for server"""
941+ argv = FLAGS(sys.argv)
942+
943+ if not FLAGS.pidfile:
944+ FLAGS.pidfile = '%s.pid' % name
945+
946+ logging.debug("Full set of FLAGS: \n\n\n")
947+ for flag in FLAGS:
948+ logging.debug("%s : %s", flag, FLAGS.get(flag, None))
949+
950+ action = 'start'
951+ if len(argv) > 1:
952+ action = argv.pop()
953+
954+ if action == 'stop':
955+ stop(FLAGS.pidfile)
956+ sys.exit()
957+ elif action == 'restart':
958+ stop(FLAGS.pidfile)
959+ elif action == 'start':
960+ pass
961+ else:
962+ print 'usage: %s [options] [start|stop|restart]' % argv[0]
963+ sys.exit(1)
964+ daemonize(argv, name, main)
965+
966+
967+def daemonize(args, name, main):
968+ """Does the work of daemonizing the process"""
969+ logging.getLogger('amqplib').setLevel(logging.WARN)
970+ if FLAGS.daemonize:
971+ logger = logging.getLogger()
972+ formatter = logging.Formatter(
973+ name + '(%(name)s): %(levelname)s %(message)s')
974+ if FLAGS.use_syslog and not FLAGS.logfile:
975+ syslog = logging.handlers.SysLogHandler(address='/dev/log')
976+ syslog.setFormatter(formatter)
977+ logger.addHandler(syslog)
978+ else:
979+ if not FLAGS.logfile:
980+ FLAGS.logfile = '%s.log' % name
981+ logfile = logging.FileHandler(FLAGS.logfile)
982+ logfile.setFormatter(formatter)
983+ logger.addHandler(logfile)
984+ stdin, stdout, stderr = None, None, None
985+ else:
986+ stdin, stdout, stderr = sys.stdin, sys.stdout, sys.stderr
987+
988+ if FLAGS.verbose:
989+ logging.getLogger().setLevel(logging.DEBUG)
990+ else:
991+ logging.getLogger().setLevel(logging.WARNING)
992+
993+ with daemon.DaemonContext(
994+ detach_process=FLAGS.daemonize,
995+ working_directory=FLAGS.working_directory,
996+ pidfile=pidlockfile.TimeoutPIDLockFile(FLAGS.pidfile,
997+ acquire_timeout=1,
998+ threaded=False),
999+ stdin=stdin,
1000+ stdout=stdout,
1001+ stderr=stderr,
1002+ uid=FLAGS.uid,
1003+ gid=FLAGS.gid
1004+ ):
1005+ main(args)
1006
1007=== added file 'glance/common/utils.py'
1008--- glance/common/utils.py 1970-01-01 00:00:00 +0000
1009+++ glance/common/utils.py 2010-09-29 20:56:47 +0000
1010@@ -0,0 +1,204 @@
1011+# vim: tabstop=4 shiftwidth=4 softtabstop=4
1012+
1013+# Copyright 2010 United States Government as represented by the
1014+# Administrator of the National Aeronautics and Space Administration.
1015+# All Rights Reserved.
1016+#
1017+# Licensed under the Apache License, Version 2.0 (the "License"); you may
1018+# not use this file except in compliance with the License. You may obtain
1019+# a copy of the License at
1020+#
1021+# http://www.apache.org/licenses/LICENSE-2.0
1022+#
1023+# Unless required by applicable law or agreed to in writing, software
1024+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
1025+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
1026+# License for the specific language governing permissions and limitations
1027+# under the License.
1028+
1029+"""
1030+System-level utilities and helper functions.
1031+"""
1032+
1033+import datetime
1034+import inspect
1035+import logging
1036+import os
1037+import random
1038+import subprocess
1039+import socket
1040+import sys
1041+
1042+from twisted.internet.threads import deferToThread
1043+
1044+from glance.common import exception
1045+from glance.common import flags
1046+from glance.common.exception import ProcessExecutionError
1047+
1048+
1049+FLAGS = flags.FLAGS
1050+TIME_FORMAT = "%Y-%m-%dT%H:%M:%SZ"
1051+
1052+def import_class(import_str):
1053+ """Returns a class from a string including module and class"""
1054+ mod_str, _sep, class_str = import_str.rpartition('.')
1055+ try:
1056+ __import__(mod_str)
1057+ return getattr(sys.modules[mod_str], class_str)
1058+ except (ImportError, ValueError, AttributeError):
1059+ raise exception.NotFound('Class %s cannot be found' % class_str)
1060+
1061+def import_object(import_str):
1062+ """Returns an object including a module or module and class"""
1063+ try:
1064+ __import__(import_str)
1065+ return sys.modules[import_str]
1066+ except ImportError:
1067+ cls = import_class(import_str)
1068+ return cls()
1069+
1070+def fetchfile(url, target):
1071+ logging.debug("Fetching %s" % url)
1072+# c = pycurl.Curl()
1073+# fp = open(target, "wb")
1074+# c.setopt(c.URL, url)
1075+# c.setopt(c.WRITEDATA, fp)
1076+# c.perform()
1077+# c.close()
1078+# fp.close()
1079+ execute("curl --fail %s -o %s" % (url, target))
1080+
1081+def execute(cmd, process_input=None, addl_env=None, check_exit_code=True):
1082+ logging.debug("Running cmd: %s", cmd)
1083+ env = os.environ.copy()
1084+ if addl_env:
1085+ env.update(addl_env)
1086+ obj = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE,
1087+ stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
1088+ result = None
1089+ if process_input != None:
1090+ result = obj.communicate(process_input)
1091+ else:
1092+ result = obj.communicate()
1093+ obj.stdin.close()
1094+ if obj.returncode:
1095+ logging.debug("Result was %s" % (obj.returncode))
1096+ if check_exit_code and obj.returncode <> 0:
1097+ (stdout, stderr) = result
1098+ raise ProcessExecutionError(exit_code=obj.returncode,
1099+ stdout=stdout,
1100+ stderr=stderr,
1101+ cmd=cmd)
1102+ return result
1103+
1104+
1105+def abspath(s):
1106+ return os.path.join(os.path.dirname(__file__), s)
1107+
1108+
1109+# TODO(sirp): when/if utils is extracted to common library, we should remove
1110+# the argument's default.
1111+#def default_flagfile(filename='nova.conf'):
1112+def default_flagfile(filename='glance.conf'):
1113+ for arg in sys.argv:
1114+ if arg.find('flagfile') != -1:
1115+ break
1116+ else:
1117+ if not os.path.isabs(filename):
1118+ # turn relative filename into an absolute path
1119+ script_dir = os.path.dirname(inspect.stack()[-1][1])
1120+ filename = os.path.abspath(os.path.join(script_dir, filename))
1121+ if os.path.exists(filename):
1122+ sys.argv = sys.argv[:1] + ['--flagfile=%s' % filename] + sys.argv[1:]
1123+
1124+
1125+def debug(arg):
1126+ logging.debug('debug in callback: %s', arg)
1127+ return arg
1128+
1129+
1130+def runthis(prompt, cmd, check_exit_code = True):
1131+ logging.debug("Running %s" % (cmd))
1132+ exit_code = subprocess.call(cmd.split(" "))
1133+ logging.debug(prompt % (exit_code))
1134+ if check_exit_code and exit_code <> 0:
1135+ raise ProcessExecutionError(exit_code=exit_code,
1136+ stdout=None,
1137+ stderr=None,
1138+ cmd=cmd)
1139+
1140+
1141+def generate_uid(topic, size=8):
1142+ return '%s-%s' % (topic, ''.join([random.choice('01234567890abcdefghijklmnopqrstuvwxyz') for x in xrange(size)]))
1143+
1144+
1145+def generate_mac():
1146+ mac = [0x02, 0x16, 0x3e, random.randint(0x00, 0x7f),
1147+ random.randint(0x00, 0xff), random.randint(0x00, 0xff)
1148+ ]
1149+ return ':'.join(map(lambda x: "%02x" % x, mac))
1150+
1151+
1152+def last_octet(address):
1153+ return int(address.split(".")[-1])
1154+
1155+
1156+def get_my_ip():
1157+ """Returns the actual ip of the local machine."""
1158+ if getattr(FLAGS, 'fake_tests', None):
1159+ return '127.0.0.1'
1160+ try:
1161+ csock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
1162+ csock.connect(('www.google.com', 80))
1163+ (addr, port) = csock.getsockname()
1164+ csock.close()
1165+ return addr
1166+ except socket.gaierror as ex:
1167+ logging.warn("Couldn't get IP, using 127.0.0.1 %s", ex)
1168+ return "127.0.0.1"
1169+
1170+
1171+def isotime(at=None):
1172+ if not at:
1173+ at = datetime.datetime.utcnow()
1174+ return at.strftime(TIME_FORMAT)
1175+
1176+
1177+def parse_isotime(timestr):
1178+ return datetime.datetime.strptime(timestr, TIME_FORMAT)
1179+
1180+
1181+class LazyPluggable(object):
1182+ """A pluggable backend loaded lazily based on some value."""
1183+
1184+ def __init__(self, pivot, **backends):
1185+ self.__backends = backends
1186+ self.__pivot = pivot
1187+ self.__backend = None
1188+
1189+ def __get_backend(self):
1190+ if not self.__backend:
1191+ backend_name = self.__pivot.value
1192+ if backend_name not in self.__backends:
1193+ raise exception.Error('Invalid backend: %s' % backend_name)
1194+
1195+ backend = self.__backends[backend_name]
1196+ if type(backend) == type(tuple()):
1197+ name = backend[0]
1198+ fromlist = backend[1]
1199+ else:
1200+ name = backend
1201+ fromlist = backend
1202+
1203+ self.__backend = __import__(name, None, None, fromlist)
1204+ logging.info('backend %s', self.__backend)
1205+ return self.__backend
1206+
1207+ def __getattr__(self, key):
1208+ backend = self.__get_backend()
1209+ return getattr(backend, key)
1210+
1211+def deferredToThread(f):
1212+ def g(*args, **kwargs):
1213+ return deferToThread(f, *args, **kwargs)
1214+ return g
1215
1216=== added file 'glance/common/wsgi.py'
1217--- glance/common/wsgi.py 1970-01-01 00:00:00 +0000
1218+++ glance/common/wsgi.py 2010-09-29 20:56:47 +0000
1219@@ -0,0 +1,298 @@
1220+# vim: tabstop=4 shiftwidth=4 softtabstop=4
1221+
1222+# Copyright 2010 United States Government as represented by the
1223+# Administrator of the National Aeronautics and Space Administration.
1224+# Copyright 2010 OpenStack LLC.
1225+# All Rights Reserved.
1226+#
1227+# Licensed under the Apache License, Version 2.0 (the "License"); you may
1228+# not use this file except in compliance with the License. You may obtain
1229+# a copy of the License at
1230+#
1231+# http://www.apache.org/licenses/LICENSE-2.0
1232+#
1233+# Unless required by applicable law or agreed to in writing, software
1234+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
1235+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
1236+# License for the specific language governing permissions and limitations
1237+# under the License.
1238+
1239+"""
1240+Utility methods for working with WSGI servers
1241+"""
1242+
1243+import logging
1244+import sys
1245+
1246+import eventlet
1247+import eventlet.wsgi
1248+eventlet.patcher.monkey_patch(all=False, socket=True)
1249+import routes
1250+import routes.middleware
1251+import webob.dec
1252+import webob.exc
1253+
1254+
1255+logging.getLogger("routes.middleware").addHandler(logging.StreamHandler())
1256+
1257+
1258+def run_server(application, port):
1259+ """Run a WSGI server with the given application."""
1260+ sock = eventlet.listen(('0.0.0.0', port))
1261+ eventlet.wsgi.server(sock, application)
1262+
1263+
1264+class Application(object):
1265+# TODO(gundlach): I think we should toss this class, now that it has no
1266+# purpose.
1267+ """Base WSGI application wrapper. Subclasses need to implement __call__."""
1268+
1269+ def __call__(self, environ, start_response):
1270+ r"""Subclasses will probably want to implement __call__ like this:
1271+
1272+ @webob.dec.wsgify
1273+ def __call__(self, req):
1274+ # Any of the following objects work as responses:
1275+
1276+ # Option 1: simple string
1277+ res = 'message\n'
1278+
1279+ # Option 2: a nicely formatted HTTP exception page
1280+ res = exc.HTTPForbidden(detail='Nice try')
1281+
1282+ # Option 3: a webob Response object (in case you need to play with
1283+ # headers, or you want to be treated like an iterable, or or or)
1284+ res = Response();
1285+ res.app_iter = open('somefile')
1286+
1287+ # Option 4: any wsgi app to be run next
1288+ res = self.application
1289+
1290+ # Option 5: you can get a Response object for a wsgi app, too, to
1291+ # play with headers etc
1292+ res = req.get_response(self.application)
1293+
1294+ # You can then just return your response...
1295+ return res
1296+ # ... or set req.response and return None.
1297+ req.response = res
1298+
1299+ See the end of http://pythonpaste.org/webob/modules/dec.html
1300+ for more info.
1301+ """
1302+ raise NotImplementedError("You must implement __call__")
1303+
1304+
1305+class Middleware(Application):
1306+ """
1307+ Base WSGI middleware wrapper. These classes require an application to be
1308+ initialized that will be called next. By default the middleware will
1309+ simply call its wrapped app, or you can override __call__ to customize its
1310+ behavior.
1311+ """
1312+
1313+ def __init__(self, application): # pylint: disable-msg=W0231
1314+ self.application = application
1315+
1316+ @webob.dec.wsgify
1317+ def __call__(self, req): # pylint: disable-msg=W0221
1318+ """Override to implement middleware behavior."""
1319+ return self.application
1320+
1321+
1322+class Debug(Middleware):
1323+ """Helper class that can be inserted into any WSGI application chain
1324+ to get information about the request and response."""
1325+
1326+ @webob.dec.wsgify
1327+ def __call__(self, req):
1328+ print ("*" * 40) + " REQUEST ENVIRON"
1329+ for key, value in req.environ.items():
1330+ print key, "=", value
1331+ print
1332+ resp = req.get_response(self.application)
1333+
1334+ print ("*" * 40) + " RESPONSE HEADERS"
1335+ for (key, value) in resp.headers.iteritems():
1336+ print key, "=", value
1337+ print
1338+
1339+ resp.app_iter = self.print_generator(resp.app_iter)
1340+
1341+ return resp
1342+
1343+ @staticmethod
1344+ def print_generator(app_iter):
1345+ """
1346+ Iterator that prints the contents of a wrapper string iterator
1347+ when iterated.
1348+ """
1349+ print ("*" * 40) + " BODY"
1350+ for part in app_iter:
1351+ sys.stdout.write(part)
1352+ sys.stdout.flush()
1353+ yield part
1354+ print
1355+
1356+
1357+class Router(object):
1358+ """
1359+ WSGI middleware that maps incoming requests to WSGI apps.
1360+ """
1361+
1362+ def __init__(self, mapper):
1363+ """
1364+ Create a router for the given routes.Mapper.
1365+
1366+ Each route in `mapper` must specify a 'controller', which is a
1367+ WSGI app to call. You'll probably want to specify an 'action' as
1368+ well and have your controller be a wsgi.Controller, who will route
1369+ the request to the action method.
1370+
1371+ Examples:
1372+ mapper = routes.Mapper()
1373+ sc = ServerController()
1374+
1375+ # Explicit mapping of one route to a controller+action
1376+ mapper.connect(None, "/svrlist", controller=sc, action="list")
1377+
1378+ # Actions are all implicitly defined
1379+ mapper.resource("server", "servers", controller=sc)
1380+
1381+ # Pointing to an arbitrary WSGI app. You can specify the
1382+ # {path_info:.*} parameter so the target app can be handed just that
1383+ # section of the URL.
1384+ mapper.connect(None, "/v1.0/{path_info:.*}", controller=BlogApp())
1385+ """
1386+ self.map = mapper
1387+ self._router = routes.middleware.RoutesMiddleware(self._dispatch,
1388+ self.map)
1389+
1390+ @webob.dec.wsgify
1391+ def __call__(self, req):
1392+ """
1393+ Route the incoming request to a controller based on self.map.
1394+ If no match, return a 404.
1395+ """
1396+ return self._router
1397+
1398+ @staticmethod
1399+ @webob.dec.wsgify
1400+ def _dispatch(req):
1401+ """
1402+ Called by self._router after matching the incoming request to a route
1403+ and putting the information into req.environ. Either returns 404
1404+ or the routed WSGI app's response.
1405+ """
1406+ match = req.environ['wsgiorg.routing_args'][1]
1407+ if not match:
1408+ return webob.exc.HTTPNotFound()
1409+ app = match['controller']
1410+ return app
1411+
1412+
1413+class Controller(object):
1414+ """
1415+ WSGI app that reads routing information supplied by RoutesMiddleware
1416+ and calls the requested action method upon itself. All action methods
1417+ must, in addition to their normal parameters, accept a 'req' argument
1418+ which is the incoming webob.Request. They raise a webob.exc exception,
1419+ or return a dict which will be serialized by requested content type.
1420+ """
1421+
1422+ @webob.dec.wsgify
1423+ def __call__(self, req):
1424+ """
1425+ Call the method specified in req.environ by RoutesMiddleware.
1426+ """
1427+ arg_dict = req.environ['wsgiorg.routing_args'][1]
1428+ action = arg_dict['action']
1429+ method = getattr(self, action)
1430+ del arg_dict['controller']
1431+ del arg_dict['action']
1432+ arg_dict['req'] = req
1433+ result = method(**arg_dict)
1434+ if type(result) is dict:
1435+ return self._serialize(result, req)
1436+ else:
1437+ return result
1438+
1439+ def _serialize(self, data, request):
1440+ """
1441+ Serialize the given dict to the response type requested in request.
1442+ Uses self._serialization_metadata if it exists, which is a dict mapping
1443+ MIME types to information needed to serialize to that type.
1444+ """
1445+ # FIXME(sirp): type(self) should just be `self`
1446+ _metadata = getattr(type(self), "_serialization_metadata", {})
1447+ serializer = Serializer(request.environ, _metadata)
1448+ return serializer.to_content_type(data)
1449+
1450+
1451+class Serializer(object):
1452+ """
1453+ Serializes a dictionary to a Content Type specified by a WSGI environment.
1454+ """
1455+
1456+ def __init__(self, environ, metadata=None):
1457+ """
1458+ Create a serializer based on the given WSGI environment.
1459+ 'metadata' is an optional dict mapping MIME types to information
1460+ needed to serialize a dictionary to that type.
1461+ """
1462+ self.environ = environ
1463+ self.metadata = metadata or {}
1464+ self._methods = {
1465+ 'application/json': self._to_json,
1466+ 'application/xml': self._to_xml}
1467+
1468+ def to_content_type(self, data):
1469+ """
1470+ Serialize a dictionary into a string. The format of the string
1471+ will be decided based on the Content Type requested in self.environ:
1472+ by Accept: header, or by URL suffix.
1473+ """
1474+ # FIXME(sirp): for now, supporting json only
1475+ #mimetype = 'application/xml'
1476+ mimetype = 'application/json'
1477+ # TODO(gundlach): determine mimetype from request
1478+ return self._methods.get(mimetype, repr)(data)
1479+
1480+ def _to_json(self, data):
1481+ import json
1482+ return json.dumps(data)
1483+
1484+ def _to_xml(self, data):
1485+ metadata = self.metadata.get('application/xml', {})
1486+ # We expect data to contain a single key which is the XML root.
1487+ root_key = data.keys()[0]
1488+ from xml.dom import minidom
1489+ doc = minidom.Document()
1490+ node = self._to_xml_node(doc, metadata, root_key, data[root_key])
1491+ return node.toprettyxml(indent=' ')
1492+
1493+ def _to_xml_node(self, doc, metadata, nodename, data):
1494+ """Recursive method to convert data members to XML nodes."""
1495+ result = doc.createElement(nodename)
1496+ if type(data) is list:
1497+ singular = metadata.get('plurals', {}).get(nodename, None)
1498+ if singular is None:
1499+ if nodename.endswith('s'):
1500+ singular = nodename[:-1]
1501+ else:
1502+ singular = 'item'
1503+ for item in data:
1504+ node = self._to_xml_node(doc, metadata, singular, item)
1505+ result.appendChild(node)
1506+ elif type(data) is dict:
1507+ attrs = metadata.get('attributes', {}).get(nodename, {})
1508+ for k, v in data.items():
1509+ if k in attrs:
1510+ result.setAttribute(k, str(v))
1511+ else:
1512+ node = self._to_xml_node(doc, metadata, k, v)
1513+ result.appendChild(node)
1514+ else: # atom
1515+ node = doc.createTextNode(str(data))
1516+ result.appendChild(node)
1517+ return result
1518
1519=== modified file 'glance/parallax/__init__.py'
1520--- glance/parallax/__init__.py 2010-08-24 04:32:57 +0000
1521+++ glance/parallax/__init__.py 2010-09-29 20:56:47 +0000
1522@@ -1,16 +1,21 @@
1523-class ImageRegistry(object):
1524- """
1525- {
1526- "uuid": uuid,
1527- "slug": slug,
1528- "name": name,
1529- "objects": [
1530- { "uri": obj1_uri },
1531- { "uri": obj2_uri }
1532- ]
1533- }
1534- """
1535- def __init__(self):
1536- pass
1537-
1538-
1539+# vim: tabstop=4 shiftwidth=4 softtabstop=4
1540+
1541+# Copyright 2010 United States Government as represented by the
1542+# Administrator of the National Aeronautics and Space Administration.
1543+# All Rights Reserved.
1544+#
1545+# Licensed under the Apache License, Version 2.0 (the "License"); you may
1546+# not use this file except in compliance with the License. You may obtain
1547+# a copy of the License at
1548+#
1549+# http://www.apache.org/licenses/LICENSE-2.0
1550+#
1551+# Unless required by applicable law or agreed to in writing, software
1552+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
1553+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
1554+# License for the specific language governing permissions and limitations
1555+# under the License.
1556+
1557+"""
1558+Parallax API
1559+"""
1560
1561=== added directory 'glance/parallax/api'
1562=== added file 'glance/parallax/api/__init__.py'
1563--- glance/parallax/api/__init__.py 1970-01-01 00:00:00 +0000
1564+++ glance/parallax/api/__init__.py 2010-09-29 20:56:47 +0000
1565@@ -0,0 +1,49 @@
1566+# vim: tabstop=4 shiftwidth=4 softtabstop=4
1567+
1568+# Copyright 2010 United States Government as represented by the
1569+# Administrator of the National Aeronautics and Space Administration.
1570+# All Rights Reserved.
1571+#
1572+# Licensed under the Apache License, Version 2.0 (the "License"); you may
1573+# not use this file except in compliance with the License. You may obtain
1574+# a copy of the License at
1575+#
1576+# http://www.apache.org/licenses/LICENSE-2.0
1577+#
1578+# Unless required by applicable law or agreed to in writing, software
1579+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
1580+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
1581+# License for the specific language governing permissions and limitations
1582+# under the License.
1583+
1584+"""
1585+Parallax API controllers.
1586+"""
1587+
1588+import json
1589+import time
1590+
1591+import routes
1592+import webob.dec
1593+import webob.exc
1594+import webob
1595+
1596+from glance.common import flags
1597+from glance.common import utils
1598+from glance.common import wsgi
1599+from glance.parallax.api import images
1600+
1601+
1602+FLAGS = flags.FLAGS
1603+
1604+
1605+class API(wsgi.Router):
1606+ """WSGI entry point for all Parallax requests."""
1607+
1608+ def __init__(self):
1609+ # TODO(sirp): should we add back the middleware for parallax
1610+ mapper = routes.Mapper()
1611+ mapper.resource("image", "images", controller=images.Controller(),
1612+ collection={'detail': 'GET'})
1613+ super(API, self).__init__(mapper)
1614+
1615
1616=== added file 'glance/parallax/api/images.py'
1617--- glance/parallax/api/images.py 1970-01-01 00:00:00 +0000
1618+++ glance/parallax/api/images.py 2010-09-29 20:56:47 +0000
1619@@ -0,0 +1,82 @@
1620+# vim: tabstop=4 shiftwidth=4 softtabstop=4
1621+
1622+# Copyright 2010 OpenStack LLC.
1623+# All Rights Reserved.
1624+#
1625+# Licensed under the Apache License, Version 2.0 (the "License"); you may
1626+# not use this file except in compliance with the License. You may obtain
1627+# a copy of the License at
1628+#
1629+# http://www.apache.org/licenses/LICENSE-2.0
1630+#
1631+# Unless required by applicable law or agreed to in writing, software
1632+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
1633+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
1634+# License for the specific language governing permissions and limitations
1635+# under the License.
1636+
1637+"""
1638+Parllax Image controller
1639+"""
1640+
1641+
1642+from glance.common import wsgi
1643+from glance.common import db
1644+from glance.common import exception
1645+from webob import exc
1646+
1647+
1648+class Controller(wsgi.Controller):
1649+ """Image Controller """
1650+
1651+ # TODO(sirp): this is not currently used, but should eventually
1652+ # incorporate it
1653+ _serialization_metadata = {
1654+ 'application/xml': {
1655+ "attributes": {
1656+ "image": [ "id", "name", "updated", "created", "status",
1657+ "serverId", "progress" ]
1658+ }
1659+ }
1660+ }
1661+
1662+ def index(self, req):
1663+ """Index is not currently supported """
1664+ raise exc.HTTPNotImplemented()
1665+
1666+ def detail(self, req):
1667+ """Detail is not currently supported """
1668+ raise exc.HTTPNotImplemented()
1669+
1670+ def show(self, req, id):
1671+ """Return data about the given image id."""
1672+ try:
1673+ image = db.image_get(None, id)
1674+ except exception.NotFound:
1675+ raise exc.HTTPNotFound()
1676+
1677+ file_dicts = [dict(location=f.location, size=f.size)
1678+ for f in image.files]
1679+
1680+ metadata_dicts = [dict(key=m.key, value=m.value)
1681+ for m in image.metadata]
1682+
1683+ return dict(id=image.id,
1684+ name=image.name,
1685+ state=image.state,
1686+ public=image.public,
1687+ files=file_dicts,
1688+ metadata=metadata_dicts)
1689+
1690+ def delete(self, req, id):
1691+ """Delete is not currently supported """
1692+ raise exc.HTTPNotImplemented()
1693+
1694+ def create(self, req):
1695+ """Create is not currently supported """
1696+ raise exc.HTTPNotImplemented()
1697+
1698+ def update(self, req, id):
1699+ """Update is not currently supported """
1700+ raise exc.HTTPNotImplemented()
1701+
1702
1703=== added file 'tests/test_data.py'
1704--- tests/test_data.py 1970-01-01 00:00:00 +0000
1705+++ tests/test_data.py 2010-09-29 20:56:47 +0000
1706@@ -0,0 +1,49 @@
1707+# vim: tabstop=4 shiftwidth=4 softtabstop=4
1708+
1709+# Copyright 2010 OpenStack, LLC
1710+# All Rights Reserved.
1711+#
1712+# Licensed under the Apache License, Version 2.0 (the "License"); you may
1713+# not use this file except in compliance with the License. You may obtain
1714+# a copy of the License at
1715+#
1716+# http://www.apache.org/licenses/LICENSE-2.0
1717+#
1718+# Unless required by applicable law or agreed to in writing, software
1719+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
1720+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
1721+# License for the specific language governing permissions and limitations
1722+# under the License.
1723+
1724+from glance.common.db import api
1725+
1726+
1727+def make_fake_image():
1728+ """Create a fake image record """
1729+ image = api.image_create(
1730+ None,
1731+ dict(name="Test Image",
1732+ state="available",
1733+ public=True,
1734+ image_type="tarball"))
1735+
1736+ api.image_file_create(
1737+ None,
1738+ dict(image_id=image.id,
1739+ location="swift://myacct/mycontainer/obj.tar.gz.0",
1740+ size=101))
1741+ api.image_file_create(
1742+ None,
1743+ dict(image_id=image.id,
1744+ location="swift://myacct/mycontainer/obj.tar.gz.1",
1745+ size=101))
1746+
1747+ api.image_metadatum_create(
1748+ None,
1749+ dict(image_id=image.id,
1750+ key="testkey",
1751+ value="testvalue"))
1752+
1753+
1754+if __name__ == "__main__":
1755+ make_fake_image()

Subscribers

People subscribed via source and target branches