Merge lp:~rconradharris/glance/registry_db_migration into lp:~glance-coresec/glance/cactus-trunk

[Rick Harris]

Adds sqlalchemy migrations.

Also adds a glance-manage utility for managing migrations.

Potentially glance-manage (and glance-upload) could be merged into glance-admin when that lands.

[Jay Pipes (jaypipes)]

Very nice work, Rick. Any chance we can figure out a way of testing this?

review: Needs Information

[Rick Harris (rconradharris)]

I can't think of any meaningful tests as this is just a CLI wrapper around sqlalchemy-migrate (which itself has been tested quite a bit ;-).

[Jay Pipes (jaypipes)]

> I can't think of any meaningful tests as this is just a CLI wrapper around
> sqlalchemy-migrate (which itself has been tested quite a bit ;-).

What about something as simple as this?

http://pastie.org/1522390

[Rick Harris (rconradharris)]

Honestly, doesn't strike me as super-useful; but that said, it's better to err on the side of caution, so I'll defer to your judgement here.

Test-mode engaged...

[Jay Pipes (jaypipes)]

OK, so I had an additional thought about where the database migration code should go. Since the database migration is only for the reference implementation Glance registry server (bin/glance-registry), I'm thinking that the bin/glance-manage program should be renamed bin/glance-registry-manage. While I understand your comment about having the glance-admin program merge with the glance-manage program in this patch, the two do different things. The database migration commands are particular only to the reference implementation registry server, and not to Glance as a whole, so I think the tool name should reflect that...thoughts?

Note that I think glance-upload can and should be merged into the glance-admin program, just not these specific registry database commands.

[Jay Pipes (jaypipes)]

OK, let's get this merged...we can always address the binary naming stuff later... I want to push on to api-image-format blueprint..

review: Approve

[Matt Dietz (cerberus)]

Seems alright to me.

review: Approve

=== modified file 'MANIFEST.in'
--- MANIFEST.in	2011-01-20 10:13:46 +0000
+++ MANIFEST.in	2011-02-03 00:03:10 +0000
@@ -6,5 +6,6 @@
 include tests/test_data.py
 include tests/utils.py
 include run_tests.py
+include glance/registry/db/migrate_repo/migrate.cfg
 graft doc
 graft tools

=== added file 'bin/glance-manage'
--- bin/glance-manage	1970-01-01 00:00:00 +0000
+++ bin/glance-manage	2011-02-03 00:03:10 +0000
@@ -0,0 +1,130 @@
+#!/usr/bin/env python
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# Copyright 2011 OpenStack LLC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""
+Glance Management Utility
+"""
+
+# FIXME(sirp): When we have glance-admin we can consider merging this into it
+# Perhaps for consistency with Nova, we would then rename glance-admin ->
+# glance-manage (or the other way around)
+
+import optparse
+import os
+import sys
+
+ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+sys.path.append(ROOT_DIR)
+
+from glance import version as glance_version
+from glance.common import config
+from glance.common import exception
+import glance.registry.db
+import glance.registry.db.migration
+
+
+def create_options(parser):
+    """
+    Sets up the CLI and config-file options that may be
+    parsed and program commands.
+
+    :param parser: The option parser
+    """
+    glance.registry.db.add_options(parser)
+    config.add_common_options(parser)
+    config.add_log_options('glance-manage', parser)
+
+
+def do_db_version(options, args):
+    """Print database's current migration level"""
+    print glance.registry.db.migration.db_version(options)
+
+
+def do_upgrade(options, args):
+    """Upgrade the database's migration level"""
+    try:
+        db_version = args[1]
+    except IndexError:
+        db_version = None
+
+    glance.registry.db.migration.upgrade(options, version=db_version)
+
+
+def do_downgrade(options, args):
+    """Downgrade the database's migration level"""
+    try:
+        db_version = args[1]
+    except IndexError:
+        raise exception.MissingArgumentError(
+            "downgrade requires a version argument")
+
+    glance.registry.db.migration.downgrade(options, version=db_version)
+
+
+def do_version_control(options, args):
+    """Place a database under migration control"""
+    glance.registry.db.migration.version_control(options)
+
+
+def do_db_sync(options, args):
+    """Place a database under migration control and upgrade"""
+    try:
+        db_version = args[1]
+    except IndexError:
+        db_version = None
+    glance.registry.db.migration.db_sync(options, version=db_version)
+
+
+def dispatch_cmd(options, args):
+    """Search for do_* cmd in this module and then run it"""
+    cmd = args[0]
+    try:
+        cmd_func = globals()['do_%s' % cmd]
+    except KeyError:
+        sys.exit("ERROR: unrecognized command '%s'" % cmd)
+
+    try:
+        cmd_func(options, args)
+    except exception.Error, e:
+        sys.exit("ERROR: %s" % e)
+
+
+def main():
+    version = '%%prog %s' % glance_version.version_string()
+    usage = "%prog [options] <cmd>"
+    oparser = optparse.OptionParser(usage, version=version)
+    create_options(oparser)
+    (options, args) = config.parse_options(oparser)
+
+    try:
+        config.setup_logging(options)
+    except RuntimeError, e:
+        sys.exit("ERROR: %s" % e)
+
+    if not args:
+        oparser.print_usage()
+        sys.exit(1)
+
+    dispatch_cmd(options, args)
+
+
+if __name__ == '__main__':
+    main()

=== modified file 'bin/glance-upload'
--- bin/glance-upload	2011-01-23 17:18:48 +0000
+++ bin/glance-upload	2011-02-03 00:03:10 +0000
@@ -33,6 +33,9 @@
                      <filename> <name>
 
 """
+
+# FIXME(sirp): This can be merged into glance-admin when that becomes
+# available
 import argparse
 import pprint
 import sys

=== modified file 'doc/source/conf.py'
--- doc/source/conf.py	2011-01-27 20:01:02 +0000
+++ doc/source/conf.py	2011-02-03 00:03:10 +0000
@@ -129,6 +129,8 @@
     ('man/glanceapi', 'glance-api', u'Glance API Server',
      [u'OpenStack'], 1),
     ('man/glanceregistry', 'glance-registry', u'Glance Registry Server',
+     [u'OpenStack'], 1),
+    ('man/glancemanage', 'glance-manage', u'Glance Management Utility',
      [u'OpenStack'], 1)
  ]
 

=== modified file 'doc/source/glanceapi.rst'
--- doc/source/glanceapi.rst	2011-01-26 17:26:54 +0000
+++ doc/source/glanceapi.rst	2011-02-03 00:03:10 +0000
@@ -24,7 +24,7 @@
 Server*.
 
 Assume there is a Glance API server running at the URL
-``http://glance.example.com``. 
+``http://glance.example.com``.
 
 Let's walk through how a user might request information from this server.
 
@@ -116,7 +116,7 @@
   x-image-meta-store            swift
   x-image-meta-created_at       2010-02-03 09:34:01
   x-image-meta-updated_at       2010-02-03 09:34:01
-  x-image-meta-deleted_at       
+  x-image-meta-deleted_at
   x-image-meta-status           available
   x-image-meta-is_public        True
   x-image-meta-property-distro  Ubuntu 10.04 LTS
@@ -126,7 +126,7 @@
   All timestamps returned are in UTC
 
   The `x-image-meta-updated_at` timestamp is the timestamp when an
-  image's metadata was last updated, not its image data, as all 
+  image's metadata was last updated, not its image data, as all
   image data is immutable once stored in Glance
 
   There may be multiple headers that begin with the prefix
@@ -165,7 +165,7 @@
   x-image-meta-store            swift
   x-image-meta-created_at       2010-02-03 09:34:01
   x-image-meta-updated_at       2010-02-03 09:34:01
-  x-image-meta-deleted_at       
+  x-image-meta-deleted_at
   x-image-meta-status           available
   x-image-meta-is_public        True
   x-image-meta-property-distro  Ubuntu 10.04 LTS
@@ -175,7 +175,7 @@
   All timestamps returned are in UTC
 
   The `x-image-meta-updated_at` timestamp is the timestamp when an
-  image's metadata was last updated, not its image data, as all 
+  image's metadata was last updated, not its image data, as all
   image data is immutable once stored in Glance
 
   There may be multiple headers that begin with the prefix
@@ -232,7 +232,7 @@
 
 * ``x-image-meta-id``
 
-  This header is optional. 
+  This header is optional.
 
   When present, Glance will use the supplied identifier for the image.
   If the identifier already exists in that Glance node, then a

=== modified file 'doc/source/man/glanceapi.rst'
--- doc/source/man/glanceapi.rst	2011-01-19 21:38:39 +0000
+++ doc/source/man/glanceapi.rst	2011-02-03 00:03:10 +0000
@@ -48,7 +48,7 @@
         running ``glance-api``
 
 FILES
-========
+=====
 
 None
 

=== added file 'doc/source/man/glancemanage.py'
--- doc/source/man/glancemanage.py	1970-01-01 00:00:00 +0000
+++ doc/source/man/glancemanage.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,54 @@
+=============
+glance-manage
+=============
+
+-------------------------
+Glance Management Utility
+-------------------------
+
+:Author: [e-mail]
+:Date:   2010-11-16
+:Copyright: OpenStack LLC
+:Version: 0.1.2
+:Manual section: 1
+:Manual group: cloud computing
+
+SYNOPSIS
+========
+
+  glance-manage [options]
+
+DESCRIPTION
+===========
+
+glance-manage is a utility for managing and configuring a Glance installation.
+One important use of glance-manage is to setup the database. To do this run::
+
+    glance-manage db_sync
+
+OPTIONS
+=======
+
+  **General options**
+
+  **-v, --verbose**
+        Print more verbose output
+
+  **--sql_connection=CONN_STRING**
+        A proper SQLAlchemy connection string as described
+        `here <http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/connections.html?highlight=engine#sqlalchemy.create_engine>`_
+
+FILES
+=====
+
+None
+
+SEE ALSO
+========
+
+* `OpenStack Glance <http://glance.openstack.org>`__
+
+BUGS
+====
+
+* Glance is sourced in Launchpad so you can view current bugs at `OpenStack Glance <http://glance.openstack.org>`__

=== modified file 'doc/source/man/glanceregistry.rst'
--- doc/source/man/glanceregistry.rst	2011-01-19 21:38:39 +0000
+++ doc/source/man/glanceregistry.rst	2011-02-03 00:03:10 +0000
@@ -43,7 +43,7 @@
         `here <http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/connections.html?highlight=engine#sqlalchemy.create_engine>`_
 
 FILES
-========
+=====
 
 None
 

=== modified file 'glance/common/config.py'
--- glance/common/config.py	2011-02-02 16:40:57 +0000
+++ glance/common/config.py	2011-02-03 00:03:10 +0000
@@ -27,9 +27,11 @@
 import os
 import sys
 
+import glance.common.exception as exception
 
 DEFAULT_LOG_FORMAT = "%(asctime)s %(levelname)8s [%(name)s] %(message)s"
 DEFAULT_LOG_DATE_FORMAT = "%Y-%m-%d %H:%M:%S"
+DEFAULT_LOG_HANDLER = 'stream'
 LOGGING_HANDLER_CHOICES = ['syslog', 'file', 'stream']
 
 
@@ -132,7 +134,8 @@
                           "any other logging options specified. Please see "
                           "the Python logging module documentation for "
                           "details on logging configuration files.")
-    group.add_option('--log-handler', default='stream', metavar="HANDLER",
+    group.add_option('--log-handler', default=DEFAULT_LOG_HANDLER,
+                     metavar="HANDLER",
                      choices=LOGGING_HANDLER_CHOICES,
                      help="What logging handler to use? "
                            "Default: %default")
@@ -159,7 +162,7 @@
     :param options: Mapping of typed option key/values
     """
 
-    if options['log_config']:
+    if options.get('log_config', None):
         # Use a logging configuration file for all settings...
         if os.path.exists(options['log_config']):
             logging.config.fileConfig(options['log_config'])
@@ -179,14 +182,16 @@
         root_logger.setLevel(logging.WARNING)
 
     # Set log configuration from options...
-    formatter = logging.Formatter(options['log_format'],
-                                  options['log_date_format'])
+    log_format = options.get('log_format', DEFAULT_LOG_FORMAT)
+    log_date_format = options.get('log_date_format', DEFAULT_LOG_DATE_FORMAT)
+    formatter = logging.Formatter(log_format, log_date_format)
 
-    if options['log_handler'] == 'syslog':
+    log_handler = options.get('log_handler', DEFAULT_LOG_HANDLER)
+    if log_handler == 'syslog':
         syslog = logging.handlers.SysLogHandler(address='/dev/log')
         syslog.setFormatter(formatter)
         root_logger.addHandler(syslog)
-    elif options['log_handler'] == 'file':
+    elif log_handler == 'file':
         logfile = options['log_file']
         logdir = options['log_dir']
         if logdir:
@@ -195,10 +200,13 @@
         logfile.setFormatter(formatter)
         logfile.setFormatter(formatter)
         root_logger.addHandler(logfile)
-    else:
+    elif log_handler == 'stream':
         handler = logging.StreamHandler(sys.stdout)
         handler.setFormatter(formatter)
         root_logger.addHandler(handler)
+    else:
+        raise exception.BadInputError(
+            "unrecognized log handler '%(log_handler)s'" % locals())
 
     # Log the options used when starting if we're in debug mode...
     if debug:

=== modified file 'glance/common/exception.py'
--- glance/common/exception.py	2011-01-26 17:26:54 +0000
+++ glance/common/exception.py	2011-02-03 00:03:10 +0000
@@ -75,6 +75,14 @@
     pass
 
 
+class MissingArgumentError(Error):
+    pass
+
+
+class DatabaseMigrationError(Error):
+    pass
+
+
 def wrap_exception(f):
     def _wrap(*args, **kw):
         try:

=== added directory 'glance/registry/db/migrate_repo'
=== added file 'glance/registry/db/migrate_repo/README'
--- glance/registry/db/migrate_repo/README	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migrate_repo/README	2011-02-03 00:03:10 +0000
@@ -0,0 +1,4 @@
+This is a database migration repository.
+
+More information at
+http://code.google.com/p/sqlalchemy-migrate/

=== added file 'glance/registry/db/migrate_repo/__init__.py'
--- glance/registry/db/migrate_repo/__init__.py	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migrate_repo/__init__.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,1 @@
+# template repository default module

=== added file 'glance/registry/db/migrate_repo/manage.py'
--- glance/registry/db/migrate_repo/manage.py	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migrate_repo/manage.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,3 @@
+#!/usr/bin/env python
+from migrate.versioning.shell import main
+main(debug='False', repository='.')

=== added file 'glance/registry/db/migrate_repo/migrate.cfg'
--- glance/registry/db/migrate_repo/migrate.cfg	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migrate_repo/migrate.cfg	2011-02-03 00:03:10 +0000
@@ -0,0 +1,20 @@
+[db_settings]
+# Used to identify which repository this database is versioned under.
+# You can use the name of your project.
+repository_id=Glance Migrations
+
+# The name of the database table used to track the schema version.
+# This name shouldn't already be used by your project.
+# If this is changed once a database is under version control, you'll need to
+# change the table name in each database too.
+version_table=migrate_version
+
+# When committing a change script, Migrate will attempt to generate the
+# sql for all supported databases; normally, if one of them fails - probably
+# because you don't have that database installed - it is ignored and the
+# commit continues, perhaps ending successfully.
+# Databases in this list MUST compile successfully during a commit, or the
+# entire commit will fail. List the databases your application will actually
+# be using to ensure your updates to that database work properly.
+# This must be a list; example: ['postgres','sqlite']
+required_dbs=[]

=== added file 'glance/registry/db/migrate_repo/schema.py'
--- glance/registry/db/migrate_repo/schema.py	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migrate_repo/schema.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,100 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2011 OpenStack LLC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""
+Various conveniences used for migration scripts
+"""
+
+import logging
+
+import sqlalchemy.types
+from sqlalchemy.schema import MetaData
+
+
+logger = logging.getLogger('glance.registry.db.migrate_repo.schema')
+
+
+String = lambda length: sqlalchemy.types.String(
+    length=length, convert_unicode=False, assert_unicode=None,
+    unicode_error=None, _warn_on_bytestring=False)
+
+
+Text = lambda: sqlalchemy.types.Text(
+    length=None, convert_unicode=False, assert_unicode=None,
+    unicode_error=None, _warn_on_bytestring=False)
+
+
+Boolean = lambda: sqlalchemy.types.Boolean(create_constraint=True, name=None)
+
+
+DateTime = lambda: sqlalchemy.types.DateTime(timezone=False)
+
+
+Integer = lambda: sqlalchemy.types.Integer()
+
+
+def from_migration_import(module_name, fromlist):
+    """Import a migration file and return the module
+
+    :param module_name: name of migration module to import from
+        (ex: 001_add_images_table)
+    :param fromlist: list of items to import (ex: define_images_table)
+    :retval: module object
+
+    This bit of ugliness warrants an explanation:
+
+        As you're writing migrations, you'll frequently want to refer to
+        tables defined in previous migrations.
+
+        In the interest of not repeating yourself, you need a way of importing
+        that table into a 'future' migration.
+
+        However, tables are bound to metadata, so what you need to import is
+        really a table factory, which you can late-bind to your current
+        metadata object.
+
+        Moreover, migrations begin with a number (001...), which means they
+        aren't valid Python identifiers. This means we can't perform a
+        'normal' import on them (the Python lexer will 'splode). Instead, we
+        need to use __import__ magic to bring the table-factory into our
+        namespace.
+
+    Example Usage:
+
+        (define_images_table,) = from_migration_import(
+            '001_add_images_table', ['define_images_table'])
+
+        images = define_images_table(meta)
+
+        # Refer to images table
+
+    """
+    module_path = 'glance.registry.db.migrate_repo.versions.%s' % module_name
+    module = __import__(module_path, globals(), locals(), fromlist, -1)
+    return [getattr(module, item) for item in fromlist]
+
+
+def create_tables(tables):
+    for table in tables:
+        logger.info("creating table %(table)s" % locals())
+        table.create()
+
+
+def drop_tables(tables):
+    for table in tables:
+        logger.info("dropping table %(table)s" % locals())
+        table.drop()

=== added directory 'glance/registry/db/migrate_repo/versions'
=== added file 'glance/registry/db/migrate_repo/versions/001_add_images_table.py'
--- glance/registry/db/migrate_repo/versions/001_add_images_table.py	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migrate_repo/versions/001_add_images_table.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,55 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2011 OpenStack LLC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from sqlalchemy.schema import (Column, MetaData, Table)
+
+from glance.registry.db.migrate_repo.schema import (
+    Boolean, DateTime, Integer, String, Text, create_tables, drop_tables)
+
+
+def define_images_table(meta):
+    images = Table('images', meta,
+        Column('id', Integer(), primary_key=True, nullable=False),
+        Column('name', String(255)),
+        Column('type', String(30)),
+        Column('size', Integer()),
+        Column('status', String(30), nullable=False),
+        Column('is_public', Boolean(), nullable=False, default=False,
+               index=True),
+        Column('location', Text()),
+        Column('created_at', DateTime(), nullable=False),
+        Column('updated_at', DateTime()),
+        Column('deleted_at', DateTime()),
+        Column('deleted', Boolean(), nullable=False, default=False,
+               index=True),
+        mysql_engine='InnoDB')
+
+    return images
+
+
+def upgrade(migrate_engine):
+    meta = MetaData()
+    meta.bind = migrate_engine
+    tables = [define_images_table(meta)]
+    create_tables(tables)
+
+
+def downgrade(migrate_engine):
+    meta = MetaData()
+    meta.bind = migrate_engine
+    tables = [define_images_table(meta)]
+    drop_tables(tables)

=== added file 'glance/registry/db/migrate_repo/versions/002_add_image_properties_table.py'
--- glance/registry/db/migrate_repo/versions/002_add_image_properties_table.py	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migrate_repo/versions/002_add_image_properties_table.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,63 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2011 OpenStack LLC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from sqlalchemy.schema import (
+    Column, ForeignKey, Index, MetaData, Table, UniqueConstraint)
+
+from glance.registry.db.migrate_repo.schema import (
+    Boolean, DateTime, Integer, String, Text, create_tables, drop_tables,
+    from_migration_import)
+
+
+def define_image_properties_table(meta):
+    (define_images_table,) = from_migration_import(
+        '001_add_images_table', ['define_images_table'])
+
+    images = define_images_table(meta)
+
+    image_properties = Table('image_properties', meta,
+        Column('id', Integer(), primary_key=True, nullable=False),
+        Column('image_id', Integer(), ForeignKey('images.id'), nullable=False,
+               index=True),
+        Column('key', String(255), nullable=False),
+        Column('value', Text()),
+        Column('created_at', DateTime(), nullable=False),
+        Column('updated_at', DateTime()),
+        Column('deleted_at', DateTime()),
+        Column('deleted', Boolean(), nullable=False, default=False,
+               index=True),
+        UniqueConstraint('image_id', 'key'),
+        mysql_engine='InnoDB')
+
+    Index('ix_image_properties_image_id_key', image_properties.c.image_id,
+          image_properties.c.key)
+
+    return image_properties
+
+
+def upgrade(migrate_engine):
+    meta = MetaData()
+    meta.bind = migrate_engine
+    tables = [define_image_properties_table(meta)]
+    create_tables(tables)
+
+
+def downgrade(migrate_engine):
+    meta = MetaData()
+    meta.bind = migrate_engine
+    tables = [define_image_properties_table(meta)]
+    drop_tables(tables)

=== added file 'glance/registry/db/migrate_repo/versions/__init__.py'
--- glance/registry/db/migrate_repo/versions/__init__.py	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migrate_repo/versions/__init__.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,1 @@
+# template repository default versions module

=== added file 'glance/registry/db/migration.py'
--- glance/registry/db/migration.py	1970-01-01 00:00:00 +0000
+++ glance/registry/db/migration.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,117 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2011 OpenStack LLC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+import logging
+import os
+
+from migrate.versioning import api as versioning_api
+from migrate.versioning import exceptions as versioning_exceptions
+
+from glance.common import exception
+
+
+def db_version(options):
+    """Return the database's current migration number
+
+    :param options: options dict
+    :retval version number
+    """
+    repo_path = _find_migrate_repo()
+    sql_connection = options['sql_connection']
+    try:
+        return versioning_api.db_version(sql_connection, repo_path)
+    except versioning_exceptions.DatabaseNotControlledError, e:
+        msg = ("database '%(sql_connection)s' is not under migration control"
+               % locals())
+        raise exception.DatabaseMigrationError(msg)
+
+
+def upgrade(options, version=None):
+    """Upgrade the database's current migration level
+
+    :param options: options dict
+    :param version: version to upgrade (defaults to latest)
+    :retval version number
+    """
+    db_version(options)  # Ensure db is under migration control
+    repo_path = _find_migrate_repo()
+    sql_connection = options['sql_connection']
+    version_str = version or 'latest'
+    logging.info("Upgrading %(sql_connection)s to version %(version_str)s" %
+                  locals())
+    return versioning_api.upgrade(sql_connection, repo_path, version)
+
+
+def downgrade(options, version):
+    """Downgrade the database's current migration level
+
+    :param options: options dict
+    :param version: version to downgrade to
+    :retval version number
+    """
+    db_version(options)  # Ensure db is under migration control
+    repo_path = _find_migrate_repo()
+    sql_connection = options['sql_connection']
+    logging.info("Downgrading %(sql_connection)s to version %(version)s" %
+                  locals())
+    return versioning_api.downgrade(sql_connection, repo_path, version)
+
+
+def version_control(options):
+    """Place a database under migration control
+
+    :param options: options dict
+    """
+    sql_connection = options['sql_connection']
+    try:
+        _version_control(options)
+    except versioning_exceptions.DatabaseAlreadyControlledError, e:
+        msg = ("database '%(sql_connection)s' is already under migration "
+               "control" % locals())
+        raise exception.DatabaseMigrationError(msg)
+
+
+def _version_control(options):
+    """Place a database under migration control
+
+    :param options: options dict
+    """
+    repo_path = _find_migrate_repo()
+    sql_connection = options['sql_connection']
+    return versioning_api.version_control(sql_connection, repo_path)
+
+
+def db_sync(options, version=None):
+    """Place a database under migration control and perform an upgrade
+
+    :param options: options dict
+    :retval version number
+    """
+    try:
+        _version_control(options)
+    except versioning_exceptions.DatabaseAlreadyControlledError, e:
+        pass
+
+    upgrade(options, version=version)
+
+
+def _find_migrate_repo():
+    """Get the path for the migrate repository."""
+    path = os.path.join(os.path.abspath(os.path.dirname(__file__)),
+                        'migrate_repo')
+    assert os.path.exists(path)
+    return path

=== modified file 'glance/registry/db/models.py'
--- glance/registry/db/models.py	2011-01-31 19:39:39 +0000
+++ glance/registry/db/models.py	2011-02-03 00:03:10 +0000
@@ -42,10 +42,11 @@
     __protected_attributes__ = set([
         "created_at", "updated_at", "deleted_at", "deleted"])
 
-    created_at = Column(DateTime, default=datetime.datetime.utcnow)
+    created_at = Column(DateTime, default=datetime.datetime.utcnow,
+                        nullable=False)
     updated_at = Column(DateTime, onupdate=datetime.datetime.utcnow)
     deleted_at = Column(DateTime)
-    deleted = Column(Boolean, default=False)
+    deleted = Column(Boolean, nullable=False, default=False)
 
     def save(self, session=None):
         """Save this object"""
@@ -96,8 +97,8 @@
     name = Column(String(255))
     type = Column(String(30))
     size = Column(Integer)
-    status = Column(String(30))
-    is_public = Column(Boolean, default=False)
+    status = Column(String(30), nullable=False)
+    is_public = Column(Boolean, nullable=False, default=False)
     location = Column(Text)
 
     @validates('type')
@@ -123,5 +124,7 @@
     image_id = Column(Integer, ForeignKey('images.id'), nullable=False)
     image = relationship(Image, backref=backref('properties'))
 
-    key = Column(String(255), index=True)
+    # FIXME(sirp): KEY is a reserved word in SQL, might be a good idea to
+    # rename this column
+    key = Column(String(255), index=True, nullable=False)
     value = Column(Text)

=== added file 'tests/unit/test_migrations.py'
--- tests/unit/test_migrations.py	1970-01-01 00:00:00 +0000
+++ tests/unit/test_migrations.py	2011-02-03 00:03:10 +0000
@@ -0,0 +1,48 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2010-2011 OpenStack, LLC
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+import os
+import unittest
+
+import glance.registry.db.migration as migration_api
+import glance.common.config as config
+
+class TestMigrations(unittest.TestCase):
+    """Test sqlalchemy-migrate migrations"""
+
+    def setUp(self):
+        self.db_path = "glance_test_migration.sqlite"
+        self.options = dict(sql_connection="sqlite:///%s" % self.db_path,
+                            verbose=False)
+        config.setup_logging(self.options)
+
+    def tearDown(self):
+        if os.path.exists(self.db_path):
+            os.unlink(self.db_path)
+
+    def test_db_sync_downgrade_then_upgrade(self):
+        migration_api.db_sync(self.options)
+        
+        latest = migration_api.db_version(self.options)
+        
+        migration_api.downgrade(self.options, latest-1)
+        cur_version = migration_api.db_version(self.options)
+        self.assertEqual(cur_version, latest-1)
+
+        migration_api.upgrade(self.options, cur_version+1)
+        cur_version = migration_api.db_version(self.options)
+        self.assertEqual(cur_version, latest)

=== modified file 'tools/pip-requires'
--- tools/pip-requires	2011-01-28 21:54:34 +0000
+++ tools/pip-requires	2011-02-03 00:03:10 +0000
@@ -14,3 +14,4 @@
 argparse
 mox==0.5.0
 -f http://pymox.googlecode.com/files/mox-0.5.0.tar.gz
+sqlalchemy-migrate>=0.6