Merge lp:~rconradharris/glance/registry_db_migration into lp:~glance-coresec/glance/cactus-trunk

Proposed by Rick Harris on 2011-02-03
Status: Merged
Approved by: Jay Pipes on 2011-02-03
Approved revision: 74
Merged at revision: 64
Proposed branch: lp:~rconradharris/glance/registry_db_migration
Merge into: lp:~glance-coresec/glance/cactus-trunk
Diff against target: 916 lines (+642/-20)
22 files modified
MANIFEST.in (+1/-0)
bin/glance-manage (+130/-0)
bin/glance-upload (+3/-0)
doc/source/conf.py (+2/-0)
doc/source/glanceapi.rst (+6/-6)
doc/source/man/glanceapi.rst (+1/-1)
doc/source/man/glancemanage.py (+54/-0)
doc/source/man/glanceregistry.rst (+1/-1)
glance/common/config.py (+15/-7)
glance/common/exception.py (+8/-0)
glance/registry/db/migrate_repo/README (+4/-0)
glance/registry/db/migrate_repo/__init__.py (+1/-0)
glance/registry/db/migrate_repo/manage.py (+3/-0)
glance/registry/db/migrate_repo/migrate.cfg (+20/-0)
glance/registry/db/migrate_repo/schema.py (+100/-0)
glance/registry/db/migrate_repo/versions/001_add_images_table.py (+55/-0)
glance/registry/db/migrate_repo/versions/002_add_image_properties_table.py (+63/-0)
glance/registry/db/migrate_repo/versions/__init__.py (+1/-0)
glance/registry/db/migration.py (+117/-0)
glance/registry/db/models.py (+8/-5)
tests/unit/test_migrations.py (+48/-0)
tools/pip-requires (+1/-0)
To merge this branch: bzr merge lp:~rconradharris/glance/registry_db_migration
Reviewer Review Type Date Requested Status
Matt Dietz (community) 2011-02-03 Approve on 2011-02-03
Jay Pipes (community) 2011-02-03 Approve on 2011-02-03
Review via email: mp+48406@code.launchpad.net

This proposal supersedes a proposal from 2011-02-02.

Description of the change

Adds sqlalchemy migrations.

Also adds a glance-manage utility for managing migrations.

Potentially glance-manage (and glance-upload) could be merged into glance-admin when that lands.

To post a comment you must log in.
Jay Pipes (jaypipes) wrote : Posted in a previous version of this proposal

Very nice work, Rick. Any chance we can figure out a way of testing this?

review: Needs Information
Rick Harris (rconradharris) wrote : Posted in a previous version of this proposal

I can't think of any meaningful tests as this is just a CLI wrapper around sqlalchemy-migrate (which itself has been tested quite a bit ;-).

Jay Pipes (jaypipes) wrote : Posted in a previous version of this proposal

> I can't think of any meaningful tests as this is just a CLI wrapper around
> sqlalchemy-migrate (which itself has been tested quite a bit ;-).

What about something as simple as this?

http://pastie.org/1522390

Rick Harris (rconradharris) wrote : Posted in a previous version of this proposal

Honestly, doesn't strike me as super-useful; but that said, it's better to err on the side of caution, so I'll defer to your judgement here.

Test-mode engaged...

Jay Pipes (jaypipes) wrote :

OK, so I had an additional thought about where the database migration code should go. Since the database migration is only for the reference implementation Glance registry server (bin/glance-registry), I'm thinking that the bin/glance-manage program should be renamed bin/glance-registry-manage. While I understand your comment about having the glance-admin program merge with the glance-manage program in this patch, the two do different things. The database migration commands are particular only to the reference implementation registry server, and not to Glance as a whole, so I think the tool name should reflect that...thoughts?

Note that I think glance-upload can and should be merged into the glance-admin program, just not these specific registry database commands.

Jay Pipes (jaypipes) wrote :

OK, let's get this merged...we can always address the binary naming stuff later... I want to push on to api-image-format blueprint..

review: Approve
Matt Dietz (cerberus) wrote :

Seems alright to me.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'MANIFEST.in'
2--- MANIFEST.in 2011-01-20 10:13:46 +0000
3+++ MANIFEST.in 2011-02-03 00:03:10 +0000
4@@ -6,5 +6,6 @@
5 include tests/test_data.py
6 include tests/utils.py
7 include run_tests.py
8+include glance/registry/db/migrate_repo/migrate.cfg
9 graft doc
10 graft tools
11
12=== added file 'bin/glance-manage'
13--- bin/glance-manage 1970-01-01 00:00:00 +0000
14+++ bin/glance-manage 2011-02-03 00:03:10 +0000
15@@ -0,0 +1,130 @@
16+#!/usr/bin/env python
17+# vim: tabstop=4 shiftwidth=4 softtabstop=4
18+
19+# Copyright 2010 United States Government as represented by the
20+# Administrator of the National Aeronautics and Space Administration.
21+# Copyright 2011 OpenStack LLC.
22+# All Rights Reserved.
23+#
24+# Licensed under the Apache License, Version 2.0 (the "License"); you may
25+# not use this file except in compliance with the License. You may obtain
26+# a copy of the License at
27+#
28+# http://www.apache.org/licenses/LICENSE-2.0
29+#
30+# Unless required by applicable law or agreed to in writing, software
31+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
32+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
33+# License for the specific language governing permissions and limitations
34+# under the License.
35+
36+"""
37+Glance Management Utility
38+"""
39+
40+# FIXME(sirp): When we have glance-admin we can consider merging this into it
41+# Perhaps for consistency with Nova, we would then rename glance-admin ->
42+# glance-manage (or the other way around)
43+
44+import optparse
45+import os
46+import sys
47+
48+ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
49+
50+sys.path.append(ROOT_DIR)
51+
52+from glance import version as glance_version
53+from glance.common import config
54+from glance.common import exception
55+import glance.registry.db
56+import glance.registry.db.migration
57+
58+
59+def create_options(parser):
60+ """
61+ Sets up the CLI and config-file options that may be
62+ parsed and program commands.
63+
64+ :param parser: The option parser
65+ """
66+ glance.registry.db.add_options(parser)
67+ config.add_common_options(parser)
68+ config.add_log_options('glance-manage', parser)
69+
70+
71+def do_db_version(options, args):
72+ """Print database's current migration level"""
73+ print glance.registry.db.migration.db_version(options)
74+
75+
76+def do_upgrade(options, args):
77+ """Upgrade the database's migration level"""
78+ try:
79+ db_version = args[1]
80+ except IndexError:
81+ db_version = None
82+
83+ glance.registry.db.migration.upgrade(options, version=db_version)
84+
85+
86+def do_downgrade(options, args):
87+ """Downgrade the database's migration level"""
88+ try:
89+ db_version = args[1]
90+ except IndexError:
91+ raise exception.MissingArgumentError(
92+ "downgrade requires a version argument")
93+
94+ glance.registry.db.migration.downgrade(options, version=db_version)
95+
96+
97+def do_version_control(options, args):
98+ """Place a database under migration control"""
99+ glance.registry.db.migration.version_control(options)
100+
101+
102+def do_db_sync(options, args):
103+ """Place a database under migration control and upgrade"""
104+ try:
105+ db_version = args[1]
106+ except IndexError:
107+ db_version = None
108+ glance.registry.db.migration.db_sync(options, version=db_version)
109+
110+
111+def dispatch_cmd(options, args):
112+ """Search for do_* cmd in this module and then run it"""
113+ cmd = args[0]
114+ try:
115+ cmd_func = globals()['do_%s' % cmd]
116+ except KeyError:
117+ sys.exit("ERROR: unrecognized command '%s'" % cmd)
118+
119+ try:
120+ cmd_func(options, args)
121+ except exception.Error, e:
122+ sys.exit("ERROR: %s" % e)
123+
124+
125+def main():
126+ version = '%%prog %s' % glance_version.version_string()
127+ usage = "%prog [options] <cmd>"
128+ oparser = optparse.OptionParser(usage, version=version)
129+ create_options(oparser)
130+ (options, args) = config.parse_options(oparser)
131+
132+ try:
133+ config.setup_logging(options)
134+ except RuntimeError, e:
135+ sys.exit("ERROR: %s" % e)
136+
137+ if not args:
138+ oparser.print_usage()
139+ sys.exit(1)
140+
141+ dispatch_cmd(options, args)
142+
143+
144+if __name__ == '__main__':
145+ main()
146
147=== modified file 'bin/glance-upload'
148--- bin/glance-upload 2011-01-23 17:18:48 +0000
149+++ bin/glance-upload 2011-02-03 00:03:10 +0000
150@@ -33,6 +33,9 @@
151 <filename> <name>
152
153 """
154+
155+# FIXME(sirp): This can be merged into glance-admin when that becomes
156+# available
157 import argparse
158 import pprint
159 import sys
160
161=== modified file 'doc/source/conf.py'
162--- doc/source/conf.py 2011-01-27 20:01:02 +0000
163+++ doc/source/conf.py 2011-02-03 00:03:10 +0000
164@@ -129,6 +129,8 @@
165 ('man/glanceapi', 'glance-api', u'Glance API Server',
166 [u'OpenStack'], 1),
167 ('man/glanceregistry', 'glance-registry', u'Glance Registry Server',
168+ [u'OpenStack'], 1),
169+ ('man/glancemanage', 'glance-manage', u'Glance Management Utility',
170 [u'OpenStack'], 1)
171 ]
172
173
174=== modified file 'doc/source/glanceapi.rst'
175--- doc/source/glanceapi.rst 2011-01-26 17:26:54 +0000
176+++ doc/source/glanceapi.rst 2011-02-03 00:03:10 +0000
177@@ -24,7 +24,7 @@
178 Server*.
179
180 Assume there is a Glance API server running at the URL
181-``http://glance.example.com``.
182+``http://glance.example.com``.
183
184 Let's walk through how a user might request information from this server.
185
186@@ -116,7 +116,7 @@
187 x-image-meta-store swift
188 x-image-meta-created_at 2010-02-03 09:34:01
189 x-image-meta-updated_at 2010-02-03 09:34:01
190- x-image-meta-deleted_at
191+ x-image-meta-deleted_at
192 x-image-meta-status available
193 x-image-meta-is_public True
194 x-image-meta-property-distro Ubuntu 10.04 LTS
195@@ -126,7 +126,7 @@
196 All timestamps returned are in UTC
197
198 The `x-image-meta-updated_at` timestamp is the timestamp when an
199- image's metadata was last updated, not its image data, as all
200+ image's metadata was last updated, not its image data, as all
201 image data is immutable once stored in Glance
202
203 There may be multiple headers that begin with the prefix
204@@ -165,7 +165,7 @@
205 x-image-meta-store swift
206 x-image-meta-created_at 2010-02-03 09:34:01
207 x-image-meta-updated_at 2010-02-03 09:34:01
208- x-image-meta-deleted_at
209+ x-image-meta-deleted_at
210 x-image-meta-status available
211 x-image-meta-is_public True
212 x-image-meta-property-distro Ubuntu 10.04 LTS
213@@ -175,7 +175,7 @@
214 All timestamps returned are in UTC
215
216 The `x-image-meta-updated_at` timestamp is the timestamp when an
217- image's metadata was last updated, not its image data, as all
218+ image's metadata was last updated, not its image data, as all
219 image data is immutable once stored in Glance
220
221 There may be multiple headers that begin with the prefix
222@@ -232,7 +232,7 @@
223
224 * ``x-image-meta-id``
225
226- This header is optional.
227+ This header is optional.
228
229 When present, Glance will use the supplied identifier for the image.
230 If the identifier already exists in that Glance node, then a
231
232=== modified file 'doc/source/man/glanceapi.rst'
233--- doc/source/man/glanceapi.rst 2011-01-19 21:38:39 +0000
234+++ doc/source/man/glanceapi.rst 2011-02-03 00:03:10 +0000
235@@ -48,7 +48,7 @@
236 running ``glance-api``
237
238 FILES
239-========
240+=====
241
242 None
243
244
245=== added file 'doc/source/man/glancemanage.py'
246--- doc/source/man/glancemanage.py 1970-01-01 00:00:00 +0000
247+++ doc/source/man/glancemanage.py 2011-02-03 00:03:10 +0000
248@@ -0,0 +1,54 @@
249+=============
250+glance-manage
251+=============
252+
253+-------------------------
254+Glance Management Utility
255+-------------------------
256+
257+:Author: glance@lists.launchpad.net
258+:Date: 2010-11-16
259+:Copyright: OpenStack LLC
260+:Version: 0.1.2
261+:Manual section: 1
262+:Manual group: cloud computing
263+
264+SYNOPSIS
265+========
266+
267+ glance-manage [options]
268+
269+DESCRIPTION
270+===========
271+
272+glance-manage is a utility for managing and configuring a Glance installation.
273+One important use of glance-manage is to setup the database. To do this run::
274+
275+ glance-manage db_sync
276+
277+OPTIONS
278+=======
279+
280+ **General options**
281+
282+ **-v, --verbose**
283+ Print more verbose output
284+
285+ **--sql_connection=CONN_STRING**
286+ A proper SQLAlchemy connection string as described
287+ `here <http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/connections.html?highlight=engine#sqlalchemy.create_engine>`_
288+
289+FILES
290+=====
291+
292+None
293+
294+SEE ALSO
295+========
296+
297+* `OpenStack Glance <http://glance.openstack.org>`__
298+
299+BUGS
300+====
301+
302+* Glance is sourced in Launchpad so you can view current bugs at `OpenStack Glance <http://glance.openstack.org>`__
303
304=== modified file 'doc/source/man/glanceregistry.rst'
305--- doc/source/man/glanceregistry.rst 2011-01-19 21:38:39 +0000
306+++ doc/source/man/glanceregistry.rst 2011-02-03 00:03:10 +0000
307@@ -43,7 +43,7 @@
308 `here <http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/connections.html?highlight=engine#sqlalchemy.create_engine>`_
309
310 FILES
311-========
312+=====
313
314 None
315
316
317=== modified file 'glance/common/config.py'
318--- glance/common/config.py 2011-02-02 16:40:57 +0000
319+++ glance/common/config.py 2011-02-03 00:03:10 +0000
320@@ -27,9 +27,11 @@
321 import os
322 import sys
323
324+import glance.common.exception as exception
325
326 DEFAULT_LOG_FORMAT = "%(asctime)s %(levelname)8s [%(name)s] %(message)s"
327 DEFAULT_LOG_DATE_FORMAT = "%Y-%m-%d %H:%M:%S"
328+DEFAULT_LOG_HANDLER = 'stream'
329 LOGGING_HANDLER_CHOICES = ['syslog', 'file', 'stream']
330
331
332@@ -132,7 +134,8 @@
333 "any other logging options specified. Please see "
334 "the Python logging module documentation for "
335 "details on logging configuration files.")
336- group.add_option('--log-handler', default='stream', metavar="HANDLER",
337+ group.add_option('--log-handler', default=DEFAULT_LOG_HANDLER,
338+ metavar="HANDLER",
339 choices=LOGGING_HANDLER_CHOICES,
340 help="What logging handler to use? "
341 "Default: %default")
342@@ -159,7 +162,7 @@
343 :param options: Mapping of typed option key/values
344 """
345
346- if options['log_config']:
347+ if options.get('log_config', None):
348 # Use a logging configuration file for all settings...
349 if os.path.exists(options['log_config']):
350 logging.config.fileConfig(options['log_config'])
351@@ -179,14 +182,16 @@
352 root_logger.setLevel(logging.WARNING)
353
354 # Set log configuration from options...
355- formatter = logging.Formatter(options['log_format'],
356- options['log_date_format'])
357+ log_format = options.get('log_format', DEFAULT_LOG_FORMAT)
358+ log_date_format = options.get('log_date_format', DEFAULT_LOG_DATE_FORMAT)
359+ formatter = logging.Formatter(log_format, log_date_format)
360
361- if options['log_handler'] == 'syslog':
362+ log_handler = options.get('log_handler', DEFAULT_LOG_HANDLER)
363+ if log_handler == 'syslog':
364 syslog = logging.handlers.SysLogHandler(address='/dev/log')
365 syslog.setFormatter(formatter)
366 root_logger.addHandler(syslog)
367- elif options['log_handler'] == 'file':
368+ elif log_handler == 'file':
369 logfile = options['log_file']
370 logdir = options['log_dir']
371 if logdir:
372@@ -195,10 +200,13 @@
373 logfile.setFormatter(formatter)
374 logfile.setFormatter(formatter)
375 root_logger.addHandler(logfile)
376- else:
377+ elif log_handler == 'stream':
378 handler = logging.StreamHandler(sys.stdout)
379 handler.setFormatter(formatter)
380 root_logger.addHandler(handler)
381+ else:
382+ raise exception.BadInputError(
383+ "unrecognized log handler '%(log_handler)s'" % locals())
384
385 # Log the options used when starting if we're in debug mode...
386 if debug:
387
388=== modified file 'glance/common/exception.py'
389--- glance/common/exception.py 2011-01-26 17:26:54 +0000
390+++ glance/common/exception.py 2011-02-03 00:03:10 +0000
391@@ -75,6 +75,14 @@
392 pass
393
394
395+class MissingArgumentError(Error):
396+ pass
397+
398+
399+class DatabaseMigrationError(Error):
400+ pass
401+
402+
403 def wrap_exception(f):
404 def _wrap(*args, **kw):
405 try:
406
407=== added directory 'glance/registry/db/migrate_repo'
408=== added file 'glance/registry/db/migrate_repo/README'
409--- glance/registry/db/migrate_repo/README 1970-01-01 00:00:00 +0000
410+++ glance/registry/db/migrate_repo/README 2011-02-03 00:03:10 +0000
411@@ -0,0 +1,4 @@
412+This is a database migration repository.
413+
414+More information at
415+http://code.google.com/p/sqlalchemy-migrate/
416
417=== added file 'glance/registry/db/migrate_repo/__init__.py'
418--- glance/registry/db/migrate_repo/__init__.py 1970-01-01 00:00:00 +0000
419+++ glance/registry/db/migrate_repo/__init__.py 2011-02-03 00:03:10 +0000
420@@ -0,0 +1,1 @@
421+# template repository default module
422
423=== added file 'glance/registry/db/migrate_repo/manage.py'
424--- glance/registry/db/migrate_repo/manage.py 1970-01-01 00:00:00 +0000
425+++ glance/registry/db/migrate_repo/manage.py 2011-02-03 00:03:10 +0000
426@@ -0,0 +1,3 @@
427+#!/usr/bin/env python
428+from migrate.versioning.shell import main
429+main(debug='False', repository='.')
430
431=== added file 'glance/registry/db/migrate_repo/migrate.cfg'
432--- glance/registry/db/migrate_repo/migrate.cfg 1970-01-01 00:00:00 +0000
433+++ glance/registry/db/migrate_repo/migrate.cfg 2011-02-03 00:03:10 +0000
434@@ -0,0 +1,20 @@
435+[db_settings]
436+# Used to identify which repository this database is versioned under.
437+# You can use the name of your project.
438+repository_id=Glance Migrations
439+
440+# The name of the database table used to track the schema version.
441+# This name shouldn't already be used by your project.
442+# If this is changed once a database is under version control, you'll need to
443+# change the table name in each database too.
444+version_table=migrate_version
445+
446+# When committing a change script, Migrate will attempt to generate the
447+# sql for all supported databases; normally, if one of them fails - probably
448+# because you don't have that database installed - it is ignored and the
449+# commit continues, perhaps ending successfully.
450+# Databases in this list MUST compile successfully during a commit, or the
451+# entire commit will fail. List the databases your application will actually
452+# be using to ensure your updates to that database work properly.
453+# This must be a list; example: ['postgres','sqlite']
454+required_dbs=[]
455
456=== added file 'glance/registry/db/migrate_repo/schema.py'
457--- glance/registry/db/migrate_repo/schema.py 1970-01-01 00:00:00 +0000
458+++ glance/registry/db/migrate_repo/schema.py 2011-02-03 00:03:10 +0000
459@@ -0,0 +1,100 @@
460+# vim: tabstop=4 shiftwidth=4 softtabstop=4
461+
462+# Copyright 2011 OpenStack LLC.
463+# All Rights Reserved.
464+#
465+# Licensed under the Apache License, Version 2.0 (the "License"); you may
466+# not use this file except in compliance with the License. You may obtain
467+# a copy of the License at
468+#
469+# http://www.apache.org/licenses/LICENSE-2.0
470+#
471+# Unless required by applicable law or agreed to in writing, software
472+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
473+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
474+# License for the specific language governing permissions and limitations
475+# under the License.
476+
477+"""
478+Various conveniences used for migration scripts
479+"""
480+
481+import logging
482+
483+import sqlalchemy.types
484+from sqlalchemy.schema import MetaData
485+
486+
487+logger = logging.getLogger('glance.registry.db.migrate_repo.schema')
488+
489+
490+String = lambda length: sqlalchemy.types.String(
491+ length=length, convert_unicode=False, assert_unicode=None,
492+ unicode_error=None, _warn_on_bytestring=False)
493+
494+
495+Text = lambda: sqlalchemy.types.Text(
496+ length=None, convert_unicode=False, assert_unicode=None,
497+ unicode_error=None, _warn_on_bytestring=False)
498+
499+
500+Boolean = lambda: sqlalchemy.types.Boolean(create_constraint=True, name=None)
501+
502+
503+DateTime = lambda: sqlalchemy.types.DateTime(timezone=False)
504+
505+
506+Integer = lambda: sqlalchemy.types.Integer()
507+
508+
509+def from_migration_import(module_name, fromlist):
510+ """Import a migration file and return the module
511+
512+ :param module_name: name of migration module to import from
513+ (ex: 001_add_images_table)
514+ :param fromlist: list of items to import (ex: define_images_table)
515+ :retval: module object
516+
517+ This bit of ugliness warrants an explanation:
518+
519+ As you're writing migrations, you'll frequently want to refer to
520+ tables defined in previous migrations.
521+
522+ In the interest of not repeating yourself, you need a way of importing
523+ that table into a 'future' migration.
524+
525+ However, tables are bound to metadata, so what you need to import is
526+ really a table factory, which you can late-bind to your current
527+ metadata object.
528+
529+ Moreover, migrations begin with a number (001...), which means they
530+ aren't valid Python identifiers. This means we can't perform a
531+ 'normal' import on them (the Python lexer will 'splode). Instead, we
532+ need to use __import__ magic to bring the table-factory into our
533+ namespace.
534+
535+ Example Usage:
536+
537+ (define_images_table,) = from_migration_import(
538+ '001_add_images_table', ['define_images_table'])
539+
540+ images = define_images_table(meta)
541+
542+ # Refer to images table
543+
544+ """
545+ module_path = 'glance.registry.db.migrate_repo.versions.%s' % module_name
546+ module = __import__(module_path, globals(), locals(), fromlist, -1)
547+ return [getattr(module, item) for item in fromlist]
548+
549+
550+def create_tables(tables):
551+ for table in tables:
552+ logger.info("creating table %(table)s" % locals())
553+ table.create()
554+
555+
556+def drop_tables(tables):
557+ for table in tables:
558+ logger.info("dropping table %(table)s" % locals())
559+ table.drop()
560
561=== added directory 'glance/registry/db/migrate_repo/versions'
562=== added file 'glance/registry/db/migrate_repo/versions/001_add_images_table.py'
563--- glance/registry/db/migrate_repo/versions/001_add_images_table.py 1970-01-01 00:00:00 +0000
564+++ glance/registry/db/migrate_repo/versions/001_add_images_table.py 2011-02-03 00:03:10 +0000
565@@ -0,0 +1,55 @@
566+# vim: tabstop=4 shiftwidth=4 softtabstop=4
567+
568+# Copyright 2011 OpenStack LLC.
569+# All Rights Reserved.
570+#
571+# Licensed under the Apache License, Version 2.0 (the "License"); you may
572+# not use this file except in compliance with the License. You may obtain
573+# a copy of the License at
574+#
575+# http://www.apache.org/licenses/LICENSE-2.0
576+#
577+# Unless required by applicable law or agreed to in writing, software
578+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
579+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
580+# License for the specific language governing permissions and limitations
581+# under the License.
582+
583+from sqlalchemy.schema import (Column, MetaData, Table)
584+
585+from glance.registry.db.migrate_repo.schema import (
586+ Boolean, DateTime, Integer, String, Text, create_tables, drop_tables)
587+
588+
589+def define_images_table(meta):
590+ images = Table('images', meta,
591+ Column('id', Integer(), primary_key=True, nullable=False),
592+ Column('name', String(255)),
593+ Column('type', String(30)),
594+ Column('size', Integer()),
595+ Column('status', String(30), nullable=False),
596+ Column('is_public', Boolean(), nullable=False, default=False,
597+ index=True),
598+ Column('location', Text()),
599+ Column('created_at', DateTime(), nullable=False),
600+ Column('updated_at', DateTime()),
601+ Column('deleted_at', DateTime()),
602+ Column('deleted', Boolean(), nullable=False, default=False,
603+ index=True),
604+ mysql_engine='InnoDB')
605+
606+ return images
607+
608+
609+def upgrade(migrate_engine):
610+ meta = MetaData()
611+ meta.bind = migrate_engine
612+ tables = [define_images_table(meta)]
613+ create_tables(tables)
614+
615+
616+def downgrade(migrate_engine):
617+ meta = MetaData()
618+ meta.bind = migrate_engine
619+ tables = [define_images_table(meta)]
620+ drop_tables(tables)
621
622=== added file 'glance/registry/db/migrate_repo/versions/002_add_image_properties_table.py'
623--- glance/registry/db/migrate_repo/versions/002_add_image_properties_table.py 1970-01-01 00:00:00 +0000
624+++ glance/registry/db/migrate_repo/versions/002_add_image_properties_table.py 2011-02-03 00:03:10 +0000
625@@ -0,0 +1,63 @@
626+# vim: tabstop=4 shiftwidth=4 softtabstop=4
627+
628+# Copyright 2011 OpenStack LLC.
629+# All Rights Reserved.
630+#
631+# Licensed under the Apache License, Version 2.0 (the "License"); you may
632+# not use this file except in compliance with the License. You may obtain
633+# a copy of the License at
634+#
635+# http://www.apache.org/licenses/LICENSE-2.0
636+#
637+# Unless required by applicable law or agreed to in writing, software
638+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
639+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
640+# License for the specific language governing permissions and limitations
641+# under the License.
642+
643+from sqlalchemy.schema import (
644+ Column, ForeignKey, Index, MetaData, Table, UniqueConstraint)
645+
646+from glance.registry.db.migrate_repo.schema import (
647+ Boolean, DateTime, Integer, String, Text, create_tables, drop_tables,
648+ from_migration_import)
649+
650+
651+def define_image_properties_table(meta):
652+ (define_images_table,) = from_migration_import(
653+ '001_add_images_table', ['define_images_table'])
654+
655+ images = define_images_table(meta)
656+
657+ image_properties = Table('image_properties', meta,
658+ Column('id', Integer(), primary_key=True, nullable=False),
659+ Column('image_id', Integer(), ForeignKey('images.id'), nullable=False,
660+ index=True),
661+ Column('key', String(255), nullable=False),
662+ Column('value', Text()),
663+ Column('created_at', DateTime(), nullable=False),
664+ Column('updated_at', DateTime()),
665+ Column('deleted_at', DateTime()),
666+ Column('deleted', Boolean(), nullable=False, default=False,
667+ index=True),
668+ UniqueConstraint('image_id', 'key'),
669+ mysql_engine='InnoDB')
670+
671+ Index('ix_image_properties_image_id_key', image_properties.c.image_id,
672+ image_properties.c.key)
673+
674+ return image_properties
675+
676+
677+def upgrade(migrate_engine):
678+ meta = MetaData()
679+ meta.bind = migrate_engine
680+ tables = [define_image_properties_table(meta)]
681+ create_tables(tables)
682+
683+
684+def downgrade(migrate_engine):
685+ meta = MetaData()
686+ meta.bind = migrate_engine
687+ tables = [define_image_properties_table(meta)]
688+ drop_tables(tables)
689
690=== added file 'glance/registry/db/migrate_repo/versions/__init__.py'
691--- glance/registry/db/migrate_repo/versions/__init__.py 1970-01-01 00:00:00 +0000
692+++ glance/registry/db/migrate_repo/versions/__init__.py 2011-02-03 00:03:10 +0000
693@@ -0,0 +1,1 @@
694+# template repository default versions module
695
696=== added file 'glance/registry/db/migration.py'
697--- glance/registry/db/migration.py 1970-01-01 00:00:00 +0000
698+++ glance/registry/db/migration.py 2011-02-03 00:03:10 +0000
699@@ -0,0 +1,117 @@
700+# vim: tabstop=4 shiftwidth=4 softtabstop=4
701+
702+# Copyright 2011 OpenStack LLC.
703+# All Rights Reserved.
704+#
705+# Licensed under the Apache License, Version 2.0 (the "License"); you may
706+# not use this file except in compliance with the License. You may obtain
707+# a copy of the License at
708+#
709+# http://www.apache.org/licenses/LICENSE-2.0
710+#
711+# Unless required by applicable law or agreed to in writing, software
712+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
713+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
714+# License for the specific language governing permissions and limitations
715+# under the License.
716+
717+import logging
718+import os
719+
720+from migrate.versioning import api as versioning_api
721+from migrate.versioning import exceptions as versioning_exceptions
722+
723+from glance.common import exception
724+
725+
726+def db_version(options):
727+ """Return the database's current migration number
728+
729+ :param options: options dict
730+ :retval version number
731+ """
732+ repo_path = _find_migrate_repo()
733+ sql_connection = options['sql_connection']
734+ try:
735+ return versioning_api.db_version(sql_connection, repo_path)
736+ except versioning_exceptions.DatabaseNotControlledError, e:
737+ msg = ("database '%(sql_connection)s' is not under migration control"
738+ % locals())
739+ raise exception.DatabaseMigrationError(msg)
740+
741+
742+def upgrade(options, version=None):
743+ """Upgrade the database's current migration level
744+
745+ :param options: options dict
746+ :param version: version to upgrade (defaults to latest)
747+ :retval version number
748+ """
749+ db_version(options) # Ensure db is under migration control
750+ repo_path = _find_migrate_repo()
751+ sql_connection = options['sql_connection']
752+ version_str = version or 'latest'
753+ logging.info("Upgrading %(sql_connection)s to version %(version_str)s" %
754+ locals())
755+ return versioning_api.upgrade(sql_connection, repo_path, version)
756+
757+
758+def downgrade(options, version):
759+ """Downgrade the database's current migration level
760+
761+ :param options: options dict
762+ :param version: version to downgrade to
763+ :retval version number
764+ """
765+ db_version(options) # Ensure db is under migration control
766+ repo_path = _find_migrate_repo()
767+ sql_connection = options['sql_connection']
768+ logging.info("Downgrading %(sql_connection)s to version %(version)s" %
769+ locals())
770+ return versioning_api.downgrade(sql_connection, repo_path, version)
771+
772+
773+def version_control(options):
774+ """Place a database under migration control
775+
776+ :param options: options dict
777+ """
778+ sql_connection = options['sql_connection']
779+ try:
780+ _version_control(options)
781+ except versioning_exceptions.DatabaseAlreadyControlledError, e:
782+ msg = ("database '%(sql_connection)s' is already under migration "
783+ "control" % locals())
784+ raise exception.DatabaseMigrationError(msg)
785+
786+
787+def _version_control(options):
788+ """Place a database under migration control
789+
790+ :param options: options dict
791+ """
792+ repo_path = _find_migrate_repo()
793+ sql_connection = options['sql_connection']
794+ return versioning_api.version_control(sql_connection, repo_path)
795+
796+
797+def db_sync(options, version=None):
798+ """Place a database under migration control and perform an upgrade
799+
800+ :param options: options dict
801+ :retval version number
802+ """
803+ try:
804+ _version_control(options)
805+ except versioning_exceptions.DatabaseAlreadyControlledError, e:
806+ pass
807+
808+ upgrade(options, version=version)
809+
810+
811+def _find_migrate_repo():
812+ """Get the path for the migrate repository."""
813+ path = os.path.join(os.path.abspath(os.path.dirname(__file__)),
814+ 'migrate_repo')
815+ assert os.path.exists(path)
816+ return path
817
818=== modified file 'glance/registry/db/models.py'
819--- glance/registry/db/models.py 2011-01-31 19:39:39 +0000
820+++ glance/registry/db/models.py 2011-02-03 00:03:10 +0000
821@@ -42,10 +42,11 @@
822 __protected_attributes__ = set([
823 "created_at", "updated_at", "deleted_at", "deleted"])
824
825- created_at = Column(DateTime, default=datetime.datetime.utcnow)
826+ created_at = Column(DateTime, default=datetime.datetime.utcnow,
827+ nullable=False)
828 updated_at = Column(DateTime, onupdate=datetime.datetime.utcnow)
829 deleted_at = Column(DateTime)
830- deleted = Column(Boolean, default=False)
831+ deleted = Column(Boolean, nullable=False, default=False)
832
833 def save(self, session=None):
834 """Save this object"""
835@@ -96,8 +97,8 @@
836 name = Column(String(255))
837 type = Column(String(30))
838 size = Column(Integer)
839- status = Column(String(30))
840- is_public = Column(Boolean, default=False)
841+ status = Column(String(30), nullable=False)
842+ is_public = Column(Boolean, nullable=False, default=False)
843 location = Column(Text)
844
845 @validates('type')
846@@ -123,5 +124,7 @@
847 image_id = Column(Integer, ForeignKey('images.id'), nullable=False)
848 image = relationship(Image, backref=backref('properties'))
849
850- key = Column(String(255), index=True)
851+ # FIXME(sirp): KEY is a reserved word in SQL, might be a good idea to
852+ # rename this column
853+ key = Column(String(255), index=True, nullable=False)
854 value = Column(Text)
855
856=== added file 'tests/unit/test_migrations.py'
857--- tests/unit/test_migrations.py 1970-01-01 00:00:00 +0000
858+++ tests/unit/test_migrations.py 2011-02-03 00:03:10 +0000
859@@ -0,0 +1,48 @@
860+# vim: tabstop=4 shiftwidth=4 softtabstop=4
861+
862+# Copyright 2010-2011 OpenStack, LLC
863+# All Rights Reserved.
864+#
865+# Licensed under the Apache License, Version 2.0 (the "License"); you may
866+# not use this file except in compliance with the License. You may obtain
867+# a copy of the License at
868+#
869+# http://www.apache.org/licenses/LICENSE-2.0
870+#
871+# Unless required by applicable law or agreed to in writing, software
872+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
873+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
874+# License for the specific language governing permissions and limitations
875+# under the License.
876+
877+import os
878+import unittest
879+
880+import glance.registry.db.migration as migration_api
881+import glance.common.config as config
882+
883+class TestMigrations(unittest.TestCase):
884+ """Test sqlalchemy-migrate migrations"""
885+
886+ def setUp(self):
887+ self.db_path = "glance_test_migration.sqlite"
888+ self.options = dict(sql_connection="sqlite:///%s" % self.db_path,
889+ verbose=False)
890+ config.setup_logging(self.options)
891+
892+ def tearDown(self):
893+ if os.path.exists(self.db_path):
894+ os.unlink(self.db_path)
895+
896+ def test_db_sync_downgrade_then_upgrade(self):
897+ migration_api.db_sync(self.options)
898+
899+ latest = migration_api.db_version(self.options)
900+
901+ migration_api.downgrade(self.options, latest-1)
902+ cur_version = migration_api.db_version(self.options)
903+ self.assertEqual(cur_version, latest-1)
904+
905+ migration_api.upgrade(self.options, cur_version+1)
906+ cur_version = migration_api.db_version(self.options)
907+ self.assertEqual(cur_version, latest)
908
909=== modified file 'tools/pip-requires'
910--- tools/pip-requires 2011-01-28 21:54:34 +0000
911+++ tools/pip-requires 2011-02-03 00:03:10 +0000
912@@ -14,3 +14,4 @@
913 argparse
914 mox==0.5.0
915 -f http://pymox.googlecode.com/files/mox-0.5.0.tar.gz
916+sqlalchemy-migrate>=0.6

Subscribers

People subscribed via source and target branches