OpenStack Compute (nova)

Merge lp:~termie/nova/db_migration into lp:~hudson-openstack/nova/trunk

  1. db_migration
  2. Merge into trunk
Proposed by termie
Status: Merged
Approved by: Vish Ishaya
Approved revision: 565
Merged at revision: 581
Proposed branch: lp:~termie/nova/db_migration
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 1141 lines (+920/-88)
15 files modified
MANIFEST.in (+1/-0)
bin/nova-manage (+18/-1)
nova/db/migration.py (+38/-0)
nova/db/sqlalchemy/__init__.py (+0/-28)
nova/db/sqlalchemy/migrate_repo/README (+4/-0)
nova/db/sqlalchemy/migrate_repo/manage.py (+4/-0)
nova/db/sqlalchemy/migrate_repo/migrate.cfg (+20/-0)
nova/db/sqlalchemy/migrate_repo/versions/001_austin.py (+547/-0)
nova/db/sqlalchemy/migrate_repo/versions/002_bexar.py (+209/-0)
nova/db/sqlalchemy/migration.py (+72/-0)
nova/db/sqlalchemy/models.py (+0/-45)
nova/service.py (+0/-13)
nova/tests/__init__.py (+5/-0)
run_tests.py (+1/-1)
tools/pip-requires (+1/-0)
To merge this branch: bzr merge lp:~termie/nova/db_migration
Reviewer Review Type Date Requested Status
termie (community) Approve
Jay Pipes (community) Approve
Thierry Carrez (community) ffe Approve
Todd Willey (community) Approve
Devin Carlen (community) Approve
Review via email: mp+46073@code.launchpad.net

Description of the change

Allows moving from the Austin-style db to the Bexar-style.

Prevents the db from being created automatically when services run.

Adds nova-manage db sync and nova manage db version to bring the db up-to-date and to check the current version. Must be run with the same privileges as the services who will access the database (or you need to chmod the db afterwards if sqlite)

To post a comment you must log in.
Revision history for this message
Devin Carlen (devcamcar) wrote :

approve, nice work

review: Approve
Revision history for this message
Todd Willey (xtoddx) wrote :

Awesome.

It would be great to have more sphinx docs about this, from the standpoint of developer needing to modify the schema, as well as a service provider needing to use nova-manage to keep their schema up to date.

review: Approve
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

The attempt to merge lp:~termie/nova/db_migration into lp:nova failed. Below is the output from the failed tests.

Revision history for this message
Soren Hansen (soren) wrote :

/me looks into this.

2011/1/13 OpenStack Hudson <email address hidden>:
> The attempt to merge lp:~termie/nova/db_migration into lp:nova failed. Below is the output from the failed tests.
>
>
>
> --
> https://code.launchpad.net/~termie/nova/db_migration/+merge/46073
> You are subscribed to branch lp:nova.
>

--
Soren Hansen <email address hidden>
Systems Architect, The Rackspace Cloud
Ubuntu Developer

Revision history for this message
Soren Hansen (soren) wrote :

python-migrate and python-tempita installed on Hudson box. Retrying.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :
Download full text (122.8 KiB)

The attempt to merge lp:~termie/nova/db_migration into lp:nova failed. Below is the output from the failed tests.

TrialTestCase
    runTest ok
AdminAPITest
    test_admin_disabled ok
    test_admin_enabled ok
APITest
    test_exceptions_are_converted_to_faults ok
Test
    test_authorize_token ok
    test_authorize_user ok
    test_bad_token ok
    test_bad_user ok
    test_no_user ok
    test_token_expiry ok
TestLimiter
    test_authorize_token ok
TestFaults
    test_fault_parts ok
    test_raise ok
    test_retry_header ok
FlavorsTest
    test_get_flavor_by_id ok
    test_get_flavor_list ok
GlanceImageServiceTest
    test_create ok
    test_create_and_show_non_existing_image ok
    test_delete ok
    test_update ok
ImageControllerWithGlanceServiceTest
    test_get_image_details ok
    test_get_image_index ok
LocalImageServiceTest
    test_create ok
    test_create_and_show_non_existing_image ok
    test_delete ok
    test_update ok
LimiterTest
    test_minute ok
    test_one_per_period ok
    test_second ok
    test_users_get_separate_buckets ok
    test_we_can_go_indefinitely_if_we_spread_out_requests ok
WSGIAppProxyTest
    test_200 ok
    test_403 ok
    test_failure ok
WSGIAppTest
    test_escaping ok
    test_good_urls ok
    test_invalid_methods ok
    test_invalid_urls ok
    test_response_to_delays ok
ServersTest
    test_create_backup_schedules ok
    test_create_instance ok
    test_delete_backup_schedules ok
    test_delete_server_instance ...

Revision history for this message
termie (termie) wrote :

looks like something new was added in trunk that found an error in the migration :) (i was dumb and didn't run the tests in the merge branch) anyway, looking into it

Revision history for this message
termie (termie) wrote :

the column creation isn't working as documented as far as i can tell, rawwwrrrr

Revision history for this message
termie (termie) wrote :

alright, column creation is working fine, there have just been changes to the models in trunk whilst i was not looking, wee.

should work fine now.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Attempt to merge into lp:nova failed due to conflicts:

text conflict in tools/pip-requires

Revision history for this message
Thierry Carrez (ttx) wrote :

This just needs a few more hours to get in. FFe approved for landing before Monday.

review: Approve (ffe)
Revision history for this message
termie (termie) wrote :

pushed new version, however there are currently pep8 violations in trunk so i expect there will be more conflicts, i have already fixed the violations in a different branch and i expect somebody else will fix them too... so.... dumb.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :
Download full text (588.5 KiB)

The attempt to merge lp:~termie/nova/db_migration into lp:nova failed. Below is the output from the failed tests.

AdminAPITest
    test_admin_disabled ok
    test_admin_enabled ok
APITest
    test_exceptions_are_converted_to_faults ok
Test
    test_authorize_token ok
    test_authorize_user ok
    test_bad_token ok
    test_bad_user ok
    test_no_user ok
    test_token_expiry ok
TestLimiter
    test_authorize_token ok
TestFaults
    test_fault_parts ok
    test_raise ok
    test_retry_header ok
FlavorsTest
    test_get_flavor_by_id ok
    test_get_flavor_list ok
GlanceImageServiceTest
    test_create ok
    test_create_and_show_non_existing_image ok
    test_delete ok
    test_update ok
ImageControllerWithGlanceServiceTest
    test_get_image_details ok
    test_get_image_index ok
LocalImageServiceTest
    test_create ok
    test_create_and_show_non_existing_image ok
    test_delete ok
    test_update ok
LimiterTest
    test_minute ok
    test_one_per_period ok
    test_second ok
    test_users_get_separate_buckets ok
    test_we_can_go_indefinitely_if_we_spread_out_requests ok
WSGIAppProxyTest
    test_200 ok
    test_403 ok
    test_failure ok
WSGIAppTest
    test_escaping ok
    test_good_urls ok
    test_invalid_methods ok
    test_invalid_urls ok
    test_response_to_delays ok
ServersTest
    test_create_backup_schedules ok
    test_create_instance ok
    test_delete_backup_schedules ok
    test_delete_server_instance ok
    test_get_all_server_details ok
    test...

Revision history for this message
termie (termie) wrote :

i'm going to wait a day or so for things to settle before I try to commit this again, tracking other people's schema changes commit-to-commit isn't really on my list of fun things.

Revision history for this message
Thierry Carrez (ttx) wrote :

@termie: due to featurefreeze you should have the necessary stability for merging this now.
I see only one branch in queue still modifying the schema (https://code.launchpad.net/~nttdata/nova/live-migration/+merge/44940)

Revision history for this message
Jay Pipes (jaypipes) wrote :

Heyo.

Mostly good (and necessary!) stuff, Andy. Nice job.

Here's a couple things to address, though (outside of the merge stuff you're currently looking at):

1) Please add sqlalchemy-migrations to tools/pip-requires otherwise tests will bomb in a virtualenv

2) Copy/paste docstring fail here :)

22 + def sync(self, version=None):
23 + """adds role to user
24 + if project is specified, adds project specific role
25 + arguments: user, role [project]"""
26 + return migration.db_sync(version)

3) Nova != Django :)

135 === added file 'nova/db/sqlalchemy/migrate_repo/__init__.py'
136 === added file 'nova/db/sqlalchemy/migrate_repo/manage.py'
137 --- nova/db/sqlalchemy/migrate_repo/manage.py 1970-01-01 00:00:00 +0000
138 +++ nova/db/sqlalchemy/migrate_repo/manage.py 2011-01-15 02:00:45 +0000
139 @@ -0,0 +1,4 @@
140 +#!/usr/bin/env python
141 +from migrate.versioning.shell import main
142 +if __name__ == '__main__':
143 + main(debug='False', repository='.')

Not sure if the above is necessary considering the nova-manage db sync. Was this just to achieve parity with Django? I didn't see this script being used anywhere.

4) This code:

951 +def db_sync(version=None):
952 + db_version()
953 + repo_path = _find_migrate_repo()
954 + return versioning_api.upgrade(FLAGS.sql_connection, repo_path, version)

Looks like there is something missing... db_version() returns the numeric version of the database, if it is versioned (returning 0 if not versioned). May I suggest changing the above code to the following:

def db_sync(version=1):
  cur_version = db_version()
  repo_path = _find_migrate_repo()
  if version > cur_version:
    return versioning_api.upgrade(FLAGS.sql_connection, repo_path, version)
  return True

Cheers!

jay

review: Needs Fixing
Revision history for this message
termie (termie) wrote :

> Heyo.
>
> Mostly good (and necessary!) stuff, Andy. Nice job.
>
> Here's a couple things to address, though (outside of the merge stuff you're
> currently looking at):
>
> 1) Please add sqlalchemy-migrations to tools/pip-requires otherwise tests will
> bomb in a virtualenv
>

sqlalchemy-migrate is already in there.

> 2) Copy/paste docstring fail here :)
>
> 22 + def sync(self, version=None):
> 23 + """adds role to user
> 24 + if project is specified, adds project specific role
> 25 + arguments: user, role [project]"""
> 26 + return migration.db_sync(version)

Nice catch.

>
> 3) Nova != Django :)
>
> 135 === added file 'nova/db/sqlalchemy/migrate_repo/__init__.py'
> 136 === added file 'nova/db/sqlalchemy/migrate_repo/manage.py'
> 137 --- nova/db/sqlalchemy/migrate_repo/manage.py 1970-01-01 00:00:00
> +0000
> 138 +++ nova/db/sqlalchemy/migrate_repo/manage.py 2011-01-15 02:00:45
> +0000
> 139 @@ -0,0 +1,4 @@
> 140 +#!/usr/bin/env python
> 141 +from migrate.versioning.shell import main
> 142 +if __name__ == '__main__':
> 143 + main(debug='False', repository='.')
>
> Not sure if the above is necessary considering the nova-manage db sync. Was
> this just to achieve parity with Django? I didn't see this script being used
> anywhere.
>

This has nothing to do with django, this is the default repository format for sqlalchemy-migrate. There is, however an indentation error.

> 4) This code:
>
> 951 +def db_sync(version=None):
> 952 + db_version()
> 953 + repo_path = _find_migrate_repo()
> 954 + return versioning_api.upgrade(FLAGS.sql_connection, repo_path,
> version)
>
> Looks like there is something missing... db_version() returns the numeric
> version of the database, if it is versioned (returning 0 if not versioned).

db_versions ensures that the db version is set correctly (1 if they already had the austin db, 0 if the db is empty).

> May I suggest changing the above code to the following:
>
> def db_sync(version=1):
> cur_version = db_version()
> repo_path = _find_migrate_repo()
> if version > cur_version:
> return versioning_api.upgrade(FLAGS.sql_connection, repo_path, version)
> return True
>
> Cheers!

Sync already knows about updates and won't update if the version is correct.

I'll fix the nova-manage documentation, however this branch is now waiting on discussion about the live_migration branch.

>
> jay

Revision history for this message
Jay Pipes (jaypipes) wrote :

> sqlalchemy-migrate is already in there.

Doh. Sorry, missed that :)

> This has nothing to do with django, this is the default repository format for
> sqlalchemy-migrate. There is, however an indentation error.

Ah, OK, sorry. Looked like it was Django-related; thx for the explanation.

> Sync already knows about updates and won't update if the version is correct.

Ah, OK, that wasn't clear...thx for the explanation.

> I'll fix the nova-manage documentation, however this branch is now waiting on
> discussion about the live_migration branch

My vote is to revert the live-migrations patch and proceed with this patch. The live-migration patch needs more baking IMHO.

review: Approve
Revision history for this message
termie (termie) wrote :

Just to make sure launchpad doesn't try to merge this again and generate an epic error log marking this as needs fixing

review: Needs Fixing
Revision history for this message
termie (termie) wrote :

aight, ready for another looksee, i added string lengths to satisfy MySQL and added the new columns

Revision history for this message
termie (termie) wrote :

approving myself, not sure whether that will make it try to merge...

review: Approve
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :
Download full text (132.2 KiB)

The attempt to merge lp:~termie/nova/db_migration into lp:nova failed. Below is the output from the failed tests.

AdminAPITest
    test_admin_disabled ok
    test_admin_enabled ok
APITest
    test_exceptions_are_converted_to_faults ok
Test
    test_authorize_token ok
    test_authorize_user ok
    test_bad_token ok
    test_bad_user ok
    test_no_user ok
    test_token_expiry ok
TestLimiter
    test_authorize_token ok
TestFaults
    test_fault_parts ok
    test_raise ok
    test_retry_header ok
FlavorsTest
    test_get_flavor_by_id ok
    test_get_flavor_list ok
GlanceImageServiceTest
    test_create ok
    test_create_and_show_non_existing_image ok
    test_delete ok
    test_update ok
ImageControllerWithGlanceServiceTest
    test_get_image_details ok
    test_get_image_index ok
LocalImageServiceTest
    test_create ok
    test_create_and_show_non_existing_image ok
    test_delete ok
    test_update ok
LimiterTest
    test_minute ok
    test_one_per_period ok
    test_second ok
    test_users_get_separate_buckets ok
    test_we_can_go_indefinitely_if_we_spread_out_requests ok
WSGIAppProxyTest
    test_200 ok
    test_403 ok
    test_failure ok
WSGIAppTest
    test_escaping ok
    test_good_urls ok
    test_invalid_methods ok
    test_invalid_urls ok
    test_response_to_delays ok
ServersTest
    test_create_backup_schedules ok
    test_create_instance ok
    test_delete_backup_schedules ok
    test_delete_server_instance ok
    test_get_all_server_details ok
    test...

Revision history for this message
termie (termie) wrote :

grr, this is all passing locally with run_tests.sh

Revision history for this message
termie (termie) wrote :

it looks like hudson is running tests twice, one of which without run_tests.sh (looks like it is running nose)

lp:~termie/nova/db_migration updated
565. By termie

move db sync into nosetests package-level fixtures so that the existing nosetests attempt in hudson will pass

Revision history for this message
termie (termie) wrote :

moved db sync into a nose package-level fixture, approving again

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'MANIFEST.in'
2--- MANIFEST.in 2010-12-22 20:59:53 +0000
3+++ MANIFEST.in 2011-01-19 01:35:10 +0000
4@@ -12,6 +12,7 @@
5 include nova/cloudpipe/client.ovpn.template
6 include nova/compute/fakevirtinstance.xml
7 include nova/compute/interfaces.template
8+include nova/db/sqlalchemy/migrate_repo/migrate.cfg
9 include nova/virt/interfaces.template
10 include nova/virt/libvirt*.xml.template
11 include nova/tests/CA/
12
13=== modified file 'bin/nova-manage'
14--- bin/nova-manage 2011-01-18 19:01:16 +0000
15+++ bin/nova-manage 2011-01-19 01:35:10 +0000
16@@ -82,6 +82,7 @@
17 from nova import utils
18 from nova.auth import manager
19 from nova.cloudpipe import pipelib
20+from nova.db import migration
21
22
23 logging.basicConfig()
24@@ -519,6 +520,21 @@
25 print re.sub('#012', "\n", "\n".join(lines))
26
27
28+class DbCommands(object):
29+ """Class for managing the database."""
30+
31+ def __init__(self):
32+ pass
33+
34+ def sync(self, version=None):
35+ """Sync the database up to the most recent version."""
36+ return migration.db_sync(version)
37+
38+ def version(self):
39+ """Print the current database version."""
40+ print migration.db_version()
41+
42+
43 CATEGORIES = [
44 ('user', UserCommands),
45 ('project', ProjectCommands),
46@@ -528,7 +544,8 @@
47 ('floating', FloatingIpCommands),
48 ('network', NetworkCommands),
49 ('service', ServiceCommands),
50- ('log', LogCommands)]
51+ ('log', LogCommands),
52+ ('db', DbCommands)]
53
54
55 def lazy_match(name, key_value_tuples):
56
57=== added file 'nova/db/migration.py'
58--- nova/db/migration.py 1970-01-01 00:00:00 +0000
59+++ nova/db/migration.py 2011-01-19 01:35:10 +0000
60@@ -0,0 +1,38 @@
61+# vim: tabstop=4 shiftwidth=4 softtabstop=4
62+
63+# Copyright 2010 United States Government as represented by the
64+# Administrator of the National Aeronautics and Space Administration.
65+# All Rights Reserved.
66+#
67+# Licensed under the Apache License, Version 2.0 (the "License"); you may
68+# not use this file except in compliance with the License. You may obtain
69+# a copy of the License at
70+#
71+# http://www.apache.org/licenses/LICENSE-2.0
72+#
73+# Unless required by applicable law or agreed to in writing, software
74+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
75+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
76+# License for the specific language governing permissions and limitations
77+# under the License.
78+"""Database setup and migration commands."""
79+
80+from nova import flags
81+from nova import utils
82+
83+FLAGS = flags.FLAGS
84+flags.DECLARE('db_backend', 'nova.db.api')
85+
86+
87+IMPL = utils.LazyPluggable(FLAGS['db_backend'],
88+ sqlalchemy='nova.db.sqlalchemy.migration')
89+
90+
91+def db_sync(version=None):
92+ """Migrate the database to `version` or the most recent version."""
93+ return IMPL.db_sync(version=version)
94+
95+
96+def db_version():
97+ """Display the current database version."""
98+ return IMPL.db_version()
99
100=== modified file 'nova/db/sqlalchemy/__init__.py'
101--- nova/db/sqlalchemy/__init__.py 2011-01-07 19:49:07 +0000
102+++ nova/db/sqlalchemy/__init__.py 2011-01-19 01:35:10 +0000
103@@ -15,31 +15,3 @@
104 # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
105 # License for the specific language governing permissions and limitations
106 # under the License.
107-
108-"""
109-SQLAlchemy database backend
110-"""
111-import time
112-
113-from sqlalchemy.exc import OperationalError
114-
115-from nova import flags
116-from nova import log as logging
117-from nova.db.sqlalchemy import models
118-
119-
120-FLAGS = flags.FLAGS
121-LOG = logging.getLogger('nova.db.sqlalchemy')
122-
123-
124-for i in xrange(FLAGS.sql_max_retries):
125- if i > 0:
126- time.sleep(FLAGS.sql_retry_interval)
127-
128- try:
129- models.register_models()
130- break
131- except OperationalError:
132- LOG.exception(_("Data store %s is unreachable."
133- " Trying again in %d seconds."),
134- FLAGS.sql_connection, FLAGS.sql_retry_interval)
135
136=== added directory 'nova/db/sqlalchemy/migrate_repo'
137=== added file 'nova/db/sqlalchemy/migrate_repo/README'
138--- nova/db/sqlalchemy/migrate_repo/README 1970-01-01 00:00:00 +0000
139+++ nova/db/sqlalchemy/migrate_repo/README 2011-01-19 01:35:10 +0000
140@@ -0,0 +1,4 @@
141+This is a database migration repository.
142+
143+More information at
144+http://code.google.com/p/sqlalchemy-migrate/
145
146=== added file 'nova/db/sqlalchemy/migrate_repo/__init__.py'
147=== added file 'nova/db/sqlalchemy/migrate_repo/manage.py'
148--- nova/db/sqlalchemy/migrate_repo/manage.py 1970-01-01 00:00:00 +0000
149+++ nova/db/sqlalchemy/migrate_repo/manage.py 2011-01-19 01:35:10 +0000
150@@ -0,0 +1,4 @@
151+#!/usr/bin/env python
152+from migrate.versioning.shell import main
153+if __name__ == '__main__':
154+ main(debug='False', repository='.')
155
156=== added file 'nova/db/sqlalchemy/migrate_repo/migrate.cfg'
157--- nova/db/sqlalchemy/migrate_repo/migrate.cfg 1970-01-01 00:00:00 +0000
158+++ nova/db/sqlalchemy/migrate_repo/migrate.cfg 2011-01-19 01:35:10 +0000
159@@ -0,0 +1,20 @@
160+[db_settings]
161+# Used to identify which repository this database is versioned under.
162+# You can use the name of your project.
163+repository_id=nova
164+
165+# The name of the database table used to track the schema version.
166+# This name shouldn't already be used by your project.
167+# If this is changed once a database is under version control, you'll need to
168+# change the table name in each database too.
169+version_table=migrate_version
170+
171+# When committing a change script, Migrate will attempt to generate the
172+# sql for all supported databases; normally, if one of them fails - probably
173+# because you don't have that database installed - it is ignored and the
174+# commit continues, perhaps ending successfully.
175+# Databases in this list MUST compile successfully during a commit, or the
176+# entire commit will fail. List the databases your application will actually
177+# be using to ensure your updates to that database work properly.
178+# This must be a list; example: ['postgres','sqlite']
179+required_dbs=[]
180
181=== added directory 'nova/db/sqlalchemy/migrate_repo/versions'
182=== added file 'nova/db/sqlalchemy/migrate_repo/versions/001_austin.py'
183--- nova/db/sqlalchemy/migrate_repo/versions/001_austin.py 1970-01-01 00:00:00 +0000
184+++ nova/db/sqlalchemy/migrate_repo/versions/001_austin.py 2011-01-19 01:35:10 +0000
185@@ -0,0 +1,547 @@
186+# vim: tabstop=4 shiftwidth=4 softtabstop=4
187+
188+# Copyright 2010 United States Government as represented by the
189+# Administrator of the National Aeronautics and Space Administration.
190+# All Rights Reserved.
191+#
192+# Licensed under the Apache License, Version 2.0 (the "License"); you may
193+# not use this file except in compliance with the License. You may obtain
194+# a copy of the License at
195+#
196+# http://www.apache.org/licenses/LICENSE-2.0
197+#
198+# Unless required by applicable law or agreed to in writing, software
199+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
200+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
201+# License for the specific language governing permissions and limitations
202+# under the License.
203+
204+## Table code mostly autogenerated by genmodel.py
205+from sqlalchemy import *
206+from migrate import *
207+
208+from nova import log as logging
209+
210+
211+meta = MetaData()
212+
213+
214+auth_tokens = Table('auth_tokens', meta,
215+ Column('created_at', DateTime(timezone=False)),
216+ Column('updated_at', DateTime(timezone=False)),
217+ Column('deleted_at', DateTime(timezone=False)),
218+ Column('deleted', Boolean(create_constraint=True, name=None)),
219+ Column('token_hash',
220+ String(length=255, convert_unicode=False, assert_unicode=None,
221+ unicode_error=None, _warn_on_bytestring=False),
222+ primary_key=True,
223+ nullable=False),
224+ Column('user_id', Integer()),
225+ Column('server_manageent_url',
226+ String(length=255, convert_unicode=False, assert_unicode=None,
227+ unicode_error=None, _warn_on_bytestring=False)),
228+ Column('storage_url',
229+ String(length=255, convert_unicode=False, assert_unicode=None,
230+ unicode_error=None, _warn_on_bytestring=False)),
231+ Column('cdn_management_url',
232+ String(length=255, convert_unicode=False, assert_unicode=None,
233+ unicode_error=None, _warn_on_bytestring=False)),
234+ )
235+
236+
237+export_devices = Table('export_devices', meta,
238+ Column('created_at', DateTime(timezone=False)),
239+ Column('updated_at', DateTime(timezone=False)),
240+ Column('deleted_at', DateTime(timezone=False)),
241+ Column('deleted', Boolean(create_constraint=True, name=None)),
242+ Column('id', Integer(), primary_key=True, nullable=False),
243+ Column('shelf_id', Integer()),
244+ Column('blade_id', Integer()),
245+ Column('volume_id',
246+ Integer(),
247+ ForeignKey('volumes.id'),
248+ nullable=True),
249+ )
250+
251+
252+fixed_ips = Table('fixed_ips', meta,
253+ Column('created_at', DateTime(timezone=False)),
254+ Column('updated_at', DateTime(timezone=False)),
255+ Column('deleted_at', DateTime(timezone=False)),
256+ Column('deleted', Boolean(create_constraint=True, name=None)),
257+ Column('id', Integer(), primary_key=True, nullable=False),
258+ Column('address',
259+ String(length=255, convert_unicode=False, assert_unicode=None,
260+ unicode_error=None, _warn_on_bytestring=False)),
261+ Column('network_id',
262+ Integer(),
263+ ForeignKey('networks.id'),
264+ nullable=True),
265+ Column('instance_id',
266+ Integer(),
267+ ForeignKey('instances.id'),
268+ nullable=True),
269+ Column('allocated', Boolean(create_constraint=True, name=None)),
270+ Column('leased', Boolean(create_constraint=True, name=None)),
271+ Column('reserved', Boolean(create_constraint=True, name=None)),
272+ )
273+
274+
275+floating_ips = Table('floating_ips', meta,
276+ Column('created_at', DateTime(timezone=False)),
277+ Column('updated_at', DateTime(timezone=False)),
278+ Column('deleted_at', DateTime(timezone=False)),
279+ Column('deleted', Boolean(create_constraint=True, name=None)),
280+ Column('id', Integer(), primary_key=True, nullable=False),
281+ Column('address',
282+ String(length=255, convert_unicode=False, assert_unicode=None,
283+ unicode_error=None, _warn_on_bytestring=False)),
284+ Column('fixed_ip_id',
285+ Integer(),
286+ ForeignKey('fixed_ips.id'),
287+ nullable=True),
288+ Column('project_id',
289+ String(length=255, convert_unicode=False, assert_unicode=None,
290+ unicode_error=None, _warn_on_bytestring=False)),
291+ Column('host',
292+ String(length=255, convert_unicode=False, assert_unicode=None,
293+ unicode_error=None, _warn_on_bytestring=False)),
294+ )
295+
296+
297+instances = Table('instances', meta,
298+ Column('created_at', DateTime(timezone=False)),
299+ Column('updated_at', DateTime(timezone=False)),
300+ Column('deleted_at', DateTime(timezone=False)),
301+ Column('deleted', Boolean(create_constraint=True, name=None)),
302+ Column('id', Integer(), primary_key=True, nullable=False),
303+ Column('internal_id', Integer()),
304+ Column('admin_pass',
305+ String(length=255, convert_unicode=False, assert_unicode=None,
306+ unicode_error=None, _warn_on_bytestring=False)),
307+ Column('user_id',
308+ String(length=255, convert_unicode=False, assert_unicode=None,
309+ unicode_error=None, _warn_on_bytestring=False)),
310+ Column('project_id',
311+ String(length=255, convert_unicode=False, assert_unicode=None,
312+ unicode_error=None, _warn_on_bytestring=False)),
313+ Column('image_id',
314+ String(length=255, convert_unicode=False, assert_unicode=None,
315+ unicode_error=None, _warn_on_bytestring=False)),
316+ Column('kernel_id',
317+ String(length=255, convert_unicode=False, assert_unicode=None,
318+ unicode_error=None, _warn_on_bytestring=False)),
319+ Column('ramdisk_id',
320+ String(length=255, convert_unicode=False, assert_unicode=None,
321+ unicode_error=None, _warn_on_bytestring=False)),
322+ Column('launch_index', Integer()),
323+ Column('key_name',
324+ String(length=255, convert_unicode=False, assert_unicode=None,
325+ unicode_error=None, _warn_on_bytestring=False)),
326+ Column('key_data',
327+ Text(length=None, convert_unicode=False, assert_unicode=None,
328+ unicode_error=None, _warn_on_bytestring=False)),
329+ Column('state', Integer()),
330+ Column('state_description',
331+ String(length=255, convert_unicode=False, assert_unicode=None,
332+ unicode_error=None, _warn_on_bytestring=False)),
333+ Column('memory_mb', Integer()),
334+ Column('vcpus', Integer()),
335+ Column('local_gb', Integer()),
336+ Column('hostname',
337+ String(length=255, convert_unicode=False, assert_unicode=None,
338+ unicode_error=None, _warn_on_bytestring=False)),
339+ Column('host',
340+ String(length=255, convert_unicode=False, assert_unicode=None,
341+ unicode_error=None, _warn_on_bytestring=False)),
342+ Column('instance_type',
343+ String(length=255, convert_unicode=False, assert_unicode=None,
344+ unicode_error=None, _warn_on_bytestring=False)),
345+ Column('user_data',
346+ Text(length=None, convert_unicode=False, assert_unicode=None,
347+ unicode_error=None, _warn_on_bytestring=False)),
348+ Column('reservation_id',
349+ String(length=255, convert_unicode=False, assert_unicode=None,
350+ unicode_error=None, _warn_on_bytestring=False)),
351+ Column('mac_address',
352+ String(length=255, convert_unicode=False, assert_unicode=None,
353+ unicode_error=None, _warn_on_bytestring=False)),
354+ Column('scheduled_at', DateTime(timezone=False)),
355+ Column('launched_at', DateTime(timezone=False)),
356+ Column('terminated_at', DateTime(timezone=False)),
357+ Column('display_name',
358+ String(length=255, convert_unicode=False, assert_unicode=None,
359+ unicode_error=None, _warn_on_bytestring=False)),
360+ Column('display_description',
361+ String(length=255, convert_unicode=False, assert_unicode=None,
362+ unicode_error=None, _warn_on_bytestring=False)),
363+ )
364+
365+
366+iscsi_targets = Table('iscsi_targets', meta,
367+ Column('created_at', DateTime(timezone=False)),
368+ Column('updated_at', DateTime(timezone=False)),
369+ Column('deleted_at', DateTime(timezone=False)),
370+ Column('deleted', Boolean(create_constraint=True, name=None)),
371+ Column('id', Integer(), primary_key=True, nullable=False),
372+ Column('target_num', Integer()),
373+ Column('host',
374+ String(length=255, convert_unicode=False, assert_unicode=None,
375+ unicode_error=None, _warn_on_bytestring=False)),
376+ Column('volume_id',
377+ Integer(),
378+ ForeignKey('volumes.id'),
379+ nullable=True),
380+ )
381+
382+
383+key_pairs = Table('key_pairs', meta,
384+ Column('created_at', DateTime(timezone=False)),
385+ Column('updated_at', DateTime(timezone=False)),
386+ Column('deleted_at', DateTime(timezone=False)),
387+ Column('deleted', Boolean(create_constraint=True, name=None)),
388+ Column('id', Integer(), primary_key=True, nullable=False),
389+ Column('name',
390+ String(length=255, convert_unicode=False, assert_unicode=None,
391+ unicode_error=None, _warn_on_bytestring=False)),
392+ Column('user_id',
393+ String(length=255, convert_unicode=False, assert_unicode=None,
394+ unicode_error=None, _warn_on_bytestring=False)),
395+ Column('fingerprint',
396+ String(length=255, convert_unicode=False, assert_unicode=None,
397+ unicode_error=None, _warn_on_bytestring=False)),
398+ Column('public_key',
399+ Text(length=None, convert_unicode=False, assert_unicode=None,
400+ unicode_error=None, _warn_on_bytestring=False)),
401+ )
402+
403+
404+networks = Table('networks', meta,
405+ Column('created_at', DateTime(timezone=False)),
406+ Column('updated_at', DateTime(timezone=False)),
407+ Column('deleted_at', DateTime(timezone=False)),
408+ Column('deleted', Boolean(create_constraint=True, name=None)),
409+ Column('id', Integer(), primary_key=True, nullable=False),
410+ Column('injected', Boolean(create_constraint=True, name=None)),
411+ Column('cidr',
412+ String(length=255, convert_unicode=False, assert_unicode=None,
413+ unicode_error=None, _warn_on_bytestring=False)),
414+ Column('netmask',
415+ String(length=255, convert_unicode=False, assert_unicode=None,
416+ unicode_error=None, _warn_on_bytestring=False)),
417+ Column('bridge',
418+ String(length=255, convert_unicode=False, assert_unicode=None,
419+ unicode_error=None, _warn_on_bytestring=False)),
420+ Column('gateway',
421+ String(length=255, convert_unicode=False, assert_unicode=None,
422+ unicode_error=None, _warn_on_bytestring=False)),
423+ Column('broadcast',
424+ String(length=255, convert_unicode=False, assert_unicode=None,
425+ unicode_error=None, _warn_on_bytestring=False)),
426+ Column('dns',
427+ String(length=255, convert_unicode=False, assert_unicode=None,
428+ unicode_error=None, _warn_on_bytestring=False)),
429+ Column('vlan', Integer()),
430+ Column('vpn_public_address',
431+ String(length=255, convert_unicode=False, assert_unicode=None,
432+ unicode_error=None, _warn_on_bytestring=False)),
433+ Column('vpn_public_port', Integer()),
434+ Column('vpn_private_address',
435+ String(length=255, convert_unicode=False, assert_unicode=None,
436+ unicode_error=None, _warn_on_bytestring=False)),
437+ Column('dhcp_start',
438+ String(length=255, convert_unicode=False, assert_unicode=None,
439+ unicode_error=None, _warn_on_bytestring=False)),
440+ Column('project_id',
441+ String(length=255, convert_unicode=False, assert_unicode=None,
442+ unicode_error=None, _warn_on_bytestring=False)),
443+ Column('host',
444+ String(length=255, convert_unicode=False, assert_unicode=None,
445+ unicode_error=None, _warn_on_bytestring=False)),
446+ )
447+
448+
449+projects = Table('projects', meta,
450+ Column('created_at', DateTime(timezone=False)),
451+ Column('updated_at', DateTime(timezone=False)),
452+ Column('deleted_at', DateTime(timezone=False)),
453+ Column('deleted', Boolean(create_constraint=True, name=None)),
454+ Column('id',
455+ String(length=255, convert_unicode=False, assert_unicode=None,
456+ unicode_error=None, _warn_on_bytestring=False),
457+ primary_key=True,
458+ nullable=False),
459+ Column('name',
460+ String(length=255, convert_unicode=False, assert_unicode=None,
461+ unicode_error=None, _warn_on_bytestring=False)),
462+ Column('description',
463+ String(length=255, convert_unicode=False, assert_unicode=None,
464+ unicode_error=None, _warn_on_bytestring=False)),
465+ Column('project_manager',
466+ String(length=255, convert_unicode=False, assert_unicode=None,
467+ unicode_error=None, _warn_on_bytestring=False),
468+ ForeignKey('users.id')),
469+ )
470+
471+
472+quotas = Table('quotas', meta,
473+ Column('created_at', DateTime(timezone=False)),
474+ Column('updated_at', DateTime(timezone=False)),
475+ Column('deleted_at', DateTime(timezone=False)),
476+ Column('deleted', Boolean(create_constraint=True, name=None)),
477+ Column('id', Integer(), primary_key=True, nullable=False),
478+ Column('project_id',
479+ String(length=255, convert_unicode=False, assert_unicode=None,
480+ unicode_error=None, _warn_on_bytestring=False)),
481+ Column('instances', Integer()),
482+ Column('cores', Integer()),
483+ Column('volumes', Integer()),
484+ Column('gigabytes', Integer()),
485+ Column('floating_ips', Integer()),
486+ )
487+
488+
489+security_groups = Table('security_groups', meta,
490+ Column('created_at', DateTime(timezone=False)),
491+ Column('updated_at', DateTime(timezone=False)),
492+ Column('deleted_at', DateTime(timezone=False)),
493+ Column('deleted', Boolean(create_constraint=True, name=None)),
494+ Column('id', Integer(), primary_key=True, nullable=False),
495+ Column('name',
496+ String(length=255, convert_unicode=False, assert_unicode=None,
497+ unicode_error=None, _warn_on_bytestring=False)),
498+ Column('description',
499+ String(length=255, convert_unicode=False, assert_unicode=None,
500+ unicode_error=None, _warn_on_bytestring=False)),
501+ Column('user_id',
502+ String(length=255, convert_unicode=False, assert_unicode=None,
503+ unicode_error=None, _warn_on_bytestring=False)),
504+ Column('project_id',
505+ String(length=255, convert_unicode=False, assert_unicode=None,
506+ unicode_error=None, _warn_on_bytestring=False)),
507+ )
508+
509+
510+security_group_inst_assoc = Table('security_group_instance_association', meta,
511+ Column('created_at', DateTime(timezone=False)),
512+ Column('updated_at', DateTime(timezone=False)),
513+ Column('deleted_at', DateTime(timezone=False)),
514+ Column('deleted', Boolean(create_constraint=True, name=None)),
515+ Column('id', Integer(), primary_key=True, nullable=False),
516+ Column('security_group_id',
517+ Integer(),
518+ ForeignKey('security_groups.id')),
519+ Column('instance_id', Integer(), ForeignKey('instances.id')),
520+ )
521+
522+
523+security_group_rules = Table('security_group_rules', meta,
524+ Column('created_at', DateTime(timezone=False)),
525+ Column('updated_at', DateTime(timezone=False)),
526+ Column('deleted_at', DateTime(timezone=False)),
527+ Column('deleted', Boolean(create_constraint=True, name=None)),
528+ Column('id', Integer(), primary_key=True, nullable=False),
529+ Column('parent_group_id',
530+ Integer(),
531+ ForeignKey('security_groups.id')),
532+ Column('protocol',
533+ String(length=255, convert_unicode=False, assert_unicode=None,
534+ unicode_error=None, _warn_on_bytestring=False)),
535+ Column('from_port', Integer()),
536+ Column('to_port', Integer()),
537+ Column('cidr',
538+ String(length=255, convert_unicode=False, assert_unicode=None,
539+ unicode_error=None, _warn_on_bytestring=False)),
540+ Column('group_id',
541+ Integer(),
542+ ForeignKey('security_groups.id')),
543+ )
544+
545+
546+services = Table('services', meta,
547+ Column('created_at', DateTime(timezone=False)),
548+ Column('updated_at', DateTime(timezone=False)),
549+ Column('deleted_at', DateTime(timezone=False)),
550+ Column('deleted', Boolean(create_constraint=True, name=None)),
551+ Column('id', Integer(), primary_key=True, nullable=False),
552+ Column('host',
553+ String(length=255, convert_unicode=False, assert_unicode=None,
554+ unicode_error=None, _warn_on_bytestring=False)),
555+ Column('binary',
556+ String(length=255, convert_unicode=False, assert_unicode=None,
557+ unicode_error=None, _warn_on_bytestring=False)),
558+ Column('topic',
559+ String(length=255, convert_unicode=False, assert_unicode=None,
560+ unicode_error=None, _warn_on_bytestring=False)),
561+ Column('report_count', Integer(), nullable=False),
562+ Column('disabled', Boolean(create_constraint=True, name=None)),
563+ )
564+
565+
566+users = Table('users', meta,
567+ Column('created_at', DateTime(timezone=False)),
568+ Column('updated_at', DateTime(timezone=False)),
569+ Column('deleted_at', DateTime(timezone=False)),
570+ Column('deleted', Boolean(create_constraint=True, name=None)),
571+ Column('id',
572+ String(length=255, convert_unicode=False, assert_unicode=None,
573+ unicode_error=None, _warn_on_bytestring=False),
574+ primary_key=True,
575+ nullable=False),
576+ Column('name',
577+ String(length=255, convert_unicode=False, assert_unicode=None,
578+ unicode_error=None, _warn_on_bytestring=False)),
579+ Column('access_key',
580+ String(length=255, convert_unicode=False, assert_unicode=None,
581+ unicode_error=None, _warn_on_bytestring=False)),
582+ Column('secret_key',
583+ String(length=255, convert_unicode=False, assert_unicode=None,
584+ unicode_error=None, _warn_on_bytestring=False)),
585+ Column('is_admin', Boolean(create_constraint=True, name=None)),
586+ )
587+
588+
589+user_project_association = Table('user_project_association', meta,
590+ Column('created_at', DateTime(timezone=False)),
591+ Column('updated_at', DateTime(timezone=False)),
592+ Column('deleted_at', DateTime(timezone=False)),
593+ Column('deleted', Boolean(create_constraint=True, name=None)),
594+ Column('user_id',
595+ String(length=255, convert_unicode=False, assert_unicode=None,
596+ unicode_error=None, _warn_on_bytestring=False),
597+ ForeignKey('users.id'),
598+ primary_key=True,
599+ nullable=False),
600+ Column('project_id',
601+ String(length=255, convert_unicode=False, assert_unicode=None,
602+ unicode_error=None, _warn_on_bytestring=False),
603+ ForeignKey('projects.id'),
604+ primary_key=True,
605+ nullable=False),
606+ )
607+
608+
609+user_project_role_association = Table('user_project_role_association', meta,
610+ Column('created_at', DateTime(timezone=False)),
611+ Column('updated_at', DateTime(timezone=False)),
612+ Column('deleted_at', DateTime(timezone=False)),
613+ Column('deleted', Boolean(create_constraint=True, name=None)),
614+ Column('user_id',
615+ String(length=255, convert_unicode=False, assert_unicode=None,
616+ unicode_error=None, _warn_on_bytestring=False),
617+ primary_key=True,
618+ nullable=False),
619+ Column('project_id',
620+ String(length=255, convert_unicode=False, assert_unicode=None,
621+ unicode_error=None, _warn_on_bytestring=False),
622+ primary_key=True,
623+ nullable=False),
624+ Column('role',
625+ String(length=255, convert_unicode=False, assert_unicode=None,
626+ unicode_error=None, _warn_on_bytestring=False),
627+ primary_key=True,
628+ nullable=False),
629+ ForeignKeyConstraint(['user_id',
630+ 'project_id'],
631+ ['user_project_association.user_id',
632+ 'user_project_association.project_id']),
633+ )
634+
635+
636+user_role_association = Table('user_role_association', meta,
637+ Column('created_at', DateTime(timezone=False)),
638+ Column('updated_at', DateTime(timezone=False)),
639+ Column('deleted_at', DateTime(timezone=False)),
640+ Column('deleted', Boolean(create_constraint=True, name=None)),
641+ Column('user_id',
642+ String(length=255, convert_unicode=False, assert_unicode=None,
643+ unicode_error=None, _warn_on_bytestring=False),
644+ ForeignKey('users.id'),
645+ primary_key=True,
646+ nullable=False),
647+ Column('role',
648+ String(length=255, convert_unicode=False, assert_unicode=None,
649+ unicode_error=None, _warn_on_bytestring=False),
650+ primary_key=True,
651+ nullable=False),
652+ )
653+
654+
655+volumes = Table('volumes', meta,
656+ Column('created_at', DateTime(timezone=False)),
657+ Column('updated_at', DateTime(timezone=False)),
658+ Column('deleted_at', DateTime(timezone=False)),
659+ Column('deleted', Boolean(create_constraint=True, name=None)),
660+ Column('id', Integer(), primary_key=True, nullable=False),
661+ Column('ec2_id',
662+ String(length=255, convert_unicode=False, assert_unicode=None,
663+ unicode_error=None, _warn_on_bytestring=False)),
664+ Column('user_id',
665+ String(length=255, convert_unicode=False, assert_unicode=None,
666+ unicode_error=None, _warn_on_bytestring=False)),
667+ Column('project_id',
668+ String(length=255, convert_unicode=False, assert_unicode=None,
669+ unicode_error=None, _warn_on_bytestring=False)),
670+ Column('host',
671+ String(length=255, convert_unicode=False, assert_unicode=None,
672+ unicode_error=None, _warn_on_bytestring=False)),
673+ Column('size', Integer()),
674+ Column('availability_zone',
675+ String(length=255, convert_unicode=False, assert_unicode=None,
676+ unicode_error=None, _warn_on_bytestring=False)),
677+ Column('instance_id',
678+ Integer(),
679+ ForeignKey('instances.id'),
680+ nullable=True),
681+ Column('mountpoint',
682+ String(length=255, convert_unicode=False, assert_unicode=None,
683+ unicode_error=None, _warn_on_bytestring=False)),
684+ Column('attach_time',
685+ String(length=255, convert_unicode=False, assert_unicode=None,
686+ unicode_error=None, _warn_on_bytestring=False)),
687+ Column('status',
688+ String(length=255, convert_unicode=False, assert_unicode=None,
689+ unicode_error=None, _warn_on_bytestring=False)),
690+ Column('attach_status',
691+ String(length=255, convert_unicode=False, assert_unicode=None,
692+ unicode_error=None, _warn_on_bytestring=False)),
693+ Column('scheduled_at', DateTime(timezone=False)),
694+ Column('launched_at', DateTime(timezone=False)),
695+ Column('terminated_at', DateTime(timezone=False)),
696+ Column('display_name',
697+ String(length=255, convert_unicode=False, assert_unicode=None,
698+ unicode_error=None, _warn_on_bytestring=False)),
699+ Column('display_description',
700+ String(length=255, convert_unicode=False, assert_unicode=None,
701+ unicode_error=None, _warn_on_bytestring=False)),
702+ )
703+
704+
705+def upgrade(migrate_engine):
706+ # Upgrade operations go here. Don't create your own engine;
707+ # bind migrate_engine to your metadata
708+ meta.bind = migrate_engine
709+
710+ for table in (auth_tokens, export_devices, fixed_ips, floating_ips,
711+ instances, iscsi_targets, key_pairs, networks,
712+ projects, quotas, security_groups, security_group_inst_assoc,
713+ security_group_rules, services, users,
714+ user_project_association, user_project_role_association,
715+ user_role_association, volumes):
716+ try:
717+ table.create()
718+ except Exception:
719+ logging.info(repr(table))
720+ logging.exception('Exception while creating table')
721+ raise
722+
723+
724+def downgrade(migrate_engine):
725+ # Operations to reverse the above upgrade go here.
726+ for table in (auth_tokens, export_devices, fixed_ips, floating_ips,
727+ instances, iscsi_targets, key_pairs, networks,
728+ projects, quotas, security_groups, security_group_inst_assoc,
729+ security_group_rules, services, users,
730+ user_project_association, user_project_role_association,
731+ user_role_association, volumes):
732+ table.drop()
733
734=== added file 'nova/db/sqlalchemy/migrate_repo/versions/002_bexar.py'
735--- nova/db/sqlalchemy/migrate_repo/versions/002_bexar.py 1970-01-01 00:00:00 +0000
736+++ nova/db/sqlalchemy/migrate_repo/versions/002_bexar.py 2011-01-19 01:35:10 +0000
737@@ -0,0 +1,209 @@
738+# vim: tabstop=4 shiftwidth=4 softtabstop=4
739+
740+# Copyright 2010 United States Government as represented by the
741+# Administrator of the National Aeronautics and Space Administration.
742+# All Rights Reserved.
743+#
744+# Licensed under the Apache License, Version 2.0 (the "License"); you may
745+# not use this file except in compliance with the License. You may obtain
746+# a copy of the License at
747+#
748+# http://www.apache.org/licenses/LICENSE-2.0
749+#
750+# Unless required by applicable law or agreed to in writing, software
751+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
752+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
753+# License for the specific language governing permissions and limitations
754+# under the License.
755+
756+from sqlalchemy import *
757+from migrate import *
758+
759+from nova import log as logging
760+
761+
762+meta = MetaData()
763+
764+
765+# Just for the ForeignKey and column creation to succeed, these are not the
766+# actual definitions of instances or services.
767+instances = Table('instances', meta,
768+ Column('id', Integer(), primary_key=True, nullable=False),
769+ )
770+
771+
772+services = Table('services', meta,
773+ Column('id', Integer(), primary_key=True, nullable=False),
774+ )
775+
776+
777+networks = Table('networks', meta,
778+ Column('id', Integer(), primary_key=True, nullable=False),
779+ )
780+
781+
782+#
783+# New Tables
784+#
785+certificates = Table('certificates', meta,
786+ Column('created_at', DateTime(timezone=False)),
787+ Column('updated_at', DateTime(timezone=False)),
788+ Column('deleted_at', DateTime(timezone=False)),
789+ Column('deleted', Boolean(create_constraint=True, name=None)),
790+ Column('id', Integer(), primary_key=True, nullable=False),
791+ Column('user_id',
792+ String(length=255, convert_unicode=False, assert_unicode=None,
793+ unicode_error=None, _warn_on_bytestring=False)),
794+ Column('project_id',
795+ String(length=255, convert_unicode=False, assert_unicode=None,
796+ unicode_error=None, _warn_on_bytestring=False)),
797+ Column('file_name',
798+ String(length=255, convert_unicode=False, assert_unicode=None,
799+ unicode_error=None, _warn_on_bytestring=False)),
800+ )
801+
802+
803+consoles = Table('consoles', meta,
804+ Column('created_at', DateTime(timezone=False)),
805+ Column('updated_at', DateTime(timezone=False)),
806+ Column('deleted_at', DateTime(timezone=False)),
807+ Column('deleted', Boolean(create_constraint=True, name=None)),
808+ Column('id', Integer(), primary_key=True, nullable=False),
809+ Column('instance_name',
810+ String(length=255, convert_unicode=False, assert_unicode=None,
811+ unicode_error=None, _warn_on_bytestring=False)),
812+ Column('instance_id', Integer()),
813+ Column('password',
814+ String(length=255, convert_unicode=False, assert_unicode=None,
815+ unicode_error=None, _warn_on_bytestring=False)),
816+ Column('port', Integer(), nullable=True),
817+ Column('pool_id',
818+ Integer(),
819+ ForeignKey('console_pools.id')),
820+ )
821+
822+
823+console_pools = Table('console_pools', meta,
824+ Column('created_at', DateTime(timezone=False)),
825+ Column('updated_at', DateTime(timezone=False)),
826+ Column('deleted_at', DateTime(timezone=False)),
827+ Column('deleted', Boolean(create_constraint=True, name=None)),
828+ Column('id', Integer(), primary_key=True, nullable=False),
829+ Column('address',
830+ String(length=255, convert_unicode=False, assert_unicode=None,
831+ unicode_error=None, _warn_on_bytestring=False)),
832+ Column('username',
833+ String(length=255, convert_unicode=False, assert_unicode=None,
834+ unicode_error=None, _warn_on_bytestring=False)),
835+ Column('password',
836+ String(length=255, convert_unicode=False, assert_unicode=None,
837+ unicode_error=None, _warn_on_bytestring=False)),
838+ Column('console_type',
839+ String(length=255, convert_unicode=False, assert_unicode=None,
840+ unicode_error=None, _warn_on_bytestring=False)),
841+ Column('public_hostname',
842+ String(length=255, convert_unicode=False, assert_unicode=None,
843+ unicode_error=None, _warn_on_bytestring=False)),
844+ Column('host',
845+ String(length=255, convert_unicode=False, assert_unicode=None,
846+ unicode_error=None, _warn_on_bytestring=False)),
847+ Column('compute_host',
848+ String(length=255, convert_unicode=False, assert_unicode=None,
849+ unicode_error=None, _warn_on_bytestring=False)),
850+ )
851+
852+
853+instance_actions = Table('instance_actions', meta,
854+ Column('created_at', DateTime(timezone=False)),
855+ Column('updated_at', DateTime(timezone=False)),
856+ Column('deleted_at', DateTime(timezone=False)),
857+ Column('deleted', Boolean(create_constraint=True, name=None)),
858+ Column('id', Integer(), primary_key=True, nullable=False),
859+ Column('instance_id',
860+ Integer(),
861+ ForeignKey('instances.id')),
862+ Column('action',
863+ String(length=255, convert_unicode=False, assert_unicode=None,
864+ unicode_error=None, _warn_on_bytestring=False)),
865+ Column('error',
866+ Text(length=None, convert_unicode=False, assert_unicode=None,
867+ unicode_error=None, _warn_on_bytestring=False)),
868+ )
869+
870+
871+#
872+# Tables to alter
873+#
874+auth_tokens = Table('auth_tokens', meta,
875+ Column('created_at', DateTime(timezone=False)),
876+ Column('updated_at', DateTime(timezone=False)),
877+ Column('deleted_at', DateTime(timezone=False)),
878+ Column('deleted', Boolean(create_constraint=True, name=None)),
879+ Column('token_hash',
880+ String(length=255, convert_unicode=False, assert_unicode=None,
881+ unicode_error=None, _warn_on_bytestring=False),
882+ primary_key=True,
883+ nullable=False),
884+ Column('user_id', Integer()),
885+ Column('server_manageent_url',
886+ String(length=255, convert_unicode=False, assert_unicode=None,
887+ unicode_error=None, _warn_on_bytestring=False)),
888+ Column('storage_url',
889+ String(length=255, convert_unicode=False, assert_unicode=None,
890+ unicode_error=None, _warn_on_bytestring=False)),
891+ Column('cdn_management_url',
892+ String(length=255, convert_unicode=False, assert_unicode=None,
893+ unicode_error=None, _warn_on_bytestring=False)),
894+ )
895+
896+
897+instances_availability_zone = Column(
898+ 'availability_zone',
899+ String(length=255, convert_unicode=False, assert_unicode=None,
900+ unicode_error=None, _warn_on_bytestring=False))
901+
902+
903+instances_locked = Column('locked',
904+ Boolean(create_constraint=True, name=None))
905+
906+
907+networks_cidr_v6 = Column(
908+ 'cidr_v6',
909+ String(length=255, convert_unicode=False, assert_unicode=None,
910+ unicode_error=None, _warn_on_bytestring=False))
911+
912+networks_ra_server = Column(
913+ 'ra_server',
914+ String(length=255, convert_unicode=False, assert_unicode=None,
915+ unicode_error=None, _warn_on_bytestring=False))
916+
917+
918+services_availability_zone = Column(
919+ 'availability_zone',
920+ String(length=255, convert_unicode=False, assert_unicode=None,
921+ unicode_error=None, _warn_on_bytestring=False))
922+
923+
924+def upgrade(migrate_engine):
925+ # Upgrade operations go here. Don't create your own engine;
926+ # bind migrate_engine to your metadata
927+ meta.bind = migrate_engine
928+ for table in (certificates, consoles, console_pools, instance_actions):
929+ try:
930+ table.create()
931+ except Exception:
932+ logging.info(repr(table))
933+ logging.exception('Exception while creating table')
934+ raise
935+
936+ auth_tokens.c.user_id.alter(type=String(length=255,
937+ convert_unicode=False,
938+ assert_unicode=None,
939+ unicode_error=None,
940+ _warn_on_bytestring=False))
941+
942+ instances.create_column(instances_availability_zone)
943+ instances.create_column(instances_locked)
944+ networks.create_column(networks_cidr_v6)
945+ networks.create_column(networks_ra_server)
946+ services.create_column(services_availability_zone)
947
948=== added file 'nova/db/sqlalchemy/migrate_repo/versions/__init__.py'
949=== added file 'nova/db/sqlalchemy/migration.py'
950--- nova/db/sqlalchemy/migration.py 1970-01-01 00:00:00 +0000
951+++ nova/db/sqlalchemy/migration.py 2011-01-19 01:35:10 +0000
952@@ -0,0 +1,72 @@
953+# vim: tabstop=4 shiftwidth=4 softtabstop=4
954+
955+# Copyright 2010 United States Government as represented by the
956+# Administrator of the National Aeronautics and Space Administration.
957+# All Rights Reserved.
958+#
959+# Licensed under the Apache License, Version 2.0 (the "License"); you may
960+# not use this file except in compliance with the License. You may obtain
961+# a copy of the License at
962+#
963+# http://www.apache.org/licenses/LICENSE-2.0
964+#
965+# Unless required by applicable law or agreed to in writing, software
966+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
967+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
968+# License for the specific language governing permissions and limitations
969+# under the License.
970+
971+import os
972+
973+from nova import flags
974+
975+import sqlalchemy
976+from migrate.versioning import api as versioning_api
977+from migrate.versioning import exceptions as versioning_exceptions
978+
979+FLAGS = flags.FLAGS
980+
981+
982+def db_sync(version=None):
983+ db_version()
984+ repo_path = _find_migrate_repo()
985+ return versioning_api.upgrade(FLAGS.sql_connection, repo_path, version)
986+
987+
988+def db_version():
989+ repo_path = _find_migrate_repo()
990+ try:
991+ return versioning_api.db_version(FLAGS.sql_connection, repo_path)
992+ except versioning_exceptions.DatabaseNotControlledError:
993+ # If we aren't version controlled we may already have the database
994+ # in the state from before we started version control, check for that
995+ # and set up version_control appropriately
996+ meta = sqlalchemy.MetaData()
997+ engine = sqlalchemy.create_engine(FLAGS.sql_connection, echo=False)
998+ meta.reflect(bind=engine)
999+ try:
1000+ for table in ('auth_tokens', 'export_devices', 'fixed_ips',
1001+ 'floating_ips', 'instances', 'iscsi_targets',
1002+ 'key_pairs', 'networks', 'projects', 'quotas',
1003+ 'security_group_rules',
1004+ 'security_group_instance_association', 'services',
1005+ 'users', 'user_project_association',
1006+ 'user_project_role_association', 'volumes'):
1007+ assert table in meta.tables
1008+ return db_version_control(1)
1009+ except AssertionError:
1010+ return db_version_control(0)
1011+
1012+
1013+def db_version_control(version=None):
1014+ repo_path = _find_migrate_repo()
1015+ versioning_api.version_control(FLAGS.sql_connection, repo_path, version)
1016+ return version
1017+
1018+
1019+def _find_migrate_repo():
1020+ """Get the path for the migrate repository."""
1021+ path = os.path.join(os.path.abspath(os.path.dirname(__file__)),
1022+ 'migrate_repo')
1023+ assert os.path.exists(path)
1024+ return path
1025
1026=== modified file 'nova/db/sqlalchemy/models.py'
1027--- nova/db/sqlalchemy/models.py 2011-01-18 19:01:16 +0000
1028+++ nova/db/sqlalchemy/models.py 2011-01-19 01:35:10 +0000
1029@@ -100,51 +100,6 @@
1030 return local.iteritems()
1031
1032
1033-# TODO(vish): Store images in the database instead of file system
1034-#class Image(BASE, NovaBase):
1035-# """Represents an image in the datastore"""
1036-# __tablename__ = 'images'
1037-# id = Column(Integer, primary_key=True)
1038-# ec2_id = Column(String(12), unique=True)
1039-# user_id = Column(String(255))
1040-# project_id = Column(String(255))
1041-# image_type = Column(String(255))
1042-# public = Column(Boolean, default=False)
1043-# state = Column(String(255))
1044-# location = Column(String(255))
1045-# arch = Column(String(255))
1046-# default_kernel_id = Column(String(255))
1047-# default_ramdisk_id = Column(String(255))
1048-#
1049-# @validates('image_type')
1050-# def validate_image_type(self, key, image_type):
1051-# assert(image_type in ['machine', 'kernel', 'ramdisk', 'raw'])
1052-#
1053-# @validates('state')
1054-# def validate_state(self, key, state):
1055-# assert(state in ['available', 'pending', 'disabled'])
1056-#
1057-# @validates('default_kernel_id')
1058-# def validate_kernel_id(self, key, val):
1059-# if val != 'machine':
1060-# assert(val is None)
1061-#
1062-# @validates('default_ramdisk_id')
1063-# def validate_ramdisk_id(self, key, val):
1064-# if val != 'machine':
1065-# assert(val is None)
1066-#
1067-#
1068-# TODO(vish): To make this into its own table, we need a good place to
1069-# create the host entries. In config somwhere? Or the first
1070-# time any object sets host? This only becomes particularly
1071-# important if we need to store per-host data.
1072-#class Host(BASE, NovaBase):
1073-# """Represents a host where services are running"""
1074-# __tablename__ = 'hosts'
1075-# id = Column(String(255), primary_key=True)
1076-
1077-
1078 class Service(BASE, NovaBase):
1079 """Represents a running service on a host."""
1080
1081
1082=== modified file 'nova/service.py'
1083--- nova/service.py 2011-01-18 19:01:16 +0000
1084+++ nova/service.py 2011-01-19 01:35:10 +0000
1085@@ -209,19 +209,6 @@
1086 self.model_disconnected = True
1087 logging.exception(_("model server went away"))
1088
1089- try:
1090- # NOTE(vish): This is late-loaded to make sure that the
1091- # database is not created before flags have
1092- # been loaded.
1093- from nova.db.sqlalchemy import models
1094- models.register_models()
1095- except OperationalError:
1096- logging.exception(_("Data store %s is unreachable."
1097- " Trying again in %d seconds.") %
1098- (FLAGS.sql_connection,
1099- FLAGS.sql_retry_interval))
1100- time.sleep(FLAGS.sql_retry_interval)
1101-
1102
1103 def serve(*services):
1104 FLAGS(sys.argv)
1105
1106=== modified file 'nova/tests/__init__.py'
1107--- nova/tests/__init__.py 2010-12-11 20:10:24 +0000
1108+++ nova/tests/__init__.py 2011-01-19 01:35:10 +0000
1109@@ -34,3 +34,8 @@
1110 # The code below enables nosetests to work with i18n _() blocks
1111 import __builtin__
1112 setattr(__builtin__, '_', lambda x: x)
1113+
1114+
1115+def setup():
1116+ from nova.db import migration
1117+ migration.db_sync()
1118
1119=== modified file 'run_tests.py'
1120--- run_tests.py 2011-01-04 20:57:18 +0000
1121+++ run_tests.py 2011-01-19 01:35:10 +0000
1122@@ -17,7 +17,7 @@
1123 # See the License for the specific language governing permissions and
1124 # limitations under the License.
1125
1126-
1127+import gettext
1128 import os
1129 import unittest
1130 import sys
1131
1132=== modified file 'tools/pip-requires'
1133--- tools/pip-requires 2011-01-14 15:51:42 +0000
1134+++ tools/pip-requires 2011-01-19 01:35:10 +0000
1135@@ -25,5 +25,6 @@
1136 Twisted>=10.1.0
1137 PasteDeploy
1138 paste
1139+sqlalchemy-migrate
1140 netaddr
1141 glance