Merge lp:~raj-abhilash1/mailman/alembic-fix into lp:mailman

Proposed by Abhilash Raj
Status: Merged
Merged at revision: 7291
Proposed branch: lp:~raj-abhilash1/mailman/alembic-fix
Merge into: lp:mailman
Diff against target: 74 lines (+30/-2)
2 files modified
src/mailman/database/alembic/env.py (+7/-2)
src/mailman/docs/DATABASE.rst (+23/-0)
To merge this branch: bzr merge lp:~raj-abhilash1/mailman/alembic-fix
Reviewer Review Type Date Requested Status
Mailman Coders Pending
Review via email: mp+246026@code.launchpad.net

Description of the change

Fixed the autogeneration of migrations using alembic.

To post a comment you must log in.
Revision history for this message
Barry Warsaw (barry) wrote :

Thanks for working on this. I think my problem is more basic. How do I run this - or maybe, *do* I even need to run this? For example, how would I create a migration for the pre-py3 branch merge to the current trunk, where a few columns changed from LargeBinary to Unicode?

Could you add some documentation on what needs to be done to make the migration work, say in src/mailman/docs/DATABASE.rst? Or is everything so automatic that I don't need to do anything when the database model changes? (I doubt that. :)

Revision history for this message
Abhilash Raj (raj-abhilash1) wrote :

On Monday 12 January 2015 06:11 AM, Barry Warsaw wrote:
> Thanks for working on this. I think my problem is more basic. How do I run this - or maybe, *do* I even need to run this? For example, how would I create a migration for the pre-py3 branch merge to the current trunk, where a few columns changed from LargeBinary to Unicode?

The basic command to generate migrations automatically would be like this:

   `alembic -c src/mailman/config/alembic.cfg revision --autogenerate`

What alembic does is compares the current state of database schema and
models, so in order to automatically generate the migrations you need to
have the database in the older state. Then the migrations would be
applied automatically when the mailman is started next. So when someone
updates the mailman, the database schema would be upgraded in the first
run automatically. In case you don't have the database in the old state
you generate an empty revision( just remove the --autogenerate flag in
the above command) and add the migrations yourself.

> Could you add some documentation on what needs to be done to make the migration work, say in src/mailman/docs/DATABASE.rst? Or is everything so automatic that I don't need to do anything when the database model changes? (I doubt that. :)

I will add what I wrote above in the documentation as well.

--
thanks,
Abhilash Raj

7290. By Abhilash Raj

Add how to migrate in DATABASE.rst

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/mailman/database/alembic/env.py'
2--- src/mailman/database/alembic/env.py 2015-01-05 01:22:39 +0000
3+++ src/mailman/database/alembic/env.py 2015-01-13 09:40:47 +0000
4@@ -25,11 +25,18 @@
5
6 from alembic import context
7 from contextlib import closing
8+from mailman.core.initialize import initialize_1
9 from mailman.config import config
10 from mailman.database.model import Model
11 from mailman.utilities.string import expand
12 from sqlalchemy import create_engine
13
14+try:
15+ url = expand(config.database.url, config.paths)
16+except AttributeError:
17+ # Initialize config object for external alembic calls
18+ initialize_1()
19+ url = expand(config.database.url, config.paths)
20
21
22
23 def run_migrations_offline():
24@@ -42,7 +49,6 @@
25 Calls to context.execute() here emit the given string to the script
26 output.
27 """
28- url = expand(config.database.url, config.paths)
29 context.configure(url=url, target_metadata=Model.metadata)
30 with context.begin_transaction():
31 context.run_migrations()
32@@ -54,7 +60,6 @@
33 In this scenario we need to create an Engine and associate a
34 connection with the context.
35 """
36- url = expand(config.database.url, config.paths)
37 engine = create_engine(url)
38
39 connection = engine.connect()
40
41=== modified file 'src/mailman/docs/DATABASE.rst'
42--- src/mailman/docs/DATABASE.rst 2014-11-09 12:52:58 +0000
43+++ src/mailman/docs/DATABASE.rst 2015-01-13 09:40:47 +0000
44@@ -59,8 +59,31 @@
45 My thanks to Stephen A. Goss for his contribution of PostgreSQL support.
46
47
48+Database Migrations
49+===================
50+
51+Mailman uses `Alembic`_ to manage database migrations. Lets say you change
52+something in the models, what steps are needed to reflect that change in
53+the database schema? All you need to do is run this command::
54+
55+ alembic -c src/mailman/config/alembic.cfg revision --autogenerate
56+
57+This would create a new migration which would automatically be migrated to the
58+database on the next run of mailman. Note that the database needs to be in the
59+older state so that alembic can track the changes in the schema and autogenerate
60+a migration. If you don't have the database in the older state you can remove
61+the `--autogenerate` flag in the above command. It would then create a new empty
62+revision which you can edit manually to reflect your changes in the database
63+schema.
64+
65+People upgrading Mailman from previous versions need not do anything manually,
66+as soon as a new migration is added in the sources, it will be automatically
67+reflected in the schema on first-run post-update.
68+
69+
70 .. _SQLAlchemy: http://www.sqlalchemy.org/
71 .. _SQLite3: http://docs.python.org/library/sqlite3.html
72 .. _PostgreSQL: http://www.postgresql.org/
73 .. _MySQL: http://dev.mysql.com/
74 .. _`Ubuntu article`: https://help.ubuntu.com/community/PostgreSQL
75+.. _`Alembic`: https://alembic.readthedocs.org/en/latest/