Merge lp:~raj-abhilash1/mailman/alembic-fix into lp:mailman

[Abhilash Raj]

Fixed the autogeneration of migrations using alembic.

[Barry Warsaw (barry)]

Thanks for working on this. I think my problem is more basic. How do I run this - or maybe, *do* I even need to run this? For example, how would I create a migration for the pre-py3 branch merge to the current trunk, where a few columns changed from LargeBinary to Unicode?

Could you add some documentation on what needs to be done to make the migration work, say in src/mailman/docs/DATABASE.rst? Or is everything so automatic that I don't need to do anything when the database model changes? (I doubt that. :)

[Abhilash Raj (raj-abhilash1)]

On Monday 12 January 2015 06:11 AM, Barry Warsaw wrote:
> Thanks for working on this. I think my problem is more basic. How do I run this - or maybe, *do* I even need to run this? For example, how would I create a migration for the pre-py3 branch merge to the current trunk, where a few columns changed from LargeBinary to Unicode?

The basic command to generate migrations automatically would be like this:

   `alembic -c src/mailman/config/alembic.cfg revision --autogenerate`

What alembic does is compares the current state of database schema and
models, so in order to automatically generate the migrations you need to
have the database in the older state. Then the migrations would be
applied automatically when the mailman is started next. So when someone
updates the mailman, the database schema would be upgraded in the first
run automatically. In case you don't have the database in the old state
you generate an empty revision( just remove the --autogenerate flag in
the above command) and add the migrations yourself.

> Could you add some documentation on what needs to be done to make the migration work, say in src/mailman/docs/DATABASE.rst? Or is everything so automatic that I don't need to do anything when the database model changes? (I doubt that. :)

I will add what I wrote above in the documentation as well.

--
thanks,
Abhilash Raj

[lp:~raj-abhilash1/mailman/alembic-fix]

7290. By Abhilash Raj on 2015-01-13

Add how to migrate in DATABASE.rst

=== modified file 'src/mailman/database/alembic/env.py'
--- src/mailman/database/alembic/env.py	2015-01-05 01:22:39 +0000
+++ src/mailman/database/alembic/env.py	2015-01-13 09:40:47 +0000
@@ -25,11 +25,18 @@
 
 from alembic import context
 from contextlib import closing
+from mailman.core.initialize import initialize_1
 from mailman.config import config
 from mailman.database.model import Model
 from mailman.utilities.string import expand
 from sqlalchemy import create_engine
 
+try:
+    url = expand(config.database.url, config.paths)
+except AttributeError:
+    # Initialize config object for external alembic calls
+    initialize_1()
+    url = expand(config.database.url, config.paths)
 
 
 
 def run_migrations_offline():
@@ -42,7 +49,6 @@
     Calls to context.execute() here emit the given string to the script
     output.
     """
-    url = expand(config.database.url, config.paths)
     context.configure(url=url, target_metadata=Model.metadata)
     with context.begin_transaction():
         context.run_migrations()
@@ -54,7 +60,6 @@
     In this scenario we need to create an Engine and associate a
     connection with the context.
     """
-    url = expand(config.database.url, config.paths)
     engine = create_engine(url)
 
     connection = engine.connect()
 
=== modified file 'src/mailman/docs/DATABASE.rst'
--- src/mailman/docs/DATABASE.rst	2014-11-09 12:52:58 +0000
+++ src/mailman/docs/DATABASE.rst	2015-01-13 09:40:47 +0000
@@ -59,8 +59,31 @@
 My thanks to Stephen A. Goss for his contribution of PostgreSQL support.
 
 
+Database Migrations
+===================
+
+Mailman uses `Alembic`_ to manage database migrations. Lets say you change
+something in the models, what steps are needed to reflect that change in
+the database schema? All you need to do is run this command::
+
+    alembic -c src/mailman/config/alembic.cfg revision --autogenerate
+
+This would create a new migration which would automatically be migrated to the
+database on the next run of mailman. Note that the database needs to be in the
+older state so that alembic can track the changes in the schema and autogenerate
+a migration. If you don't have the database in the older state you can remove
+the `--autogenerate` flag in the above command. It would then create a new empty
+revision which you can edit manually to reflect your changes in the database
+schema.
+
+People upgrading Mailman from previous versions need not do anything manually,
+as soon as a new migration is added in the sources, it will be automatically
+reflected in the schema on first-run post-update.
+
+
 .. _SQLAlchemy: http://www.sqlalchemy.org/
 .. _SQLite3: http://docs.python.org/library/sqlite3.html
 .. _PostgreSQL: http://www.postgresql.org/
 .. _MySQL: http://dev.mysql.com/
 .. _`Ubuntu article`: https://help.ubuntu.com/community/PostgreSQL
+.. _`Alembic`: https://alembic.readthedocs.org/en/latest/