Merge lp:~stefanor/ibid/sqlalchemy-0.6-trunk into lp:ibid

[Stefano Rivera]

The massive SQLALchemy 0.6 + 0.7 patch. Only 18 months late :)
Main points:
* Use SQLALchemy 0.6 APIs, dropping 0.5 support.
* Create fullblown SQLAlchemy types for IbidUnicode(Text) rather than use TypeDecorator.

We should be talking about merging this

[Max Rabkin (max-rabkin)]

bucket now uses the database

review: Needs Fixing

[Keegan Carruthers-Smith (keegan-csmith)]

Don't know much about sqlalchemy, so gonna abstain.

review: Abstain

[Stefano Rivera (stefanor)]

I see from uncommitted diffs on my laptop, that I started trying to get to the bottom of the test failures with SQLAlchemy > 0.5, but I think that's non-trivial.

How about we land this now, and fix it up in trunk?

[Keegan Carruthers-Smith (keegan-csmith)]

Approving this because ibid has been running this branch for a while, and there have not been any problems. I did do a quick look over the diff and couldn't see any problems.

review: Approve

[Keegan Carruthers-Smith (keegan-csmith)]

by ibid, i mean tibid.

[Ibid Branch Auto-Lander (ibid-tarmac)]

Voting does not meet specified criteria. Required: Approve >= 1, Disapprove == 0. Got: 2 Pending.

[Max Rabkin (max-rabkin)]

per Keegan

review: Approve

[Ibid Branch Auto-Lander (ibid-tarmac)]

The attempt to merge lp:~stefanor/ibid/sqlalchemy-0.6-trunk into lp:ibid failed. Below is the output from the failed tests.

                                                                        [ERROR]

===============================================================================
[ERROR]: ibid

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/twisted/trial/runner.py", line 651, in loadByNames
    things.append(self.findByName(name))
  File "/usr/lib/python2.6/dist-packages/twisted/trial/runner.py", line 460, in findByName
    return filenameToModule(name)
  File "/usr/lib/python2.6/dist-packages/twisted/trial/runner.py", line 95, in filenameToModule
    ret = reflect.namedAny(reflect.filenameToModuleName(fn))
  File "/usr/lib/python2.6/dist-packages/twisted/python/reflect.py", line 464, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/home/stefanor/ibid/tarmac-trunk/ibid/__init__.py", line 16, in <module>
    import ibid.core
  File "/home/stefanor/ibid/tarmac-trunk/ibid/core.py", line 21, in <module>
    from ibid.db import SchemaVersionException, schema_version_check
  File "/home/stefanor/ibid/tarmac-trunk/ibid/db/__init__.py", line 5, in <module>
    from ibid.db.types import Integer, DateTime, Boolean, \
  File "/home/stefanor/ibid/tarmac-trunk/ibid/db/types.py", line 75, in <module>
    monkey_patch()
  File "/home/stefanor/ibid/tarmac-trunk/ibid/db/types.py", line 9, in monkey_patch
    import sqlalchemy.dialects.postgresql
exceptions.ImportError: No module named dialects.postgresql
-------------------------------------------------------------------------------

FAILED (errors=1)

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/twisted/python/usage.py", line 241, in parseOptions
    self.postOptions()
  File "/usr/lib/python2.6/dist-packages/twisted/scripts/trial.py", line 295, in postOptions
    self['reporter'] = self._loadReporterByName(self['reporter'])
  File "/usr/lib/python2.6/dist-packages/twisted/scripts/trial.py", line 281, in _loadReporterByName
    for p in plugin.getPlugins(itrial.IReporter):
  File "/usr/lib/python2.6/dist-packages/twisted/plugin.py", line 200, in getPlugins
    allDropins = getCache(package)
--- <exception caught here> ---
  File "/usr/lib/python2.6/dist-packages/twisted/plugin.py", line 165, in getCache
    provider = pluginModule.load()
  File "/usr/lib/python2.6/dist-packages/twisted/python/modules.py", line 381, in load
    return self.pathEntry.pythonPath.moduleLoader(self.name)
  File "/usr/lib/python2.6/dist-packages/twisted/python/reflect.py", line 464, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/home/stefanor/ibid/tarmac-trunk/twisted/plugins/ibid_plugin.py", line 10, in <module>
    import ibid
  File "/home/stefanor/ibid/tarmac-trunk/ibid/__init__.py", line 16, in <module>
    import ibid.core
  File "/home/stefanor/ibid/tarmac-trunk/ibid/core.py", line 21, in <module>
    from ibid.db import SchemaVersionException, schema_version_check
  File "/home/stefanor/ibid/tarmac-trunk/ibid/db/__init__.py", line 5, in <module>
    from ibid.db.types import Integer, DateTime, Boolean, \
 ...

[Jonathan Hitchcock (vhata)]

Surewhynot.

review: Approve

[Ibid Branch Auto-Lander (ibid-tarmac)]

Attempt to merge into lp:ibid failed due to conflicts:

text conflict in ibid/test/__init__.py

[Ibid Branch Auto-Lander (ibid-tarmac)]

There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.

[Stefano Rivera (stefanor)]

Here's a biscuit, tarmac

review: Approve

[Keegan Carruthers-Smith (keegan-csmith)]

blind approvals ftw

review: Approve

=== modified file 'docs/install.rst'
--- docs/install.rst	2010-09-26 22:55:15 +0000
+++ docs/install.rst	2012-07-31 10:05:22 +0000
@@ -50,7 +50,7 @@
 * `Twisted framework <http://twistedmatrix.com/>`_ (core sources)
 * Twisted Words (IRC, XMPP)
 * `Wokkel <http://wokkel.ik.nu/>`_. (XMPP)
-* `SQLAlchemy <http://www.sqlalchemy.org/>`_ 0.5 preferred, 0.4 compatible.
+* `SQLAlchemy <http://www.sqlalchemy.org/>`_ 0.6 or later.
 * `ConfigObj <http://www.voidspace.org.uk/python/configobj.html>`_ >= 4.7.0
 * `python-dateutil <http://labix.org/python-dateutil>`_
 * `SOAPpy <http://pywebsvcs.sourceforge.net/>`_ [#soappy-install]_

=== modified file 'ibid/core.py'
--- ibid/core.py	2012-07-08 00:38:43 +0000
+++ ibid/core.py	2012-07-31 10:05:22 +0000
@@ -14,7 +14,7 @@
 from twisted.python.modules import getModule
 from sqlalchemy import create_engine
 from sqlalchemy.orm import sessionmaker, scoped_session
-from sqlalchemy.exceptions import IntegrityError
+from sqlalchemy.exc import IntegrityError
 
 import ibid
 from ibid.event import Event
@@ -348,39 +348,47 @@
                 assert_unicode=True, echo=echo
             )
 
-        else:
-            if uri.startswith(u'mysql://'):
-                if u'?' not in uri:
-                    uri += u'?charset=utf8'
-                else:
-                    params = parse_qs(uri.split(u'?', 1)[1])
-                    if u'charset' not in params:
-                        uri += u'&charset=utf8'
-
-            engine = create_engine(uri, encoding='utf-8',
-                convert_unicode=True, assert_unicode=True, echo=echo)
-
-            if uri.startswith('mysql://'):
-                class MySQLModeListener(object):
-                    def connect(self, dbapi_con, con_record):
-                        dbapi_con.set_sql_mode("ANSI")
-                        mysql_engine = ibid.config.get('mysql_engine', 'InnoDB')
-                        c = dbapi_con.cursor()
-                        c.execute("SET storage_engine=%s;" % mysql_engine)
-                        c.execute("SET time_zone='+0:00';")
-                        c.close()
-                engine.pool.add_listener(MySQLModeListener())
-
-                engine.dialect.use_ansiquotes = True
-
-            elif uri.startswith('postgres://'):
-                class PGSQLModeListener(object):
-                    def connect(self, dbapi_con, con_record):
-                        c = dbapi_con.cursor()
-                        c.execute("SET TIME ZONE UTC")
-                        c.close()
-
-                engine.pool.add_listener(PGSQLModeListener())
+        elif uri.startswith(u'mysql://'):
+            if u'?' not in uri:
+                uri += u'?'
+            params = parse_qs(uri.split(u'?', 1)[1])
+            if u'charset' not in params:
+                uri += u'&charset=utf8'
+            if u'sql_mode' not in params:
+                uri += u'&sql_mode=ANSI_QUOTES'
+            # As recommended by SQLAlchemy due to a memory leak:
+            # http://www.sqlalchemy.org/trac/wiki/DatabaseNotes
+            if u'use_unicode' not in params:
+                uri += u'&use_unicode=0'
+
+            engine = create_engine(uri, encoding='utf-8',
+                convert_unicode=True, assert_unicode=True, echo=echo,
+                # MySQL closes 8hr old connections:
+                pool_recycle=3600)
+
+            class MySQLModeListener(object):
+                def connect(self, dbapi_con, con_record):
+                    mysql_engine = ibid.config.get('mysql_engine', 'InnoDB')
+                    c = dbapi_con.cursor()
+                    c.execute("SET SESSION storage_engine=%s;"
+                              % mysql_engine)
+                    c.execute("SET SESSION time_zone='+0:00';")
+                    c.close()
+            engine.pool.add_listener(MySQLModeListener())
+
+        elif uri.startswith('postgres://'):
+            engine = create_engine(uri, encoding='utf-8',
+                convert_unicode=True, assert_unicode=True, echo=echo,
+                # Ensure decoded unicode values are returned:
+                use_native_unicode=False)
+
+            class PGSQLModeListener(object):
+                def connect(self, dbapi_con, con_record):
+                    c = dbapi_con.cursor()
+                    c.execute("SET TIME ZONE UTC")
+                    c.close()
+
+            engine.pool.add_listener(PGSQLModeListener())
 
         self[name] = scoped_session(sessionmaker(bind=engine))
 

=== modified file 'ibid/db/__init__.py'
--- ibid/db/__init__.py	2011-01-24 11:38:54 +0000
+++ ibid/db/__init__.py	2012-07-31 10:05:22 +0000
@@ -2,7 +2,7 @@
 # Released under terms of the MIT/X/Expat Licence. See COPYING for details.
 import warnings as _warnings
 
-from ibid.db.types import TypeDecorator, Integer, DateTime, Boolean, \
+from ibid.db.types import Integer, DateTime, Boolean, \
                           IbidUnicode, IbidUnicodeText
 
 from sqlalchemy import Table, Column, ForeignKey, Index, UniqueConstraint, \
@@ -11,7 +11,7 @@
 from sqlalchemy.sql import func
 from sqlalchemy.ext.declarative import declarative_base as _declarative_base
 
-from sqlalchemy.exceptions import IntegrityError, SADeprecationWarning
+from sqlalchemy.exc import IntegrityError, SADeprecationWarning
 
 metadata = _MetaData()
 Base = _declarative_base(metadata=metadata)
@@ -19,9 +19,6 @@
 from ibid.db.versioned_schema import VersionedSchema, SchemaVersionException, \
                                      schema_version_check, upgrade_schemas
 
-# We use SQLAlchemy 0.4 compatible .save_or_update() functions
-_warnings.filterwarnings('ignore', 'Use session.add\(\)', SADeprecationWarning)
-
 def get_regexp_op(session):
     "Return a regexp operator"
     if session.bind.engine.name in ('postgres', 'postgresql'):

=== modified file 'ibid/db/types.py'
--- ibid/db/types.py	2010-01-17 20:28:33 +0000
+++ ibid/db/types.py	2012-07-31 10:05:22 +0000
@@ -1,65 +1,77 @@
-# Copyright (c) 2009-2010, Stefano Rivera
+# Copyright (c) 2009-2011, Stefano Rivera
 # Released under terms of the MIT/X/Expat Licence. See COPYING for details.
 
-from sqlalchemy.types import TypeDecorator, Integer, DateTime, Boolean, \
+from sqlalchemy.types import Integer, DateTime, Boolean, \
                              Unicode as _Unicode, UnicodeText as _UnicodeText
 
-class _CIDecorator(TypeDecorator):
-    "Abstract class for collation aware columns"
-
-    def __init__(self, length=None, case_insensitive=False):
-        self.case_insensitive = case_insensitive
-        super(_CIDecorator, self).__init__(length=length)
-
-    def load_dialect_impl(self, dialect):
-        if hasattr(dialect, 'name'):
-            self.dialect = dialect.name
-        # SQLAlchemy 0.4:
-        else:
-            self.dialect = {
-                'SQLiteDialect': 'sqlite',
-                'PGDialect': 'postgres',
-                'MySQLDialect': 'mysql',
-            }[dialect.__class__.__name__]
-
-        return dialect.type_descriptor(self.impl)
-
-    def get_col_spec(self):
-        colspec = self.impl.get_col_spec()
-        if hasattr(self, 'case_insensitive'):
-            collation = None
-            if self.dialect == 'mysql':
-                if self.case_insensitive:
-                    collation = 'utf8_general_ci'
-                else:
-                    collation = 'utf8_bin'
-            elif self.dialect == 'sqlite':
-                if self.case_insensitive:
-                    collation = 'NOCASE'
-                else:
-                    collation = 'BINARY'
-            elif self.dialect == 'postgres' and self.case_insensitive:
-                return 'CITEXT'
-
-            if collation is not None:
-                return colspec + ' COLLATE ' + collation
-        return colspec
-
-class IbidUnicode(_CIDecorator):
+
+def monkey_patch():
+    import sqlalchemy.dialects.postgresql
+    sqlalchemy.dialects.postgresql.dialect.ischema_names['citext'] = IbidUnicodeText
+    def postgres_visit_IBID_VARCHAR(self, type_):
+        if type_.case_insensitive:
+            return 'CITEXT'
+        else:
+            return self.visit_VARCHAR(type_)
+    sqlalchemy.dialects.postgresql.dialect.type_compiler.visit_IBID_VARCHAR = postgres_visit_IBID_VARCHAR
+    def postgres_visit_IBID_TEXT(self, type_):
+        if type_.case_insensitive:
+            return 'CITEXT'
+        else:
+            return self.visit_TEXT(type_)
+    sqlalchemy.dialects.postgresql.dialect.type_compiler.visit_IBID_TEXT = postgres_visit_IBID_TEXT
+
+    import sqlalchemy.dialects.sqlite
+    def sqlite_visit_IBID_VARCHAR(self, type_):
+        if type_.case_insensitive:
+            collation = 'NOCASE'
+        else:
+            collation = 'BINARY'
+        return self.visit_VARCHAR(type_) + ' COLLATE ' + collation
+    sqlalchemy.dialects.sqlite.dialect.type_compiler.visit_IBID_VARCHAR = sqlite_visit_IBID_VARCHAR
+    def sqlite_visit_IBID_TEXT(self, type_):
+        if type_.case_insensitive:
+            collation = 'NOCASE'
+        else:
+            collation = 'BINARY'
+        return self.visit_TEXT(type_) + ' COLLATE ' + collation
+    sqlalchemy.dialects.sqlite.dialect.type_compiler.visit_IBID_TEXT = sqlite_visit_IBID_TEXT
+
+    import sqlalchemy.dialects.mysql
+    def mysql_visit_IBID_VARCHAR(self, type_):
+        if type_.case_insensitive:
+            collation = 'utf8_general_ci'
+        else:
+            collation = 'utf8_bin'
+        return self.visit_VARCHAR(type_) + ' COLLATE ' + collation
+    sqlalchemy.dialects.mysql.dialect.type_compiler.visit_IBID_VARCHAR = sqlite_visit_IBID_VARCHAR
+    def mysql_visit_IBID_TEXT(self, type_):
+        if type_.case_insensitive:
+            collation = 'utf8_general_ci'
+        else:
+            collation = 'utf8_bin'
+        return self.visit_TEXT(type_) + ' COLLATE ' + collation
+    sqlalchemy.dialects.mysql.dialect.type_compiler.visit_IBID_TEXT = sqlite_visit_IBID_TEXT
+
+class IbidUnicode(_Unicode):
     "Collaiton aware Unicode"
 
-    impl = _Unicode
+    __visit_name__ = 'IBID_VARCHAR'
 
-    def __init__(self, length, **kwargs):
+    def __init__(self, length, case_insensitive=False, **kwargs):
+        self.case_insensitive = case_insensitive
         super(IbidUnicode, self).__init__(length, **kwargs)
 
-class IbidUnicodeText(_CIDecorator):
+class IbidUnicodeText(_UnicodeText):
     "Collation aware UnicodeText"
 
-    impl = _UnicodeText
+    __visit_name__ = 'IBID_TEXT'
 
-    def __init__(self, index_length=8, **kwargs):
+    def __init__(self, index_length=8, case_insensitive=False, **kwargs):
+        self.case_insensitive = case_insensitive
         self.index_length = index_length
-        super(IbidUnicodeText, self).__init__(length=None, **kwargs)
+        super(IbidUnicodeText, self).__init__(**kwargs)
+
+monkey_patch()
 
 # vi: set et sta sw=4 ts=4:

=== modified file 'ibid/db/versioned_schema.py'
--- ibid/db/versioned_schema.py	2010-06-22 00:04:13 +0000
+++ ibid/db/versioned_schema.py	2012-07-31 10:05:22 +0000
@@ -4,14 +4,11 @@
 import logging
 import re
 
-from sqlalchemy import Column, Index, UniqueConstraint, MetaData, \
-                       __version__ as _sqlalchemy_version
-from sqlalchemy.exceptions import InvalidRequestError, OperationalError, \
+from sqlalchemy import Column, Index, CheckConstraint, UniqueConstraint, \
+                       MetaData, __version__ as _sqlalchemy_version
+from sqlalchemy.exc import InvalidRequestError, OperationalError, \
                                   ProgrammingError, InternalError
-if _sqlalchemy_version < '0.5':
-    NoResultFound = InvalidRequestError
-else:
-    from sqlalchemy.orm.exc import NoResultFound
+from sqlalchemy.orm.exc import NoResultFound
 
 from ibid.db.types import Integer, IbidUnicodeText, IbidUnicode
 
@@ -19,6 +16,11 @@
 
 log = logging.getLogger('ibid.db.versioned_schema')
 
+if _sqlalchemy_version < '0.6':
+    pg_engine = 'postgres'
+else:
+    pg_engine = 'postgresql'
+
 class VersionedSchema(object):
     """For an initial table schema, set
     table.versioned_schema = VersionedSchema(__table__, 1)
@@ -77,7 +79,7 @@
                 self._create_table()
 
                 schema = Schema(unicode(self.table.name), self.version)
-                session.save_or_update(schema)
+                session.add(schema)
                 return
             Schema.__table__ = self._get_reflected_model()
 
@@ -91,7 +93,7 @@
                 self._create_table()
 
                 schema = Schema(unicode(self.table.name), self.version)
-                session.save_or_update(schema)
+                session.add(schema)
 
             elif self.version > schema.version:
                 for version in range(schema.version + 1, self.version + 1):
@@ -103,7 +105,7 @@
                     getattr(self, 'upgrade_%i_to_%i' % (version - 1, version))()
 
                     schema.version = version
-                    session.save_or_update(schema)
+                    session.add(schema)
 
                     self.upgrade_reflected_model = \
                             MetaData(session.bind, reflect=True)
@@ -126,7 +128,7 @@
 
         if session.bind.engine.name == 'sqlite':
             return 'ix_%s_%s' % (self.table.name, col.name)
-        elif session.bind.engine.name == 'postgres':
+        elif session.bind.engine.name == pg_engine:
             return '%s_%s_key' % (self.table.name, col.name)
         elif session.bind.engine.name == 'mysql':
             return col.name
@@ -175,6 +177,8 @@
                     ('constraints', old_constraints),
                     ('indexes', old_indexes)):
                 for constraint in old_list:
+                    if isinstance(constraint, CheckConstraint):
+                        continue
                     if any(True for column in constraint.columns
                             if isinstance(column.type, IbidUnicodeText)):
                         indices.append((
@@ -248,7 +252,7 @@
             query = 'ALTER TABLE "%s" ADD %s INDEX "%s" ("%s"(%i));' % (
                     self.table.name, col.unique and 'UNIQUE' or '',
                     self._index_name(col), col.name, col.type.index_length)
-        elif engine == 'postgres':
+        elif engine == pg_engine:
             # SQLAlchemy hangs if it tries to do this, because it forgets the ;
             query = 'CREATE %s INDEX "%s" ON "%s" ("%s")' % (
                     col.unique and 'UNIQUE' or '',self._index_name(col),
@@ -271,7 +275,7 @@
                 return
             raise
         except ProgrammingError, e:
-            if engine == 'postgres' and u'already exists' in unicode(e):
+            if engine == pg_engine and u'already exists' in unicode(e):
                 return
             raise
 
@@ -296,21 +300,13 @@
             raise
 
         except ProgrammingError, e:
-            if engine == 'postgres' and u'does not exist' in unicode(e):
-                return
-            # In SQLAlchemy 0.4, the InternalError below is a ProgrammingError
-            # and can't be executed in the upgrade transaction:
-            if engine == 'postgres' and u'requires' in unicode(e):
-                self.upgrade_session.bind.execute(
-                        'ALTER TABLE "%s" DROP CONSTRAINT "%s"' % (
-                        self.table.name, self._index_name(col)))
-                return
-            raise
+            if engine == pg_engine and u'does not exist' in unicode(e):
+                return
 
         # Postgres constraints can be attached to tables and can't be dropped
         # at DB level.
         except InternalError, e:
-            if engine == 'postgres':
+            if engine == pg_engine:
                 self.upgrade_session.execute(
                         'ALTER TABLE "%s" DROP CONSTRAINT "%s"' % (
                         self.table.name, self._index_name(col)))

=== modified file 'ibid/plugins/codecontest.py'
--- ibid/plugins/codecontest.py	2011-10-30 17:19:07 +0000
+++ ibid/plugins/codecontest.py	2012-07-31 10:05:22 +0000
@@ -239,7 +239,7 @@
             usaco_account[0].value = usaco_user
         else:
             account.attributes.append(Attribute('usaco_account', usaco_user))
-        event.session.save_or_update(account)
+        event.session.add(account)
         event.session.commit()
 
         event.addresponse(u'Done')

=== modified file 'ibid/plugins/factoid.py'
--- ibid/plugins/factoid.py	2011-08-12 10:04:25 +0000
+++ ibid/plugins/factoid.py	2012-07-31 10:05:22 +0000
@@ -82,7 +82,7 @@
                     .filter(FactoidName.name.like('%#_#%%', escape='#')) \
                     .all():
                 row.wild = True
-                self.upgrade_session.save_or_update(row)
+                self.upgrade_session.add(row)
         def upgrade_7_to_8(self):
             self.drop_index(self.table.c._name)
             self.alter_column(Column('name',
@@ -401,7 +401,7 @@
 
             name = FactoidName(unicode(target), event.identity)
             factoid.names.append(name)
-            event.session.save_or_update(factoid)
+            event.session.add(factoid)
             event.session.commit()
             event.addresponse(True)
             log.info(u"Added name '%s' to factoid %s (%s) by %s/%s (%s)",
@@ -607,7 +607,7 @@
             factoid = Factoid()
             fname = FactoidName(unicode(name), event.identity)
             factoid.names.append(fname)
-            event.session.save_or_update(factoid)
+            event.session.add(factoid)
             event.session.flush()
             log.info(u"Creating factoid %s with name '%s' by %s", factoid.id, fname.name, event.identity)
 
@@ -615,7 +615,7 @@
             value = '%s %s' % (verb, value)
         fvalue = FactoidValue(unicode(value), event.identity)
         factoid.values.append(fvalue)
-        event.session.save_or_update(factoid)
+        event.session.add(factoid)
         event.session.commit()
         self.last_set_factoid=factoid.names[0].name
         log.info(u"Added value '%s' to factoid %s (%s) by %s/%s (%s)",
@@ -676,7 +676,7 @@
 
             oldvalue = factoid[2].value
             factoid[2].value += suffix
-            event.session.save_or_update(factoid[2])
+            event.session.add(factoid[2])
             event.session.commit()
 
             log.info(u"Appended '%s' to value %s (%s) of factoid %s (%s) by %s/%s (%s)",
@@ -778,7 +778,7 @@
                     event.addresponse(u"That operation makes no sense. Try something like y/abcdef/ABCDEF/")
                     return
 
-            event.session.save_or_update(factoid[2])
+            event.session.add(factoid[2])
             event.session.commit()
 
             log.info(u"Applying '%s' to value %s (%s) of factoid %s (%s) by %s/%s (%s)",

=== modified file 'ibid/plugins/feeds.py'
--- ibid/plugins/feeds.py	2011-04-05 00:31:49 +0000
+++ ibid/plugins/feeds.py	2012-07-31 10:05:22 +0000
@@ -147,7 +147,7 @@
             return
 
         feed = Feed(unicode(name), unicode(url), event.identity)
-        event.session.save(feed)
+        event.session.add(feed)
         event.session.commit()
         event.addresponse(True)
         log.info(u"Added feed '%s' by %s/%s (%s): %s (Found %s entries)",

=== modified file 'ibid/plugins/fun.py'
--- ibid/plugins/fun.py	2011-03-11 21:59:06 +0000
+++ ibid/plugins/fun.py	2012-07-31 10:05:22 +0000
@@ -393,7 +393,7 @@
             raise EmptyBucketException
 
         item.carried = False
-        session.save_or_update(item)
+        session.add(item)
 
         return item
 
@@ -503,7 +503,7 @@
         if items:
             item = choice(items)
             item.carried = False
-            event.session.save_or_update(item)
+            event.session.add(item)
 
             if kind == 'owned' and yours and yours != 'your':
                 item.determiner = yours
@@ -623,6 +623,6 @@
     else:
         item = Item(object, None, event.identity)
 
-    event.session.save_or_update(item)
+    event.session.add(item)
 
 # vi: set et sta sw=4 ts=4:

=== modified file 'ibid/plugins/identity.py'
--- ibid/plugins/identity.py	2011-03-16 21:41:51 +0000
+++ ibid/plugins/identity.py	2012-07-31 10:05:22 +0000
@@ -54,7 +54,7 @@
             return
 
         account = Account(username)
-        event.session.save_or_update(account)
+        event.session.add(account)
         event.session.commit()
         log.info(u"Created account %s (%s) by %s/%s (%s)",
                 account.id, account.username, event.account, event.identity, event.sender['connection'])
@@ -64,14 +64,14 @@
                     .filter_by(identity=username, source=event.source).first()
             if identity:
                 identity.account_id = account.id
-                event.session.save_or_update(identity)
+                event.session.add(identity)
                 event.session.commit()
                 log.info(u"Attached identity %s (%s on %s) to account %s (%s)",
                         identity.id, identity.identity, identity.source, account.id, account.username)
         else:
             identity = event.session.query(Identity).get(event.identity)
             identity.account_id = account.id
-            event.session.save_or_update(identity)
+            event.session.add(identity)
             event.session.commit()
             log.info(u"Attached identity %s (%s on %s) to account %s (%s)",
                     identity.id, identity.identity, identity.source, account.id, account.username)
@@ -132,7 +132,7 @@
         oldname = account.username
         account.username = newname
 
-        event.session.save_or_update(account)
+        event.session.add(account)
         event.session.commit()
         identify_cache.clear()
 
@@ -183,12 +183,12 @@
                         return
 
                     account = Account(username)
-                    event.session.save_or_update(account)
+                    event.session.add(account)
 
                     currentidentity = event.session.query(Identity) \
                             .get(event.identity)
                     currentidentity.account_id = account.id
-                    event.session.save_or_update(currentidentity)
+                    event.session.add(currentidentity)
 
                     identify_cache.clear()
 
@@ -250,7 +250,7 @@
             if not ident:
                 ident = Identity(source, identity)
             ident.account_id = account.id
-            event.session.save_or_update(ident)
+            event.session.add(ident)
             event.session.commit()
 
             identify_cache.clear()
@@ -276,7 +276,7 @@
             if not identity:
                 identity = Identity(source, user)
             identity.account_id = account_id
-            event.session.save_or_update(identity)
+            event.session.add(identity)
             identify_cache.clear()
 
             del self.tokens[token]
@@ -308,7 +308,7 @@
             event.addresponse(u"I don't know about that identity")
         else:
             identity.account_id = None
-            event.session.save_or_update(identity)
+            event.session.add(identity)
             event.session.commit()
 
             identify_cache.clear()
@@ -344,7 +344,7 @@
                 return
 
         account.attributes.append(Attribute(name, value))
-        event.session.save_or_update(account)
+        event.session.add(account)
         event.session.commit()
 
         event.addresponse(True)
@@ -467,7 +467,7 @@
                     .first()
             if not identity:
                 identity = Identity(event.source, event.sender['id'])
-                event.session.save_or_update(identity)
+                event.session.add(identity)
                 try:
                     event.session.commit()
                     log.info(u'Created identity %s for %s on %s', identity.id, identity.identity, identity.source)
@@ -546,7 +546,7 @@
             credential = password
 
         credential = Credential(method, credential, source, account.id)
-        event.session.save_or_update(credential)
+        event.session.add(credential)
         event.session.commit()
         log.info(u"Added %s credential %s for account %s (%s) on %s by account %s",
                 method, credential.credential, account.id, account.username, source, event.account)
@@ -605,7 +605,7 @@
                 return
 
             permission.value = value
-            event.session.save_or_update(permission)
+            event.session.add(permission)
 
         event.session.commit()
         ibid.auth.drop_caches()

=== modified file 'ibid/plugins/karma.py'
--- ibid/plugins/karma.py	2011-01-23 13:19:55 +0000
+++ ibid/plugins/karma.py	2012-07-31 10:05:22 +0000
@@ -133,7 +133,7 @@
 
             event.session.delete(karma)
         else:
-            event.session.save_or_update(karma)
+            event.session.add(karma)
         event.session.commit()
 
         log.info(u"%s karma for '%s' by %s/%s (%s) because: %s",
@@ -168,11 +168,18 @@
 
     @match(r'^(reverse\s+)?karmaladder$')
     def ladder(self, event, reverse):
-        karmas = event.session.query(Karma) \
-                .order_by(reverse and Karma.value.asc() or Karma.value.desc()) \
-                .limit(30).all()
+        karmas = event.session.query(Karma)
+        if reverse:
+            karmas = karmas.order_by(Karma.value.asc())
+        else:
+            karmas = karmas.order_by(Karma.value.desc())
+        karmas = karmas.limit(30).all()
+
         if karmas:
-            event.addresponse(', '.join(['%s: %s (%s)' % (karmas.index(karma), karma.subject, karma.value) for karma in karmas]))
+            event.addresponse(u', '.join(
+                u'%s: %s (%s)'
+                % (karmas.index(karma), karma.subject, karma.value)
+                for karma in karmas))
         else:
             event.addresponse(u"I don't really care about anything")
 

=== modified file 'ibid/plugins/memo.py'
--- ibid/plugins/memo.py	2011-03-23 16:11:13 +0000
+++ ibid/plugins/memo.py	2012-07-31 10:05:22 +0000
@@ -118,7 +118,7 @@
                 event.addresponse(u'I am not connected to %s', source)
                 return
             to = Identity(source, who)
-            event.session.save(to)
+            event.session.add(to)
             event.session.commit()
 
             log.info(u"Created identity %s for %s on %s", to.id, to.identity,
@@ -133,7 +133,7 @@
 
         memo = Memo(event.identity, to.id, memo,
                     how.lower() in (u'pm', u'privmsg', u'msg'))
-        event.session.save_or_update(memo)
+        event.session.add(memo)
 
         event.session.commit()
         log.info(u"Stored memo %s for %s (%s) from %s (%s): %s",
@@ -286,7 +286,7 @@
                 event.addresponse(message)
 
             memo.delivered = True
-            event.session.save_or_update(memo)
+            event.session.add(memo)
             event.session.commit()
             log.info(u"Delivered memo %s to %s (%s)",
                     memo.id, event.identity, event.sender['connection'])

=== modified file 'ibid/plugins/seen.py'
--- ibid/plugins/seen.py	2010-03-27 15:50:58 +0000
+++ ibid/plugins/seen.py	2012-07-31 10:05:22 +0000
@@ -86,7 +86,7 @@
         sighting.time = event.time
         sighting.count = sighting.count + 1
 
-        event.session.save_or_update(sighting)
+        event.session.add(sighting)
         try:
             event.session.commit()
         except IntegrityError:

=== modified file 'ibid/plugins/urlgrab.py'
--- ibid/plugins/urlgrab.py	2011-01-26 20:28:17 +0000
+++ ibid/plugins/urlgrab.py	2012-07-31 10:05:22 +0000
@@ -76,7 +76,7 @@
                 url = 'http://%s' % url
 
         u = URL(url, event.channel, event.identity)
-        event.session.save_or_update(u)
+        event.session.add(u)
 
         if self.service and self.username:
             self._post_url(event, url)

=== modified file 'ibid/test/__init__.py'
--- ibid/test/__init__.py	2011-08-14 15:26:08 +0000
+++ ibid/test/__init__.py	2012-07-31 10:05:22 +0000
@@ -10,6 +10,8 @@
 import sys
 import tempfile
 
+import sqlalchemy
+
 from twisted.python import log
 from twisted.python.modules import getModule
 from twisted.trial import unittest
@@ -103,6 +105,13 @@
 
     def setUp(self):
         super(PluginTestCase, self).setUp()
+        if sqlalchemy.__version__ > '0.6.0':
+            raise unittest.SkipTest(
+                    "PluginTestCase doesn't work with SQLAlchemy 0.6")
+        if self.network and os.getenv('IBID_NETWORKLESS_TEST') is not None:
+            raise unittest.SkipTest('test uses network')
+
+        ibid.config = FileConfig(locate_resource('ibid.test', 'test.ini'))
 
         if self.load_configured is None:
             self.load_configured = not self.load
@@ -127,7 +136,7 @@
         session = ibid.databases.ibid()
 
         self.identity = Identity(self.source, self.username)
-        session.save(self.identity)
+        session.add(self.identity)
         session.commit()
         self.identity = session.query(Identity) \
             .filter_by(identity=self.username).one()

=== modified file 'scripts/ibid-factpack'
--- scripts/ibid-factpack	2010-02-16 09:41:11 +0000
+++ scripts/ibid-factpack	2012-07-31 10:05:22 +0000
@@ -85,7 +85,7 @@
     exit(5)
 
 factpack = Factpack(name)
-session.save(factpack)
+session.add(factpack)
 session.flush()
 
 existing = []
@@ -102,7 +102,7 @@
         fvalue = FactoidValue(unicode(value), None, factpack=factpack.id)
         factoid.values.append(fvalue)
     if len(factoid.names) > 0:
-        session.save(factoid)
+        session.add(factoid)
 
 if existing and not options.skip:
     print >> stderr, u'The following factoids already exist in the database. ' \

=== modified file 'scripts/ibid-knab-import'
--- scripts/ibid-knab-import	2011-02-20 18:31:50 +0000
+++ scripts/ibid-knab-import	2012-07-31 10:05:22 +0000
@@ -130,7 +130,7 @@
     if not identity:
         identity = Identity(source, user)
         identity.created = created
-        session.save(identity)
+        session.add(identity)
         session.flush()
     elif identity.created > created:
         identity.created = created
@@ -187,7 +187,7 @@
         fvalue.time = kfactoid.utc_time
 
         factoid.values.append(fvalue)
-        ibid.save_or_update(factoid)
+        ibid.add(factoid)
 
     ibid.commit()
 
@@ -297,7 +297,7 @@
         config.merge(FileConfig('local.ini'))
 
     ibidengine = create_engine(config.databases['ibid'], encoding='utf-8')
-    IbidSession = sessionmaker(bind=ibidengine, transactional=False)
+    IbidSession = sessionmaker(bind=ibidengine, autocommit=True)
     ibid = IbidSession()
 
     import_factoids(knab, ibid, source)

=== modified file 'scripts/ibid-plugin'
--- scripts/ibid-plugin	2011-03-15 09:56:25 +0000
+++ scripts/ibid-plugin	2012-07-31 10:05:22 +0000
@@ -102,7 +102,7 @@
 identity = session.query(Identity).filter_by(identity=username, source=u'test_source').first()
 if not identity:
     identity = Identity(u'test_source',username)
-    session.save(identity)
+    session.add(identity)
     session.commit()
     identity = session.query(Identity).filter_by(identity=username).first()
 identity_id = identity.id

=== modified file 'scripts/ibid-setup'
--- scripts/ibid-setup	2011-02-25 23:43:15 +0000
+++ scripts/ibid-setup	2012-07-31 10:05:22 +0000
@@ -158,8 +158,8 @@
 credential = Credential(u'password', hash(unicode(pass1)))
 account.credentials.append(credential)
 
-session.save_or_update(account)
-session.save_or_update(identity)
+session.add(account)
+session.add(identity)
 session.commit()
 session.close()
 

=== modified file 'setup.py'
--- setup.py	2010-09-26 22:55:15 +0000
+++ setup.py	2012-07-31 10:05:22 +0000
@@ -18,7 +18,7 @@
     'PyStemmer',
     'python-dateutil',
     'SOAPpy',
-    'SQLAlchemy>=0.5,<0.6a', # Works with >=0.4.6 except on OS X
+    'SQLAlchemy>=0.6',
     'Twisted',
     'wokkel>=0.6.3',
 ]