Merge lp:~iffy/storm/lazycolumns into lp:storm

[Matt Haggard]

Implemented the lazy-loading feature as described in the TODO file. Instead of having separate "lazy" and "lazy_group" arguments, however, I opted for just requiring "lazy" which can be one of the following:
 - True: The column should be lazy all by itself.
 - True-ish values: The column should be grouped with all other columns using this same value. If any of them are accessed, all of them are loaded.

Rationale: We need this to speed up access of our bad tables with huge text fields that are rarely used.

[Robert Collins (lifeless)]

I'm glad you are working on this.

I have not done a full review at this point; I'd like to clarify
something though - it looks like you are making it so that all queries
will assume these columns are lazy. It is more efficient to grab the
column if needed in a single query rather than one-per-object. It
would be nice if users had the ability to control that.

For instance, say you have a changelog field in a table, which is a
big text field. Most pages in the website may not need it, but some
do. If its marked lazy, the pages that do need it will end up doing
(number of changelogs shown) separate queries to lazy-load the
changelog. If there is someway to say 'for this query, the changelog
field is needed', then that overhead can be eliminated.

And, if we have such a way to force laziness-into-eagerness, we
probably can do the reverse trivially - force eagerness into laziness
to prevent a bunch of fields being loaded on a query by query basis.

What do you think?

-Rob

[Matt Haggard (iffy)]

Rob,

I know what you mean. I'm working on this to fix a problem we currently have with a big table (126 fields). 90% of the time, we need about 4 fields; 10% of the time we need some combination of the others.

My current patch allows you to define column laziness globally, which improves 90% of our code. What you're suggesting is that in addition to (instead of?) defining laziness globally, you can override laziness when using a store.find, store.get, Reference or ReferenceSet? This would allow one to avoid double queries for the other 10%.

What would the syntax look like?

store.find(Foo, eager=(Foo.name, Foo.kind))
bar = Reference(bar_id, 'Bar.id', eager=('Bar.name', 'Bar.kind'))

Or would you do some kind of state thing (this tastes bad)?

Bar.eagers.push('name', 'kind')
store.find(Bar)
Bar.eagers.pop()

Maybe this is better:

store.eager_load(Bar.name, Bar.kind)
store.find(Bar)
...
store.lazy_load(Bar.name, Bar.kind)

Also, is laziness overriding (your suggestion) something that should be built in from the beginning, or can it be added as a new feature after global laziness (my patch) is added? Maybe the answer depends on the implementation.

Thanks,

Matt

[Jamu Kakar (jkakar)]

There are some notes in the TODO file that you might draw inspiration
from.

[Matt Haggard (iffy)]

> There are some notes in the TODO file that you might draw inspiration
> from.

Yes, those are great notes! This branch implements laziness as described there.

[Robert Collins (lifeless)]

On Wed, Jan 4, 2012 at 5:33 AM, Matt Haggard <email address hidden> wrote:
> Rob,
> Also, is laziness overriding (your suggestion) something that should be built in from the beginning, or can it be added as a new feature after global laziness (my patch) is added?  Maybe the answer depends on the implementation.

I think adding it later is fine, but we should consider what it needs
early, so that we don't conflict with the global laziness thing, which
will itself be useful.

As for syntax, perhaps
# do not load attribute1 or attribute2, and load attribute3 which is
globally set to lazy
store.find(Load(Foo, lazy=['attribute1', 'attribute2'], eager=['attribute3'])) ?

[Matt Haggard (iffy)]

So what's the next step on this?

[Robert Collins (lifeless)]

On Fri, Jan 6, 2012 at 11:19 AM, Matt Haggard <email address hidden> wrote:
> So what's the next step on this?

Well, I think for you to spend a little time coming up with an answer to
'does the patch need to change to gracefully permit per-field laziness
in future'

if the answer is no, then we review and land; if it is yes, you make
such changes as you consider necessary to future proof, then we review
and land.

-Rob

[Matt Haggard (iffy)]

Rob,

No, this patch doesn't need to change to support per-query laziness in the future. Per-query laziness would use some of the stuff in this patch as it is. The overlap point is probably the LazyGroupMember class, which would probably be changed to Unloaded, NotLoadedYet, NotFetchedYet or Unfetched or something like that. But that's used internally and not in the public API.

Thanks for your help on this!

Matt

[Gustavo Niemeyer (niemeyer)]

Thanks a lot for pushing this forward.

Given the question on how the syntax should look like, I'm concerned the
spirit described in the notes isn't being implemented. It doesn't seem
practical to define per field granularity for what should be loaded.

I'll have a look at the branch, hopefully next week.
On Jan 3, 2012 2:33 PM, "Matt Haggard" <email address hidden> wrote:

> Rob,
>
> I know what you mean. I'm working on this to fix a problem we currently
> have with a big table (126 fields). 90% of the time, we need about 4
> fields; 10% of the time we need some combination of the others.
>
> My current patch allows you to define column laziness globally, which
> improves 90% of our code. What you're suggesting is that in addition to
> (instead of?) defining laziness globally, you can override laziness when
> using a store.find, store.get, Reference or ReferenceSet? This would allow
> one to avoid double queries for the other 10%.
>
> What would the syntax look like?
>
> store.find(Foo, eager=(Foo.name, Foo.kind))
> bar = Reference(bar_id, 'Bar.id', eager=('Bar.name', 'Bar.kind'))
>
> Or would you do some kind of state thing (this tastes bad)?
>
> Bar.eagers.push('name', 'kind')
> store.find(Bar)
> Bar.eagers.pop()
>
> Maybe this is better:
>
> store.eager_load(Bar.name, Bar.kind)
> store.find(Bar)
> ...
> store.lazy_load(Bar.name, Bar.kind)
>
> Also, is laziness overriding (your suggestion) something that should be
> built in from the beginning, or can it be added as a new feature after
> global laziness (my patch) is added? Maybe the answer depends on the
> implementation.
>
>
> Thanks,
>
> Matt
> --
> https://code.launchpad.net/~magmatt/storm/lazycolumns/+merge/87092
> You are subscribed to branch lp:storm.
>

[Matt Haggard (iffy)]

Gustavo,

I'm excited for your review.

To clarify, this branch implements laziness as described in the TODO. Rob mentioned that it would be nice to sometimes override that behavior per query. That (overriding per query) is not implemented here. So the syntax closely mirrors the TODO.

A single, lazy attr:

    class C(object):
        ...
        attr = Unicode(lazy=True)

or a group of lazy attrs (loaded together if either is accessed):

    class C(object):
        ...
        attr = Unicode(lazy=2)
        attr2 = Unicode(lazy=2)

    class D(object):
        ...
        attr = Int(lazy='group 1')
        attr2 = Int(lazy='group 2')

^ Is the syntax supported by this branch. I opted for a single "lazy" arg instead of having a "lazy" and "lazy_group." It seems simpler to me.

Unmerged revisions

431. By Matt Haggard on 2011-12-29

Added docstring for lazy attribute in expr.Column and removed laziness from the
TODO file

430. By Matt Haggard on 2011-12-29

Lazy-loaded columns will not overwrite a previously set value.

429. By Matt Haggard on 2011-12-29

Finished lazy-loading of columns including some Store tests.

428. By Matt Haggard on 2011-12-28

Added lazy keyword as __init__ to Property and changed ClassInfo to
gather appropriate laziness information from that.

=== modified file 'TODO'
--- TODO	2008-10-29 15:49:19 +0000
+++ TODO	2011-12-29 16:23:23 +0000
@@ -7,32 +7,6 @@
 
 - Unicode(autoreload=True) will mark the field as autoreload by default.
 
-- Lazy-by-default attributes:
-
-  class C(object):
-      ...
-      attr = Unicode(lazy=True)
-
-  This would make attr be loaded only if touched.
-
-  Or maybe lazy groups:
-
-  class C(object):
-      ...
-      lazy_group = LazyGroup()
-      attr = Unicode(lazy=True, lazy_group=lazy_group)
-
-  Once one of the group attributes are accessed all of them are retrieved
-  at the same time.
-
-  Lazy groups may be integers as well:
-
-  class C(object):
-      ...
-      attr = Unicode(lazy_group=1)
-
-  lazy_group=None means not lazy.
-
 - Implement ResultSet.reverse[d]() to invert order_by()?
 
 - Add support to cyclic references when all of elements of the cycle are

=== modified file 'storm/expr.py'
--- storm/expr.py	2011-09-15 13:07:55 +0000
+++ storm/expr.py	2011-12-29 16:23:23 +0000
@@ -799,15 +799,21 @@
         a bool.
     @ivar variable_factory: Factory producing C{Variable} instances typed
         according to this column.
+    @ivar lazy: Anything trueish indicates that this column should not be
+        loaded from the database when an object is fetched.  If this is
+        C{True} the column will be loaded on first access.  If the value is
+        an integer or string, then all columns with the same value will be
+        loaded when any of them is first accessed.
     """
-    __slots__ = ("name", "table", "primary", "variable_factory",
+    __slots__ = ("name", "table", "primary", "lazy", "variable_factory",
                  "compile_cache", "compile_id")
 
     def __init__(self, name=Undef, table=Undef, primary=False,
-                 variable_factory=None):
+                 variable_factory=None, lazy=None):
         self.name = name
         self.table = table
         self.primary = int(primary)
+        self.lazy = lazy
         self.variable_factory = variable_factory or Variable
         self.compile_cache = None
         self.compile_id = None

=== modified file 'storm/info.py'
--- storm/info.py	2011-08-14 08:55:15 +0000
+++ storm/info.py	2011-12-29 16:23:23 +0000
@@ -62,11 +62,16 @@
     @ivar table: Expression from where columns will be looked up.
     @ivar cls: Class which should be used to build objects.
     @ivar columns: Tuple of column properties found in the class.
+    @ivar eager_columns: Tuple of columns that are not lazy.
+    @ivar lazy_columns: Tuple of columns that are loaded lazily.
+    @ivar lazy_groups: Dictionary whose keys are columns and values are lists
+        of columns that should be lazily loaded together.
     @ivar primary_key: Tuple of column properties used to form the primary key
     @ivar primary_key_pos: Position of primary_key items in the columns tuple.
     """
 
     def __init__(self, cls):
+        from storm.properties import Group
         self.table = getattr(cls, "__storm_table__", None)
         if self.table is None:
             raise ClassInfoError("%s.__storm_table__ missing" % repr(cls))
@@ -81,11 +86,28 @@
             column = getattr(cls, attr, None)
             if isinstance(column, Column):
                 pairs.append((attr, column))
+                                            
 
         pairs.sort()
 
         self.columns = tuple(pair[1] for pair in pairs)
         self.attributes = dict(pairs)
+        self.eager_columns = tuple(pair[1] for pair in pairs
+                                   if not(pair[1].lazy))
+        self.lazy_columns = tuple(pair[1] for pair in pairs
+                                  if pair[1].lazy)
+        
+        common_laziness = {}
+        self.lazy_groups = {}
+        for column in self.lazy_columns:
+            if column.lazy is True:
+                # lazy alone
+                self.lazy_groups[column] = [column]
+            else:
+                # lazy, perhaps with some other columns
+                common = common_laziness.setdefault(column.lazy, [])
+                common.append(column)
+                self.lazy_groups[column] = common
 
         storm_primary = getattr(cls, "__storm_primary__", None)
         if storm_primary is not None:

=== modified file 'storm/properties.py'
--- storm/properties.py	2011-02-28 21:16:29 +0000
+++ storm/properties.py	2011-12-29 16:23:23 +0000
@@ -36,17 +36,18 @@
 __all__ = ["Property", "SimpleProperty",
            "Bool", "Int", "Float", "Decimal", "RawStr", "Unicode",
            "DateTime", "Date", "Time", "TimeDelta", "UUID", "Enum",
-           "Pickle", "JSON", "List", "PropertyRegistry"]
+           "Pickle", "JSON", "List", "PropertyRegistry", "Group"]
 
 
 class Property(object):
 
     def __init__(self, name=None, primary=False,
-                 variable_class=Variable, variable_kwargs={}):
+                 variable_class=Variable, variable_kwargs={}, lazy=None):
         self._name = name
         self._primary = primary
         self._variable_class = variable_class
         self._variable_kwargs = variable_kwargs
+        self._lazy = lazy
 
     def __get__(self, obj, cls=None):
         if obj is None:
@@ -100,6 +101,7 @@
             else:
                 name = self._name
             column = PropertyColumn(self, cls, attr, name, self._primary,
+                                    self._lazy,
                                     self._variable_class,
                                     self._variable_kwargs)
             cls._storm_columns[self] = column
@@ -108,12 +110,13 @@
 
 class PropertyColumn(Column):
 
-    def __init__(self, prop, cls, attr, name, primary,
+    def __init__(self, prop, cls, attr, name, primary, lazy,
                  variable_class, variable_kwargs):
         Column.__init__(self, name, cls, primary,
                         VariableFactory(variable_class, column=self,
                                         validator_attribute=attr,
-                                        **variable_kwargs))
+                                        **variable_kwargs),
+                        lazy)
 
         self.cls = cls # Used by references
 
@@ -127,10 +130,11 @@
 
     variable_class = None
 
-    def __init__(self, name=None, primary=False, **kwargs):
+    def __init__(self, name=None, primary=False, lazy=None, **kwargs):
         kwargs["value"] = kwargs.pop("default", Undef)
         kwargs["value_factory"] = kwargs.pop("default_factory", Undef)
-        Property.__init__(self, name, primary, self.variable_class, kwargs)
+        Property.__init__(self, name, primary, self.variable_class, kwargs,
+                          lazy)
 
 
 class Bool(SimpleProperty):
@@ -340,3 +344,13 @@
             self._storm_property_registry = PropertyRegistry()
         elif hasattr(self, "__storm_table__"):
             self._storm_property_registry.add_class(self)
+
+
+
+class Group(object):
+    """I am used to group L{Property}s to be lazily loaded together.
+    """
+
+
+
+

=== modified file 'storm/store.py'
--- storm/store.py	2011-05-16 10:45:52 +0000
+++ storm/store.py	2011-12-29 16:23:23 +0000
@@ -42,7 +42,7 @@
 from storm.event import EventSystem
 
 
-__all__ = ["Store", "AutoReload", "EmptyResultSet"]
+__all__ = ["Store", "AutoReload", "EmptyResultSet", "LazyGroupMember"]
 
 
 PENDING_ADD = 1
@@ -172,7 +172,7 @@
 
         where = compare_columns(cls_info.primary_key, primary_vars)
 
-        select = Select(cls_info.columns, where,
+        select = Select(cls_info.eager_columns, where,
                         default_tables=cls_info.table, limit=1)
 
         result = self._connection.execute(select)
@@ -667,7 +667,7 @@
 
         # Prepare cache key.
         primary_vars = []
-        columns = cls_info.columns
+        columns = cls_info.eager_columns + cls_info.lazy_columns
 
         for value in values:
             if value is not None:
@@ -677,6 +677,8 @@
             # wasn't found.  This is useful for joins, where non-existent
             # rows are represented like that.
             return None
+        
+        values += tuple([LazyGroupMember] * len(cls_info.lazy_columns))
 
         for i in cls_info.primary_key_pos:
             value = values[i]
@@ -694,7 +696,7 @@
 
             # Take that chance and fill up any undefined variables
             # with fresh data, since we got it anyway.
-            self._set_values(obj_info, cls_info.columns, result,
+            self._set_values(obj_info, columns, result,
                              values, keep_defined=True)
 
             # We're not sure if the obj is still in memory at this
@@ -707,7 +709,7 @@
             obj_info = get_obj_info(obj)
             obj_info["store"] = self
 
-            self._set_values(obj_info, cls_info.columns, result, values,
+            self._set_values(obj_info, columns, result, values,
                              replace_unknown_lazy=True)
 
             self._add_to_alive(obj_info)
@@ -750,7 +752,8 @@
             variable = obj_info.variables[column]
             lazy_value = variable.get_lazy()
             is_unknown_lazy = not (lazy_value is None or
-                                   lazy_value is AutoReload)
+                                   lazy_value is AutoReload or
+                                   lazy_value is LazyGroupMember)
             if keep_defined:
                 if variable.is_defined() or is_unknown_lazy:
                     continue
@@ -871,6 +874,14 @@
         the store, and then set all variables set to AutoReload to
         their database values.
         """
+        if lazy_value is LazyGroupMember:
+            fellow_columns = obj_info.cls_info.lazy_groups[variable.column]
+            for column in fellow_columns:
+                if obj_info.variables[column].get_lazy() is LazyGroupMember:
+                    obj_info.variables[column].set(AutoReload)
+            # now that it's an AutoReload, resolve the lazy value
+            return self._resolve_lazy_value(obj_info, variable, AutoReload)
+        
         if lazy_value is not AutoReload and not isinstance(lazy_value, Expr):
             # It's not something we handle.
             return
@@ -1678,7 +1689,7 @@
                 if isinstance(info, Column):
                     default_tables.append(info.table)
             else:
-                columns.extend(info.columns)
+                columns.extend(info.eager_columns)
                 default_tables.append(info.table)
         return columns, default_tables
 
@@ -1706,7 +1717,7 @@
                     value=values[values_start], from_db=True)
                 objects.append(variable.get())
             else:
-                values_end += len(info.columns)
+                values_end += len(info.eager_columns)
                 obj = store._load_object(info, result,
                                          values[values_start:values_end])
                 objects.append(obj)
@@ -1809,3 +1820,21 @@
     pass
 
 AutoReload = AutoReload()
+
+
+class LazyGroupMember(LazyValue):
+    """A marker for indicating that this value is loaded on first access.
+    
+    This is used internally when you make a property lazy, as in::
+    
+        class Person(object):
+            __storm_table__ = "person"
+            id = Int(primary=True)
+            name = Unicode(lazy=True)
+    
+    When a C{Person} is retrieved from the store, it's C{name} property
+    will be set to L{LazyGroupMember}.
+    """
+    pass
+
+LazyGroupMember = LazyGroupMember()
\ No newline at end of file

=== modified file 'tests/info.py'
--- tests/info.py	2011-12-07 11:57:07 +0000
+++ tests/info.py	2011-12-29 16:23:23 +0000
@@ -22,7 +22,7 @@
 import gc
 
 from storm.exceptions import ClassInfoError
-from storm.properties import Property
+from storm.properties import Property, Group
 from storm.variables import Variable
 from storm.expr import Undef, Select, compile
 from storm.info import *
@@ -168,6 +168,40 @@
         cls_info = ClassInfo(Class)
         self.assertEquals(cls_info.primary_key_pos, (2, 0))
 
+    def test_eager_columns(self):
+        class Class(object):
+            __storm_table__ = 'table'
+            prop1 = Property(primary=True)
+            prop2 = Property()
+            prop3 = Property(lazy=True)
+            prop4 = Property(lazy=1)
+            prop5 = Property(lazy=Group())
+        cls_info = ClassInfo(Class)
+        self.assertEquals(cls_info.eager_columns, (Class.prop1, Class.prop2))
+
+    def test_lazy_columns(self):
+        class Class(object):
+            __storm_table__ = 'table'
+            prop1 = Property(primary=True)
+            prop2 = Property()
+            prop3 = Property(lazy=True)
+            prop4 = Property(lazy=1)
+            prop5 = Property(lazy=Group())
+            prop6 = Property(lazy=1)
+        cls_info = ClassInfo(Class)
+        self.assertEquals(cls_info.lazy_columns,
+                          (Class.prop3, Class.prop4, Class.prop5, Class.prop6))
+        self.assertEqual(cls_info.lazy_groups[Class.prop3],
+                         [Class.prop3])
+        self.assertEqual(cls_info.lazy_groups[Class.prop4],
+                         [Class.prop4, Class.prop6])
+        self.assertEqual(cls_info.lazy_groups[Class.prop5],
+                         [Class.prop5])
+        self.assertEqual(cls_info.lazy_groups[Class.prop6],
+                         cls_info.lazy_groups[Class.prop4])
+                        
+            
+
 
 class ObjectInfoTest(TestHelper):
 

=== modified file 'tests/store/base.py'
--- tests/store/base.py	2011-08-01 12:54:36 +0000
+++ tests/store/base.py	2011-12-29 16:23:23 +0000
@@ -42,7 +42,8 @@
     NoStoreError, NotFlushedError, NotOneError, OrderLoopError, UnorderedError,
     WrongStoreError, DisconnectionError)
 from storm.cache import Cache
-from storm.store import AutoReload, EmptyResultSet, Store, ResultSet
+from storm.store import (
+    AutoReload, EmptyResultSet, Store, ResultSet, LazyGroupMember)
 from storm.tracer import debug
 
 from tests.info import Wrapper
@@ -130,6 +131,14 @@
     id = Int(primary=True)
     value = Decimal()
 
+class Sloth(object):
+    __storm_table__ = "sloth"
+    id = Int(primary=True)
+    val1 = Int()
+    val2 = Int(lazy=2)
+    val3 = Int(lazy=2)
+    val4 = Int(lazy=True)
+
 
 class DecorateVariable(Variable):
 
@@ -244,6 +253,8 @@
                            " VALUES (8, 20, 1, 4)")
         connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
                            " VALUES (9, 20, 1, 2)")
+        connection.execute("INSERT INTO sloth (id, val1, val2, val3, val4)"
+                           " VALUES (1, 1, 2, 3, 4)")
 
         connection.commit()
 
@@ -5291,6 +5302,64 @@
         self.assertEquals(lazy_value, AutoReload)
         self.assertEquals(foo.title, u"Default Title")
 
+    def test_lazy_loading_find(self):
+        """
+        Accessing a lazily-loaded attribute will cause it to load as well as
+        any other attribute in its group.
+        """
+        sloth = self.store.find(Sloth, id=1).one()
+        variables = get_obj_info(sloth).variables
+        self.assertEqual(variables[Sloth.val2].get_lazy(), LazyGroupMember)
+        self.assertEqual(variables[Sloth.val3].get_lazy(), LazyGroupMember)
+        self.assertEqual(variables[Sloth.val4].get_lazy(), LazyGroupMember)
+        val2 = sloth.val2
+        # val2 and val3 are in a group together
+        self.assertEqual(variables[Sloth.val2].get_lazy(), None)
+        self.assertEqual(variables[Sloth.val3].get_lazy(), None)
+        self.assertEqual(variables[Sloth.val4].get_lazy(), LazyGroupMember)
+        self.assertEqual(val2, 2)
+        self.assertEqual(sloth.val3, 3)
+        val4 = sloth.val4
+        self.assertEqual(variables[Sloth.val4].get_lazy(), None)
+        self.assertEqual(val4, 4)
+
+    def test_lazy_loading_get(self):
+        """
+        get() should honor lazy attributes and not get them
+        """
+        sloth = self.store.get(Sloth, 1)
+        variables = get_obj_info(sloth).variables
+        self.assertEqual(variables[Sloth.val2].get_lazy(), LazyGroupMember)
+        self.assertEqual(variables[Sloth.val3].get_lazy(), LazyGroupMember)
+        self.assertEqual(variables[Sloth.val4].get_lazy(), LazyGroupMember)
+        val2 = sloth.val2
+        # val2 and val3 are in a group together
+        self.assertEqual(variables[Sloth.val2].get_lazy(), None)
+        self.assertEqual(variables[Sloth.val3].get_lazy(), None)
+        self.assertEqual(variables[Sloth.val4].get_lazy(), LazyGroupMember)
+        self.assertEqual(val2, 2)
+        self.assertEqual(sloth.val3, 3)
+        val4 = sloth.val4
+        self.assertEqual(variables[Sloth.val4].get_lazy(), None)
+        self.assertEqual(val4, 4)
+
+    def test_lazy_loading_set(self):
+        """
+        Setting an attribute should not cause a group to load, but should
+        prevent that value from being overwritten by a later load.
+        """
+        sloth = self.store.find(Sloth, id=1).one()
+        sloth.val2 = 200
+        variables = get_obj_info(sloth).variables
+        self.assertEqual(variables[Sloth.val2].get_lazy(), None)
+        self.assertEqual(variables[Sloth.val3].get_lazy(), LazyGroupMember,
+                         "The other column in this lazy group should not yet "
+                         "be loaded.")
+        val3 = sloth.val3
+        self.assertEqual(val3, 3)
+        self.assertEqual(sloth.val2, 200, "Attribute should not be overwritten"
+                         " by database value: %r" % sloth.val2)
+
     def test_reference_break_on_local_diverged_doesnt_autoreload(self):
         foo = self.store.get(Foo, 10)
         self.store.autoreload(foo)

=== modified file 'tests/store/mysql.py'
--- tests/store/mysql.py	2011-02-14 12:17:54 +0000
+++ tests/store/mysql.py	2011-12-29 16:23:23 +0000
@@ -79,6 +79,11 @@
         connection.execute("CREATE TABLE unique_id "
                            "(id VARCHAR(36) PRIMARY KEY) "
                            "ENGINE=InnoDB")
+        connection.execute("CREATE TABLE sloth "
+                           "(id INT PRIMARY KEY AUTO_INCREMENT,"
+                           " val1 INTEGER, val2 INTEGER, val3 INTEGER,"
+                           " val4 INTEGER) "
+                           "ENGINE=InnoDB")
         connection.commit()
 
 

=== modified file 'tests/store/postgres.py'
--- tests/store/postgres.py	2011-02-14 12:17:54 +0000
+++ tests/store/postgres.py	2011-12-29 16:23:23 +0000
@@ -93,6 +93,9 @@
                            " value1 INTEGER, value2 INTEGER)")
         connection.execute("CREATE TABLE unique_id "
                            "(id UUID PRIMARY KEY)")
+        connection.execute("CREATE TABLE sloth "
+                           "(id SERIAL PRIMARY KEY, val1 INTEGER,"
+                           " val2 INTEGER, val3 INTEGER, val4 INTEGER)")
         connection.commit()
 
     def drop_tables(self):

=== modified file 'tests/store/sqlite.py'
--- tests/store/sqlite.py	2011-02-14 12:17:54 +0000
+++ tests/store/sqlite.py	2011-12-29 16:23:23 +0000
@@ -65,6 +65,9 @@
                            " value1 INTEGER, value2 INTEGER)")
         connection.execute("CREATE TABLE unique_id "
                            "(id VARCHAR PRIMARY KEY)")
+        connection.execute("CREATE TABLE sloth "
+                           "(id INTEGER PRIMARY KEY, val1 INTEGER,"
+                           " val2 INTEGER, val3 INTEGER, val4 INTEGER)")
         connection.commit()
 
     def drop_tables(self):