Merge lp:~iffy/storm/lazycolumns into lp:storm

Proposed by Matt Haggard on 2011-12-29
Status: Needs review
Proposed branch: lp:~iffy/storm/lazycolumns
Merge into: lp:storm
Diff against target: 516 lines (+203/-44)
10 files modified
TODO (+0/-26)
storm/expr.py (+8/-2)
storm/info.py (+22/-0)
storm/properties.py (+20/-6)
storm/store.py (+37/-8)
tests/info.py (+35/-1)
tests/store/base.py (+70/-1)
tests/store/mysql.py (+5/-0)
tests/store/postgres.py (+3/-0)
tests/store/sqlite.py (+3/-0)
To merge this branch: bzr merge lp:~iffy/storm/lazycolumns
Reviewer Review Type Date Requested Status
Storm Developers 2011-12-29 Pending
Review via email: mp+87092@code.launchpad.net

Description of the change

Implemented the lazy-loading feature as described in the TODO file. Instead of having separate "lazy" and "lazy_group" arguments, however, I opted for just requiring "lazy" which can be one of the following:
 - True: The column should be lazy all by itself.
 - True-ish values: The column should be grouped with all other columns using this same value. If any of them are accessed, all of them are loaded.

Rationale: We need this to speed up access of our bad tables with huge text fields that are rarely used.

To post a comment you must log in.
Robert Collins (lifeless) wrote :

I'm glad you are working on this.

I have not done a full review at this point; I'd like to clarify
something though - it looks like you are making it so that all queries
will assume these columns are lazy. It is more efficient to grab the
column if needed in a single query rather than one-per-object. It
would be nice if users had the ability to control that.

For instance, say you have a changelog field in a table, which is a
big text field. Most pages in the website may not need it, but some
do. If its marked lazy, the pages that do need it will end up doing
(number of changelogs shown) separate queries to lazy-load the
changelog. If there is someway to say 'for this query, the changelog
field is needed', then that overhead can be eliminated.

And, if we have such a way to force laziness-into-eagerness, we
probably can do the reverse trivially - force eagerness into laziness
to prevent a bunch of fields being loaded on a query by query basis.

What do you think?

-Rob

Matt Haggard (iffy) wrote :

Rob,

I know what you mean. I'm working on this to fix a problem we currently have with a big table (126 fields). 90% of the time, we need about 4 fields; 10% of the time we need some combination of the others.

My current patch allows you to define column laziness globally, which improves 90% of our code. What you're suggesting is that in addition to (instead of?) defining laziness globally, you can override laziness when using a store.find, store.get, Reference or ReferenceSet? This would allow one to avoid double queries for the other 10%.

What would the syntax look like?

store.find(Foo, eager=(Foo.name, Foo.kind))
bar = Reference(bar_id, 'Bar.id', eager=('Bar.name', 'Bar.kind'))

Or would you do some kind of state thing (this tastes bad)?

Bar.eagers.push('name', 'kind')
store.find(Bar)
Bar.eagers.pop()

Maybe this is better:

store.eager_load(Bar.name, Bar.kind)
store.find(Bar)
...
store.lazy_load(Bar.name, Bar.kind)

Also, is laziness overriding (your suggestion) something that should be built in from the beginning, or can it be added as a new feature after global laziness (my patch) is added? Maybe the answer depends on the implementation.

Thanks,

Matt

Jamu Kakar (jkakar) wrote :

There are some notes in the TODO file that you might draw inspiration
from.

Matt Haggard (iffy) wrote :

> There are some notes in the TODO file that you might draw inspiration
> from.

Yes, those are great notes! This branch implements laziness as described there.

Robert Collins (lifeless) wrote :

On Wed, Jan 4, 2012 at 5:33 AM, Matt Haggard <email address hidden> wrote:
> Rob,
> Also, is laziness overriding (your suggestion) something that should be built in from the beginning, or can it be added as a new feature after global laziness (my patch) is added?  Maybe the answer depends on the implementation.

I think adding it later is fine, but we should consider what it needs
early, so that we don't conflict with the global laziness thing, which
will itself be useful.

As for syntax, perhaps
# do not load attribute1 or attribute2, and load attribute3 which is
globally set to lazy
store.find(Load(Foo, lazy=['attribute1', 'attribute2'], eager=['attribute3'])) ?

Matt Haggard (iffy) wrote :

So what's the next step on this?

Robert Collins (lifeless) wrote :

On Fri, Jan 6, 2012 at 11:19 AM, Matt Haggard <email address hidden> wrote:
> So what's the next step on this?

Well, I think for you to spend a little time coming up with an answer to
'does the patch need to change to gracefully permit per-field laziness
in future'

if the answer is no, then we review and land; if it is yes, you make
such changes as you consider necessary to future proof, then we review
and land.

-Rob

Matt Haggard (iffy) wrote :

Rob,

No, this patch doesn't need to change to support per-query laziness in the future. Per-query laziness would use some of the stuff in this patch as it is. The overlap point is probably the LazyGroupMember class, which would probably be changed to Unloaded, NotLoadedYet, NotFetchedYet or Unfetched or something like that. But that's used internally and not in the public API.

Thanks for your help on this!

Matt

Gustavo Niemeyer (niemeyer) wrote :

Thanks a lot for pushing this forward.

Given the question on how the syntax should look like, I'm concerned the
spirit described in the notes isn't being implemented. It doesn't seem
practical to define per field granularity for what should be loaded.

I'll have a look at the branch, hopefully next week.
On Jan 3, 2012 2:33 PM, "Matt Haggard" <email address hidden> wrote:

> Rob,
>
> I know what you mean. I'm working on this to fix a problem we currently
> have with a big table (126 fields). 90% of the time, we need about 4
> fields; 10% of the time we need some combination of the others.
>
> My current patch allows you to define column laziness globally, which
> improves 90% of our code. What you're suggesting is that in addition to
> (instead of?) defining laziness globally, you can override laziness when
> using a store.find, store.get, Reference or ReferenceSet? This would allow
> one to avoid double queries for the other 10%.
>
> What would the syntax look like?
>
> store.find(Foo, eager=(Foo.name, Foo.kind))
> bar = Reference(bar_id, 'Bar.id', eager=('Bar.name', 'Bar.kind'))
>
> Or would you do some kind of state thing (this tastes bad)?
>
> Bar.eagers.push('name', 'kind')
> store.find(Bar)
> Bar.eagers.pop()
>
> Maybe this is better:
>
> store.eager_load(Bar.name, Bar.kind)
> store.find(Bar)
> ...
> store.lazy_load(Bar.name, Bar.kind)
>
> Also, is laziness overriding (your suggestion) something that should be
> built in from the beginning, or can it be added as a new feature after
> global laziness (my patch) is added? Maybe the answer depends on the
> implementation.
>
>
> Thanks,
>
> Matt
> --
> https://code.launchpad.net/~magmatt/storm/lazycolumns/+merge/87092
> You are subscribed to branch lp:storm.
>

Matt Haggard (iffy) wrote :

Gustavo,

I'm excited for your review.

To clarify, this branch implements laziness as described in the TODO. Rob mentioned that it would be nice to sometimes override that behavior per query. That (overriding per query) is not implemented here. So the syntax closely mirrors the TODO.

A single, lazy attr:

    class C(object):
        ...
        attr = Unicode(lazy=True)

or a group of lazy attrs (loaded together if either is accessed):

    class C(object):
        ...
        attr = Unicode(lazy=2)
        attr2 = Unicode(lazy=2)

    class D(object):
        ...
        attr = Int(lazy='group 1')
        attr2 = Int(lazy='group 2')

^ Is the syntax supported by this branch. I opted for a single "lazy" arg instead of having a "lazy" and "lazy_group." It seems simpler to me.

Unmerged revisions

431. By Matt Haggard on 2011-12-29

Added docstring for lazy attribute in expr.Column and removed laziness from the
TODO file

430. By Matt Haggard on 2011-12-29

Lazy-loaded columns will not overwrite a previously set value.

429. By Matt Haggard on 2011-12-29

Finished lazy-loading of columns including some Store tests.

428. By Matt Haggard on 2011-12-28

Added lazy keyword as __init__ to Property and changed ClassInfo to
gather appropriate laziness information from that.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'TODO'
2--- TODO 2008-10-29 15:49:19 +0000
3+++ TODO 2011-12-29 16:23:23 +0000
4@@ -7,32 +7,6 @@
5
6 - Unicode(autoreload=True) will mark the field as autoreload by default.
7
8-- Lazy-by-default attributes:
9-
10- class C(object):
11- ...
12- attr = Unicode(lazy=True)
13-
14- This would make attr be loaded only if touched.
15-
16- Or maybe lazy groups:
17-
18- class C(object):
19- ...
20- lazy_group = LazyGroup()
21- attr = Unicode(lazy=True, lazy_group=lazy_group)
22-
23- Once one of the group attributes are accessed all of them are retrieved
24- at the same time.
25-
26- Lazy groups may be integers as well:
27-
28- class C(object):
29- ...
30- attr = Unicode(lazy_group=1)
31-
32- lazy_group=None means not lazy.
33-
34 - Implement ResultSet.reverse[d]() to invert order_by()?
35
36 - Add support to cyclic references when all of elements of the cycle are
37
38=== modified file 'storm/expr.py'
39--- storm/expr.py 2011-09-15 13:07:55 +0000
40+++ storm/expr.py 2011-12-29 16:23:23 +0000
41@@ -799,15 +799,21 @@
42 a bool.
43 @ivar variable_factory: Factory producing C{Variable} instances typed
44 according to this column.
45+ @ivar lazy: Anything trueish indicates that this column should not be
46+ loaded from the database when an object is fetched. If this is
47+ C{True} the column will be loaded on first access. If the value is
48+ an integer or string, then all columns with the same value will be
49+ loaded when any of them is first accessed.
50 """
51- __slots__ = ("name", "table", "primary", "variable_factory",
52+ __slots__ = ("name", "table", "primary", "lazy", "variable_factory",
53 "compile_cache", "compile_id")
54
55 def __init__(self, name=Undef, table=Undef, primary=False,
56- variable_factory=None):
57+ variable_factory=None, lazy=None):
58 self.name = name
59 self.table = table
60 self.primary = int(primary)
61+ self.lazy = lazy
62 self.variable_factory = variable_factory or Variable
63 self.compile_cache = None
64 self.compile_id = None
65
66=== modified file 'storm/info.py'
67--- storm/info.py 2011-08-14 08:55:15 +0000
68+++ storm/info.py 2011-12-29 16:23:23 +0000
69@@ -62,11 +62,16 @@
70 @ivar table: Expression from where columns will be looked up.
71 @ivar cls: Class which should be used to build objects.
72 @ivar columns: Tuple of column properties found in the class.
73+ @ivar eager_columns: Tuple of columns that are not lazy.
74+ @ivar lazy_columns: Tuple of columns that are loaded lazily.
75+ @ivar lazy_groups: Dictionary whose keys are columns and values are lists
76+ of columns that should be lazily loaded together.
77 @ivar primary_key: Tuple of column properties used to form the primary key
78 @ivar primary_key_pos: Position of primary_key items in the columns tuple.
79 """
80
81 def __init__(self, cls):
82+ from storm.properties import Group
83 self.table = getattr(cls, "__storm_table__", None)
84 if self.table is None:
85 raise ClassInfoError("%s.__storm_table__ missing" % repr(cls))
86@@ -81,11 +86,28 @@
87 column = getattr(cls, attr, None)
88 if isinstance(column, Column):
89 pairs.append((attr, column))
90+
91
92 pairs.sort()
93
94 self.columns = tuple(pair[1] for pair in pairs)
95 self.attributes = dict(pairs)
96+ self.eager_columns = tuple(pair[1] for pair in pairs
97+ if not(pair[1].lazy))
98+ self.lazy_columns = tuple(pair[1] for pair in pairs
99+ if pair[1].lazy)
100+
101+ common_laziness = {}
102+ self.lazy_groups = {}
103+ for column in self.lazy_columns:
104+ if column.lazy is True:
105+ # lazy alone
106+ self.lazy_groups[column] = [column]
107+ else:
108+ # lazy, perhaps with some other columns
109+ common = common_laziness.setdefault(column.lazy, [])
110+ common.append(column)
111+ self.lazy_groups[column] = common
112
113 storm_primary = getattr(cls, "__storm_primary__", None)
114 if storm_primary is not None:
115
116=== modified file 'storm/properties.py'
117--- storm/properties.py 2011-02-28 21:16:29 +0000
118+++ storm/properties.py 2011-12-29 16:23:23 +0000
119@@ -36,17 +36,18 @@
120 __all__ = ["Property", "SimpleProperty",
121 "Bool", "Int", "Float", "Decimal", "RawStr", "Unicode",
122 "DateTime", "Date", "Time", "TimeDelta", "UUID", "Enum",
123- "Pickle", "JSON", "List", "PropertyRegistry"]
124+ "Pickle", "JSON", "List", "PropertyRegistry", "Group"]
125
126
127 class Property(object):
128
129 def __init__(self, name=None, primary=False,
130- variable_class=Variable, variable_kwargs={}):
131+ variable_class=Variable, variable_kwargs={}, lazy=None):
132 self._name = name
133 self._primary = primary
134 self._variable_class = variable_class
135 self._variable_kwargs = variable_kwargs
136+ self._lazy = lazy
137
138 def __get__(self, obj, cls=None):
139 if obj is None:
140@@ -100,6 +101,7 @@
141 else:
142 name = self._name
143 column = PropertyColumn(self, cls, attr, name, self._primary,
144+ self._lazy,
145 self._variable_class,
146 self._variable_kwargs)
147 cls._storm_columns[self] = column
148@@ -108,12 +110,13 @@
149
150 class PropertyColumn(Column):
151
152- def __init__(self, prop, cls, attr, name, primary,
153+ def __init__(self, prop, cls, attr, name, primary, lazy,
154 variable_class, variable_kwargs):
155 Column.__init__(self, name, cls, primary,
156 VariableFactory(variable_class, column=self,
157 validator_attribute=attr,
158- **variable_kwargs))
159+ **variable_kwargs),
160+ lazy)
161
162 self.cls = cls # Used by references
163
164@@ -127,10 +130,11 @@
165
166 variable_class = None
167
168- def __init__(self, name=None, primary=False, **kwargs):
169+ def __init__(self, name=None, primary=False, lazy=None, **kwargs):
170 kwargs["value"] = kwargs.pop("default", Undef)
171 kwargs["value_factory"] = kwargs.pop("default_factory", Undef)
172- Property.__init__(self, name, primary, self.variable_class, kwargs)
173+ Property.__init__(self, name, primary, self.variable_class, kwargs,
174+ lazy)
175
176
177 class Bool(SimpleProperty):
178@@ -340,3 +344,13 @@
179 self._storm_property_registry = PropertyRegistry()
180 elif hasattr(self, "__storm_table__"):
181 self._storm_property_registry.add_class(self)
182+
183+
184+
185+class Group(object):
186+ """I am used to group L{Property}s to be lazily loaded together.
187+ """
188+
189+
190+
191+
192
193=== modified file 'storm/store.py'
194--- storm/store.py 2011-05-16 10:45:52 +0000
195+++ storm/store.py 2011-12-29 16:23:23 +0000
196@@ -42,7 +42,7 @@
197 from storm.event import EventSystem
198
199
200-__all__ = ["Store", "AutoReload", "EmptyResultSet"]
201+__all__ = ["Store", "AutoReload", "EmptyResultSet", "LazyGroupMember"]
202
203
204 PENDING_ADD = 1
205@@ -172,7 +172,7 @@
206
207 where = compare_columns(cls_info.primary_key, primary_vars)
208
209- select = Select(cls_info.columns, where,
210+ select = Select(cls_info.eager_columns, where,
211 default_tables=cls_info.table, limit=1)
212
213 result = self._connection.execute(select)
214@@ -667,7 +667,7 @@
215
216 # Prepare cache key.
217 primary_vars = []
218- columns = cls_info.columns
219+ columns = cls_info.eager_columns + cls_info.lazy_columns
220
221 for value in values:
222 if value is not None:
223@@ -677,6 +677,8 @@
224 # wasn't found. This is useful for joins, where non-existent
225 # rows are represented like that.
226 return None
227+
228+ values += tuple([LazyGroupMember] * len(cls_info.lazy_columns))
229
230 for i in cls_info.primary_key_pos:
231 value = values[i]
232@@ -694,7 +696,7 @@
233
234 # Take that chance and fill up any undefined variables
235 # with fresh data, since we got it anyway.
236- self._set_values(obj_info, cls_info.columns, result,
237+ self._set_values(obj_info, columns, result,
238 values, keep_defined=True)
239
240 # We're not sure if the obj is still in memory at this
241@@ -707,7 +709,7 @@
242 obj_info = get_obj_info(obj)
243 obj_info["store"] = self
244
245- self._set_values(obj_info, cls_info.columns, result, values,
246+ self._set_values(obj_info, columns, result, values,
247 replace_unknown_lazy=True)
248
249 self._add_to_alive(obj_info)
250@@ -750,7 +752,8 @@
251 variable = obj_info.variables[column]
252 lazy_value = variable.get_lazy()
253 is_unknown_lazy = not (lazy_value is None or
254- lazy_value is AutoReload)
255+ lazy_value is AutoReload or
256+ lazy_value is LazyGroupMember)
257 if keep_defined:
258 if variable.is_defined() or is_unknown_lazy:
259 continue
260@@ -871,6 +874,14 @@
261 the store, and then set all variables set to AutoReload to
262 their database values.
263 """
264+ if lazy_value is LazyGroupMember:
265+ fellow_columns = obj_info.cls_info.lazy_groups[variable.column]
266+ for column in fellow_columns:
267+ if obj_info.variables[column].get_lazy() is LazyGroupMember:
268+ obj_info.variables[column].set(AutoReload)
269+ # now that it's an AutoReload, resolve the lazy value
270+ return self._resolve_lazy_value(obj_info, variable, AutoReload)
271+
272 if lazy_value is not AutoReload and not isinstance(lazy_value, Expr):
273 # It's not something we handle.
274 return
275@@ -1678,7 +1689,7 @@
276 if isinstance(info, Column):
277 default_tables.append(info.table)
278 else:
279- columns.extend(info.columns)
280+ columns.extend(info.eager_columns)
281 default_tables.append(info.table)
282 return columns, default_tables
283
284@@ -1706,7 +1717,7 @@
285 value=values[values_start], from_db=True)
286 objects.append(variable.get())
287 else:
288- values_end += len(info.columns)
289+ values_end += len(info.eager_columns)
290 obj = store._load_object(info, result,
291 values[values_start:values_end])
292 objects.append(obj)
293@@ -1809,3 +1820,21 @@
294 pass
295
296 AutoReload = AutoReload()
297+
298+
299+class LazyGroupMember(LazyValue):
300+ """A marker for indicating that this value is loaded on first access.
301+
302+ This is used internally when you make a property lazy, as in::
303+
304+ class Person(object):
305+ __storm_table__ = "person"
306+ id = Int(primary=True)
307+ name = Unicode(lazy=True)
308+
309+ When a C{Person} is retrieved from the store, it's C{name} property
310+ will be set to L{LazyGroupMember}.
311+ """
312+ pass
313+
314+LazyGroupMember = LazyGroupMember()
315\ No newline at end of file
316
317=== modified file 'tests/info.py'
318--- tests/info.py 2011-12-07 11:57:07 +0000
319+++ tests/info.py 2011-12-29 16:23:23 +0000
320@@ -22,7 +22,7 @@
321 import gc
322
323 from storm.exceptions import ClassInfoError
324-from storm.properties import Property
325+from storm.properties import Property, Group
326 from storm.variables import Variable
327 from storm.expr import Undef, Select, compile
328 from storm.info import *
329@@ -168,6 +168,40 @@
330 cls_info = ClassInfo(Class)
331 self.assertEquals(cls_info.primary_key_pos, (2, 0))
332
333+ def test_eager_columns(self):
334+ class Class(object):
335+ __storm_table__ = 'table'
336+ prop1 = Property(primary=True)
337+ prop2 = Property()
338+ prop3 = Property(lazy=True)
339+ prop4 = Property(lazy=1)
340+ prop5 = Property(lazy=Group())
341+ cls_info = ClassInfo(Class)
342+ self.assertEquals(cls_info.eager_columns, (Class.prop1, Class.prop2))
343+
344+ def test_lazy_columns(self):
345+ class Class(object):
346+ __storm_table__ = 'table'
347+ prop1 = Property(primary=True)
348+ prop2 = Property()
349+ prop3 = Property(lazy=True)
350+ prop4 = Property(lazy=1)
351+ prop5 = Property(lazy=Group())
352+ prop6 = Property(lazy=1)
353+ cls_info = ClassInfo(Class)
354+ self.assertEquals(cls_info.lazy_columns,
355+ (Class.prop3, Class.prop4, Class.prop5, Class.prop6))
356+ self.assertEqual(cls_info.lazy_groups[Class.prop3],
357+ [Class.prop3])
358+ self.assertEqual(cls_info.lazy_groups[Class.prop4],
359+ [Class.prop4, Class.prop6])
360+ self.assertEqual(cls_info.lazy_groups[Class.prop5],
361+ [Class.prop5])
362+ self.assertEqual(cls_info.lazy_groups[Class.prop6],
363+ cls_info.lazy_groups[Class.prop4])
364+
365+
366+
367
368 class ObjectInfoTest(TestHelper):
369
370
371=== modified file 'tests/store/base.py'
372--- tests/store/base.py 2011-08-01 12:54:36 +0000
373+++ tests/store/base.py 2011-12-29 16:23:23 +0000
374@@ -42,7 +42,8 @@
375 NoStoreError, NotFlushedError, NotOneError, OrderLoopError, UnorderedError,
376 WrongStoreError, DisconnectionError)
377 from storm.cache import Cache
378-from storm.store import AutoReload, EmptyResultSet, Store, ResultSet
379+from storm.store import (
380+ AutoReload, EmptyResultSet, Store, ResultSet, LazyGroupMember)
381 from storm.tracer import debug
382
383 from tests.info import Wrapper
384@@ -130,6 +131,14 @@
385 id = Int(primary=True)
386 value = Decimal()
387
388+class Sloth(object):
389+ __storm_table__ = "sloth"
390+ id = Int(primary=True)
391+ val1 = Int()
392+ val2 = Int(lazy=2)
393+ val3 = Int(lazy=2)
394+ val4 = Int(lazy=True)
395+
396
397 class DecorateVariable(Variable):
398
399@@ -244,6 +253,8 @@
400 " VALUES (8, 20, 1, 4)")
401 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
402 " VALUES (9, 20, 1, 2)")
403+ connection.execute("INSERT INTO sloth (id, val1, val2, val3, val4)"
404+ " VALUES (1, 1, 2, 3, 4)")
405
406 connection.commit()
407
408@@ -5291,6 +5302,64 @@
409 self.assertEquals(lazy_value, AutoReload)
410 self.assertEquals(foo.title, u"Default Title")
411
412+ def test_lazy_loading_find(self):
413+ """
414+ Accessing a lazily-loaded attribute will cause it to load as well as
415+ any other attribute in its group.
416+ """
417+ sloth = self.store.find(Sloth, id=1).one()
418+ variables = get_obj_info(sloth).variables
419+ self.assertEqual(variables[Sloth.val2].get_lazy(), LazyGroupMember)
420+ self.assertEqual(variables[Sloth.val3].get_lazy(), LazyGroupMember)
421+ self.assertEqual(variables[Sloth.val4].get_lazy(), LazyGroupMember)
422+ val2 = sloth.val2
423+ # val2 and val3 are in a group together
424+ self.assertEqual(variables[Sloth.val2].get_lazy(), None)
425+ self.assertEqual(variables[Sloth.val3].get_lazy(), None)
426+ self.assertEqual(variables[Sloth.val4].get_lazy(), LazyGroupMember)
427+ self.assertEqual(val2, 2)
428+ self.assertEqual(sloth.val3, 3)
429+ val4 = sloth.val4
430+ self.assertEqual(variables[Sloth.val4].get_lazy(), None)
431+ self.assertEqual(val4, 4)
432+
433+ def test_lazy_loading_get(self):
434+ """
435+ get() should honor lazy attributes and not get them
436+ """
437+ sloth = self.store.get(Sloth, 1)
438+ variables = get_obj_info(sloth).variables
439+ self.assertEqual(variables[Sloth.val2].get_lazy(), LazyGroupMember)
440+ self.assertEqual(variables[Sloth.val3].get_lazy(), LazyGroupMember)
441+ self.assertEqual(variables[Sloth.val4].get_lazy(), LazyGroupMember)
442+ val2 = sloth.val2
443+ # val2 and val3 are in a group together
444+ self.assertEqual(variables[Sloth.val2].get_lazy(), None)
445+ self.assertEqual(variables[Sloth.val3].get_lazy(), None)
446+ self.assertEqual(variables[Sloth.val4].get_lazy(), LazyGroupMember)
447+ self.assertEqual(val2, 2)
448+ self.assertEqual(sloth.val3, 3)
449+ val4 = sloth.val4
450+ self.assertEqual(variables[Sloth.val4].get_lazy(), None)
451+ self.assertEqual(val4, 4)
452+
453+ def test_lazy_loading_set(self):
454+ """
455+ Setting an attribute should not cause a group to load, but should
456+ prevent that value from being overwritten by a later load.
457+ """
458+ sloth = self.store.find(Sloth, id=1).one()
459+ sloth.val2 = 200
460+ variables = get_obj_info(sloth).variables
461+ self.assertEqual(variables[Sloth.val2].get_lazy(), None)
462+ self.assertEqual(variables[Sloth.val3].get_lazy(), LazyGroupMember,
463+ "The other column in this lazy group should not yet "
464+ "be loaded.")
465+ val3 = sloth.val3
466+ self.assertEqual(val3, 3)
467+ self.assertEqual(sloth.val2, 200, "Attribute should not be overwritten"
468+ " by database value: %r" % sloth.val2)
469+
470 def test_reference_break_on_local_diverged_doesnt_autoreload(self):
471 foo = self.store.get(Foo, 10)
472 self.store.autoreload(foo)
473
474=== modified file 'tests/store/mysql.py'
475--- tests/store/mysql.py 2011-02-14 12:17:54 +0000
476+++ tests/store/mysql.py 2011-12-29 16:23:23 +0000
477@@ -79,6 +79,11 @@
478 connection.execute("CREATE TABLE unique_id "
479 "(id VARCHAR(36) PRIMARY KEY) "
480 "ENGINE=InnoDB")
481+ connection.execute("CREATE TABLE sloth "
482+ "(id INT PRIMARY KEY AUTO_INCREMENT,"
483+ " val1 INTEGER, val2 INTEGER, val3 INTEGER,"
484+ " val4 INTEGER) "
485+ "ENGINE=InnoDB")
486 connection.commit()
487
488
489
490=== modified file 'tests/store/postgres.py'
491--- tests/store/postgres.py 2011-02-14 12:17:54 +0000
492+++ tests/store/postgres.py 2011-12-29 16:23:23 +0000
493@@ -93,6 +93,9 @@
494 " value1 INTEGER, value2 INTEGER)")
495 connection.execute("CREATE TABLE unique_id "
496 "(id UUID PRIMARY KEY)")
497+ connection.execute("CREATE TABLE sloth "
498+ "(id SERIAL PRIMARY KEY, val1 INTEGER,"
499+ " val2 INTEGER, val3 INTEGER, val4 INTEGER)")
500 connection.commit()
501
502 def drop_tables(self):
503
504=== modified file 'tests/store/sqlite.py'
505--- tests/store/sqlite.py 2011-02-14 12:17:54 +0000
506+++ tests/store/sqlite.py 2011-12-29 16:23:23 +0000
507@@ -65,6 +65,9 @@
508 " value1 INTEGER, value2 INTEGER)")
509 connection.execute("CREATE TABLE unique_id "
510 "(id VARCHAR PRIMARY KEY)")
511+ connection.execute("CREATE TABLE sloth "
512+ "(id INTEGER PRIMARY KEY, val1 INTEGER,"
513+ " val2 INTEGER, val3 INTEGER, val4 INTEGER)")
514 connection.commit()
515
516 def drop_tables(self):

Subscribers

People subscribed via source and target branches

to status/vote changes: