Merge lp:~rye/python-oops-tools/oopsloader-update-date-when-needed into lp:python-oops-tools

[Roman Yepishev]

Tune dboopsloader to avoid pointless commits to the last-processed date marker.

Tune dboopsloader to avoid pointless commits to the last-processed date marker.

[Robert Collins (lifeless)]

.save does not do what you think it does :)

You need the .save, but what you want to avoid is the commits; that means:
 - taking the ORM out of autocommit mode
 - explicitly doing a commit (I suggest one every 5 seconds work)

review: Needs Fixing

[Robert Collins (lifeless)]

Oh bah, I see, I misread. So I'm going to +1 this instead, but I'll note that there is an equal performance win waiting if we move out of autocommit mode.

review: Approve

=== modified file 'src/oopstools/oops/dboopsloader.py'
--- src/oopstools/oops/dboopsloader.py	2011-10-13 20:18:51 +0000
+++ src/oopstools/oops/dboopsloader.py	2011-10-21 15:18:26 +0000
@@ -151,10 +151,12 @@
                         oops = self._load_oops(datedir, filename)
                         if oops is not None:
                             yield oops
-                        # We do this every time because scripts that use
-                        # it (update_db.py) might stop after any oops.
-                        entry.last_date = date
-                        entry.save()
+                        # We update the last_date only when oops 
+                        # has the date different to what we already have
+                        # This speeds up the loading process
+                        if entry.last_date != date:
+                            entry.last_date = date
+                            entry.save()
                 date += datetime.timedelta(days=1)
 
     def _load_oops(self, datedir, filename):