Comment 10 for bug 808930

Running a version of cronscripts/scan_branches.py modified for profiling:

--- cronscripts/scan_branches.py 2011-06-20 20:07:33 +0000
+++ cronscripts/scan_branches.py 2011-10-10 11:13:03 +0000
@@ -8,11 +8,12 @@
 __metaclass__ = type

 import _pythonpath
+import cProfile

 from lp.code.interfaces.branchjob import IBranchScanJobSource
 from lp.services.job.runner import (
     JobCronScript,
- TwistedJobRunner,
+ JobRunner,
     )

@@ -23,10 +24,17 @@
     source_interface = IBranchScanJobSource

     def __init__(self):
- super(RunScanBranches, self).__init__(runner_class=TwistedJobRunner)
+ super(RunScanBranches, self).__init__(runner_class=JobRunner)

 if __name__ == '__main__':

     script = RunScanBranches()
- script.lock_and_run()
+ import os
+ all_names = os.listdir('.')
+ index = 0
+ fname = 'scan-branches-%04i.profile' % index
+ while fname in all_names:
+ index += 1
+ fname = 'scan-branches-%04i.profile' % index
+ cProfile.run('script.lock_and_run()', fname)

for this branch: for this branch
bzr+ssh://bazaar.launchpad.net/~ramana/gcc-linaro/fix-lp-838994/

shows that approximately 50% (604s) of the total run time (1196s)
is spent in Storm. (note: of this time, only 268s are spent in
psycopg2._psycopg.cursor.execute(), and 417s in
storm.database.raw_execute())

The only obvious hotspot I am seeing is the method
acquireRevisionAuthor() in lp.code.mode.revision, which needs
464s.

Danilo dug up several OOPSes, where the timeout errors always
occur in methods called by planDatabaseChanges(). Interestingly,
this method has a runtime of only 4.3s in my test run.