rosetta export queue appears stuck, and not progressing

There's a tough request on the queue that makes the script keep a transaction open for more than 3 hours, at which point the master store connection gets killed by one of our watchdog processes.

There's no good reason why there should be a transaction open on the master store in the first place, until the very end where we remove the request and send email. We're particularly hitting the instance in POTMsgSet._conflictsExistingSourceFileFormats here, which still uses cursor().

Instating a SlaveOnlyDatabasePolicy would make cursor() (and other default stores) use the slave store. That should stop the script from opening a transaction on the master until the end.

We could also consider committing between files, rather than between requests. The place for that would be in the TranslationExporter.exportTranslationFiles loop.

Triaging low - we need to get the queue unblocked. Do we get an OOPS when this happens?

We don't get an oops; we get emails on error-reports.

To deal with the immediate problem we stretched the timeout for long master transactions. As far as translation exports are concerned we can undo that as soon as my fix rolls out.

Fixed in stable r12215 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12215>.

We no longer open master-store transactions for export requests until after the bulk of its work is done.