Code review comment for lp:~jtv/launchpad/bug-452107

= Bug 452107 =

We have a script that exports the latest translations for a product
series to a bzr branch. Another script allows users to import their
translations from a bzr branch. There's nothing stopping a user from
setting up both on the same product series and the same branch, creating
a full-duplex exchange of translations between the branch and Rosetta.

To avoid all sorts of race conditions, the export-to-bzr script checks
that there are no pending BranchJobs tasked with putting changes coming
in from the branch onto the translations import queue. If there are
pending jobs of this kind, it skips the export.

Alas! BranchJobs have no garbage collection or status monitoring of any
kind. So if a job dies in a way that circumvents its status change to
"failed" (e.g. because of a server reset or a script kill), the zombie
job sticks around in uncompleted state and haunts us forever.

This branch copes with that by assuming that a RosettaUploadJob (the
type of BranchJob in question) will always be completed, one way or the
other, within one day. Any jobs older than that are ignored.

I implemented this by adding a "since" parameter to the method that
finds unfinished jobs. It still defaults to an unlimited search because
the method may conceivably be used in the future to find zombie jobs
instead of the pending ones that aren't zombies.

Test: {{{./bin/test -vv -t translations.*to.branch}}}

Staging Q/A is a pain in the arse:

 * Pick a project that you have edit rights for. Ensure that it's set
   up to use Launchpad Translations.

 * Register a branch for the project on staging. Populate it by pushing
   a real branch to it. Set the branch as the development branch for a
   translatable release series of your product.

 * Set up translations export for the product series to your branch.

 * Set up translations import from bzr for the product series (of both
   template and translations).

 * Add a .pot file to your branch, commit, and push.

 * Wait for the rosetta-branches script to run on staging. (Make sure
   this has been set up; mthaddon started on it 2009-10-19 but I haven't
   checked up at the time of writing). This can take up to an hour.

 * Verify that your template appears on the project's import queue.
   Wait for it to be imported.

 * Translate something in the UI.

 * Have cronscripts/translations-export-to-branch.py run on the staging
   codehosting server. Never mind the horrible failures for the other
   branches; they're not available on this server.

 * Check that this committed your UI translation to your branch.

So far we've verified that I haven't broken exports, but not the actual
fix yet. Have a LOSA execute this SQL against the staging db to
simulate zombification of the job that just imported your template:

    UPDATE Job
    SET status = 1
    FROM Product, ProductSeries, BranchJob
    WHERE
        Job.id = BranchJob.job AND
        ProductSeries.branch = BranchJob.branch AND
        Product.id = ProductSeries.product AND
        Product.name = '<your-product-name>' AND
        ProductSeries.name = '<your-productseries-name>' AND
        BranchJob.job_type = 3;

This should update 1 record.

 * Now have cronscripts/translations-export-to-branch.py run on the
   staging codehosting server again. It will skip your branch because
   it has a pending translations change.

 * Have a LOSA make your zombie job look old:

    UPDATE Job
    SET date_created = '2009-10-19'::date
    FROM Product, ProductSeries, BranchJob
    WHERE
        Job.id = BranchJob.job AND
        ProductSeries.branch = BranchJob.branch AND
        Product.id = ProductSeries.product AND
        Product.name = '<your-product-name>' AND
        ProductSeries.name = '<your-productseries-name>' AND
        BranchJob.job_type = 3;

This should update 1 record or 2; another job may have been created for
the translations that we just committed.

 * Now have cronscripts/translations-export-to-branch.py run again.
   (On the staging codehosting server, remember). This time it will not
   skip your branch; the script recognizes the zombie job for what it
   is. There probably aren't any new translations to export, but that
   doesn't matter.

No lint (after one small drive-by cleanup).

Jeroen