Code review comment for lp:~cjwatson/launchpad/refactor-cron-germinate

Revision history for this message
Colin Watson (cjwatson) wrote :

I've tested the core logic on mawson now (stripped out the database interaction and hardcoded paths to a local mirror so that I could run it in isolation). Compared to an output tree generated by the old cron.germinate with germinate 2.0, the substantive output is identical: various files are deliberately no longer generated, and the log output is different, but the actual germinate output and the more-extra.override.precise.main file are identical. It runs in about 7 minutes compared with about 14/15 minutes for the old cron.germinate, which I call a win (although these are single-run figures).

The weird part is that something is wrong that means we aren't even getting a lot of the benefit yet! It isn't actually sharing the results of germinating common seeds. Therefore, that 7/8 minutes of win must essentially come from avoiding duplicated seed parsing, avoiding duplicated Packages/Sources parsing, and a reduction in the number of files we write. I wasn't expecting that to account for so much of the runtime, but I'm not going to complain.

I will investigate further to find out why common seed output isn't being shared. That should bring the runtime down further.

« Back to merge proposal