Launchpad itself

BugWatch updates on bug trackers with large numbers of watches fall behind quickly

Bug #383467 reported by Graham Binns
38
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Gavin Panella
Launchpad itself 3.1.11

Bug Description

If a bug tracker has a lot of watches (for example: http://launchpad.net/bugs/bugtrackers/debbugs or http://launchpad.net/bugs/bugtrackers/gnome-bugs) it can very quickly get out of date, with some watches not being updated for well over the standard 24 hour period that we'd normally expect all watches to be checked over.

This may be because checkwatches picks more-or-less at random from the list of watches that have already been updated but which still need updating (watches which have just been created should *always* be updated within a few checkwatches cycles, depending on volume, though we should do some research to discover whether this is actually the case).

To fix this, we should make checkwatches select the 'old' bug watches to update in a given run in order of date last checked, oldest first. Furthermore it may be worth looking into bug 302529 and bug 67480, though these are non-trivial to fix given the complexity of checkwatches at this point in time.

Related branches

lp:~allenap/launchpad/multithreaded-checkwatches
Abel Deuring (community): Approve (code)
Diff: 600 lines (+284/-122)
5 files modified
lib/lp/bugs/doc/checkwatches-cli-switches.txt (+3/-0)
lib/lp/bugs/doc/checkwatches.txt (+4/-4)
lib/lp/bugs/doc/externalbugtracker.txt (+97/-1)
lib/lp/bugs/scripts/checkwatches.py (+177/-114)
lib/lp/bugs/scripts/tests/test_bugimport.py (+3/-3)
Revision history for this message
Graham Binns (gmb) wrote :

This was fixed in r8623.

Changed in malone:
assignee: nobody → Graham Binns (gmb)
status: Triaged → Fix Committed
Revision history for this message
Graham Binns (gmb) wrote :

Argh. No it wasn't. My bad.

Changed in malone:
assignee: Graham Binns (gmb) → nobody
status: Fix Committed → Triaged
Björn Tillenius (bjornt)
Changed in malone:
milestone: 2.2.6 → 2.2.7
Björn Tillenius (bjornt)
Changed in malone:
milestone: 2.2.7 → 2.2.8
Deryck Hodge (deryck)
Changed in malone:
milestone: 2.2.8 → 3.1.10
Deryck Hodge (deryck)
Changed in malone:
milestone: 3.1.10 → none
Deryck Hodge (deryck)
Changed in malone:
milestone: none → 3.1.11
Revision history for this message
Deryck Hodge (deryck) wrote :

Gavin's current multi-threaded checkwatches work should fix this bug.

Changed in malone:
assignee: nobody → Gavin Panella (allenap)
Gavin Panella (allenap)
Changed in malone:
status: Triaged → In Progress
Revision history for this message
Diogo Matsubara (matsubara) wrote : Bug fixed by a commit

Fixed in devel r9957 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/9957>

Changed in malone:
status: In Progress → Fix Committed
Gavin Panella (allenap)
Changed in malone:
status: Fix Committed → Fix Released
Revision history for this message
Martin Mai (mrkanister-deactivatedaccount-deactivatedaccount) wrote :

I am curious: is it possible for someone to guess when the bug watches for large bug trackers (like bugzilla.gnome.org) will be in sync again? Thanks.

Revision history for this message
Graham Binns (gmb) wrote : Re: [Bug 383467] Re: BugWatch updates on bug trackers with large numbers of watches fall behind quickly

2009/12/16 Martin Mai <email address hidden>:
> I am curious: is it possible for someone to guess when the bug watches
> for large bug trackers (like bugzilla.gnome.org) will be in sync again?
> Thanks.

Hopefully within the next couple of weeks. We're working hard to get
bug watches against large bug trackers working properly, but it's an
uphill task. We'll let you know how things go.

Revision history for this message
Deryck Hodge (deryck) wrote :

On Wed, Dec 16, 2009 at 12:10 PM, Martin Mai <email address hidden> wrote:
> I am curious: is it possible for someone to guess when the bug watches
> for large bug trackers (like bugzilla.gnome.org) will be in sync again?
> Thanks.

Hi, Martin.

It's a bit of an unknown right now. The work that was released to fix
this bug was to fix what we estimated to be the worst issue -- large
numbers of bugwatches causing the checkwatches run to fall behind for
gnome-bugs and debbugs -- but running concurrent updates revealed
other faults in the checkwatches system.

We're actively working on this issue. We should know more in the next
day or two, and I'll update this bug at that point.

Cheers,
deryck

Revision history for this message
Martin Mai (mrkanister-deactivatedaccount-deactivatedaccount) wrote :

Wow, that was quick. Thanks you two, for the information.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.