Merge lp:~wgrant/launchpad/bug-390543 into lp:launchpad

[William Grant]

[r=stevenk][bug=390543] Don't log OOPSes for UnknownBugTrackerTypeErrors.

UnknownBugTrackerTypeErrors are logged to BugWatchActivity and all have bugs filed, but they cause hundreds of OOPSes every day. They are known, non-critical issues, so this branch removes OOPS logging for them.

[Steve Kowalik (stevenk)]

William has addressed my concerns on IRC.

review: Approve

=== modified file 'lib/lp/bugs/doc/checkwatches-cli-switches.txt'
--- lib/lp/bugs/doc/checkwatches-cli-switches.txt	2011-02-28 08:59:27 +0000
+++ lib/lp/bugs/doc/checkwatches-cli-switches.txt	2011-03-16 05:38:26 +0000
@@ -108,8 +108,6 @@
     INFO Resetting 5 bug watches for bug tracker 'savannah'
     INFO Updating 5 watches on bug tracker 'savannah'
     INFO 'Unsupported Bugtracker' error updating http://savannah.gnu.org/: SAVANE
-    WARNING ExternalBugtracker for BugTrackerType 'SAVANE' is not known.
-    (OOPS-...)
     INFO 0 watches left to check on bug tracker 'savannah'
     INFO Time for this run...
 

=== modified file 'lib/lp/bugs/doc/checkwatches.txt'
--- lib/lp/bugs/doc/checkwatches.txt	2011-03-10 02:15:00 +0000
+++ lib/lp/bugs/doc/checkwatches.txt	2011-03-16 05:38:26 +0000
@@ -14,90 +14,6 @@
     >>> import transaction
     >>> transaction.commit()
 
-We'll add a bug watch to the Gnu Savannah bug tracker so that
-checkwatches will try to get an ExternalBugTracker with which to update
-the watch.
-
-    >>> from canonical.config import config
-    >>> from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities
-    >>> from canonical.testing.layers import LaunchpadZopelessLayer
-    >>> from lp.testing.factory import LaunchpadObjectFactory
-
-    >>> LaunchpadZopelessLayer.switchDbUser('launchpad')
-
-    >>> factory = LaunchpadObjectFactory()
-    >>> savannah = getUtility(ILaunchpadCelebrities).savannah_tracker
-    >>> bug_watch = factory.makeBugWatch(bugtracker=savannah)
-
-    >>> transaction.commit()
-    >>> LaunchpadZopelessLayer.switchDbUser(config.checkwatches.dbuser)
-
-    >>> import subprocess
-    >>> process = subprocess.Popen(
-    ...     ['cronscripts/checkwatches.py', '-v'],
-    ...     stdin=subprocess.PIPE, stdout=subprocess.PIPE,
-    ...     stderr=subprocess.PIPE)
-    >>> (out, err) = process.communicate()
-    >>> out
-    ''
-    >>> process.returncode
-    0
-
-    >>> print err
-    DEBUG   ...
-    DEBUG   No global batch size specified.
-    DEBUG   Skipping updating Ubuntu Bugzilla watches.
-    DEBUG   No watches to update on http://bugs.debian.org
-    DEBUG   No watches to update on mailto:[e-mail]
-    DEBUG   No watches to update on http://bugzilla.gnome.org/
-    DEBUG   No watches to update on http://bugzilla.gnome.org/bugs
-    DEBUG   No watches to update on https://bugzilla.mozilla.org/
-    INFO    'Unsupported Bugtracker' error updating http://savannah.gnu.org/: SAVANE
-    WARNING ExternalBugtracker for BugTrackerType 'SAVANE' is not known.
-    (OOPS-...)
-    DEBUG   No watches to update on http://sourceforge.net/
-    INFO    Time for this run: ... seconds.
-    DEBUG   Removing lock file:...
-    <BLANKLINE>
-
-OOPSes reported by the checkwatches process are raised using the
-CheckWatchesErrorUtility. The errors have 'CW' in their oops prefix
-and their type is 'BugWatchUpdateWarning'.
-
-    >>> from lp.bugs.scripts.checkwatches.base import (
-    ...     CheckWatchesErrorUtility)
-    >>> error_utility = CheckWatchesErrorUtility()
-    >>> report = error_utility.getLastOopsReport()
-    >>> report.id
-    'OOPS-...TCW...'
-    >>> report.type
-    'BugWatchUpdateWarning'
-
-    >>> from sys import stdout
-    >>> def dump_last_oops():
-    ...     error_utility.getLastOopsReport().write(stdout)
-
-    >>> dump_last_oops()
-    Oops-Id: OOPS-...TCW...
-    Exception-Type: BugWatchUpdateWarning
-    Exception-Value: ExternalBugtracker for BugTrackerType 'SAVANE' ...
-    Date: ...
-    Page-Id:
-    Branch: ...
-    Revision: ...
-    User: None
-    URL: None
-    Duration: ...
-    Informational: False
-    <BLANKLINE>
-    error-explanation=ExternalBugtracker for ... is not known.
-    ...SELECT BugTracker...
-    ...Transaction completed, status: Active...
-    Traceback (most recent call last):
-    ...
-    BugWatchUpdateWarning: ExternalBugtracker for BugTrackerType 'SAVANE'
-    is not known.
-
 We set a default timeout on checkwatches to 30 seconds. In order to test
 this, we can monkey-patch urllib2.urlopen so that it always raises a
 timeout and call the checkwatches cronscript machinery directly.
@@ -105,6 +21,8 @@
 First, we create some bug watches to test with:
 
     >>> from datetime import datetime
+    >>> from canonical.launchpad.interfaces.launchpad import (
+    ...     ILaunchpadCelebrities)
     >>> from lp.bugs.interfaces.bug import IBugSet
     >>> from lp.bugs.interfaces.bugtracker import BugTrackerType
     >>> from lp.bugs.model.bugtracker import BugTracker
@@ -144,7 +62,10 @@
     >>> import urllib2
     >>> urlopen = urllib2.urlopen
 
-    >>> transaction.commit()
+    >>> from lp.bugs.scripts.checkwatches.base import (
+    ...     CheckWatchesErrorUtility)
+    >>> error_utility = CheckWatchesErrorUtility()
+    >>> last_oops_id = error_utility.getLastOopsReport().id
     >>> def do_not_urlopen(url=None, data=None):
     ...     raise socket.timeout("Connection timed out.")
     >>> try:
@@ -154,11 +75,8 @@
     ...         bug_tracker_names=[example_bug_tracker_name])
     ... finally:
     ...     urllib2.urlopen = urlopen
-    >>> dump_last_oops()
-    Oops-Id: OOPS-...TCW...
-    Exception-Type: BugWatchUpdateWarning
-    Exception-Value: ExternalBugtracker for BugTrackerType 'SAVANE' ...
-    ...
+    >>> last_oops_id == error_utility.getLastOopsReport().id
+    True
 
 Errors that occur when updating a bug watch are recorded against that
 bug watch. The timeout will be recorded against the bug watch we just
@@ -196,8 +114,6 @@
     ... finally:
     ...     externalbugtracker.get_external_bugtracker = (
     ...         real_get_external_bugtracker)
-    WARNING...:ExternalBugtracker for BugTrackerType 'ROUNDUP' is not
-    known. (OOPS-...)
 
 The bug watch's last error type field will have been updated to reflect
 the error that was raised:
@@ -266,9 +182,12 @@
 called by the checkwatches script, which doesn't know or care about
 IBugTracker and instances thereof.
 
-If we look at our Savannah bug tracker, we'll see that it's got one bug
-watch.
+Let's first create a watch on our Savannah bug tracker.
 
+    >>> from lp.testing.dbuser import dbuser
+    >>> savannah = getUtility(ILaunchpadCelebrities).savannah_tracker
+    >>> with dbuser('launchpad'):
+    ...     bug_watch = factory.makeBugWatch(bugtracker=savannah)
     >>> savannah.watches.count()
     1
 
@@ -276,8 +195,7 @@
 that it looks as though it has been updated recently
 
     >>> login('[e-mail]')
-    >>> for watch in savannah.watches:
-    ...     watch.lastchecked = datetime.now(utc)
+    >>> savannah.resetWatches()
 
 So our Savannah instance now has no watches that need checking.
 
@@ -300,8 +218,6 @@
     INFO Resetting 1 bug watches for bug tracker 'savannah'
     INFO Updating 1 watches on bug tracker 'savannah'
     INFO 'Unsupported Bugtracker' error updating http://savannah.gnu.org/: SAVANE
-    WARNING ExternalBugtracker for BugTrackerType 'SAVANE' is not known.
-    (OOPS-...)
     INFO 0 watches left to check on bug tracker 'savannah'
 
 We can see that the Savannah bug watch has been updated recently. Also,
@@ -316,13 +232,10 @@
 If a bug tracker doesn't have any watches to update, forceUpdateAll()
 will ignore it.
 
-    >>> transaction.commit()
-    >>> LaunchpadZopelessLayer.switchDbUser('launchpad')
-    >>> login('[e-mail]')
-    >>> empty_tracker = factory.makeBugTracker(
-    ...     'http://example.com', BugTrackerType.ROUNDUP)
-    >>> transaction.commit()
-    >>> LaunchpadZopelessLayer.switchDbUser(config.checkwatches.dbuser)
+    >>> with dbuser('launchpad'):
+    ...     login('[e-mail]')
+    ...     empty_tracker = factory.makeBugTracker(
+    ...         'http://example.com', BugTrackerType.ROUNDUP)
     >>> empty_tracker_name = empty_tracker.name
     >>> update_all(empty_tracker_name)
     INFO Bug tracker 'auto-example.com' doesn't have any watches. Ignoring.
@@ -336,11 +249,9 @@
 We'll add some more watches in order to demonstrate this.
 
     >>> transaction.commit()
-    >>> LaunchpadZopelessLayer.switchDbUser('launchpad')
-    >>> for i in range(5):
-    ...     bug_watch = factory.makeBugWatch(bugtracker=empty_tracker)
-    >>> transaction.commit()
-    >>> LaunchpadZopelessLayer.switchDbUser(config.checkwatches.dbuser)
+    >>> with dbuser('launchpad'):
+    ...     for i in range(5):
+    ...         bug_watch = factory.makeBugWatch(bugtracker=empty_tracker)
 
     >>> empty_tracker.watches.count()
     5
@@ -443,13 +354,11 @@
 We'll generate a bug watch with which to test this. The bug watch must
 be associated with at least one bug task to enable syncing.
 
-    >>> LaunchpadZopelessLayer.switchDbUser('launchpad')
-    >>> login('[e-mail]')
-    >>> bug_tracker = factory.makeBugTracker()
-    >>> bug_watch = factory.makeBugWatch(bugtracker=bug_tracker)
-    >>> bug_watch.bug.default_bugtask.bugwatch = bug_watch
-    >>> transaction.commit()
-    >>> LaunchpadZopelessLayer.switchDbUser(config.checkwatches.dbuser)
+    >>> with dbuser('launchpad'):
+    ...     login('[e-mail]')
+    ...     bug_tracker = factory.makeBugTracker()
+    ...     bug_watch = factory.makeBugWatch(bugtracker=bug_tracker)
+    ...     bug_watch.bug.default_bugtask.bugwatch = bug_watch
 
 If we pass our UselessExternalBugTracker and the bug watch we just
 generated to updateBugWatches we can see that its comments will be
@@ -469,8 +378,7 @@
 another bug, comments won't be synced and the bug won't be linked back
 to the remote bug.
 
-    >>> LaunchpadZopelessLayer.switchDbUser('launchpad')
-    >>> bug_15 = getUtility(IBugSet).get(15)
-    >>> bug_watch.bug.markAsDuplicate(bug_15)
-
-    >>> updater.updateBugWatches(remote_system, [bug_watch], now=nowish)
+    >>> with dbuser('launchpad'):
+    ...     bug_15 = getUtility(IBugSet).get(15)
+    ...     bug_watch.bug.markAsDuplicate(bug_15)
+    ...     updater.updateBugWatches(remote_system, [bug_watch], now=nowish)

=== modified file 'lib/lp/bugs/scripts/checkwatches/core.py'
--- lib/lp/bugs/scripts/checkwatches/core.py	2011-03-10 04:17:48 +0000
+++ lib/lp/bugs/scripts/checkwatches/core.py	2011-03-16 05:38:26 +0000
@@ -439,14 +439,6 @@
                     self.logger.info(
                         "'%s' error updating %s: %s" % (
                             error_type.title, bug_tracker.baseurl, error))
-                # This is retained to make tests happy. It will be
-                # removed later when the checkwatches OOPS
-                # infrastructure can be.
-                if isinstance(error, UnknownBugTrackerTypeError):
-                    message = (
-                        "ExternalBugtracker for BugTrackerType '%s' is not "
-                        "known." % (error.bugtrackertypename))
-                    self.warning(message)
             else:
                 for remotesystem, bug_watch_batch in trackers_and_watches:
                     self.updateBugWatches(