Merge lp:~gmb/launchpad/bugzilla-3.4-init-db-bug-416514 into lp:launchpad

Proposed by Graham Binns
Status: Merged
Merged at revision: not available
Proposed branch: lp:~gmb/launchpad/bugzilla-3.4-init-db-bug-416514
Merge into: lp:launchpad
Diff against target: None lines
To merge this branch: bzr merge lp:~gmb/launchpad/bugzilla-3.4-init-db-bug-416514
Reviewer Review Type Date Requested Status
Abel Deuring (community) code Approve
Review via email: mp+10516@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Graham Binns (gmb) wrote :
Download full text (18.7 KiB)

This branch adds an initializeRemoteBugDB() method to the BugzillaAPI
ExternalBugTracker, in accordance with IExternalBugTracker. I've
refactored a couple of methods out of BugzillaLPPlugin, which descends
from BugzillaAPI, and into the BugzillaAPI class because they are useful
utilities.

I've also refactored the associated tests into the bugzilla-api doctest.

Note that this branch is based on a branch that hasn't yet landed (but
which is r=cprov). Please use the diff below rather than the one
generated by the merge proposal.

Launchpad lint
--------------

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt
  lib/lp/bugs/doc/externalbugtracker-bugzilla-lp-plugin.txt
  lib/lp/bugs/externalbugtracker/bugzilla.py
  lib/lp/bugs/tests/bugzilla-xmlrpc-transport.txt
  lib/lp/bugs/tests/externalbugtracker.py

Diff of changes
---------------

=== modified file 'lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt'
--- lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt 2009-08-20 15:43:37 +0000
+++ lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt 2009-08-21 09:51:57 +0000
@@ -111,3 +111,111 @@
     >>> bugzilla.getCurrentDBTime()
     CALLED Bugzilla.time()
     datetime.datetime(2009, 8, 19, 22, 2, 2, tzinfo=<UTC>)
+
+
+Initializing the bug database
+-----------------------------
+
+BugzillaAPI implements IExternalBugTracker.initializeRemoteBugDB(),
+which takes a list of bug IDs to fetch from the remote server and stores
+those bugs locally for future use.
+
+ >>> bugzilla.initializeRemoteBugDB([1, 2])
+ CALLED Bug.get({'ids': [1, 2], 'permissive': True})
+
+The bug data is stored as a list of dicts:
+
+ >>> def print_bugs(bugs):
+ ... for bug in sorted(bugs):
+ ... print "Bug %s:" % bug
+ ... for key in sorted(bugzilla._bugs[bug]):
+ ... print " %s: %s" % (key, bugzilla._bugs[bug][key])
+ ... print "\n"
+
+ >>> print_bugs(bugzilla._bugs)
+ Bug 1:
+ alias:
+ assigned_to: <email address hidden>
+ component: GPPSystems
+ creation_time: 20080610T16:19:53
+ id: 1
+ internals:...
+ is_open: True
+ last_change_time: 20080610T16:19:53
+ priority: P1
+ product: Marvin
+ resolution: FIXED
+ severity: normal
+ status: RESOLVED
+ summary: That bloody robot still exists.
+ <BLANKLINE>
+ Bug 2:
+ alias: bug-two
+ assigned_to: <email address hidden>
+ component: Crew
+ creation_time: 20080611T09:23:12
+ id: 2
+ internals:...
+ is_open: True
+ last_change_time: 20080611T09:24:29
+ priority: P1
+ product: HeartOfGold
+ resolution:
+ severity: high
+ status: NEW
+ summary: Collect unknown persons in docking bay 2.
+ <BLANKLINE>
+ <BLANKLINE>
+
+
+Storing bugs
+------------
+
+initializeRemoteBugDB() uses the _storeBugs() method to store bug data.
+_storeBugs() will only store a bug once, even if it is requested both by
+alias and ID...

Revision history for this message
Abel Deuring (adeuring) wrote :
Download full text (4.7 KiB)

Hi Graham,

nice branch. No complaints, but I found a few lines with trailing spaces.
Could you remove them?

r=me

Abel

> === modified file 'lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt'
> --- lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt 2009-08-20 15:43:37 +0000
> +++ lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt 2009-08-21 09:51:57 +0000
> @@ -111,3 +111,111 @@
> >>> bugzilla.getCurrentDBTime()
> CALLED Bugzilla.time()
> datetime.datetime(2009, 8, 19, 22, 2, 2, tzinfo=<UTC>)
> +
> +
> +Initializing the bug database
> +-----------------------------
> +
> +BugzillaAPI implements IExternalBugTracker.initializeRemoteBugDB(),
> +which takes a list of bug IDs to fetch from the remote server and stores
> +those bugs locally for future use.
> +
> + >>> bugzilla.initializeRemoteBugDB([1, 2])
> + CALLED Bug.get({'ids': [1, 2], 'permissive': True})
> +
> +The bug data is stored as a list of dicts:
> +
> + >>> def print_bugs(bugs):
> + ... for bug in sorted(bugs):
> + ... print "Bug %s:" % bug
> + ... for key in sorted(bugzilla._bugs[bug]):
> + ... print " %s: %s" % (key, bugzilla._bugs[bug][key])
> + ... print "\n"
> +
> + >>> print_bugs(bugzilla._bugs)
> + Bug 1:
> + alias:

trailing space.

> + assigned_to: <email address hidden>
> + component: GPPSystems
> + creation_time: 20080610T16:19:53
> + id: 1
> + internals:...
> + is_open: True
> + last_change_time: 20080610T16:19:53
> + priority: P1
> + product: Marvin
> + resolution: FIXED
> + severity: normal
> + status: RESOLVED
> + summary: That bloody robot still exists.
> + <BLANKLINE>
> + Bug 2:
> + alias: bug-two
> + assigned_to: <email address hidden>
> + component: Crew
> + creation_time: 20080611T09:23:12
> + id: 2
> + internals:...
> + is_open: True
> + last_change_time: 20080611T09:24:29
> + priority: P1
> + product: HeartOfGold
> + resolution:

trailing space.

> + severity: high
> + status: NEW
> + summary: Collect unknown persons in docking bay 2.
> + <BLANKLINE>
> + <BLANKLINE>
> +
> +
> +Storing bugs
> +------------
> +
> +initializeRemoteBugDB() uses the _storeBugs() method to store bug data.
> +_storeBugs() will only store a bug once, even if it is requested both by
> +alias and ID. We'll reset the test BugzillaAPI's _bugs and _bug_aliases
> +dicts to demonstrate this.
> +
> + >>> bugzilla._bugs = {}
> + >>> bugzilla._bug_aliases = {}
> + >>> bugzilla.initializeRemoteBugDB([2, 'bug-two'])
> + CALLED Bug.get({'ids': [2, 'bug-two'], 'permissive': True})
> +
> + >>> for bug in sorted(bugzilla._bugs):
> + ... print "Bug %r:" % bug
> + ... for key in sorted(bugzilla._bugs[bug]):
> + ... print " %s: %s" % (key, bugzilla._bugs[bug][key])
> + ... print "\n"
> + Bug 2:
> + alias: bug-two
> + assigned_to: <email address hidden>
> + component: Crew
> + creation_time: 20080611T09:23:12
> + id: 2
> +...

Read more...

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt'
--- lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt 2009-08-19 13:12:30 +0000
+++ lib/lp/bugs/doc/externalbugtracker-bugzilla-api.txt 2009-08-21 09:32:57 +0000
@@ -75,3 +75,147 @@
75 ...75 ...
76 BugTrackerAuthenticationError: http://thiswillfail.example.com:76 BugTrackerAuthenticationError: http://thiswillfail.example.com:
77 Fault 300: The username or password you entered is not valid.77 Fault 300: The username or password you entered is not valid.
78
79
80Getting the server time
81-----------------------
82
83To be able to accurately sync with a bug tracker, we need to be able to
84check the time on the remote server. We use BugzillaAPI.getCurrentDBTime()
85to get the current time on the remote server.
86
87 # There's no way to create a UTC timestamp without monkey-patching
88 # the TZ environment variable. Rather than do that, we create our
89 # own datetime and work with that.
90 >>> from datetime import datetime
91 >>> remote_time = datetime(2009, 8, 19, 17, 2, 2)
92
93 >>> test_transport.local_datetime = remote_time
94 >>> bugzilla.getCurrentDBTime()
95 CALLED Bugzilla.time()
96 datetime.datetime(2009, 8, 19, 17, 2, 2, tzinfo=<UTC>)
97
98If the remote system is in a different timezone, getCurrentDBTime() will
99convert its time to UTC before returning it.
100
101 >>> test_transport.utc_offset = 60**2
102 >>> test_transport.timezone = 'CET'
103 >>> bugzilla.getCurrentDBTime()
104 CALLED Bugzilla.time()
105 datetime.datetime(2009, 8, 19, 16, 2, 2, tzinfo=<UTC>)
106
107This works whether the UTC offset is positive or negative.
108
109 >>> test_transport.utc_offset = -5 * 60**2
110 >>> test_transport.timezone = 'US/Eastern'
111 >>> bugzilla.getCurrentDBTime()
112 CALLED Bugzilla.time()
113 datetime.datetime(2009, 8, 19, 22, 2, 2, tzinfo=<UTC>)
114
115
116Initializing the bug database
117-----------------------------
118
119BugzillaAPI implements IExternalBugTracker.initializeRemoteBugDB(),
120which takes a list of bug IDs to fetch from the remote server and stores
121those bugs locally for future use.
122
123 >>> bugzilla.initializeRemoteBugDB([1, 2])
124 CALLED Bug.get({'ids': [1, 2], 'permissive': True})
125
126The bug data is stored as a list of dicts:
127
128 >>> def print_bugs(bugs):
129 ... for bug in sorted(bugs):
130 ... print "Bug %s:" % bug
131 ... for key in sorted(bugzilla._bugs[bug]):
132 ... print " %s: %s" % (key, bugzilla._bugs[bug][key])
133 ... print "\n"
134
135 >>> print_bugs(bugzilla._bugs)
136 Bug 1:
137 alias:
138 assigned_to: test@canonical.com
139 component: GPPSystems
140 creation_time: 20080610T16:19:53
141 id: 1
142 internals:...
143 is_open: True
144 last_change_time: 20080610T16:19:53
145 priority: P1
146 product: Marvin
147 resolution: FIXED
148 severity: normal
149 status: RESOLVED
150 summary: That bloody robot still exists.
151 <BLANKLINE>
152 Bug 2:
153 alias: bug-two
154 assigned_to: marvin@heartofgold.ship
155 component: Crew
156 creation_time: 20080611T09:23:12
157 id: 2
158 internals:...
159 is_open: True
160 last_change_time: 20080611T09:24:29
161 priority: P1
162 product: HeartOfGold
163 resolution:
164 severity: high
165 status: NEW
166 summary: Collect unknown persons in docking bay 2.
167 <BLANKLINE>
168 <BLANKLINE>
169
170
171Storing bugs
172------------
173
174initializeRemoteBugDB() uses the _storeBugs() method to store bug data.
175_storeBugs() will only store a bug once, even if it is requested both by
176alias and ID. We'll reset the test BugzillaAPI's _bugs and _bug_aliases
177dicts to demonstrate this.
178
179 >>> bugzilla._bugs = {}
180 >>> bugzilla._bug_aliases = {}
181 >>> bugzilla.initializeRemoteBugDB([2, 'bug-two'])
182 CALLED Bug.get({'ids': [2, 'bug-two'], 'permissive': True})
183
184 >>> for bug in sorted(bugzilla._bugs):
185 ... print "Bug %r:" % bug
186 ... for key in sorted(bugzilla._bugs[bug]):
187 ... print " %s: %s" % (key, bugzilla._bugs[bug][key])
188 ... print "\n"
189 Bug 2:
190 alias: bug-two
191 assigned_to: marvin@heartofgold.ship
192 component: Crew
193 creation_time: 20080611T09:23:12
194 id: 2
195 internals:...
196 is_open: True
197 last_change_time: 20080611T09:24:29
198 priority: P1
199 product: HeartOfGold
200 resolution:
201 severity: high
202 status: NEW
203 summary: Collect unknown persons in docking bay 2.
204 <BLANKLINE>
205 <BLANKLINE>
206
207Aliases are stored in a separate dict, which contains a mapping between
208the alias and the bug's actual ID.
209
210 >>> for alias, bug_id in bugzilla._bug_aliases.items():
211 ... print "%s: %s" % (alias, bug_id)
212 bug-two: 2
213
214The method _getActualBugId() returns the correct bug ID for a passed bug
215ID or alias.
216
217 >>> bugzilla._getActualBugId('bug-two')
218 2
219
220 >>> bugzilla._getActualBugId(2)
221 2
78222
=== modified file 'lib/lp/bugs/doc/externalbugtracker-bugzilla-lp-plugin.txt'
--- lib/lp/bugs/doc/externalbugtracker-bugzilla-lp-plugin.txt 2009-08-19 09:22:08 +0000
+++ lib/lp/bugs/doc/externalbugtracker-bugzilla-lp-plugin.txt 2009-08-21 09:32:57 +0000
@@ -1,4 +1,5 @@
1= Bugzilla bugtrackers with the Launchpad plugin =1Bugzilla bugtrackers with the Launchpad plugin
2==============================================
23
3These tests cover the BugzillaLPPlugin ExternalBugTracker, which handles4These tests cover the BugzillaLPPlugin ExternalBugTracker, which handles
4Bugzilla instances that have the Launchpad plugin installed.5Bugzilla instances that have the Launchpad plugin installed.
@@ -25,7 +26,8 @@
25 True26 True
2627
2728
28== Authentication ==29Authentication
30--------------
2931
30XML-RPC methods that modify data on the remote server require32XML-RPC methods that modify data on the remote server require
31authentication. To authenticate, we create a LoginToken of type33authentication. To authenticate, we create a LoginToken of type
@@ -154,7 +156,8 @@
154 Protocol error: 500 "Internal server error"156 Protocol error: 500 "Internal server error"
155157
156158
157== Getting the current time ==159Getting the current time
160------------------------
158161
159The BugzillaLPPlugin ExternalBugTracker, like all other162The BugzillaLPPlugin ExternalBugTracker, like all other
160ExternalBugTrackers, has a getCurrentDBTime() method, which returns the163ExternalBugTrackers, has a getCurrentDBTime() method, which returns the
@@ -173,7 +176,8 @@
173 datetime.datetime(2008, 5, 16, 15, 53, 20, tzinfo=<UTC>)176 datetime.datetime(2008, 5, 16, 15, 53, 20, tzinfo=<UTC>)
174177
175178
176== Initializing the remote bug database ==179Initializing the remote bug database
180------------------------------------
177181
178The BugzillaLPPlugin implements the standard initializeRemoteBugDB()182The BugzillaLPPlugin implements the standard initializeRemoteBugDB()
179method, taking a list of the bug ids that need to be updated. It uses183method, taking a list of the bug ids that need to be updated. It uses
@@ -228,56 +232,13 @@
228 <BLANKLINE>232 <BLANKLINE>
229 <BLANKLINE>233 <BLANKLINE>
230234
231initializeRemoteBugDB() will only store a bug once, even if it is235BugzillaLPPlugin.initializeRemoteBugDB() uses its _storeBugs() method to
232requested both by alias and ID. We'll reset the test BugzillaLPPlugin's236store bugs. See externalbugtracker-bugzilla-api.txt for details of
233_bugs and _bug_aliases dicts to demonstrate this.237_storeBugs().
234238
235 >>> bugzilla._bugs = {}239
236 >>> bugzilla._bug_aliases = {}240Getting a list of changed bugs
237 >>> bugzilla.initializeRemoteBugDB([2, 'bug-two'])241------------------------------
238 CALLED Launchpad.get_bugs({'ids': [2, 'bug-two'], 'permissive': True})
239
240 >>> for bug in sorted(bugzilla._bugs):
241 ... print "Bug %r:" % bug
242 ... for key in sorted(bugzilla._bugs[bug]):
243 ... print " %s: %s" % (key, bugzilla._bugs[bug][key])
244 ... print "\n"
245 Bug 2:
246 alias: bug-two
247 assigned_to: marvin@heartofgold.ship
248 component: Crew
249 creation_time: 20080611T09:23:12
250 id: 2
251 internals:...
252 is_open: True
253 last_change_time: 20080611T09:24:29
254 priority: P1
255 product: HeartOfGold
256 resolution:
257 severity: high
258 status: NEW
259 summary: Collect unknown persons in docking bay 2.
260 <BLANKLINE>
261 <BLANKLINE>
262
263Aliases are stored in a separate dict, which contains a mapping between
264the alias and the bug's actual ID.
265
266 >>> for alias, bug_id in bugzilla._bug_aliases.items():
267 ... print "%s: %s" % (alias, bug_id)
268 bug-two: 2
269
270The method _getActualBugId() returns the correct bug ID for a passed bug
271ID or alias.
272
273 >>> bugzilla._getActualBugId('bug-two')
274 2
275
276 >>> bugzilla._getActualBugId(2)
277 2
278
279
280== Getting a list of changed bugs ==
281242
282IExternalBugTracker defines a method, getModifiedRemoteBugs(), which243IExternalBugTracker defines a method, getModifiedRemoteBugs(), which
283accepts a list of bug IDs and a datetime as a parameter and returns the244accepts a list of bug IDs and a datetime as a parameter and returns the
@@ -337,7 +298,8 @@
337 CALLED Launchpad.get_bugs({'ids': [3], 'permissive': True})298 CALLED Launchpad.get_bugs({'ids': [3], 'permissive': True})
338299
339300
340== Getting remote statuses ==301Getting remote statuses
302-----------------------
341303
342BugzillaLPPlugin.getRemoteStatus() will return the remote status of a304BugzillaLPPlugin.getRemoteStatus() will return the remote status of a
343given bug as a string. If the bug has a resolution, that will be305given bug as a string. If the bug has a resolution, that will be
@@ -369,7 +331,8 @@
369 >>> del bugzilla._bugs[999]331 >>> del bugzilla._bugs[999]
370332
371333
372== Getting the remote product ==334Getting the remote product
335--------------------------
373336
374getRemoteProduct() returns the product a remote bug is associated with337getRemoteProduct() returns the product a remote bug is associated with
375in Bugzilla. 338in Bugzilla.
@@ -393,7 +356,8 @@
393 BugNotFound: 12345356 BugNotFound: 12345
394357
395358
396== Retrieving remote comments ==359Retrieving remote comments
360--------------------------
397361
398BugzillaLPPlugin implments the ISupportsCommentImport interface, which362BugzillaLPPlugin implments the ISupportsCommentImport interface, which
399means that we can use it to import comments from the remote Bugzilla363means that we can use it to import comments from the remote Bugzilla
@@ -435,7 +399,8 @@
435 >>> LaunchpadZopelessLayer.switchDbUser(config.checkwatches.dbuser)399 >>> LaunchpadZopelessLayer.switchDbUser(config.checkwatches.dbuser)
436400
437401
438=== getCommentIds() ===402getCommentIds()
403---------------
439404
440ISupportsCommentImport.getCommentIds() is the method used to get all the405ISupportsCommentImport.getCommentIds() is the method used to get all the
441comment IDs for a given bug on a remote bugtracker.406comment IDs for a given bug on a remote bugtracker.
@@ -456,7 +421,8 @@
456 BugNotFound: 42421 BugNotFound: 42
457422
458423
459=== fetchComments() ===424fetchComments()
425---------------
460426
461ISupportsCommentImport.fetchComments() is the method used to fetch a427ISupportsCommentImport.fetchComments() is the method used to fetch a
462given set of comments from the remote bugtracker. It takes a bug watch428given set of comments from the remote bugtracker. It takes a bug watch
@@ -488,7 +454,8 @@
488 time: 20080616T13:22:29454 time: 20080616T13:22:29
489455
490456
491=== getPosterForComment() ===457getPosterForComment()
458---------------------
492459
493ISupportsCommentImport.getPosterForComment() returns a tuple of460ISupportsCommentImport.getPosterForComment() returns a tuple of
494(displayname, email) for the author of a remote comment.461(displayname, email) for the author of a remote comment.
@@ -507,7 +474,8 @@
507 None trillian474 None trillian
508475
509476
510=== getMessageForComment() ===477getMessageForComment()
478----------------------
511479
512ISupportsCommentImport.getMessageForComment() returns a Launchpad480ISupportsCommentImport.getMessageForComment() returns a Launchpad
513IMessage instance for a given comment. It takes a bug watch, a comment481IMessage instance for a given comment. It takes a bug watch, a comment
@@ -533,7 +501,8 @@
533 2008-06-16 13:08:08+00:00501 2008-06-16 13:08:08+00:00
534502
535503
536== Pushing comments to remote systems ==504Pushing comments to remote systems
505----------------------------------
537506
538BugzillaLPPlugin implements the ISupportsCommentPushing interface, which507BugzillaLPPlugin implements the ISupportsCommentPushing interface, which
539defines the necessary methods for pushing comments to remote servers.508defines the necessary methods for pushing comments to remote servers.
@@ -583,7 +552,8 @@
583 <BLANKLINE>552 <BLANKLINE>
584553
585554
586== Linking remote bugs to Launchpad bugs ==555Linking remote bugs to Launchpad bugs
556-------------------------------------
587557
588BugzillaLPPlugin implements the ISupportsBackLinking interface, which558BugzillaLPPlugin implements the ISupportsBackLinking interface, which
589provides methods to set and retrieve the Launchpad bug that links to a559provides methods to set and retrieve the Launchpad bug that links to a
@@ -644,7 +614,8 @@
644 10614 10
645615
646616
647== Working with a specified set of Bugzilla products ==617Working with a specified set of Bugzilla products
618-------------------------------------------------
648619
649BugzillaLPPlugin can be instructed to only get the data for a set of620BugzillaLPPlugin can be instructed to only get the data for a set of
650bug IDs if those bugs belong to one of a given set of products.621bug IDs if those bugs belong to one of a given set of products.
@@ -691,7 +662,8 @@
691 0662 0
692663
693664
694== Getting the products for a set of remote bugs ==665Getting the products for a set of remote bugs
666---------------------------------------------
695667
696BugzillaLPPlugin provides a helper method, getProductsForRemoteBugs(),668BugzillaLPPlugin provides a helper method, getProductsForRemoteBugs(),
697which takes a list of bug IDs or aliases and returns the products to669which takes a list of bug IDs or aliases and returns the products to
698670
=== modified file 'lib/lp/bugs/externalbugtracker/bugzilla.py'
--- lib/lp/bugs/externalbugtracker/bugzilla.py 2009-08-19 13:19:22 +0000
+++ lib/lp/bugs/externalbugtracker/bugzilla.py 2009-08-21 09:32:57 +0000
@@ -430,6 +430,98 @@
430 self.baseurl,430 self.baseurl,
431 "Fault %s: %s" % (fault.faultCode, fault.faultString))431 "Fault %s: %s" % (fault.faultCode, fault.faultString))
432432
433 def _storeBugs(self, remote_bugs):
434 """Store remote bugs in the local `bugs` dict."""
435 for remote_bug in remote_bugs:
436 self._bugs[remote_bug['id']] = remote_bug
437
438 # The bug_aliases dict is a mapping between aliases and bug
439 # IDs. We use the aliases dict to look up the correct ID for
440 # a bug. This allows us to reference a bug by either ID or
441 # alias.
442 if remote_bug['alias'] != '':
443 self._bug_aliases[remote_bug['alias']] = remote_bug['id']
444
445 def getCurrentDBTime(self):
446 """See `IExternalBugTracker`."""
447 time_dict = self.xmlrpc_proxy.Bugzilla.time()
448
449 # Convert the XML-RPC DateTime we get back into a regular Python
450 # datetime.
451 server_db_timetuple = time.strptime(
452 str(time_dict['db_time']), '%Y%m%dT%H:%M:%S')
453 server_db_datetime = datetime(*server_db_timetuple[:6])
454
455 # The server's DB time is the one that we want to use. However,
456 # this may not be in UTC, so we need to convert it. Since we
457 # can't guarantee that the timezone data returned by the server
458 # is sane, we work out the server's offset from UTC by looking
459 # at the difference between the web_time and the web_time_utc
460 # values.
461 server_web_time = time.strptime(
462 str(time_dict['web_time']), '%Y%m%dT%H:%M:%S')
463 server_web_datetime = datetime(*server_web_time[:6])
464 server_web_time_utc = time.strptime(
465 str(time_dict['web_time_utc']), '%Y%m%dT%H:%M:%S')
466 server_web_datetime_utc = datetime(*server_web_time_utc[:6])
467
468 server_utc_offset = server_web_datetime - server_web_datetime_utc
469 server_utc_datetime = server_db_datetime - server_utc_offset
470
471 return server_utc_datetime.replace(tzinfo=pytz.timezone('UTC'))
472
473 def _getActualBugId(self, bug_id):
474 """Return the actual bug id for an alias or id."""
475 # See if bug_id is actually an alias.
476 actual_bug_id = self._bug_aliases.get(bug_id)
477
478 # bug_id isn't an alias, so try turning it into an int and
479 # looking the bug up by ID.
480 if actual_bug_id is not None:
481 return actual_bug_id
482 else:
483 try:
484 actual_bug_id = int(bug_id)
485 except ValueError:
486 # If bug_id can't be int()'d then it's likely an alias
487 # that doesn't exist, so raise BugNotFound.
488 raise BugNotFound(bug_id)
489
490 # Check that the bug does actually exist. That way we're
491 # treating integer bug IDs and aliases in the same way.
492 if actual_bug_id not in self._bugs:
493 raise BugNotFound(bug_id)
494
495 return actual_bug_id
496
497 def _getBugIdsToRetrieve(self, bug_ids):
498 """For a set of bug IDs, return those for which we have no data."""
499 bug_ids_to_retrieve = []
500 for bug_id in bug_ids:
501 try:
502 actual_bug_id = self._getActualBugId(bug_id)
503 except BugNotFound:
504 bug_ids_to_retrieve.append(bug_id)
505
506 return bug_ids_to_retrieve
507
508 def initializeRemoteBugDB(self, bug_ids):
509 """See `IExternalBugTracker`."""
510 # First, discard all those bug IDs about which we already have
511 # data.
512 bug_ids_to_retrieve = self._getBugIdsToRetrieve(bug_ids)
513
514 # Pull the bug data from the remote server. permissive=True here
515 # prevents Bugzilla from erroring if we ask for a bug it doesn't
516 # have.
517 response_dict = self.xmlrpc_proxy.Bug.get({
518 'ids': bug_ids_to_retrieve,
519 'permissive': True,
520 })
521 remote_bugs = response_dict['bugs']
522
523 self._storeBugs(remote_bugs)
524
433525
434class BugzillaLPPlugin(BugzillaAPI):526class BugzillaLPPlugin(BugzillaAPI):
435 """An `ExternalBugTracker` to handle Bugzillas using the LP Plugin."""527 """An `ExternalBugTracker` to handle Bugzillas using the LP Plugin."""
@@ -472,18 +564,6 @@
472 raise BugTrackerAuthenticationError(564 raise BugTrackerAuthenticationError(
473 self.baseurl, message)565 self.baseurl, message)
474566
475 def _storeBugs(self, remote_bugs):
476 """Store remote bugs in the local `bugs` dict."""
477 for remote_bug in remote_bugs:
478 self._bugs[remote_bug['id']] = remote_bug
479
480 # The bug_aliases dict is a mapping between aliases and bug
481 # IDs. We use the aliases dict to look up the correct ID for
482 # a bug. This allows us to reference a bug by either ID or
483 # alias.
484 if remote_bug['alias'] != '':
485 self._bug_aliases[remote_bug['alias']] = remote_bug['id']
486
487 def getModifiedRemoteBugs(self, bug_ids, last_checked):567 def getModifiedRemoteBugs(self, bug_ids, last_checked):
488 """See `IExternalBugTracker`."""568 """See `IExternalBugTracker`."""
489 # We marshal last_checked into an xmlrpclib.DateTime since569 # We marshal last_checked into an xmlrpclib.DateTime since
@@ -538,12 +618,7 @@
538 """See `IExternalBugTracker`."""618 """See `IExternalBugTracker`."""
539 # First, discard all those bug IDs about which we already have619 # First, discard all those bug IDs about which we already have
540 # data.620 # data.
541 bug_ids_to_retrieve = []621 bug_ids_to_retrieve = self._getBugIdsToRetrieve(bug_ids)
542 for bug_id in bug_ids:
543 try:
544 actual_bug_id = self._getActualBugId(bug_id)
545 except BugNotFound:
546 bug_ids_to_retrieve.append(bug_id)
547622
548 # Next, grab the bugs we still need from the remote server.623 # Next, grab the bugs we still need from the remote server.
549 # We pass permissive=True to ensure that Bugzilla won't error if624 # We pass permissive=True to ensure that Bugzilla won't error if
@@ -573,30 +648,6 @@
573 server_utc_time = datetime(*server_timetuple[:6])648 server_utc_time = datetime(*server_timetuple[:6])
574 return server_utc_time.replace(tzinfo=pytz.timezone('UTC'))649 return server_utc_time.replace(tzinfo=pytz.timezone('UTC'))
575650
576 def _getActualBugId(self, bug_id):
577 """Return the actual bug id for an alias or id."""
578 # See if bug_id is actually an alias.
579 actual_bug_id = self._bug_aliases.get(bug_id)
580
581 # bug_id isn't an alias, so try turning it into an int and
582 # looking the bug up by ID.
583 if actual_bug_id is not None:
584 return actual_bug_id
585 else:
586 try:
587 actual_bug_id = int(bug_id)
588 except ValueError:
589 # If bug_id can't be int()'d then it's likely an alias
590 # that doesn't exist, so raise BugNotFound.
591 raise BugNotFound(bug_id)
592
593 # Check that the bug does actually exist. That way we're
594 # treating integer bug IDs and aliases in the same way.
595 if actual_bug_id not in self._bugs:
596 raise BugNotFound(bug_id)
597
598 return actual_bug_id
599
600 def getRemoteStatus(self, bug_id):651 def getRemoteStatus(self, bug_id):
601 """See `IExternalBugTracker`."""652 """See `IExternalBugTracker`."""
602 actual_bug_id = self._getActualBugId(bug_id)653 actual_bug_id = self._getActualBugId(bug_id)
603654
=== modified file 'lib/lp/bugs/tests/bugzilla-xmlrpc-transport.txt'
--- lib/lp/bugs/tests/bugzilla-xmlrpc-transport.txt 2009-08-18 16:39:17 +0000
+++ lib/lp/bugs/tests/bugzilla-xmlrpc-transport.txt 2009-08-21 09:32:57 +0000
@@ -576,3 +576,68 @@
576 Traceback (most recent call last):576 Traceback (most recent call last):
577 ...577 ...
578 Fault: <Fault 300: 'The username or password you entered is not valid.'>578 Fault: <Fault 300: 'The username or password you entered is not valid.'>
579
580
581Getting the current time
582------------------------
583
584The Bugzilla.time() method allows us to retrieve a dict of the time on
585the remote server.
586
587 >>> time_dict = server.Bugzilla.time()
588 >>> for key in sorted(time_dict):
589 ... print "%s: %s" % (key, time_dict[key])
590 db_time: 20080501T01:01:01
591 tz_name: UTC
592 tz_offset: +0000
593 tz_short_name: UTC
594 web_time: 20080501T01:01:01
595 web_time_utc: 20080501T01:01:01
596
597If the remote server is in a different timezone, the db_time and
598web_time items will be in the server's local timezone whilst
599web_time_utc will be in UTC.
600
601 >>> bugzilla_transport.utc_offset = 60**2
602 >>> bugzilla_transport.timezone = 'CET'
603 >>> time_dict = server.Bugzilla.time()
604 >>> for key in sorted(time_dict):
605 ... print "%s: %s" % (key, time_dict[key])
606 db_time: 20080501T01:01:01
607 tz_name: CET
608 tz_offset: +0100
609 tz_short_name: CET
610 web_time: 20080501T01:01:01
611 web_time_utc: 20080501T00:01:01
612
613
614Getting bugs from the server
615----------------------------
616
617The Bugzilla API method Bug.get() allows us to get one or more bugs from
618the remote server. It takes a list of bug IDs to return and returns a
619list of dicts containing those bugs' data.
620
621 >>> return_value = server.Bug.get(
622 ... {'ids': [1], 'permissive': True})
623 >>> [bug_dict] = return_value['bugs']
624 >>> for key in sorted(bug_dict):
625 ... print "%s: %s" % (key, bug_dict[key])
626 alias:
627 assigned_to: test@canonical.com
628 component: GPPSystems
629 creation_time: 20080610T16:19:53
630 id: 1
631 internals:...
632 is_open: True
633 last_change_time: 20080610T16:19:53
634 priority: P1
635 product: Marvin
636 resolution: FIXED
637 severity: normal
638 status: RESOLVED
639 summary: That bloody robot still exists.
640
641Note that further tests for this functionality can be found in the
642"Launchpad.get_bugs()" section, above. This is because these two methods
643are synonymous.
579644
=== modified file 'lib/lp/bugs/tests/externalbugtracker.py'
--- lib/lp/bugs/tests/externalbugtracker.py 2009-08-18 16:41:18 +0000
+++ lib/lp/bugs/tests/externalbugtracker.py 2009-08-21 09:32:57 +0000
@@ -522,7 +522,9 @@
522 # what BugZilla will return.522 # what BugZilla will return.
523 local_time = xmlrpclib.DateTime(local_datetime.timetuple())523 local_time = xmlrpclib.DateTime(local_datetime.timetuple())
524524
525 utc_date_time = local_datetime - timedelta(seconds=self.utc_offset)525 utc_offset_delta = timedelta(seconds=self.utc_offset)
526 utc_date_time = local_datetime - utc_offset_delta
527
526 utc_time = xmlrpclib.DateTime(utc_date_time.timetuple())528 utc_time = xmlrpclib.DateTime(utc_date_time.timetuple())
527 return {529 return {
528 'local_time': local_time,530 'local_time': local_time,
@@ -757,7 +759,11 @@
757759
758 # Map namespaces onto method names.760 # Map namespaces onto method names.
759 methods = {761 methods = {
760 'Bugzilla': ['version'],762 'Bug': ['get'],
763 'Bugzilla': [
764 'time',
765 'version',
766 ],
761 'Test': ['login_required'],767 'Test': ['login_required'],
762 'User': ['login'],768 'User': ['login'],
763 }769 }
@@ -794,6 +800,29 @@
794 300,800 300,
795 "The username or password you entered is not valid.")801 "The username or password you entered is not valid.")
796802
803 def time(self):
804 """Return a dict of the local time and associated data."""
805 # We cheat slightly by calling the superclass to get the time
806 # data. We do this the old fashioned way because XML-RPC
807 # Transports don't support new-style classes.
808 time_dict = TestBugzillaXMLRPCTransport.time(self)
809 offset_hours = (self.utc_offset / 60) / 60
810 offset_string = '+%02d00' % offset_hours
811
812 return {
813 'db_time': time_dict['local_time'],
814 'tz_name': time_dict['tz_name'],
815 'tz_offset': offset_string,
816 'tz_short_name': time_dict['tz_name'],
817 'web_time': time_dict['local_time'],
818 'web_time_utc': time_dict['utc_time'],
819 }
820
821 def get(self, arguments):
822 """Return a list of bug dicts for a given set of bug ids."""
823 # This method is actually just a synonym for get_bugs().
824 return self.get_bugs(arguments)
825
797826
798class TestMantis(Mantis):827class TestMantis(Mantis):
799 """Mantis ExternalSystem for use in tests.828 """Mantis ExternalSystem for use in tests.