Merge ~andrey-fedoseev/launchpad:jira-bug-watch into launchpad:master

Proposed by Andrey Fedoseev
Status: Needs review
Proposed branch: ~andrey-fedoseev/launchpad:jira-bug-watch
Merge into: launchpad:master
Diff against target: 873 lines (+742/-5)
9 files modified
lib/lp/bugs/externalbugtracker/__init__.py (+2/-0)
lib/lp/bugs/externalbugtracker/base.py (+8/-3)
lib/lp/bugs/externalbugtracker/jira.py (+268/-0)
lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py (+1/-1)
lib/lp/bugs/externalbugtracker/tests/test_jira.py (+432/-0)
lib/lp/bugs/interfaces/bugtracker.py (+9/-0)
lib/lp/bugs/model/bugwatch.py (+11/-0)
lib/lp/bugs/tests/test_bugwatch.py (+9/-0)
lib/lp/services/config/schema-lazr.conf (+2/-1)
Reviewer Review Type Date Requested Status
Colin Watson (community) Approve
Review via email: mp+433103@code.launchpad.net

Commit message

Add external bug tracker for JIRA

To post a comment you must log in.
Revision history for this message
Colin Watson (cjwatson) wrote :

This looks like a perfectly reasonable first-pass implementation, and I'm fine with landing it as far as it goes.

We'll need to be careful about the credentials handling. The existing `checkwatches.credentials` stuff was intended for cases where the bug tracker itself is essentially public but we need some kind of credentials for Launchpad to connect to it anyway, either out of politeness (credentials allow our sync script to be identified unambiguously), or to gain access to higher rate limits (as in the GitHub/GitLab cases), or because we need to push comments (for Bugzilla). In this case, though, the credentials are partly also being used because the remote bug tracker is private, which is a very different matter: if we were to configure these credentials on production, it would mean that anyone could discover information about the status of a given Jira issue by guessing its URL and adding a bug watch for it. Not a very serious information leak since it only tells you the remote status and importance, but nevertheless probably not something we should leave designed into the system in case somebody wants to extend it in future to gather more information. I think it's fine to leave an XXX comment about this for now, though, as it doesn't become a problem until we configure credentials; perhaps we could change some other part of the system to restrict who can add such bug watches, or restrict them to certain projects, or something like that.

Having gathered requirements for Launchpad/Jira integration, I also think this will probably not address those requirements on its own (though it may be a component of the eventual solution). I've belatedly written down what I know so far here: https://docs.google.com/document/d/1CiEgo-CHX8Go0lTAdryqKCeFxaVnBq49QVkfBshX28M

review: Approve

Unmerged commits

9f871dc... by Andrey Fedoseev

Add external bug tracker for JIRA

Succeeded
[SUCCEEDED] docs:0 (build)
[SUCCEEDED] lint:0 (build)
[SUCCEEDED] mypy:0 (build)
13 of 3 results

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/lib/lp/bugs/externalbugtracker/__init__.py b/lib/lp/bugs/externalbugtracker/__init__.py
2index 65c5333..d4b887d 100644
3--- a/lib/lp/bugs/externalbugtracker/__init__.py
4+++ b/lib/lp/bugs/externalbugtracker/__init__.py
5@@ -51,6 +51,7 @@ from lp.bugs.externalbugtracker.bugzilla import Bugzilla
6 from lp.bugs.externalbugtracker.debbugs import DebBugs, DebBugsDatabaseNotFound
7 from lp.bugs.externalbugtracker.github import GitHub
8 from lp.bugs.externalbugtracker.gitlab import GitLab
9+from lp.bugs.externalbugtracker.jira import Jira
10 from lp.bugs.externalbugtracker.mantis import Mantis
11 from lp.bugs.externalbugtracker.roundup import Roundup
12 from lp.bugs.externalbugtracker.rt import RequestTracker
13@@ -68,6 +69,7 @@ BUG_TRACKER_CLASSES = {
14 BugTrackerType.ROUNDUP: Roundup,
15 BugTrackerType.RT: RequestTracker,
16 BugTrackerType.SOURCEFORGE: SourceForge,
17+ BugTrackerType.JIRA: Jira,
18 }
19
20
21diff --git a/lib/lp/bugs/externalbugtracker/base.py b/lib/lp/bugs/externalbugtracker/base.py
22index ee7be38..0d2efd4 100644
23--- a/lib/lp/bugs/externalbugtracker/base.py
24+++ b/lib/lp/bugs/externalbugtracker/base.py
25@@ -279,16 +279,17 @@ class ExternalBugTracker:
26 except requests.RequestException as e:
27 raise BugTrackerConnectError(self.baseurl, e)
28
29- def _postPage(self, page, form, repost_on_redirect=False):
30+ def _postPage(self, page, data, repost_on_redirect=False, json=False):
31 """POST to the specified page and form.
32
33- :param form: is a dict of form variables being POSTed.
34+ :param data: is a dict of form variables being POSTed.
35 :param repost_on_redirect: override RFC-compliant redirect handling.
36 By default, if the POST receives a redirect response, the
37 request to the redirection's target URL will be a GET. If
38 `repost_on_redirect` is True, this method will do a second POST
39 instead. Do this only if you are sure that repeated POST to
40 this page is safe, as is usually the case with search forms.
41+ :param json: if True, the data will be JSON encoded.
42 :return: A `requests.Response` object.
43 """
44 hooks = (
45@@ -301,8 +302,12 @@ class ExternalBugTracker:
46 if not url.endswith("/"):
47 url += "/"
48 url = urljoin(url, page)
49+ if json:
50+ kwargs = {"json": data}
51+ else:
52+ kwargs = {"data": data}
53 response = self.makeRequest(
54- "POST", url, headers=self._getHeaders(), data=form, hooks=hooks
55+ "POST", url, headers=self._getHeaders(), hooks=hooks, **kwargs
56 )
57 raise_for_status_redacted(response)
58 return response
59diff --git a/lib/lp/bugs/externalbugtracker/jira.py b/lib/lp/bugs/externalbugtracker/jira.py
60new file mode 100644
61index 0000000..b529c5b
62--- /dev/null
63+++ b/lib/lp/bugs/externalbugtracker/jira.py
64@@ -0,0 +1,268 @@
65+# Copyright 2022 Canonical Ltd. This software is licensed under the
66+# GNU Affero General Public License version 3 (see the file LICENSE).
67+
68+"""Jira ExternalBugTracker utility."""
69+
70+__all__ = [
71+ "Jira",
72+ "JiraCredentials",
73+ "JiraBug",
74+ "JiraStatus",
75+ "JiraPriority",
76+]
77+
78+import base64
79+import datetime
80+from enum import Enum
81+from typing import Dict, Iterable, NamedTuple, Optional, Tuple
82+from urllib.parse import urlunsplit
83+
84+import dateutil.parser
85+
86+from lp.bugs.externalbugtracker import (
87+ BugTrackerConnectError,
88+ ExternalBugTracker,
89+)
90+from lp.bugs.interfaces.bugtask import BugTaskImportance, BugTaskStatus
91+from lp.services.config import config
92+from lp.services.webapp.url import urlsplit
93+
94+JiraCredentials = NamedTuple(
95+ "JiraCredentials",
96+ (
97+ ("username", str),
98+ ("password", str),
99+ ),
100+)
101+
102+
103+class JiraStatus(Enum):
104+
105+ UNDEFINED = "undefined"
106+ NEW = "new"
107+ INDETERMINATE = "indeterminate"
108+ DONE = "done"
109+
110+ @property
111+ def launchpad_status(self):
112+ if self == JiraStatus.UNDEFINED:
113+ return BugTaskStatus.UNKNOWN
114+ elif self == JiraStatus.NEW:
115+ return BugTaskStatus.NEW
116+ elif self == JiraStatus.INDETERMINATE:
117+ return BugTaskStatus.INPROGRESS
118+ elif self == JiraStatus.DONE:
119+ return BugTaskStatus.FIXRELEASED
120+ else:
121+ raise AssertionError()
122+
123+
124+class JiraPriority(Enum):
125+
126+ UNDEFINED = "undefined"
127+ LOWEST = "Lowest"
128+ LOW = "Low"
129+ MEDIUM = "Medium"
130+ HIGH = "High"
131+ HIGHEST = "Highest"
132+
133+ @property
134+ def launchpad_importance(self):
135+ if self == JiraPriority.UNDEFINED:
136+ return BugTaskImportance.UNKNOWN
137+ elif self == JiraPriority.LOWEST:
138+ return BugTaskImportance.WISHLIST
139+ elif self == JiraPriority.LOW:
140+ return BugTaskImportance.LOW
141+ elif self == JiraPriority.MEDIUM:
142+ return BugTaskImportance.MEDIUM
143+ elif self == JiraPriority.HIGH:
144+ return BugTaskImportance.HIGH
145+ elif self == JiraPriority.HIGHEST:
146+ return BugTaskImportance.CRITICAL
147+ else:
148+ raise AssertionError()
149+
150+
151+class JiraBug:
152+ def __init__(self, key: str, status: JiraStatus, priority: JiraPriority):
153+ self.key = key
154+ self.status = status
155+ self.priority = priority
156+
157+ @classmethod
158+ def from_api_data(cls, bug_data) -> "JiraBug":
159+ try:
160+ status = JiraStatus(
161+ bug_data["fields"]["status"]["statusCategory"]["key"]
162+ )
163+ except ValueError:
164+ status = JiraStatus.UNDEFINED
165+
166+ try:
167+ priority = JiraPriority(bug_data["fields"]["priority"]["name"])
168+ except ValueError:
169+ priority = JiraPriority.UNDEFINED
170+
171+ return cls(
172+ key=bug_data["key"],
173+ status=status,
174+ priority=priority,
175+ )
176+
177+ def __eq__(self, other):
178+ if not isinstance(other, JiraBug):
179+ raise ValueError()
180+ return (
181+ self.key == other.key
182+ and self.status == other.status
183+ and self.priority == other.priority
184+ )
185+
186+
187+class Jira(ExternalBugTracker):
188+ """An `ExternalBugTracker` for dealing with Jira issues."""
189+
190+ batch_query_threshold = 0 # Always use the batch method.
191+
192+ def __init__(self, baseurl):
193+ _, host, path, query, fragment = urlsplit(baseurl)
194+ path = "/rest/api/2/"
195+ baseurl = urlunsplit(("https", host, path, "", ""))
196+ super().__init__(baseurl)
197+ self.cached_bugs = {} # type: Dict[str, Optional[JiraBug]]
198+
199+ @property
200+ def credentials(self) -> Optional[JiraCredentials]:
201+ credentials_config = config["checkwatches.credentials"]
202+ # lazr.config.Section doesn't support get().
203+ try:
204+ username = credentials_config["{}.username".format(self.basehost)]
205+ password = credentials_config["{}.password".format(self.basehost)]
206+ return JiraCredentials(
207+ username=username,
208+ password=password,
209+ )
210+ except KeyError:
211+ return
212+
213+ def getCurrentDBTime(self):
214+ # See https://docs.atlassian.com/software/jira/docs/api/REST/9.3.1/#api/2/serverInfo-getServerInfo # noqa
215+ response_data = self._getPage("serverInfo").json()
216+ return dateutil.parser.parse(response_data["serverTime"]).astimezone(
217+ datetime.timezone.utc
218+ )
219+
220+ def getModifiedRemoteBugs(self, bug_ids, last_accessed):
221+ """See `IExternalBugTracker`."""
222+ modified_bugs = self.getRemoteBugBatch(
223+ bug_ids, last_accessed=last_accessed
224+ )
225+ self.cached_bugs.update(modified_bugs)
226+ return list(modified_bugs)
227+
228+ def getRemoteBug(self, bug_id: str) -> Tuple[str, Optional[JiraBug]]:
229+ """See `ExternalBugTracker`."""
230+ if bug_id not in self.cached_bugs:
231+ self.cached_bugs[bug_id] = self._loadJiraBug(bug_id)
232+ return bug_id, self.cached_bugs[bug_id]
233+
234+ def getRemoteBugBatch(
235+ self, bug_ids, last_accessed=None
236+ ) -> Dict[str, Optional[JiraBug]]:
237+ """See `ExternalBugTracker`."""
238+ bugs = {
239+ bug_id: self.cached_bugs[bug_id]
240+ for bug_id in bug_ids
241+ if bug_id in self.cached_bugs
242+ }
243+ if set(bugs) == set(bug_ids):
244+ return bugs
245+
246+ for jira_bug in self._loadJiraBugs(
247+ bug_ids, last_accessed=last_accessed
248+ ):
249+ if jira_bug.key not in bug_ids:
250+ continue
251+ bugs[jira_bug.key] = self.cached_bugs[jira_bug.key] = jira_bug
252+
253+ return bugs
254+
255+ def getRemoteImportance(self, bug_id) -> str:
256+ """See `ExternalBugTracker`."""
257+ remote_bug = self.bugs[bug_id] # type: JiraBug
258+ return remote_bug.priority.value
259+
260+ def getRemoteStatus(self, bug_id) -> str:
261+ """See `ExternalBugTracker`."""
262+ remote_bug = self.bugs[bug_id] # type: JiraBug
263+ return remote_bug.status.value
264+
265+ def convertRemoteImportance(
266+ self, remote_importance: str
267+ ) -> BugTaskImportance:
268+ """See `IExternalBugTracker`."""
269+ return JiraPriority(remote_importance).launchpad_importance
270+
271+ def convertRemoteStatus(self, remote_status: str) -> BugTaskStatus:
272+ """See `IExternalBugTracker`."""
273+ return JiraStatus(remote_status).launchpad_status
274+
275+ def _getHeaders(self):
276+ headers = super()._getHeaders()
277+ credentials = self.credentials
278+ if credentials:
279+ headers["Authorization"] = "Basic {}".format(
280+ base64.b64encode(
281+ "{}:{}".format(
282+ credentials.username, credentials.password
283+ ).encode()
284+ ).decode()
285+ )
286+ return headers
287+
288+ def _loadJiraBug(self, bug_id: str) -> Optional[JiraBug]:
289+ # See https://docs.atlassian.com/software/jira/docs/api/REST/9.3.1/#api/2/issue-getIssue # noqa
290+ try:
291+ response = self._getPage(
292+ "issue/{}".format(bug_id),
293+ params={
294+ "fields": "status,priority",
295+ },
296+ )
297+ except BugTrackerConnectError:
298+ return
299+
300+ return JiraBug.from_api_data(response.json())
301+
302+ def _loadJiraBugs(
303+ self, bug_ids, last_accessed=None, start_at=0
304+ ) -> Iterable[JiraBug]:
305+ # See https://docs.atlassian.com/software/jira/docs/api/REST/9.3.1/#api/2/search-searchUsingSearchRequest # noqa
306+
307+ jql_query = "id in ({})".format(",".join(bug_ids))
308+ if last_accessed is not None:
309+ jql_query = "{} AND updated >= {}".format(
310+ jql_query, last_accessed.strftime("%Y-%m-%d %H:%M")
311+ )
312+
313+ params = {
314+ "jql": jql_query,
315+ "fields": ["status", "priority"],
316+ "startAt": start_at,
317+ }
318+
319+ response_data = self._postPage("search", data=params, json=True).json()
320+
321+ max_results = response_data["maxResults"]
322+ total = response_data["total"]
323+
324+ for bug_data in response_data["issues"]:
325+ yield JiraBug.from_api_data(bug_data)
326+
327+ if total > (start_at + max_results):
328+ yield from self._loadJiraBugs(
329+ bug_ids,
330+ last_accessed=last_accessed,
331+ start_at=start_at + max_results,
332+ )
333diff --git a/lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py b/lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py
334index 03b3197..2da0866 100644
335--- a/lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py
336+++ b/lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py
337@@ -167,7 +167,7 @@ class TestCheckwatchesConfig(TestCase):
338 )
339 responses.add("POST", base_url + target, body=fake_form)
340
341- bugtracker._postPage(form, form={}, repost_on_redirect=True)
342+ bugtracker._postPage(form, {}, repost_on_redirect=True)
343
344 requests = [call.request for call in responses.calls]
345 self.assertThat(
346diff --git a/lib/lp/bugs/externalbugtracker/tests/test_jira.py b/lib/lp/bugs/externalbugtracker/tests/test_jira.py
347new file mode 100644
348index 0000000..1301dce
349--- /dev/null
350+++ b/lib/lp/bugs/externalbugtracker/tests/test_jira.py
351@@ -0,0 +1,432 @@
352+# Copyright 2022 Canonical Ltd. This software is licensed under the
353+# GNU Affero General Public License version 3 (see the file LICENSE).
354+import datetime
355+import json
356+
357+import responses
358+import transaction
359+from testtools.matchers import (
360+ ContainsDict,
361+ Equals,
362+ MatchesListwise,
363+ MatchesStructure,
364+ StartsWith,
365+)
366+from zope.component import getUtility
367+
368+from lp.app.interfaces.launchpad import ILaunchpadCelebrities
369+from lp.bugs.externalbugtracker import get_external_bugtracker
370+from lp.bugs.externalbugtracker.jira import (
371+ Jira,
372+ JiraBug,
373+ JiraCredentials,
374+ JiraPriority,
375+ JiraStatus,
376+)
377+from lp.bugs.interfaces.bugtask import BugTaskStatus
378+from lp.bugs.interfaces.bugtracker import BugTrackerType
379+from lp.bugs.interfaces.externalbugtracker import IExternalBugTracker
380+from lp.bugs.scripts.checkwatches import CheckwatchesMaster
381+from lp.services.log.logger import BufferLogger
382+from lp.testing import TestCase, TestCaseWithFactory, verifyObject
383+from lp.testing.layers import ZopelessDatabaseLayer, ZopelessLayer
384+
385+
386+class TestJira(TestCase):
387+
388+ layer = ZopelessLayer
389+
390+ def setUp(self):
391+ super().setUp()
392+ self.jira = Jira("https://warthogs.atlassian.net")
393+ self.pushConfig(
394+ "checkwatches.credentials",
395+ **{
396+ "warthogs.atlassian.net.username": "launchpad",
397+ "warthogs.atlassian.net.password": "launchpad",
398+ },
399+ )
400+
401+ def test_implements_interface(self):
402+ self.assertTrue(verifyObject(IExternalBugTracker, self.jira))
403+
404+ def test_convert_jira_url_to_api_endpoint(self):
405+ self.assertEqual(
406+ "https://warthogs.atlassian.net/rest/api/2", self.jira.baseurl
407+ )
408+
409+ def test_credentials(self):
410+ self.assertEqual(
411+ JiraCredentials(
412+ username="launchpad",
413+ password="launchpad",
414+ ),
415+ self.jira.credentials,
416+ )
417+
418+ def test_getHeaders(self):
419+ headers = self.jira._getHeaders()
420+ self.assertThat(
421+ headers,
422+ ContainsDict(
423+ {"Authorization": Equals("Basic bGF1bmNocGFkOmxhdW5jaHBhZA==")}
424+ ),
425+ )
426+
427+ @responses.activate
428+ def test_getCurrentDBTime(self):
429+ responses.add(
430+ "GET",
431+ self.jira.baseurl + "/serverInfo",
432+ json={
433+ "baseUrl": "https://warthogs.atlassian.net",
434+ "buildDate": "2022-11-15T06:27:18.000+0800",
435+ "buildNumber": 100210,
436+ "defaultLocale": {"locale": "en_US"},
437+ "deploymentType": "Cloud",
438+ "scmInfo": "28a36363a81be3fec088cc03de57ea0d3b868a26",
439+ "serverTime": "2022-11-15T14:11:11.818+0800",
440+ "serverTitle": "Jira",
441+ "version": "1001.0.0-SNAPSHOT",
442+ "versionNumbers": [1001, 0, 0],
443+ },
444+ )
445+ self.assertEqual(
446+ self.jira.getCurrentDBTime(),
447+ datetime.datetime(
448+ 2022, 11, 15, 6, 11, 11, 818000, tzinfo=datetime.timezone.utc
449+ ),
450+ )
451+ requests = [call.request for call in responses.calls]
452+ self.assertThat(
453+ requests,
454+ MatchesListwise(
455+ [
456+ MatchesStructure(
457+ method=Equals("GET"),
458+ path_url=Equals("/rest/api/2/serverInfo"),
459+ headers=ContainsDict(
460+ {"Authorization": StartsWith("Basic ")}
461+ ),
462+ ),
463+ ]
464+ ),
465+ )
466+
467+ @responses.activate
468+ def test_getRemoteBug(self):
469+ responses.add(
470+ "GET",
471+ self.jira.baseurl + "/issue/LP-984",
472+ json={
473+ "fields": {
474+ "priority": {"name": "Medium"},
475+ "status": {"statusCategory": {"key": "indeterminate"}},
476+ },
477+ "key": "LP-984",
478+ },
479+ )
480+ responses.add("GET", self.jira.baseurl + "/issue/LP-123", status=404)
481+ self.assertEqual(
482+ (
483+ "LP-984",
484+ JiraBug(
485+ key="LP-984",
486+ status=JiraStatus.INDETERMINATE,
487+ priority=JiraPriority.MEDIUM,
488+ ),
489+ ),
490+ self.jira.getRemoteBug("LP-984"),
491+ )
492+ self.assertEqual(("LP-123", None), self.jira.getRemoteBug("LP-123"))
493+
494+ requests = [call.request for call in responses.calls]
495+ self.assertThat(
496+ requests,
497+ MatchesListwise(
498+ [
499+ MatchesStructure(
500+ method=Equals("GET"),
501+ path_url=Equals(
502+ "/rest/api/2/issue/LP-984?fields=status%2Cpriority"
503+ ),
504+ ),
505+ MatchesStructure(
506+ method=Equals("GET"),
507+ path_url=Equals(
508+ "/rest/api/2/issue/LP-123?fields=status%2Cpriority"
509+ ),
510+ ),
511+ ]
512+ ),
513+ )
514+
515+ # Getting the same bug the second time should fetch it from the cache
516+ # without making another request to JIRA API
517+ self.assertEqual(
518+ (
519+ "LP-984",
520+ JiraBug(
521+ key="LP-984",
522+ status=JiraStatus.INDETERMINATE,
523+ priority=JiraPriority.MEDIUM,
524+ ),
525+ ),
526+ self.jira.getRemoteBug("LP-984"),
527+ )
528+ self.assertEqual(("LP-123", None), self.jira.getRemoteBug("LP-123"))
529+ self.assertEqual(2, len(responses.calls))
530+
531+ @responses.activate
532+ def test_getRemoteBugBatch(self):
533+
534+ existing_bugs = [
535+ {
536+ "fields": {
537+ "priority": {"name": "High"},
538+ "status": {"statusCategory": {"key": "indeterminate"}},
539+ },
540+ "key": "1",
541+ },
542+ {
543+ "fields": {
544+ "priority": {"name": "Medium"},
545+ "status": {"statusCategory": {"key": "done"}},
546+ },
547+ "key": "2",
548+ },
549+ ]
550+
551+ def search_callback(request):
552+ payload = json.loads(request.body.decode())
553+ start_at = payload["startAt"]
554+
555+ if start_at >= len(existing_bugs):
556+ return 404, {}, ""
557+
558+ return (
559+ 200,
560+ {},
561+ json.dumps(
562+ {
563+ "issues": existing_bugs[start_at : start_at + 1],
564+ "total": len(existing_bugs),
565+ "startAt": start_at,
566+ "maxResults": 1,
567+ }
568+ ),
569+ )
570+
571+ responses.add_callback(
572+ "POST",
573+ self.jira.baseurl + "/search",
574+ callback=search_callback,
575+ content_type="application/json",
576+ )
577+
578+ self.assertDictEqual(
579+ {
580+ "1": JiraBug(
581+ key="1",
582+ status=JiraStatus.INDETERMINATE,
583+ priority=JiraPriority.HIGH,
584+ ),
585+ "2": JiraBug(
586+ key="2",
587+ status=JiraStatus.DONE,
588+ priority=JiraPriority.MEDIUM,
589+ ),
590+ },
591+ self.jira.getRemoteBugBatch(["1", "2"]),
592+ )
593+
594+ requests = [call.request for call in responses.calls]
595+ self.assertThat(
596+ requests,
597+ MatchesListwise(
598+ [
599+ MatchesStructure(
600+ method=Equals("POST"),
601+ path_url=Equals("/rest/api/2/search"),
602+ ),
603+ MatchesStructure(
604+ method=Equals("POST"),
605+ path_url=Equals("/rest/api/2/search"),
606+ ),
607+ ]
608+ ),
609+ )
610+
611+ for i, call in enumerate(responses.calls):
612+ payload = json.loads(call.request.body.decode())
613+ self.assertEqual("id in (1,2)", payload["jql"])
614+ self.assertEqual(["status", "priority"], payload["fields"])
615+ self.assertEqual(i, payload["startAt"])
616+
617+ # Getting the same bugs the second time should fetch it from the cache
618+ # without making another request to JIRA API
619+ self.assertDictEqual(
620+ {
621+ "1": JiraBug(
622+ key="1",
623+ status=JiraStatus.INDETERMINATE,
624+ priority=JiraPriority.HIGH,
625+ ),
626+ "2": JiraBug(
627+ key="2",
628+ status=JiraStatus.DONE,
629+ priority=JiraPriority.MEDIUM,
630+ ),
631+ },
632+ self.jira.getRemoteBugBatch(["1", "2"]),
633+ )
634+ self.assertEqual(2, len(responses.calls))
635+
636+ # Verify JQL query when `last_accessed` is specified
637+ self.jira.getRemoteBugBatch(
638+ ["3"], last_accessed=datetime.datetime(2000, 1, 1, 1, 2, 3)
639+ )
640+ payload = json.loads(responses.calls[-1].request.body.decode())
641+ self.assertEqual(
642+ "id in (3) AND updated >= 2000-01-01 01:02", payload["jql"]
643+ )
644+
645+
646+class TestJiraUpdateBugWatches(TestCaseWithFactory):
647+
648+ layer = ZopelessDatabaseLayer
649+
650+ @responses.activate
651+ def test_process_one(self):
652+ responses.add(
653+ "GET",
654+ "https://warthogs.atlassian.net/rest/api/2/issue/LP-984",
655+ json={
656+ "fields": {
657+ "priority": {"name": "Medium"},
658+ "status": {"statusCategory": {"key": "indeterminate"}},
659+ },
660+ "key": "LP-984",
661+ },
662+ )
663+ responses.add(
664+ "GET",
665+ "https://warthogs.atlassian.net/rest/api/2/serverInfo",
666+ json={
667+ "serverTime": datetime.datetime.now(
668+ tz=datetime.timezone.utc
669+ ).isoformat()
670+ },
671+ )
672+ bug_tracker = self.factory.makeBugTracker(
673+ base_url="https://warthogs.atlassian.net",
674+ bugtrackertype=BugTrackerType.JIRA,
675+ )
676+ bug = self.factory.makeBug()
677+ bug.addWatch(
678+ bug_tracker, "LP-984", getUtility(ILaunchpadCelebrities).janitor
679+ )
680+ self.assertEqual(
681+ [("LP-984", None)],
682+ [
683+ (watch.remotebug, watch.remotestatus)
684+ for watch in bug_tracker.watches
685+ ],
686+ )
687+ transaction.commit()
688+ logger = BufferLogger()
689+ bug_watch_updater = CheckwatchesMaster(transaction, logger=logger)
690+ jira = get_external_bugtracker(bug_tracker)
691+ jira.batch_query_threshold = 1
692+ bug_watch_updater.updateBugWatches(jira, bug_tracker.watches)
693+ self.assertEqual(
694+ "INFO Updating 1 watches for 1 bugs on "
695+ "https://warthogs.atlassian.net/rest/api/2\n",
696+ logger.getLogBuffer(),
697+ )
698+ self.assertEqual(
699+ [("LP-984", BugTaskStatus.INPROGRESS)],
700+ [
701+ (
702+ watch.remotebug,
703+ jira.convertRemoteStatus(watch.remotestatus),
704+ )
705+ for watch in bug_tracker.watches
706+ ],
707+ )
708+
709+ @responses.activate
710+ def test_process_many(self):
711+ remote_bugs = [
712+ {
713+ "fields": {
714+ "priority": {"name": "Medium"},
715+ "status": {
716+ "statusCategory": {
717+ "key": "indeterminate"
718+ if (bug_id % 2) == 0
719+ else "done"
720+ }
721+ },
722+ },
723+ "key": str(bug_id),
724+ }
725+ for bug_id in range(1000, 1010)
726+ ]
727+ responses.add(
728+ "POST",
729+ "https://warthogs.atlassian.net/rest/api/2/search",
730+ json={
731+ "startAt": 0,
732+ "maxResults": 100,
733+ "total": len(remote_bugs),
734+ "issues": remote_bugs,
735+ },
736+ )
737+ responses.add(
738+ "GET",
739+ "https://warthogs.atlassian.net/rest/api/2/serverInfo",
740+ json={
741+ "serverTime": datetime.datetime.now(
742+ tz=datetime.timezone.utc
743+ ).isoformat()
744+ },
745+ )
746+ bug = self.factory.makeBug()
747+ bug_tracker = self.factory.makeBugTracker(
748+ base_url="https://warthogs.atlassian.net",
749+ bugtrackertype=BugTrackerType.JIRA,
750+ )
751+ for remote_bug in remote_bugs:
752+ bug.addWatch(
753+ bug_tracker,
754+ remote_bug["key"],
755+ getUtility(ILaunchpadCelebrities).janitor,
756+ )
757+ transaction.commit()
758+ logger = BufferLogger()
759+ bug_watch_updater = CheckwatchesMaster(transaction, logger=logger)
760+ jira = get_external_bugtracker(bug_tracker)
761+ bug_watch_updater.updateBugWatches(jira, bug_tracker.watches)
762+ self.assertEqual(
763+ "INFO Updating 10 watches for 10 bugs on "
764+ "https://warthogs.atlassian.net/rest/api/2\n",
765+ logger.getLogBuffer(),
766+ )
767+ self.assertContentEqual(
768+ [
769+ (str(bug_id), BugTaskStatus.INPROGRESS)
770+ for bug_id in (1000, 1002, 1004, 1006, 1008)
771+ ]
772+ + [
773+ (str(bug_id), BugTaskStatus.FIXRELEASED)
774+ for bug_id in (1001, 1003, 1005, 1007, 1009)
775+ ],
776+ [
777+ (
778+ watch.remotebug,
779+ jira.convertRemoteStatus(watch.remotestatus),
780+ )
781+ for watch in bug_tracker.watches
782+ ],
783+ )
784diff --git a/lib/lp/bugs/interfaces/bugtracker.py b/lib/lp/bugs/interfaces/bugtracker.py
785index 3f383e3..a44c99b 100644
786--- a/lib/lp/bugs/interfaces/bugtracker.py
787+++ b/lib/lp/bugs/interfaces/bugtracker.py
788@@ -219,6 +219,15 @@ class BugTrackerType(DBEnumeratedType):
789 """,
790 )
791
792+ JIRA = DBItem(
793+ 14,
794+ """
795+ JIRA Issues
796+
797+ The issue tracker for JIRA-based projects.
798+ """,
799+ )
800+
801
802 # A list of the BugTrackerTypes that don't need a remote product to be
803 # able to return a bug filing URL. We use a whitelist rather than a
804diff --git a/lib/lp/bugs/model/bugwatch.py b/lib/lp/bugs/model/bugwatch.py
805index 2bbef55..83fe3cd 100644
806--- a/lib/lp/bugs/model/bugwatch.py
807+++ b/lib/lp/bugs/model/bugwatch.py
808@@ -70,6 +70,7 @@ BUG_TRACKER_URL_FORMATS = {
809 BugTrackerType.TRAC: "ticket/%s",
810 BugTrackerType.SAVANE: "bugs/?%s",
811 BugTrackerType.PHPPROJECT: "bug.php?id=%s",
812+ BugTrackerType.JIRA: "%s",
813 }
814
815
816@@ -418,6 +419,7 @@ class BugWatchSet:
817 BugTrackerType.SAVANE: self.parseSavaneURL,
818 BugTrackerType.SOURCEFORGE: self.parseSourceForgeLikeURL,
819 BugTrackerType.TRAC: self.parseTracURL,
820+ BugTrackerType.JIRA: self.parseJiraURL,
821 }
822
823 def get(self, watch_id):
824@@ -745,6 +747,15 @@ class BugWatchSet:
825 base_url = urlunsplit((scheme, host, base_path, "", ""))
826 return base_url, remote_bug
827
828+ def parseJiraURL(self, scheme, host, path, query):
829+ """Extract a JIRA issue base URL and bug ID."""
830+ match = re.match(r"^/browse/([A-Z]{1,10}-\d+)$", path)
831+ if not match:
832+ return None
833+ remote_bug = match.group(1)
834+ base_url = urlunsplit((scheme, host, "/", "", ""))
835+ return base_url, remote_bug
836+
837 def extractBugTrackerAndBug(self, url):
838 """See `IBugWatchSet`."""
839 for trackertype, parse_func in self.bugtracker_parse_functions.items():
840diff --git a/lib/lp/bugs/tests/test_bugwatch.py b/lib/lp/bugs/tests/test_bugwatch.py
841index a71a7d0..275761b 100644
842--- a/lib/lp/bugs/tests/test_bugwatch.py
843+++ b/lib/lp/bugs/tests/test_bugwatch.py
844@@ -224,6 +224,15 @@ class ExtractBugTrackerAndBugTest(WithScenarios, TestCase):
845 "bug_id": "12345",
846 },
847 ),
848+ (
849+ "JIRA",
850+ {
851+ "bugtracker_type": BugTrackerType.JIRA,
852+ "bug_url": "https://warthogs.atlassian.net/browse/LP-984",
853+ "base_url": "https://warthogs.atlassian.net/",
854+ "bug_id": "LP-984",
855+ },
856+ ),
857 ]
858
859 layer = LaunchpadFunctionalLayer
860diff --git a/lib/lp/services/config/schema-lazr.conf b/lib/lp/services/config/schema-lazr.conf
861index 640865d..da9477e 100644
862--- a/lib/lp/services/config/schema-lazr.conf
863+++ b/lib/lp/services/config/schema-lazr.conf
864@@ -237,7 +237,8 @@ api.github.com.token: none
865 gitlab.com.token: none
866 gitlab.gnome.org.token: none
867 salsa.debian.org.token: none
868-
869+warthogs.atlassian.net.username: none
870+warthogs.atlassian.net.password: none
871
872 [cibuild.soss]
873 # value is a JSON Object

Subscribers

People subscribed via source and target branches

to status/vote changes: