Merge lp:~chad.smith/lp2kanban/branch-to-card into lp:~launchpad/lp2kanban/trunk
- branch-to-card
- Merge into trunk
Status: | Superseded |
---|---|
Proposed branch: | lp:~chad.smith/lp2kanban/branch-to-card |
Merge into: | lp:~launchpad/lp2kanban/trunk |
Diff against target: |
964 lines (+460/-83) 14 files modified
.bzrignore (+1/-0) Makefile (+2/-2) README.landscape (+17/-0) bootstrap.py (+66/-42) buildout.cfg (+2/-2) configs/groups.ini (+16/-0) configs/sync.ini (+106/-0) create_creds.py (+1/-1) jenkins.sh (+32/-0) run.sh (+21/-0) src/lp2kanban/bugs2cards.py (+57/-10) src/lp2kanban/kanban.py (+78/-16) src/lp2kanban/tests/common.py (+5/-2) src/lp2kanban/tests/test_bugs2cards.py (+56/-8) |
To merge this branch: | bzr merge lp:~chad.smith/lp2kanban/branch-to-card |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Данило Шеган (community) | Needs Resubmitting | ||
Landscape | Pending | ||
Review via email: mp+280669@code.launchpad.net |
Commit message
Description of the change
Add initial support for branch cards and taskboards. This branch only handles syncing and moving branch "MP/blue" cards on the Landscape 2016 board.
Support was added for branch cards inside bugs2card because a lot of the logic is common and can hopefully be consolidated in a subsequent branch.
There will be a followup branch with added logic in bugs2cards to call moveToTaskBoard for feature cards that are manually dragged to the "$Group::Landed" lane.
src/lp2kanban/
- Define initial LeankitTaskBoard class
- Add optional CurrentTaskBoardId property to LeankitCard
- Add LeankitCard.
- Since Taskboard API calls live on a different route, rework API routes defined on the LeankitConnector class
src/lp2kanban/
- _get_mp_info to handle non-bug branch records when parsing linked MPs
- should_sync_card to return true if external_system_url is a branch
- get_card_status returns proper status for cards that have external_
- add a loop to sync_board function which processes all "branch cards" through board.getCardsW
config/sync.ini:
- comment out [Landscape Cisco] board definition (as that should live in this board now
To test:
buildout
./bin/test
# to test against landscape 201
./create_creds.py (this will create your creds to modify the shared 2016 board)
Select "Change anything" button in your browser to allow updating cards
./bin/py src/lp2kanban/
# Tweak branches or drag cards at https:/
What's lacking:
unit tests to cover bugs2card sync_board (coming in a followup branch)
Данило Шеган (danilo) wrote : | # |
Note that I've also split out configs now since it was too easy to leak credentials (a bzr push in the lp2kanban namespace creates a public branch). Please review my drop-configs branch.
The kanban-sync job has been updated to use the new credentials (I've updated the password as well).
- 135. By Chad Smith
-
simplify moveToTaskBoard card method
- 136. By Chad Smith
-
merge ~landscape/
lp2kanban/ landscape- deploy to drop configs - 137. By Chad Smith
-
address review comments: approved branches that have not yet been merged will remain in the status LANDING. update copyright.
- 138. By Chad Smith
-
fix has_branch conditional to first check if card.external_
system_ url is not None before trying to match BRANCH_REGEX - 139. By Chad Smith
-
card.save needs to be taskboard aware. When calling UpdateCard the parent_board.id needs to be specified, not the taskboard.id
- 140. By Chad Smith
-
add parent_card param to LeankitTaskBoard so that we have a refernce to the taskboard's card on the parent_board. Use the parent_card for card path printing
- 141. By Chad Smith
-
merge landscape-deploy to get latest changes to jenkins.sh
- 142. By Chad Smith
-
revert test makefile change
Unmerged revisions
- 142. By Chad Smith
-
revert test makefile change
- 141. By Chad Smith
-
merge landscape-deploy to get latest changes to jenkins.sh
- 140. By Chad Smith
-
add parent_card param to LeankitTaskBoard so that we have a refernce to the taskboard's card on the parent_board. Use the parent_card for card path printing
- 139. By Chad Smith
-
card.save needs to be taskboard aware. When calling UpdateCard the parent_board.id needs to be specified, not the taskboard.id
- 138. By Chad Smith
-
fix has_branch conditional to first check if card.external_
system_ url is not None before trying to match BRANCH_REGEX - 137. By Chad Smith
-
address review comments: approved branches that have not yet been merged will remain in the status LANDING. update copyright.
- 136. By Chad Smith
-
merge ~landscape/
lp2kanban/ landscape- deploy to drop configs - 135. By Chad Smith
-
simplify moveToTaskBoard card method
- 134. By Chad Smith
-
process cards with linked branches in bugs2cards since a lot of the logic is common
- 133. By Chad Smith
-
landing_lanes == Dev::::Landed for config
Preview Diff
1 | === modified file '.bzrignore' |
2 | --- .bzrignore 2012-09-15 18:20:28 +0000 |
3 | +++ .bzrignore 2015-12-16 04:22:55 +0000 |
4 | @@ -14,3 +14,4 @@ |
5 | .emacs.desktop |
6 | credentials |
7 | .emacs.desktop.lock |
8 | +logs/* |
9 | |
10 | === modified file 'Makefile' |
11 | --- Makefile 2013-03-18 15:45:56 +0000 |
12 | +++ Makefile 2015-12-16 04:22:55 +0000 |
13 | @@ -23,12 +23,12 @@ |
14 | |
15 | |
16 | bin/buildout: download-cache |
17 | - $(PYTHON) bootstrap.py -v 1.7.0 |
18 | + $(PYTHON) bootstrap.py -v 1.7.1 -f eggs -c buildout.cfg |
19 | touch --no-create $@ |
20 | |
21 | |
22 | $(PY): bin/buildout $(BUILDOUT_CFG) setup.py |
23 | - PYTHONPATH=. ./bin/buildout -c $(BUILDOUT_CFG) |
24 | + ./bin/buildout -c $(BUILDOUT_CFG) |
25 | |
26 | |
27 | $(subst $(PY),,$(BUILDOUT_BIN)): $(PY) |
28 | |
29 | === added file 'README.landscape' |
30 | --- README.landscape 1970-01-01 00:00:00 +0000 |
31 | +++ README.landscape 2015-12-16 04:22:55 +0000 |
32 | @@ -0,0 +1,17 @@ |
33 | +This branch is based off lp:~bjornt/lp2kanban/landscape and contains the |
34 | +configuration files needed for lp2kanban to process bugs for the Landscape |
35 | +project. |
36 | + |
37 | +To deploy, first run ./create_creds.py to create the credentials file. You |
38 | +should be logged into Launchpad as the lp2kanban user when doing this, unless |
39 | +for testing. |
40 | + |
41 | +Next you need to add cron entries to run the bugs2kanban.py script. You will |
42 | +need to run it twice, one for creating new bugs (create.ini) and one for |
43 | +syncing the card information with bugs (sync.ini): |
44 | + |
45 | + bin/py src/lp2kanban/bugs2kanban.py -c create.ini |
46 | + bin/py src/lp2kanban/bugs2kanban.py -c sync.ini |
47 | + |
48 | +The former is quite quick and can be run every minute. The latter is slower |
49 | +and shouldn't be run more often than every 5 minutes. |
50 | |
51 | === modified file 'bootstrap.py' |
52 | --- bootstrap.py 2013-03-18 15:45:56 +0000 |
53 | +++ bootstrap.py 2015-12-16 04:22:55 +0000 |
54 | @@ -18,7 +18,11 @@ |
55 | use the -c option to specify an alternate configuration file. |
56 | """ |
57 | |
58 | -import os, shutil, sys, tempfile |
59 | +import os |
60 | +import shutil |
61 | +import sys |
62 | +import tempfile |
63 | + |
64 | from optparse import OptionParser |
65 | |
66 | tmpeggs = tempfile.mkdtemp() |
67 | @@ -31,8 +35,8 @@ |
68 | Simply run this script in a directory containing a buildout.cfg, using the |
69 | Python that you want bin/buildout to use. |
70 | |
71 | -Note that by using --setup-source and --download-base to point to |
72 | -local resources, you can keep this script from going over the network. |
73 | +Note that by using --find-links to point to local resources, you can keep |
74 | +this script from going over the network. |
75 | ''' |
76 | |
77 | parser = OptionParser(usage=usage) |
78 | @@ -48,48 +52,63 @@ |
79 | "bootstrap and buildout will get the newest releases " |
80 | "even if they are alphas or betas.")) |
81 | parser.add_option("-c", "--config-file", |
82 | - help=("Specify the path to the buildout configuration " |
83 | - "file to be used.")) |
84 | + help=("Specify the path to the buildout configuration " |
85 | + "file to be used.")) |
86 | parser.add_option("-f", "--find-links", |
87 | - help=("Specify a URL to search for buildout releases")) |
88 | + help=("Specify a URL to search for buildout releases")) |
89 | +parser.add_option("--allow-site-packages", |
90 | + action="store_true", default=False, |
91 | + help=("Let bootstrap.py use existing site packages")) |
92 | +parser.add_option("--setuptools-version", |
93 | + help="use a specific setuptools version") |
94 | |
95 | |
96 | options, args = parser.parse_args() |
97 | |
98 | ###################################################################### |
99 | -# load/install distribute |
100 | +# load/install setuptools |
101 | |
102 | -to_reload = False |
103 | try: |
104 | - import pkg_resources, setuptools |
105 | - if not hasattr(pkg_resources, '_distribute'): |
106 | - to_reload = True |
107 | - raise ImportError |
108 | + if options.allow_site_packages: |
109 | + import setuptools |
110 | + import pkg_resources |
111 | + from urllib.request import urlopen |
112 | except ImportError: |
113 | - ez = {} |
114 | - |
115 | - try: |
116 | - from urllib.request import urlopen |
117 | - except ImportError: |
118 | - from urllib2 import urlopen |
119 | - |
120 | - exec(urlopen('http://python-distribute.org/distribute_setup.py').read(), ez) |
121 | - setup_args = dict(to_dir=tmpeggs, download_delay=0, no_fake=True) |
122 | - ez['use_setuptools'](**setup_args) |
123 | - |
124 | - if to_reload: |
125 | - reload(pkg_resources) |
126 | - import pkg_resources |
127 | - # This does not (always?) update the default working set. We will |
128 | - # do it. |
129 | - for path in sys.path: |
130 | - if path not in pkg_resources.working_set.entries: |
131 | - pkg_resources.working_set.add_entry(path) |
132 | + from urllib2 import urlopen |
133 | + |
134 | +ez = {} |
135 | +exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez) |
136 | + |
137 | +if not options.allow_site_packages: |
138 | + # ez_setup imports site, which adds site packages |
139 | + # this will remove them from the path to ensure that incompatible versions |
140 | + # of setuptools are not in the path |
141 | + import site |
142 | + # inside a virtualenv, there is no 'getsitepackages'. |
143 | + # We can't remove these reliably |
144 | + if hasattr(site, 'getsitepackages'): |
145 | + for sitepackage_path in site.getsitepackages(): |
146 | + sys.path[:] = [x for x in sys.path if sitepackage_path not in x] |
147 | + |
148 | +setup_args = dict(to_dir=tmpeggs, download_delay=0) |
149 | + |
150 | +if options.setuptools_version is not None: |
151 | + setup_args['version'] = options.setuptools_version |
152 | + |
153 | +ez['use_setuptools'](**setup_args) |
154 | +import setuptools |
155 | +import pkg_resources |
156 | + |
157 | +# This does not (always?) update the default working set. We will |
158 | +# do it. |
159 | +for path in sys.path: |
160 | + if path not in pkg_resources.working_set.entries: |
161 | + pkg_resources.working_set.add_entry(path) |
162 | |
163 | ###################################################################### |
164 | # Install buildout |
165 | |
166 | -ws = pkg_resources.working_set |
167 | +ws = pkg_resources.working_set |
168 | |
169 | cmd = [sys.executable, '-c', |
170 | 'from setuptools.command.easy_install import main; main()', |
171 | @@ -104,8 +123,8 @@ |
172 | if find_links: |
173 | cmd.extend(['-f', find_links]) |
174 | |
175 | -distribute_path = ws.find( |
176 | - pkg_resources.Requirement.parse('distribute')).location |
177 | +setuptools_path = ws.find( |
178 | + pkg_resources.Requirement.parse('setuptools')).location |
179 | |
180 | requirement = 'zc.buildout' |
181 | version = options.version |
182 | @@ -113,13 +132,19 @@ |
183 | # Figure out the most recent final version of zc.buildout. |
184 | import setuptools.package_index |
185 | _final_parts = '*final-', '*final' |
186 | + |
187 | def _final_version(parsed_version): |
188 | - for part in parsed_version: |
189 | - if (part[:1] == '*') and (part not in _final_parts): |
190 | - return False |
191 | - return True |
192 | + try: |
193 | + return not parsed_version.is_prerelease |
194 | + except AttributeError: |
195 | + # Older setuptools |
196 | + for part in parsed_version: |
197 | + if (part[:1] == '*') and (part not in _final_parts): |
198 | + return False |
199 | + return True |
200 | + |
201 | index = setuptools.package_index.PackageIndex( |
202 | - search_path=[distribute_path]) |
203 | + search_path=[setuptools_path]) |
204 | if find_links: |
205 | index.add_find_links((find_links,)) |
206 | req = pkg_resources.Requirement.parse(requirement) |
207 | @@ -142,10 +167,9 @@ |
208 | cmd.append(requirement) |
209 | |
210 | import subprocess |
211 | -if subprocess.call(cmd, env=dict(os.environ, PYTHONPATH=distribute_path)) != 0: |
212 | +if subprocess.call(cmd, env=dict(os.environ, PYTHONPATH=setuptools_path)) != 0: |
213 | raise Exception( |
214 | - "Failed to execute command:\n%s", |
215 | - repr(cmd)[1:-1]) |
216 | + "Failed to execute command:\n%s" % repr(cmd)[1:-1]) |
217 | |
218 | ###################################################################### |
219 | # Import and run buildout |
220 | |
221 | === modified file 'buildout.cfg' |
222 | --- buildout.cfg 2013-06-19 19:44:20 +0000 |
223 | +++ buildout.cfg 2015-12-16 04:22:55 +0000 |
224 | @@ -11,6 +11,7 @@ |
225 | exec-sitecustomize = true |
226 | develop = . |
227 | eggs-directory = eggs |
228 | +offline = false |
229 | |
230 | [test] |
231 | recipe = zc.recipe.testrunner |
232 | @@ -38,13 +39,12 @@ |
233 | lazr.restfulclient = 0.11.2 |
234 | lazr.uri = 1.0.2 |
235 | oauth = 1.0.1 |
236 | -setuptools = 0.6c11 |
237 | simplejson = 2.1.6 |
238 | wadllib = 1.2.0 |
239 | wsgi-intercept = 0.5.0 |
240 | z3c.recipe.scripts = 1.0.1 |
241 | z3c.recipe.tag = 0.4.0 |
242 | -zc.buildout = 1.5.2 |
243 | +zc.buildout = 1.7.1 |
244 | zc.recipe.egg = 1.3.2 |
245 | zc.recipe.testrunner = 1.4.0 |
246 | zope.exceptions = 3.6.1 |
247 | |
248 | === added directory 'configs' |
249 | === added file 'configs/groups.ini' |
250 | --- configs/groups.ini 1970-01-01 00:00:00 +0000 |
251 | +++ configs/groups.ini 2015-12-16 04:22:55 +0000 |
252 | @@ -0,0 +1,16 @@ |
253 | +# The group name, which will replace ${group} in lane configs. |
254 | +# The Launchpad account names of the group members. |
255 | +[Alpha] |
256 | +members = benji,simpoir |
257 | +[Beta] |
258 | +members = fcorrea,chad.smith |
259 | +[Gamma] |
260 | +members = adam-collard,bogdana |
261 | +[Delta] |
262 | +members = tealeg,danilo |
263 | +[Epsilon] |
264 | +members = free.ekanayaka,ack |
265 | +[Zeta] |
266 | +members = bjornt,tribaal |
267 | +[QA] |
268 | +members = ahasenack |
269 | |
270 | === added file 'configs/sync.ini' |
271 | --- configs/sync.ini 1970-01-01 00:00:00 +0000 |
272 | +++ configs/sync.ini 2015-12-16 04:22:55 +0000 |
273 | @@ -0,0 +1,106 @@ |
274 | +[GLOBAL] |
275 | +# This is the account part of https://<account>.leankitkanban.com/ URL. |
276 | +account = canonical |
277 | +# LeankitKanban user name and password. |
278 | +user = landscape-crew@lists.canonical.com |
279 | +password = fShv0K4h@LV:!{ |
280 | + |
281 | +# Defaults are common values overridable in each of the boards definition. |
282 | +[DEFAULT] |
283 | +lp_server = production |
284 | +launchpadlib_dir = .launchpadlib.create |
285 | + |
286 | +# Should all cards with external IDs be synced or not. Set to 'on' if |
287 | +# you want them synced automatically (and use "(no-sync)" as the marker |
288 | +# for cards you don't want synced). |
289 | +autosync = on |
290 | + |
291 | +# Should the cards be moved? If not specified it is off. |
292 | +move_cards = on |
293 | + |
294 | +# Should the cards be synced? If not specified it is off. |
295 | +sync_cards = on |
296 | + |
297 | +# Bug tag for which new cards should be created. If a bug in Launchpad |
298 | +# has this tag, a new card will be created for it in the first kanban lane. |
299 | +# If this config option isn't set, no cards will be created. |
300 | +# After the card has been created, the tag will be removed from the bug, so a |
301 | +# credentials_file has to be specified as well, to authenticate against LP. |
302 | +bug_to_card_tag = kanban |
303 | + |
304 | +# Which card type should cards created from bugs have. If not set, the default |
305 | +# card type is used. |
306 | +bug_to_card_type = Bug |
307 | + |
308 | +bug_to_card_lane = Backlog |
309 | + |
310 | +# By default, omnidirectional card moves are off, which means that cards will |
311 | +# be moved only to the lane that is considered to be next of the card's |
312 | +# current lane. I.e. a card will move only within a "swim lane". If |
313 | +# omnidirectional card moves are turned on, a card can move to any lane |
314 | +# on the board. |
315 | +omnidirectional_card_moves = on |
316 | + |
317 | +# Lanes are assigned a "role" by looking for sublanes matching |
318 | +# values given here. "Coding" will match all lanes which have |
319 | +# a title of "Coding", and "QA::Ready" will match all lanes |
320 | +# titled "Ready" that have a parent lane with title "QA". |
321 | +# |
322 | +# Note that bugs2cards.py analyzes the lane structure and will flow |
323 | +# cards only within the same horizontal "lane" (iow, if you have |
324 | +# parallel 'Feature' and 'Bugs' development lanes, cards will flow |
325 | +# only from Coding -> Review inside them, and then will move to the |
326 | +# next lane horizontally (eg. Landing lane). |
327 | +#coding_lanes = |
328 | +#review_lanes = |
329 | +#landing_lanes = |
330 | +#qa_lanes = |
331 | +#deploy_lanes = |
332 | +#downtime_deploy_lanes = |
333 | +#done_lanes = |
334 | + |
335 | + |
336 | +# If you track multiple groups of people on one board, you can specify a |
337 | +# groups config that will map a card to a group according to its |
338 | +#assignee. You can then make the lane configs to include the group |
339 | +# name, by using ${group} in the lane name. For example: |
340 | +# |
341 | +# coding_lanes = ${group}::Coding |
342 | +# |
343 | +groups_config_file = ./configs/groups.ini |
344 | + |
345 | +credentials_file = ./credentials |
346 | + |
347 | + |
348 | +#[Landscape Cisco] |
349 | +# Keep [Landscape Cisco] board before [Landscape] so it gets the "falkor" cards |
350 | +# (we can only use external ID for one card across all of Leankit). |
351 | +#projects = falkor |
352 | + |
353 | +#bug_to_card_lane = Backlog |
354 | + |
355 | +#coding_lanes = Doing |
356 | +#review_lanes = Doing |
357 | +#landing_lanes = Done |
358 | +#deploy_lanes = Done |
359 | +#done_lanes = Done |
360 | + |
361 | +#no_move_lanes = |
362 | +#no_sync_lanes = |
363 | + |
364 | + |
365 | +[Landscape 2016] |
366 | +project_group = landscape-project |
367 | + |
368 | +# TODO: This should put charm bugs in ::Charm and server/client bugs |
369 | +# in ::Landscape, for now stick everything in ::Landscape |
370 | +bug_to_card_lane = Backlog::Engineering |
371 | + |
372 | +coding_lanes = Development::${group}::Doing |
373 | +review_lanes = Development::${group}::Review |
374 | +landing_lanes = Development::${group}::Landed |
375 | +deploy_lanes = QA::Landed |
376 | +done_lanes = Done |
377 | + |
378 | +no_move_lanes = Done,QA::Landed,QA::In progress |
379 | +no_sync_lanes = Backlog::Engineering,Done,QA::Landed,QA::In progress,Archive,Ready for QA,Release Ready,Staging,Production,LDS |
380 | |
381 | === modified file 'create_creds.py' |
382 | --- create_creds.py 2012-12-10 15:21:44 +0000 |
383 | +++ create_creds.py 2015-12-16 04:22:55 +0000 |
384 | @@ -7,7 +7,7 @@ |
385 | |
386 | web_root="https://launchpad.net/" |
387 | |
388 | -creds = Credentials("tarmac") |
389 | +creds = Credentials("lp2kanban-landscape") |
390 | url = creds.get_request_token(web_root=web_root) |
391 | |
392 | subprocess.call(['xdg-open', url]) |
393 | |
394 | === added file 'jenkins.sh' |
395 | --- jenkins.sh 1970-01-01 00:00:00 +0000 |
396 | +++ jenkins.sh 2015-12-16 04:22:55 +0000 |
397 | @@ -0,0 +1,32 @@ |
398 | +#!/bin/bash -xe |
399 | + |
400 | +ini="$1.ini" |
401 | +date "+START: %c" |
402 | + |
403 | +#export https_proxy=http://squid.external:3128 |
404 | +#export http_proxy=http://squid.external:3128 |
405 | + |
406 | +make |
407 | + |
408 | +if [ ! -e credentials ]; then |
409 | + if [ -e ~/credentials ]; then |
410 | + cp ~/credentials credentials |
411 | + else |
412 | + echo "ERROR: try make credentials first!" |
413 | + exit 1 |
414 | + fi |
415 | +fi |
416 | + |
417 | +# default encoding of stdout/stderr when not attached to a terminal is set to |
418 | +# 'None'. 'PYTHONIOENCODING' overrides, could also be overridden in the |
419 | +# python program by modifying 'sys.stdout.encoding' directly. This feels |
420 | +# cleaner. Note setting LANG (which the terminal respects) wont fix this, |
421 | +# since there is no terminal attached. |
422 | +# -------- |
423 | +# See: |
424 | +# - https://wiki.python.org/moin/PrintFails |
425 | +# - https://stackoverflow.com/questions/1473577 |
426 | +# - https://stackoverflow.com/questions/492483 |
427 | +# - https://stackoverflow.com/questions/9932406 |
428 | +PYTHONIOENCODING="utf-8" bin/py src/lp2kanban/bugs2cards.py -c configs/${1}.ini |
429 | +date "+END: %c" |
430 | |
431 | === added directory 'logs' |
432 | === added file 'run.sh' |
433 | --- run.sh 1970-01-01 00:00:00 +0000 |
434 | +++ run.sh 2015-12-16 04:22:55 +0000 |
435 | @@ -0,0 +1,21 @@ |
436 | +#!/bin/bash |
437 | + |
438 | +LOGFILE="logs/${1}-`date '+%Y-%m-%d'`.log" |
439 | +LOCKFILE="$HOME/lp2kanban/run-$1.lock" |
440 | +exit_status=0 |
441 | +export https_proxy=http://squid.external:3128 |
442 | + |
443 | +cd $HOME/lp2kanban |
444 | +echo -n "START " >> $LOGFILE |
445 | +date "+%c" >> $LOGFILE |
446 | +lockfile -10 -r 4 $LOCKFILE |
447 | +if [ "$?" -eq 0 ]; then |
448 | + bin/py src/lp2kanban/bugs2cards.py -c configs/${1}.ini 2>&1 >> $LOGFILE |
449 | + exit_status=$? |
450 | + rm -f $LOCKFILE |
451 | +else |
452 | + echo "Failed to grab lockfile $LOCKFILE" >> $LOGFILE |
453 | + exit_status=1 |
454 | +fi |
455 | +echo -n "END " >> $LOGFILE; date "+%c" >> $LOGFILE |
456 | +exit $? |
457 | |
458 | === modified file 'src/lp2kanban/bugs2cards.py' |
459 | --- src/lp2kanban/bugs2cards.py 2013-06-19 19:44:20 +0000 |
460 | +++ src/lp2kanban/bugs2cards.py 2015-12-16 04:22:55 +0000 |
461 | @@ -1,4 +1,4 @@ |
462 | -# Copyright 2011 Canonical Ltd |
463 | +# Copyright 2015 Canonical Ltd |
464 | # |
465 | from ConfigParser import ConfigParser |
466 | from argparse import ArgumentParser |
467 | @@ -9,6 +9,7 @@ |
468 | update_blueprints_from_work_items, |
469 | ) |
470 | from lp2kanban.kanban import ( |
471 | + BRANCH_REGEX, |
472 | LeankitKanban, |
473 | Record, |
474 | ) |
475 | @@ -117,6 +118,9 @@ |
476 | for user in board.users: |
477 | lp_user = lp.people.getByEmail(email=user) |
478 | if lp_user is None: |
479 | + print ( |
480 | + "WARNING: There is no matching Launchpad user for kanban" |
481 | + " user %s." % (user,)) |
482 | continue |
483 | self.lp_to_kanban[lp_user.name] = board.users[user] |
484 | self.kanban_to_lp[board.users[user]] = lp_user |
485 | @@ -161,8 +165,10 @@ |
486 | |
487 | def _get_mp_info(): |
488 | mps = [] |
489 | - for bug_branch in branches: |
490 | - for mp in bug_branch.branch.landing_targets: |
491 | + for branch in branches: |
492 | + if hasattr(branch, 'branch') : |
493 | + branch = branch.branch |
494 | + for mp in branch.landing_targets: |
495 | mp_info = Record(rank=None, status=None, mp=None) |
496 | status = mp.queue_status |
497 | mp_info.rank = ORDERED_STATUSES.get(status, 1) |
498 | @@ -222,8 +228,10 @@ |
499 | if conf.get('autosync', None) == 'on': |
500 | has_id = (card.external_card_id is not None and |
501 | card.external_card_id.strip() != '') |
502 | - return (has_id and NO_SYNC_MARKER not in card.title and |
503 | - NO_SYNC_MARKER not in card.description) |
504 | + has_branch = BRANCH_REGEX.match(card.external_system_url) |
505 | + if NO_SYNC_MARKER in card.title or NO_SYNC_MARKER in card.description: |
506 | + return False |
507 | + return (has_id or has_branch) |
508 | else: |
509 | return (card.title.startswith(TITLE_MARKER) or |
510 | DESCRIPTION_MARKER in card.description) |
511 | @@ -296,15 +304,21 @@ |
512 | """Return the status of the card as one of CardStatus values.""" |
513 | # status of None means card needs not to be moved. |
514 | status = None |
515 | - if bug_status in IN_PROGRESS_BUG_STATUSES: |
516 | - status = CardStatus.CODING |
517 | + branch_card_status = None |
518 | + if branch_info: |
519 | if branch_info.status == 'In Progress': |
520 | - status = CardStatus.CODING |
521 | + branch_card_status = CardStatus.CODING |
522 | elif branch_info.status == 'In Review': |
523 | - status = CardStatus.REVIEW |
524 | + branch_card_status = CardStatus.REVIEW |
525 | elif branch_info.status == 'Approved': |
526 | - status = CardStatus.LANDING |
527 | + branch_card_status = CardStatus.LANDING |
528 | elif branch_info.status == 'Merged': |
529 | + branch_card_status = CardStatus.LANDING |
530 | + if bug_status is None: |
531 | + return branch_card_status |
532 | + if bug_status in IN_PROGRESS_BUG_STATUSES: |
533 | + status = CardStatus.CODING |
534 | + if branch_info.status == 'Merged': |
535 | if bug_status == 'Fix Committed': |
536 | if 'qa-needstesting' in bug_tags: |
537 | status = CardStatus.QA |
538 | @@ -315,6 +329,8 @@ |
539 | status = CardStatus.DEPLOY |
540 | else: |
541 | status = CardStatus.LANDING |
542 | + elif branch_card_status: |
543 | + return branch_card_status |
544 | elif bug_status in DONE_BUG_STATUSES: |
545 | status = CardStatus.DONE |
546 | return status |
547 | @@ -454,6 +470,37 @@ |
548 | board, project, new_bug_tag, bconf.get("bug_to_card_type"), |
549 | bconf.get("bug_to_card_lane")) |
550 | print " Syncing cards:" |
551 | + # Process linked branch cards |
552 | + for card in board.getCardsWithExternalLinks(only_branches=True): |
553 | + if card.external_card_id: |
554 | + # Cards with external_id are processed by getCardsWithExternalIds |
555 | + continue |
556 | + branch_name = card.external_system_url.replace( |
557 | + "https://code.launchpad.net/", "") |
558 | + branch = lp.branches.getByUniqueName(unique_name=branch_name) |
559 | + if not branch: |
560 | + print "Invalid branch url ({}) for card '{}'".format( |
561 | + card.external_system_url, card.title) |
562 | + continue |
563 | + if should_sync_card(card, bconf): |
564 | + branch_info = get_branch_info([branch]) |
565 | + owner_name = BRANCH_REGEX.match(card.external_system_url).group(1) |
566 | + card_status = get_card_status(None, '', branch_info) |
567 | + if branch_info.status and should_move_card(card, bconf): |
568 | + assignee = Record(name=owner_name) |
569 | + try: |
570 | + move_card(card, card_status, bconf, [assignee], lp_users) |
571 | + except IOError as e: |
572 | + print " * %s (in %s)" % (card.title, card.lane.path) |
573 | + print " >>> Error moving card:" |
574 | + print " ", e |
575 | + continue |
576 | + kanban_user = lp_users.lp_to_kanban.get(assignee.name, None) |
577 | + if kanban_user: |
578 | + card.assigned_user_id = kanban_user.id |
579 | + card.kanban_user = kanban_user |
580 | + print " * %s (in %s)" % (card.title, card.lane.path) |
581 | + |
582 | for card in board.getCardsWithExternalIds(): |
583 | if should_sync_card(card, bconf): |
584 | try: |
585 | |
586 | === modified file 'src/lp2kanban/kanban.py' |
587 | --- src/lp2kanban/kanban.py 2013-05-24 14:48:07 +0000 |
588 | +++ src/lp2kanban/kanban.py 2015-12-16 04:22:55 +0000 |
589 | @@ -11,6 +11,8 @@ |
590 | |
591 | |
592 | ANNOTATION_REGEX = re.compile('^\s*{.*}\s*$', re.MULTILINE|re.DOTALL) |
593 | +BRANCH_REGEX = re.compile( |
594 | + '^https://code.launchpad.net/~([.\w]*)/([-\w]*)/([-\w]*)$') |
595 | |
596 | |
597 | class Record(dict): |
598 | @@ -55,8 +57,7 @@ |
599 | |
600 | class LeankitConnector(object): |
601 | def __init__(self, account, username=None, password=None, throttle=1): |
602 | - host = 'https://' + account + '.leankitkanban.com' |
603 | - self.base_api_url = host + '/Kanban/Api' |
604 | + self.base_api_url = 'https://' + account + '.leankit.com' |
605 | self.http = self._configure_auth(username, password) |
606 | self.last_request_time = time.time() - throttle |
607 | self.throttle = throttle |
608 | @@ -97,6 +98,10 @@ |
609 | config={}, |
610 | return_response=False) |
611 | sent = request.send() |
612 | + except AttributeError as e: |
613 | + # Weirdly, httplib2 has a habit of throwing an AttributeError |
614 | + # when it can't connect, so we handle that nicely. |
615 | + raise IOError("Unable to connect to LeanKitKanban server.") |
616 | except Exception as e: |
617 | raise IOError("Unable to make HTTP request: %s" % e.message) |
618 | |
619 | @@ -193,14 +198,17 @@ |
620 | optional_attributes = [ |
621 | 'ExternalCardID', 'AssignedUserId', 'Size', 'IsBlocked', |
622 | 'BlockReason', 'ExternalSystemName', 'ExternalSystemUrl', |
623 | - 'ClassOfServiceId', 'DueDate', |
624 | + 'ClassOfServiceId', 'DueDate', 'CurrentTaskBoardId' |
625 | ] |
626 | |
627 | def __init__(self, card_dict, lane): |
628 | super(LeankitCard, self).__init__(card_dict) |
629 | |
630 | self.lane = lane |
631 | - self.tags_list = set([tag.strip() for tag in self.tags.split(',')]) |
632 | + if not self.tags: |
633 | + self.tags_list = [] |
634 | + else: |
635 | + self.tags_list = set([tag.strip() for tag in self.tags.split(',')]) |
636 | if '' in self.tags_list: |
637 | self.tags_list.remove('') |
638 | self.type = lane.board.cardtypes[self.type_id] |
639 | @@ -214,7 +222,7 @@ |
640 | tag = tag.strip() |
641 | if tag not in self.tags_list or self.tags.startswith(','): |
642 | if tag != '': |
643 | - self.tags_list.add(tag) |
644 | + self.tags_list.append(tag) |
645 | self.tags = ', '.join(self.tags_list) |
646 | |
647 | def save(self): |
648 | @@ -241,10 +249,12 @@ |
649 | del data['Id'] |
650 | del data['LaneId'] |
651 | position = len(self.lane.cards) |
652 | - url_parts = ['/Board', str(self.lane.board.id), 'AddCard', |
653 | - 'Lane', str(self.lane.id), 'Position', str(position)] |
654 | + url_parts = ['/Kanban/Api/Board', str(self.lane.board.id), |
655 | + 'AddCard', 'Lane', str(self.lane.id), |
656 | + 'Position', str(position)] |
657 | else: |
658 | - url_parts = ['/Board', str(self.lane.board.id), 'UpdateCard'] |
659 | + url_parts = ['/Kanban/Api/Board', str(self.lane.board.id), |
660 | + 'UpdateCard'] |
661 | |
662 | url = '/'.join(url_parts) |
663 | |
664 | @@ -267,9 +277,23 @@ |
665 | else: |
666 | return None |
667 | |
668 | + def moveToTaskBoard(self, parent_board, target_card): |
669 | + url = '/Api/Card/MoveCardToTaskboard' |
670 | + result = self.lane.board.connector.post(url, data={ |
671 | + 'boardId': parent_board.id, |
672 | + 'destCardId': target_card.id, |
673 | + 'srcCardId': [self.id]}) |
674 | + if result.ReplyCode in LeankitResponseCodes.SUCCESS_CODES: |
675 | + return result.ReplyData[0] |
676 | + else: |
677 | + raise Exception( |
678 | + "Moving card %s (%s) to %s failed. " % ( |
679 | + self.title, self.id, self.lane.path) + |
680 | + "Error %s: %s" % (result.ReplyCode, result.ReplyText)) |
681 | + |
682 | def _moveCard(self): |
683 | target_pos = len(self.lane.cards) |
684 | - url = '/Board/%d/MoveCard/%d/Lane/%d/Position/%d' % ( |
685 | + url = '/Kanban/Api/Board/%d/MoveCard/%d/Lane/%d/Position/%d' % ( |
686 | self.lane.board.id, self.id, self.lane.id, target_pos) |
687 | result = self.lane.board.connector.post(url, data=None) |
688 | if result.ReplyCode in LeankitResponseCodes.SUCCESS_CODES: |
689 | @@ -323,6 +347,8 @@ |
690 | text_after_json), where json_annotations contains the |
691 | JSON loaded with json.loads(). |
692 | """ |
693 | + if not self.description: |
694 | + self.description = "" |
695 | match = ANNOTATION_REGEX.search(self.description) |
696 | if match: |
697 | start = match.start() |
698 | @@ -330,7 +356,7 @@ |
699 | try: |
700 | annotations = Record(json.loads(self.description[start:end])) |
701 | except ValueError, ex: |
702 | - print "Unable to parse card %i: %s" % (self.id, ex.message) |
703 | + print "Unable to parse card %s: %s" % (self.id, ex.message) |
704 | annotations = Record() |
705 | return ( |
706 | annotations, |
707 | @@ -450,7 +476,7 @@ |
708 | |
709 | attributes = ['Id', 'Title', 'CreationDate', 'IsArchived'] |
710 | |
711 | - base_uri = '/Boards/' |
712 | + base_uri = '/Kanban/Api/Boards/' |
713 | |
714 | def __init__(self, board_dict, connector): |
715 | super(LeankitBoard, self).__init__(board_dict) |
716 | @@ -469,12 +495,20 @@ |
717 | self._cards_with_external_ids = set() |
718 | self._cards_with_description_annotations = set() |
719 | self._cards_with_external_links = set() |
720 | + self._cards_with_branches = set() |
721 | self.default_cardtype = None |
722 | |
723 | def getCardsWithExternalIds(self): |
724 | return self._cards_with_external_ids |
725 | |
726 | - def getCardsWithExternalLinks(self): |
727 | + def getCardsWithExternalLinks(self, only_branches=False): |
728 | + """Return cards with external links |
729 | + |
730 | + @param only_merge_proposals: Only return cards that have merge |
731 | + proposals specified as the card's external link |
732 | + """ |
733 | + if only_branches: |
734 | + return self._cards_with_branches |
735 | return self._cards_with_external_links |
736 | |
737 | def getCardsWithDescriptionAnnotations(self): |
738 | @@ -487,15 +521,23 @@ |
739 | self._populateUsers(self.details['BoardUsers']) |
740 | self._populateCardTypes(self.details['CardTypes']) |
741 | self._archive = self.connector.get( |
742 | - "/Board/" + str(self.id) + "/Archive").ReplyData[0] |
743 | + "/Kanban/Api/Board/" + str(self.id) + "/Archive").ReplyData[0] |
744 | archive_lanes = [lane_dict['Lane'] for lane_dict in self._archive] |
745 | archive_lanes.extend( |
746 | [lane_dict['Lane'] for |
747 | lane_dict in self._archive[0]['ChildLanes']]) |
748 | self._backlog = self.connector.get( |
749 | - "/Board/" + str(self.id) + "/Backlog").ReplyData[0] |
750 | + "/Kanban/Api/Board/" + str(self.id) + "/Backlog").ReplyData[0] |
751 | self._populateLanes( |
752 | self.details['Lanes'] + archive_lanes + self._backlog) |
753 | + for card in self.cards: |
754 | + if card.current_task_board_id: # We are a task board |
755 | + taskboard_data = self.connector.get( |
756 | + "/Kanban/Api/v1/board/%s/card/%s/taskboard" % (self.id, card.id)) |
757 | + card.taskboard = LeankitTaskBoard( |
758 | + taskboard_data["ReplyData"][0], self) |
759 | + card.taskboard.fetchDetails() |
760 | + self.cards.extend(card.taskboard.cards) |
761 | self._classifyCards() |
762 | |
763 | def _classifyCards(self): |
764 | @@ -508,12 +550,16 @@ |
765 | self._cards_with_description_annotations.add(card) |
766 | if card.external_system_url: |
767 | self._cards_with_external_links.add(card) |
768 | + if BRANCH_REGEX.match(card.external_system_url): |
769 | + self._cards_with_branches.add(card) |
770 | print " - %s cards with external ids" % len( |
771 | self._cards_with_external_ids) |
772 | print " - %s cards with external links" % len( |
773 | self._cards_with_external_links) |
774 | print " - %s cards with description annotations" % len( |
775 | self._cards_with_description_annotations) |
776 | + print " - %s cards with branches" % len( |
777 | + self._cards_with_branches) |
778 | |
779 | def _populateUsers(self, user_data): |
780 | self.users = {} |
781 | @@ -625,6 +671,21 @@ |
782 | self._printLanes(lane, indent, include_cards) |
783 | |
784 | |
785 | +class LeankitTaskBoard(LeankitBoard): |
786 | + |
787 | + attributes = ['Id', 'Title'] |
788 | + |
789 | + def __init__(self, taskboard_dict, board): |
790 | + super(LeankitTaskBoard, self).__init__(taskboard_dict, board.connector) |
791 | + self.base_uri = '/Api/Board/%d/TaskBoard/%s/Get' % (board.id, self.id) |
792 | + self.cardtypes = board.cardtypes |
793 | + self.parent_board = board |
794 | + |
795 | + def fetchDetails(self): |
796 | + self.details = self.connector.get(self.base_uri).ReplyData[0] |
797 | + self._populateLanes(self.details['Lanes']) |
798 | + |
799 | + |
800 | class LeankitKanban(object): |
801 | |
802 | def __init__(self, account, username=None, password=None): |
803 | @@ -638,7 +699,7 @@ |
804 | |
805 | :param include_archived: if True, include archived boards as well. |
806 | """ |
807 | - boards_data = self.connector.get('/Boards').ReplyData |
808 | + boards_data = self.connector.get('/Kanban/Api/Boards').ReplyData |
809 | boards = [] |
810 | for board_dict in boards_data[0]: |
811 | board = LeankitBoard(board_dict, self.connector) |
812 | @@ -683,13 +744,14 @@ |
813 | if __name__ == '__main__': |
814 | kanban = LeankitKanban('launchpad.leankitkanban.com', |
815 | 'user@email', 'password') |
816 | + |
817 | print "Active boards:" |
818 | boards = kanban.getBoards() |
819 | for board in boards: |
820 | print " * %s (%d)" % (board.title, board.id) |
821 | |
822 | # Get a board by the title. |
823 | - board_name = 'lp2kanban test' |
824 | + board_name = 'Landscape 2016' |
825 | print "Getting board '%s'..." % board_name |
826 | board = kanban.getBoard(title=board_name) |
827 | board.printLanes() |
828 | |
829 | === modified file 'src/lp2kanban/tests/common.py' |
830 | --- src/lp2kanban/tests/common.py 2013-04-08 12:46:05 +0000 |
831 | +++ src/lp2kanban/tests/common.py 2015-12-16 04:22:55 +0000 |
832 | @@ -33,13 +33,16 @@ |
833 | self.is_archived = is_archived |
834 | self._cards_with_description_annotations = set() |
835 | self._cards_with_external_links = set() |
836 | + self._cards_with_branches = set() |
837 | self.lanes = {} |
838 | self.root_lane = self.addLane('ROOT LANE') |
839 | |
840 | def getCardsWithDescriptionAnnotations(self): |
841 | return self._cards_with_description_annotations |
842 | |
843 | - def getCardsWithExternalLinks(self): |
844 | + def getCardsWithExternalLinks(self, only_branches=False): |
845 | + if only_branches: |
846 | + return self._cards_with_branches |
847 | return self._cards_with_external_links |
848 | |
849 | def getLaneByPath(self, path): |
850 | @@ -93,7 +96,7 @@ |
851 | def __init__(self, external_card_id=None, title=u"", description=u"", |
852 | description_annotations=None, lane=None, |
853 | assigned_user_id=None, external_system_name=None, |
854 | - external_system_url=None): |
855 | + external_system_url=u""): |
856 | self.external_card_id = external_card_id |
857 | self.title = title |
858 | self.description = description |
859 | |
860 | === modified file 'src/lp2kanban/tests/test_bugs2cards.py' |
861 | --- src/lp2kanban/tests/test_bugs2cards.py 2013-04-08 12:50:46 +0000 |
862 | +++ src/lp2kanban/tests/test_bugs2cards.py 2015-12-16 04:22:55 +0000 |
863 | @@ -346,6 +346,7 @@ |
864 | # in CODING. |
865 | self.assertEqual(CardStatus.CODING, status) |
866 | |
867 | +BRANCH_URL = "https://code.launchpad.net/~me/project/branch-name" |
868 | |
869 | class CardStatusTest(unittest.TestCase): |
870 | |
871 | @@ -375,23 +376,43 @@ |
872 | |
873 | def test_should_sync_card_autosync_no(self): |
874 | # When autosync is 'on', cards with no external card IDs |
875 | - # are not synced. |
876 | + # and no branches are not synced. |
877 | card = Record(title=u'no sync', description=u'', |
878 | - external_card_id=None) |
879 | + external_card_id=None, external_system_url=None) |
880 | self.assertFalse(should_sync_card(card, {'sync_cards': 'on'})) |
881 | |
882 | - def test_should_sync_card_autosync_yes(self): |
883 | + def test_should_sync_card_autosync_no_sync_with_non_branch_urls(self): |
884 | + # When autosync is 'on', cards with external_system_urls which are not |
885 | + # valid launchpad branches are not synced. |
886 | + non_branch_urls = [ |
887 | + "http://www.google.com/", |
888 | + "https://bugs.launchpad.net/charms/+source/hacluster/+bug/1", |
889 | + "https://code.launchpad.net/~person/project/blah/+merge/111" |
890 | + ] |
891 | + for url in non_branch_urls: |
892 | + card = Record(title=u'no sync', description=u'', |
893 | + external_card_id=None, external_system_url=url) |
894 | + self.assertFalse(should_sync_card(card, {'sync_cards': 'on'})) |
895 | + |
896 | + def test_should_sync_card_autosync_synced_with_external_card_id(self): |
897 | # When autosync is 'on', cards with external card IDs are synced. |
898 | card = Record(title=u'no sync', description=u'', |
899 | - external_card_id='11') |
900 | + external_card_id=u'11', external_system_url=u'') |
901 | + self.assertTrue(should_sync_card(card, {'autosync': 'on', |
902 | + 'sync_cards': 'on'})) |
903 | + |
904 | + def test_should_sync_card_autosync_synced_with_branch(self): |
905 | + # When autosync is 'on', cards with a valid branch are synced. |
906 | + card = Record(title=u'no sync', description=u'', |
907 | + external_card_id=u'11', external_system_url=BRANCH_URL) |
908 | self.assertTrue(should_sync_card(card, {'autosync': 'on', |
909 | 'sync_cards': 'on'})) |
910 | |
911 | def test_should_sync_card_autosync_nosync(self): |
912 | - # When autosync is 'on', cards with external card IDs are not |
913 | - # synced when they contain the no-sync marker. |
914 | + # When autosync is 'on', cards with external card IDs or a valid |
915 | + # branch are not synced when they contain the no-sync marker. |
916 | card = Record(title=u'(no-sync)', description=u'(no-sync)', |
917 | - external_card_id='11') |
918 | + external_card_id=u'11', external_system_url=BRANCH_URL) |
919 | self.assertFalse(should_sync_card(card, {'autosync': 'on', |
920 | 'sync_cards': 'on'})) |
921 | |
922 | @@ -452,7 +473,7 @@ |
923 | self.assertFalse(should_sync_card(card1, conf)) |
924 | self.assertFalse(should_sync_card(card2, conf)) |
925 | |
926 | - def test_get_card_status_noop(self): |
927 | + def test_get_card_status_noop_no_branch(self): |
928 | # For a bug in 'New', 'Triaged', 'Confirmed' or 'Incomplete' statuses, |
929 | # the status is unknown. |
930 | self.assertEqual(None, get_card_status('New', [], None)) |
931 | @@ -460,6 +481,33 @@ |
932 | self.assertEqual(None, get_card_status('Incomplete', [], None)) |
933 | self.assertEqual(None, get_card_status('Triaged', [], None)) |
934 | |
935 | + def test_get_card_status_coding_no_bug(self): |
936 | + # For a card with no associated bug, an in progress branch status will |
937 | + # be in the coding state. |
938 | + branch_info = Record(status='In Progress', target=None) |
939 | + self.assertEqual( |
940 | + CardStatus.CODING, |
941 | + get_card_status(None, [], branch_info)) |
942 | + |
943 | + def test_get_card_status_review_no_bug(self): |
944 | + # For a card with no associated bug, an in review branch status will |
945 | + # be in the review state. |
946 | + branch_info = Record(status='In Review', target=None) |
947 | + self.assertEqual( |
948 | + CardStatus.REVIEW, |
949 | + get_card_status(None, [], branch_info)) |
950 | + |
951 | + def test_get_card_status_landing_no_bug(self): |
952 | + # For a card with no associated bug, an approved or merged branch |
953 | + # status will be in the landing state. |
954 | + branch_infos = [ |
955 | + Record(status='Approved', target=None), |
956 | + Record(status='Merged', target=None)] |
957 | + for branch_info in branch_infos: |
958 | + self.assertEqual( |
959 | + CardStatus.LANDING, |
960 | + get_card_status(None, [], branch_info)) |
961 | + |
962 | def test_get_card_status_coding_no_branch(self): |
963 | # For a bug in 'In Progress' or 'Fix Committed' status, it is |
964 | # considered to be in the coding state if there is no branch. |
Please re-submit this against lp:~landscape/lp2kanban/landscape-deploy.