Merge lp:~chad.smith/lp2kanban/branch-to-card into lp:~launchpad/lp2kanban/trunk

Proposed by Chad Smith on 2015-12-16
Status: Superseded
Proposed branch: lp:~chad.smith/lp2kanban/branch-to-card
Merge into: lp:~launchpad/lp2kanban/trunk
Diff against target: 964 lines (+460/-83)
14 files modified
.bzrignore (+1/-0)
Makefile (+2/-2)
README.landscape (+17/-0)
bootstrap.py (+66/-42)
buildout.cfg (+2/-2)
configs/groups.ini (+16/-0)
configs/sync.ini (+106/-0)
create_creds.py (+1/-1)
jenkins.sh (+32/-0)
run.sh (+21/-0)
src/lp2kanban/bugs2cards.py (+57/-10)
src/lp2kanban/kanban.py (+78/-16)
src/lp2kanban/tests/common.py (+5/-2)
src/lp2kanban/tests/test_bugs2cards.py (+56/-8)
To merge this branch: bzr merge lp:~chad.smith/lp2kanban/branch-to-card
Reviewer Review Type Date Requested Status
Данило Шеган (community) 2015-12-16 Resubmit on 2015-12-16
Landscape 2015-12-16 Pending
Review via email: mp+280669@code.launchpad.net

Description of the change

Add initial support for branch cards and taskboards. This branch only handles syncing and moving branch "MP/blue" cards on the Landscape 2016 board.

 Support was added for branch cards inside bugs2card because a lot of the logic is common and can hopefully be consolidated in a subsequent branch.

There will be a followup branch with added logic in bugs2cards to call moveToTaskBoard for feature cards that are manually dragged to the "$Group::Landed" lane.

src/lp2kanban/kanban.py:
- Define initial LeankitTaskBoard class
- Add optional CurrentTaskBoardId property to LeankitCard
- Add LeankitCard.moveToTaskBoard method to move a card as a subtask of another card
- Since Taskboard API calls live on a different route, rework API routes defined on the LeankitConnector class

src/lp2kanban/bugs2cards.py:
- _get_mp_info to handle non-bug branch records when parsing linked MPs
- should_sync_card to return true if external_system_url is a branch
- get_card_status returns proper status for cards that have external_system_urls linked but no bugs (external_system_id) linked
- add a loop to sync_board function which processes all "branch cards" through board.getCardsWithExternalLinks(only_branches=True)

config/sync.ini:
  - comment out [Landscape Cisco] board definition (as that should live in this board now

To test:
buildout
./bin/test
# to test against landscape 201
./create_creds.py (this will create your creds to modify the shared 2016 board)
Select "Change anything" button in your browser to allow updating cards

./bin/py src/lp2kanban/bugs2cards.py -c configs/sync.ini -b 'Landscape 2016'

# Tweak branches or drag cards at https://canonical.leankit.com/Boards/View/102392996#workflow-view that you want to see automatically moved back to the proper lanes.

What's lacking:
 unit tests to cover bugs2card sync_board (coming in a followup branch)

To post a comment you must log in.
Данило Шеган (danilo) wrote :

Please re-submit this against lp:~landscape/lp2kanban/landscape-deploy.

review: Resubmit
Данило Шеган (danilo) wrote :

Note that I've also split out configs now since it was too easy to leak credentials (a bzr push in the lp2kanban namespace creates a public branch). Please review my drop-configs branch.

The kanban-sync job has been updated to use the new credentials (I've updated the password as well).

135. By Chad Smith on 2015-12-16

simplify moveToTaskBoard card method

136. By Chad Smith on 2015-12-16

merge ~landscape/lp2kanban/landscape-deploy to drop configs

137. By Chad Smith on 2015-12-16

address review comments: approved branches that have not yet been merged will remain in the status LANDING. update copyright.

138. By Chad Smith on 2015-12-16

fix has_branch conditional to first check if card.external_system_url is not None before trying to match BRANCH_REGEX

139. By Chad Smith on 2015-12-16

card.save needs to be taskboard aware. When calling UpdateCard the parent_board.id needs to be specified, not the taskboard.id

140. By Chad Smith on 2015-12-16

add parent_card param to LeankitTaskBoard so that we have a refernce to the taskboard's card on the parent_board. Use the parent_card for card path printing

141. By Chad Smith on 2015-12-16

merge landscape-deploy to get latest changes to jenkins.sh

142. By Chad Smith on 2015-12-17

revert test makefile change

Unmerged revisions

142. By Chad Smith on 2015-12-17

revert test makefile change

141. By Chad Smith on 2015-12-16

merge landscape-deploy to get latest changes to jenkins.sh

140. By Chad Smith on 2015-12-16

add parent_card param to LeankitTaskBoard so that we have a refernce to the taskboard's card on the parent_board. Use the parent_card for card path printing

139. By Chad Smith on 2015-12-16

card.save needs to be taskboard aware. When calling UpdateCard the parent_board.id needs to be specified, not the taskboard.id

138. By Chad Smith on 2015-12-16

fix has_branch conditional to first check if card.external_system_url is not None before trying to match BRANCH_REGEX

137. By Chad Smith on 2015-12-16

address review comments: approved branches that have not yet been merged will remain in the status LANDING. update copyright.

136. By Chad Smith on 2015-12-16

merge ~landscape/lp2kanban/landscape-deploy to drop configs

135. By Chad Smith on 2015-12-16

simplify moveToTaskBoard card method

134. By Chad Smith on 2015-12-16

process cards with linked branches in bugs2cards since a lot of the logic is common

133. By Chad Smith on 2015-12-16

landing_lanes == Dev::::Landed for config

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2012-09-15 18:20:28 +0000
3+++ .bzrignore 2015-12-16 04:22:55 +0000
4@@ -14,3 +14,4 @@
5 .emacs.desktop
6 credentials
7 .emacs.desktop.lock
8+logs/*
9
10=== modified file 'Makefile'
11--- Makefile 2013-03-18 15:45:56 +0000
12+++ Makefile 2015-12-16 04:22:55 +0000
13@@ -23,12 +23,12 @@
14
15
16 bin/buildout: download-cache
17- $(PYTHON) bootstrap.py -v 1.7.0
18+ $(PYTHON) bootstrap.py -v 1.7.1 -f eggs -c buildout.cfg
19 touch --no-create $@
20
21
22 $(PY): bin/buildout $(BUILDOUT_CFG) setup.py
23- PYTHONPATH=. ./bin/buildout -c $(BUILDOUT_CFG)
24+ ./bin/buildout -c $(BUILDOUT_CFG)
25
26
27 $(subst $(PY),,$(BUILDOUT_BIN)): $(PY)
28
29=== added file 'README.landscape'
30--- README.landscape 1970-01-01 00:00:00 +0000
31+++ README.landscape 2015-12-16 04:22:55 +0000
32@@ -0,0 +1,17 @@
33+This branch is based off lp:~bjornt/lp2kanban/landscape and contains the
34+configuration files needed for lp2kanban to process bugs for the Landscape
35+project.
36+
37+To deploy, first run ./create_creds.py to create the credentials file. You
38+should be logged into Launchpad as the lp2kanban user when doing this, unless
39+for testing.
40+
41+Next you need to add cron entries to run the bugs2kanban.py script. You will
42+need to run it twice, one for creating new bugs (create.ini) and one for
43+syncing the card information with bugs (sync.ini):
44+
45+ bin/py src/lp2kanban/bugs2kanban.py -c create.ini
46+ bin/py src/lp2kanban/bugs2kanban.py -c sync.ini
47+
48+The former is quite quick and can be run every minute. The latter is slower
49+and shouldn't be run more often than every 5 minutes.
50
51=== modified file 'bootstrap.py'
52--- bootstrap.py 2013-03-18 15:45:56 +0000
53+++ bootstrap.py 2015-12-16 04:22:55 +0000
54@@ -18,7 +18,11 @@
55 use the -c option to specify an alternate configuration file.
56 """
57
58-import os, shutil, sys, tempfile
59+import os
60+import shutil
61+import sys
62+import tempfile
63+
64 from optparse import OptionParser
65
66 tmpeggs = tempfile.mkdtemp()
67@@ -31,8 +35,8 @@
68 Simply run this script in a directory containing a buildout.cfg, using the
69 Python that you want bin/buildout to use.
70
71-Note that by using --setup-source and --download-base to point to
72-local resources, you can keep this script from going over the network.
73+Note that by using --find-links to point to local resources, you can keep
74+this script from going over the network.
75 '''
76
77 parser = OptionParser(usage=usage)
78@@ -48,48 +52,63 @@
79 "bootstrap and buildout will get the newest releases "
80 "even if they are alphas or betas."))
81 parser.add_option("-c", "--config-file",
82- help=("Specify the path to the buildout configuration "
83- "file to be used."))
84+ help=("Specify the path to the buildout configuration "
85+ "file to be used."))
86 parser.add_option("-f", "--find-links",
87- help=("Specify a URL to search for buildout releases"))
88+ help=("Specify a URL to search for buildout releases"))
89+parser.add_option("--allow-site-packages",
90+ action="store_true", default=False,
91+ help=("Let bootstrap.py use existing site packages"))
92+parser.add_option("--setuptools-version",
93+ help="use a specific setuptools version")
94
95
96 options, args = parser.parse_args()
97
98 ######################################################################
99-# load/install distribute
100+# load/install setuptools
101
102-to_reload = False
103 try:
104- import pkg_resources, setuptools
105- if not hasattr(pkg_resources, '_distribute'):
106- to_reload = True
107- raise ImportError
108+ if options.allow_site_packages:
109+ import setuptools
110+ import pkg_resources
111+ from urllib.request import urlopen
112 except ImportError:
113- ez = {}
114-
115- try:
116- from urllib.request import urlopen
117- except ImportError:
118- from urllib2 import urlopen
119-
120- exec(urlopen('http://python-distribute.org/distribute_setup.py').read(), ez)
121- setup_args = dict(to_dir=tmpeggs, download_delay=0, no_fake=True)
122- ez['use_setuptools'](**setup_args)
123-
124- if to_reload:
125- reload(pkg_resources)
126- import pkg_resources
127- # This does not (always?) update the default working set. We will
128- # do it.
129- for path in sys.path:
130- if path not in pkg_resources.working_set.entries:
131- pkg_resources.working_set.add_entry(path)
132+ from urllib2 import urlopen
133+
134+ez = {}
135+exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)
136+
137+if not options.allow_site_packages:
138+ # ez_setup imports site, which adds site packages
139+ # this will remove them from the path to ensure that incompatible versions
140+ # of setuptools are not in the path
141+ import site
142+ # inside a virtualenv, there is no 'getsitepackages'.
143+ # We can't remove these reliably
144+ if hasattr(site, 'getsitepackages'):
145+ for sitepackage_path in site.getsitepackages():
146+ sys.path[:] = [x for x in sys.path if sitepackage_path not in x]
147+
148+setup_args = dict(to_dir=tmpeggs, download_delay=0)
149+
150+if options.setuptools_version is not None:
151+ setup_args['version'] = options.setuptools_version
152+
153+ez['use_setuptools'](**setup_args)
154+import setuptools
155+import pkg_resources
156+
157+# This does not (always?) update the default working set. We will
158+# do it.
159+for path in sys.path:
160+ if path not in pkg_resources.working_set.entries:
161+ pkg_resources.working_set.add_entry(path)
162
163 ######################################################################
164 # Install buildout
165
166-ws = pkg_resources.working_set
167+ws = pkg_resources.working_set
168
169 cmd = [sys.executable, '-c',
170 'from setuptools.command.easy_install import main; main()',
171@@ -104,8 +123,8 @@
172 if find_links:
173 cmd.extend(['-f', find_links])
174
175-distribute_path = ws.find(
176- pkg_resources.Requirement.parse('distribute')).location
177+setuptools_path = ws.find(
178+ pkg_resources.Requirement.parse('setuptools')).location
179
180 requirement = 'zc.buildout'
181 version = options.version
182@@ -113,13 +132,19 @@
183 # Figure out the most recent final version of zc.buildout.
184 import setuptools.package_index
185 _final_parts = '*final-', '*final'
186+
187 def _final_version(parsed_version):
188- for part in parsed_version:
189- if (part[:1] == '*') and (part not in _final_parts):
190- return False
191- return True
192+ try:
193+ return not parsed_version.is_prerelease
194+ except AttributeError:
195+ # Older setuptools
196+ for part in parsed_version:
197+ if (part[:1] == '*') and (part not in _final_parts):
198+ return False
199+ return True
200+
201 index = setuptools.package_index.PackageIndex(
202- search_path=[distribute_path])
203+ search_path=[setuptools_path])
204 if find_links:
205 index.add_find_links((find_links,))
206 req = pkg_resources.Requirement.parse(requirement)
207@@ -142,10 +167,9 @@
208 cmd.append(requirement)
209
210 import subprocess
211-if subprocess.call(cmd, env=dict(os.environ, PYTHONPATH=distribute_path)) != 0:
212+if subprocess.call(cmd, env=dict(os.environ, PYTHONPATH=setuptools_path)) != 0:
213 raise Exception(
214- "Failed to execute command:\n%s",
215- repr(cmd)[1:-1])
216+ "Failed to execute command:\n%s" % repr(cmd)[1:-1])
217
218 ######################################################################
219 # Import and run buildout
220
221=== modified file 'buildout.cfg'
222--- buildout.cfg 2013-06-19 19:44:20 +0000
223+++ buildout.cfg 2015-12-16 04:22:55 +0000
224@@ -11,6 +11,7 @@
225 exec-sitecustomize = true
226 develop = .
227 eggs-directory = eggs
228+offline = false
229
230 [test]
231 recipe = zc.recipe.testrunner
232@@ -38,13 +39,12 @@
233 lazr.restfulclient = 0.11.2
234 lazr.uri = 1.0.2
235 oauth = 1.0.1
236-setuptools = 0.6c11
237 simplejson = 2.1.6
238 wadllib = 1.2.0
239 wsgi-intercept = 0.5.0
240 z3c.recipe.scripts = 1.0.1
241 z3c.recipe.tag = 0.4.0
242-zc.buildout = 1.5.2
243+zc.buildout = 1.7.1
244 zc.recipe.egg = 1.3.2
245 zc.recipe.testrunner = 1.4.0
246 zope.exceptions = 3.6.1
247
248=== added directory 'configs'
249=== added file 'configs/groups.ini'
250--- configs/groups.ini 1970-01-01 00:00:00 +0000
251+++ configs/groups.ini 2015-12-16 04:22:55 +0000
252@@ -0,0 +1,16 @@
253+# The group name, which will replace ${group} in lane configs.
254+# The Launchpad account names of the group members.
255+[Alpha]
256+members = benji,simpoir
257+[Beta]
258+members = fcorrea,chad.smith
259+[Gamma]
260+members = adam-collard,bogdana
261+[Delta]
262+members = tealeg,danilo
263+[Epsilon]
264+members = free.ekanayaka,ack
265+[Zeta]
266+members = bjornt,tribaal
267+[QA]
268+members = ahasenack
269
270=== added file 'configs/sync.ini'
271--- configs/sync.ini 1970-01-01 00:00:00 +0000
272+++ configs/sync.ini 2015-12-16 04:22:55 +0000
273@@ -0,0 +1,106 @@
274+[GLOBAL]
275+# This is the account part of https://<account>.leankitkanban.com/ URL.
276+account = canonical
277+# LeankitKanban user name and password.
278+user = landscape-crew@lists.canonical.com
279+password = fShv0K4h@LV:!{
280+
281+# Defaults are common values overridable in each of the boards definition.
282+[DEFAULT]
283+lp_server = production
284+launchpadlib_dir = .launchpadlib.create
285+
286+# Should all cards with external IDs be synced or not. Set to 'on' if
287+# you want them synced automatically (and use "(no-sync)" as the marker
288+# for cards you don't want synced).
289+autosync = on
290+
291+# Should the cards be moved? If not specified it is off.
292+move_cards = on
293+
294+# Should the cards be synced? If not specified it is off.
295+sync_cards = on
296+
297+# Bug tag for which new cards should be created. If a bug in Launchpad
298+# has this tag, a new card will be created for it in the first kanban lane.
299+# If this config option isn't set, no cards will be created.
300+# After the card has been created, the tag will be removed from the bug, so a
301+# credentials_file has to be specified as well, to authenticate against LP.
302+bug_to_card_tag = kanban
303+
304+# Which card type should cards created from bugs have. If not set, the default
305+# card type is used.
306+bug_to_card_type = Bug
307+
308+bug_to_card_lane = Backlog
309+
310+# By default, omnidirectional card moves are off, which means that cards will
311+# be moved only to the lane that is considered to be next of the card's
312+# current lane. I.e. a card will move only within a "swim lane". If
313+# omnidirectional card moves are turned on, a card can move to any lane
314+# on the board.
315+omnidirectional_card_moves = on
316+
317+# Lanes are assigned a "role" by looking for sublanes matching
318+# values given here. "Coding" will match all lanes which have
319+# a title of "Coding", and "QA::Ready" will match all lanes
320+# titled "Ready" that have a parent lane with title "QA".
321+#
322+# Note that bugs2cards.py analyzes the lane structure and will flow
323+# cards only within the same horizontal "lane" (iow, if you have
324+# parallel 'Feature' and 'Bugs' development lanes, cards will flow
325+# only from Coding -> Review inside them, and then will move to the
326+# next lane horizontally (eg. Landing lane).
327+#coding_lanes =
328+#review_lanes =
329+#landing_lanes =
330+#qa_lanes =
331+#deploy_lanes =
332+#downtime_deploy_lanes =
333+#done_lanes =
334+
335+
336+# If you track multiple groups of people on one board, you can specify a
337+# groups config that will map a card to a group according to its
338+#assignee. You can then make the lane configs to include the group
339+# name, by using ${group} in the lane name. For example:
340+#
341+# coding_lanes = ${group}::Coding
342+#
343+groups_config_file = ./configs/groups.ini
344+
345+credentials_file = ./credentials
346+
347+
348+#[Landscape Cisco]
349+# Keep [Landscape Cisco] board before [Landscape] so it gets the "falkor" cards
350+# (we can only use external ID for one card across all of Leankit).
351+#projects = falkor
352+
353+#bug_to_card_lane = Backlog
354+
355+#coding_lanes = Doing
356+#review_lanes = Doing
357+#landing_lanes = Done
358+#deploy_lanes = Done
359+#done_lanes = Done
360+
361+#no_move_lanes =
362+#no_sync_lanes =
363+
364+
365+[Landscape 2016]
366+project_group = landscape-project
367+
368+# TODO: This should put charm bugs in ::Charm and server/client bugs
369+# in ::Landscape, for now stick everything in ::Landscape
370+bug_to_card_lane = Backlog::Engineering
371+
372+coding_lanes = Development::${group}::Doing
373+review_lanes = Development::${group}::Review
374+landing_lanes = Development::${group}::Landed
375+deploy_lanes = QA::Landed
376+done_lanes = Done
377+
378+no_move_lanes = Done,QA::Landed,QA::In progress
379+no_sync_lanes = Backlog::Engineering,Done,QA::Landed,QA::In progress,Archive,Ready for QA,Release Ready,Staging,Production,LDS
380
381=== modified file 'create_creds.py'
382--- create_creds.py 2012-12-10 15:21:44 +0000
383+++ create_creds.py 2015-12-16 04:22:55 +0000
384@@ -7,7 +7,7 @@
385
386 web_root="https://launchpad.net/"
387
388-creds = Credentials("tarmac")
389+creds = Credentials("lp2kanban-landscape")
390 url = creds.get_request_token(web_root=web_root)
391
392 subprocess.call(['xdg-open', url])
393
394=== added file 'jenkins.sh'
395--- jenkins.sh 1970-01-01 00:00:00 +0000
396+++ jenkins.sh 2015-12-16 04:22:55 +0000
397@@ -0,0 +1,32 @@
398+#!/bin/bash -xe
399+
400+ini="$1.ini"
401+date "+START: %c"
402+
403+#export https_proxy=http://squid.external:3128
404+#export http_proxy=http://squid.external:3128
405+
406+make
407+
408+if [ ! -e credentials ]; then
409+ if [ -e ~/credentials ]; then
410+ cp ~/credentials credentials
411+ else
412+ echo "ERROR: try make credentials first!"
413+ exit 1
414+ fi
415+fi
416+
417+# default encoding of stdout/stderr when not attached to a terminal is set to
418+# 'None'. 'PYTHONIOENCODING' overrides, could also be overridden in the
419+# python program by modifying 'sys.stdout.encoding' directly. This feels
420+# cleaner. Note setting LANG (which the terminal respects) wont fix this,
421+# since there is no terminal attached.
422+# --------
423+# See:
424+# - https://wiki.python.org/moin/PrintFails
425+# - https://stackoverflow.com/questions/1473577
426+# - https://stackoverflow.com/questions/492483
427+# - https://stackoverflow.com/questions/9932406
428+PYTHONIOENCODING="utf-8" bin/py src/lp2kanban/bugs2cards.py -c configs/${1}.ini
429+date "+END: %c"
430
431=== added directory 'logs'
432=== added file 'run.sh'
433--- run.sh 1970-01-01 00:00:00 +0000
434+++ run.sh 2015-12-16 04:22:55 +0000
435@@ -0,0 +1,21 @@
436+#!/bin/bash
437+
438+LOGFILE="logs/${1}-`date '+%Y-%m-%d'`.log"
439+LOCKFILE="$HOME/lp2kanban/run-$1.lock"
440+exit_status=0
441+export https_proxy=http://squid.external:3128
442+
443+cd $HOME/lp2kanban
444+echo -n "START " >> $LOGFILE
445+date "+%c" >> $LOGFILE
446+lockfile -10 -r 4 $LOCKFILE
447+if [ "$?" -eq 0 ]; then
448+ bin/py src/lp2kanban/bugs2cards.py -c configs/${1}.ini 2>&1 >> $LOGFILE
449+ exit_status=$?
450+ rm -f $LOCKFILE
451+else
452+ echo "Failed to grab lockfile $LOCKFILE" >> $LOGFILE
453+ exit_status=1
454+fi
455+echo -n "END " >> $LOGFILE; date "+%c" >> $LOGFILE
456+exit $?
457
458=== modified file 'src/lp2kanban/bugs2cards.py'
459--- src/lp2kanban/bugs2cards.py 2013-06-19 19:44:20 +0000
460+++ src/lp2kanban/bugs2cards.py 2015-12-16 04:22:55 +0000
461@@ -1,4 +1,4 @@
462-# Copyright 2011 Canonical Ltd
463+# Copyright 2015 Canonical Ltd
464 #
465 from ConfigParser import ConfigParser
466 from argparse import ArgumentParser
467@@ -9,6 +9,7 @@
468 update_blueprints_from_work_items,
469 )
470 from lp2kanban.kanban import (
471+ BRANCH_REGEX,
472 LeankitKanban,
473 Record,
474 )
475@@ -117,6 +118,9 @@
476 for user in board.users:
477 lp_user = lp.people.getByEmail(email=user)
478 if lp_user is None:
479+ print (
480+ "WARNING: There is no matching Launchpad user for kanban"
481+ " user %s." % (user,))
482 continue
483 self.lp_to_kanban[lp_user.name] = board.users[user]
484 self.kanban_to_lp[board.users[user]] = lp_user
485@@ -161,8 +165,10 @@
486
487 def _get_mp_info():
488 mps = []
489- for bug_branch in branches:
490- for mp in bug_branch.branch.landing_targets:
491+ for branch in branches:
492+ if hasattr(branch, 'branch') :
493+ branch = branch.branch
494+ for mp in branch.landing_targets:
495 mp_info = Record(rank=None, status=None, mp=None)
496 status = mp.queue_status
497 mp_info.rank = ORDERED_STATUSES.get(status, 1)
498@@ -222,8 +228,10 @@
499 if conf.get('autosync', None) == 'on':
500 has_id = (card.external_card_id is not None and
501 card.external_card_id.strip() != '')
502- return (has_id and NO_SYNC_MARKER not in card.title and
503- NO_SYNC_MARKER not in card.description)
504+ has_branch = BRANCH_REGEX.match(card.external_system_url)
505+ if NO_SYNC_MARKER in card.title or NO_SYNC_MARKER in card.description:
506+ return False
507+ return (has_id or has_branch)
508 else:
509 return (card.title.startswith(TITLE_MARKER) or
510 DESCRIPTION_MARKER in card.description)
511@@ -296,15 +304,21 @@
512 """Return the status of the card as one of CardStatus values."""
513 # status of None means card needs not to be moved.
514 status = None
515- if bug_status in IN_PROGRESS_BUG_STATUSES:
516- status = CardStatus.CODING
517+ branch_card_status = None
518+ if branch_info:
519 if branch_info.status == 'In Progress':
520- status = CardStatus.CODING
521+ branch_card_status = CardStatus.CODING
522 elif branch_info.status == 'In Review':
523- status = CardStatus.REVIEW
524+ branch_card_status = CardStatus.REVIEW
525 elif branch_info.status == 'Approved':
526- status = CardStatus.LANDING
527+ branch_card_status = CardStatus.LANDING
528 elif branch_info.status == 'Merged':
529+ branch_card_status = CardStatus.LANDING
530+ if bug_status is None:
531+ return branch_card_status
532+ if bug_status in IN_PROGRESS_BUG_STATUSES:
533+ status = CardStatus.CODING
534+ if branch_info.status == 'Merged':
535 if bug_status == 'Fix Committed':
536 if 'qa-needstesting' in bug_tags:
537 status = CardStatus.QA
538@@ -315,6 +329,8 @@
539 status = CardStatus.DEPLOY
540 else:
541 status = CardStatus.LANDING
542+ elif branch_card_status:
543+ return branch_card_status
544 elif bug_status in DONE_BUG_STATUSES:
545 status = CardStatus.DONE
546 return status
547@@ -454,6 +470,37 @@
548 board, project, new_bug_tag, bconf.get("bug_to_card_type"),
549 bconf.get("bug_to_card_lane"))
550 print " Syncing cards:"
551+ # Process linked branch cards
552+ for card in board.getCardsWithExternalLinks(only_branches=True):
553+ if card.external_card_id:
554+ # Cards with external_id are processed by getCardsWithExternalIds
555+ continue
556+ branch_name = card.external_system_url.replace(
557+ "https://code.launchpad.net/", "")
558+ branch = lp.branches.getByUniqueName(unique_name=branch_name)
559+ if not branch:
560+ print "Invalid branch url ({}) for card '{}'".format(
561+ card.external_system_url, card.title)
562+ continue
563+ if should_sync_card(card, bconf):
564+ branch_info = get_branch_info([branch])
565+ owner_name = BRANCH_REGEX.match(card.external_system_url).group(1)
566+ card_status = get_card_status(None, '', branch_info)
567+ if branch_info.status and should_move_card(card, bconf):
568+ assignee = Record(name=owner_name)
569+ try:
570+ move_card(card, card_status, bconf, [assignee], lp_users)
571+ except IOError as e:
572+ print " * %s (in %s)" % (card.title, card.lane.path)
573+ print " >>> Error moving card:"
574+ print " ", e
575+ continue
576+ kanban_user = lp_users.lp_to_kanban.get(assignee.name, None)
577+ if kanban_user:
578+ card.assigned_user_id = kanban_user.id
579+ card.kanban_user = kanban_user
580+ print " * %s (in %s)" % (card.title, card.lane.path)
581+
582 for card in board.getCardsWithExternalIds():
583 if should_sync_card(card, bconf):
584 try:
585
586=== modified file 'src/lp2kanban/kanban.py'
587--- src/lp2kanban/kanban.py 2013-05-24 14:48:07 +0000
588+++ src/lp2kanban/kanban.py 2015-12-16 04:22:55 +0000
589@@ -11,6 +11,8 @@
590
591
592 ANNOTATION_REGEX = re.compile('^\s*{.*}\s*$', re.MULTILINE|re.DOTALL)
593+BRANCH_REGEX = re.compile(
594+ '^https://code.launchpad.net/~([.\w]*)/([-\w]*)/([-\w]*)$')
595
596
597 class Record(dict):
598@@ -55,8 +57,7 @@
599
600 class LeankitConnector(object):
601 def __init__(self, account, username=None, password=None, throttle=1):
602- host = 'https://' + account + '.leankitkanban.com'
603- self.base_api_url = host + '/Kanban/Api'
604+ self.base_api_url = 'https://' + account + '.leankit.com'
605 self.http = self._configure_auth(username, password)
606 self.last_request_time = time.time() - throttle
607 self.throttle = throttle
608@@ -97,6 +98,10 @@
609 config={},
610 return_response=False)
611 sent = request.send()
612+ except AttributeError as e:
613+ # Weirdly, httplib2 has a habit of throwing an AttributeError
614+ # when it can't connect, so we handle that nicely.
615+ raise IOError("Unable to connect to LeanKitKanban server.")
616 except Exception as e:
617 raise IOError("Unable to make HTTP request: %s" % e.message)
618
619@@ -193,14 +198,17 @@
620 optional_attributes = [
621 'ExternalCardID', 'AssignedUserId', 'Size', 'IsBlocked',
622 'BlockReason', 'ExternalSystemName', 'ExternalSystemUrl',
623- 'ClassOfServiceId', 'DueDate',
624+ 'ClassOfServiceId', 'DueDate', 'CurrentTaskBoardId'
625 ]
626
627 def __init__(self, card_dict, lane):
628 super(LeankitCard, self).__init__(card_dict)
629
630 self.lane = lane
631- self.tags_list = set([tag.strip() for tag in self.tags.split(',')])
632+ if not self.tags:
633+ self.tags_list = []
634+ else:
635+ self.tags_list = set([tag.strip() for tag in self.tags.split(',')])
636 if '' in self.tags_list:
637 self.tags_list.remove('')
638 self.type = lane.board.cardtypes[self.type_id]
639@@ -214,7 +222,7 @@
640 tag = tag.strip()
641 if tag not in self.tags_list or self.tags.startswith(','):
642 if tag != '':
643- self.tags_list.add(tag)
644+ self.tags_list.append(tag)
645 self.tags = ', '.join(self.tags_list)
646
647 def save(self):
648@@ -241,10 +249,12 @@
649 del data['Id']
650 del data['LaneId']
651 position = len(self.lane.cards)
652- url_parts = ['/Board', str(self.lane.board.id), 'AddCard',
653- 'Lane', str(self.lane.id), 'Position', str(position)]
654+ url_parts = ['/Kanban/Api/Board', str(self.lane.board.id),
655+ 'AddCard', 'Lane', str(self.lane.id),
656+ 'Position', str(position)]
657 else:
658- url_parts = ['/Board', str(self.lane.board.id), 'UpdateCard']
659+ url_parts = ['/Kanban/Api/Board', str(self.lane.board.id),
660+ 'UpdateCard']
661
662 url = '/'.join(url_parts)
663
664@@ -267,9 +277,23 @@
665 else:
666 return None
667
668+ def moveToTaskBoard(self, parent_board, target_card):
669+ url = '/Api/Card/MoveCardToTaskboard'
670+ result = self.lane.board.connector.post(url, data={
671+ 'boardId': parent_board.id,
672+ 'destCardId': target_card.id,
673+ 'srcCardId': [self.id]})
674+ if result.ReplyCode in LeankitResponseCodes.SUCCESS_CODES:
675+ return result.ReplyData[0]
676+ else:
677+ raise Exception(
678+ "Moving card %s (%s) to %s failed. " % (
679+ self.title, self.id, self.lane.path) +
680+ "Error %s: %s" % (result.ReplyCode, result.ReplyText))
681+
682 def _moveCard(self):
683 target_pos = len(self.lane.cards)
684- url = '/Board/%d/MoveCard/%d/Lane/%d/Position/%d' % (
685+ url = '/Kanban/Api/Board/%d/MoveCard/%d/Lane/%d/Position/%d' % (
686 self.lane.board.id, self.id, self.lane.id, target_pos)
687 result = self.lane.board.connector.post(url, data=None)
688 if result.ReplyCode in LeankitResponseCodes.SUCCESS_CODES:
689@@ -323,6 +347,8 @@
690 text_after_json), where json_annotations contains the
691 JSON loaded with json.loads().
692 """
693+ if not self.description:
694+ self.description = ""
695 match = ANNOTATION_REGEX.search(self.description)
696 if match:
697 start = match.start()
698@@ -330,7 +356,7 @@
699 try:
700 annotations = Record(json.loads(self.description[start:end]))
701 except ValueError, ex:
702- print "Unable to parse card %i: %s" % (self.id, ex.message)
703+ print "Unable to parse card %s: %s" % (self.id, ex.message)
704 annotations = Record()
705 return (
706 annotations,
707@@ -450,7 +476,7 @@
708
709 attributes = ['Id', 'Title', 'CreationDate', 'IsArchived']
710
711- base_uri = '/Boards/'
712+ base_uri = '/Kanban/Api/Boards/'
713
714 def __init__(self, board_dict, connector):
715 super(LeankitBoard, self).__init__(board_dict)
716@@ -469,12 +495,20 @@
717 self._cards_with_external_ids = set()
718 self._cards_with_description_annotations = set()
719 self._cards_with_external_links = set()
720+ self._cards_with_branches = set()
721 self.default_cardtype = None
722
723 def getCardsWithExternalIds(self):
724 return self._cards_with_external_ids
725
726- def getCardsWithExternalLinks(self):
727+ def getCardsWithExternalLinks(self, only_branches=False):
728+ """Return cards with external links
729+
730+ @param only_merge_proposals: Only return cards that have merge
731+ proposals specified as the card's external link
732+ """
733+ if only_branches:
734+ return self._cards_with_branches
735 return self._cards_with_external_links
736
737 def getCardsWithDescriptionAnnotations(self):
738@@ -487,15 +521,23 @@
739 self._populateUsers(self.details['BoardUsers'])
740 self._populateCardTypes(self.details['CardTypes'])
741 self._archive = self.connector.get(
742- "/Board/" + str(self.id) + "/Archive").ReplyData[0]
743+ "/Kanban/Api/Board/" + str(self.id) + "/Archive").ReplyData[0]
744 archive_lanes = [lane_dict['Lane'] for lane_dict in self._archive]
745 archive_lanes.extend(
746 [lane_dict['Lane'] for
747 lane_dict in self._archive[0]['ChildLanes']])
748 self._backlog = self.connector.get(
749- "/Board/" + str(self.id) + "/Backlog").ReplyData[0]
750+ "/Kanban/Api/Board/" + str(self.id) + "/Backlog").ReplyData[0]
751 self._populateLanes(
752 self.details['Lanes'] + archive_lanes + self._backlog)
753+ for card in self.cards:
754+ if card.current_task_board_id: # We are a task board
755+ taskboard_data = self.connector.get(
756+ "/Kanban/Api/v1/board/%s/card/%s/taskboard" % (self.id, card.id))
757+ card.taskboard = LeankitTaskBoard(
758+ taskboard_data["ReplyData"][0], self)
759+ card.taskboard.fetchDetails()
760+ self.cards.extend(card.taskboard.cards)
761 self._classifyCards()
762
763 def _classifyCards(self):
764@@ -508,12 +550,16 @@
765 self._cards_with_description_annotations.add(card)
766 if card.external_system_url:
767 self._cards_with_external_links.add(card)
768+ if BRANCH_REGEX.match(card.external_system_url):
769+ self._cards_with_branches.add(card)
770 print " - %s cards with external ids" % len(
771 self._cards_with_external_ids)
772 print " - %s cards with external links" % len(
773 self._cards_with_external_links)
774 print " - %s cards with description annotations" % len(
775 self._cards_with_description_annotations)
776+ print " - %s cards with branches" % len(
777+ self._cards_with_branches)
778
779 def _populateUsers(self, user_data):
780 self.users = {}
781@@ -625,6 +671,21 @@
782 self._printLanes(lane, indent, include_cards)
783
784
785+class LeankitTaskBoard(LeankitBoard):
786+
787+ attributes = ['Id', 'Title']
788+
789+ def __init__(self, taskboard_dict, board):
790+ super(LeankitTaskBoard, self).__init__(taskboard_dict, board.connector)
791+ self.base_uri = '/Api/Board/%d/TaskBoard/%s/Get' % (board.id, self.id)
792+ self.cardtypes = board.cardtypes
793+ self.parent_board = board
794+
795+ def fetchDetails(self):
796+ self.details = self.connector.get(self.base_uri).ReplyData[0]
797+ self._populateLanes(self.details['Lanes'])
798+
799+
800 class LeankitKanban(object):
801
802 def __init__(self, account, username=None, password=None):
803@@ -638,7 +699,7 @@
804
805 :param include_archived: if True, include archived boards as well.
806 """
807- boards_data = self.connector.get('/Boards').ReplyData
808+ boards_data = self.connector.get('/Kanban/Api/Boards').ReplyData
809 boards = []
810 for board_dict in boards_data[0]:
811 board = LeankitBoard(board_dict, self.connector)
812@@ -683,13 +744,14 @@
813 if __name__ == '__main__':
814 kanban = LeankitKanban('launchpad.leankitkanban.com',
815 'user@email', 'password')
816+
817 print "Active boards:"
818 boards = kanban.getBoards()
819 for board in boards:
820 print " * %s (%d)" % (board.title, board.id)
821
822 # Get a board by the title.
823- board_name = 'lp2kanban test'
824+ board_name = 'Landscape 2016'
825 print "Getting board '%s'..." % board_name
826 board = kanban.getBoard(title=board_name)
827 board.printLanes()
828
829=== modified file 'src/lp2kanban/tests/common.py'
830--- src/lp2kanban/tests/common.py 2013-04-08 12:46:05 +0000
831+++ src/lp2kanban/tests/common.py 2015-12-16 04:22:55 +0000
832@@ -33,13 +33,16 @@
833 self.is_archived = is_archived
834 self._cards_with_description_annotations = set()
835 self._cards_with_external_links = set()
836+ self._cards_with_branches = set()
837 self.lanes = {}
838 self.root_lane = self.addLane('ROOT LANE')
839
840 def getCardsWithDescriptionAnnotations(self):
841 return self._cards_with_description_annotations
842
843- def getCardsWithExternalLinks(self):
844+ def getCardsWithExternalLinks(self, only_branches=False):
845+ if only_branches:
846+ return self._cards_with_branches
847 return self._cards_with_external_links
848
849 def getLaneByPath(self, path):
850@@ -93,7 +96,7 @@
851 def __init__(self, external_card_id=None, title=u"", description=u"",
852 description_annotations=None, lane=None,
853 assigned_user_id=None, external_system_name=None,
854- external_system_url=None):
855+ external_system_url=u""):
856 self.external_card_id = external_card_id
857 self.title = title
858 self.description = description
859
860=== modified file 'src/lp2kanban/tests/test_bugs2cards.py'
861--- src/lp2kanban/tests/test_bugs2cards.py 2013-04-08 12:50:46 +0000
862+++ src/lp2kanban/tests/test_bugs2cards.py 2015-12-16 04:22:55 +0000
863@@ -346,6 +346,7 @@
864 # in CODING.
865 self.assertEqual(CardStatus.CODING, status)
866
867+BRANCH_URL = "https://code.launchpad.net/~me/project/branch-name"
868
869 class CardStatusTest(unittest.TestCase):
870
871@@ -375,23 +376,43 @@
872
873 def test_should_sync_card_autosync_no(self):
874 # When autosync is 'on', cards with no external card IDs
875- # are not synced.
876+ # and no branches are not synced.
877 card = Record(title=u'no sync', description=u'',
878- external_card_id=None)
879+ external_card_id=None, external_system_url=None)
880 self.assertFalse(should_sync_card(card, {'sync_cards': 'on'}))
881
882- def test_should_sync_card_autosync_yes(self):
883+ def test_should_sync_card_autosync_no_sync_with_non_branch_urls(self):
884+ # When autosync is 'on', cards with external_system_urls which are not
885+ # valid launchpad branches are not synced.
886+ non_branch_urls = [
887+ "http://www.google.com/",
888+ "https://bugs.launchpad.net/charms/+source/hacluster/+bug/1",
889+ "https://code.launchpad.net/~person/project/blah/+merge/111"
890+ ]
891+ for url in non_branch_urls:
892+ card = Record(title=u'no sync', description=u'',
893+ external_card_id=None, external_system_url=url)
894+ self.assertFalse(should_sync_card(card, {'sync_cards': 'on'}))
895+
896+ def test_should_sync_card_autosync_synced_with_external_card_id(self):
897 # When autosync is 'on', cards with external card IDs are synced.
898 card = Record(title=u'no sync', description=u'',
899- external_card_id='11')
900+ external_card_id=u'11', external_system_url=u'')
901+ self.assertTrue(should_sync_card(card, {'autosync': 'on',
902+ 'sync_cards': 'on'}))
903+
904+ def test_should_sync_card_autosync_synced_with_branch(self):
905+ # When autosync is 'on', cards with a valid branch are synced.
906+ card = Record(title=u'no sync', description=u'',
907+ external_card_id=u'11', external_system_url=BRANCH_URL)
908 self.assertTrue(should_sync_card(card, {'autosync': 'on',
909 'sync_cards': 'on'}))
910
911 def test_should_sync_card_autosync_nosync(self):
912- # When autosync is 'on', cards with external card IDs are not
913- # synced when they contain the no-sync marker.
914+ # When autosync is 'on', cards with external card IDs or a valid
915+ # branch are not synced when they contain the no-sync marker.
916 card = Record(title=u'(no-sync)', description=u'(no-sync)',
917- external_card_id='11')
918+ external_card_id=u'11', external_system_url=BRANCH_URL)
919 self.assertFalse(should_sync_card(card, {'autosync': 'on',
920 'sync_cards': 'on'}))
921
922@@ -452,7 +473,7 @@
923 self.assertFalse(should_sync_card(card1, conf))
924 self.assertFalse(should_sync_card(card2, conf))
925
926- def test_get_card_status_noop(self):
927+ def test_get_card_status_noop_no_branch(self):
928 # For a bug in 'New', 'Triaged', 'Confirmed' or 'Incomplete' statuses,
929 # the status is unknown.
930 self.assertEqual(None, get_card_status('New', [], None))
931@@ -460,6 +481,33 @@
932 self.assertEqual(None, get_card_status('Incomplete', [], None))
933 self.assertEqual(None, get_card_status('Triaged', [], None))
934
935+ def test_get_card_status_coding_no_bug(self):
936+ # For a card with no associated bug, an in progress branch status will
937+ # be in the coding state.
938+ branch_info = Record(status='In Progress', target=None)
939+ self.assertEqual(
940+ CardStatus.CODING,
941+ get_card_status(None, [], branch_info))
942+
943+ def test_get_card_status_review_no_bug(self):
944+ # For a card with no associated bug, an in review branch status will
945+ # be in the review state.
946+ branch_info = Record(status='In Review', target=None)
947+ self.assertEqual(
948+ CardStatus.REVIEW,
949+ get_card_status(None, [], branch_info))
950+
951+ def test_get_card_status_landing_no_bug(self):
952+ # For a card with no associated bug, an approved or merged branch
953+ # status will be in the landing state.
954+ branch_infos = [
955+ Record(status='Approved', target=None),
956+ Record(status='Merged', target=None)]
957+ for branch_info in branch_infos:
958+ self.assertEqual(
959+ CardStatus.LANDING,
960+ get_card_status(None, [], branch_info))
961+
962 def test_get_card_status_coding_no_branch(self):
963 # For a bug in 'In Progress' or 'Fix Committed' status, it is
964 # considered to be in the coding state if there is no branch.

Subscribers

People subscribed via source and target branches