Merge ~morphis/snappy-hwe-snaps/+git/jenkins-jobs:f/add-snap-docs-build into ~snappy-hwe-team/snappy-hwe-snaps/+git/jenkins-jobs:master

Proposed by Simon Fels
Status: Merged
Approved by: Konrad Zapałowicz
Approved revision: de9c52d08a653180f242938c7cd9ae2915f89f65
Merged at revision: 14e6ed725e26c6c417dd991d7c994062ccc10462
Proposed branch: ~morphis/snappy-hwe-snaps/+git/jenkins-jobs:f/add-snap-docs-build
Merge into: ~snappy-hwe-team/snappy-hwe-snaps/+git/jenkins-jobs:master
Diff against target: 299 lines (+234/-0)
6 files modified
jobs/snap/snap-build.yaml (+3/-0)
jobs/snap/snap-docs.sh (+38/-0)
jobs/snap/snap-docs.yaml (+38/-0)
jobs/snap/snap-project-jobs.yaml (+1/-0)
tools/se_utils/__init__.py (+90/-0)
tools/update-merge-proposal.py (+64/-0)
Reviewer Review Type Date Requested Status
Konrad Zapałowicz (community) continuous-integration Approve
Alfonso Sanchez-Beato Approve
System Enablement Bot continuous-integration Approve
Review via email: mp+330205@code.launchpad.net

Description of the change

Implement *-snap-docs build job

The new build job will build the documentation for every project and stage the generated documentation as build artifact for further inspection during the review process.

To post a comment you must log in.
Revision history for this message
System Enablement Bot (system-enablement-ci-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
System Enablement Bot (system-enablement-ci-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote :

LGTM

review: Approve
Revision history for this message
Konrad Zapałowicz (kzapalowicz) wrote :

ack

review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/jobs/snap/snap-build.yaml b/jobs/snap/snap-build.yaml
2index dec3b19..e5b22f5 100644
3--- a/jobs/snap/snap-build.yaml
4+++ b/jobs/snap/snap-build.yaml
5@@ -57,6 +57,9 @@
6 RESULTS_ID=$BUILD_TAG
7 ARCHITECTURE=$ARCHITECTURE
8 block: true
9+ - project: '{name}-snap-docs'
10+ current-parameters: true
11+ block: true
12 - project: '{name}-snap-test'
13 current-parameters: true
14 predefined-parameters: |
15diff --git a/jobs/snap/snap-docs.sh b/jobs/snap/snap-docs.sh
16new file mode 100644
17index 0000000..e256294
18--- /dev/null
19+++ b/jobs/snap/snap-docs.sh
20@@ -0,0 +1,38 @@
21+#!/bin/sh
22+#
23+# Copyright (C) 2017 Canonical Ltd
24+#
25+# This program is free software: you can redistribute it and/or modify
26+# it under the terms of the GNU General Public License version 3 as
27+# published by the Free Software Foundation.
28+#
29+# This program is distributed in the hope that it will be useful,
30+# but WITHOUT ANY WARRANTY; without even the implied warranty of
31+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
32+# GNU General Public License for more details.
33+#
34+# You should have received a copy of the GNU General Public License
35+# along with this program. If not, see <http://www.gnu.org/licenses/>.
36+
37+set -ex
38+
39+. "$WORKSPACE/.build_env"
40+
41+git clone -b $GIT_REPO_BRANCH $GIT_REPO $WORKSPACE/src
42+if [ ! -e $WORKSPACE/src/docs ]; then
43+ echo "WARNING: repository does not contain any documentation we can build. Aborting."
44+ exit 0
45+fi
46+
47+# Run documentation through the builder and ensure everything builds nicely.
48+cd $WORKSPACE/src/docs
49+documentation-builder
50+
51+# Generate an archive which will be available as build artifact
52+DATE=$(date --rfc-3339=date --utc)
53+(cd $WORKSPACE/src/docs/build ; tar cJf $WORKSPACE/docs-$DATE-$BUILD_ID.tar.xz *)
54+
55+$BUILD_SCRIPTS/tools/update-merge-proposal.py \
56+ --merge-proposal=$MERGE_PROPOSAL \
57+ --approve \
58+ --message="PASSED: Successfully build documentation, rev: $REVISION\n\nGenerated documentation is available at $BUILD_URL"
59diff --git a/jobs/snap/snap-docs.yaml b/jobs/snap/snap-docs.yaml
60new file mode 100644
61index 0000000..3954f13
62--- /dev/null
63+++ b/jobs/snap/snap-docs.yaml
64@@ -0,0 +1,38 @@
65+- job-template:
66+ name: '{name}-snap-docs'
67+ project-type: freestyle
68+ defaults: global
69+ description: "Build documentation bits of the snap"
70+ display-name: "{name}-snap-docs"
71+ concurrent: true
72+ node: snap && build
73+ parameters:
74+ - string:
75+ name: GIT_REPO
76+ default:
77+ description: "Source git repository"
78+ - string:
79+ name: GIT_REPO_BRANCH
80+ default:
81+ description: "Branch of the source git repository to use"
82+ - string:
83+ name: MERGE_PROPOSAL
84+ default:
85+ description: "Link to the merge proposal this build relates to"
86+ - string:
87+ name: REVISION
88+ default:
89+ description: "Revision from the git repository to use"
90+ builders:
91+ - shell:
92+ !include-raw:
93+ - common-job-prepare.sh
94+ - shell:
95+ !include-raw-escape:
96+ - snap-docs.sh
97+ publishers:
98+ - archive:
99+ artifacts: 'docs-*.tar.xz'
100+ allow-empty: 'true'
101+ fingerprint: true
102+ default-excludes: false
103\ No newline at end of file
104diff --git a/jobs/snap/snap-project-jobs.yaml b/jobs/snap/snap-project-jobs.yaml
105index b693af3..05e4f3f 100644
106--- a/jobs/snap/snap-project-jobs.yaml
107+++ b/jobs/snap/snap-project-jobs.yaml
108@@ -11,3 +11,4 @@
109 - '{name}-snap-trigger-ci'
110 - '{name}-snap-update-mp'
111 - '{name}-snap-automerger'
112+ - '{name}-snap-docs'
113diff --git a/tools/se_utils/__init__.py b/tools/se_utils/__init__.py
114index 280450c..64cc0cc 100644
115--- a/tools/se_utils/__init__.py
116+++ b/tools/se_utils/__init__.py
117@@ -20,6 +20,7 @@ import sys
118 import time
119 import logging
120 import os
121+import re
122 import yaml
123 from shutil import rmtree
124 from launchpadlib.credentials import RequestTokenAuthorizationEngine
125@@ -27,6 +28,11 @@ from lazr.restfulclient.errors import HTTPError
126 from launchpadlib.launchpad import Launchpad
127 from launchpadlib.credentials import UnencryptedFileCredentialStore
128
129+class LaunchpadVote():
130+ APPROVE = 'Approve'
131+ DISAPPROVE = 'Disapprove'
132+ NEEDS_FIXING = 'Needs Fixing'
133+
134 ACCESS_TOKEN_POLL_TIME = 10
135 WAITING_FOR_USER = """Open this link:
136 {}
137@@ -120,3 +126,87 @@ def load_config():
138 def get_config_option(name):
139 config = load_config()
140 return config[name]
141+
142+def get_branch_handle_from_url(lp_handle, url):
143+ """ Return a branch/repo handle for the given url.
144+ Returns a launchpad branch or git repository handle for the given url.
145+ :param lp_handle: launchpad API handle/instance
146+ :param url: url of the branch or git repository
147+ """
148+ if '+git' in url:
149+ name = url.replace('https://code.launchpad.net/', '')
150+ print('fetching repo: ' + name)
151+ try:
152+ return lp_handle.git_repositories.getByPath(path=name)
153+ except AttributeError:
154+ print('git_repositories.getByPath was not found. You may need to set lp_version=devel in the config')
155+ return None
156+ else:
157+ name = url.replace('https://code.launchpad.net/', 'lp:')
158+ name = name.replace('https://code.staging.launchpad.net/', 'lp://staging/')
159+ print('fetching branch: ' + name)
160+ return lp_handle.branches.getByUrl(url=name)
161+
162+def get_branch_from_mp(merge_proposal):
163+ """Return a link to branch given a link to a merge proposal.
164+
165+ If merge_proposal is:
166+ https://copde.launchpad.net/~user/project/name/+merge/12345
167+ then the result will be:
168+ https://copde.launchpad.net/~user/project/name/
169+
170+ :param merge_proposal: url of a launchpad merge proposal
171+ """
172+ m = re.search('(.*)\+merge/[0-9]+$', merge_proposal)
173+ if m:
174+ return m.group(1)
175+ return None
176+
177+def get_mp_handle_from_url(lp_handle, merge_proposal_link):
178+ """ Get launchpad handle for merge proposal given a merge proposal URL.
179+
180+ Returns None in case the merge proposal can't be found.
181+ :param merge_proposal_link: URL of the merge proposal
182+ """
183+ branch_link = get_branch_from_mp(merge_proposal_link)
184+ if not branch_link:
185+ print('Unable to get branch link from merge proposal link.')
186+ return None
187+
188+ branch = get_branch_handle_from_url(lp_handle, branch_link)
189+ if not branch:
190+ print('Branch {} does not exist'.format(branch_link))
191+ return None
192+
193+ print('mp_link: {}.'.format(merge_proposal_link))
194+
195+ for mp in branch.landing_targets:
196+ print('mp.web_link: {}'.format(mp.web_link))
197+ if mp.web_link == merge_proposal_link:
198+ return mp
199+
200+ return None
201+
202+def get_vote_subject(mp):
203+ """Given a mp handle return a subject for the vote message
204+
205+ Unfortunately there is no method in the API that gives you the "standard"
206+ subject that launchapd is using and some email clients (gmail) are
207+ grouping conversations into threads based on subject.
208+
209+ This returns what seems to be the launchpad way of doing subjects.
210+ :param mp: launchpad merge proposal handle
211+ """
212+
213+ if '+git' in mp.web_link:
214+ source = mp.source_git_repository.display_name.replace('lp:', '') + \
215+ ':' + \
216+ mp.source_git_path.replace('refs/heads/', '')
217+ target = mp.target_git_repository.display_name.replace('lp:', '') + \
218+ ':' + \
219+ mp.target_git_path.replace('refs/heads/', '')
220+ return 'Re: [Merge] {} into {}'.format(source, target)
221+ else:
222+ return 'Re: [Merge] {} into {}'.format(
223+ mp.source_branch.display_name,
224+ mp.target_branch.display_name)
225\ No newline at end of file
226diff --git a/tools/se_utils/__init__.pyc b/tools/se_utils/__init__.pyc
227deleted file mode 100644
228index f1be9ca..0000000
229Binary files a/tools/se_utils/__init__.pyc and /dev/null differ
230diff --git a/tools/update-merge-proposal.py b/tools/update-merge-proposal.py
231new file mode 100755
232index 0000000..9e0b95a
233--- /dev/null
234+++ b/tools/update-merge-proposal.py
235@@ -0,0 +1,64 @@
236+#!/usr/bin/env python
237+# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
238+#
239+# Copyright (C) 2017 Canonical Ltd
240+#
241+# This program is free software: you can redistribute it and/or modify
242+# it under the terms of the GNU General Public License version 3 as
243+# published by the Free Software Foundation.
244+#
245+# This program is distributed in the hope that it will be useful,
246+# but WITHOUT ANY WARRANTY; without even the implied warranty of
247+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
248+# GNU General Public License for more details.
249+#
250+# You should have received a copy of the GNU General Public License
251+# along with this program. If not, see <http://www.gnu.org/licenses/>.
252+
253+import atexit
254+import sys
255+import time
256+import logging
257+import os
258+import yaml
259+import re
260+from shutil import rmtree
261+from argparse import ArgumentParser
262+
263+import se_utils
264+
265+parser = ArgumentParser(description="Vote on a Launchpad merge proposal.")
266+parser.add_argument('-p', '--merge-proposal', required=True, help="URL of the merge proposal to update")
267+parser.add_argument('-a', '--approve', action='store_true', help="approve merge proposal")
268+parser.add_argument('-d', '--disapprove', action='store_true', help="disapprove merge proposal")
269+parser.add_argument('-m', '--message', help="message to add as a comment to the merge-proposal")
270+
271+args = vars(parser.parse_args())
272+
273+# Get us connected with launchpad
274+lp_app = se_utils.get_config_option("lp_app")
275+lp_env = se_utils.get_config_option("lp_env")
276+credential_store_path = se_utils.get_config_option('credential_store_path')
277+lp_handle = se_utils.get_launchpad(None, credential_store_path, lp_app, lp_env)
278+
279+mp = se_utils.get_mp_handle_from_url(lp_handle, args["merge_proposal"])
280+if not mp:
281+ parser.error('merge proposal related to this branch was not found')
282+
283+content = args["message"]
284+if content is None or len(content) == 0:
285+ parser.error('no message was provided')
286+# Unescape message to ensure we get all new lines etc.
287+content = content.decode('string_escape')
288+
289+if not args['approve'] and not args['disapprove']:
290+ parser.error('neither approved or disapproved the MP')
291+
292+vote=""
293+if args["approve"]:
294+ vote = se_utils.LaunchpadVote.APPROVE
295+else if args["disapprove"]:
296+ vote = se_utils.LaunchpadVote.DISAPPROVE
297+
298+subject = se_utils.get_vote_subject(mp)
299+mp.createComment(content=content, subject=subject)

Subscribers

People subscribed via source and target branches

to all changes: