Merge ~albaguirre/jenkaas-jobs:add-git-support into jenkaas-jobs:master

Proposed by Alberto Aguirre on 2016-08-13
Status: Needs review
Proposed branch: ~albaguirre/jenkaas-jobs:add-git-support
Merge into: jenkaas-jobs:master
Diff against target: 824 lines (+388/-80)
22 files modified
.gitignore (+3/-0)
dev/null (+0/-22)
jenkaas_jobs/yaml/build/build-0-fetch-bazaar.sh (+48/-0)
jenkaas_jobs/yaml/build/build-0-fetch-git.sh (+57/-0)
jenkaas_jobs/yaml/build/build-0-fetch.yaml (+4/-6)
jenkaas_jobs/yaml/build/build-1-sourcepkg.yaml (+1/-1)
jenkaas_jobs/yaml/build/build.yaml (+5/-6)
jenkaas_jobs/yaml/build/lp-generic-update-mp.sh (+1/-1)
jenkaas_jobs/yaml/build/lp-generic-update-mp.yaml (+53/-15)
jenkaas_jobs/yaml/build/macros.yaml (+42/-2)
jenkaas_jobs/yaml/credentials/credentials-0-ssh.sh (+4/-4)
jenkaas_jobs/yaml/credentials/credentials-1-git.sh (+8/-0)
jenkaas_jobs/yaml/credentials/credentials-1-vcs.yaml (+37/-0)
jenkaas_jobs/yaml/credentials/credentials-3-launchpad-plugin.sh (+5/-1)
jenkaas_jobs/yaml/credentials/credentials.yaml (+1/-1)
jenkaas_jobs/yaml/defaults.yaml (+8/-0)
jenkaas_jobs/yaml/launchpad/lp-ci.yaml (+97/-16)
jenkaas_jobs/yaml/launchpad/lp-generic-land-mp.yaml (+2/-2)
jenkaas_jobs/yaml/launchpad/lp-trigger-autolanding.sh (+1/-1)
jenkaas_jobs/yaml/launchpad/lp-trigger.sh (+1/-1)
jenkaas_jobs/yaml/prepare/prepare-0-install.sh (+2/-0)
project-example.yaml (+8/-1)
Reviewer Review Type Date Requested Status
Michał Sawicz 2016-08-13 Needs Information on 2016-09-09
Review via email: mp+302870@code.launchpad.net

Commit message

Add git support

Add support for git based launchpad projects.
It depends on an updated jenkins-launchpad-plugin that also understands git
based merge proposals and projects.

Breakdown of changes:

Credentials
- Parametrize credentials-0-ssh for bazaar only or git only access
- Rename credentials-1-bazaar to credentials-1-vcs
- Parametrize credentials-1-vcs so that it sets up bazaar or git for ssh
  access and additional config that makes 'git clone lp:project' work

Build
- Parametrize build-0-fetch so that it can also create source packages for git
  based projects
- Add additional "source_repo" parameter for git based projects as git merge
  proposals specify both a source repository + a source branch due to the
  nature of git.

Launchpad
- Specify repo type in lp-trigger.sh
- Add "landing_candidate_branch" parameter for git based projects
  so that we can propagate both the source repo + source branch to the build
  job

Other
- Build name customization depending on repo_type so that git based projects use
  <source_repo>:<source_branch>

Limitations
 - All projects defined need to be either bazaar or git - mixing and matching
   of project types is not currently possible.

To post a comment you must log in.
7421e65... by Alberto Aguirre on 2016-08-15

Remove extra curly bracket in ci-build parameter.

Michał Sawicz (saviq) wrote :

From a quick glance my problem with this is that it won't allow you mixing bzr and git projects within a single instance, that would mean you'd have to migrate all projects at once :/

Couldn't we instead set up for both VCS and parametrize on the project level (or even better, decide based on source/target branches and MP data)?

review: Needs Information

Unmerged commits

7421e65... by Alberto Aguirre on 2016-08-15

Remove extra curly bracket in ci-build parameter.

1559cd9... by Alberto Aguirre on 2016-08-13

Add git support

Add support for git based launchpad projects.
It depends on an updated jenkins-launchpad-plugin that also understands git
based merge proposals and projects.

Breakdown of changes:

Credentials
- Parametrize credentials-0-ssh for bazaar only or git only access
- Rename credentials-1-bazaar to credentials-1-vcs
- Parametrize credentials-1-vcs so that it sets up bazaar or git for ssh
  access and additional config that makes 'git clone lp:project' work

Build
- Parametrize build-0-fetch so that it can also create source packages for git
  based projects
- Add additional "source_repo" parameter for git based projects as git merge
  proposals specify both a source repository + a source branch due to the
  nature of git.

Launchpad
- Specify repo type in lp-trigger.sh
- Add "landing_candidate_branch" parameter for git based projects
  so that we can propagate both the source repo + source branch to the build
  job

Other
- Build name customization depending on repo_type so that git based projects use
  <source_repo>:<source_branch>

Limitations
 - All projects defined need to be either bazaar or git - mixing and matching
   of project types is not currently possible.

9acb946... by Alberto Aguirre on 2016-08-13

Don't ignore new files introduced in jenkaas_jobs/yaml/build/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/.gitignore b/.gitignore
2index 0b904cd..2cd6059 100644
3--- a/.gitignore
4+++ b/.gitignore
5@@ -26,3 +26,6 @@ var/
6
7 # Temporary files
8 .*.swp
9+
10+# But don't ignore new files here
11+!jenkaas_jobs/yaml/build/
12diff --git a/jenkaas_jobs/yaml/build/build-0-fetch-bazaar.sh b/jenkaas_jobs/yaml/build/build-0-fetch-bazaar.sh
13new file mode 100644
14index 0000000..006ef92
15--- /dev/null
16+++ b/jenkaas_jobs/yaml/build/build-0-fetch-bazaar.sh
17@@ -0,0 +1,48 @@
18+#!/bin/bash
19+
20+# log all commands and abort on error
21+set -xe
22+
23+# sanitize input
24+bzr revno ${source_branch} || (echo "ERROR: source branch incorrect"; exit 1)
25+[ -z "${source_revision}" ] || bzr revno -r ${source_revision} ${source_branch} || (echo "ERROR: source revision incorrect"; exit 2)
26+[ -z "${target_branch}" ] || bzr revno ${target_branch} || (echo "ERROR: target branch incorrect"; exit 3)
27+
28+WORKDIR=${PWD}/source
29+OUTDIR=${PWD}/output
30+
31+# get source branch revision if not provided
32+[ -z "${source_revision}" ] && source_revision=`bzr revno ${source_branch}`
33+
34+echo "INFO: @r${source_revision}"
35+
36+if [ -n "${target_branch}" ]; then
37+ # branch target and merge source
38+ bzr branch ${target_branch} ${WORKDIR}
39+ bzr merge -d ${WORKDIR} -r ${source_revision} ${source_branch}
40+else
41+ # or branch source alone
42+ bzr branch -r ${source_revision} ${source_branch} ${WORKDIR}
43+fi
44+
45+cd ${WORKDIR}
46+
47+# determine upstream version
48+UPSTREAM_VERSION=`dpkg-parsechangelog | perl -n -e '/^Version:\s(.*?)(-[0-9+](.*)?)?$/ && print $1'`
49+
50+# build version number
51+VERSION="${UPSTREAM_VERSION}+fetch${BUILD_NUMBER}bzr${source_revision}"
52+
53+NONNATIVE=`dpkg-parsechangelog | perl -n -e '/^Version:\s(.*?)(-.*)?$/ && print $2'`
54+if [ -n "${NONNATIVE}" ]; then
55+ VERSION="${VERSION}-0ubuntu1"
56+fi
57+
58+# add version to changelog
59+dch -v ${VERSION} "Automatic build of ${source_branch} at revision ${source_revision}"
60+
61+# add base information
62+[ -n "${target_branch}" ] && dch "Based on ${target_branch}"
63+
64+# build source package
65+bzr builddeb --builder="debuild -uc -us" --source --result-dir=${OUTDIR}
66diff --git a/jenkaas_jobs/yaml/build/build-0-fetch-git.sh b/jenkaas_jobs/yaml/build/build-0-fetch-git.sh
67new file mode 100644
68index 0000000..8e9bebc
69--- /dev/null
70+++ b/jenkaas_jobs/yaml/build/build-0-fetch-git.sh
71@@ -0,0 +1,57 @@
72+#!/bin/bash
73+
74+# log all commands and abort on error
75+set -xe
76+
77+WORKDIR=${PWD}/source
78+OUTDIR=${PWD}/output
79+
80+if [ -n "${target_branch}" ]; then
81+ # branch target and merge source
82+ git clone ${target_branch} ${WORKDIR}
83+ cd ${WORKDIR}
84+
85+ git remote add source ${source_repo}
86+ git fetch source ${source_branch}
87+
88+ if [ -z "${source_revision}" ]; then
89+ git merge source/${source_branch} -m "Merge ${source_branch}"
90+ else
91+ git merge ${source_revision} -m "Merge ${source_revision}"
92+ fi
93+else
94+ # or branch source alone
95+ git clone ${source_repo} ${WORKDIR}
96+ cd ${WORKDIR}
97+ # no revision provided, just checkout the latest branch
98+ if [ -z "${source_revision}" ]; then
99+ git checkout -b ${source_branch}
100+ else
101+ git checkout ${source_revision}
102+ fi
103+fi
104+
105+# get source branch revision if not provided
106+[ -z "${source_revision}" ] && source_revision=`git log --pretty=%H -n 1`
107+
108+echo "INFO: @commit:${source_revision}"
109+
110+# determine upstream version
111+UPSTREAM_VERSION=`dpkg-parsechangelog | perl -n -e '/^Version:\s(.*?)(-[0-9+](.*)?)?$/ && print $1'`
112+
113+# build version number
114+VERSION="${UPSTREAM_VERSION}+fetch${BUILD_NUMBER}bzr${source_revision}"
115+
116+NONNATIVE=`dpkg-parsechangelog | perl -n -e '/^Version:\s(.*?)(-.*)?$/ && print $2'`
117+if [ -n "${NONNATIVE}" ]; then
118+ VERSION="${VERSION}-0ubuntu1"
119+fi
120+
121+# add version to changelog
122+dch -v ${VERSION} "Automatic build of ${source_branch} at revision ${source_revision}"
123+
124+# add base information
125+[ -n "${target_branch}" ] && dch "Based on ${target_branch}"
126+
127+# build source package
128+yes | gbp buildpackage --git-ignore-new --git-builder="debuild -uc -us -S" --git-export-dir=${OUTDIR}
129diff --git a/jenkaas_jobs/yaml/build/build-0-fetch.sh b/jenkaas_jobs/yaml/build/build-0-fetch.sh
130deleted file mode 100644
131index 006ef92..0000000
132--- a/jenkaas_jobs/yaml/build/build-0-fetch.sh
133+++ /dev/null
134@@ -1,48 +0,0 @@
135-#!/bin/bash
136-
137-# log all commands and abort on error
138-set -xe
139-
140-# sanitize input
141-bzr revno ${source_branch} || (echo "ERROR: source branch incorrect"; exit 1)
142-[ -z "${source_revision}" ] || bzr revno -r ${source_revision} ${source_branch} || (echo "ERROR: source revision incorrect"; exit 2)
143-[ -z "${target_branch}" ] || bzr revno ${target_branch} || (echo "ERROR: target branch incorrect"; exit 3)
144-
145-WORKDIR=${PWD}/source
146-OUTDIR=${PWD}/output
147-
148-# get source branch revision if not provided
149-[ -z "${source_revision}" ] && source_revision=`bzr revno ${source_branch}`
150-
151-echo "INFO: @r${source_revision}"
152-
153-if [ -n "${target_branch}" ]; then
154- # branch target and merge source
155- bzr branch ${target_branch} ${WORKDIR}
156- bzr merge -d ${WORKDIR} -r ${source_revision} ${source_branch}
157-else
158- # or branch source alone
159- bzr branch -r ${source_revision} ${source_branch} ${WORKDIR}
160-fi
161-
162-cd ${WORKDIR}
163-
164-# determine upstream version
165-UPSTREAM_VERSION=`dpkg-parsechangelog | perl -n -e '/^Version:\s(.*?)(-[0-9+](.*)?)?$/ && print $1'`
166-
167-# build version number
168-VERSION="${UPSTREAM_VERSION}+fetch${BUILD_NUMBER}bzr${source_revision}"
169-
170-NONNATIVE=`dpkg-parsechangelog | perl -n -e '/^Version:\s(.*?)(-.*)?$/ && print $2'`
171-if [ -n "${NONNATIVE}" ]; then
172- VERSION="${VERSION}-0ubuntu1"
173-fi
174-
175-# add version to changelog
176-dch -v ${VERSION} "Automatic build of ${source_branch} at revision ${source_revision}"
177-
178-# add base information
179-[ -n "${target_branch}" ] && dch "Based on ${target_branch}"
180-
181-# build source package
182-bzr builddeb --builder="debuild -uc -us" --source --result-dir=${OUTDIR}
183diff --git a/jenkaas_jobs/yaml/build/build-0-fetch.yaml b/jenkaas_jobs/yaml/build/build-0-fetch.yaml
184index 7e1649f..d74ddc9 100644
185--- a/jenkaas_jobs/yaml/build/build-0-fetch.yaml
186+++ b/jenkaas_jobs/yaml/build/build-0-fetch.yaml
187@@ -1,11 +1,11 @@
188 - job-template:
189- name: build-0-fetch
190+ name: 'build-0-fetch-{repo_type}'
191 project-type: freestyle
192 description: 'This job fetches the source and builds a base source package.'
193 concurrent: true
194 node: build
195 parameters:
196- - branch_parameters
197+ - '{repo_type}_build_parameters'
198 properties:
199 - build-discarder:
200 days-to-keep: 60
201@@ -14,11 +14,9 @@
202 wrappers:
203 - workspace-cleanup
204 - timestamps
205- - build-name:
206- name: ${{ENV,var="source_branch"}}
207+ - '{repo_type}_build_name_wrapper'
208 builders:
209- - shell:
210- !include-raw-escape: build-0-fetch.sh
211+ - '{repo_type}_builder'
212 publishers:
213 - description_setter:
214 set_for_matrix: 'false'
215diff --git a/jenkaas_jobs/yaml/build/build-1-sourcepkg.yaml b/jenkaas_jobs/yaml/build/build-1-sourcepkg.yaml
216index d49a1d8..f25a31e 100644
217--- a/jenkaas_jobs/yaml/build/build-1-sourcepkg.yaml
218+++ b/jenkaas_jobs/yaml/build/build-1-sourcepkg.yaml
219@@ -7,7 +7,7 @@
220 parameters:
221 - run:
222 name: fetch_run
223- project-name: build-0-fetch
224+ project-name: build-0-fetch-{repo_type}
225 description: The run which to take the base source package from
226 properties:
227 - build-discarder:
228diff --git a/jenkaas_jobs/yaml/build/build.yaml b/jenkaas_jobs/yaml/build/build.yaml
229index 0742cec..0adc7a9 100644
230--- a/jenkaas_jobs/yaml/build/build.yaml
231+++ b/jenkaas_jobs/yaml/build/build.yaml
232@@ -2,7 +2,7 @@
233 name: build-jobs
234 jobs:
235 - build
236- - build-0-fetch
237+ - build-0-fetch-{repo_type}
238 - build-1-sourcepkg
239 - build-2-binpkg
240
241@@ -21,23 +21,22 @@
242 concurrent: true
243 node: monitor
244 parameters:
245- - branch_parameters
246+ - '{repo_type}_build_parameters'
247 properties:
248 - build-discarder:
249 days-to-keep: 60
250 - rebuild
251 wrappers:
252 - timestamps
253- - build-name:
254- name: ${{ENV,var="source_branch"}}
255+ - '{repo_type}_build_name_wrapper'
256 builders:
257 - trigger-builds:
258- - project: build-0-fetch
259+ - project: build-0-fetch-{repo_type}
260 current-parameters: true
261 <<: *trigger_defaults
262 - trigger-builds:
263 - project: build-1-sourcepkg
264- predefined-parameters: fetch_run=build-0-fetch#${{TRIGGERED_BUILD_NUMBERS_build_0_fetch}}
265+ predefined-parameters: fetch_run=build-0-fetch-{repo_type}#${{TRIGGERED_BUILD_NUMBERS_build_0_fetch_{repo_type}}}
266 <<: *trigger_defaults
267 - trigger-builds:
268 - project: build-2-binpkg
269diff --git a/jenkaas_jobs/yaml/build/lp-generic-update-mp.sh b/jenkaas_jobs/yaml/build/lp-generic-update-mp.sh
270index 1e23eda..e4ebef7 100644
271--- a/jenkaas_jobs/yaml/build/lp-generic-update-mp.sh
272+++ b/jenkaas_jobs/yaml/build/lp-generic-update-mp.sh
273@@ -3,4 +3,4 @@
274 # log all commands and abort on error
275 set -ex
276
277-voteOnMergeProposal --status="${ci_result}" --build-url="${ci_build}" --branch="${ci_branch}" --merge-proposal="${ci_merge_proposal}" --revision="${ci_revision}"
278\ No newline at end of file
279+voteOnMergeProposal --status="${ci_result}" --build-url="${ci_build}" --merge-proposal="${ci_merge_proposal}" --revision="${ci_revision}"
280diff --git a/jenkaas_jobs/yaml/build/lp-generic-update-mp.yaml b/jenkaas_jobs/yaml/build/lp-generic-update-mp.yaml
281index 6d5ba0e..30c6f32 100644
282--- a/jenkaas_jobs/yaml/build/lp-generic-update-mp.yaml
283+++ b/jenkaas_jobs/yaml/build/lp-generic-update-mp.yaml
284@@ -1,13 +1,5 @@
285-- job-template:
286- name: lp-generic-update-mp
287- project-type: freestyle
288- description: 'Update merge proposal on Launchpad.'
289- node: launchpad
290- properties:
291- - build-discarder:
292- num-to-keep: 100
293- - rebuild:
294- rebuild-disabled: true
295+- parameter:
296+ name: bazaar_mp_update_parameters
297 parameters:
298 - string:
299 name: ci_result
300@@ -20,14 +12,60 @@
301 description: Launchpad branch that was processed
302 - string:
303 name: ci_merge_proposal
304- description: Launchpad merge proposal that was processed
305+ description: Launchpad merge proposal that was processed
306+ - string:
307+ name: ci_revision
308+ description: Revision of the processed branch
309+
310+- parameter:
311+ name: git_mp_update_parameters
312+ parameters:
313+ - string:
314+ name: ci_result
315+ description: Result of the CI build
316+ - string:
317+ name: ci_build
318+ description: Jenkins URL of the build
319+ - string:
320+ name: ci_repo
321+ description: Launchpad git repo that was processed
322+ - string:
323+ name: ci_branch
324+ description: Launchpad git branch from ci_repo that was processed
325+ - string:
326+ name: ci_merge_proposal
327+ description: Launchpad merge proposal that was processed
328 - string:
329 name: ci_revision
330- description: Revision of the processed branch
331+ description: Revision of the processed branch
332+
333+- wrapper:
334+ name: git_mp_update_build_name_wrapper
335 wrappers:
336- - timestamps
337 - build-name:
338- name: ${{ENV,var="ci_branch"}}
339+ name: ${ENV,var="ci_repo"}:${ENV,var="ci_branch"}
340+
341+- wrapper:
342+ name: bazaar_mp_update_build_name_wrapper
343+ wrappers:
344+ - build-name:
345+ name: ${ENV,var="ci_branch"}
346+
347+- job-template:
348+ name: lp-generic-update-mp
349+ project-type: freestyle
350+ description: 'Update merge proposal on Launchpad.'
351+ node: launchpad
352+ properties:
353+ - build-discarder:
354+ num-to-keep: 100
355+ - rebuild:
356+ rebuild-disabled: true
357+ parameters:
358+ - '{repo_type}_mp_update_parameters'
359+ wrappers:
360+ - timestamps
361+ - '{repo_type}_mp_update_build_name_wrapper'
362 builders:
363 - shell:
364- !include-raw-escape: lp-generic-update-mp.sh
365\ No newline at end of file
366+ !include-raw-escape: lp-generic-update-mp.sh
367diff --git a/jenkaas_jobs/yaml/build/macros.yaml b/jenkaas_jobs/yaml/build/macros.yaml
368index d352acd..c715167 100644
369--- a/jenkaas_jobs/yaml/build/macros.yaml
370+++ b/jenkaas_jobs/yaml/build/macros.yaml
371@@ -1,5 +1,5 @@
372 - parameter:
373- name: branch_parameters
374+ name: bazaar_build_parameters
375 parameters:
376 - string:
377 name: source_branch
378@@ -9,4 +9,44 @@
379 description: Optional revision to fetch, defaults to pull the last one
380 - string:
381 name: target_branch
382- description: Optional target branch to merge the source on top of
383\ No newline at end of file
384+ description: Optional target branch to merge the source on top of
385+
386+- parameter:
387+ name: git_build_parameters
388+ parameters:
389+ - string:
390+ name: source_repo
391+ description: Git repository to fetch
392+ - string:
393+ name: source_branch
394+ description: Which git branch to fetch from source repo
395+ - string:
396+ name: source_revision
397+ description: Optional revision to fetch, defaults to pull the last one
398+ - string:
399+ name: target_branch
400+ description: Optional target branch to merge the source on top of
401+
402+- builder:
403+ name: git_builder
404+ builders:
405+ - shell:
406+ !include-raw: build-0-fetch-git.sh
407+
408+- builder:
409+ name: bazaar_builder
410+ builders:
411+ - shell:
412+ !include-raw: build-0-fetch-bazaar.sh
413+
414+- wrapper:
415+ name: git_build_name_wrapper
416+ wrappers:
417+ - build-name:
418+ name: ${ENV,var="source_repo"}:${ENV,var="source_branch"}
419+
420+- wrapper:
421+ name: bazaar_build_name_wrapper
422+ wrappers:
423+ - build-name:
424+ name: ${ENV,var="source_branch"}
425diff --git a/jenkaas_jobs/yaml/credentials/credentials-0-ssh.sh b/jenkaas_jobs/yaml/credentials/credentials-0-ssh.sh
426index 9dd20a5..3d06e00 100644
427--- a/jenkaas_jobs/yaml/credentials/credentials-0-ssh.sh
428+++ b/jenkaas_jobs/yaml/credentials/credentials-0-ssh.sh
429@@ -6,8 +6,8 @@ set -xe
430 # put it in $JENKINS_HOME so it doesn't get purged with the workspace
431 SSH_PATH="${{JENKINS_HOME}}/.ssh/"
432
433-# this is so that this key's only used with bazaar
434-SSH_KEY_PATH="${{SSH_PATH}}/bazaar.launchpad.net/{bot_username}"
435+# this is so that this key's only used with bazaar or git
436+SSH_KEY_PATH="${{SSH_PATH}}/{repo_type}.launchpad.net/{bot_username}"
437
438 if [ ! -d "${{SSH_KEY_PATH}}" ]; then
439 mkdir -p "${{SSH_KEY_PATH}}"
440@@ -19,7 +19,7 @@ cat > "${{SSH_PATH}}/config" << EOF
441 Host *
442 StrictHostKeyChecking no
443
444-Host bazaar.launchpad.net
445+Host {repo_type}.launchpad.net
446 IdentityFile ~/.ssh/%h/%r/id_rsa
447 EOF
448
449@@ -40,4 +40,4 @@ fi
450 echo "The public key for this jenkins user is:"
451 echo "----------"
452 cat "${{SSH_KEY_PATH}}/id_rsa.pub"
453-echo "----------"
454\ No newline at end of file
455+echo "----------"
456diff --git a/jenkaas_jobs/yaml/credentials/credentials-1-bazaar.yaml b/jenkaas_jobs/yaml/credentials/credentials-1-bazaar.yaml
457deleted file mode 100644
458index c1b9c73..0000000
459--- a/jenkaas_jobs/yaml/credentials/credentials-1-bazaar.yaml
460+++ /dev/null
461@@ -1,22 +0,0 @@
462-- job-template:
463- name: credentials-1-bazaar
464- project-type: matrix
465- description: This job configures Bazaar for Launchpad and branch access.
466- node: monitor
467- properties:
468- - build-discarder:
469- num-to-keep: 1
470- - rebuild
471- parameters:
472- - matrix-combinations:
473- name: nodes
474- axes:
475- - axis:
476- type: slave
477- name: node
478- values: '{obj:build_slaves}'
479- wrappers:
480- - timestamps
481- builders:
482- - shell:
483- !include-raw: credentials-1-bazaar.sh
484diff --git a/jenkaas_jobs/yaml/credentials/credentials-1-git.sh b/jenkaas_jobs/yaml/credentials/credentials-1-git.sh
485new file mode 100644
486index 0000000..235272d
487--- /dev/null
488+++ b/jenkaas_jobs/yaml/credentials/credentials-1-git.sh
489@@ -0,0 +1,8 @@
490+#!/bin/bash
491+
492+# log all commands and abort on error
493+set -xe
494+
495+git config --global url.git+ssh://{bot_username}@git.launchpad.net/.insteadof lp:
496+git config --global user.email "{bot_email}"
497+git config --global user.name "{bot_username}"
498diff --git a/jenkaas_jobs/yaml/credentials/credentials-1-vcs.yaml b/jenkaas_jobs/yaml/credentials/credentials-1-vcs.yaml
499new file mode 100644
500index 0000000..c9349d6
501--- /dev/null
502+++ b/jenkaas_jobs/yaml/credentials/credentials-1-vcs.yaml
503@@ -0,0 +1,37 @@
504+- builder:
505+ name: git_credentials
506+ dummy: "test"
507+ builders:
508+ - shell:
509+ !include-raw: credentials-1-git.sh
510+
511+- builder:
512+ name: bazaar_credentials
513+ dummy: "test"
514+ builders:
515+ - shell:
516+ !include-raw: credentials-1-bazaar.sh
517+
518+- job-template:
519+ name: credentials-1-{repo_type}
520+ project-type: matrix
521+ description: This job configures {repo_type} for Launchpad and branch access.
522+ node: monitor
523+ properties:
524+ - build-discarder:
525+ num-to-keep: 1
526+ - rebuild
527+ parameters:
528+ - matrix-combinations:
529+ name: nodes
530+ axes:
531+ - axis:
532+ type: slave
533+ name: node
534+ values: '{obj:build_slaves}'
535+ wrappers:
536+ - timestamps
537+ builders:
538+ - '{repo_type}_credentials':
539+ bot_username: '{bot_username}'
540+ bot_email: '{bot_email}'
541diff --git a/jenkaas_jobs/yaml/credentials/credentials-3-launchpad-plugin.sh b/jenkaas_jobs/yaml/credentials/credentials-3-launchpad-plugin.sh
542index 5a5cd23..cad6333 100644
543--- a/jenkaas_jobs/yaml/credentials/credentials-3-launchpad-plugin.sh
544+++ b/jenkaas_jobs/yaml/credentials/credentials-3-launchpad-plugin.sh
545@@ -81,6 +81,10 @@ lp_app: launchpad-trigger
546 #you don't need to change this
547 lp_env: production
548
549+#which launchpad api version to use
550+#devel version includes additional querying support for git repos
551+lp_version: {lp_version}
552+
553 #URL of your public jenkins in case you are publishing your jobs to some
554 #other jenkins
555 public_jenkins_url:
556@@ -100,4 +104,4 @@ urls_to_hide: []
557 # verbosity of the commands
558 # one of: debug, info, warning, error, critical
559 log_level: debug
560-EOF
561\ No newline at end of file
562+EOF
563diff --git a/jenkaas_jobs/yaml/credentials/credentials.yaml b/jenkaas_jobs/yaml/credentials/credentials.yaml
564index d22bd25..292c2b3 100644
565--- a/jenkaas_jobs/yaml/credentials/credentials.yaml
566+++ b/jenkaas_jobs/yaml/credentials/credentials.yaml
567@@ -2,6 +2,6 @@
568 name: credentials-jobs
569 jobs:
570 - credentials-0-ssh
571- - credentials-1-bazaar
572+ - credentials-1-{repo_type}
573 - credentials-2-launchpad
574 - credentials-3-launchpad-plugin
575diff --git a/jenkaas_jobs/yaml/defaults.yaml b/jenkaas_jobs/yaml/defaults.yaml
576index 805878b..8150f9f 100644
577--- a/jenkaas_jobs/yaml/defaults.yaml
578+++ b/jenkaas_jobs/yaml/defaults.yaml
579@@ -1,5 +1,13 @@
580 - defaults:
581 name: global
582+
583+ # Type of repositry the project uses
584+ repo_type: bazaar
585+
586+ # Launchpad API version the jenkins-launchpad-plugin will use. Set
587+ # to 'devel' if using a git repository; leave empty to use the default API version
588+ lp_version:
589+
590 # Path to launchpad credentials file
591 credentials_path: /var/lib/jenkins/.launchpad.credentials
592
593diff --git a/jenkaas_jobs/yaml/launchpad/lp-ci.yaml b/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
594index 3119c3b..585e856 100644
595--- a/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
596+++ b/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
597@@ -10,6 +10,25 @@
598 days-to-keep: 60
599 - rebuild
600 parameters:
601+ - '{repo_type}_ci_parameters'
602+ builders:
603+ - '{repo_type}_trigger_build':
604+ branch: '{branch}'
605+ - matrix-trigger:
606+ job: test-0-autopkgtest
607+ parameters: |
608+ build_run=build#${{TRIGGERED_BUILD_NUMBER_build}}
609+ testnames={autopkgtest_names}
610+ filter: '{autopkgtest_filter}'
611+ wrappers:
612+ - timestamps
613+ - '{repo_type}_ci_build_name_wrapper'
614+ publishers:
615+ - '{repo_type}_publisher'
616+
617+- parameter:
618+ name: bazaar_ci_parameters
619+ parameters:
620 - string:
621 name: landing_candidate
622 description: Launchpad branch to process
623@@ -19,6 +38,25 @@
624 - string:
625 name: merge_proposal
626 description: Optional merge proposal to process
627+
628+- parameter:
629+ name: git_ci_parameters
630+ parameters:
631+ - string:
632+ name: landing_candidate
633+ description: Launchpad git repo to process
634+ - string:
635+ name: landing_candidate_branch
636+ description: Git branch from landing_candidate to process
637+ - string:
638+ name: candidate_revision
639+ description: Optional branch revision to process
640+ - string:
641+ name: merge_proposal
642+ description: Optional merge proposal to process
643+
644+- builder:
645+ name: bazaar_trigger_build
646 builders:
647 - trigger-builds:
648 - project:
649@@ -28,33 +66,76 @@
650 source_branch=${{landing_candidate}}
651 source_revision=${{candidate_revision}}
652 target_branch={branch}
653- - matrix-trigger:
654- job: test-0-autopkgtest
655- parameters: |
656- build_run=build#${{TRIGGERED_BUILD_NUMBER_build}}
657- testnames={autopkgtest_names}
658- filter: '{autopkgtest_filter}'
659+
660+- builder:
661+ name: git_trigger_build
662+ builders:
663+ - trigger-builds:
664+ - project:
665+ - build
666+ block: true
667+ predefined-parameters: |
668+ source_repo=${{landing_candidate}}
669+ source_branch=${{landing_candidate_branch}}
670+ source_revision=${{candidate_revision}}
671+ target_branch={branch}
672+
673+- wrapper:
674+ name: git_ci_build_name_wrapper
675+ wrappers:
676+ - build-name:
677+ name: ${ENV,var="landing_candidate"}:${ENV,var="landing_candidate_branch"}
678+
679+- wrapper:
680+ name: bazaar_ci_build_name_wrapper
681 wrappers:
682- - timestamps
683 - build-name:
684- name: ${{ENV,var="landing_candidate"}}
685+ name: ${ENV,var="landing_candidate"}
686+
687+- publisher:
688+ name: bazaar_publisher
689+ publishers:
690+ - trigger-parameterized-builds:
691+ - project:
692+ - lp-generic-update-mp
693+ predefined-parameters: |
694+ ci_result=PASSED
695+ ci_build=${BUILD_URL}
696+ ci_branch=${landing_candidate}
697+ ci_merge_proposal=${merge_proposal}
698+ ci_revision=${candidate_revision}
699+ condition: SUCCESS
700+ - project:
701+ - lp-generic-update-mp
702+ predefined-parameters: |
703+ ci_result=FAILED
704+ ci_build=${BUILD_URL}
705+ ci_branch=${landing_candidate}
706+ ci_merge_proposal=${merge_proposal}
707+ ci_revision=${candidate_revision}
708+ condition: UNSTABLE_OR_WORSE
709+
710+- publisher:
711+ name: git_publisher
712 publishers:
713 - trigger-parameterized-builds:
714 - project:
715 - lp-generic-update-mp
716 predefined-parameters: |
717 ci_result=PASSED
718- ci_build=${{BUILD_URL}}
719- ci_branch=${{landing_candidate}}
720- ci_merge_proposal=${{merge_proposal}}
721- ci_revision=${{candidate_revision}}
722+ ci_build=${BUILD_URL}
723+ ci_repo=${landing_candidate}
724+ ci_branch=${landing_candidate_branch}
725+ ci_merge_proposal=${merge_proposal}
726+ ci_revision=${candidate_revision}
727 condition: SUCCESS
728 - project:
729 - lp-generic-update-mp
730 predefined-parameters: |
731 ci_result=FAILED
732- ci_build=${{BUILD_URL}}
733- ci_branch=${{landing_candidate}}
734- ci_merge_proposal=${{merge_proposal}}
735- ci_revision=${{candidate_revision}}
736+ ci_build=${BUILD_URL}
737+ ci_repo=${landing_candidate}
738+ ci_branch=${landing_candidate_branch}
739+ ci_merge_proposal=${merge_proposal}
740+ ci_revision=${candidate_revision}
741 condition: UNSTABLE_OR_WORSE
742diff --git a/jenkaas_jobs/yaml/launchpad/lp-generic-land-mp.yaml b/jenkaas_jobs/yaml/launchpad/lp-generic-land-mp.yaml
743index 3aa557e..3b608d5 100644
744--- a/jenkaas_jobs/yaml/launchpad/lp-generic-land-mp.yaml
745+++ b/jenkaas_jobs/yaml/launchpad/lp-generic-land-mp.yaml
746@@ -15,10 +15,10 @@
747 description: Launchpad branch that was processed
748 - string:
749 name: ci_merge_proposal
750- description: Launchpad merge proposal that was processed
751+ description: Launchpad merge proposal that was processed
752 - string:
753 name: ci_revision
754- description: Revision of the processed branch
755+ description: Revision of the processed branch
756 properties:
757 - build-discarder:
758 num-to-keep: 100
759diff --git a/jenkaas_jobs/yaml/launchpad/lp-trigger-autolanding.sh b/jenkaas_jobs/yaml/launchpad/lp-trigger-autolanding.sh
760index 488a91b..359d9e2 100644
761--- a/jenkaas_jobs/yaml/launchpad/lp-trigger-autolanding.sh
762+++ b/jenkaas_jobs/yaml/launchpad/lp-trigger-autolanding.sh
763@@ -3,4 +3,4 @@
764 # log all commands and abort on error
765 set -ex
766
767-launchpadTrigger --lock-name=${{JOB_NAME}} --job=lp-{project}-autoland --branch={branch} --autoland || true
768+launchpadTrigger --lock-name=${{JOB_NAME}} --job=lp-{project}-autoland --branch={branch} --repo-type={repo_type} --autoland || true
769diff --git a/jenkaas_jobs/yaml/launchpad/lp-trigger.sh b/jenkaas_jobs/yaml/launchpad/lp-trigger.sh
770index 15c7a10..de9dcd5 100644
771--- a/jenkaas_jobs/yaml/launchpad/lp-trigger.sh
772+++ b/jenkaas_jobs/yaml/launchpad/lp-trigger.sh
773@@ -4,4 +4,4 @@
774 set -ex
775
776 # ignore errors from launchpadTrigger as it might just be "unauthorized"
777-launchpadTrigger --lock-name=${{JOB_NAME}} --job=lp-{project}-ci --branch={branch} --trigger-ci || true
778\ No newline at end of file
779+launchpadTrigger --lock-name=${{JOB_NAME}} --job=lp-{project}-ci --branch={branch} --repo-type={repo_type} --trigger-ci || true
780diff --git a/jenkaas_jobs/yaml/prepare/prepare-0-install.sh b/jenkaas_jobs/yaml/prepare/prepare-0-install.sh
781index 24a2294..397f5bc 100644
782--- a/jenkaas_jobs/yaml/prepare/prepare-0-install.sh
783+++ b/jenkaas_jobs/yaml/prepare/prepare-0-install.sh
784@@ -21,6 +21,7 @@ sudo apt update
785
786 # autopkgtest to run autopkgtests
787 # bzr-builddeb for building source packages
788+# git-buildpackage for building source packages from git repos
789 # ccache to prepare the shared ccache dir
790 # debhelper for debian/rules clean
791 # debootstrap to create the chroots
792@@ -33,6 +34,7 @@ sudo apt update
793 sudo apt-get install --yes \
794 autopkgtest/trusty-backports \
795 bzr-builddeb \
796+ git-buildpackage \
797 ccache \
798 debhelper \
799 debootstrap \
800diff --git a/project-example.yaml b/project-example.yaml
801index dbd7205..62001ce 100644
802--- a/project-example.yaml
803+++ b/project-example.yaml
804@@ -16,6 +16,13 @@
805 ########## Global variables ##########
806 # The variables below apply globally to the whole jenkins setup.
807
808+ # The type of repository used - git or bzr. Defaults to bazaar (see defaults.yaml)
809+ #repo_type: git
810+
811+ # Launchpad API version the jenkins-launchpad-plugin will use. Set
812+ # to 'devel' if using a git repository, or use the default (see defaults.yaml)
813+ #lp_version:
814+
815 # Your internal jenkins address.
816 backend_url: http://your-project-jenkins-be.internal:8080/
817
818@@ -167,4 +174,4 @@
819 <<: *even_another_project
820 autopkgtest_names: test-name.sh autopilot.sh integration.sh
821 jobs:
822- - lp-{project}-autolanding
823\ No newline at end of file
824+ - lp-{project}-autolanding

Subscribers

People subscribed via source and target branches