Merge ~saviq/jenkaas-jobs:use-upstream-matrix-trigger into jenkaas-jobs:master

Proposed by Michał Sawicz
Status: Merged
Approved by: Michał Sawicz
Approved revision: 34465877ebeeda5d8c75caa5238139177f2aa7ff
Merged at revision: 34465877ebeeda5d8c75caa5238139177f2aa7ff
Proposed branch: ~saviq/jenkaas-jobs:use-upstream-matrix-trigger
Merge into: jenkaas-jobs:master
Diff against target: 420 lines (+64/-103)
20 files modified
README.rst (+24/-2)
dev/null (+0/-39)
jenkaas_jobs/__init__.py (+1/-0)
jenkaas_jobs/yaml/launchpad/lp-autoland.yaml (+9/-8)
jenkaas_jobs/yaml/launchpad/lp-ci.yaml (+9/-8)
jenkaas_jobs/yaml/macros.yaml (+5/-13)
project-example.yaml (+2/-8)
requirements.txt (+1/-0)
test-requirements.txt (+3/-1)
tests/fixtures/launchpad/lp-four-autoland.xml (+1/-1)
tests/fixtures/launchpad/lp-four-ci.xml (+1/-1)
tests/fixtures/launchpad/lp-one-ci.xml (+1/-1)
tests/fixtures/launchpad/lp-three-autoland.xml (+1/-1)
tests/fixtures/launchpad/lp-three-ci.xml (+1/-1)
tests/fixtures/launchpad/lp-two-autoland.xml (+1/-1)
tests/fixtures/launchpad/lp-two-ci.xml (+1/-1)
tests/fixtures/maintain/maintain.xml (+0/-6)
tests/fixtures/prepare/prepare.xml (+0/-9)
tests/fixtures/test/test-0-autopkgtest.xml (+1/-0)
tests/project.yaml (+2/-2)
Reviewer Review Type Date Requested Status
Olivier Tilloy (community) Approve
Review via email: mp+305610@code.launchpad.net

Commit message

Drop raw matrix-trigger

Cherry-picked jenkins-job-builder has native support for it.

NOTE: You can now drop the repeated, escaped autopkgtest_filter, see
changes in project-example.yaml

Description of the change

FIXME--! Thanks Olivier for nudging me in this direction ;)

To post a comment you must log in.
Revision history for this message
Olivier Tilloy (osomon) wrote :

LGTM, thanks for that!
I’ve deployed to the system-apps jenkaas instance and everything seems to work fine.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/README.rst b/README.rst
2index 8c0e9a1..2a5a9ea 100644
3--- a/README.rst
4+++ b/README.rst
5@@ -53,18 +53,40 @@ Documentation <http://docs.openstack.org/infra/jenkins-job-builder/>`_. The
6 example-project.yaml file has comments that should help you set things for your
7 project just-so.
8
9+Using jenkaas-jobs
10+------------------
11+You'll need to use a virtualenv for all the dependencies to be set up in a
12+predictable way:
13+
14+.. code-block::
15+
16+ $ virtualenv -p python3 .venv
17+ $ source .venv/bin/activate
18+ (.venv)$ pip install -r requirements.txt
19+
20+To get out of the virtualenv:
21+
22+.. code-block::
23+ (.venv)$ deactivate
24+
25+To run jenkins-jobs:
26+
27+.. code-block::
28+ $ source .venv/bin/activate
29+ (.venv)$ jenkins-jobs
30+
31 Testing your job definitions
32 ----------------------------
33 .. code-block::
34
35- $ jenkins-job-builder --conf your-project.conf test \
36+ (.venv)$ jenkins-jobs --conf your-project.conf test \
37 your-project.yaml:jenkaas-jobs/jenkaas_jobs/yaml --output test
38
39 Creating/Updating your job definitions
40 --------------------------------------
41 .. code-block::
42
43- $ jenkins-job-builder --conf your-project.conf update \
44+ (.venv)$ jenkins-jobs --conf your-project.conf update \
45 your-project.yaml:jenkaas-jobs/jenkaas_jobs/yaml
46
47 Extending
48diff --git a/jenkaas_jobs/__init__.py b/jenkaas_jobs/__init__.py
49index 81ad8c9..5ca2982 100644
50--- a/jenkaas_jobs/__init__.py
51+++ b/jenkaas_jobs/__init__.py
52@@ -44,6 +44,7 @@ jenkins_rules = {
53 "project",
54 "matrix-project",
55 "properties",
56+ "configs",
57 "hudson.model.StringParameterDefinition",
58 "hudson.model.RunParameterDefinition",
59 "hudson.tasks.ArtifactArchiver",
60diff --git a/jenkaas_jobs/yaml/launchpad/lp-autoland.yaml b/jenkaas_jobs/yaml/launchpad/lp-autoland.yaml
61index 34df322..0110145 100644
62--- a/jenkaas_jobs/yaml/launchpad/lp-autoland.yaml
63+++ b/jenkaas_jobs/yaml/launchpad/lp-autoland.yaml
64@@ -32,14 +32,15 @@
65 source_branch=${{landing_candidate}}
66 source_revision=${{candidate_revision}}
67 target_branch={branch}
68- - matrix-trigger:
69- job: test-0-autopkgtest
70- parameters: |
71- build_run=build#${{TRIGGERED_BUILD_NUMBER_build}}
72- testnames={autopkgtest_names}
73- filter: '{autopkgtest_filter}'
74- raw_parameters: |
75- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
76+ - trigger-builds:
77+ - project:
78+ - test-0-autopkgtest
79+ block: true
80+ current-parameters: true
81+ predefined-parameters: |
82+ build_run=build#${{TRIGGERED_BUILD_NUMBER_build}}
83+ testnames={autopkgtest_names}
84+ restrict-matrix-project: '{autopkgtest_filter}'
85 wrappers:
86 - timestamps
87 - build-name:
88diff --git a/jenkaas_jobs/yaml/launchpad/lp-ci.yaml b/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
89index 2e72df5..5a8f05d 100644
90--- a/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
91+++ b/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
92@@ -32,14 +32,15 @@
93 source_branch=${{landing_candidate}}
94 source_revision=${{candidate_revision}}
95 target_branch={branch}
96- - matrix-trigger:
97- job: test-0-autopkgtest
98- parameters: |
99- build_run=build#${{TRIGGERED_BUILD_NUMBER_build}}
100- testnames={autopkgtest_names}
101- filter: '{autopkgtest_filter}'
102- raw_parameters: |
103- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
104+ - trigger-builds:
105+ - project:
106+ - test-0-autopkgtest
107+ block: true
108+ current-parameters: true
109+ predefined-parameters: |
110+ build_run=build#${{TRIGGERED_BUILD_NUMBER_build}}
111+ testnames={autopkgtest_names}
112+ restrict-matrix-project: '{autopkgtest_filter}'
113 wrappers:
114 - timestamps
115 - build-name:
116diff --git a/jenkaas_jobs/yaml/macros.yaml b/jenkaas_jobs/yaml/macros.yaml
117index c9f8064..b6c724d 100644
118--- a/jenkaas_jobs/yaml/macros.yaml
119+++ b/jenkaas_jobs/yaml/macros.yaml
120@@ -9,22 +9,14 @@
121 <setForMatrix>{set_for_matrix}</setForMatrix>
122 </hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
123
124-- builder: # FIXME: JJB can't do matrix combinations parameter on the trigger
125- name: matrix-trigger
126- builders:
127- - raw:
128- raw_parameters: ''
129- xml:
130- !include-raw: matrix-trigger.xml
131-
132 - builder:
133 name: node-trigger
134 builders:
135- - matrix-trigger:
136- job: '{job}'
137- filter: node=="${{node}}"
138- parameters: ''
139- raw_parameters: ''
140+ - trigger-builds:
141+ - project:
142+ - '{job}'
143+ block: true
144+ restrict-matrix-project: node=="${{node}}"
145
146 - parameter:
147 name: extra_repositories_parameters
148diff --git a/jenkaas_jobs/yaml/matrix-trigger.xml b/jenkaas_jobs/yaml/matrix-trigger.xml
149deleted file mode 100644
150index 892c792..0000000
151--- a/jenkaas_jobs/yaml/matrix-trigger.xml
152+++ /dev/null
153@@ -1,39 +0,0 @@
154-<hudson.plugins.parameterizedtrigger.TriggerBuilder>
155- <configs>
156- <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
157- <configs>
158- <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
159- <properties>{parameters}</properties>
160- </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
161- <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
162- <filter>{filter}</filter>
163- </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
164- {raw_parameters}
165- </configs>
166- <projects>{job}</projects>
167- <condition>ALWAYS</condition>
168- <triggerWithNoParameters>false</triggerWithNoParameters>
169- <buildAllNodesWithLabel>false</buildAllNodesWithLabel>
170- <block>
171- <buildStepFailureThreshold>
172- <name>FAILURE</name>
173- <ordinal>2</ordinal>
174- <color>RED</color>
175- <completeBuild>true</completeBuild>
176- </buildStepFailureThreshold>
177- <unstableThreshold>
178- <name>UNSTABLE</name>
179- <ordinal>1</ordinal>
180- <color>YELLOW</color>
181- <completeBuild>true</completeBuild>
182- </unstableThreshold>
183- <failureThreshold>
184- <name>FAILURE</name>
185- <ordinal>2</ordinal>
186- <color>RED</color>
187- <completeBuild>true</completeBuild>
188- </failureThreshold>
189- </block>
190- </hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
191- </configs>
192-</hudson.plugins.parameterizedtrigger.TriggerBuilder>
193diff --git a/project-example.yaml b/project-example.yaml
194index 443b61f..21cda2f 100644
195--- a/project-example.yaml
196+++ b/project-example.yaml
197@@ -96,11 +96,8 @@
198 # Which autopktests to run on which architectures.
199 #
200 # Available axes: label, release and testname
201- #
202- # FIXME: this has to use XML entities for & due to a Jenkins Job Builder
203- # limitation.
204 autopkgtest_filter: &autopkgtest_filter
205- (label=="amd64" &amp;&amp; testname=="test-name.sh") || (label=="phone-armhf" &amp;&amp; testname=="autopilot.sh" &amp;&amp; release=="vivid+overlay")
206+ (label=="amd64" && testname=="test-name.sh") || (label=="phone-armhf" && testname=="autopilot.sh" && release=="vivid+overlay")
207
208 # A list of Launchpad projects to configure for CI, each of which can have
209 # overrides for the per-project variables above as key-value pairs.
210@@ -135,10 +132,7 @@
211 - build-jobs
212
213 # Run tests on the built packages.
214- - test-jobs:
215- # FIXME: this is the same filter as above, but without entities.
216- autopkgtest_filter:
217- (label=="amd64" && testname=="test-name.sh") || (label=="phone-armhf" && testname=="autopilot.sh" && release=="vivid+overlay")
218+ - test-jobs
219
220 # Generic jobs for interacting with launchpad.
221 - launchpad-jobs
222diff --git a/requirements.txt b/requirements.txt
223index e7c9015..5317e3a 100644
224--- a/requirements.txt
225+++ b/requirements.txt
226@@ -1 +1,2 @@
227 pbr>=1.0.0,<2.0
228+git+https://git.launchpad.net/~jenkaas-hackers/+git/jenkins-job-builder@5822a2fb
229diff --git a/test-requirements.txt b/test-requirements.txt
230index ff5d80a..9bf7549 100644
231--- a/test-requirements.txt
232+++ b/test-requirements.txt
233@@ -1 +1,3 @@
234-jenkins-job-builder>=1.4
235+nose
236+nose-ignore-docstring
237+ddt
238diff --git a/tests/fixtures/launchpad/lp-four-autoland.xml b/tests/fixtures/launchpad/lp-four-autoland.xml
239index 8413199..1c1e385 100644
240--- a/tests/fixtures/launchpad/lp-four-autoland.xml
241+++ b/tests/fixtures/launchpad/lp-four-autoland.xml
242@@ -103,7 +103,7 @@ target_branch=lp:four
243 testnames=</properties>
244 </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
245 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
246- <filter/>
247+ <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
248 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
249 <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
250 </configs>
251diff --git a/tests/fixtures/launchpad/lp-four-ci.xml b/tests/fixtures/launchpad/lp-four-ci.xml
252index 3953b46..18a9c78 100644
253--- a/tests/fixtures/launchpad/lp-four-ci.xml
254+++ b/tests/fixtures/launchpad/lp-four-ci.xml
255@@ -104,7 +104,7 @@ target_branch=lp:four</properties>
256 testnames=</properties>
257 </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
258 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
259- <filter></filter>
260+ <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
261 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
262 <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
263 </configs>
264diff --git a/tests/fixtures/launchpad/lp-one-ci.xml b/tests/fixtures/launchpad/lp-one-ci.xml
265index 2d940c3..9c71787 100644
266--- a/tests/fixtures/launchpad/lp-one-ci.xml
267+++ b/tests/fixtures/launchpad/lp-one-ci.xml
268@@ -103,7 +103,7 @@ target_branch=lp:one</properties>
269 testnames=</properties>
270 </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
271 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
272- <filter></filter>
273+ <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
274 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
275 <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
276 </configs>
277diff --git a/tests/fixtures/launchpad/lp-three-autoland.xml b/tests/fixtures/launchpad/lp-three-autoland.xml
278index c1b70f3..88f4934 100644
279--- a/tests/fixtures/launchpad/lp-three-autoland.xml
280+++ b/tests/fixtures/launchpad/lp-three-autoland.xml
281@@ -103,7 +103,7 @@ testnames=test-three
282 </properties>
283 </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
284 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
285- <filter/>
286+ <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
287 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
288 <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
289 </configs>
290diff --git a/tests/fixtures/launchpad/lp-three-ci.xml b/tests/fixtures/launchpad/lp-three-ci.xml
291index 7de1709..4c0bb36 100644
292--- a/tests/fixtures/launchpad/lp-three-ci.xml
293+++ b/tests/fixtures/launchpad/lp-three-ci.xml
294@@ -103,7 +103,7 @@ target_branch=lp:three/devel</properties>
295 testnames=</properties>
296 </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
297 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
298- <filter></filter>
299+ <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
300 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
301 <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
302 </configs>
303diff --git a/tests/fixtures/launchpad/lp-two-autoland.xml b/tests/fixtures/launchpad/lp-two-autoland.xml
304index e563d1f..7e7b52d 100644
305--- a/tests/fixtures/launchpad/lp-two-autoland.xml
306+++ b/tests/fixtures/launchpad/lp-two-autoland.xml
307@@ -103,7 +103,7 @@ testnames=test-one test-two
308 </properties>
309 </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
310 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
311- <filter>(label=="armhf").implies(testname=="test-one")</filter>
312+ <filter>(label==&quot;armhf&quot;) &amp;&amp; (release==&quot;stable&quot;)</filter>
313 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
314 <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
315 </configs>
316diff --git a/tests/fixtures/launchpad/lp-two-ci.xml b/tests/fixtures/launchpad/lp-two-ci.xml
317index f79c7f2..d1b8a46 100644
318--- a/tests/fixtures/launchpad/lp-two-ci.xml
319+++ b/tests/fixtures/launchpad/lp-two-ci.xml
320@@ -103,7 +103,7 @@ target_branch=lp:two</properties>
321 testnames=test-one test-two</properties>
322 </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
323 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
324- <filter>(label==&quot;armhf&quot;).implies(testname==&quot;test-one&quot;)</filter>
325+ <filter>(label==&quot;armhf&quot;) &amp;&amp; (release==&quot;stable&quot;)</filter>
326 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
327 <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
328 </configs>
329diff --git a/tests/fixtures/maintain/maintain.xml b/tests/fixtures/maintain/maintain.xml
330index 861a1eb..7fdcb33 100644
331--- a/tests/fixtures/maintain/maintain.xml
332+++ b/tests/fixtures/maintain/maintain.xml
333@@ -55,9 +55,6 @@
334 <configs>
335 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
336 <configs>
337- <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
338- <properties></properties>
339- </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
340 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
341 <filter>node==&quot;${node}&quot;</filter>
342 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
343@@ -93,9 +90,6 @@
344 <configs>
345 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
346 <configs>
347- <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
348- <properties></properties>
349- </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
350 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
351 <filter>node==&quot;${node}&quot;</filter>
352 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
353diff --git a/tests/fixtures/prepare/prepare.xml b/tests/fixtures/prepare/prepare.xml
354index 95677eb..0a53c3d 100644
355--- a/tests/fixtures/prepare/prepare.xml
356+++ b/tests/fixtures/prepare/prepare.xml
357@@ -51,9 +51,6 @@
358 <configs>
359 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
360 <configs>
361- <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
362- <properties></properties>
363- </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
364 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
365 <filter>node==&quot;${node}&quot;</filter>
366 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
367@@ -89,9 +86,6 @@
368 <configs>
369 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
370 <configs>
371- <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
372- <properties></properties>
373- </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
374 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
375 <filter>node==&quot;${node}&quot;</filter>
376 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
377@@ -127,9 +121,6 @@
378 <configs>
379 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
380 <configs>
381- <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
382- <properties></properties>
383- </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
384 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
385 <filter>node==&quot;${node}&quot;</filter>
386 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
387diff --git a/tests/fixtures/test/test-0-autopkgtest.xml b/tests/fixtures/test/test-0-autopkgtest.xml
388index 4e4d993..8391283 100644
389--- a/tests/fixtures/test/test-0-autopkgtest.xml
390+++ b/tests/fixtures/test/test-0-autopkgtest.xml
391@@ -27,6 +27,7 @@
392 <hudson.plugins.matrix__configuration__parameter.MatrixCombinationsParameterDefinition plugin="matrix-combinations-parameter@1.0.9">
393 <name>configurations</name>
394 <description/>
395+ <defaultCombinationFilter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</defaultCombinationFilter>
396 </hudson.plugins.matrix__configuration__parameter.MatrixCombinationsParameterDefinition>
397 <hudson.model.StringParameterDefinition>
398 <name>testnames</name>
399diff --git a/tests/project.yaml b/tests/project.yaml
400index ffc40c8..df730b4 100644
401--- a/tests/project.yaml
402+++ b/tests/project.yaml
403@@ -19,7 +19,7 @@
404 chroot_filter:
405 (node=="jenkins-slave-2").implies(arch=="amd64" || arch=="i386") && (node=="cyclops-node02").implies(arch=="armhf")
406 autopkgtest_filter: &autopkgtest_filter
407- ''
408+ (label=="amd64") && (release=="devel")
409 extra_repositories: |
410 deb http://ppa.launchpad.net/test-ci-team/appa RELEASE main
411 extra_repository_keys: AA22CC44
412@@ -27,7 +27,7 @@
413 - one
414 - two: &two
415 autopkgtest_names: test-one test-two
416- autopkgtest_filter: (label==&quot;armhf&quot;).implies(testname==&quot;test-one&quot;)
417+ autopkgtest_filter: (label=="armhf") && (release=="stable")
418 - three: &three
419 branch: lp:three/devel
420 - four: &four

Subscribers

People subscribed via source and target branches