Jenkaas Jobs

Merge ~saviq/jenkaas-jobs:parse-mp-description into jenkaas-jobs:master

  1. Git
  2. lp:~saviq/jenkaas-jobs
  3. parse-mp-description
  4. Merge into master
Proposed by Michał Sawicz
Status: Merged
Approved by: Olivier Tilloy
Approved revision: dcaf147f7dc219abeb4f86e02bd74dc6433e632a
Merged at revision: dcaf147f7dc219abeb4f86e02bd74dc6433e632a
Proposed branch: ~saviq/jenkaas-jobs:parse-mp-description
Merge into: jenkaas-jobs:master
Diff against target: 831 lines (+478/-27)
10 files modified
jenkaas_jobs/yaml/launchpad/lp-autoland.yaml (+5/-3)
jenkaas_jobs/yaml/launchpad/lp-ci.yaml (+5/-3)
jenkaas_jobs/yaml/launchpad/lp_params.py (+48/-0)
tests/fixtures/launchpad/lp-four-autoland.xml (+60/-3)
tests/fixtures/launchpad/lp-four-ci.xml (+60/-3)
tests/fixtures/launchpad/lp-one-ci.xml (+60/-3)
tests/fixtures/launchpad/lp-three-autoland.xml (+60/-3)
tests/fixtures/launchpad/lp-three-ci.xml (+60/-3)
tests/fixtures/launchpad/lp-two-autoland.xml (+60/-3)
tests/fixtures/launchpad/lp-two-ci.xml (+60/-3)
Reviewer Review Type Date Requested Status
Olivier Tilloy (community) Approve
Review via email: mp+306224@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Olivier Tilloy (osomon) wrote :

First of all, I’m *very* sorry for taking that long to review your merge request.

I’ve added one minor comment/question inline. Otherwise this looks good, and even though I haven’t tested it IRL myself, I know it’s been running smoothly on the unity8 instance, so that’s proof enough.

It’s a shame we don’t have a way to test with mock merge requests.

Revision history for this message
Olivier Tilloy (osomon) wrote :

LGTM.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/jenkaas_jobs/yaml/launchpad/lp-autoland.yaml b/jenkaas_jobs/yaml/launchpad/lp-autoland.yaml
2index 0110145..a97523b 100644
3--- a/jenkaas_jobs/yaml/launchpad/lp-autoland.yaml
4+++ b/jenkaas_jobs/yaml/launchpad/lp-autoland.yaml
5@@ -2,7 +2,7 @@
6 name: lp-{project}-autoland
7 project-type: freestyle
8 description: 'Auto-land top-approved merge proposal in {branch}'
9- node: monitor
10+ node: monitor && launchpad
11 concurrent: true
12 auth-token: BUILD_ME
13 properties:
14@@ -23,24 +23,26 @@
15 extra_repositories: '{extra_repositories}'
16 extra_repository_keys: '{extra_repository_keys}'
17 builders:
18+ - shell:
19+ !include-raw-escape: lp_params.py
20 - trigger-builds:
21 - project:
22 - build
23 block: true
24- current-parameters: true
25 predefined-parameters: |
26 source_branch=${{landing_candidate}}
27 source_revision=${{candidate_revision}}
28 target_branch={branch}
29+ property-file: lp.properties
30 - trigger-builds:
31 - project:
32 - test-0-autopkgtest
33 block: true
34- current-parameters: true
35 predefined-parameters: |
36 build_run=build#${{TRIGGERED_BUILD_NUMBER_build}}
37 testnames={autopkgtest_names}
38 restrict-matrix-project: '{autopkgtest_filter}'
39+ property-file: lp.properties
40 wrappers:
41 - timestamps
42 - build-name:
43diff --git a/jenkaas_jobs/yaml/launchpad/lp-ci.yaml b/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
44index 5a8f05d..b99872e 100644
45--- a/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
46+++ b/jenkaas_jobs/yaml/launchpad/lp-ci.yaml
47@@ -2,7 +2,7 @@
48 name: lp-{project}-ci
49 project-type: freestyle
50 description: 'Run continuous integration steps for {branch}'
51- node: monitor
52+ node: monitor && launchpad
53 concurrent: true
54 auth-token: BUILD_ME
55 properties:
56@@ -23,24 +23,26 @@
57 extra_repositories: '{extra_repositories}'
58 extra_repository_keys: '{extra_repository_keys}'
59 builders:
60+ - shell:
61+ !include-raw-escape: lp_params.py
62 - trigger-builds:
63 - project:
64 - build
65 block: true
66- current-parameters: true
67 predefined-parameters: |
68 source_branch=${{landing_candidate}}
69 source_revision=${{candidate_revision}}
70 target_branch={branch}
71+ property-file: lp.properties
72 - trigger-builds:
73 - project:
74 - test-0-autopkgtest
75 block: true
76- current-parameters: true
77 predefined-parameters: |
78 build_run=build#${{TRIGGERED_BUILD_NUMBER_build}}
79 testnames={autopkgtest_names}
80 restrict-matrix-project: '{autopkgtest_filter}'
81+ property-file: lp.properties
82 wrappers:
83 - timestamps
84 - build-name:
85diff --git a/jenkaas_jobs/yaml/launchpad/lp_params.py b/jenkaas_jobs/yaml/launchpad/lp_params.py
86new file mode 100644
87index 0000000..335e30e
88--- /dev/null
89+++ b/jenkaas_jobs/yaml/launchpad/lp_params.py
90@@ -0,0 +1,48 @@
91+#!/usr/bin/env python2
92+
93+import atexit
94+import os
95+import re
96+from shutil import rmtree
97+import sys
98+from tempfile import mkdtemp
99+
100+from jlp import get_launchpad, launchpadutils
101+
102+# match lines starting with Prereq-archive: or Prereq-archive-keys:, followed by
103+# zero or more whitespace, then archive definition and any trailing whitespace
104+archive_re = re.compile(r"^Prereq-archive:\s*(.*?)\s*$", re.MULTILINE)
105+keys_re = re.compile(r"^Prereq-archive-keys:\s*(.*?)\s*$", re.MULTILINE)
106+
107+# retrieve values for parameters passed in the build
108+mp_url = os.getenv("merge_proposal", None)
109+archives = set(os.getenv("extra_repositories", "").splitlines())
110+archive_keys = set(os.getenv("extra_repository_keys", "").split())
111+
112+# if mp_url is not empty, try and get the MP description and parse Prereq-*
113+# declarations
114+if mp_url:
115+ # launchpadlib is not thread/process safe so we are creating launchpadlib
116+ # cache in /tmp per process which gets cleaned up at the end
117+ # see also lp:459418 and lp:1025153
118+ launchpad_cachedir = mkdtemp()
119+
120+ # make sure to drop the cache dir on exit
121+ atexit.register(rmtree, launchpad_cachedir, ignore_errors=True)
122+
123+ lp_handle = get_launchpad(launchpadlib_dir=launchpad_cachedir)
124+ mp = launchpadutils.get_mp_handle_from_url(lp_handle, mp_url)
125+ if not mp:
126+ sys.stderr.write("ERROR: Merge proposal '{}' not found".format(mp_url))
127+ sys.exit(1)
128+
129+ if mp.description:
130+ # add all Prereq-archive and Prereq-archive-keys to extra archives and keys
131+ archives.update(archive_re.findall(mp.description))
132+ archive_keys.update(keys_re.findall(mp.description))
133+
134+with open("lp.properties", "w") as params:
135+ # Java property file allows multi-line props with backslash at the end,
136+ # the \n is needed to actually have newlines in the resulting value
137+ params.write("extra_repositories={}\n".format("\\n\\\n".join(archives)))
138+ params.write("extra_repository_keys={}\n".format(" ".join(archive_keys)))
139diff --git a/tests/fixtures/launchpad/lp-four-autoland.xml b/tests/fixtures/launchpad/lp-four-autoland.xml
140index 1c1e385..738dfd9 100644
141--- a/tests/fixtures/launchpad/lp-four-autoland.xml
142+++ b/tests/fixtures/launchpad/lp-four-autoland.xml
143@@ -7,7 +7,7 @@
144 <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
145 <authToken>BUILD_ME</authToken>
146 <concurrentBuild>true</concurrentBuild>
147- <assignedNode>monitor</assignedNode>
148+ <assignedNode>monitor &amp;&amp; launchpad</assignedNode>
149 <canRoam>false</canRoam>
150 <properties>
151 <jenkins.model.BuildDiscarderProperty>
152@@ -55,11 +55,65 @@ deb https://user:pwd@private-ppa.launchpad.net/test-ci-team/fourprivate/ubuntu R
153 </properties>
154 <scm class="hudson.scm.NullSCM"/>
155 <builders>
156+ <hudson.tasks.Shell>
157+ <command>#!/usr/bin/env python2
158+
159+import atexit
160+import os
161+import re
162+from shutil import rmtree
163+import sys
164+from tempfile import mkdtemp
165+
166+from jlp import get_launchpad, launchpadutils
167+
168+# match lines starting with Prereq-archive: or Prereq-archive-keys:, followed by
169+# zero or more whitespace, then archive definition and any trailing whitespace
170+archive_re = re.compile(r&quot;^Prereq-archive:\s*(.*?)\s*$&quot;, re.MULTILINE)
171+keys_re = re.compile(r&quot;^Prereq-archive-keys:\s*(.*?)\s*$&quot;, re.MULTILINE)
172+
173+# retrieve values for parameters passed in the build
174+mp_url = os.getenv(&quot;merge_proposal&quot;, None)
175+archives = set(os.getenv(&quot;extra_repositories&quot;, &quot;&quot;).splitlines())
176+archive_keys = set(os.getenv(&quot;extra_repository_keys&quot;, &quot;&quot;).split())
177+
178+# if mp_url is not empty, try and get the MP description and parse Prereq-*
179+# declarations
180+if mp_url:
181+ # launchpadlib is not thread/process safe so we are creating launchpadlib
182+ # cache in /tmp per process which gets cleaned up at the end
183+ # see also lp:459418 and lp:1025153
184+ launchpad_cachedir = mkdtemp()
185+
186+ # make sure to drop the cache dir on exit
187+ atexit.register(rmtree, launchpad_cachedir, ignore_errors=True)
188+
189+ lp_handle = get_launchpad(launchpadlib_dir=launchpad_cachedir)
190+ mp = launchpadutils.get_mp_handle_from_url(lp_handle, mp_url)
191+ if not mp:
192+ sys.stderr.write(&quot;ERROR: Merge proposal '{}' not found&quot;.format(mp_url))
193+ sys.exit(1)
194+
195+ if mp.description:
196+ # add all Prereq-archive and Prereq-archive-keys to extra archives and keys
197+ archives.update(archive_re.findall(mp.description))
198+ archive_keys.update(keys_re.findall(mp.description))
199+
200+with open(&quot;lp.properties&quot;, &quot;w&quot;) as params:
201+ # Java property file allows multi-line props with backslash at the end,
202+ # the \n is needed to actually have newlines in the resulting value
203+ params.write(&quot;extra_repositories={}\n&quot;.format(&quot;\\n\\\n&quot;.join(archives)))
204+ params.write(&quot;extra_repository_keys={}\n&quot;.format(&quot; &quot;.join(archive_keys)))
205+</command>
206+ </hudson.tasks.Shell>
207 <hudson.plugins.parameterizedtrigger.TriggerBuilder>
208 <configs>
209 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
210 <configs>
211- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
212+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
213+ <propertiesFile>lp.properties</propertiesFile>
214+ <failTriggerOnMissing>true</failTriggerOnMissing>
215+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
216 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
217 <properties>source_branch=${landing_candidate}
218 source_revision=${candidate_revision}
219@@ -98,6 +152,10 @@ target_branch=lp:four
220 <configs>
221 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
222 <configs>
223+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
224+ <propertiesFile>lp.properties</propertiesFile>
225+ <failTriggerOnMissing>true</failTriggerOnMissing>
226+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
227 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
228 <properties>build_run=build#${TRIGGERED_BUILD_NUMBER_build}
229 testnames=</properties>
230@@ -105,7 +163,6 @@ testnames=</properties>
231 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
232 <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
233 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
234- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
235 </configs>
236 <projects>test-0-autopkgtest</projects>
237 <condition>ALWAYS</condition>
238diff --git a/tests/fixtures/launchpad/lp-four-ci.xml b/tests/fixtures/launchpad/lp-four-ci.xml
239index 18a9c78..4afa8e5 100644
240--- a/tests/fixtures/launchpad/lp-four-ci.xml
241+++ b/tests/fixtures/launchpad/lp-four-ci.xml
242@@ -48,7 +48,7 @@ deb https://user:pwd@private-ppa.launchpad.net/test-ci-team/fourprivate/ubuntu R
243 </hudson.model.ParametersDefinitionProperty>
244 </properties>
245 <scm class="hudson.scm.NullSCM"/>
246- <assignedNode>monitor</assignedNode>
247+ <assignedNode>monitor &amp;&amp; launchpad</assignedNode>
248 <canRoam>false</canRoam>
249 <disabled>false</disabled>
250 <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
251@@ -57,11 +57,65 @@ deb https://user:pwd@private-ppa.launchpad.net/test-ci-team/fourprivate/ubuntu R
252 <triggers/>
253 <concurrentBuild>true</concurrentBuild>
254 <builders>
255+ <hudson.tasks.Shell>
256+ <command>#!/usr/bin/env python2
257+
258+import atexit
259+import os
260+import re
261+from shutil import rmtree
262+import sys
263+from tempfile import mkdtemp
264+
265+from jlp import get_launchpad, launchpadutils
266+
267+# match lines starting with Prereq-archive: or Prereq-archive-keys:, followed by
268+# zero or more whitespace, then archive definition and any trailing whitespace
269+archive_re = re.compile(r"^Prereq-archive:\s*(.*?)\s*$", re.MULTILINE)
270+keys_re = re.compile(r"^Prereq-archive-keys:\s*(.*?)\s*$", re.MULTILINE)
271+
272+# retrieve values for parameters passed in the build
273+mp_url = os.getenv("merge_proposal", None)
274+archives = set(os.getenv("extra_repositories", "").splitlines())
275+archive_keys = set(os.getenv("extra_repository_keys", "").split())
276+
277+# if mp_url is not empty, try and get the MP description and parse Prereq-*
278+# declarations
279+if mp_url:
280+ # launchpadlib is not thread/process safe so we are creating launchpadlib
281+ # cache in /tmp per process which gets cleaned up at the end
282+ # see also lp:459418 and lp:1025153
283+ launchpad_cachedir = mkdtemp()
284+
285+ # make sure to drop the cache dir on exit
286+ atexit.register(rmtree, launchpad_cachedir, ignore_errors=True)
287+
288+ lp_handle = get_launchpad(launchpadlib_dir=launchpad_cachedir)
289+ mp = launchpadutils.get_mp_handle_from_url(lp_handle, mp_url)
290+ if not mp:
291+ sys.stderr.write("ERROR: Merge proposal '{}' not found".format(mp_url))
292+ sys.exit(1)
293+
294+ if mp.description:
295+ # add all Prereq-archive and Prereq-archive-keys to extra archives and keys
296+ archives.update(archive_re.findall(mp.description))
297+ archive_keys.update(keys_re.findall(mp.description))
298+
299+with open("lp.properties", "w") as params:
300+ # Java property file allows multi-line props with backslash at the end,
301+ # the \n is needed to actually have newlines in the resulting value
302+ params.write("extra_repositories={}\n".format("\\n\\\n".join(archives)))
303+ params.write("extra_repository_keys={}\n".format(" ".join(archive_keys)))
304+</command>
305+ </hudson.tasks.Shell>
306 <hudson.plugins.parameterizedtrigger.TriggerBuilder plugin="parameterized-trigger@2.29">
307 <configs>
308 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
309 <configs>
310- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
311+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
312+ <propertiesFile>lp.properties</propertiesFile>
313+ <failTriggerOnMissing>true</failTriggerOnMissing>
314+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
315 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
316 <properties>source_branch=${landing_candidate}
317 source_revision=${candidate_revision}
318@@ -99,6 +153,10 @@ target_branch=lp:four</properties>
319 <configs>
320 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
321 <configs>
322+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
323+ <propertiesFile>lp.properties</propertiesFile>
324+ <failTriggerOnMissing>true</failTriggerOnMissing>
325+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
326 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
327 <properties>build_run=build#${TRIGGERED_BUILD_NUMBER_build}
328 testnames=</properties>
329@@ -106,7 +164,6 @@ testnames=</properties>
330 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
331 <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
332 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
333- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
334 </configs>
335 <projects>test-0-autopkgtest</projects>
336 <condition>ALWAYS</condition>
337diff --git a/tests/fixtures/launchpad/lp-one-ci.xml b/tests/fixtures/launchpad/lp-one-ci.xml
338index 9c71787..e60f5e8 100644
339--- a/tests/fixtures/launchpad/lp-one-ci.xml
340+++ b/tests/fixtures/launchpad/lp-one-ci.xml
341@@ -47,7 +47,7 @@
342 </hudson.model.ParametersDefinitionProperty>
343 </properties>
344 <scm class="hudson.scm.NullSCM"/>
345- <assignedNode>monitor</assignedNode>
346+ <assignedNode>monitor &amp;&amp; launchpad</assignedNode>
347 <canRoam>false</canRoam>
348 <disabled>false</disabled>
349 <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
350@@ -56,11 +56,65 @@
351 <triggers/>
352 <concurrentBuild>true</concurrentBuild>
353 <builders>
354+ <hudson.tasks.Shell>
355+ <command>#!/usr/bin/env python2
356+
357+import atexit
358+import os
359+import re
360+from shutil import rmtree
361+import sys
362+from tempfile import mkdtemp
363+
364+from jlp import get_launchpad, launchpadutils
365+
366+# match lines starting with Prereq-archive: or Prereq-archive-keys:, followed by
367+# zero or more whitespace, then archive definition and any trailing whitespace
368+archive_re = re.compile(r"^Prereq-archive:\s*(.*?)\s*$", re.MULTILINE)
369+keys_re = re.compile(r"^Prereq-archive-keys:\s*(.*?)\s*$", re.MULTILINE)
370+
371+# retrieve values for parameters passed in the build
372+mp_url = os.getenv("merge_proposal", None)
373+archives = set(os.getenv("extra_repositories", "").splitlines())
374+archive_keys = set(os.getenv("extra_repository_keys", "").split())
375+
376+# if mp_url is not empty, try and get the MP description and parse Prereq-*
377+# declarations
378+if mp_url:
379+ # launchpadlib is not thread/process safe so we are creating launchpadlib
380+ # cache in /tmp per process which gets cleaned up at the end
381+ # see also lp:459418 and lp:1025153
382+ launchpad_cachedir = mkdtemp()
383+
384+ # make sure to drop the cache dir on exit
385+ atexit.register(rmtree, launchpad_cachedir, ignore_errors=True)
386+
387+ lp_handle = get_launchpad(launchpadlib_dir=launchpad_cachedir)
388+ mp = launchpadutils.get_mp_handle_from_url(lp_handle, mp_url)
389+ if not mp:
390+ sys.stderr.write("ERROR: Merge proposal '{}' not found".format(mp_url))
391+ sys.exit(1)
392+
393+ if mp.description:
394+ # add all Prereq-archive and Prereq-archive-keys to extra archives and keys
395+ archives.update(archive_re.findall(mp.description))
396+ archive_keys.update(keys_re.findall(mp.description))
397+
398+with open("lp.properties", "w") as params:
399+ # Java property file allows multi-line props with backslash at the end,
400+ # the \n is needed to actually have newlines in the resulting value
401+ params.write("extra_repositories={}\n".format("\\n\\\n".join(archives)))
402+ params.write("extra_repository_keys={}\n".format(" ".join(archive_keys)))
403+</command>
404+ </hudson.tasks.Shell>
405 <hudson.plugins.parameterizedtrigger.TriggerBuilder plugin="parameterized-trigger@2.29">
406 <configs>
407 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
408 <configs>
409- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
410+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
411+ <propertiesFile>lp.properties</propertiesFile>
412+ <failTriggerOnMissing>true</failTriggerOnMissing>
413+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
414 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
415 <properties>source_branch=${landing_candidate}
416 source_revision=${candidate_revision}
417@@ -98,6 +152,10 @@ target_branch=lp:one</properties>
418 <configs>
419 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
420 <configs>
421+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
422+ <propertiesFile>lp.properties</propertiesFile>
423+ <failTriggerOnMissing>true</failTriggerOnMissing>
424+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
425 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
426 <properties>build_run=build#${TRIGGERED_BUILD_NUMBER_build}
427 testnames=</properties>
428@@ -105,7 +163,6 @@ testnames=</properties>
429 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
430 <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
431 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
432- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
433 </configs>
434 <projects>test-0-autopkgtest</projects>
435 <condition>ALWAYS</condition>
436diff --git a/tests/fixtures/launchpad/lp-three-autoland.xml b/tests/fixtures/launchpad/lp-three-autoland.xml
437index 88f4934..8e84bc3 100644
438--- a/tests/fixtures/launchpad/lp-three-autoland.xml
439+++ b/tests/fixtures/launchpad/lp-three-autoland.xml
440@@ -7,7 +7,7 @@
441 <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
442 <authToken>BUILD_ME</authToken>
443 <concurrentBuild>true</concurrentBuild>
444- <assignedNode>monitor</assignedNode>
445+ <assignedNode>monitor &amp;&amp; launchpad</assignedNode>
446 <canRoam>false</canRoam>
447 <properties>
448 <jenkins.model.BuildDiscarderProperty>
449@@ -54,11 +54,65 @@
450 </properties>
451 <scm class="hudson.scm.NullSCM"/>
452 <builders>
453+ <hudson.tasks.Shell>
454+ <command>#!/usr/bin/env python2
455+
456+import atexit
457+import os
458+import re
459+from shutil import rmtree
460+import sys
461+from tempfile import mkdtemp
462+
463+from jlp import get_launchpad, launchpadutils
464+
465+# match lines starting with Prereq-archive: or Prereq-archive-keys:, followed by
466+# zero or more whitespace, then archive definition and any trailing whitespace
467+archive_re = re.compile(r&quot;^Prereq-archive:\s*(.*?)\s*$&quot;, re.MULTILINE)
468+keys_re = re.compile(r&quot;^Prereq-archive-keys:\s*(.*?)\s*$&quot;, re.MULTILINE)
469+
470+# retrieve values for parameters passed in the build
471+mp_url = os.getenv(&quot;merge_proposal&quot;, None)
472+archives = set(os.getenv(&quot;extra_repositories&quot;, &quot;&quot;).splitlines())
473+archive_keys = set(os.getenv(&quot;extra_repository_keys&quot;, &quot;&quot;).split())
474+
475+# if mp_url is not empty, try and get the MP description and parse Prereq-*
476+# declarations
477+if mp_url:
478+ # launchpadlib is not thread/process safe so we are creating launchpadlib
479+ # cache in /tmp per process which gets cleaned up at the end
480+ # see also lp:459418 and lp:1025153
481+ launchpad_cachedir = mkdtemp()
482+
483+ # make sure to drop the cache dir on exit
484+ atexit.register(rmtree, launchpad_cachedir, ignore_errors=True)
485+
486+ lp_handle = get_launchpad(launchpadlib_dir=launchpad_cachedir)
487+ mp = launchpadutils.get_mp_handle_from_url(lp_handle, mp_url)
488+ if not mp:
489+ sys.stderr.write(&quot;ERROR: Merge proposal '{}' not found&quot;.format(mp_url))
490+ sys.exit(1)
491+
492+ if mp.description:
493+ # add all Prereq-archive and Prereq-archive-keys to extra archives and keys
494+ archives.update(archive_re.findall(mp.description))
495+ archive_keys.update(keys_re.findall(mp.description))
496+
497+with open(&quot;lp.properties&quot;, &quot;w&quot;) as params:
498+ # Java property file allows multi-line props with backslash at the end,
499+ # the \n is needed to actually have newlines in the resulting value
500+ params.write(&quot;extra_repositories={}\n&quot;.format(&quot;\\n\\\n&quot;.join(archives)))
501+ params.write(&quot;extra_repository_keys={}\n&quot;.format(&quot; &quot;.join(archive_keys)))
502+</command>
503+ </hudson.tasks.Shell>
504 <hudson.plugins.parameterizedtrigger.TriggerBuilder>
505 <configs>
506 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
507 <configs>
508- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
509+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
510+ <propertiesFile>lp.properties</propertiesFile>
511+ <failTriggerOnMissing>true</failTriggerOnMissing>
512+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
513 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
514 <properties>source_branch=${landing_candidate}
515 source_revision=${candidate_revision}
516@@ -97,6 +151,10 @@ target_branch=lp:three/devel
517 <configs>
518 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
519 <configs>
520+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
521+ <propertiesFile>lp.properties</propertiesFile>
522+ <failTriggerOnMissing>true</failTriggerOnMissing>
523+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
524 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
525 <properties>build_run=build#${TRIGGERED_BUILD_NUMBER_build}
526 testnames=test-three
527@@ -105,7 +163,6 @@ testnames=test-three
528 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
529 <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
530 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
531- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
532 </configs>
533 <projects>test-0-autopkgtest</projects>
534 <condition>ALWAYS</condition>
535diff --git a/tests/fixtures/launchpad/lp-three-ci.xml b/tests/fixtures/launchpad/lp-three-ci.xml
536index 4c0bb36..2dde932 100644
537--- a/tests/fixtures/launchpad/lp-three-ci.xml
538+++ b/tests/fixtures/launchpad/lp-three-ci.xml
539@@ -47,7 +47,7 @@
540 </hudson.model.ParametersDefinitionProperty>
541 </properties>
542 <scm class="hudson.scm.NullSCM"/>
543- <assignedNode>monitor</assignedNode>
544+ <assignedNode>monitor &amp;&amp; launchpad</assignedNode>
545 <canRoam>false</canRoam>
546 <disabled>false</disabled>
547 <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
548@@ -56,11 +56,65 @@
549 <triggers/>
550 <concurrentBuild>true</concurrentBuild>
551 <builders>
552+ <hudson.tasks.Shell>
553+ <command>#!/usr/bin/env python2
554+
555+import atexit
556+import os
557+import re
558+from shutil import rmtree
559+import sys
560+from tempfile import mkdtemp
561+
562+from jlp import get_launchpad, launchpadutils
563+
564+# match lines starting with Prereq-archive: or Prereq-archive-keys:, followed by
565+# zero or more whitespace, then archive definition and any trailing whitespace
566+archive_re = re.compile(r"^Prereq-archive:\s*(.*?)\s*$", re.MULTILINE)
567+keys_re = re.compile(r"^Prereq-archive-keys:\s*(.*?)\s*$", re.MULTILINE)
568+
569+# retrieve values for parameters passed in the build
570+mp_url = os.getenv("merge_proposal", None)
571+archives = set(os.getenv("extra_repositories", "").splitlines())
572+archive_keys = set(os.getenv("extra_repository_keys", "").split())
573+
574+# if mp_url is not empty, try and get the MP description and parse Prereq-*
575+# declarations
576+if mp_url:
577+ # launchpadlib is not thread/process safe so we are creating launchpadlib
578+ # cache in /tmp per process which gets cleaned up at the end
579+ # see also lp:459418 and lp:1025153
580+ launchpad_cachedir = mkdtemp()
581+
582+ # make sure to drop the cache dir on exit
583+ atexit.register(rmtree, launchpad_cachedir, ignore_errors=True)
584+
585+ lp_handle = get_launchpad(launchpadlib_dir=launchpad_cachedir)
586+ mp = launchpadutils.get_mp_handle_from_url(lp_handle, mp_url)
587+ if not mp:
588+ sys.stderr.write("ERROR: Merge proposal '{}' not found".format(mp_url))
589+ sys.exit(1)
590+
591+ if mp.description:
592+ # add all Prereq-archive and Prereq-archive-keys to extra archives and keys
593+ archives.update(archive_re.findall(mp.description))
594+ archive_keys.update(keys_re.findall(mp.description))
595+
596+with open("lp.properties", "w") as params:
597+ # Java property file allows multi-line props with backslash at the end,
598+ # the \n is needed to actually have newlines in the resulting value
599+ params.write("extra_repositories={}\n".format("\\n\\\n".join(archives)))
600+ params.write("extra_repository_keys={}\n".format(" ".join(archive_keys)))
601+</command>
602+ </hudson.tasks.Shell>
603 <hudson.plugins.parameterizedtrigger.TriggerBuilder plugin="parameterized-trigger@2.29">
604 <configs>
605 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
606 <configs>
607- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
608+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
609+ <propertiesFile>lp.properties</propertiesFile>
610+ <failTriggerOnMissing>true</failTriggerOnMissing>
611+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
612 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
613 <properties>source_branch=${landing_candidate}
614 source_revision=${candidate_revision}
615@@ -98,6 +152,10 @@ target_branch=lp:three/devel</properties>
616 <configs>
617 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
618 <configs>
619+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
620+ <propertiesFile>lp.properties</propertiesFile>
621+ <failTriggerOnMissing>true</failTriggerOnMissing>
622+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
623 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
624 <properties>build_run=build#${TRIGGERED_BUILD_NUMBER_build}
625 testnames=</properties>
626@@ -105,7 +163,6 @@ testnames=</properties>
627 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
628 <filter>(label==&quot;amd64&quot;) &amp;&amp; (release==&quot;devel&quot;)</filter>
629 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
630- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
631 </configs>
632 <projects>test-0-autopkgtest</projects>
633 <condition>ALWAYS</condition>
634diff --git a/tests/fixtures/launchpad/lp-two-autoland.xml b/tests/fixtures/launchpad/lp-two-autoland.xml
635index 7e7b52d..48a84ee 100644
636--- a/tests/fixtures/launchpad/lp-two-autoland.xml
637+++ b/tests/fixtures/launchpad/lp-two-autoland.xml
638@@ -7,7 +7,7 @@
639 <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
640 <authToken>BUILD_ME</authToken>
641 <concurrentBuild>true</concurrentBuild>
642- <assignedNode>monitor</assignedNode>
643+ <assignedNode>monitor &amp;&amp; launchpad</assignedNode>
644 <canRoam>false</canRoam>
645 <properties>
646 <jenkins.model.BuildDiscarderProperty>
647@@ -54,11 +54,65 @@
648 </properties>
649 <scm class="hudson.scm.NullSCM"/>
650 <builders>
651+ <hudson.tasks.Shell>
652+ <command>#!/usr/bin/env python2
653+
654+import atexit
655+import os
656+import re
657+from shutil import rmtree
658+import sys
659+from tempfile import mkdtemp
660+
661+from jlp import get_launchpad, launchpadutils
662+
663+# match lines starting with Prereq-archive: or Prereq-archive-keys:, followed by
664+# zero or more whitespace, then archive definition and any trailing whitespace
665+archive_re = re.compile(r"^Prereq-archive:\s*(.*?)\s*$", re.MULTILINE)
666+keys_re = re.compile(r"^Prereq-archive-keys:\s*(.*?)\s*$", re.MULTILINE)
667+
668+# retrieve values for parameters passed in the build
669+mp_url = os.getenv("merge_proposal", None)
670+archives = set(os.getenv("extra_repositories", "").splitlines())
671+archive_keys = set(os.getenv("extra_repository_keys", "").split())
672+
673+# if mp_url is not empty, try and get the MP description and parse Prereq-*
674+# declarations
675+if mp_url:
676+ # launchpadlib is not thread/process safe so we are creating launchpadlib
677+ # cache in /tmp per process which gets cleaned up at the end
678+ # see also lp:459418 and lp:1025153
679+ launchpad_cachedir = mkdtemp()
680+
681+ # make sure to drop the cache dir on exit
682+ atexit.register(rmtree, launchpad_cachedir, ignore_errors=True)
683+
684+ lp_handle = get_launchpad(launchpadlib_dir=launchpad_cachedir)
685+ mp = launchpadutils.get_mp_handle_from_url(lp_handle, mp_url)
686+ if not mp:
687+ sys.stderr.write("ERROR: Merge proposal '{}' not found".format(mp_url))
688+ sys.exit(1)
689+
690+ if mp.description:
691+ # add all Prereq-archive and Prereq-archive-keys to extra archives and keys
692+ archives.update(archive_re.findall(mp.description))
693+ archive_keys.update(keys_re.findall(mp.description))
694+
695+with open("lp.properties", "w") as params:
696+ # Java property file allows multi-line props with backslash at the end,
697+ # the \n is needed to actually have newlines in the resulting value
698+ params.write("extra_repositories={}\n".format("\\n\\\n".join(archives)))
699+ params.write("extra_repository_keys={}\n".format(" ".join(archive_keys)))
700+</command>
701+ </hudson.tasks.Shell>
702 <hudson.plugins.parameterizedtrigger.TriggerBuilder>
703 <configs>
704 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
705 <configs>
706- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
707+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
708+ <propertiesFile>lp.properties</propertiesFile>
709+ <failTriggerOnMissing>true</failTriggerOnMissing>
710+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
711 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
712 <properties>source_branch=${landing_candidate}
713 source_revision=${candidate_revision}
714@@ -97,6 +151,10 @@ target_branch=lp:two
715 <configs>
716 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
717 <configs>
718+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
719+ <propertiesFile>lp.properties</propertiesFile>
720+ <failTriggerOnMissing>true</failTriggerOnMissing>
721+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
722 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
723 <properties>build_run=build#${TRIGGERED_BUILD_NUMBER_build}
724 testnames=test-one test-two
725@@ -105,7 +163,6 @@ testnames=test-one test-two
726 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
727 <filter>(label==&quot;armhf&quot;) &amp;&amp; (release==&quot;stable&quot;)</filter>
728 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
729- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
730 </configs>
731 <projects>test-0-autopkgtest</projects>
732 <condition>ALWAYS</condition>
733diff --git a/tests/fixtures/launchpad/lp-two-ci.xml b/tests/fixtures/launchpad/lp-two-ci.xml
734index d1b8a46..10f0e10 100644
735--- a/tests/fixtures/launchpad/lp-two-ci.xml
736+++ b/tests/fixtures/launchpad/lp-two-ci.xml
737@@ -47,7 +47,7 @@
738 </hudson.model.ParametersDefinitionProperty>
739 </properties>
740 <scm class="hudson.scm.NullSCM"/>
741- <assignedNode>monitor</assignedNode>
742+ <assignedNode>monitor &amp;&amp; launchpad</assignedNode>
743 <canRoam>false</canRoam>
744 <disabled>false</disabled>
745 <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
746@@ -56,11 +56,65 @@
747 <triggers/>
748 <concurrentBuild>true</concurrentBuild>
749 <builders>
750+ <hudson.tasks.Shell>
751+ <command>#!/usr/bin/env python2
752+
753+import atexit
754+import os
755+import re
756+from shutil import rmtree
757+import sys
758+from tempfile import mkdtemp
759+
760+from jlp import get_launchpad, launchpadutils
761+
762+# match lines starting with Prereq-archive: or Prereq-archive-keys:, followed by
763+# zero or more whitespace, then archive definition and any trailing whitespace
764+archive_re = re.compile(r"^Prereq-archive:\s*(.*?)\s*$", re.MULTILINE)
765+keys_re = re.compile(r"^Prereq-archive-keys:\s*(.*?)\s*$", re.MULTILINE)
766+
767+# retrieve values for parameters passed in the build
768+mp_url = os.getenv("merge_proposal", None)
769+archives = set(os.getenv("extra_repositories", "").splitlines())
770+archive_keys = set(os.getenv("extra_repository_keys", "").split())
771+
772+# if mp_url is not empty, try and get the MP description and parse Prereq-*
773+# declarations
774+if mp_url:
775+ # launchpadlib is not thread/process safe so we are creating launchpadlib
776+ # cache in /tmp per process which gets cleaned up at the end
777+ # see also lp:459418 and lp:1025153
778+ launchpad_cachedir = mkdtemp()
779+
780+ # make sure to drop the cache dir on exit
781+ atexit.register(rmtree, launchpad_cachedir, ignore_errors=True)
782+
783+ lp_handle = get_launchpad(launchpadlib_dir=launchpad_cachedir)
784+ mp = launchpadutils.get_mp_handle_from_url(lp_handle, mp_url)
785+ if not mp:
786+ sys.stderr.write("ERROR: Merge proposal '{}' not found".format(mp_url))
787+ sys.exit(1)
788+
789+ if mp.description:
790+ # add all Prereq-archive and Prereq-archive-keys to extra archives and keys
791+ archives.update(archive_re.findall(mp.description))
792+ archive_keys.update(keys_re.findall(mp.description))
793+
794+with open("lp.properties", "w") as params:
795+ # Java property file allows multi-line props with backslash at the end,
796+ # the \n is needed to actually have newlines in the resulting value
797+ params.write("extra_repositories={}\n".format("\\n\\\n".join(archives)))
798+ params.write("extra_repository_keys={}\n".format(" ".join(archive_keys)))
799+</command>
800+ </hudson.tasks.Shell>
801 <hudson.plugins.parameterizedtrigger.TriggerBuilder plugin="parameterized-trigger@2.29">
802 <configs>
803 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
804 <configs>
805- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
806+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
807+ <propertiesFile>lp.properties</propertiesFile>
808+ <failTriggerOnMissing>true</failTriggerOnMissing>
809+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
810 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
811 <properties>source_branch=${landing_candidate}
812 source_revision=${candidate_revision}
813@@ -98,6 +152,10 @@ target_branch=lp:two</properties>
814 <configs>
815 <hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig>
816 <configs>
817+ <hudson.plugins.parameterizedtrigger.FileBuildParameters>
818+ <propertiesFile>lp.properties</propertiesFile>
819+ <failTriggerOnMissing>true</failTriggerOnMissing>
820+ </hudson.plugins.parameterizedtrigger.FileBuildParameters>
821 <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
822 <properties>build_run=build#${TRIGGERED_BUILD_NUMBER_build}
823 testnames=test-one test-two</properties>
824@@ -105,7 +163,6 @@ testnames=test-one test-two</properties>
825 <hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
826 <filter>(label==&quot;armhf&quot;) &amp;&amp; (release==&quot;stable&quot;)</filter>
827 </hudson.plugins.parameterizedtrigger.matrix.MatrixSubsetBuildParameters>
828- <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
829 </configs>
830 <projects>test-0-autopkgtest</projects>
831 <condition>ALWAYS</condition>

Subscribers

People subscribed via source and target branches