Merge lp:~hloeung/jenkins-agent-charm/reactive-rewrite into lp:jenkins-agent-charm
- reactive-rewrite
- Merge into trunk
Proposed by
Haw Loeung
Status: | Merged |
---|---|
Approved by: | Tom Haddon |
Approved revision: | 51 |
Merged at revision: | 22 |
Proposed branch: | lp:~hloeung/jenkins-agent-charm/reactive-rewrite |
Merge into: | lp:jenkins-agent-charm |
Diff against target: |
1152 lines (+673/-311) 25 files modified
.bzrignore (+11/-0) Makefile (+30/-0) README.md (+5/-0) files/jenkins-slave-sudoers (+5/-0) hooks/configure-slave (+0/-27) hooks/install.d/add_sudoers (+0/-18) hooks/install.d/canonical_ci_utils.py (+0/-91) hooks/nrpe-external-master-relation-changed (+0/-7) hooks/slave-relation-changed (+0/-19) hooks/slave-relation-departed (+0/-3) hooks/slave-relation-joined (+0/-20) hooks/start (+0/-3) hooks/stop (+0/-3) layer.yaml (+5/-0) metadata.yaml (+5/-1) reactive/jenkins_slave.py (+228/-118) requirements.txt (+1/-0) revision (+0/-1) templates/jenkins-slave-default (+8/-0) tests/functional/requirements.txt (+6/-0) tests/functional/test_jenkins_slave.py (+50/-0) tests/unit/files/jenkins-slave-default (+59/-0) tests/unit/requirements.txt (+5/-0) tests/unit/test_jenkins_slave.py (+218/-0) tox.ini (+37/-0) |
To merge this branch: | bzr merge lp:~hloeung/jenkins-agent-charm/reactive-rewrite |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Tom Haddon | Approve | ||
Haw Loeung | Needs Resubmitting | ||
Canonical IS Reviewers | Pending | ||
Review via email: mp+363649@code.launchpad.net |
Commit message
Charm rewrite using Python and the Reactive framework
Description of the change
To post a comment you must log in.
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
Revision history for this message
Tom Haddon (mthaddon) wrote : | # |
Some comments and questions inline.
review:
Needs Fixing
Revision history for this message
Tom Haddon (mthaddon) wrote : | # |
Missed one other thing as well.
Revision history for this message
Tom Haddon (mthaddon) wrote : | # |
Some comments inline with suggestions for testing.
Revision history for this message
Tom Haddon (mthaddon) wrote : | # |
Er, we we obviously *do* want the max-complexity check
Revision history for this message
Tom Haddon (mthaddon) wrote : | # |
A few comments inline
Revision history for this message
Haw Loeung (hloeung) : | # |
review:
Needs Resubmitting
- 50. By Haw Loeung
-
Added basic functional test
- 51. By Haw Loeung
-
Fixed to use the module itself instead of sys.modules
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
Change successfully merged at revision 22
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === added file '.bzrignore' | |||
2 | --- .bzrignore 1970-01-01 00:00:00 +0000 | |||
3 | +++ .bzrignore 2019-03-01 05:39:53 +0000 | |||
4 | @@ -0,0 +1,11 @@ | |||
5 | 1 | *.pyc | ||
6 | 2 | *.swp | ||
7 | 3 | *~ | ||
8 | 4 | .coverage | ||
9 | 5 | .pytest_cache/ | ||
10 | 6 | .tox/ | ||
11 | 7 | .unit-state.db | ||
12 | 8 | __pycache__/ | ||
13 | 9 | builds/ | ||
14 | 10 | deps/ | ||
15 | 11 | revision | ||
16 | 0 | 12 | ||
17 | === added file 'Makefile' | |||
18 | --- Makefile 1970-01-01 00:00:00 +0000 | |||
19 | +++ Makefile 2019-03-01 05:39:53 +0000 | |||
20 | @@ -0,0 +1,30 @@ | |||
21 | 1 | help: | ||
22 | 2 | @echo "This project supports the following targets" | ||
23 | 3 | @echo "" | ||
24 | 4 | @echo " make help - show this text" | ||
25 | 5 | @echo " make lint - run flake8" | ||
26 | 6 | @echo " make test - run the functional test and unittests" | ||
27 | 7 | @echo " make unittest - run the the unittest" | ||
28 | 8 | @echo " make functionaltest - run the functional tests" | ||
29 | 9 | @echo " make clean - remove unneeded files" | ||
30 | 10 | @echo "" | ||
31 | 11 | |||
32 | 12 | lint: | ||
33 | 13 | @echo "Running flake8" | ||
34 | 14 | @tox -e lint | ||
35 | 15 | |||
36 | 16 | test: unittest functionaltest lint | ||
37 | 17 | |||
38 | 18 | unittest: | ||
39 | 19 | @tox -e unit | ||
40 | 20 | |||
41 | 21 | functionaltest: | ||
42 | 22 | @tox -e functional | ||
43 | 23 | |||
44 | 24 | clean: | ||
45 | 25 | @echo "Cleaning files" | ||
46 | 26 | @if [ -d ./.tox ] ; then rm -r ./.tox ; fi | ||
47 | 27 | @if [ -d ./.pytest_cache ] ; then rm -r ./.pytest_cache ; fi | ||
48 | 28 | |||
49 | 29 | # The targets below don't depend on a file | ||
50 | 30 | .PHONY: lint test unittest functionaltest clean help | ||
51 | 0 | 31 | ||
52 | === modified file 'README.md' | |||
53 | --- README.md 2014-05-15 16:06:29 +0000 | |||
54 | +++ README.md 2019-03-01 05:39:53 +0000 | |||
55 | @@ -21,3 +21,8 @@ | |||
56 | 21 | juju deploy --to <special-mabine-number> jenkins-slave ppc-slave | 21 | juju deploy --to <special-mabine-number> jenkins-slave ppc-slave |
57 | 22 | 22 | ||
58 | 23 | See the Jenkins charm for more details. | 23 | See the Jenkins charm for more details. |
59 | 24 | |||
60 | 25 | |||
61 | 26 | # Notes | ||
62 | 27 | |||
63 | 28 | We can't use interface:jenkins-slave yet as it's not fully implemented. | ||
64 | 24 | 29 | ||
65 | === added directory 'actions' | |||
66 | === added file 'files/jenkins-slave-sudoers' | |||
67 | --- files/jenkins-slave-sudoers 1970-01-01 00:00:00 +0000 | |||
68 | +++ files/jenkins-slave-sudoers 2019-03-01 05:39:53 +0000 | |||
69 | @@ -0,0 +1,5 @@ | |||
70 | 1 | # Created automatically during charm installation | ||
71 | 2 | # Any manual changes to it will be lost | ||
72 | 3 | |||
73 | 4 | # User rules for jenkins | ||
74 | 5 | jenkins ALL=(ALL) NOPASSWD: ALL | ||
75 | 0 | 6 | ||
76 | === removed symlink 'hooks/config-changed' | |||
77 | === target was u'install' | |||
78 | === removed file 'hooks/configure-slave' | |||
79 | --- hooks/configure-slave 2016-06-03 02:54:52 +0000 | |||
80 | +++ hooks/configure-slave 1970-01-01 00:00:00 +0000 | |||
81 | @@ -1,27 +0,0 @@ | |||
82 | 1 | #!/bin/bash | ||
83 | 2 | |||
84 | 3 | set -e | ||
85 | 4 | |||
86 | 5 | # Grab the jenkins master url as a passed in parameter | ||
87 | 6 | url="$1" | ||
88 | 7 | |||
89 | 8 | # Set the slave hostname to match the juju unit | ||
90 | 9 | slavehost="$(echo ${JUJU_UNIT_NAME} | sed s,/,-,)" | ||
91 | 10 | |||
92 | 11 | # Set the slave and url fields | ||
93 | 12 | juju-log "Configuring jenkins-slave with ${url}..." | ||
94 | 13 | sed -i -e "s!^JENKINS_HOSTNAME.*!JENKINS_HOSTNAME=${slavehost}!" \ | ||
95 | 14 | -e "s!^#*JENKINS_URL.*!JENKINS_URL=${url}!" \ | ||
96 | 15 | /etc/default/jenkins-slave | ||
97 | 16 | |||
98 | 17 | # Startup the jenkins-slave service | ||
99 | 18 | # This is called in the install, config-changed and upgrade-charm paths | ||
100 | 19 | # It needs to be tolerant of a running or stopped jenkins-slave | ||
101 | 20 | status="$(service jenkins-slave status || true)" | ||
102 | 21 | if echo "${status}" | egrep -q "stop|inactive"; then | ||
103 | 22 | juju-log "Starting jenkins-slave..." | ||
104 | 23 | service jenkins-slave start | ||
105 | 24 | else | ||
106 | 25 | juju-log "Restarting jenkins-slave..." | ||
107 | 26 | service jenkins-slave restart | ||
108 | 27 | fi | ||
109 | 28 | 0 | ||
110 | === removed symlink 'hooks/install.d/03_nrpe_relation_changed' | |||
111 | === target was u'canonical_ci_utils.py' | |||
112 | === removed file 'hooks/install.d/add_sudoers' | |||
113 | --- hooks/install.d/add_sudoers 2015-08-13 18:04:16 +0000 | |||
114 | +++ hooks/install.d/add_sudoers 1970-01-01 00:00:00 +0000 | |||
115 | @@ -1,18 +0,0 @@ | |||
116 | 1 | #!/bin/bash | ||
117 | 2 | |||
118 | 3 | set -eu | ||
119 | 4 | |||
120 | 5 | temp_sudoers="$(mktemp /tmp/jenkins.XXXXXX)" | ||
121 | 6 | |||
122 | 7 | cat > "$temp_sudoers" << EOF | ||
123 | 8 | # Created automatically during charm installation | ||
124 | 9 | # Any manual changes to it will be lost | ||
125 | 10 | |||
126 | 11 | # User rules for jenkins | ||
127 | 12 | jenkins ALL=(ALL) NOPASSWD: ALL | ||
128 | 13 | EOF | ||
129 | 14 | |||
130 | 15 | visudo -c -f "$temp_sudoers" | ||
131 | 16 | install -m 440 "$temp_sudoers" /etc/sudoers.d/jenkins | ||
132 | 17 | rm "$temp_sudoers" | ||
133 | 18 | visudo -c | ||
134 | 19 | 0 | ||
135 | === removed file 'hooks/install.d/canonical_ci_utils.py' | |||
136 | --- hooks/install.d/canonical_ci_utils.py 2015-12-03 10:40:55 +0000 | |||
137 | +++ hooks/install.d/canonical_ci_utils.py 1970-01-01 00:00:00 +0000 | |||
138 | @@ -1,91 +0,0 @@ | |||
139 | 1 | #!/usr/bin/python | ||
140 | 2 | |||
141 | 3 | import os | ||
142 | 4 | import sys | ||
143 | 5 | |||
144 | 6 | from charmhelpers.canonical_ci import nrpe | ||
145 | 7 | |||
146 | 8 | |||
147 | 9 | from charmhelpers.core.hookenv import ( | ||
148 | 10 | config, | ||
149 | 11 | local_unit, | ||
150 | 12 | log, | ||
151 | 13 | relation_ids, | ||
152 | 14 | unit_get, | ||
153 | 15 | ERROR, | ||
154 | 16 | INFO, | ||
155 | 17 | ) | ||
156 | 18 | |||
157 | 19 | from charmhelpers.core.host import ( | ||
158 | 20 | service_reload, | ||
159 | 21 | ) | ||
160 | 22 | |||
161 | 23 | # Constants set in jenkins/hooks/install. | ||
162 | 24 | JENKINS_HOME = '/var/lib/jenkins' | ||
163 | 25 | JENKINS_USER = 'jenkins' | ||
164 | 26 | JENKINS_GROUP = 'nogroup' | ||
165 | 27 | JENKINS_PORT = 8080 | ||
166 | 28 | |||
167 | 29 | # Local constants | ||
168 | 30 | NRPE_CHECK_PS = "/etc/nagios/nrpe.d/check_jenkins_slave_ps.cfg" | ||
169 | 31 | |||
170 | 32 | |||
171 | 33 | def update_nrpe_config(): | ||
172 | 34 | if not relation_ids('nrpe-external-master'): | ||
173 | 35 | log('No relation to an nrpe-external-master, not configuring ' | ||
174 | 36 | 'nagios.') | ||
175 | 37 | return | ||
176 | 38 | with open(NRPE_CHECK_PS, "w") as conf: | ||
177 | 39 | log("Writing config: %s." % NRPE_CHECK_PS, INFO) | ||
178 | 40 | conf.write(nrpe.CONF_HEADER) | ||
179 | 41 | conf.write( | ||
180 | 42 | "command[check_jenkins_slave_ps]=" | ||
181 | 43 | "/usr/lib/nagios/plugins/check_procs -c 1:1 -a slave.jar") | ||
182 | 44 | |||
183 | 45 | # config | ||
184 | 46 | unit_name = local_unit().replace('/', '-') | ||
185 | 47 | n_hostname = "%s-%s" % (config("nagios_context"), unit_name) | ||
186 | 48 | n_servicegroup = config("nagios_servicegroups") if config("nagios_servicegroups") else config("nagios_context") | ||
187 | 49 | service_file = ( | ||
188 | 50 | '/var/lib/nagios/export/service__%s_check_jenkins_slave.cfg' % | ||
189 | 51 | n_hostname) | ||
190 | 52 | |||
191 | 53 | # There is a race between this code and the nrpe_external_master install | ||
192 | 54 | # hook. Both need to write files to /var/lib/nagios/export, but only one | ||
193 | 55 | # needs to create it. So tolerate failure if the directory exists. | ||
194 | 56 | try: | ||
195 | 57 | os.makedirs(os.path.dirname(service_file)) | ||
196 | 58 | except os.error: | ||
197 | 59 | pass | ||
198 | 60 | |||
199 | 61 | with open(service_file, "w") as conf: | ||
200 | 62 | log("Writing config: %s." % conf, INFO) | ||
201 | 63 | conf.write(nrpe.CONF_HEADER) | ||
202 | 64 | conf.write(nrpe.NRPE_SERVICE_ENTRY % { | ||
203 | 65 | 'nagios_hostname': n_hostname, | ||
204 | 66 | 'check_name': 'check_jenkins_slave_ps', | ||
205 | 67 | 'nagios_servicegroup': n_servicegroup, | ||
206 | 68 | }) | ||
207 | 69 | |||
208 | 70 | # reboot | ||
209 | 71 | if os.path.isfile('/etc/init.d/nagios-nrpe-server'): | ||
210 | 72 | service_reload('nagios-nrpe-server') | ||
211 | 73 | |||
212 | 74 | |||
213 | 75 | hooks = { | ||
214 | 76 | '03_nrpe_relation_changed': update_nrpe_config | ||
215 | 77 | } | ||
216 | 78 | |||
217 | 79 | |||
218 | 80 | def main(hook): | ||
219 | 81 | try: | ||
220 | 82 | hooks[hook]() | ||
221 | 83 | except KeyError: | ||
222 | 84 | e = 'Invalid install.d hook: %s.' % hook | ||
223 | 85 | log(e, ERROR) | ||
224 | 86 | raise Exception(e) | ||
225 | 87 | |||
226 | 88 | |||
227 | 89 | if __name__ == '__main__': | ||
228 | 90 | if os.path.islink(sys.argv[0]): | ||
229 | 91 | main(os.path.basename(sys.argv[0])) | ||
230 | 92 | 0 | ||
231 | === removed file 'hooks/nrpe-external-master-relation-changed' | |||
232 | --- hooks/nrpe-external-master-relation-changed 2015-08-19 20:00:15 +0000 | |||
233 | +++ hooks/nrpe-external-master-relation-changed 1970-01-01 00:00:00 +0000 | |||
234 | @@ -1,7 +0,0 @@ | |||
235 | 1 | #!/bin/sh | ||
236 | 2 | set -e | ||
237 | 3 | |||
238 | 4 | home=`dirname $0` | ||
239 | 5 | |||
240 | 6 | juju-log "Running install hook once more to pick up new NRPE relation changes" | ||
241 | 7 | exec $home/install | ||
242 | 8 | 0 | ||
243 | === removed symlink 'hooks/slave-relation-broken' | |||
244 | === target was u'slave-relation-departed' | |||
245 | === removed file 'hooks/slave-relation-changed' | |||
246 | --- hooks/slave-relation-changed 2015-08-05 13:20:44 +0000 | |||
247 | +++ hooks/slave-relation-changed 1970-01-01 00:00:00 +0000 | |||
248 | @@ -1,19 +0,0 @@ | |||
249 | 1 | #!/bin/bash | ||
250 | 2 | |||
251 | 3 | set -e | ||
252 | 4 | |||
253 | 5 | # Setup connection to master instance once set | ||
254 | 6 | url="$(relation-get url)" | ||
255 | 7 | |||
256 | 8 | if [ "x$url" = "x" ]; then | ||
257 | 9 | juju-log "Master hasn't exported its url yet, exiting..." | ||
258 | 10 | exit 0 | ||
259 | 11 | fi | ||
260 | 12 | |||
261 | 13 | master_url="$(config-get master_url)" | ||
262 | 14 | if [ -n "${master_url}" ]; then | ||
263 | 15 | juju-log "Config option 'master_url' is set. Can't use slave relation." | ||
264 | 16 | exit 0 | ||
265 | 17 | fi | ||
266 | 18 | |||
267 | 19 | hooks/configure-slave "${url}" | ||
268 | 20 | 0 | ||
269 | === removed file 'hooks/slave-relation-departed' | |||
270 | --- hooks/slave-relation-departed 2012-07-27 11:23:24 +0000 | |||
271 | +++ hooks/slave-relation-departed 1970-01-01 00:00:00 +0000 | |||
272 | @@ -1,3 +0,0 @@ | |||
273 | 1 | #!/bin/bash | ||
274 | 2 | |||
275 | 3 | stop jenkins-slave || true | ||
276 | 4 | 0 | ||
277 | === removed file 'hooks/slave-relation-joined' | |||
278 | --- hooks/slave-relation-joined 2015-08-04 16:46:21 +0000 | |||
279 | +++ hooks/slave-relation-joined 1970-01-01 00:00:00 +0000 | |||
280 | @@ -1,20 +0,0 @@ | |||
281 | 1 | #!/bin/bash | ||
282 | 2 | |||
283 | 3 | set -e | ||
284 | 4 | |||
285 | 5 | # Set the slave hostname to match the juju unit | ||
286 | 6 | # in the jenkins master instance | ||
287 | 7 | slavehost=`echo ${JUJU_UNIT_NAME} | sed s,/,-,` | ||
288 | 8 | noexecutors=`cat /proc/cpuinfo | grep processor | wc -l` | ||
289 | 9 | config_labels=`config-get labels` | ||
290 | 10 | labels=`uname -p` | ||
291 | 11 | |||
292 | 12 | if [ -n "$config_labels" ]; then | ||
293 | 13 | labels=$config_labels | ||
294 | 14 | fi | ||
295 | 15 | |||
296 | 16 | # Set all relations | ||
297 | 17 | relation-set executors=$noexecutors | ||
298 | 18 | relation-set labels="$labels" | ||
299 | 19 | relation-set slavehost=$slavehost | ||
300 | 20 | relation-set slaveaddress=`unit-get private-address` | ||
301 | 21 | 0 | ||
302 | === removed file 'hooks/start' | |||
303 | --- hooks/start 2011-08-02 08:53:54 +0000 | |||
304 | +++ hooks/start 1970-01-01 00:00:00 +0000 | |||
305 | @@ -1,3 +0,0 @@ | |||
306 | 1 | #!/bin/bash | ||
307 | 2 | |||
308 | 3 | start jenkins-slave || true | ||
309 | 4 | 0 | ||
310 | === removed file 'hooks/stop' | |||
311 | --- hooks/stop 2011-08-02 08:53:54 +0000 | |||
312 | +++ hooks/stop 1970-01-01 00:00:00 +0000 | |||
313 | @@ -1,3 +0,0 @@ | |||
314 | 1 | #!/bin/bash | ||
315 | 2 | |||
316 | 3 | stop jenkins-slave | ||
317 | 4 | 0 | ||
318 | === removed symlink 'hooks/upgrade-charm' | |||
319 | === target was u'install' | |||
320 | === added file 'layer.yaml' | |||
321 | --- layer.yaml 1970-01-01 00:00:00 +0000 | |||
322 | +++ layer.yaml 2019-03-01 05:39:53 +0000 | |||
323 | @@ -0,0 +1,5 @@ | |||
324 | 1 | includes: | ||
325 | 2 | - layer:basic | ||
326 | 3 | - layer:apt | ||
327 | 4 | - layer:nagios | ||
328 | 5 | repo: lp:jenkins-slave-charm | ||
329 | 0 | 6 | ||
330 | === added directory 'lib' | |||
331 | === modified file 'metadata.yaml' | |||
332 | --- metadata.yaml 2019-02-18 00:41:09 +0000 | |||
333 | +++ metadata.yaml 2019-03-01 05:39:53 +0000 | |||
334 | @@ -9,8 +9,12 @@ | |||
335 | 9 | This charm provides support for jenkins slaves | 9 | This charm provides support for jenkins slaves |
336 | 10 | . | 10 | . |
337 | 11 | https://launchpad.net/jenkins-slave-charm | 11 | https://launchpad.net/jenkins-slave-charm |
339 | 12 | categories: | 12 | tags: |
340 | 13 | - applications | 13 | - applications |
341 | 14 | series: | ||
342 | 15 | - bionic | ||
343 | 16 | - xenial | ||
344 | 17 | - trusty | ||
345 | 14 | provides: | 18 | provides: |
346 | 15 | slave: | 19 | slave: |
347 | 16 | interface: jenkins-slave | 20 | interface: jenkins-slave |
348 | 17 | 21 | ||
349 | === added directory 'reactive' | |||
350 | === renamed file 'hooks/install' => 'reactive/jenkins_slave.py' | |||
351 | --- hooks/install 2016-06-14 01:04:43 +0000 | |||
352 | +++ reactive/jenkins_slave.py 2019-03-01 05:39:53 +0000 | |||
353 | @@ -1,121 +1,231 @@ | |||
385 | 1 | #!/bin/bash | 1 | import os |
386 | 2 | 2 | ||
387 | 3 | set -eu | 3 | from charmhelpers.core import hookenv, host, templating, unitdata |
388 | 4 | 4 | from charmhelpers.contrib.charmsupport import nrpe | |
389 | 5 | 5 | from charmhelpers.fetch import apt_purge | |
390 | 6 | install_exec_d () { | 6 | from charms import apt, reactive |
391 | 7 | if [[ -d exec.d ]]; then | 7 | |
392 | 8 | shopt -s nullglob | 8 | |
393 | 9 | for f in exec.d/*/charm-pre-install; do | 9 | @reactive.hook('upgrade-charm') |
394 | 10 | [[ -x "$f" ]] || continue | 10 | def upgrade_charm(): |
395 | 11 | ${SHELL} -c "$f"|| { | 11 | hookenv.status_set('maintenance', 'forcing reconfiguration on upgrade-charm') |
396 | 12 | ## bail out if anyone fails | 12 | reactive.clear_flag('jenkins-slave.active') |
397 | 13 | juju-log -l ERROR "$f: returned exit_status=$? " | 13 | reactive.clear_flag('jenkins-slave.installed') |
398 | 14 | } | 14 | |
399 | 15 | done | 15 | |
400 | 16 | shopt -u nullglob | 16 | @reactive.when_not('jenkins-slave.installed') |
401 | 17 | fi | 17 | def install(): |
402 | 18 | } | 18 | hookenv.status_set('maintenance', 'installing jenkins-slave') |
403 | 19 | 19 | reactive.clear_flag('jenkins-slave.active') | |
404 | 20 | # Get rid of the legacy jenkins-slave package, including config files. | 20 | |
405 | 21 | clean_up_old_package () { | 21 | config = hookenv.config() |
406 | 22 | juju-log "Removing the old jenkin-slave package... (obsoleted by this charm)" | 22 | |
407 | 23 | dpkg --purge jenkins-slave | 23 | hookenv.log('Adding jenkins-slave dependencies to be installed') |
408 | 24 | juju-log "Removing the old jenkin-slave package... done." | 24 | packages = ['wget', 'default-jre-headless'] |
409 | 25 | } | 25 | |
410 | 26 | 26 | # Install extra packages needed by the slave. | |
411 | 27 | # Install the slave if it is not installed already. | 27 | tools = config.get('tools') |
412 | 28 | install_slave () { | 28 | if tools: |
413 | 29 | juju-log "Installing jenkins-slave..." | 29 | hookenv.log('Adding jenkins-slave additional tools to be installed: {}'.format(tools)) |
414 | 30 | juju-log "Installing jenkins-slave (dependencies)..." | 30 | for package in tools.split(): |
415 | 31 | apt-get -y install -qq wget adduser default-jre-headless | 31 | packages.append(package) |
416 | 32 | apt.queue_install(packages) | ||
417 | 33 | if not apt.install_queued(): | ||
418 | 34 | return # apt layer already set blocked state. | ||
419 | 35 | |||
420 | 36 | # Get rid of the legacy jenkins-slave package, including config files. | ||
421 | 37 | hookenv.log('Removing the old jenkins-slave package... (obsoleted by this charm)') | ||
422 | 38 | apt_purge(['jenkins-slave']) | ||
423 | 32 | 39 | ||
424 | 33 | # Create jenkins user if it doesn't exist. | 40 | # Create jenkins user if it doesn't exist. |
434 | 34 | if ! id jenkins > /dev/null 2>&1 ; then | 41 | if host.user_exists('jenkins'): |
435 | 35 | juju-log "Installing jenkins-slave (user account)..." | 42 | hookenv.log('Installing jenkins-slave (user account already exists)...') |
436 | 36 | adduser --system --home /var/lib/jenkins --group \ | 43 | else: |
437 | 37 | --disabled-password --quiet --shell /bin/bash \ | 44 | hookenv.log('Installing jenkins-slave (user account)...') |
438 | 38 | jenkins | 45 | host.adduser(username='jenkins', system_user=True, home_dir='/var/lib/jenkins') |
439 | 39 | else | 46 | |
431 | 40 | juju-log "Installing jenkins-slave (user account already exists)..." | ||
432 | 41 | fi | ||
433 | 42 | juju-log "Installing jenkins-slave (directories)..." | ||
440 | 43 | # And ensure required directories exist and are set up. | 47 | # And ensure required directories exist and are set up. |
519 | 44 | mkdir -p /var/lib/jenkins | 48 | hookenv.log('Installing jenkins-slave (directories)...') |
520 | 45 | chown -R jenkins:jenkins /var/lib/jenkins || true | 49 | host.mkdir('/var/lib/jenkins', owner='jenkins', group='jenkins') |
521 | 46 | mkdir -p /var/log/jenkins | 50 | host.mkdir('/var/log/jenkins', owner='jenkins', group='jenkins') |
522 | 47 | chown -R jenkins:jenkins /var/log/jenkins || true | 51 | |
523 | 48 | juju-log "Installing jenkins-slave (common files)..." | 52 | hookenv.log('Installing jenkins-slave (common files)...') |
524 | 49 | install -m 0555 files/download-slave.sh /usr/local/sbin/download-slave.sh | 53 | write_default_conf() |
525 | 50 | # XXX obviously we lose conffile handling here... | 54 | file_to_units('files/download-slave.sh', '/usr/local/sbin/download-slave.sh') |
526 | 51 | install -m 0444 files/jenkins-slave-default /etc/default/jenkins-slave | 55 | file_to_units('files/jenkins-slave-logrotate-config', '/etc/logrotate.d/jenkins-slave') |
527 | 52 | install -m 0444 files/jenkins-slave-logrotate-config /etc/logrotate.d/jenkins-slave | 56 | |
528 | 53 | 57 | if host.lsb_release()['DISTRIB_CODENAME'] == 'trusty': | |
529 | 54 | distro=$(source /etc/lsb-release ; echo $DISTRIB_CODENAME) | 58 | hookenv.log('Installing jenkins-slave (upstart job)...') |
530 | 55 | case $distro in | 59 | file_to_units('files/jenkins-slave-upstart-config', '/etc/init/jenkins-slave.conf') |
531 | 56 | xenial) | 60 | else: |
532 | 57 | # LTS or bust! | 61 | hookenv.log('Installing jenkins-slave (system unit)...') |
533 | 58 | juju-log "Installing jenkins-slave (system unit)..." | 62 | file_to_units('files/jenkins-slave-systemd-config', '/lib/systemd/system/jenkins-slave.service') |
534 | 59 | install -m 0444 files/jenkins-slave-systemd-config /lib/systemd/system/jenkins-slave.service | 63 | host.service('enable', 'jenkins-slave') |
535 | 60 | systemctl enable jenkins-slave | 64 | |
536 | 61 | ;; | 65 | hookenv.log('Installing jenkins-slave... done.') |
537 | 62 | *) | 66 | reactive.clear_flag('jenkins-slave.blocked') |
538 | 63 | # Probably an LTS, and it's not, then too bad. | 67 | reactive.clear_flag('jenkins-slave.configured') |
539 | 64 | juju-log "Installing jenkins-slave (upstart job)..." | 68 | reactive.set_flag('jenkins-slave.installed') |
540 | 65 | install -m 0444 files/jenkins-slave-upstart-config /etc/init/jenkins-slave.conf | 69 | |
541 | 66 | ;; | 70 | |
542 | 67 | esac | 71 | @reactive.when('config.changed') |
543 | 68 | juju-log "Installing jenkins-slave... done." | 72 | def config_changed(): |
544 | 69 | } | 73 | reactive.clear_flag('jenkins-slave.blocked') |
545 | 70 | 74 | reactive.clear_flag('jenkins-slave.configured') | |
546 | 71 | 75 | reactive.clear_flag('nagios-nrpe.configured') | |
547 | 72 | # Install extra packages needed by the slave. | 76 | |
548 | 73 | install_tools () { | 77 | |
549 | 74 | juju-log "Installing tools..." | 78 | @reactive.when('jenkins-slave.installed') |
550 | 75 | apt-get -y install -qq $(config-get tools) | 79 | @reactive.when_not('jenkins-slave.configured') |
551 | 76 | } | 80 | @reactive.when_not('jenkins-slave.blocked') |
552 | 77 | 81 | def configure_jenkins_slave(): | |
553 | 78 | 82 | hookenv.status_set('maintenance', 'configuring jenkins-slave') | |
554 | 79 | # Configure slave | 83 | reactive.clear_flag('jenkins-slave.active') |
555 | 80 | set_up_slave () { | 84 | |
556 | 81 | # If a master_url value is specified, use that to configure the slave. | 85 | config = hookenv.config() |
557 | 82 | master_url="$(config-get master_url)" | 86 | kv = unitdata.kv() |
558 | 83 | if [ -n "${master_url}" ]; then | 87 | |
559 | 84 | juju-log "Using 'master_url' to configure the slave." | 88 | if config.get('master_url'): |
560 | 85 | hooks/configure-slave "${master_url}" | 89 | hookenv.log("Using 'master_url' to configure the slave.") |
561 | 86 | else | 90 | write_default_conf(config.get('master_url')) |
562 | 87 | juju-log "No 'master_url' set; not configuring slave at this time." | 91 | elif kv.get('url'): |
563 | 88 | fi | 92 | hookenv.log("Using url from relation as 'master_url'") |
564 | 89 | } | 93 | write_default_conf(kv.get('url')) |
565 | 90 | 94 | else: | |
566 | 91 | # Execute any hook overlay which may be provided | 95 | hookenv.log("No 'master_url' set; not configuring slave at this time.") |
567 | 92 | # by forks of this charm. | 96 | hookenv.status_set('blocked', "requires either slave relation or 'master_url'") |
568 | 93 | install_extra_hooks () { | 97 | reactive.set_flag('jenkins-slave.blocked') |
569 | 94 | # XXX for canonical_ci_utils.py | 98 | return |
570 | 95 | apt-get -y install -qq python-yaml | 99 | |
571 | 96 | 100 | file_to_units('files/jenkins-slave-sudoers', '/etc/sudoers.d/jenkins', perms=0o440) | |
572 | 97 | juju-log "Installing hooks..." | 101 | |
573 | 98 | if [[ -d hooks/install.d ]] | 102 | reactive.clear_flag('nagios-nrpe.configured') |
574 | 99 | then | 103 | reactive.set_flag('jenkins-slave.configured') |
575 | 100 | for i in $(ls -1 hooks/install.d/*) | 104 | |
576 | 101 | do | 105 | |
577 | 102 | if [[ -x "$i" ]] | 106 | @reactive.when('jenkins-slave.blocked') |
578 | 103 | then | 107 | def blocked_on_jenkins_url(): |
579 | 104 | ./$i | 108 | reactive.clear_flag('jenkins-slave.active') |
580 | 105 | fi | 109 | |
581 | 106 | done | 110 | |
582 | 107 | else | 111 | @reactive.when('jenkins-slave.configured') |
583 | 108 | juju-log "No extra hooks found." | 112 | @reactive.when('nrpe-external-master.available') |
584 | 109 | fi | 113 | @reactive.when('jenkins-slave.active') |
585 | 110 | } | 114 | @reactive.when_not('nagios-nrpe.configured') |
586 | 111 | 115 | def configure_nagios(nagios): | |
587 | 112 | 116 | hookenv.status_set('maintenance', 'setting up NRPE checks') | |
588 | 113 | apt-get update -qq | 117 | |
589 | 114 | install_exec_d | 118 | # Use charmhelpers.contrib.charmsupport's nrpe to determine hostname |
590 | 115 | clean_up_old_package | 119 | hostname = nrpe.get_nagios_hostname() |
591 | 116 | install_slave | 120 | nrpe_setup = nrpe.NRPE(hostname=hostname, primary=True) |
592 | 117 | install_tools | 121 | |
593 | 118 | set_up_slave | 122 | cmd = '/usr/lib/nagios/plugins/check_procs -c 1:1 -a slave.jar' |
594 | 119 | install_extra_hooks | 123 | nrpe_setup.add_check('jenkins_slave_ps', 'Jenkins Slave Process', cmd) |
595 | 120 | 124 | ||
596 | 121 | exit 0 | 125 | nrpe_setup.write() |
597 | 126 | reactive.set_flag('nagios-nrpe.configured') | ||
598 | 127 | |||
599 | 128 | |||
600 | 129 | @reactive.when('jenkins-slave.configured') | ||
601 | 130 | @reactive.when_not('jenkins-slave.active') | ||
602 | 131 | def set_active(): | ||
603 | 132 | # Startup the jenkins-slave service. This is called in the | ||
604 | 133 | # install, config-changed and upgrade-charm paths. It needs to be | ||
605 | 134 | # tolerant of a running or stopped jenkins-slave. | ||
606 | 135 | if host.service_running('jenkins-slave'): | ||
607 | 136 | hookenv.log('Restarting jenkins-slave...') | ||
608 | 137 | host.service_restart('jenkins-slave') | ||
609 | 138 | else: | ||
610 | 139 | hookenv.log('Starting jenkins-slave...') | ||
611 | 140 | host.service_start('jenkins-slave') | ||
612 | 141 | |||
613 | 142 | hookenv.status_set('active', 'ready') | ||
614 | 143 | reactive.set_flag('jenkins-slave.active') | ||
615 | 144 | |||
616 | 145 | |||
617 | 146 | # We can't use interface:jenkins-slave yet as it's not implemented. | ||
618 | 147 | @reactive.hook('slave-relation-joined', 'slave-relation-changed') | ||
619 | 148 | def slave_relation_changed(): | ||
620 | 149 | reactive.set_flag('slave-relation.available') | ||
621 | 150 | reactive.clear_flag('jenkins-slave.blocked') | ||
622 | 151 | reactive.clear_flag('jenkins-slave.configured') | ||
623 | 152 | reactive.clear_flag('slave-relation.configured') | ||
624 | 153 | |||
625 | 154 | |||
626 | 155 | @reactive.hook('slave-relation-departed', 'slave-relation-broken') | ||
627 | 156 | def slave_relation_removed(): | ||
628 | 157 | kv = unitdata.kv() | ||
629 | 158 | kv.set('url', None) | ||
630 | 159 | reactive.clear_flag('slave-relation.available') | ||
631 | 160 | |||
632 | 161 | |||
633 | 162 | @reactive.when('slave-relation.available') | ||
634 | 163 | @reactive.when_not('slave-relation.configured') | ||
635 | 164 | def slave_relation(): | ||
636 | 165 | hookenv.status_set('maintenance', 'setting up jenkins via slave relation') | ||
637 | 166 | config = hookenv.config() | ||
638 | 167 | kv = unitdata.kv() | ||
639 | 168 | |||
640 | 169 | if config.get('master_url'): | ||
641 | 170 | hookenv.log("Config option 'master_url' is set. Can't use slave relation.") | ||
642 | 171 | reactive.set_flag('slave-relation.configured') | ||
643 | 172 | return | ||
644 | 173 | |||
645 | 174 | url = hookenv.relation_get('url') | ||
646 | 175 | if url: | ||
647 | 176 | kv.set('url', url) | ||
648 | 177 | write_default_conf(url) | ||
649 | 178 | else: | ||
650 | 179 | hookenv.log("Master hasn't exported its url yet, exiting...") | ||
651 | 180 | return | ||
652 | 181 | |||
653 | 182 | reactive.clear_flag('jenkins-slave.active') | ||
654 | 183 | |||
655 | 184 | # Set the slave hostname to match the juju unit | ||
656 | 185 | # in the jenkins master instance | ||
657 | 186 | slave_host = hookenv.local_unit().replace('/', '-') | ||
658 | 187 | slave_address = hookenv.unit_private_ip() | ||
659 | 188 | noexecutors = os.cpu_count() | ||
660 | 189 | config_labels = config.get('labels') | ||
661 | 190 | |||
662 | 191 | if config_labels: | ||
663 | 192 | labels = config_labels | ||
664 | 193 | else: | ||
665 | 194 | labels = os.uname()[4] | ||
666 | 195 | |||
667 | 196 | # Set all relations | ||
668 | 197 | hookenv.relation_set(executors=noexecutors) | ||
669 | 198 | hookenv.relation_set(labels=labels) | ||
670 | 199 | hookenv.relation_set(slavehost=slave_host) | ||
671 | 200 | hookenv.relation_set(slaveaddress=slave_address) | ||
672 | 201 | reactive.set_flag('slave-relation.configured') | ||
673 | 202 | |||
674 | 203 | |||
675 | 204 | def file_to_units(local_path, unit_path, perms=None, owner='root', group='root'): | ||
676 | 205 | """ copy a file from the charm onto our unit(s) """ | ||
677 | 206 | file_perms = perms | ||
678 | 207 | if not perms: | ||
679 | 208 | # Let's try manually work it out | ||
680 | 209 | if local_path[-3:] == '.py' or local_path[-3:] == '.sh': | ||
681 | 210 | file_perms = 0o755 | ||
682 | 211 | else: | ||
683 | 212 | file_perms = 0o644 | ||
684 | 213 | |||
685 | 214 | with open(local_path, 'r') as fh: | ||
686 | 215 | host.write_file( | ||
687 | 216 | path=unit_path, | ||
688 | 217 | content=fh.read().encode(), | ||
689 | 218 | owner=owner, | ||
690 | 219 | group=group, | ||
691 | 220 | perms=file_perms, | ||
692 | 221 | ) | ||
693 | 222 | |||
694 | 223 | |||
695 | 224 | def write_default_conf(master_url=None, owner='root', group='root', | ||
696 | 225 | conf_path='/etc/default/jenkins-slave'): | ||
697 | 226 | templates_dir = os.path.join(hookenv.charm_dir(), 'templates') | ||
698 | 227 | slave_host = hookenv.local_unit().replace('/', '-') | ||
699 | 228 | templating.render('jenkins-slave-default', conf_path, | ||
700 | 229 | {'master_url': master_url, 'slave_host': slave_host}, | ||
701 | 230 | owner=owner, group=group, perms=0o444, | ||
702 | 231 | templates_dir=templates_dir) | ||
703 | 122 | 232 | ||
704 | === added file 'requirements.txt' | |||
705 | --- requirements.txt 1970-01-01 00:00:00 +0000 | |||
706 | +++ requirements.txt 2019-03-01 05:39:53 +0000 | |||
707 | @@ -0,0 +1,1 @@ | |||
708 | 1 | # Include python requirements here | ||
709 | 0 | 2 | ||
710 | === removed file 'revision' | |||
711 | --- revision 2015-08-04 16:46:21 +0000 | |||
712 | +++ revision 1970-01-01 00:00:00 +0000 | |||
713 | @@ -1,1 +0,0 @@ | |||
714 | 1 | 9 | ||
715 | 2 | 0 | ||
716 | === added directory 'templates' | |||
717 | === renamed file 'files/jenkins-slave-default' => 'templates/jenkins-slave-default' | |||
718 | --- files/jenkins-slave-default 2016-06-03 03:37:31 +0000 | |||
719 | +++ templates/jenkins-slave-default 2019-03-01 05:39:53 +0000 | |||
720 | @@ -32,12 +32,20 @@ | |||
721 | 32 | # URL of jenkins server to connect to | 32 | # URL of jenkins server to connect to |
722 | 33 | # Not specifying this parameter will stop the slave | 33 | # Not specifying this parameter will stop the slave |
723 | 34 | # job from running. | 34 | # job from running. |
724 | 35 | {% if master_url %} | ||
725 | 36 | JENKINS_URL="{{ master_url }}" | ||
726 | 37 | {% else %} | ||
727 | 35 | #JENKINS_URL="" | 38 | #JENKINS_URL="" |
728 | 39 | {% endif %} | ||
729 | 36 | 40 | ||
730 | 37 | # Name of slave configuration to use at JENKINS_URL | 41 | # Name of slave configuration to use at JENKINS_URL |
731 | 38 | # Override if it need to be something other than the | 42 | # Override if it need to be something other than the |
732 | 39 | # hostname of the server the slave is running on. | 43 | # hostname of the server the slave is running on. |
733 | 44 | {% if slave_host %} | ||
734 | 45 | JENKINS_HOSTNAME="{{ slave_host }}" | ||
735 | 46 | {% else %} | ||
736 | 40 | JENKINS_HOSTNAME="$(hostname)" | 47 | JENKINS_HOSTNAME="$(hostname)" |
737 | 48 | {% endif %} | ||
738 | 41 | 49 | ||
739 | 42 | # Log file location for use in Debian init script | 50 | # Log file location for use in Debian init script |
740 | 43 | JENKINS_SLAVE_LOG=/var/log/jenkins/$NAME.log | 51 | JENKINS_SLAVE_LOG=/var/log/jenkins/$NAME.log |
741 | 44 | 52 | ||
742 | === added directory 'tests' | |||
743 | === added directory 'tests/functional' | |||
744 | === added file 'tests/functional/requirements.txt' | |||
745 | --- tests/functional/requirements.txt 1970-01-01 00:00:00 +0000 | |||
746 | +++ tests/functional/requirements.txt 2019-03-01 05:39:53 +0000 | |||
747 | @@ -0,0 +1,6 @@ | |||
748 | 1 | flake8 | ||
749 | 2 | juju | ||
750 | 3 | mock | ||
751 | 4 | pytest | ||
752 | 5 | pytest-asyncio | ||
753 | 6 | requests | ||
754 | 0 | 7 | ||
755 | === added file 'tests/functional/test_jenkins_slave.py' | |||
756 | --- tests/functional/test_jenkins_slave.py 1970-01-01 00:00:00 +0000 | |||
757 | +++ tests/functional/test_jenkins_slave.py 2019-03-01 05:39:53 +0000 | |||
758 | @@ -0,0 +1,50 @@ | |||
759 | 1 | import os | ||
760 | 2 | import pytest | ||
761 | 3 | from juju.model import Model | ||
762 | 4 | |||
763 | 5 | # Treat tests as coroutines | ||
764 | 6 | pytestmark = pytest.mark.asyncio | ||
765 | 7 | |||
766 | 8 | series = ['bionic'] | ||
767 | 9 | juju_repository = os.getenv('JUJU_REPOSITORY', '.').rstrip('/') | ||
768 | 10 | |||
769 | 11 | |||
770 | 12 | @pytest.fixture | ||
771 | 13 | async def model(): | ||
772 | 14 | model = Model() | ||
773 | 15 | await model.connect_current() | ||
774 | 16 | yield model | ||
775 | 17 | await model.disconnect() | ||
776 | 18 | |||
777 | 19 | |||
778 | 20 | @pytest.fixture | ||
779 | 21 | async def apps(model): | ||
780 | 22 | apps = [] | ||
781 | 23 | for entry in series: | ||
782 | 24 | app = model.applications['jenkins-slave-{}'.format(entry)] | ||
783 | 25 | apps.append(app) | ||
784 | 26 | return apps | ||
785 | 27 | |||
786 | 28 | |||
787 | 29 | @pytest.fixture | ||
788 | 30 | async def units(apps): | ||
789 | 31 | units = [] | ||
790 | 32 | for app in apps: | ||
791 | 33 | units.extend(app.units) | ||
792 | 34 | return units | ||
793 | 35 | |||
794 | 36 | |||
795 | 37 | @pytest.mark.parametrize('series', series) | ||
796 | 38 | async def test_jenkins_slave_deploy(model, series): | ||
797 | 39 | # Starts a deploy for each series | ||
798 | 40 | await model.deploy('{}/builds/jenkins-slave'.format(juju_repository), | ||
799 | 41 | series=series, | ||
800 | 42 | application_name='jenkins-slave-{}'.format(series)) | ||
801 | 43 | assert True | ||
802 | 44 | |||
803 | 45 | |||
804 | 46 | async def test_jenkins_slave_status(apps, model): | ||
805 | 47 | # Verifies status for all deployed series of the charm | ||
806 | 48 | for app in apps: | ||
807 | 49 | await model.block_until(lambda: app.status == 'active') | ||
808 | 50 | assert True | ||
809 | 0 | 51 | ||
810 | === added directory 'tests/unit' | |||
811 | === added directory 'tests/unit/files' | |||
812 | === added file 'tests/unit/files/jenkins-slave-default' | |||
813 | --- tests/unit/files/jenkins-slave-default 1970-01-01 00:00:00 +0000 | |||
814 | +++ tests/unit/files/jenkins-slave-default 2019-03-01 05:39:53 +0000 | |||
815 | @@ -0,0 +1,59 @@ | |||
816 | 1 | # | ||
817 | 2 | # This file is managed by Juju. Attempt no changes here. | ||
818 | 3 | # | ||
819 | 4 | |||
820 | 5 | # defaults for jenkins-slave component of the jenkins continuous integration | ||
821 | 6 | # system | ||
822 | 7 | |||
823 | 8 | # pulled in from the init script; makes things easier. | ||
824 | 9 | NAME=jenkins-slave | ||
825 | 10 | |||
826 | 11 | # location of java | ||
827 | 12 | JAVA=/usr/bin/java | ||
828 | 13 | |||
829 | 14 | # arguments to pass to java - optional | ||
830 | 15 | #JAVA_ARGS="-Xmx256m" | ||
831 | 16 | |||
832 | 17 | # for daemon to use | ||
833 | 18 | PIDFILE=/var/run/jenkins/$NAME.pid | ||
834 | 19 | |||
835 | 20 | # user id to be invoked as (otherwise will run as root; not wise!) | ||
836 | 21 | JENKINS_USER=jenkins | ||
837 | 22 | |||
838 | 23 | # location of jenkins arch indep files | ||
839 | 24 | JENKINS_ROOT=/usr/share/jenkins | ||
840 | 25 | |||
841 | 26 | # jenkins home location | ||
842 | 27 | JENKINS_HOME=/var/lib/jenkins | ||
843 | 28 | |||
844 | 29 | # jenkins /run location | ||
845 | 30 | JENKINS_RUN=/var/run/jenkins | ||
846 | 31 | |||
847 | 32 | # URL of jenkins server to connect to | ||
848 | 33 | # Not specifying this parameter will stop the slave | ||
849 | 34 | # job from running. | ||
850 | 35 | |||
851 | 36 | #JENKINS_URL="" | ||
852 | 37 | |||
853 | 38 | |||
854 | 39 | # Name of slave configuration to use at JENKINS_URL | ||
855 | 40 | # Override if it need to be something other than the | ||
856 | 41 | # hostname of the server the slave is running on. | ||
857 | 42 | |||
858 | 43 | JENKINS_HOSTNAME="jenkins-slave-3" | ||
859 | 44 | |||
860 | 45 | |||
861 | 46 | # Log file location for use in Debian init script | ||
862 | 47 | JENKINS_SLAVE_LOG=/var/log/jenkins/$NAME.log | ||
863 | 48 | |||
864 | 49 | # OS LIMITS SETUP | ||
865 | 50 | # comment this out to observe /etc/security/limits.conf | ||
866 | 51 | # this is on by default because http://github.com/feniix/hudson/commit/d13c08ea8f5a3fa730ba174305e6429b74853927 | ||
867 | 52 | # reported that Ubuntu's PAM configuration doesn't include pam_limits.so, and as a result the # of file | ||
868 | 53 | # descriptors are forced to 1024 regardless of /etc/security/limits.confa | ||
869 | 54 | # NOTE - Ubuntu Users - this is not used by the upstart configuration - please use an upstart overrides file | ||
870 | 55 | # to change the OS limits setup. | ||
871 | 56 | MAXOPENFILES=8192 | ||
872 | 57 | |||
873 | 58 | # Arguments to pass to jenkins slave on startup | ||
874 | 59 | JENKINS_ARGS="-jnlpUrl $JENKINS_URL/computer/$JENKINS_HOSTNAME/slave-agent.jnlp" | ||
875 | 0 | \ No newline at end of file | 60 | \ No newline at end of file |
876 | 1 | 61 | ||
877 | === added file 'tests/unit/files/somefile' | |||
878 | === added file 'tests/unit/files/somefile.py' | |||
879 | === added file 'tests/unit/requirements.txt' | |||
880 | --- tests/unit/requirements.txt 1970-01-01 00:00:00 +0000 | |||
881 | +++ tests/unit/requirements.txt 2019-03-01 05:39:53 +0000 | |||
882 | @@ -0,0 +1,5 @@ | |||
883 | 1 | charmhelpers | ||
884 | 2 | charms.reactive | ||
885 | 3 | mock | ||
886 | 4 | pytest | ||
887 | 5 | pytest-cov | ||
888 | 0 | 6 | ||
889 | === added file 'tests/unit/test_jenkins_slave.py' | |||
890 | --- tests/unit/test_jenkins_slave.py 1970-01-01 00:00:00 +0000 | |||
891 | +++ tests/unit/test_jenkins_slave.py 2019-03-01 05:39:53 +0000 | |||
892 | @@ -0,0 +1,218 @@ | |||
893 | 1 | import grp | ||
894 | 2 | import os | ||
895 | 3 | import pwd | ||
896 | 4 | import shutil | ||
897 | 5 | import sys | ||
898 | 6 | import tempfile | ||
899 | 7 | import unittest | ||
900 | 8 | from unittest import mock | ||
901 | 9 | |||
902 | 10 | sys.modules['charms.apt'] = mock.MagicMock() | ||
903 | 11 | from charms import apt # NOQA: E402 | ||
904 | 12 | |||
905 | 13 | # Add path to where our reactive layer lives and import. | ||
906 | 14 | sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))) | ||
907 | 15 | from reactive.jenkins_slave import ( | ||
908 | 16 | config_changed, | ||
909 | 17 | install, | ||
910 | 18 | configure_jenkins_slave, | ||
911 | 19 | file_to_units, | ||
912 | 20 | write_default_conf, | ||
913 | 21 | ) # NOQA: E402 | ||
914 | 22 | |||
915 | 23 | |||
916 | 24 | INITIAL_CONF = 'tests/unit/files/jenkins-slave-default' | ||
917 | 25 | |||
918 | 26 | |||
919 | 27 | class TestSetDefaultConf(unittest.TestCase): | ||
920 | 28 | def setUp(self): | ||
921 | 29 | self.tmpdir = tempfile.mkdtemp(prefix='charm-unittests-') | ||
922 | 30 | temp_file = tempfile.NamedTemporaryFile(delete=False, dir=self.tmpdir) | ||
923 | 31 | with open(INITIAL_CONF, 'rb') as f: | ||
924 | 32 | conf = f.read().decode('utf-8') | ||
925 | 33 | temp_file.write(conf.encode()) | ||
926 | 34 | temp_file.close() | ||
927 | 35 | self.conf_file = temp_file.name | ||
928 | 36 | self.user = pwd.getpwuid(os.getuid()).pw_name | ||
929 | 37 | self.group = grp.getgrgid(os.getgid()).gr_name | ||
930 | 38 | self.charm_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))) | ||
931 | 39 | |||
932 | 40 | # charmhelpers is getting difficult to test against, as it writes | ||
933 | 41 | # to system directories even for things that should be idempotent, | ||
934 | 42 | # like accessing config options. | ||
935 | 43 | patcher = mock.patch('charmhelpers.core.hookenv.charm_dir') | ||
936 | 44 | self.mock_charm_dir = patcher.start() | ||
937 | 45 | self.addCleanup(patcher.stop) | ||
938 | 46 | self.mock_charm_dir.return_value = self.charm_dir | ||
939 | 47 | |||
940 | 48 | patcher = mock.patch('charmhelpers.core.hookenv.config') | ||
941 | 49 | self.mock_config = patcher.start() | ||
942 | 50 | self.addCleanup(patcher.stop) | ||
943 | 51 | self.mock_config.return_value = {'tools': []} | ||
944 | 52 | |||
945 | 53 | patcher = mock.patch('charmhelpers.core.hookenv.local_unit') | ||
946 | 54 | self.mock_local_unit = patcher.start() | ||
947 | 55 | self.addCleanup(patcher.stop) | ||
948 | 56 | self.mock_local_unit.return_value = 'mock-jenkins-slave/0' | ||
949 | 57 | |||
950 | 58 | patcher = mock.patch('charmhelpers.core.hookenv.log') | ||
951 | 59 | self.mock_log = patcher.start() | ||
952 | 60 | self.addCleanup(patcher.stop) | ||
953 | 61 | self.mock_log.return_value = '' | ||
954 | 62 | |||
955 | 63 | def tearDown(self): | ||
956 | 64 | shutil.rmtree(self.tmpdir) | ||
957 | 65 | |||
958 | 66 | @mock.patch('charms.reactive.clear_flag') | ||
959 | 67 | def test_hook_config_changed(self, clear_flag): | ||
960 | 68 | config_changed() | ||
961 | 69 | expected = [mock.call('jenkins-slave.blocked'), | ||
962 | 70 | mock.call('jenkins-slave.configured'), | ||
963 | 71 | mock.call('nagios-nrpe.configured')] | ||
964 | 72 | self.assertEqual(clear_flag.call_args_list, expected) | ||
965 | 73 | |||
966 | 74 | @mock.patch('charmhelpers.core.host.adduser') | ||
967 | 75 | @mock.patch('charmhelpers.core.host.mkdir') | ||
968 | 76 | @mock.patch('charmhelpers.core.host.service') | ||
969 | 77 | @mock.patch('reactive.jenkins_slave.apt_purge') | ||
970 | 78 | @mock.patch('reactive.jenkins_slave.file_to_units') | ||
971 | 79 | @mock.patch('reactive.jenkins_slave.write_default_conf') | ||
972 | 80 | def test_hook_install(self, write_default_conf, file_to_units, apt_purge, service, mkdir, adduser): | ||
973 | 81 | install() | ||
974 | 82 | expected = [mock.call(home_dir='/var/lib/jenkins', system_user=True, username='jenkins')] | ||
975 | 83 | self.assertEqual(adduser.call_args_list, expected) | ||
976 | 84 | expected = [mock.call('/var/lib/jenkins', group='jenkins', owner='jenkins'), | ||
977 | 85 | mock.call('/var/log/jenkins', group='jenkins', owner='jenkins')] | ||
978 | 86 | self.assertEqual(mkdir.call_args_list, expected) | ||
979 | 87 | self.assertEqual(service.call_args_list, [mock.call('enable', 'jenkins-slave')]) | ||
980 | 88 | self.assertEqual(apt_purge.call_args_list, [mock.call(['jenkins-slave'])]) | ||
981 | 89 | expected = [mock.call('files/download-slave.sh', '/usr/local/sbin/download-slave.sh'), | ||
982 | 90 | mock.call('files/jenkins-slave-logrotate-config', '/etc/logrotate.d/jenkins-slave'), | ||
983 | 91 | mock.call('files/jenkins-slave-systemd-config', '/lib/systemd/system/jenkins-slave.service')] | ||
984 | 92 | self.assertEqual(file_to_units.call_args_list, expected) | ||
985 | 93 | self.assertEqual(write_default_conf.call_args_list, [mock.call()]) | ||
986 | 94 | expected = [mock.call.queue_install(['wget', 'default-jre-headless']), | ||
987 | 95 | mock.call.install_queued()] | ||
988 | 96 | self.assertEqual(apt.method_calls, expected) | ||
989 | 97 | |||
990 | 98 | @mock.patch('charms.reactive.clear_flag') | ||
991 | 99 | @mock.patch('charms.reactive.set_flag') | ||
992 | 100 | @mock.patch('charmhelpers.core.hookenv.config') | ||
993 | 101 | @mock.patch('charmhelpers.core.unitdata.kv') | ||
994 | 102 | @mock.patch('reactive.jenkins_slave.write_default_conf') | ||
995 | 103 | @mock.patch('reactive.jenkins_slave.file_to_units') | ||
996 | 104 | def test_configure_jenkins_slave_no_url(self, file_to_units, write_default_conf, unitdata_kv, config, | ||
997 | 105 | set_flag, clear_flag): | ||
998 | 106 | config.return_value = {} | ||
999 | 107 | unitdata_kv.return_value = {} | ||
1000 | 108 | configure_jenkins_slave() | ||
1001 | 109 | self.assertEqual(write_default_conf.call_args_list, []) | ||
1002 | 110 | self.assertEqual(set_flag.call_args_list, [mock.call('jenkins-slave.blocked')]) | ||
1003 | 111 | self.assertEqual(clear_flag.call_args_list, [mock.call('jenkins-slave.active')]) | ||
1004 | 112 | |||
1005 | 113 | @mock.patch('charms.reactive.clear_flag') | ||
1006 | 114 | @mock.patch('charms.reactive.set_flag') | ||
1007 | 115 | @mock.patch('charmhelpers.core.hookenv.config') | ||
1008 | 116 | @mock.patch('charmhelpers.core.unitdata.kv') | ||
1009 | 117 | @mock.patch('reactive.jenkins_slave.write_default_conf') | ||
1010 | 118 | @mock.patch('reactive.jenkins_slave.file_to_units') | ||
1011 | 119 | def test_configure_jenkins_slave_master_url(self, file_to_units, write_default_conf, unitdata_kv, config, | ||
1012 | 120 | set_flag, clear_flag): | ||
1013 | 121 | config.return_value = {'master_url': 'http://10.1.1.1:8080'} | ||
1014 | 122 | unitdata_kv.return_value = {} | ||
1015 | 123 | configure_jenkins_slave() | ||
1016 | 124 | self.assertEqual(write_default_conf.call_args_list, [mock.call('http://10.1.1.1:8080')]) | ||
1017 | 125 | self.assertEqual(set_flag.call_args_list, [mock.call('jenkins-slave.configured')]) | ||
1018 | 126 | expected = [mock.call('jenkins-slave.active'), mock.call('nagios-nrpe.configured')] | ||
1019 | 127 | self.assertEqual(clear_flag.call_args_list, expected) | ||
1020 | 128 | |||
1021 | 129 | @mock.patch('charms.reactive.clear_flag') | ||
1022 | 130 | @mock.patch('charms.reactive.set_flag') | ||
1023 | 131 | @mock.patch('charmhelpers.core.hookenv.config') | ||
1024 | 132 | @mock.patch('charmhelpers.core.unitdata.kv') | ||
1025 | 133 | @mock.patch('reactive.jenkins_slave.write_default_conf') | ||
1026 | 134 | @mock.patch('reactive.jenkins_slave.file_to_units') | ||
1027 | 135 | def test_configure_jenkins_slave_relation_url(self, file_to_units, write_default_conf, unitdata_kv, config, | ||
1028 | 136 | set_flag, clear_flag): | ||
1029 | 137 | config.return_value = {} | ||
1030 | 138 | unitdata_kv.return_value = {'url': 'http://10.22.22.22:8080'} | ||
1031 | 139 | configure_jenkins_slave() | ||
1032 | 140 | print(write_default_conf.call_args_list) | ||
1033 | 141 | self.assertEqual(write_default_conf.call_args_list, [mock.call('http://10.22.22.22:8080')]) | ||
1034 | 142 | self.assertEqual(set_flag.call_args_list, [mock.call('jenkins-slave.configured')]) | ||
1035 | 143 | expected = [mock.call('jenkins-slave.active'), mock.call('nagios-nrpe.configured')] | ||
1036 | 144 | self.assertEqual(clear_flag.call_args_list, expected) | ||
1037 | 145 | |||
1038 | 146 | def test_write_default_conf_update(self): | ||
1039 | 147 | write_default_conf('http://10.1.1.1:8080', self.user, self.group, self.conf_file) | ||
1040 | 148 | self.assertTrue(conf_match(self.conf_file, 'JENKINS_URL', 'http://10.1.1.1:8080')) | ||
1041 | 149 | |||
1042 | 150 | def test_write_default_conf_reset(self): | ||
1043 | 151 | write_default_conf(None, self.user, self.group, self.conf_file) | ||
1044 | 152 | self.assertTrue(conf_match(self.conf_file, '#JENKINS_URL', '')) | ||
1045 | 153 | |||
1046 | 154 | def test_file_to_units_executable_sh(self): | ||
1047 | 155 | source = os.path.join(self.charm_dir, 'files/download-slave.sh') | ||
1048 | 156 | dest = os.path.join(self.tmpdir, os.path.basename(source)) | ||
1049 | 157 | file_to_units(source, dest, owner=self.user, group=self.group) | ||
1050 | 158 | with open(dest, 'rb') as fh: | ||
1051 | 159 | want = fh.read().decode('utf-8') | ||
1052 | 160 | self.assertTrue(conf_equals(source, want)) | ||
1053 | 161 | self.assertEqual(pwd.getpwuid(os.stat(dest).st_uid).pw_name, self.user) | ||
1054 | 162 | self.assertEqual(grp.getgrgid(os.stat(dest).st_gid).gr_name, self.group) | ||
1055 | 163 | self.assertTrue(os.access(dest, os.X_OK)) | ||
1056 | 164 | |||
1057 | 165 | def test_file_to_units_executable_py(self): | ||
1058 | 166 | source = os.path.join(self.charm_dir, 'tests/unit/files/somefile.py') | ||
1059 | 167 | dest = os.path.join(self.tmpdir, os.path.basename(source)) | ||
1060 | 168 | file_to_units(source, dest, owner=self.user, group=self.group) | ||
1061 | 169 | with open(dest, 'rb') as fh: | ||
1062 | 170 | want = fh.read().decode('utf-8') | ||
1063 | 171 | self.assertTrue(conf_equals(source, want)) | ||
1064 | 172 | self.assertEqual(pwd.getpwuid(os.stat(dest).st_uid).pw_name, self.user) | ||
1065 | 173 | self.assertEqual(grp.getgrgid(os.stat(dest).st_gid).gr_name, self.group) | ||
1066 | 174 | self.assertTrue(os.access(dest, os.X_OK)) | ||
1067 | 175 | |||
1068 | 176 | def test_file_to_units_non_executable(self): | ||
1069 | 177 | source = os.path.join(self.charm_dir, 'files/jenkins-slave-logrotate-config') | ||
1070 | 178 | dest = os.path.join(self.tmpdir, os.path.basename(source)) | ||
1071 | 179 | file_to_units(source, dest, owner=self.user, group=self.group) | ||
1072 | 180 | with open(dest, 'rb') as fh: | ||
1073 | 181 | want = fh.read().decode('utf-8') | ||
1074 | 182 | self.assertTrue(conf_equals(dest, want)) | ||
1075 | 183 | self.assertEqual(pwd.getpwuid(os.stat(dest).st_uid).pw_name, self.user) | ||
1076 | 184 | self.assertEqual(grp.getgrgid(os.stat(dest).st_gid).gr_name, self.group) | ||
1077 | 185 | self.assertFalse(os.access(dest, os.X_OK)) | ||
1078 | 186 | |||
1079 | 187 | def test_file_to_units_non_executable_x_on_disk(self): | ||
1080 | 188 | source = os.path.join(self.charm_dir, 'tests/unit/files/somefile') | ||
1081 | 189 | dest = os.path.join(self.tmpdir, os.path.basename(source)) | ||
1082 | 190 | file_to_units(source, dest, owner=self.user, group=self.group) | ||
1083 | 191 | with open(dest, 'rb') as fh: | ||
1084 | 192 | want = fh.read().decode('utf-8') | ||
1085 | 193 | self.assertTrue(conf_equals(source, want)) | ||
1086 | 194 | self.assertEqual(pwd.getpwuid(os.stat(dest).st_uid).pw_name, self.user) | ||
1087 | 195 | self.assertEqual(grp.getgrgid(os.stat(dest).st_gid).gr_name, self.group) | ||
1088 | 196 | self.assertFalse(os.access(dest, os.X_OK)) | ||
1089 | 197 | |||
1090 | 198 | |||
1091 | 199 | def conf_equals(conf_file, want): | ||
1092 | 200 | with open(conf_file, 'rb') as conf: | ||
1093 | 201 | got = conf.read().decode('utf-8') | ||
1094 | 202 | if got == want: | ||
1095 | 203 | return True | ||
1096 | 204 | print('{}\n != \n{}'.format(got, want)) | ||
1097 | 205 | return False | ||
1098 | 206 | |||
1099 | 207 | |||
1100 | 208 | def conf_match(conf_file, key, value): | ||
1101 | 209 | with open(conf_file, 'rb') as conf: | ||
1102 | 210 | for line in conf.readlines(): | ||
1103 | 211 | line = line.decode('utf-8').rstrip('\n') | ||
1104 | 212 | if line == '{}="{}"'.format(key, value): | ||
1105 | 213 | return True | ||
1106 | 214 | return False | ||
1107 | 215 | |||
1108 | 216 | |||
1109 | 217 | if __name__ == '__main__': | ||
1110 | 218 | unittest.main() | ||
1111 | 0 | 219 | ||
1112 | === added file 'tox.ini' | |||
1113 | --- tox.ini 1970-01-01 00:00:00 +0000 | |||
1114 | +++ tox.ini 2019-03-01 05:39:53 +0000 | |||
1115 | @@ -0,0 +1,37 @@ | |||
1116 | 1 | [tox] | ||
1117 | 2 | skipsdist=True | ||
1118 | 3 | envlist = unit, functional | ||
1119 | 4 | skip_missing_interpreters = True | ||
1120 | 5 | |||
1121 | 6 | [testenv] | ||
1122 | 7 | basepython = python3 | ||
1123 | 8 | setenv = | ||
1124 | 9 | PYTHONPATH = . | ||
1125 | 10 | |||
1126 | 11 | [testenv:unit] | ||
1127 | 12 | commands = pytest -v --ignore {toxinidir}/tests/functional --cov=lib --cov=reactive --cov=actions --cov-report=term | ||
1128 | 13 | deps = -r{toxinidir}/tests/unit/requirements.txt | ||
1129 | 14 | -r{toxinidir}/requirements.txt | ||
1130 | 15 | setenv = PYTHONPATH={toxinidir}/lib | ||
1131 | 16 | |||
1132 | 17 | [testenv:functional] | ||
1133 | 18 | passenv = | ||
1134 | 19 | HOME | ||
1135 | 20 | JUJU_REPOSITORY | ||
1136 | 21 | PATH | ||
1137 | 22 | commands = pytest -v --ignore {toxinidir}/tests/unit | ||
1138 | 23 | deps = -r{toxinidir}/tests/functional/requirements.txt | ||
1139 | 24 | -r{toxinidir}/requirements.txt | ||
1140 | 25 | |||
1141 | 26 | [testenv:lint] | ||
1142 | 27 | commands = flake8 | ||
1143 | 28 | deps = flake8 | ||
1144 | 29 | |||
1145 | 30 | [flake8] | ||
1146 | 31 | exclude = | ||
1147 | 32 | .git, | ||
1148 | 33 | __pycache__, | ||
1149 | 34 | .tox, | ||
1150 | 35 | hooks/install.d/, | ||
1151 | 36 | max-line-length = 120 | ||
1152 | 37 | max-complexity = 10 |
This merge proposal is being monitored by mergebot. Change the status to Approved to merge.