Merge lp:~hloeung/jenkins-agent-charm/reactive-rewrite into lp:jenkins-agent-charm

Proposed by Haw Loeung
Status: Merged
Approved by: Tom Haddon
Approved revision: 51
Merged at revision: 22
Proposed branch: lp:~hloeung/jenkins-agent-charm/reactive-rewrite
Merge into: lp:jenkins-agent-charm
Diff against target: 1152 lines (+673/-311)
25 files modified
.bzrignore (+11/-0)
Makefile (+30/-0)
README.md (+5/-0)
files/jenkins-slave-sudoers (+5/-0)
hooks/configure-slave (+0/-27)
hooks/install.d/add_sudoers (+0/-18)
hooks/install.d/canonical_ci_utils.py (+0/-91)
hooks/nrpe-external-master-relation-changed (+0/-7)
hooks/slave-relation-changed (+0/-19)
hooks/slave-relation-departed (+0/-3)
hooks/slave-relation-joined (+0/-20)
hooks/start (+0/-3)
hooks/stop (+0/-3)
layer.yaml (+5/-0)
metadata.yaml (+5/-1)
reactive/jenkins_slave.py (+228/-118)
requirements.txt (+1/-0)
revision (+0/-1)
templates/jenkins-slave-default (+8/-0)
tests/functional/requirements.txt (+6/-0)
tests/functional/test_jenkins_slave.py (+50/-0)
tests/unit/files/jenkins-slave-default (+59/-0)
tests/unit/requirements.txt (+5/-0)
tests/unit/test_jenkins_slave.py (+218/-0)
tox.ini (+37/-0)
To merge this branch: bzr merge lp:~hloeung/jenkins-agent-charm/reactive-rewrite
Reviewer Review Type Date Requested Status
Tom Haddon Approve
Haw Loeung Needs Resubmitting
Canonical IS Reviewers Pending
Review via email: mp+363649@code.launchpad.net

Commit message

Charm rewrite using Python and the Reactive framework

To post a comment you must log in.
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote :

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
Tom Haddon (mthaddon) wrote :

Some comments and questions inline.

review: Needs Fixing
Revision history for this message
Tom Haddon (mthaddon) wrote :

Missed one other thing as well.

Revision history for this message
Tom Haddon (mthaddon) wrote :

Some comments inline with suggestions for testing.

Revision history for this message
Tom Haddon (mthaddon) wrote :

Er, we we obviously *do* want the max-complexity check

Revision history for this message
Tom Haddon (mthaddon) wrote :

A few comments inline

Revision history for this message
Haw Loeung (hloeung) :
review: Needs Resubmitting
50. By Haw Loeung

Added basic functional test

51. By Haw Loeung

Fixed to use the module itself instead of sys.modules

Revision history for this message
Tom Haddon (mthaddon) wrote :

LGTM, thanks

review: Approve
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote :

Change successfully merged at revision 22

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== added file '.bzrignore'
--- .bzrignore 1970-01-01 00:00:00 +0000
+++ .bzrignore 2019-03-01 05:39:53 +0000
@@ -0,0 +1,11 @@
1*.pyc
2*.swp
3*~
4.coverage
5.pytest_cache/
6.tox/
7.unit-state.db
8__pycache__/
9builds/
10deps/
11revision
012
=== added file 'Makefile'
--- Makefile 1970-01-01 00:00:00 +0000
+++ Makefile 2019-03-01 05:39:53 +0000
@@ -0,0 +1,30 @@
1help:
2 @echo "This project supports the following targets"
3 @echo ""
4 @echo " make help - show this text"
5 @echo " make lint - run flake8"
6 @echo " make test - run the functional test and unittests"
7 @echo " make unittest - run the the unittest"
8 @echo " make functionaltest - run the functional tests"
9 @echo " make clean - remove unneeded files"
10 @echo ""
11
12lint:
13 @echo "Running flake8"
14 @tox -e lint
15
16test: unittest functionaltest lint
17
18unittest:
19 @tox -e unit
20
21functionaltest:
22 @tox -e functional
23
24clean:
25 @echo "Cleaning files"
26 @if [ -d ./.tox ] ; then rm -r ./.tox ; fi
27 @if [ -d ./.pytest_cache ] ; then rm -r ./.pytest_cache ; fi
28
29# The targets below don't depend on a file
30.PHONY: lint test unittest functionaltest clean help
031
=== modified file 'README.md'
--- README.md 2014-05-15 16:06:29 +0000
+++ README.md 2019-03-01 05:39:53 +0000
@@ -21,3 +21,8 @@
21 juju deploy --to <special-mabine-number> jenkins-slave ppc-slave21 juju deploy --to <special-mabine-number> jenkins-slave ppc-slave
2222
23See the Jenkins charm for more details.23See the Jenkins charm for more details.
24
25
26# Notes
27
28We can't use interface:jenkins-slave yet as it's not fully implemented.
2429
=== added directory 'actions'
=== added file 'files/jenkins-slave-sudoers'
--- files/jenkins-slave-sudoers 1970-01-01 00:00:00 +0000
+++ files/jenkins-slave-sudoers 2019-03-01 05:39:53 +0000
@@ -0,0 +1,5 @@
1# Created automatically during charm installation
2# Any manual changes to it will be lost
3
4# User rules for jenkins
5jenkins ALL=(ALL) NOPASSWD: ALL
06
=== removed symlink 'hooks/config-changed'
=== target was u'install'
=== removed file 'hooks/configure-slave'
--- hooks/configure-slave 2016-06-03 02:54:52 +0000
+++ hooks/configure-slave 1970-01-01 00:00:00 +0000
@@ -1,27 +0,0 @@
1#!/bin/bash
2
3set -e
4
5# Grab the jenkins master url as a passed in parameter
6url="$1"
7
8# Set the slave hostname to match the juju unit
9slavehost="$(echo ${JUJU_UNIT_NAME} | sed s,/,-,)"
10
11# Set the slave and url fields
12juju-log "Configuring jenkins-slave with ${url}..."
13sed -i -e "s!^JENKINS_HOSTNAME.*!JENKINS_HOSTNAME=${slavehost}!" \
14 -e "s!^#*JENKINS_URL.*!JENKINS_URL=${url}!" \
15 /etc/default/jenkins-slave
16
17# Startup the jenkins-slave service
18# This is called in the install, config-changed and upgrade-charm paths
19# It needs to be tolerant of a running or stopped jenkins-slave
20status="$(service jenkins-slave status || true)"
21if echo "${status}" | egrep -q "stop|inactive"; then
22 juju-log "Starting jenkins-slave..."
23 service jenkins-slave start
24else
25 juju-log "Restarting jenkins-slave..."
26 service jenkins-slave restart
27fi
280
=== removed symlink 'hooks/install.d/03_nrpe_relation_changed'
=== target was u'canonical_ci_utils.py'
=== removed file 'hooks/install.d/add_sudoers'
--- hooks/install.d/add_sudoers 2015-08-13 18:04:16 +0000
+++ hooks/install.d/add_sudoers 1970-01-01 00:00:00 +0000
@@ -1,18 +0,0 @@
1#!/bin/bash
2
3set -eu
4
5temp_sudoers="$(mktemp /tmp/jenkins.XXXXXX)"
6
7cat > "$temp_sudoers" << EOF
8# Created automatically during charm installation
9# Any manual changes to it will be lost
10
11# User rules for jenkins
12jenkins ALL=(ALL) NOPASSWD: ALL
13EOF
14
15visudo -c -f "$temp_sudoers"
16install -m 440 "$temp_sudoers" /etc/sudoers.d/jenkins
17rm "$temp_sudoers"
18visudo -c
190
=== removed file 'hooks/install.d/canonical_ci_utils.py'
--- hooks/install.d/canonical_ci_utils.py 2015-12-03 10:40:55 +0000
+++ hooks/install.d/canonical_ci_utils.py 1970-01-01 00:00:00 +0000
@@ -1,91 +0,0 @@
1#!/usr/bin/python
2
3import os
4import sys
5
6from charmhelpers.canonical_ci import nrpe
7
8
9from charmhelpers.core.hookenv import (
10 config,
11 local_unit,
12 log,
13 relation_ids,
14 unit_get,
15 ERROR,
16 INFO,
17)
18
19from charmhelpers.core.host import (
20 service_reload,
21)
22
23# Constants set in jenkins/hooks/install.
24JENKINS_HOME = '/var/lib/jenkins'
25JENKINS_USER = 'jenkins'
26JENKINS_GROUP = 'nogroup'
27JENKINS_PORT = 8080
28
29# Local constants
30NRPE_CHECK_PS = "/etc/nagios/nrpe.d/check_jenkins_slave_ps.cfg"
31
32
33def update_nrpe_config():
34 if not relation_ids('nrpe-external-master'):
35 log('No relation to an nrpe-external-master, not configuring '
36 'nagios.')
37 return
38 with open(NRPE_CHECK_PS, "w") as conf:
39 log("Writing config: %s." % NRPE_CHECK_PS, INFO)
40 conf.write(nrpe.CONF_HEADER)
41 conf.write(
42 "command[check_jenkins_slave_ps]="
43 "/usr/lib/nagios/plugins/check_procs -c 1:1 -a slave.jar")
44
45 # config
46 unit_name = local_unit().replace('/', '-')
47 n_hostname = "%s-%s" % (config("nagios_context"), unit_name)
48 n_servicegroup = config("nagios_servicegroups") if config("nagios_servicegroups") else config("nagios_context")
49 service_file = (
50 '/var/lib/nagios/export/service__%s_check_jenkins_slave.cfg' %
51 n_hostname)
52
53 # There is a race between this code and the nrpe_external_master install
54 # hook. Both need to write files to /var/lib/nagios/export, but only one
55 # needs to create it. So tolerate failure if the directory exists.
56 try:
57 os.makedirs(os.path.dirname(service_file))
58 except os.error:
59 pass
60
61 with open(service_file, "w") as conf:
62 log("Writing config: %s." % conf, INFO)
63 conf.write(nrpe.CONF_HEADER)
64 conf.write(nrpe.NRPE_SERVICE_ENTRY % {
65 'nagios_hostname': n_hostname,
66 'check_name': 'check_jenkins_slave_ps',
67 'nagios_servicegroup': n_servicegroup,
68 })
69
70 # reboot
71 if os.path.isfile('/etc/init.d/nagios-nrpe-server'):
72 service_reload('nagios-nrpe-server')
73
74
75hooks = {
76 '03_nrpe_relation_changed': update_nrpe_config
77}
78
79
80def main(hook):
81 try:
82 hooks[hook]()
83 except KeyError:
84 e = 'Invalid install.d hook: %s.' % hook
85 log(e, ERROR)
86 raise Exception(e)
87
88
89if __name__ == '__main__':
90 if os.path.islink(sys.argv[0]):
91 main(os.path.basename(sys.argv[0]))
920
=== removed file 'hooks/nrpe-external-master-relation-changed'
--- hooks/nrpe-external-master-relation-changed 2015-08-19 20:00:15 +0000
+++ hooks/nrpe-external-master-relation-changed 1970-01-01 00:00:00 +0000
@@ -1,7 +0,0 @@
1#!/bin/sh
2set -e
3
4home=`dirname $0`
5
6juju-log "Running install hook once more to pick up new NRPE relation changes"
7exec $home/install
80
=== removed symlink 'hooks/slave-relation-broken'
=== target was u'slave-relation-departed'
=== removed file 'hooks/slave-relation-changed'
--- hooks/slave-relation-changed 2015-08-05 13:20:44 +0000
+++ hooks/slave-relation-changed 1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
1#!/bin/bash
2
3set -e
4
5# Setup connection to master instance once set
6url="$(relation-get url)"
7
8if [ "x$url" = "x" ]; then
9 juju-log "Master hasn't exported its url yet, exiting..."
10 exit 0
11fi
12
13master_url="$(config-get master_url)"
14if [ -n "${master_url}" ]; then
15 juju-log "Config option 'master_url' is set. Can't use slave relation."
16 exit 0
17fi
18
19hooks/configure-slave "${url}"
200
=== removed file 'hooks/slave-relation-departed'
--- hooks/slave-relation-departed 2012-07-27 11:23:24 +0000
+++ hooks/slave-relation-departed 1970-01-01 00:00:00 +0000
@@ -1,3 +0,0 @@
1#!/bin/bash
2
3stop jenkins-slave || true
40
=== removed file 'hooks/slave-relation-joined'
--- hooks/slave-relation-joined 2015-08-04 16:46:21 +0000
+++ hooks/slave-relation-joined 1970-01-01 00:00:00 +0000
@@ -1,20 +0,0 @@
1#!/bin/bash
2
3set -e
4
5# Set the slave hostname to match the juju unit
6# in the jenkins master instance
7slavehost=`echo ${JUJU_UNIT_NAME} | sed s,/,-,`
8noexecutors=`cat /proc/cpuinfo | grep processor | wc -l`
9config_labels=`config-get labels`
10labels=`uname -p`
11
12if [ -n "$config_labels" ]; then
13 labels=$config_labels
14fi
15
16# Set all relations
17relation-set executors=$noexecutors
18relation-set labels="$labels"
19relation-set slavehost=$slavehost
20relation-set slaveaddress=`unit-get private-address`
210
=== removed file 'hooks/start'
--- hooks/start 2011-08-02 08:53:54 +0000
+++ hooks/start 1970-01-01 00:00:00 +0000
@@ -1,3 +0,0 @@
1#!/bin/bash
2
3start jenkins-slave || true
40
=== removed file 'hooks/stop'
--- hooks/stop 2011-08-02 08:53:54 +0000
+++ hooks/stop 1970-01-01 00:00:00 +0000
@@ -1,3 +0,0 @@
1#!/bin/bash
2
3stop jenkins-slave
40
=== removed symlink 'hooks/upgrade-charm'
=== target was u'install'
=== added file 'layer.yaml'
--- layer.yaml 1970-01-01 00:00:00 +0000
+++ layer.yaml 2019-03-01 05:39:53 +0000
@@ -0,0 +1,5 @@
1includes:
2 - layer:basic
3 - layer:apt
4 - layer:nagios
5repo: lp:jenkins-slave-charm
06
=== added directory 'lib'
=== modified file 'metadata.yaml'
--- metadata.yaml 2019-02-18 00:41:09 +0000
+++ metadata.yaml 2019-03-01 05:39:53 +0000
@@ -9,8 +9,12 @@
9 This charm provides support for jenkins slaves9 This charm provides support for jenkins slaves
10 .10 .
11 https://launchpad.net/jenkins-slave-charm11 https://launchpad.net/jenkins-slave-charm
12categories:12tags:
13 - applications13 - applications
14series:
15 - bionic
16 - xenial
17 - trusty
14provides:18provides:
15 slave:19 slave:
16 interface: jenkins-slave20 interface: jenkins-slave
1721
=== added directory 'reactive'
=== renamed file 'hooks/install' => 'reactive/jenkins_slave.py'
--- hooks/install 2016-06-14 01:04:43 +0000
+++ reactive/jenkins_slave.py 2019-03-01 05:39:53 +0000
@@ -1,121 +1,231 @@
1#!/bin/bash1import os
22
3set -eu3from charmhelpers.core import hookenv, host, templating, unitdata
44from charmhelpers.contrib.charmsupport import nrpe
55from charmhelpers.fetch import apt_purge
6install_exec_d () {6from charms import apt, reactive
7 if [[ -d exec.d ]]; then7
8 shopt -s nullglob8
9 for f in exec.d/*/charm-pre-install; do9@reactive.hook('upgrade-charm')
10 [[ -x "$f" ]] || continue10def upgrade_charm():
11 ${SHELL} -c "$f"|| {11 hookenv.status_set('maintenance', 'forcing reconfiguration on upgrade-charm')
12 ## bail out if anyone fails12 reactive.clear_flag('jenkins-slave.active')
13 juju-log -l ERROR "$f: returned exit_status=$? "13 reactive.clear_flag('jenkins-slave.installed')
14 }14
15 done15
16 shopt -u nullglob16@reactive.when_not('jenkins-slave.installed')
17 fi17def install():
18}18 hookenv.status_set('maintenance', 'installing jenkins-slave')
1919 reactive.clear_flag('jenkins-slave.active')
20# Get rid of the legacy jenkins-slave package, including config files.20
21clean_up_old_package () {21 config = hookenv.config()
22 juju-log "Removing the old jenkin-slave package... (obsoleted by this charm)"22
23 dpkg --purge jenkins-slave23 hookenv.log('Adding jenkins-slave dependencies to be installed')
24 juju-log "Removing the old jenkin-slave package... done."24 packages = ['wget', 'default-jre-headless']
25}25
2626 # Install extra packages needed by the slave.
27# Install the slave if it is not installed already.27 tools = config.get('tools')
28install_slave () {28 if tools:
29 juju-log "Installing jenkins-slave..."29 hookenv.log('Adding jenkins-slave additional tools to be installed: {}'.format(tools))
30 juju-log "Installing jenkins-slave (dependencies)..."30 for package in tools.split():
31 apt-get -y install -qq wget adduser default-jre-headless31 packages.append(package)
32 apt.queue_install(packages)
33 if not apt.install_queued():
34 return # apt layer already set blocked state.
35
36 # Get rid of the legacy jenkins-slave package, including config files.
37 hookenv.log('Removing the old jenkins-slave package... (obsoleted by this charm)')
38 apt_purge(['jenkins-slave'])
3239
33 # Create jenkins user if it doesn't exist.40 # Create jenkins user if it doesn't exist.
34 if ! id jenkins > /dev/null 2>&1 ; then41 if host.user_exists('jenkins'):
35 juju-log "Installing jenkins-slave (user account)..."42 hookenv.log('Installing jenkins-slave (user account already exists)...')
36 adduser --system --home /var/lib/jenkins --group \43 else:
37 --disabled-password --quiet --shell /bin/bash \44 hookenv.log('Installing jenkins-slave (user account)...')
38 jenkins 45 host.adduser(username='jenkins', system_user=True, home_dir='/var/lib/jenkins')
39 else46
40 juju-log "Installing jenkins-slave (user account already exists)..."
41 fi
42 juju-log "Installing jenkins-slave (directories)..."
43 # And ensure required directories exist and are set up.47 # And ensure required directories exist and are set up.
44 mkdir -p /var/lib/jenkins48 hookenv.log('Installing jenkins-slave (directories)...')
45 chown -R jenkins:jenkins /var/lib/jenkins || true49 host.mkdir('/var/lib/jenkins', owner='jenkins', group='jenkins')
46 mkdir -p /var/log/jenkins50 host.mkdir('/var/log/jenkins', owner='jenkins', group='jenkins')
47 chown -R jenkins:jenkins /var/log/jenkins || true51
48 juju-log "Installing jenkins-slave (common files)..."52 hookenv.log('Installing jenkins-slave (common files)...')
49 install -m 0555 files/download-slave.sh /usr/local/sbin/download-slave.sh53 write_default_conf()
50 # XXX obviously we lose conffile handling here...54 file_to_units('files/download-slave.sh', '/usr/local/sbin/download-slave.sh')
51 install -m 0444 files/jenkins-slave-default /etc/default/jenkins-slave55 file_to_units('files/jenkins-slave-logrotate-config', '/etc/logrotate.d/jenkins-slave')
52 install -m 0444 files/jenkins-slave-logrotate-config /etc/logrotate.d/jenkins-slave56
5357 if host.lsb_release()['DISTRIB_CODENAME'] == 'trusty':
54 distro=$(source /etc/lsb-release ; echo $DISTRIB_CODENAME)58 hookenv.log('Installing jenkins-slave (upstart job)...')
55 case $distro in59 file_to_units('files/jenkins-slave-upstart-config', '/etc/init/jenkins-slave.conf')
56 xenial)60 else:
57 # LTS or bust!61 hookenv.log('Installing jenkins-slave (system unit)...')
58 juju-log "Installing jenkins-slave (system unit)..."62 file_to_units('files/jenkins-slave-systemd-config', '/lib/systemd/system/jenkins-slave.service')
59 install -m 0444 files/jenkins-slave-systemd-config /lib/systemd/system/jenkins-slave.service63 host.service('enable', 'jenkins-slave')
60 systemctl enable jenkins-slave64
61 ;;65 hookenv.log('Installing jenkins-slave... done.')
62 *)66 reactive.clear_flag('jenkins-slave.blocked')
63 # Probably an LTS, and it's not, then too bad.67 reactive.clear_flag('jenkins-slave.configured')
64 juju-log "Installing jenkins-slave (upstart job)..."68 reactive.set_flag('jenkins-slave.installed')
65 install -m 0444 files/jenkins-slave-upstart-config /etc/init/jenkins-slave.conf69
66 ;;70
67 esac71@reactive.when('config.changed')
68 juju-log "Installing jenkins-slave... done."72def config_changed():
69}73 reactive.clear_flag('jenkins-slave.blocked')
7074 reactive.clear_flag('jenkins-slave.configured')
7175 reactive.clear_flag('nagios-nrpe.configured')
72# Install extra packages needed by the slave.76
73install_tools () {77
74 juju-log "Installing tools..."78@reactive.when('jenkins-slave.installed')
75 apt-get -y install -qq $(config-get tools)79@reactive.when_not('jenkins-slave.configured')
76}80@reactive.when_not('jenkins-slave.blocked')
7781def configure_jenkins_slave():
7882 hookenv.status_set('maintenance', 'configuring jenkins-slave')
79# Configure slave83 reactive.clear_flag('jenkins-slave.active')
80set_up_slave () {84
81 # If a master_url value is specified, use that to configure the slave.85 config = hookenv.config()
82 master_url="$(config-get master_url)"86 kv = unitdata.kv()
83 if [ -n "${master_url}" ]; then87
84 juju-log "Using 'master_url' to configure the slave."88 if config.get('master_url'):
85 hooks/configure-slave "${master_url}"89 hookenv.log("Using 'master_url' to configure the slave.")
86 else90 write_default_conf(config.get('master_url'))
87 juju-log "No 'master_url' set; not configuring slave at this time."91 elif kv.get('url'):
88 fi92 hookenv.log("Using url from relation as 'master_url'")
89}93 write_default_conf(kv.get('url'))
9094 else:
91# Execute any hook overlay which may be provided95 hookenv.log("No 'master_url' set; not configuring slave at this time.")
92# by forks of this charm.96 hookenv.status_set('blocked', "requires either slave relation or 'master_url'")
93install_extra_hooks () {97 reactive.set_flag('jenkins-slave.blocked')
94 # XXX for canonical_ci_utils.py98 return
95 apt-get -y install -qq python-yaml99
96100 file_to_units('files/jenkins-slave-sudoers', '/etc/sudoers.d/jenkins', perms=0o440)
97 juju-log "Installing hooks..."101
98 if [[ -d hooks/install.d ]]102 reactive.clear_flag('nagios-nrpe.configured')
99 then103 reactive.set_flag('jenkins-slave.configured')
100 for i in $(ls -1 hooks/install.d/*)104
101 do105
102 if [[ -x "$i" ]]106@reactive.when('jenkins-slave.blocked')
103 then107def blocked_on_jenkins_url():
104 ./$i108 reactive.clear_flag('jenkins-slave.active')
105 fi109
106 done110
107 else111@reactive.when('jenkins-slave.configured')
108 juju-log "No extra hooks found."112@reactive.when('nrpe-external-master.available')
109 fi113@reactive.when('jenkins-slave.active')
110}114@reactive.when_not('nagios-nrpe.configured')
111115def configure_nagios(nagios):
112116 hookenv.status_set('maintenance', 'setting up NRPE checks')
113apt-get update -qq117
114install_exec_d118 # Use charmhelpers.contrib.charmsupport's nrpe to determine hostname
115clean_up_old_package119 hostname = nrpe.get_nagios_hostname()
116install_slave120 nrpe_setup = nrpe.NRPE(hostname=hostname, primary=True)
117install_tools121
118set_up_slave122 cmd = '/usr/lib/nagios/plugins/check_procs -c 1:1 -a slave.jar'
119install_extra_hooks123 nrpe_setup.add_check('jenkins_slave_ps', 'Jenkins Slave Process', cmd)
120124
121exit 0125 nrpe_setup.write()
126 reactive.set_flag('nagios-nrpe.configured')
127
128
129@reactive.when('jenkins-slave.configured')
130@reactive.when_not('jenkins-slave.active')
131def set_active():
132 # Startup the jenkins-slave service. This is called in the
133 # install, config-changed and upgrade-charm paths. It needs to be
134 # tolerant of a running or stopped jenkins-slave.
135 if host.service_running('jenkins-slave'):
136 hookenv.log('Restarting jenkins-slave...')
137 host.service_restart('jenkins-slave')
138 else:
139 hookenv.log('Starting jenkins-slave...')
140 host.service_start('jenkins-slave')
141
142 hookenv.status_set('active', 'ready')
143 reactive.set_flag('jenkins-slave.active')
144
145
146# We can't use interface:jenkins-slave yet as it's not implemented.
147@reactive.hook('slave-relation-joined', 'slave-relation-changed')
148def slave_relation_changed():
149 reactive.set_flag('slave-relation.available')
150 reactive.clear_flag('jenkins-slave.blocked')
151 reactive.clear_flag('jenkins-slave.configured')
152 reactive.clear_flag('slave-relation.configured')
153
154
155@reactive.hook('slave-relation-departed', 'slave-relation-broken')
156def slave_relation_removed():
157 kv = unitdata.kv()
158 kv.set('url', None)
159 reactive.clear_flag('slave-relation.available')
160
161
162@reactive.when('slave-relation.available')
163@reactive.when_not('slave-relation.configured')
164def slave_relation():
165 hookenv.status_set('maintenance', 'setting up jenkins via slave relation')
166 config = hookenv.config()
167 kv = unitdata.kv()
168
169 if config.get('master_url'):
170 hookenv.log("Config option 'master_url' is set. Can't use slave relation.")
171 reactive.set_flag('slave-relation.configured')
172 return
173
174 url = hookenv.relation_get('url')
175 if url:
176 kv.set('url', url)
177 write_default_conf(url)
178 else:
179 hookenv.log("Master hasn't exported its url yet, exiting...")
180 return
181
182 reactive.clear_flag('jenkins-slave.active')
183
184 # Set the slave hostname to match the juju unit
185 # in the jenkins master instance
186 slave_host = hookenv.local_unit().replace('/', '-')
187 slave_address = hookenv.unit_private_ip()
188 noexecutors = os.cpu_count()
189 config_labels = config.get('labels')
190
191 if config_labels:
192 labels = config_labels
193 else:
194 labels = os.uname()[4]
195
196 # Set all relations
197 hookenv.relation_set(executors=noexecutors)
198 hookenv.relation_set(labels=labels)
199 hookenv.relation_set(slavehost=slave_host)
200 hookenv.relation_set(slaveaddress=slave_address)
201 reactive.set_flag('slave-relation.configured')
202
203
204def file_to_units(local_path, unit_path, perms=None, owner='root', group='root'):
205 """ copy a file from the charm onto our unit(s) """
206 file_perms = perms
207 if not perms:
208 # Let's try manually work it out
209 if local_path[-3:] == '.py' or local_path[-3:] == '.sh':
210 file_perms = 0o755
211 else:
212 file_perms = 0o644
213
214 with open(local_path, 'r') as fh:
215 host.write_file(
216 path=unit_path,
217 content=fh.read().encode(),
218 owner=owner,
219 group=group,
220 perms=file_perms,
221 )
222
223
224def write_default_conf(master_url=None, owner='root', group='root',
225 conf_path='/etc/default/jenkins-slave'):
226 templates_dir = os.path.join(hookenv.charm_dir(), 'templates')
227 slave_host = hookenv.local_unit().replace('/', '-')
228 templating.render('jenkins-slave-default', conf_path,
229 {'master_url': master_url, 'slave_host': slave_host},
230 owner=owner, group=group, perms=0o444,
231 templates_dir=templates_dir)
122232
=== added file 'requirements.txt'
--- requirements.txt 1970-01-01 00:00:00 +0000
+++ requirements.txt 2019-03-01 05:39:53 +0000
@@ -0,0 +1,1 @@
1# Include python requirements here
02
=== removed file 'revision'
--- revision 2015-08-04 16:46:21 +0000
+++ revision 1970-01-01 00:00:00 +0000
@@ -1,1 +0,0 @@
19
20
=== added directory 'templates'
=== renamed file 'files/jenkins-slave-default' => 'templates/jenkins-slave-default'
--- files/jenkins-slave-default 2016-06-03 03:37:31 +0000
+++ templates/jenkins-slave-default 2019-03-01 05:39:53 +0000
@@ -32,12 +32,20 @@
32# URL of jenkins server to connect to 32# URL of jenkins server to connect to
33# Not specifying this parameter will stop the slave 33# Not specifying this parameter will stop the slave
34# job from running.34# job from running.
35{% if master_url %}
36JENKINS_URL="{{ master_url }}"
37{% else %}
35#JENKINS_URL=""38#JENKINS_URL=""
39{% endif %}
3640
37# Name of slave configuration to use at JENKINS_URL41# Name of slave configuration to use at JENKINS_URL
38# Override if it need to be something other than the42# Override if it need to be something other than the
39# hostname of the server the slave is running on.43# hostname of the server the slave is running on.
44{% if slave_host %}
45JENKINS_HOSTNAME="{{ slave_host }}"
46{% else %}
40JENKINS_HOSTNAME="$(hostname)"47JENKINS_HOSTNAME="$(hostname)"
48{% endif %}
4149
42# Log file location for use in Debian init script50# Log file location for use in Debian init script
43JENKINS_SLAVE_LOG=/var/log/jenkins/$NAME.log51JENKINS_SLAVE_LOG=/var/log/jenkins/$NAME.log
4452
=== added directory 'tests'
=== added directory 'tests/functional'
=== added file 'tests/functional/requirements.txt'
--- tests/functional/requirements.txt 1970-01-01 00:00:00 +0000
+++ tests/functional/requirements.txt 2019-03-01 05:39:53 +0000
@@ -0,0 +1,6 @@
1flake8
2juju
3mock
4pytest
5pytest-asyncio
6requests
07
=== added file 'tests/functional/test_jenkins_slave.py'
--- tests/functional/test_jenkins_slave.py 1970-01-01 00:00:00 +0000
+++ tests/functional/test_jenkins_slave.py 2019-03-01 05:39:53 +0000
@@ -0,0 +1,50 @@
1import os
2import pytest
3from juju.model import Model
4
5# Treat tests as coroutines
6pytestmark = pytest.mark.asyncio
7
8series = ['bionic']
9juju_repository = os.getenv('JUJU_REPOSITORY', '.').rstrip('/')
10
11
12@pytest.fixture
13async def model():
14 model = Model()
15 await model.connect_current()
16 yield model
17 await model.disconnect()
18
19
20@pytest.fixture
21async def apps(model):
22 apps = []
23 for entry in series:
24 app = model.applications['jenkins-slave-{}'.format(entry)]
25 apps.append(app)
26 return apps
27
28
29@pytest.fixture
30async def units(apps):
31 units = []
32 for app in apps:
33 units.extend(app.units)
34 return units
35
36
37@pytest.mark.parametrize('series', series)
38async def test_jenkins_slave_deploy(model, series):
39 # Starts a deploy for each series
40 await model.deploy('{}/builds/jenkins-slave'.format(juju_repository),
41 series=series,
42 application_name='jenkins-slave-{}'.format(series))
43 assert True
44
45
46async def test_jenkins_slave_status(apps, model):
47 # Verifies status for all deployed series of the charm
48 for app in apps:
49 await model.block_until(lambda: app.status == 'active')
50 assert True
051
=== added directory 'tests/unit'
=== added directory 'tests/unit/files'
=== added file 'tests/unit/files/jenkins-slave-default'
--- tests/unit/files/jenkins-slave-default 1970-01-01 00:00:00 +0000
+++ tests/unit/files/jenkins-slave-default 2019-03-01 05:39:53 +0000
@@ -0,0 +1,59 @@
1#
2# This file is managed by Juju. Attempt no changes here.
3#
4
5# defaults for jenkins-slave component of the jenkins continuous integration
6# system
7
8# pulled in from the init script; makes things easier.
9NAME=jenkins-slave
10
11# location of java
12JAVA=/usr/bin/java
13
14# arguments to pass to java - optional
15#JAVA_ARGS="-Xmx256m"
16
17# for daemon to use
18PIDFILE=/var/run/jenkins/$NAME.pid
19
20# user id to be invoked as (otherwise will run as root; not wise!)
21JENKINS_USER=jenkins
22
23# location of jenkins arch indep files
24JENKINS_ROOT=/usr/share/jenkins
25
26# jenkins home location
27JENKINS_HOME=/var/lib/jenkins
28
29# jenkins /run location
30JENKINS_RUN=/var/run/jenkins
31
32# URL of jenkins server to connect to
33# Not specifying this parameter will stop the slave
34# job from running.
35
36#JENKINS_URL=""
37
38
39# Name of slave configuration to use at JENKINS_URL
40# Override if it need to be something other than the
41# hostname of the server the slave is running on.
42
43JENKINS_HOSTNAME="jenkins-slave-3"
44
45
46# Log file location for use in Debian init script
47JENKINS_SLAVE_LOG=/var/log/jenkins/$NAME.log
48
49# OS LIMITS SETUP
50# comment this out to observe /etc/security/limits.conf
51# this is on by default because http://github.com/feniix/hudson/commit/d13c08ea8f5a3fa730ba174305e6429b74853927
52# reported that Ubuntu's PAM configuration doesn't include pam_limits.so, and as a result the # of file
53# descriptors are forced to 1024 regardless of /etc/security/limits.confa
54# NOTE - Ubuntu Users - this is not used by the upstart configuration - please use an upstart overrides file
55# to change the OS limits setup.
56MAXOPENFILES=8192
57
58# Arguments to pass to jenkins slave on startup
59JENKINS_ARGS="-jnlpUrl $JENKINS_URL/computer/$JENKINS_HOSTNAME/slave-agent.jnlp"
0\ No newline at end of file60\ No newline at end of file
161
=== added file 'tests/unit/files/somefile'
=== added file 'tests/unit/files/somefile.py'
=== added file 'tests/unit/requirements.txt'
--- tests/unit/requirements.txt 1970-01-01 00:00:00 +0000
+++ tests/unit/requirements.txt 2019-03-01 05:39:53 +0000
@@ -0,0 +1,5 @@
1charmhelpers
2charms.reactive
3mock
4pytest
5pytest-cov
06
=== added file 'tests/unit/test_jenkins_slave.py'
--- tests/unit/test_jenkins_slave.py 1970-01-01 00:00:00 +0000
+++ tests/unit/test_jenkins_slave.py 2019-03-01 05:39:53 +0000
@@ -0,0 +1,218 @@
1import grp
2import os
3import pwd
4import shutil
5import sys
6import tempfile
7import unittest
8from unittest import mock
9
10sys.modules['charms.apt'] = mock.MagicMock()
11from charms import apt # NOQA: E402
12
13# Add path to where our reactive layer lives and import.
14sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__)))))
15from reactive.jenkins_slave import (
16 config_changed,
17 install,
18 configure_jenkins_slave,
19 file_to_units,
20 write_default_conf,
21) # NOQA: E402
22
23
24INITIAL_CONF = 'tests/unit/files/jenkins-slave-default'
25
26
27class TestSetDefaultConf(unittest.TestCase):
28 def setUp(self):
29 self.tmpdir = tempfile.mkdtemp(prefix='charm-unittests-')
30 temp_file = tempfile.NamedTemporaryFile(delete=False, dir=self.tmpdir)
31 with open(INITIAL_CONF, 'rb') as f:
32 conf = f.read().decode('utf-8')
33 temp_file.write(conf.encode())
34 temp_file.close()
35 self.conf_file = temp_file.name
36 self.user = pwd.getpwuid(os.getuid()).pw_name
37 self.group = grp.getgrgid(os.getgid()).gr_name
38 self.charm_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))
39
40 # charmhelpers is getting difficult to test against, as it writes
41 # to system directories even for things that should be idempotent,
42 # like accessing config options.
43 patcher = mock.patch('charmhelpers.core.hookenv.charm_dir')
44 self.mock_charm_dir = patcher.start()
45 self.addCleanup(patcher.stop)
46 self.mock_charm_dir.return_value = self.charm_dir
47
48 patcher = mock.patch('charmhelpers.core.hookenv.config')
49 self.mock_config = patcher.start()
50 self.addCleanup(patcher.stop)
51 self.mock_config.return_value = {'tools': []}
52
53 patcher = mock.patch('charmhelpers.core.hookenv.local_unit')
54 self.mock_local_unit = patcher.start()
55 self.addCleanup(patcher.stop)
56 self.mock_local_unit.return_value = 'mock-jenkins-slave/0'
57
58 patcher = mock.patch('charmhelpers.core.hookenv.log')
59 self.mock_log = patcher.start()
60 self.addCleanup(patcher.stop)
61 self.mock_log.return_value = ''
62
63 def tearDown(self):
64 shutil.rmtree(self.tmpdir)
65
66 @mock.patch('charms.reactive.clear_flag')
67 def test_hook_config_changed(self, clear_flag):
68 config_changed()
69 expected = [mock.call('jenkins-slave.blocked'),
70 mock.call('jenkins-slave.configured'),
71 mock.call('nagios-nrpe.configured')]
72 self.assertEqual(clear_flag.call_args_list, expected)
73
74 @mock.patch('charmhelpers.core.host.adduser')
75 @mock.patch('charmhelpers.core.host.mkdir')
76 @mock.patch('charmhelpers.core.host.service')
77 @mock.patch('reactive.jenkins_slave.apt_purge')
78 @mock.patch('reactive.jenkins_slave.file_to_units')
79 @mock.patch('reactive.jenkins_slave.write_default_conf')
80 def test_hook_install(self, write_default_conf, file_to_units, apt_purge, service, mkdir, adduser):
81 install()
82 expected = [mock.call(home_dir='/var/lib/jenkins', system_user=True, username='jenkins')]
83 self.assertEqual(adduser.call_args_list, expected)
84 expected = [mock.call('/var/lib/jenkins', group='jenkins', owner='jenkins'),
85 mock.call('/var/log/jenkins', group='jenkins', owner='jenkins')]
86 self.assertEqual(mkdir.call_args_list, expected)
87 self.assertEqual(service.call_args_list, [mock.call('enable', 'jenkins-slave')])
88 self.assertEqual(apt_purge.call_args_list, [mock.call(['jenkins-slave'])])
89 expected = [mock.call('files/download-slave.sh', '/usr/local/sbin/download-slave.sh'),
90 mock.call('files/jenkins-slave-logrotate-config', '/etc/logrotate.d/jenkins-slave'),
91 mock.call('files/jenkins-slave-systemd-config', '/lib/systemd/system/jenkins-slave.service')]
92 self.assertEqual(file_to_units.call_args_list, expected)
93 self.assertEqual(write_default_conf.call_args_list, [mock.call()])
94 expected = [mock.call.queue_install(['wget', 'default-jre-headless']),
95 mock.call.install_queued()]
96 self.assertEqual(apt.method_calls, expected)
97
98 @mock.patch('charms.reactive.clear_flag')
99 @mock.patch('charms.reactive.set_flag')
100 @mock.patch('charmhelpers.core.hookenv.config')
101 @mock.patch('charmhelpers.core.unitdata.kv')
102 @mock.patch('reactive.jenkins_slave.write_default_conf')
103 @mock.patch('reactive.jenkins_slave.file_to_units')
104 def test_configure_jenkins_slave_no_url(self, file_to_units, write_default_conf, unitdata_kv, config,
105 set_flag, clear_flag):
106 config.return_value = {}
107 unitdata_kv.return_value = {}
108 configure_jenkins_slave()
109 self.assertEqual(write_default_conf.call_args_list, [])
110 self.assertEqual(set_flag.call_args_list, [mock.call('jenkins-slave.blocked')])
111 self.assertEqual(clear_flag.call_args_list, [mock.call('jenkins-slave.active')])
112
113 @mock.patch('charms.reactive.clear_flag')
114 @mock.patch('charms.reactive.set_flag')
115 @mock.patch('charmhelpers.core.hookenv.config')
116 @mock.patch('charmhelpers.core.unitdata.kv')
117 @mock.patch('reactive.jenkins_slave.write_default_conf')
118 @mock.patch('reactive.jenkins_slave.file_to_units')
119 def test_configure_jenkins_slave_master_url(self, file_to_units, write_default_conf, unitdata_kv, config,
120 set_flag, clear_flag):
121 config.return_value = {'master_url': 'http://10.1.1.1:8080'}
122 unitdata_kv.return_value = {}
123 configure_jenkins_slave()
124 self.assertEqual(write_default_conf.call_args_list, [mock.call('http://10.1.1.1:8080')])
125 self.assertEqual(set_flag.call_args_list, [mock.call('jenkins-slave.configured')])
126 expected = [mock.call('jenkins-slave.active'), mock.call('nagios-nrpe.configured')]
127 self.assertEqual(clear_flag.call_args_list, expected)
128
129 @mock.patch('charms.reactive.clear_flag')
130 @mock.patch('charms.reactive.set_flag')
131 @mock.patch('charmhelpers.core.hookenv.config')
132 @mock.patch('charmhelpers.core.unitdata.kv')
133 @mock.patch('reactive.jenkins_slave.write_default_conf')
134 @mock.patch('reactive.jenkins_slave.file_to_units')
135 def test_configure_jenkins_slave_relation_url(self, file_to_units, write_default_conf, unitdata_kv, config,
136 set_flag, clear_flag):
137 config.return_value = {}
138 unitdata_kv.return_value = {'url': 'http://10.22.22.22:8080'}
139 configure_jenkins_slave()
140 print(write_default_conf.call_args_list)
141 self.assertEqual(write_default_conf.call_args_list, [mock.call('http://10.22.22.22:8080')])
142 self.assertEqual(set_flag.call_args_list, [mock.call('jenkins-slave.configured')])
143 expected = [mock.call('jenkins-slave.active'), mock.call('nagios-nrpe.configured')]
144 self.assertEqual(clear_flag.call_args_list, expected)
145
146 def test_write_default_conf_update(self):
147 write_default_conf('http://10.1.1.1:8080', self.user, self.group, self.conf_file)
148 self.assertTrue(conf_match(self.conf_file, 'JENKINS_URL', 'http://10.1.1.1:8080'))
149
150 def test_write_default_conf_reset(self):
151 write_default_conf(None, self.user, self.group, self.conf_file)
152 self.assertTrue(conf_match(self.conf_file, '#JENKINS_URL', ''))
153
154 def test_file_to_units_executable_sh(self):
155 source = os.path.join(self.charm_dir, 'files/download-slave.sh')
156 dest = os.path.join(self.tmpdir, os.path.basename(source))
157 file_to_units(source, dest, owner=self.user, group=self.group)
158 with open(dest, 'rb') as fh:
159 want = fh.read().decode('utf-8')
160 self.assertTrue(conf_equals(source, want))
161 self.assertEqual(pwd.getpwuid(os.stat(dest).st_uid).pw_name, self.user)
162 self.assertEqual(grp.getgrgid(os.stat(dest).st_gid).gr_name, self.group)
163 self.assertTrue(os.access(dest, os.X_OK))
164
165 def test_file_to_units_executable_py(self):
166 source = os.path.join(self.charm_dir, 'tests/unit/files/somefile.py')
167 dest = os.path.join(self.tmpdir, os.path.basename(source))
168 file_to_units(source, dest, owner=self.user, group=self.group)
169 with open(dest, 'rb') as fh:
170 want = fh.read().decode('utf-8')
171 self.assertTrue(conf_equals(source, want))
172 self.assertEqual(pwd.getpwuid(os.stat(dest).st_uid).pw_name, self.user)
173 self.assertEqual(grp.getgrgid(os.stat(dest).st_gid).gr_name, self.group)
174 self.assertTrue(os.access(dest, os.X_OK))
175
176 def test_file_to_units_non_executable(self):
177 source = os.path.join(self.charm_dir, 'files/jenkins-slave-logrotate-config')
178 dest = os.path.join(self.tmpdir, os.path.basename(source))
179 file_to_units(source, dest, owner=self.user, group=self.group)
180 with open(dest, 'rb') as fh:
181 want = fh.read().decode('utf-8')
182 self.assertTrue(conf_equals(dest, want))
183 self.assertEqual(pwd.getpwuid(os.stat(dest).st_uid).pw_name, self.user)
184 self.assertEqual(grp.getgrgid(os.stat(dest).st_gid).gr_name, self.group)
185 self.assertFalse(os.access(dest, os.X_OK))
186
187 def test_file_to_units_non_executable_x_on_disk(self):
188 source = os.path.join(self.charm_dir, 'tests/unit/files/somefile')
189 dest = os.path.join(self.tmpdir, os.path.basename(source))
190 file_to_units(source, dest, owner=self.user, group=self.group)
191 with open(dest, 'rb') as fh:
192 want = fh.read().decode('utf-8')
193 self.assertTrue(conf_equals(source, want))
194 self.assertEqual(pwd.getpwuid(os.stat(dest).st_uid).pw_name, self.user)
195 self.assertEqual(grp.getgrgid(os.stat(dest).st_gid).gr_name, self.group)
196 self.assertFalse(os.access(dest, os.X_OK))
197
198
199def conf_equals(conf_file, want):
200 with open(conf_file, 'rb') as conf:
201 got = conf.read().decode('utf-8')
202 if got == want:
203 return True
204 print('{}\n != \n{}'.format(got, want))
205 return False
206
207
208def conf_match(conf_file, key, value):
209 with open(conf_file, 'rb') as conf:
210 for line in conf.readlines():
211 line = line.decode('utf-8').rstrip('\n')
212 if line == '{}="{}"'.format(key, value):
213 return True
214 return False
215
216
217if __name__ == '__main__':
218 unittest.main()
0219
=== added file 'tox.ini'
--- tox.ini 1970-01-01 00:00:00 +0000
+++ tox.ini 2019-03-01 05:39:53 +0000
@@ -0,0 +1,37 @@
1[tox]
2skipsdist=True
3envlist = unit, functional
4skip_missing_interpreters = True
5
6[testenv]
7basepython = python3
8setenv =
9 PYTHONPATH = .
10
11[testenv:unit]
12commands = pytest -v --ignore {toxinidir}/tests/functional --cov=lib --cov=reactive --cov=actions --cov-report=term
13deps = -r{toxinidir}/tests/unit/requirements.txt
14 -r{toxinidir}/requirements.txt
15setenv = PYTHONPATH={toxinidir}/lib
16
17[testenv:functional]
18passenv =
19 HOME
20 JUJU_REPOSITORY
21 PATH
22commands = pytest -v --ignore {toxinidir}/tests/unit
23deps = -r{toxinidir}/tests/functional/requirements.txt
24 -r{toxinidir}/requirements.txt
25
26[testenv:lint]
27commands = flake8
28deps = flake8
29
30[flake8]
31exclude =
32 .git,
33 __pycache__,
34 .tox,
35 hooks/install.d/,
36max-line-length = 120
37max-complexity = 10

Subscribers

People subscribed via source and target branches