Merge charm-k8s-jenkins-agent:relations-config into charm-k8s-jenkins-agent:master
- Git
- lp:charm-k8s-jenkins-agent
- relations-config
- Merge into master
Status: | Merged |
---|---|
Approved by: | Tom Haddon |
Approved revision: | 7c58cfd617d7f7a001ce9307a898dd33cbd7a482 |
Merged at revision: | 3d32fe405416600da88ceb51f21453f9fbe2aec9 |
Proposed branch: | charm-k8s-jenkins-agent:relations-config |
Merge into: | charm-k8s-jenkins-agent:master |
Diff against target: |
722 lines (+475/-83) 8 files modified
.gitignore (+1/-0) Makefile (+7/-0) config.yaml (+16/-18) dockerfile/Dockerfile (+3/-1) dockerfile/files/entrypoint.sh (+12/-9) metadata.yaml (+3/-0) src/charm.py (+134/-47) tests/unit/test_charm.py (+299/-8) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Tom Haddon | Approve | ||
Facundo Batista (community) | Approve | ||
Alexandre Gomes | Needs Resubmitting | ||
Canonical IS Reviewers | Pending | ||
Canonical IS Reviewers | Pending | ||
Review via email: mp+389685@code.launchpad.net |
This proposal supersedes a proposal from 2020-08-15.
Commit message
Add feature to configure agents through juju relations
Description of the change
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : Posted in a previous version of this proposal | # |
Tom Haddon (mthaddon) wrote : Posted in a previous version of this proposal | # |
Some comments inline. Tests will need updating a little based on what changes you make.
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
This merge proposal is being monitored by mergebot. Change the status to Approved to merge.
Alexandre Gomes (alejdg) wrote : | # |
All comments have been addressed and tests have been updated too.
Tom Haddon (mthaddon) wrote : | # |
Some comments/questions inline.
Alexandre Gomes (alejdg) wrote : | # |
The interface is still called jenkins-slave, so we need to keep all the references to 'slave' until the interface gets updated.
Tom Haddon (mthaddon) wrote : | # |
Thanks, a few more comments (sorry for not catching those before). With these addressed I think we can put this up for review with ~charmcrafters.
Alexandre Gomes (alejdg) wrote : | # |
All comments have been addressed. @mthaddon could you check if the changes in the _is_valid_config are more in line with the pattern you mentioned?
Tom Haddon (mthaddon) wrote : | # |
A few more comments inline. Hopefully final changes now, and then we can get this up for review with the charmcrafters. Thanks.
Alexandre Gomes (alejdg) wrote : | # |
All addressed in the last commit.
Tom Haddon (mthaddon) wrote : | # |
LGTM, thx. Have requested a review from ~charmcrafters.
Facundo Batista (facundo) wrote : | # |
Annotated a couple of small comments, thanks!
Alexandre Gomes (alejdg) wrote : | # |
> Annotated a couple of small comments, thanks!
Hey Facundo, the _spec thing was just something I got from another charm and I thought it was a best practice. I've addressed it and the other comments as well.
Facundo Batista (facundo) wrote : | # |
Awesome, thanks!
Tom Haddon (mthaddon) wrote : | # |
Adding an approving comment so we can merge, having had charmcraft review (thanks!)
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
Change successfully merged at revision 3d32fe405416600
Preview Diff
1 | diff --git a/.gitignore b/.gitignore |
2 | index bf7fffd..eea14ad 100644 |
3 | --- a/.gitignore |
4 | +++ b/.gitignore |
5 | @@ -6,3 +6,4 @@ __pycache__ |
6 | build |
7 | lib/ops |
8 | lib/ops-*.dist-info |
9 | +unit/ |
10 | diff --git a/Makefile b/Makefile |
11 | index 86bd8d3..8c8056a 100644 |
12 | --- a/Makefile |
13 | +++ b/Makefile |
14 | @@ -18,6 +18,13 @@ build-image: | $(dockerfile_dir) |
15 | --build-arg REVISION="$(revision)" \ |
16 | -t $(JENKINS_IMAGE) $(dockerfile_dir) |
17 | |
18 | +build-image-no-cache: | $(dockerfile_dir) |
19 | + docker build \ |
20 | + --build-arg AUTHOR=$(author) \ |
21 | + --build-arg REVISION="$(revision)" \ |
22 | + --no-cache \ |
23 | + -t $(JENKINS_IMAGE) $(dockerfile_dir) |
24 | + |
25 | build-release-image: | $(dockerfile_dir) |
26 | docker build \ |
27 | --build-arg DATE_CREATED="$(date_created)" \ |
28 | diff --git a/config.yaml b/config.yaml |
29 | index 7ad8e02..c419381 100644 |
30 | --- a/config.yaml |
31 | +++ b/config.yaml |
32 | @@ -3,32 +3,30 @@ options: |
33 | type: string |
34 | default: "jenkins-agent-operator" |
35 | description: "The docker image to install. Required. Defaults to jenkins-agent-operator from rocks.canonical.com" |
36 | - tools: |
37 | - type: string |
38 | - default: git gcc make bzr |
39 | - description: Tooling to deploy on jenkins agent node |
40 | - labels: |
41 | - type: string |
42 | - description: Jenkins labels to associate with jenkins agent node |
43 | - default: "" |
44 | jenkins_master_url: |
45 | type: string |
46 | default: "" |
47 | description: | |
48 | - Configure the agent to use an explicit jenkins master instead of using |
49 | - the jenkins-agent relation. This allows for the agent and master to |
50 | - be deployed in different environments. |
51 | + Configure the agent to use an explicit Jenkins master instead of using |
52 | + the jenkins-agent relation. This allows the agent to connect to a Jenkins instance not managed |
53 | + by Juju. |
54 | jenkins_agent_name: |
55 | type: string |
56 | default: "" |
57 | description: | |
58 | - Jenkins agent name that will be seen on the jenkins master. |
59 | - Will default to the hostname of the container. |
60 | - jenkins_api_token: |
61 | + Agent name as configured in Jenkins. Multiple names can be input by using `:` as a separator. |
62 | + Example: "agent-one:agent-two:agent-three" |
63 | + jenkins_agent_token: |
64 | type: string |
65 | default: "" |
66 | - description: "Jenkins API token used to join the master" |
67 | - jenkins_user: |
68 | + description: | |
69 | + Agent token provided by Jenkins. Can be found in your Jenkins instance at |
70 | + ${JENKINS_URL}/computer/${AGENT_NAME}/. Multiple tokens can be input by |
71 | + using `:` as a separator matching the order of the agents in `jenkins_agent_name`. |
72 | + Example: "token-one:token-two:token-three" |
73 | + jenkins_agent_labels: |
74 | type: string |
75 | - default: "admin" |
76 | - description: "Jenkins user to download agent.jar from master" |
77 | + default: "" |
78 | + description: | |
79 | + Comma-separated list of labels to be assigned to the agent in Jenkins. If not set it will default to |
80 | + the agents hardware identifier, e.g.: 'x86_64' |
81 | diff --git a/dockerfile/Dockerfile b/dockerfile/Dockerfile |
82 | index 1abd7e6..700e70f 100644 |
83 | --- a/dockerfile/Dockerfile |
84 | +++ b/dockerfile/Dockerfile |
85 | @@ -47,7 +47,9 @@ RUN apt-get update -y \ |
86 | |
87 | WORKDIR /var/lib/jenkins |
88 | |
89 | -USER ${USER} |
90 | +# Using root due to https://bugs.launchpad.net/juju/+bug/1879598 |
91 | +#USER ${USER} |
92 | +USER root |
93 | |
94 | COPY files/entrypoint.sh / |
95 | |
96 | diff --git a/dockerfile/files/entrypoint.sh b/dockerfile/files/entrypoint.sh |
97 | index c4e879c..a2f820e 100755 |
98 | --- a/dockerfile/files/entrypoint.sh |
99 | +++ b/dockerfile/files/entrypoint.sh |
100 | @@ -3,6 +3,7 @@ |
101 | set -eu -o pipefail |
102 | |
103 | export LC_ALL=C |
104 | +export TERM=xterm |
105 | |
106 | # defaults for jenkins-agent component of the jenkins continuous integration |
107 | # system |
108 | @@ -20,17 +21,11 @@ typeset JAVA_ARGS=${JAVA_ARGS:-""} |
109 | # job from running. |
110 | typeset JENKINS_URL="${JENKINS_URL:?"URL of a jenkins server must be provided"}" |
111 | |
112 | -# Name of agent configuration to use at JENKINS_URL |
113 | -# Override if it need to be something other than the |
114 | -# hostname of the server the agent is running on. |
115 | -typeset JENKINS_HOSTNAME="${JENKINS_HOSTNAME:-$(hostname)}" |
116 | +typeset JENKINS_WORKDIR="/var/lib/jenkins" |
117 | |
118 | # Arguments to pass to jenkins agent on startup |
119 | typeset -a JENKINS_ARGS |
120 | |
121 | -JENKINS_ARGS+=(-jnlpUrl "${JENKINS_URL}"/computer/"${JENKINS_HOSTNAME}"/slave-agent.jnlp) |
122 | -JENKINS_ARGS+=(-jnlpCredentials "${JENKINS_API_USER:?Please specify JENKINS_API_USER}:${JENKINS_API_TOKEN:?Please specify JENKINS_API_TOKEN}") |
123 | - |
124 | # Path of the agent.jar |
125 | typeset AGENT_JAR=/var/lib/jenkins/agent.jar |
126 | |
127 | @@ -55,5 +50,13 @@ download_agent |
128 | # Specify the pod as ready |
129 | touch /var/lib/jenkins/agents/.ready |
130 | |
131 | -#shellcheck disable=SC2086 |
132 | -"${JAVA}" ${JAVA_ARGS} -jar "${AGENT_JAR}" "${JENKINS_ARGS[@]}" |
133 | +# Transform the env variables in arrays to iterate through it |
134 | +IFS=':' read -r -a AGENTS <<< ${JENKINS_AGENTS} |
135 | +IFS=':' read -r -a TOKENS <<< ${JENKINS_TOKENS} |
136 | + |
137 | +echo ${!AGENTS[@]} |
138 | + |
139 | +for index in ${!AGENTS[@]}; do |
140 | + echo "About to run ${JAVA}" "${JAVA_ARGS}" -jar "${AGENT_JAR}" -jnlpUrl "${JENKINS_URL}"/computer/"${AGENTS[$index]}"/slave-agent.jnlp -workDir "${JENKINS_WORKDIR}" -noReconnect -secret "${TOKENS[$index]}" |
141 | + ${JAVA} ${JAVA_ARGS} -jar ${AGENT_JAR} -jnlpUrl ${JENKINS_URL}/computer/${AGENTS[$index]}/slave-agent.jnlp -workDir ${JENKINS_WORKDIR} -noReconnect -secret ${TOKENS[$index]} || echo "Invalid or already used credentials." |
142 | +done |
143 | diff --git a/metadata.yaml b/metadata.yaml |
144 | index b53a613..dd92ccb 100644 |
145 | --- a/metadata.yaml |
146 | +++ b/metadata.yaml |
147 | @@ -9,3 +9,6 @@ series: |
148 | - kubernetes |
149 | deployment: |
150 | service: omit |
151 | +provides: |
152 | + slave: |
153 | + interface: jenkins-slave |
154 | diff --git a/src/charm.py b/src/charm.py |
155 | index 4ba8782..9ade818 100755 |
156 | --- a/src/charm.py |
157 | +++ b/src/charm.py |
158 | @@ -4,8 +4,9 @@ |
159 | # Licensed under the GPLv3, see LICENCE file for details. |
160 | |
161 | import io |
162 | -import pprint |
163 | import logging |
164 | +import os |
165 | +import pprint |
166 | |
167 | from ops.charm import CharmBase |
168 | from ops.framework import StoredState |
169 | @@ -16,59 +17,58 @@ from ops.model import ActiveStatus, BlockedStatus, MaintenanceStatus |
170 | logger = logging.getLogger() |
171 | |
172 | |
173 | -def generate_pod_config(config, secured=True): |
174 | - """Kubernetes pod config generator. |
175 | - |
176 | - generate_pod_config generates Kubernetes deployment config. |
177 | - If the secured keyword is set then it will return a sanitised copy |
178 | - without exposing secrets. |
179 | - """ |
180 | - pod_config = {} |
181 | - |
182 | - pod_config["JENKINS_API_USER"] = config["jenkins_user"] |
183 | - if config.get("jenkins_master_url", None): |
184 | - pod_config["JENKINS_URL"] = config["jenkins_master_url"] |
185 | - if config.get("jenkins_agent_name", None): |
186 | - pod_config["JENKINS_HOSTNAME"] = config["jenkins_agent_name"] |
187 | - |
188 | - if secured: |
189 | - return pod_config |
190 | +class JenkinsAgentCharm(CharmBase): |
191 | + _stored = StoredState() |
192 | |
193 | - # Add secrets from charm config |
194 | - pod_config["JENKINS_API_TOKEN"] = config["jenkins_api_token"] |
195 | + def __init__(self, *args): |
196 | + super().__init__(*args) |
197 | |
198 | - return pod_config |
199 | + self.framework.observe(self.on.start, self.configure_pod) |
200 | + self.framework.observe(self.on.config_changed, self.configure_pod) |
201 | + self.framework.observe(self.on.upgrade_charm, self.configure_pod) |
202 | + self.framework.observe(self.on.slave_relation_joined, self.on_agent_relation_joined) |
203 | + self.framework.observe(self.on.slave_relation_changed, self.on_agent_relation_changed) |
204 | |
205 | + self._stored.set_default(spec=None, jenkins_url=None, agent_tokens=None, agents=None) |
206 | |
207 | -class JenkinsAgentCharm(CharmBase): |
208 | - state = StoredState() |
209 | + def generate_pod_config(self, config, secured=True): |
210 | + """Kubernetes pod config generator. |
211 | |
212 | - def __init__(self, framework, parent): |
213 | - super().__init__(framework, parent) |
214 | + generate_pod_config generates Kubernetes deployment config. |
215 | + If the secured keyword is set then it will return a sanitised copy |
216 | + without exposing secrets. |
217 | + """ |
218 | + pod_config = {} |
219 | |
220 | - framework.observe(self.on.start, self.configure_pod) |
221 | - framework.observe(self.on.config_changed, self.configure_pod) |
222 | - framework.observe(self.on.upgrade_charm, self.configure_pod) |
223 | + if self._stored.jenkins_url: |
224 | + pod_config["JENKINS_URL"] = self._stored.jenkins_url |
225 | + else: |
226 | + pod_config["JENKINS_URL"] = config["jenkins_master_url"] |
227 | |
228 | - self.state.set_default(_spec=None) |
229 | + if secured: |
230 | + return pod_config |
231 | |
232 | - def on_upgrade_charm(self, event): |
233 | - pass |
234 | + if self._stored.agent_tokens and self._stored.agents: |
235 | + pod_config["JENKINS_AGENTS"] = ":".join(self._stored.agents) |
236 | + pod_config["JENKINS_TOKENS"] = ":".join(self._stored.agent_tokens) |
237 | + else: |
238 | + pod_config["JENKINS_AGENTS"] = config["jenkins_agent_name"] |
239 | + pod_config["JENKINS_TOKENS"] = config["jenkins_agent_token"] |
240 | |
241 | - def on_config_changed(self, event): |
242 | - pass |
243 | + return pod_config |
244 | |
245 | def configure_pod(self, event): |
246 | - is_valid = self.is_valid_config() |
247 | + """Assemble the pod spec and apply it, if possible.""" |
248 | + is_valid = self._is_valid_config() |
249 | if not is_valid: |
250 | return |
251 | |
252 | - spec = self.make_pod_spec() |
253 | - if spec != self.state._spec: |
254 | - self.state._spec = spec |
255 | + spec = self._make_pod_spec() |
256 | + if spec != self._stored.spec: |
257 | + self._stored.spec = spec |
258 | # only the leader can set_spec() |
259 | if self.model.unit.is_leader(): |
260 | - spec = self.make_pod_spec() |
261 | + spec = self._make_pod_spec() |
262 | |
263 | logger.info("Configuring pod") |
264 | self.model.unit.status = MaintenanceStatus("Configuring pod") |
265 | @@ -81,13 +81,15 @@ class JenkinsAgentCharm(CharmBase): |
266 | else: |
267 | logger.info("Pod spec unchanged") |
268 | |
269 | - self.state.is_started = True |
270 | + self._stored.is_started = True |
271 | self.model.unit.status = ActiveStatus() |
272 | |
273 | - def make_pod_spec(self): |
274 | + def _make_pod_spec(self): |
275 | + """Prepare and return a pod spec.""" |
276 | config = self.model.config |
277 | - full_pod_config = generate_pod_config(config, secured=False) |
278 | - secure_pod_config = generate_pod_config(config, secured=True) |
279 | + |
280 | + full_pod_config = self.generate_pod_config(config, secured=False) |
281 | + secure_pod_config = self.generate_pod_config(config, secured=True) |
282 | |
283 | spec = { |
284 | "containers": [ |
285 | @@ -105,15 +107,31 @@ class JenkinsAgentCharm(CharmBase): |
286 | logger.info("This is the Kubernetes Pod spec config (sans secrets) <<EOM\n{}\nEOM".format(out.getvalue())) |
287 | |
288 | secure_pod_config.update(full_pod_config) |
289 | - |
290 | return spec |
291 | |
292 | - def is_valid_config(self): |
293 | - is_valid = True |
294 | + def _missing_charm_settings(self): |
295 | + """Check configuration setting dependencies |
296 | + |
297 | + Return a list of missing settings; otherwise return an empty list.""" |
298 | config = self.model.config |
299 | + if self._stored.agent_tokens: |
300 | + required_settings = ("image",) |
301 | + else: |
302 | + required_settings = ("image", "jenkins_master_url", "jenkins_agent_name", "jenkins_agent_token") |
303 | |
304 | - want = ("image", "jenkins_user", "jenkins_api_token") |
305 | - missing = [k for k in want if config[k].rstrip() == ""] |
306 | + missing = [setting for setting in required_settings if not config[setting]] |
307 | + |
308 | + return sorted(missing) |
309 | + |
310 | + def _is_valid_config(self): |
311 | + """Validate required configuration. |
312 | + |
313 | + When not configuring the agent through relations |
314 | + 'jenkins_master_url', 'jenkins_agent_name' and 'jenkins_agent_token' |
315 | + are required.""" |
316 | + is_valid = True |
317 | + |
318 | + missing = self._missing_charm_settings() |
319 | if missing: |
320 | message = "Missing required config: {}".format(" ".join(missing)) |
321 | logger.info(message) |
322 | @@ -122,6 +140,75 @@ class JenkinsAgentCharm(CharmBase): |
323 | |
324 | return is_valid |
325 | |
326 | + def on_agent_relation_joined(self, event): |
327 | + """Set relation data for the unit once an agent has connected.""" |
328 | + logger.info("Jenkins relation joined") |
329 | + num_executors = os.cpu_count() |
330 | + config_labels = self.model.config.get('jenkins_agent_labels') |
331 | + agent_name = self._gen_agent_name() |
332 | + |
333 | + if config_labels: |
334 | + labels = config_labels |
335 | + else: |
336 | + labels = os.uname().machine |
337 | + |
338 | + event.relation.data[self.model.unit]["executors"] = str(num_executors) |
339 | + event.relation.data[self.model.unit]["labels"] = labels |
340 | + event.relation.data[self.model.unit]["slavehost"] = agent_name |
341 | + |
342 | + def on_agent_relation_changed(self, event): |
343 | + """Populate local configuration with data from relation.""" |
344 | + logger.info("Jenkins relation changed") |
345 | + try: |
346 | + self._stored.jenkins_url = event.relation.data[event.unit]['url'] |
347 | + except KeyError: |
348 | + pass |
349 | + |
350 | + try: |
351 | + self._stored.agent_tokens = self._stored.agent_tokens or [] |
352 | + self._stored.agent_tokens.append(event.relation.data[event.unit]['secret']) |
353 | + self._gen_agent_name(store=True) |
354 | + except KeyError: |
355 | + pass |
356 | + |
357 | + self.configure_through_relation(event) |
358 | + |
359 | + def configure_through_relation(self, event): |
360 | + """Configure the agent through data from relation.""" |
361 | + logger.info("Setting up jenkins via agent relation") |
362 | + self.model.unit.status = MaintenanceStatus("Configuring jenkins agent") |
363 | + |
364 | + if self.model.config.get("jenkins_master_url"): |
365 | + logger.info("Config option 'jenkins_master_url' is set. Can't use agent relation.") |
366 | + self.model.unit.status = ActiveStatus() |
367 | + return |
368 | + |
369 | + if self._stored.jenkins_url is None: |
370 | + logger.info("Jenkins hasn't exported its URL yet. Skipping setup for now.") |
371 | + self.model.unit.status = ActiveStatus() |
372 | + return |
373 | + |
374 | + if self._stored.agent_tokens is None: |
375 | + logger.info("Jenkins hasn't exported the agent secret yet. Skipping setup for now.") |
376 | + self.model.unit.status = ActiveStatus() |
377 | + return |
378 | + |
379 | + self.configure_pod(event) |
380 | + |
381 | + def _gen_agent_name(self, store=False): |
382 | + """Generate the agent name or get the one already in use.""" |
383 | + agent_name = "" |
384 | + if self._stored.agents: |
385 | + name, number = self._stored.agents[-1].rsplit('-', 1) |
386 | + agent_name = "{}-{}".format(name, int(number) + 1) |
387 | + if store: |
388 | + self._stored.agents.append(agent_name) |
389 | + else: |
390 | + agent_name = self.unit.name.replace('/', '-') |
391 | + if store: |
392 | + self._stored.agents = [agent_name] |
393 | + return agent_name |
394 | + |
395 | |
396 | if __name__ == '__main__': |
397 | main(JenkinsAgentCharm) |
398 | diff --git a/tests/unit/test_charm.py b/tests/unit/test_charm.py |
399 | index 65705e7..d029934 100644 |
400 | --- a/tests/unit/test_charm.py |
401 | +++ b/tests/unit/test_charm.py |
402 | @@ -1,21 +1,312 @@ |
403 | # Copyright 2020 Canonical Ltd. |
404 | # Licensed under the GPLv3, see LICENCE file for details. |
405 | |
406 | +from mock import MagicMock, patch |
407 | import unittest |
408 | |
409 | -import sys |
410 | +from ops import testing |
411 | +from ops.model import ActiveStatus, BlockedStatus, MaintenanceStatus |
412 | |
413 | -sys.path.append('src') # noqa: E402 |
414 | +from charm import JenkinsAgentCharm |
415 | |
416 | -from charm import generate_pod_config |
417 | +CONFIG_DEFAULT = { |
418 | + "image": "jenkins-agent-operator", |
419 | + "jenkins_master_url": "", |
420 | + "jenkins_agent_name": "", |
421 | + "jenkins_agent_token": "", |
422 | + "jenkins_agent_label": "" |
423 | +} |
424 | + |
425 | +CONFIG_ONE_AGENT = { |
426 | + "jenkins_master_url": "http://test", |
427 | + "jenkins_agent_name": "agent-one", |
428 | + "jenkins_agent_token": "token-one" |
429 | +} |
430 | + |
431 | +CONFIG_ONE_AGENT_CUSTOM_IMAGE = { |
432 | + "image": "image-name", |
433 | + "jenkins_master_url": "http://test", |
434 | + "jenkins_agent_name": "agent-one", |
435 | + "jenkins_agent_token": "token-one" |
436 | +} |
437 | + |
438 | +SPEC_EXPECTED = { |
439 | + 'containers': [{ |
440 | + 'config': { |
441 | + 'JENKINS_AGENTS': 'agent-one', |
442 | + 'JENKINS_TOKENS': 'token-one', |
443 | + 'JENKINS_URL': 'http://test' |
444 | + }, |
445 | + 'imageDetails': { |
446 | + 'imagePath': 'image-name' |
447 | + }, |
448 | + 'name': 'jenkins-agent', |
449 | + 'readinessProbe': { |
450 | + 'exec': { |
451 | + 'command': [ |
452 | + '/bin/cat', |
453 | + '/var/lib/jenkins/agents/.ready' |
454 | + ] |
455 | + } |
456 | + } |
457 | + }] |
458 | +} |
459 | |
460 | |
461 | class TestJenkinsAgentCharm(unittest.TestCase): |
462 | - def test_generate_pod_config(self): |
463 | - config = { |
464 | - "jenkins_user": "test", |
465 | + def setUp(self): |
466 | + self.harness = testing.Harness(JenkinsAgentCharm) |
467 | + self.harness.begin() |
468 | + self.harness.disable_hooks() |
469 | + self.harness.update_config(CONFIG_DEFAULT) |
470 | + |
471 | + def test__generate_pod_config(self): |
472 | + """Test generate_pod_config""" |
473 | + expected = { |
474 | + "JENKINS_URL": "http://test", |
475 | } |
476 | + self.assertEqual(self.harness.charm.generate_pod_config(CONFIG_ONE_AGENT, secured=True), expected) |
477 | + |
478 | + expected["JENKINS_AGENTS"] = "agent-one" |
479 | + expected["JENKINS_TOKENS"] = "token-one" |
480 | + self.assertEqual(self.harness.charm.generate_pod_config(CONFIG_ONE_AGENT, secured=False), expected) |
481 | + |
482 | + def test__generate_pod_config__with__relation__data(self): |
483 | + """Test generate_pod_config with relation data.""" |
484 | expected = { |
485 | - "JENKINS_API_USER": "test", |
486 | + "JENKINS_URL": "http://test", |
487 | } |
488 | - self.assertEqual(generate_pod_config(config), expected) |
489 | + agent_name = "jenkins-agent-0" |
490 | + url = "http://test" |
491 | + token = "token" |
492 | + |
493 | + self.harness.charm._state.jenkins_url = url |
494 | + self.harness.charm._state.agent_tokens = [token] |
495 | + self.harness.charm._state.agents = [agent_name] |
496 | + |
497 | + self.assertEqual(self.harness.charm.generate_pod_config(CONFIG_ONE_AGENT, secured=True), expected) |
498 | + |
499 | + expected["JENKINS_AGENTS"] = "jenkins-agent-0" |
500 | + expected["JENKINS_TOKENS"] = "token" |
501 | + self.assertEqual(self.harness.charm.generate_pod_config(CONFIG_ONE_AGENT, secured=False), expected) |
502 | + |
503 | + def test__configure_pod__invalid__config(self): |
504 | + """Test configure_pod when the config is invalid.""" |
505 | + self.harness.charm.on.config_changed.emit() |
506 | + message = "Missing required config: jenkins_agent_name jenkins_agent_token jenkins_master_url" |
507 | + print(self.harness.model.unit.status) |
508 | + print(BlockedStatus(message)) |
509 | + self.assertEqual(self.harness.model.unit.status, BlockedStatus(message)) |
510 | + self.assertEqual(self.harness.get_pod_spec(), None) |
511 | + |
512 | + def test__configure_pod__unit__not__leader(self): |
513 | + """Test configure_pod when the unit isn't leader.""" |
514 | + self.harness.set_leader(is_leader=False) |
515 | + self.harness.update_config(CONFIG_ONE_AGENT) |
516 | + with self.assertLogs(level='INFO') as logger: |
517 | + self.harness.charm.on.config_changed.emit() |
518 | + self.assertEqual(self.harness.model.unit.status, ActiveStatus()) |
519 | + message = "INFO:root:Spec changes ignored by non-leader" |
520 | + self.assertEqual(logger.output[-1], message) |
521 | + self.assertEqual(self.harness.get_pod_spec(), None) |
522 | + |
523 | + def test__configure_pod(self): |
524 | + """Test configure_pod.""" |
525 | + expected = (SPEC_EXPECTED, None) |
526 | + |
527 | + self.harness.set_leader(is_leader=True) |
528 | + self.harness.update_config(CONFIG_ONE_AGENT_CUSTOM_IMAGE) |
529 | + self.harness.charm.on.config_changed.emit() |
530 | + self.assertEqual(self.harness.model.unit.status, ActiveStatus()) |
531 | + self.assertEqual(self.harness.get_pod_spec(), expected) |
532 | + |
533 | + def test__configure_pod__no__spec_change(self): |
534 | + """Test configure_pod when there is no change in the spec.""" |
535 | + self.harness.set_leader(is_leader=True) |
536 | + self.harness.update_config(CONFIG_ONE_AGENT_CUSTOM_IMAGE) |
537 | + self.harness.charm._state._spec = self.harness.charm._make_pod_spec() |
538 | + with self.assertLogs(level='INFO') as logger: |
539 | + self.harness.charm.on.config_changed.emit() |
540 | + self.assertEqual(self.harness.model.unit.status, ActiveStatus()) |
541 | + message = "INFO:root:Pod spec unchanged" |
542 | + self.assertEqual(logger.output[-1], message) |
543 | + self.assertEqual(self.harness.get_pod_spec(), None) |
544 | + |
545 | + def test__make_pod_spec(self): |
546 | + """Test the construction of the spec based on juju config.""" |
547 | + self.harness.update_config(CONFIG_ONE_AGENT_CUSTOM_IMAGE) |
548 | + self.assertEqual(self.harness.charm._make_pod_spec(), SPEC_EXPECTED) |
549 | + |
550 | + def test__is_valid_config(self): |
551 | + """Test config validation.""" |
552 | + config = { |
553 | + "image": "image-name", |
554 | + "jenkins_master_url": "http://test", |
555 | + "jenkins_agent_name": "agent-one", |
556 | + "jenkins_agent_token": "token-one" |
557 | + } |
558 | + self.assertEqual(self.harness.charm._is_valid_config(), False) |
559 | + with self.subTest("Config from relation"): |
560 | + self.harness.charm._state.agent_tokens = "token" |
561 | + self.assertEqual(self.harness.charm._is_valid_config(), True) |
562 | + with self.subTest("Config from juju config"): |
563 | + self.harness.update_config(config) |
564 | + self.assertEqual(self.harness.charm._is_valid_config(), True) |
565 | + |
566 | + @patch("os.uname") |
567 | + @patch("os.cpu_count") |
568 | + def test__on_agent_relation_joined(self, mock_os_cpu_count, mock_os_uname): |
569 | + """Test relation_data is set when a new relation joins.""" |
570 | + mock_os_cpu_count.return_value = 8 |
571 | + mock_os_uname.return_value.machine = "x86_64" |
572 | + expected_relation_data = { |
573 | + 'executors': '8', |
574 | + 'labels': 'x86_64', |
575 | + 'slavehost': 'jenkins-agent-0' |
576 | + } |
577 | + self.harness.enable_hooks() |
578 | + rel_id = self.harness.add_relation("slave", "jenkins") |
579 | + self.harness.add_relation_unit(rel_id, "jenkins/0") |
580 | + |
581 | + self.assertEqual(self.harness.get_relation_data(rel_id, "jenkins-agent/0"), expected_relation_data) |
582 | + |
583 | + @patch("charm.JenkinsAgentCharm.configure_pod") |
584 | + def test__configure_through_relation__no__jenkins_master_url(self, mock_configure_pod): |
585 | + """Test configure_through_relation when no configuration has been provided.""" |
586 | + mock_event = MagicMock() |
587 | + with self.assertLogs(level='INFO') as logger: |
588 | + self.harness.charm.configure_through_relation(mock_event) |
589 | + expected_output = [ |
590 | + "INFO:root:Setting up jenkins via agent relation", |
591 | + "INFO:root:Jenkins hasn't exported its URL yet. Skipping setup for now." |
592 | + ] |
593 | + self.assertEqual(logger.output, expected_output) |
594 | + mock_configure_pod.assert_not_called() |
595 | + self.assertEqual(self.harness.model.unit.status, ActiveStatus()) |
596 | + |
597 | + @patch("charm.JenkinsAgentCharm.configure_pod") |
598 | + def test__configure_through_relation__no__jenkins_agent_tokens(self, mock_configure_pod): |
599 | + """Test configure_through_relation when no tokens have been provided.""" |
600 | + mock_event = MagicMock() |
601 | + self.harness.charm._state.jenkins_url = "http://test" |
602 | + with self.assertLogs(level='INFO') as logger: |
603 | + self.harness.charm.configure_through_relation(mock_event) |
604 | + expected_output = [ |
605 | + "INFO:root:Setting up jenkins via agent relation", |
606 | + "INFO:root:Jenkins hasn't exported the agent secret yet. Skipping setup for now." |
607 | + ] |
608 | + self.assertEqual(logger.output, expected_output) |
609 | + mock_configure_pod.assert_not_called() |
610 | + self.assertEqual(self.harness.model.unit.status, ActiveStatus()) |
611 | + |
612 | + @patch("charm.JenkinsAgentCharm.configure_pod") |
613 | + def test__configure_through_relation__manual__config(self, mock_configure_pod): |
614 | + """Test configure_through_relation when no configuration has been provided.""" |
615 | + mock_event = MagicMock() |
616 | + self.harness.update_config({"jenkins_master_url": "http://test"}) |
617 | + with self.assertLogs(level='INFO') as logger: |
618 | + self.harness.charm.configure_through_relation(mock_event) |
619 | + expected_output = [ |
620 | + "INFO:root:Setting up jenkins via agent relation", |
621 | + "INFO:root:Config option 'jenkins_master_url' is set. Can't use agent relation." |
622 | + ] |
623 | + self.assertEqual(logger.output, expected_output) |
624 | + mock_configure_pod.assert_not_called() |
625 | + self.assertEqual(self.harness.model.unit.status, ActiveStatus()) |
626 | + |
627 | + @patch("charm.JenkinsAgentCharm.configure_pod") |
628 | + def test__configure_through_relation(self, mock_configure_pod): |
629 | + """Test configure_through_relation when the relation have provided all needed data.""" |
630 | + mock_event = MagicMock() |
631 | + self.harness.charm._state.jenkins_url = "http://test" |
632 | + self.harness.charm._state.agent_tokens = "token" |
633 | + with self.assertLogs(level='INFO') as logger: |
634 | + self.harness.charm.configure_through_relation(mock_event) |
635 | + expected_output = [ |
636 | + "INFO:root:Setting up jenkins via agent relation" |
637 | + ] |
638 | + self.assertEqual(logger.output, expected_output) |
639 | + mock_configure_pod.assert_called() |
640 | + self.assertEqual(self.harness.model.unit.status, MaintenanceStatus("Configuring jenkins agent")) |
641 | + |
642 | + @patch("os.uname") |
643 | + @patch("os.cpu_count") |
644 | + def test__on_agent_relation_joined__custom__label(self, mock_os_cpu_count, mock_os_uname): |
645 | + """Test relation_data is set when a new relation joins |
646 | + and custom labels are set""" |
647 | + mock_os_cpu_count.return_value = 8 |
648 | + mock_os_uname.return_value.machine = "x86_64" |
649 | + labels = "test, label" |
650 | + expected_relation_data = { |
651 | + 'executors': '8', |
652 | + 'labels': labels, |
653 | + 'slavehost': 'jenkins-agent-0' |
654 | + } |
655 | + self.harness.update_config({"jenkins_agent_labels": labels}) |
656 | + self.harness.enable_hooks() |
657 | + rel_id = self.harness.add_relation("slave", "jenkins") |
658 | + self.harness.add_relation_unit(rel_id, "jenkins/0") |
659 | + self.assertEqual(self.harness.get_relation_data(rel_id, "jenkins-agent/0"), expected_relation_data) |
660 | + |
661 | + @patch("charm.JenkinsAgentCharm.configure_through_relation") |
662 | + @patch("os.uname") |
663 | + @patch("os.cpu_count") |
664 | + def test__on_agent_relation_changed__noop(self, mock_os_cpu_count, mock_os_uname, mock_configure_through_relation): |
665 | + """Test on_agent_relation_changed when jenkins hasn't provided information yet.""" |
666 | + mock_os_cpu_count.return_value = 8 |
667 | + mock_os_uname.return_value.machine = "x86_64" |
668 | + remote_unit = "jenkins/0" |
669 | + self.harness.enable_hooks() |
670 | + rel_id = self.harness.add_relation("slave", "jenkins") |
671 | + self.harness.add_relation_unit(rel_id, remote_unit) |
672 | + self.harness.update_relation_data(rel_id, remote_unit, {}) |
673 | + |
674 | + mock_configure_through_relation.assert_called() |
675 | + |
676 | + @patch("charm.JenkinsAgentCharm.configure_through_relation") |
677 | + @patch("os.uname") |
678 | + @patch("os.cpu_count") |
679 | + def test__on_agent_relation_changed_old(self, mock_os_cpu_count, mock_os_uname, mock_configure_through_relation): |
680 | + """Test relation_data is set when a new relation joins.""" |
681 | + mock_os_cpu_count.return_value = 8 |
682 | + mock_os_uname.return_value.machine = "x86_64" |
683 | + remote_unit = "jenkins/0" |
684 | + agent_name = "jenkins-agent-0" |
685 | + url = "http://test" |
686 | + secret = "token" |
687 | + self.harness.enable_hooks() |
688 | + rel_id = self.harness.add_relation("slave", "jenkins") |
689 | + self.harness.add_relation_unit(rel_id, remote_unit) |
690 | + self.harness.update_relation_data(rel_id, remote_unit, {"url": url, "secret": secret}) |
691 | + self.assertEqual(self.harness.charm._state.jenkins_url, url) |
692 | + self.assertEqual(self.harness.charm._state.agent_tokens[-1], secret) |
693 | + self.assertEqual(self.harness.charm._state.agents[-1], agent_name) |
694 | + mock_configure_through_relation.assert_called() |
695 | + |
696 | + @patch("charm.JenkinsAgentCharm.configure_through_relation") |
697 | + @patch("os.uname") |
698 | + @patch("os.cpu_count") |
699 | + def test__on_agent_relation_changed__multiple__agents( |
700 | + self, |
701 | + mock_os_cpu_count, |
702 | + mock_os_uname, |
703 | + mock_configure_through_relation |
704 | + ): |
705 | + """Test relation_data is set when a new relation joins.""" |
706 | + mock_os_cpu_count.return_value = 8 |
707 | + mock_os_uname.return_value.machine = "x86_64" |
708 | + remote_unit = "jenkins/0" |
709 | + agent_name = "jenkins-agent-0" |
710 | + expected_new_agent = "jenkins-agent-1" |
711 | + url = "http://test" |
712 | + secret = "token" |
713 | + |
714 | + self.harness.charm._state.agents = [agent_name] |
715 | + self.harness.enable_hooks() |
716 | + rel_id = self.harness.add_relation("slave", "jenkins") |
717 | + self.harness.add_relation_unit(rel_id, remote_unit) |
718 | + self.harness.update_relation_data(rel_id, remote_unit, {"url": url, "secret": secret}) |
719 | + self.assertEqual(self.harness.charm._state.jenkins_url, url) |
720 | + self.assertEqual(self.harness.charm._state.agent_tokens[-1], secret) |
721 | + self.assertEqual(self.harness.charm._state.agents[-1], expected_new_agent) |
722 | + mock_configure_through_relation.assert_called() |
This merge proposal is being monitored by mergebot. Change the status to Approved to merge.