lp:~dannf/charms/+source/scalebot-jenkins

Author: dann frazier
Author Date: 2022-11-14 23:22:18 UTC

charmcraft.yaml: Build on 22.04

Building with a 20.04 base has started to fail because of:
  https://github.com/canonical/charmcraft/issues/738
TLDR: some pip packages have been updated to need a newer version of
setuptools than focal provides.

Ubuntu 22.04 does have a new enough setuptools to avoid this, so
we can workaround the problem by building the charm on 22.04.

Note that the above bug suggests an alternative option of asking the
charm plugin to install a newer setuptools via pip:

parts:
  charm:
    charm-python-packages: [setuptools, pip]

That won't work for us though, as the charm-python-packages interface
appears to not be implemented for the reactive charm plugin.

Author: dann frazier
Author Date: 2022-05-06 21:16:19 UTC

Consolidate juju bootstrap code

We can consolidate the "controller already exists" logic into a
common path, increasing code coverage and potentially avoiding situations
where the charm loses track of the bootstrap state. This also replaces the
difficult to differentiate scalebot_configure_juju() and
scalebot_configure_juju_for_lab() with just scalebot_juju_bootstrap()
(you can guess what that does, right?). This consolidation avoids
some redundant and buggy behavior with how we check to see if
configs that changed require a bootstrap.

It is still the case that rebootstrapping will fail if there are
active models, but we can't automatically deal with that without
data loss, so a user will need to manually clean those up.

Author: dann frazier
Author Date: 2022-04-12 22:39:16 UTC

Drop CrumbIssuer workaround, which appears to no longer be needed

While I haven't identified *what* fixed it, a fresh deploy of scalebot
without this workaround seems to work fine. LP: #1842030.

Author: dann frazier
Author Date: 2022-04-12 22:21:32 UTC

symlink the juju snap's jujud binary into /usr/local/bin

juju apparently needs to be in the user's $PATH in order to bootstrap
a new controller:

unit-scalebot-0: 21:57:24 ERROR unit.scalebot/0.juju-log Fail to bootstrap a controller. It may be because: Failed for: failed to acquire node: No available machine matches constraints.
unit-scalebot-0: 21:57:25 WARNING unit.scalebot/0.update-status ERROR No controllers registered.
unit-scalebot-0: 21:57:25 WARNING unit.scalebot/0.update-status
unit-scalebot-0: 21:57:25 WARNING unit.scalebot/0.update-status Please either create a new controller using "juju bootstrap" or connect to
unit-scalebot-0: 21:57:25 WARNING unit.scalebot/0.update-status another controller that you have been given access to using "juju register".
unit-scalebot-0: 21:57:25 WARNING unit.scalebot/0.update-status
unit-scalebot-0: 21:57:25 INFO unit.scalebot/0.juju-log Invoking reactive handler: reactive/scalebot-jenkins.py:471:scalebot_start
unit-scalebot-0: 21:57:25 INFO unit.scalebot/0.juju-log Invoking reactive handler: reactive/apt.py:50:ensure_package_status
unit-scalebot-0: 21:57:25 INFO unit.scalebot/0.juju-log status-set: blocked: Fail to bootstrap a controller. It may be because: Failed for: failed to acquire node: No available machine matches constraints.
unit-scalebot-0: 21:57:25 INFO juju.worker.uniter.operation ran "update-status" hook (via explicit, bespoke hook script)

Author: dann frazier
Author Date: 2022-04-07 19:36:26 UTC

Don't fail if systemd override directory already exists

Fixes this issue I saw after a charm upgrade:

unit-scalebot-1: 15:33:45 ERROR unit.scalebot/1.juju-log Hook error:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/usr/local/lib/python3.8/dist-packages/charms/reactive/bus.py", line 379, in dispatch
    _invoke(hook_handlers)
  File "/usr/local/lib/python3.8/dist-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/usr/local/lib/python3.8/dist-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-scalebot-1/charm/reactive/scalebot-jenkins.py", line 114, in configure_scalebot_env
    os.makedirs(unit_override_dir)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/etc/systemd/system/jenkins.service.d'

Author: dann frazier
Author Date: 2022-03-23 21:13:46 UTC

Install juju from the snap, but execute it without the snap wrapper

Cut & pasting of the included comment:

    New versions of juju are only distributed as a snap, but the jenkins
    charm doesn't support running juju from a snap:
      https://github.com/jenkinsci/jenkins-charm/issues/59
    Let's install juju via the snap, but purely as a way to get the juju
    binary onto the system. Instead of executing it through the snap wrapper
    (/snap/bin/juju), we install a symlink to the juju binary itself
    (/snap/juju/current/bin/juju) into /usr/local/bin. /usr/local/bin
    precedes the /snap/bin wrapper in $PATH. This method allows us to run
    it without any of the snap confinement.

LP: #1965450

Author: dann frazier
Author Date: 2022-03-18 22:47:51 UTC

Remove the crumb workaround insertion for initscript-based jenkins installs

With this we can no longer support Jenkins debs prior to 2.335 where
they migrated to systemd. Doing so lets us get rid of an ugly hack
that will otherwise likely bit-rot.

Author: dann frazier
Author Date: 2021-10-05 22:52:58 UTC

Reapply jenkins-job-builder workaround on charm upgrade

The last charm release made some changes to the code we inject into
/etc/default/jenkins, but provided no mechanism to re-trigger the
injection. Add a hook to do it on charm upgrade.

Author: dann frazier
Author Date: 2021-10-05 21:44:05 UTC

black automatic reformatting

Author: dann frazier
Author Date: 2021-04-15 21:41:36 UTC

Block unnecessary reactions when the charm is stopping

If a user removes the scalebot application (`juju remove-application`), juju
will trigger the stop hook. We provide a stop hook which destroys the lab
controller, freeing up resources in the cloud. However, the charm continues
to react to other events, which can lead to it *re*-bootstrapping a
controller. This can lead to orphaned resources in the cloud.

Introduce a new pair of states[*], "scalebot.go" and "scalebot.stop", which
are intended to be mutually exclusive. The charm will start out and remain
in the "scalebot.go" state until the stop hook is called, at which point
we'll transition to the "scalebot.stop" state. Add guards to prevent
various reactions from executing in "scalebot.stop".

[*] The "state" API has been apparently deprecated for a "flag" API for
    some time. We should switch over to the "flag" API - but I'll leave
    that for another time.

Author: dann frazier
Author Date: 2021-04-15 22:41:45 UTC

README.devel: Remind developers to tag the repo after pushing a new charm

Author: dann frazier
Author Date: 2021-03-30 18:10:11 UTC

Fix typo and simplify log message

The main goal here is to correct spelling of "bootstrapping", but I also
decided to shorten the message because I'm not sure the second sentence
was adding to the first.

Author: dann frazier
Author Date: 2021-03-24 01:10:03 UTC

Add focal support

This charm deploys fine on Ubuntu 20.04 ('focal'), so designate it to
be a supported series.

Author: dann frazier
Author Date: 2019-10-15 13:45:38 UTC

README.devel: Update built charm path

charm-tools changed where it outputs built charms sometime ago.

Author: dann frazier
Author Date: 2019-10-11 14:57:01 UTC

Rename 'mkjobs' hook to 'refresh'

mkjobs implies what it does, not when it does it. There are other things
a user might want to do after refreshing a repo - e.g. installing new
dependencies. Make the naming more reflective of that.

Author: dann frazier
Author Date: 2019-10-11 13:26:47 UTC

Add a scalebot_production config

This simply sets an environment variable for tests to determine whether
they are in a production environment or not. What that means is really
up to an individual deployment. Our need, for example, is that we have
a "test" that just frees up nodes in the lab. We only want the production
environment doing that.

Author: dann frazier
Author Date: 2019-09-27 15:52:09 UTC

Expose the path to the juju cloud/credentials files via environment variables

We have a job that wants to release all unlocked MAAS machines, but to do so,
it needs to talk to MAAS directly.

Author: dann frazier
Author Date: 2019-09-12 22:30:29 UTC

Call refreshjobs action script when doing initial jenkins job import

Eliminates duplicate code, and also allows the mkjobs hook to be called
during initial import. It will cause a git pull to happen right after
the initial clone - likely a no-op, but adds a bit of overhead.

Author: dann frazier
Author Date: 2019-09-10 23:13:26 UTC

Disable improved CSRF tokens (crumbs) until jobbuilder supports them

The current version of jenkins stable has grown incompatible with
jjb: https://storyboard.openstack.org/#!/story/2006489

Workaround it for now.

Caveat: Trying to automatically edit config files sucks, this should
definitely only be a short term thing. I also haven't tried to make
this a great charm upgrade experience. If you upgrade from this charm
to a later one w/o the workaround, it will not restore better CSRF.

Author: dann frazier
Author Date: 2019-08-20 19:25:01 UTC

Add support for executing an initialization script from the config repo

This can be used to e.g. install test dependencies.

Author: dann frazier
Author Date: 2019-08-17 14:44:49 UTC

Correct state name

The state is scalebot.configured.jobbuilder, not jobbuilder.configured.

Author: dann frazier
Author Date: 2019-08-16 15:23:21 UTC

Add bionic support

The only thing that appears to be missing to support bionic is that juju
is no longer in the Ubuntu archive for bionic - it has moved to a snap.
Switching to the snap will take some work. snap binaries don't work when
$HOME isn't under /home, which is the case for the jenkins user
(HOME=/var/lib/jenkins). Fixing - or even working around that - turns out
to be complicated. For now, pull it from the stable PPA, which appears to
be kept up-to-date.

Author: dann frazier
Author Date: 2018-01-02 23:41:40 UTC

Add README.devel, initially describing how to build/publish the charm

Author: dann frazier
Author Date: 2017-10-12 22:17:09 UTC

Use the sample config repo as the default

Author: dann frazier
Author Date: 2017-10-12 21:20:44 UTC

Add a quickstart section to README.md

Author: dann frazier
Author Date: 2017-10-12 17:44:13 UTC

Make lp_ssh_key config optional

An ssh key is only needed when your scalebot config repo has a git+ssh://
URI. If you're accessing a public repo anonymously (git://), you should
be able to leave this empty.

Author: dann frazier
Author Date: 2017-09-29 19:08:37 UTC

Add empty default settings for config

Fixes the following warnings from 'charm build':

proof: W: config.yaml: option scalebot_juju_bootstrap_constraints does not have the keys: default
proof: W: config.yaml: option scalebot_repo does not have the keys: default
proof: W: config.yaml: option scalebot_juju_model_defaults does not have the keys: default
proof: W: config.yaml: option scalebot_juju_credentials does not have the keys: default
proof: W: config.yaml: option lp_ssh_key does not have the keys: default
proof: W: config.yaml: option scalebot_juju_clouds does not have the keys: default
proof: W: config.yaml: option scalebot_branch does not have the keys: default

Author: dann frazier
Author Date: 2017-09-29 16:40:07 UTC

Destroy the lab controller on charm stop hook

Author: dann frazier
Author Date: 2017-09-27 17:54:01 UTC

Add links to referenced projects

Author: dann frazier
Author Date: 2017-09-18 22:47:44 UTC

 Add config to allow passing in constraints for scalebot's juju controller

Author: dann frazier
Author Date: 2017-09-01 22:48:41 UTC

Improve config descriptions

Author: dann frazier
Author Date: 2017-02-07 17:04:27 UTC

Replace SCALEBOT_ROOT w/ SCALEBOT_HOME and SCALEBOT_REPO

Originally we had planned to expose SCALEBOT_ROOT to users, pointing to
the top of the scalebot git repository, which would be at /srv/scalebot.
However, when implementing the git clone support, the repo ended up being
cloned to /srv/scalebot/repo as a way to leave room to add other artifacts
under /srv/scalebot. Let's be more explicit and expose the repo location
as $SCALEBOT_REPO.

Author: dann frazier
Author Date: 2017-02-04 16:47:14 UTC

Create a local clone of the scalebot repo

This repo contains the jenkins job builder files and the tests
that they execute. Make the repository/branch/key configurable
via juju config so that users can test code before merging.

Author: dann frazier
Author Date: 2017-01-20 23:30:19 UTC

Add repo field

Cleans up a warning reported during 'charm build'.

Author: dann frazier
Author Date: 2017-01-13 20:47:46 UTC

Use jenkins-charm layer instead of installing jenkins ourselves

jenkins-charm isn't an official/published layer at this time, so
you have to clone that layer to a local directory, and set LAYER_PATH to
that directory, so 'charm build' can find it.

The jenkins-charm layer can be cloned from:
  git clone https://github.com/jenkinsci/jenkins-charm

Author: dann frazier
Author Date: 2017-01-13 20:47:46 UTC

Use jenkins-charm layer instead of installing jenkins ourselves

jenkins-charm isn't an official/published layer at this time, so
you have to clone that layer to a local directory, and set LAYER_PATH to
that directory, so 'charm build' can find it.

The jenkins-charm layer can be cloned from:
  git clone https://github.com/jenkinsci/jenkins-charm

Author: dann frazier
Author Date: 2017-01-11 16:51:11 UTC

Restrict series to xenial

subprocess.run() wasn't available in trusty.