Merge lp:~lutostag/charms/trusty/jenkins/jenkins+executor+remote-deb+plugins-move into lp:charms/trusty/jenkins

[Greg Lutostanski]

Use a fully specified plugins-site -- don't change plugins-url to */latest/, just have it use the url directly, and change the default to take care of this.

Allow installing from remote deb link -- means you can change which version of jenkins you are using without bundling or having external net access for the ci-jenkins deb repos.

Allow setting number of executors on master explicitly with a config option.

[Review Queue (review-queue)]

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/1867/

review: Needs Fixing (automated testing)

[Review Queue (review-queue)]

The results (PASS) are in and available here: http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/1847/

review: Approve (automated testing)

[Ryan Beisner (1chb1n)]

@lutostag

Thanks for you work on this! Everything seems pretty sane to me. The only holdback I have, is that slaves also have executors, and it may be misleading to have a charm config option named 'executors' which is pinned to just the master. Later, someone may want a knob to control the executors on slaves, independent from the master executor setting (ex. I know my test lab has differing quantities of executors on the master than on the slaves).

Perhaps "master-executors" for the charm config option name? That leaves room for slave-executors or something along those lines.

review: Needs Information

[lp:~lutostag/charms/trusty/jenkins/jenkins+executor+remote-deb+plugins-move]

38. By Greg Lutostanski on 2016-01-12

config option name change executors->master-executors

[Greg Lutostanski (lutostag)]

Agreed on name change, thanks for the review.

[lp:~lutostag/charms/trusty/jenkins/jenkins+executor+remote-deb+plugins-move]

39. By Greg Lutostanski on 2016-01-12

fix because jinja doesnt like "-" in variable names

[Review Queue (review-queue)]

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/2174/

review: Needs Fixing (automated testing)

[Review Queue (review-queue)]

The results (PASS) are in and available here: http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/2155/

review: Approve (automated testing)

[Adam Israel (aisrael)]

Hi Greg,

Thanks for your work on this patch so far! There are a few minor concerns, like proxy access in certain network configurations and payload verification (i.e., checksums) but jenkins isn't currently publishing an sha1 to compare against. Neither of those are immediately solvable, and not enough to hold this back.

+1

review: Approve

=== modified file 'config.yaml'
--- config.yaml	2015-11-11 23:32:08 +0000
+++ config.yaml	2016-01-12 17:52:04 +0000
@@ -14,6 +14,7 @@
                manually copied over to the charm files/ dir before deployment
                and named jenkins.deb. It can be downloaded from
                http://pkg.jenkins-ci.org/debian/
+     - http(s)://...: http(s) link to a retrievable jenkins deb
   username:
     type: string
     default: admin
@@ -30,7 +31,7 @@
     description: Space separated list of plugins to install
   plugins-site:
     type: string
-    default: "https://updates.jenkins-ci.org/"
+    default: "https://updates.jenkins-ci.org/latest/"
     description: Site to download plugin .hpi files from.
   plugins-check-certificate:
     type: string
@@ -42,3 +43,8 @@
     description: |
       Set this to yes to remove any plugins not listed in plugins
       from the installation.
+  master-executors:
+    type: int
+    default: 1
+    description: |
+      Number of executors to configure for jenkins master.

=== modified file 'hooks/jenkins_hooks.py'
--- hooks/jenkins_hooks.py	2015-11-17 21:39:23 +0000
+++ hooks/jenkins_hooks.py	2016-01-12 17:52:04 +0000
@@ -32,6 +32,7 @@
     service_stop,
 )
 from charmhelpers.payload.execd import execd_preinstall
+from charmhelpers.core.templating import render
 from jenkins_utils import (
     JENKINS_HOME,
     JENKINS_USERS,
@@ -41,6 +42,7 @@
     get_jenkins_password,
     setup_source,
     install_from_bundle,
+    install_from_remote_deb,
     install_jenkins_plugins,
 )
 
@@ -52,6 +54,8 @@
     execd_preinstall('hooks/install.d')
     if config('release') == 'bundle':
         install_from_bundle()
+    elif config('release').startswith('http'):
+        install_from_remote_deb(config('release'))
     else:
         # Only setup the source if jenkins is not already installed i.e. makes
         # the config 'release' immutable so you can't change source once
@@ -122,10 +126,10 @@
     if not os.path.exists(jenkins_bootstrap_flag):
         log("Bootstrapping secure initial configuration in Jenkins.",
             level=DEBUG)
-        src = os.path.join(TEMPLATES_DIR, 'jenkins-config.xml')
         dst = os.path.join(JENKINS_HOME, 'config.xml')
-        shutil.copy(src, dst)
-        os.chown(dst, jenkins_uid, nogroup_gid)
+        context = {'master_executors': config('master-executors')}
+        render('jenkins-config.xml', dst, context, owner='jenkins',
+               group='nogroup')
         # Touch
         with open(jenkins_bootstrap_flag, 'w'):
             pass

=== modified file 'hooks/jenkins_utils.py'
--- hooks/jenkins_utils.py	2015-11-11 23:32:08 +0000
+++ hooks/jenkins_utils.py	2016-01-12 17:52:04 +0000
@@ -62,6 +62,15 @@
         log("Node '%s' does not exist - not deleting" % (host), level=INFO)
 
 
+def install_local_deb(filename):
+    """Install the given local jenkins deb"""
+    # Install jenkins deps.
+    apt_install(['daemon', 'adduser', 'psmisc', 'default-jre'], fatal=True)
+    # Run dpkg to install bundled deb.
+    env = os.environ.copy()
+    subprocess.call(['dpkg', '-i', filename], env=env)
+
+
 def install_from_bundle():
     """Install Jenkins from bundled package."""
     # Check bundled package exists.
@@ -70,11 +79,17 @@
         errmsg = "'%s' doesn't exist. No package bundled." % (bundle_path)
         raise Exception(errmsg)
     log('Installing from bundled Jenkins package: %s' % bundle_path)
-    # Install bundle deps.
-    apt_install(['daemon', 'adduser', 'psmisc', 'default-jre'], fatal=True)
-    # Run dpkg to install bundled deb.
-    env = os.environ.copy()
-    subprocess.call(['dpkg', '-i', bundle_path], env=env)
+    install_local_deb(bundle_path)
+
+
+def install_from_remote_deb(link):
+    """Install Jenkins from http(s) deb file."""
+    log('Getting remote jenkins package: %s' % link)
+    tempdir = tempfile.mkdtemp()
+    target = os.path.join(tempdir, 'jenkins.deb')
+    subprocess.check_call(['wget', '-q', '-O', target, link])
+    install_local_deb(target)
+    shutil.rmtree(tempdir)
 
 
 def setup_source(release):
@@ -158,7 +173,7 @@
 
         for plugin in plugins:
             plugin_filename = "%s.hpi" % (plugin)
-            url = os.path.join(plugins_site, 'latest', plugin_filename)
+            url = os.path.join(plugins_site, plugin_filename)
             plugin_path = os.path.join(JENKINS_PLUGINS, plugin_filename)
             if not os.path.isfile(plugin_path):
                 log("Installing plugin %s" % (plugin_filename), level=DEBUG)

=== modified file 'templates/jenkins-config.xml'
--- templates/jenkins-config.xml	2011-09-22 14:46:56 +0000
+++ templates/jenkins-config.xml	2016-01-12 17:52:04 +0000
@@ -1,6 +1,7 @@
 <?xml version='1.0' encoding='UTF-8'?>
 <hudson>
   <useSecurity>true</useSecurity>
+  <numExecutors>{{master_executors}}</numExecutors>
   <authorizationStrategy class="hudson.security.FullControlOnceLoggedInAuthorizationStrategy"/>
   <securityRealm class="hudson.security.HudsonPrivateSecurityRealm">
     <disableSignup>true</disableSignup>