Merge ~aittner/charms/+source/scalebot-autopkgtest:add-autopkgtest-charm into ~ce-hyperscale/charms/+source/scalebot-autopkgtest:master

[Alexandre Erwin Ittner]

[dann frazier (dannf)]

This looks awesome, thanks Alexandre!

A few comments inline - and one nit about the name, this charm isn't really glued to scalebot at all AFAICT, so I'd suggest we just call it autopkgtest or similar.

[dann frazier (dannf)]

Nice! I added a couple of very minor comments from check tools - incorporate as you see fit. I followed the steps in your README and it worked perfectly (on lakitu w/ lxd cloud). +1.

review: Approve

diff --git a/README.md b/README.md
 new file mode 100644
index 0000000..aa2aa31
--- /dev/null
+++ b/README.md
@@ -0,0 +1,82 @@
+# Overview
+
+This charm is a thin automation layer to run DEP8 tests in a target environment.
+
+This charm will deploy a basic environment to run DEP8 tests (with autopkgtest) on the target MAAS-provisioned machine, VM or container. Packages will be build and run in the target itself to improve test coverage.
+
+Basic idea is to reuse the tests already available in the Debian packages to validate new hardware configurations and run these tests in existing infrastructure while using Juju to abstract the nature of the target environment. It will not matter, for the charm, if the test is running in a VM or in bare metal, but the intention is run them on metal to minimize the interference of virtualization providers on test results.
+
+The test run directly on the target environment using the "null" autopkgtest virtualization provider and it is assumed the  environment is "disposable", i.e. it will released or recreated once the tests are finished.
+
+
+
+# Usage
+
+This charm is intended to be deployed and used through test automation but, in development enviroments, the deployment procedure is:
+
+- Install the charm with
+```$ juju deploy cs:~ce-hyperscale/autopkgtest testbed1```.
+Notice that this example assumes a testbed named "testbed1".
+
+- Run a test with the Juju action ``run-test``, passing the Debian package name in argument ``pkgname``:
+
+        $ juju run-action testbed1/0 run-test pkgname=nginx
+        Action queued with id: d13d1822-724f-4719-8ed1-35dd0d124474
+
+- Use the action-id to check if the execution is already finished:
+
+        $ juju show-action-status d13d1822-724f-4719-8ed1-35dd0d124474
+        actions:
+        - action: run-test
+          completed at: n/a
+          id: d13d1822-724f-4719-8ed1-35dd0d124474
+          status: running
+          unit: testbed1/0
+
+- Use ``juju debug-log`` to see the outputs of the test (stdout and stderr) in realtime;
+
+- Once the test is finished, grab the output with:
+
+        $ juju show-action-output d13d1822-724f-4719-8ed1-35dd0d124474
+          results:
+            output-file: /tmp/test-nginx-kz6TT/output.txt
+            retcode: "0"
+            summary-file: /tmp/test-nginx-kz6TT/summary.txt
+          status: completed
+          timing:
+            completed: 2019-10-22 21:54:33 +0000 UTC
+            enqueued: 2019-10-22 21:48:01 +0000 UTC
+            started: 2019-10-22 21:48:06 +0000 UTC
+
+Optionally, run the action with the "--wait" command line switch. This spares the user from polling the test state, but blocks the Juju client process. 
+
+
+## Interpreting outputs
+
+The exit code returned by ``autopkgtest`` will be saved in variable ``retcode`` (see autpkgtest manpage for more information on it meaning). This only happens if ``autopkgtest`` gets a chance to run and won't be created if the test script fails before this step.
+
+Variables ``summary`` and ``output`` will have paths to files containing the summary of the test results and a copy of stdout and stderr of the test execution.  Use ``juju ssh`` to see their contents or copy them to the host machine.
+
+        $ juju ssh testbed1/0 cat /tmp/test-nginx-kz6TT/summary.txt
+        light-simple         PASS
+        core-simple          PASS
+        lua                  PASS
+        full-simple          PASS
+        extras-simple        PASS
+        full-module-deps     PASS
+        light-module-deps    PASS
+        core-module-deps     PASS
+        extras-module-deps   PASS
+        Connection to 10.214.19.195 closed.
+
+Notice that file output.txt can be pretty large.
+
+
+
+
+# Known Limitations and Issues
+
+- Some tests may refuse to run in containers; 
+
+- Autopkgtest has a protection against running tests that can break the testbed (restriction ``breaks-testbed``). It does not matter for us as the target will be recreated anyway, but these tests will require patching to force them to run.
+
diff --git a/actions.yaml b/actions.yaml
 new file mode 100644
index 0000000..a77ee65
--- /dev/null
+++ b/actions.yaml
@@ -0,0 +1,7 @@
+run-test:
+  description: Run autopkgtest for a single package.
+  params:
+    pkgname:
+      type: string
+      description: Name of the Debian package to test.
+  required: [pkgname]
diff --git a/actions/run-test b/actions/run-test
 new file mode 100755
index 0000000..1ee6857
--- /dev/null
+++ b/actions/run-test
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+set -e
+
+PKGNAME=$(action-get pkgname)
+BASEDIR=$(mktemp -d /tmp/test-$PKGNAME-XXXXX)
+
+OUTFILE=$BASEDIR/output.txt
+action-set output-file="$OUTFILE"
+
+SUMMARY=$BASEDIR/summary.txt
+action-set summary-file="$SUMMARY"
+
+touch $OUTFILE $SUMMARY
+chmod a+r $OUTFILE $SUMMARY
+chmod a+x $BASEDIR
+
+mkdir $BASEDIR/tmp
+cd $BASEDIR/tmp
+
+{
+    echo "===== AUTOPKGTEST CHARM TEST STARTING: $PKGNAME ====="
+    set +e
+    autopkgtest --summary-file=$SUMMARY -B $PKGNAME -- null
+    TESTRC=$?
+    action-set retcode="$TESTRC"
+    echo "===== AUTOPKGTEST CHARM TEST FINISHED: $PKGNAME ====="
+} 2>&1 | tee $OUTFILE
+
+rm -rf $BASEDIR/tmp
+exit $TESTRC
diff --git a/config.yaml b/config.yaml
 new file mode 100644
index 0000000..7e66e9c
--- /dev/null
+++ b/config.yaml
@@ -0,0 +1 @@
+options: { }
diff --git a/icon.svg b/icon.svg
 new file mode 100644
index 0000000..559154b
--- /dev/null
+++ b/icon.svg
@@ -0,0 +1,292 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:xlink="http://www.w3.org/1999/xlink"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   width="96"
+   height="96"
+   id="svg6517"
+   version="1.1"
+   inkscape:version="0.92.3 (2405546, 2018-03-11)"
+   sodipodi:docname="icon.svg">
+  <defs
+     id="defs6519">
+    <linearGradient
+       inkscape:collect="always"
+       xlink:href="#Background"
+       id="linearGradient6461"
+       gradientUnits="userSpaceOnUse"
+       x1="0"
+       y1="970.29498"
+       x2="144"
+       y2="970.29498"
+       gradientTransform="matrix(0,-0.66666669,0.6660448,0,-866.25992,731.29077)" />
+    <linearGradient
+       id="Background">
+      <stop
+         id="stop4178"
+         offset="0"
+         style="stop-color:#b8b8b8;stop-opacity:1" />
+      <stop
+         id="stop4180"
+         offset="1"
+         style="stop-color:#c9c9c9;stop-opacity:1" />
+    </linearGradient>
+    <filter
+       style="color-interpolation-filters:sRGB;"
+       inkscape:label="Inner Shadow"
+       id="filter1121">
+      <feFlood
+         flood-opacity="0.59999999999999998"
+         flood-color="rgb(0,0,0)"
+         result="flood"
+         id="feFlood1123" />
+      <feComposite
+         in="flood"
+         in2="SourceGraphic"
+         operator="out"
+         result="composite1"
+         id="feComposite1125" />
+      <feGaussianBlur
+         in="composite1"
+         stdDeviation="1"
+         result="blur"
+         id="feGaussianBlur1127" />
+      <feOffset
+         dx="0"
+         dy="2"
+         result="offset"
+         id="feOffset1129" />
+      <feComposite
+         in="offset"
+         in2="SourceGraphic"
+         operator="atop"
+         result="composite2"
+         id="feComposite1131" />
+    </filter>
+    <filter
+       style="color-interpolation-filters:sRGB;"
+       inkscape:label="Drop Shadow"
+       id="filter950">
+      <feFlood
+         flood-opacity="0.25"
+         flood-color="rgb(0,0,0)"
+         result="flood"
+         id="feFlood952" />
+      <feComposite
+         in="flood"
+         in2="SourceGraphic"
+         operator="in"
+         result="composite1"
+         id="feComposite954" />
+      <feGaussianBlur
+         in="composite1"
+         stdDeviation="1"
+         result="blur"
+         id="feGaussianBlur956" />
+      <feOffset
+         dx="0"
+         dy="1"
+         result="offset"
+         id="feOffset958" />
+      <feComposite
+         in="SourceGraphic"
+         in2="offset"
+         operator="over"
+         result="composite2"
+         id="feComposite960" />
+    </filter>
+    <clipPath
+       clipPathUnits="userSpaceOnUse"
+       id="clipPath873">
+      <g
+         transform="matrix(0,-0.66666667,0.66604479,0,-258.25992,677.00001)"
+         id="g875"
+         inkscape:label="Layer 1"
+         style="fill:#ff00ff;fill-opacity:1;stroke:none;display:inline">
+        <path
+           style="fill:#ff00ff;fill-opacity:1;stroke:none;display:inline"
+           d="m 46.702703,898.22775 50.594594,0 C 138.16216,898.22775 144,904.06497 144,944.92583 l 0,50.73846 c 0,40.86071 -5.83784,46.69791 -46.702703,46.69791 l -50.594594,0 C 5.8378378,1042.3622 0,1036.525 0,995.66429 L 0,944.92583 C 0,904.06497 5.8378378,898.22775 46.702703,898.22775 Z"
+           id="path877"
+           inkscape:connector-curvature="0"
+           sodipodi:nodetypes="sssssssss" />
+      </g>
+    </clipPath>
+    <filter
+       inkscape:collect="always"
+       id="filter891"
+       inkscape:label="Badge Shadow">
+      <feGaussianBlur
+         inkscape:collect="always"
+         stdDeviation="0.71999962"
+         id="feGaussianBlur893" />
+    </filter>
+  </defs>
+  <sodipodi:namedview
+     id="base"
+     pagecolor="#ffffff"
+     bordercolor="#666666"
+     borderopacity="1.0"
+     inkscape:pageopacity="0.0"
+     inkscape:pageshadow="2"
+     inkscape:zoom="1.0186341"
+     inkscape:cx="176.12112"
+     inkscape:cy="-118.41312"
+     inkscape:document-units="px"
+     inkscape:current-layer="text150"
+     showgrid="true"
+     fit-margin-top="0"
+     fit-margin-left="0"
+     fit-margin-right="0"
+     fit-margin-bottom="0"
+     inkscape:window-width="1920"
+     inkscape:window-height="1016"
+     inkscape:window-x="0"
+     inkscape:window-y="27"
+     inkscape:window-maximized="1"
+     showborder="true"
+     showguides="true"
+     inkscape:guide-bbox="true"
+     inkscape:showpageshadow="false">
+    <inkscape:grid
+       type="xygrid"
+       id="grid821" />
+    <sodipodi:guide
+       orientation="1,0"
+       position="16,48"
+       id="guide823"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       orientation="0,1"
+       position="64,80"
+       id="guide825"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       orientation="1,0"
+       position="80,40"
+       id="guide827"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       orientation="0,1"
+       position="64,16"
+       id="guide829"
+       inkscape:locked="false" />
+  </sodipodi:namedview>
+  <metadata
+     id="metadata6522">
+    <rdf:RDF>
+      <cc:Work
+         rdf:about="">
+        <dc:format>image/svg+xml</dc:format>
+        <dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+        <dc:title />
+      </cc:Work>
+    </rdf:RDF>
+  </metadata>
+  <g
+     inkscape:label="BACKGROUND"
+     inkscape:groupmode="layer"
+     id="layer1"
+     transform="translate(268,-635.29076)"
+     style="display:inline">
+    <path
+       style="fill:url(#linearGradient6461);fill-opacity:1;stroke:none;display:inline;filter:url(#filter1121)"
+       d="m -268,700.15563 0,-33.72973 c 0,-27.24324 3.88785,-31.13513 31.10302,-31.13513 l 33.79408,0 c 27.21507,0 31.1029,3.89189 31.1029,31.13513 l 0,33.72973 c 0,27.24325 -3.88783,31.13514 -31.1029,31.13514 l -33.79408,0 C -264.11215,731.29077 -268,727.39888 -268,700.15563 Z"
+       id="path6455"
+       inkscape:connector-curvature="0"
+       sodipodi:nodetypes="sssssssss" />
+    <g
+       aria-label="8"
+       style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:74.66666412px;line-height:125%;font-family:Arial;-inkscape-font-specification:'Arial Bold';text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:#000000;stroke-width:0.60000002;stroke-linecap:round;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+       id="text150">
+      <path
+         d="m -229.41907,679.75616 q -5.10937,-2.15625 -7.45312,-5.90625 -2.29688,-3.79687 -2.29688,-8.29687 0,-7.6875 5.34375,-12.70313 5.39063,-5.01562 15.28125,-5.01562 9.79688,0 15.1875,5.01562 5.4375,5.01563 5.4375,12.70313 0,4.78125 -2.48437,8.53125 -2.48438,3.70312 -6.98438,5.67187 5.71875,2.29688 8.67188,6.70313 3,4.40625 3,10.17187 0,9.51563 -6.09375,15.46875 -6.04688,5.95313 -16.125,5.95313 -9.375,0 -15.60938,-4.92188 -7.35937,-5.8125 -7.35937,-15.9375 0,-5.57812 2.76562,-10.21875 2.76563,-4.6875 8.71875,-7.21875 z m 2.71875,-13.26562 q 0,3.9375 2.20313,6.14062 2.25,2.20313 5.95312,2.20313 3.75,0 6,-2.20313 2.25,-2.25 2.25,-6.1875 0,-3.70312 -2.25,-5.90625 -2.20312,-2.25 -5.85937,-2.25 -3.79688,0 -6.04688,2.25 -2.25,2.25 -2.25,5.95313 z m -1.21875,29.4375 q 0,5.4375 2.76563,8.48437 2.8125,3.04688 6.98437,3.04688 4.07813,0 6.75,-2.90625 2.67188,-2.95313 2.67188,-8.48438 0,-4.82812 -2.71875,-7.73437 -2.71875,-2.95313 -6.89063,-2.95313 -4.82812,0 -7.21875,3.32813 -2.34375,3.32812 -2.34375,7.21875 z"
+         style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:96px;font-family:Arial;-inkscape-font-specification:'Arial Bold';fill:#ff0000;stroke:#000000;stroke-width:0.60000002;stroke-linecap:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
+         id="path959" />
+    </g>
+  </g>
+  <g
+     inkscape:groupmode="layer"
+     id="layer3"
+     inkscape:label="PLACE YOUR PICTOGRAM HERE"
+     style="display:inline" />
+  <g
+     inkscape:groupmode="layer"
+     id="layer2"
+     inkscape:label="BADGE"
+     style="display:none"
+     sodipodi:insensitive="true">
+    <g
+       style="display:inline"
+       transform="translate(-340.00001,-581)"
+       id="g4394"
+       clip-path="none">
+      <g
+         id="g855">
+        <g
+           inkscape:groupmode="maskhelper"
+           id="g870"
+           clip-path="url(#clipPath873)"
+           style="opacity:0.6;filter:url(#filter891)">
+          <path
+             transform="matrix(1.4999992,0,0,1.4999992,-29.999795,-237.54282)"
+             d="m 264,552.36218 a 12,12 0 0 1 -12,12 12,12 0 0 1 -12,-12 12,12 0 0 1 12,-12 12,12 0 0 1 12,12 z"
+             sodipodi:ry="12"
+             sodipodi:rx="12"
+             sodipodi:cy="552.36218"
+             sodipodi:cx="252"
+             id="path844"
+             style="color:#000000;fill:#000000;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:4;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
+             sodipodi:type="arc" />
+        </g>
+        <g
+           id="g862">
+          <path
+             sodipodi:type="arc"
+             style="color:#000000;fill:#f5f5f5;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:4;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
+             id="path4398"
+             sodipodi:cx="252"
+             sodipodi:cy="552.36218"
+             sodipodi:rx="12"
+             sodipodi:ry="12"
+             d="m 264,552.36218 a 12,12 0 0 1 -12,12 12,12 0 0 1 -12,-12 12,12 0 0 1 12,-12 12,12 0 0 1 12,12 z"
+             transform="matrix(1.4999992,0,0,1.4999992,-29.999795,-238.54282)" />
+          <path
+             transform="matrix(1.25,0,0,1.25,33,-100.45273)"
+             d="m 264,552.36218 a 12,12 0 0 1 -12,12 12,12 0 0 1 -12,-12 12,12 0 0 1 12,-12 12,12 0 0 1 12,12 z"
+             sodipodi:ry="12"
+             sodipodi:rx="12"
+             sodipodi:cy="552.36218"
+             sodipodi:cx="252"
+             id="path4400"
+             style="color:#000000;fill:#dd4814;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:4;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
+             sodipodi:type="arc" />
+          <path
+             sodipodi:type="star"
+             style="color:#000000;fill:#f5f5f5;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:3;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
+             id="path4459"
+             sodipodi:sides="5"
+             sodipodi:cx="666.19574"
+             sodipodi:cy="589.50385"
+             sodipodi:r1="7.2431178"
+             sodipodi:r2="4.3458705"
+             sodipodi:arg1="1.0471976"
+             sodipodi:arg2="1.6755161"
+             inkscape:flatsided="false"
+             inkscape:rounded="0.1"
+             inkscape:randomized="0"
+             d="m 669.8173,595.77657 c -0.39132,0.22593 -3.62645,-1.90343 -4.07583,-1.95066 -0.44938,-0.0472 -4.05653,1.36297 -4.39232,1.06062 -0.3358,-0.30235 0.68963,-4.03715 0.59569,-4.47913 -0.0939,-0.44198 -2.5498,-3.43681 -2.36602,-3.8496 0.18379,-0.41279 4.05267,-0.59166 4.44398,-0.81759 0.39132,-0.22593 2.48067,-3.48704 2.93005,-3.4398 0.44938,0.0472 1.81505,3.67147 2.15084,3.97382 0.3358,0.30236 4.08294,1.2817 4.17689,1.72369 0.0939,0.44198 -2.9309,2.86076 -3.11469,3.27355 -0.18379,0.41279 0.0427,4.27917 -0.34859,4.5051 z"
+             transform="matrix(1.511423,-0.16366377,0.16366377,1.511423,-755.37346,-191.93651)" />
+        </g>
+      </g>
+    </g>
+  </g>
+</svg>
diff --git a/layer.yaml b/layer.yaml
 new file mode 100644
index 0000000..6581470
--- /dev/null
+++ b/layer.yaml
@@ -0,0 +1,5 @@
+includes: ['layer:basic']  # if you use any interfaces, add them here
+
+options:
+  basic:
+    packages: ['autopkgtest', 'dpkg-dev']
diff --git a/metadata.yaml b/metadata.yaml
 new file mode 100644
index 0000000..11dde37
--- /dev/null
+++ b/metadata.yaml
@@ -0,0 +1,18 @@
+name: autopkgtest
+summary: Run DEP8 tests on Juju infrastructure
+maintainer: Canonical Server and Hardware Enablement Team <[e-mail]>
+description: |
+  This charm will deploy a basic environment to run DEP8 tests (with
+  autopkgtest) on the target machine, VM or container. Basic idea is to
+  reuse the tests already available in the packages and run them, using
+  Juju to automate target selection and deployment. It is assumed the
+  target environment is "disposable", i.e.  it will released or recreated
+  once the tests are finished.
+
+series:
+  - xenial
+  - bionic
+  - eoan
+tags:
+  - ops
+  - misc
diff --git a/reactive/autopkgtest.py b/reactive/autopkgtest.py
 new file mode 100644
index 0000000..beb31b8
--- /dev/null
+++ b/reactive/autopkgtest.py
@@ -0,0 +1,54 @@
+import os
+import subprocess
+
+from charmhelpers.core.hookenv import log, status_set
+from charms.reactive import (hook, when, when_not, when_any, set_flag,
+                             set_state, remove_state)
+
+APT_SOURCE_FNAME = "/etc/apt/sources.list.d/charm-autopkgtest.list"
+
+
+def apt_update():
+    subprocess.run(["apt-get", "-y", "update"], check=True)
+
+
+def get_release_codename():
+    with os.popen("lsb_release -cs") as fp:
+        return fp.read().strip()
+
+
+def add_apt_sources():
+    """Add an APT source list with the required packages.
+
+    This may duplicate some existing entries, but apt seems to handle
+    this situation pretty well.
+    """
+
+    codename = get_release_codename()
+    channels = "main universe restricted"
+    with open(APT_SOURCE_FNAME, "w") as fp:
+        fp.write("deb-src http://archive.ubuntu.com/ubuntu %s %s\n"
+                 % (codename, channels))
+        fp.write("deb-src http://security.ubuntu.com/ubuntu %s-security %s\n"
+                 % (codename, channels))
+    apt_update()
+
+
+def remove_apt_sources():
+    try:
+        os.unlink(APT_SOURCE_FNAME)
+    except FileNotFoundError:
+        # Ignore non-existing files.
+        pass
+    apt_update()
+
+
+@when_not('autopkgtest.ready')
+def autopkgtest_ready():
+    try:
+        add_apt_sources()
+    except subprocess.CalledProcessError:
+        status_set('blocked', "Failed to install dependencies")
+    else:
+        status_set('active', 'ready')
+        set_flag('autopkgtest.ready')