Merge ~alfonsosanchezbeato/snappy-hwe-snaps/+git/alsa-utils:run-on-hw into ~snappy-hwe-team/snappy-hwe-snaps/+git/alsa-utils:master

[Alfonso Sanchez-Beato]

Adapt spread tests so we can run them on real HW

Adapt spread tests so we can run them on real HW

[Konrad Zapałowicz (kzapalowicz)]

ack

review: Approve (code)

[System Enablement Bot (system-enablement-ci-bot)]

FAILED: Continuous integration, rev:5943c6b9c6b76c29a5970463c12294b891939ae6
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1424/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/system-enablement/job/generic-build-snap-worker/893/console
    None: https://jenkins.canonical.com/system-enablement/job/generic-update-snap-mp/1333/console
    FAILURE: https://jenkins.canonical.com/system-enablement/job/generic-test-snap/1520/console
    None: https://jenkins.canonical.com/system-enablement/job/generic-cleanup-snap/532/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1424/rebuild

review: Needs Fixing (continuous-integration)

[System Enablement Bot (system-enablement-ci-bot)]

PASSED: Continuous integration, rev:5943c6b9c6b76c29a5970463c12294b891939ae6
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1426/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-build-snap-worker/895
    None: https://jenkins.canonical.com/system-enablement/job/generic-update-snap-mp/1334/console
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-test-snap/1521
    None: https://jenkins.canonical.com/system-enablement/job/generic-cleanup-snap/533/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1426/rebuild

review: Approve (continuous-integration)

[Simon Fels (morphis)]

Two comments in line, otherwise LGTM

review: Needs Fixing

[Alfonso Sanchez-Beato (alfonsosanchezbeato)]

Branch refreshed after addressing review comments. See also comments below.

[System Enablement Bot (system-enablement-ci-bot)]

FAILED: Continuous integration, rev:dad6c2218a071687f48defb94975ef3ec7162dc9
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1429/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/system-enablement/job/generic-build-snap-worker/898/console
    None: https://jenkins.canonical.com/system-enablement/job/generic-update-snap-mp/1337/console
    FAILURE: https://jenkins.canonical.com/system-enablement/job/generic-test-snap/1524/console
    None: https://jenkins.canonical.com/system-enablement/job/generic-cleanup-snap/536/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1429/rebuild

review: Needs Fixing (continuous-integration)

[System Enablement Bot (system-enablement-ci-bot)]

FAILED: Continuous integration, rev:c679c8d54f0866da0f0291fe92a1465df1011c40
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1430/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/system-enablement/job/generic-build-snap-worker/899/console
    None: https://jenkins.canonical.com/system-enablement/job/generic-update-snap-mp/1338/console
    FAILURE: https://jenkins.canonical.com/system-enablement/job/generic-test-snap/1525/console
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-cleanup-snap/537

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1430/rebuild

review: Needs Fixing (continuous-integration)

[System Enablement Bot (system-enablement-ci-bot)]

PASSED: Continuous integration, rev:c679c8d54f0866da0f0291fe92a1465df1011c40
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1431/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-build-snap-worker/900
    None: https://jenkins.canonical.com/system-enablement/job/generic-update-snap-mp/1339/console
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-test-snap/1526
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-cleanup-snap/538

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1431/rebuild

review: Approve (continuous-integration)

[Jim Hodapp (jhodapp)]

LGTM

review: Approve

[System Enablement Bot (system-enablement-ci-bot)]

FAILED: Continuous integration, rev:25692b96f03f37294e4bb38abdcc69fb5f9ead40
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1434/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/system-enablement/job/generic-build-snap-worker/913/console
    None: https://jenkins.canonical.com/system-enablement/job/generic-update-snap-mp/1342/console
    FAILURE: https://jenkins.canonical.com/system-enablement/job/generic-test-snap/1539/console
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-cleanup-snap/551

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1434/rebuild

review: Needs Fixing (continuous-integration)

[System Enablement Bot (system-enablement-ci-bot)]

PASSED: Continuous integration, rev:25692b96f03f37294e4bb38abdcc69fb5f9ead40
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1435/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-build-snap-worker/914
    None: https://jenkins.canonical.com/system-enablement/job/generic-update-snap-mp/1343/console
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-test-snap/1540
    None: https://jenkins.canonical.com/system-enablement/job/generic-cleanup-snap/552/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1435/rebuild

review: Approve (continuous-integration)

[Simon Fels (morphis)]

review: Needs Fixing

[Alfonso Sanchez-Beato (alfonsosanchezbeato)]

Branch refreshed

[System Enablement Bot (system-enablement-ci-bot)]

PASSED: Continuous integration, rev:4f7f9e9139f19ad3261d7743ff74a2bcce047030
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1448/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-build-snap-worker/947
    None: https://jenkins.canonical.com/system-enablement/job/generic-update-snap-mp/1356/console
    SUCCESS: https://jenkins.canonical.com/system-enablement/job/generic-test-snap/1573
    None: https://jenkins.canonical.com/system-enablement/job/generic-cleanup-snap/581/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/system-enablement/job/generic-build-snap/1448/rebuild

review: Approve (continuous-integration)

[Jim Hodapp (jhodapp)]

Still looks good to me.

[Simon Fels (morphis)]

LGTM

review: Approve

diff --git a/run-tests.sh b/run-tests.sh
index f030283..bb879ba 100755
--- a/run-tests.sh
+++ b/run-tests.sh
@@ -22,20 +22,31 @@ TESTS_EXTRAS_PATH=".tests-extras"
 # Display help.
 # This has to be in sync with the tests-extras/test-runner.sh script
 # functionalities as the parameters to this one are passed directly there
+# this function will quit the script because of the 'exec' keyword
 show_help() {
-    echo "Usage: run-tests.sh [OPTIONS]"
-    echo
-    echo "This is fetch & forget script and what it does is to fetch the"
-    echo "tests-extras repository and execute the run-tests.sh script from"
-    echo "there passing arguments as-is."
-    echo
-    echo "optional arguments:"
-    echo "  --help                 Show this help message and exit"
-    echo "  --channel              Select another channel to build the base image from (default: stable)"
-    echo "  --snap                 Extra snap to install"
-    echo "  --debug                Enable verbose debugging output"
-    echo "  --test-from-channel    Pull network-manager snap from the specified channel instead of building it from source"
-    echo "  --force-new-image      Force generating a new image used for testing"
+    exec cat <<'EOF'
+Usage: run-tests.sh [OPTIONS]
+
+This is fetch & forget script and what it does is to fetch the
+tests-extras repository and execute the run-tests.sh script from
+there passing arguments as-is.
+
+optional arguments:
+  --help                           Show this help message and exit
+  --channel=<channel>              Select another channel to build the base image from (default: $channel)
+  --snap=<name>                    Extra snap to install
+  --debug                          Enable verbose debugging output
+  --test-from-channel=<channel>    Pull snap from the specified channel instead of building it from source
+  --spread-system=<system>         Name of a spread system to use for testing
+  --spread-filter=<filter>         Spread filter expressions for test cases
+  --external-address=<IP address>  Address of external backend used for testing
+  --external-user=<user name>      User name to use on external test backend (default: $external_user)
+  --external-password=<password>   Password to use for specified external user (default: $external_password)
+  --force-new-image                Force generating a new image used for testing
+  --image-unsigned-core            Use an unsigned core snap for the generated image
+  --image-unsigned-gadget          Use an unsigned gadget snap for the generated image
+  --image-unsigned-kernel          Use an unsigned kernel snap for the generated image
+EOF
 }
 
 # Clone the tests-extras repository
diff --git a/spread.yaml b/spread.yaml
index c5e1e1b..a4920c2 100644
--- a/spread.yaml
+++ b/spread.yaml
@@ -13,15 +13,17 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-
 project: alsa-utils
 
+kill-timeout: 2h
+warn-timeout: 40m
+
 environment:
     SNAP_NAME: alsa-utils
     PROJECT_PATH: /home/${SNAP_NAME}
     TESTSLIB: $PROJECT_PATH/tests/integration/lib
     SNAP_CHANNEL: $(HOST:echo $SNAP_CHANNEL)
-    SNAP_ARCH: amd64
+    SNAP_ARCH: $(dpkg --print-architecture)
     SNAP_AUTOCONNECT_CORE_PLUGS: alsa home
     SNAP_AUTO_ALIASES: arecord aplay alsamixer amixer alsaucm alsactl speaker-test aseqnet alsatplg arecordmidi amidi alsaloop aconnect iecset alsabat aseqdump aplaymidi
 
@@ -31,6 +33,16 @@ backends:
             - ubuntu-core-16:
                 username: test
                 password: test
+    external:
+        type: adhoc
+        environment:
+            SPREAD_EXTERNAL_ADDRESS: "$(HOST: echo ${SPREAD_EXTERNAL_ADDRESS:-localhost:8022})"
+        allocate: |
+            ADDRESS $SPREAD_EXTERNAL_ADDRESS
+        systems:
+            - hw-ubuntu-core-16:
+                username: $(HOST:echo $SPREAD_EXTERNAL_USER)
+                password: $(HOST:echo $SPREAD_EXTERNAL_PASSWORD)
 
 path: /home/alsa-utils
 
@@ -45,6 +57,7 @@ suites:
         summary: Full-system tests for the alsa-utils snap
         systems:
             - ubuntu-core-16
+            - hw-ubuntu-core-16
         prepare: |
             . $TESTSLIB/prepare.sh
         restore-each: |
diff --git a/tests/integration/main/content-statefile/task.yaml b/tests/integration/main/content-statefile/task.yaml
index 2782c35..6227ef9 100644
--- a/tests/integration/main/content-statefile/task.yaml
+++ b/tests/integration/main/content-statefile/task.yaml
@@ -1,5 +1,10 @@
 summary: Verify that initial state file can be pulled over the content interface
 
+systems:
+    # Do not run on real hardware as this test requires an unsigned gadget snap
+    # in the image and we cannot guarantee that in this case.
+    - -hw-ubuntu-core-16
+
 prepare: |
     . $TESTSLIB/snap-names.sh
     snap list | grep $gadget_name | awk '{print $2}' > gadget_version
diff --git a/tests/integration/main/documentation-builds/task.yaml b/tests/integration/main/documentation-builds/task.yaml
index 7fef7b1..ccac8ae 100644
--- a/tests/integration/main/documentation-builds/task.yaml
+++ b/tests/integration/main/documentation-builds/task.yaml
@@ -1,5 +1,10 @@
 summary: Verify the project documentation is building without errors
 
+systems:
+    # Do not run on real hardware as documentation-builder is not available in
+    # all archs.
+    - -hw-ubuntu-core-16
+
 execute: |
     # Need to install in devmode as otherwise the snap can't access our project
     # home which is outside of the home directory of our current user.
diff --git a/tests/integration/main/validate-manual-tests/task.yaml b/tests/integration/main/validate-manual-tests/task.yaml
index 1b8b481..8edd337 100644
--- a/tests/integration/main/validate-manual-tests/task.yaml
+++ b/tests/integration/main/validate-manual-tests/task.yaml
@@ -2,7 +2,7 @@ summary: Validate all manual plainbox test cases so that we don't ship any broke
 
 execute: |
     snap install --devmode --beta classic
-    cat <<-EOF > /home/test/validate-manual-tests.sh
+    cat <<-EOF > ${PROJECT_PATH}/validate-manual-tests.sh
     #!/bin/sh
     set -ex
     apt update
@@ -13,5 +13,5 @@ execute: |
     python3 manage.py develop
     python3 manage.py validate
     EOF
-    chmod +x /home/test/validate-manual-tests.sh
-    sudo classic /home/test/validate-manual-tests.sh
+    chmod +x ${PROJECT_PATH}/validate-manual-tests.sh
+    sudo classic ${PROJECT_PATH}/validate-manual-tests.sh