Merge lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-setup-script into lp:snappy-ecosystem-tests

[Heber Parrucci]

Adding setup shell script for setting up the environment

Add a setup shell script in order to install the python dependencies needed for setting up the environment.

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:32
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/247/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2056/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/247/rebuild

review: Needs Fixing (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-setup-script]

33. By Heber Parrucci on 2017-03-06

Fixing pylint issue

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:33
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/249/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2058/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/249/rebuild

review: Approve (continuous-integration)

[lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-setup-script]

34. By Heber Parrucci on 2017-03-07

changing pip for pip3

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:34
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/254/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2066/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/254/rebuild

review: Approve (continuous-integration)

[Omer Akram (om26er)]

Setup failed for me, am I missing something ? http://paste.ubuntu.com/24131475/

Also I had to install virtualenv manually.

review: Needs Fixing

[Heber Parrucci (heber013)]

> Setup failed for me, am I missing something ?
> http://paste.ubuntu.com/24131475/
>
> Also I had to install virtualenv manually.

Seems that the same error we got some days ago when creating the virtual env for running the tests.
The problem is when building wheel for cryptography. Check if you have the following packages in your system:

$ sudo apt-get install libffi-dev
$ sudo apt-get install libssl-dev

Notice that the virtual env is created properly on Jenkins (though it fails in another part because of a proxy issue that is being resolved by IS people):
https://platform-qa-jenkins.ubuntu.com/view/QA%20Jobs/view/System%20Tests/job/run_snappy_ecosystem_tests/8/

[Omer Akram (om26er)]

Would be nice to have a fully reproducible environment without the need to fiddle with package installation manually. So we need to first check if pip is installed, otherwise exit the script. After that we should make sure to install libffi-dev and libssl-dev. Should also install virtualenv with pip. After that the script should proceed. Obviously not something for this MR to block on but something to aim for.

After installing the package I was able to get the environment setup.

review: Approve

=== modified file 'mk-venv'
--- mk-venv	2017-02-09 20:09:40 +0000
+++ mk-venv	2017-03-07 15:11:37 +0000
@@ -32,13 +32,21 @@
     proxy="$2"
     shift # past argument
     ;;
+    -r|--requirements)
+    requirements="$2"
+    shift # past argument
+    ;;
 esac
 shift # past argument or value
 done
 
-if [ "$proxy" ]; then
-    echo "Using proxy: $proxy to install dependencies"
-    pip install --proxy $proxy -r requirements.txt
+if [ -z "$requirements" ]; then
+    requirements=requirements.txt
+fi
+
+if [ ! -z "$proxy" ]; then
+    echo "Using proxy: ${proxy} to install dependencies"
+    pip3 install --proxy ${proxy} -r ${requirements}
 else
-    pip install -r requirements.txt
+    pip3 install -r ${requirements}
 fi

=== added file 'requirements-setup.txt'
--- requirements-setup.txt	1970-01-01 00:00:00 +0000
+++ requirements-setup.txt	2017-03-07 15:11:37 +0000
@@ -0,0 +1,2 @@
+pylxd
+retrying

=== added file 'run_setup'
--- run_setup	1970-01-01 00:00:00 +0000
+++ run_setup	2017-03-07 15:11:37 +0000
@@ -0,0 +1,62 @@
+#!/bin/bash
+
+#
+# Snappy Ecosystem Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+while [[ $# -gt 1 ]]
+do
+
+key="$1"
+
+case $key in
+    --proxy)
+    proxy="$2"
+    shift # past argument
+    ;;
+    -m|--mode)
+    mode="$2"
+    shift # past argument
+    ;;
+    --profile)
+    profile="$2"
+    shift # past argument
+    ;;
+esac
+shift # past argument or value
+done
+
+. ./mk-venv -r requirements-setup.txt --proxy ${proxy}
+
+if [ -z "$mode" ]; then
+    mode=lxd
+fi
+
+if [ -z "$profile" ]; then
+    profile=staging
+fi
+ve/bin/python3 -m snappy_ecosystem_tests.environment.setup --mode ${mode} --profile ${profile}
+
+result=$?
+deactivate
+
+if [ $result != 0 ]; then
+    echo -e -n "Environment setup FAILED.\n"
+    exit 1;
+else
+    echo -e -n "Environment setup PASSED.\n"
+fi

=== modified file 'snappy_ecosystem_tests/environment/managers.py'
--- snappy_ecosystem_tests/environment/managers.py	2017-03-03 18:09:16 +0000
+++ snappy_ecosystem_tests/environment/managers.py	2017-03-07 15:11:37 +0000
@@ -114,8 +114,7 @@
         # Allow export of environment variables over ssh.
         container.execute(shlex.split(COMMAND_ALLOW_ENV_VARS.format(
             ' '.join(PROFILES[profile][container_type]
-                     ['environment_variables'].keys()))
-        ))
+                     ['environment_variables'].keys()))))
         container.execute(shlex.split('service ssh restart'))
 
     def setup(self, profile):