Merge lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment into lp:snappy-ecosystem-tests

[Santiago Baldassin]

Scripts to build a couple of lxd containers to deploy snapcraft and snapd

Scripts to build a couple of lxd containers to deploy snapcraft and snapd

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/94/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1683/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/94/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/97/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1707/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/97/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/99/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1712/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/99/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/101/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1718/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/101/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/104/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1720/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/104/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/107/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1723/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/107/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/110/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1726/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/110/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/114/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1730/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/114/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/118/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1734/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/118/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/122/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1738/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/122/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/126/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1742/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/126/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/130/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1746/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/130/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/134/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1750/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/134/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/138/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1754/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/138/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/142/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1758/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/142/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/146/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1762/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/146/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/150/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1766/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/150/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/154/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1770/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/154/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/158/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1774/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/158/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/162/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1778/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/162/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/166/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1782/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/166/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/170/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1786/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/170/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/174/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1790/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/174/rebuild

review: Needs Fixing (continuous-integration)

[Heber Parrucci (heber013)]

Comments inline.

Additionally:
* We already have in trunk some code for building a snapd container for staging:
snappy_ecosystem_tests/helpers/snapd/staging_builder.py
and some container helpers:
snappy_ecosystem_tests/utils/lxd.py
Some parts of that code seem to overlap with this changes. My suggestion is to merge that code with yours to have one place for building containers.

Update the README with instructions to build the environment.

Add logging to see the activities and progress while building the containers... When I run the setup I just see the cursor for some minutes and have no idea what is happening behind the scene.

I am getting an error when executing the setup: http://paste.ubuntu.com/24052852/
When it tries to enable ssh, it tries to put the public key in the container, and it seems that the file /home/ubuntu/.ssh/authorized_keys does not exist

review: Needs Fixing

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/180/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1814/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/180/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/180/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1815/console
    SUCCESS: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1814

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/180/rebuild

review: Needs Fixing (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/183/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1849/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/183/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/188/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1864/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/188/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/191/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1870/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/191/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/194/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1881/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/194/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/198/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1901/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/198/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/202/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1930/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/202/rebuild

review: Approve (continuous-integration)

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:18
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/206/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1952/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/206/rebuild

review: Approve (continuous-integration)

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

19. By Santiago Baldassin on 2017-02-25

Addressing comments from the reviews

[Santiago Baldassin (sbaldassin)]

All comments addressed

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:19
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/211/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/1997/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/211/rebuild

review: Needs Fixing (continuous-integration)

[I Ahmad (iahmad)]

Could this be landed soon as heber's other mp is waiting on it. Please update readme with instructions before landing.

[Omer Akram (om26er)]

Minor cosmetic review. Need to do a thorough review later.

[Omer Akram (om26er)]

review: Needs Fixing

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

20. By Santiago Baldassin on 2017-03-01

Fixing the script

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:20
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/213/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2005/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/213/rebuild

review: Needs Fixing (continuous-integration)

[Santiago Baldassin (sbaldassin)]

Thanks Omer. All comments were addressed

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

21. By Santiago Baldassin on 2017-03-01

Addressing comments from the reviews

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:21
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/214/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2006/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/214/rebuild

review: Needs Fixing (continuous-integration)

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

22. By Santiago Baldassin on 2017-03-01

Merge from trunk

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:22
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/215/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2007/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/215/rebuild

review: Approve (continuous-integration)

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

23. By Santiago Baldassin on 2017-03-01

Adding a get_ip api

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:23
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/217/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2010/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/217/rebuild

review: Approve (continuous-integration)

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

24. By Santiago Baldassin on 2017-03-01

Minor fix

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:24
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/221/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2014/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/221/rebuild

review: Needs Fixing (continuous-integration)

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

25. By Santiago Baldassin on 2017-03-01

Minor change

[platform-qa-bot (platform-qa-bot)]

FAILED: Continuous integration, rev:25
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/222/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2016/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/222/rebuild

review: Needs Fixing (continuous-integration)

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

26. By Santiago Baldassin on 2017-03-01

fixing pylint

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:26
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/223/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2017/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/223/rebuild

review: Approve (continuous-integration)

[Heber Parrucci (heber013)]

Code LGTM

review: Approve

[Omer Akram (om26er)]

I have done the code review of the LxdManager class. The problems that were previously mentioned for the Container class were either non existent(for example a container getting out of sync, found after more research, pylxd is intelligent), or duplicate object creation, which I explained in a separate email, detailed, that its not an issue on the memory as we were only passing in a reference to an already existing object and the one that we were passing that object to was not more than the size of an empty object (a class that inherits from object but does not add any override methods).

review: Needs Fixing

[Santiago Baldassin (sbaldassin)]

> I have done the code review of the LxdManager class. The problems that were
> previously mentioned for the Container class were either non existent(for
> example a container getting out of sync, found after more research, pylxd is
> intelligent), or duplicate object creation, which I explained in a separate
> email, detailed, that its not an issue on the memory as we were only passing
> in a reference to an already existing object and the one that we were passing
> that object to was not more than the size of an empty object (a class that
> inherits from object but does not add any override methods).

Let's go through the approach over the phone. Heber and I clearly agree that the best course of action is to use a manager and to avoid the use of the Container class. Once agreed on the approach we'll get rid of the either the manager or the container class to avoid code duplication

[Omer Akram (om26er)]

Phone call sounds good to me and as an added bonus this is what the new(lean) code would look like: http://paste.ubuntu.com/24092575/ is pseudo code, so may not work as-is.

I do understand that you have certain preference for the "manager" paradigm but in this specific case that doesn't feel like solving any problem :)

[lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment]

27. By Santiago Baldassin on 2017-03-03

Minor fix

[platform-qa-bot (platform-qa-bot)]

PASSED: Continuous integration, rev:27
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/237/
Executed test runs:
    None: https://platform-qa-jenkins.ubuntu.com/job/generic-update-mp/2042/console

Click here to trigger a rebuild:
https://platform-qa-jenkins.ubuntu.com/job/snappy-ecosystem-tests-ci/237/rebuild

review: Approve (continuous-integration)

[Omer Akram (om26er)]

Lets land this.

review: Approve

=== modified file 'requirements.txt'
--- requirements.txt	2017-02-21 21:07:33 +0000
+++ requirements.txt	2017-03-03 10:19:52 +0000
@@ -14,7 +14,7 @@
 pexpect
 pymacaroons==0.9.2
 requests-toolbelt==0.6.0
-#chromedriver_installer
 pylxd
 pyyaml
+retrying
 paramiko

=== modified file 'run_checks'
--- run_checks	2017-02-20 13:10:18 +0000
+++ run_checks	2017-03-03 10:19:52 +0000
@@ -51,9 +51,9 @@
 
     if [ "$proxy" ]; then
         echo "Using proxy: $proxy to install dependencies"
-        pip install --proxy $proxy -r requirements-unit-tests.txt
+        pip3 install --proxy $proxy -r requirements-unit-tests.txt
     else
-        pip install -r requirements-unit-tests.txt
+        pip3 install -r requirements-unit-tests.txt
     fi
 
     ve_unit_tests/bin/python3 -m unittest discover snappy_ecosystem_tests.unittests 

=== added directory 'snappy_ecosystem_tests/environment'
=== added file 'snappy_ecosystem_tests/environment/__init__.py'
--- snappy_ecosystem_tests/environment/__init__.py	1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/__init__.py	2017-03-03 10:19:52 +0000
@@ -0,0 +1,19 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Snappy Ecosystem Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#

=== added file 'snappy_ecosystem_tests/environment/constants.py'
--- snappy_ecosystem_tests/environment/constants.py	1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/constants.py	2017-03-03 10:19:52 +0000
@@ -0,0 +1,66 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Snappy Ecosystem Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+"""Constants used accross the environment setup"""
+
+SNAPD = 'snapd'
+SNAPCRAFT = 'snapcraft'
+
+DEPENDENCIES = {
+    SNAPD : [],
+    SNAPCRAFT : [],
+    "shared" : ["expect"]
+}
+
+PROFILES = {
+    "staging":{
+        SNAPD:{
+            "channel": "xenial",
+            "package_name": 'snapd',
+            "environment_variables": {
+                "UBUNTU_STORE_API_ROOT_URL":
+                    "https://myapps.developer.ubuntu.com/dev/api/",
+                "UBUNTU_STORE_UPLOAD_ROOT_URL":
+                    "https://upload.apps.ubuntu.com",
+                "UBUNTU_SSO_API_ROOT_URL":
+                    "https://login.ubuntu.com/api/v2/",
+                "UBUNTU_STORE_SEARCH_ROOT_URL":
+                    "https://search.apps.ubuntu.com/"
+            }
+        },
+        SNAPCRAFT: {
+            "channel": "xenial",
+            "package_name": 'snapcraft',
+            "environment_variables": {
+                "UBUNTU_STORE_API_ROOT_URL":
+                    "https://myapps.developer.ubuntu.com/dev/api/",
+                "UBUNTU_STORE_UPLOAD_ROOT_URL":
+                    "https://upload.apps.ubuntu.com",
+                "UBUNTU_SSO_API_ROOT_URL":
+                    "https://login.ubuntu.com/api/v2/",
+                "UBUNTU_STORE_SEARCH_ROOT_URL":
+                    "https://search.apps.ubuntu.com/"
+            }
+        },
+        "snapweb":{
+            "url": "https://myapps.developer.ubuntu.com/"
+        },
+    }
+}

=== added directory 'snappy_ecosystem_tests/environment/data'
=== added file 'snappy_ecosystem_tests/environment/data/__init__.py'
--- snappy_ecosystem_tests/environment/data/__init__.py	1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/data/__init__.py	2017-03-03 10:19:52 +0000
@@ -0,0 +1,19 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Snappy Ecosystem Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#

=== added file 'snappy_ecosystem_tests/environment/data/snapcraft.py'
--- snappy_ecosystem_tests/environment/data/snapcraft.py	1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/data/snapcraft.py	2017-03-03 10:19:52 +0000
@@ -0,0 +1,21 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Snappy Ecosystem Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+"""Snapcraft data"""
+SNAPCRAFT_CONTAINER_NAME = 'snapcraft'

=== added file 'snappy_ecosystem_tests/environment/data/snapd.py'
--- snappy_ecosystem_tests/environment/data/snapd.py	1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/data/snapd.py	2017-03-03 10:19:52 +0000
@@ -0,0 +1,88 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Snappy Ecosystem Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+"""snapd data"""
+SNAPD_CONTAINER_NAME = 'snapd'
+
+CONTAINER_HOME = '/root'
+DIRECTORY_CLONE = '{}/src/github.com/snapcore/snapd'.format(CONTAINER_HOME)
+DOMAIN_PING = 'ubuntu.com'
+ENV_STORE_API_ROOT = 'UBUNTU_STORE_API_ROOT_URL'
+ENV_STORE_SEARCH_ROOT = 'UBUNTU_STORE_SEARCH_ROOT_URL'
+ENV_STORE_UPLOAD_ROOT = 'UBUNTU_STORE_UPLOAD_ROOT_URL'
+ENV_SSO_API_ROOT = 'UBUNTU_SSO_API_ROOT_URL'
+PACKAGES_SNAPD = ['snapd', 'snap-confine', 'ubuntu-core-launcher']
+COMMAND_REMOVE_SNAPD = 'apt purge -y {}'.format(' '.join(PACKAGES_SNAPD))
+COMMAND_APT_BUILD_DEP = 'apt build-dep -y ./'
+COMMAND_APT_INSTALL = 'apt install -y {}'
+COMMAND_APT_INSTALL_STAGING_DEBS = COMMAND_APT_INSTALL.format(
+    ' '.join(['../{}*.deb'.format(pkg) for pkg in PACKAGES_SNAPD]))
+COMMAND_APT_UPDATE = 'apt update'
+COMMAND_APT_UPGRADE = 'apt dist-upgrade -y'
+COMMAND_BUILD_SNAPD = 'DEB_BUILD_OPTIONS="nocheck testkeys" ' \
+                      'dpkg-buildpackage -tc -b'
+COMMAND_EXPORT_STAGING_STORE_VAR = 'echo SNAPPY_USE_STAGING_STORE=1 >> ' \
+                                   '/etc/environment'
+COMMAND_GET_GOVENDOR = 'go get -v github.com/kardianos/govendor'
+COMMAND_GIT_CLONE = 'git clone {} {}'
+COMMAND_GOVENDOR_SYNC = 'govendor sync'
+REPOSITORY_GIT = 'https://github.com/snapcore/snapd'
+URL_API_ROOT = 'https://myapps.developer.staging.ubuntu.com/dev/api/'
+URL_SEARCH_ROOT = 'https://search.apps.staging.ubuntu.com/'
+URL_UPLOAD_ROOT = 'https://upload.apps.staging.ubuntu.com/'
+URL_SSO_API_ROOT = 'https://login.staging.ubuntu.com/api/v2/'
+
+CONTAINER_SETUP = [
+    COMMAND_REMOVE_SNAPD,
+    COMMAND_APT_UPDATE,
+    # Hold open-iscsi which is causing apt upgrade errors while running
+    # inside container.
+    'apt-mark hold open-iscsi',
+    COMMAND_APT_UPGRADE,
+    # Needed for snaps to work inside a lxd container
+    #  ref: https://stgraber.org/2016/12/07/running-snaps-in-lxd-containers/
+    COMMAND_APT_INSTALL.format('squashfuse'),
+    COMMAND_GIT_CLONE.format(REPOSITORY_GIT, DIRECTORY_CLONE),
+    {'command': COMMAND_APT_BUILD_DEP, 'cwd': DIRECTORY_CLONE},
+    {'command': COMMAND_GET_GOVENDOR, 'cwd': DIRECTORY_CLONE},
+    {'command': COMMAND_GOVENDOR_SYNC, 'cwd': DIRECTORY_CLONE},
+    {'command': COMMAND_BUILD_SNAPD, 'cwd': DIRECTORY_CLONE},
+    COMMAND_EXPORT_STAGING_STORE_VAR,
+    {'command': COMMAND_APT_INSTALL_STAGING_DEBS, 'cwd': DIRECTORY_CLONE}
+]
+
+CONTAINER_ENV_VARS = {
+    'GOPATH': CONTAINER_HOME,
+    'SNAPPY_USE_STAGING_STORE': '1',
+    ENV_STORE_API_ROOT: URL_API_ROOT,
+    ENV_STORE_SEARCH_ROOT: URL_SEARCH_ROOT,
+    ENV_STORE_UPLOAD_ROOT: URL_UPLOAD_ROOT,
+    ENV_SSO_API_ROOT: URL_SSO_API_ROOT,
+}
+
+DEFAULT_CONTAINER_CONFIG = {
+    "name": "snapd",
+    "source": {
+        "type": "image",
+        "mode": "pull",
+        "server": "https://cloud-images.ubuntu.com/releases",
+        "protocol": "simplestreams",
+        "fingerprint": "16.04"
+    }
+}

=== added file 'snappy_ecosystem_tests/environment/managers.py'
--- snappy_ecosystem_tests/environment/managers.py	1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/managers.py	2017-03-03 10:19:52 +0000
@@ -0,0 +1,143 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+"""Managers to setup the environment"""
+
+import os
+import shlex
+from pylxd import Client
+from pylxd.exceptions import NotFound, LXDAPIException
+from retrying import retry
+
+from snappy_ecosystem_tests.environment import constants
+from snappy_ecosystem_tests.environment.constants import (
+    PROFILES, DEPENDENCIES)
+from snappy_ecosystem_tests.environment.data.snapcraft import (
+    SNAPCRAFT_CONTAINER_NAME)
+from snappy_ecosystem_tests.environment.data.snapd import (
+    DEFAULT_CONTAINER_CONFIG, SNAPD_CONTAINER_NAME)
+
+def get_manager(mode):
+    """Get the manager for the given mode"""
+    supported_modules = {
+        "lxd": LxdManager
+    }
+    try:
+        manager = supported_modules[mode]
+        return manager()
+    except KeyError:
+        raise RuntimeError("Mode: {} is not supported".format(mode))
+
+
+class LxdManager:
+    """Manage lxd containers"""
+
+    def __init__(self):
+        self.client = Client()
+
+    def launch(self, container_name, release='16.04'):
+        """Launch a container"""
+        try:
+            container = self.client.containers.get(container_name)
+            if container.status == "Running":
+                container.stop(wait=True)
+            container.delete(wait=True)
+        except NotFound:
+            pass
+        container_config = DEFAULT_CONTAINER_CONFIG.copy()
+        container_config['source']['alias'] = release
+        container_config['name'] = container_name
+        container = self.client.containers.create(container_config, wait=True)
+        container.start(wait=True)
+
+    def execute_command(self, container_name, command):
+        """execute command"""
+        container = self.client.containers.get(container_name)
+        container.execute(command)
+
+    def install(self, container_name, packages, channel=None):
+        """Install a packages"""
+        container = self.client.containers.get(container_name)
+        container.execute(shlex.split('apt-get update'))
+        install_command = 'apt-get install {} -y'.format(packages)
+        if channel:
+            install_command += ' -t {}'.format(channel)
+        container.execute(shlex.split(install_command))
+
+    def install_dependencies(self, container_name, container_type):
+        """Install a dependencies"""
+        self.install(
+            container_name,
+            DEPENDENCIES[container_type] + DEPENDENCIES['shared'])
+
+
+    def configure(self, container_name, container_type, profile):
+        """Configure a given container"""
+        container = self.client.containers.get(container_name)
+        for key, value in \
+                PROFILES[profile][container_type]['environment_variables'].\
+                        items():
+            container.config.update({'environment.{}'.format(
+                key): value})
+        container.save(wait=True)
+
+    @retry(stop_max_attempt_number=5, wait_fixed=3000,
+           retry_on_exception=lambda exception: isinstance(
+               exception, LXDAPIException))
+    def enable_ssh(self, container_name):
+        """Enable the ssh connection on the container"""
+        container = self.client.containers.get(container_name)
+        pub_key = open(
+            os.path.expanduser('~') + '/.ssh/id_rsa.pub').read().strip('\n')
+        container.files.put('/home/ubuntu/.ssh/authorized_keys', pub_key)
+
+
+    def setup(self, profile):
+        """setup the container based on the profile"""
+        self._setup(SNAPD_CONTAINER_NAME, constants.SNAPD, profile)
+        self._setup(
+            SNAPCRAFT_CONTAINER_NAME, constants.SNAPCRAFT, profile)
+
+
+    def _setup(self, container_name, container_type, profile):
+        """
+        Launch a container, enable ssh, install dependencies
+        and setup the needed environment variables
+        """
+        self.launch(container_name)
+        self.enable_ssh(container_name)
+        self.install(
+            container_name,
+            packages=PROFILES[profile][container_type]["package_name"],
+            channel=PROFILES[profile][container_type]["channel"])
+        self.install_dependencies(container_name, container_type)
+        self.configure(container_name, container_type, profile)
+
+    def get_ip(self, container_name):
+        """Gets the ip address for a given container"""
+        networks = self.client.containers.get(
+            container_name).state().network['eth0']['addresses']
+
+        for network in networks:
+            if network['address']:
+                return network['address']
+        raise RuntimeError(
+            "The container {} does not have an IPV4 connection".format(
+                container_name))

=== added file 'snappy_ecosystem_tests/environment/setup.py'
--- snappy_ecosystem_tests/environment/setup.py	1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/setup.py	2017-03-03 10:19:52 +0000
@@ -0,0 +1,46 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+
+#
+# Ubuntu System Tests
+# Copyright (C) 2017 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+"""Standalone script to setup an ecosystem environment"""
+
+import argparse
+
+import logging
+
+from snappy_ecosystem_tests.environment.managers import get_manager
+
+logging.basicConfig(level=logging.DEBUG)
+
+def main(mode, profile):
+    """Main script"""
+    manager = get_manager(mode)
+    manager.setup(profile)
+
+
+if __name__ == '__main__':
+    PARSER = argparse.ArgumentParser(description=
+                                     'Setup a snappy ecosystem environment.')
+    PARSER.add_argument('--mode', default="lxd",
+                        help='lxd')
+    PARSER.add_argument('--profile', default="staging",
+                        help='Profile to configure the environment')
+    ARGS = PARSER.parse_args()
+
+    main(ARGS.mode, ARGS.profile)

=== modified file 'snappy_ecosystem_tests/helpers/snapd/staging_builder.py'
--- snappy_ecosystem_tests/helpers/snapd/staging_builder.py	2017-02-21 13:46:08 +0000
+++ snappy_ecosystem_tests/helpers/snapd/staging_builder.py	2017-03-03 10:19:52 +0000
@@ -24,63 +24,10 @@
 import sys
 from time import sleep
 
+from snappy_ecosystem_tests.environment.data.snapd import (
+    CONTAINER_HOME, CONTAINER_ENV_VARS, CONTAINER_SETUP)
 from snappy_ecosystem_tests.utils.lxd import Container, launch_container
 
-CONTAINER_HOME = '/root'
-DIRECTORY_CLONE = '{}/src/github.com/snapcore/snapd'.format(CONTAINER_HOME)
-ENV_STORE_API_ROOT = 'UBUNTU_STORE_API_ROOT_URL'
-ENV_STORE_SEARCH_ROOT = 'UBUNTU_STORE_SEARCH_ROOT_URL'
-ENV_STORE_UPLOAD_ROOT = 'UBUNTU_STORE_UPLOAD_ROOT_URL'
-ENV_SSO_API_ROOT = 'UBUNTU_SSO_API_ROOT_URL'
-PACKAGES_SNAPD = ['snapd', 'snap-confine', 'ubuntu-core-launcher']
-COMMAND_REMOVE_SNAPD = 'apt purge -y {}'.format(' '.join(PACKAGES_SNAPD))
-COMMAND_APT_BUILD_DEP = 'apt build-dep -y ./'
-COMMAND_APT_INSTALL = 'apt install -y {}'
-COMMAND_APT_INSTALL_STAGING_DEBS = COMMAND_APT_INSTALL.format(
-    ' '.join(['../{}*.deb'.format(pkg) for pkg in PACKAGES_SNAPD]))
-COMMAND_APT_UPDATE = 'apt update'
-COMMAND_APT_UPGRADE = 'apt dist-upgrade -y'
-COMMAND_BUILD_SNAPD = 'DEB_BUILD_OPTIONS="nocheck testkeys" ' \
-                      'dpkg-buildpackage -tc -b'
-COMMAND_EXPORT_STAGING_STORE_VAR = 'echo SNAPPY_USE_STAGING_STORE=1 >> ' \
-                                   '/etc/environment'
-COMMAND_GET_GOVENDOR = 'go get -v github.com/kardianos/govendor'
-COMMAND_GIT_CLONE = 'git clone {} {}'
-COMMAND_GOVENDOR_SYNC = 'govendor sync'
-REPOSITORY_GIT = 'https://github.com/snapcore/snapd'
-URL_API_ROOT = 'https://myapps.developer.staging.ubuntu.com/dev/api/'
-URL_SEARCH_ROOT = 'https://search.apps.staging.ubuntu.com/'
-URL_UPLOAD_ROOT = 'https://upload.apps.staging.ubuntu.com/'
-URL_SSO_API_ROOT = 'https://login.staging.ubuntu.com/api/v2/'
-
-CONTAINER_SETUP = [
-    COMMAND_REMOVE_SNAPD,
-    COMMAND_APT_UPDATE,
-    # Hold open-iscsi which is causing apt upgrade errors while running
-    # inside container.
-    'apt-mark hold open-iscsi',
-    COMMAND_APT_UPGRADE,
-    # Needed for snaps to work inside a lxd container
-    #  ref: https://stgraber.org/2016/12/07/running-snaps-in-lxd-containers/
-    COMMAND_APT_INSTALL.format('squashfuse'),
-    COMMAND_GIT_CLONE.format(REPOSITORY_GIT, DIRECTORY_CLONE),
-    {'command': COMMAND_APT_BUILD_DEP, 'cwd': DIRECTORY_CLONE},
-    {'command': COMMAND_GET_GOVENDOR, 'cwd': DIRECTORY_CLONE},
-    {'command': COMMAND_GOVENDOR_SYNC, 'cwd': DIRECTORY_CLONE},
-    {'command': COMMAND_BUILD_SNAPD, 'cwd': DIRECTORY_CLONE},
-    COMMAND_EXPORT_STAGING_STORE_VAR,
-    {'command': COMMAND_APT_INSTALL_STAGING_DEBS, 'cwd': DIRECTORY_CLONE}
-]
-
-CONTAINER_ENV_VARS = {
-    'GOPATH': CONTAINER_HOME,
-    'SNAPPY_USE_STAGING_STORE': '1',
-    ENV_STORE_API_ROOT: URL_API_ROOT,
-    ENV_STORE_SEARCH_ROOT: URL_SEARCH_ROOT,
-    ENV_STORE_UPLOAD_ROOT: URL_UPLOAD_ROOT,
-    ENV_SSO_API_ROOT: URL_SSO_API_ROOT,
-}
-
 LOGGER = logging.getLogger(__name__)