Merge lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment into lp:snappy-ecosystem-tests

Proposed by Santiago Baldassin
Status: Merged
Approved by: Omer Akram
Approved revision: 27
Merged at revision: 25
Proposed branch: lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment
Merge into: lp:snappy-ecosystem-tests
Diff against target: 535 lines (+407/-58)
10 files modified
requirements.txt (+1/-1)
run_checks (+2/-2)
snappy_ecosystem_tests/environment/__init__.py (+19/-0)
snappy_ecosystem_tests/environment/constants.py (+66/-0)
snappy_ecosystem_tests/environment/data/__init__.py (+19/-0)
snappy_ecosystem_tests/environment/data/snapcraft.py (+21/-0)
snappy_ecosystem_tests/environment/data/snapd.py (+88/-0)
snappy_ecosystem_tests/environment/managers.py (+143/-0)
snappy_ecosystem_tests/environment/setup.py (+46/-0)
snappy_ecosystem_tests/helpers/snapd/staging_builder.py (+2/-55)
To merge this branch: bzr merge lp:~sbaldassin/snappy-ecosystem-tests/ecosystem_deployment
Reviewer Review Type Date Requested Status
Omer Akram (community) Approve
platform-qa-bot continuous-integration Approve
Heber Parrucci (community) Approve
prod-platform-qa continuous-integration Pending
Review via email: mp+318018@code.launchpad.net

Commit message

Scripts to build a couple of lxd containers to deploy snapcraft and snapd

Description of the change

Scripts to build a couple of lxd containers to deploy snapcraft and snapd

To post a comment you must log in.
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Heber Parrucci (heber013) wrote :

Comments inline.

Additionally:
* We already have in trunk some code for building a snapd container for staging:
snappy_ecosystem_tests/helpers/snapd/staging_builder.py
and some container helpers:
snappy_ecosystem_tests/utils/lxd.py
Some parts of that code seem to overlap with this changes. My suggestion is to merge that code with yours to have one place for building containers.

Update the README with instructions to build the environment.

Add logging to see the activities and progress while building the containers... When I run the setup I just see the cursor for some minutes and have no idea what is happening behind the scene.

I am getting an error when executing the setup: http://paste.ubuntu.com/24052852/
When it tries to enable ssh, it tries to put the public key in the container, and it seems that the file /home/ubuntu/.ssh/authorized_keys does not exist

review: Needs Fixing
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
19. By Santiago Baldassin

Addressing comments from the reviews

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

All comments addressed

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
I Ahmad (iahmad) wrote :

Could this be landed soon as heber's other mp is waiting on it. Please update readme with instructions before landing.

Revision history for this message
Omer Akram (om26er) wrote :

Minor cosmetic review. Need to do a thorough review later.

Revision history for this message
Omer Akram (om26er) :
review: Needs Fixing
20. By Santiago Baldassin

Fixing the script

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Thanks Omer. All comments were addressed

21. By Santiago Baldassin

Addressing comments from the reviews

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
22. By Santiago Baldassin

Merge from trunk

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
23. By Santiago Baldassin

Adding a get_ip api

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
24. By Santiago Baldassin

Minor fix

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
25. By Santiago Baldassin

Minor change

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Needs Fixing (continuous-integration)
26. By Santiago Baldassin

fixing pylint

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Heber Parrucci (heber013) wrote :

Code LGTM

review: Approve
Revision history for this message
Omer Akram (om26er) wrote :

I have done the code review of the LxdManager class. The problems that were previously mentioned for the Container class were either non existent(for example a container getting out of sync, found after more research, pylxd is intelligent), or duplicate object creation, which I explained in a separate email, detailed, that its not an issue on the memory as we were only passing in a reference to an already existing object and the one that we were passing that object to was not more than the size of an empty object (a class that inherits from object but does not add any override methods).

review: Needs Fixing
Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

> I have done the code review of the LxdManager class. The problems that were
> previously mentioned for the Container class were either non existent(for
> example a container getting out of sync, found after more research, pylxd is
> intelligent), or duplicate object creation, which I explained in a separate
> email, detailed, that its not an issue on the memory as we were only passing
> in a reference to an already existing object and the one that we were passing
> that object to was not more than the size of an empty object (a class that
> inherits from object but does not add any override methods).

Let's go through the approach over the phone. Heber and I clearly agree that the best course of action is to use a manager and to avoid the use of the Container class. Once agreed on the approach we'll get rid of the either the manager or the container class to avoid code duplication

Revision history for this message
Omer Akram (om26er) wrote :

Phone call sounds good to me and as an added bonus this is what the new(lean) code would look like: http://paste.ubuntu.com/24092575/ is pseudo code, so may not work as-is.

I do understand that you have certain preference for the "manager" paradigm but in this specific case that doesn't feel like solving any problem :)

27. By Santiago Baldassin

Minor fix

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Omer Akram (om26er) wrote :

Lets land this.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'requirements.txt'
--- requirements.txt 2017-02-21 21:07:33 +0000
+++ requirements.txt 2017-03-03 10:19:52 +0000
@@ -14,7 +14,7 @@
14pexpect14pexpect
15pymacaroons==0.9.215pymacaroons==0.9.2
16requests-toolbelt==0.6.016requests-toolbelt==0.6.0
17#chromedriver_installer
18pylxd17pylxd
19pyyaml18pyyaml
19retrying
20paramiko20paramiko
2121
=== modified file 'run_checks'
--- run_checks 2017-02-20 13:10:18 +0000
+++ run_checks 2017-03-03 10:19:52 +0000
@@ -51,9 +51,9 @@
5151
52 if [ "$proxy" ]; then52 if [ "$proxy" ]; then
53 echo "Using proxy: $proxy to install dependencies"53 echo "Using proxy: $proxy to install dependencies"
54 pip install --proxy $proxy -r requirements-unit-tests.txt54 pip3 install --proxy $proxy -r requirements-unit-tests.txt
55 else55 else
56 pip install -r requirements-unit-tests.txt56 pip3 install -r requirements-unit-tests.txt
57 fi57 fi
5858
59 ve_unit_tests/bin/python3 -m unittest discover snappy_ecosystem_tests.unittests 59 ve_unit_tests/bin/python3 -m unittest discover snappy_ecosystem_tests.unittests
6060
=== added directory 'snappy_ecosystem_tests/environment'
=== added file 'snappy_ecosystem_tests/environment/__init__.py'
--- snappy_ecosystem_tests/environment/__init__.py 1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/__init__.py 2017-03-03 10:19:52 +0000
@@ -0,0 +1,19 @@
1# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2
3#
4# Snappy Ecosystem Tests
5# Copyright (C) 2017 Canonical
6#
7# This program is free software: you can redistribute it and/or modify
8# it under the terms of the GNU General Public License as published by
9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program. If not, see <http://www.gnu.org/licenses/>.
19#
020
=== added file 'snappy_ecosystem_tests/environment/constants.py'
--- snappy_ecosystem_tests/environment/constants.py 1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/constants.py 2017-03-03 10:19:52 +0000
@@ -0,0 +1,66 @@
1# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2
3#
4# Snappy Ecosystem Tests
5# Copyright (C) 2017 Canonical
6#
7# This program is free software: you can redistribute it and/or modify
8# it under the terms of the GNU General Public License as published by
9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program. If not, see <http://www.gnu.org/licenses/>.
19#
20
21"""Constants used accross the environment setup"""
22
23SNAPD = 'snapd'
24SNAPCRAFT = 'snapcraft'
25
26DEPENDENCIES = {
27 SNAPD : [],
28 SNAPCRAFT : [],
29 "shared" : ["expect"]
30}
31
32PROFILES = {
33 "staging":{
34 SNAPD:{
35 "channel": "xenial",
36 "package_name": 'snapd',
37 "environment_variables": {
38 "UBUNTU_STORE_API_ROOT_URL":
39 "https://myapps.developer.ubuntu.com/dev/api/",
40 "UBUNTU_STORE_UPLOAD_ROOT_URL":
41 "https://upload.apps.ubuntu.com",
42 "UBUNTU_SSO_API_ROOT_URL":
43 "https://login.ubuntu.com/api/v2/",
44 "UBUNTU_STORE_SEARCH_ROOT_URL":
45 "https://search.apps.ubuntu.com/"
46 }
47 },
48 SNAPCRAFT: {
49 "channel": "xenial",
50 "package_name": 'snapcraft',
51 "environment_variables": {
52 "UBUNTU_STORE_API_ROOT_URL":
53 "https://myapps.developer.ubuntu.com/dev/api/",
54 "UBUNTU_STORE_UPLOAD_ROOT_URL":
55 "https://upload.apps.ubuntu.com",
56 "UBUNTU_SSO_API_ROOT_URL":
57 "https://login.ubuntu.com/api/v2/",
58 "UBUNTU_STORE_SEARCH_ROOT_URL":
59 "https://search.apps.ubuntu.com/"
60 }
61 },
62 "snapweb":{
63 "url": "https://myapps.developer.ubuntu.com/"
64 },
65 }
66}
067
=== added directory 'snappy_ecosystem_tests/environment/data'
=== added file 'snappy_ecosystem_tests/environment/data/__init__.py'
--- snappy_ecosystem_tests/environment/data/__init__.py 1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/data/__init__.py 2017-03-03 10:19:52 +0000
@@ -0,0 +1,19 @@
1# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2
3#
4# Snappy Ecosystem Tests
5# Copyright (C) 2017 Canonical
6#
7# This program is free software: you can redistribute it and/or modify
8# it under the terms of the GNU General Public License as published by
9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program. If not, see <http://www.gnu.org/licenses/>.
19#
020
=== added file 'snappy_ecosystem_tests/environment/data/snapcraft.py'
--- snappy_ecosystem_tests/environment/data/snapcraft.py 1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/data/snapcraft.py 2017-03-03 10:19:52 +0000
@@ -0,0 +1,21 @@
1# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2
3#
4# Snappy Ecosystem Tests
5# Copyright (C) 2017 Canonical
6#
7# This program is free software: you can redistribute it and/or modify
8# it under the terms of the GNU General Public License as published by
9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program. If not, see <http://www.gnu.org/licenses/>.
19#
20"""Snapcraft data"""
21SNAPCRAFT_CONTAINER_NAME = 'snapcraft'
022
=== added file 'snappy_ecosystem_tests/environment/data/snapd.py'
--- snappy_ecosystem_tests/environment/data/snapd.py 1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/data/snapd.py 2017-03-03 10:19:52 +0000
@@ -0,0 +1,88 @@
1# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2
3#
4# Snappy Ecosystem Tests
5# Copyright (C) 2017 Canonical
6#
7# This program is free software: you can redistribute it and/or modify
8# it under the terms of the GNU General Public License as published by
9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program. If not, see <http://www.gnu.org/licenses/>.
19#
20"""snapd data"""
21SNAPD_CONTAINER_NAME = 'snapd'
22
23CONTAINER_HOME = '/root'
24DIRECTORY_CLONE = '{}/src/github.com/snapcore/snapd'.format(CONTAINER_HOME)
25DOMAIN_PING = 'ubuntu.com'
26ENV_STORE_API_ROOT = 'UBUNTU_STORE_API_ROOT_URL'
27ENV_STORE_SEARCH_ROOT = 'UBUNTU_STORE_SEARCH_ROOT_URL'
28ENV_STORE_UPLOAD_ROOT = 'UBUNTU_STORE_UPLOAD_ROOT_URL'
29ENV_SSO_API_ROOT = 'UBUNTU_SSO_API_ROOT_URL'
30PACKAGES_SNAPD = ['snapd', 'snap-confine', 'ubuntu-core-launcher']
31COMMAND_REMOVE_SNAPD = 'apt purge -y {}'.format(' '.join(PACKAGES_SNAPD))
32COMMAND_APT_BUILD_DEP = 'apt build-dep -y ./'
33COMMAND_APT_INSTALL = 'apt install -y {}'
34COMMAND_APT_INSTALL_STAGING_DEBS = COMMAND_APT_INSTALL.format(
35 ' '.join(['../{}*.deb'.format(pkg) for pkg in PACKAGES_SNAPD]))
36COMMAND_APT_UPDATE = 'apt update'
37COMMAND_APT_UPGRADE = 'apt dist-upgrade -y'
38COMMAND_BUILD_SNAPD = 'DEB_BUILD_OPTIONS="nocheck testkeys" ' \
39 'dpkg-buildpackage -tc -b'
40COMMAND_EXPORT_STAGING_STORE_VAR = 'echo SNAPPY_USE_STAGING_STORE=1 >> ' \
41 '/etc/environment'
42COMMAND_GET_GOVENDOR = 'go get -v github.com/kardianos/govendor'
43COMMAND_GIT_CLONE = 'git clone {} {}'
44COMMAND_GOVENDOR_SYNC = 'govendor sync'
45REPOSITORY_GIT = 'https://github.com/snapcore/snapd'
46URL_API_ROOT = 'https://myapps.developer.staging.ubuntu.com/dev/api/'
47URL_SEARCH_ROOT = 'https://search.apps.staging.ubuntu.com/'
48URL_UPLOAD_ROOT = 'https://upload.apps.staging.ubuntu.com/'
49URL_SSO_API_ROOT = 'https://login.staging.ubuntu.com/api/v2/'
50
51CONTAINER_SETUP = [
52 COMMAND_REMOVE_SNAPD,
53 COMMAND_APT_UPDATE,
54 # Hold open-iscsi which is causing apt upgrade errors while running
55 # inside container.
56 'apt-mark hold open-iscsi',
57 COMMAND_APT_UPGRADE,
58 # Needed for snaps to work inside a lxd container
59 # ref: https://stgraber.org/2016/12/07/running-snaps-in-lxd-containers/
60 COMMAND_APT_INSTALL.format('squashfuse'),
61 COMMAND_GIT_CLONE.format(REPOSITORY_GIT, DIRECTORY_CLONE),
62 {'command': COMMAND_APT_BUILD_DEP, 'cwd': DIRECTORY_CLONE},
63 {'command': COMMAND_GET_GOVENDOR, 'cwd': DIRECTORY_CLONE},
64 {'command': COMMAND_GOVENDOR_SYNC, 'cwd': DIRECTORY_CLONE},
65 {'command': COMMAND_BUILD_SNAPD, 'cwd': DIRECTORY_CLONE},
66 COMMAND_EXPORT_STAGING_STORE_VAR,
67 {'command': COMMAND_APT_INSTALL_STAGING_DEBS, 'cwd': DIRECTORY_CLONE}
68]
69
70CONTAINER_ENV_VARS = {
71 'GOPATH': CONTAINER_HOME,
72 'SNAPPY_USE_STAGING_STORE': '1',
73 ENV_STORE_API_ROOT: URL_API_ROOT,
74 ENV_STORE_SEARCH_ROOT: URL_SEARCH_ROOT,
75 ENV_STORE_UPLOAD_ROOT: URL_UPLOAD_ROOT,
76 ENV_SSO_API_ROOT: URL_SSO_API_ROOT,
77}
78
79DEFAULT_CONTAINER_CONFIG = {
80 "name": "snapd",
81 "source": {
82 "type": "image",
83 "mode": "pull",
84 "server": "https://cloud-images.ubuntu.com/releases",
85 "protocol": "simplestreams",
86 "fingerprint": "16.04"
87 }
88}
089
=== added file 'snappy_ecosystem_tests/environment/managers.py'
--- snappy_ecosystem_tests/environment/managers.py 1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/managers.py 2017-03-03 10:19:52 +0000
@@ -0,0 +1,143 @@
1# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2
3#
4# Ubuntu System Tests
5# Copyright (C) 2017 Canonical
6#
7# This program is free software: you can redistribute it and/or modify
8# it under the terms of the GNU General Public License as published by
9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program. If not, see <http://www.gnu.org/licenses/>.
19#
20
21"""Managers to setup the environment"""
22
23import os
24import shlex
25from pylxd import Client
26from pylxd.exceptions import NotFound, LXDAPIException
27from retrying import retry
28
29from snappy_ecosystem_tests.environment import constants
30from snappy_ecosystem_tests.environment.constants import (
31 PROFILES, DEPENDENCIES)
32from snappy_ecosystem_tests.environment.data.snapcraft import (
33 SNAPCRAFT_CONTAINER_NAME)
34from snappy_ecosystem_tests.environment.data.snapd import (
35 DEFAULT_CONTAINER_CONFIG, SNAPD_CONTAINER_NAME)
36
37def get_manager(mode):
38 """Get the manager for the given mode"""
39 supported_modules = {
40 "lxd": LxdManager
41 }
42 try:
43 manager = supported_modules[mode]
44 return manager()
45 except KeyError:
46 raise RuntimeError("Mode: {} is not supported".format(mode))
47
48
49class LxdManager:
50 """Manage lxd containers"""
51
52 def __init__(self):
53 self.client = Client()
54
55 def launch(self, container_name, release='16.04'):
56 """Launch a container"""
57 try:
58 container = self.client.containers.get(container_name)
59 if container.status == "Running":
60 container.stop(wait=True)
61 container.delete(wait=True)
62 except NotFound:
63 pass
64 container_config = DEFAULT_CONTAINER_CONFIG.copy()
65 container_config['source']['alias'] = release
66 container_config['name'] = container_name
67 container = self.client.containers.create(container_config, wait=True)
68 container.start(wait=True)
69
70 def execute_command(self, container_name, command):
71 """execute command"""
72 container = self.client.containers.get(container_name)
73 container.execute(command)
74
75 def install(self, container_name, packages, channel=None):
76 """Install a packages"""
77 container = self.client.containers.get(container_name)
78 container.execute(shlex.split('apt-get update'))
79 install_command = 'apt-get install {} -y'.format(packages)
80 if channel:
81 install_command += ' -t {}'.format(channel)
82 container.execute(shlex.split(install_command))
83
84 def install_dependencies(self, container_name, container_type):
85 """Install a dependencies"""
86 self.install(
87 container_name,
88 DEPENDENCIES[container_type] + DEPENDENCIES['shared'])
89
90
91 def configure(self, container_name, container_type, profile):
92 """Configure a given container"""
93 container = self.client.containers.get(container_name)
94 for key, value in \
95 PROFILES[profile][container_type]['environment_variables'].\
96 items():
97 container.config.update({'environment.{}'.format(
98 key): value})
99 container.save(wait=True)
100
101 @retry(stop_max_attempt_number=5, wait_fixed=3000,
102 retry_on_exception=lambda exception: isinstance(
103 exception, LXDAPIException))
104 def enable_ssh(self, container_name):
105 """Enable the ssh connection on the container"""
106 container = self.client.containers.get(container_name)
107 pub_key = open(
108 os.path.expanduser('~') + '/.ssh/id_rsa.pub').read().strip('\n')
109 container.files.put('/home/ubuntu/.ssh/authorized_keys', pub_key)
110
111
112 def setup(self, profile):
113 """setup the container based on the profile"""
114 self._setup(SNAPD_CONTAINER_NAME, constants.SNAPD, profile)
115 self._setup(
116 SNAPCRAFT_CONTAINER_NAME, constants.SNAPCRAFT, profile)
117
118
119 def _setup(self, container_name, container_type, profile):
120 """
121 Launch a container, enable ssh, install dependencies
122 and setup the needed environment variables
123 """
124 self.launch(container_name)
125 self.enable_ssh(container_name)
126 self.install(
127 container_name,
128 packages=PROFILES[profile][container_type]["package_name"],
129 channel=PROFILES[profile][container_type]["channel"])
130 self.install_dependencies(container_name, container_type)
131 self.configure(container_name, container_type, profile)
132
133 def get_ip(self, container_name):
134 """Gets the ip address for a given container"""
135 networks = self.client.containers.get(
136 container_name).state().network['eth0']['addresses']
137
138 for network in networks:
139 if network['address']:
140 return network['address']
141 raise RuntimeError(
142 "The container {} does not have an IPV4 connection".format(
143 container_name))
0144
=== added file 'snappy_ecosystem_tests/environment/setup.py'
--- snappy_ecosystem_tests/environment/setup.py 1970-01-01 00:00:00 +0000
+++ snappy_ecosystem_tests/environment/setup.py 2017-03-03 10:19:52 +0000
@@ -0,0 +1,46 @@
1# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2
3#
4# Ubuntu System Tests
5# Copyright (C) 2017 Canonical
6#
7# This program is free software: you can redistribute it and/or modify
8# it under the terms of the GNU General Public License as published by
9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program. If not, see <http://www.gnu.org/licenses/>.
19#
20
21"""Standalone script to setup an ecosystem environment"""
22
23import argparse
24
25import logging
26
27from snappy_ecosystem_tests.environment.managers import get_manager
28
29logging.basicConfig(level=logging.DEBUG)
30
31def main(mode, profile):
32 """Main script"""
33 manager = get_manager(mode)
34 manager.setup(profile)
35
36
37if __name__ == '__main__':
38 PARSER = argparse.ArgumentParser(description=
39 'Setup a snappy ecosystem environment.')
40 PARSER.add_argument('--mode', default="lxd",
41 help='lxd')
42 PARSER.add_argument('--profile', default="staging",
43 help='Profile to configure the environment')
44 ARGS = PARSER.parse_args()
45
46 main(ARGS.mode, ARGS.profile)
047
=== modified file 'snappy_ecosystem_tests/helpers/snapd/staging_builder.py'
--- snappy_ecosystem_tests/helpers/snapd/staging_builder.py 2017-02-21 13:46:08 +0000
+++ snappy_ecosystem_tests/helpers/snapd/staging_builder.py 2017-03-03 10:19:52 +0000
@@ -24,63 +24,10 @@
24import sys24import sys
25from time import sleep25from time import sleep
2626
27from snappy_ecosystem_tests.environment.data.snapd import (
28 CONTAINER_HOME, CONTAINER_ENV_VARS, CONTAINER_SETUP)
27from snappy_ecosystem_tests.utils.lxd import Container, launch_container29from snappy_ecosystem_tests.utils.lxd import Container, launch_container
2830
29CONTAINER_HOME = '/root'
30DIRECTORY_CLONE = '{}/src/github.com/snapcore/snapd'.format(CONTAINER_HOME)
31ENV_STORE_API_ROOT = 'UBUNTU_STORE_API_ROOT_URL'
32ENV_STORE_SEARCH_ROOT = 'UBUNTU_STORE_SEARCH_ROOT_URL'
33ENV_STORE_UPLOAD_ROOT = 'UBUNTU_STORE_UPLOAD_ROOT_URL'
34ENV_SSO_API_ROOT = 'UBUNTU_SSO_API_ROOT_URL'
35PACKAGES_SNAPD = ['snapd', 'snap-confine', 'ubuntu-core-launcher']
36COMMAND_REMOVE_SNAPD = 'apt purge -y {}'.format(' '.join(PACKAGES_SNAPD))
37COMMAND_APT_BUILD_DEP = 'apt build-dep -y ./'
38COMMAND_APT_INSTALL = 'apt install -y {}'
39COMMAND_APT_INSTALL_STAGING_DEBS = COMMAND_APT_INSTALL.format(
40 ' '.join(['../{}*.deb'.format(pkg) for pkg in PACKAGES_SNAPD]))
41COMMAND_APT_UPDATE = 'apt update'
42COMMAND_APT_UPGRADE = 'apt dist-upgrade -y'
43COMMAND_BUILD_SNAPD = 'DEB_BUILD_OPTIONS="nocheck testkeys" ' \
44 'dpkg-buildpackage -tc -b'
45COMMAND_EXPORT_STAGING_STORE_VAR = 'echo SNAPPY_USE_STAGING_STORE=1 >> ' \
46 '/etc/environment'
47COMMAND_GET_GOVENDOR = 'go get -v github.com/kardianos/govendor'
48COMMAND_GIT_CLONE = 'git clone {} {}'
49COMMAND_GOVENDOR_SYNC = 'govendor sync'
50REPOSITORY_GIT = 'https://github.com/snapcore/snapd'
51URL_API_ROOT = 'https://myapps.developer.staging.ubuntu.com/dev/api/'
52URL_SEARCH_ROOT = 'https://search.apps.staging.ubuntu.com/'
53URL_UPLOAD_ROOT = 'https://upload.apps.staging.ubuntu.com/'
54URL_SSO_API_ROOT = 'https://login.staging.ubuntu.com/api/v2/'
55
56CONTAINER_SETUP = [
57 COMMAND_REMOVE_SNAPD,
58 COMMAND_APT_UPDATE,
59 # Hold open-iscsi which is causing apt upgrade errors while running
60 # inside container.
61 'apt-mark hold open-iscsi',
62 COMMAND_APT_UPGRADE,
63 # Needed for snaps to work inside a lxd container
64 # ref: https://stgraber.org/2016/12/07/running-snaps-in-lxd-containers/
65 COMMAND_APT_INSTALL.format('squashfuse'),
66 COMMAND_GIT_CLONE.format(REPOSITORY_GIT, DIRECTORY_CLONE),
67 {'command': COMMAND_APT_BUILD_DEP, 'cwd': DIRECTORY_CLONE},
68 {'command': COMMAND_GET_GOVENDOR, 'cwd': DIRECTORY_CLONE},
69 {'command': COMMAND_GOVENDOR_SYNC, 'cwd': DIRECTORY_CLONE},
70 {'command': COMMAND_BUILD_SNAPD, 'cwd': DIRECTORY_CLONE},
71 COMMAND_EXPORT_STAGING_STORE_VAR,
72 {'command': COMMAND_APT_INSTALL_STAGING_DEBS, 'cwd': DIRECTORY_CLONE}
73]
74
75CONTAINER_ENV_VARS = {
76 'GOPATH': CONTAINER_HOME,
77 'SNAPPY_USE_STAGING_STORE': '1',
78 ENV_STORE_API_ROOT: URL_API_ROOT,
79 ENV_STORE_SEARCH_ROOT: URL_SEARCH_ROOT,
80 ENV_STORE_UPLOAD_ROOT: URL_UPLOAD_ROOT,
81 ENV_SSO_API_ROOT: URL_SSO_API_ROOT,
82}
83
84LOGGER = logging.getLogger(__name__)31LOGGER = logging.getLogger(__name__)
8532
8633

Subscribers

People subscribed via source and target branches

to all changes: