Merge lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver into lp:snappy-ecosystem-tests

Proposed by Heber Parrucci
Status: Merged
Approved by: Santiago Baldassin
Approved revision: 33
Merged at revision: 33
Proposed branch: lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver
Merge into: lp:snappy-ecosystem-tests
Diff against target: 520 lines (+168/-171)
6 files modified
snappy_ecosystem_tests/environment/setup.py (+1/-0)
snappy_ecosystem_tests/helpers/snapcraft/build_snap.py (+17/-9)
snappy_ecosystem_tests/helpers/snapcraft/client.py (+22/-24)
snappy_ecosystem_tests/helpers/snapd/snapd.py (+112/-103)
snappy_ecosystem_tests/tests/test_snapd.py (+5/-4)
snappy_ecosystem_tests/utils/ssh.py (+11/-31)
To merge this branch: bzr merge lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver
Reviewer Review Type Date Requested Status
Santiago Baldassin (community) Approve
platform-qa-bot continuous-integration Approve
Omer Akram (community) Approve
Review via email: mp+318948@code.launchpad.net

Commit message

Adding SSHClientDriver to:
* Avoid passing the host, username, port in every run_command execution.
* Customize run command with cwd option
* Add pull method
* Ensure we execute each command against an active connection provided by the SSHManager

Description of the change

Centralize a way to do operations against an ssh client to:

* Avoid passing the host, username, port in every run_command execution.
* Customize run command with cwd option
* Add pull method
* Ensure we execute each command against an active connection provided by the SSHManager

To post a comment you must log in.
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Omer Akram (om26er) wrote :

Excuse my ignorance, can you tell what problem this branch tries to solve ? From the look of it, it seems to be making things more complex. The SSHClientDriver is now the only consumer of SSHManager.

I do see the SSHClientDriver gets a connected instance from the manager each time a command is run, this does solve a problem, though that will only be relevant if our tests were running over the internet, right now they are running on the local network.

Do you know if in future our tests will be running over the internet instead of the local network where the risk of ssh connection drop practically diminishes.

review: Needs Information
Revision history for this message
Heber Parrucci (heber013) wrote :

> Excuse my ignorance, can you tell what problem this branch tries to solve ?
> From the look of it, it seems to be making things more complex. The
> SSHClientDriver is now the only consumer of SSHManager.
>
> I do see the SSHClientDriver gets a connected instance from the manager each
> time a command is run, this does solve a problem, though that will only be
> relevant if our tests were running over the internet, right now they are
> running on the local network.
>
> Do you know if in future our tests will be running over the internet instead
> of the local network where the risk of ssh connection drop practically
> diminishes.

Sure no problem!
The change description might not be clear enough. So I will provide more details on each item:

* Avoid passing the host, username, port in every run_command execution. --> Currently we have a run_command at module level so we need to provide host, username, etc every time we call it. Encapsulating the logic in a class and providing the parameters in the object initialization, we ensure that the ssh specific parameters will be provided once per remote helper.
* Customize run command with cwd option --> per your request, I added cwd named parameter in run_command so you can specify the dir in which the command will be executed.
* Add pull method --> the pull was also at module level.
* Ensure we execute each command against an active connection provided by the SSHManager --> the fact that the ssh connections are more likely to drop if tests are executed over internet, does not alter the design (the keywords are *more likely*). I mean, we could use the same client object as well, but in this case we would me assuming the risk. Do you note any performance issues of requesting an active connection in every command? Remember that it is stored in the Manager connections pool. Furthermore, if I change the logic to use the same connection on the ssh driver, I will not be using the feature that the SshManager provides.

Finally, we currently have on trunk a custom SShClient class that extends the paramiko one and overrides exec_command just to provide a dir in which the command will be executed, and it is not using our manager to get the ssh connection.

Let me know if that make more sense for you now.

Revision history for this message
Heber Parrucci (heber013) wrote :

> > Excuse my ignorance, can you tell what problem this branch tries to solve ?
> > From the look of it, it seems to be making things more complex. The
> > SSHClientDriver is now the only consumer of SSHManager.
> >
> > I do see the SSHClientDriver gets a connected instance from the manager each
> > time a command is run, this does solve a problem, though that will only be
> > relevant if our tests were running over the internet, right now they are
> > running on the local network.
> >
> > Do you know if in future our tests will be running over the internet instead
> > of the local network where the risk of ssh connection drop practically
> > diminishes.
>
> Sure no problem!
> The change description might not be clear enough. So I will provide more
> details on each item:
>
>
> * Avoid passing the host, username, port in every run_command execution. -->
> Currently we have a run_command at module level so we need to provide host,
> username, etc every time we call it. Encapsulating the logic in a class and
> providing the parameters in the object initialization, we ensure that the ssh
> specific parameters will be provided once per remote helper.
> * Customize run command with cwd option --> per your request, I added cwd
> named parameter in run_command so you can specify the dir in which the command
> will be executed.
> * Add pull method --> the pull was also at module level.
> * Ensure we execute each command against an active connection provided by the
> SSHManager --> the fact that the ssh connections are more likely to drop if
> tests are executed over internet, does not alter the design (the keywords are
> *more likely*). I mean, we could use the same client object as well, but in
> this case we would me assuming the risk. Do you note any performance issues of
> requesting an active connection in every command? Remember that it is stored
> in the Manager connections pool. Furthermore, if I change the logic to use the
> same connection on the ssh driver, I will not be using the feature that the
> SshManager provides.
>
>
> Finally, we currently have on trunk a custom SShClient class that extends the
> paramiko one and overrides exec_command just to provide a dir in which the
> command will be executed, and it is not using our manager to get the ssh
> connection.
>
> Let me know if that make more sense for you now.

I forgot, we currently have on trunk two alternatives to run a ssh command:

1- Calling exec_command of our custom SSHClient (does not use the manager to get an active connection)
2- calling run_command in snappy_ecosystem_tests/utils/ssh.py Which actually uses the manager.

Why would we need 2 ways to execute a ssh command? This change unifies them.

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Looks good in general. Minor comment inline

review: Needs Fixing
Revision history for this message
Omer Akram (om26er) wrote :
Download full text (3.3 KiB)

On Mon, Mar 6, 2017 at 10:22 PM, Heber Parrucci <
<email address hidden>> wrote:

> > Excuse my ignorance, can you tell what problem this branch tries to
> solve ?
> > From the look of it, it seems to be making things more complex. The
> > SSHClientDriver is now the only consumer of SSHManager.
> >
> > I do see the SSHClientDriver gets a connected instance from the manager
> each
> > time a command is run, this does solve a problem, though that will only
> be
> > relevant if our tests were running over the internet, right now they are
> > running on the local network.
> >
> > Do you know if in future our tests will be running over the internet
> instead
> > of the local network where the risk of ssh connection drop practically
> > diminishes.
>
> Sure no problem!
> The change description might not be clear enough. So I will provide more
> details on each item:
>
>
> * Avoid passing the host, username, port in every run_command execution.
> --> Currently we have a run_command at module level so we need to provide
> host, username, etc every time we call it. Encapsulating the logic in a
> class and providing the parameters in the object initialization, we ensure
> that the ssh specific parameters will be provided once per remote helper.
>

That can be done on current trunk right now, SSHManager.get_instance() is
for that.

> * Customize run command with cwd option --> per your request, I added cwd
> named parameter in run_command so you can specify the dir in which the
> command will be executed.
> * Add pull method --> the pull was also at module level.
>

I thought these methods already existed in the extended SSHClient, does
putting them behind a driver make things simpler/easier to consume ?

> * Ensure we execute each command against an active connection provided by
> the SSHManager --> the fact that the ssh connections are more likely to
> drop if tests are executed over internet, does not alter the design (the
> keywords are *more likely*). I mean, we could use the same client object as
> well, but in this case we would me assuming the risk. Do you note any
> performance issues of requesting an active connection in every command?
> Remember that it is stored in the Manager connections pool. Furthermore, if
> I change the logic to use the same connection on the ssh driver, I will not
> be using the feature that the SshManager provides.
>

If we don't run tests over the internet, the chances of connection drop are
practically zero, unless we are hit by a nasty bug.

>
>
> Finally, we currently have on trunk a custom SShClient class that extends
> the paramiko one and overrides exec_command just to provide a dir in which
> the command will be executed, and it is not using our manager to get the
> ssh connection.
>

run_command is indeed redundant in the presence of
SSHManager.get_instance(), we can remove that and *not* introduce
SSHClientDriver if it does not solve any problems we are facing.

> Let me know if that make more sense for you now.
>

Thanks for the detailed response. :)

>
> --
> https://code.launchpad.net/~canonical-platform-qa/snappy-
> ecosystem-tests/adding-ssh-client-driver/+merge/318948
> You are reviewing the ...

Read more...

Revision history for this message
Omer Akram (om26er) wrote :
Download full text (3.6 KiB)

On Mon, Mar 6, 2017 at 10:30 PM, Heber Parrucci <
<email address hidden>> wrote:

> > > Excuse my ignorance, can you tell what problem this branch tries to
> solve ?
> > > From the look of it, it seems to be making things more complex. The
> > > SSHClientDriver is now the only consumer of SSHManager.
> > >
> > > I do see the SSHClientDriver gets a connected instance from the
> manager each
> > > time a command is run, this does solve a problem, though that will
> only be
> > > relevant if our tests were running over the internet, right now they
> are
> > > running on the local network.
> > >
> > > Do you know if in future our tests will be running over the internet
> instead
> > > of the local network where the risk of ssh connection drop practically
> > > diminishes.
> >
> > Sure no problem!
> > The change description might not be clear enough. So I will provide more
> > details on each item:
> >
> >
> > * Avoid passing the host, username, port in every run_command execution.
> -->
> > Currently we have a run_command at module level so we need to provide
> host,
> > username, etc every time we call it. Encapsulating the logic in a class
> and
> > providing the parameters in the object initialization, we ensure that
> the ssh
> > specific parameters will be provided once per remote helper.
> > * Customize run command with cwd option --> per your request, I added cwd
> > named parameter in run_command so you can specify the dir in which the
> command
> > will be executed.
> > * Add pull method --> the pull was also at module level.
> > * Ensure we execute each command against an active connection provided
> by the
> > SSHManager --> the fact that the ssh connections are more likely to drop
> if
> > tests are executed over internet, does not alter the design (the
> keywords are
> > *more likely*). I mean, we could use the same client object as well, but
> in
> > this case we would me assuming the risk. Do you note any performance
> issues of
> > requesting an active connection in every command? Remember that it is
> stored
> > in the Manager connections pool. Furthermore, if I change the logic to
> use the
> > same connection on the ssh driver, I will not be using the feature that
> the
> > SshManager provides.
> >
> >
> > Finally, we currently have on trunk a custom SShClient class that
> extends the
> > paramiko one and overrides exec_command just to provide a dir in which
> the
> > command will be executed, and it is not using our manager to get the ssh
> > connection.
> >
> > Let me know if that make more sense for you now.
>
> I forgot, we currently have on trunk two alternatives to run a ssh command:
>
> 1- Calling exec_command of our custom SSHClient (does not use the manager
> to get an active connection)
>

If a function uses SSHManager.get_instance() it would indeed be getting a
connected instance. If a class instance stores a reference to the
SSHClient, then in case tests are run over the internet we have the issue,
given we will be running them locally the issue does not exist to solve :)

> 2- calling run_command in snappy_ecosystem_tests/utils/ssh.py Which
> actually uses the manager.

> Why would we need 2 ways to execute...

Read more...

Revision history for this message
Heber Parrucci (heber013) wrote :
Download full text (4.6 KiB)

> On Mon, Mar 6, 2017 at 10:30 PM, Heber Parrucci <
> <email address hidden>> wrote:
>
> > > > Excuse my ignorance, can you tell what problem this branch tries to
> > solve ?
> > > > From the look of it, it seems to be making things more complex. The
> > > > SSHClientDriver is now the only consumer of SSHManager.
> > > >
> > > > I do see the SSHClientDriver gets a connected instance from the
> > manager each
> > > > time a command is run, this does solve a problem, though that will
> > only be
> > > > relevant if our tests were running over the internet, right now they
> > are
> > > > running on the local network.
> > > >
> > > > Do you know if in future our tests will be running over the internet
> > instead
> > > > of the local network where the risk of ssh connection drop practically
> > > > diminishes.
> > >
> > > Sure no problem!
> > > The change description might not be clear enough. So I will provide more
> > > details on each item:
> > >
> > >
> > > * Avoid passing the host, username, port in every run_command execution.
> > -->
> > > Currently we have a run_command at module level so we need to provide
> > host,
> > > username, etc every time we call it. Encapsulating the logic in a class
> > and
> > > providing the parameters in the object initialization, we ensure that
> > the ssh
> > > specific parameters will be provided once per remote helper.
> > > * Customize run command with cwd option --> per your request, I added cwd
> > > named parameter in run_command so you can specify the dir in which the
> > command
> > > will be executed.
> > > * Add pull method --> the pull was also at module level.
> > > * Ensure we execute each command against an active connection provided
> > by the
> > > SSHManager --> the fact that the ssh connections are more likely to drop
> > if
> > > tests are executed over internet, does not alter the design (the
> > keywords are
> > > *more likely*). I mean, we could use the same client object as well, but
> > in
> > > this case we would me assuming the risk. Do you note any performance
> > issues of
> > > requesting an active connection in every command? Remember that it is
> > stored
> > > in the Manager connections pool. Furthermore, if I change the logic to
> > use the
> > > same connection on the ssh driver, I will not be using the feature that
> > the
> > > SshManager provides.
> > >
> > >
> > > Finally, we currently have on trunk a custom SShClient class that
> > extends the
> > > paramiko one and overrides exec_command just to provide a dir in which
> > the
> > > command will be executed, and it is not using our manager to get the ssh
> > > connection.
> > >
> > > Let me know if that make more sense for you now.
> >
> > I forgot, we currently have on trunk two alternatives to run a ssh command:
> >
> > 1- Calling exec_command of our custom SSHClient (does not use the manager
> > to get an active connection)
> >
>
> If a function uses SSHManager.get_instance() it would indeed be getting a
> connected instance. If a class instance stores a reference to the
> SSHClient, then in case tests are run over the internet we have the issue,
> given we will be running them locally the issue does not ex...

Read more...

Revision history for this message
Heber Parrucci (heber013) wrote :

> Looks good in general. Minor comment inline
Applies the same response that for Omer:
We are going to run the tests over internet in the future for devices like: raspberry Pi, dragonboards, etc. And those devices will be on the lab (at least for running tests on Jenkins). In those cases we have to ensure that every time we execute a command over ssh, the connection is active.
That's why we cannot save the SshClient instance in the helpers, because we cannot ask to the client itself to get an active connection in every command execution.
The driver was created to resolve that problem. You store an instance in your helper and then execute commands, for every command you execute, it ensures that the ssh connection for that client is active by requesting the manager.
How can we achieve that using the SshClient instead of the driver?

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Thanks for the explanation. What about keeping the ssh client and have something like this?

class EcosystemObject:

    def __init__(ip, user, port):
        self.ip = ip
        self.user = user
        self.port = port

    @property
    def ssh(self):
       return SSHManager.get_instance(self.ip, self.user, self.port)

class Snapcraft(EcosystemObject):

class Snapd(EcosystemObject)

That way every time you instantiate Snapcraft or Snapd, you'll get a property ssh that will always point to an active ssh connection

32. By Heber Parrucci

Addressing review comments:
* Deleting SShDriver and adding back Sshclient
* Cwd parameter removed from client and adding in helper (To mantain paramiko's method signature)
* adding ssh property in each helper that needs ssh, to ensure an active connection for every command execution.

Revision history for this message
Heber Parrucci (heber013) wrote :

> Thanks for the explanation. What about keeping the ssh client and have
> something like this?
>
> class EcosystemObject:
>
> def __init__(ip, user, port):
> self.ip = ip
> self.user = user
> self.port = port
>
> @property
> def ssh(self):
> return SSHManager.get_instance(self.ip, self.user, self.port)
>
> class Snapcraft(EcosystemObject):
>
> class Snapd(EcosystemObject)
>
> That way every time you instantiate Snapcraft or Snapd, you'll get a property
> ssh that will always point to an active ssh connection

Thanks for the feedback. Comments addressed:
* Deleting SShDriver and adding back Sshclient
* Cwd parameter removed from client and adding in helper (To mantain paramiko's method signature)
* adding ssh property in each helper that needs ssh, to ensure an active connection for every command execution.

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Omer Akram (om26er) wrote :

Code looks good to me.

review: Approve
33. By Heber Parrucci

merge from trunk

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Looks good

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'snappy_ecosystem_tests/environment/setup.py'
2--- snappy_ecosystem_tests/environment/setup.py 2017-03-01 14:52:43 +0000
3+++ snappy_ecosystem_tests/environment/setup.py 2017-03-07 17:41:57 +0000
4@@ -28,6 +28,7 @@
5
6 logging.basicConfig(level=logging.DEBUG)
7
8+
9 def main(mode, profile):
10 """Main script"""
11 manager = get_manager(mode)
12
13=== modified file 'snappy_ecosystem_tests/helpers/snapcraft/build_snap.py'
14--- snappy_ecosystem_tests/helpers/snapcraft/build_snap.py 2017-03-02 16:24:59 +0000
15+++ snappy_ecosystem_tests/helpers/snapcraft/build_snap.py 2017-03-07 17:41:57 +0000
16@@ -53,13 +53,21 @@
17 class SnapBuilder:
18 """Class to build a snap on a remote machine and download its content."""
19 def __init__(self, hostname, username, port=22):
20- self.client = SSHManager.get_instance(hostname, username, port)
21+ self.hostname = hostname
22+ self.username = username
23+ self.port = port
24+
25+ @property
26+ def ssh(self):
27+ """Property tu ensure we get an active connection for every command
28+ execution over ssh"""
29+ return SSHManager.get_instance(self.hostname, self.username, self.port)
30
31 def is_host_setup(self):
32 """Return bool representing whether the remote host is setup by
33 checking if snapcraft package is installed."""
34 try:
35- self.client.exec_command('which snapcraft')
36+ self.ssh.exec_command('which snapcraft')
37 return True
38 except ValueError:
39 return False
40@@ -68,7 +76,7 @@
41 """Setup the host to build a snap, currently only installs
42 snapcraft."""
43 for command in COMMANDS_SETUP:
44- self.client.exec_command(command)
45+ self.ssh.exec_command(command)
46
47 def build(self, name, version, summary, description, grade='stable',
48 confinement='strict'):
49@@ -86,9 +94,9 @@
50 """
51 if not self.is_host_setup():
52 self.setup_host()
53- tempdir = self.client.exec_command('mktemp -d')
54- self.client.exec_command('mkdir snap', cwd=tempdir)
55- self.client.exec_command('echo "{}" > snap/snapcraft.yaml'.format(
56+ tempdir = self.ssh.exec_command('mktemp -d')
57+ self.ssh.exec_command('mkdir snap', cwd=tempdir)
58+ self.ssh.exec_command('echo "{}" > snap/snapcraft.yaml'.format(
59 SNAPCRAFT_YAML.format(
60 snap_name=name,
61 version=version,
62@@ -98,13 +106,13 @@
63 confinement=confinement
64 )
65 ), cwd=tempdir)
66- self.client.exec_command('snapcraft', cwd=tempdir)
67+ self.ssh.exec_command('snapcraft', cwd=tempdir)
68 return os.path.join(
69 tempdir,
70 '{name}_{version}_{arch}.snap'.format(
71 name=name,
72 version=version,
73- arch=self.client.exec_command('dpkg --print-architecture')
74+ arch=self.ssh.exec_command('dpkg --print-architecture')
75 )
76 )
77
78@@ -136,5 +144,5 @@
79 output_path_absolute = os.path.abspath(
80 os.path.join(output_location, PurePath(remote_snap_path).name)
81 )
82- self.client.pull(remote_snap_path, output_path_absolute)
83+ self.ssh.pull(remote_snap_path, output_path_absolute)
84 return output_path_absolute
85
86=== modified file 'snappy_ecosystem_tests/helpers/snapcraft/client.py'
87--- snappy_ecosystem_tests/helpers/snapcraft/client.py 2017-03-03 14:06:17 +0000
88+++ snappy_ecosystem_tests/helpers/snapcraft/client.py 2017-03-07 17:41:57 +0000
89@@ -25,9 +25,9 @@
90 import time
91
92 from snappy_ecosystem_tests.models.snap import Snap
93-from snappy_ecosystem_tests.utils import ssh
94 from snappy_ecosystem_tests.utils.commands import build_command
95 from snappy_ecosystem_tests.utils.filters import filter_list
96+from snappy_ecosystem_tests.utils.ssh import SSHManager
97 from snappy_ecosystem_tests.utils.storeconfig import get_current_store
98
99 from snappy_ecosystem_tests.commons.config import CONFIG_STACK
100@@ -72,8 +72,13 @@
101 self._login = False
102 self._cleanup()
103
104- @staticmethod
105- def _run_snapcraft_command_ssh(command, debug=True):
106+ @property
107+ def ssh(self):
108+ """Property tu ensure we get an active connection for every command
109+ execution over ssh"""
110+ return SSHManager.get_instance(HOSTNAME, USERNAME, PORT)
111+
112+ def _run_snapcraft_command_ssh(self, command, debug=True):
113 """
114 Run snapcraft command via ssh
115 :param command: the command to be executed.
116@@ -85,25 +90,21 @@
117 command = build_snapcraft_command(command)
118 if debug:
119 command += ' -d'
120- return ssh.run_command(command, hostname=HOSTNAME, username=USERNAME,
121- port=PORT)
122+ return self.ssh.exec_command(command)
123
124- @staticmethod
125- def assert_logged_in():
126+ def assert_logged_in(self):
127 """Assert that an user is logged.
128 :raise ValueError: in case login is false
129 """
130- if Snapcraft.is_logged_in() is False:
131+ if self.is_logged_in() is False:
132 raise ValueError("User is not logged in, "
133 "please login before using this command")
134
135- @staticmethod
136- def is_logged_in():
137+ def is_logged_in(self):
138 """Return bool representing if the user is logged into snapcraft."""
139 try:
140- output = ssh.run_command(
141- 'cat %s' % SNAPCRAFT_CONFIG_FILE_PATH,
142- hostname=HOSTNAME, username=USERNAME, port=PORT)
143+ output = self.ssh.exec_command(
144+ 'cat %s' % SNAPCRAFT_CONFIG_FILE_PATH)
145 return output is not ''
146 except ValueError:
147 return False
148@@ -112,26 +113,23 @@
149 """Perform cleanup actions"""
150 self.logout()
151
152- @staticmethod
153- def logout():
154+ def logout(self):
155 """logout of snapcraft store session
156 :return: True if logout was successful, False otherwise
157 """
158- Snapcraft._run_snapcraft_command_ssh(COMMAND_LOGOUT)
159- return not Snapcraft.is_logged_in()
160+ self._run_snapcraft_command_ssh(COMMAND_LOGOUT)
161+ return not self.is_logged_in()
162
163- @staticmethod
164- def login(email=LOGIN_EMAIL, password=LOGIN_PASSWORD):
165+ def login(self, email=LOGIN_EMAIL, password=LOGIN_PASSWORD):
166 """Login to Snapcraft. If email and/or password are not provided,
167 use the default ones.
168 :param email: the SSO email
169 :param password: the email password
170 """
171- ssh.run_command(
172+ self.ssh.exec_command((
173 COMMANDS_LOGIN.format(email=email, password=password,
174- snapcraft=PATH_SNAPCRAFT),
175- hostname=HOSTNAME, username=USERNAME, port=PORT)
176- return Snapcraft.is_logged_in()
177+ snapcraft=PATH_SNAPCRAFT)))
178+ return self.is_logged_in()
179
180 def list_registered(self, debug=True, raw=False):
181 """Call snapcraft list-registered command,
182@@ -180,7 +178,7 @@
183 if private:
184 command += ' --private'
185 try:
186- Snapcraft._run_snapcraft_command_ssh(command)
187+ self._run_snapcraft_command_ssh(command)
188 except subprocess.CalledProcessError as _e:
189 LOGGER.error('Unable to register snap: %s', _e.output)
190 return False
191
192=== modified file 'snappy_ecosystem_tests/helpers/snapd/snapd.py'
193--- snappy_ecosystem_tests/helpers/snapd/snapd.py 2017-03-02 18:36:24 +0000
194+++ snappy_ecosystem_tests/helpers/snapd/snapd.py 2017-03-07 17:41:57 +0000
195@@ -25,7 +25,7 @@
196
197 import yaml
198
199-from snappy_ecosystem_tests.utils import ssh
200+from snappy_ecosystem_tests.utils.ssh import SSHManager
201 from snappy_ecosystem_tests.utils.user import get_snapd_remote_host_credentials
202
203 PATH_SNAP = '/usr/bin/snap'
204@@ -51,107 +51,116 @@
205 HOSTNAME, USERNAME, PORT = get_snapd_remote_host_credentials()
206
207
208-def run_snapd_command_ssh(parameters, cwd=''):
209- """Run snapd command over ssh.
210-
211- :param parameters: a string containing parameters for the `snap` command.
212- :param cwd: the current working directory on the remote where the command
213- should run.
214- :returns: stdout of the command
215- """
216- return ssh.run_command('{} {}'.format(PATH_SNAP, parameters), cwd=cwd,
217- hostname=HOSTNAME, username=USERNAME, port=PORT)
218-
219-
220-def login(email, password):
221- """Login to snapd.
222-
223- :param email: Ubuntu SSO account email address.
224- :param password: Ubuntu SSO account password.
225- """
226- ssh.run_command(COMMANDS_LOGIN.format(email=email, password=password))
227- return is_logged_in(email)
228-
229-
230-def is_logged_in(email):
231- """Return bool representing if the user is logged into snapd."""
232- try:
233- return json.loads(ssh.run_command('cat ~/.snap/auth.json')).get(
234- 'email') == email
235- except ValueError:
236+class Snapd:
237+ """Contain Snapd specific functionality to use via command
238+ line interface"""
239+
240+ @property
241+ def ssh(self):
242+ """Property tu ensure we get an active connection for every command
243+ execution over ssh"""
244+ return SSHManager.get_instance(HOSTNAME, USERNAME, PORT)
245+
246+ def run_snapd_command_ssh(self, command, cwd=''):
247+ """Run snapd command over ssh.
248+
249+ :param command: a string containing parameters for the `snap` command.
250+ :param cwd: the current working directory on the remote where
251+ the command should run.
252+ :returns: stdout of the command
253+ """
254+ if cwd:
255+ command = '{};{}'.format(cwd, command)
256+ return self.ssh.exec_command(command)
257+
258+ def login(self, email, password):
259+ """Login to snapd.
260+
261+ :param email: Ubuntu SSO account email address.
262+ :param password: Ubuntu SSO account password.
263+ """
264+ self.ssh.exec_command(COMMANDS_LOGIN.format(email=email,
265+ password=password))
266+ return self.is_logged_in(email)
267+
268+ def is_logged_in(self, email):
269+ """Return bool representing if the user is logged into snapd."""
270+ try:
271+ return json.loads(
272+ self.ssh.exec_command(
273+ 'cat ~/.snap/auth.json')).get('email') == email
274+ except ValueError:
275+ return False
276+
277+ def logout(self, email):
278+ """Logout snapd current user."""
279+ if self.is_logged_in(email):
280+ self.ssh.exec_command(COMMAND_LOGOUT)
281+
282+ def download(self, snap, channel=CHANNEL_STABLE):
283+ """Download the requested snap.
284+
285+ :param snap: name of the snap to download.
286+ :param channel: name of the release channel to download from.
287+ """
288+ command = COMMAND_DOWNLOAD.format(snap=snap, channel=channel)
289+ self.run_snapd_command_ssh(command,
290+ cwd=self.ssh.exec_command('mktemp -d'))
291+
292+ def install(self, snap, channel=CHANNEL_STABLE):
293+ """Install the requested snap."""
294+ self.ssh.exec_command(COMMAND_INSTALL.format(snap=snap,
295+ channel=channel))
296+
297+ def _is_installed(self, snap):
298+ """Return bool representing whether a snap is installed."""
299+ for installed_snap in Snapd._parse_output(
300+ self.run_snapd_command_ssh(COMMAND_LIST)):
301+ if installed_snap['name'] == snap:
302+ return True
303 return False
304
305-
306-def logout(email):
307- """Logout snapd current user."""
308- if is_logged_in(email):
309- run_snapd_command_ssh(COMMAND_LOGOUT)
310-
311-
312-def download(snap, channel=CHANNEL_STABLE):
313- """Download the requested snap.
314-
315- :param snap: name of the snap to download.
316- :param channel: name of the release channel to download from.
317- """
318- command = COMMAND_DOWNLOAD.format(snap=snap, channel=channel)
319- run_snapd_command_ssh(command, cwd=ssh.run_command('mktemp -d'))
320-
321-
322-def install(snap, channel=CHANNEL_STABLE):
323- """Install the requested snap."""
324- run_snapd_command_ssh(COMMAND_INSTALL.format(snap=snap, channel=channel))
325-
326-
327-def _is_installed(snap):
328- """Return bool representing whether a snap is installed."""
329- for installed_snap in _parse_output(run_snapd_command_ssh(COMMAND_LIST)):
330- if installed_snap['name'] == snap:
331- return True
332- return False
333-
334-
335-def remove(snap):
336- """Remove a snap, if its already installed."""
337- if _is_installed(snap):
338- run_snapd_command_ssh(COMMAND_REMOVE.format(snap=snap))
339-
340-
341-def info(snap, verbose=False):
342- """Query the Ubuntu store of the information about a snap.
343-
344- :param snap: Name of the snap for which the info is required.
345- :param verbose: Whether to information should be detailed.
346- :return: Return a dictionary containing information about the snap.
347- """
348- command = COMMAND_INFO.format(snap=snap)
349- if verbose:
350- command = ' '.join([command, '--verbose'])
351- return yaml.load("""{}""".format(run_snapd_command_ssh(command)))
352-
353-
354-def refresh(snap, channel=CHANNEL_STABLE):
355- """Refresh the requested snap."""
356- run_snapd_command_ssh(COMMAND_REFRESH.format(snap=snap, channel=channel))
357-
358-
359-def _parse_output(raw_output):
360- """Pretty parse the output from snapd commands like `find` and `list`.
361-
362- :param raw_output: The raw output returned from the command
363- :return: A list of dictionaries containing sorted results from the output.
364- """
365- split_output = raw_output.split('\n')
366- headers = [header.lower() for header in split_output.pop(0).split()]
367- return [dict(zip(headers, line.split())) for line in split_output]
368-
369-
370-def find(keyword):
371- """Find snaps based on the provided filters
372-
373- :param keyword: Keyword to use for the query.
374- :return: Return a list of dictionaries containing information about
375- snaps matching the `keyword`.
376- """
377- return _parse_output(
378- run_snapd_command_ssh(COMMAND_FIND.format(search_term=keyword)))
379+ def remove(self, snap):
380+ """Remove a snap, if its already installed."""
381+ if self._is_installed(snap):
382+ self.run_snapd_command_ssh(COMMAND_REMOVE.format(snap=snap))
383+
384+ def info(self, snap, verbose=False):
385+ """Query the Ubuntu store of the information about a snap.
386+
387+ :param snap: Name of the snap for which the info is required.
388+ :param verbose: Whether to information should be detailed.
389+ :return: Return a dictionary containing information about the snap.
390+ """
391+ command = COMMAND_INFO.format(snap=snap)
392+ if verbose:
393+ command = ' '.join([command, '--verbose'])
394+ return yaml.load("""{}""".format(self.run_snapd_command_ssh(command)))
395+
396+ def refresh(self, snap, channel=CHANNEL_STABLE):
397+ """Refresh the requested snap."""
398+ self.run_snapd_command_ssh(COMMAND_REFRESH.format(snap=snap,
399+ channel=channel))
400+
401+ @staticmethod
402+ def _parse_output(raw_output):
403+ """Pretty parse the output from snapd commands like `find` and `list`.
404+
405+ :param raw_output: The raw output returned from the command
406+ :return: A list of dictionaries containing sorted results
407+ from the output.
408+ """
409+ split_output = raw_output.split('\n')
410+ headers = [header.lower() for header in split_output.pop(0).split()]
411+ return [dict(zip(headers, line.split())) for line in split_output]
412+
413+ def find(self, keyword):
414+ """Find snaps based on the provided filters
415+
416+ :param keyword: Keyword to use for the query.
417+ :return: Return a list of dictionaries containing information about
418+ snaps matching the `keyword`.
419+ """
420+ return Snapd._parse_output(
421+ self.run_snapd_command_ssh(COMMAND_FIND.format(
422+ search_term=keyword)))
423
424=== modified file 'snappy_ecosystem_tests/tests/test_snapd.py'
425--- snappy_ecosystem_tests/tests/test_snapd.py 2017-02-22 15:47:09 +0000
426+++ snappy_ecosystem_tests/tests/test_snapd.py 2017-03-07 17:41:57 +0000
427@@ -22,7 +22,7 @@
428
429 import testtools
430
431-from snappy_ecosystem_tests.helpers.snapd import snapd
432+from snappy_ecosystem_tests.helpers.snapd.snapd import Snapd
433 from snappy_ecosystem_tests.utils.storeconfig import get_store_credentials
434
435
436@@ -30,13 +30,14 @@
437 """Tests for snapd."""
438 def setUp(self):
439 super().setUp()
440+ self.snapd = Snapd()
441
442 def test_info(self):
443 """Ensure the publisher of core snap in canonical."""
444- self.assertTrue('canonical', snapd.info('core')['publisher'])
445+ self.assertTrue('canonical', self.snapd.info('core')['publisher'])
446
447 def test_login(self):
448 """Login the snapd user."""
449 email, password = get_store_credentials()
450- self.assertTrue(snapd.login(email, password))
451- self.addCleanup(snapd.logout, email)
452+ self.assertTrue(self.snapd.login(email, password))
453+ self.addCleanup(self.snapd.logout, email)
454
455=== modified file 'snappy_ecosystem_tests/utils/ssh.py'
456--- snappy_ecosystem_tests/utils/ssh.py 2017-03-02 20:38:49 +0000
457+++ snappy_ecosystem_tests/utils/ssh.py 2017-03-07 17:41:57 +0000
458@@ -21,9 +21,9 @@
459 """Module to connect to ssh client for running tests."""
460
461 import logging
462+
463 import paramiko
464
465-from snappy_ecosystem_tests.utils.user import get_snapd_remote_host_credentials
466 from snappy_ecosystem_tests.utils.storeconfig import get_current_store
467 from snappy_ecosystem_tests.environment.data.snapd import STAGING_VARIABLES
468
469@@ -33,13 +33,17 @@
470 class SSHClient(paramiko.SSHClient):
471 """Extended SSHClient."""
472
473- def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False, # pylint: disable=arguments-differ
474- environment=None, cwd=''):
475+ def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False,
476+ environment=None):
477 """Execute the given command over ssh."""
478- if cwd:
479- command = 'cd {}; {}'.format(cwd, command)
480- _, stdout, stderr = super().exec_command(command, bufsize, timeout,
481- get_pty, environment)
482+ if environment is None:
483+ # FIXME: get environment variables dinamically
484+ if get_current_store() == 'staging':
485+ environment = STAGING_VARIABLES
486+ _, stdout, stderr = super().exec_command(
487+ command, bufsize, timeout,
488+ get_pty,
489+ environment=environment)
490 if stdout.channel.recv_exit_status() != 0:
491 raise ValueError(stderr.read().decode().strip())
492 response = stdout.read().decode().strip()
493@@ -102,27 +106,3 @@
494 username == client.get_transport().get_username()
495 ]
496 return connections[0] if connections else None
497-
498-
499-def run_command(command, cwd=None, hostname=None, username=None, port=None):
500- """Run the given command on remote machine over ssh.
501-
502- :param command: a string of the command to run.
503- :param cwd: Current working directory for the command.
504- :param hostname: The host to run command on.
505- :param username: Name of the user on the remote host to login to.
506- :param port: SSH port number.
507- :raises ValueError: if command exits with non-zero status.
508- :return: the stdout of the command.
509- """
510- # Take snapd remote credentials as default if they are not provided.
511- # Just arbitrary... Could have been other ones.
512- _hostname, _username, _port = get_snapd_remote_host_credentials()
513- ssh = SSHManager.get_instance(
514- hostname or _hostname, username or _username, port or _port)
515- return ssh.exec_command(
516- command,
517- cwd=cwd,
518- environment=STAGING_VARIABLES if get_current_store() == 'staging'
519- else None
520- )

Subscribers

People subscribed via source and target branches