Merge lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver into lp:snappy-ecosystem-tests
- adding-ssh-client-driver
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Santiago Baldassin |
Approved revision: | 33 |
Merged at revision: | 33 |
Proposed branch: | lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver |
Merge into: | lp:snappy-ecosystem-tests |
Diff against target: |
520 lines (+168/-171) 6 files modified
snappy_ecosystem_tests/environment/setup.py (+1/-0) snappy_ecosystem_tests/helpers/snapcraft/build_snap.py (+17/-9) snappy_ecosystem_tests/helpers/snapcraft/client.py (+22/-24) snappy_ecosystem_tests/helpers/snapd/snapd.py (+112/-103) snappy_ecosystem_tests/tests/test_snapd.py (+5/-4) snappy_ecosystem_tests/utils/ssh.py (+11/-31) |
To merge this branch: | bzr merge lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Santiago Baldassin (community) | Approve | ||
platform-qa-bot | continuous-integration | Approve | |
Omer Akram (community) | Approve | ||
Review via email: mp+318948@code.launchpad.net |
Commit message
Adding SSHClientDriver to:
* Avoid passing the host, username, port in every run_command execution.
* Customize run command with cwd option
* Add pull method
* Ensure we execute each command against an active connection provided by the SSHManager
Description of the change
Centralize a way to do operations against an ssh client to:
* Avoid passing the host, username, port in every run_command execution.
* Customize run command with cwd option
* Add pull method
* Ensure we execute each command against an active connection provided by the SSHManager
platform-qa-bot (platform-qa-bot) wrote : | # |
Omer Akram (om26er) wrote : | # |
Excuse my ignorance, can you tell what problem this branch tries to solve ? From the look of it, it seems to be making things more complex. The SSHClientDriver is now the only consumer of SSHManager.
I do see the SSHClientDriver gets a connected instance from the manager each time a command is run, this does solve a problem, though that will only be relevant if our tests were running over the internet, right now they are running on the local network.
Do you know if in future our tests will be running over the internet instead of the local network where the risk of ssh connection drop practically diminishes.
Heber Parrucci (heber013) wrote : | # |
> Excuse my ignorance, can you tell what problem this branch tries to solve ?
> From the look of it, it seems to be making things more complex. The
> SSHClientDriver is now the only consumer of SSHManager.
>
> I do see the SSHClientDriver gets a connected instance from the manager each
> time a command is run, this does solve a problem, though that will only be
> relevant if our tests were running over the internet, right now they are
> running on the local network.
>
> Do you know if in future our tests will be running over the internet instead
> of the local network where the risk of ssh connection drop practically
> diminishes.
Sure no problem!
The change description might not be clear enough. So I will provide more details on each item:
* Avoid passing the host, username, port in every run_command execution. --> Currently we have a run_command at module level so we need to provide host, username, etc every time we call it. Encapsulating the logic in a class and providing the parameters in the object initialization, we ensure that the ssh specific parameters will be provided once per remote helper.
* Customize run command with cwd option --> per your request, I added cwd named parameter in run_command so you can specify the dir in which the command will be executed.
* Add pull method --> the pull was also at module level.
* Ensure we execute each command against an active connection provided by the SSHManager --> the fact that the ssh connections are more likely to drop if tests are executed over internet, does not alter the design (the keywords are *more likely*). I mean, we could use the same client object as well, but in this case we would me assuming the risk. Do you note any performance issues of requesting an active connection in every command? Remember that it is stored in the Manager connections pool. Furthermore, if I change the logic to use the same connection on the ssh driver, I will not be using the feature that the SshManager provides.
Finally, we currently have on trunk a custom SShClient class that extends the paramiko one and overrides exec_command just to provide a dir in which the command will be executed, and it is not using our manager to get the ssh connection.
Let me know if that make more sense for you now.
Heber Parrucci (heber013) wrote : | # |
> > Excuse my ignorance, can you tell what problem this branch tries to solve ?
> > From the look of it, it seems to be making things more complex. The
> > SSHClientDriver is now the only consumer of SSHManager.
> >
> > I do see the SSHClientDriver gets a connected instance from the manager each
> > time a command is run, this does solve a problem, though that will only be
> > relevant if our tests were running over the internet, right now they are
> > running on the local network.
> >
> > Do you know if in future our tests will be running over the internet instead
> > of the local network where the risk of ssh connection drop practically
> > diminishes.
>
> Sure no problem!
> The change description might not be clear enough. So I will provide more
> details on each item:
>
>
> * Avoid passing the host, username, port in every run_command execution. -->
> Currently we have a run_command at module level so we need to provide host,
> username, etc every time we call it. Encapsulating the logic in a class and
> providing the parameters in the object initialization, we ensure that the ssh
> specific parameters will be provided once per remote helper.
> * Customize run command with cwd option --> per your request, I added cwd
> named parameter in run_command so you can specify the dir in which the command
> will be executed.
> * Add pull method --> the pull was also at module level.
> * Ensure we execute each command against an active connection provided by the
> SSHManager --> the fact that the ssh connections are more likely to drop if
> tests are executed over internet, does not alter the design (the keywords are
> *more likely*). I mean, we could use the same client object as well, but in
> this case we would me assuming the risk. Do you note any performance issues of
> requesting an active connection in every command? Remember that it is stored
> in the Manager connections pool. Furthermore, if I change the logic to use the
> same connection on the ssh driver, I will not be using the feature that the
> SshManager provides.
>
>
> Finally, we currently have on trunk a custom SShClient class that extends the
> paramiko one and overrides exec_command just to provide a dir in which the
> command will be executed, and it is not using our manager to get the ssh
> connection.
>
> Let me know if that make more sense for you now.
I forgot, we currently have on trunk two alternatives to run a ssh command:
1- Calling exec_command of our custom SSHClient (does not use the manager to get an active connection)
2- calling run_command in snappy_
Why would we need 2 ways to execute a ssh command? This change unifies them.
Santiago Baldassin (sbaldassin) wrote : | # |
Looks good in general. Minor comment inline
Omer Akram (om26er) wrote : | # |
On Mon, Mar 6, 2017 at 10:22 PM, Heber Parrucci <
<email address hidden>> wrote:
> > Excuse my ignorance, can you tell what problem this branch tries to
> solve ?
> > From the look of it, it seems to be making things more complex. The
> > SSHClientDriver is now the only consumer of SSHManager.
> >
> > I do see the SSHClientDriver gets a connected instance from the manager
> each
> > time a command is run, this does solve a problem, though that will only
> be
> > relevant if our tests were running over the internet, right now they are
> > running on the local network.
> >
> > Do you know if in future our tests will be running over the internet
> instead
> > of the local network where the risk of ssh connection drop practically
> > diminishes.
>
> Sure no problem!
> The change description might not be clear enough. So I will provide more
> details on each item:
>
>
> * Avoid passing the host, username, port in every run_command execution.
> --> Currently we have a run_command at module level so we need to provide
> host, username, etc every time we call it. Encapsulating the logic in a
> class and providing the parameters in the object initialization, we ensure
> that the ssh specific parameters will be provided once per remote helper.
>
That can be done on current trunk right now, SSHManager.
for that.
> * Customize run command with cwd option --> per your request, I added cwd
> named parameter in run_command so you can specify the dir in which the
> command will be executed.
> * Add pull method --> the pull was also at module level.
>
I thought these methods already existed in the extended SSHClient, does
putting them behind a driver make things simpler/easier to consume ?
> * Ensure we execute each command against an active connection provided by
> the SSHManager --> the fact that the ssh connections are more likely to
> drop if tests are executed over internet, does not alter the design (the
> keywords are *more likely*). I mean, we could use the same client object as
> well, but in this case we would me assuming the risk. Do you note any
> performance issues of requesting an active connection in every command?
> Remember that it is stored in the Manager connections pool. Furthermore, if
> I change the logic to use the same connection on the ssh driver, I will not
> be using the feature that the SshManager provides.
>
If we don't run tests over the internet, the chances of connection drop are
practically zero, unless we are hit by a nasty bug.
>
>
> Finally, we currently have on trunk a custom SShClient class that extends
> the paramiko one and overrides exec_command just to provide a dir in which
> the command will be executed, and it is not using our manager to get the
> ssh connection.
>
run_command is indeed redundant in the presence of
SSHManager.
SSHClientDriver if it does not solve any problems we are facing.
> Let me know if that make more sense for you now.
>
Thanks for the detailed response. :)
>
> --
> https:/
> ecosystem-
> You are reviewing the ...
Omer Akram (om26er) wrote : | # |
On Mon, Mar 6, 2017 at 10:30 PM, Heber Parrucci <
<email address hidden>> wrote:
> > > Excuse my ignorance, can you tell what problem this branch tries to
> solve ?
> > > From the look of it, it seems to be making things more complex. The
> > > SSHClientDriver is now the only consumer of SSHManager.
> > >
> > > I do see the SSHClientDriver gets a connected instance from the
> manager each
> > > time a command is run, this does solve a problem, though that will
> only be
> > > relevant if our tests were running over the internet, right now they
> are
> > > running on the local network.
> > >
> > > Do you know if in future our tests will be running over the internet
> instead
> > > of the local network where the risk of ssh connection drop practically
> > > diminishes.
> >
> > Sure no problem!
> > The change description might not be clear enough. So I will provide more
> > details on each item:
> >
> >
> > * Avoid passing the host, username, port in every run_command execution.
> -->
> > Currently we have a run_command at module level so we need to provide
> host,
> > username, etc every time we call it. Encapsulating the logic in a class
> and
> > providing the parameters in the object initialization, we ensure that
> the ssh
> > specific parameters will be provided once per remote helper.
> > * Customize run command with cwd option --> per your request, I added cwd
> > named parameter in run_command so you can specify the dir in which the
> command
> > will be executed.
> > * Add pull method --> the pull was also at module level.
> > * Ensure we execute each command against an active connection provided
> by the
> > SSHManager --> the fact that the ssh connections are more likely to drop
> if
> > tests are executed over internet, does not alter the design (the
> keywords are
> > *more likely*). I mean, we could use the same client object as well, but
> in
> > this case we would me assuming the risk. Do you note any performance
> issues of
> > requesting an active connection in every command? Remember that it is
> stored
> > in the Manager connections pool. Furthermore, if I change the logic to
> use the
> > same connection on the ssh driver, I will not be using the feature that
> the
> > SshManager provides.
> >
> >
> > Finally, we currently have on trunk a custom SShClient class that
> extends the
> > paramiko one and overrides exec_command just to provide a dir in which
> the
> > command will be executed, and it is not using our manager to get the ssh
> > connection.
> >
> > Let me know if that make more sense for you now.
>
> I forgot, we currently have on trunk two alternatives to run a ssh command:
>
> 1- Calling exec_command of our custom SSHClient (does not use the manager
> to get an active connection)
>
If a function uses SSHManager.
connected instance. If a class instance stores a reference to the
SSHClient, then in case tests are run over the internet we have the issue,
given we will be running them locally the issue does not exist to solve :)
> 2- calling run_command in snappy_
> actually uses the manager.
> Why would we need 2 ways to execute...
Heber Parrucci (heber013) wrote : | # |
> On Mon, Mar 6, 2017 at 10:30 PM, Heber Parrucci <
> <email address hidden>> wrote:
>
> > > > Excuse my ignorance, can you tell what problem this branch tries to
> > solve ?
> > > > From the look of it, it seems to be making things more complex. The
> > > > SSHClientDriver is now the only consumer of SSHManager.
> > > >
> > > > I do see the SSHClientDriver gets a connected instance from the
> > manager each
> > > > time a command is run, this does solve a problem, though that will
> > only be
> > > > relevant if our tests were running over the internet, right now they
> > are
> > > > running on the local network.
> > > >
> > > > Do you know if in future our tests will be running over the internet
> > instead
> > > > of the local network where the risk of ssh connection drop practically
> > > > diminishes.
> > >
> > > Sure no problem!
> > > The change description might not be clear enough. So I will provide more
> > > details on each item:
> > >
> > >
> > > * Avoid passing the host, username, port in every run_command execution.
> > -->
> > > Currently we have a run_command at module level so we need to provide
> > host,
> > > username, etc every time we call it. Encapsulating the logic in a class
> > and
> > > providing the parameters in the object initialization, we ensure that
> > the ssh
> > > specific parameters will be provided once per remote helper.
> > > * Customize run command with cwd option --> per your request, I added cwd
> > > named parameter in run_command so you can specify the dir in which the
> > command
> > > will be executed.
> > > * Add pull method --> the pull was also at module level.
> > > * Ensure we execute each command against an active connection provided
> > by the
> > > SSHManager --> the fact that the ssh connections are more likely to drop
> > if
> > > tests are executed over internet, does not alter the design (the
> > keywords are
> > > *more likely*). I mean, we could use the same client object as well, but
> > in
> > > this case we would me assuming the risk. Do you note any performance
> > issues of
> > > requesting an active connection in every command? Remember that it is
> > stored
> > > in the Manager connections pool. Furthermore, if I change the logic to
> > use the
> > > same connection on the ssh driver, I will not be using the feature that
> > the
> > > SshManager provides.
> > >
> > >
> > > Finally, we currently have on trunk a custom SShClient class that
> > extends the
> > > paramiko one and overrides exec_command just to provide a dir in which
> > the
> > > command will be executed, and it is not using our manager to get the ssh
> > > connection.
> > >
> > > Let me know if that make more sense for you now.
> >
> > I forgot, we currently have on trunk two alternatives to run a ssh command:
> >
> > 1- Calling exec_command of our custom SSHClient (does not use the manager
> > to get an active connection)
> >
>
> If a function uses SSHManager.
> connected instance. If a class instance stores a reference to the
> SSHClient, then in case tests are run over the internet we have the issue,
> given we will be running them locally the issue does not ex...
Heber Parrucci (heber013) wrote : | # |
> Looks good in general. Minor comment inline
Applies the same response that for Omer:
We are going to run the tests over internet in the future for devices like: raspberry Pi, dragonboards, etc. And those devices will be on the lab (at least for running tests on Jenkins). In those cases we have to ensure that every time we execute a command over ssh, the connection is active.
That's why we cannot save the SshClient instance in the helpers, because we cannot ask to the client itself to get an active connection in every command execution.
The driver was created to resolve that problem. You store an instance in your helper and then execute commands, for every command you execute, it ensures that the ssh connection for that client is active by requesting the manager.
How can we achieve that using the SshClient instead of the driver?
Santiago Baldassin (sbaldassin) wrote : | # |
Thanks for the explanation. What about keeping the ssh client and have something like this?
class EcosystemObject:
def __init__(ip, user, port):
self.ip = ip
self.user = user
self.port = port
@property
def ssh(self):
return SSHManager.
class Snapcraft(
class Snapd(Ecosystem
That way every time you instantiate Snapcraft or Snapd, you'll get a property ssh that will always point to an active ssh connection
- 32. By Heber Parrucci
-
Addressing review comments:
* Deleting SShDriver and adding back Sshclient
* Cwd parameter removed from client and adding in helper (To mantain paramiko's method signature)
* adding ssh property in each helper that needs ssh, to ensure an active connection for every command execution.
Heber Parrucci (heber013) wrote : | # |
> Thanks for the explanation. What about keeping the ssh client and have
> something like this?
>
> class EcosystemObject:
>
> def __init__(ip, user, port):
> self.ip = ip
> self.user = user
> self.port = port
>
> @property
> def ssh(self):
> return SSHManager.
>
> class Snapcraft(
>
> class Snapd(Ecosystem
>
> That way every time you instantiate Snapcraft or Snapd, you'll get a property
> ssh that will always point to an active ssh connection
Thanks for the feedback. Comments addressed:
* Deleting SShDriver and adding back Sshclient
* Cwd parameter removed from client and adding in helper (To mantain paramiko's method signature)
* adding ssh property in each helper that needs ssh, to ensure an active connection for every command execution.
platform-qa-bot (platform-qa-bot) wrote : | # |
PASSED: Continuous integration, rev:32
https:/
Executed test runs:
None: https:/
Click here to trigger a rebuild:
https:/
Omer Akram (om26er) wrote : | # |
Code looks good to me.
- 33. By Heber Parrucci
-
merge from trunk
platform-qa-bot (platform-qa-bot) wrote : | # |
PASSED: Continuous integration, rev:33
https:/
Executed test runs:
None: https:/
Click here to trigger a rebuild:
https:/
Santiago Baldassin (sbaldassin) wrote : | # |
Looks good
Preview Diff
1 | === modified file 'snappy_ecosystem_tests/environment/setup.py' |
2 | --- snappy_ecosystem_tests/environment/setup.py 2017-03-01 14:52:43 +0000 |
3 | +++ snappy_ecosystem_tests/environment/setup.py 2017-03-07 17:41:57 +0000 |
4 | @@ -28,6 +28,7 @@ |
5 | |
6 | logging.basicConfig(level=logging.DEBUG) |
7 | |
8 | + |
9 | def main(mode, profile): |
10 | """Main script""" |
11 | manager = get_manager(mode) |
12 | |
13 | === modified file 'snappy_ecosystem_tests/helpers/snapcraft/build_snap.py' |
14 | --- snappy_ecosystem_tests/helpers/snapcraft/build_snap.py 2017-03-02 16:24:59 +0000 |
15 | +++ snappy_ecosystem_tests/helpers/snapcraft/build_snap.py 2017-03-07 17:41:57 +0000 |
16 | @@ -53,13 +53,21 @@ |
17 | class SnapBuilder: |
18 | """Class to build a snap on a remote machine and download its content.""" |
19 | def __init__(self, hostname, username, port=22): |
20 | - self.client = SSHManager.get_instance(hostname, username, port) |
21 | + self.hostname = hostname |
22 | + self.username = username |
23 | + self.port = port |
24 | + |
25 | + @property |
26 | + def ssh(self): |
27 | + """Property tu ensure we get an active connection for every command |
28 | + execution over ssh""" |
29 | + return SSHManager.get_instance(self.hostname, self.username, self.port) |
30 | |
31 | def is_host_setup(self): |
32 | """Return bool representing whether the remote host is setup by |
33 | checking if snapcraft package is installed.""" |
34 | try: |
35 | - self.client.exec_command('which snapcraft') |
36 | + self.ssh.exec_command('which snapcraft') |
37 | return True |
38 | except ValueError: |
39 | return False |
40 | @@ -68,7 +76,7 @@ |
41 | """Setup the host to build a snap, currently only installs |
42 | snapcraft.""" |
43 | for command in COMMANDS_SETUP: |
44 | - self.client.exec_command(command) |
45 | + self.ssh.exec_command(command) |
46 | |
47 | def build(self, name, version, summary, description, grade='stable', |
48 | confinement='strict'): |
49 | @@ -86,9 +94,9 @@ |
50 | """ |
51 | if not self.is_host_setup(): |
52 | self.setup_host() |
53 | - tempdir = self.client.exec_command('mktemp -d') |
54 | - self.client.exec_command('mkdir snap', cwd=tempdir) |
55 | - self.client.exec_command('echo "{}" > snap/snapcraft.yaml'.format( |
56 | + tempdir = self.ssh.exec_command('mktemp -d') |
57 | + self.ssh.exec_command('mkdir snap', cwd=tempdir) |
58 | + self.ssh.exec_command('echo "{}" > snap/snapcraft.yaml'.format( |
59 | SNAPCRAFT_YAML.format( |
60 | snap_name=name, |
61 | version=version, |
62 | @@ -98,13 +106,13 @@ |
63 | confinement=confinement |
64 | ) |
65 | ), cwd=tempdir) |
66 | - self.client.exec_command('snapcraft', cwd=tempdir) |
67 | + self.ssh.exec_command('snapcraft', cwd=tempdir) |
68 | return os.path.join( |
69 | tempdir, |
70 | '{name}_{version}_{arch}.snap'.format( |
71 | name=name, |
72 | version=version, |
73 | - arch=self.client.exec_command('dpkg --print-architecture') |
74 | + arch=self.ssh.exec_command('dpkg --print-architecture') |
75 | ) |
76 | ) |
77 | |
78 | @@ -136,5 +144,5 @@ |
79 | output_path_absolute = os.path.abspath( |
80 | os.path.join(output_location, PurePath(remote_snap_path).name) |
81 | ) |
82 | - self.client.pull(remote_snap_path, output_path_absolute) |
83 | + self.ssh.pull(remote_snap_path, output_path_absolute) |
84 | return output_path_absolute |
85 | |
86 | === modified file 'snappy_ecosystem_tests/helpers/snapcraft/client.py' |
87 | --- snappy_ecosystem_tests/helpers/snapcraft/client.py 2017-03-03 14:06:17 +0000 |
88 | +++ snappy_ecosystem_tests/helpers/snapcraft/client.py 2017-03-07 17:41:57 +0000 |
89 | @@ -25,9 +25,9 @@ |
90 | import time |
91 | |
92 | from snappy_ecosystem_tests.models.snap import Snap |
93 | -from snappy_ecosystem_tests.utils import ssh |
94 | from snappy_ecosystem_tests.utils.commands import build_command |
95 | from snappy_ecosystem_tests.utils.filters import filter_list |
96 | +from snappy_ecosystem_tests.utils.ssh import SSHManager |
97 | from snappy_ecosystem_tests.utils.storeconfig import get_current_store |
98 | |
99 | from snappy_ecosystem_tests.commons.config import CONFIG_STACK |
100 | @@ -72,8 +72,13 @@ |
101 | self._login = False |
102 | self._cleanup() |
103 | |
104 | - @staticmethod |
105 | - def _run_snapcraft_command_ssh(command, debug=True): |
106 | + @property |
107 | + def ssh(self): |
108 | + """Property tu ensure we get an active connection for every command |
109 | + execution over ssh""" |
110 | + return SSHManager.get_instance(HOSTNAME, USERNAME, PORT) |
111 | + |
112 | + def _run_snapcraft_command_ssh(self, command, debug=True): |
113 | """ |
114 | Run snapcraft command via ssh |
115 | :param command: the command to be executed. |
116 | @@ -85,25 +90,21 @@ |
117 | command = build_snapcraft_command(command) |
118 | if debug: |
119 | command += ' -d' |
120 | - return ssh.run_command(command, hostname=HOSTNAME, username=USERNAME, |
121 | - port=PORT) |
122 | + return self.ssh.exec_command(command) |
123 | |
124 | - @staticmethod |
125 | - def assert_logged_in(): |
126 | + def assert_logged_in(self): |
127 | """Assert that an user is logged. |
128 | :raise ValueError: in case login is false |
129 | """ |
130 | - if Snapcraft.is_logged_in() is False: |
131 | + if self.is_logged_in() is False: |
132 | raise ValueError("User is not logged in, " |
133 | "please login before using this command") |
134 | |
135 | - @staticmethod |
136 | - def is_logged_in(): |
137 | + def is_logged_in(self): |
138 | """Return bool representing if the user is logged into snapcraft.""" |
139 | try: |
140 | - output = ssh.run_command( |
141 | - 'cat %s' % SNAPCRAFT_CONFIG_FILE_PATH, |
142 | - hostname=HOSTNAME, username=USERNAME, port=PORT) |
143 | + output = self.ssh.exec_command( |
144 | + 'cat %s' % SNAPCRAFT_CONFIG_FILE_PATH) |
145 | return output is not '' |
146 | except ValueError: |
147 | return False |
148 | @@ -112,26 +113,23 @@ |
149 | """Perform cleanup actions""" |
150 | self.logout() |
151 | |
152 | - @staticmethod |
153 | - def logout(): |
154 | + def logout(self): |
155 | """logout of snapcraft store session |
156 | :return: True if logout was successful, False otherwise |
157 | """ |
158 | - Snapcraft._run_snapcraft_command_ssh(COMMAND_LOGOUT) |
159 | - return not Snapcraft.is_logged_in() |
160 | + self._run_snapcraft_command_ssh(COMMAND_LOGOUT) |
161 | + return not self.is_logged_in() |
162 | |
163 | - @staticmethod |
164 | - def login(email=LOGIN_EMAIL, password=LOGIN_PASSWORD): |
165 | + def login(self, email=LOGIN_EMAIL, password=LOGIN_PASSWORD): |
166 | """Login to Snapcraft. If email and/or password are not provided, |
167 | use the default ones. |
168 | :param email: the SSO email |
169 | :param password: the email password |
170 | """ |
171 | - ssh.run_command( |
172 | + self.ssh.exec_command(( |
173 | COMMANDS_LOGIN.format(email=email, password=password, |
174 | - snapcraft=PATH_SNAPCRAFT), |
175 | - hostname=HOSTNAME, username=USERNAME, port=PORT) |
176 | - return Snapcraft.is_logged_in() |
177 | + snapcraft=PATH_SNAPCRAFT))) |
178 | + return self.is_logged_in() |
179 | |
180 | def list_registered(self, debug=True, raw=False): |
181 | """Call snapcraft list-registered command, |
182 | @@ -180,7 +178,7 @@ |
183 | if private: |
184 | command += ' --private' |
185 | try: |
186 | - Snapcraft._run_snapcraft_command_ssh(command) |
187 | + self._run_snapcraft_command_ssh(command) |
188 | except subprocess.CalledProcessError as _e: |
189 | LOGGER.error('Unable to register snap: %s', _e.output) |
190 | return False |
191 | |
192 | === modified file 'snappy_ecosystem_tests/helpers/snapd/snapd.py' |
193 | --- snappy_ecosystem_tests/helpers/snapd/snapd.py 2017-03-02 18:36:24 +0000 |
194 | +++ snappy_ecosystem_tests/helpers/snapd/snapd.py 2017-03-07 17:41:57 +0000 |
195 | @@ -25,7 +25,7 @@ |
196 | |
197 | import yaml |
198 | |
199 | -from snappy_ecosystem_tests.utils import ssh |
200 | +from snappy_ecosystem_tests.utils.ssh import SSHManager |
201 | from snappy_ecosystem_tests.utils.user import get_snapd_remote_host_credentials |
202 | |
203 | PATH_SNAP = '/usr/bin/snap' |
204 | @@ -51,107 +51,116 @@ |
205 | HOSTNAME, USERNAME, PORT = get_snapd_remote_host_credentials() |
206 | |
207 | |
208 | -def run_snapd_command_ssh(parameters, cwd=''): |
209 | - """Run snapd command over ssh. |
210 | - |
211 | - :param parameters: a string containing parameters for the `snap` command. |
212 | - :param cwd: the current working directory on the remote where the command |
213 | - should run. |
214 | - :returns: stdout of the command |
215 | - """ |
216 | - return ssh.run_command('{} {}'.format(PATH_SNAP, parameters), cwd=cwd, |
217 | - hostname=HOSTNAME, username=USERNAME, port=PORT) |
218 | - |
219 | - |
220 | -def login(email, password): |
221 | - """Login to snapd. |
222 | - |
223 | - :param email: Ubuntu SSO account email address. |
224 | - :param password: Ubuntu SSO account password. |
225 | - """ |
226 | - ssh.run_command(COMMANDS_LOGIN.format(email=email, password=password)) |
227 | - return is_logged_in(email) |
228 | - |
229 | - |
230 | -def is_logged_in(email): |
231 | - """Return bool representing if the user is logged into snapd.""" |
232 | - try: |
233 | - return json.loads(ssh.run_command('cat ~/.snap/auth.json')).get( |
234 | - 'email') == email |
235 | - except ValueError: |
236 | +class Snapd: |
237 | + """Contain Snapd specific functionality to use via command |
238 | + line interface""" |
239 | + |
240 | + @property |
241 | + def ssh(self): |
242 | + """Property tu ensure we get an active connection for every command |
243 | + execution over ssh""" |
244 | + return SSHManager.get_instance(HOSTNAME, USERNAME, PORT) |
245 | + |
246 | + def run_snapd_command_ssh(self, command, cwd=''): |
247 | + """Run snapd command over ssh. |
248 | + |
249 | + :param command: a string containing parameters for the `snap` command. |
250 | + :param cwd: the current working directory on the remote where |
251 | + the command should run. |
252 | + :returns: stdout of the command |
253 | + """ |
254 | + if cwd: |
255 | + command = '{};{}'.format(cwd, command) |
256 | + return self.ssh.exec_command(command) |
257 | + |
258 | + def login(self, email, password): |
259 | + """Login to snapd. |
260 | + |
261 | + :param email: Ubuntu SSO account email address. |
262 | + :param password: Ubuntu SSO account password. |
263 | + """ |
264 | + self.ssh.exec_command(COMMANDS_LOGIN.format(email=email, |
265 | + password=password)) |
266 | + return self.is_logged_in(email) |
267 | + |
268 | + def is_logged_in(self, email): |
269 | + """Return bool representing if the user is logged into snapd.""" |
270 | + try: |
271 | + return json.loads( |
272 | + self.ssh.exec_command( |
273 | + 'cat ~/.snap/auth.json')).get('email') == email |
274 | + except ValueError: |
275 | + return False |
276 | + |
277 | + def logout(self, email): |
278 | + """Logout snapd current user.""" |
279 | + if self.is_logged_in(email): |
280 | + self.ssh.exec_command(COMMAND_LOGOUT) |
281 | + |
282 | + def download(self, snap, channel=CHANNEL_STABLE): |
283 | + """Download the requested snap. |
284 | + |
285 | + :param snap: name of the snap to download. |
286 | + :param channel: name of the release channel to download from. |
287 | + """ |
288 | + command = COMMAND_DOWNLOAD.format(snap=snap, channel=channel) |
289 | + self.run_snapd_command_ssh(command, |
290 | + cwd=self.ssh.exec_command('mktemp -d')) |
291 | + |
292 | + def install(self, snap, channel=CHANNEL_STABLE): |
293 | + """Install the requested snap.""" |
294 | + self.ssh.exec_command(COMMAND_INSTALL.format(snap=snap, |
295 | + channel=channel)) |
296 | + |
297 | + def _is_installed(self, snap): |
298 | + """Return bool representing whether a snap is installed.""" |
299 | + for installed_snap in Snapd._parse_output( |
300 | + self.run_snapd_command_ssh(COMMAND_LIST)): |
301 | + if installed_snap['name'] == snap: |
302 | + return True |
303 | return False |
304 | |
305 | - |
306 | -def logout(email): |
307 | - """Logout snapd current user.""" |
308 | - if is_logged_in(email): |
309 | - run_snapd_command_ssh(COMMAND_LOGOUT) |
310 | - |
311 | - |
312 | -def download(snap, channel=CHANNEL_STABLE): |
313 | - """Download the requested snap. |
314 | - |
315 | - :param snap: name of the snap to download. |
316 | - :param channel: name of the release channel to download from. |
317 | - """ |
318 | - command = COMMAND_DOWNLOAD.format(snap=snap, channel=channel) |
319 | - run_snapd_command_ssh(command, cwd=ssh.run_command('mktemp -d')) |
320 | - |
321 | - |
322 | -def install(snap, channel=CHANNEL_STABLE): |
323 | - """Install the requested snap.""" |
324 | - run_snapd_command_ssh(COMMAND_INSTALL.format(snap=snap, channel=channel)) |
325 | - |
326 | - |
327 | -def _is_installed(snap): |
328 | - """Return bool representing whether a snap is installed.""" |
329 | - for installed_snap in _parse_output(run_snapd_command_ssh(COMMAND_LIST)): |
330 | - if installed_snap['name'] == snap: |
331 | - return True |
332 | - return False |
333 | - |
334 | - |
335 | -def remove(snap): |
336 | - """Remove a snap, if its already installed.""" |
337 | - if _is_installed(snap): |
338 | - run_snapd_command_ssh(COMMAND_REMOVE.format(snap=snap)) |
339 | - |
340 | - |
341 | -def info(snap, verbose=False): |
342 | - """Query the Ubuntu store of the information about a snap. |
343 | - |
344 | - :param snap: Name of the snap for which the info is required. |
345 | - :param verbose: Whether to information should be detailed. |
346 | - :return: Return a dictionary containing information about the snap. |
347 | - """ |
348 | - command = COMMAND_INFO.format(snap=snap) |
349 | - if verbose: |
350 | - command = ' '.join([command, '--verbose']) |
351 | - return yaml.load("""{}""".format(run_snapd_command_ssh(command))) |
352 | - |
353 | - |
354 | -def refresh(snap, channel=CHANNEL_STABLE): |
355 | - """Refresh the requested snap.""" |
356 | - run_snapd_command_ssh(COMMAND_REFRESH.format(snap=snap, channel=channel)) |
357 | - |
358 | - |
359 | -def _parse_output(raw_output): |
360 | - """Pretty parse the output from snapd commands like `find` and `list`. |
361 | - |
362 | - :param raw_output: The raw output returned from the command |
363 | - :return: A list of dictionaries containing sorted results from the output. |
364 | - """ |
365 | - split_output = raw_output.split('\n') |
366 | - headers = [header.lower() for header in split_output.pop(0).split()] |
367 | - return [dict(zip(headers, line.split())) for line in split_output] |
368 | - |
369 | - |
370 | -def find(keyword): |
371 | - """Find snaps based on the provided filters |
372 | - |
373 | - :param keyword: Keyword to use for the query. |
374 | - :return: Return a list of dictionaries containing information about |
375 | - snaps matching the `keyword`. |
376 | - """ |
377 | - return _parse_output( |
378 | - run_snapd_command_ssh(COMMAND_FIND.format(search_term=keyword))) |
379 | + def remove(self, snap): |
380 | + """Remove a snap, if its already installed.""" |
381 | + if self._is_installed(snap): |
382 | + self.run_snapd_command_ssh(COMMAND_REMOVE.format(snap=snap)) |
383 | + |
384 | + def info(self, snap, verbose=False): |
385 | + """Query the Ubuntu store of the information about a snap. |
386 | + |
387 | + :param snap: Name of the snap for which the info is required. |
388 | + :param verbose: Whether to information should be detailed. |
389 | + :return: Return a dictionary containing information about the snap. |
390 | + """ |
391 | + command = COMMAND_INFO.format(snap=snap) |
392 | + if verbose: |
393 | + command = ' '.join([command, '--verbose']) |
394 | + return yaml.load("""{}""".format(self.run_snapd_command_ssh(command))) |
395 | + |
396 | + def refresh(self, snap, channel=CHANNEL_STABLE): |
397 | + """Refresh the requested snap.""" |
398 | + self.run_snapd_command_ssh(COMMAND_REFRESH.format(snap=snap, |
399 | + channel=channel)) |
400 | + |
401 | + @staticmethod |
402 | + def _parse_output(raw_output): |
403 | + """Pretty parse the output from snapd commands like `find` and `list`. |
404 | + |
405 | + :param raw_output: The raw output returned from the command |
406 | + :return: A list of dictionaries containing sorted results |
407 | + from the output. |
408 | + """ |
409 | + split_output = raw_output.split('\n') |
410 | + headers = [header.lower() for header in split_output.pop(0).split()] |
411 | + return [dict(zip(headers, line.split())) for line in split_output] |
412 | + |
413 | + def find(self, keyword): |
414 | + """Find snaps based on the provided filters |
415 | + |
416 | + :param keyword: Keyword to use for the query. |
417 | + :return: Return a list of dictionaries containing information about |
418 | + snaps matching the `keyword`. |
419 | + """ |
420 | + return Snapd._parse_output( |
421 | + self.run_snapd_command_ssh(COMMAND_FIND.format( |
422 | + search_term=keyword))) |
423 | |
424 | === modified file 'snappy_ecosystem_tests/tests/test_snapd.py' |
425 | --- snappy_ecosystem_tests/tests/test_snapd.py 2017-02-22 15:47:09 +0000 |
426 | +++ snappy_ecosystem_tests/tests/test_snapd.py 2017-03-07 17:41:57 +0000 |
427 | @@ -22,7 +22,7 @@ |
428 | |
429 | import testtools |
430 | |
431 | -from snappy_ecosystem_tests.helpers.snapd import snapd |
432 | +from snappy_ecosystem_tests.helpers.snapd.snapd import Snapd |
433 | from snappy_ecosystem_tests.utils.storeconfig import get_store_credentials |
434 | |
435 | |
436 | @@ -30,13 +30,14 @@ |
437 | """Tests for snapd.""" |
438 | def setUp(self): |
439 | super().setUp() |
440 | + self.snapd = Snapd() |
441 | |
442 | def test_info(self): |
443 | """Ensure the publisher of core snap in canonical.""" |
444 | - self.assertTrue('canonical', snapd.info('core')['publisher']) |
445 | + self.assertTrue('canonical', self.snapd.info('core')['publisher']) |
446 | |
447 | def test_login(self): |
448 | """Login the snapd user.""" |
449 | email, password = get_store_credentials() |
450 | - self.assertTrue(snapd.login(email, password)) |
451 | - self.addCleanup(snapd.logout, email) |
452 | + self.assertTrue(self.snapd.login(email, password)) |
453 | + self.addCleanup(self.snapd.logout, email) |
454 | |
455 | === modified file 'snappy_ecosystem_tests/utils/ssh.py' |
456 | --- snappy_ecosystem_tests/utils/ssh.py 2017-03-02 20:38:49 +0000 |
457 | +++ snappy_ecosystem_tests/utils/ssh.py 2017-03-07 17:41:57 +0000 |
458 | @@ -21,9 +21,9 @@ |
459 | """Module to connect to ssh client for running tests.""" |
460 | |
461 | import logging |
462 | + |
463 | import paramiko |
464 | |
465 | -from snappy_ecosystem_tests.utils.user import get_snapd_remote_host_credentials |
466 | from snappy_ecosystem_tests.utils.storeconfig import get_current_store |
467 | from snappy_ecosystem_tests.environment.data.snapd import STAGING_VARIABLES |
468 | |
469 | @@ -33,13 +33,17 @@ |
470 | class SSHClient(paramiko.SSHClient): |
471 | """Extended SSHClient.""" |
472 | |
473 | - def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False, # pylint: disable=arguments-differ |
474 | - environment=None, cwd=''): |
475 | + def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False, |
476 | + environment=None): |
477 | """Execute the given command over ssh.""" |
478 | - if cwd: |
479 | - command = 'cd {}; {}'.format(cwd, command) |
480 | - _, stdout, stderr = super().exec_command(command, bufsize, timeout, |
481 | - get_pty, environment) |
482 | + if environment is None: |
483 | + # FIXME: get environment variables dinamically |
484 | + if get_current_store() == 'staging': |
485 | + environment = STAGING_VARIABLES |
486 | + _, stdout, stderr = super().exec_command( |
487 | + command, bufsize, timeout, |
488 | + get_pty, |
489 | + environment=environment) |
490 | if stdout.channel.recv_exit_status() != 0: |
491 | raise ValueError(stderr.read().decode().strip()) |
492 | response = stdout.read().decode().strip() |
493 | @@ -102,27 +106,3 @@ |
494 | username == client.get_transport().get_username() |
495 | ] |
496 | return connections[0] if connections else None |
497 | - |
498 | - |
499 | -def run_command(command, cwd=None, hostname=None, username=None, port=None): |
500 | - """Run the given command on remote machine over ssh. |
501 | - |
502 | - :param command: a string of the command to run. |
503 | - :param cwd: Current working directory for the command. |
504 | - :param hostname: The host to run command on. |
505 | - :param username: Name of the user on the remote host to login to. |
506 | - :param port: SSH port number. |
507 | - :raises ValueError: if command exits with non-zero status. |
508 | - :return: the stdout of the command. |
509 | - """ |
510 | - # Take snapd remote credentials as default if they are not provided. |
511 | - # Just arbitrary... Could have been other ones. |
512 | - _hostname, _username, _port = get_snapd_remote_host_credentials() |
513 | - ssh = SSHManager.get_instance( |
514 | - hostname or _hostname, username or _username, port or _port) |
515 | - return ssh.exec_command( |
516 | - command, |
517 | - cwd=cwd, |
518 | - environment=STAGING_VARIABLES if get_current_store() == 'staging' |
519 | - else None |
520 | - ) |
PASSED: Continuous integration, rev:31 /platform- qa-jenkins. ubuntu. com/job/ snappy- ecosystem- tests-ci/ 246/ /platform- qa-jenkins. ubuntu. com/job/ generic- update- mp/2053/ console
https:/
Executed test runs:
None: https:/
Click here to trigger a rebuild: /platform- qa-jenkins. ubuntu. com/job/ snappy- ecosystem- tests-ci/ 246/rebuild
https:/