Merge lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver into lp:snappy-ecosystem-tests

Proposed by Heber Parrucci
Status: Merged
Approved by: Santiago Baldassin
Approved revision: 33
Merged at revision: 33
Proposed branch: lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver
Merge into: lp:snappy-ecosystem-tests
Diff against target: 520 lines (+168/-171)
6 files modified
snappy_ecosystem_tests/environment/setup.py (+1/-0)
snappy_ecosystem_tests/helpers/snapcraft/build_snap.py (+17/-9)
snappy_ecosystem_tests/helpers/snapcraft/client.py (+22/-24)
snappy_ecosystem_tests/helpers/snapd/snapd.py (+112/-103)
snappy_ecosystem_tests/tests/test_snapd.py (+5/-4)
snappy_ecosystem_tests/utils/ssh.py (+11/-31)
To merge this branch: bzr merge lp:~canonical-platform-qa/snappy-ecosystem-tests/adding-ssh-client-driver
Reviewer Review Type Date Requested Status
Santiago Baldassin (community) Approve
platform-qa-bot continuous-integration Approve
Omer Akram (community) Approve
Review via email: mp+318948@code.launchpad.net

Commit message

Adding SSHClientDriver to:
* Avoid passing the host, username, port in every run_command execution.
* Customize run command with cwd option
* Add pull method
* Ensure we execute each command against an active connection provided by the SSHManager

Description of the change

Centralize a way to do operations against an ssh client to:

* Avoid passing the host, username, port in every run_command execution.
* Customize run command with cwd option
* Add pull method
* Ensure we execute each command against an active connection provided by the SSHManager

To post a comment you must log in.
Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Omer Akram (om26er) wrote :

Excuse my ignorance, can you tell what problem this branch tries to solve ? From the look of it, it seems to be making things more complex. The SSHClientDriver is now the only consumer of SSHManager.

I do see the SSHClientDriver gets a connected instance from the manager each time a command is run, this does solve a problem, though that will only be relevant if our tests were running over the internet, right now they are running on the local network.

Do you know if in future our tests will be running over the internet instead of the local network where the risk of ssh connection drop practically diminishes.

review: Needs Information
Revision history for this message
Heber Parrucci (heber013) wrote :

> Excuse my ignorance, can you tell what problem this branch tries to solve ?
> From the look of it, it seems to be making things more complex. The
> SSHClientDriver is now the only consumer of SSHManager.
>
> I do see the SSHClientDriver gets a connected instance from the manager each
> time a command is run, this does solve a problem, though that will only be
> relevant if our tests were running over the internet, right now they are
> running on the local network.
>
> Do you know if in future our tests will be running over the internet instead
> of the local network where the risk of ssh connection drop practically
> diminishes.

Sure no problem!
The change description might not be clear enough. So I will provide more details on each item:

* Avoid passing the host, username, port in every run_command execution. --> Currently we have a run_command at module level so we need to provide host, username, etc every time we call it. Encapsulating the logic in a class and providing the parameters in the object initialization, we ensure that the ssh specific parameters will be provided once per remote helper.
* Customize run command with cwd option --> per your request, I added cwd named parameter in run_command so you can specify the dir in which the command will be executed.
* Add pull method --> the pull was also at module level.
* Ensure we execute each command against an active connection provided by the SSHManager --> the fact that the ssh connections are more likely to drop if tests are executed over internet, does not alter the design (the keywords are *more likely*). I mean, we could use the same client object as well, but in this case we would me assuming the risk. Do you note any performance issues of requesting an active connection in every command? Remember that it is stored in the Manager connections pool. Furthermore, if I change the logic to use the same connection on the ssh driver, I will not be using the feature that the SshManager provides.

Finally, we currently have on trunk a custom SShClient class that extends the paramiko one and overrides exec_command just to provide a dir in which the command will be executed, and it is not using our manager to get the ssh connection.

Let me know if that make more sense for you now.

Revision history for this message
Heber Parrucci (heber013) wrote :

> > Excuse my ignorance, can you tell what problem this branch tries to solve ?
> > From the look of it, it seems to be making things more complex. The
> > SSHClientDriver is now the only consumer of SSHManager.
> >
> > I do see the SSHClientDriver gets a connected instance from the manager each
> > time a command is run, this does solve a problem, though that will only be
> > relevant if our tests were running over the internet, right now they are
> > running on the local network.
> >
> > Do you know if in future our tests will be running over the internet instead
> > of the local network where the risk of ssh connection drop practically
> > diminishes.
>
> Sure no problem!
> The change description might not be clear enough. So I will provide more
> details on each item:
>
>
> * Avoid passing the host, username, port in every run_command execution. -->
> Currently we have a run_command at module level so we need to provide host,
> username, etc every time we call it. Encapsulating the logic in a class and
> providing the parameters in the object initialization, we ensure that the ssh
> specific parameters will be provided once per remote helper.
> * Customize run command with cwd option --> per your request, I added cwd
> named parameter in run_command so you can specify the dir in which the command
> will be executed.
> * Add pull method --> the pull was also at module level.
> * Ensure we execute each command against an active connection provided by the
> SSHManager --> the fact that the ssh connections are more likely to drop if
> tests are executed over internet, does not alter the design (the keywords are
> *more likely*). I mean, we could use the same client object as well, but in
> this case we would me assuming the risk. Do you note any performance issues of
> requesting an active connection in every command? Remember that it is stored
> in the Manager connections pool. Furthermore, if I change the logic to use the
> same connection on the ssh driver, I will not be using the feature that the
> SshManager provides.
>
>
> Finally, we currently have on trunk a custom SShClient class that extends the
> paramiko one and overrides exec_command just to provide a dir in which the
> command will be executed, and it is not using our manager to get the ssh
> connection.
>
> Let me know if that make more sense for you now.

I forgot, we currently have on trunk two alternatives to run a ssh command:

1- Calling exec_command of our custom SSHClient (does not use the manager to get an active connection)
2- calling run_command in snappy_ecosystem_tests/utils/ssh.py Which actually uses the manager.

Why would we need 2 ways to execute a ssh command? This change unifies them.

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Looks good in general. Minor comment inline

review: Needs Fixing
Revision history for this message
Omer Akram (om26er) wrote :
Download full text (3.3 KiB)

On Mon, Mar 6, 2017 at 10:22 PM, Heber Parrucci <
<email address hidden>> wrote:

> > Excuse my ignorance, can you tell what problem this branch tries to
> solve ?
> > From the look of it, it seems to be making things more complex. The
> > SSHClientDriver is now the only consumer of SSHManager.
> >
> > I do see the SSHClientDriver gets a connected instance from the manager
> each
> > time a command is run, this does solve a problem, though that will only
> be
> > relevant if our tests were running over the internet, right now they are
> > running on the local network.
> >
> > Do you know if in future our tests will be running over the internet
> instead
> > of the local network where the risk of ssh connection drop practically
> > diminishes.
>
> Sure no problem!
> The change description might not be clear enough. So I will provide more
> details on each item:
>
>
> * Avoid passing the host, username, port in every run_command execution.
> --> Currently we have a run_command at module level so we need to provide
> host, username, etc every time we call it. Encapsulating the logic in a
> class and providing the parameters in the object initialization, we ensure
> that the ssh specific parameters will be provided once per remote helper.
>

That can be done on current trunk right now, SSHManager.get_instance() is
for that.

> * Customize run command with cwd option --> per your request, I added cwd
> named parameter in run_command so you can specify the dir in which the
> command will be executed.
> * Add pull method --> the pull was also at module level.
>

I thought these methods already existed in the extended SSHClient, does
putting them behind a driver make things simpler/easier to consume ?

> * Ensure we execute each command against an active connection provided by
> the SSHManager --> the fact that the ssh connections are more likely to
> drop if tests are executed over internet, does not alter the design (the
> keywords are *more likely*). I mean, we could use the same client object as
> well, but in this case we would me assuming the risk. Do you note any
> performance issues of requesting an active connection in every command?
> Remember that it is stored in the Manager connections pool. Furthermore, if
> I change the logic to use the same connection on the ssh driver, I will not
> be using the feature that the SshManager provides.
>

If we don't run tests over the internet, the chances of connection drop are
practically zero, unless we are hit by a nasty bug.

>
>
> Finally, we currently have on trunk a custom SShClient class that extends
> the paramiko one and overrides exec_command just to provide a dir in which
> the command will be executed, and it is not using our manager to get the
> ssh connection.
>

run_command is indeed redundant in the presence of
SSHManager.get_instance(), we can remove that and *not* introduce
SSHClientDriver if it does not solve any problems we are facing.

> Let me know if that make more sense for you now.
>

Thanks for the detailed response. :)

>
> --
> https://code.launchpad.net/~canonical-platform-qa/snappy-
> ecosystem-tests/adding-ssh-client-driver/+merge/318948
> You are reviewing the ...

Read more...

Revision history for this message
Omer Akram (om26er) wrote :
Download full text (3.6 KiB)

On Mon, Mar 6, 2017 at 10:30 PM, Heber Parrucci <
<email address hidden>> wrote:

> > > Excuse my ignorance, can you tell what problem this branch tries to
> solve ?
> > > From the look of it, it seems to be making things more complex. The
> > > SSHClientDriver is now the only consumer of SSHManager.
> > >
> > > I do see the SSHClientDriver gets a connected instance from the
> manager each
> > > time a command is run, this does solve a problem, though that will
> only be
> > > relevant if our tests were running over the internet, right now they
> are
> > > running on the local network.
> > >
> > > Do you know if in future our tests will be running over the internet
> instead
> > > of the local network where the risk of ssh connection drop practically
> > > diminishes.
> >
> > Sure no problem!
> > The change description might not be clear enough. So I will provide more
> > details on each item:
> >
> >
> > * Avoid passing the host, username, port in every run_command execution.
> -->
> > Currently we have a run_command at module level so we need to provide
> host,
> > username, etc every time we call it. Encapsulating the logic in a class
> and
> > providing the parameters in the object initialization, we ensure that
> the ssh
> > specific parameters will be provided once per remote helper.
> > * Customize run command with cwd option --> per your request, I added cwd
> > named parameter in run_command so you can specify the dir in which the
> command
> > will be executed.
> > * Add pull method --> the pull was also at module level.
> > * Ensure we execute each command against an active connection provided
> by the
> > SSHManager --> the fact that the ssh connections are more likely to drop
> if
> > tests are executed over internet, does not alter the design (the
> keywords are
> > *more likely*). I mean, we could use the same client object as well, but
> in
> > this case we would me assuming the risk. Do you note any performance
> issues of
> > requesting an active connection in every command? Remember that it is
> stored
> > in the Manager connections pool. Furthermore, if I change the logic to
> use the
> > same connection on the ssh driver, I will not be using the feature that
> the
> > SshManager provides.
> >
> >
> > Finally, we currently have on trunk a custom SShClient class that
> extends the
> > paramiko one and overrides exec_command just to provide a dir in which
> the
> > command will be executed, and it is not using our manager to get the ssh
> > connection.
> >
> > Let me know if that make more sense for you now.
>
> I forgot, we currently have on trunk two alternatives to run a ssh command:
>
> 1- Calling exec_command of our custom SSHClient (does not use the manager
> to get an active connection)
>

If a function uses SSHManager.get_instance() it would indeed be getting a
connected instance. If a class instance stores a reference to the
SSHClient, then in case tests are run over the internet we have the issue,
given we will be running them locally the issue does not exist to solve :)

> 2- calling run_command in snappy_ecosystem_tests/utils/ssh.py Which
> actually uses the manager.

> Why would we need 2 ways to execute...

Read more...

Revision history for this message
Heber Parrucci (heber013) wrote :
Download full text (4.6 KiB)

> On Mon, Mar 6, 2017 at 10:30 PM, Heber Parrucci <
> <email address hidden>> wrote:
>
> > > > Excuse my ignorance, can you tell what problem this branch tries to
> > solve ?
> > > > From the look of it, it seems to be making things more complex. The
> > > > SSHClientDriver is now the only consumer of SSHManager.
> > > >
> > > > I do see the SSHClientDriver gets a connected instance from the
> > manager each
> > > > time a command is run, this does solve a problem, though that will
> > only be
> > > > relevant if our tests were running over the internet, right now they
> > are
> > > > running on the local network.
> > > >
> > > > Do you know if in future our tests will be running over the internet
> > instead
> > > > of the local network where the risk of ssh connection drop practically
> > > > diminishes.
> > >
> > > Sure no problem!
> > > The change description might not be clear enough. So I will provide more
> > > details on each item:
> > >
> > >
> > > * Avoid passing the host, username, port in every run_command execution.
> > -->
> > > Currently we have a run_command at module level so we need to provide
> > host,
> > > username, etc every time we call it. Encapsulating the logic in a class
> > and
> > > providing the parameters in the object initialization, we ensure that
> > the ssh
> > > specific parameters will be provided once per remote helper.
> > > * Customize run command with cwd option --> per your request, I added cwd
> > > named parameter in run_command so you can specify the dir in which the
> > command
> > > will be executed.
> > > * Add pull method --> the pull was also at module level.
> > > * Ensure we execute each command against an active connection provided
> > by the
> > > SSHManager --> the fact that the ssh connections are more likely to drop
> > if
> > > tests are executed over internet, does not alter the design (the
> > keywords are
> > > *more likely*). I mean, we could use the same client object as well, but
> > in
> > > this case we would me assuming the risk. Do you note any performance
> > issues of
> > > requesting an active connection in every command? Remember that it is
> > stored
> > > in the Manager connections pool. Furthermore, if I change the logic to
> > use the
> > > same connection on the ssh driver, I will not be using the feature that
> > the
> > > SshManager provides.
> > >
> > >
> > > Finally, we currently have on trunk a custom SShClient class that
> > extends the
> > > paramiko one and overrides exec_command just to provide a dir in which
> > the
> > > command will be executed, and it is not using our manager to get the ssh
> > > connection.
> > >
> > > Let me know if that make more sense for you now.
> >
> > I forgot, we currently have on trunk two alternatives to run a ssh command:
> >
> > 1- Calling exec_command of our custom SSHClient (does not use the manager
> > to get an active connection)
> >
>
> If a function uses SSHManager.get_instance() it would indeed be getting a
> connected instance. If a class instance stores a reference to the
> SSHClient, then in case tests are run over the internet we have the issue,
> given we will be running them locally the issue does not ex...

Read more...

Revision history for this message
Heber Parrucci (heber013) wrote :

> Looks good in general. Minor comment inline
Applies the same response that for Omer:
We are going to run the tests over internet in the future for devices like: raspberry Pi, dragonboards, etc. And those devices will be on the lab (at least for running tests on Jenkins). In those cases we have to ensure that every time we execute a command over ssh, the connection is active.
That's why we cannot save the SshClient instance in the helpers, because we cannot ask to the client itself to get an active connection in every command execution.
The driver was created to resolve that problem. You store an instance in your helper and then execute commands, for every command you execute, it ensures that the ssh connection for that client is active by requesting the manager.
How can we achieve that using the SshClient instead of the driver?

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Thanks for the explanation. What about keeping the ssh client and have something like this?

class EcosystemObject:

    def __init__(ip, user, port):
        self.ip = ip
        self.user = user
        self.port = port

    @property
    def ssh(self):
       return SSHManager.get_instance(self.ip, self.user, self.port)

class Snapcraft(EcosystemObject):

class Snapd(EcosystemObject)

That way every time you instantiate Snapcraft or Snapd, you'll get a property ssh that will always point to an active ssh connection

32. By Heber Parrucci

Addressing review comments:
* Deleting SShDriver and adding back Sshclient
* Cwd parameter removed from client and adding in helper (To mantain paramiko's method signature)
* adding ssh property in each helper that needs ssh, to ensure an active connection for every command execution.

Revision history for this message
Heber Parrucci (heber013) wrote :

> Thanks for the explanation. What about keeping the ssh client and have
> something like this?
>
> class EcosystemObject:
>
> def __init__(ip, user, port):
> self.ip = ip
> self.user = user
> self.port = port
>
> @property
> def ssh(self):
> return SSHManager.get_instance(self.ip, self.user, self.port)
>
> class Snapcraft(EcosystemObject):
>
> class Snapd(EcosystemObject)
>
> That way every time you instantiate Snapcraft or Snapd, you'll get a property
> ssh that will always point to an active ssh connection

Thanks for the feedback. Comments addressed:
* Deleting SShDriver and adding back Sshclient
* Cwd parameter removed from client and adding in helper (To mantain paramiko's method signature)
* adding ssh property in each helper that needs ssh, to ensure an active connection for every command execution.

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Omer Akram (om26er) wrote :

Code looks good to me.

review: Approve
33. By Heber Parrucci

merge from trunk

Revision history for this message
platform-qa-bot (platform-qa-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Looks good

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'snappy_ecosystem_tests/environment/setup.py'
--- snappy_ecosystem_tests/environment/setup.py 2017-03-01 14:52:43 +0000
+++ snappy_ecosystem_tests/environment/setup.py 2017-03-07 17:41:57 +0000
@@ -28,6 +28,7 @@
2828
29logging.basicConfig(level=logging.DEBUG)29logging.basicConfig(level=logging.DEBUG)
3030
31
31def main(mode, profile):32def main(mode, profile):
32 """Main script"""33 """Main script"""
33 manager = get_manager(mode)34 manager = get_manager(mode)
3435
=== modified file 'snappy_ecosystem_tests/helpers/snapcraft/build_snap.py'
--- snappy_ecosystem_tests/helpers/snapcraft/build_snap.py 2017-03-02 16:24:59 +0000
+++ snappy_ecosystem_tests/helpers/snapcraft/build_snap.py 2017-03-07 17:41:57 +0000
@@ -53,13 +53,21 @@
53class SnapBuilder:53class SnapBuilder:
54 """Class to build a snap on a remote machine and download its content."""54 """Class to build a snap on a remote machine and download its content."""
55 def __init__(self, hostname, username, port=22):55 def __init__(self, hostname, username, port=22):
56 self.client = SSHManager.get_instance(hostname, username, port)56 self.hostname = hostname
57 self.username = username
58 self.port = port
59
60 @property
61 def ssh(self):
62 """Property tu ensure we get an active connection for every command
63 execution over ssh"""
64 return SSHManager.get_instance(self.hostname, self.username, self.port)
5765
58 def is_host_setup(self):66 def is_host_setup(self):
59 """Return bool representing whether the remote host is setup by67 """Return bool representing whether the remote host is setup by
60 checking if snapcraft package is installed."""68 checking if snapcraft package is installed."""
61 try:69 try:
62 self.client.exec_command('which snapcraft')70 self.ssh.exec_command('which snapcraft')
63 return True71 return True
64 except ValueError:72 except ValueError:
65 return False73 return False
@@ -68,7 +76,7 @@
68 """Setup the host to build a snap, currently only installs76 """Setup the host to build a snap, currently only installs
69 snapcraft."""77 snapcraft."""
70 for command in COMMANDS_SETUP:78 for command in COMMANDS_SETUP:
71 self.client.exec_command(command)79 self.ssh.exec_command(command)
7280
73 def build(self, name, version, summary, description, grade='stable',81 def build(self, name, version, summary, description, grade='stable',
74 confinement='strict'):82 confinement='strict'):
@@ -86,9 +94,9 @@
86 """94 """
87 if not self.is_host_setup():95 if not self.is_host_setup():
88 self.setup_host()96 self.setup_host()
89 tempdir = self.client.exec_command('mktemp -d')97 tempdir = self.ssh.exec_command('mktemp -d')
90 self.client.exec_command('mkdir snap', cwd=tempdir)98 self.ssh.exec_command('mkdir snap', cwd=tempdir)
91 self.client.exec_command('echo "{}" > snap/snapcraft.yaml'.format(99 self.ssh.exec_command('echo "{}" > snap/snapcraft.yaml'.format(
92 SNAPCRAFT_YAML.format(100 SNAPCRAFT_YAML.format(
93 snap_name=name,101 snap_name=name,
94 version=version,102 version=version,
@@ -98,13 +106,13 @@
98 confinement=confinement106 confinement=confinement
99 )107 )
100 ), cwd=tempdir)108 ), cwd=tempdir)
101 self.client.exec_command('snapcraft', cwd=tempdir)109 self.ssh.exec_command('snapcraft', cwd=tempdir)
102 return os.path.join(110 return os.path.join(
103 tempdir,111 tempdir,
104 '{name}_{version}_{arch}.snap'.format(112 '{name}_{version}_{arch}.snap'.format(
105 name=name,113 name=name,
106 version=version,114 version=version,
107 arch=self.client.exec_command('dpkg --print-architecture')115 arch=self.ssh.exec_command('dpkg --print-architecture')
108 )116 )
109 )117 )
110118
@@ -136,5 +144,5 @@
136 output_path_absolute = os.path.abspath(144 output_path_absolute = os.path.abspath(
137 os.path.join(output_location, PurePath(remote_snap_path).name)145 os.path.join(output_location, PurePath(remote_snap_path).name)
138 )146 )
139 self.client.pull(remote_snap_path, output_path_absolute)147 self.ssh.pull(remote_snap_path, output_path_absolute)
140 return output_path_absolute148 return output_path_absolute
141149
=== modified file 'snappy_ecosystem_tests/helpers/snapcraft/client.py'
--- snappy_ecosystem_tests/helpers/snapcraft/client.py 2017-03-03 14:06:17 +0000
+++ snappy_ecosystem_tests/helpers/snapcraft/client.py 2017-03-07 17:41:57 +0000
@@ -25,9 +25,9 @@
25import time25import time
2626
27from snappy_ecosystem_tests.models.snap import Snap27from snappy_ecosystem_tests.models.snap import Snap
28from snappy_ecosystem_tests.utils import ssh
29from snappy_ecosystem_tests.utils.commands import build_command28from snappy_ecosystem_tests.utils.commands import build_command
30from snappy_ecosystem_tests.utils.filters import filter_list29from snappy_ecosystem_tests.utils.filters import filter_list
30from snappy_ecosystem_tests.utils.ssh import SSHManager
31from snappy_ecosystem_tests.utils.storeconfig import get_current_store31from snappy_ecosystem_tests.utils.storeconfig import get_current_store
3232
33from snappy_ecosystem_tests.commons.config import CONFIG_STACK33from snappy_ecosystem_tests.commons.config import CONFIG_STACK
@@ -72,8 +72,13 @@
72 self._login = False72 self._login = False
73 self._cleanup()73 self._cleanup()
7474
75 @staticmethod75 @property
76 def _run_snapcraft_command_ssh(command, debug=True):76 def ssh(self):
77 """Property tu ensure we get an active connection for every command
78 execution over ssh"""
79 return SSHManager.get_instance(HOSTNAME, USERNAME, PORT)
80
81 def _run_snapcraft_command_ssh(self, command, debug=True):
77 """82 """
78 Run snapcraft command via ssh83 Run snapcraft command via ssh
79 :param command: the command to be executed.84 :param command: the command to be executed.
@@ -85,25 +90,21 @@
85 command = build_snapcraft_command(command)90 command = build_snapcraft_command(command)
86 if debug:91 if debug:
87 command += ' -d'92 command += ' -d'
88 return ssh.run_command(command, hostname=HOSTNAME, username=USERNAME,93 return self.ssh.exec_command(command)
89 port=PORT)
9094
91 @staticmethod95 def assert_logged_in(self):
92 def assert_logged_in():
93 """Assert that an user is logged.96 """Assert that an user is logged.
94 :raise ValueError: in case login is false97 :raise ValueError: in case login is false
95 """98 """
96 if Snapcraft.is_logged_in() is False:99 if self.is_logged_in() is False:
97 raise ValueError("User is not logged in, "100 raise ValueError("User is not logged in, "
98 "please login before using this command")101 "please login before using this command")
99102
100 @staticmethod103 def is_logged_in(self):
101 def is_logged_in():
102 """Return bool representing if the user is logged into snapcraft."""104 """Return bool representing if the user is logged into snapcraft."""
103 try:105 try:
104 output = ssh.run_command(106 output = self.ssh.exec_command(
105 'cat %s' % SNAPCRAFT_CONFIG_FILE_PATH,107 'cat %s' % SNAPCRAFT_CONFIG_FILE_PATH)
106 hostname=HOSTNAME, username=USERNAME, port=PORT)
107 return output is not ''108 return output is not ''
108 except ValueError:109 except ValueError:
109 return False110 return False
@@ -112,26 +113,23 @@
112 """Perform cleanup actions"""113 """Perform cleanup actions"""
113 self.logout()114 self.logout()
114115
115 @staticmethod116 def logout(self):
116 def logout():
117 """logout of snapcraft store session117 """logout of snapcraft store session
118 :return: True if logout was successful, False otherwise118 :return: True if logout was successful, False otherwise
119 """119 """
120 Snapcraft._run_snapcraft_command_ssh(COMMAND_LOGOUT)120 self._run_snapcraft_command_ssh(COMMAND_LOGOUT)
121 return not Snapcraft.is_logged_in()121 return not self.is_logged_in()
122122
123 @staticmethod123 def login(self, email=LOGIN_EMAIL, password=LOGIN_PASSWORD):
124 def login(email=LOGIN_EMAIL, password=LOGIN_PASSWORD):
125 """Login to Snapcraft. If email and/or password are not provided,124 """Login to Snapcraft. If email and/or password are not provided,
126 use the default ones.125 use the default ones.
127 :param email: the SSO email126 :param email: the SSO email
128 :param password: the email password127 :param password: the email password
129 """128 """
130 ssh.run_command(129 self.ssh.exec_command((
131 COMMANDS_LOGIN.format(email=email, password=password,130 COMMANDS_LOGIN.format(email=email, password=password,
132 snapcraft=PATH_SNAPCRAFT),131 snapcraft=PATH_SNAPCRAFT)))
133 hostname=HOSTNAME, username=USERNAME, port=PORT)132 return self.is_logged_in()
134 return Snapcraft.is_logged_in()
135133
136 def list_registered(self, debug=True, raw=False):134 def list_registered(self, debug=True, raw=False):
137 """Call snapcraft list-registered command,135 """Call snapcraft list-registered command,
@@ -180,7 +178,7 @@
180 if private:178 if private:
181 command += ' --private'179 command += ' --private'
182 try:180 try:
183 Snapcraft._run_snapcraft_command_ssh(command)181 self._run_snapcraft_command_ssh(command)
184 except subprocess.CalledProcessError as _e:182 except subprocess.CalledProcessError as _e:
185 LOGGER.error('Unable to register snap: %s', _e.output)183 LOGGER.error('Unable to register snap: %s', _e.output)
186 return False184 return False
187185
=== modified file 'snappy_ecosystem_tests/helpers/snapd/snapd.py'
--- snappy_ecosystem_tests/helpers/snapd/snapd.py 2017-03-02 18:36:24 +0000
+++ snappy_ecosystem_tests/helpers/snapd/snapd.py 2017-03-07 17:41:57 +0000
@@ -25,7 +25,7 @@
2525
26import yaml26import yaml
2727
28from snappy_ecosystem_tests.utils import ssh28from snappy_ecosystem_tests.utils.ssh import SSHManager
29from snappy_ecosystem_tests.utils.user import get_snapd_remote_host_credentials29from snappy_ecosystem_tests.utils.user import get_snapd_remote_host_credentials
3030
31PATH_SNAP = '/usr/bin/snap'31PATH_SNAP = '/usr/bin/snap'
@@ -51,107 +51,116 @@
51HOSTNAME, USERNAME, PORT = get_snapd_remote_host_credentials()51HOSTNAME, USERNAME, PORT = get_snapd_remote_host_credentials()
5252
5353
54def run_snapd_command_ssh(parameters, cwd=''):54class Snapd:
55 """Run snapd command over ssh.55 """Contain Snapd specific functionality to use via command
5656 line interface"""
57 :param parameters: a string containing parameters for the `snap` command.57
58 :param cwd: the current working directory on the remote where the command58 @property
59 should run.59 def ssh(self):
60 :returns: stdout of the command60 """Property tu ensure we get an active connection for every command
61 """61 execution over ssh"""
62 return ssh.run_command('{} {}'.format(PATH_SNAP, parameters), cwd=cwd,62 return SSHManager.get_instance(HOSTNAME, USERNAME, PORT)
63 hostname=HOSTNAME, username=USERNAME, port=PORT)63
6464 def run_snapd_command_ssh(self, command, cwd=''):
6565 """Run snapd command over ssh.
66def login(email, password):66
67 """Login to snapd.67 :param command: a string containing parameters for the `snap` command.
6868 :param cwd: the current working directory on the remote where
69 :param email: Ubuntu SSO account email address.69 the command should run.
70 :param password: Ubuntu SSO account password.70 :returns: stdout of the command
71 """71 """
72 ssh.run_command(COMMANDS_LOGIN.format(email=email, password=password))72 if cwd:
73 return is_logged_in(email)73 command = '{};{}'.format(cwd, command)
7474 return self.ssh.exec_command(command)
7575
76def is_logged_in(email):76 def login(self, email, password):
77 """Return bool representing if the user is logged into snapd."""77 """Login to snapd.
78 try:78
79 return json.loads(ssh.run_command('cat ~/.snap/auth.json')).get(79 :param email: Ubuntu SSO account email address.
80 'email') == email80 :param password: Ubuntu SSO account password.
81 except ValueError:81 """
82 self.ssh.exec_command(COMMANDS_LOGIN.format(email=email,
83 password=password))
84 return self.is_logged_in(email)
85
86 def is_logged_in(self, email):
87 """Return bool representing if the user is logged into snapd."""
88 try:
89 return json.loads(
90 self.ssh.exec_command(
91 'cat ~/.snap/auth.json')).get('email') == email
92 except ValueError:
93 return False
94
95 def logout(self, email):
96 """Logout snapd current user."""
97 if self.is_logged_in(email):
98 self.ssh.exec_command(COMMAND_LOGOUT)
99
100 def download(self, snap, channel=CHANNEL_STABLE):
101 """Download the requested snap.
102
103 :param snap: name of the snap to download.
104 :param channel: name of the release channel to download from.
105 """
106 command = COMMAND_DOWNLOAD.format(snap=snap, channel=channel)
107 self.run_snapd_command_ssh(command,
108 cwd=self.ssh.exec_command('mktemp -d'))
109
110 def install(self, snap, channel=CHANNEL_STABLE):
111 """Install the requested snap."""
112 self.ssh.exec_command(COMMAND_INSTALL.format(snap=snap,
113 channel=channel))
114
115 def _is_installed(self, snap):
116 """Return bool representing whether a snap is installed."""
117 for installed_snap in Snapd._parse_output(
118 self.run_snapd_command_ssh(COMMAND_LIST)):
119 if installed_snap['name'] == snap:
120 return True
82 return False121 return False
83122
84123 def remove(self, snap):
85def logout(email):124 """Remove a snap, if its already installed."""
86 """Logout snapd current user."""125 if self._is_installed(snap):
87 if is_logged_in(email):126 self.run_snapd_command_ssh(COMMAND_REMOVE.format(snap=snap))
88 run_snapd_command_ssh(COMMAND_LOGOUT)127
89128 def info(self, snap, verbose=False):
90129 """Query the Ubuntu store of the information about a snap.
91def download(snap, channel=CHANNEL_STABLE):130
92 """Download the requested snap.131 :param snap: Name of the snap for which the info is required.
93132 :param verbose: Whether to information should be detailed.
94 :param snap: name of the snap to download.133 :return: Return a dictionary containing information about the snap.
95 :param channel: name of the release channel to download from.134 """
96 """135 command = COMMAND_INFO.format(snap=snap)
97 command = COMMAND_DOWNLOAD.format(snap=snap, channel=channel)136 if verbose:
98 run_snapd_command_ssh(command, cwd=ssh.run_command('mktemp -d'))137 command = ' '.join([command, '--verbose'])
99138 return yaml.load("""{}""".format(self.run_snapd_command_ssh(command)))
100139
101def install(snap, channel=CHANNEL_STABLE):140 def refresh(self, snap, channel=CHANNEL_STABLE):
102 """Install the requested snap."""141 """Refresh the requested snap."""
103 run_snapd_command_ssh(COMMAND_INSTALL.format(snap=snap, channel=channel))142 self.run_snapd_command_ssh(COMMAND_REFRESH.format(snap=snap,
104143 channel=channel))
105144
106def _is_installed(snap):145 @staticmethod
107 """Return bool representing whether a snap is installed."""146 def _parse_output(raw_output):
108 for installed_snap in _parse_output(run_snapd_command_ssh(COMMAND_LIST)):147 """Pretty parse the output from snapd commands like `find` and `list`.
109 if installed_snap['name'] == snap:148
110 return True149 :param raw_output: The raw output returned from the command
111 return False150 :return: A list of dictionaries containing sorted results
112151 from the output.
113152 """
114def remove(snap):153 split_output = raw_output.split('\n')
115 """Remove a snap, if its already installed."""154 headers = [header.lower() for header in split_output.pop(0).split()]
116 if _is_installed(snap):155 return [dict(zip(headers, line.split())) for line in split_output]
117 run_snapd_command_ssh(COMMAND_REMOVE.format(snap=snap))156
118157 def find(self, keyword):
119158 """Find snaps based on the provided filters
120def info(snap, verbose=False):159
121 """Query the Ubuntu store of the information about a snap.160 :param keyword: Keyword to use for the query.
122161 :return: Return a list of dictionaries containing information about
123 :param snap: Name of the snap for which the info is required.162 snaps matching the `keyword`.
124 :param verbose: Whether to information should be detailed.163 """
125 :return: Return a dictionary containing information about the snap.164 return Snapd._parse_output(
126 """165 self.run_snapd_command_ssh(COMMAND_FIND.format(
127 command = COMMAND_INFO.format(snap=snap)166 search_term=keyword)))
128 if verbose:
129 command = ' '.join([command, '--verbose'])
130 return yaml.load("""{}""".format(run_snapd_command_ssh(command)))
131
132
133def refresh(snap, channel=CHANNEL_STABLE):
134 """Refresh the requested snap."""
135 run_snapd_command_ssh(COMMAND_REFRESH.format(snap=snap, channel=channel))
136
137
138def _parse_output(raw_output):
139 """Pretty parse the output from snapd commands like `find` and `list`.
140
141 :param raw_output: The raw output returned from the command
142 :return: A list of dictionaries containing sorted results from the output.
143 """
144 split_output = raw_output.split('\n')
145 headers = [header.lower() for header in split_output.pop(0).split()]
146 return [dict(zip(headers, line.split())) for line in split_output]
147
148
149def find(keyword):
150 """Find snaps based on the provided filters
151
152 :param keyword: Keyword to use for the query.
153 :return: Return a list of dictionaries containing information about
154 snaps matching the `keyword`.
155 """
156 return _parse_output(
157 run_snapd_command_ssh(COMMAND_FIND.format(search_term=keyword)))
158167
=== modified file 'snappy_ecosystem_tests/tests/test_snapd.py'
--- snappy_ecosystem_tests/tests/test_snapd.py 2017-02-22 15:47:09 +0000
+++ snappy_ecosystem_tests/tests/test_snapd.py 2017-03-07 17:41:57 +0000
@@ -22,7 +22,7 @@
2222
23import testtools23import testtools
2424
25from snappy_ecosystem_tests.helpers.snapd import snapd25from snappy_ecosystem_tests.helpers.snapd.snapd import Snapd
26from snappy_ecosystem_tests.utils.storeconfig import get_store_credentials26from snappy_ecosystem_tests.utils.storeconfig import get_store_credentials
2727
2828
@@ -30,13 +30,14 @@
30 """Tests for snapd."""30 """Tests for snapd."""
31 def setUp(self):31 def setUp(self):
32 super().setUp()32 super().setUp()
33 self.snapd = Snapd()
3334
34 def test_info(self):35 def test_info(self):
35 """Ensure the publisher of core snap in canonical."""36 """Ensure the publisher of core snap in canonical."""
36 self.assertTrue('canonical', snapd.info('core')['publisher'])37 self.assertTrue('canonical', self.snapd.info('core')['publisher'])
3738
38 def test_login(self):39 def test_login(self):
39 """Login the snapd user."""40 """Login the snapd user."""
40 email, password = get_store_credentials()41 email, password = get_store_credentials()
41 self.assertTrue(snapd.login(email, password))42 self.assertTrue(self.snapd.login(email, password))
42 self.addCleanup(snapd.logout, email)43 self.addCleanup(self.snapd.logout, email)
4344
=== modified file 'snappy_ecosystem_tests/utils/ssh.py'
--- snappy_ecosystem_tests/utils/ssh.py 2017-03-02 20:38:49 +0000
+++ snappy_ecosystem_tests/utils/ssh.py 2017-03-07 17:41:57 +0000
@@ -21,9 +21,9 @@
21"""Module to connect to ssh client for running tests."""21"""Module to connect to ssh client for running tests."""
2222
23import logging23import logging
24
24import paramiko25import paramiko
2526
26from snappy_ecosystem_tests.utils.user import get_snapd_remote_host_credentials
27from snappy_ecosystem_tests.utils.storeconfig import get_current_store27from snappy_ecosystem_tests.utils.storeconfig import get_current_store
28from snappy_ecosystem_tests.environment.data.snapd import STAGING_VARIABLES28from snappy_ecosystem_tests.environment.data.snapd import STAGING_VARIABLES
2929
@@ -33,13 +33,17 @@
33class SSHClient(paramiko.SSHClient):33class SSHClient(paramiko.SSHClient):
34 """Extended SSHClient."""34 """Extended SSHClient."""
3535
36 def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False, # pylint: disable=arguments-differ36 def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False,
37 environment=None, cwd=''):37 environment=None):
38 """Execute the given command over ssh."""38 """Execute the given command over ssh."""
39 if cwd:39 if environment is None:
40 command = 'cd {}; {}'.format(cwd, command)40 # FIXME: get environment variables dinamically
41 _, stdout, stderr = super().exec_command(command, bufsize, timeout,41 if get_current_store() == 'staging':
42 get_pty, environment)42 environment = STAGING_VARIABLES
43 _, stdout, stderr = super().exec_command(
44 command, bufsize, timeout,
45 get_pty,
46 environment=environment)
43 if stdout.channel.recv_exit_status() != 0:47 if stdout.channel.recv_exit_status() != 0:
44 raise ValueError(stderr.read().decode().strip())48 raise ValueError(stderr.read().decode().strip())
45 response = stdout.read().decode().strip()49 response = stdout.read().decode().strip()
@@ -102,27 +106,3 @@
102 username == client.get_transport().get_username()106 username == client.get_transport().get_username()
103 ]107 ]
104 return connections[0] if connections else None108 return connections[0] if connections else None
105
106
107def run_command(command, cwd=None, hostname=None, username=None, port=None):
108 """Run the given command on remote machine over ssh.
109
110 :param command: a string of the command to run.
111 :param cwd: Current working directory for the command.
112 :param hostname: The host to run command on.
113 :param username: Name of the user on the remote host to login to.
114 :param port: SSH port number.
115 :raises ValueError: if command exits with non-zero status.
116 :return: the stdout of the command.
117 """
118 # Take snapd remote credentials as default if they are not provided.
119 # Just arbitrary... Could have been other ones.
120 _hostname, _username, _port = get_snapd_remote_host_credentials()
121 ssh = SSHManager.get_instance(
122 hostname or _hostname, username or _username, port or _port)
123 return ssh.exec_command(
124 command,
125 cwd=cwd,
126 environment=STAGING_VARIABLES if get_current_store() == 'staging'
127 else None
128 )

Subscribers

People subscribed via source and target branches