Merge lp:~milo/lava-tool/lava-165 into lp:~linaro-validation/lava-tool/trunk

Proposed by Milo Casagrande
Status: Merged
Merged at revision: 188
Proposed branch: lp:~milo/lava-tool/lava-165
Merge into: lp:~linaro-validation/lava-tool/trunk
Prerequisite: lp:~milo/lava-tool/device-parameters
Diff against target: 3405 lines (+2196/-459)
30 files modified
.coveragerc (+1/-0)
entry_points.ini (+8/-0)
lava/commands.py (+240/-0)
lava/config.py (+50/-17)
lava/device/__init__.py (+3/-3)
lava/device/commands.py (+8/-6)
lava/device/tests/test_commands.py (+25/-18)
lava/device/tests/test_device.py (+16/-11)
lava/helper/command.py (+137/-60)
lava/helper/template.py (+80/-0)
lava/helper/tests/helper_test.py (+24/-4)
lava/helper/tests/test_command.py (+0/-89)
lava/helper/tests/test_template.py (+102/-0)
lava/job/__init__.py (+40/-12)
lava/job/commands.py (+21/-77)
lava/job/templates.py (+84/-19)
lava/job/tests/test_commands.py (+2/-9)
lava/job/tests/test_job.py (+40/-29)
lava/parameter.py (+199/-17)
lava/testdef/__init__.py (+60/-0)
lava/testdef/commands.py (+72/-0)
lava/testdef/templates.py (+75/-0)
lava/testdef/tests/test_commands.py (+153/-0)
lava/tests/test_commands.py (+128/-0)
lava/tests/test_config.py (+59/-65)
lava/tests/test_parameter.py (+92/-11)
lava_tool/tests/__init__.py (+13/-10)
lava_tool/tests/test_utils.py (+177/-2)
lava_tool/utils.py (+286/-0)
setup.py (+1/-0)
To merge this branch: bzr merge lp:~milo/lava-tool/lava-165
Reviewer Review Type Date Requested Status
Antonio Terceiro Pending
Review via email: mp+174942@code.launchpad.net

Description of the change

This adds final work towards completing the lava helper tools for card-15.
The merge adds the following commands:

lava init [DIR]
lava run [JOB|DIR]
lava submit [JOB|DIR]
lava update [JOB|DIR]

The "init" command initializes a directory with a job definition file, and a default subdirectory called "tests/" that will contain the test definition file(s), and a default shell script, called "mytest.sh". The test definition file steps, by default, will execute the shell script. The user is prompted to fill in the shell script.

The "run" command runs a job file on the local dispatcher, the "submit" one will send the job file to a remote LAVA server.

The "update" command can be used, in case the user modifies the content of the "tests/" directory, to update the job file.

The "run", "submit" and "update" commands can accept a job file (.json extension), a directory (it will pick up the first .json file) or nothing to use the current working directory.

New Parameter class have been introduced, not used at this stage, but where used with the first implementation. New attributes have also been added in order not to store the parameter on disk at program exit.

Tests have been added to cover most of the new features.

To post a comment you must log in.
lp:~milo/lava-tool/lava-165 updated
389. By Milo Casagrande

Updated copyright info, fixed typo.

390. By Milo Casagrande

Reworked the default job tempalte.

    * Remove the UrlListParameter and introduced the TarRepoParameter.
    * Reworked the key of the job templates.
    * Fixed the tests.

391. By Milo Casagrande

Fixed testdef_repos parameter: it is an array.

392. By Milo Casagrande

Fixed template access.

393. By Milo Casagrande

New functions to get and set a key in a template.

     * Added new tests too.

394. By Milo Casagrande

Used device_type, not target.

395. By Milo Casagrande

Make sure shell script is executable.

    * Removed unused imports.

396. By Milo Casagrande

Fixed docstring.

397. By Milo Casagrande

Removed unused imports.

398. By Milo Casagrande

Added submit_results command.

    * Added new step in job template to submit the results.
      It will ask two parameters: the stream and the server.
    * Fixed timeout value in lava_test_shell command.

399. By Milo Casagrande

Added default parse parttern for testdef, updated tests.

Revision history for this message
Antonio Terceiro (terceiro) wrote :
Download full text (57.8 KiB)

On Tue, Jul 16, 2013 at 07:41:29AM -0000, Milo Casagrande wrote:
> Milo Casagrande has proposed merging lp:~milo/lava-tool/lava-165 into lp:lava-tool with lp:~milo/lava-tool/device-parameters as a prerequisite.
>
> Requested reviews:
> Antonio Terceiro (terceiro)
>
> For more details, see:
> https://code.launchpad.net/~milo/lava-tool/lava-165/+merge/174942
>
> This adds final work towards completing the lava helper tools for card-15.
> The merge adds the following commands:
>
> lava init [DIR]
> lava run [JOB|DIR]
> lava submit [JOB|DIR]
> lava update [JOB|DIR]
>
> The "init" command initializes a directory with a job definition file, and a default subdirectory called "tests/" that will contain the test definition file(s), and a default shell script, called "mytest.sh". The test definition file steps, by default, will execute the shell script. The user is prompted to fill in the shell script.
>
> The "run" command runs a job file on the local dispatcher, the "submit" one will send the job file to a remote LAVA server.
>
> The "update" command can be used, in case the user modifies the content of the "tests/" directory, to update the job file.
>
> The "run", "submit" and "update" commands can accept a job file (.json extension), a directory (it will pick up the first .json file) or nothing to use the current working directory.
>
> New Parameter class have been introduced, not used at this stage, but where used with the first implementation. New attributes have also been added in order not to store the parameter on disk at program exit.
>
> Tests have been added to cover most of the new features.

arf ... this one was big!

> === added file 'lava/commands.py'
> --- lava/commands.py 1970-01-01 00:00:00 +0000
> +++ lava/commands.py 2013-07-22 09:32:26 +0000
> @@ -0,0 +1,293 @@
> +# Copyright (C) 2013 Linaro Limited
> +#
> +# Author: Milo Casagrande <email address hidden>
> +#
> +# This file is part of lava-tool.
> +#
> +# lava-tool is free software: you can redistribute it and/or modify
> +# it under the terms of the GNU Lesser General Public License version 3
> +# as published by the Free Software Foundation
> +#
> +# lava-tool is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public License
> +# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
> +
> +"""
> +Lava init commands.
> +
> +When invoking:
> +
> + `lava init [DIR]`
> +
> +the command will create a default directory and files structure as follows:
> +
> +DIR/
> + |
> + +- JOB_FILE.json
> + +- tests/
> + |
> + + mytest.sh
> + + lavatest.yaml
> +
> +If DIR is not passed, it will use the current working directory.
> +JOB_FILE is a file name that will be asked to the user, along with
> +other necessary information to define the tests.
> +
> +If the user manually updates either the lavatest.yaml or mytest.sh file, it is
> +necessary to run the following command in order t...

Revision history for this message
Milo Casagrande (milo) wrote :
Download full text (12.0 KiB)

On Wed, Jul 24, 2013 at 2:49 AM, Antonio Terceiro
<email address hidden> wrote:
>> +
>> +INIT_TEMPLATE = {
>> + JOBFILE_ID: JOBFILE_PARAMETER,
>> +}
>
> Do we really need a dictionary with a single key?

If we want to maintain the same pattern as for the other commands,
yes: go through the template, look up the parameter in the config, and
ask for the values there.
Or we need to change the "expand_template" function signature and also
the name to accept single Parameter objects.

>> + def _create_files(self, data, full_path, test_path):
>> + # This is the default script file as defined in the testdef template.
>> + default_script = os.path.join(test_path, DEFAULT_TESTDEF_SCRIPT)
>> +
>> + if not os.path.isfile(default_script):
>> + # We do not have the default testdef script. Create it, but
>> + # remind the user to update it.
>> + print >> sys.stdout, ("\nCreating default test script "
>> + "'{0}'.".format(DEFAULT_TESTDEF_SCRIPT))
>> +
>> + with open(default_script, "w") as write_file:
>> + write_file.write(DEFAULT_TESTDEF_SCRIPT_CONTENT)
>
> if this behaviour of "create a file with this content here" starts to repeat,
> we probably want to encapsulatet it somewhere else (I noted it at least once in
> other parts of this MP).

Sounds good, even if at the moment should be only used here.
It was used also in the tests to fill up files with fake content.

>> + # Prompt the user to write the script file.
>> + self.edit_file(default_script)
>> +
>> + # Make sure the script is executable.
>> + os.chmod(default_script,
>> + stat.S_IRWXU | stat.S_IRWXG | stat.S_IROTH | stat.S_IXOTH)
>> +
>> + print >> sys.stdout, ("\nCreating test definition "
>> + "'{0}':".format(DEFAULT_TESTDEF_FILE))
>
> print already writes to stdout by default. is it really necessary to be that explicit?

I used the same behavior found in most of lava-tool (in particular in
lava_dashboard_tool) where it is using the explicit redirect.
Albeit, there is a mixed use of it even in the old code.

> I’m not sure about this approach of calling other commands. IMO the best
> approach would be to concentrate as much of the logic as possible in the
> model/backend classes, them call them from both commands.

Some refactoring will be needed, not sure if in this merge proposal though.

> IIUC you are assuming that you will be always storing the tarball in the job
> file?

True only if you go through the "lava init" step.
If you want to use the other commands, this does not happen. It might
need some refactoring since "testdef|script submit" is still missing
though.

> My initial thinking was to always generate a tarball dinamically, so that
> you don’t have to explicitly update a job file. It would go like this:
>
> $ lava job submit FOO.json
>
> submits FOO.json as ir

This works.

> $ lava testdef submit FOO.yaml
>
> creates the tarball and a job file dinamically; submit those
>
> $ lava script submit FOO.sh
>
> creates testdef, tarball and job file dynamically; submit th...

Revision history for this message
Antonio Terceiro (terceiro) wrote :
Download full text (3.6 KiB)

> > My initial thinking was to always generate a tarball dinamically, so that
> > you don’t have to explicitly update a job file. It would go like this:
> >
> > $ lava job submit FOO.json
> >
> > submits FOO.json as ir
>
> This works.
>
> > $ lava testdef submit FOO.yaml
> >
> > creates the tarball and a job file dinamically; submit those
> >
> > $ lava script submit FOO.sh
> >
> > creates testdef, tarball and job file dynamically; submit them
>
> These two are still missing.
>
> After showing this at Connect though, the common patterns were:
> - I will not use this (mostly from QA or people that knows how to write testdef)
> - There are too many commands (from a couple of Linaro employees and
> ARM ones too, hence the "lava init" stuff that will do everything
> almost automatically).

OK, that's good to know. I think we can reduce the number of commands,
but still keep the functionality - because craeting testdefs
automatically is *not* intended at people who already know how to create
them by themselves. :-)

> >> - user_input = raw_input(prompt).strip()
> >> + data = raw_input(prompt).strip()
> >> except EOFError:
> >> - pass
> >> + # Force to return None.
> >> + data = None
> >> except KeyboardInterrupt:
> >> sys.exit(-1)
> >> -
> >> - if user_input is not None:
> >> + return data
> >> +
> >> + @classmethod
> >> + def serialize(cls, value):
> >> + """Serializes the passed value to be friendly written to file.
> >> +
> >> + Lists are serialized as a comma separated string of values.
> >
> > I already saw something about serializing lists above. Can we keep all this
> > list serialization/deserialization just here?
>
> What do you mean? Having a single method to serialize and deserialize
> instead of the two we have right now?
> "serialize" and "deserialize" are in the same class (Parameter), the
> LIST_SERIALIZE_DELIMITER is a module constant.

Above there was a call to serialize, and maybe this should be handled
transparently inside this class. The factoring in serialize/deserialize
is good already.

> >> + :param value: The value to serialize.
> >> + :return The serialized value as string.
> >> + """
> >> + serialized = ""
> >> + if isinstance(value, list):
> >> + serialized = LIST_SERIALIZE_DELIMITER.join(
> >> + str(x) for x in value if x)
> >> + else:
> >> + serialized = str(value)
> >> + return serialized
> >
> > I wonder if we can we use a existing serialization instead of using a arbitrary
> > delimiter ... e.g. json + base64, this way we don’t have to limit ourselves to
> > values that do not contain a comma. But OTOH we want something that is readable
> > (and editable!) in the config file ...
>
> I wanted to use something that could have been human readable
> (avoiding also all hashing mechanism), and easily to modify.
> I considered also pickle, but then I would have preferred to pass
> everything through it rather than just a small set of values.
> We can probably use just json.dumps() and json.loads(), even if ...

Read more...

Revision history for this message
Milo Casagrande (milo) wrote :

On Wed, Jul 24, 2013 at 3:16 PM, Antonio Terceiro
<email address hidden> wrote:
> Agreed. We only have to consider whether we expect people to edit that
> file manually or not, because the serialization of the entire file in
> either JSON or YAML might mess up with the order in which a user would
> write the configuration.

I wouldn't be much concerned with how a user would write it. As long
as the file is valid JSON/YAML, we should be safe using it.
When it is written it is subject to changes anyway.

If we start considering it a cache, well, I do not even expect the
user to fiddle with it. :-)

> But maybe in the end what we are calling configuration is actually just
> a cache of pre-answered questions instead of an actual configuration
> file ...

It is actually, only two of those values I can call "configuration":
the server and the rpc_endpoint to talk to a remote server.
All the others, not really configuration values.

But still, even for those two, I would not expect to be asked each
time to insert or confirm them. A configuration for me is something
that should not be asked each time.

--
Milo Casagrande | Automation Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs

lp:~milo/lava-tool/lava-165 updated
400. By Milo Casagrande

Removed unused UrlSchemeParameter.

401. By Milo Casagrande

Fixed doc string and behaviour for the retrieve_file method.

402. By Milo Casagrande

Fixed doc string.

403. By Milo Casagrande

Fixed exception handling.

404. By Milo Casagrande

Use single quotes to avoid escaping.

405. By Milo Casagrande

Added lazy property for Config config_file, and fixed tests.

406. By Milo Casagrande

Fixed doc string in test method.

407. By Milo Casagrande

Removed constants.

408. By Milo Casagrande

Removed singleton, added getter and setter, fixed tests.

409. By Milo Casagrande

Fixed template helper functions.

410. By Milo Casagrande

Refactored testdef init, refactored utils functions, fixed tests.

411. By Milo Casagrande

Refactored methods and functions, fixed tests.

    * Refactored job templates.
    * Removed TarRepoParameter.

412. By Milo Casagrande

Fixed function call: missing parameter.

413. By Milo Casagrande

Removed job_ids save on file, removed constants.

414. By Milo Casagrande

Refacotred Job class.

    * Introduced a new cmd arg "--typ" to specify the kind of job type
      to create.
    * Fixed tests.

415. By Milo Casagrande

Completed utils, run and submit refactoring.

     * Fixed tests.

416. By Milo Casagrande

Removed constants.

417. By Milo Casagrande

Fixed help string.

418. By Milo Casagrande

Refactored the lava init command.

419. By Milo Casagrande

Fixed tests.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file '.coveragerc'
--- .coveragerc 2013-07-25 15:43:25 +0000
+++ .coveragerc 2013-07-25 15:43:26 +0000
@@ -3,6 +3,7 @@
3source = .3source = .
4omit =4omit =
5 setup*5 setup*
6 */tests/*
67
7[report]8[report]
8precision = 29precision = 2
910
=== modified file 'entry_points.ini'
--- entry_points.ini 2013-07-25 15:43:25 +0000
+++ entry_points.ini 2013-07-25 15:43:26 +0000
@@ -9,6 +9,11 @@
9dashboard = lava_dashboard_tool.commands:dashboard9dashboard = lava_dashboard_tool.commands:dashboard
10job = lava.job.commands:job10job = lava.job.commands:job
11device = lava.device.commands:device11device = lava.device.commands:device
12testdef = lava.testdef.commands:testdef
13init = lava.commands:init
14submit = lava.commands:submit
15run = lava.commands:run
16update = lava.commands:update
1217
13[lava_tool.commands]18[lava_tool.commands]
14help = lava.tool.commands.help:help19help = lava.tool.commands.help:help
@@ -78,3 +83,6 @@
78add = lava.device.commands:add83add = lava.device.commands:add
79remove = lava.device.commands:remove84remove = lava.device.commands:remove
80config = lava.device.commands:config85config = lava.device.commands:config
86
87[lava.testdef.commands]
88new = lava.testdef.commands:new
8189
=== added file 'lava/commands.py'
--- lava/commands.py 1970-01-01 00:00:00 +0000
+++ lava/commands.py 2013-07-25 15:43:26 +0000
@@ -0,0 +1,240 @@
1# Copyright (C) 2013 Linaro Limited
2#
3# Author: Milo Casagrande <milo.casagrande@linaro.org>
4#
5# This file is part of lava-tool.
6#
7# lava-tool is free software: you can redistribute it and/or modify
8# it under the terms of the GNU Lesser General Public License version 3
9# as published by the Free Software Foundation
10#
11# lava-tool is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
18
19"""
20Lava init commands.
21
22When invoking:
23
24 `lava init [DIR]`
25
26the command will create a default directory and files structure as follows:
27
28DIR/
29 |
30 +- JOB_FILE.json
31 +- tests/
32 |
33 + mytest.sh
34 + lavatest.yaml
35
36If DIR is not passed, it will use the current working directory.
37JOB_FILE is a file name that will be asked to the user, along with
38other necessary information to define the tests.
39
40If the user manually updates either the lavatest.yaml or mytest.sh file, it is
41necessary to run the following command in order to update the job definition:
42
43 `lava update [JOB|DIR]`
44"""
45
46import copy
47import json
48import os
49import stat
50import sys
51
52from lava.helper.command import BaseCommand
53from lava.helper.template import (
54 expand_template,
55 set_value
56)
57from lava.job import (
58 JOB_FILE_EXTENSIONS,
59)
60from lava.job.templates import (
61 LAVA_TEST_SHELL_TAR_REPO_KEY,
62)
63from lava.parameter import (
64 Parameter,
65)
66from lava.testdef.templates import (
67 DEFAULT_TESTDEF_FILE,
68 DEFAULT_TESTDEF_SCRIPT,
69 DEFAULT_TESTDEF_SCRIPT_CONTENT,
70)
71from lava.tool.errors import CommandError
72from lava_tool.utils import (
73 edit_file,
74 retrieve_file,
75 create_dir,
76 write_file,
77)
78
79# Default directory structure name.
80TESTS_DIR = "tests"
81
82# Internal parameter ids.
83JOBFILE_ID = "jobfile"
84
85JOBFILE_PARAMETER = Parameter(JOBFILE_ID)
86JOBFILE_PARAMETER.store = False
87
88INIT_TEMPLATE = {
89 JOBFILE_ID: JOBFILE_PARAMETER,
90}
91
92
93class init(BaseCommand):
94 """Set-ups the base directory structure."""
95
96 @classmethod
97 def register_arguments(cls, parser):
98 super(init, cls).register_arguments(parser)
99 parser.add_argument("DIR",
100 help=("The name of the directory to initialize. "
101 "Defaults to current working directory."),
102 nargs="?",
103 default=os.getcwd())
104
105 def invoke(self):
106 full_path = os.path.abspath(self.args.DIR)
107
108 if os.path.isfile(full_path):
109 raise CommandError("'{0}' already exists, and is a "
110 "file.".format(self.args.DIR))
111
112 create_dir(full_path)
113
114 data = self._update_data()
115
116 test_path = create_dir(full_path, TESTS_DIR)
117 # TODO
118 self._create_script(test_path)
119
120 testdef_file = self.create_test_definition(
121 os.path.join(test_path, DEFAULT_TESTDEF_FILE))
122
123 job = data[JOBFILE_ID]
124 self.create_tar_repo_job(
125 os.path.join(full_path, job), testdef_file, test_path)
126
127 def _update_data(self):
128 """Updates the template and ask values to the user.
129
130 The template in this case is a layout of the directory structure as it
131 would be written to disk.
132
133 :return A dictionary containing all the necessary file names to create.
134 """
135 data = copy.deepcopy(INIT_TEMPLATE)
136 expand_template(data, self.config)
137
138 return data
139
140 def _create_script(self, test_path):
141 # This is the default script file as defined in the testdef template.
142 default_script = os.path.join(test_path, DEFAULT_TESTDEF_SCRIPT)
143
144 if not os.path.isfile(default_script):
145 # We do not have the default testdef script. Create it, but
146 # remind the user to update it.
147 print >> sys.stdout, ("\nCreating default test script "
148 "'{0}'.".format(DEFAULT_TESTDEF_SCRIPT))
149
150 write_file(default_script, DEFAULT_TESTDEF_SCRIPT_CONTENT)
151
152 # Prompt the user to write the script file.
153 edit_file(default_script)
154
155 # Make sure the script is executable.
156 os.chmod(default_script,
157 stat.S_IRWXU | stat.S_IRWXG | stat.S_IROTH | stat.S_IXOTH)
158
159
160class run(BaseCommand):
161 """Runs a job on the local dispatcher."""
162
163 @classmethod
164 def register_arguments(cls, parser):
165 super(run, cls).register_arguments(parser)
166 parser.add_argument("JOB",
167 help=("The job file to run, or a directory "
168 "containing a job file. If nothing is "
169 "passed, it uses the current working "
170 "directory."),
171 nargs="?",
172 default=os.getcwd())
173
174 def invoke(self):
175 full_path = os.path.abspath(self.args.JOB)
176 job_file = retrieve_file(full_path, JOB_FILE_EXTENSIONS)
177
178 super(run, self).run(job_file)
179
180
181class submit(BaseCommand):
182 """Submits a job to LAVA."""
183
184 @classmethod
185 def register_arguments(cls, parser):
186 super(submit, cls).register_arguments(parser)
187 parser.add_argument("JOB",
188 help=("The job file to send, or a directory "
189 "containing a job file. If nothing is "
190 "passed, it uses the current working "
191 "directory."),
192 nargs="?",
193 default=os.getcwd())
194
195 def invoke(self):
196 full_path = os.path.abspath(self.args.JOB)
197 job_file = self.retrieve_file(full_path, JOB_FILE_EXTENSIONS)
198
199 super(submit, self).submit(job_file)
200
201
202class update(BaseCommand):
203 """Updates a job file with the correct data."""
204
205 @classmethod
206 def register_arguments(cls, parser):
207 super(update, cls).register_arguments(parser)
208 parser.add_argument("JOB",
209 help=("Automatically updates a job file "
210 "definition. If nothing is passed, it uses"
211 "the current working directory."),
212 nargs="?",
213 default=os.getcwd())
214
215 def invoke(self):
216 full_path = os.path.abspath(self.args.JOB)
217 job_file = self.retrieve_file(full_path, JOB_FILE_EXTENSIONS)
218 job_dir = os.path.dirname(job_file)
219 tests_dir = os.path.join(job_dir, TESTS_DIR)
220
221 if os.path.isdir(tests_dir):
222 # TODO
223 encoded_tests = None
224
225 json_data = None
226 with open(job_file, "r") as json_file:
227 try:
228 json_data = json.load(json_file)
229 set_value(
230 json_data, LAVA_TEST_SHELL_TAR_REPO_KEY, encoded_tests)
231 except Exception:
232 raise CommandError("Cannot read job file '{0}'.".format(
233 job_file))
234
235 content = json.dumps(json_data, indent=4)
236 write_file(job_file, content)
237
238 print >> sys.stdout, "Job definition updated."
239 else:
240 raise CommandError("Cannot find tests directory.")
0241
=== modified file 'lava/config.py'
--- lava/config.py 2013-07-25 15:43:25 +0000
+++ lava/config.py 2013-07-25 15:43:26 +0000
@@ -30,8 +30,10 @@
30 NoSectionError,30 NoSectionError,
31)31)
3232
33from lava.parameter import Parameter
33from lava.tool.errors import CommandError34from lava.tool.errors import CommandError
3435
36
35__all__ = ['Config', 'InteractiveConfig']37__all__ = ['Config', 'InteractiveConfig']
3638
37# Store for function calls to be made at exit time.39# Store for function calls to be made at exit time.
@@ -53,19 +55,35 @@
53 call()55 call()
54atexit.register(_run_at_exit)56atexit.register(_run_at_exit)
5557
56
57class Config(object):58class Config(object):
58 """A generic config object."""59 """A generic config object."""
60
59 def __init__(self):61 def __init__(self):
60 # The cache where to store parameters.62 # The cache where to store parameters.
61 self._cache = {}63 self._cache = {}
62 self._config_file = (os.environ.get('LAVACONFIG') or64 self._config_file = None
63 os.path.join(os.path.expanduser('~'),65 self._config_backend = None
64 '.lavaconfig'))
65 self._config_backend = ConfigParser()
66 self._config_backend.read([self._config_file])
67 AT_EXIT_CALLS.add(self.save)66 AT_EXIT_CALLS.add(self.save)
6867
68 @property
69 def config_file(self):
70 if self._config_file is None:
71 self._config_file = (os.environ.get('LAVACONFIG') or
72 os.path.join(os.path.expanduser('~'),
73 '.lavaconfig'))
74 return self._config_file
75
76 @config_file.setter
77 def config_file(self, value):
78 self._config_file = value
79
80 @property
81 def config_backend(self):
82 if self._config_backend is None:
83 self._config_backend = ConfigParser()
84 self._config_backend.read([self.config_file])
85 return self._config_backend
86
69 def _calculate_config_section(self, parameter):87 def _calculate_config_section(self, parameter):
70 """Calculates the config section of the specified parameter.88 """Calculates the config section of the specified parameter.
7189
@@ -92,7 +110,7 @@
92 if not section:110 if not section:
93 section = self._calculate_config_section(parameter)111 section = self._calculate_config_section(parameter)
94 # Try to get the parameter value first if it has one.112 # Try to get the parameter value first if it has one.
95 if parameter.value:113 if parameter.value is not None:
96 value = parameter.value114 value = parameter.value
97 else:115 else:
98 value = self._get_from_cache(parameter, section)116 value = self._get_from_cache(parameter, section)
@@ -109,7 +127,7 @@
109 """127 """
110 value = None128 value = None
111 try:129 try:
112 value = self._config_backend.get(section, parameter.id)130 value = self.config_backend.get(section, parameter.id)
113 except (NoOptionError, NoSectionError):131 except (NoOptionError, NoSectionError):
114 # Ignore, we return None.132 # Ignore, we return None.
115 pass133 pass
@@ -147,10 +165,17 @@
147 :param value: The value to add.165 :param value: The value to add.
148 :param section: The name of the section as in the config file.166 :param section: The name of the section as in the config file.
149 """167 """
150 if (not self._config_backend.has_section(section) and168 if (not self.config_backend.has_section(section) and
151 section != DEFAULT_SECTION):169 section != DEFAULT_SECTION):
152 self._config_backend.add_section(section)170 self.config_backend.add_section(section)
153 self._config_backend.set(section, key, value)171
172 # This is done to serialize a list when ConfigParser is written to
173 # file. Since there is no real support for list in ConfigParser, we
174 # serialized it in a common way that can get easily deserialized.
175 if isinstance(value, list):
176 value = Parameter.serialize(value)
177
178 self.config_backend.set(section, key, value)
154 # Store in the cache too.179 # Store in the cache too.
155 self._put_in_cache(key, value, section)180 self._put_in_cache(key, value, section)
156181
@@ -158,6 +183,7 @@
158 """Adds a Parameter to the config file and cache.183 """Adds a Parameter to the config file and cache.
159184
160 :param Parameter: The parameter to add.185 :param Parameter: The parameter to add.
186 :type Parameter
161 :param value: The value of the parameter. Defaults to None.187 :param value: The value of the parameter. Defaults to None.
162 :param section: The section where this parameter should be stored.188 :param section: The section where this parameter should be stored.
163 Defaults to None.189 Defaults to None.
@@ -174,8 +200,8 @@
174200
175 def save(self):201 def save(self):
176 """Saves the config to file."""202 """Saves the config to file."""
177 with open(self._config_file, "w") as write_file:203 with open(self.config_file, "w") as write_file:
178 self._config_backend.write(write_file)204 self.config_backend.write(write_file)
179205
180206
181class InteractiveConfig(Config):207class InteractiveConfig(Config):
@@ -188,6 +214,14 @@
188 super(InteractiveConfig, self).__init__()214 super(InteractiveConfig, self).__init__()
189 self._force_interactive = force_interactive215 self._force_interactive = force_interactive
190216
217 @property
218 def force_interactive(self):
219 return self._force_interactive
220
221 @force_interactive.setter
222 def force_interactive(self, value):
223 self._force_interactive = value
224
191 def get(self, parameter, section=None):225 def get(self, parameter, section=None):
192 """Overrides the parent one.226 """Overrides the parent one.
193227
@@ -198,10 +232,9 @@
198 section = self._calculate_config_section(parameter)232 section = self._calculate_config_section(parameter)
199 value = super(InteractiveConfig, self).get(parameter, section)233 value = super(InteractiveConfig, self).get(parameter, section)
200234
201 if not (value is not None and parameter.asked):235 if value is None or self.force_interactive:
202 if not value or self._force_interactive:236 value = parameter.prompt(old_value=value)
203 value = parameter.prompt(old_value=value)
204237
205 if value is not None:238 if value is not None and parameter.store:
206 self.put(parameter.id, value, section)239 self.put(parameter.id, value, section)
207 return value240 return value
208241
=== modified file 'lava/device/__init__.py'
--- lava/device/__init__.py 2013-07-25 15:43:25 +0000
+++ lava/device/__init__.py 2013-07-25 15:43:26 +0000
@@ -67,8 +67,8 @@
67 # given on the command line for the config file.67 # given on the command line for the config file.
68 if self.hostname is not None:68 if self.hostname is not None:
69 # We do not ask the user again this parameter.69 # We do not ask the user again this parameter.
70 self.data[HOSTNAME_PARAMETER.id].value = self.hostname
70 self.data[HOSTNAME_PARAMETER.id].asked = True71 self.data[HOSTNAME_PARAMETER.id].asked = True
71 config.put(HOSTNAME_PARAMETER.id, self.hostname)
7272
73 expand_template(self.data, config)73 expand_template(self.data, config)
7474
@@ -85,9 +85,9 @@
85 :param name: The name of the device we want matched to a real device.85 :param name: The name of the device we want matched to a real device.
86 :return A Device instance.86 :return A Device instance.
87 """87 """
88 instance = Device(DEFAULT_TEMPLATE, name)88 instance = Device(DEFAULT_TEMPLATE, hostname=name)
89 for known_dev, (matcher, dev_template) in KNOWN_DEVICES.iteritems():89 for known_dev, (matcher, dev_template) in KNOWN_DEVICES.iteritems():
90 if matcher.match(name):90 if matcher.match(name):
91 instance = Device(dev_template, name)91 instance = Device(dev_template, hostname=name)
92 break92 break
93 return instance93 return instance
9494
=== modified file 'lava/device/commands.py'
--- lava/device/commands.py 2013-07-25 15:43:25 +0000
+++ lava/device/commands.py 2013-07-25 15:43:26 +0000
@@ -23,18 +23,20 @@
23import os23import os
24import sys24import sys
2525
26from lava.device import get_known_device
26from lava.helper.command import (27from lava.helper.command import (
27 BaseCommand,28 BaseCommand,
28)29)
29
30from lava.helper.dispatcher import (30from lava.helper.dispatcher import (
31 get_device_file,31 get_device_file,
32 get_devices_path,32 get_devices_path,
33)33)
34
35from lava.device import get_known_device
36from lava.tool.command import CommandGroup34from lava.tool.command import CommandGroup
37from lava.tool.errors import CommandError35from lava.tool.errors import CommandError
36from lava_tool.utils import (
37 can_edit_file,
38 edit_file,
39)
3840
39DEVICE_FILE_SUFFIX = "conf"41DEVICE_FILE_SUFFIX = "conf"
4042
@@ -73,7 +75,7 @@
7375
74 print >> sys.stdout, ("Created device file '{0}' in: {1}".format(76 print >> sys.stdout, ("Created device file '{0}' in: {1}".format(
75 real_file_name, devices_path))77 real_file_name, devices_path))
76 self.edit_file(device_conf_file)78 edit_file(device_conf_file)
7779
7880
79class remove(BaseCommand):81class remove(BaseCommand):
@@ -114,7 +116,7 @@
114 real_file_name = ".".join([self.args.DEVICE, DEVICE_FILE_SUFFIX])116 real_file_name = ".".join([self.args.DEVICE, DEVICE_FILE_SUFFIX])
115 device_conf = get_device_file(real_file_name)117 device_conf = get_device_file(real_file_name)
116118
117 if device_conf and self.can_edit_file(device_conf):119 if device_conf and can_edit_file(device_conf):
118 self.edit_file(device_conf)120 edit_file(device_conf)
119 else:121 else:
120 raise CommandError("Cannot edit file '{0}'".format(real_file_name))122 raise CommandError("Cannot edit file '{0}'".format(real_file_name))
121123
=== modified file 'lava/device/tests/test_commands.py'
--- lava/device/tests/test_commands.py 2013-07-25 15:43:25 +0000
+++ lava/device/tests/test_commands.py 2013-07-25 15:43:26 +0000
@@ -50,35 +50,39 @@
50 name, args, kwargs = self.parser.method_calls[1]50 name, args, kwargs = self.parser.method_calls[1]
51 self.assertIn("DEVICE", args)51 self.assertIn("DEVICE", args)
5252
53 @patch("lava.device.Device.__str__", new=MagicMock(return_value=""))53 @patch("lava.device.commands.edit_file", create=True)
54 @patch("lava.device.Device.update", new=MagicMock())54 @patch("lava.device.Device.__str__")
55 @patch("lava.device.commands.get_device_file",55 @patch("lava.device.Device.update")
56 new=MagicMock(return_value=None))56 @patch("lava.device.commands.get_device_file")
57 @patch("lava.device.commands.get_devices_path")57 @patch("lava.device.commands.get_devices_path")
58 def test_add_invoke_0(self, get_devices_path_mock):58 def test_add_invoke_0(self, mocked_get_devices_path,
59 mocked_get_device_file, mocked_update, mocked_str,
60 mocked_edit_file):
59 # Tests invocation of the add command. Verifies that the conf file is61 # Tests invocation of the add command. Verifies that the conf file is
60 # written to disk.62 # written to disk.
61 get_devices_path_mock.return_value = self.temp_dir63 mocked_get_devices_path.return_value = self.temp_dir
64 mocked_get_device_file.return_value = None
65 mocked_str.return_value = ""
6266
63 add_command = add(self.parser, self.args)67 add_command = add(self.parser, self.args)
64 add_command.edit_file = MagicMock()
65 add_command.invoke()68 add_command.invoke()
6669
67 expected_path = os.path.join(self.temp_dir,70 expected_path = os.path.join(self.temp_dir,
68 ".".join([self.device, "conf"]))71 ".".join([self.device, "conf"]))
69 self.assertTrue(os.path.isfile(expected_path))72 self.assertTrue(os.path.isfile(expected_path))
7073
74 @patch("lava.device.commands.edit_file", create=True)
71 @patch("lava.device.commands.get_known_device")75 @patch("lava.device.commands.get_known_device")
72 @patch("lava.device.commands.get_devices_path")76 @patch("lava.device.commands.get_devices_path")
73 @patch("lava.device.commands.sys.exit")77 @patch("lava.device.commands.sys.exit")
74 @patch("lava.device.commands.get_device_file")78 @patch("lava.device.commands.get_device_file")
75 def test_add_invoke_1(self, mocked_get_device_file, mocked_sys_exit,79 def test_add_invoke_1(self, mocked_get_device_file, mocked_sys_exit,
76 mocked_get_devices_path, mocked_get_known_device):80 mocked_get_devices_path, mocked_get_known_device,
81 mocked_edit_file):
77 mocked_get_devices_path.return_value = self.temp_dir82 mocked_get_devices_path.return_value = self.temp_dir
78 mocked_get_device_file.return_value = self.temp_file.name83 mocked_get_device_file.return_value = self.temp_file.name
7984
80 add_command = add(self.parser, self.args)85 add_command = add(self.parser, self.args)
81 add_command.edit_file = MagicMock()
82 add_command.invoke()86 add_command.invoke()
8387
84 self.assertTrue(mocked_sys_exit.called)88 self.assertTrue(mocked_sys_exit.called)
@@ -97,11 +101,13 @@
97 name, args, kwargs = self.parser.method_calls[1]101 name, args, kwargs = self.parser.method_calls[1]
98 self.assertIn("DEVICE", args)102 self.assertIn("DEVICE", args)
99103
100 @patch("lava.device.Device.__str__", new=MagicMock(return_value=""))104 @patch("lava.device.commands.edit_file", create=True)
101 @patch("lava.device.Device.update", new=MagicMock())105 @patch("lava.device.Device.__str__", return_value="")
106 @patch("lava.device.Device.update")
102 @patch("lava.device.commands.get_device_file")107 @patch("lava.device.commands.get_device_file")
103 @patch("lava.device.commands.get_devices_path")108 @patch("lava.device.commands.get_devices_path")
104 def test_remove_invoke(self, get_devices_path_mock, get_device_file_mock):109 def test_remove_invoke(self, get_devices_path_mock, get_device_file_mock,
110 mocked_update, mocked_str, mocked_edit_file):
105 # Tests invocation of the remove command. Verifies that the conf file111 # Tests invocation of the remove command. Verifies that the conf file
106 # has been correctly removed.112 # has been correctly removed.
107 # First we add a new conf file, then we remove it.113 # First we add a new conf file, then we remove it.
@@ -109,7 +115,6 @@
109 get_devices_path_mock.return_value = self.temp_dir115 get_devices_path_mock.return_value = self.temp_dir
110116
111 add_command = add(self.parser, self.args)117 add_command = add(self.parser, self.args)
112 add_command.edit_file = MagicMock()
113 add_command.invoke()118 add_command.invoke()
114119
115 expected_path = os.path.join(self.temp_dir,120 expected_path = os.path.join(self.temp_dir,
@@ -145,18 +150,20 @@
145 name, args, kwargs = self.parser.method_calls[1]150 name, args, kwargs = self.parser.method_calls[1]
146 self.assertIn("DEVICE", args)151 self.assertIn("DEVICE", args)
147152
153 @patch("lava.device.commands.can_edit_file", create=True)
154 @patch("lava.device.commands.edit_file", create=True)
148 @patch("lava.device.commands.get_device_file")155 @patch("lava.device.commands.get_device_file")
149 def test_config_invoke_0(self, mocked_get_device_file):156 def test_config_invoke_0(self, mocked_get_device_file, mocked_edit_file,
157 mocked_can_edit_file):
150 command = config(self.parser, self.args)158 command = config(self.parser, self.args)
151159
160 mocked_can_edit_file.return_value = True
152 mocked_get_device_file.return_value = self.temp_file.name161 mocked_get_device_file.return_value = self.temp_file.name
153 command.can_edit_file = MagicMock(return_value=True)
154 command.edit_file = MagicMock()
155 command.invoke()162 command.invoke()
156163
157 self.assertTrue(command.edit_file.called)164 self.assertTrue(mocked_edit_file.called)
158 self.assertEqual([call(self.temp_file.name)],165 self.assertEqual([call(self.temp_file.name)],
159 command.edit_file.call_args_list)166 mocked_edit_file.call_args_list)
160167
161 @patch("lava.device.commands.get_device_file",168 @patch("lava.device.commands.get_device_file",
162 new=MagicMock(return_value=None))169 new=MagicMock(return_value=None))
163170
=== modified file 'lava/device/tests/test_device.py'
--- lava/device/tests/test_device.py 2013-07-25 15:43:25 +0000
+++ lava/device/tests/test_device.py 2013-07-25 15:43:26 +0000
@@ -20,19 +20,20 @@
20Device class unit tests.20Device class unit tests.
21"""21"""
2222
23from lava.parameter import Parameter23from mock import patch
24
25from lava.config import Config
26from lava.device import (
27 Device,
28 get_known_device,
29)
24from lava.device.templates import (30from lava.device.templates import (
25 HOSTNAME_PARAMETER,31 HOSTNAME_PARAMETER,
26 PANDA_DEVICE_TYPE,32 PANDA_DEVICE_TYPE,
27 PANDA_CONNECTION_COMMAND,33 PANDA_CONNECTION_COMMAND,
28)34)
29from lava.tests.test_config import MockedConfig
30from lava.device import (
31 Device,
32 get_known_device,
33)
34from lava.tool.errors import CommandError
35from lava.helper.tests.helper_test import HelperTest35from lava.helper.tests.helper_test import HelperTest
36from lava.parameter import Parameter
3637
3738
38class DeviceTest(HelperTest):39class DeviceTest(HelperTest):
@@ -64,12 +65,14 @@
64 self.assertIsInstance(instance.data['device_type'], Parameter)65 self.assertIsInstance(instance.data['device_type'], Parameter)
65 self.assertEqual(instance.data['device_type'].value, 'vexpress')66 self.assertEqual(instance.data['device_type'].value, 'vexpress')
6667
67 def test_device_update_1(self):68 @patch("lava.config.Config.save")
69 def test_device_update_1(self, patched_save):
68 # Tests that when calling update() on a Device, the template gets70 # Tests that when calling update() on a Device, the template gets
69 # updated with the correct values from a Config instance.71 # updated with the correct values from a Config instance.
70 hostname = "panda_device"72 hostname = "panda_device"
7173
72 config = MockedConfig(self.temp_file.name)74 config = Config()
75 config._config_file = self.temp_file.name
73 config.put_parameter(HOSTNAME_PARAMETER, hostname)76 config.put_parameter(HOSTNAME_PARAMETER, hostname)
74 config.put_parameter(PANDA_DEVICE_TYPE, "panda")77 config.put_parameter(PANDA_DEVICE_TYPE, "panda")
75 config.put_parameter(PANDA_CONNECTION_COMMAND, "test")78 config.put_parameter(PANDA_CONNECTION_COMMAND, "test")
@@ -85,12 +88,14 @@
8588
86 self.assertEqual(expected, instance.data)89 self.assertEqual(expected, instance.data)
8790
88 def test_device_write(self):91 @patch("lava.config.Config.save")
92 def test_device_write(self, mocked_save):
89 # User tries to create a new panda device. The conf file is written93 # User tries to create a new panda device. The conf file is written
90 # and contains the expected results.94 # and contains the expected results.
91 hostname = "panda_device"95 hostname = "panda_device"
9296
93 config = MockedConfig(self.temp_file.name)97 config = Config()
98 config._config_file = self.temp_file.name
94 config.put_parameter(HOSTNAME_PARAMETER, hostname)99 config.put_parameter(HOSTNAME_PARAMETER, hostname)
95 config.put_parameter(PANDA_DEVICE_TYPE, "panda")100 config.put_parameter(PANDA_DEVICE_TYPE, "panda")
96 config.put_parameter(PANDA_CONNECTION_COMMAND, "test")101 config.put_parameter(PANDA_CONNECTION_COMMAND, "test")
97102
=== modified file 'lava/helper/command.py'
--- lava/helper/command.py 2013-07-25 15:43:25 +0000
+++ lava/helper/command.py 2013-07-25 15:43:26 +0000
@@ -19,22 +19,49 @@
19"""Base command class common to lava commands series."""19"""Base command class common to lava commands series."""
2020
21import os21import os
22import subprocess
23import sys22import sys
2423import xmlrpclib
2524
26from lava.config import InteractiveConfig25from lava.config import InteractiveConfig
26from lava.helper.dispatcher import get_devices
27from lava.parameter import (
28 Parameter,
29 SingleChoiceParameter,
30)
27from lava.tool.command import Command31from lava.tool.command import Command
28from lava.tool.errors import CommandError32from lava.tool.errors import CommandError
29from lava_tool.utils import has_command33from lava_tool.authtoken import (
34 AuthenticatingServerProxy,
35 KeyringAuthBackend
36)
37from lava_tool.utils import (
38 has_command,
39 verify_and_create_url,
40 create_tar,
41 base64_encode,
42)
43from lava.job import Job
44from lava.job.templates import (
45 LAVA_TEST_SHELL_TAR_REPO,
46 LAVA_TEST_SHELL_TAR_REPO_KEY,
47 LAVA_TEST_SHELL_TESDEF_KEY,
48)
49
50from lava.testdef import TestDefinition
51from lava.testdef.templates import (
52 TESTDEF_TEMPLATE,
53)
54CONFIG = InteractiveConfig()
3055
3156
32class BaseCommand(Command):57class BaseCommand(Command):
58
33 """Base command class for all lava commands."""59 """Base command class for all lava commands."""
60
34 def __init__(self, parser, args):61 def __init__(self, parser, args):
35 super(BaseCommand, self).__init__(parser, args)62 super(BaseCommand, self).__init__(parser, args)
36 self.config = InteractiveConfig(63 self.config = CONFIG
37 force_interactive=self.args.non_interactive)64 self.config.force_interactive = self.args.non_interactive
3865
39 @classmethod66 @classmethod
40 def register_arguments(cls, parser):67 def register_arguments(cls, parser):
@@ -43,59 +70,109 @@
43 action='store_false',70 action='store_false',
44 help=("Do not ask for input parameters."))71 help=("Do not ask for input parameters."))
4572
46 @classmethod73 def authenticated_server(self):
47 def can_edit_file(cls, conf_file):74 """Returns a connection to a LAVA server.
48 """Checks if a file can be opend in write mode.75
4976 It will ask the user the necessary parameters to establish the
50 :param conf_file: The path to the file.77 connection.
51 :return True if it is possible to write on the file, False otherwise.78 """
52 """79 server_name_parameter = Parameter("server")
53 can_edit = True80 rpc_endpoint_parameter = Parameter("rpc_endpoint",
54 try:81 depends=server_name_parameter)
55 fp = open(conf_file, "a")82
56 fp.close()83 server_url = self.config.get(server_name_parameter)
57 except IOError:84 endpoint = self.config.get(rpc_endpoint_parameter)
58 can_edit = False85
59 return can_edit86 rpc_url = verify_and_create_url(server_url, endpoint)
6087 server = AuthenticatingServerProxy(rpc_url,
61 @classmethod88 auth_backend=KeyringAuthBackend())
62 def edit_file(cls, config_file):89 return server
63 """Opens the specified file with the default file editor.90
6491 def submit(self, job_file):
65 :param config_file: The file to edit.92 """Submits a job file to a LAVA server.
66 """93
67 editor = os.environ.get("EDITOR", None)94 :param job_file: The job file to submit.
68 if editor is None:95 :return The job ID on success.
69 if has_command("sensible-editor"):96 """
70 editor = "sensible-editor"97 if os.path.isfile(job_file):
71 elif has_command("xdg-open"):98 try:
72 editor = "xdg-open"99 jobdata = open(job_file, 'rb').read()
100 server = self.authenticated_server()
101
102 job_id = server.scheduler.submit_job(jobdata)
103 print >> sys.stdout, ("Job submitted with job "
104 "ID {0}.".format(job_id))
105
106 return job_id
107 except xmlrpclib.Fault, exc:
108 raise CommandError(str(exc))
109 else:
110 raise CommandError("Job file '{0}' does not exists, or is not "
111 "a file.".format(job_file))
112
113 def run(self, job_file):
114 """Runs a job file on the local LAVA dispatcher.
115
116 :param job_file: The job file to run.
117 """
118 if os.path.isfile(job_file):
119 if has_command("lava-dispatch"):
120 devices = get_devices()
121 if devices:
122 if len(devices) > 1:
123 device_names = [device.hostname for device in devices]
124 device_param = SingleChoiceParameter("device",
125 device_names)
126 device = device_param.prompt("Device to use: ")
127 else:
128 device = devices[0].hostname
129 self.execute(
130 ["lava-dispatch", "--target", device, job_file])
73 else:131 else:
74 # We really do not know how to open a file.132 raise CommandError("Cannot find lava-dispatcher installation.")
75 print >> sys.stdout, ("Cannot find an editor to open the "133 else:
76 "file '{0}'.".format(config_file))134 raise CommandError("Job file '{0}' does not exists, or it is not "
77 print >> sys.stdout, ("Either set the 'EDITOR' environment "135 "a file.".format(job_file))
78 "variable, or install 'sensible-editor' "136
79 "or 'xdg-open'.")137 def create_tar_repo_job(self, job_file, testdef_file, tar_content):
80 sys.exit(-1)138 """Creates a job file based on the tar-repo template.
81 try:139
82 subprocess.Popen([editor, config_file]).wait()140 The tar repo is not kept on the file system.
83 except Exception:141
84 raise CommandError("Error opening the file '{0}' with the "142 :param job_file: The path of the job file to create.
85 "following editor: {1}.".format(config_file,143 :param testdef_file: The path of the test definition file.
86 editor))144 :param tar_content: What should go into the tarball repository.
87145 :return The path of the job file created.
88 @classmethod146 """
89 def run(cls, cmd_args):147 try:
90 """Runs the supplied command args.148 tar_repo = create_tar(tar_content)
91149
92 :param cmd_args: The command, and its optional arguments, to run.150 job_instance = Job(LAVA_TEST_SHELL_TAR_REPO, job_file)
93 :return The command execution return code.151 job_instance.update(self.config)
94 """152
95 if not isinstance(cmd_args, list):153 job_instance.set(LAVA_TEST_SHELL_TAR_REPO_KEY,
96 cmd_args = [cmd_args]154 base64_encode(tar_repo))
97 try:155 job_instance.set(LAVA_TEST_SHELL_TESDEF_KEY,
98 return subprocess.check_call(cmd_args)156 os.path.basename(testdef_file))
99 except subprocess.CalledProcessError:157
100 raise CommandError("Error running the following command: "158 job_instance.write()
101 "{0}".format(" ".join(cmd_args)))159
160 return job_instance.file_name
161 finally:
162 if os.path.isfile(tar_repo):
163 os.unlink(tar_repo)
164
165 def create_test_definition(self, testdef_file, template=TESTDEF_TEMPLATE):
166 """Creates a test definition YAML file.
167
168 :param testdef_file: The file to create.
169 :return The path of the file created.
170 """
171 testdef = TestDefinition(template, testdef_file)
172 testdef.update(self.config)
173 testdef.write()
174
175 print >> sys.stdout, ("Create test definition "
176 "'{0}'.".format(testdef.file_name))
177
178 return testdef.file_name
102179
=== modified file 'lava/helper/template.py'
--- lava/helper/template.py 2013-07-25 15:43:25 +0000
+++ lava/helper/template.py 2013-07-25 15:43:26 +0000
@@ -16,6 +16,8 @@
16# You should have received a copy of the GNU Lesser General Public License16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
1818
19"""Helper functions for a template."""
20
19from lava.parameter import Parameter21from lava.parameter import Parameter
2022
2123
@@ -42,3 +44,81 @@
42 update(entry)44 update(entry)
4345
44 update(template)46 update(template)
47
48
49def get_key(data, search_key):
50 """Goes through a template looking for a key.
51
52 :param data: The template to traverse.
53 :param search_key: The key to look for.
54 :return The key value.
55 """
56 return_value = None
57 found = False
58
59 if isinstance(data, dict):
60 bucket = []
61
62 for key, value in data.iteritems():
63 if key == search_key:
64 return_value = value
65 found = True
66 break
67 else:
68 bucket.append(value)
69
70 if bucket and not found:
71 for element in bucket:
72 if isinstance(element, list):
73 for element in element:
74 bucket.append(element)
75 elif isinstance(element, dict):
76 for key, value in element.iteritems():
77 if key == search_key:
78 return_value = value
79 found = True
80 break
81 else:
82 bucket.append(value)
83 if found:
84 break
85
86 return return_value
87
88
89def set_value(data, search_key, new_value):
90 """Sets a new value for a template key.
91
92 :param data: The data structure to update.
93 :type dict
94 :param search_key: The key to search and update.
95 :param new_value: The new value to set.
96 """
97 is_set = False
98
99 if isinstance(data, dict):
100 bucket = []
101
102 for key, value in data.iteritems():
103 if key == search_key:
104 data[key] = new_value
105 is_set = True
106 break
107 else:
108 bucket.append(value)
109
110 if bucket and not is_set:
111 for element in bucket:
112 if isinstance(element, list):
113 for element in element:
114 bucket.append(element)
115 elif isinstance(element, dict):
116 for key, value in element.iteritems():
117 if key == search_key:
118 element[key] = new_value
119 is_set = True
120 break
121 else:
122 bucket.append(value)
123 if is_set:
124 break
45125
=== modified file 'lava/helper/tests/helper_test.py'
--- lava/helper/tests/helper_test.py 2013-07-25 15:43:25 +0000
+++ lava/helper/tests/helper_test.py 2013-07-25 15:43:26 +0000
@@ -29,12 +29,20 @@
29import tempfile29import tempfile
3030
31from unittest import TestCase31from unittest import TestCase
32from mock import MagicMock32from mock import (
33 MagicMock,
34 patch
35)
3336
3437
35class HelperTest(TestCase):38class HelperTest(TestCase):
36 """Helper test class that all tests under the lava package can inherit."""39 """Helper test class that all tests under the lava package can inherit."""
40
37 def setUp(self):41 def setUp(self):
42 # Need to patch it here, not as a decorator, or running the tests
43 # via `./setup.py test` will fail.
44 self.at_exit_patcher = patch("lava.config.AT_EXIT_CALLS", spec=set)
45 self.at_exit_patcher.start()
38 self.original_stdout = sys.stdout46 self.original_stdout = sys.stdout
39 sys.stdout = open("/dev/null", "w")47 sys.stdout = open("/dev/null", "w")
40 self.original_stderr = sys.stderr48 self.original_stderr = sys.stderr
@@ -50,12 +58,24 @@
50 self.args.interactive = MagicMock(return_value=False)58 self.args.interactive = MagicMock(return_value=False)
51 self.args.DEVICE = self.device59 self.args.DEVICE = self.device
5260
53 self.config = MagicMock()
54 self.config.get = MagicMock(return_value=self.temp_dir)
55
56 def tearDown(self):61 def tearDown(self):
62 self.at_exit_patcher.stop()
57 sys.stdin = self.original_stdin63 sys.stdin = self.original_stdin
58 sys.stdout = self.original_stdout64 sys.stdout = self.original_stdout
59 sys.stderr = self.original_stderr65 sys.stderr = self.original_stderr
60 shutil.rmtree(self.temp_dir)66 shutil.rmtree(self.temp_dir)
61 os.unlink(self.temp_file.name)67 os.unlink(self.temp_file.name)
68
69 def tmp(self, name):
70 """
71 Returns the full path to a file, or directory, called `name` in a
72 temporary directory.
73
74 This method does not create the file, it only gives a full filename
75 where you can actually write some data. The file will not be removed
76 by this method.
77
78 :param name: The name the file/directory should have.
79 :return A path.
80 """
81 return os.path.join(tempfile.gettempdir(), name)
6282
=== modified file 'lava/helper/tests/test_command.py'
--- lava/helper/tests/test_command.py 2013-07-25 15:43:25 +0000
+++ lava/helper/tests/test_command.py 2013-07-25 15:43:26 +0000
@@ -18,14 +18,7 @@
1818
19"""lava.herlp.command module tests."""19"""lava.herlp.command module tests."""
2020
21import subprocess
22from mock import (
23 MagicMock,
24 call,
25 patch,
26)
2721
28from lava.tool.errors import CommandError
29from lava.helper.command import BaseCommand22from lava.helper.command import BaseCommand
30from lava.helper.tests.helper_test import HelperTest23from lava.helper.tests.helper_test import HelperTest
3124
@@ -39,85 +32,3 @@
39 command.register_arguments(self.parser)32 command.register_arguments(self.parser)
40 name, args, kwargs = self.parser.method_calls[0]33 name, args, kwargs = self.parser.method_calls[0]
41 self.assertIn("--non-interactive", args)34 self.assertIn("--non-interactive", args)
42
43 def test_can_edit_file(self):
44 # Tests the can_edit_file method of the config command.
45 # This is to make sure the device config file is not erased when
46 # checking if it is possible to open it.
47 expected = ("hostname = a_fake_panda02\nconnection_command = \n"
48 "device_type = panda\n")
49
50 command = BaseCommand(self.parser, self.args)
51 conf_file = self.temp_file
52
53 with open(conf_file.name, "w") as f:
54 f.write(expected)
55
56 self.assertTrue(command.can_edit_file(conf_file.name))
57 obtained = ""
58 with open(conf_file.name) as f:
59 obtained = f.read()
60
61 self.assertEqual(expected, obtained)
62
63 @patch("lava.helper.command.subprocess")
64 def test_run_0(self, mocked_subprocess):
65 mocked_subprocess.check_call = MagicMock()
66 BaseCommand.run("foo")
67 self.assertEqual(mocked_subprocess.check_call.call_args_list,
68 [call(["foo"])])
69 self.assertTrue(mocked_subprocess.check_call.called)
70
71 @patch("lava.helper.command.subprocess.check_call")
72 def test_run_1(self, mocked_check_call):
73 mocked_check_call.side_effect = subprocess.CalledProcessError(1, "foo")
74 self.assertRaises(CommandError, BaseCommand.run, ["foo"])
75
76 @patch("lava.helper.command.subprocess")
77 @patch("lava.helper.command.has_command", return_value=False)
78 @patch("lava.helper.command.os.environ.get", return_value=None)
79 @patch("lava.helper.command.sys.exit")
80 def test_edit_file_0(self, mocked_sys_exit, mocked_env_get,
81 mocked_has_command, mocked_subprocess):
82 BaseCommand.edit_file(self.temp_file.name)
83 self.assertTrue(mocked_sys_exit.called)
84
85 @patch("lava.helper.command.subprocess")
86 @patch("lava.helper.command.has_command", side_effect=[True, False])
87 @patch("lava.helper.command.os.environ.get", return_value=None)
88 def test_edit_file_1(self, mocked_env_get, mocked_has_command,
89 mocked_subprocess):
90 mocked_subprocess.Popen = MagicMock()
91 BaseCommand.edit_file(self.temp_file.name)
92 expected = [call(["sensible-editor", self.temp_file.name])]
93 self.assertEqual(expected, mocked_subprocess.Popen.call_args_list)
94
95 @patch("lava.helper.command.subprocess")
96 @patch("lava.helper.command.has_command", side_effect=[False, True])
97 @patch("lava.helper.command.os.environ.get", return_value=None)
98 def test_edit_file_2(self, mocked_env_get, mocked_has_command,
99 mocked_subprocess):
100 mocked_subprocess.Popen = MagicMock()
101 BaseCommand.edit_file(self.temp_file.name)
102 expected = [call(["xdg-open", self.temp_file.name])]
103 self.assertEqual(expected, mocked_subprocess.Popen.call_args_list)
104
105 @patch("lava.helper.command.subprocess")
106 @patch("lava.helper.command.has_command", return_value=False)
107 @patch("lava.helper.command.os.environ.get", return_value="vim")
108 def test_edit_file_3(self, mocked_env_get, mocked_has_command,
109 mocked_subprocess):
110 mocked_subprocess.Popen = MagicMock()
111 BaseCommand.edit_file(self.temp_file.name)
112 expected = [call(["vim", self.temp_file.name])]
113 self.assertEqual(expected, mocked_subprocess.Popen.call_args_list)
114
115 @patch("lava.helper.command.subprocess")
116 @patch("lava.helper.command.has_command", return_value=False)
117 @patch("lava.helper.command.os.environ.get", return_value="vim")
118 def test_edit_file_4(self, mocked_env_get, mocked_has_command,
119 mocked_subprocess):
120 mocked_subprocess.Popen = MagicMock()
121 mocked_subprocess.Popen.side_effect = Exception()
122 self.assertRaises(CommandError, BaseCommand.edit_file,
123 self.temp_file.name)
12435
=== added file 'lava/helper/tests/test_template.py'
--- lava/helper/tests/test_template.py 1970-01-01 00:00:00 +0000
+++ lava/helper/tests/test_template.py 2013-07-25 15:43:26 +0000
@@ -0,0 +1,102 @@
1# Copyright (C) 2013 Linaro Limited
2#
3# Author: Milo Casagrande <milo.casagrande@linaro.org>
4#
5# This file is part of lava-tool.
6#
7# lava-tool is free software: you can redistribute it and/or modify
8# it under the terms of the GNU Lesser General Public License version 3
9# as published by the Free Software Foundation
10#
11# lava-tool is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
18
19""" """
20
21import copy
22from unittest import TestCase
23
24from lava.helper.template import (
25 get_key,
26 set_value
27)
28
29
30TEST_TEMPLATE = {
31 "key1": "value1",
32 "key2": [
33 "value2", "value3"
34 ],
35 "key3": [
36 {
37 "key4": "value4",
38 "key5": "value5"
39 },
40 {
41 "key6": "value6",
42 "key7": "value7"
43 },
44 [
45 {
46 "key8": "value8"
47 }
48 ]
49 ],
50 "key10": {
51 "key11": "value11"
52 }
53}
54
55
56class TestParameter(TestCase):
57
58 def test_get_key_simple_key(self):
59 expected = "value1"
60 obtained = get_key(TEST_TEMPLATE, "key1")
61 self.assertEquals(expected, obtained)
62
63 def test_get_key_nested_key(self):
64 expected = "value4"
65 obtained = get_key(TEST_TEMPLATE, "key4")
66 self.assertEquals(expected, obtained)
67
68 def test_get_key_nested_key_1(self):
69 expected = "value7"
70 obtained = get_key(TEST_TEMPLATE, "key7")
71 self.assertEquals(expected, obtained)
72
73 def test_get_key_nested_key_2(self):
74 expected = "value8"
75 obtained = get_key(TEST_TEMPLATE, "key8")
76 self.assertEquals(expected, obtained)
77
78 def test_get_key_nested_key_3(self):
79 expected = "value11"
80 obtained = get_key(TEST_TEMPLATE, "key11")
81 self.assertEquals(expected, obtained)
82
83 def test_set_value_0(self):
84 data = copy.deepcopy(TEST_TEMPLATE)
85 expected = "foo"
86 set_value(data, "key1", expected)
87 obtained = get_key(data, "key1")
88 self.assertEquals(expected, obtained)
89
90 def test_set_value_1(self):
91 data = copy.deepcopy(TEST_TEMPLATE)
92 expected = "foo"
93 set_value(data, "key6", expected)
94 obtained = get_key(data, "key6")
95 self.assertEquals(expected, obtained)
96
97 def test_set_value_2(self):
98 data = copy.deepcopy(TEST_TEMPLATE)
99 expected = "foo"
100 set_value(data, "key11", expected)
101 obtained = get_key(data, "key11")
102 self.assertEquals(expected, obtained)
0103
=== modified file 'lava/job/__init__.py'
--- lava/job/__init__.py 2013-07-25 15:43:25 +0000
+++ lava/job/__init__.py 2013-07-25 15:43:26 +0000
@@ -16,18 +16,46 @@
16# You should have received a copy of the GNU Lesser General Public License16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
1818
19import json
20
19from copy import deepcopy21from copy import deepcopy
20import json22
2123from lava.helper.template import (
22from lava.helper.template import expand_template24 expand_template,
2325 set_value,
2426)
25class Job:27from lava_tool.utils import (
26 def __init__(self, template):28 verify_file_extension,
27 self.data = deepcopy(template)29 verify_path_existance,
2830 write_file
29 def fill_in(self, config):31)
32
33# Default job file extension.
34DEFAULT_JOB_EXTENSION = "json"
35# Possible extension for a job file.
36JOB_FILE_EXTENSIONS = [DEFAULT_JOB_EXTENSION]
37
38
39class Job(object):
40 def __init__(self, data, file_name):
41 self.file_name = verify_file_extension(file_name,
42 DEFAULT_JOB_EXTENSION,
43 JOB_FILE_EXTENSIONS)
44 verify_path_existance(self.file_name)
45 self.data = deepcopy(data)
46
47 def set(self, key, value):
48 """Set key to the specified value.
49
50 :param key: The key to look in the object data.
51 :param value: The value to set.
52 """
53 set_value(self.data, key, value)
54
55 def update(self, config):
56 """Updates the Job object based on the provided config."""
30 expand_template(self.data, config)57 expand_template(self.data, config)
3158
32 def write(self, stream):59 def write(self):
33 stream.write(json.dumps(self.data, indent=4))60 """Writes the Job object to file."""
61 write_file(self.file_name, json.dumps(self.data, indent=4))
3462
=== modified file 'lava/job/commands.py'
--- lava/job/commands.py 2013-07-25 15:43:25 +0000
+++ lava/job/commands.py 2013-07-25 15:43:26 +0000
@@ -21,21 +21,14 @@
21"""21"""
2222
23import os23import os
24import sys
25import xmlrpclib
2624
27from lava.helper.command import BaseCommand25from lava.helper.command import BaseCommand
28from lava.helper.dispatcher import get_devices
29
30from lava.job import Job26from lava.job import Job
31from lava.job.templates import (27from lava.job.templates import (
32 BOOT_TEST,28 BOOT_TEST_KEY,
29 JOB_TYPES,
33)30)
34from lava.parameter import Parameter
35from lava.tool.command import CommandGroup31from lava.tool.command import CommandGroup
36from lava.tool.errors import CommandError
37from lava_tool.authtoken import AuthenticatingServerProxy, KeyringAuthBackend
38from lava_tool.utils import has_command
3932
4033
41class job(CommandGroup):34class job(CommandGroup):
@@ -50,18 +43,25 @@
50 def register_arguments(cls, parser):43 def register_arguments(cls, parser):
51 super(new, cls).register_arguments(parser)44 super(new, cls).register_arguments(parser)
52 parser.add_argument("FILE", help=("Job file to be created."))45 parser.add_argument("FILE", help=("Job file to be created."))
5346 parser.add_argument("--type",
54 def invoke(self):47 help=("The type of job to create. Defaults to "
55 if os.path.exists(self.args.FILE):48 "'{0}'.".format(BOOT_TEST_KEY)),
56 raise CommandError('{0} already exists.'.format(self.args.FILE))49 choices=JOB_TYPES.keys(),
5750 default=BOOT_TEST_KEY)
58 with open(self.args.FILE, 'w') as job_file:51
59 job_instance = Job(BOOT_TEST)52 def invoke(self, job_template=None):
60 job_instance.fill_in(self.config)53 if not job_template:
61 job_instance.write(job_file)54 job_template = JOB_TYPES.get(self.args.type)
55
56 full_path = os.path.abspath(self.args.FILE)
57
58 job_instance = Job(job_template, full_path)
59 job_instance.update(self.config)
60 job_instance.write()
6261
6362
64class submit(BaseCommand):63class submit(BaseCommand):
64
65 """Submits the specified job file."""65 """Submits the specified job file."""
6666
67 @classmethod67 @classmethod
@@ -70,24 +70,11 @@
70 parser.add_argument("FILE", help=("The job file to submit."))70 parser.add_argument("FILE", help=("The job file to submit."))
7171
72 def invoke(self):72 def invoke(self):
73 jobfile = self.args.FILE73 super(submit, self).submit(self.args.FILE)
74 jobdata = open(jobfile, 'rb').read()
75
76 server_name = Parameter('server')
77 rpc_endpoint = Parameter('rpc_endpoint', depends=server_name)
78 self.config.get(server_name)
79 endpoint = self.config.get(rpc_endpoint)
80
81 server = AuthenticatingServerProxy(endpoint,
82 auth_backend=KeyringAuthBackend())
83 try:
84 job_id = server.scheduler.submit_job(jobdata)
85 print >> sys.stdout, "Job submitted with job ID {0}".format(job_id)
86 except xmlrpclib.Fault, exc:
87 raise CommandError(str(exc))
8874
8975
90class run(BaseCommand):76class run(BaseCommand):
77
91 """Runs the specified job file on the local dispatcher."""78 """Runs the specified job file on the local dispatcher."""
9279
93 @classmethod80 @classmethod
@@ -95,48 +82,5 @@
95 super(run, cls).register_arguments(parser)82 super(run, cls).register_arguments(parser)
96 parser.add_argument("FILE", help=("The job file to submit."))83 parser.add_argument("FILE", help=("The job file to submit."))
9784
98 @classmethod
99 def _choose_device(cls, devices):
100 """Let the user choose the device to use.
101
102 :param devices: The list of available devices.
103 :return The selected device.
104 """
105 devices_len = len(devices)
106 output_list = []
107 for device, number in zip(devices, range(1, devices_len + 1)):
108 output_list.append("\t{0}. {1}\n".format(number, device.hostname))
109
110 print >> sys.stdout, ("More than one local device found. "
111 "Please choose one:\n")
112 print >> sys.stdout, "".join(output_list)
113
114 while True:
115 try:
116 user_input = raw_input("Device number to use: ").strip()
117
118 if user_input in [str(x) for x in range(1, devices_len + 1)]:
119 return devices[int(user_input) - 1].hostname
120 else:
121 continue
122 except EOFError:
123 user_input = None
124 except KeyboardInterrupt:
125 sys.exit(-1)
126
127 def invoke(self):85 def invoke(self):
128 if os.path.isfile(self.args.FILE):86 super(run, self).run(self.args.FILE)
129 if has_command("lava-dispatch"):
130 devices = get_devices()
131 if devices:
132 if len(devices) > 1:
133 device = self._choose_device(devices)
134 else:
135 device = devices[0].hostname
136 self.run(["lava-dispatch", "--target", device,
137 self.args.FILE])
138 else:
139 raise CommandError("Cannot find lava-dispatcher installation.")
140 else:
141 raise CommandError("The file '{0}' does not exists. or is not "
142 "a file.".format(self.args.FILE))
14387
=== modified file 'lava/job/templates.py'
--- lava/job/templates.py 2013-07-25 15:43:25 +0000
+++ lava/job/templates.py 2013-07-25 15:43:26 +0000
@@ -16,19 +16,33 @@
16# You should have received a copy of the GNU Lesser General Public License16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
1818
19from lava.parameter import Parameter19from lava.parameter import (
2020 ListParameter,
21device_type = Parameter("device_type")21 Parameter,
22prebuilt_image = Parameter("prebuilt_image", depends=device_type)22)
23
24LAVA_TEST_SHELL_TAR_REPO_KEY = "tar-repo"
25LAVA_TEST_SHELL_TESDEF_KEY = "testdef"
26
27DEVICE_TYPE_PARAMETER = Parameter("device_type")
28PREBUILT_IMAGE_PARAMETER = Parameter("image", depends=DEVICE_TYPE_PARAMETER)
29
30TESTDEF_URLS_PARAMETER = ListParameter("testdef_urls")
31TESTDEF_URLS_PARAMETER.store = False
32
33# Use another ID for the server parameter, might be different.
34SERVER_PARAMETER = Parameter("stream_server")
35STREAM_PARAMETER = Parameter("stream")
2336
24BOOT_TEST = {37BOOT_TEST = {
38 "timeout": 18000,
25 "job_name": "Boot test",39 "job_name": "Boot test",
26 "device_type": device_type,40 "device_type": DEVICE_TYPE_PARAMETER,
27 "actions": [41 "actions": [
28 {42 {
29 "command": "deploy_linaro_image",43 "command": "deploy_linaro_image",
30 "parameters": {44 "parameters": {
31 "image": prebuilt_image45 "image": PREBUILT_IMAGE_PARAMETER
32 }46 }
33 },47 },
34 {48 {
@@ -39,21 +53,72 @@
3953
40LAVA_TEST_SHELL = {54LAVA_TEST_SHELL = {
41 "job_name": "LAVA Test Shell",55 "job_name": "LAVA Test Shell",
42 "device_type": device_type,56 "timeout": 18000,
43 "actions": [57 "device_type": DEVICE_TYPE_PARAMETER,
44 {58 "actions": [
45 "command": "deploy_linaro_image",59 {
46 "parameters": {60 "command": "deploy_linaro_image",
47 "image": prebuilt_image,61 "parameters": {
48 }62 "image": PREBUILT_IMAGE_PARAMETER,
49 },63 }
50 {64 },
51 "command": "lava_test_shell",65 {
52 "parameters": {66 "command": "lava_test_shell",
53 "testdef_urls": [67 "parameters": {
54 Parameter("testdef_url")68 "timeout": 1800,
69 "testdef_urls": TESTDEF_URLS_PARAMETER,
70 }
71 },
72 {
73 "command": "submit_results",
74 "parameters" : {
75 "stream": STREAM_PARAMETER,
76 "server": SERVER_PARAMETER
77 }
78 }
79 ]
80}
81
82# This is a special case template, only use when automatically create job files
83# starting from a testdef or a script. Never to be used directly by the user.
84LAVA_TEST_SHELL_TAR_REPO = {
85 "job_name": "LAVA Test Shell",
86 "timeout": 18000,
87 "device_type": DEVICE_TYPE_PARAMETER,
88 "actions": [
89 {
90 "command": "deploy_linaro_image",
91 "parameters": {
92 "image": PREBUILT_IMAGE_PARAMETER,
93 }
94 },
95 {
96 "command": "lava_test_shell",
97 "parameters": {
98 "timeout": 1800,
99 "testdef_repos": [
100 {
101 LAVA_TEST_SHELL_TESDEF_KEY: None,
102 LAVA_TEST_SHELL_TAR_REPO_KEY: None,
103 }
55 ]104 ]
56 }105 }
106 },
107 {
108 "command": "submit_results",
109 "parameters" : {
110 "stream": STREAM_PARAMETER,
111 "server": SERVER_PARAMETER
112 }
57 }113 }
58 ]114 ]
59}115}
116
117BOOT_TEST_KEY = "boot-test"
118LAVA_TEST_SHELL_KEY = "lava-test-shell"
119
120# Dict with all the user available job templates.
121JOB_TYPES = {
122 BOOT_TEST_KEY: BOOT_TEST,
123 LAVA_TEST_SHELL_KEY: LAVA_TEST_SHELL,
124}
60125
=== modified file 'lava/job/tests/test_commands.py'
--- lava/job/tests/test_commands.py 2013-07-25 15:43:25 +0000
+++ lava/job/tests/test_commands.py 2013-07-25 15:43:26 +0000
@@ -41,6 +41,7 @@
41 def setUp(self):41 def setUp(self):
42 super(CommandTest, self).setUp()42 super(CommandTest, self).setUp()
43 self.args.FILE = self.temp_file.name43 self.args.FILE = self.temp_file.name
44 self.args.type = "boot-test"
4445
45 self.device_type = Parameter('device_type')46 self.device_type = Parameter('device_type')
46 self.prebuilt_image = Parameter('prebuilt_image',47 self.prebuilt_image = Parameter('prebuilt_image',
@@ -49,14 +50,6 @@
49 self.config.put_parameter(self.device_type, 'foo')50 self.config.put_parameter(self.device_type, 'foo')
50 self.config.put_parameter(self.prebuilt_image, 'bar')51 self.config.put_parameter(self.prebuilt_image, 'bar')
5152
52 def tmp(self, filename):
53 """Returns a path to a non existent file.
54
55 :param filename: The name the file should have.
56 :return A path.
57 """
58 return os.path.join(self.temp_dir, filename)
59
6053
61class JobNewTest(CommandTest):54class JobNewTest(CommandTest):
6255
@@ -106,7 +99,7 @@
106 command = run(self.parser, self.args)99 command = run(self.parser, self.args)
107 self.assertRaises(CommandError, command.invoke)100 self.assertRaises(CommandError, command.invoke)
108101
109 @patch("lava.job.commands.has_command", new=MagicMock(return_value=False))102 @patch("lava_tool.utils.has_command", new=MagicMock(return_value=False))
110 def test_invoke_raises_1(self):103 def test_invoke_raises_1(self):
111 # Users passes a valid file to the run command, but she does not have104 # Users passes a valid file to the run command, but she does not have
112 # the dispatcher installed.105 # the dispatcher installed.
113106
=== modified file 'lava/job/tests/test_job.py'
--- lava/job/tests/test_job.py 2013-07-25 15:43:25 +0000
+++ lava/job/tests/test_job.py 2013-07-25 15:43:26 +0000
@@ -20,62 +20,73 @@
20Unit tests for the Job class20Unit tests for the Job class
21"""21"""
2222
23import os
23import json24import json
24import os
25import tempfile25import tempfile
2626
27from StringIO import StringIO27from mock import patch
28from unittest import TestCase
2928
30from lava.config import Config29from lava.config import Config
30from lava.helper.tests.helper_test import HelperTest
31from lava.job import Job31from lava.job import Job
32from lava.job.templates import BOOT_TEST32from lava.job.templates import BOOT_TEST
33from lava.parameter import Parameter33from lava.parameter import Parameter
3434
3535
36class JobTest(TestCase):36class JobTest(HelperTest):
3737
38 def setUp(self):38 @patch("lava.config.Config.save")
39 self.config_file = tempfile.NamedTemporaryFile(delete=False)39 def setUp(self, mocked_config):
40 super(JobTest, self).setUp()
40 self.config = Config()41 self.config = Config()
41 self.config._config_file = self.config_file.name42 self.config.config_file = self.temp_file.name
42
43 def tearDown(self):
44 if os.path.isfile(self.config_file.name):
45 os.unlink(self.config_file.name)
4643
47 def test_from_template(self):44 def test_from_template(self):
48 template = {}45 template = {}
49 job = Job(template)46 job = Job(template, self.temp_file.name)
50 self.assertEqual(job.data, template)47 self.assertEqual(job.data, template)
51 self.assertIsNot(job.data, template)48 self.assertIsNot(job.data, template)
5249
53 def test_fill_in_data(self):50 def test_update_data(self):
54 image = "/path/to/panda.img"51 image = "/path/to/panda.img"
55 param1 = Parameter("device_type")52 param1 = Parameter("device_type")
56 param2 = Parameter("prebuilt_image", depends=param1)53 param2 = Parameter("image", depends=param1)
57 self.config.put_parameter(param1, "panda")54 self.config.put_parameter(param1, "panda")
58 self.config.put_parameter(param2, image)55 self.config.put_parameter(param2, image)
5956
60 job = Job(BOOT_TEST)57 job = Job(BOOT_TEST, self.temp_file.name)
61 job.fill_in(self.config)58 job.update(self.config)
6259
63 self.assertEqual(job.data['device_type'], "panda")60 self.assertEqual(job.data['device_type'], "panda")
64 self.assertEqual(job.data['actions'][0]["parameters"]["image"], image)61 self.assertEqual(job.data['actions'][0]["parameters"]["image"], image)
6562
66 def test_write(self):63 def test_write(self):
67 orig_data = {"foo": "bar"}64 try:
68 job = Job(orig_data)65 orig_data = {"foo": "bar"}
69 output = StringIO()66 job_file = os.path.join(tempfile.gettempdir(), "a_json_file.json")
70 job.write(output)67 job = Job(orig_data, job_file)
7168 job.write()
72 data = json.loads(output.getvalue())69
73 self.assertEqual(data, orig_data)70 output = ""
71 with open(job_file) as read_file:
72 output = read_file.read()
73
74 data = json.loads(output)
75 self.assertEqual(data, orig_data)
76 finally:
77 os.unlink(job_file)
7478
75 def test_writes_nicely_formatted_json(self):79 def test_writes_nicely_formatted_json(self):
76 orig_data = {"foo": "bar"}80 try:
77 job = Job(orig_data)81 orig_data = {"foo": "bar"}
78 output = StringIO()82 job_file = os.path.join(tempfile.gettempdir(), "b_json_file.json")
79 job.write(output)83 job = Job(orig_data, job_file)
8084 job.write()
81 self.assertTrue(output.getvalue().startswith("{\n"))85
86 output = ""
87 with open(job_file) as read_file:
88 output = read_file.read()
89
90 self.assertTrue(output.startswith("{\n"))
91 finally:
92 os.unlink(job_file)
8293
=== modified file 'lava/parameter.py'
--- lava/parameter.py 2013-07-25 15:43:25 +0000
+++ lava/parameter.py 2013-07-25 15:43:26 +0000
@@ -20,7 +20,19 @@
20Parameter class and its accessory methods/functions.20Parameter class and its accessory methods/functions.
21"""21"""
2222
23import StringIO
24import base64
25import os
23import sys26import sys
27import tarfile
28import tempfile
29import types
30
31from lava.tool.errors import CommandError
32from lava_tool.utils import to_list
33
34# Character used to join serialized list parameters.
35LIST_SERIALIZE_DELIMITER = ","
2436
2537
26class Parameter(object):38class Parameter(object):
@@ -38,6 +50,15 @@
38 self.value = value50 self.value = value
39 self.depends = depends51 self.depends = depends
40 self.asked = False52 self.asked = False
53 # Whether to store or not the parameter in the user config file.
54 self.store = True
55
56 def set(self, value):
57 """Sets the value of the parameter.
58
59 :param value: The value to set.
60 """
61 self.value = value
4162
42 def prompt(self, old_value=None):63 def prompt(self, old_value=None):
43 """Gets the parameter value from the user.64 """Gets the parameter value from the user.
@@ -50,26 +71,187 @@
50 :param old_value: The old parameter value.71 :param old_value: The old parameter value.
51 :return The input as typed by the user, or the old value.72 :return The input as typed by the user, or the old value.
52 """73 """
53 if old_value is not None:74 if not self.asked:
54 prompt = "{0} [{1}]: ".format(self.id, old_value)75 if old_value is not None:
55 else:76 prompt = "{0} [{1}]: ".format(self.id, old_value)
56 prompt = "{0}: ".format(self.id)77 else:
5778 prompt = "{0}: ".format(self.id)
58 user_input = None79
80 user_input = self.get_user_input(prompt)
81
82 if user_input is not None:
83 if len(user_input) == 0 and old_value:
84 # Keep the old value when user press enter or another
85 # whitespace char.
86 self.value = old_value
87 else:
88 self.value = user_input
89
90 self.asked = True
91
92 return self.value
93
94 @classmethod
95 def get_user_input(cls, prompt=""):
96 """Asks the user for input data.
97
98 :param prompt: The prompt that should be given to the user.
99 :return A string with what the user typed.
100 """
101 data = None
59 try:102 try:
60 user_input = raw_input(prompt).strip()103 data = raw_input(prompt).strip()
61 except EOFError:104 except EOFError:
62 pass105 # Force to return None.
106 data = None
63 except KeyboardInterrupt:107 except KeyboardInterrupt:
64 sys.exit(-1)108 sys.exit(-1)
65109 return data
66 if user_input is not None:110
111 @classmethod
112 def serialize(cls, value):
113 """Serializes the passed value to be friendly written to file.
114
115 Lists are serialized as a comma separated string of values.
116
117 :param value: The value to serialize.
118 :return The serialized value as string.
119 """
120 serialized = ""
121 if isinstance(value, list):
122 serialized = LIST_SERIALIZE_DELIMITER.join(
123 str(x) for x in value if x)
124 else:
125 serialized = str(value)
126 return serialized
127
128 @classmethod
129 def deserialize(cls, value):
130 """Deserialize a value into a list.
131
132 The value must have been serialized with the class instance serialize()
133 method.
134
135 :param value: The string value to be deserialized.
136 :type str
137 :return A list of values.
138 """
139 deserialized = []
140 if isinstance(value, types.StringTypes):
141 deserialized = filter(None, (x.strip() for x in value.split(
142 LIST_SERIALIZE_DELIMITER)))
143 else:
144 deserialized = list(value)
145 return deserialized
146
147
148class SingleChoiceParameter(Parameter):
149 """A parameter implemeting a single choice between multiple choices."""
150 def __init__(self, id, choices):
151 super(SingleChoiceParameter, self).__init__(id)
152 self.choices = to_list(choices)
153
154 def prompt(self, prompt, old_value=None):
155 """Asks the user for their choice."""
156 # Sliglty different than the other parameters: here we first present
157 # the user with what the choices are about.
158 print >> sys.stdout, prompt
159
160 index = 1
161 for choice in self.choices:
162 print >> sys.stdout, "\t{0:d}. {1}".format(index, choice)
163 index += 1
164
165 choices_len = len(self.choices)
166 while True:
167 user_input = self.get_user_input("Choice: ")
168
67 if len(user_input) == 0 and old_value:169 if len(user_input) == 0 and old_value:
68 # Keep the old value when user press enter or another170 choice = old_value
69 # whitespace char.171 break
70 self.value = old_value172 elif user_input in [str(x) for x in range(1, choices_len + 1)]:
71 else:173 choice = self.choices[int(user_input) - 1]
72 self.value = user_input174 break
73175
74 self.asked = True176 return choice
177
178
179class ListParameter(Parameter):
180 """A specialized Parameter to handle list values."""
181
182 # This is used as a deletion character. When we have an old value and the
183 # user enters this char, it sort of deletes the value.
184 DELETE_CHAR = "-"
185
186 def __init__(self, id, value=None, depends=None):
187 super(ListParameter, self).__init__(id, depends=depends)
188 self.value = []
189 if value:
190 self.set(value)
191
192 def set(self, value):
193 """Sets the value of the parameter.
194
195 :param value: The value to set.
196 """
197 self.value = to_list(value)
198
199 def add(self, value):
200 """Adds a new value to the list of values of this parameter.
201
202 :param value: The value to add.
203 """
204 if isinstance(value, list):
205 self.value.extend(value)
206 else:
207 self.value.append(value)
208
209 def prompt(self, old_value=None):
210 """Gets the parameter in a list form.
211
212 To exit the input procedure it is necessary to insert an empty line.
213
214 :return The list of values.
215 """
216
217 if not self.asked:
218 if old_value is not None:
219 # We might get the old value read from file via ConfigParser,
220 # and usually it comes in string format.
221 old_value = self.deserialize(old_value)
222
223 print >> sys.stdout, "Values for '{0}': ".format(self.id)
224
225 index = 1
226 while True:
227 user_input = None
228 if old_value is not None and (0 < len(old_value) >= index):
229 prompt = "{0:>3d}.\n\told: {1}\n\tnew: ".format(
230 index, old_value[index-1])
231 user_input = self.get_user_input(prompt)
232 else:
233 prompt = "{0:>3d}. ".format(index)
234 user_input = self.get_user_input(prompt)
235
236 if user_input is not None:
237 # The user has pressed Enter.
238 if len(user_input) == 0:
239 if old_value is not None and \
240 (0 < len(old_value) >= index):
241 user_input = old_value[index-1]
242 else:
243 break
244
245 if len(user_input) == 1 and user_input == \
246 self.DELETE_CHAR and (0 < len(old_value) >= index):
247 # We have an old value, user presses the DELETE_CHAR
248 # and we do not store anything. This is done to delete
249 # an old entry.
250 pass
251 else:
252 self.value.append(user_input)
253 index += 1
254
255 self.asked = True
256
75 return self.value257 return self.value
76258
=== added directory 'lava/testdef'
=== added file 'lava/testdef/__init__.py'
--- lava/testdef/__init__.py 1970-01-01 00:00:00 +0000
+++ lava/testdef/__init__.py 2013-07-25 15:43:26 +0000
@@ -0,0 +1,60 @@
1# Copyright (C) 2013 Linaro Limited
2#
3# Author: Milo Casagrande <milo.casagrande@linaro.org>
4#
5# This file is part of lava-tool.
6#
7# lava-tool is free software: you can redistribute it and/or modify
8# it under the terms of the GNU Lesser General Public License version 3
9# as published by the Free Software Foundation
10#
11# lava-tool is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
18
19import yaml
20
21from copy import deepcopy
22
23from lava.helper.template import expand_template
24from lava_tool.utils import (
25 write_file,
26 verify_path_existance,
27 verify_file_extension
28)
29
30# Default test def file extension.
31DEFAULT_TESTDEF_EXTENSION = "yaml"
32# Possible extensions for a test def file.
33TESTDEF_FILE_EXTENSIONS = [DEFAULT_TESTDEF_EXTENSION]
34
35
36class TestDefinition(object):
37
38 def __init__(self, data, file_name):
39 """Initialize the object.
40
41 :param data: The serializable data to be used, usually a template.
42 :type dict
43 :param file_name: Where the test definition will be written.
44 :type str
45 """
46 self.file_name = verify_file_extension(file_name,
47 DEFAULT_TESTDEF_EXTENSION,
48 TESTDEF_FILE_EXTENSIONS)
49 verify_path_existance(self.file_name)
50
51 self.data = deepcopy(data)
52
53 def write(self):
54 """Writes the test definition to file."""
55 content = yaml.dump(self.data, default_flow_style=False, indent=4)
56 write_file(self.file_name, content)
57
58 def update(self, config):
59 """Updates the TestDefinition object based on the provided config."""
60 expand_template(self.data, config)
061
=== added file 'lava/testdef/commands.py'
--- lava/testdef/commands.py 1970-01-01 00:00:00 +0000
+++ lava/testdef/commands.py 2013-07-25 15:43:26 +0000
@@ -0,0 +1,72 @@
1# Copyright (C) 2013 Linaro Limited
2#
3# Author: Milo Casagrande <milo.casagrande@linaro.org>
4#
5# This file is part of lava-tool.
6#
7# lava-tool is free software: you can redistribute it and/or modify
8# it under the terms of the GNU Lesser General Public License version 3
9# as published by the Free Software Foundation
10#
11# lava-tool is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
18
19"""
20Test definition commands class.
21"""
22
23import os
24
25from lava.helper.command import BaseCommand
26from lava.tool.command import CommandGroup
27
28
29class testdef(CommandGroup):
30
31 """LAVA test definitions handling."""
32
33 namespace = "lava.testdef.commands"
34
35
36class new(BaseCommand):
37
38 """Creates a new test definition file."""
39
40 @classmethod
41 def register_arguments(cls, parser):
42 super(new, cls).register_arguments(parser)
43 parser.add_argument("FILE", help="Test definition file to create.")
44
45 def invoke(self):
46 full_path = os.path.abspath(self.args.FILE)
47 self.create_test_definition(full_path)
48
49
50class run(BaseCommand):
51
52 """Runs the specified test definition on a local device."""
53
54 @classmethod
55 def register_arguments(cls, parser):
56 super(run, cls).register_arguments(parser)
57 parser.add_argument("FILE", help="Test definition file to run.")
58
59 def invoke(self):
60 pass
61
62
63def submit(BaseCommand):
64 """Submits the specified test definition to a remove LAVA server."""
65
66 @classmethod
67 def register_arguments(cls, parser):
68 super(submit, cls).register_arguments(parser)
69 parser.add_argument("FILE", help="Test definition file to send.")
70
71 def invoke(self):
72 pass
073
=== added file 'lava/testdef/templates.py'
--- lava/testdef/templates.py 1970-01-01 00:00:00 +0000
+++ lava/testdef/templates.py 2013-07-25 15:43:26 +0000
@@ -0,0 +1,75 @@
1# Copyright (C) 2013 Linaro Limited
2#
3# Author: Milo Casagrande <milo.casagrande@linaro.org>
4#
5# This file is part of lava-tool.
6#
7# lava-tool is free software: you can redistribute it and/or modify
8# it under the terms of the GNU Lesser General Public License version 3
9# as published by the Free Software Foundation
10#
11# lava-tool is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
18
19"""Test definition templates."""
20
21from lava.parameter import (
22 ListParameter,
23 Parameter,
24)
25
26DEFAULT_TESTDEF_FILE = "lavatest.yaml"
27
28DEFAULT_TESTDEF_VERSION = "1.0"
29DEFAULT_TESTDEF_FORMAT = "Lava-Test Test Definition 1.0"
30
31# This is what will be called by default by the test definition yaml file.
32DEFAULT_TESTDEF_SCRIPT = "mytest.sh"
33DEFAULT_TESTDEF_SCRIPT_CONTENT = """#!/bin/sh
34# Automatic generated content by lava-tool.
35# Please add your own instructions.
36"""
37DEFAULT_TESTDEF_STEP = "./mytest.sh"
38
39DEFAULT_ENVIRONMET_VALUE = "lava_test_shell"
40
41# All these parameters will not be stored on the local config file.
42NAME_PARAMETER = Parameter("name")
43NAME_PARAMETER.store = False
44
45DESCRIPTION_PARAMETER = Parameter("description", depends=NAME_PARAMETER)
46DESCRIPTION_PARAMETER.store = False
47
48ENVIRONMENT_PARAMETER = ListParameter("environment",
49 depends=NAME_PARAMETER)
50ENVIRONMENT_PARAMETER.add(DEFAULT_ENVIRONMET_VALUE)
51ENVIRONMENT_PARAMETER.asked = True
52ENVIRONMENT_PARAMETER.store = False
53
54# Steps parameter. Default to a local shell script that the user defines.
55# We do not ask this parameter, and we do not store it either.
56STEPS_PARAMETER = ListParameter("steps", depends=NAME_PARAMETER)
57STEPS_PARAMETER.add(DEFAULT_TESTDEF_STEP)
58STEPS_PARAMETER.asked = True
59STEPS_PARAMETER.store = False
60
61TESTDEF_TEMPLATE = {
62 "metadata": {
63 "name": NAME_PARAMETER,
64 "format": DEFAULT_TESTDEF_FORMAT,
65 "version": DEFAULT_TESTDEF_VERSION,
66 "description": DESCRIPTION_PARAMETER,
67 "environment": ENVIRONMENT_PARAMETER,
68 },
69 "run": {
70 "steps": STEPS_PARAMETER,
71 },
72 "parse": {
73 "pattern": '^\s*(?P<test_case_id>\w+)=(?P<result>\w+)\s*$'
74 }
75}
076
=== added directory 'lava/testdef/tests'
=== added file 'lava/testdef/tests/__init__.py'
=== added file 'lava/testdef/tests/test_commands.py'
--- lava/testdef/tests/test_commands.py 1970-01-01 00:00:00 +0000
+++ lava/testdef/tests/test_commands.py 2013-07-25 15:43:26 +0000
@@ -0,0 +1,153 @@
1# Copyright (C) 2013 Linaro Limited
2#
3# Author: Milo Casagrande <milo.casagrande@linaro.org>
4#
5# This file is part of lava-tool.
6#
7# lava-tool is free software: you can redistribute it and/or modify
8# it under the terms of the GNU Lesser General Public License version 3
9# as published by the Free Software Foundation
10#
11# lava-tool is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
18
19"""
20Tests for lava.testdef.commands.
21"""
22
23import os
24import tempfile
25import yaml
26
27from mock import (
28 patch,
29)
30
31from lava.config import InteractiveConfig
32from lava.helper.tests.helper_test import HelperTest
33from lava.testdef.commands import (
34 new,
35)
36from lava.tool.errors import CommandError
37
38
39class NewCommandTest(HelperTest):
40 """Class for the lava.testdef new command tests."""
41
42 @patch("lava.config.Config.save")
43 def setUp(self, mocked_save):
44 super(NewCommandTest, self).setUp()
45 self.file_name = "fake_testdef.yaml"
46 self.file_path = os.path.join(tempfile.gettempdir(), self.file_name)
47 self.args.FILE = self.file_path
48
49 self.temp_yaml = tempfile.NamedTemporaryFile(suffix=".yaml",
50 delete=False)
51
52 self.config_file = tempfile.NamedTemporaryFile(delete=False)
53 self.config = InteractiveConfig()
54 self.config.config_file = self.config_file.name
55 # Patch class raw_input, start it, and stop it on tearDown.
56 self.patcher1 = patch("lava.parameter.raw_input", create=True)
57 self.mocked_raw_input = self.patcher1.start()
58
59 def tearDown(self):
60 super(NewCommandTest, self).tearDown()
61 if os.path.isfile(self.file_path):
62 os.unlink(self.file_path)
63 os.unlink(self.config_file.name)
64 os.unlink(self.temp_yaml.name)
65 self.patcher1.stop()
66
67 def test_register_arguments(self):
68 # Make sure that the parser add_argument is called and we have the
69 # correct argument.
70 new_command = new(self.parser, self.args)
71 new_command.register_arguments(self.parser)
72
73 # Make sure we do not forget about this test.
74 self.assertEqual(2, len(self.parser.method_calls))
75
76 _, args, _ = self.parser.method_calls[0]
77 self.assertIn("--non-interactive", args)
78
79 _, args, _ = self.parser.method_calls[1]
80 self.assertIn("FILE", args)
81
82 def test_invoke_0(self):
83 # Test that passing a file on the command line, it is created on the
84 # file system.
85 self.mocked_raw_input.return_value = "\n"
86 new_command = new(self.parser, self.args)
87 new_command.invoke()
88 self.assertTrue(os.path.exists(self.file_path))
89
90 def test_invoke_1(self):
91 # Test that when passing an already existing file, an exception is
92 # thrown.
93 self.args.FILE = self.temp_yaml.name
94 new_command = new(self.parser, self.args)
95 self.assertRaises(CommandError, new_command.invoke)
96
97 def test_invoke_2(self):
98 # Tests that when adding a new test definition and writing it to file
99 # a correct YAML structure is created.
100 self.mocked_raw_input.return_value = "\n"
101 new_command = new(self.parser, self.args)
102 new_command.config = self.config
103 new_command.invoke()
104 expected = {'run': {'steps': ["./mytest.sh"]},
105 'metadata': {
106 'environment': ['lava_test_shell'],
107 'format': 'Lava-Test Test Definition 1.0',
108 'version': '1.0',
109 'description': '',
110 'name': ''},
111 'parse': {
112 'pattern':
113 '^\\s*(?P<test_case_id>\\w+)=(?P<result>\\w+)\\s*$'
114 },
115 }
116 obtained = None
117 with open(self.file_path, 'r') as read_file:
118 obtained = yaml.load(read_file)
119 self.assertEqual(expected, obtained)
120
121 def test_invoke_3(self):
122 # Tests that when adding a new test definition and writing it to a file
123 # in a directory withour permissions, exception is raised.
124 self.args.FILE = "/test_file.yaml"
125 self.mocked_raw_input.return_value = "\n"
126 new_command = new(self.parser, self.args)
127 self.assertRaises(CommandError, new_command.invoke)
128 self.assertFalse(os.path.exists(self.args.FILE))
129
130 def test_invoke_4(self):
131 # Tests that when passing values for the "steps" ListParameter, we get
132 # back the correct data structure.
133 self.mocked_raw_input.side_effect = ["foo", "\n", "\n", "\n", "\n",
134 "\n"]
135 new_command = new(self.parser, self.args)
136 new_command.invoke()
137 expected = {'run': {'steps': ["./mytest.sh"]},
138 'metadata': {
139 'environment': ['lava_test_shell'],
140 'format': 'Lava-Test Test Definition 1.0',
141 'version': '1.0',
142 'description': '',
143 'name': 'foo'
144 },
145 'parse': {
146 'pattern':
147 '^\\s*(?P<test_case_id>\\w+)=(?P<result>\\w+)\\s*$'
148 },
149 }
150 obtained = None
151 with open(self.file_path, 'r') as read_file:
152 obtained = yaml.load(read_file)
153 self.assertEqual(expected, obtained)
0154
=== added file 'lava/tests/test_commands.py'
--- lava/tests/test_commands.py 1970-01-01 00:00:00 +0000
+++ lava/tests/test_commands.py 2013-07-25 15:43:26 +0000
@@ -0,0 +1,128 @@
1# Copyright (C) 2013 Linaro Limited
2#
3# Author: Milo Casagrande <milo.casagrande@linaro.org>
4#
5# This file is part of lava-tool.
6#
7# lava-tool is free software: you can redistribute it and/or modify
8# it under the terms of the GNU Lesser General Public License version 3
9# as published by the Free Software Foundation
10#
11# lava-tool is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
18
19"""
20Tests for lava.commands.
21"""
22
23import os
24import tempfile
25
26from mock import (
27 MagicMock,
28 patch
29)
30
31from lava.commands import (
32 init,
33 submit,
34)
35from lava.config import Config
36from lava.helper.tests.helper_test import HelperTest
37from lava.tool.errors import CommandError
38
39
40class InitCommandTests(HelperTest):
41
42 def setUp(self):
43 super(InitCommandTests, self).setUp()
44 self.config_file = self.tmp("init_command_tests")
45 self.config = Config()
46 self.config.config_file = self.config_file
47
48 def tearDown(self):
49 super(InitCommandTests, self).tearDown()
50 if os.path.isfile(self.config_file):
51 os.unlink(self.config_file)
52
53 def test_register_arguments(self):
54 self.args.DIR = os.path.join(tempfile.gettempdir(), "a_fake_dir")
55 init_command = init(self.parser, self.args)
56 init_command.register_arguments(self.parser)
57
58 # Make sure we do not forget about this test.
59 self.assertEqual(2, len(self.parser.method_calls))
60
61 _, args, _ = self.parser.method_calls[0]
62 self.assertIn("--non-interactive", args)
63
64 _, args, _ = self.parser.method_calls[1]
65 self.assertIn("DIR", args)
66
67
68 @patch("lava.commands.edit_file", create=True)
69 def test_command_invoke_0(self, mocked_edit_file):
70 # Invoke the init command passing a path to a file. Should raise an
71 # exception.
72 self.args.DIR = self.temp_file.name
73 init_command = init(self.parser, self.args)
74 self.assertRaises(CommandError, init_command.invoke)
75
76 def test_command_invoke_2(self):
77 # Invoke the init command passing a path where the user cannot write.
78 try:
79 self.args.DIR = "/root/a_temp_dir"
80 init_command = init(self.parser, self.args)
81 self.assertRaises(CommandError, init_command.invoke)
82 finally:
83 if os.path.exists(self.args.DIR):
84 os.removedirs(self.args.DIR)
85
86 def test_update_data(self):
87 # Make sure the template is updated accordingly with the provided data.
88 self.args.DIR = self.temp_file.name
89
90 init_command = init(self.parser, self.args)
91 init_command.config.get = MagicMock()
92 init_command.config.save = MagicMock()
93 init_command.config.get.side_effect = ["a_job.json"]
94
95 expected = {
96 "jobfile": "a_job.json",
97 }
98
99 obtained = init_command._update_data()
100 self.assertEqual(expected, obtained)
101
102
103class SubmitCommandTests(HelperTest):
104 def setUp(self):
105 super(SubmitCommandTests, self).setUp()
106 self.config_file = self.tmp("submit_command_tests")
107 self.config = Config()
108 self.config.config_file = self.config_file
109 self.config.save = MagicMock()
110
111 def tearDown(self):
112 super(SubmitCommandTests, self).tearDown()
113 if os.path.isfile(self.config_file):
114 os.unlink(self.config_file)
115
116 def test_register_arguments(self):
117 self.args.JOB = os.path.join(tempfile.gettempdir(), "a_fake_file")
118 submit_command = submit(self.parser, self.args)
119 submit_command.register_arguments(self.parser)
120
121 # Make sure we do not forget about this test.
122 self.assertEqual(2, len(self.parser.method_calls))
123
124 _, args, _ = self.parser.method_calls[0]
125 self.assertIn("--non-interactive", args)
126
127 _, args, _ = self.parser.method_calls[1]
128 self.assertIn("JOB", args)
0129
=== modified file 'lava/tests/test_config.py'
--- lava/tests/test_config.py 2013-07-25 15:43:25 +0000
+++ lava/tests/test_config.py 2013-07-25 15:43:26 +0000
@@ -20,69 +20,46 @@
20lava.config unit tests.20lava.config unit tests.
21"""21"""
2222
23import os
24import sys23import sys
25import tempfile
2624
27from StringIO import StringIO25from StringIO import StringIO
28from mock import MagicMock, patch, call26from mock import (
27 MagicMock,
28 call,
29 patch,
30)
2931
30from lava.config import (32from lava.config import (
31 Config,33 Config,
32 InteractiveConfig,34 InteractiveConfig,
33 ConfigParser,
34)35)
35from lava.helper.tests.helper_test import HelperTest36from lava.helper.tests.helper_test import HelperTest
36from lava.parameter import Parameter37from lava.parameter import (
38 Parameter,
39 ListParameter,
40)
37from lava.tool.errors import CommandError41from lava.tool.errors import CommandError
3842
3943
40class MockedConfig(Config):
41 """A subclass of the original Config class.
42
43 Used to test the Config class, but to not have the same constructor in
44 order to use temporary files for the configuration.
45 """
46 def __init__(self, config_file):
47 self._cache = {}
48 self._config_file = config_file
49 self._config_backend = ConfigParser()
50 self._config_backend.read([self._config_file])
51
52
53class MockedInteractiveConfig(InteractiveConfig):
54 def __init__(self, config_file, force_interactive=False):
55 self._cache = {}
56 self._config_file = config_file
57 self._config_backend = ConfigParser()
58 self._config_backend.read([self._config_file])
59 self._force_interactive = force_interactive
60
61
62class ConfigTestCase(HelperTest):44class ConfigTestCase(HelperTest):
63 """General test case class for the different Config classes."""45 """General test case class for the different Config classes."""
64 def setUp(self):46 def setUp(self):
65 super(ConfigTestCase, self).setUp()47 super(ConfigTestCase, self).setUp()
66 self.config_file = tempfile.NamedTemporaryFile(delete=False)
67
68 self.param1 = Parameter("foo")48 self.param1 = Parameter("foo")
69 self.param2 = Parameter("bar", depends=self.param1)49 self.param2 = Parameter("bar", depends=self.param1)
7050
71 def tearDown(self):
72 super(ConfigTestCase, self).tearDown()
73 if os.path.isfile(self.config_file.name):
74 os.unlink(self.config_file.name)
75
7651
77class ConfigTest(ConfigTestCase):52class ConfigTest(ConfigTestCase):
7853
79 def setUp(self):54 @patch("lava.config.Config.save")
55 def setUp(self, mocked_save):
80 super(ConfigTest, self).setUp()56 super(ConfigTest, self).setUp()
81 self.config = MockedConfig(self.config_file.name)57 self.config = Config()
58 self.config.config_file = self.temp_file.name
8259
83 def test_assert_temp_config_file(self):60 def test_assert_temp_config_file(self):
84 # Dummy test to make sure we are overriding correctly the Config class.61 # Dummy test to make sure we are overriding correctly the Config class.
85 self.assertEqual(self.config._config_file, self.config_file.name)62 self.assertEqual(self.config.config_file, self.temp_file.name)
8663
87 def test_config_put_in_cache_0(self):64 def test_config_put_in_cache_0(self):
88 self.config._put_in_cache("key", "value", "section")65 self.config._put_in_cache("key", "value", "section")
@@ -167,46 +144,38 @@
167144
168 expected = "[DEFAULT]\nfoo = foo\n\n"145 expected = "[DEFAULT]\nfoo = foo\n\n"
169 obtained = ""146 obtained = ""
170 with open(self.config_file.name) as tmp_file:147 with open(self.temp_file.name) as tmp_file:
171 obtained = tmp_file.read()148 obtained = tmp_file.read()
172 self.assertEqual(expected, obtained)149 self.assertEqual(expected, obtained)
173150
174 @patch("lava.config.AT_EXIT_CALLS", spec=set)
175 def test_config_atexit_call_list(self, mocked_calls):
176 # Tests that the save() method is added to the set of atexit calls.
177 config = Config()
178 config._config_file = self.config_file.name
179 config.put_parameter(self.param1, "foo")
180
181 expected = [call.add(config.save)]
182
183 self.assertEqual(expected, mocked_calls.mock_calls)
184
185151
186class InteractiveConfigTest(ConfigTestCase):152class InteractiveConfigTest(ConfigTestCase):
187153
188 def setUp(self):154 @patch("lava.config.Config.save")
155 def setUp(self, mocked_save):
189 super(InteractiveConfigTest, self).setUp()156 super(InteractiveConfigTest, self).setUp()
190 self.config = MockedInteractiveConfig(157 self.config = InteractiveConfig()
191 config_file=self.config_file.name)158 self.config.config_file = self.temp_file.name
192159
193 @patch("lava.config.Config.get", new=MagicMock(return_value=None))160 @patch("lava.config.Config.get", new=MagicMock(return_value=None))
194 def test_non_interactive_config_0(self):161 def test_non_interactive_config_0(self):
195 # Mocked config default is not to be interactive.
196 # Try to get a value that does not exists, users just press enter when162 # Try to get a value that does not exists, users just press enter when
197 # asked for a value. Value will be empty.163 # asked for a value. Value will be empty.
164 self.config.force_interactive = False
198 sys.stdin = StringIO("\n")165 sys.stdin = StringIO("\n")
199 value = self.config.get(Parameter("foo"))166 value = self.config.get(Parameter("foo"))
200 self.assertEqual("", value)167 self.assertEqual("", value)
201168
202 @patch("lava.config.Config.get", new=MagicMock(return_value="value"))169 @patch("lava.config.Config.get", new=MagicMock(return_value="value"))
203 def test_non_interactive_config_1(self):170 def test_non_interactive_config_1(self):
204 # Parent class config returns a value, but we are not interactive.171 # Parent class config returns value, but we are not interactive.
172 self.config.force_interactive = False
205 value = self.config.get(Parameter("foo"))173 value = self.config.get(Parameter("foo"))
206 self.assertEqual("value", value)174 self.assertEqual("value", value)
207175
208 @patch("lava.config.Config.get", new=MagicMock(return_value=None))176 @patch("lava.config.Config.get", new=MagicMock(return_value=None))
209 def test_non_interactive_config_2(self):177 def test_non_interactive_config_2(self):
178 self.config.force_interactive = False
210 expected = "bar"179 expected = "bar"
211 sys.stdin = StringIO(expected)180 sys.stdin = StringIO(expected)
212 value = self.config.get(Parameter("foo"))181 value = self.config.get(Parameter("foo"))
@@ -216,7 +185,7 @@
216 def test_interactive_config_0(self):185 def test_interactive_config_0(self):
217 # We force to be interactive, meaning that even if a value is found,186 # We force to be interactive, meaning that even if a value is found,
218 # it will be asked anyway.187 # it will be asked anyway.
219 self.config._force_interactive = True188 self.config.force_interactive = True
220 expected = "a_new_value"189 expected = "a_new_value"
221 sys.stdin = StringIO(expected)190 sys.stdin = StringIO(expected)
222 value = self.config.get(Parameter("foo"))191 value = self.config.get(Parameter("foo"))
@@ -226,28 +195,28 @@
226 def test_interactive_config_1(self):195 def test_interactive_config_1(self):
227 # Force to be interactive, but when asked for the new value press196 # Force to be interactive, but when asked for the new value press
228 # Enter. The old value should be returned.197 # Enter. The old value should be returned.
229 self.config._force_interactive = True198 self.config.force_interactive = True
230 sys.stdin = StringIO("\n")199 sys.stdin = StringIO("\n")
231 value = self.config.get(Parameter("foo"))200 value = self.config.get(Parameter("foo"))
232 self.assertEqual("value", value)201 self.assertEqual("value", value)
233202
234 def test_calculate_config_section_0(self):203 def test_calculate_config_section_0(self):
235 self.config._force_interactive = True204 self.config.force_interactive = True
236 obtained = self.config._calculate_config_section(self.param1)205 obtained = self.config._calculate_config_section(self.param1)
237 expected = "DEFAULT"206 expected = "DEFAULT"
238 self.assertEqual(expected, obtained)207 self.assertEqual(expected, obtained)
239208
240 def test_calculate_config_section_1(self):209 def test_calculate_config_section_1(self):
210 self.param1.set("foo")
241 self.param2.depends.asked = True211 self.param2.depends.asked = True
242 self.config._force_interactive = True212 self.config.force_interactive = True
243 self.config.put(self.param1.id, "foo")
244 obtained = self.config._calculate_config_section(self.param2)213 obtained = self.config._calculate_config_section(self.param2)
245 expected = "foo=foo"214 expected = "foo=foo"
246 self.assertEqual(expected, obtained)215 self.assertEqual(expected, obtained)
247216
248 def test_calculate_config_section_2(self):217 def test_calculate_config_section_2(self):
249 self.config._force_interactive = True218 self.config.force_interactive = True
250 self.config._config_backend.get = MagicMock(return_value=None)219 self.config.config_backend.get = MagicMock(return_value=None)
251 sys.stdin = StringIO("baz")220 sys.stdin = StringIO("baz")
252 expected = "foo=baz"221 expected = "foo=baz"
253 obtained = self.config._calculate_config_section(self.param2)222 obtained = self.config._calculate_config_section(self.param2)
@@ -256,10 +225,9 @@
256 def test_calculate_config_section_3(self):225 def test_calculate_config_section_3(self):
257 # Tests that when a parameter has its value in the cache and also on226 # Tests that when a parameter has its value in the cache and also on
258 # file, we honor the cached version.227 # file, we honor the cached version.
228 self.param1.set("bar")
259 self.param2.depends.asked = True229 self.param2.depends.asked = True
260 self.config._force_interactive = True230 self.config.force_interactive = True
261 self.config._get_from_cache = MagicMock(return_value="bar")
262 self.config._config_backend.get = MagicMock(return_value="baz")
263 expected = "foo=bar"231 expected = "foo=bar"
264 obtained = self.config._calculate_config_section(self.param2)232 obtained = self.config._calculate_config_section(self.param2)
265 self.assertEqual(expected, obtained)233 self.assertEqual(expected, obtained)
@@ -273,6 +241,32 @@
273241
274 mocked_raw.side_effect = KeyboardInterrupt()242 mocked_raw.side_effect = KeyboardInterrupt()
275243
276 self.config._force_interactive = True244 self.config.force_interactive = True
277 self.config.get(self.param1)245 self.config.get(self.param1)
278 self.assertTrue(mocked_sys_exit.called)246 self.assertTrue(mocked_sys_exit.called)
247
248 @patch("lava.parameter.raw_input", create=True)
249 def test_interactive_config_with_list_parameter(self, mocked_raw_input):
250 # Tests that we get a list back in the Config class when using
251 # ListParameter and that it contains the expected values.
252 expected = ["foo", "bar"]
253 mocked_raw_input.side_effect = expected + ["\n"]
254 obtained = self.config.get(ListParameter("list"))
255 self.assertIsInstance(obtained, list)
256 self.assertEqual(expected, obtained)
257
258 def test_interactive_save_list_param(self):
259 # Tests that when saved to file, the ListParameter parameter is stored
260 # correctly.
261 param_values = ["foo", "more than one words", "bar"]
262 list_param = ListParameter("list")
263 list_param.set(param_values)
264
265 self.config.put_parameter(list_param, param_values)
266 self.config.save()
267
268 expected = "[DEFAULT]\nlist = " + ",".join(param_values) + "\n\n"
269 obtained = ""
270 with open(self.temp_file.name, "r") as read_file:
271 obtained = read_file.read()
272 self.assertEqual(expected, obtained)
279273
=== modified file 'lava/tests/test_parameter.py'
--- lava/tests/test_parameter.py 2013-07-25 15:43:25 +0000
+++ lava/tests/test_parameter.py 2013-07-25 15:43:26 +0000
@@ -20,16 +20,32 @@
20lava.parameter unit tests.20lava.parameter unit tests.
21"""21"""
2222
23import sys
24
25from StringIO import StringIO
26from mock import patch23from mock import patch
2724
28from lava.helper.tests.helper_test import HelperTest25from lava.helper.tests.helper_test import HelperTest
29from lava.parameter import Parameter26from lava.parameter import (
3027 ListParameter,
3128 Parameter,
32class ParameterTest(HelperTest):29)
30
31from lava_tool.utils import to_list
32
33
34class GeneralParameterTest(HelperTest):
35 """General class with setUp and tearDown methods for Parameter tests."""
36 def setUp(self):
37 super(GeneralParameterTest, self).setUp()
38 # Patch class raw_input, start it, and stop it on tearDown.
39 self.patcher1 = patch("lava.parameter.raw_input", create=True)
40 self.mocked_raw_input = self.patcher1.start()
41
42 def tearDown(self):
43 super(GeneralParameterTest, self).tearDown()
44 self.patcher1.stop()
45
46
47class ParameterTest(GeneralParameterTest):
48 """Tests for the Parameter class."""
3349
34 def setUp(self):50 def setUp(self):
35 super(ParameterTest, self).setUp()51 super(ParameterTest, self).setUp()
@@ -38,14 +54,79 @@
38 def test_prompt_0(self):54 def test_prompt_0(self):
39 # Tests that when we have a value in the parameters and the user press55 # Tests that when we have a value in the parameters and the user press
40 # Enter, we get the old value back.56 # Enter, we get the old value back.
41 sys.stdin = StringIO("\n")57 self.mocked_raw_input.return_value = "\n"
42 obtained = self.parameter1.prompt()58 obtained = self.parameter1.prompt()
43 self.assertEqual(self.parameter1.value, obtained)59 self.assertEqual(self.parameter1.value, obtained)
4460
45 @patch("lava.parameter.raw_input", create=True)61 def test_prompt_1(self,):
46 def test_prompt_1(self, mocked_raw_input):
47 # Tests that with a value stored in the parameter, if and EOFError is62 # Tests that with a value stored in the parameter, if and EOFError is
48 # raised when getting user input, we get back the old value.63 # raised when getting user input, we get back the old value.
49 mocked_raw_input.side_effect = EOFError()64 self.mocked_raw_input.side_effect = EOFError()
50 obtained = self.parameter1.prompt()65 obtained = self.parameter1.prompt()
51 self.assertEqual(self.parameter1.value, obtained)66 self.assertEqual(self.parameter1.value, obtained)
67
68 def test_to_list_0(self):
69 value = "a_value"
70 expected = [value]
71 obtained = to_list(value)
72 self.assertIsInstance(obtained, list)
73 self.assertEquals(expected, obtained)
74
75 def test_to_list_1(self):
76 expected = ["a_value", "b_value"]
77 obtained = to_list(expected)
78 self.assertIsInstance(obtained, list)
79 self.assertEquals(expected, obtained)
80
81class ListParameterTest(GeneralParameterTest):
82 """Tests for the specialized ListParameter class."""
83
84 def setUp(self):
85 super(ListParameterTest, self).setUp()
86 self.list_parameter = ListParameter("list")
87
88 def test_prompt_0(self):
89 # Test that when pressing Enter, the prompt stops and the list is
90 # returned.
91 expected = []
92 self.mocked_raw_input.return_value = "\n"
93 obtained = self.list_parameter.prompt()
94 self.assertEqual(expected, obtained)
95
96 def test_prompt_1(self):
97 # Tests that when passing 3 values, a list with those values
98 # is returned
99 expected = ["foo", "bar", "foobar"]
100 self.mocked_raw_input.side_effect = expected + ["\n"]
101 obtained = self.list_parameter.prompt()
102 self.assertEqual(expected, obtained)
103
104 def test_serialize_0(self):
105 # Tests the serialize method of ListParameter passing a list.
106 expected = "foo,bar,baz,1"
107 to_serialize = ["foo", "bar", "baz", "", 1]
108
109 obtained = self.list_parameter.serialize(to_serialize)
110 self.assertEqual(expected, obtained)
111
112 def test_serialize_1(self):
113 # Tests the serialize method of ListParameter passing an int.
114 expected = "1"
115 to_serialize = 1
116
117 obtained = self.list_parameter.serialize(to_serialize)
118 self.assertEqual(expected, obtained)
119
120 def test_deserialize_0(self):
121 # Tests the deserialize method of ListParameter with a string
122 # of values.
123 expected = ["foo", "bar", "baz"]
124 to_deserialize = "foo,bar,,baz,"
125 obtained = self.list_parameter.deserialize(to_deserialize)
126 self.assertEqual(expected, obtained)
127
128 def test_deserialize_1(self):
129 # Tests the deserialization method of ListParameter passing a list.
130 expected = ["foo", 1, "", "bar"]
131 obtained = self.list_parameter.deserialize(expected)
132 self.assertEqual(expected, obtained)
52133
=== modified file 'lava_tool/tests/__init__.py'
--- lava_tool/tests/__init__.py 2013-07-25 15:43:25 +0000
+++ lava_tool/tests/__init__.py 2013-07-25 15:43:26 +0000
@@ -35,19 +35,22 @@
3535
36def test_modules():36def test_modules():
37 return [37 return [
38 'lava.device.tests.test_commands',
39 'lava.device.tests.test_device',
40 'lava.helper.tests.test_command',
41 'lava.helper.tests.test_dispatcher',
42 'lava.helper.tests.test_template',
43 'lava.job.tests.test_commands',
44 'lava.job.tests.test_job',
45 'lava.testdef.tests.test_commands',
46 'lava.tests.test_commands',
47 'lava.tests.test_config',
48 'lava.tests.test_parameter',
49 'lava_dashboard_tool.tests.test_commands',
50 'lava_tool.tests.test_auth_commands',
38 'lava_tool.tests.test_authtoken',51 'lava_tool.tests.test_authtoken',
39 'lava_tool.tests.test_auth_commands',
40 'lava_tool.tests.test_commands',52 'lava_tool.tests.test_commands',
41 'lava_tool.tests.test_utils',53 'lava_tool.tests.test_utils',
42 'lava_dashboard_tool.tests.test_commands',
43 'lava.job.tests.test_job',
44 'lava.job.tests.test_commands',
45 'lava.device.tests.test_device',
46 'lava.device.tests.test_commands',
47 'lava.tests.test_config',
48 'lava.tests.test_parameter',
49 'lava.helper.tests.test_command',
50 'lava.helper.tests.test_dispatcher',
51 ]54 ]
5255
5356
5457
=== modified file 'lava_tool/tests/test_utils.py'
--- lava_tool/tests/test_utils.py 2013-07-25 15:43:25 +0000
+++ lava_tool/tests/test_utils.py 2013-07-25 15:43:26 +0000
@@ -18,16 +18,45 @@
1818
19"""lava_tool.utils tests."""19"""lava_tool.utils tests."""
2020
21import sys
22import os
21import subprocess23import subprocess
24import tempfile
2225
23from unittest import TestCase26from unittest import TestCase
24from mock import patch27from mock import (
28 MagicMock,
29 call,
30 patch,
31)
2532
26from lava_tool.utils import has_command33from lava.tool.errors import CommandError
34from lava_tool.utils import (
35 can_edit_file,
36 edit_file,
37 execute,
38 has_command,
39 retrieve_file,
40 verify_file_extension,
41)
2742
2843
29class UtilTests(TestCase):44class UtilTests(TestCase):
3045
46 def setUp(self):
47 self.original_stdout = sys.stdout
48 sys.stdout = open("/dev/null", "w")
49 self.original_stderr = sys.stderr
50 sys.stderr = open("/dev/null", "w")
51 self.original_stdin = sys.stdin
52 self.temp_file = tempfile.NamedTemporaryFile(delete=False)
53
54 def tearDown(self):
55 sys.stdin = self.original_stdin
56 sys.stdout = self.original_stdout
57 sys.stderr = self.original_stderr
58 os.unlink(self.temp_file.name)
59
31 @patch("lava_tool.utils.subprocess.check_call")60 @patch("lava_tool.utils.subprocess.check_call")
32 def test_has_command_0(self, mocked_check_call):61 def test_has_command_0(self, mocked_check_call):
33 # Make sure we raise an exception when the subprocess is called.62 # Make sure we raise an exception when the subprocess is called.
@@ -39,3 +68,149 @@
39 # Check that a "command" exists. The call to subprocess is mocked.68 # Check that a "command" exists. The call to subprocess is mocked.
40 mocked_check_call.return_value = 069 mocked_check_call.return_value = 0
41 self.assertTrue(has_command(""))70 self.assertTrue(has_command(""))
71
72 def test_verify_file_extension_with_extension(self):
73 extension = ".test"
74 supported = [extension[1:]]
75 try:
76 temp_file = tempfile.NamedTemporaryFile(suffix=extension,
77 delete=False)
78 obtained = verify_file_extension(
79 temp_file.name, extension[1:], supported)
80 self.assertEquals(temp_file.name, obtained)
81 finally:
82 if os.path.isfile(temp_file.name):
83 os.unlink(temp_file.name)
84
85 def test_verify_file_extension_without_extension(self):
86 extension = "json"
87 supported = [extension]
88 expected = "/tmp/a_fake.json"
89 obtained = verify_file_extension("/tmp/a_fake", extension, supported)
90 self.assertEquals(expected, obtained)
91
92 def test_verify_file_extension_with_unsupported_extension(self):
93 extension = "json"
94 supported = [extension]
95 expected = "/tmp/a_fake.json"
96 obtained = verify_file_extension(
97 "/tmp/a_fake.extension", extension, supported)
98 self.assertEquals(expected, obtained)
99
100 @patch("os.listdir")
101 def test_retrieve_job_file_0(self, mocked_os_listdir):
102 # Make sure that exception is raised if we go through all the elements
103 # returned by os.listdir().
104 mocked_os_listdir.return_value = ["a_file"]
105 self.assertRaises(CommandError, retrieve_file,
106 "a_path", ["ext"])
107
108 @patch("os.listdir")
109 def test_retrieve_job_file_1(self, mocked_os_listdir):
110 # Pass some files and directories to retrieve_file(), and make
111 # sure a file with .json suffix is returned.
112 # Pass also a hidden file.
113 try:
114 json_file = tempfile.NamedTemporaryFile(suffix=".json")
115 json_file_name = os.path.basename(json_file.name)
116
117 file_name_no_suffix = tempfile.NamedTemporaryFile(delete=False)
118 file_name_with_suffix = tempfile.NamedTemporaryFile(
119 suffix=".bork", delete=False)
120
121 temp_dir_name = "submit_command_test_tmp_dir"
122 temp_dir_path = os.path.join(tempfile.gettempdir(), temp_dir_name)
123 os.makedirs(temp_dir_path)
124
125 hidden_file = tempfile.NamedTemporaryFile(
126 prefix=".tmp", delete=False)
127
128 mocked_os_listdir.return_value = [
129 temp_dir_name, file_name_no_suffix.name,
130 file_name_with_suffix.name, json_file_name, hidden_file.name]
131
132 obtained = retrieve_file(tempfile.gettempdir(), ["json"])
133 self.assertEqual(json_file.name, obtained)
134 finally:
135 os.removedirs(temp_dir_path)
136 os.unlink(file_name_no_suffix.name)
137 os.unlink(file_name_with_suffix.name)
138 os.unlink(hidden_file.name)
139
140 @patch("lava_tool.utils.subprocess")
141 def test_execute_0(self, mocked_subprocess):
142 mocked_subprocess.check_call = MagicMock()
143 execute("foo")
144 self.assertEqual(mocked_subprocess.check_call.call_args_list,
145 [call(["foo"])])
146 self.assertTrue(mocked_subprocess.check_call.called)
147
148 @patch("lava_tool.utils.subprocess.check_call")
149 def test_execute_1(self, mocked_check_call):
150 mocked_check_call.side_effect = subprocess.CalledProcessError(1, "foo")
151 self.assertRaises(CommandError, execute, ["foo"])
152
153 @patch("lava_tool.utils.subprocess")
154 @patch("lava_tool.utils.has_command", return_value=False)
155 @patch("lava_tool.utils.os.environ.get", return_value=None)
156 @patch("lava_tool.utils.sys.exit")
157 def test_edit_file_0(self, mocked_sys_exit, mocked_env_get,
158 mocked_has_command, mocked_subprocess):
159 edit_file(self.temp_file.name)
160 self.assertTrue(mocked_sys_exit.called)
161
162 @patch("lava_tool.utils.subprocess")
163 @patch("lava_tool.utils.has_command", side_effect=[True, False])
164 @patch("lava_tool.utils.os.environ.get", return_value=None)
165 def test_edit_file_1(self, mocked_env_get, mocked_has_command,
166 mocked_subprocess):
167 mocked_subprocess.Popen = MagicMock()
168 edit_file(self.temp_file.name)
169 expected = [call(["sensible-editor", self.temp_file.name])]
170 self.assertEqual(expected, mocked_subprocess.Popen.call_args_list)
171
172 @patch("lava_tool.utils.subprocess")
173 @patch("lava_tool.utils.has_command", side_effect=[False, True])
174 @patch("lava_tool.utils.os.environ.get", return_value=None)
175 def test_edit_file_2(self, mocked_env_get, mocked_has_command,
176 mocked_subprocess):
177 mocked_subprocess.Popen = MagicMock()
178 edit_file(self.temp_file.name)
179 expected = [call(["xdg-open", self.temp_file.name])]
180 self.assertEqual(expected, mocked_subprocess.Popen.call_args_list)
181
182 @patch("lava_tool.utils.subprocess")
183 @patch("lava_tool.utils.has_command", return_value=False)
184 @patch("lava_tool.utils.os.environ.get", return_value="vim")
185 def test_edit_file_3(self, mocked_env_get, mocked_has_command,
186 mocked_subprocess):
187 mocked_subprocess.Popen = MagicMock()
188 edit_file(self.temp_file.name)
189 expected = [call(["vim", self.temp_file.name])]
190 self.assertEqual(expected, mocked_subprocess.Popen.call_args_list)
191
192 @patch("lava_tool.utils.subprocess")
193 @patch("lava_tool.utils.has_command", return_value=False)
194 @patch("lava_tool.utils.os.environ.get", return_value="vim")
195 def test_edit_file_4(self, mocked_env_get, mocked_has_command,
196 mocked_subprocess):
197 mocked_subprocess.Popen = MagicMock()
198 mocked_subprocess.Popen.side_effect = Exception()
199 self.assertRaises(CommandError, edit_file, self.temp_file.name)
200
201 def test_can_edit_file(self):
202 # Tests the can_edit_file method of the config command.
203 # This is to make sure the device config file is not erased when
204 # checking if it is possible to open it.
205 expected = ("hostname = a_fake_panda02\nconnection_command = \n"
206 "device_type = panda\n")
207
208 with open(self.temp_file.name, "w") as f:
209 f.write(expected)
210
211 self.assertTrue(can_edit_file(self.temp_file.name))
212 obtained = ""
213 with open(self.temp_file.name) as f:
214 obtained = f.read()
215
216 self.assertEqual(expected, obtained)
42217
=== modified file 'lava_tool/utils.py'
--- lava_tool/utils.py 2013-07-25 15:43:25 +0000
+++ lava_tool/utils.py 2013-07-25 15:43:26 +0000
@@ -16,8 +16,17 @@
16# You should have received a copy of the GNU Lesser General Public License16# You should have received a copy of the GNU Lesser General Public License
17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.17# along with lava-tool. If not, see <http://www.gnu.org/licenses/>.
1818
19import StringIO
20import base64
19import os21import os
22import tarfile
23import tempfile
24import types
20import subprocess25import subprocess
26import sys
27import urlparse
28
29from lava.tool.errors import CommandError
2130
2231
23def has_command(command):32def has_command(command):
@@ -32,3 +41,280 @@
32 except subprocess.CalledProcessError:41 except subprocess.CalledProcessError:
33 command_available = False42 command_available = False
34 return command_available43 return command_available
44
45
46def to_list(value):
47 """Return a list from the passed value.
48
49 :param value: The parameter to turn into a list.
50 """
51 return_value = []
52 if isinstance(value, types.StringType):
53 return_value = [value]
54 else:
55 return_value = list(value)
56 return return_value
57
58
59def create_tar(paths):
60 """Creates a temporary tar file with the provided paths.
61
62 The tar file is not deleted at the end, it has to be delete by who calls
63 this function.
64
65 If just a directory is passed, it will be flattened out: its contents will
66 be added, but not the directory itself.
67
68 :param paths: List of paths to be included in the tar archive.
69 :type list
70 :return The path to the temporary tar file.
71 """
72 paths = to_list(paths)
73 try:
74 temp_tar_file = tempfile.NamedTemporaryFile(suffix=".tar",
75 delete=False)
76 with tarfile.open(temp_tar_file.name, "w") as tar_file:
77 for path in paths:
78 full_path = os.path.abspath(path)
79 if os.path.isfile(full_path):
80 arcname = os.path.basename(full_path)
81 tar_file.add(full_path, arcname=arcname)
82 elif os.path.isdir(full_path):
83 # If we pass a directory, flatten it out.
84 # List its contents, and add them as they are.
85 for element in os.listdir(full_path):
86 arcname = element
87 tar_file.add(os.path.join(full_path, element),
88 arcname=arcname)
89 return temp_tar_file.name
90 except tarfile.TarError:
91 raise CommandError("Error creating the temporary tar archive.")
92
93
94def base64_encode(path):
95 """Encode in base64 the provided file.
96
97 :param path: The path to a file.
98 :return The file content encoded in base64.
99 """
100 if os.path.isfile(path):
101 encoded_content = StringIO.StringIO()
102
103 try:
104 with open(path) as read_file:
105 base64.encode(read_file, encoded_content)
106
107 return encoded_content.getvalue().strip()
108 except IOError:
109 raise CommandError("Cannot read file "
110 "'{0}'.".format(path))
111 else:
112 raise CommandError("Provided path does not exists or is not a file: "
113 "{0}.".format(path))
114
115
116def retrieve_file(path, extensions):
117 """Searches for a file that has one of the supported extensions.
118
119 The path of the first file that matches one of the supported provided
120 extensions will be returned. The files are examined in alphabetical
121 order.
122
123 :param path: Where to look for the file.
124 :param extensions: A list of extensions the file to look for should
125 have.
126 :return The full path of the file.
127 """
128 if os.path.isfile(path):
129 if check_valid_extension(path, extensions):
130 retrieved_path = path
131 else:
132 raise CommandError("The provided file '{0}' is not "
133 "valid: extension not supported.".format(path))
134 else:
135 dir_listing = os.listdir(path)
136 dir_listing.sort()
137
138 for element in dir_listing:
139 if element.startswith("."):
140 continue
141
142 element_path = os.path.join(path, element)
143 if os.path.isdir(element_path):
144 continue
145 elif os.path.isfile(element_path):
146 if check_valid_extension(element_path, extensions):
147 retrieved_path = element_path
148 break
149 else:
150 raise CommandError("No suitable file found in '{0}'".format(path))
151
152 return retrieved_path
153
154
155def check_valid_extension(path, extensions):
156 """Checks that a file has one of the supported extensions.
157
158 :param path: The file to check.
159 :param extensions: A list of supported extensions.
160 """
161 is_valid = False
162
163 local_path, file_name = os.path.split(path)
164 name, full_extension = os.path.splitext(file_name)
165
166 if full_extension:
167 extension = full_extension[1:].strip().lower()
168 if extension in extensions:
169 is_valid = True
170 return is_valid
171
172
173def verify_file_extension(path, default, supported):
174 """Verifies if a file has a supported extensions.
175
176 If the file does not have one, it will add the default extension
177 provided.
178
179 :param path: The path of a file to verify.
180 :param default: The default extension to use.
181 :param supported: A list of supported extensions to check against.
182 :return The path of the file.
183 """
184 full_path, file_name = os.path.split(path)
185 name, extension = os.path.splitext(file_name)
186 if not extension:
187 path = ".".join([path, default])
188 elif extension[1:].lower() not in supported:
189 path = os.path.join(full_path, ".".join([name, default]))
190 return path
191
192
193def verify_path_existance(path):
194 """Verifies if a given path exists or not on the file system.
195
196 Raises a CommandError in case it exists.
197
198 :param path: The path to verify."""
199 if os.path.exists(path):
200 raise CommandError("{0} already exists.".format(path))
201
202
203def write_file(path, content):
204 """Creates a file with the specified content.
205
206 :param path: The path of the file to write.
207 :param content: What to write in the file.
208 """
209 try:
210 with open(path, "w") as to_write:
211 to_write.write(content)
212 except (OSError, IOError):
213 raise CommandError("Error writing file '{0}'".format(path))
214
215
216def execute(cmd_args):
217 """Executes the supplied command args.
218
219 :param cmd_args: The command, and its optional arguments, to run.
220 :return The command execution return code.
221 """
222 cmd_args = to_list(cmd_args)
223 try:
224 return subprocess.check_call(cmd_args)
225 except subprocess.CalledProcessError:
226 raise CommandError("Error running the following command: "
227 "{0}".format(" ".join(cmd_args)))
228
229
230def can_edit_file(path):
231 """Checks if a file can be opend in write mode.
232
233 :param path: The path to the file.
234 :return True if it is possible to write on the file, False otherwise.
235 """
236 can_edit = True
237 try:
238 fp = open(path, "a")
239 fp.close()
240 except IOError:
241 can_edit = False
242 return can_edit
243
244
245def edit_file(file_to_edit):
246 """Opens the specified file with the default file editor.
247
248 :param file_to_edit: The file to edit.
249 """
250 editor = os.environ.get("EDITOR", None)
251 if editor is None:
252 if has_command("sensible-editor"):
253 editor = "sensible-editor"
254 elif has_command("xdg-open"):
255 editor = "xdg-open"
256 else:
257 # We really do not know how to open a file.
258 print >> sys.stdout, ("Cannot find an editor to open the "
259 "file '{0}'.".format(file_to_edit))
260 print >> sys.stdout, ("Either set the 'EDITOR' environment "
261 "variable, or install 'sensible-editor' "
262 "or 'xdg-open'.")
263 sys.exit(-1)
264 try:
265 subprocess.Popen([editor, file_to_edit]).wait()
266 except Exception:
267 raise CommandError("Error opening the file '{0}' with the "
268 "following editor: {1}.".format(file_to_edit,
269 editor))
270
271
272def verify_and_create_url(server, endpoint=""):
273 """Checks that the provided values make a correct URL.
274
275 If the server address does not contain a scheme, by default it will use
276 HTTPS.
277 The endpoint is then added at the URL.
278
279 :param server: A server URL to verify.
280 :return A URL.
281 """
282 scheme, netloc, path, params, query, fragment = \
283 urlparse.urlparse(server)
284 if not scheme:
285 scheme = "https"
286 if not netloc:
287 netloc, path = path, ""
288
289 if not netloc[-1:] == "/":
290 netloc += "/"
291
292 if endpoint:
293 if endpoint[0] == "/":
294 endpoint = endpoint[1:]
295 if not endpoint[-1:] == "/":
296 endpoint += "/"
297 netloc += endpoint
298
299 return urlparse.urlunparse(
300 (scheme, netloc, path, params, query, fragment))
301
302
303def create_dir(path, dir_name=None):
304 """Checks if a directory does not exists, and creates it.
305
306 :param path: The path where the directory should be created.
307 :param dir_name: An optional name for a directory to be created at
308 path (dir_name will be joined with path).
309 :return The path of the created directory."""
310 created_dir = path
311 if dir_name:
312 created_dir = os.path.join(path, dir_name)
313
314 if not os.path.isdir(created_dir):
315 try:
316 os.makedirs(created_dir)
317 except OSError:
318 raise CommandError("Cannot create directory "
319 "'{0}'.".format(created_dir))
320 return created_dir
35321
=== modified file 'setup.py'
--- setup.py 2013-07-25 15:43:25 +0000
+++ setup.py 2013-07-25 15:43:26 +0000
@@ -46,6 +46,7 @@
46 "Topic :: Software Development :: Testing",46 "Topic :: Software Development :: Testing",
47 ],47 ],
48 install_requires=[48 install_requires=[
49 'PyYAML >= 3.10',
49 'argparse >= 1.1',50 'argparse >= 1.1',
50 'argcomplete >= 0.3',51 'argcomplete >= 0.3',
51 'keyring',52 'keyring',

Subscribers

People subscribed via source and target branches