Merge lp:~nuclearbob/utah/docstring-cleanup into lp:utah

Proposed by Max Brustkern
Status: Merged
Merged at revision: 855
Proposed branch: lp:~nuclearbob/utah/docstring-cleanup
Merge into: lp:utah
Diff against target: 7966 lines (+1933/-2048)
77 files modified
client.py (+7/-8)
debian/changelog (+18/-1)
docs/source/conf.py (+22/-14)
examples/run_install_test.py (+4/-1)
examples/run_test_bamboo_feeder.py (+4/-1)
examples/run_test_cobbler.py (+4/-1)
examples/run_test_vm.py (+4/-1)
examples/run_utah_tests.py (+4/-1)
tests/test_rsyslog.py (+16/-17)
utah-done.py (+1/-0)
utah/__init__.py (+1/-3)
utah/cleanup.py (+5/-4)
utah/client/__init__.py (+1/-3)
utah/client/battery.py (+21/-1)
utah/client/common.py (+59/-298)
utah/client/examples/examples/test_one/test_one.py (+3/-0)
utah/client/examples/examples/test_two/test_two.py (+3/-0)
utah/client/examples/utah_tests/test_one/test_one.py (+3/-0)
utah/client/examples/utah_tests/test_two/test_two.py (+3/-0)
utah/client/examples/utah_tests_sample/sample_one/sample.py (+3/-0)
utah/client/exceptions.py (+37/-26)
utah/client/phoenix.py (+2/-1)
utah/client/result.py (+73/-78)
utah/client/runner.py (+47/-54)
utah/client/state_agent.py (+23/-21)
utah/client/testcase.py (+43/-48)
utah/client/tests/__init__.py (+2/-0)
utah/client/tests/common.py (+19/-15)
utah/client/tests/manual_privileges.py (+0/-188)
utah/client/tests/test_common.py (+36/-21)
utah/client/tests/test_jsonschema.py (+10/-16)
utah/client/tests/test_misc.py (+8/-2)
utah/client/tests/test_phoenix.py (+18/-29)
utah/client/tests/test_result.py (+21/-22)
utah/client/tests/test_runner.py (+17/-18)
utah/client/tests/test_state_agent.py (+17/-21)
utah/client/tests/test_testcase.py (+20/-20)
utah/client/tests/test_testsuite.py (+19/-11)
utah/client/tests/test_vcs_bzr.py (+15/-18)
utah/client/tests/test_vcs_dev.py (+15/-18)
utah/client/tests/test_vcs_git.py (+15/-18)
utah/client/tests/test_yaml.py (+9/-0)
utah/client/testsuite.py (+39/-37)
utah/client/vcs.py (+144/-0)
utah/commandstr.py (+5/-7)
utah/config.py (+12/-10)
utah/exceptions.py (+8/-7)
utah/group.py (+7/-11)
utah/iso.py (+108/-67)
utah/isotest/iso_static_validation.py (+18/-22)
utah/orderedcollections.py (+14/-3)
utah/parser.py (+2/-0)
utah/preseed.py (+6/-0)
utah/process.py (+2/-0)
utah/provisioning/__init__.py (+1/-3)
utah/provisioning/baremetal/__init__.py (+1/-3)
utah/provisioning/baremetal/bamboofeeder.py (+64/-76)
utah/provisioning/baremetal/cobbler.py (+91/-72)
utah/provisioning/baremetal/exceptions.py (+5/-6)
utah/provisioning/baremetal/inventory.py (+38/-19)
utah/provisioning/baremetal/power.py (+20/-22)
utah/provisioning/exceptions.py (+7/-6)
utah/provisioning/inventory/__init__.py (+1/-3)
utah/provisioning/inventory/exceptions.py (+5/-6)
utah/provisioning/inventory/inventory.py (+33/-34)
utah/provisioning/inventory/sqlite.py (+10/-7)
utah/provisioning/provisioning.py (+347/-258)
utah/provisioning/rsyslog.py (+2/-2)
utah/provisioning/ssh.py (+57/-35)
utah/provisioning/vm/__init__.py (+1/-3)
utah/provisioning/vm/exceptions.py (+5/-6)
utah/provisioning/vm/vm.py (+147/-145)
utah/publish.py (+0/-146)
utah/retry.py (+3/-3)
utah/run.py (+55/-15)
utah/timeout.py (+17/-15)
utah/url.py (+6/-0)
To merge this branch: bzr merge lp:~nuclearbob/utah/docstring-cleanup
Reviewer Review Type Date Requested Status
Andy Doan (community) Approve
Max Brustkern (community) Needs Resubmitting
Review via email: mp+157162@code.launchpad.net

Description of the change

This branch cleans up docstrings to be PEP257 compliant, updates string formatting, and addresses a number of minor issues raised during the code review while attempting to minimize potential functionality changes.

To post a comment you must log in.
Revision history for this message
Max Brustkern (nuclearbob) wrote :

I'm still testing provisioning and some client self tests are failing, but I'd still like to get some other eyes on this since it's a big one.

Revision history for this message
Max Brustkern (nuclearbob) wrote :

All the provisioning bugs are fixed now, so if anyone wants to look things over or help with the client self-test issues, that would be rad.

Revision history for this message
Andy Doan (doanac) wrote :

Looks good. How bout we add a unit test to cover the get_children_process change. here's something i hacked together:

 http://paste.ubuntu.com/5680415/

961. By Max Brustkern

Accidentally removed a function when converting to private, reinstating

962. By Max Brustkern

Converting pid to string before passing it to subprocess

963. By Max Brustkern

I misunderstood the difference between revision and get_revision, and get_revision was only used by tests, so I'm using the correct revision as revision now

Revision history for this message
Max Brustkern (nuclearbob) wrote :

I've fixed all but one of the client test issues. I'm still not raising MissingData, because we decided in the code review to take out the block of code that raises that. I think some other exception should be raised, however, and that doesn't appear to be happening. I think the lack of another exception is not a problem with this merge, but we should either fix that problem, or reinstate the MissingData block before moving forward. I'm in favor of fixing the problem, so I'm going to try for that.

964. By Max Brustkern

utah/client/tests/test_testcase.py

Revision history for this message
Max Brustkern (nuclearbob) wrote :

I've removed that test. The DefaultValidator isn't working, and I think we should have some different tests for it. I've opened a bug:
https://bugs.launchpad.net/utah/+bug/1165175
and I'll work on that separately. I think we should have probably at least three tests for the DefaultValidator:
test that it validates valid data
test that it fails to validate bad data
test that it populates a default if needed
and maybe
test that not supplying a default doesn't cause validation to fail?

Additionally, in test_testcase, I think we should test for:
missing field
extra field
bad reboot
bad timeout
bad type

I'll add that info to the bug as well.

965. By Max Brustkern

Removed unneeded exception and backed out my testing changes to DefaultValidator

966. By Max Brustkern

Fixed static analysis errors, added test suggested by Andy, removed correct exception

Revision history for this message
Max Brustkern (nuclearbob) wrote :

> Looks good. How bout we add a unit test to cover the get_children_process
> change. here's something i hacked together:
>
> http://paste.ubuntu.com/5680415/

Added that. Good idea.

Revision history for this message
Andy Doan (doanac) wrote :

common.py:validate_properties has some extra "print" statements that seem to be left over from debugging.

Revision history for this message
Andy Doan (doanac) wrote :

You removed the MissingData check:

        # Do not proceed if we are missing data
        if (self.description is None
                or self.dependencies is None
                or self.action is None
                or self.expected_results is None):
            raise MissingData

However, I discovered a bug the other data in this same place. Our DefaultValidator is broken and doesn't raise errors when this data is missing. So this can't be removed until we fix that code. I glanced at fixing it, but haven't gotten around to making it work. NOTE: we did our own DefaultValidator so that we could set default attributes on our object.

Revision history for this message
Andy Doan (doanac) wrote :

rsyslogd.py
            # TODO: Find out if this can/should be None
            self._port = 0

the TODO line that was added isn't needed. This line is required and safe.

Revision history for this message
Andy Doan (doanac) wrote :

should _UTAHISOException really be listed as private? We don't seem to catch this in our code, so I *think* its a user visible exception?

Revision history for this message
Max Brustkern (nuclearbob) wrote :

I'll fix the print statements.

I removed the MissingData check because we said in the code review that we wanted to remove that since we're using the validator now. The validator code that's in dev right now (and I think production) is already broken, and I don't think removing that check will make things worse. We can leave it in until that's fixed if you want, but either way, we should fix it soon, since we're currently behaving in confusing and bad ways when you try to run a bad tc_control file.

That TODO was for me, and I forget to get to it before I finished the merge. Whoops. I did want to ask you about that, when we're using a file, do we want port to be 0 or would it be better as None? Does the code to send the rsyslog arguments to the kernel command line still check if that value is None? I don't think we need those parameters if we're using a file, so I wanted to check if we were still setting them.

You make a good point about the exception. I think I went a little overboard making things private. I'll roll that back.

Revision history for this message
Andy Doan (doanac) wrote :

On 04/08/2013 11:03 AM, Max Brustkern wrote:
> I removed the MissingData check because we said in the code review
> that we wanted to remove that since we're using the validator now.
> The validator code that's in dev right now (and I think production)
> is already broken, and I don't think removing that check will make
> things worse.

If you remove this, then we need to assign those attributes default
values in the schema so that they get set. Otherwise, I think a "bad"
control file could cause us to try and reference variables that haven't
been initialized.

> We can leave it in until that's fixed if you want, but
> either way, we should fix it soon, since we're currently behaving in
> confusing and bad ways when you try to run a bad tc_control file.

I would like to fix it soon. Just not sure we want to fit it in this.

Revision history for this message
Andy Doan (doanac) wrote :

On 04/08/2013 11:03 AM, Max Brustkern wrote:
> That TODO was for me, and I forget to get to it before I finished the
> merge. Whoops. I did want to ask you about that, when we're using a
> file, do we want port to be 0 or would it be better as None? Does
> the code to send the rsyslog arguments to the kernel command line
> still check if that value is None? I don't think we need those
> parameters if we're using a file, so I wanted to check if we were
> still setting them.

I think it can be None or 0. If you look at provisioning.py we have a check:

  if self.rsyslog.port:
    ... (setup rsylog'ing)
  else:
    ... (set up serial console logging)

Maybe a smarter thing would be to make this more explicit like:

  if self.rsyslog.local_file()

not sure.

967. By Max Brustkern

Implemented some fixes suggested by Andy

Revision history for this message
Max Brustkern (nuclearbob) wrote :

> On 04/08/2013 11:03 AM, Max Brustkern wrote:
> > That TODO was for me, and I forget to get to it before I finished the
> > merge. Whoops. I did want to ask you about that, when we're using a
> > file, do we want port to be 0 or would it be better as None? Does
> > the code to send the rsyslog arguments to the kernel command line
> > still check if that value is None? I don't think we need those
> > parameters if we're using a file, so I wanted to check if we were
> > still setting them.
>
> I think it can be None or 0. If you look at provisioning.py we have a check:
>
> if self.rsyslog.port:
> ... (setup rsylog'ing)
> else:
> ... (set up serial console logging)
>
> Maybe a smarter thing would be to make this more explicit like:
>
> if self.rsyslog.local_file()
>
> not sure.

I think I originally wrote things as 'if config.rsyslogport is not None' and I forgot that you had improved on that by testing the value directly, which will return False on a 0. Good stuff. I implemented the three things you said other than the MissingData, which I'll comment on separately.

Revision history for this message
Max Brustkern (nuclearbob) wrote :

> On 04/08/2013 11:03 AM, Max Brustkern wrote:
> > I removed the MissingData check because we said in the code review
> > that we wanted to remove that since we're using the validator now.
> > The validator code that's in dev right now (and I think production)
> > is already broken, and I don't think removing that check will make
> > things worse.
>
> If you remove this, then we need to assign those attributes default
> values in the schema so that they get set. Otherwise, I think a "bad"
> control file could cause us to try and reference variables that haven't
> been initialized.
>
> > We can leave it in until that's fixed if you want, but
> > either way, we should fix it soon, since we're currently behaving in
> > confusing and bad ways when you try to run a bad tc_control file.
>
> I would like to fix it soon. Just not sure we want to fit it in this.

My feeling is, this is already broken. I don't see a MissingData exception get raised when I feed in a file with bad data, even without that change in place, so I don't think that code is doing anything useful. Maybe that means we should leave it in until the branch that fixes it (which I'm hoping to propose today or tomorrow.) As I'm typing this, that actually makes more sense. If nobody disagrees, I can put that block and the test back in. That will mean the test is failing, but the correction for that is something we're going to want as a bugfix anyway, so landing it separately makes sense.

Revision history for this message
Andy Doan (doanac) wrote :

On 04/08/2013 12:33 PM, Max Brustkern wrote:
> My feeling is, this is already broken. I don't see a MissingData
> exception get raised when I feed in a file with bad data, even
> without that change in place, so I don't think that code is doing
> anything useful.

hmm - I've seen it. it was what prompted me to start on:

  https://code.launchpad.net/~doanac/utah/schema-bug/+merge/156215

> Maybe that means we should leave it in until the
> branch that fixes it (which I'm hoping to propose today or tomorrow.)
> As I'm typing this, that actually makes more sense. If nobody
> disagrees, I can put that block and the test back in. That will mean
> the test is failing, but the correction for that is something we're
> going to want as a bugfix anyway, so landing it separately makes
> sense.

okay

Revision history for this message
Andy Doan (doanac) wrote :

+1

968. By Max Brustkern

Put MissingData bits back in. We'll resolve that in a separate branch.

969. By Max Brustkern

Static analysis fixed

Revision history for this message
Max Brustkern (nuclearbob) wrote :

Okay, all of the issues you raised should be fixed now. We can handle the MissingData thing with a new validator in a separate branch.

review: Needs Resubmitting
Revision history for this message
Andy Doan (doanac) wrote :

+1 for me.

review: Approve
Revision history for this message
Javier Collado (javier.collado) wrote :

Thanks for all the effort on this branch.

The changes look good and I've been able to successfully execute the pass
runlist. However, there are still a few issues reported by the pep257 tool:
------------
provisioning.py:76:8: PEP257 Return value type should be mentioned.
runner.py:415:8: PEP257 Return value type should be mentioned.
setup.py:1:0: PEP257 Modules should have docstrings.
------------

I don't think they are really important, but if you could fix them before
merging to the development branch, that would be great.

Revision history for this message
Max Brustkern (nuclearbob) wrote :

I didn't see this before I merged, but the first two are false positives. Those functions both define functions internally that have return statements, but those functions themselves do not have return statements. I think we should separate the internal functions to get rid of those warnings.

As to setup.py, I was mostly ignoring that, along with distribute_setup.py, but I forgot we actually do maintain setup.py. I'll address that issue before release, and we can take care of the other ones later since they involve restructuing.

Revision history for this message
Javier Collado (javier.collado) wrote :

That's interesting, I didn't know why the error was triggered just for some
methods.

Looking at them I see that:
- one of them could be a function at the module level because is defined there
  just for the convenience of having it close to where it's used, but it's not
  really part of the method.
- the other one uses a variable from the outer scope. Hence the signature must
  be change if it's moved as an instance method in the class itself.

I don't like much the second bullet (changing the signature), so I've made a
couple of tests to verify that adding 'return' to the documentation is enough
to get rid of the error message. Consequently, if we stick to use sphinx
markup, then we could just use `:returns: None` after all the `:param:` and
`:type:` tags and the error message would be gone. What do you think about this
option?

By the way, some other detail is that the private methods are not added by
default to the documentation, which I think is fine, but that means that the
documentation for `__init__` sometimes has to be moved to the class docstring
to make it available in the documentation. I've done that for a couple of
classes in the code and I think it applies to the `Machine` class as well.

Revision history for this message
Max Brustkern (nuclearbob) wrote :

I don't like that second bullet either. `:returns: None` sounds like a good solution.

That makes me wonder what, if any, other private methods we do want documented. We can make another merge proposal to move __init__ documentation up to the class level for things that need it.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'client.py'
--- client.py 2013-04-02 13:56:33 +0000
+++ client.py 2013-04-08 19:17:22 +0000
@@ -15,6 +15,9 @@
15# You should have received a copy of the GNU General Public License along15# You should have received a copy of the GNU General Public License along
16# with this program. If not, see <http://www.gnu.org/licenses/>.16# with this program. If not, see <http://www.gnu.org/licenses/>.
1717
18"UTAH client used to run tests."""
19
20
18import logging21import logging
19import platform22import platform
20import sys23import sys
@@ -33,9 +36,7 @@
3336
3437
35def get_parser():38def get_parser():
36 """39 """Process the command line arguments."""
37 Process the command line arguments.
38 """
39 # requires Python2.7+40 # requires Python2.7+
40 import argparse41 import argparse
4142
@@ -48,8 +49,8 @@
48 parser.add_argument('--resume', action='store_true',49 parser.add_argument('--resume', action='store_true',
49 help='Continue a previous run. Used after a reboot')50 help='Continue a previous run. Used after a reboot')
50 parser.add_argument('-s', '--state-file',51 parser.add_argument('-s', '--state-file',
51 help=('File to use for storing state (default "%s")'52 help=('File to use for storing state (default "{}")'
52 % DEFAULT_STATE_FILE))53 .format(DEFAULT_STATE_FILE)))
53 parser.add_argument('-f', '--format',54 parser.add_argument('-f', '--format',
54 choices=['text', 'yaml', 'json'],55 choices=['text', 'yaml', 'json'],
55 default='yaml',56 default='yaml',
@@ -87,9 +88,7 @@
8788
8889
89def main():90def main():
90 """91 """Interpret command line arguments and run tests."""
91 The main driver for the 'utah' application.
92 """
93 # TODO: write <2.7 optparse version and set based on version of python92 # TODO: write <2.7 optparse version and set based on version of python
94 # being used.93 # being used.
95 parser = get_parser()94 parser = get_parser()
9695
=== modified file 'debian/changelog'
--- debian/changelog 2013-04-04 14:58:12 +0000
+++ debian/changelog 2013-04-08 19:17:22 +0000
@@ -1,5 +1,6 @@
1utah (0.10ubuntu1) UNRELEASED; urgency=low1utah (0.10ubuntu1) UNRELEASED; urgency=low
22
3 [ Javier Collado ]
3 * Return error code on unhandled error (LP: #1160857)4 * Return error code on unhandled error (LP: #1160857)
4 * Remove temporary files downloaded based on URL (LP: #1101186)5 * Remove temporary files downloaded based on URL (LP: #1101186)
5 * Stop server on installation failure (LP: #1161855)6 * Stop server on installation failure (LP: #1161855)
@@ -9,7 +10,23 @@
9 * Use sys.exit when SIGTERM is received (LP: #1160247)10 * Use sys.exit when SIGTERM is received (LP: #1160247)
10 * Write syslog pattern matched and timeouts timestamps (LP: #1162862)11 * Write syslog pattern matched and timeouts timestamps (LP: #1162862)
1112
12 -- Javier Collado <javier.collado@canonical.com> Wed, 27 Mar 2013 13:00:47 +010013 [ Max Brustkern ]
14 * Cleaned up docstrings to be PEP257 compliant
15 * Improved string formatting
16 * Harmonized Machine provision method signatures and returns
17 * Separated VCS handling into its own file and simplified revision
18 handling
19 * Privatized public functions
20 * Removed publishing code
21 * Cleaned up import ordering
22 * Refactored kernel command line processing
23 * Added local copy of specified kernel and initrd
24 * Removed useless comments
25 * Improved exception handling
26 * Improved logging
27 * Cleaned up issues raised during code review
28
29 -- Max Brustkern <max@canonical.com> Thu, 04 Apr 2013 12:29:54 -0400
1330
14utah (0.9.2ubuntu1) quantal; urgency=low31utah (0.9.2ubuntu1) quantal; urgency=low
1532
1633
=== modified file 'docs/source/conf.py'
--- docs/source/conf.py 2012-12-06 16:08:13 +0000
+++ docs/source/conf.py 2013-04-08 19:17:22 +0000
@@ -28,10 +28,13 @@
28# All configuration values have a default; values that are commented out28# All configuration values have a default; values that are commented out
29# serve to show the default.29# serve to show the default.
3030
31import sys31"""Provide documentation configuration."""
32
33
34import itertools
32import os35import os
33import itertools
34import re36import re
37import sys
3538
36# If extensions (or modules to document with autodoc) are in another directory,39# If extensions (or modules to document with autodoc) are in another directory,
37# add these directories to sys.path here. If the directory is relative to the40# add these directories to sys.path here. If the directory is relative to the
@@ -41,10 +44,9 @@
4144
4245
43class ModuleMock(object):46class ModuleMock(object):
44 """47
45 Mock class to avoid import errors when building the documentation in48 """Mock class to avoid import errors."""
46 readthedocs.org49
47 """
48 def __init__(self, name):50 def __init__(self, name):
49 self.__name__ = name51 self.__name__ = name
5052
@@ -69,10 +71,9 @@
6971
7072
71class ClassMock(object):73class ClassMock(object):
72 """74
73 Mock class to avoid module member access errors when building the75 """Mock class to avoid import errors."""
74 documentation in readthedocs.org76
75 """
76 def __init__(self, *args, **kwargs):77 def __init__(self, *args, **kwargs):
77 pass78 pass
7879
@@ -84,8 +85,13 @@
84 sys.modules[mod_name] = ModuleMock(mod_name)85 sys.modules[mod_name] = ModuleMock(mod_name)
8586
8687
87# Autogenerate help contents for manual pages when building manual pages
88def get_parser_strings(script_name, parser):88def get_parser_strings(script_name, parser):
89 """Autogenerate help contents for manual pages when building them.
90
91 :returns: description, options, and epilog
92 :rtype: tuple
93
94 """
89 description = parser.description95 description = parser.description
9096
91 help_lines = parser.format_help().splitlines()97 help_lines = parser.format_help().splitlines()
@@ -98,9 +104,11 @@
98 options_lines.next() # Drop first line104 options_lines.next() # Drop first line
99105
100 def fix_line(line):106 def fix_line(line):
101 """107 """Format argparse output.
102 Remove indentation and replace curly braces with angle brackets108
103 to match the expected format109 Remove indentation and replace curly braces with angle brackets to
110 match the expected format.
111
104 """112 """
105 line = line[2:]113 line = line[2:]
106114
107115
=== modified file 'examples/run_install_test.py'
--- examples/run_install_test.py 2013-04-03 14:25:21 +0000
+++ examples/run_install_test.py 2013-04-08 19:17:22 +0000
@@ -1,4 +1,4 @@
1#!/usr/bin/python1#!/usr/bin/env python
22
3# Ubuntu Testing Automation Harness3# Ubuntu Testing Automation Harness
4# Copyright 2012 Canonical Ltd.4# Copyright 2012 Canonical Ltd.
@@ -15,6 +15,9 @@
15# You should have received a copy of the GNU General Public License along15# You should have received a copy of the GNU General Public License along
16# with this program. If not, see <http://www.gnu.org/licenses/>.16# with this program. If not, see <http://www.gnu.org/licenses/>.
1717
18"""Create a VM and run a test."""
19
20
18import argparse21import argparse
19import logging22import logging
20import sys23import sys
2124
=== modified file 'examples/run_test_bamboo_feeder.py'
--- examples/run_test_bamboo_feeder.py 2013-04-03 14:25:21 +0000
+++ examples/run_test_bamboo_feeder.py 2013-04-08 19:17:22 +0000
@@ -1,4 +1,4 @@
1#!/usr/bin/python1#!/usr/bin/env python
22
3# Ubuntu Testing Automation Harness3# Ubuntu Testing Automation Harness
4# Copyright 2012 Canonical Ltd.4# Copyright 2012 Canonical Ltd.
@@ -15,6 +15,9 @@
15# You should have received a copy of the GNU General Public License along15# You should have received a copy of the GNU General Public License along
16# with this program. If not, see <http://www.gnu.org/licenses/>.16# with this program. If not, see <http://www.gnu.org/licenses/>.
1717
18"""Provision a panda board in a bamboo-feeder setup and run a test."""
19
20
18import argparse21import argparse
19import os22import os
20import sys23import sys
2124
=== modified file 'examples/run_test_cobbler.py'
--- examples/run_test_cobbler.py 2013-04-03 14:25:21 +0000
+++ examples/run_test_cobbler.py 2013-04-08 19:17:22 +0000
@@ -1,4 +1,4 @@
1#!/usr/bin/python1#!/usr/bin/env python
22
3# Ubuntu Testing Automation Harness3# Ubuntu Testing Automation Harness
4# Copyright 2012 Canonical Ltd.4# Copyright 2012 Canonical Ltd.
@@ -15,6 +15,9 @@
15# You should have received a copy of the GNU General Public License along15# You should have received a copy of the GNU General Public License along
16# with this program. If not, see <http://www.gnu.org/licenses/>.16# with this program. If not, see <http://www.gnu.org/licenses/>.
1717
18"""Provision a machine with cobbler and run a test."""
19
20
18import argparse21import argparse
19import sys22import sys
20import logging23import logging
2124
=== modified file 'examples/run_test_vm.py'
--- examples/run_test_vm.py 2013-04-03 14:25:21 +0000
+++ examples/run_test_vm.py 2013-04-08 19:17:22 +0000
@@ -1,4 +1,4 @@
1#!/usr/bin/python1#!/usr/bin/env python
22
3# Ubuntu Testing Automation Harness3# Ubuntu Testing Automation Harness
4# Copyright 2012 Canonical Ltd.4# Copyright 2012 Canonical Ltd.
@@ -15,6 +15,9 @@
15# You should have received a copy of the GNU General Public License along15# You should have received a copy of the GNU General Public License along
16# with this program. If not, see <http://www.gnu.org/licenses/>.16# with this program. If not, see <http://www.gnu.org/licenses/>.
1717
18"""Create a VM and run a test."""
19
20
18import argparse21import argparse
19import logging22import logging
20import sys23import sys
2124
=== modified file 'examples/run_utah_tests.py'
--- examples/run_utah_tests.py 2013-04-03 14:25:21 +0000
+++ examples/run_utah_tests.py 2013-04-08 19:17:22 +0000
@@ -1,4 +1,4 @@
1#!/usr/bin/python1#!/usr/bin/env python
22
3# Ubuntu Testing Automation Harness3# Ubuntu Testing Automation Harness
4# Copyright 2012 Canonical Ltd.4# Copyright 2012 Canonical Ltd.
@@ -15,6 +15,9 @@
15# You should have received a copy of the GNU General Public License along15# You should have received a copy of the GNU General Public License along
16# with this program. If not, see <http://www.gnu.org/licenses/>.16# with this program. If not, see <http://www.gnu.org/licenses/>.
1717
18"""Provision a machine a run a test."""
19
20
18import argparse21import argparse
19import logging22import logging
20import sys23import sys
2124
=== modified file 'tests/test_rsyslog.py'
--- tests/test_rsyslog.py 2013-03-29 16:22:16 +0000
+++ tests/test_rsyslog.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test syslog processing functionality."""
17
18
16import logging19import logging
17import socket20import socket
18import time21import time
@@ -25,13 +28,17 @@
2528
2629
27class TestRsyslog(unittest.TestCase):30class TestRsyslog(unittest.TestCase):
31
32 """Test syslog processing code."""
33
28 logger = logging.getLogger()34 logger = logging.getLogger()
29 logger.setLevel(logging.DEBUG)35 logger.setLevel(logging.DEBUG)
30 logging.getLogger('rsyslog').addHandler(logging.StreamHandler())36 logging.getLogger('rsyslog').addHandler(logging.StreamHandler())
3137
32 @staticmethod38 @staticmethod
33 def producer(port, messages):39 def producer(port, messages):
34 print 'sending fake messages to port: %d' % port40 """Send messages to a port for testing."""
41 print('sending fake messages to port: {}'.format(port))
35 s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)42 s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
36 s.connect(('localhost', port))43 s.connect(('localhost', port))
37 for m in messages:44 for m in messages:
@@ -40,6 +47,7 @@
4047
41 @staticmethod48 @staticmethod
42 def file_producer(f, messages, truncate=False):49 def file_producer(f, messages, truncate=False):
50 """Produce a file for testing."""
43 for m in messages:51 for m in messages:
44 f.write(m)52 f.write(m)
45 f.write('\n')53 f.write('\n')
@@ -52,9 +60,7 @@
52 truncate = False60 truncate = False
5361
54 def test_easy(self):62 def test_easy(self):
55 """63 """Make sure we can match a typical first message."""
56 minimal test to make sure we can match a typical first message
57 """
58 steps = [64 steps = [
59 {65 {
60 "message": "test_easy",66 "message": "test_easy",
@@ -94,10 +100,7 @@
94 self.assertRaises(UTAHException, r.wait_for_install, steps)100 self.assertRaises(UTAHException, r.wait_for_install, steps)
95101
96 def test_future(self):102 def test_future(self):
97 """103 """Test handling a missing message."""
98 test to make sure we can handle missing a message and understanding
99 where things are in the future
100 """
101 steps = [104 steps = [
102 {105 {
103 "message": "test_future",106 "message": "test_future",
@@ -134,9 +137,7 @@
134 self.assertTrue(self.test_future_booted, 'booted callback not made')137 self.assertTrue(self.test_future_booted, 'booted callback not made')
135138
136 def test_callbacks(self):139 def test_callbacks(self):
137 """140 """Make sure wait_for_steps works."""
138 minimal test to make sure wait_for_steps works
139 """
140 steps = [141 steps = [
141 {142 {
142 "message": "test callbacks",143 "message": "test callbacks",
@@ -182,9 +183,7 @@
182 self.assertTrue(self.test_callbacks_blah, 'blah callback not made')183 self.assertTrue(self.test_callbacks_blah, 'blah callback not made')
183184
184 def test_booted(self):185 def test_booted(self):
185 """186 """Make sure wait_for_booted works."""
186 minimal test to make sure wait_for_booted works
187 """
188 steps = [187 steps = [
189 {188 {
190 "message": "test booted",189 "message": "test booted",
@@ -203,9 +202,7 @@
203 r.wait_for_booted(steps)202 r.wait_for_booted(steps)
204203
205 def usefile(self, truncate):204 def usefile(self, truncate):
206 """205 """Make sure wait_for_booted works when tailing a file."""
207 minimal test to make sure wait_for_booted works when tailing a file
208 """
209 steps = [206 steps = [
210 {207 {
211 "message": "truncate test",208 "message": "truncate test",
@@ -229,7 +226,9 @@
229 r.wait_for_booted(steps)226 r.wait_for_booted(steps)
230227
231 def test_usefile(self):228 def test_usefile(self):
229 """Test tailing a file."""
232 self.usefile(False)230 self.usefile(False)
233231
234 def test_usefile_truncation(self):232 def test_usefile_truncation(self):
233 """Test tailing a file with truncation."""
235 self.usefile(True)234 self.usefile(True)
236235
=== modified file 'utah-done.py'
--- utah-done.py 2013-03-15 13:13:28 +0000
+++ utah-done.py 2013-04-08 19:17:22 +0000
@@ -10,6 +10,7 @@
10- Fetch command failure10- Fetch command failure
11- Setup command failure11- Setup command failure
12UNKNOWN: Unable to retrieve state file to check if client finished properly.12UNKNOWN: Unable to retrieve state file to check if client finished properly.
13
13"""14"""
1415
1516
1617
=== modified file 'utah/__init__.py'
--- utah/__init__.py 2012-12-03 14:14:15 +0000
+++ utah/__init__.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,4 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""utah"""
17utah package
18"""
1917
=== modified file 'utah/cleanup.py'
--- utah/cleanup.py 2013-04-03 14:25:21 +0000
+++ utah/cleanup.py 2013-04-08 19:17:22 +0000
@@ -15,6 +15,7 @@
1515
16"""Generic functionality to execute callbacks on exit."""16"""Generic functionality to execute callbacks on exit."""
1717
18
18import atexit19import atexit
19import logging20import logging
20import os21import os
@@ -67,7 +68,7 @@
6768
68 """69 """
69 timeout, command, args, kw = function70 timeout, command, args, kw = function
70 self.logger.debug('Running: ' +71 self.logger.debug('Running: %s',
71 commandstr(command, *args, **kw))72 commandstr(command, *args, **kw))
72 try:73 try:
73 utah.timeout.timeout(timeout, command, *args, **kw)74 utah.timeout.timeout(timeout, command, *args, **kw)
@@ -102,7 +103,7 @@
102103
103 """104 """
104 if os.path.islink(path):105 if os.path.islink(path):
105 self.logger.debug('Removing link ' + path)106 self.logger.debug('Removing link %s', path)
106 os.unlink(path)107 os.unlink(path)
107 elif os.path.isfile(path):108 elif os.path.isfile(path):
108 self._clean_file(path)109 self._clean_file(path)
@@ -121,14 +122,14 @@
121 :type path: str122 :type path: str
122123
123 """124 """
124 self.logger.debug('Changing permissions of ' + path)125 self.logger.debug('Changing permissions of %s', path)
125 try:126 try:
126 os.chmod(path, 0664)127 os.chmod(path, 0664)
127 except OSError as err:128 except OSError as err:
128 self.logger.warning(129 self.logger.warning(
129 'OSError when changing file permissions: {}'130 'OSError when changing file permissions: {}'
130 .format(str(err)))131 .format(str(err)))
131 self.logger.debug('Removing file ' + path)132 self.logger.debug('Removing file %s', path)
132 try:133 try:
133 os.unlink(path)134 os.unlink(path)
134 except OSError as err:135 except OSError as err:
135136
=== modified file 'utah/client/__init__.py'
--- utah/client/__init__.py 2012-12-03 14:31:27 +0000
+++ utah/client/__init__.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,4 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""utah.client"""
17utah.client
18"""
1917
=== modified file 'utah/client/battery.py'
--- utah/client/battery.py 2013-03-13 12:13:12 +0000
+++ utah/client/battery.py 2013-04-08 19:17:22 +0000
@@ -1,11 +1,30 @@
1# Ubuntu Testing Automation Harness
2# Copyright 2013 Canonical Ltd.
3
4# This program is free software: you can redistribute it and/or modify it
5# under the terms of the GNU General Public License version 3, as published
6# by the Free Software Foundation.
7
8# This program is distributed in the hope that it will be useful, but
9# WITHOUT ANY WARRANTY; without even the implied warranties of
10# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11# PURPOSE. See the GNU General Public License for more details.
12
13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.
15
1"""Helpers for battery measurements."""16"""Helpers for battery measurements."""
17
18
2import os19import os
3import re20import re
4import logging21import logging
522
623
7class _Battery(object):24class _Battery(object):
25
8 """Battery information gathering."""26 """Battery information gathering."""
27
9 POWER_SUPPLY_DIR = '/sys/class/power_supply'28 POWER_SUPPLY_DIR = '/sys/class/power_supply'
10 BATTERY_DIR_REGEX = re.compile(r'bat.*', re.IGNORECASE)29 BATTERY_DIR_REGEX = re.compile(r'bat.*', re.IGNORECASE)
11 BATTERY_FILE_REGEX = re.compile(r'(.*_now|capacity|status)')30 BATTERY_FILE_REGEX = re.compile(r'(.*_now|capacity|status)')
@@ -14,13 +33,14 @@
14 self.filenames = None33 self.filenames = None
1534
16 def get_data(self):35 def get_data(self):
17 """Get data from the battery information files36 """Get data from the battery information files.
1837
19 Every file is read and the contents of the file is written to a38 Every file is read and the contents of the file is written to a
20 dictionary using as key the basename of the file.39 dictionary using as key the basename of the file.
2140
22 :returns: Data as read from the battery information files41 :returns: Data as read from the battery information files
23 :rtype: dict42 :rtype: dict
43
24 """44 """
25 def read_file(filename):45 def read_file(filename):
26 """Read contents of a file and try to convert data to integer46 """Read contents of a file and try to convert data to integer
2747
=== modified file 'utah/client/common.py'
--- utah/client/common.py 2013-04-02 13:56:33 +0000
+++ utah/client/common.py 2013-04-08 19:17:22 +0000
@@ -15,25 +15,27 @@
1515
16"""UTAH client common classes and functions."""16"""UTAH client common classes and functions."""
1717
18
18import datetime19import datetime
19import re
20import grp
21import getpass20import getpass
22import json
23import os21import os
24import platform22import platform
25import pwd23import pwd
24import re
26import signal25import signal
27import subprocess26import subprocess
27import time
28
29import jsonschema
28import yaml30import yaml
29import jsonschema31
32from utah.client.battery import battery
3033
31from utah.client.exceptions import (34from utah.client.exceptions import (
32 YAMLParsingError,35 YAMLParsingError,
33 YAMLEmptyFile,36 YAMLEmptyFile,
34)37)
3538
36from utah.client.battery import battery
3739
38CONFIG = {40CONFIG = {
39 'DEBUG': False,41 'DEBUG': False,
@@ -52,19 +54,29 @@
52CLIENT_CONFIG = os.path.join(UTAH_DIR, 'config', 'client.json')54CLIENT_CONFIG = os.path.join(UTAH_DIR, 'config', 'client.json')
53DEFAULT_STATE_FILE = os.path.join(UTAH_DIR, "state.yaml")55DEFAULT_STATE_FILE = os.path.join(UTAH_DIR, "state.yaml")
5456
5557MEDIA_INFO = '/var/log/installer/media-info'
56# UTAH client return codes58PRODUCT_UUID = '/sys/class/dmi/id/product_uuid'
57# PASS: All test cases were executed and passed59
58# FAIL: All test cases were executed, but at least one of them failed60
59# ERROR: At least one error was detected that prevented a test case from being
60# executed. Examples of situations that are considered an error are:
61# - Fetch command failure
62# - Setup command failure
63# REBOOT: The system under test rebooted as required by a test case. To get
64# final result the state file has to be checked.
65# UNKNOWN: Unable to retrieve state file to check if client finished properly.
66# INVALID_GROUP: The client was launched with a user other than root.
67class ReturnCodes:61class ReturnCodes:
62
63 """Provide consistent return codes for UTAH client.
64
65 PASS: All test cases were executed and passed
66 FAIL: All test cases were executed, but at least one of them failed
67 ERROR: At least one error was detected that prevented a test case from
68 being executed. Examples of situations that are considered an error are:
69 - Fetch command failure
70 - Setup command failure
71 REBOOT: The system under test rebooted as required by a test case. To get
72 final result the state file has to be checked.
73 UNKNOWN: Unable to retrieve state file to check if client finished
74 properly.
75 INVALID_USER: The client was launched with a user other than root.
76 EXCEPTION_ERROR: An exception error was encountered.
77
78 """
79
68 PASS = 080 PASS = 0
69 FAIL = 181 FAIL = 1
70 ERROR = 282 ERROR = 2
@@ -73,6 +85,7 @@
73 INVALID_USER = 585 INVALID_USER = 5
74 EXCEPTION_ERROR = 686 EXCEPTION_ERROR = 6
7587
88
76CMD_TC_BUILD = 'testcase_build'89CMD_TC_BUILD = 'testcase_build'
77CMD_TC_SETUP = 'testcase_setup'90CMD_TC_SETUP = 'testcase_setup'
78CMD_TC_TEST = 'testcase_test'91CMD_TC_TEST = 'testcase_test'
@@ -164,20 +177,22 @@
164177
165 try:178 try:
166 stdout, stderr = p.communicate()179 stdout, stderr = p.communicate()
167 stdout = normalize_encoding(stdout)180 stdout = _normalize_encoding(stdout)
168 stderr = normalize_encoding(stderr)181 stderr = _normalize_encoding(stderr)
169182
170 if timeout != 0:183 if timeout != 0:
171 signal.alarm(0)184 signal.alarm(0)
172 except TimeoutAlarm:185 except TimeoutAlarm:
173 pids = [p.pid]186 pids = [p.pid]
174 # Kill p's children too.187 # Kill p's children too.
175 pids.extend(get_process_children(p.pid))188 pids.extend(_get_process_children(p.pid))
176189
177 for pid in pids:190 for pid in pids:
178 # process might have died before getting to this line191 # process might have died before getting to this line
179 # so wrap to avoid OSError: no such process192 # so wrap to avoid OSError: no such process
180 try:193 try:
194 os.kill(pid, signal.SIGTERM)
195 time.sleep(5)
181 os.kill(pid, signal.SIGKILL)196 os.kill(pid, signal.SIGKILL)
182 except OSError:197 except OSError:
183 pass198 pass
@@ -212,7 +227,7 @@
212 return make_result(**kwargs)227 return make_result(**kwargs)
213228
214229
215def normalize_encoding(value, encoding='utf-8'):230def _normalize_encoding(value, encoding='utf-8'):
216 """Normalize string encoding.231 """Normalize string encoding.
217232
218 Make sure that byte strings are used only for ascii data and unicode233 Make sure that byte strings are used only for ascii data and unicode
@@ -226,7 +241,10 @@
226 :rtype: str241 :rtype: str
227242
228 """243 """
229 unicode_value = value.decode(encoding)244 try:
245 unicode_value = value.decode(encoding)
246 except UnicodeDecodeError:
247 return value
230 try:248 try:
231 output = unicode_value.encode('ascii')249 output = unicode_value.encode('ascii')
232 except UnicodeEncodeError:250 except UnicodeEncodeError:
@@ -291,7 +309,7 @@
291 return res309 return res
292310
293311
294def get_process_children(pid):312def _get_process_children(pid):
295 """Get the children processes of a given one.313 """Get the children processes of a given one.
296314
297 :param pid: Process ID of the parent process315 :param pid: Process ID of the parent process
@@ -300,11 +318,12 @@
300 :rtype: list(int)318 :rtype: list(int)
301319
302 """320 """
303 p = subprocess.Popen('ps --no-headers -o pid --ppid %d' % pid, shell=True,321 try:
304 stdout=subprocess.PIPE, stderr=subprocess.PIPE)322 pids = subprocess.check_output(['ps', '--no-headers', '-o', 'pid',
305 stdout, _stderr = p.communicate()323 '--ppid', str(pid)]).split()
306324 return [int(pid) for pid in pids]
307 return [int(pid) for pid in stdout.split()]325 except subprocess.CalledProcessError:
326 return []
308327
309328
310def parse_yaml_file(filename):329def parse_yaml_file(filename):
@@ -409,102 +428,6 @@
409 return debug428 return debug
410429
411430
412def raise_privileges(orig_privs):
413 """Raise privileges to original effective privileges.
414
415 :param orig_privs: Original privileges
416 :type orig_privs: dict
417
418 """
419 os.seteuid(orig_privs['euid'])
420 os.setegid(orig_privs['egid'])
421 os.setgroups(orig_privs['groups'])
422
423
424def drop_privileges(user='nobody', group='nogroup', full=False):
425 """Change user and/or group to decrease privileges.
426
427 Currently only drops effective privileges.
428
429 :param user: User name to change to.
430 :type user: str
431 :param group: Group name to change to.
432 :type group: str
433
434 :returns: Original prrivileges
435 :rtype: dict
436
437 """
438
439 orig_privs = {}
440
441 orig_privs['uid'] = os.getuid()
442 orig_privs['euid'] = os.geteuid()
443 orig_privs['gid'] = os.getgid()
444 orig_privs['egid'] = os.getegid()
445 orig_privs['groups'] = os.getgroups()
446
447 # If root, drop privileges
448 # inspired by: http://stackoverflow.com/a/2699996
449 if os.getuid() == 0 or os.geteuid() == 0:
450
451 running_uid = pwd.getpwnam(user).pw_uid
452 running_gid = grp.getgrnam(group).gr_gid
453 os.setgroups([])
454
455 if full:
456 os.setgid(running_gid)
457 else:
458 os.setegid(running_gid)
459
460 if full:
461 os.setuid(running_uid)
462 else:
463 os.seteuid(running_uid)
464
465 os.umask(077)
466
467 return orig_privs
468
469
470def gather_artifacts(extras=None, excludes=None):
471 """Print files contents, so that it's included in the logs.
472
473 :param extras: Path to files to be included in the artifacts
474 :type extras: list(str)
475 :param excludes: Path to files to be exluded from the artifacts
476 :type excludes: list(str)
477
478 """
479 if extras is None:
480 extras = []
481 if excludes is None:
482 excludes = []
483
484 artifacts = ['/var/log/syslog', '/proc/cpuinfo']
485
486 artifacts += extras
487
488 # if excludes is a single item make it a list
489 if not isinstance(excludes, list):
490 excludes = [excludes]
491
492 for exc in excludes:
493 try:
494 artifacts.remove(exc)
495 except ValueError:
496 print("{} is not already an artifact".format(exc))
497 pass
498
499 for artifact in artifacts:
500 print("### Start {} ###".format(artifact))
501 with open(artifact, 'r') as fp:
502 for line in fp:
503 print(line.strip())
504 fp.close()
505 print("### End {} ###".format(artifact))
506
507
508def get_media_info():431def get_media_info():
509 """Get the contents of the media-info file if available.432 """Get the contents of the media-info file if available.
510433
@@ -516,13 +439,14 @@
516439
517 """440 """
518441
519 filename = '/var/log/installer/media-info'
520
521 media_info = 'unknown'442 media_info = 'unknown'
522443
523 if os.path.exists(filename) and os.access(filename, os.R_OK):444 try:
524 with open(filename, 'r') as f:445 with open(MEDIA_INFO, 'r') as f:
525 media_info = f.read()446 media_info = f.read()
447 except IOError:
448 # If this fails, return the default
449 pass
526450
527 return media_info451 return media_info
528452
@@ -537,13 +461,14 @@
537461
538 """462 """
539463
540 filename = '/sys/class/dmi/id/product_uuid'
541
542 product_uuid = None464 product_uuid = None
543465
544 if os.path.exists(filename) and os.access(filename, os.R_OK):466 try:
545 with open(filename) as f:467 with open(PRODUCT_UUID) as f:
546 product_uuid = f.read().strip()468 product_uuid = f.read().strip()
469 except IOError:
470 # If this fails, return the default
471 pass
547472
548 return product_uuid473 return product_uuid
549474
@@ -581,7 +506,8 @@
581 'i686': 'i386',506 'i686': 'i386',
582 }507 }
583508
584 return arches.get(platform.machine(), 'unknown')509 arch = platform.machine()
510 return arches.get(arch, arch)
585511
586512
587def get_release():513def get_release():
@@ -594,22 +520,6 @@
594 return platform.linux_distribution()[2]520 return platform.linux_distribution()[2]
595521
596522
597def get_api_config():
598 """Parse the client configuration file for API configuration data.
599
600 :returns: The value for the ``API`` field in the client configuration file.
601
602 """
603
604 data = {}
605 data['API'] = {}
606
607 with open(CLIENT_CONFIG, 'r') as fp:
608 data = json.load(fp)
609
610 return data['API']
611
612
613def get_build_number():523def get_build_number():
614 """Get build number.524 """Get build number.
615525
@@ -624,152 +534,3 @@
624 match = pattern.match(host_info['media-info'])534 match = pattern.match(host_info['media-info'])
625535
626 return match.group(1) if match else '?'536 return match.group(1) if match else '?'
627
628# VCS Support
629# XXX: This maybe ought to be moved to another file
630REPO_DEFAULT_DIR = "."
631
632
633class VCSHandler(object):
634
635 """Base class for Version Control System support.
636
637 :param repo: Repository location
638 :type repo: str
639 :param destination:
640 Local directory where the repository should be made available
641 :type destination: str
642 :param battery_measurements:
643 Whether battery information should be gathered when running any of the
644 VCS commands
645 :type battery_measurements: bool
646
647 """
648
649 def __init__(self, repo, destination='', battery_measurements=False):
650 self.repo = repo
651 self.destination = destination
652 self.battery_measurements = battery_measurements
653
654 def get(self, directory=REPO_DEFAULT_DIR):
655 """Execute VCS get command.
656
657 The get command will make a copy of a given repository to the directory
658 specified as destination.
659
660 :param directory: Working directory
661 :type directory: str
662 :returns: Get command execution result
663 :rtype: dict
664
665 .. seealso:: :func:`run_cmd`
666
667 """
668 return run_cmd(self.get_command,
669 cwd=directory,
670 cmd_type=CMD_TS_FETCH,
671 battery_measurements=self.battery_measurements)
672
673 def revision(self, directory=REPO_DEFAULT_DIR):
674 """Execute revision command.
675
676 :param directory: Working directory
677 :type directory: str
678 :returns: Revision command execution result.
679 :rtype: dict
680
681 .. seealso:: :func:`run_cmd`, :meth:`get_revision`
682
683 """
684 return run_cmd(self.rev_command,
685 cwd=directory,
686 cmd_type=CMD_TS_FETCH,
687 battery_measurements=self.battery_measurements)
688
689 def get_revision(self, directory=REPO_DEFAULT_DIR):
690 """Print revision information.
691
692 This is a wrapper around :meth:`revision` to make sure that the command
693 returned a success code.
694
695 :param directory: Working directory
696 :type directory: str
697 :returns: Revision command returncode
698 :rtype: int
699
700 .. seealso:: :meth:`revision`
701
702 """
703 retval = None
704 result = self.revision(directory=directory)
705
706 if result['returncode'] == 0:
707 retval = result['stdout'].strip()
708 else:
709 print(result)
710
711 return retval
712
713
714class BzrHandler(VCSHandler):
715
716 """Bazaar VCS handler."""
717
718 def __init__(self, repo, branch=True, options="", destination="",
719 **kwargs):
720 super(BzrHandler, self).__init__(repo, destination, **kwargs)
721 self.options = options
722
723 if branch:
724 self.get_command = "bzr branch {} {} {}".format(
725 options,
726 self.repo,
727 self.destination,
728 )
729
730 self.rev_command = "bzr revno"
731 else:
732 self.get_command = "bzr export {} {} {}".format(
733 options,
734 self.destination,
735 self.repo,
736 )
737
738 self.rev_command = "bzr revno {}".format(self.repo)
739
740
741class GitHandler(VCSHandler):
742
743 """Git VCS handler."""
744
745 def __init__(self, repo, options="", destination="", **kwargs):
746 super(GitHandler, self).__init__(repo, destination, **kwargs)
747 self.options = options
748
749 self.get_command = "git clone {} {} {}".format(
750 options,
751 self.repo,
752 self.destination,
753 )
754
755 self.rev_command = "git rev-parse HEAD"
756
757
758class DevHandler(VCSHandler):
759
760 """Development VCS handler.
761
762 This isn't a handler for any VCS, but a helper to ease development and let
763 runlists point to a path that can be copied locally.
764
765 """
766
767 def __init__(self, repo, destination=".", **kwargs):
768 super(DevHandler, self).__init__(repo, destination, **kwargs)
769
770 self.get_command = "cp -r {} {}".format(
771 self.repo,
772 self.destination,
773 )
774
775 self.rev_command = "echo 'DEVELOPMENT'"
776537
=== modified file 'utah/client/examples/examples/test_one/test_one.py'
--- utah/client/examples/examples/test_one/test_one.py 2012-03-27 21:22:53 +0000
+++ utah/client/examples/examples/test_one/test_one.py 2013-04-08 19:17:22 +0000
@@ -1,5 +1,8 @@
1#!/usr/bin/env python1#!/usr/bin/env python
22
3"""Basic example test script."""
4
5
3import time6import time
47
58
69
=== modified file 'utah/client/examples/examples/test_two/test_two.py'
--- utah/client/examples/examples/test_two/test_two.py 2012-03-27 15:58:56 +0000
+++ utah/client/examples/examples/test_two/test_two.py 2013-04-08 19:17:22 +0000
@@ -1,3 +1,6 @@
1#!/usr/bin/env python1#!/usr/bin/env python
22
3"""An even more basic example test script."""
4
5
3print "test_two"6print "test_two"
47
=== modified file 'utah/client/examples/utah_tests/test_one/test_one.py'
--- utah/client/examples/utah_tests/test_one/test_one.py 2012-04-10 20:23:50 +0000
+++ utah/client/examples/utah_tests/test_one/test_one.py 2013-04-08 19:17:22 +0000
@@ -1,5 +1,8 @@
1#!/usr/bin/env python1#!/usr/bin/env python
22
3"""Basic example test script."""
4
5
3import time6import time
47
58
69
=== modified file 'utah/client/examples/utah_tests/test_two/test_two.py'
--- utah/client/examples/utah_tests/test_two/test_two.py 2012-04-10 20:23:50 +0000
+++ utah/client/examples/utah_tests/test_two/test_two.py 2013-04-08 19:17:22 +0000
@@ -1,3 +1,6 @@
1#!/usr/bin/env python1#!/usr/bin/env python
22
3"""An even more basic example test script."""
4
5
3print "test_two"6print "test_two"
47
=== modified file 'utah/client/examples/utah_tests_sample/sample_one/sample.py'
--- utah/client/examples/utah_tests_sample/sample_one/sample.py 2012-04-10 20:23:50 +0000
+++ utah/client/examples/utah_tests_sample/sample_one/sample.py 2013-04-08 19:17:22 +0000
@@ -1,3 +1,6 @@
1#!/usr/bin/env python1#!/usr/bin/env python
22
3"""A very basic example test script."""
4
5
3print("sample_one test output goes here")6print("sample_one test output goes here")
47
=== modified file 'utah/client/exceptions.py'
--- utah/client/exceptions.py 2013-03-14 20:23:09 +0000
+++ utah/client/exceptions.py 2013-04-08 19:17:22 +0000
@@ -13,59 +13,70 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Provide UTAH client exceptions."""
17UTAH client exceptions
18"""
1917
2018
21class UTAHClientError(Exception):19class UTAHClientError(Exception):
22 """20
23 Base class of all exceptions in the client21 """Base class of all exceptions in the client."""
24 """22
23 pass
2524
2625
27class BadDir(UTAHClientError):26class BadDir(UTAHClientError):
28 """27
28 """Directory error.
29
29 Raised when some test directory isn't found30 Raised when some test directory isn't found
30 or an error happens when trying to change to it31 or an error happens when trying to change to it
32
31 """33 """
3234
35 pass
36
3337
34class MissingFile(UTAHClientError):38class MissingFile(UTAHClientError):
35 """39
36 Raised when yaml file with metadata about tests40 """Raised when yaml file with metadata about tests cannot be found."""
37 cannot be found41
38 """42 pass
3943
4044
41class BadMasterRunlist(UTAHClientError):45class BadMasterRunlist(UTAHClientError):
42 """46
43 Raised when master runlist isn't in the expected format47 """Raised when master runlist isn't in the expected format."""
44 """48
49 pass
4550
4651
47class YAMLParsingError(UTAHClientError):52class YAMLParsingError(UTAHClientError):
48 """53
54 """Provide more detailed yaml.load exception handling.
55
49 Used to provide the filename and the location56 Used to provide the filename and the location
50 in which the parsing error happened when calling yaml.load57 in which the parsing error happened when calling yaml.load
58
51 """59 """
5260
61 pass
62
5363
54class YAMLEmptyFile(UTAHClientError):64class YAMLEmptyFile(UTAHClientError):
55 """65
56 Used to signal that a file that was supposed to contain yaml data66 """Raised when a file that was supposed to contain yaml data is empty."""
57 is actually empty67
58 """68 pass
5969
6070
61class ValidationError(UTAHClientError):71class ValidationError(UTAHClientError):
62 """72
63 Used to provide additional information when schema validation fails73 """Used to provide additional information when schema validation fails."""
64 """74
75 pass
6576
6677
67class MissingData(UTAHClientError):78class MissingData(UTAHClientError):
68 """79
69 Raised when there is missing data that is required for an object80 """Raised when there is missing data required for an object to proceed."""
70 to proceed.81
71 """82 pass
7283
=== modified file 'utah/client/phoenix.py'
--- utah/client/phoenix.py 2012-12-11 15:18:01 +0000
+++ utah/client/phoenix.py 2013-04-08 19:17:22 +0000
@@ -15,7 +15,8 @@
15# You should have received a copy of the GNU General Public License along15# You should have received a copy of the GNU General Public License along
16# with this program. If not, see <http://www.gnu.org/licenses/>.16# with this program. If not, see <http://www.gnu.org/licenses/>.
1717
18# phoenix.py - A bootstrap script for testsuite/testcase authors18"""phoenix.py - A bootstrap script for testsuite/testcase authors"""
19
1920
20import argparse21import argparse
21import os22import os
2223
=== modified file 'utah/client/result.py'
--- utah/client/result.py 2013-03-01 13:04:22 +0000
+++ utah/client/result.py 2013-04-08 19:17:22 +0000
@@ -13,11 +13,14 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Provide functionality for test result handling."""
17
18
16import json19import json
17import sys20import sys
18import yaml21import yaml
1922
20from .common import (23from utah.client.common import (
21 get_host_info,24 get_host_info,
22 get_build_number,25 get_build_number,
23 get_release,26 get_release,
@@ -25,41 +28,10 @@
25)28)
2629
2730
28def get_smoke_data(_result):
29 data = {}
30 data['build_no'] = get_build_number()
31
32 return data
33
34
35def get_kernel_sru_data(_result):
36 data = {}
37
38 return data
39
40
41# XXX: this logic should probably be moved into the __init__ method
42# of the result object since there will be one result object per
43# master.run
44def create_payload(result):
45 publish_type = result.publish_type
46
47 if publish_type is None:
48 return None
49
50 type_map = {
51 'smoke': get_smoke_data,
52 'kernel-sru': get_kernel_sru_data,
53 }
54
55 if publish_type and publish_type in type_map.iterkeys():
56 return type_map[publish_type](result)
57
58
59class Result(object):31class Result(object):
60 """32
61 Result collection class.33 """Result collection class."""
62 """34
63 def __init__(self, name=None, testsuite=None, testcase=None,35 def __init__(self, name=None, testsuite=None, testcase=None,
64 runlist=None, publish_type=None, install_type=None):36 runlist=None, publish_type=None, install_type=None):
65 self.results = []37 self.results = []
@@ -85,14 +57,13 @@
85 :param old_results: Old results as loaded from the report file57 :param old_results: Old results as loaded from the report file
86 :type old_results: str58 :type old_results: str
8759
88 .. seealso:: :meth:`payload`60 .. seealso:: :meth:`_payload`
8961
90 """62 """
91 raise NotImplementedError63 raise NotImplementedError
9264
93 def add_result(self, result, extra_info=None):65 def add_result(self, result, extra_info=None):
94 """66 """Add a result to the object.
95 Add a result to the object.
9667
97 Note: 'result' is expected to be a dictionary like this::68 Note: 'result' is expected to be a dictionary like this::
9869
@@ -104,6 +75,7 @@
104 'start_time': '',75 'start_time': '',
105 'time_delta': '',76 'time_delta': '',
106 }77 }
78
107 """79 """
108 if result is None:80 if result is None:
109 return81 return
@@ -127,8 +99,13 @@
127 self.status = 'FAIL'99 self.status = 'FAIL'
128100
129 def result(self, verbose=False):101 def result(self, verbose=False):
130 """102 """Output a text based result.
131 Output a text based result.103
104 :param verbose: Enable verbose mode
105 :type verbose: bool
106 :returns: Status 'PASS' 'FAIL' or 'ERROR'
107 :rtype: str
108
132 """109 """
133 status = self.status110 status = self.status
134 sep = '-' * 70111 sep = '-' * 70
@@ -140,19 +117,19 @@
140 print sep117 print sep
141118
142 for result in self.results:119 for result in self.results:
143 print "command: %s" % result['command']120 print 'command: {}'.format(result['command'])
144 print "returned: %d" % result['returncode']121 print 'returned: {}'.format(str(result['returncode']))
145 print "started: %s" % result['start_time']122 print 'started: {}'.format(result['start_time'])
146 print "runtime: %s" % result['time_delta']123 print 'runtime: {}'.format(result['time_delta'])
147124
148 if self.testsuite is not None:125 if self.testsuite is not None:
149 print "testsuite: %s" % self.testsuite126 print "testsuite: {}".format(self.testsuite)
150127
151 if self.testcase is not None:128 if self.testcase is not None:
152 print "testcase: %s" % self.testcase129 print "testcase: {}".format(self.testcase)
153130
154 if self.runlist is not None:131 if self.runlist is not None:
155 print "runlist: %s" % self.runlist132 print "runlist: {}".format(self.runlist)
156133
157 if self.status != 'PASS' or verbose and result['stdout'] != '':134 if self.status != 'PASS' or verbose and result['stdout'] != '':
158 print "stdout: "135 print "stdout: "
@@ -164,37 +141,33 @@
164141
165 print142 print
166143
167 data = self.payload()144 data = self._payload()
168145
169 for key, value in data.iteritems():146 for key, value in data.iteritems():
170 print '{}: {}'.format(key, value)147 print '{}: {}'.format(key, value)
171148
172 self.clear_results()149 self._clear_results()
173150
174 return status151 return status
175152
176 def clear_results(self):153 def _clear_results(self):
177 # reset the list of results so this object can be reused.154 """Reset the list of results so this object can be reused."""
178 self.results = []155 self.results = []
179156
180 # reset the status, this should be safe since the only places that157 # reset the status, this should be safe since the only places that
181 # should be calling clear_results() is testcase.run(), testsuite.run(),158 # should be calling _clear_results() is testcase.run(),
182 # and runner after attempting to fetch the testsuite.159 # testsuite.run(), and runner after attempting to fetch the testsuite.
183 self.status = 'PASS'160 self.status = 'PASS'
184161
185 def count_results(self):162 def _count_results(self):
186 return len(self.results)163 return len(self.results)
187164
188 def publish_results(self):165 def _payload(self):
189 # TBD: Use url and token166 """Construct the result payload.
190 # url = self.publish_url167
191 # token = self.publish_token168 :returns: Test result
192169 :rtype: dict
193 create_payload(self)170
194
195 def payload(self):
196 """
197 Construct the result payload.
198 """171 """
199 host_info = get_host_info()172 host_info = get_host_info()
200 data = {173 data = {
@@ -229,27 +202,35 @@
229202
230# Trick to get strings printed as literal blocks203# Trick to get strings printed as literal blocks
231# inspired by: http://stackoverflow.com/a/7445560/183066204# inspired by: http://stackoverflow.com/a/7445560/183066
232class LiteralString(object):205class _LiteralString(object):
233 def __init__(self, str_data):206 def __init__(self, str_data):
234 self.str_data = str_data207 self.str_data = str_data
235208
236209
237def literal_block(dumper, data):210def _literal_block(dumper, data):
238 return dumper.represent_scalar('tag:yaml.org,2002:str',211 return dumper.represent_scalar('tag:yaml.org,2002:str',
239 data.str_data, style='|')212 data.str_data, style='|')
240yaml.add_representer(LiteralString, literal_block)213yaml.add_representer(_LiteralString, _literal_block)
241214
242215
243class ResultYAML(Result):216class ResultYAML(Result):
217
218 """Return results in a YAML format."""
219
244 def _literalize(self, data):220 def _literalize(self, data):
245 """221 """Transform long strings into literal blocks.
246 Transform long strings into literal blocks222
223 :param data: Data to literalize
224 :type data: string, dict, or list
225 :returns: literalized form of data
226 :rtype: str, dict, or list
227
247 """228 """
248 if isinstance(data, basestring) and '\n' in data:229 if isinstance(data, basestring) and '\n' in data:
249 # Remove trailing whitespace to serialize in yaml230 # Remove trailing whitespace to serialize in yaml
250 # as a literal string231 # as a literal string
251 lines = [line.rstrip() for line in data.splitlines()]232 lines = [line.rstrip() for line in data.splitlines()]
252 return LiteralString('\n'.join(lines))233 return _LiteralString('\n'.join(lines))
253234
254 if isinstance(data, dict):235 if isinstance(data, dict):
255 for key, value in data.iteritems():236 for key, value in data.iteritems():
@@ -259,16 +240,22 @@
259 for element in data]240 for element in data]
260 return data241 return data
261242
262 def result(self, verbose=False):243 def result(self, _verbose=False):
244 """Output a YAML result.
245
246 :returns: Status 'PASS' 'FAIL' or 'ERROR'
247 :rtype: str
248
249 """
263 if self.results:250 if self.results:
264 data = self._literalize(self.payload())251 data = self._literalize(self._payload())
265 yaml.dump(data, sys.stdout,252 yaml.dump(data, sys.stdout,
266 explicit_start='---',253 explicit_start='---',
267 default_flow_style=False,254 default_flow_style=False,
268 allow_unicode=True)255 allow_unicode=True)
269256
270 status = self.status257 status = self.status
271 self.clear_results()258 self._clear_results()
272259
273 return status260 return status
274261
@@ -278,7 +265,7 @@
278 :param old_results: Old results as loaded from the report file265 :param old_results: Old results as loaded from the report file
279 :type old_results: str266 :type old_results: str
280267
281 .. seealso:: :meth:`payload`268 .. seealso:: :meth:`_payload`
282269
283 """270 """
284 data = yaml.load(old_results)271 data = yaml.load(old_results)
@@ -293,13 +280,21 @@
293280
294281
295class ResultJSON(Result):282class ResultJSON(Result):
296 def result(self, verbose=False):283
297284 """Return results in a JSON format."""
285
286 def result(self, _verbose=False):
287 """Output a YAML result.
288
289 :returns: Status 'PASS' 'FAIL' or 'ERROR'
290 :rtype: str
291
292 """
298 if self.results:293 if self.results:
299 data = self.payload()294 data = self._payload()
300 json.dump(data, sys.stdout, indent=4)295 json.dump(data, sys.stdout, indent=4)
301296
302 status = self.status297 status = self.status
303 self.clear_results()298 self._clear_results()
304299
305 return status300 return status
306301
=== modified file 'utah/client/runner.py'
--- utah/client/runner.py 2013-03-21 15:43:29 +0000
+++ utah/client/runner.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,8 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Provide code to actually run the tests."""
17
1618
17import datetime19import datetime
18import jsonschema20import jsonschema
@@ -25,14 +27,11 @@
25from utah.client import exceptions27from utah.client import exceptions
26from utah.client.battery import battery28from utah.client.battery import battery
27from utah.client.common import (29from utah.client.common import (
28 BzrHandler,
29 CMD_TC_REBOOT,30 CMD_TC_REBOOT,
30 DATE_FORMAT,31 DATE_FORMAT,
31 DEFAULT_STATE_FILE,32 DEFAULT_STATE_FILE,
32 DEFAULT_TSLIST,33 DEFAULT_TSLIST,
33 DefaultValidator,34 DefaultValidator,
34 DevHandler,
35 GitHandler,
36 make_result,35 make_result,
37 MASTER_RUNLIST,36 MASTER_RUNLIST,
38 parse_yaml_file,37 parse_yaml_file,
@@ -42,12 +41,17 @@
42from utah.client.result import Result41from utah.client.result import Result
43from utah.client.state_agent import StateAgentYAML42from utah.client.state_agent import StateAgentYAML
44from utah.client.testsuite import TestSuite43from utah.client.testsuite import TestSuite
44from utah.client.vcs import (
45 BzrHandler,
46 DevHandler,
47 GitHandler,
48)
45from utah.exceptions import UTAHException49from utah.exceptions import UTAHException
46from utah.retry import retry50from utah.retry import retry
47from utah.timeout import timeout51from utah.timeout import timeout
4852
49RC_LOCAL = '/etc/rc.local'53RC_LOCAL = '/etc/rc.local'
50RC_LOCAL_BACKUP = '%s-utah.bak' % RC_LOCAL54RC_LOCAL_BACKUP = '{}-utah.bak'.format(RC_LOCAL)
5155
52# XXX: need to output to the same file that was supplied on the original56# XXX: need to output to the same file that was supplied on the original
53# run.57# run.
@@ -58,10 +62,11 @@
5862
5963
60class Runner(object):64class Runner(object):
61 """65
62 The main runner.66 """Provide The main runner class.
6367
64 Parses a master runlist, builds a list of TestSuites, and runs them.68 Parses a master runlist, builds a list of TestSuites, and runs them.
69
65 """70 """
6671
67 status = "NOTRUN"72 status = "NOTRUN"
@@ -216,10 +221,7 @@
216 self.result.name = self.name221 self.result.name = self.name
217222
218 def backup_rc_local(self):223 def backup_rc_local(self):
219 """224 """Backup /etc/rc.local if it exists."""
220 Backup /etc/rc.local if it exists.
221 """
222
223 # Ignore permission denied errors since we only care if a reboot is225 # Ignore permission denied errors since we only care if a reboot is
224 # pending whether or not we can write to RC_LOCAL(_BACKUP).226 # pending whether or not we can write to RC_LOCAL(_BACKUP).
225 try:227 try:
@@ -230,11 +232,7 @@
230 'Failed to backup rc.local: {}'.format(e))232 'Failed to backup rc.local: {}'.format(e))
231233
232 def reset_rc_local(self):234 def reset_rc_local(self):
233 """235 """Restore /etc/rc.local if there is a backup, remove it otherwise."""
234 Restore original /etc/rc.local if there is a backup
235 Otherwise remove it.
236 """
237
238 # Ignore permission denied errors since we only care if a reboot is236 # Ignore permission denied errors since we only care if a reboot is
239 # pending whether or not we can write to RC_LOCAL(_BACKUP).237 # pending whether or not we can write to RC_LOCAL(_BACKUP).
240 try:238 try:
@@ -248,9 +246,7 @@
248 'Failed to backup rc.local: {}'.format(e))246 'Failed to backup rc.local: {}'.format(e))
249247
250 def setup_rc_local(self, rc_local=RC_LOCAL, runlist=None):248 def setup_rc_local(self, rc_local=RC_LOCAL, runlist=None):
251 """249 """Setup /etc/rc.local to kick-off a --resume on successful boot."""
252 Setup /etc/rc.local to kick-off a --resume on successful boot.
253 """
254 runlist = runlist or self.master_runlist or MASTER_RUNLIST250 runlist = runlist or self.master_runlist or MASTER_RUNLIST
255251
256 self.rc_local_content = rc_local_content252 self.rc_local_content = rc_local_content
@@ -265,6 +261,12 @@
265 'Error setting up rc.local: {}'.format(err))261 'Error setting up rc.local: {}'.format(err))
266262
267 def process_results(self):263 def process_results(self):
264 """Add stats to results and process them.
265
266 :returns: A return code based on the test status.
267 :rtype: int
268
269 """
268 # Add stats to the results270 # Add stats to the results
269 self.result.failures = self.failures271 self.result.failures = self.failures
270 self.result.passes = self.passes272 self.result.passes = self.passes
@@ -276,8 +278,13 @@
276 return self.returncode()278 return self.returncode()
277279
278 def run(self):280 def run(self):
279 """281 """Run the test suites we've parsed.
280 Run the test suites we've parsed.282
283 :returns: The result of process_results, which is a return code.
284 :rtype: int
285
286 .. seealso:: :meth:`process_results`
287
281 """288 """
282 if self.battery_measurements:289 if self.battery_measurements:
283 self.result.start_battery = battery.get_data()290 self.result.start_battery = battery.get_data()
@@ -315,12 +322,11 @@
315 return self.process_results()322 return self.process_results()
316323
317 def add_suite(self, suite):324 def add_suite(self, suite):
325 """Add a test suite to run."""
318 self.suites.append(suite)326 self.suites.append(suite)
319327
320 def get_fetched_suites(self):328 def get_fetched_suites(self):
321 """329 """Return a list of fetched suites from the state_agent."""
322 Return a list of fetched suites from the state_agent.
323 """
324 state = self.state_agent.load_state()330 state = self.state_agent.load_state()
325331
326 fetched_suites = []332 fetched_suites = []
@@ -331,6 +337,7 @@
331 return fetched_suites337 return fetched_suites
332338
333 def load_state(self):339 def load_state(self):
340 """Load the state saved by a previous partial run (i.e., a reboot)."""
334 state = self.state_agent.load_state()341 state = self.state_agent.load_state()
335342
336 self.master_runlist = state['master_runlist']343 self.master_runlist = state['master_runlist']
@@ -351,10 +358,12 @@
351 self.fetched_suites = state['fetched_suites']358 self.fetched_suites = state['fetched_suites']
352359
353 def save_state(self):360 def save_state(self):
354 """361 """Save the list of tests we are to run and whether we've run them.
355 Save the list of tests we are to run and whether we've run them.362
356 """363 :returns: state of currently run tests
357364 :rtype: dict
365
366 """
358 self.backup_runlist = os.path.join(self.testdir, 'master.run-reboot')367 self.backup_runlist = os.path.join(self.testdir, 'master.run-reboot')
359368
360 if (os.path.exists(369 if (os.path.exists(
@@ -390,9 +399,11 @@
390 return state399 return state
391400
392 def count_suites(self):401 def count_suites(self):
402 """Return the number of test suites in the runner."""
393 return len(self.suites)403 return len(self.suites)
394404
395 def count_tests(self):405 def count_tests(self):
406 """Return the number of test cases in the runner."""
396 tests = 0407 tests = 0
397408
398 for suite in self.suites:409 for suite in self.suites:
@@ -401,10 +412,7 @@
401 return tests412 return tests
402413
403 def process_master_runlist(self, runlist=None, resume=False):414 def process_master_runlist(self, runlist=None, resume=False):
404 """415 """Parse a master runlist and build a list of suites from the data."""
405 Parse a master runlist building a list of suites from the parsed data.
406
407 """
408 runlist = runlist or self.master_runlist416 runlist = runlist or self.master_runlist
409417
410 # Download runlist using the URL passed through the commmand line to418 # Download runlist using the URL passed through the commmand line to
@@ -511,6 +519,7 @@
511 except OSError as err:519 except OSError as err:
512 raise exceptions.BadDir(err)520 raise exceptions.BadDir(err)
513521
522 # TODO: move this somewhere else
514 def vcs_get_retriable():523 def vcs_get_retriable():
515 result = vcs_handler.get(directory=name)524 result = vcs_handler.get(directory=name)
516 if (isinstance(vcs_handler, BzrHandler) and525 if (isinstance(vcs_handler, BzrHandler) and
@@ -555,12 +564,11 @@
555 self.add_suite(s)564 self.add_suite(s)
556565
557 def get_next_suite(self):566 def get_next_suite(self):
558 """567 """Return the next suite to be run.
559 Return the next suite to be run.
560568
561 Mainly used for debugging.569 Mainly used for debugging.
570
562 """571 """
563
564 suite = None572 suite = None
565573
566 for s in self.suites:574 for s in self.suites:
@@ -571,18 +579,18 @@
571 return suite579 return suite
572580
573 def get_next_test(self):581 def get_next_test(self):
574 """582 """Return the next test to be run.
575 Return the next test to be run.
576583
577 Mainly used for debugging.584 Mainly used for debugging.
585
578 """586 """
579 return self.get_next_suite().get_next_test()587 return self.get_next_suite().get_next_test()
580588
581 def reboot(self):589 def reboot(self):
582 """590 """Reboot the machine.
583 Reboot the machine.
584591
585 Save state, setup /etc/rc.local, and shutdown.592 Save state, setup /etc/rc.local, and shutdown.
593
586 """594 """
587 # Create fake result for logging purposes595 # Create fake result for logging purposes
588 command = 'shutdown -r now'596 command = 'shutdown -r now'
@@ -613,25 +621,10 @@
613 # End of execution621 # End of execution
614622
615 def returncode(self):623 def returncode(self):
624 """Provide return code based on test status."""
616 if self.errors > 0 or self.fetch_errors > 0:625 if self.errors > 0 or self.fetch_errors > 0:
617 return ReturnCodes.ERROR626 return ReturnCodes.ERROR
618 elif self.failures > 0:627 elif self.failures > 0:
619 return ReturnCodes.FAIL628 return ReturnCodes.FAIL
620 else:629 else:
621 return ReturnCodes.PASS630 return ReturnCodes.PASS
622
623 def report(self):
624 tests_run = self.passes + self.errors + self.failures
625
626 output = ("total: %d, passes: %d, failure: %d, error: %d"
627 % (tests_run, self.passes, self.failures,
628 self.errors + self.fetch_errors))
629
630 if self.errors > 0 or self.fetch_errors > 0:
631 result = "ERROR"
632 elif self.failures > 0:
633 result = "FAIL"
634 else:
635 result = "PASS"
636
637 return "%s - %s" % (result, output)
638631
=== modified file 'utah/client/state_agent.py'
--- utah/client/state_agent.py 2012-12-03 14:02:18 +0000
+++ utah/client/state_agent.py 2013-04-08 19:17:22 +0000
@@ -13,17 +13,22 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Provide functionality for saving and restoring run state."""
17
18
16import os19import os
20
17import yaml21import yaml
1822
19from utah.client.common import DEFAULT_STATE_FILE, parse_yaml_file23from utah.client.common import DEFAULT_STATE_FILE, parse_yaml_file
2024
2125
22class StateAgent(object):26class StateAgent(object):
23 """27
24 State saving base class.28 """State saving base class.
2529
26 Accepts a dictionary of state info and prints it to STDOUT.30 Accepts a dictionary of state info and prints it to STDOUT.
31
27 """32 """
2833
29 state_file = DEFAULT_STATE_FILE34 state_file = DEFAULT_STATE_FILE
@@ -33,9 +38,7 @@
33 self.state_file = state_file38 self.state_file = state_file
3439
35 def clean(self):40 def clean(self):
36 """41 """Clean up the state file if it exists."""
37 Clean up the state file if it exists.
38 """
39 # Don't fail if the state_file doesn't exists.42 # Don't fail if the state_file doesn't exists.
40 try:43 try:
41 os.remove(self.state_file)44 os.remove(self.state_file)
@@ -44,17 +47,17 @@
44 pass47 pass
4548
46 def save_state(self, state):49 def save_state(self, state):
47 """50 """Save state to the state_file."""
48 Save state to the state_file.
49 """
50
51 fp = open(self.state_file, 'w')51 fp = open(self.state_file, 'w')
52 fp.write(str(state))52 fp.write(str(state))
53 fp.close()53 fp.close()
5454
55 def load_state(self):55 def load_state(self):
56 """56 """Load state from the state_file.
57 Load state from the state_file.57
58 :returns: state information from file.
59 :rtype: dict
60
58 """61 """
59 state = {}62 state = {}
6063
@@ -67,22 +70,21 @@
6770
6871
69class StateAgentYAML(StateAgent):72class StateAgentYAML(StateAgent):
70 """73
71 YAML based state saver.74 """YAML based state saver."""
72 """
7375
74 def save_state(self, state):76 def save_state(self, state):
75 """77 """Output the state as YAML."""
76 Output the state as YAML.
77 """
78 yaml_state = yaml.dump(state, default_flow_style=False)78 yaml_state = yaml.dump(state, default_flow_style=False)
79 super(StateAgentYAML, self).save_state(yaml_state)79 super(StateAgentYAML, self).save_state(yaml_state)
8080
81 def load_state(self):81 def load_state(self):
82 """82 """Load state from YAML state_file.
83 Load state from YAML state_file.83
84 """84 :returns: state information from YAML file
8585 :rtype: dict
86
87 """
86 state = {}88 state = {}
8789
88 if os.path.exists(self.state_file):90 if os.path.exists(self.state_file):
8991
=== modified file 'utah/client/testcase.py'
--- utah/client/testcase.py 2013-03-14 20:53:20 +0000
+++ utah/client/testcase.py 2013-04-08 19:17:22 +0000
@@ -13,35 +13,35 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Testcase specific code."""
17Testcase specific code.17
18"""18
1919import os
2020
21import jsonschema21import jsonschema
22import os
2322
24from utah.client.common import (23from utah.client.common import (
25 run_cmd,
26 parse_control_file,
27 do_nothing,
28 CMD_TC_BUILD,24 CMD_TC_BUILD,
25 CMD_TC_CLEANUP,
29 CMD_TC_SETUP,26 CMD_TC_SETUP,
30 CMD_TC_TEST,27 CMD_TC_TEST,
31 CMD_TC_CLEANUP,28 do_nothing,
29 parse_control_file,
30 run_cmd,
32)31)
33from utah.client.exceptions import (32from utah.client.exceptions import (
33 MissingData,
34 MissingFile,34 MissingFile,
35 ValidationError,35 ValidationError,
36 MissingData,
37)36)
3837
3938
40class TestCase(object):39class TestCase(object):
41 """40
42 The TestCase class.41 """Base class describing a test case.
4342
44 status is one of 'NOTRUN', 'BUILD', 'SETUP', 'RUN', 'CLEANUP', or 'DONE'43 status is one of 'NOTRUN', 'BUILD', 'SETUP', 'RUN', 'CLEANUP', or 'DONE'
44
45 """45 """
4646
47 status = 'NOTRUN'47 status = 'NOTRUN'
@@ -100,15 +100,15 @@
100 battery_measurements=False,100 battery_measurements=False,
101 _control_data=None, _save_state_callback=None,101 _control_data=None, _save_state_callback=None,
102 _reboot_callback=None):102 _reboot_callback=None):
103 """103 """Build a TestCase from a control file's data.
104 Build a TestCase from a control file's data.
105104
106 'name' is a directory where the test case resides.105 'name' is a directory where the test case resides.
106
107 """107 """
108 self.name = name108 self.name = name
109 self.path = path109 self.path = path
110 self.working_dir = path110 self.working_dir = path
111 self.filename = "%s/tc_control" % path111 self.filename = "{}/tc_control".format(path)
112 self.results = []112 self.results = []
113 self.result = result113 self.result = result
114 self.timeout = timeout114 self.timeout = timeout
@@ -158,20 +158,16 @@
158 self.command = command158 self.command = command
159159
160 def __str__(self):160 def __str__(self):
161 return ("%s: %s, %s, %s"161 return ("{}: {}, {}, {}".format(
162 % (self.name, self.description, self.command, self.timeout))162 self.name, self.description, self.command, self.timeout))
163163
164 def set_status(self, status):164 def set_status(self, status):
165 """165 """Set the status and call the save state callback."""
166 Set the status for the test case and call the save state callback.
167 """
168 self.status = status166 self.status = status
169 self.save_state_callback()167 self.save_state_callback()
170168
171 def build(self, result):169 def build(self, result):
172 """170 """Run build, but only if we haven't started a run yet."""
173 Run build, but only if we haven't started a run yet.
174 """
175 if self.status == 'NOTRUN':171 if self.status == 'NOTRUN':
176 self.set_status('BUILD')172 self.set_status('BUILD')
177 cmd_result = run_cmd(173 cmd_result = run_cmd(
@@ -182,9 +178,7 @@
182 result.add_result(cmd_result)178 result.add_result(cmd_result)
183179
184 def setup(self, result):180 def setup(self, result):
185 """181 """Run tc_setup, but only if build() has just passed."""
186 Run tc_setup, but only if build() has just passed.
187 """
188 if self.status == 'BUILD' and result.status == 'PASS':182 if self.status == 'BUILD' and result.status == 'PASS':
189 self.set_status('SETUP')183 self.set_status('SETUP')
190 cmd_result = run_cmd(184 cmd_result = run_cmd(
@@ -195,9 +189,7 @@
195 result.add_result(cmd_result)189 result.add_result(cmd_result)
196190
197 def cleanup(self, result):191 def cleanup(self, result):
198 """192 """Run tc_cleanup after a run."""
199 Run tc_cleanup after a run.
200 """
201 if self.status == 'RUN':193 if self.status == 'RUN':
202 self.set_status('CLEANUP')194 self.set_status('CLEANUP')
203 cmd_result = run_cmd(195 cmd_result = run_cmd(
@@ -208,10 +200,14 @@
208 result.add_result(cmd_result)200 result.add_result(cmd_result)
209201
210 def run(self):202 def run(self):
211 """203 """Run the complete test case.
212 Run the test case; including any build, setup, and cleanup commands.204
213 """205 This includes any build, setup, and cleanup commands.
214206
207 :returns: Whether to keep running tests (True) or reboot (False)
208 :rtype: bool
209
210 """
215 if self.is_done():211 if self.is_done():
216 return 'PASS'212 return 'PASS'
217213
@@ -280,9 +276,8 @@
280276
281 need_reboot = False277 need_reboot = False
282 if (278 if (
283 self.reboot == 'always'279 self.reboot == 'always' or
284 or self.reboot == 'pass'280 (self.reboot == 'pass' and result.status == 'PASS')
285 and result.status == 'PASS'
286 ):281 ):
287 need_reboot = True282 need_reboot = True
288283
@@ -299,25 +294,21 @@
299 return keep_going294 return keep_going
300295
301 def process_overrides(self, overrides):296 def process_overrides(self, overrides):
302 """297 """Set override values from a TestSuite runlist for this test case."""
303 Sets override values from a TestSuite runlist for this test case.
304 """
305 for key, value in overrides.iteritems():298 for key, value in overrides.iteritems():
306 setattr(self, key, value)299 setattr(self, key, value)
307300
308 def load_state(self, state):301 def load_state(self, state):
309 """302 """Restore state from the supplied dictionary.
310 Restore state from the supplied dictionary.
311303
312 Requires that 'state' has the same fieldnames as the TestCase class.304 Requires that 'state' has the same fieldnames as the TestCase class.
305
313 """306 """
314 for key, value in state.iteritems():307 for key, value in state.iteritems():
315 setattr(self, key, value)308 setattr(self, key, value)
316309
317 def save_state(self):310 def save_state(self):
318 """311 """Return a dictionary representing the test's state."""
319 Returns a dictionary representing the test's state.
320 """
321 state = {312 state = {
322 'name': self.name,313 'name': self.name,
323 'path': self.path,314 'path': self.path,
@@ -338,9 +329,13 @@
338 return state329 return state
339330
340 def is_done(self):331 def is_done(self):
341 """332 """Determine if the case is done.
342 Determine if the case is done. This might mean that something has333
343 failed. Used by suite to determine if the suite needs to be re-run334 This might mean that something has failed.
344 on resume.335 Used by suite to determine if the suite needs to be re-run on resume.
336
337 :returns: Whether the test case is finished (done or cleaned up)
338 :rtype: bool
339
345 """340 """
346 return self.status == 'DONE' or self.status == 'CLEANUP'341 return self.status == 'DONE' or self.status == 'CLEANUP'
347342
=== modified file 'utah/client/tests/__init__.py'
--- utah/client/tests/__init__.py 2012-12-08 02:10:12 +0000
+++ utah/client/tests/__init__.py 2013-04-08 19:17:22 +0000
@@ -12,3 +12,5 @@
1212
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
15
16"""utah.client.tests"""
1517
=== modified file 'utah/client/tests/common.py'
--- utah/client/tests/common.py 2013-03-21 20:46:21 +0000
+++ utah/client/tests/common.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Provide common functions and content for self tests."""
17
18
16import os19import os
17import shutil20import shutil
1821
@@ -20,8 +23,7 @@
2023
2124
22def get_module_path():25def get_module_path():
23 """26 """Return the path for the 'utah' directory.
24 Return the path for the 'utah' directory.
2527
26 This is used in tests to refer to the location where the branch resides.28 This is used in tests to refer to the location where the branch resides.
27 It is used in conjunction with running the self tests via:29 It is used in conjunction with running the self tests via:
@@ -29,6 +31,7 @@
29 sudo nosetests utah/client/tests31 sudo nosetests utah/client/tests
3032
31 from the top-level directory in the repo.33 from the top-level directory in the repo.
34
32 """35 """
33 import utah36 import utah
3437
@@ -38,6 +41,7 @@
3841
39 return module_path42 return module_path
4043
44
41master_runlist_content = """# utah/self_test.py master runlist45master_runlist_content = """# utah/self_test.py master runlist
42# needed for utah/self_test.py runs46# needed for utah/self_test.py runs
43---47---
@@ -64,11 +68,12 @@
6468
6569
66def setUp():70def setUp():
67 """71 """Create a sample master.run file.
68 Set up a master.run file that will copy our 'examples' directory to the72
69 testing directory (usually /var/lib/utah).73 This file will copy our 'examples' directory to the testing directory
70 """74 (usually /var/lib/utah).
7175
76 """
72 # If we're not root don't bother77 # If we're not root don't bother
73 if os.geteuid() != 0 and os.getuid() != 0:78 if os.geteuid() != 0 and os.getuid() != 0:
74 raise RuntimeError('These tests must be run as root')79 raise RuntimeError('These tests must be run as root')
@@ -92,10 +97,7 @@
9297
9398
94def tearDown():99def tearDown():
95 """100 """Clean up after ourselves."""
96 Cleanup after ourselves.
97 """
98
99 # restore the copy of the master runlist101 # restore the copy of the master runlist
100 if os.path.exists(master_runlist_bak):102 if os.path.exists(master_runlist_bak):
101 os.rename(master_runlist_bak, master_runlist)103 os.rename(master_runlist_bak, master_runlist)
@@ -107,10 +109,12 @@
107109
108110
109def _get_partial_state_file(filename):111def _get_partial_state_file(filename):
110 """112 """Read a partial state file from disk.
111 Read a partial state file from disk.113
112 """114 :returns: A pre-defined partially run test suite state.
113115 :rtype: dict
116
117 """
114 state = None118 state = None
115119
116 with open(filename, 'r') as fp:120 with open(filename, 'r') as fp:
117121
=== removed file 'utah/client/tests/manual_privileges.py'
--- utah/client/tests/manual_privileges.py 2012-12-08 02:10:12 +0000
+++ utah/client/tests/manual_privileges.py 1970-01-01 00:00:00 +0000
@@ -1,188 +0,0 @@
1# Ubuntu Testing Automation Harness
2# Copyright 2012 Canonical Ltd.
3
4# This program is free software: you can redistribute it and/or modify it
5# under the terms of the GNU General Public License version 3, as published
6# by the Free Software Foundation.
7
8# This program is distributed in the hope that it will be useful, but
9# WITHOUT ANY WARRANTY; without even the implied warranties of
10# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11# PURPOSE. See the GNU General Public License for more details.
12
13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.
15
16######################################################################
17### Privileges Tests
18######################################################################
19
20import os
21import pwd
22import grp
23import shlex
24import subprocess
25import unittest
26
27from utah.client.common import drop_privileges, raise_privileges
28
29NOBODY_UID = pwd.getpwnam('nobody').pw_uid
30NOGROUP_GID = grp.getgrnam('nogroup').gr_gid
31
32TMP_CONTENT = """Some data"""
33TMP_FILENAME = '/tmp/priv_test.tmp'
34
35
36def print_priv(msg=None):
37 """
38 Print out the current privileges.
39 """
40 if msg is not None:
41 print(msg)
42
43 print("uid: {}, euid: {}".format(os.getuid(), os.geteuid()))
44 print("gid: {}, egid: {}".format(os.getgid(), os.getegid()))
45
46
47def write_temp_file(filename=TMP_FILENAME):
48 """
49 Write the contents to a temp file.
50 """
51 with open(filename, 'w') as fp:
52 fp.write(TMP_CONTENT)
53 fp.close()
54
55
56def read_temp_file(filename=TMP_FILENAME):
57 """
58 Print the contents of a temp file.
59 """
60 with open(filename, 'r') as fp:
61 for line in fp:
62 print(line.strip())
63 fp.close()
64
65
66def _do_cmd(cmd, quiet=False, shell=False):
67 """
68 Run a command.
69 """
70 cmd_args = cmd
71
72 if not shell:
73 cmd_args = shlex.split(cmd)
74
75 proc = subprocess.Popen(
76 cmd_args,
77 shell=shell,
78 stdout=subprocess.PIPE,
79 stderr=subprocess.PIPE
80 )
81
82 (stdout, _stderr) = proc.communicate()
83
84 if not quiet:
85 print("{}:\n{}".format(cmd, stdout))
86
87
88def stat_file(filename=TMP_FILENAME):
89 proc = subprocess.Popen(['ls', '-al', filename], stdout=subprocess.PIPE,
90 stderr=subprocess.PIPE)
91
92 stdout, _stderr = proc.communicate()
93 print("{}: {}".format(filename, stdout))
94
95
96def get_privs():
97 privs = {}
98
99 privs['uid'] = os.getuid()
100 privs['euid'] = os.geteuid()
101 privs['gid'] = os.getgid()
102 privs['egid'] = os.getegid()
103
104 return privs
105
106
107class baseTestPriv(unittest.TestCase):
108
109 def assertRoot(self, msg="Must be root"):
110 self.assertEqual(os.geteuid(), 0, msg=msg)
111
112 def assertPrivs(self, privs, msg=None):
113 # Only check privileges passed in
114 if 'uid' in privs:
115 self.assertEqual(os.getuid(), privs['uid'], msg)
116
117 if 'euid' in privs:
118 self.assertEqual(os.geteuid(), privs['euid'], msg)
119
120 if 'gid' in privs:
121 self.assertEqual(os.getuid(), privs['uid'], msg)
122
123 if 'egid' in privs:
124 self.assertEqual(os.geteuid(), privs['euid'], msg)
125
126 def setUp(self):
127 print_priv("before")
128 _do_cmd("id")
129
130 def tearDown(self):
131 print_priv("after")
132 _do_cmd("id")
133
134
135class TestPriv(baseTestPriv):
136 def test_drop_priv(self):
137 """
138 Test that we drop effective privileges appropriately.
139 """
140 self.assertRoot()
141 old_privs = drop_privileges()
142
143 self.assertPrivs({'euid': NOBODY_UID, 'egid': NOGROUP_GID})
144
145 print_priv("middle")
146 _do_cmd("id")
147
148 write_temp_file()
149 read_temp_file()
150 stat_file()
151
152 raise_privileges(old_privs)
153
154 def test_restore_priv(self):
155 """
156 Test that we restore effective privileges appropriately.
157 """
158 self.assertRoot()
159 old_privs = drop_privileges()
160
161 self.assertPrivs({'euid': NOBODY_UID, 'egid': NOGROUP_GID})
162
163 print_priv("middle")
164
165 raise_privileges(old_privs)
166
167 self.assertPrivs(old_privs)
168
169
170class TestXXXLast(baseTestPriv):
171 """
172 Test's to be run last.
173 """
174 def test_xxx_drop_priv_full(self):
175 """
176 Test dropping all privileges.
177 """
178
179 self.assertRoot()
180 drop_privileges(full=True)
181 self.assertPrivs({
182 'uid': NOBODY_UID,
183 'euid': NOBODY_UID,
184 'gid': NOGROUP_GID,
185 'egid': NOGROUP_GID,
186 })
187
188 _do_cmd("id")
1890
=== modified file 'utah/client/tests/test_common.py'
--- utah/client/tests/test_common.py 2013-01-29 14:56:33 +0000
+++ utah/client/tests/test_common.py 2013-04-08 19:17:22 +0000
@@ -13,11 +13,15 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test utah client common functions."""
17
16import os18import os
19import signal
17import unittest20import unittest
1821
19# REQUIRES that the top level utah package be in the Python path.22# REQUIRES that the top level utah package be in the Python path.
20from utah.client.common import (23from utah.client.common import (
24 _get_process_children,
21 get_media_info,25 get_media_info,
22 get_product_uuid,26 get_product_uuid,
23 get_host_info,27 get_host_info,
@@ -28,12 +32,11 @@
2832
2933
30class TestCommon(unittest.TestCase):34class TestCommon(unittest.TestCase):
31 """35
32 Tests for utah.client.common.36 """Tests for utah.client.common."""
33 """
3437
35 def test_get_product_uuid(self):38 def test_get_product_uuid(self):
36 """ Test that product_uuid is returned when possible. """39 """Test that product_uuid is returned when possible."""
3740
38 product_uuid = get_product_uuid()41 product_uuid = get_product_uuid()
3942
@@ -41,9 +44,7 @@
41 self.assertIsNotNone(product_uuid)44 self.assertIsNotNone(product_uuid)
4245
43 def test_get_media_info(self):46 def test_get_media_info(self):
44 """47 """Test that if there is a media-info file that it is parsed."""
45 Test that if there is a media-info file that it is parsed.
46 """
47 media_info = get_media_info()48 media_info = get_media_info()
4849
49 print("media_info: {}".format(media_info))50 print("media_info: {}".format(media_info))
@@ -54,10 +55,7 @@
54 self.assertTrue(media_info == 'unknown')55 self.assertTrue(media_info == 'unknown')
5556
56 def test_get_host_info(self):57 def test_get_host_info(self):
57 """58 """Test that get_host_info returns results."""
58 Test that get_host_info returns results.
59 """
60
61 host_info = get_host_info()59 host_info = get_host_info()
6260
63 print("host_info: {}".format(host_info))61 print("host_info: {}".format(host_info))
@@ -65,14 +63,17 @@
65 self.assertTrue(host_info is not None)63 self.assertTrue(host_info is not None)
6664
67 def test_run_cmd(self):65 def test_run_cmd(self):
66 """Test running a command."""
68 result = run_cmd('find . -name "*.py"')67 result = run_cmd('find . -name "*.py"')
6968
70 self.assertEqual(result['returncode'], 0)69 self.assertEqual(result['returncode'], 0)
7170
72 def test_debug_print(self):71 def test_debug_print(self):
73 """72 """Test that debug_print prints at the appropriate times.
74 Test that debug_print doesn't print when DEBUG is False unless73
75 'debug' is passed in and is True.74 It shouldn't print when DEBUG is False unless 'debug' is passed in and
75 is True.
76
76 """77 """
77 old_debug = CONFIG['DEBUG']78 old_debug = CONFIG['DEBUG']
7879
@@ -94,9 +95,7 @@
94 CONFIG['DEBUG'] = old_debug95 CONFIG['DEBUG'] = old_debug
9596
96 def test_run_as(self):97 def test_run_as(self):
97 """98 """Test that run_as functions correctly."""
98 Test that run_as functions correctly.
99 """
100 user = "nobody"99 user = "nobody"
101 result = run_cmd("touch /tmp/run_as_test.tmp", run_as=user)100 result = run_cmd("touch /tmp/run_as_test.tmp", run_as=user)
102 print("result: {}".format(result))101 print("result: {}".format(result))
@@ -111,10 +110,7 @@
111 self.assertEqual(result['user'], user)110 self.assertEqual(result['user'], user)
112111
113 def test_run_as_default(self):112 def test_run_as_default(self):
114 """113 """Test that the default user is correctly identified."""
115 Test that the default user is correctly identified.
116 """
117
118 result = run_cmd("touch /tmp/run_as_test.tmp")114 result = run_cmd("touch /tmp/run_as_test.tmp")
119 print("result: {}".format(result))115 print("result: {}".format(result))
120116
@@ -126,3 +122,22 @@
126122
127 self.assertEqual(result['returncode'], 0)123 self.assertEqual(result['returncode'], 0)
128 self.assertNotEqual(result['user'], "unknown")124 self.assertNotEqual(result['user'], "unknown")
125
126 def test_get_process_children(self):
127 """Test that the get_process_children function works."""
128 children = []
129 for _ in xrange(3):
130 pid = os.fork()
131 if pid == 0:
132 # child to wait
133 os.execv('/bin/sleep', ['/bin/sleep', '5'])
134 children.append(pid)
135
136 ret = _get_process_children(str(os.getpid()))
137 # the last PID from childre will be the "ps" process we ran, so
138 # we can ignore that one
139 ret.pop()
140 for pid in children:
141 os.kill(pid, signal.SIGTERM)
142 os.waitpid(pid, 0)
143 self.assertEqual(children, ret)
129144
=== modified file 'utah/client/tests/test_jsonschema.py'
--- utah/client/tests/test_jsonschema.py 2012-12-11 09:30:00 +0000
+++ utah/client/tests/test_jsonschema.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test schema parsing functionality."""
17
18
16import jsonschema19import jsonschema
17import os20import os
18import unittest21import unittest
@@ -80,25 +83,22 @@
8083
81class TestJSONSchema(unittest.TestCase):84class TestJSONSchema(unittest.TestCase):
8285
86 """Test schema validation."""
87
83 def test_orig_schema(self):88 def test_orig_schema(self):
84 """89 """Test that the original master.run validates correctly."""
85 Test that the original master.run is correctly validated by the schema.
86 """
87
88 data = yaml.load(yaml_content)90 data = yaml.load(yaml_content)
89 print("data: {}".format(data))91 print("data: {}".format(data))
90 jsonschema.validate(data, Runner.MASTER_RUNLIST_SCHEMA)92 jsonschema.validate(data, Runner.MASTER_RUNLIST_SCHEMA)
9193
92 def test_multi_schema(self):94 def test_multi_schema(self):
93 """95 """Test that the new master.run validates correctly."""
94 Test that the new master.run is correctly validated by the schema.
95 """
96
97 data_new = yaml.load(yaml_content_new)96 data_new = yaml.load(yaml_content_new)
98 print("data_new: {}".format(data_new))97 print("data_new: {}".format(data_new))
99 jsonschema.validate(data_new, Runner.MASTER_RUNLIST_SCHEMA)98 jsonschema.validate(data_new, Runner.MASTER_RUNLIST_SCHEMA)
10099
101 def test_example_runlist(self):100 def test_example_runlist(self):
101 """Test that the example runlist validates correctly."""
102 example_runlist = os.path.join(os.path.dirname(__file__),102 example_runlist = os.path.join(os.path.dirname(__file__),
103 "../examples/master.run")103 "../examples/master.run")
104104
@@ -111,19 +111,13 @@
111 fp.close()111 fp.close()
112112
113 def test_include_schema(self):113 def test_include_schema(self):
114 """114 """Test that include works correctly."""
115 Test that include works correctly.
116 """
117
118 data = yaml.load(yaml_content_include)115 data = yaml.load(yaml_content_include)
119 print("data: {}".format(data))116 print("data: {}".format(data))
120 jsonschema.validate(data, Runner.MASTER_RUNLIST_SCHEMA)117 jsonschema.validate(data, Runner.MASTER_RUNLIST_SCHEMA)
121118
122 def test_bad_schemas(self):119 def test_bad_schemas(self):
123 """120 """Test that required fields are enforced by the schema."""
124 Test that required fields are enforced by the schema.
125 """
126
127 for content in yaml_content_bad:121 for content in yaml_content_bad:
128 data = yaml.load(content)122 data = yaml.load(content)
129 print("data: {}".format(data))123 print("data: {}".format(data))
130124
=== modified file 'utah/client/tests/test_misc.py'
--- utah/client/tests/test_misc.py 2012-12-03 14:02:18 +0000
+++ utah/client/tests/test_misc.py 2013-04-08 19:17:22 +0000
@@ -13,15 +13,21 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test miscellaneous functionality."""
17
18
16import os19import os
17import unittest20import unittest
1821
19from .common import get_module_path22from utah.client.tests.common import get_module_path
2023
2124
22class TestMisc(unittest.TestCase):25class TestMisc(unittest.TestCase):
26
27 """Test miscellaneous functionality."""
28
23 def test_get_module_path(self):29 def test_get_module_path(self):
2430 """Test getting the module path."""
25 print("__file__: {}".format(__file__))31 print("__file__: {}".format(__file__))
2632
27 # assumes this file is in 'utah/client/tests'33 # assumes this file is in 'utah/client/tests'
2834
=== modified file 'utah/client/tests/test_phoenix.py'
--- utah/client/tests/test_phoenix.py 2013-03-21 15:51:53 +0000
+++ utah/client/tests/test_phoenix.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test phoenix metadata generation tool."""
17
18
16import os19import os
17import shutil20import shutil
18import unittest21import unittest
@@ -23,21 +26,27 @@
2326
2427
25def setUp():28def setUp():
29 """Set up test prerequisite."""
26 if os.path.exists(directory_name):30 if os.path.exists(directory_name):
27 raise Exception("directory '{}' already exists".format(directory_name))31 raise Exception("directory '{}' already exists".format(directory_name))
2832
2933
30def tearDown():34def tearDown():
35 """Remove test resources."""
31 if os.path.exists(directory_name):36 if os.path.exists(directory_name):
32 shutil.rmtree(directory_name)37 shutil.rmtree(directory_name)
3338
3439
35class TestPhoenix(unittest.TestCase):40class TestPhoenix(unittest.TestCase):
41
42 """Test phoenix test suite creation utility."""
43
36 testsuite_name = "testsuite_name"44 testsuite_name = "testsuite_name"
37 testcases = ['test_one', 'test_two']45 testcases = ['test_one', 'test_two']
38 directory_name = directory_name46 directory_name = directory_name
3947
40 def setUp(self):48 def setUp(self):
49 """Set up test suite creation information."""
41 self.phoenix = Phoenix(50 self.phoenix = Phoenix(
42 self.testsuite_name,51 self.testsuite_name,
43 directory=self.directory_name,52 directory=self.directory_name,
@@ -45,61 +54,43 @@
45 )54 )
4655
47 def test_missing_required_args(self):56 def test_missing_required_args(self):
48 """57 """Test that __init__ fails without the required arguments."""
49 Test that __init__ fails without the required arguments
50 """
51 self.assertRaises(58 self.assertRaises(
52 TypeError,59 TypeError,
53 Phoenix,60 Phoenix,
54 )61 )
5562
56 def test_providing_required_args(self):63 def test_providing_required_args(self):
57 """64 """Test that __init__ succeeds with the required arguments."""
58 Test that __init__ succeeds with the required arguments.
59 """
60
61 self.assertTrue(self.phoenix is not None)65 self.assertTrue(self.phoenix is not None)
6266
63 def test_suite_name(self):67 def test_suite_name(self):
64 """68 """Test that the testsuite name is correctly set."""
65 Test that the testsuite name is correctly set.
66 """
67
68 self.assertEqual(self.phoenix.testsuite, self.testsuite_name)69 self.assertEqual(self.phoenix.testsuite, self.testsuite_name)
6970
70 def test_directory_name(self):71 def test_directory_name(self):
71 """72 """Test that the directory name is correctly set."""
72 Test that the directory name is correctly set.
73 """
74
75 self.assertEqual(self.phoenix.directory, self.directory_name)73 self.assertEqual(self.phoenix.directory, self.directory_name)
7674
77 def test_default_directory_name(self):75 def test_default_directory_name(self):
78 """76 """Test that the default directory name is correctly set."""
79 Test that the default directory name is correctly set.
80 """
81
82 phoenix = Phoenix(self.testsuite_name)77 phoenix = Phoenix(self.testsuite_name)
8378
84 self.assertEqual(phoenix.directory, '.')79 self.assertEqual(phoenix.directory, '.')
8580
86 def test_testcases(self):81 def test_testcases(self):
87 """82 """Test that the testcases are correctly set."""
88 Test that the testcases are correctly set.
89 """
90
91 self.assertListEqual(self.phoenix.testcases, self.testcases)83 self.assertListEqual(self.phoenix.testcases, self.testcases)
9284
93 def test_testcases_default(self):85 def test_testcases_default(self):
94 """86 """Test that the default testcase is correctly set."""
95 Test that the default testcase is correctly set.
96 """
9787
98 phoenix = Phoenix(self.testsuite_name)88 phoenix = Phoenix(self.testsuite_name)
99 # there should be only one default testcase.89 # there should be only one default testcase.
100 self.assertEqual(len(phoenix.testcases), 1)90 self.assertEqual(len(phoenix.testcases), 1)
10191
102 def test_build_suite(self):92 def test_build_suite(self):
93 """Test building a test suite."""
103 self.phoenix.build_suite()94 self.phoenix.build_suite()
10495
105 dir_path = self.directory_name96 dir_path = self.directory_name
@@ -118,9 +109,7 @@
118 "{} doesn't exist".format(path))109 "{} doesn't exist".format(path))
119110
120 def test_create_file(self):111 def test_create_file(self):
121 """112 """Test that Phoenix create file works correctly."""
122 Test that Phoenix create file works correctly.
123 """
124 fn = '/tmp/phoenix_create_file_test.file'113 fn = '/tmp/phoenix_create_file_test.file'
125 fc = 'a test file'114 fc = 'a test file'
126115
127116
=== modified file 'utah/client/tests/test_result.py'
--- utah/client/tests/test_result.py 2013-03-20 19:20:13 +0000
+++ utah/client/tests/test_result.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test our ability to report results."""
17
18
16import unittest19import unittest
17import yaml20import yaml
1821
@@ -23,7 +26,11 @@
2326
2427
25class TestResult(unittest.TestCase):28class TestResult(unittest.TestCase):
29
30 """Test result processing and display functionality."""
31
26 def setUp(self):32 def setUp(self):
33 """Create needed data structures."""
27 self.result_yaml = ResultYAML("result_yaml")34 self.result_yaml = ResultYAML("result_yaml")
28 self.result_text = Result("result")35 self.result_text = Result("result")
2936
@@ -44,58 +51,49 @@
44 }51 }
4552
46 def test_add_result(self):53 def test_add_result(self):
54 """Test adding test results."""
47 self.result_yaml.add_result(self.result)55 self.result_yaml.add_result(self.result)
4856
49 self.assertEqual(self.result_yaml.count_results(), 1)57 self.assertEqual(self.result_yaml._count_results(), 1)
5058
51 self.result_yaml.result()59 self.result_yaml.result()
5260
53 def test_result(self):61 def test_result(self):
62 """Test generating results."""
54 self.result_yaml.add_result(self.result)63 self.result_yaml.add_result(self.result)
5564
56 # check that the result was added.65 # check that the result was added.
57 self.assertEqual(self.result_yaml.count_results(), 1)66 self.assertEqual(self.result_yaml._count_results(), 1)
5867
59 self.result_yaml.result()68 self.result_yaml.result()
6069
61 # check that the result was processed and removed.70 # check that the result was processed and removed.
62 self.assertEqual(self.result_yaml.count_results(), 0)71 self.assertEqual(self.result_yaml._count_results(), 0)
6372
64 def test_result_pass(self):73 def test_result_pass(self):
65 """74 """Test that adding a pass result sets the status to 'PASS'."""
66 Test that adding a pass result sets the Result object's status
67 to 'PASS'.
68 """
69 self.result_yaml.add_result(self.result)75 self.result_yaml.add_result(self.result)
7076
71 self.assertEqual(self.result_yaml.status, 'PASS')77 self.assertEqual(self.result_yaml.status, 'PASS')
7278
73 def test_result_failure(self):79 def test_result_failure(self):
74 """80 """Test that adding a failure result sets the status to 'FAIL'."""
75 Test that adding a failure result sets the Result object's status
76 to 'ERROR'.
77 """
78 self.result_yaml.add_result(self.result_fail)81 self.result_yaml.add_result(self.result_fail)
7982
80 self.assertEqual(self.result_yaml.status, 'FAIL')83 self.assertEqual(self.result_yaml.status, 'FAIL')
8184
82 def test_result_clear_result(self):85 def test_result_clear_result(self):
83 """86 """Test that the result is actually cleared."""
84 Test that the result is actually cleared.
85 """
86 self.result_yaml.add_result(self.result_fail)87 self.result_yaml.add_result(self.result_fail)
87 self.assertEqual(self.result_yaml.status, 'FAIL')88 self.assertEqual(self.result_yaml.status, 'FAIL')
8889
89 self.result_yaml.clear_results()90 self.result_yaml._clear_results()
9091
91 self.assertEqual(self.result_yaml.status, 'PASS')92 self.assertEqual(self.result_yaml.status, 'PASS')
92 self.assertEqual(self.result_yaml.count_results(), 0)93 self.assertEqual(self.result_yaml._count_results(), 0)
9394
94 def test_result_result(self):95 def test_result_result(self):
95 """96 """Test that ERROR status is retained after adding pass results."""
96 Test that ERROR status is retained after adding pass
97 results.
98 """
99 self.result_yaml.add_result(self.result_fail)97 self.result_yaml.add_result(self.result_fail)
10098
101 self.assertEqual(self.result_yaml.status, 'FAIL')99 self.assertEqual(self.result_yaml.status, 'FAIL')
@@ -105,6 +103,7 @@
105 self.assertEqual(self.result_yaml.status, 'FAIL')103 self.assertEqual(self.result_yaml.status, 'FAIL')
106104
107 def test_result_yaml(self):105 def test_result_yaml(self):
106 """Test generating YAML results."""
108 self.result_yaml.add_result(self.result)107 self.result_yaml.add_result(self.result)
109108
110 import sys109 import sys
@@ -122,7 +121,7 @@
122121
123 result_str = str_output.getvalue()122 result_str = str_output.getvalue()
124123
125 print("result_str: %s" % result_str)124 print('result_str: {}'.format(result_str))
126125
127 data = yaml.load(result_str)126 data = yaml.load(result_str)
128127
@@ -130,7 +129,7 @@
130129
131 result = data['commands'][0]130 result = data['commands'][0]
132131
133 print("data: %s" % data)132 print('data: {}'.format(data))
134133
135 self.assertEqual(result['command'], self.result['command'])134 self.assertEqual(result['command'], self.result['command'])
136 self.assertEqual(result['returncode'], self.result['returncode'])135 self.assertEqual(result['returncode'], self.result['returncode'])
137136
=== modified file 'utah/client/tests/test_runner.py'
--- utah/client/tests/test_runner.py 2013-03-21 21:04:14 +0000
+++ utah/client/tests/test_runner.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Who tests the test runners?"""
17
18
16import os19import os
17import shutil20import shutil
18import unittest21import unittest
@@ -24,7 +27,7 @@
24from utah.client.common import ReturnCodes27from utah.client.common import ReturnCodes
25from utah.client import exceptions28from utah.client import exceptions
2629
27from .common import ( # NOQA30from utah.client.tests.common import ( # NOQA
28 setUp, # Used by nosetests31 setUp, # Used by nosetests
29 tearDown, # Used by nosetests32 tearDown, # Used by nosetests
30 master_runlist_content,33 master_runlist_content,
@@ -33,7 +36,11 @@
3336
3437
35class TestRunner(unittest.TestCase):38class TestRunner(unittest.TestCase):
39
40 """Test the TestRunner class."""
41
36 def setUp(self):42 def setUp(self):
43 """Set up test prerequisites."""
37 self.result_class = ResultYAML44 self.result_class = ResultYAML
38 self.state_file = '/tmp/state.yaml'45 self.state_file = '/tmp/state.yaml'
39 self.state_agent = StateAgentYAML(state_file=self.state_file)46 self.state_agent = StateAgentYAML(state_file=self.state_file)
@@ -46,7 +53,7 @@
46 "{} shouldn't exist".format(self.bad_dir))53 "{} shouldn't exist".format(self.bad_dir))
4754
48 def tearDown(self):55 def tearDown(self):
4956 """Clean up test resources."""
50 # Remove self.bad_dir if it got created.57 # Remove self.bad_dir if it got created.
51 try:58 try:
52 shutil.rmtree(self.bad_dir)59 shutil.rmtree(self.bad_dir)
@@ -58,6 +65,7 @@
58 raise65 raise
5966
60 def test_fetch_failed(self):67 def test_fetch_failed(self):
68 """Test handling of failed fetch commands."""
61 bad_runlist_content = """---69 bad_runlist_content = """---
62testsuites:70testsuites:
63 - name: fake_tests71 - name: fake_tests
@@ -75,17 +83,18 @@
75 os.remove(runlist)83 os.remove(runlist)
7684
77 runner.run()85 runner.run()
78 print runner.report()
7986
80 self.assertEqual(runner.fetch_errors, 1)87 self.assertEqual(runner.fetch_errors, 1)
8188
82 def test_rc_local(self):89 def test_rc_local(self):
90 """Test rc.local functionality for reboots."""
83 tmp_rc_local = '/tmp/rc.local'91 tmp_rc_local = '/tmp/rc.local'
84 self.runner.setup_rc_local(tmp_rc_local)92 self.runner.setup_rc_local(tmp_rc_local)
8593
86 self.assertTrue(os.path.exists(tmp_rc_local))94 self.assertTrue(os.path.exists(tmp_rc_local))
8795
88 def test_run(self):96 def test_run(self):
97 """Test running all test suites."""
89 retcode = self.runner.run()98 retcode = self.runner.run()
9099
91 for suite in self.runner.suites:100 for suite in self.runner.suites:
@@ -94,9 +103,7 @@
94 self.assertEqual(retcode, ReturnCodes.FAIL)103 self.assertEqual(retcode, ReturnCodes.FAIL)
95104
96 def test_missing_testdir(self):105 def test_missing_testdir(self):
97 """106 """Test that the runner fails gracefully on missing testdir."""
98 Test that the runner fails gracefully on missing testdir.
99 """
100 self.assertRaises(exceptions.BadDir,107 self.assertRaises(exceptions.BadDir,
101 Runner,108 Runner,
102 install_type="desktop",109 install_type="desktop",
@@ -104,10 +111,7 @@
104 testdir=self.bad_dir)111 testdir=self.bad_dir)
105112
106 def test_missing_master_runlist(self):113 def test_missing_master_runlist(self):
107 """114 """Test that the runner fails gracefully on missing 'master.run'."""
108 Test that the runner fails gracefully on missing 'master.run'
109 """
110
111 tmp_master_runlist = os.path.join(self.bad_dir, 'master.run')115 tmp_master_runlist = os.path.join(self.bad_dir, 'master.run')
112116
113 if not os.path.exists(self.bad_dir):117 if not os.path.exists(self.bad_dir):
@@ -125,9 +129,7 @@
125 testdir=self.bad_dir)129 testdir=self.bad_dir)
126130
127 def test_runlist(self):131 def test_runlist(self):
128 """132 """Test that passing a runlist to the Runner works properly."""
129 Test that passing a runlist to the Runner works properly.
130 """
131 runlist = '/tmp/master_runlist_test'133 runlist = '/tmp/master_runlist_test'
132134
133 with open(runlist, 'w') as fp:135 with open(runlist, 'w') as fp:
@@ -145,17 +147,14 @@
145 os.remove(runlist)147 os.remove(runlist)
146148
147 def test_testdir(self):149 def test_testdir(self):
148 """150 """Test that the default test directory is created."""
149 Test that the default test directory is created.
150 """
151
152 testdir = self.runner.testdir151 testdir = self.runner.testdir
153152
154 self.assertTrue(os.path.exists(testdir))153 self.assertTrue(os.path.exists(testdir))
155154
156 @patch('utah.client.runner.TestSuite')155 @patch('utah.client.runner.TestSuite')
157 def test_include_exclude(self, mock_class):156 def test_include_exclude(self, mock_class):
158 """Test that includes/excludes are passed to the test suite"""157 """Test that includes/excludes are passed to the test suite."""
159 include_runlist_content = """---158 include_runlist_content = """---
160testsuites:159testsuites:
161 - name: utah_tests160 - name: utah_tests
162161
=== modified file 'utah/client/tests/test_state_agent.py'
--- utah/client/tests/test_state_agent.py 2013-03-21 21:22:10 +0000
+++ utah/client/tests/test_state_agent.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test our ability to save and restore state information."""
17
18
16import os19import os
17import unittest20import unittest
18import yaml21import yaml
@@ -21,7 +24,7 @@
21from utah.client.runner import Runner24from utah.client.runner import Runner
22from utah.client.result import ResultYAML25from utah.client.result import ResultYAML
2326
24from utah.client.tests.common import (27from utah.client.tests.common import ( # NOQA
25 setUp, # Used by nosetests28 setUp, # Used by nosetests
26 tearDown, # Used by nosetests29 tearDown, # Used by nosetests
27 partial_state_file_content,30 partial_state_file_content,
@@ -31,10 +34,11 @@
3134
3235
33class TestStateAgentYAML(unittest.TestCase):36class TestStateAgentYAML(unittest.TestCase):
37
38 """Test the YAML state agent."""
39
34 def setUp(self):40 def setUp(self):
35 """41 """Create a state agent for testing."""
36 Create a state agent for testing.
37 """
38 self.state_file = '/tmp/state.yaml'42 self.state_file = '/tmp/state.yaml'
3943
40 if os.path.exists(self.state_file):44 if os.path.exists(self.state_file):
@@ -46,19 +50,18 @@
46 state_agent=self.state_agent)50 state_agent=self.state_agent)
4751
48 def tearDown(self):52 def tearDown(self):
49 """53 """Clean up the test state file."""
50 Clean up the test state file.
51 """
52
53 if os.path.exists(self.state_file):54 if os.path.exists(self.state_file):
54 os.remove(self.state_file)55 os.remove(self.state_file)
5556
56 def test_creation(self):57 def test_creation(self):
58 """Test that state file is correctly created."""
57 self.runner.run()59 self.runner.run()
5860
59 self.assertTrue(os.path.exists(self.state_file))61 self.assertTrue(os.path.exists(self.state_file))
6062
61 def test_cleanup(self):63 def test_cleanup(self):
64 """Test that cleanup removes state file."""
62 self.runner.save_state()65 self.runner.save_state()
6366
64 self.assertTrue(os.path.exists(self.state_file),67 self.assertTrue(os.path.exists(self.state_file),
@@ -67,12 +70,11 @@
67 self.state_agent.clean()70 self.state_agent.clean()
6871
69 self.assertFalse(os.path.exists(self.state_file),72 self.assertFalse(os.path.exists(self.state_file),
70 "Failed to remove state file (%s)" % self.state_file)73 'Failed to remove state file ({})'
74 .format(self.state_file))
7175
72 def test_status(self):76 def test_status(self):
73 """77 """Test that a runner properly saves its state during test runs."""
74 Test that a runner properly saves its state during test runs.
75 """
76 suite_count = self.runner.count_suites()78 suite_count = self.runner.count_suites()
77 test_count = self.runner.count_tests()79 test_count = self.runner.count_tests()
7880
@@ -121,21 +123,15 @@
121 self.assertEqual(runner.passes + runner.failures + runner.errors, t)123 self.assertEqual(runner.passes + runner.failures + runner.errors, t)
122124
123 def test_load_state_partial(self):125 def test_load_state_partial(self):
124 """126 """Test that a partially run state file can be resumed."""
125 Test that a partially run state file can be resumed.
126 """
127 self._test_partial(partial_state_file_content, 'test_two', 4, 1, 0)127 self._test_partial(partial_state_file_content, 'test_two', 4, 1, 0)
128128
129 def test_load_state_partial_run_all(self):129 def test_load_state_partial_run_all(self):
130 """130 """Test partially run state file - run all jobs."""
131 Test partially run state file - run all jobs.
132 """
133 self._test_partial(131 self._test_partial(
134 partial_state_file_content_run_all, 'test_one', 3, 2, 0)132 partial_state_file_content_run_all, 'test_one', 3, 2, 0)
135133
136 def test_load_state_partial_done_all_failed(self):134 def test_load_state_partial_done_all_failed(self):
137 """135 """Test partially run state file - all jobs failed."""
138 Test partially run state file - all jobs failed.
139 """
140 self._test_partial(136 self._test_partial(
141 partial_state_file_content_done_all_failed, None, 0, 5, 0)137 partial_state_file_content_done_all_failed, None, 0, 5, 0)
142138
=== modified file 'utah/client/tests/test_testcase.py'
--- utah/client/tests/test_testcase.py 2013-02-11 19:41:45 +0000
+++ utah/client/tests/test_testcase.py 2013-04-08 19:17:22 +0000
@@ -13,27 +13,30 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test handling of test cases."""
17
18
16import os19import os
17import unittest20import unittest
1821
22from utah.client import testcase
23from utah.client.common import (
24 UTAH_DIR,
25)
19from utah.client.exceptions import MissingData26from utah.client.exceptions import MissingData
20from utah.client.result import ResultYAML27from utah.client.result import ResultYAML
21from utah.client.common import (28from utah.client.tests.common import ( # NOQA
22 UTAH_DIR,
23)
24from utah.client import testcase
25
26from .common import ( # NOQA
27 setUp, # Used by nosetests29 setUp, # Used by nosetests
28 tearDown, # Used by nosetests30 tearDown, # Used by nosetests
29)31)
3032
3133
32class TestTestCase(unittest.TestCase):34class TestTestCase(unittest.TestCase):
33 """35
34 Tests for the TestCase class.36 """Tests for the TestCase class."""
35 """37
36 def setUp(self):38 def setUp(self):
39 """Initialize basic test case data."""
37 self.timeout = 1040 self.timeout = 10
38 self.command = 'echo "command"'41 self.command = 'echo "command"'
39 self.result = ResultYAML()42 self.result = ResultYAML()
@@ -53,9 +56,7 @@
53 )56 )
5457
55 def test_run(self):58 def test_run(self):
56 """59 """Test that a TestCase can be run."""
57 Test that a TestCase can be run.
58 """
59 self.case.run()60 self.case.run()
6061
61 # make sure after a testcase run that the cwd hasn't changed.62 # make sure after a testcase run that the cwd hasn't changed.
@@ -63,12 +64,15 @@
63 self.assertTrue(self.case.is_done())64 self.assertTrue(self.case.is_done())
6465
65 def test_timeout(self):66 def test_timeout(self):
67 """Test that timeouts are parsed correctly."""
66 self.assertEqual(self.case.timeout, self.timeout)68 self.assertEqual(self.case.timeout, self.timeout)
6769
68 def test_command(self):70 def test_command(self):
71 """Test that commands are parsed correctly."""
69 self.assertEqual(self.case.command, self.command)72 self.assertEqual(self.case.command, self.command)
7073
71 def test_save_state(self):74 def test_save_state(self):
75 """Test that state can be saved."""
72 state = self.case.save_state()76 state = self.case.save_state()
7377
74 self.assertEqual(state['status'], 'NOTRUN')78 self.assertEqual(state['status'], 'NOTRUN')
@@ -76,6 +80,7 @@
76 self.assertEqual(state['timeout'], self.timeout)80 self.assertEqual(state['timeout'], self.timeout)
7781
78 def test_save_and_load_state(self):82 def test_save_and_load_state(self):
83 """Test that state can be saved and loaded."""
79 state = self.case.save_state()84 state = self.case.save_state()
8085
81 case = testcase.TestCase(86 case = testcase.TestCase(
@@ -97,6 +102,7 @@
97 self.assertTrue(case.is_done())102 self.assertTrue(case.is_done())
98103
99 def test_timeout_override(self):104 def test_timeout_override(self):
105 """Test that overriding timeouts works."""
100 timeout = 99106 timeout = 99
101 case = testcase.TestCase(107 case = testcase.TestCase(
102 name=self.name,108 name=self.name,
@@ -108,10 +114,7 @@
108 self.assertEqual(timeout, case.timeout)114 self.assertEqual(timeout, case.timeout)
109115
110 def test_control_data(self):116 def test_control_data(self):
111 """117 """Test that valid control_data doesn't raise an exception."""
112 Test that a correctly formed control_data that's passed in
113 doesn't raise an exception.
114 """
115 control_data = {118 control_data = {
116 'description': 'a test case',119 'description': 'a test case',
117 'command': '/bin/true',120 'command': '/bin/true',
@@ -137,10 +140,7 @@
137 self.assertTrue(case.is_done())140 self.assertTrue(case.is_done())
138141
139 def test_bad_control_data(self):142 def test_bad_control_data(self):
140 """143 """Test that bad control_data does raise an exception."""
141 Test that an incorrectly formed control_data that's passed in
142 does raise an exception.
143 """
144 control_data = {144 control_data = {
145 'description': 'a test case',145 'description': 'a test case',
146 'command': '/bin/true',146 'command': '/bin/true',
147147
=== modified file 'utah/client/tests/test_testsuite.py'
--- utah/client/tests/test_testsuite.py 2012-12-11 09:47:51 +0000
+++ utah/client/tests/test_testsuite.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test handling of test suites."""
17
18
16import os19import os
17import unittest20import unittest
1821
@@ -22,24 +25,23 @@
22from utah.client.result import ResultYAML25from utah.client.result import ResultYAML
23from utah.client import testsuite26from utah.client import testsuite
2427
25from .common import ( # NOQA28from utah.client.tests.common import ( # NOQA
26 setUp, # Used by nosetests29 setUp, # Used by nosetests
27 tearDown, # Used by nosetests30 tearDown, # Used by nosetests
28)31)
2932
3033
31def state_saver():34def state_saver():
35 """Provide state saving functionality for testing."""
32 debug_print("Saving state", force=True)36 debug_print("Saving state", force=True)
3337
3438
35class TestTestSuite(unittest.TestCase):39class TestTestSuite(unittest.TestCase):
36 """40
37 Tests for the TestSuite class.41 """Tests for the TestSuite class."""
38 """42
39 def setUp(self):43 def setUp(self):
40 """44 """Use the 'examples' test suite that's part of the UTAH package."""
41 Use the 'examples' test suite that's part of the UTAH package.
42 """
43 self.name = 'examples'45 self.name = 'examples'
44 self.path = '/var/lib/utah/testsuites'46 self.path = '/var/lib/utah/testsuites'
45 self.assertTrue(os.path.exists(os.path.join(self.path, self.name)))47 self.assertTrue(os.path.exists(os.path.join(self.path, self.name)))
@@ -49,23 +51,28 @@
49 _save_state_callback=state_saver)51 _save_state_callback=state_saver)
5052
51 def test_run(self):53 def test_run(self):
52 print "found %d tests" % self.suite.count_tests()54 """Test running a test suite."""
55 print 'found {} tests'.format(str(self.suite.count_tests()))
53 self.suite.run()56 self.suite.run()
54 self.assertEqual(self.suite.result.status, 'PASS')57 self.assertEqual(self.suite.result.status, 'PASS')
5558
56 def test_count_tests(self):59 def test_count_tests(self):
60 """Test ability of test suite to count tests."""
57 self.assertEquals(self.suite.count_tests(), 2)61 self.assertEquals(self.suite.count_tests(), 2)
5862
59 def test_next_test(self):63 def test_next_test(self):
64 """Test ability of test suite to indicate next test to run."""
60 next_test = self.suite.get_next_test()65 next_test = self.suite.get_next_test()
6166
62 self.assertTrue(hasattr(next_test, 'name'))67 self.assertTrue(hasattr(next_test, 'name'))
63 self.assertEquals(self.suite.get_next_test().name, 'test_one')68 self.assertEquals(self.suite.get_next_test().name, 'test_one')
6469
65 def test_is_done(self):70 def test_is_done(self):
71 """Test test suite completion reporting."""
66 self.assertFalse(self.suite.is_done())72 self.assertFalse(self.suite.is_done())
6773
68 def test_save_state(self):74 def test_save_state(self):
75 """Test saving test suite state."""
69 state = self.suite.save_state()76 state = self.suite.save_state()
7077
71 print state78 print state
@@ -79,6 +86,7 @@
79 self.assertEqual(tests, self.suite.count_tests())86 self.assertEqual(tests, self.suite.count_tests())
8087
81 def test_timeout_override(self):88 def test_timeout_override(self):
89 """Test overriding a timeout value."""
82 timeout = 10190 timeout = 101
83 suite = testsuite.TestSuite(91 suite = testsuite.TestSuite(
84 name=self.name,92 name=self.name,
@@ -94,7 +102,7 @@
94 self.assertEqual(timeout, test.timeout)102 self.assertEqual(timeout, test.timeout)
95103
96 def test_include(self):104 def test_include(self):
97 """Included tests are executed"""105 """Verify that included tests are executed."""
98 included_test_name = 'test_one'106 included_test_name = 'test_one'
99107
100 suite = testsuite.TestSuite(108 suite = testsuite.TestSuite(
@@ -112,7 +120,7 @@
112 self.assertEqual(case.name, included_test_name)120 self.assertEqual(case.name, included_test_name)
113121
114 def test_exclude(self):122 def test_exclude(self):
115 """Excluded tests are *not* executed"""123 """Verify that excluded tests are not executed."""
116 excluded_test_name = 'test_two'124 excluded_test_name = 'test_two'
117125
118 suite = testsuite.TestSuite(126 suite = testsuite.TestSuite(
@@ -130,7 +138,7 @@
130 self.assertNotEqual(case.name, excluded_test_name)138 self.assertNotEqual(case.name, excluded_test_name)
131139
132 def test_include_exclude(self):140 def test_include_exclude(self):
133 """Include/exclude combination works fine"""141 """Verify that include/exclude combination works fine."""
134 included_test_name = 'test_one'142 included_test_name = 'test_one'
135 excluded_test_name = 'test_two'143 excluded_test_name = 'test_two'
136144
137145
=== modified file 'utah/client/tests/test_vcs_bzr.py'
--- utah/client/tests/test_vcs_bzr.py 2012-12-12 11:22:58 +0000
+++ utah/client/tests/test_vcs_bzr.py 2013-04-08 19:17:22 +0000
@@ -13,20 +13,22 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test bzr interface."""
17
18
16import os19import os
17import shutil20import shutil
18import unittest21import unittest
1922
20from utah.client.common import BzrHandler, run_cmd23from utah.client.common import run_cmd
24from utah.client.vcs import BzrHandler
25
2126
22BZR_TEST_REPO = "/tmp/utahbzrtest"27BZR_TEST_REPO = "/tmp/utahbzrtest"
2328
2429
25def setUp():30def setUp():
26 """31 """Create a local bzr repository for testing."""
27 Create a local bzr repository to test with.
28 """
29
30 # should fail if the repo already exists.32 # should fail if the repo already exists.
31 os.mkdir(BZR_TEST_REPO)33 os.mkdir(BZR_TEST_REPO)
3234
@@ -37,24 +39,17 @@
3739
3840
39def tearDown():41def tearDown():
40 """42 """Remove the test repo."""
41 Remove the test repo.
42 """
43
44 # We should only ever get here if the repo was created in setUp().43 # We should only ever get here if the repo was created in setUp().
45 shutil.rmtree(BZR_TEST_REPO)44 shutil.rmtree(BZR_TEST_REPO)
4645
4746
48class TestBzrHandler(unittest.TestCase):47class TestBzrHandler(unittest.TestCase):
49 """48
50 Test the bazaar VCS handler.49 """Test the bazaar VCS handler."""
51 """
5250
53 def test_init(self):51 def test_init(self):
54 """52 """Test the initial setup."""
55 Test the initial setup.
56 """
57
58 repo = BZR_TEST_REPO53 repo = BZR_TEST_REPO
59 bzr = BzrHandler(repo=repo)54 bzr = BzrHandler(repo=repo)
6055
@@ -64,6 +59,7 @@
64 self.assertEqual(bzr.rev_command, "bzr revno")59 self.assertEqual(bzr.rev_command, "bzr revno")
6560
66 def test_bzr_get(self):61 def test_bzr_get(self):
62 """Test retrieving a bzr repo."""
67 repo = BZR_TEST_REPO63 repo = BZR_TEST_REPO
68 dest = "utah-dev"64 dest = "utah-dev"
69 bzr = BzrHandler(repo=repo, destination=dest)65 bzr = BzrHandler(repo=repo, destination=dest)
@@ -79,6 +75,7 @@
79 shutil.rmtree(path)75 shutil.rmtree(path)
8076
81 def test_bzr_revision(self):77 def test_bzr_revision(self):
78 """Test getting revision information from a bzr repo."""
82 repo = BZR_TEST_REPO79 repo = BZR_TEST_REPO
83 dest = "utah-dev"80 dest = "utah-dev"
84 bzr = BzrHandler(repo=repo, destination=dest)81 bzr = BzrHandler(repo=repo, destination=dest)
@@ -92,7 +89,7 @@
9289
93 self.assertTrue(os.path.exists(path), path)90 self.assertTrue(os.path.exists(path), path)
9491
95 result = bzr.get_revision(directory=path)92 result = bzr.revision(directory=path)
9693
97 shutil.rmtree(path)94 shutil.rmtree(path)
98 self.assertIsInstance(int(result), int)95 self.assertEqual(result['returncode'], 0)
9996
=== modified file 'utah/client/tests/test_vcs_dev.py'
--- utah/client/tests/test_vcs_dev.py 2012-12-12 11:22:58 +0000
+++ utah/client/tests/test_vcs_dev.py 2013-04-08 19:17:22 +0000
@@ -13,20 +13,22 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test the dev pseudo-VCS interface."""
17
18
16import os19import os
17import shutil20import shutil
18import unittest21import unittest
1922
20from utah.client.common import DevHandler, run_cmd23from utah.client.common import run_cmd
24from utah.client.vcs import DevHandler
25
2126
22DEV_TEST_REPO = "/tmp/utahdevtest"27DEV_TEST_REPO = "/tmp/utahdevtest"
2328
2429
25def setUp():30def setUp():
26 """31 """Create a local directory for testing."""
27 Create a local directory to test on.
28 """
29
30 # should fail if the repo already exists.32 # should fail if the repo already exists.
31 os.mkdir(DEV_TEST_REPO)33 os.mkdir(DEV_TEST_REPO)
3234
@@ -34,24 +36,17 @@
3436
3537
36def tearDown():38def tearDown():
37 """39 """Remove the test repo."""
38 Remove the test repo.
39 """
40
41 # We should only ever get here if the repo was created in setUp().40 # We should only ever get here if the repo was created in setUp().
42 shutil.rmtree(DEV_TEST_REPO)41 shutil.rmtree(DEV_TEST_REPO)
4342
4443
45class TestDevHandler(unittest.TestCase):44class TestDevHandler(unittest.TestCase):
46 """45
47 Test the bazaar VCS handler.46 """Test the dev pseudo-VCS handler."""
48 """
4947
50 def test_init(self):48 def test_init(self):
51 """49 """Test the initial setup."""
52 Test the initial setup.
53 """
54
55 repo = DEV_TEST_REPO50 repo = DEV_TEST_REPO
56 dev = DevHandler(repo=repo)51 dev = DevHandler(repo=repo)
5752
@@ -61,6 +56,7 @@
61 self.assertEqual(dev.rev_command, "echo 'DEVELOPMENT'")56 self.assertEqual(dev.rev_command, "echo 'DEVELOPMENT'")
6257
63 def test_dev_get(self):58 def test_dev_get(self):
59 """Test copying files."""
64 repo = DEV_TEST_REPO60 repo = DEV_TEST_REPO
65 dest = "utah-dev"61 dest = "utah-dev"
66 dev = DevHandler(repo=repo, destination=dest)62 dev = DevHandler(repo=repo, destination=dest)
@@ -76,6 +72,7 @@
76 shutil.rmtree(path)72 shutil.rmtree(path)
7773
78 def test_dev_revision(self):74 def test_dev_revision(self):
75 """Test fake revision gathering support."""
79 repo = DEV_TEST_REPO76 repo = DEV_TEST_REPO
80 dest = "utah-dev"77 dest = "utah-dev"
81 dev = DevHandler(repo=repo, destination=dest)78 dev = DevHandler(repo=repo, destination=dest)
@@ -87,7 +84,7 @@
87 self.assertEqual(result['returncode'], 0)84 self.assertEqual(result['returncode'], 0)
88 self.assertTrue(os.path.exists(path))85 self.assertTrue(os.path.exists(path))
8986
90 result = dev.get_revision(directory=path)87 result = dev.revision(directory=path)
9188
92 shutil.rmtree(path)89 shutil.rmtree(path)
93 self.assertIsInstance(result, str)90 self.assertEqual(result['returncode'], 0)
9491
=== modified file 'utah/client/tests/test_vcs_git.py'
--- utah/client/tests/test_vcs_git.py 2012-12-12 11:22:58 +0000
+++ utah/client/tests/test_vcs_git.py 2013-04-08 19:17:22 +0000
@@ -13,20 +13,22 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test the git interface."""
17
18
16import os19import os
17import shutil20import shutil
18import unittest21import unittest
1922
20from utah.client.common import GitHandler, run_cmd23from utah.client.common import run_cmd
24from utah.client.vcs import GitHandler
25
2126
22GIT_TEST_REPO = "/tmp/utahgittest"27GIT_TEST_REPO = "/tmp/utahgittest"
2328
2429
25def setUp():30def setUp():
26 """31 """Create a local git repository for testing."""
27 Create a local git repository to test on.
28 """
29
30 # should fail if the repo already exists.32 # should fail if the repo already exists.
31 os.mkdir(GIT_TEST_REPO)33 os.mkdir(GIT_TEST_REPO)
3234
@@ -37,24 +39,17 @@
3739
3840
39def tearDown():41def tearDown():
40 """42 """Remove the test repo."""
41 Remove the test repo.
42 """
43
44 # We should only ever get here if the repo was created in setUp().43 # We should only ever get here if the repo was created in setUp().
45 shutil.rmtree(GIT_TEST_REPO)44 shutil.rmtree(GIT_TEST_REPO)
4645
4746
48class TestGitHandler(unittest.TestCase):47class TestGitHandler(unittest.TestCase):
49 """48
50 Test the bazaar VCS handler.49 """Test the bazaar VCS handler."""
51 """
5250
53 def test_init(self):51 def test_init(self):
54 """52 """Test the initial setup."""
55 Test the initial setup.
56 """
57
58 repo = GIT_TEST_REPO53 repo = GIT_TEST_REPO
59 git = GitHandler(repo=repo)54 git = GitHandler(repo=repo)
6055
@@ -64,6 +59,7 @@
64 self.assertEqual(git.rev_command, "git rev-parse HEAD")59 self.assertEqual(git.rev_command, "git rev-parse HEAD")
6560
66 def test_git_get(self):61 def test_git_get(self):
62 """Test retrieving a git repo."""
67 repo = GIT_TEST_REPO63 repo = GIT_TEST_REPO
68 dest = "utah-dev"64 dest = "utah-dev"
69 git = GitHandler(repo=repo, destination=dest)65 git = GitHandler(repo=repo, destination=dest)
@@ -79,6 +75,7 @@
79 shutil.rmtree(path)75 shutil.rmtree(path)
8076
81 def test_git_revision(self):77 def test_git_revision(self):
78 """Test getting revision information from a git repo."""
82 repo = GIT_TEST_REPO79 repo = GIT_TEST_REPO
83 dest = "utah-dev"80 dest = "utah-dev"
84 git = GitHandler(repo=repo, destination=dest)81 git = GitHandler(repo=repo, destination=dest)
@@ -90,7 +87,7 @@
90 self.assertEqual(result['returncode'], 0)87 self.assertEqual(result['returncode'], 0)
91 self.assertTrue(os.path.exists(path))88 self.assertTrue(os.path.exists(path))
9289
93 result = git.get_revision(directory=path)90 result = git.revision(directory=path)
9491
95 shutil.rmtree(path)92 shutil.rmtree(path)
96 self.assertIsInstance(result, str)93 self.assertEqual(result['returncode'], 0)
9794
=== modified file 'utah/client/tests/test_yaml.py'
--- utah/client/tests/test_yaml.py 2012-12-08 02:10:12 +0000
+++ utah/client/tests/test_yaml.py 2013-04-08 19:17:22 +0000
@@ -13,12 +13,19 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Test YAML functionality."""
17
18
16import unittest19import unittest
17import yaml20import yaml
1821
1922
20class TestYAML(unittest.TestCase):23class TestYAML(unittest.TestCase):
24
25 """Test YAML processing."""
26
21 def test_parts(self):27 def test_parts(self):
28 """Test initial YAML processing."""
22 content = {}29 content = {}
23 content['metadata'] = {30 content['metadata'] = {
24 'runlist': '/tmp/master.run',31 'runlist': '/tmp/master.run',
@@ -41,6 +48,7 @@
41 print(dec_data)48 print(dec_data)
4249
43 def test_parts_2(self):50 def test_parts_2(self):
51 """Test additional YAML processing."""
44 content = {52 content = {
45 'file': '/tmp/file'53 'file': '/tmp/file'
46 }54 }
@@ -76,6 +84,7 @@
76 print(dec_data)84 print(dec_data)
7785
78 def test_parts_3(self):86 def test_parts_3(self):
87 """Test final YAML processing."""
79 extra_args = {}88 extra_args = {}
80 content = [89 content = [
81 {90 {
8291
=== modified file 'utah/client/testsuite.py'
--- utah/client/testsuite.py 2013-03-14 20:36:41 +0000
+++ utah/client/testsuite.py 2013-04-08 19:17:22 +0000
@@ -13,9 +13,8 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Testsuite specific code."""
17Testsuite specific code.17
18"""
1918
20import jsonschema19import jsonschema
21import os20import os
@@ -36,8 +35,11 @@
3635
3736
38def parse_runlist_file(runlist_file):37def parse_runlist_file(runlist_file):
39 """38 """Parse a tslist.run runlist file and check against schema.
40 Parse a tslist.run runlist file and check agains schema39
40 :returns: Parsed data from the runlist
41 :rtype: dict
42
41 """43 """
42 with open(runlist_file, 'r') as fp:44 with open(runlist_file, 'r') as fp:
43 data = yaml.load(fp)45 data = yaml.load(fp)
@@ -48,9 +50,8 @@
4850
4951
50class TestSuite(object):52class TestSuite(object):
51 """53
52 The TestSuite class.54 """Base class describing a test suite."""
53 """
5455
55 build_cmd = None56 build_cmd = None
56 timeout = None57 timeout = None
@@ -138,7 +139,7 @@
138 control_data = None139 control_data = None
139 except jsonschema.ValidationError as exception:140 except jsonschema.ValidationError as exception:
140 raise exceptions.ValidationError(141 raise exceptions.ValidationError(
141 '{!r} test suite control file invalid: {!r}\n1'142 '{!r} test suite control file invalid: {!r}\n'
142 'Detailed information: {}'143 'Detailed information: {}'
143 .format(self.name, self.control_file, exception))144 .format(self.name, self.control_file, exception))
144145
@@ -192,19 +193,15 @@
192 self.tests.append(tc)193 self.tests.append(tc)
193194
194 def __str__(self):195 def __str__(self):
195 return "%s: %s" % (self.control_file, self.runlist_file)196 return "{}: {}".format(self.control_file, self.runlist_file)
196197
197 def set_status(self, status):198 def set_status(self, status):
198 """199 """Set the status and call the save state callback."""
199 Set the status for the test suite and call the save state callback.
200 """
201 self.status = status200 self.status = status
202 self.save_state_callback()201 self.save_state_callback()
203202
204 def build(self, result):203 def build(self, result):
205 """204 """Run build, but only if we haven't started a run yet."""
206 Run build, but only if we haven't started a run yet.
207 """
208 if self.status == 'NOTRUN':205 if self.status == 'NOTRUN':
209 self.set_status('BUILD')206 self.set_status('BUILD')
210 cmd_result = run_cmd(207 cmd_result = run_cmd(
@@ -215,9 +212,7 @@
215 result.add_result(cmd_result)212 result.add_result(cmd_result)
216213
217 def setup(self, result):214 def setup(self, result):
218 """215 """Run ts_setup, but only if build() has just passed."""
219 Run ts_setup, but only if build() has just passed.
220 """
221 if self.status == 'BUILD' and result.status == 'PASS':216 if self.status == 'BUILD' and result.status == 'PASS':
222 self.set_status('SETUP')217 self.set_status('SETUP')
223 cmd_result = run_cmd(218 cmd_result = run_cmd(
@@ -228,9 +223,7 @@
228 result.add_result(cmd_result)223 result.add_result(cmd_result)
229224
230 def cleanup(self, result):225 def cleanup(self, result):
231 """226 """Run ts_cleanup after a run."""
232 Run ts_cleanup after a run.
233 """
234 if self.status == 'INPROGRESS':227 if self.status == 'INPROGRESS':
235 self.set_status('CLEANUP')228 self.set_status('CLEANUP')
236 cmd_result = run_cmd(229 cmd_result = run_cmd(
@@ -241,8 +234,14 @@
241 result.add_result(cmd_result)234 result.add_result(cmd_result)
242235
243 def run(self):236 def run(self):
244 """237 """Run the complete test suite.
245 Run the test cases; including any build, setup, and cleanup commands.238
239 This includes any build, setup, and cleanup commands, as well as all
240 test cases (including build, setup, and cleanup.)
241
242 :returns: Whether to keep running tests (True) or reboot (False)
243 :rtype: bool
244
246 """245 """
247246
248 # we gather this info from the testcases247 # we gather this info from the testcases
@@ -291,9 +290,7 @@
291 return keep_going290 return keep_going
292291
293 def add_test(self, tests):292 def add_test(self, tests):
294 """293 """Add a single test or list of tests to this suite."""
295 Add a single test or list of tests to this suite.
296 """
297 if isinstance(tests, TestCase):294 if isinstance(tests, TestCase):
298 self.tests.append(tests)295 self.tests.append(tests)
299 else:296 else:
@@ -304,16 +301,17 @@
304 pass301 pass
305302
306 def count_tests(self):303 def count_tests(self):
304 """Return the number of test cases in the suite."""
307 return len(self.tests)305 return len(self.tests)
308306
309 def load_state(self, state):307 def load_state(self, state):
310 """308 """Restore our state from the supplied dictionary.
311 Restore our state from the supplied dictionary.
312309
313 Requires that the fieldnames in the dictionary match the class310 Requires that the fieldnames in the dictionary match the class
314 properties.311 properties.
312
315 """313 """
316 # XXX: Should this be done explicitly?314 # TODO: do this explicitly
317 self.__dict__.update(state)315 self.__dict__.update(state)
318316
319 self.tests = []317 self.tests = []
@@ -326,6 +324,7 @@
326 self.tests.append(test)324 self.tests.append(test)
327325
328 def save_state(self):326 def save_state(self):
327 """Return a dictionary representing the suite's state."""
329 state = {328 state = {
330 'name': self.name,329 'name': self.name,
331 'status': self.status,330 'status': self.status,
@@ -345,20 +344,23 @@
345 return state344 return state
346345
347 def is_done(self):346 def is_done(self):
348 """347 """Determine if the suite is done.
349 Determine if the suite is done. This might mean that something has348
350 failed. Used by Runner to determine if the suite needs to be re-run349 This might mean that something has failed.
351 on resume.350 Used by Runner to determine if the suite needs to be re-run on resume.
351
352 :returns: Whether the test suite is finished (done or cleaned up)
353 :rtype: bool
354
352 """355 """
353 return self.status == 'DONE' or self.status == 'CLEANUP'356 return self.status == 'DONE' or self.status == 'CLEANUP'
354357
355 def get_next_test(self):358 def get_next_test(self):
356 """359 """Return the next test to be run.
357 Return the next test to be run.
358360
359 Mainly used for debugging.361 Mainly used for debugging.
362
360 """363 """
361
362 test = None364 test = None
363365
364 for t in self.tests:366 for t in self.tests:
365367
=== added file 'utah/client/vcs.py'
--- utah/client/vcs.py 1970-01-01 00:00:00 +0000
+++ utah/client/vcs.py 2013-04-08 19:17:22 +0000
@@ -0,0 +1,144 @@
1# Ubuntu Testing Automation Harness
2# Copyright 2012 Canonical Ltd.
3
4# This program is free software: you can redistribute it and/or modify it
5# under the terms of the GNU General Public License version 3, as published
6# by the Free Software Foundation.
7
8# This program is distributed in the hope that it will be useful, but
9# WITHOUT ANY WARRANTY; without even the implied warranties of
10# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11# PURPOSE. See the GNU General Public License for more details.
12
13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.
15
16"""UTAH client VCS support."""
17
18
19from utah.client.common import (
20 CMD_TS_FETCH,
21 run_cmd,
22)
23
24
25REPO_DEFAULT_DIR = "."
26
27
28class VCSHandler(object):
29
30 """Base class for Version Control System support.
31
32 :param repo: Repository location
33 :type repo: str
34 :param destination:
35 Local directory where the repository should be made available
36 :type destination: str
37 :param battery_measurements:
38 Whether battery information should be gathered when running any of the
39 VCS commands
40 :type battery_measurements: bool
41
42 """
43
44 def __init__(self, repo, destination='', battery_measurements=False):
45 self.repo = repo
46 self.destination = destination
47 self.battery_measurements = battery_measurements
48
49 def get(self, directory=REPO_DEFAULT_DIR):
50 """Execute VCS get command.
51
52 The get command will make a copy of a given repository to the directory
53 specified as destination.
54
55 :param directory: Working directory
56 :type directory: str
57 :returns: Get command execution result
58 :rtype: dict
59
60 .. seealso:: :func:`run_cmd`
61
62 """
63 return run_cmd(self.get_command,
64 cwd=directory,
65 cmd_type=CMD_TS_FETCH,
66 battery_measurements=self.battery_measurements)
67
68 def revision(self, directory=REPO_DEFAULT_DIR):
69 """Execute revision command.
70
71 :param directory: Working directory
72 :type directory: str
73 :returns: Revision command execution result.
74 :rtype: dict
75
76 """
77 return run_cmd(self.rev_command,
78 cwd=directory,
79 cmd_type=CMD_TS_FETCH,
80 battery_measurements=self.battery_measurements)
81
82
83class BzrHandler(VCSHandler):
84
85 """Bazaar VCS handler."""
86
87 def __init__(self, repo, branch=True, options="", destination="",
88 **kwargs):
89 super(BzrHandler, self).__init__(repo, destination, **kwargs)
90 self.options = options
91
92 if branch:
93 self.get_command = "bzr branch {} {} {}".format(
94 options,
95 self.repo,
96 self.destination,
97 )
98
99 self.rev_command = "bzr revno"
100 else:
101 self.get_command = "bzr export {} {} {}".format(
102 options,
103 self.destination,
104 self.repo,
105 )
106
107 self.rev_command = "bzr revno {}".format(self.repo)
108
109
110class GitHandler(VCSHandler):
111
112 """Git VCS handler."""
113
114 def __init__(self, repo, options="", destination="", **kwargs):
115 super(GitHandler, self).__init__(repo, destination, **kwargs)
116 self.options = options
117
118 self.get_command = "git clone {} {} {}".format(
119 options,
120 self.repo,
121 self.destination,
122 )
123
124 self.rev_command = "git rev-parse HEAD"
125
126
127class DevHandler(VCSHandler):
128
129 """Development VCS handler.
130
131 This isn't a handler for any VCS, but a helper to ease development and let
132 runlists point to a path that can be copied locally.
133
134 """
135
136 def __init__(self, repo, destination=".", **kwargs):
137 super(DevHandler, self).__init__(repo, destination, **kwargs)
138
139 self.get_command = "cp -r {} {}".format(
140 self.repo,
141 self.destination,
142 )
143
144 self.rev_command = "echo 'DEVELOPMENT'"
0145
=== modified file 'utah/commandstr.py'
--- utah/commandstr.py 2012-12-03 14:02:18 +0000
+++ utah/commandstr.py 2013-04-08 19:17:22 +0000
@@ -13,19 +13,17 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Provide stringification for commands used by timeout and retry."""
17Provide stringification for commands used by timeout and retry.
18"""
1917
2018
21def commandstr(command, *args, **kw):19def commandstr(command, *args, **kw):
22 """20 """Provide a user-oriented command display.
23 Convert a command and argument lists into a representation of that command21
24 suitable for display to a user.
25 i.e. commandstr(command, arg1, arg2, kw1=1, kw2=2)22 i.e. commandstr(command, arg1, arg2, kw1=1, kw2=2)
26 should return 'command(arg1, arg2, kw1=1, kw2=2)'23 should return 'command(arg1, arg2, kw1=1, kw2=2)'
24
27 """25 """
28 msg = command.__name__ + '('26 msg = '{}('.format(command.__name__)
29 for arg in args:27 for arg in args:
30 try:28 try:
31 msg += arg.__name__29 msg += arg.__name__
3230
=== modified file 'utah/config.py'
--- utah/config.py 2013-03-29 12:26:45 +0000
+++ utah/config.py 2013-04-08 19:17:22 +0000
@@ -1,4 +1,4 @@
1#!/usr/bin/python1#!/usr/bin/env python
22
3# Ubuntu Testing Automation Harness3# Ubuntu Testing Automation Harness
4# Copyright 2012 Canonical Ltd.4# Copyright 2012 Canonical Ltd.
@@ -15,10 +15,11 @@
15# You should have received a copy of the GNU General Public License along15# You should have received a copy of the GNU General Public License along
16# with this program. If not, see <http://www.gnu.org/licenses/>.16# with this program. If not, see <http://www.gnu.org/licenses/>.
1717
18"""18"""Provide config variables via utah.config.
19Provide config variables via utah.config.19
20When run directly as part of the package build, output machine-independent20When run directly as part of the package build, output machine-independent
21defaults as json for writing a config file.21defaults as json for writing a config file.
22
22"""23"""
2324
24import getpass25import getpass
@@ -36,9 +37,10 @@
3637
3738
38def getbridge():39def getbridge():
39 """40 """Get virbr0 if it is set in the qemu config or the utah config.
40 Get virbr0 if it is set in the qemu config or the utah config.41
41 Use this (or the default virbr0) as the bridge for bridged network VMs.42 Use this (or the default virbr0) as the bridge for bridged network VMs.
43
42 """44 """
43 dns = os.path.join('/', 'etc', 'utah', 'dns')45 dns = os.path.join('/', 'etc', 'utah', 'dns')
44 if os.path.isfile(dns):46 if os.path.isfile(dns):
@@ -235,13 +237,12 @@
235237
236# These depend on other config options, so they're added last.238# These depend on other config options, so they're added last.
237# Default logfile is /var/log/utah/{hostname}.log239# Default logfile is /var/log/utah/{hostname}.log
238LOCALDEFAULTS['logfile'] = os.path.join(DEFAULTS['logpath'],240LOCALDEFAULTS['logfile'] = os.path.join(
239 LOCALDEFAULTS['hostname'] + '.log')241 DEFAULTS['logpath'], '{}.log'.format(LOCALDEFAULTS['hostname']))
240# Default logfile is /var/log/utah/{hostname}-debug.log242# Default logfile is /var/log/utah/{hostname}-debug.log
241# Set to None to disable separate debug log243# Set to None to disable separate debug log
242LOCALDEFAULTS['debuglog'] = os.path.join(244LOCALDEFAULTS['debuglog'] = os.path.join(
243 DEFAULTS['logpath'],245 DEFAULTS['logpath'], '{}-debug.log'.format(LOCALDEFAULTS['hostname']))
244 LOCALDEFAULTS['hostname'] + '-debug.log')
245246
246# Log file used to log SSH related information247# Log file used to log SSH related information
247LOCALDEFAULTS['ssh_logfile'] = \248LOCALDEFAULTS['ssh_logfile'] = \
@@ -314,7 +315,8 @@
314 try:315 try:
315 CONFIG.update(json.load(fp))316 CONFIG.update(json.load(fp))
316 except ValueError:317 except ValueError:
317 sys.stderr.write(conffile + ' is not a valid JSON file')318 sys.stderr.write('{} is not a valid JSON file'
319 .format(conffile))
318320
319321
320# Support ~-based paths on variables that use paths322# Support ~-based paths on variables that use paths
321323
=== modified file 'utah/exceptions.py'
--- utah/exceptions.py 2013-03-15 14:59:35 +0000
+++ utah/exceptions.py 2013-04-08 19:17:22 +0000
@@ -13,16 +13,17 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Provide exceptions for UTAH."""
17Provide exceptions for the rest of the tool.
18"""
1917
2018
21class UTAHException(Exception):19class UTAHException(Exception):
22 """20
23 Provide a foundation class for UTAH exceptions.21 """Provide a foundation class for UTAH exceptions.
24 Support retry argument, but default to False.22
25 """23 Support retry and external arguments, but default to False.
24
25 """
26
26 def __init__(self, *args, **kw):27 def __init__(self, *args, **kw):
27 self.retry = kw.pop('retry', False)28 self.retry = kw.pop('retry', False)
28 self.external = kw.pop('external', False)29 self.external = kw.pop('external', False)
2930
=== modified file 'utah/group.py'
--- utah/group.py 2012-12-03 14:02:18 +0000
+++ utah/group.py 2013-04-08 19:17:22 +0000
@@ -13,9 +13,9 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Provide unix group checking functionality."""
17Unix group checking functionality17
18"""18
19import grp19import grp
20import os20import os
21import sys21import sys
@@ -25,9 +25,7 @@
2525
2626
27def check_user_group(group=config.group):27def check_user_group(group=config.group):
28 """28 """Return whether the user is a member of the given group."""
29 Return if the user is a member of the given group
30 """
31 user_gids = os.getgroups()29 user_gids = os.getgroups()
32 gid = grp.getgrnam(group).gr_gid30 gid = grp.getgrnam(group).gr_gid
3331
@@ -35,16 +33,14 @@
3533
3634
37def print_group_error_message(script):35def print_group_error_message(script):
38 """36 """Print error message to stderr to be used by scripts."""
39 Print error message to stderr to be used by scripts
40 """
41 argv = list(sys.argv)37 argv = list(sys.argv)
42 argv[0] = os.path.abspath(script)38 argv[0] = os.path.abspath(script)
43 message = ["Error: you are not in the utah group.",39 message = ["Error: you are not in the utah group.",
44 ("If you believe you have properly configured "40 ("If you believe you have properly configured "
45 "your user account for UTAH use, try:"),41 "your user account for UTAH use, try:"),
46 ' sudo usermod -a -G utah ' + getpass.getuser(),42 ' sudo usermod -a -G utah {}'.format(getpass.getuser()),
47 "Otherwise, please run this script as the utah user, i.e.:",43 "Otherwise, please run this script as the utah user, i.e.:",
48 " sudo su - utah -c '" + ' '.join(argv) + "'\n"]44 " sudo su - utah -c '{}'\n".format(' '.join(argv))]
49 message = '\n'.join(message)45 message = '\n'.join(message)
50 sys.stderr.write(message)46 sys.stderr.write(message)
5147
=== modified file 'utah/iso.py'
--- utah/iso.py 2013-03-20 19:17:23 +0000
+++ utah/iso.py 2013-04-08 19:17:22 +0000
@@ -33,10 +33,13 @@
3333
3434
35class UTAHISOException(UTAHException):35class UTAHISOException(UTAHException):
36
37 """Provide an exception specific to ISO errors."""
38
36 pass39 pass
3740
3841
39def get_resource(method, url, *args, **kw):42def _get_resource(method, url, *args, **kw):
40 try:43 try:
41 resource = method(url, *args, **kw)44 resource = method(url, *args, **kw)
42 return resource45 return resource
@@ -51,9 +54,9 @@
5154
5255
53class ISO(object):56class ISO(object):
54 """57
55 Provide a simplified method of interfacing with images.58 """Provide a simplified method of interfacing with images."""
56 """59
57 def __init__(self, arch=None, dlpercentincrement=None, dlretries=None,60 def __init__(self, arch=None, dlpercentincrement=None, dlretries=None,
58 image=None, installtype=None, logger=None, series=None):61 image=None, installtype=None, logger=None, series=None):
59 if dlpercentincrement is None:62 if dlpercentincrement is None:
@@ -69,22 +72,21 @@
69 else:72 else:
70 self.logger = logger73 self.logger = logger
71 if image is None:74 if image is None:
72 self.logger.debug('Need to retrieve image')
73 path = self.downloadiso(arch=arch, installtype=installtype,75 path = self.downloadiso(arch=arch, installtype=installtype,
74 series=series)76 series=series)
75 else:77 else:
76 if image.startswith('~'):78 if image.startswith('~'):
77 path = os.path.expanduser(image)79 path = os.path.expanduser(image)
78 self.logger.debug('Rewriting ~-based path: ' + image +80 self.logger.debug('Rewriting ~-based path: %s to: %s', image,
79 ' to: ' + path)81 path)
80 else:82 else:
81 path = image83 path = image
8284
83 self.percent = 085 self.percent = 0
84 self.logger.info('Preparing image: ' + path)86 self.logger.info('Preparing image: %s', path)
85 self.image = get_resource(urllib.urlretrieve, path,87 self.image = _get_resource(urllib.urlretrieve, path,
86 reporthook=self.dldisplay)[0]88 reporthook=self.dldisplay)[0]
87 self.logger.debug(path + ' is locally available as ' + self.image)89 self.logger.info('%s is locally available as %s', path, self.image)
88 self.installtype = self.getinstalltype()90 self.installtype = self.getinstalltype()
89 if self.installtype == 'mini':91 if self.installtype == 'mini':
90 self.logger.debug('Using mini image')92 self.logger.debug('Using mini image')
@@ -104,9 +106,7 @@
104 self.buildnumber = self.getbuildnumber()106 self.buildnumber = self.getbuildnumber()
105107
106 def _loggersetup(self):108 def _loggersetup(self):
107 """109 """Initialize the logging for the image."""
108 Initialize the logging for the image.
109 """
110 self.logger = logging.getLogger('utah.iso')110 self.logger = logging.getLogger('utah.iso')
111 self.logger.propagate = False111 self.logger.propagate = False
112 self.logger.setLevel(logging.INFO)112 self.logger.setLevel(logging.INFO)
@@ -119,8 +119,8 @@
119 self.consolehandler.setLevel(config.consoleloglevel)119 self.consolehandler.setLevel(config.consoleloglevel)
120 self.logger.addHandler(self.consolehandler)120 self.logger.addHandler(self.consolehandler)
121 self.filehandler = logging.handlers.WatchedFileHandler(config.logfile)121 self.filehandler = logging.handlers.WatchedFileHandler(config.logfile)
122 file_formatter = logging.Formatter('%(asctime)s iso '122 file_formatter = logging.Formatter(
123 + ' %(levelname)s: %(message)s')123 '%(asctime)s iso %(levelname)s: %(message)s')
124 self.filehandler.setFormatter(file_formatter)124 self.filehandler.setFormatter(file_formatter)
125 self.filehandler.setLevel(config.fileloglevel)125 self.filehandler.setLevel(config.fileloglevel)
126 self.logger.addHandler(self.filehandler)126 self.logger.addHandler(self.filehandler)
@@ -133,13 +133,16 @@
133 self.logger.addHandler(self.debughandler)133 self.logger.addHandler(self.debughandler)
134134
135 def getinstalltype(self):135 def getinstalltype(self):
136 """136 """Inspect the image's files to get the image type.
137 Inspect the image's files to get the image type.137
138 If .disk/mini-info exists, it's mini.138 If .disk/mini-info exists, it's mini.
139 If the casper directory exists, it's desktop.139 If the casper directory exists, it's desktop.
140 If ubuntu-server.seed exists in the preseeds directory, it's server.140 If ubuntu-server.seed exists in the preseeds directory, it's server.
141 :returns: Image type
142 :rtype: str
143
141 """144 """
142 self.logger.info('Getting image type of ' + self.image)145 self.logger.info('Getting image type of %s', self.image)
143 files = set(self.listfiles(returnlist=True))146 files = set(self.listfiles(returnlist=True))
144 installtype = 'alternate'147 installtype = 'alternate'
145 if '.disk/mini-info' in files:148 if '.disk/mini-info' in files:
@@ -149,53 +152,76 @@
149 elif ('preseed/ubuntu-server.seed' in files or152 elif ('preseed/ubuntu-server.seed' in files or
150 './preseed/ubuntu-server.seed' in files):153 './preseed/ubuntu-server.seed' in files):
151 installtype = 'server'154 installtype = 'server'
152 self.logger.info('Image type is: ' + installtype)155 self.logger.info('Image type is: %s', installtype)
153 return installtype156 return installtype
154157
155 def getarch(self):158 def getarch(self):
156 """159 """Unpack the image's info file to get the arch.
157 Unpack the image's info file to get the arch.160
161 :returns: Image architecture
162 :rtype: str
163
158 """164 """
159 arch = self.media_info.split()[-2]165 arch = self.media_info.split()[-2]
160 self.logger.info('Arch is: ' + arch)166 self.logger.info('Arch is: %s', arch)
161 return arch167 return arch
162168
163 def getseries(self):169 def getseries(self):
164 """170 """Unpack the image's info file to get the series.
165 Unpack the image's info file to get the series.171
172 :returns: Image series
173 :rtype: str
174
166 """175 """
167 for word in self.media_info.split():176 for word in self.media_info.split():
168 if word.startswith('"'):177 if word.startswith('"'):
169 series = word.strip('"').lower()178 series = word.strip('"').lower()
170 break179 break
171 self.logger.info('Series is ' + series)180 self.logger.info('Series is %s', series)
172 return series181 return series
173182
174 def getbuildnumber(self):183 def getbuildnumber(self):
175 """Unpack the image's info file to get the build number."""184 """Unpack the image's info file to get the build number.
185
186 :returns: Build number of the image.
187 :rtype: str
188
189 """
176 match = re.search('.*\(([0-9.]+)\)$', self.media_info)190 match = re.search('.*\(([0-9.]+)\)$', self.media_info)
177 build_number = match.group(1) if match else '?'191 build_number = match.group(1) if match else '?'
178 return build_number192 return build_number
179193
180 def dldisplay(self, blocks, size, total):194 def dldisplay(self, blocks, size, total):
181 """195 """Log download information (i.e., as a urllib callback).
182 Display download information.196
183 Intended for use as a urllib callback.197 :param blocks: Number of blocks downloaded
198 :type blocks: int
199 :param size: Size of blocks downloaded
200 :type size: int
201 :param total: Total size of download
202 :type size: int
203
184 """204 """
185 read = blocks * size205 read = blocks * size
186 percent = 100 * read / total206 percent = 100 * read / total
187 if percent >= self.percent:207 if percent >= self.percent:
188 self.logger.info('File ' + str(percent) + '% downloaded')208 self.logger.info('File %s%% downloaded', percent)
189 self.percent += self.dlpercentincrement209 self.percent += self.dlpercentincrement
190 self.logger.debug("%s read, %s%% of %s total" % (read, percent, total))210 self.logger.debug('%s read, %s%% of %s total', read, percent, total)
191211
192 def listfiles(self, returnlist=False):212 def listfiles(self, returnlist=False):
193 """213 """Return the contents of the ISO.
214
194 Return either a subprocess instance listing the contents of an ISO, or215 Return either a subprocess instance listing the contents of an ISO, or
195 a list of files in the ISO if returnlist is True.216 a list of files in the ISO if returnlist is True.
217 :returns: The contents of the ISO
218 :rtype: list or object
219
196 """220 """
221 # TODO: See if we still need this outside of validation
222 # move it to validation if it's only needed there
197 cmd = ['bsdtar', '-t', '-f', self.image]223 cmd = ['bsdtar', '-t', '-f', self.image]
198 self.logger.debug('bsdtar list command: ' + ' '.join(cmd))224 self.logger.debug('bsdtar list command: %s', ' '.join(cmd))
199 if returnlist:225 if returnlist:
200 try:226 try:
201 proc = subprocess.check_output(cmd).strip().split('\n')227 proc = subprocess.check_output(cmd).strip().split('\n')
@@ -223,7 +249,7 @@
223249
224 """250 """
225 cmd = ['bsdtar', '-t', '-v', '-f', self.image, filename]251 cmd = ['bsdtar', '-t', '-v', '-f', self.image, filename]
226 self.logger.debug('bsdtar list command: ' + ' '.join(cmd))252 self.logger.debug('bsdtar list command: %s', ' '.join(cmd))
227 try:253 try:
228 output = subprocess.check_output(cmd)254 output = subprocess.check_output(cmd)
229 except (OSError, subprocess.CalledProcessError) as err:255 except (OSError, subprocess.CalledProcessError) as err:
@@ -237,7 +263,7 @@
237 'for {} in {}: {}'263 'for {} in {}: {}'
238 .format(filename, self.image, output))264 .format(filename, self.image, output))
239 cmd = ['bsdtar', '-x', '-f', self.image, '-O', realfile]265 cmd = ['bsdtar', '-x', '-f', self.image, '-O', realfile]
240 self.logger.debug('bsdtar extract command: ' + ' '.join(cmd))266 self.logger.debug('bsdtar extract command: %s', ' '.join(cmd))
241 return cmd267 return cmd
242268
243 def extract(self, filename, outdir='', outfile=None, **kw):269 def extract(self, filename, outdir='', outfile=None, **kw):
@@ -293,25 +319,39 @@
293319
294 return stdout320 return stdout
295321
296 def getmd5(self, path):322 def getmd5(self, path=None):
297 """323 """Return the MD5 checksum of a file.
298 Return the md5 checksum of a file.324
299 Default file is this image.325 Default file is this image.
326 :param path: Path of file to checksum
327 :type path: str
328 :returns: MD5 checksum of file
329 :rtype: str
330
300 """331 """
301 if path is None:332 if path is None:
302 path = self.image333 path = self.image
303 self.logger.debug('Getting md5 of ' + path)334 self.logger.debug('Getting md5 of %s', path)
304 isohash = md5()335 isohash = md5()
305 with open(path) as myfile:336 with open(path) as myfile:
306 for block in iter(lambda: myfile.read(128), ""):337 for block in iter(lambda: myfile.read(128), ""):
307 isohash.update(block)338 isohash.update(block)
308 filemd5 = isohash.hexdigest()339 filemd5 = isohash.hexdigest()
309 self.logger.debug('md5 of ' + path + ' is ' + filemd5)340 self.logger.debug('md5 of %s is %s', path, filemd5)
310 return filemd5341 return filemd5
311342
312 def downloadiso(self, arch=None, installtype=None, series=None):343 def downloadiso(self, arch=None, installtype=None, series=None):
313 """344 """Download an ISO given series, type, and arch.
314 Download an ISO given series, type, and arch.345
346 :param arch: Architecture of image to download
347 :type arch: str
348 :param installtype: Install type of image to download
349 :type installtype: str
350 :param series: Series codename of image to download
351 :type series: str
352 :returns: Local path of downloaded image
353 :rtype: str
354
315 """355 """
316 if arch is None:356 if arch is None:
317 arch = config.arch357 arch = config.arch
@@ -319,26 +359,26 @@
319 installtype = config.installtype359 installtype = config.installtype
320 if series is None:360 if series is None:
321 series = config.series361 series = config.series
322 filename = '-'.join([series, installtype, arch]) + '.iso'362 filename = '{}.iso'.format('-'.join([series, installtype, arch]))
323 self.logger.info('Attempting to retrieve ' + filename)363 self.logger.info('Attempting to retrieve %s', filename)
324 with open(os.devnull, "w") as fnull:364 with open(os.devnull, "w") as fnull:
325 # If dlcommand (default dl-ubuntu-test-iso) is available, use it365 # If dlcommand (default dl-ubuntu-test-iso) is available, use it
326 cmd = ['which', config.dlcommand]366 cmd = ['which', config.dlcommand]
327 try:367 try:
328 if subprocess.call(cmd,368 if subprocess.call(cmd,
329 stdout=fnull, stderr=fnull) == 0:369 stdout=fnull, stderr=fnull) == 0:
330 self.logger.debug('Using ' + config.dlcommand)370 self.logger.debug('Using %s', config.dlcommand)
331 if installtype == 'server':371 if installtype == 'server':
332 flavor = 'ubuntu-server'372 flavor = 'ubuntu-server'
333 else:373 else:
334 flavor = 'ubuntu'374 flavor = 'ubuntu'
335 cmd = [config.dlcommand,375 cmd = [config.dlcommand,
336 '-q',376 '-q',
337 '--flavor=' + flavor,377 '--flavor={}'.format(flavor),
338 '--release=' + series,378 '--release={}'.format(series),
339 '--variant=' + installtype,379 '--variant={}'.format(installtype),
340 '--arch=' + arch,380 '--arch={}'.format(arch),
341 '--isoroot=' + config.isodir]381 '--isoroot={}'.format(config.isodir)]
342 self.logger.info('Downloading ISO')382 self.logger.info('Downloading ISO')
343 self.logger.debug(' '.join(cmd))383 self.logger.debug(' '.join(cmd))
344 try:384 try:
@@ -361,7 +401,7 @@
361 # First, we'll check the image. If it matches, we return401 # First, we'll check the image. If it matches, we return
362 # If not, we'll download it, and start the loop over402 # If not, we'll download it, and start the loop over
363 for attempt in range(config.dlretries):403 for attempt in range(config.dlretries):
364 self.logger.info('Download attempt ' + str(attempt))404 self.logger.info('Download attempt %s', str(attempt))
365 # Set the path to our files and the name of the iso we want405 # Set the path to our files and the name of the iso we want
366 if installtype == 'mini':406 if installtype == 'mini':
367 remotepath = ('http://archive.ubuntu.com/ubuntu/dists/'407 remotepath = ('http://archive.ubuntu.com/ubuntu/dists/'
@@ -371,39 +411,34 @@
371 else:411 else:
372 remotepath = ('http://releases.ubuntu.com/'412 remotepath = ('http://releases.ubuntu.com/'
373 '{series}/'.format(series=series))413 '{series}/'.format(series=series))
374 isopattern = installtype + '-' + arch + '.iso'414 isopattern = '{}-{}.iso'.format(installtype, arch)
375415
376 # Scan the MD5SUMS file for our file
377 servermd5 = None416 servermd5 = None
378 md5path = '{}/MD5SUMS'.format(remotepath)417 md5path = '{}/MD5SUMS'.format(remotepath)
379 md5list = get_resource(urllib.urlopen, md5path)418 md5list = _get_resource(urllib.urlopen, md5path)
380 for line in md5list:419 for line in md5list:
381 if isopattern in line:420 if isopattern in line:
382 servermd5, isofile = line.split()421 servermd5, isofile = line.split()
383 isofile = isofile.strip('*')422 isofile = isofile.strip('*')
384 break423 break
385424
386 # If we can't find the ISO, raise an exception
387 if servermd5 is None:425 if servermd5 is None:
388 raise UTAHISOException(426 raise UTAHISOException(
389 'Specified ISO: {} not found on mirrors.'427 'Specified ISO: {} not found on mirrors.'
390 .format(filename), external=True)428 .format(filename), external=True)
391 self.logger.debug('Server md5 is ' + servermd5)429 self.logger.debug('Server md5 is %s', servermd5)
392430
393 # If the ISO exists locally and the md5 matches, use it
394 if os.path.isfile(path) and servermd5 == self.getmd5(path):431 if os.path.isfile(path) and servermd5 == self.getmd5(path):
395 self.logger.info('Using ISO at ' + path)432 self.logger.info('Using ISO at %s', path)
396 return path433 return path
397434
398 # Try to download the ISO
399 isopath = os.path.join(remotepath, isofile)435 isopath = os.path.join(remotepath, isofile)
400 self.percent = 0436 self.percent = 0
401 self.logger.info('Attempting to download ' + isopath)437 self.logger.info('Attempting to download %s', isopath)
402 temppath = get_resource(urllib.urlretrieve, isopath,438 temppath = _get_resource(urllib.urlretrieve, isopath,
403 reporthook=self.dldisplay)[0]439 reporthook=self.dldisplay)[0]
404440
405 # Try to copy it into our cache441 self.logger.debug('Copying %s to %s', temppath, path)
406 self.logger.debug('Copying ' + temppath + ' to ' + path)
407 shutil.copyfile(temppath, path)442 shutil.copyfile(temppath, path)
408 else:443 else:
409 if os.path.isfile(path) and servermd5 == self.getmd5(path):444 if os.path.isfile(path) and servermd5 == self.getmd5(path):
@@ -414,6 +449,12 @@
414 .format(config.dlretries), external=True)449 .format(config.dlretries), external=True)
415450
416 def kernelpath(self):451 def kernelpath(self):
452 """Return the path of the kernel inside the image.
453
454 :returns: Path to the kernel pulled from bootloader config
455 :rtype: str
456
457 """
417 kernelpath = './install/vmlinuz'458 kernelpath = './install/vmlinuz'
418 if self.installtype == 'mini':459 if self.installtype == 'mini':
419 self.logger.debug('Getting kernel for mini image')460 self.logger.debug('Getting kernel for mini image')
@@ -437,8 +478,8 @@
437 self.logger.debug('Rejecting '478 self.logger.debug('Rejecting '
438 'memtest kernel')479 'memtest kernel')
439 else:480 else:
440 self.logger.debug('Found kernel: '481 self.logger.debug(
441 + newkernel)482 'Found kernel: %s', newkernel)
442 kernels[newkernel] += 1483 kernels[newkernel] += 1
443 # Now we have a list of kernel paths and the number of times484 # Now we have a list of kernel paths and the number of times
444 # each once occurs. We'll use the one that occurs most.485 # each once occurs. We'll use the one that occurs most.
445486
=== modified file 'utah/isotest/iso_static_validation.py'
--- utah/isotest/iso_static_validation.py 2013-01-03 11:51:00 +0000
+++ utah/isotest/iso_static_validation.py 2013-04-08 19:17:22 +0000
@@ -1,4 +1,4 @@
1#!/usr/bin/python1#!/usr/bin/env python
22
3# Ubuntu Testing Automation Harness3# Ubuntu Testing Automation Harness
4# Copyright 2012 Canonical Ltd.4# Copyright 2012 Canonical Ltd.
@@ -35,14 +35,14 @@
35# <http://www.gnu.org/licenses/>.35# <http://www.gnu.org/licenses/>.
36#36#
3737
38# TODO: support for non-daily images
39
40"""ISO static validation.38"""ISO static validation.
4139
42The test cases in this scripts are used to make sure that an ISO image isn't40The test cases in this scripts are used to make sure that an ISO image isn't
43broken and can be used for smoke testing.41broken and can be used for smoke testing.
4442
45"""43"""
44# TODO: support for non-daily images
45
4646
47import os47import os
48import argparse48import argparse
@@ -143,14 +143,16 @@
143 description=self.shortDescription() or ''))143 description=self.shortDescription() or ''))
144 self.block_size = ONE_MB_BLOCK144 self.block_size = ONE_MB_BLOCK
145 self.iso_location = iso_location145 self.iso_location = iso_location
146 logging.debug('Using iso at: ' + self.iso_location)146 logging.debug('Using iso at: %s', self.iso_location)
147 self.iso = ISO(image=self.iso_location, logger=logging)147 self.iso = ISO(image=self.iso_location, logger=logging)
148 self.st_release = self.iso.series148 self.st_release = self.iso.series
149 self.st_variant = self.iso.installtype149 self.st_variant = self.iso.installtype
150 self.st_arch = self.iso.arch150 self.st_arch = self.iso.arch
151 self.iso_name = '-'.join([self.st_release, self.st_variant,151 self.iso_name = ('{}.iso'
152 self.st_arch]) + '.iso'152 .format('-'.join([self.st_release,
153 logging.debug('Standard name for this iso is: ' + self.iso_name)153 self.st_variant,
154 self.st_arch])))
155 logging.debug('Standard name for this iso is: %s', self.iso_name)
154156
155 if self.st_release != 'precise':157 if self.st_release != 'precise':
156 self.url = DEFAULT_URL158 self.url = DEFAULT_URL
@@ -212,15 +214,9 @@
212214
213 def test_files_list(self):215 def test_files_list(self):
214 """Test if the list repository file matches the content of the ISO."""216 """Test if the list repository file matches the content of the ISO."""
215 if self.st_variant == 'server':217 list_url = os.path.join(
216 list_url = os.path.join(self.url, 'current',218 self.url, 'current', '{}.list'
217 self.st_release + '-' + 'server' + '-' +219 .format('-'.join([self.st_release, st_variant, self.st_arch])))
218 self.st_arch + '.list')
219 else:
220 list_url = os.path.join(self.url, 'current',
221 self.st_release + '-' +
222 self.st_variant + '-' +
223 self.st_arch + '.list')
224220
225 try:221 try:
226 list_repository = urllib2.urlopen(list_url)222 list_repository = urllib2.urlopen(list_url)
@@ -247,10 +243,10 @@
247 "manifest only for ubiquity based images")243 "manifest only for ubiquity based images")
248 def test_manifest(self):244 def test_manifest(self):
249 """Test if the ISO manifest matches the one in the server."""245 """Test if the ISO manifest matches the one in the server."""
250 manifest_url = os.path.join(self.url, 'current',246 manifest_url = os.path.join(
251 self.st_release + '-' +247 self.url, 'current', '{}.manifest'
252 self.st_variant + '-' +248 .format('-'.join([self.st_release, st_variant, self.st_arch])))
253 self.st_arch + '.manifest')249
254 try:250 try:
255 manifest = urllib2.urlopen(manifest_url)251 manifest = urllib2.urlopen(manifest_url)
256 except urllib2.HTTPError, e:252 except urllib2.HTTPError, e:
@@ -326,7 +322,7 @@
326 for list_server in files_list:322 for list_server in files_list:
327 path = list_server.rstrip()323 path = list_server.rstrip()
328 if path in exclude_files:324 if path in exclude_files:
329 logging.debug(path + ' excluded based on release')325 logging.debug('%s excluded based on release', path)
330 else:326 else:
331 self.assertIn(path, stdout)327 self.assertIn(path, stdout)
332328
@@ -370,7 +366,7 @@
370 logging.debug('check if important d-i files are present in iso')366 logging.debug('check if important d-i files are present in iso')
371 path = list_server.rstrip()367 path = list_server.rstrip()
372 if path in exclude_files:368 if path in exclude_files:
373 logging.debug(path + ' excluded based on release')369 logging.debug('%s excluded based on release', path)
374 else:370 else:
375 self.assertIn(path, stdout)371 self.assertIn(path, stdout)
376372
377373
=== modified file 'utah/orderedcollections.py'
--- utah/orderedcollections.py 2012-12-08 02:10:12 +0000
+++ utah/orderedcollections.py 2013-04-08 19:17:22 +0000
@@ -13,9 +13,7 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Provide additional collections to support cleanup functions."""
17Provide additional collections to support cleanup functions.
18"""
1917
2018
21import collections19import collections
@@ -25,6 +23,8 @@
2523
26class OrderedSet(collections.MutableSet):24class OrderedSet(collections.MutableSet):
2725
26 """Provide a set with ordering."""
27
28 def __init__(self, iterable=None):28 def __init__(self, iterable=None):
29 self.end = end = []29 self.end = end = []
30 end += [None, end, end] # sentinel node for doubly linked list30 end += [None, end, end] # sentinel node for doubly linked list
@@ -39,12 +39,14 @@
39 return key in self.map39 return key in self.map
4040
41 def add(self, key):41 def add(self, key):
42 """add an item to the set."""
42 if key not in self.map:43 if key not in self.map:
43 end = self.end44 end = self.end
44 curr = end[PREV]45 curr = end[PREV]
45 curr[NEXT] = end[PREV] = self.map[key] = [key, curr, end]46 curr[NEXT] = end[PREV] = self.map[key] = [key, curr, end]
4647
47 def discard(self, key):48 def discard(self, key):
49 """remove an item from the set."""
48 if key in self.map:50 if key in self.map:
49 key, prev, next = self.map.pop(key)51 key, prev, next = self.map.pop(key)
50 prev[NEXT] = next52 prev[NEXT] = next
@@ -65,6 +67,12 @@
65 curr = curr[PREV]67 curr = curr[PREV]
6668
67 def pop(self, last=True):69 def pop(self, last=True):
70 """Remove an item from the set and return that item.
71
72 :param last: Remove the last added item (instead of the first)
73 :type last: bool
74
75 """
68 if not self:76 if not self:
69 raise KeyError('set is empty')77 raise KeyError('set is empty')
70 key = next(reversed(self)) if last else next(iter(self))78 key = next(reversed(self)) if last else next(iter(self))
@@ -86,5 +94,8 @@
8694
8795
88class HashableDict(dict):96class HashableDict(dict):
97
98 """Provide a hashable dict."""
99
89 def __hash__(self):100 def __hash__(self):
90 return hash(tuple(sorted(self.items())))101 return hash(tuple(sorted(self.items())))
91102
=== modified file 'utah/parser.py'
--- utah/parser.py 2013-03-14 13:37:03 +0000
+++ utah/parser.py 2013-04-08 19:17:22 +0000
@@ -14,6 +14,8 @@
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""UTAH Client log parser."""16"""UTAH Client log parser."""
17
18
17import argparse19import argparse
18import jsonschema20import jsonschema
19import logging21import logging
2022
=== modified file 'utah/preseed.py'
--- utah/preseed.py 2012-12-03 14:02:18 +0000
+++ utah/preseed.py 2013-04-08 19:17:22 +0000
@@ -63,6 +63,8 @@
63<BLANKLINE>63<BLANKLINE>
6464
65"""65"""
66
67
66import string68import string
6769
6870
@@ -885,6 +887,8 @@
885887
886 """888 """
887889
890 pass
891
888892
889class DuplicatedQuestionName(Exception):893class DuplicatedQuestionName(Exception):
890894
@@ -904,3 +908,5 @@
904 DuplicatedQuestionName: passwd/username908 DuplicatedQuestionName: passwd/username
905909
906 """910 """
911
912 pass
907913
=== modified file 'utah/process.py'
--- utah/process.py 2013-03-29 09:00:34 +0000
+++ utah/process.py 2013-04-08 19:17:22 +0000
@@ -14,6 +14,8 @@
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""Process related utilities."""16"""Process related utilities."""
17
18
17import logging19import logging
18import subprocess20import subprocess
1921
2022
=== modified file 'utah/provisioning/__init__.py'
--- utah/provisioning/__init__.py 2012-12-03 14:02:18 +0000
+++ utah/provisioning/__init__.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,4 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""utah.provisioning"""
17utah.provisioning
18"""
1917
=== modified file 'utah/provisioning/baremetal/__init__.py'
--- utah/provisioning/baremetal/__init__.py 2012-12-03 14:02:18 +0000
+++ utah/provisioning/baremetal/__init__.py 2013-04-08 19:17:22 +0000
@@ -13,6 +13,4 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""utah.provisioning.baremetal"""
17utah.provisioning.baremetal
18"""
1917
=== modified file 'utah/provisioning/baremetal/bamboofeeder.py'
--- utah/provisioning/baremetal/bamboofeeder.py 2013-03-28 10:58:47 +0000
+++ utah/provisioning/baremetal/bamboofeeder.py 2013-04-08 19:17:22 +0000
@@ -13,9 +13,8 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Support provisioning of bamboo-feeder-based systems."""
17Support bare-metal provisioning of bamboo-feeder-based systems.17
18"""
1918
20import os19import os
21import pipes20import pipes
@@ -36,12 +35,13 @@
3635
3736
38class BambooFeederMachine(CustomInstallMixin, SSHMixin, PowerMixin, Machine):37class BambooFeederMachine(CustomInstallMixin, SSHMixin, PowerMixin, Machine):
39 """38
40 Provide a class to provision an ARM board from a bamboo-feeder setup.39 """Provision and manage an ARM board in a bamboo-feeder setup."""
41 """40
41 # TODO: raise more exceptions if ProcessRunner fails
42 # maybe get some easy way to do that like failok
42 def __init__(self, inventory=None, machineinfo=None, name=None,43 def __init__(self, inventory=None, machineinfo=None, name=None,
43 preboot=None, *args, **kw):44 preboot=None, *args, **kw):
44 # TODO: change the name of cargs here and elsewhere
45 # TODO: respect rewrite setting45 # TODO: respect rewrite setting
46 if name is None:46 if name is None:
47 raise UTAHBMProvisioningException(47 raise UTAHBMProvisioningException(
@@ -64,8 +64,8 @@
64 self.logger.debug('Configuring for %s with IP %s',64 self.logger.debug('Configuring for %s with IP %s',
65 config.wwwiface, self.ip)65 config.wwwiface, self.ip)
66 self._cmdlinesetup()66 self._cmdlinesetup()
67 imageurl = 'http://' + self.ip + '/utah/' + self.name + '.img'67 imageurl = 'http://{}/utah/{}.img'.format(self.ip, self.name)
68 preenvurl = 'http://' + self.ip + '/utah/' + self.name + '.preEnv'68 preenvurl = 'http://{}/utah/{}.preEnv'.format(self.ip, self.name)
69 if preboot is None:69 if preboot is None:
70 self.preboot = ('console=ttyO2,115200n8 imageurl={imageurl} '70 self.preboot = ('console=ttyO2,115200n8 imageurl={imageurl} '
71 'bootcfg={preenvurl}'.format(71 'bootcfg={preenvurl}'.format(
@@ -78,20 +78,22 @@
78 self.logger.debug('BambooFeederMachine init finished')78 self.logger.debug('BambooFeederMachine init finished')
7979
80 def _prepareimage(self):80 def _prepareimage(self):
81 """81 """Make a copy of the image and share it via www.
82 Make a copy of the image shared via wwa so we can edit it.82
83 :returns: Path to www-available image
84 :rtype: str
85
83 """86 """
84 self.logger.info('Making copy of install image')87 self.logger.info('Making copy of install image')
85 self.wwwimage = os.path.join(config.wwwdir, self.name + '.img')88 self.wwwimage = os.path.join(config.wwwdir,
86 self.logger.debug('Copying ' + self.image + ' to ' + self.wwwimage)89 '{}.img'.format(self.name))
90 self.logger.debug('Copying %s to %s', self.image, self.wwwimage)
87 self.cleanfile(self.wwwimage)91 self.cleanfile(self.wwwimage)
88 shutil.copyfile(self.image, self.wwwimage)92 shutil.copyfile(self.image, self.wwwimage)
89 return self.wwwimage93 return self.wwwimage
9094
91 def _mountimage(self):95 def _mountimage(self):
92 """96 """Mount an ARM boot image."""
93 Mount an ARM boot image so we can edit it.
94 """
95 self.logger.info('Mounting install image')97 self.logger.info('Mounting install image')
96 self.imagedir = os.path.join(self.tmpdir, 'image.d')98 self.imagedir = os.path.join(self.tmpdir, 'image.d')
97 if not os.path.isdir(self.imagedir):99 if not os.path.isdir(self.imagedir):
@@ -102,28 +104,24 @@
102 except (OSError, subprocess.CalledProcessError) as err:104 except (OSError, subprocess.CalledProcessError) as err:
103 raise UTAHBMProvisioningException('Failed to get image info: {}'105 raise UTAHBMProvisioningException('Failed to get image info: {}'
104 .format(err))106 .format(err))
105 self.logger.debug('Image start sector is ' + str(self.sector))107 self.logger.debug('Image start sector is %s', str(self.sector))
106 self.cleanfunction(self._umountimage)108 self.cleanfunction(self._umountimage)
107 self.logger.debug('Mounting ' + self.wwwimage + ' at ' +109 self.logger.debug('Mounting %s at %s', self.wwwimage, self.imagedir)
108 self.imagedir)
109 ProcessRunner(['sudo', 'mount', self.wwwimage, self.imagedir,110 ProcessRunner(['sudo', 'mount', self.wwwimage, self.imagedir,
110 '-o', 'loop', '-o', 'offset=' + str(self.sector * 512),111 '-o', 'loop',
111 '-o', 'uid=' + config.user])112 '-o', 'offset={}'.format(str(self.sector * 512)),
113 '-o', 'uid={}'.format(config.user)])
112114
113 def _setupconsole(self):115 def _setupconsole(self):
114 """116 """Setup the install to use a serial console."""
115 Setup the install to use a serial console."
116 """
117 self.logger.info('Setting up the install to use the serial console')117 self.logger.info('Setting up the install to use the serial console')
118 preenvfile = os.path.join(self.imagedir, 'preEnv.txt')118 preenvfile = os.path.join(self.imagedir, 'preEnv.txt')
119 serialpreenvfile = os.path.join(self.imagedir, 'preEnv.txt-serial')119 serialpreenvfile = os.path.join(self.imagedir, 'preEnv.txt-serial')
120 self.logger.debug('Copying ' + serialpreenvfile + ' to ' + preenvfile)120 self.logger.debug('Copying %s to %s', serialpreenvfile, preenvfile)
121 shutil.copyfile(serialpreenvfile, preenvfile)121 shutil.copyfile(serialpreenvfile, preenvfile)
122122
123 def _unpackinitrd(self):123 def _unpackinitrd(self):
124 """124 """Unpack the uInitrd file into a directory."""
125 Unpack the uInitrd file into a directory so we can modify it.
126 """
127 self.logger.info('Unpacking uInitrd')125 self.logger.info('Unpacking uInitrd')
128 if not os.path.isdir(os.path.join(self.tmpdir, 'initrd.d')):126 if not os.path.isdir(os.path.join(self.tmpdir, 'initrd.d')):
129 os.makedirs(os.path.join(self.tmpdir, 'initrd.d'))127 os.makedirs(os.path.join(self.tmpdir, 'initrd.d'))
@@ -139,10 +137,10 @@
139 raise UTAHBMProvisioningException('Failed to get initrd info: {}'137 raise UTAHBMProvisioningException('Failed to get initrd info: {}'
140 .format(err))138 .format(err))
141 headersize = filesize - datasize139 headersize = filesize - datasize
142 self.logger.debug('uInitrd header size is ' + str(headersize))140 self.logger.debug('uInitrd header size is %s', str(headersize))
143 pipe = pipes.Template()141 pipe = pipes.Template()
144 pipe.prepend('dd if=$IN bs=1 skip=' + str(headersize) +142 pipe.prepend('dd if=$IN bs=1 skip={} 2>/dev/null'
145 ' 2>/dev/null', 'f-')143 .format(str(headersize)), 'f-')
146 pipe.append('gunzip', '--')144 pipe.append('gunzip', '--')
147 pipe.append('cpio -ivd 2>/dev/null', '-.')145 pipe.append('cpio -ivd 2>/dev/null', '-.')
148 exitstatus = pipe.copy(self.initrd, '/dev/null')146 exitstatus = pipe.copy(self.initrd, '/dev/null')
@@ -153,9 +151,7 @@
153 'Unpacking initrd exited with status {}'.format(exitstatus))151 'Unpacking initrd exited with status {}'.format(exitstatus))
154152
155 def _repackinitrd(self):153 def _repackinitrd(self):
156 """154 """Repack the uInitrd file from the initrd.d directory."""
157 Repack the uInitrd file from the initrd.d directory.
158 """
159 self.logger.info('Repacking uInitrd')155 self.logger.info('Repacking uInitrd')
160 os.chdir(os.path.join(self.tmpdir, 'initrd.d'))156 os.chdir(os.path.join(self.tmpdir, 'initrd.d'))
161 pipe = pipes.Template()157 pipe = pipes.Template()
@@ -181,44 +177,40 @@
181 self.initrd])177 self.initrd])
182178
183 def _umountimage(self):179 def _umountimage(self):
184 """180 """Unmount the image after we're done with it."""
185 Unmount the image after we're done with it."
186 """
187 self.logger.info('Unmounting image')181 self.logger.info('Unmounting image')
188 ProcessRunner(['sudo', 'umount', self.imagedir])182 ProcessRunner(['sudo', 'umount', self.imagedir])
189183
190 def _cmdlinesetup(self, boot=None):184 def _cmdlinesetup(self, boot=None):
191 """185 """Add options to the command line for an automatic install."""
192 Add the needed options to the command line for an automatic install.
193 """
194 if boot is None:186 if boot is None:
195 boot = self.boot187 boot = self.boot
196 super(BambooFeederMachine, self)._cmdlinesetup(boot=boot)188 super(BambooFeederMachine, self)._cmdlinesetup(boot=boot)
197 # TODO: minimize these189 # TODO: minimize these
198 for option in ('auto', 'ro', 'text'):190 for option in ('auto', 'ro', 'text'):
199 if option not in self.cmdline:191 if option not in self.cmdline:
200 self.logger.info('Adding boot option: ' + option)192 self.logger.info('Adding boot option: %s', option)
201 self.cmdline += ' ' + option193 self.cmdline += ' {}'.format(option)
202 for parameter in (('cdrom-detect/try_usb', 'true'),194 for parameter in (
203 ('console', 'ttyO2,115200n8'),195 ('cdrom-detect/try_usb', 'true'),
204 ('country', 'US'),196 ('console', 'ttyO2,115200n8'),
205 ('hostname', self.name),197 ('country', 'US'),
206 ('language', 'en'),198 ('hostname', self.name),
207 ('locale', 'en_US'),199 ('language', 'en'),
208 ('loghost', self.ip),200 ('locale', 'en_US'),
209 ('log_port', '10514'),201 ('loghost', self.ip),
210 ('netcfg/choose_interface', 'auto'),202 ('log_port', '10514'),
211 ('url', 'http://' + self.ip + '/utah/' + self.name +203 ('netcfg/choose_interface', 'auto'),
212 '.cfg')):204 ('url', 'http://{}/utah/{}.cfg'.format(self.ip, self.name)),
205 ):
213 if parameter[0] not in self.cmdline:206 if parameter[0] not in self.cmdline:
214 self.logger.info('Adding boot option: ' + '='.join(parameter))207 self.logger.info('Adding boot option: %s',
215 self.cmdline += ' ' + '='.join(parameter)208 '='.join(parameter))
209 self.cmdline += ' {}'.format('='.join(parameter))
216 self.cmdline.strip()210 self.cmdline.strip()
217211
218 def _configurepxe(self):212 def _configurepxe(self):
219 """213 """Setup PXE configuration to boot remote image."""
220 Setup PXE configuration to boot remote image.
221 """
222 # TODO: Maybe move this into pxe.py214 # TODO: Maybe move this into pxe.py
223 # TODO: look into cutting out the middleman/215 # TODO: look into cutting out the middleman/
224 # booting straight into the installer? (maybe nfs?)216 # booting straight into the installer? (maybe nfs?)
@@ -234,37 +226,36 @@
234""".format(preboot=self.preboot)226""".format(preboot=self.preboot)
235 tmppxefile = os.path.join(self.tmpdir, 'pxe')227 tmppxefile = os.path.join(self.tmpdir, 'pxe')
236 open(tmppxefile, 'w').write(pxeconfig)228 open(tmppxefile, 'w').write(pxeconfig)
237 self.logger.debug('PXE info written to ' + tmppxefile)229 self.logger.debug('PXE info written to %s', tmppxefile)
238 pxefile = os.path.join(config.pxedir,230 pxefile = os.path.join(
239 '01-' + self.macaddress.replace(':', '-'))231 config.pxedir, '01-{}'.format(self.macaddress.replace(':', '-')))
240 self.cleancommand(('sudo', 'rm', '-f', pxefile))232 self.cleancommand(('sudo', 'rm', '-f', pxefile))
241 self.logger.debug('Copying ' + tmppxefile + ' to ' + pxefile)233 self.logger.debug('Copying %s to %s', tmppxefile, pxefile)
242 ProcessRunner(['sudo', 'cp', tmppxefile, pxefile])234 ProcessRunner(['sudo', 'cp', tmppxefile, pxefile])
243 preenvfile = os.path.join(config.wwwdir, self.name + '.preEnv')235 preenvfile = os.path.join(config.wwwdir,
236 '{}.preEnv'.format(self.name))
244 # TODO: sort this out with self.boot237 # TODO: sort this out with self.boot
245 # figure out which one should be what and customizable238 # figure out which one should be what and customizable
246 self.preenv = 'bootargs=' + self.cmdline239 self.preenv = 'bootargs={}'.format(self.cmdline)
247 self.logger.debug('Preenv setup:')240 self.logger.debug('Preenv setup:')
248 self.logger.debug(self.preenv)241 self.logger.debug(self.preenv)
249 self.cleanfile(preenvfile)242 self.cleanfile(preenvfile)
250 self.logger.debug('Writing preenv setup to ' + preenvfile)243 self.logger.debug('Writing preenv setup to %s', preenvfile)
251 open(preenvfile, 'w').write(self.preenv)244 open(preenvfile, 'w').write(self.preenv)
252245
253 def _create(self):246 def _create(self):
254 """247 """Install the OS on the system."""
255 Install the OS on the system.
256 """
257 # TODO: more checks and exceptions for failures248 # TODO: more checks and exceptions for failures
258 self.logger.info('Preparing system install')249 self.logger.info('Preparing system install')
259 self.tmpdir = tempfile.mkdtemp(prefix='/tmp/' + self.name + '_')250 self.tmpdir = tempfile.mkdtemp(prefix='/tmp/{}_'.format(self.name))
260 self.cleanfile(self.tmpdir)251 self.cleanfile(self.tmpdir)
261 self.logger.debug('Using ' + self.tmpdir + ' as temp dir')252 self.logger.debug('Using %s as temp dir', self.tmpdir)
262 os.chdir(self.tmpdir)253 os.chdir(self.tmpdir)
263 self._prepareimage()254 self._prepareimage()
264 self._mountimage()255 self._mountimage()
265 self._setupconsole()256 self._setupconsole()
266 self.initrd = os.path.join(self.imagedir, 'uInitrd')257 self.initrd = os.path.join(self.imagedir, 'uInitrd')
267 self.logger.debug('uInitrd is ' + self.initrd)258 self.logger.debug('uInitrd is %s', self.initrd)
268 self._unpackinitrd()259 self._unpackinitrd()
269 self._setuplatecommand()260 self._setuplatecommand()
270 # TODO: check if this is still needed261 # TODO: check if this is still needed
@@ -278,7 +269,7 @@
278 myfile.close()269 myfile.close()
279 self._setuppreseed()270 self._setuppreseed()
280 self.logger.debug('Copying preseed to download location')271 self.logger.debug('Copying preseed to download location')
281 preseedfile = os.path.join(config.wwwdir, self.name + '.cfg')272 preseedfile = os.path.join(config.wwwdir, '{}.cfg'.format(self.name))
282 self.cleanfile(preseedfile)273 self.cleanfile(preseedfile)
283 shutil.copyfile(os.path.join(self.tmpdir, 'initrd.d', 'preseed.cfg'),274 shutil.copyfile(os.path.join(self.tmpdir, 'initrd.d', 'preseed.cfg'),
284 preseedfile)275 preseedfile)
@@ -300,12 +291,9 @@
300 self.cleanfunction(self.run, (291 self.cleanfunction(self.run, (
301 'dd', 'bs=512k', 'count=10', 'if=/dev/zero', 'of=/dev/mmcblk0'),292 'dd', 'bs=512k', 'count=10', 'if=/dev/zero', 'of=/dev/mmcblk0'),
302 root=True)293 root=True)
303 return True
304294
305 def _depcheck(self):295 def _depcheck(self):
306 """296 """Check for dependencies that are in Recommends or Suggests."""
307 Check for dependencies that are in Recommends or Suggests.
308 """
309 super(BambooFeederMachine, self)._depcheck()297 super(BambooFeederMachine, self)._depcheck()
310 cmd = ['which', 'mkimage']298 cmd = ['which', 'mkimage']
311 if ProcessRunner(cmd).returncode != 0:299 if ProcessRunner(cmd).returncode != 0:
312300
=== modified file 'utah/provisioning/baremetal/cobbler.py'
--- utah/provisioning/baremetal/cobbler.py 2013-03-29 15:55:46 +0000
+++ utah/provisioning/baremetal/cobbler.py 2013-04-08 19:17:22 +0000
@@ -13,9 +13,8 @@
13# You should have received a copy of the GNU General Public License along13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.14# with this program. If not, see <http://www.gnu.org/licenses/>.
1515
16"""16"""Support bare metal provisioning through cobbler."""
17Support bare metal provisioning through cobbler.17
18"""
1918
20import os19import os
21import pipes20import pipes
@@ -35,9 +34,11 @@
3534
3635
37class CobblerMachine(CustomInstallMixin, SSHMixin, PowerMixin, Machine):36class CobblerMachine(CustomInstallMixin, SSHMixin, PowerMixin, Machine):
38 """37
39 Provide a class to provision a machine via cobbler.38 """Provision and manage a machine via cobbler."""
40 """39
40 # TODO: raise more exceptions if ProcessRunner fails
41 # maybe get some easy way to do that like failok
41 def __init__(self, inventory=None, machineinfo=None,42 def __init__(self, inventory=None, machineinfo=None,
42 name=None, *args, **kw):43 name=None, *args, **kw):
43 # TODO: support for reusing existing machines44 # TODO: support for reusing existing machines
@@ -73,27 +74,22 @@
73 self.carch = 'x86_64'74 self.carch = 'x86_64'
74 else:75 else:
75 self.carch = self.arch76 self.carch = self.arch
76 self.cname = self.name + '-' + self.carch77 self.cname = '-'.join([self.name, self.carch])
77 self.logger.debug('Cobbler arch is ' + self.carch)78 self.logger.debug('Cobbler arch is %s', self.carch)
78 self.logger.debug('Cobbler machine init finished')79 self.logger.debug('Cobbler machine init finished')
7980
80 def _load(self):81 def _load(self):
81 """82 """Verify the machine is in cobbler."""
82 Verify the machine is in cobbler.
83 """
84 # TODO: consider reworking _cobble to provide this
85 # (only if we'll be using cobbler for a while longer)
86 machines = self._cobble(['system', 'find'])['output'].splitlines()83 machines = self._cobble(['system', 'find'])['output'].splitlines()
87 if self.name not in machines:84 if self.name not in machines:
88 raise UTAHBMProvisioningException(85 raise UTAHBMProvisioningException(
89 'No machine named {} exists in cobbler'.format(self.name))86 'No machine named {} exists in cobbler'.format(self.name))
87 else:
88 self.provisioned = True
9089
91 def _create(self, checktimeout=config.checktimeout,90 def _create(self):
92 installtimeout=config.installtimeout):91 """Install the OS on the machine."""
93 """92 self.tmpdir = tempfile.mkdtemp(prefix='/tmp/{}_'.format(self.name))
94 Install a machine.
95 """
96 self.tmpdir = tempfile.mkdtemp(prefix='/tmp/' + self.name + '_')
97 self.cleanfile(self.tmpdir)93 self.cleanfile(self.tmpdir)
98 os.chmod(self.tmpdir, 0775)94 os.chmod(self.tmpdir, 0775)
9995
@@ -134,7 +130,7 @@
134 if self.rewrite == 'all':130 if self.rewrite == 'all':
135 self._setuplogging(tmpdir=self.tmpdir)131 self._setuplogging(tmpdir=self.tmpdir)
136 else:132 else:
137 self.logger.debug('Skipping logging setup because rewrite is' +133 self.logger.debug('Skipping logging setup because rewrite is %s',
138 self.rewrite)134 self.rewrite)
139135
140 initrd = self._repackinitrd()136 initrd = self._repackinitrd()
@@ -146,27 +142,34 @@
146142
147 if self.installtype in ['alternate', 'server']:143 if self.installtype in ['alternate', 'server']:
148 self.logger.info('Importing image')144 self.logger.info('Importing image')
149 self._cobble(['import', '--name=' + self.cname,145 self._cobble(['import',
150 '--path=' + self.isodir, '--arch=' + self.carch])146 '--name={}'.format(self.cname),
151 self._cobble(['distro', 'edit', '--name=' + self.cname,147 '--path={}'.format(self.isodir),
152 '--kernel=' + kernel, '--initrd=' + initrd])148 '--arch={}'.format(self.carch)])
149 self._cobble(['distro', 'edit',
150 '--name={}'.format(self.cname),
151 '--kernel={}'.format(kernel),
152 '--initrd={}'.format(initrd)])
153 elif self.installtype in ['mini', 'desktop']:153 elif self.installtype in ['mini', 'desktop']:
154 self.logger.info('Creating distro')154 self.logger.info('Creating distro')
155 self._cobble(['distro', 'add', '--name=' + self.cname,
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches