Merge lp:~veebers/autopilot/fix_for_config_init into lp:autopilot

Proposed by Christopher Lee
Status: Merged
Approved by: Christopher Lee
Approved revision: 547
Merged at revision: 545
Proposed branch: lp:~veebers/autopilot/fix_for_config_init
Merge into: lp:autopilot
Diff against target: 74 lines (+36/-10)
2 files modified
autopilot/run.py (+11/-10)
autopilot/tests/functional/test_autopilot_functional.py (+25/-0)
To merge this branch: bzr merge lp:~veebers/autopilot/fix_for_config_init
Reviewer Review Type Date Requested Status
PS Jenkins bot continuous-integration Approve
Richard Huddie (community) Approve
Leo Arias (community) Approve
Review via email: mp+251862@code.launchpad.net

Commit message

Move config parsing to happen before test import.

Description of the change

Move config parsing to happen before test import. Ensures config details available within decorators.

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:545
http://jenkins.qa.ubuntu.com/job/autopilot-ci/1017/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-amd64-ci/80
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-amd64-ci/80/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-armhf-ci/80
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-armhf-ci/80/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-i386-ci/80
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-i386-ci/80/artifact/work/output/*zip*/output.zip
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-vivid-touch/1657
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-vivid-autopilot/118
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-vivid-mako/1470
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/1655
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/1655/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/18573
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-vivid-autopilot/114
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-amd64/802
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-amd64/802/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/1017/rebuild

review: Approve (continuous-integration)
Revision history for this message
Leo Arias (elopio) wrote :

This looks good. Can we write a test for it?
I think that using a mock and checking the order of the calls would be good.
Even better if we add a skip decorator in a test that uses a config value.

546. By Christopher Lee

Failing test for config available in decorator

547. By Christopher Lee

test is passing (no functional code change, demos that config now available)

Revision history for this message
Christopher Lee (veebers) wrote :

@Leo, I've added a first pass at a functional test for this as requested.

I'm not son keen on doing a test using mocks etc. to check the order of the calls. The calls are static and we can see them in the source file (e.g. they aren't conditionally run etc.) as well as the whole suite of tests that cover the execution of this part of the code which should give us coverage already.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Leo Arias (elopio) wrote :

Without the fix the test fails. With it the test passes.
I actually prefer this to be tested in the functional suite, so +2.

review: Approve
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Richard Huddie (rhuddie) wrote :

This all looks good, very interesting how the new test case is done. Test passes for me.

review: Approve
Revision history for this message
Leo Arias (elopio) wrote :

Asking ci again about the tests. They are stuck.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:547
http://jenkins.qa.ubuntu.com/job/autopilot-ci/1021/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-amd64-ci/84
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-amd64-ci/84/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-armhf-ci/84
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-armhf-ci/84/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-i386-ci/84
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-i386-ci/84/artifact/work/output/*zip*/output.zip
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-vivid-touch/1734
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-vivid-autopilot/123
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-vivid-mako/1540
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/1732
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/1732/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/18772
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-vivid-autopilot/121
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-amd64/843
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-amd64/843/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/1021/rebuild

review: Approve (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:547
http://jenkins.qa.ubuntu.com/job/autopilot-ci/1022/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-amd64-ci/85
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-amd64-ci/85/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-armhf-ci/85
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-armhf-ci/85/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-i386-ci/85
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-i386-ci/85/artifact/work/output/*zip*/output.zip
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-vivid-touch/1736
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-vivid-autopilot/124
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-vivid-mako/1542
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/1734
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/1734/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/18774
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-vivid-autopilot/122
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-amd64/844
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-amd64/844/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/1022/rebuild

review: Approve (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:547
http://jenkins.qa.ubuntu.com/job/autopilot-ci/1023/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-amd64-ci/86
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-amd64-ci/86/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-armhf-ci/86
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-armhf-ci/86/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-i386-ci/86
        deb: http://jenkins.qa.ubuntu.com/job/autopilot-vivid-i386-ci/86/artifact/work/output/*zip*/output.zip
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-vivid-touch/1737
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-vivid-autopilot/125
    UNSTABLE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-runner-vivid-mako/1543
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/1735
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-armhf/1735/artifact/work/output/*zip*/output.zip
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/18776
    SUCCESS: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-vivid-autopilot/123
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-amd64/845
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-vivid-amd64/845/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/autopilot-ci/1023/rebuild

review: Approve (continuous-integration)
Revision history for this message
Christopher Lee (veebers) wrote :

Top approving as it has the required 2x approves and now that CI is resolved they are passing.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'autopilot/run.py'
--- autopilot/run.py 2014-09-05 04:55:50 +0000
+++ autopilot/run.py 2015-03-10 00:23:59 +0000
@@ -662,16 +662,6 @@
662662
663 def run_tests(self):663 def run_tests(self):
664 """Run tests, using input from `args`."""664 """Run tests, using input from `args`."""
665 test_suite, error_encountered = load_test_suite_from_name(
666 self.args.suite
667 )
668
669 if not test_suite.countTestCases():
670 raise RuntimeError('Did not find any tests')
671
672 if self.args.random_order:
673 shuffle(test_suite._tests)
674 print("Running tests in random order")
675665
676 _configure_debug_profile(self.args)666 _configure_debug_profile(self.args)
677 _configure_timeout_profile(self.args)667 _configure_timeout_profile(self.args)
@@ -688,6 +678,17 @@
688 if self.args.verbose:678 if self.args.verbose:
689 autopilot.globals.set_log_verbose(True)679 autopilot.globals.set_log_verbose(True)
690680
681 test_suite, error_encountered = load_test_suite_from_name(
682 self.args.suite
683 )
684
685 if not test_suite.countTestCases():
686 raise RuntimeError('Did not find any tests')
687
688 if self.args.random_order:
689 shuffle(test_suite._tests)
690 print("Running tests in random order")
691
691 result = construct_test_result(self.args)692 result = construct_test_result(self.args)
692 result.startTestRun()693 result.startTestRun()
693 try:694 try:
694695
=== modified file 'autopilot/tests/functional/test_autopilot_functional.py'
--- autopilot/tests/functional/test_autopilot_functional.py 2014-07-24 00:24:41 +0000
+++ autopilot/tests/functional/test_autopilot_functional.py 2015-03-10 00:23:59 +0000
@@ -94,6 +94,31 @@
9494
95class FunctionalTestMain(AutopilotFunctionalTestsBase):95class FunctionalTestMain(AutopilotFunctionalTestsBase):
9696
97 def test_config_available_in_decorator(self):
98 """Any commandline config values must be available for decorators."""
99 unique_config_value = self.getUniqueString()
100 self.create_test_file(
101 'test_config_skip.py', dedent("""\
102 from testtools import skipIf
103 import autopilot
104 from autopilot.testcase import AutopilotTestCase
105
106 class ConfigTest(AutopilotTestCase):
107 @skipIf(
108 autopilot.get_test_configuration().get('skipme', None)
109 == '{unique_config_value}',
110 'Skipping Test')
111 def test_config(self):
112 self.fail('Should not run.')
113 """.format(unique_config_value=unique_config_value))
114 )
115
116 config_string = 'skipme={}'.format(unique_config_value)
117 code, output, error = self.run_autopilot(
118 ['run', 'tests', '--config', config_string])
119
120 self.assertThat(code, Equals(0))
121
97 def test_can_list_empty_test_dir(self):122 def test_can_list_empty_test_dir(self):
98 """Autopilot list must report 0 tests found with an empty test123 """Autopilot list must report 0 tests found with an empty test
99 module."""124 module."""

Subscribers

People subscribed via source and target branches