Merge lp:~zyga/checkbox/mock-armageddon into lp:checkbox

[Zygmunt Krynicki]

This merge requests, currently, triggers the visibility of 65 test cases that silently don't test anything because of the older version of mock we were developing against did not include the precaution for using random, made-up mock methods that silently don't do anything.

It can only land after we fix everything that was silently broken throughout this time.

[Zygmunt Krynicki (zyga)]

I think we are all there now. Trying to land.

review: Approve

[Daniel Manrique (roadmr)]

The attempt to merge lp:~zyga/checkbox/mock-armageddon into lp:checkbox failed. Below is the output from the failed tests.

[precise] starting container
[precise] (timing) 0.08user 0.02system 0:05.96elapsed 1%CPU (0avgtext+0avgdata 7852maxresident)k
[precise] (timing) 0inputs+32outputs (0major+6992minor)pagefaults 0swaps
[precise] provisioning container
[precise] (timing) 52.47user 15.07system 1:40.05elapsed 67%CPU (0avgtext+0avgdata 52776maxresident)k
[precise] (timing) 0inputs+19232outputs (0major+6609933minor)pagefaults 0swaps
[precise-testing] Starting tests...
Found a test script: ./checkbox-gui/requirements/container-tests-checkbox-gui-build
[precise-testing] container-tests-checkbox-gui-build: PASS
[precise-testing] (timing) 34.66user 2.42system 0:37.51elapsed 98%CPU (0avgtext+0avgdata 116632maxresident)k
[precise-testing] (timing) 0inputs+4288outputs (0major+478359minor)pagefaults 0swaps
Found a test script: ./checkbox-ng/requirements/container-tests-checkbox-ng-unit
[precise-testing] container-tests-checkbox-ng-unit: PASS
[precise-testing] (timing) 0.54user 0.14system 0:00.72elapsed 95%CPU (0avgtext+0avgdata 39720maxresident)k
[precise-testing] (timing) 0inputs+1488outputs (0major+20872minor)pagefaults 0swaps
Found a test script: ./checkbox-support/requirements/container-tests-checkbox-support
[precise-testing] container-tests-checkbox-support: PASS
[precise-testing] (timing) 14.57user 0.21system 0:14.90elapsed 99%CPU (0avgtext+0avgdata 83840maxresident)k
[precise-testing] (timing) 0inputs+1320outputs (0major+31758minor)pagefaults 0swaps
Found a test script: ./checkbox-touch/requirements/container-tests-touch-unit-tests
[precise-testing] container-tests-touch-unit-tests: PASS
[precise-testing] (timing) 0.00user 0.00system 0:00.02elapsed 33%CPU (0avgtext+0avgdata 2040maxresident)k
[precise-testing] (timing) 0inputs+8outputs (0major+2378minor)pagefaults 0swaps
Found a test script: ./plainbox/plainbox/impl/providers/categories/requirements/container-tests-provider-categories
[precise-testing] container-tests-provider-categories: PASS
[precise-testing] (timing) 0.71user 0.06system 0:00.81elapsed 95%CPU (0avgtext+0avgdata 29920maxresident)k
[precise-testing] (timing) 0inputs+64outputs (0major+10109minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/001-container-tests-plainbox-egg-info
[precise-testing] 001-container-tests-plainbox-egg-info: PASS
[precise-testing] (timing) 0.14user 0.05system 0:00.21elapsed 92%CPU (0avgtext+0avgdata 10620maxresident)k
[precise-testing] (timing) 0inputs+96outputs (0major+5055minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox
[precise-testing] container-tests-plainbox: FAIL
[precise-testing] stdout: http://paste.ubuntu.com/12246037/
[precise-testing] stderr: http://paste.ubuntu.com/12246038/
[precise-testing] (timing) Command exited with non-zero status 1
[precise-testing] (timing) 17.30user 0.57system 0:18.04elapsed 99%CPU (0avgtext+0avgdata 87816maxresident)k
[precise-testing] (timing) 0inputs+3344outputs (0major+70379minor)pagefaults 0swaps
Found a ...

[lp:~zyga/checkbox/mock-armageddon]

3970. By Zygmunt Krynicki on 2015-09-02

plainbox:secure:plugins: fix broken mock asserion

Signed-off-by: Zygmunt Krynicki <email address hidden>

3971. By Zygmunt Krynicki on 2015-09-02

plainbox:vendor:mock: detect non-existing assertion methods

This patch cherry-picks a two-line feature from the more recent version
of mock (as found in Python 3.5) that prevents silent, useless tests
that call non-existing assertion methods.

This patch can only land (via CI) after all such issues have been fixed.

Signed-off-by: Zygmunt Krynicki <email address hidden>

3972. By Zygmunt Krynicki on 2015-09-02

checkbox-support:vendor:mock: detect non-existing assertion methods

This patch cherry-picks a two-line feature from the more recent version
of mock (as found in Python 3.5) that prevents silent, useless tests
that call non-existing assertion methods.

Signed-off-by: Zygmunt Krynicki <email address hidden>

[Sylvain Pineau (sylvain-pineau)]

+1

review: Approve

=== modified file 'checkbox-support/checkbox_support/vendor/mock.py'
--- checkbox-support/checkbox_support/vendor/mock.py	2014-01-07 13:39:38 +0000
+++ checkbox-support/checkbox_support/vendor/mock.py	2015-09-02 06:59:05 +0000
@@ -658,6 +658,8 @@
                 raise AttributeError("Mock object has no attribute %r" % name)
         elif _is_magic(name):
             raise AttributeError(name)
+        if name.startswith(('assert', 'assret')):
+            raise AttributeError(name)
 
         result = self._mock_children.get(name)
         if result is _deleted:

=== modified file 'plainbox/plainbox/impl/secure/test_plugins.py'
--- plainbox/plainbox/impl/secure/test_plugins.py	2015-08-31 10:05:19 +0000
+++ plainbox/plainbox/impl/secure/test_plugins.py	2015-09-02 06:59:05 +0000
@@ -514,7 +514,7 @@
                  {'encoding': 'UTF-8'}),
             ])
         # Ensure that no exception was logged
-        mock_logger.error.assert_not_called()
+        self.assertEqual(mock_logger.error.mock_calls, [])
         # Ensure that everything was okay
         self.assertEqual(col.problem_list, [])
         # Ensure that both files got added

=== modified file 'plainbox/plainbox/vendor/mock.py'
--- plainbox/plainbox/vendor/mock.py	2013-10-17 15:03:37 +0000
+++ plainbox/plainbox/vendor/mock.py	2015-09-02 06:59:05 +0000
@@ -658,6 +658,8 @@
                 raise AttributeError("Mock object has no attribute %r" % name)
         elif _is_magic(name):
             raise AttributeError(name)
+        if name.startswith(('assert', 'assret')):
+            raise AttributeError(name)
 
         result = self._mock_children.get(name)
         if result is _deleted: