Merge lp:~gocept/landscape-client/py3-watchdog into lp:~landscape/landscape-client/trunk

[Steffen Allner]

This is the Python 3 port of landscape.watchdog.

The key change is moving from the builtin filter(), which changed in Python 3, to a list comprehension.

This MP fixes the tests for landscape.watchdog module, which were mainly broken by a behaviour change in the built in filter() function. It now returned a not so well iterable, which did not yield anything in some loops, but a list comprehension for 3 items is acceptable, I suppose.

Additionally, I removed all assertItemsEqual() in the project, which implies the dependency on the py3-broker-exchange branch.

Some smaller test fixes concerning bytes have been necessary as well.

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 978
Branch: lp:~gocept/landscape-client/py3-watchdog
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3786/

review: Approve (test results)

[Alberto Donato (ack)]

+1

Just a nit inline

review: Approve

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[Steffen Allner (sallner)]

I added more explicit variable names, but see the reason inline.

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 979
Branch: lp:~gocept/landscape-client/py3-watchdog
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3787/

review: Approve (test results)

[Eric Snow (ericsnowcurrently)]

LGTM

review: Approve

[Daniel Havlik (nilo)]

+1

review: Approve

[🤖 Landscape Builder (landscape-builder)]

The attempt to merge lp:~gocept/landscape-client/py3-watchdog into lp:landscape-client failed. Below is the output from the failed tests.

sudo apt -y install python-twisted-core python-distutils-extra python-mock python-configobj python-passlib
Reading package lists...
Building dependency tree...
Reading state information...
python-configobj is already the newest version (5.0.6-2).
python-configobj set to manually installed.
python-mock is already the newest version (1.3.0-2.1ubuntu1).
python-passlib is already the newest version (1.6.5-4).
python-twisted-core is already the newest version (16.0.0-1).
python-distutils-extra is already the newest version (2.39-1).
The following packages were automatically installed and are no longer required:
  golang-check.v1-dev golang-context-dev golang-github-gorilla-mux-dev
  golang-yaml.v2-dev linux-headers-4.4.0-64 linux-headers-4.4.0-64-generic
  linux-image-4.4.0-64-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 27 not upgraded.
sudo apt -y install python3-twisted python3-distutils-extra python3-mock python3-configobj python3-passlib
Reading package lists...
Building dependency tree...
Reading state information...
python3-configobj is already the newest version (5.0.6-2).
python3-twisted is already the newest version (16.0.0-1).
python3-distutils-extra is already the newest version (2.39-1).
python3-mock is already the newest version (1.3.0-2.1ubuntu1).
python3-passlib is already the newest version (1.6.5-4).
The following packages were automatically installed and are no longer required:
  golang-check.v1-dev golang-context-dev golang-github-gorilla-mux-dev
  golang-yaml.v2-dev linux-headers-4.4.0-64 linux-headers-4.4.0-64-generic
  linux-image-4.4.0-64-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 27 not upgraded.
python setup.py build_ext -i
running build_ext
python3 setup.py build_ext -i
running build_ext
LC_ALL=C trial --unclean-warnings -j4 landscape
Running 2179 tests.
landscape.broker.tests.test_amp
  RemoteBrokerTest
    test_call_if_accepted_with_not_accepted ... [OK]
    test_fire_event ... [OK]
    test_get_accepted_message_types ... [OK]
    test_get_server_uuid ... [OK]
    test_is_message_pending ... [OK]
    test_listen_events ... [OK]
    test_call_if_accepted ... [OK]
    test_call_on_events ... [OK]
    test_method_call_error ... [OK]
    test_register_client ... [OK]
    test_ping ... [OK]
    test_register_client_accepted_message_type ... [OK]
    test_send_message ... [OK]
    test_reload_configuration ... ...

[Eric Snow (ericsnowcurrently)]

Looks like there was a timeout issue. I'll try it again.

[🤖 Landscape Builder (landscape-builder)]

Attempt to merge into lp:landscape-client failed due to conflicts:

text conflict in py3_ready_tests

[🤖 Landscape Builder (landscape-builder)]

No approved revision specified.

[lp:~gocept/landscape-client/py3-watchdog]

980. By Steffen Allner on 2017-03-28

Backmerge from trunk

[Steffen Allner (sallner)]

I resolved the conflict.

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make ci-check
Result: Success
Revno: 980
Branch: lp:~gocept/landscape-client/py3-watchdog
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3800/

review: Approve (test results)

=== modified file 'landscape/tests/test_watchdog.py'
--- landscape/tests/test_watchdog.py	2017-03-13 08:51:36 +0000
+++ landscape/tests/test_watchdog.py	2017-03-28 18:33:15 +0000
@@ -11,6 +11,7 @@
 from twisted.internet import reactor
 from twisted.python.fakepwd import UserDatabase
 
+from landscape.lib.fs import read_text_file
 from landscape.tests.clock import Clock
 from landscape.tests.helpers import (
     LandscapeTest, EnvironSaverHelper, FakeBrokerServiceHelper)
@@ -25,9 +26,6 @@
 
 import landscape.watchdog
 
-from unittest import skipIf
-from twisted.python.compat import _PY3
-
 
 class StubDaemon(object):
     program = "program-name"
@@ -800,7 +798,7 @@
 
             self.assertTrue("Can't keep landscape-broker running." in
                             self.logfile.getvalue())
-            self.assertItemsEqual([True], stopped)
+            self.assertCountEqual([True], stopped)
             reactor.stop = stop[0]
 
         result = Deferred()
@@ -844,7 +842,7 @@
 
             self.assertTrue("Can't keep landscape-broker running." in
                             self.logfile.getvalue())
-            self.assertItemsEqual([True], stopped)
+            self.assertCountEqual([True], stopped)
             reactor.stop = stop[0]
 
         result = Deferred()
@@ -950,7 +948,6 @@
             mock.ANY, mock.ANY, args=mock.ANY, env=mock.ANY, uid=None,
             gid=None)
 
-    @skipIf(_PY3, 'Takes long with Python3, probably unclean Reactor')
     def test_request_exit(self):
         """The request_exit() method calls exit() on the broker process."""
 
@@ -973,8 +970,8 @@
         self.daemon.request_exit()
 
         def got_result(result):
-            self.assertEqual(result, "")
-            self.assertEqual(open(output_filename).read(), "CALLED")
+            self.assertEqual(result, b"")
+            self.assertEqual(read_text_file(output_filename), "CALLED")
 
         return process_result.addCallback(got_result)
 

=== modified file 'landscape/user/tests/test_changes.py'
--- landscape/user/tests/test_changes.py	2017-03-22 14:56:26 +0000
+++ landscape/user/tests/test_changes.py	2017-03-28 18:33:15 +0000
@@ -1,5 +1,3 @@
-from twisted.python.compat import _PY3
-
 from landscape.lib.persist import Persist
 from landscape.user.changes import UserChanges
 from landscape.user.tests.helpers import FakeUserInfo, FakeUserProvider
@@ -292,14 +290,12 @@
         groups.append(("sales", "x", 70, ["bo"]))
         # Remove user "jdoe"
         users.pop(0)
-        if _PY3:
-            assertion = self.assertCountEqual
-        else:
-            assertion = self.assertItemsEqual
-        assertion(changes.create_diff(),
-                  {"create-groups": [{"gid": 50, "name": "developers"},
-                                     {"gid": 70, "name": "sales"}],
-                   "delete-users": ["jdoe"],
-                   "delete-groups": ["webdev"],
-                   "create-group-members": {"developers": ["bo"],
-                                            "sales": ["bo"]}})
+
+        self.assertCountEqual(
+            changes.create_diff(),
+            {"create-groups": [{"gid": 50, "name": "developers"},
+                               {"gid": 70, "name": "sales"}],
+             "delete-users": ["jdoe"],
+             "delete-groups": ["webdev"],
+             "create-group-members": {"developers": ["bo"],
+                                      "sales": ["bo"]}})

=== modified file 'landscape/watchdog.py'
--- landscape/watchdog.py	2017-01-19 08:34:52 +0000
+++ landscape/watchdog.py	2017-03-28 18:33:15 +0000
@@ -339,7 +339,9 @@
         self.broker = broker
         self.monitor = monitor
         self.manager = manager
-        self.daemons = filter(None, [self.broker, self.monitor, self.manager])
+        self.daemons = [daemon
+                        for daemon in [self.broker, self.monitor, self.manager]
+                        if daemon]
         self.reactor = reactor
         self._checking = None
         self._stopping = False

=== modified file 'py3_ready_tests'
--- py3_ready_tests	2017-03-28 17:05:21 +0000
+++ py3_ready_tests	2017-03-28 18:33:15 +0000
@@ -2,6 +2,7 @@
 landscape.sysinfo.tests
 landscape.user.tests
 landscape.package.tests
+landscape.tests.test_watchdog
 
 landscape.tests.test_configuration
 landscape.tests.test_schema