Merge lp:~gocept/landscape-client/py3-user into lp:~landscape/landscape-client/trunk

[Steffen Allner]

Make landscape.user module py2/py3 compatible.

This MP considers the landscape.user module and makes it py2/py3 compatible.

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Fail
Revno: 969
Branch: lp:~gocept/landscape-client/py3-user
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3703/

review: Needs Fixing (test results)

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Fail
Revno: 969
Branch: lp:~gocept/landscape-client/py3-user
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3705/

review: Needs Fixing (test results)

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Fail
Revno: 969
Branch: lp:~gocept/landscape-client/py3-user
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3706/

review: Needs Fixing (test results)

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[Данило Шеган (danilo)]

We are missing python3-passlib from depends3 rule: I've manually installed it on a few slaves, but probably best to add it to depends3 so the tests can run.

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 969
Branch: lp:~gocept/landscape-client/py3-user
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3707/

review: Approve (test results)

[Данило Шеган (danilo)]

Looks good, a few nits inline.

review: Needs Information

[Steffen Allner (sallner)]

Added some inline comments, I will change the dependencies in the Makefile though.

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[Данило Шеган (danilo)]

review: Approve

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 970
Branch: lp:~gocept/landscape-client/py3-user
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3709/

review: Approve (test results)

[Daniel Havlik (nilo)]

+1

review: Approve

[🤖 Landscape Builder (landscape-builder)]

Attempt to merge into lp:landscape-client failed due to conflicts:

text conflict in py3_ready_tests

[🤖 Landscape Builder (landscape-builder)]

No approved revision specified.

[lp:~gocept/landscape-client/py3-user]

971. By Steffen Allner on 2017-03-23

Backmerge from trunk.

[Steffen Allner (sallner)]

I resolved the conflict.

[🤖 Landscape Builder (landscape-builder)]

review: Abstain (executing tests)

[🤖 Landscape Builder (landscape-builder)]

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 971
Branch: lp:~gocept/landscape-client/py3-user
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3716/

review: Approve (test results)

=== modified file 'Makefile'
--- Makefile	2017-03-17 09:57:31 +0000
+++ Makefile	2017-03-23 14:26:32 +0000
@@ -46,7 +46,7 @@
 	LC_ALL=C $(TEST_COMMAND_PY3_READY)
 
 depends3:
-	sudo apt -y install python3-twisted python3-distutils-extra python3-mock python3-configobj
+	sudo apt -y install python3-twisted python3-distutils-extra python3-mock python3-configobj python3-passlib
 
 lint:
 	bzr ls-lint

=== modified file 'landscape/user/changes.py'
--- landscape/user/changes.py	2017-01-23 12:13:14 +0000
+++ landscape/user/changes.py	2017-03-23 14:26:32 +0000
@@ -136,6 +136,6 @@
                 members.extend(update_groups)
 
         if deletes:
-            changes["delete-groups"] = deletes.keys()
+            changes["delete-groups"] = sorted(deletes.keys())
 
         return changes

=== modified file 'landscape/user/management.py'
--- landscape/user/management.py	2017-03-10 10:31:30 +0000
+++ landscape/user/management.py	2017-03-23 14:26:32 +0000
@@ -48,9 +48,7 @@
         return output
 
     def _set_password(self, username, password):
-        # XXX temporary workaround? We're getting unicode here.
-        username = username.encode("ascii")
-        password = password.encode("ascii")
+        password = password.encode("utf-8")
         crypted = md5_crypt.encrypt(password)
         result, output = self.call_popen(["usermod", "-p", crypted, username])
         if result != 0:

=== modified file 'landscape/user/provider.py'
--- landscape/user/provider.py	2017-03-10 14:39:38 +0000
+++ landscape/user/provider.py	2017-03-23 14:26:32 +0000
@@ -119,8 +119,13 @@
         directory, path to the user's shell)
         """
         user_data = []
+        # The DictReader takes bytes in Python 2 and unicode in Python 3 so we
+        # have to pass Python 3 specific parameters to open() and do the
+        # decoding for Python 2 later after we have parsed the rows. We have to
+        # explicitly indicate the encoding as we cannot rely on the system
+        # default encoding.
         if _PY3:
-            open_params = dict(errors='replace')
+            open_params = dict(encoding="utf-8", errors='replace')
         else:
             open_params = dict()
         with open(self._passwd_file, "r", **open_params) as passwd_file:

=== modified file 'landscape/user/tests/test_changes.py'
--- landscape/user/tests/test_changes.py	2016-01-11 18:34:49 +0000
+++ landscape/user/tests/test_changes.py	2017-03-23 14:26:32 +0000
@@ -1,3 +1,5 @@
+from twisted.python.compat import _PY3
+
 from landscape.lib.persist import Persist
 from landscape.user.changes import UserChanges
 from landscape.user.tests.helpers import FakeUserInfo, FakeUserProvider
@@ -290,10 +292,14 @@
         groups.append(("sales", "x", 70, ["bo"]))
         # Remove user "jdoe"
         users.pop(0)
-        self.assertEqual(changes.create_diff(),
-                         {"create-groups": [{"gid": 50, "name": "developers"},
-                                            {"gid": 70, "name": "sales"}],
-                          "delete-users": ["jdoe"],
-                          "delete-groups": ["webdev"],
-                          "create-group-members": {"developers": ["bo"],
-                                                   "sales": ["bo"]}})
+        if _PY3:
+            assertion = self.assertCountEqual
+        else:
+            assertion = self.assertItemsEqual
+        assertion(changes.create_diff(),
+                  {"create-groups": [{"gid": 50, "name": "developers"},
+                                     {"gid": 70, "name": "sales"}],
+                   "delete-users": ["jdoe"],
+                   "delete-groups": ["webdev"],
+                   "create-group-members": {"developers": ["bo"],
+                                            "sales": ["bo"]}})

=== modified file 'landscape/user/tests/test_provider.py'
--- landscape/user/tests/test_provider.py	2017-03-10 15:25:44 +0000
+++ landscape/user/tests/test_provider.py	2017-03-23 14:26:32 +0000
@@ -130,19 +130,16 @@
     def test_utf8_gecos_data(self):
         """Gecos fields should be decoded from utf-8 to unicode."""
         name = u"Jos\N{LATIN SMALL LETTER E WITH ACUTE}"
-        location = "F\N{LATIN SMALL LETTER I WITH DIAERESIS}nland"
-        number = "N\N{LATIN SMALL LETTER AE}ver"
-        if _PY3:
-            gecos = "{},{},{},{},".format(name, location, number, number)
-        else:
-            gecos = "{},{},{},{},".format(name.encode("utf-8"),
-                                          location.encode("utf-8"),
-                                          number.encode("utf-8"),
-                                          number.encode("utf-8"))
-        passwd_file = self.makeFile("""\
+        location = u"F\N{LATIN SMALL LETTER I WITH DIAERESIS}nland"
+        number = u"N\N{LATIN SMALL LETTER AE}ver"
+        gecos = u"{},{},{},{},".format(name, location, number, number)
+        # We explicitly want to encode this file with utf-8 so we can write in
+        # binary mode and do not rely on the default encoding.
+        utf8_content = u"""\
 jdoe:x:1000:1000:{}:/home/jdoe:/bin/zsh
 root:x:0:0:root:/root:/bin/bash
-""".format(gecos))
+""".format(gecos).encode("utf-8")
+        passwd_file = self.makeFile(utf8_content, mode="wb")
         provider = UserProvider(passwd_file=passwd_file,
                                 group_file=self.group_file)
         users = provider.get_users()

=== modified file 'py3_ready_tests'
--- py3_ready_tests	2017-03-23 13:44:26 +0000
+++ py3_ready_tests	2017-03-23 14:26:32 +0000
@@ -1,5 +1,6 @@
 landscape.lib.tests
 landscape.sysinfo.tests
+landscape.user.tests
 landscape.package.tests