Landscape Client

Merge lp:~gocept/landscape-client/py3-package-store-reporter into lp:~landscape/landscape-client/trunk

  1. py3-package-store-reporter
  2. Merge into trunk
Proposed by Steffen Allner
Status: Merged
Approved by: Данило Шеган
Approved revision: 987
Merged at revision: 965
Proposed branch: lp:~gocept/landscape-client/py3-package-store-reporter
Merge into: lp:~landscape/landscape-client/trunk
Diff against target: 664 lines (+122/-118)
11 files modified
landscape/broker/store.py (+3/-10)
landscape/compat.py (+0/-13)
landscape/package/facade.py (+2/-1)
landscape/package/reporter.py (+4/-2)
landscape/package/skeleton.py (+6/-2)
landscape/package/store.py (+10/-9)
landscape/package/tests/helpers.py (+32/-17)
landscape/package/tests/test_reporter.py (+25/-27)
landscape/package/tests/test_store.py (+35/-35)
landscape/schema.py (+2/-2)
py3_ready_tests (+3/-0)
To merge this branch: bzr merge lp:~gocept/landscape-client/py3-package-store-reporter
Reviewer Review Type Date Requested Status
Данило Шеган (community) Approve
🤖 Landscape Builder test results Approve
Daniel Havlik (community) Approve
Review via email: mp+320148@code.launchpad.net

Commit message

Support py2/3 in landscape.package.store and landscape.package.reporter modules.

Description of the change

We are on the journey to get the Bytes / Unicode story solved for Python 2/3 compatibility.

This MP considers code in landscape.package.store and landscape.package.reporter and their respective tests. A major point was the explicit declaration, that hashes have to be bytes, as the hashing functions in Python do not automatically encode with ascii. Additionally other APIs required to be explicit with unicode and bytes declaration.

To post a comment you must log in.
Revision history for this message
🤖 Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
lp:~gocept/landscape-client/py3-package-store-reporter updated
982. By Steffen Allner

Comply with PEP-8 and coding style.

Revision history for this message
🤖 Landscape Builder (landscape-builder) wrote :

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 981
Branch: lp:~gocept/landscape-client/py3-package-store-reporter
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3632/

review: Approve (test results)
Revision history for this message
🤖 Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
🤖 Landscape Builder (landscape-builder) wrote :

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 982
Branch: lp:~gocept/landscape-client/py3-package-store-reporter
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3633/

review: Approve (test results)
Revision history for this message
Данило Шеган (danilo) wrote :

FWIW, bpickle is now back to landscape.lib.bpickle, so there are a few conflicts in the imports now (Launchpad does not regenerate the diff when the target changes).

lp:~gocept/landscape-client/py3-package-store-reporter updated
983. By Steffen Allner

Backmerge from trunk with py2/3 bpickle.

Revision history for this message
Steffen Allner (sallner) wrote :

> FWIW, bpickle is now back to landscape.lib.bpickle, so there are a few
> conflicts in the imports now (Launchpad does not regenerate the diff when the
> target changes).

I have merged the trunk back into this branch. Should be updated now.

Revision history for this message
🤖 Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
🤖 Landscape Builder (landscape-builder) wrote :

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 983
Branch: lp:~gocept/landscape-client/py3-package-store-reporter
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3639/

review: Approve (test results)
Revision history for this message
Данило Шеган (danilo) wrote :

Looks good: a few minor nits inline. I'll just do a test run with this before I mark it approved.

Revision history for this message
Данило Шеган (danilo) :
review: Approve
Revision history for this message
Steffen Allner (sallner) wrote :

See inline comments. I would especially interested in a direction for the StrictVersion case.

lp:~gocept/landscape-client/py3-package-store-reporter updated
984. By Steffen Allner

Remove no longer needed helper method.

985. By Steffen Allner

Remove superfluous assignment.

986. By Steffen Allner

Use undocumented parameter to ensure bytes as return value.

Revision history for this message
Steffen Allner (sallner) wrote :

I cleaned up and documented the mentioned parts.

Additionally, I opened up a new MP which uses only bytes for software comparison[0].

[0] https://code.launchpad.net/~gocept/landscape-client/bytes-versioning/+merge/320494

Revision history for this message
Данило Шеган (danilo) wrote :

Replied inline. If you want, we can deal with the is_version_higher() fix in a follow-up branch and get this one landed, but please get a Gocept review as well.

Revision history for this message
Данило Шеган (danilo) wrote :

Cool, haven't refreshed with inline comments for a while: there's a conflict in this branch now. Please fix a conflict and get a ~gocept review and I'll get this branch landed.

Revision history for this message
Данило Шеган (danilo) wrote :

After looking at the other branch, I guess you might want to land that one first. Then we can merge trunk here, resolve a conflict and land this one too.

Revision history for this message
Daniel Havlik (nilo) wrote :

If conflict is fixed: +1

review: Approve
Revision history for this message
Steffen Allner (sallner) wrote :

Then let's wait for the other branch to arrive and I can fix both afterwards.

Revision history for this message
Данило Шеган (danilo) wrote :

Landed the other branch, please re-merge trunk into this one and resolve the conflict.

lp:~gocept/landscape-client/py3-package-store-reporter updated
987. By Steffen Allner

Backmerge from trunk.

Revision history for this message
🤖 Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
🤖 Landscape Builder (landscape-builder) wrote :

Command: TRIAL_ARGS=-j4 make check
Result: Fail
Revno: 987
Branch: lp:~gocept/landscape-client/py3-package-store-reporter
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3689/

review: Needs Fixing (test results)
Revision history for this message
Steffen Allner (sallner) wrote :

It seems there is again the same flaky test, we have seen before. Danilo, do you have an idea, how we could make it more robust? I can never reproduce that failre with Python 2. Maybe you want to restart it.

Revision history for this message
🤖 Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
🤖 Landscape Builder (landscape-builder) wrote :

Command: TRIAL_ARGS=-j4 make check
Result: Success
Revno: 987
Branch: lp:~gocept/landscape-client/py3-package-store-reporter
Jenkins: https://ci.lscape.net/job/latch-test-xenial/3692/

review: Approve (test results)
Revision history for this message
Данило Шеган (danilo) wrote :

Landing.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'landscape/broker/store.py'
--- landscape/broker/store.py 2017-03-17 09:26:45 +0000
+++ landscape/broker/store.py 2017-03-21 17:01:15 +0000
@@ -101,7 +101,7 @@
101101
102from landscape import DEFAULT_SERVER_API102from landscape import DEFAULT_SERVER_API
103from landscape.lib import bpickle103from landscape.lib import bpickle
104from landscape.lib.fs import create_binary_file104from landscape.lib.fs import create_binary_file, read_binary_file
105from landscape.lib.versioning import sort_versions, is_version_higher105from landscape.lib.versioning import sort_versions, is_version_higher
106106
107107
@@ -260,7 +260,7 @@
260 for filename in self._walk_pending_messages():260 for filename in self._walk_pending_messages():
261 if max is not None and len(messages) >= max:261 if max is not None and len(messages) >= max:
262 break262 break
263 data = self._get_content(self._message_dir(filename))263 data = read_binary_file(self._message_dir(filename))
264 try:264 try:
265 message = bpickle.loads(data)265 message = bpickle.loads(data)
266 except ValueError as e:266 except ValueError as e:
@@ -436,13 +436,6 @@
436 def _message_dir(self, *args):436 def _message_dir(self, *args):
437 return os.path.join(self._directory, *args)437 return os.path.join(self._directory, *args)
438438
439 def _get_content(self, filename):
440 file = open(filename, 'rb')
441 try:
442 return file.read()
443 finally:
444 file.close()
445
446 def _reprocess_holding(self):439 def _reprocess_holding(self):
447 """440 """
448 Unhold accepted messages left behind, and hold unaccepted441 Unhold accepted messages left behind, and hold unaccepted
@@ -454,7 +447,7 @@
454 for old_filename in self._walk_messages():447 for old_filename in self._walk_messages():
455 flags = self._get_flags(old_filename)448 flags = self._get_flags(old_filename)
456 try:449 try:
457 message = bpickle.loads(self._get_content(old_filename))450 message = bpickle.loads(read_binary_file(old_filename))
458 except ValueError as e:451 except ValueError as e:
459 logging.exception(e)452 logging.exception(e)
460 if HELD not in flags:453 if HELD not in flags:
461454
=== modified file 'landscape/compat.py'
--- landscape/compat.py 2017-03-17 09:26:45 +0000
+++ landscape/compat.py 2017-03-21 17:01:15 +0000
@@ -46,16 +46,3 @@
46 return s.decode(encoding, errors)46 return s.decode(encoding, errors)
47 else:47 else:
48 return s48 return s
49
50
51def convert_buffer_to_string(mem_view):
52 """
53 Converts a buffer in Python 2 or a memoryview in Python 3 to str.
54
55 @param mem_view: The view to convert.
56 """
57 if _PY3:
58 result = mem_view.decode('ascii')
59 else:
60 result = str(mem_view)
61 return result
6249
=== modified file 'landscape/package/facade.py'
--- landscape/package/facade.py 2017-03-13 15:38:09 +0000
+++ landscape/package/facade.py 2017-03-21 17:01:15 +0000
@@ -196,7 +196,8 @@
196 process = subprocess.Popen(196 process = subprocess.Popen(
197 ["dpkg", "--set-selections"] + self._dpkg_args,197 ["dpkg", "--set-selections"] + self._dpkg_args,
198 stdin=subprocess.PIPE)198 stdin=subprocess.PIPE)
199 process.communicate(selection)199 # We need bytes here to communicate with the process.
200 process.communicate(selection.encode("utf-8"))
200201
201 def set_package_hold(self, version):202 def set_package_hold(self, version):
202 """Add a dpkg hold for a package.203 """Add a dpkg hold for a package.
203204
=== modified file 'landscape/package/reporter.py'
--- landscape/package/reporter.py 2017-03-20 09:43:08 +0000
+++ landscape/package/reporter.py 2017-03-21 17:01:15 +0000
@@ -20,7 +20,6 @@
20from landscape.lib.fs import touch_file20from landscape.lib.fs import touch_file
21from landscape.lib.lsb_release import parse_lsb_release, LSB_RELEASE_FILENAME21from landscape.lib.lsb_release import parse_lsb_release, LSB_RELEASE_FILENAME
2222
23from landscape.compat import convert_buffer_to_string
24from landscape.package.taskhandler import (23from landscape.package.taskhandler import (
25 PackageTaskHandlerConfiguration, PackageTaskHandler, run_task_handler)24 PackageTaskHandlerConfiguration, PackageTaskHandler, run_task_handler)
26from landscape.package.store import UnknownHashIDRequest, FakePackageStore25from landscape.package.store import UnknownHashIDRequest, FakePackageStore
@@ -231,6 +230,9 @@
231 self._reactor.call_later(230 self._reactor.call_later(
232 LOCK_RETRY_DELAYS[retry], self._apt_update, deferred)231 LOCK_RETRY_DELAYS[retry], self._apt_update, deferred)
233 out, err, code = yield deferred232 out, err, code = yield deferred
233 out = out.decode("utf-8")
234 err = err.decode("utf-8")
235
234 timestamp = self._reactor.time()236 timestamp = self._reactor.time()
235237
236 touch_file(self._config.update_stamp_filename)238 touch_file(self._config.update_stamp_filename)
@@ -730,7 +732,7 @@
730 messages = global_store.get_messages_by_ids(not_sent)732 messages = global_store.get_messages_by_ids(not_sent)
731 sent = []733 sent = []
732 for message_id, message in messages:734 for message_id, message in messages:
733 message = bpickle.loads(convert_buffer_to_string(message))735 message = bpickle.loads(message)
734 if message["type"] not in got_type:736 if message["type"] not in got_type:
735 got_type.add(message["type"])737 got_type.add(message["type"])
736 sent.append(message_id)738 sent.append(message_id)
737739
=== modified file 'landscape/package/skeleton.py'
--- landscape/package/skeleton.py 2017-03-10 12:19:57 +0000
+++ landscape/package/skeleton.py 2017-03-21 17:01:15 +0000
@@ -50,10 +50,14 @@
50 """50 """
51 if self._hash is not None:51 if self._hash is not None:
52 return self._hash52 return self._hash
53 digest = sha1("[%d %s %s]" % (self.type, self.name, self.version))53 # We use ascii here as encoding for backwards compatibility as it was
54 # default encoding for conversion from unicode to bytes in Python 2.7.
55 package_info = b"[%d %s %s]" % (
56 self.type, self.name.encode("ascii"), self.version.encode("ascii"))
57 digest = sha1(package_info)
54 self.relations.sort()58 self.relations.sort()
55 for pair in self.relations:59 for pair in self.relations:
56 digest.update("[%d %s]" % pair)60 digest.update(b"[%d %s]" % (pair[0], pair[1].encode("ascii")))
57 return digest.digest()61 return digest.digest()
5862
59 def set_hash(self, package_hash):63 def set_hash(self, package_hash):
6064
=== modified file 'landscape/package/store.py'
--- landscape/package/store.py 2017-03-17 09:26:45 +0000
+++ landscape/package/store.py 2017-03-21 17:01:15 +0000
@@ -7,9 +7,7 @@
7 from pysqlite2 import dbapi2 as sqlite37 from pysqlite2 import dbapi2 as sqlite3
88
9from twisted.python.compat import iteritems, long9from twisted.python.compat import iteritems, long
10from twisted.python.compat import StringType as basestring
1110
12from landscape.compat import convert_buffer_to_string
13from landscape.lib import bpickle11from landscape.lib import bpickle
14from landscape.lib.store import with_cursor12from landscape.lib.store import with_cursor
1513
@@ -50,7 +48,10 @@
5048
51 @with_cursor49 @with_cursor
52 def get_hash_id(self, cursor, hash):50 def get_hash_id(self, cursor, hash):
53 """Return the id associated to C{hash}, or C{None} if not available."""51 """Return the id associated to C{hash}, or C{None} if not available.
52
53 @param hash: a C{bytes} representing a hash.
54 """
54 cursor.execute("SELECT id FROM hash WHERE hash=?",55 cursor.execute("SELECT id FROM hash WHERE hash=?",
55 (sqlite3.Binary(hash),))56 (sqlite3.Binary(hash),))
56 value = cursor.fetchone()57 value = cursor.fetchone()
@@ -62,7 +63,7 @@
62 def get_hash_ids(self, cursor):63 def get_hash_ids(self, cursor):
63 """Return a C{dict} holding all the available hash=>id mappings."""64 """Return a C{dict} holding all the available hash=>id mappings."""
64 cursor.execute("SELECT hash, id FROM hash")65 cursor.execute("SELECT hash, id FROM hash")
65 return dict([(str(row[0]), row[1]) for row in cursor.fetchall()])66 return {bytes(row[0]): row[1] for row in cursor.fetchall()}
6667
67 @with_cursor68 @with_cursor
68 def get_id_hash(self, cursor, id):69 def get_id_hash(self, cursor, id):
@@ -71,7 +72,7 @@
71 cursor.execute("SELECT hash FROM hash WHERE id=?", (id,))72 cursor.execute("SELECT hash FROM hash WHERE id=?", (id,))
72 value = cursor.fetchone()73 value = cursor.fetchone()
73 if value:74 if value:
74 return str(value[0])75 return bytes(value[0])
75 return None76 return None
7677
77 @with_cursor78 @with_cursor
@@ -149,7 +150,7 @@
149 the attached lookaside databases, falling back to the main one, as150 the attached lookaside databases, falling back to the main one, as
150 described in L{add_hash_id_db}.151 described in L{add_hash_id_db}.
151 """152 """
152 assert isinstance(hash, basestring)153 assert isinstance(hash, bytes)
153154
154 # Check if we can find the hash=>id mapping in the lookaside stores155 # Check if we can find the hash=>id mapping in the lookaside stores
155 for store in self._hash_id_stores:156 for store in self._hash_id_stores:
@@ -332,7 +333,7 @@
332 result = cursor.execute(333 result = cursor.execute(
333 "SELECT id, data FROM message WHERE id IN (%s) "334 "SELECT id, data FROM message WHERE id IN (%s) "
334 "ORDER BY id" % params, tuple(message_ids)).fetchall()335 "ORDER BY id" % params, tuple(message_ids)).fetchall()
335 return [(row[0], row[1]) for row in result]336 return [(row[0], bytes(row[1])) for row in result]
336337
337338
338class HashIDRequest(object):339class HashIDRequest(object):
@@ -346,7 +347,7 @@
346 def hashes(self, cursor):347 def hashes(self, cursor):
347 cursor.execute("SELECT hashes FROM hash_id_request WHERE id=?",348 cursor.execute("SELECT hashes FROM hash_id_request WHERE id=?",
348 (self.id,))349 (self.id,))
349 return bpickle.loads(convert_buffer_to_string(cursor.fetchone()[0]))350 return bpickle.loads(bytes(cursor.fetchone()[0]))
350351
351 @with_cursor352 @with_cursor
352 def _get_timestamp(self, cursor):353 def _get_timestamp(self, cursor):
@@ -395,7 +396,7 @@
395396
396 self.queue = row[0]397 self.queue = row[0]
397 self.timestamp = row[1]398 self.timestamp = row[1]
398 self.data = bpickle.loads(convert_buffer_to_string(row[2]))399 self.data = bpickle.loads(bytes(row[2]))
399400
400 @with_cursor401 @with_cursor
401 def remove(self, cursor):402 def remove(self, cursor):
402403
=== modified file 'landscape/package/tests/helpers.py'
--- landscape/package/tests/helpers.py 2017-03-13 15:15:46 +0000
+++ landscape/package/tests/helpers.py 2017-03-21 17:01:15 +0000
@@ -6,7 +6,7 @@
6import apt_inst6import apt_inst
7import apt_pkg7import apt_pkg
88
9from landscape.lib.fs import append_binary_file, append_text_file9from landscape.lib.fs import append_binary_file
10from landscape.lib.fs import create_binary_file10from landscape.lib.fs import create_binary_file
11from landscape.package.facade import AptFacade11from landscape.package.facade import AptFacade
1212
@@ -49,11 +49,17 @@
49 """ % {49 """ % {
50 "name": name, "version": version,50 "name": name, "version": version,
51 "architecture": architecture,51 "architecture": architecture,
52 "description": description.encode("utf-8")})52 "description": description}).encode("utf-8")
53 # We want to re-order the TagSection, but it requires bytes as input.
54 # As we also want to write a binary file, we have to explicitly pass
55 # the hardly documented `bytes=True` to TagSection as it would be
56 # returned as unicode in Python 3 otherwise. In future versions of
57 # apt_pkg there should be a TagSection.write() which is recommended.
53 package_stanza = apt_pkg.rewrite_section(58 package_stanza = apt_pkg.rewrite_section(
54 apt_pkg.TagSection(package_stanza), apt_pkg.REWRITE_PACKAGE_ORDER,59 apt_pkg.TagSection(package_stanza, bytes=True),
55 control_fields.items())60 apt_pkg.REWRITE_PACKAGE_ORDER,
56 append_binary_file(packages_file, "\n" + package_stanza + "\n")61 list(control_fields.items()))
62 append_binary_file(packages_file, b"\n" + package_stanza + b"\n")
5763
58 def _add_system_package(self, name, architecture="all", version="1.0",64 def _add_system_package(self, name, architecture="all", version="1.0",
59 control_fields=None):65 control_fields=None):
@@ -72,9 +78,9 @@
72 control = deb.control.extractdata("control")78 control = deb.control.extractdata("control")
73 deb_file.close()79 deb_file.close()
74 lines = control.splitlines()80 lines = control.splitlines()
75 lines.insert(1, "Status: install ok installed")81 lines.insert(1, b"Status: install ok installed")
76 status = "\n".join(lines)82 status = b"\n".join(lines)
77 append_text_file(self.dpkg_status, status + "\n\n")83 append_binary_file(self.dpkg_status, status + b"\n\n")
7884
79 def _add_package_to_deb_dir(self, path, name, architecture="all",85 def _add_package_to_deb_dir(self, path, name, architecture="all",
80 version="1.0", description="description",86 version="1.0", description="description",
@@ -161,7 +167,8 @@
161 "Mzepwz6J7y5jpkIOH6sDKssF1rmUqYzBX2piZj9zyFad5RHv8dLoXsqua2spF3v+PQ"167 "Mzepwz6J7y5jpkIOH6sDKssF1rmUqYzBX2piZj9zyFad5RHv8dLoXsqua2spF3v+PQ"
162 "ffXIlN8aYepsu3x2u0202VX+QFC10st6vvMfDdacgtdzKtpe5G5tuFYx5elcpXm27O"168 "ffXIlN8aYepsu3x2u0202VX+QFC10st6vvMfDdacgtdzKtpe5G5tuFYx5elcpXm27O"
163 "d8LH7Oj3mqP7VgD8P6dTmJ33dsPnpuBnPO3SvLDNlu6ay9It6yZon0BIZRMApGwSgY"169 "d8LH7Oj3mqP7VgD8P6dTmJ33dsPnpuBnPO3SvLDNlu6ay9It6yZon0BIZRMApGwSgY"
164 "BaNgFIyCUTAKRsEoGAWjYBSMglEwCkbBKBgFo2AUjIJRMApGAUkAADhX8vgAKAAA ")170 "BaNgFIyCUTAKRsEoGAWjYBSMglEwCkbBKBgFo2AUjIJRMApGAUkAADhX8vgAKAAA "
171 ).encode("ascii")
165172
166PKGDEB2 = ("ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTE2NjExNDUyMiAgMCAgICAgMCAgICAgMT"173PKGDEB2 = ("ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTE2NjExNDUyMiAgMCAgICAgMCAgICAgMT"
167 "AwNjQ0ICA0ICAgICAgICAgYAoyLjAKY29udHJvbC50YXIuZ3ogIDExNjYxMTQ1MjIg"174 "AwNjQ0ICA0ICAgICAgICAgYAoyLjAKY29udHJvbC50YXIuZ3ogIDExNjYxMTQ1MjIg"
@@ -183,7 +190,8 @@
183 "jR45xB99RGrkMGEq4Pbf0L3UWDL4XIRIk6Hjx7Urzj6SSxS/YTzKbu28sqe/64oPmF"190 "jR45xB99RGrkMGEq4Pbf0L3UWDL4XIRIk6Hjx7Urzj6SSxS/YTzKbu28sqe/64oPmF"
184 "JGPj3lqR1cLMdz12u04rLHp/gM2y0mv3HOc/GqxvCl7PqWh7kbux6VrFk69zlefZsu"191 "JGPj3lqR1cLMdz12u04rLHp/gM2y0mv3HOc/GqxvCl7PqWh7kbux6VrFk69zlefZsu"
185 "v5WPycH/NUv7VgF8N6vfeBcgXp3NlnBFNDw5eZsd1as/aK+JzyvZ0TGEbBKBgFo2AU"192 "v5WPycH/NUv7VgF8N6vfeBcgXp3NlnBFNDw5eZsd1as/aK+JzyvZ0TGEbBKBgFo2AU"
186 "jIJRMApGwSgYBaNgFIyCUTAKRsEoGAWjYBSMglEwCkbBKBgFJAEAu4OlKQAoAAAK")193 "jIJRMApGwSgYBaNgFIyCUTAKRsEoGAWjYBSMglEwCkbBKBgFJAEAu4OlKQAoAAAK"
194 ).encode("ascii")
187195
188PKGDEB3 = ("ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTE2OTE0ODIwMyAgMCAgICAgMCAgICAgMT"196PKGDEB3 = ("ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTE2OTE0ODIwMyAgMCAgICAgMCAgICAgMT"
189 "AwNjQ0ICA0ICAgICAgICAgYAoyLjAKY29udHJvbC50YXIuZ3ogIDExNjkxNDgyMDMg"197 "AwNjQ0ICA0ICAgICAgICAgYAoyLjAKY29udHJvbC50YXIuZ3ogIDExNjkxNDgyMDMg"
@@ -206,7 +214,8 @@
206 "bOTd7zh0Xz0y5bdGmDrbLp/dbhNtdpU/EFSt9LKe7/xHgzWn4PWcirYXuVsbrlVMeT"214 "bOTd7zh0Xz0y5bdGmDrbLp/dbhNtdpU/EFSt9LKe7/xHgzWn4PWcirYXuVsbrlVMeT"
207 "pXaZ4t+zkfi5/zY57qTy3Yw7B+XU7g+8L07rmG7Fe2bVxmyHZLZ+0V8Sl2Xj8mMIyC"215 "pXaZ4t+zkfi5/zY57qTy3Yw7B+XU7g+8L07rmG7Fe2bVxmyHZLZ+0V8Sl2Xj8mMIyC"
208 "UTAKRsEoGAWjYBSMglEwCkbBKBgFo2AUjIJRMApGwSgYBaNgFIyCUTAKSAIAY/FOKA"216 "UTAKRsEoGAWjYBSMglEwCkbBKBgFo2AUjIJRMApGwSgYBaNgFIyCUTAKSAIAY/FOKA"
209 "AoAAAK")217 "AoAAAK"
218 ).encode("ascii")
210219
211PKGDEB4 = ("ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTI3NjUxMTU3OC41MCAgICAgMCAgICAgNj"220PKGDEB4 = ("ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTI3NjUxMTU3OC41MCAgICAgMCAgICAgNj"
212 "Q0ICAgICA0\nICAgICAgICAgYAoyLjAKY29udHJvbC50YXIuZ3ogIDEyNzY1MTE1Nz"221 "Q0ICAgICA0\nICAgICAgICAgYAoyLjAKY29udHJvbC50YXIuZ3ogIDEyNzY1MTE1Nz"
@@ -221,7 +230,8 @@
221 "ICAgYAofiwgAWgUWTAL/7dFBCsMgEEDRWfcUniCZ\nsU57kJ5ASJdFSOz9K9kULLQr"230 "ICAgYAofiwgAWgUWTAL/7dFBCsMgEEDRWfcUniCZ\nsU57kJ5ASJdFSOz9K9kULLQr"
222 "C4H/NiPqQvnTLMNpc3XfZ9PPfW2W1JOae9s3i5okuPzBc6t5bU9Z\nS6nf7v067z93"231 "C4H/NiPqQvnTLMNpc3XfZ9PPfW2W1JOae9s3i5okuPzBc6t5bU9Z\nS6nf7v067z93"
223 "ENO8lcd9fP/LZ/d3f4td/6h+lqD0H+7W6ocl13wSAAAAAAAAAAAAAAAAAAfzAqr5\n"232 "ENO8lcd9fP/LZ/d3f4td/6h+lqD0H+7W6ocl13wSAAAAAAAAAAAAAAAAAAfzAqr5\n"
224 "GFYAKAAACg==\n")233 "GFYAKAAACg==\n"
234 ).encode("ascii")
225235
226PKGDEB_MINIMAL = (236PKGDEB_MINIMAL = (
227 "ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTMxNzg5MDQ3OSAgMCAgICAgMCAgICAgMTAwNj"237 "ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTMxNzg5MDQ3OSAgMCAgICAgMCAgICAgMTAwNj"
@@ -234,7 +244,8 @@
234 "AAAAAAAAAAAAAAAAAAAAAMBF70s1/foAKAAAZGF0YS50YXIu Z3ogICAgIDEzMTc4OTA0N"244 "AAAAAAAAAAAAAAAAAAAAAMBF70s1/foAKAAAZGF0YS50YXIu Z3ogICAgIDEzMTc4OTA0N"
235 "zkgIDAgICAgIDAgICAgIDEwMDY0NCAgMTA3ICAgICAgIGAKH4sIAAAA AAACA+3KsQ3CQB"245 "zkgIDAgICAgIDAgICAgIDEwMDY0NCAgMTA3ICAgICAgIGAKH4sIAAAA AAACA+3KsQ3CQB"
236 "AEwCvlK4D/N4frMSGBkQz0jwmQiHCEo5lkpd09HOPv6mrMfGcbs37nR7R2Pg01"246 "AEwCvlK4D/N4frMSGBkQz0jwmQiHCEo5lkpd09HOPv6mrMfGcbs37nR7R2Pg01"
237 "ew5r32rvNUrGDp73x7SUEpfrbZl//LZ2AAAAAAAAAAAA2NELx33R7wAoAAAK")247 "ew5r32rvNUrGDp73x7SUEpfrbZl//LZ2AAAAAAAAAAAA2NELx33R7wAoAAAK"
248).encode("ascii")
238249
239PKGDEB_SIMPLE_RELATIONS = (250PKGDEB_SIMPLE_RELATIONS = (
240 "ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTMxODUxNjMyMiAgMCAgICAgMCAgICAgMTAwNj"251 "ITxhcmNoPgpkZWJpYW4tYmluYXJ5ICAgMTMxODUxNjMyMiAgMCAgICAgMCAgICAgMTAwNj"
@@ -249,7 +260,8 @@
249 "EKgcHt1gAoAABkYXRhLnRhci5neiAgICAgMTMxODUxNjMyMiAgMCAgICAgMCAg ICAgMTA"260 "EKgcHt1gAoAABkYXRhLnRhci5neiAgICAgMTMxODUxNjMyMiAgMCAgICAgMCAg ICAgMTA"
250 "wNjQ0ICAxMDcgICAgICAgYAofiwgAAAAAAAID7cqxDcJQEETBK8UVwH2b+64HQgIjGegf "261 "wNjQ0ICAxMDcgICAgICAgYAofiwgAAAAAAAID7cqxDcJQEETBK8UVwH2b+64HQgIjGegf "
251 "CJCIIMLRTPKC3d0+/i6f5qpX21z52bdorR+m7Fl9imw5jhVDxQbu19txHYY4nS/r8uX3aw"262 "CJCIIMLRTPKC3d0+/i6f5qpX21z52bdorR+m7Fl9imw5jhVDxQbu19txHYY4nS/r8uX3aw"
252 "cAAAAA AAAAAIANPQALnD6FACgAAAo=")263 "cAAAAA AAAAAIANPQALnD6FACgAAAo="
264).encode("ascii")
253265
254266
255PKGDEB_VERSION_RELATIONS = (267PKGDEB_VERSION_RELATIONS = (
@@ -265,7 +277,8 @@
265 "AAAACAy/sAwTtOtwAoAABkYXRhLnRhci5neiAgICAgMTMxODUxNjQ5OCAgMCAg ICAgMCA"277 "AAAACAy/sAwTtOtwAoAABkYXRhLnRhci5neiAgICAgMTMxODUxNjQ5OCAgMCAg ICAgMCA"
266 "gICAgMTAwNjQ0ICAxMDcgICAgICAgYAofiwgAAAAAAAID7cqxEcIwEETRK0UVgCT7UD0Q "278 "gICAgMTAwNjQ0ICAxMDcgICAgICAgYAofiwgAAAAAAAID7cqxEcIwEETRK0UVgCT7UD0Q "
267 "EpgZA/0DATNEEOHoveQHu7t9/F19GpmvtpH1s2/R2mGeemYfc9RW+9SjZGzgfr0d11LidL"279 "EpgZA/0DATNEEOHoveQHu7t9/F19GpmvtpH1s2/R2mGeemYfc9RW+9SjZGzgfr0d11LidL"
268 "6sy5ff rx0AAAAAAAAAAAA29AD/ixlwACgAAAo=")280 "6sy5ff rx0AAAAAAAAAAAA29AD/ixlwACgAAAo="
281).encode("ascii")
269282
270283
271PKGDEB_MULTIPLE_RELATIONS = (284PKGDEB_MULTIPLE_RELATIONS = (
@@ -282,7 +295,8 @@
282 "0YS50YXIuZ3ogICAgIDEzMTg1ODAwNzkgIDAgICAgIDAgICAgIDEwMDY0NCAgMTA3ICAg "295 "0YS50YXIuZ3ogICAgIDEzMTg1ODAwNzkgIDAgICAgIDAgICAgIDEwMDY0NCAgMTA3ICAg "
283 "ICAgIGAKH4sIAAAAAAACA+3KsRHCMBBE0StFFYBkfFY9EBKYGWP3DwTMEEGEo/eSH+wejv"296 "ICAgIGAKH4sIAAAAAAACA+3KsRHCMBBE0StFFYBkfFY9EBKYGWP3DwTMEEGEo/eSH+wejv"
284 "F39aln vtp61s++RWvTeBpy6tmjtjqMLUrGDrb7el5Kicv1tsxffr92AAAAAAAAAAAA2NE"297 "F39aln vtp61s++RWvTeBpy6tmjtjqMLUrGDrb7el5Kicv1tsxffr92AAAAAAAAAAAA2NE"
285 "Db6L1AQAoAAAK")298 "Db6L1AQAoAAAK"
299).encode("ascii")
286300
287301
288PKGDEB_OR_RELATIONS = (302PKGDEB_OR_RELATIONS = (
@@ -299,7 +313,8 @@
299 "6ICAgICAxMzE3ODg4ODY5ICAwICAgICAwICAgICAxMDA2NDQgIDEwNyAgICAgICBgCh+L "313 "6ICAgICAxMzE3ODg4ODY5ICAwICAgICAwICAgICAxMDA2NDQgIDEwNyAgICAgICBgCh+L "
300 "CAAAAAAAAgPtyrsRwjAURNFXiioAfZBcjwkJzIyB/oGAGSIc4eic5Aa7h2P8XX6Zen+3TD"314 "CAAAAAAAAgPtyrsRwjAURNFXiioAfZBcjwkJzIyB/oGAGSIc4eic5Aa7h2P8XX6Zen+3TD"
301 "1/9yNK"315 "1/9yNK"
302 "GadWR2ltRC651hGpxw4et/u8phTny3Vdfvy2dgAAAAAAAAAAANjRE6Lr2rEAKAAACg==")316 "GadWR2ltRC651hGpxw4et/u8phTny3Vdfvy2dgAAAAAAAAAAANjRE6Lr2rEAKAAACg=="
317).encode("ascii")
303318
304319
305HASH1 = base64.decodestring(b"/ezv4AefpJJ8DuYFSq4RiEHJYP4=")320HASH1 = base64.decodestring(b"/ezv4AefpJJ8DuYFSq4RiEHJYP4=")
306321
=== modified file 'landscape/package/tests/test_reporter.py'
--- landscape/package/tests/test_reporter.py 2017-03-20 09:43:08 +0000
+++ landscape/package/tests/test_reporter.py 2017-03-21 17:01:15 +0000
@@ -27,7 +27,6 @@
27 LandscapeTest, BrokerServiceHelper, EnvironSaverHelper)27 LandscapeTest, BrokerServiceHelper, EnvironSaverHelper)
28from landscape.reactor import FakeReactor28from landscape.reactor import FakeReactor
2929
30from landscape.compat import convert_buffer_to_string
3130
32SAMPLE_LSB_RELEASE = "DISTRIB_CODENAME=codename\n"31SAMPLE_LSB_RELEASE = "DISTRIB_CODENAME=codename\n"
3332
@@ -96,21 +95,21 @@
96 os.chmod(self.reporter.apt_update_filename, 0o755)95 os.chmod(self.reporter.apt_update_filename, 0o755)
9796
98 def test_set_package_ids_with_all_known(self):97 def test_set_package_ids_with_all_known(self):
99 self.store.add_hash_id_request(["hash1", "hash2"])98 self.store.add_hash_id_request([b"hash1", b"hash2"])
100 request2 = self.store.add_hash_id_request(["hash3", "hash4"])99 request2 = self.store.add_hash_id_request([b"hash3", b"hash4"])
101 self.store.add_hash_id_request(["hash5", "hash6"])100 self.store.add_hash_id_request([b"hash5", b"hash6"])
102101
103 self.store.add_task("reporter",102 self.store.add_task("reporter",
104 {"type": "package-ids", "ids": [123, 456],103 {"type": "package-ids", "ids": [123, 456],
105 "request-id": request2.id})104 "request-id": request2.id})
106105
107 def got_result(result):106 def got_result(result):
108 self.assertEqual(self.store.get_hash_id("hash1"), None)107 self.assertEqual(self.store.get_hash_id(b"hash1"), None)
109 self.assertEqual(self.store.get_hash_id("hash2"), None)108 self.assertEqual(self.store.get_hash_id(b"hash2"), None)
110 self.assertEqual(self.store.get_hash_id("hash3"), 123)109 self.assertEqual(self.store.get_hash_id(b"hash3"), 123)
111 self.assertEqual(self.store.get_hash_id("hash4"), 456)110 self.assertEqual(self.store.get_hash_id(b"hash4"), 456)
112 self.assertEqual(self.store.get_hash_id("hash5"), None)111 self.assertEqual(self.store.get_hash_id(b"hash5"), None)
113 self.assertEqual(self.store.get_hash_id("hash6"), None)112 self.assertEqual(self.store.get_hash_id(b"hash6"), None)
114113
115 deferred = self.reporter.handle_tasks()114 deferred = self.reporter.handle_tasks()
116 return deferred.addCallback(got_result)115 return deferred.addCallback(got_result)
@@ -129,7 +128,7 @@
129128
130 message_store.set_accepted_types(["add-packages"])129 message_store.set_accepted_types(["add-packages"])
131130
132 request1 = self.store.add_hash_id_request(["foo", HASH1, "bar"])131 request1 = self.store.add_hash_id_request([b"foo", HASH1, b"bar"])
133132
134 self.store.add_task("reporter",133 self.store.add_task("reporter",
135 {"type": "package-ids",134 {"type": "package-ids",
@@ -184,7 +183,7 @@
184183
185 message_store.set_accepted_types(["add-packages"])184 message_store.set_accepted_types(["add-packages"])
186185
187 request1 = self.store.add_hash_id_request(["foo", HASH1, "bar"])186 request1 = self.store.add_hash_id_request([b"foo", HASH1, b"bar"])
188187
189 self.store.add_task("reporter",188 self.store.add_task("reporter",
190 {"type": "package-ids",189 {"type": "package-ids",
@@ -238,7 +237,7 @@
238 deferred = Deferred()237 deferred = Deferred()
239 deferred.errback(Boom())238 deferred.errback(Boom())
240239
241 request_id = self.store.add_hash_id_request(["foo", HASH1, "bar"]).id240 request_id = self.store.add_hash_id_request([b"foo", HASH1, b"bar"]).id
242241
243 self.store.add_task("reporter", {"type": "package-ids",242 self.store.add_task("reporter", {"type": "package-ids",
244 "ids": [123, None, 456],243 "ids": [123, None, 456],
@@ -259,7 +258,7 @@
259 return result.addCallback(got_result, send_mock)258 return result.addCallback(got_result, send_mock)
260259
261 def test_set_package_ids_removes_request_id_when_done(self):260 def test_set_package_ids_removes_request_id_when_done(self):
262 request = self.store.add_hash_id_request(["hash1"])261 request = self.store.add_hash_id_request([b"hash1"])
263 self.store.add_task("reporter", {"type": "package-ids", "ids": [123],262 self.store.add_task("reporter", {"type": "package-ids", "ids": [123],
264 "request-id": request.id})263 "request-id": request.id})
265264
@@ -562,7 +561,7 @@
562 self.assertTrue(self.reporter._apt_sources_have_changed())561 self.assertTrue(self.reporter._apt_sources_have_changed())
563562
564 def test_remove_expired_hash_id_request(self):563 def test_remove_expired_hash_id_request(self):
565 request = self.store.add_hash_id_request(["hash1"])564 request = self.store.add_hash_id_request([b"hash1"])
566 request.message_id = 9999565 request.message_id = 9999
567566
568 request.timestamp -= HASH_ID_REQUEST_TIMEOUT567 request.timestamp -= HASH_ID_REQUEST_TIMEOUT
@@ -575,7 +574,7 @@
575 return result.addCallback(got_result)574 return result.addCallback(got_result)
576575
577 def test_remove_expired_hash_id_request_wont_remove_before_timeout(self):576 def test_remove_expired_hash_id_request_wont_remove_before_timeout(self):
578 request1 = self.store.add_hash_id_request(["hash1"])577 request1 = self.store.add_hash_id_request([b"hash1"])
579 request1.message_id = 9999578 request1.message_id = 9999
580 request1.timestamp -= HASH_ID_REQUEST_TIMEOUT / 2579 request1.timestamp -= HASH_ID_REQUEST_TIMEOUT / 2
581580
@@ -592,7 +591,7 @@
592 return result.addCallback(got_result)591 return result.addCallback(got_result)
593592
594 def test_remove_expired_hash_id_request_updates_timestamps(self):593 def test_remove_expired_hash_id_request_updates_timestamps(self):
595 request = self.store.add_hash_id_request(["hash1"])594 request = self.store.add_hash_id_request([b"hash1"])
596 message_store = self.broker_service.message_store595 message_store = self.broker_service.message_store
597 message_id = message_store.add({"type": "add-packages",596 message_id = message_store.add({"type": "add-packages",
598 "packages": [],597 "packages": [],
@@ -607,7 +606,7 @@
607 return result.addCallback(got_result)606 return result.addCallback(got_result)
608607
609 def test_remove_expired_hash_id_request_removes_when_no_message_id(self):608 def test_remove_expired_hash_id_request_removes_when_no_message_id(self):
610 request = self.store.add_hash_id_request(["hash1"])609 request = self.store.add_hash_id_request([b"hash1"])
611610
612 def got_result(result):611 def got_result(result):
613 self.assertRaises(UnknownHashIDRequest,612 self.assertRaises(UnknownHashIDRequest,
@@ -1305,9 +1304,9 @@
1305 spawn_patcher = mock.patch.object(reporter, "spawn_process",1304 spawn_patcher = mock.patch.object(reporter, "spawn_process",
1306 side_effect=[1305 side_effect=[
1307 # Simulate series of failures to acquire the apt lock.1306 # Simulate series of failures to acquire the apt lock.
1308 succeed(('', '', 100)),1307 succeed((b'', b'', 100)),
1309 succeed(('', '', 100)),1308 succeed((b'', b'', 100)),
1310 succeed(('', '', 100))])1309 succeed((b'', b'', 100))])
1311 spawn_patcher.start()1310 spawn_patcher.start()
1312 self.addCleanup(spawn_patcher.stop)1311 self.addCleanup(spawn_patcher.stop)
13131312
@@ -1343,8 +1342,8 @@
1343 spawn_patcher = mock.patch.object(reporter, "spawn_process",1342 spawn_patcher = mock.patch.object(reporter, "spawn_process",
1344 side_effect=[1343 side_effect=[
1345 # Simulate a failed apt lock grab then a successful one.1344 # Simulate a failed apt lock grab then a successful one.
1346 succeed(('', '', 100)),1345 succeed((b'', b'', 100)),
1347 succeed(('output', 'error', 0))])1346 succeed((b'output', b'error', 0))])
1348 spawn_patcher.start()1347 spawn_patcher.start()
1349 self.addCleanup(spawn_patcher.stop)1348 self.addCleanup(spawn_patcher.stop)
13501349
@@ -1628,7 +1627,7 @@
1628 return deferred1627 return deferred
16291628
1630 @mock.patch("landscape.package.reporter.spawn_process",1629 @mock.patch("landscape.package.reporter.spawn_process",
1631 return_value=succeed(("", "", 0)))1630 return_value=succeed((b"", b"", 0)))
1632 def test_run_apt_update_honors_http_proxy(self, mock_spawn_process):1631 def test_run_apt_update_honors_http_proxy(self, mock_spawn_process):
1633 """1632 """
1634 The PackageReporter.run_apt_update method honors the http_proxy1633 The PackageReporter.run_apt_update method honors the http_proxy
@@ -1647,7 +1646,7 @@
1647 env={"http_proxy": "http://proxy_server:8080"})1646 env={"http_proxy": "http://proxy_server:8080"})
16481647
1649 @mock.patch("landscape.package.reporter.spawn_process",1648 @mock.patch("landscape.package.reporter.spawn_process",
1650 return_value=succeed(("", "", 0)))1649 return_value=succeed((b"", b"", 0)))
1651 def test_run_apt_update_honors_https_proxy(self, mock_spawn_process):1650 def test_run_apt_update_honors_https_proxy(self, mock_spawn_process):
1652 """1651 """
1653 The PackageReporter.run_apt_update method honors the https_proxy1652 The PackageReporter.run_apt_update method honors the https_proxy
@@ -1869,8 +1868,7 @@
1869 "SELECT id, data FROM message").fetchall())1868 "SELECT id, data FROM message").fetchall())
1870 self.assertEqual(1, len(stored))1869 self.assertEqual(1, len(stored))
1871 self.assertEqual(1, stored[0][0])1870 self.assertEqual(1, stored[0][0])
1872 self.assertEqual(message,1871 self.assertEqual(message, bpickle.loads(bytes(stored[0][1])))
1873 bpickle.loads(convert_buffer_to_string(stored[0][1])))
1874 result.addCallback(callback)1872 result.addCallback(callback)
1875 result.chainDeferred(deferred)1873 result.chainDeferred(deferred)
18761874
18771875
=== modified file 'landscape/package/tests/test_store.py'
--- landscape/package/tests/test_store.py 2017-01-09 14:29:54 +0000
+++ landscape/package/tests/test_store.py 2017-03-21 17:01:15 +0000
@@ -19,12 +19,12 @@
19 self.store2 = HashIdStore(self.filename)19 self.store2 = HashIdStore(self.filename)
2020
21 def test_set_and_get_hash_id(self):21 def test_set_and_get_hash_id(self):
22 self.store1.set_hash_ids({"ha\x00sh1": 123, "ha\x00sh2": 456})22 self.store1.set_hash_ids({b"ha\x00sh1": 123, b"ha\x00sh2": 456})
23 self.assertEqual(self.store1.get_hash_id("ha\x00sh1"), 123)23 self.assertEqual(self.store1.get_hash_id(b"ha\x00sh1"), 123)
24 self.assertEqual(self.store1.get_hash_id("ha\x00sh2"), 456)24 self.assertEqual(self.store1.get_hash_id(b"ha\x00sh2"), 456)
2525
26 def test_get_hash_ids(self):26 def test_get_hash_ids(self):
27 hash_ids = {"hash1": 123, "hash2": 456}27 hash_ids = {b"hash1": 123, b"hash2": 456}
28 self.store1.set_hash_ids(hash_ids)28 self.store1.set_hash_ids(hash_ids)
29 self.assertEqual(self.store1.get_hash_ids(), hash_ids)29 self.assertEqual(self.store1.get_hash_ids(), hash_ids)
3030
@@ -80,33 +80,33 @@
80 self.assertEqual([None], rollbacks)80 self.assertEqual([None], rollbacks)
8181
82 def test_get_id_hash(self):82 def test_get_id_hash(self):
83 self.store1.set_hash_ids({"hash1": 123, "hash2": 456})83 self.store1.set_hash_ids({b"hash1": 123, b"hash2": 456})
84 self.assertEqual(self.store2.get_id_hash(123), "hash1")84 self.assertEqual(self.store2.get_id_hash(123), b"hash1")
85 self.assertEqual(self.store2.get_id_hash(456), "hash2")85 self.assertEqual(self.store2.get_id_hash(456), b"hash2")
8686
87 def test_clear_hash_ids(self):87 def test_clear_hash_ids(self):
88 self.store1.set_hash_ids({"ha\x00sh1": 123, "ha\x00sh2": 456})88 self.store1.set_hash_ids({b"ha\x00sh1": 123, b"ha\x00sh2": 456})
89 self.store1.clear_hash_ids()89 self.store1.clear_hash_ids()
90 self.assertEqual(self.store2.get_hash_id("ha\x00sh1"), None)90 self.assertEqual(self.store2.get_hash_id(b"ha\x00sh1"), None)
91 self.assertEqual(self.store2.get_hash_id("ha\x00sh2"), None)91 self.assertEqual(self.store2.get_hash_id(b"ha\x00sh2"), None)
9292
93 def test_get_unexistent_hash(self):93 def test_get_unexistent_hash(self):
94 self.assertEqual(self.store1.get_hash_id("hash1"), None)94 self.assertEqual(self.store1.get_hash_id(b"hash1"), None)
9595
96 def test_get_unexistent_id(self):96 def test_get_unexistent_id(self):
97 self.assertEqual(self.store1.get_id_hash(123), None)97 self.assertEqual(self.store1.get_id_hash(123), None)
9898
99 def test_overwrite_id_hash(self):99 def test_overwrite_id_hash(self):
100 self.store1.set_hash_ids({"hash1": 123})100 self.store1.set_hash_ids({b"hash1": 123})
101 self.store2.set_hash_ids({"hash2": 123})101 self.store2.set_hash_ids({b"hash2": 123})
102 self.assertEqual(self.store1.get_hash_id("hash1"), None)102 self.assertEqual(self.store1.get_hash_id(b"hash1"), None)
103 self.assertEqual(self.store1.get_hash_id("hash2"), 123)103 self.assertEqual(self.store1.get_hash_id(b"hash2"), 123)
104104
105 def test_overwrite_hash_id(self):105 def test_overwrite_hash_id(self):
106 self.store1.set_hash_ids({"hash1": 123})106 self.store1.set_hash_ids({b"hash1": 123})
107 self.store2.set_hash_ids({"hash1": 456})107 self.store2.set_hash_ids({b"hash1": 456})
108 self.assertEqual(self.store1.get_id_hash(123), None)108 self.assertEqual(self.store1.get_id_hash(123), None)
109 self.assertEqual(self.store1.get_id_hash(456), "hash1")109 self.assertEqual(self.store1.get_id_hash(456), b"hash1")
110110
111 def test_check_sanity(self):111 def test_check_sanity(self):
112112
@@ -184,22 +184,22 @@
184184
185 def test_get_hash_id_using_hash_id_dbs(self):185 def test_get_hash_id_using_hash_id_dbs(self):
186 # Without hash=>id dbs186 # Without hash=>id dbs
187 self.assertEqual(self.store1.get_hash_id("hash1"), None)187 self.assertEqual(self.store1.get_hash_id(b"hash1"), None)
188 self.assertEqual(self.store1.get_hash_id("hash2"), None)188 self.assertEqual(self.store1.get_hash_id(b"hash2"), None)
189189
190 # This hash=>id will be overriden190 # This hash=>id will be overriden
191 self.store1.set_hash_ids({"hash1": 1})191 self.store1.set_hash_ids({b"hash1": 1})
192192
193 # Add a couple of hash=>id dbs193 # Add a couple of hash=>id dbs
194 self.store1.add_hash_id_db(self.hash_id_db_factory({"hash1": 2,194 self.store1.add_hash_id_db(self.hash_id_db_factory({b"hash1": 2,
195 "hash2": 3}))195 b"hash2": 3}))
196 self.store1.add_hash_id_db(self.hash_id_db_factory({"hash2": 4,196 self.store1.add_hash_id_db(self.hash_id_db_factory({b"hash2": 4,
197 "ha\x00sh1": 5}))197 b"ha\x00sh1": 5}))
198198
199 # Check look-up priorities and binary hashes199 # Check look-up priorities and binary hashes
200 self.assertEqual(self.store1.get_hash_id("hash1"), 2)200 self.assertEqual(self.store1.get_hash_id(b"hash1"), 2)
201 self.assertEqual(self.store1.get_hash_id("hash2"), 3)201 self.assertEqual(self.store1.get_hash_id(b"hash2"), 3)
202 self.assertEqual(self.store1.get_hash_id("ha\x00sh1"), 5)202 self.assertEqual(self.store1.get_hash_id(b"ha\x00sh1"), 5)
203203
204 def test_get_id_hash_using_hash_id_db(self):204 def test_get_id_hash_using_hash_id_db(self):
205 """205 """
@@ -207,13 +207,13 @@
207 to query them first, falling back to the regular db in case207 to query them first, falling back to the regular db in case
208 the desired mapping is not found.208 the desired mapping is not found.
209 """209 """
210 self.store1.add_hash_id_db(self.hash_id_db_factory({"hash1": 123}))210 self.store1.add_hash_id_db(self.hash_id_db_factory({b"hash1": 123}))
211 self.store1.add_hash_id_db(self.hash_id_db_factory({"hash1": 999,211 self.store1.add_hash_id_db(self.hash_id_db_factory({b"hash1": 999,
212 "hash2": 456}))212 b"hash2": 456}))
213 self.store1.set_hash_ids({"hash3": 789})213 self.store1.set_hash_ids({b"hash3": 789})
214 self.assertEqual(self.store1.get_id_hash(123), "hash1")214 self.assertEqual(self.store1.get_id_hash(123), b"hash1")
215 self.assertEqual(self.store1.get_id_hash(456), "hash2")215 self.assertEqual(self.store1.get_id_hash(456), b"hash2")
216 self.assertEqual(self.store1.get_id_hash(789), "hash3")216 self.assertEqual(self.store1.get_id_hash(789), b"hash3")
217217
218 def test_add_and_get_available_packages(self):218 def test_add_and_get_available_packages(self):
219 self.store1.add_available([1, 2])219 self.store1.add_available([1, 2])
220220
=== modified file 'landscape/schema.py'
--- landscape/schema.py 2017-03-10 12:40:17 +0000
+++ landscape/schema.py 2017-03-21 17:01:15 +0000
@@ -67,8 +67,8 @@
67class Bytes(object):67class Bytes(object):
68 """A binary string."""68 """A binary string."""
69 def coerce(self, value):69 def coerce(self, value):
70 if not isinstance(value, str):70 if not isinstance(value, bytes):
71 raise InvalidError("%r isn't a str" % (value,))71 raise InvalidError("%r isn't a bytestring" % (value,))
72 return value72 return value
7373
7474
7575
=== modified file 'py3_ready_tests'
--- py3_ready_tests 2017-03-15 08:40:11 +0000
+++ py3_ready_tests 2017-03-21 17:01:15 +0000
@@ -1,2 +1,5 @@
1landscape.lib.tests1landscape.lib.tests
2landscape.sysinfo.tests2landscape.sysinfo.tests
3landscape.package.tests.test_store
4landscape.package.tests.test_reporter
5

Subscribers

People subscribed via source and target branches

to all changes: