Merge lp:~laney/ubuntu-archive-tools/cm-utf8 into lp:ubuntu-archive-tools

[Iain Lane]

Fix component-mismatches crashing due to the usual python unicode weirdness

[Andy Whitcroft (apw)]

Can I just say yeuch. But it looks to do what is claimed.

review: Approve

[lp:~laney/ubuntu-archive-tools/cm-utf8]

1225. By Andy Whitcroft on 2019-07-15

Merge: lp:~laney/ubuntu-archive-tools/cm-utf8

Fixes component-missmatches in the face of utf-8 user 'names' in launchpad.

Signed-off-by: Andy Whitcroft <email address hidden>

=== modified file 'component-mismatches'
--- component-mismatches	2018-03-09 04:11:21 +0000
+++ component-mismatches	2019-07-15 10:05:25 +0000
@@ -889,6 +889,23 @@
     else:
         options.html_output = None
 
+    if sys.version >= '3':
+        # Force encoding to UTF-8 even in non-UTF-8 locales.
+        import io
+        sys.stdout = io.TextIOWrapper(
+            sys.stdout.detach(), encoding="UTF-8", line_buffering=True)
+    else:
+        # Avoid having to do .encode('UTF-8') everywhere. This is a pain; I wish
+        # Python supported something like "sys.stdout.encoding = 'UTF-8'".
+        def fix_stdout():
+            import codecs
+            sys.stdout = codecs.EncodedFile(sys.stdout, 'UTF-8')
+            def null_decode(input, errors='strict'):
+                return input, len(input)
+            sys.stdout.decode = null_decode
+
+        fix_stdout()
+
     options.time = time.time()
     options.timestamp = time.strftime(
         '%a %b %e %H:%M:%S %Z %Y', time.gmtime(options.time))