Merge lp:~gz/bzr/2.5_help_translation_unicode_error_930919 into lp:bzr/2.5

[Martin Packman]

Default help output to ui wrapped stream rather than raw stdout for better encoding handling

Make the display of unicode help via the help command use a default stream that knows how to deal with unicode, rather than stdout directly which may not. The --help switch is not affected as it uses a completely different code path with a different set of issues.

There are still some niggles unaddressed. Callers could still pass in a dumb stream that it's not safe to write arbitrary unicode to. Also, as gettext is a little more forgiving than system locales, it's possible to get things translated that then bzr has no idea how to encode. In this case, the user does get a nice clear error message and non-ascii characters replaced with question marks, so it should be clear enough that they need to install the relevant locale. Possibly the fallback could be smarter, but moving that choice into bzrlib.ui seems like an improvement regardless.

[IWATA Hidetaka (hid-iwata)]

First, My report is a bit wrong.
I said that `bzr command-name --help` works, but it also can fail depending on the target command.
e.g. If LANG=ja, `bzr commit --help` works, but `bzr diff --help` does not work.

I’ve tested this patch under the environment of Windows + Python 2.6.6, and confirmed this fixes
problem about `bzr help command-name`.

Thank you.

[Alexander Belchenko (bialix)]

IWATA Hidetaka пишет:
> First, My report is a bit wrong.
> I said that `bzr command-name --help` works, but it also can fail depending on the target command.
> e.g. If LANG=ja, `bzr commit --help` works, but `bzr diff --help` does not work.

I think I understand why `bzr diff --help` does not work.

We have strict encoding for diff output, and most likely --help uses
the same output stream, so we get problem for help and unicode.

That should be fixed globally by replacing the stream for *all*
commands if there is --help or --usage options given. (Don't forget
about --usage ;-)

Maybe it's worth to pre-process command-line options before initiate
the command object, or maybe in setup_outf_stream (or how it named
today?) method of the command object.

--
All the dude wanted was his rug back

[Martin Packman (gz)]

I've filed bug 943260 for the issue with writing to outf in commands, as it's a wider problem than just help switches and wants a different fix.

Lots of commands to output like that without being very careful about it, for instance:

C:\bzr\bzr\2.5>%PY27% bzr init B:\testbranch
standalone treeを作成しました。(フォーマット:2a)

C:\bzr\bzr\2.5>python -c "file(u'B:/testbranch/\xe9','w').close()"

C:\bzr\bzr\2.5>%PY27% bzr unknowns -d B:\testbranch
bzr: ERROR: exceptions.UnicodeEncodeError: 'cp932' codec can't encode character u'\xe9' in position 1: illegal multibyte sequence

[Martin Packman (gz)]

sent to pqm by email

=== modified file 'bzrlib/help.py'
--- bzrlib/help.py	2011-12-18 15:28:38 +0000
+++ bzrlib/help.py	2012-02-27 12:26:18 +0000
@@ -31,6 +31,7 @@
     help_topics,
     osutils,
     plugin,
+    ui,
     utextwrap,
     )
 
@@ -38,7 +39,7 @@
 def help(topic=None, outfile=None):
     """Write the help for the specific topic to outfile"""
     if outfile is None:
-        outfile = sys.stdout
+        outfile = ui.ui_factory.make_output_stream()
 
     indices = HelpIndices()
 
@@ -63,7 +64,7 @@
 def help_commands(outfile=None):
     """List all commands"""
     if outfile is None:
-        outfile = sys.stdout
+        outfile = ui.ui_factory.make_output_stream()
     outfile.write(_help_commands_to_text('commands'))
 
 

=== modified file 'bzrlib/tests/blackbox/test_help.py'
--- bzrlib/tests/blackbox/test_help.py	2011-01-12 00:58:05 +0000
+++ bzrlib/tests/blackbox/test_help.py	2012-02-27 12:26:18 +0000
@@ -19,12 +19,16 @@
 """
 
 
-import bzrlib
-from bzrlib import config
-from bzrlib.tests import TestCaseWithTransport
-
-
-class TestHelp(TestCaseWithTransport):
+from bzrlib import (
+    config,
+    i18n,
+    tests,
+    )
+
+from bzrlib.tests.test_i18n import ZzzTranslations
+
+
+class TestHelp(tests.TestCaseWithTransport):
 
     def test_help_basic(self):
         for cmd in ['--help', 'help', '-h', '-?']:
@@ -175,3 +179,27 @@
 
         self.assertEqual("'bzr c' is an alias for 'bzr cat'.\n",
                          self.run_bzr('help c')[0])
+
+
+class TestTranslatedHelp(tests.TestCaseWithTransport):
+    """Tests for display of translated help topics"""
+
+    def setUp(self):
+        super(TestTranslatedHelp, self).setUp()
+        self.overrideAttr(i18n, '_translations', ZzzTranslations())
+
+    def test_help_command_utf8(self):
+        out, err = self.run_bzr(["help", "push"], encoding="utf-8")
+        self.assertContainsRe(out, "zz\xc3\xa5{{:See also:")
+
+    def test_help_switch_utf8(self):
+        out, err = self.run_bzr(["push", "--help"], encoding="utf-8")
+        self.assertContainsRe(out, "zz\xc3\xa5{{:See also:")
+
+    def test_help_command_ascii(self):
+        out, err = self.run_bzr(["help", "push"], encoding="ascii")
+        self.assertContainsRe(out, "zz\\?{{:See also:")
+
+    def test_help_switch_ascii(self):
+        out, err = self.run_bzr(["push", "--help"], encoding="ascii")
+        self.assertContainsRe(out, "zz\\?{{:See also:")