Merge lp:~mikemc/ubuntu-sso-client/cross-platform-translations-1074116 into lp:ubuntu-sso-client

[Mike McCracken]

- Add util func returning translation function for platform/pyversion/frozen status. (LP: #1074116)

- Add util func returning translation function for platform/pyversion/frozen status. (LP: #1074116)

Consolidates checking for python version and giving gettext platform-specific search paths for translation files on frozen mac/win.

Also adds fallback to built translation files in source tree for testing translation support from source.

Windows frozen support is TBD.

Testing IRL from source:
- 'setup.py build' should've been run (as part of tests, or just run it now) to get compiled translation files.
- set language:
 -- on OSX, reorder the list in system preferences > Language & Text
 -- on linux, just set LANG='es' or whatever in the shell
- run sso-login-qt
- observe translated strings in UI.

Testing IRL mac frozen requires a currently unlanded setup-mac branch.
I will be pushing that soon, and will update this MP when I do.

[Michał Karnicki (karni)]

Looks good.

review: Approve

[Brian Curtin (brian.curtin)]

134 + if fallback:
135 + g_func = translation.get_gettext(TEST_DOMAIN, fallback)
136 + else:
137 + g_func = translation.get_gettext(TEST_DOMAIN)

There's nothing functionally wrong with this, but it's a bit repetitive. Would something like the below part work?

args = [TEST_DOMAIN]
args.append(fallback) if fallback else None
gfunc = translation.get_gettext(*args)

** I don't know if that's readable to everyone so I'm not marking this "Needs Fixing". I prefer to avoid if/else blocks that only choose one line or a very similar other line.

review: Approve

[Ubuntu One Auto Pilot (otto-pilot)]

The attempt to merge lp:~mikemc/ubuntu-sso-client/cross-platform-translations-1074116 into lp:ubuntu-sso-client failed. Below is the output from the failed tests.

*** Running QT test suite for ubuntu_sso ***
running build
Compiled data/qt/ssl_dialog.ui into ubuntu_sso/qt/ui/ssl_dialog_ui.py
compiled data/qt/resources.qrc into ubuntu_sso/qt/ui/resources_rc.py
Compiled data/qt/loadingoverlay.ui into ubuntu_sso/qt/ui/loadingoverlay_ui.py
Compiled data/qt/proxy_credentials_dialog.ui into ubuntu_sso/qt/ui/proxy_credentials_dialog_ui.py
Compiled data/qt/reset_password.ui into ubuntu_sso/qt/ui/reset_password_ui.py
Compiled data/qt/network_detection.ui into ubuntu_sso/qt/ui/network_detection_ui.py
Compiled data/qt/email_verification.ui into ubuntu_sso/qt/ui/email_verification_ui.py
Compiled data/qt/setup_account.ui into ubuntu_sso/qt/ui/setup_account_ui.py
Compiled data/qt/forgotten_password.ui into ubuntu_sso/qt/ui/forgotten_password_ui.py
Compiled data/qt/current_user_sign_in.ui into ubuntu_sso/qt/ui/current_user_sign_in_ui.py
Compiled data/qt/success_message.ui into ubuntu_sso/qt/ui/success_message_ui.py
Compiled data/qt/error_message.ui into ubuntu_sso/qt/ui/error_message_ui.py
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/ubuntu_sso
copying ubuntu_sso/account.py -> build/lib.linux-x86_64-2.7/ubuntu_sso
copying ubuntu_sso/logger.py -> build/lib.linux-x86_64-2.7/ubuntu_sso
copying ubuntu_sso/__init__.py -> build/lib.linux-x86_64-2.7/ubuntu_sso
copying ubuntu_sso/credentials.py -> build/lib.linux-x86_64-2.7/ubuntu_sso
creating build/lib.linux-x86_64-2.7/ubuntu_sso/tests
copying ubuntu_sso/tests/linux.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/tests
copying ubuntu_sso/tests/__init__.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/tests
copying ubuntu_sso/tests/test_credentials.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/tests
copying ubuntu_sso/tests/test_account.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/tests
creating build/lib.linux-x86_64-2.7/ubuntu_sso/keyring
copying ubuntu_sso/keyring/linux.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/keyring
copying ubuntu_sso/keyring/__init__.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/keyring
copying ubuntu_sso/keyring/pykeyring.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/keyring
creating build/lib.linux-x86_64-2.7/ubuntu_sso/keyring/tests
copying ubuntu_sso/keyring/tests/__init__.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/keyring/tests
copying ubuntu_sso/keyring/tests/test_pykeyring.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/keyring/tests
copying ubuntu_sso/keyring/tests/test_linux.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/keyring/tests
copying ubuntu_sso/keyring/tests/test_common.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/keyring/tests
creating build/lib.linux-x86_64-2.7/ubuntu_sso/main
copying ubuntu_sso/main/glib.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/main
copying ubuntu_sso/main/perspective_broker.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/main
copying ubuntu_sso/main/linux.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/main
copying ubuntu_sso/main/windows.py -> build/lib.linux-x86_64-2.7/ubuntu_sso/main
copying ubuntu_sso/main/qt.py -> b...

=== added file 'ubuntu_sso/utils/tests/test_translation.py'
--- ubuntu_sso/utils/tests/test_translation.py	1970-01-01 00:00:00 +0000
+++ ubuntu_sso/utils/tests/test_translation.py	2012-12-12 18:25:24 +0000
@@ -0,0 +1,221 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# In addition, as a special exception, the copyright holders give
+# permission to link the code of portions of this program with the
+# OpenSSL library under certain conditions as described in each
+# individual source file, and distribute linked combinations
+# including the two.
+# You must obey the GNU General Public License in all respects
+# for all of the code used other than OpenSSL.  If you modify
+# file(s) with this exception, you may extend this exception to your
+# version of the file(s), but you are not obligated to do so.  If you
+# do not wish to do so, delete this exception statement from your
+# version.  If you delete this exception statement from all source
+# files in the program, then also delete it here.
+"""Test the platform-specific translation functions."""
+
+import os
+import sys
+
+from twisted.internet import defer
+from ubuntuone.devtools.testcases import TestCase, skipIfNotOS
+from ubuntu_sso.utils import translation
+
+TEST_DOMAIN = 'test-domain'
+TEST_LANG_DEFAULTS_EN_FIRST = ['en', 'pt-PT']
+TEST_LANG_DEFAULTS = ['es', 'en', 'pt-PT']
+TEST_FALLBACK_PATH = 'test-path/to/mofiles'
+TEST_FROZEN_PATH = 'frozen-test-path/to/mofiles'
+
+
+class MockGettextTranslations(object):
+    """Mock translations to test properties"""
+    ugettext = 'ugettext'
+    gettext = 'gettext'
+
+
+class MockNSUserDefaults(object):
+    """Mock defaults for _get_languages on darwin."""
+
+    def standardUserDefaults(self):
+        return {'AppleLanguages': TEST_LANG_DEFAULTS}
+
+
+class TranslationsTestCase(TestCase):
+    """Test getting the right gettext translations."""
+
+    @defer.inlineCallbacks
+    def setUp(self):
+        yield super(TranslationsTestCase, self).setUp()
+        self._called = []
+
+    def _set_called(self, *args, **kwargs):
+        """call recorder"""
+        self._called.append((args, kwargs))
+        return MockGettextTranslations()
+
+    def test_import(self):
+        """Test whether import translation defines _ as a builtin."""
+        import ubuntu_sso.utils.translation
+        assert ubuntu_sso.utils.translation
+        self.assertFalse('_' in __builtins__)
+
+    @skipIfNotOS('darwin', 'Test requires pyobjc-Cocoa')
+    def test_get_languages_darwin(self):
+        """Test getting the user's list of preferred languages."""
+        import Cocoa
+        assert Cocoa
+        self.patch(Cocoa, 'NSUserDefaults', MockNSUserDefaults())
+        langs = translation._get_languages()
+        self.assertEqual(langs, TEST_LANG_DEFAULTS)
+
+    def test_get_languages_linux(self):
+        """Test that we will use gettext defaults on linux."""
+        self.patch(sys, 'platform', 'linux2')
+        langs = translation._get_languages()
+        self.assertEqual(langs, None)
+
+    def test_get_translations_data_path_darwin_frozen(self):
+        """Test getting the location of the compiled translation files."""
+        self.patch(sys, 'platform', 'darwin')
+        sys.frozen = 'yes'
+        self.addCleanup(delattr, sys, 'frozen')
+        self.patch(translation, '__file__',
+                   os.path.join('path', 'to', 'Main.app', 'ignore', 'me'))
+
+        path = translation._get_translations_data_path()
+
+        self.assertEqual(path, os.path.join('path', 'to', 'Main.app',
+                                            'Contents', 'Resources',
+                                            'translations'))
+
+    def test_get_translations_data_path_darwin_unfrozen_nofallback(self):
+        """Test that we use gettext defaults on darwin when not frozen."""
+        self.patch(sys, 'platform', 'darwin')
+        path = translation._get_translations_data_path()
+        self.assertEqual(path, None)
+
+    def test_get_translations_data_path_darwin_unfrozen_fallback(self):
+        """Test that we use fallback on darwin when not frozen."""
+        self.patch(sys, 'platform', 'darwin')
+        expected = "a-test-path"
+        path = translation._get_translations_data_path(expected)
+        self.assertEqual(path, expected)
+
+    def test_get_translations_data_path_linux(self):
+        """Test that we use gettext defaults on linux."""
+        path = translation._get_translations_data_path()
+        self.assertEqual(path, None)
+
+    def _call_get_gettext(self, platform, py_version, fallback=None):
+        """Helper function to patch and call translation.get_gettext."""
+        self.patch(sys, 'platform', platform)
+        self.patch(sys, 'version_info', py_version)
+        self.patch(translation.gettext, 'translation', self._set_called)
+
+        if fallback:
+            g_func = translation.get_gettext(TEST_DOMAIN, fallback)
+        else:
+            g_func = translation.get_gettext(TEST_DOMAIN)
+
+        if py_version == (2,):
+            self.assertEqual(g_func, 'ugettext')
+        else:
+            self.assertEqual(g_func, 'gettext')
+
+    def test_get_gettext_linux_py2(self):
+        """test get_gettext on linux py2"""
+        self._call_get_gettext('linux2', py_version=(2,))
+
+    def test_get_gettext_linux_py3(self):
+        """test get_gettext on linux py3"""
+        self._call_get_gettext('linux2', py_version=(3,))
+
+    def _call_get_gettext_nonlinux(self, frozen, py_version,
+                                   lang_en_first=False):
+        """Helper function for non-linux runs of get_gettext."""
+        if lang_en_first:
+            lang_rv = TEST_LANG_DEFAULTS_EN_FIRST
+        else:
+            lang_rv = TEST_LANG_DEFAULTS
+        self.patch(translation, '_get_languages', lambda: lang_rv)
+        if frozen:
+            expected_path = TEST_FROZEN_PATH
+            sys.frozen = 'yes'
+            self.addCleanup(delattr, sys, 'frozen')
+        else:
+            expected_path = TEST_FALLBACK_PATH
+
+        self.patch(translation, '_get_translations_data_path',
+                   lambda x: expected_path)
+
+        self._call_get_gettext('notlinux', py_version)
+
+        # This tests a special case if 'en' is the first language, we
+        # don't give the path, so we will get the fallback
+        # NullTranslation instance that will use the keys and not look
+        # for en translations, which we do not ship.
+        if lang_en_first:
+            expected_arg = ((TEST_DOMAIN,), {'fallback': True})
+            self.assertEqual(self._called, [expected_arg])
+        else:
+            # Check for lang_rv[:1] (must be a list) because we only
+            # send first language, in order to fall back to 'en' even
+            # though we don't ship an en.mo
+            expected_arg = ((TEST_DOMAIN, expected_path),
+                              {'languages': lang_rv[:1],
+                               'fallback': True})
+
+            self.assertEqual(self._called, [expected_arg])
+
+    def test_get_gettext_nonlinux_frozen_py2(self):
+        """test get_gettext on nonlinux frozen and py2"""
+        self._call_get_gettext_nonlinux(frozen=True, py_version=(2,))
+
+    def test_get_gettext_nonlinux_frozen_py3(self):
+        """test get_gettext on nonlinux frozen and py3"""
+        self._call_get_gettext_nonlinux(frozen=True, py_version=(3,))
+
+    def test_get_gettext_nonlinux_unfrozen_py2(self):
+        """test get_gettext on nonlinux un-frozen and py2"""
+        self._call_get_gettext_nonlinux(frozen=False, py_version=(2,))
+
+    def test_get_gettext_nonlinux_unfrozen_py3(self):
+        """test get_gettext on nonlinux un-frozen and py3"""
+        self._call_get_gettext_nonlinux(frozen=False, py_version=(3,))
+
+    def test_get_gettext_nonlinux_frozen_py2_enfirst(self):
+        """test get_gettext returning nulltranslations when lang[0] = en"""
+        self._call_get_gettext_nonlinux(frozen=True,
+                                        py_version=(2,),
+                                        lang_en_first=True)
+
+    def test_get_gettext_darwin_unfrozen_fallback(self):
+        """test using fallback path from source"""
+        self.patch(translation, '_get_languages', lambda: ['not-en'])
+        self._call_get_gettext('darwin', (2,), TEST_FALLBACK_PATH)
+        self.assertEqual(self._called, [((TEST_DOMAIN, TEST_FALLBACK_PATH),
+                                       {'languages': ['not-en'],
+                                        'fallback': True})])
+
+    def test_get_gettext_win32_unfrozen_fallback(self):
+        """test using fallback path from source"""
+        self.patch(translation, '_get_languages', lambda: ['not-en'])
+        self._call_get_gettext('win32', (2,), TEST_FALLBACK_PATH)
+        self.assertEqual(self._called, [((TEST_DOMAIN, TEST_FALLBACK_PATH),
+                                       {'languages': ['not-en'],
+                                        'fallback': True})])

=== added file 'ubuntu_sso/utils/translation.py'
--- ubuntu_sso/utils/translation.py	1970-01-01 00:00:00 +0000
+++ ubuntu_sso/utils/translation.py	2012-12-12 18:25:24 +0000
@@ -0,0 +1,91 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright 2012 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU General Public License version 3, as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# In addition, as a special exception, the copyright holders give
+# permission to link the code of portions of this program with the
+# OpenSSL library under certain conditions as described in each
+# individual source file, and distribute linked combinations
+# including the two.
+# You must obey the GNU General Public License in all respects
+# for all of the code used other than OpenSSL.  If you modify
+# file(s) with this exception, you may extend this exception to your
+# version of the file(s), but you are not obligated to do so.  If you
+# do not wish to do so, delete this exception statement from your
+# version.  If you delete this exception statement from all source
+# files in the program, then also delete it here.
+"""Platform-specific translation functions."""
+
+import gettext
+import os
+import sys
+
+from ubuntu_sso.logger import setup_logging
+
+logger = setup_logging('ubuntu_sso.utils.translation')
+
+
+def _get_languages():
+    """list of langs ordered by preference, or None for gettext defaults."""
+    if sys.platform == 'darwin':
+        from Cocoa import NSUserDefaults
+        su = NSUserDefaults.standardUserDefaults()
+        return su['AppleLanguages']
+    else:
+        if sys.platform == 'win32':
+            return None
+        return None
+
+
+def _get_translations_data_path(fallback_path=None):
+    """path to compiled translation files, or None for gettext defaults"""
+    if getattr(sys, 'frozen', None) is not None:
+        if sys.platform == 'darwin':
+            main_app_dir = ''.join(__file__.partition('.app')[:-1])
+            path = os.path.join(main_app_dir, 'Contents', 'Resources',
+                                'translations')
+            return path
+        elif sys.platform == 'win32':
+            return None         # TODO
+
+    exists = (os.path.exists(fallback_path)
+              if fallback_path else False)
+    logger.debug("Using fallback translation path %r "
+                 "which does %s exist." %
+                 (fallback_path, "" if exists else "**NOT**"))
+
+    return fallback_path
+
+
+def get_gettext(translation_domain, fallback_path=None):
+    """Get proper gettext translation function for platform and py version."""
+    languages = _get_languages()
+    translations_path = _get_translations_data_path(fallback_path)
+
+    if languages is None or translations_path is None or languages[0] == 'en':
+        logger.debug('Using default gettext translation search paths')
+
+        translation = gettext.translation(translation_domain, fallback=True)
+    else:
+        translation = gettext.translation(translation_domain,
+                                          translations_path,
+                                          languages=languages[:1],
+                                          fallback=True)
+    if isinstance(translation, gettext.NullTranslations):
+        logger.warn('Translations not found, using null translator.')
+    if sys.version_info < (3,):
+        return translation.ugettext
+    else:
+        return translation.gettext

=== modified file 'ubuntu_sso/utils/ui.py'
--- ubuntu_sso/utils/ui.py	2012-09-18 20:45:03 +0000
+++ ubuntu_sso/utils/ui.py	2012-12-12 18:25:24 +0000
@@ -29,17 +29,18 @@
 """Utils to be used by the UI modules."""
 
 import argparse
+import os
 import re
-import gettext
-import sys
 
 from ubuntu_sso.logger import setup_logging
 
-TRANSLATION = gettext.translation('ubuntu-sso-client', fallback=True)
-if sys.version_info < (3,):
-    _ = TRANSLATION.ugettext
-else:
-    _ = TRANSLATION.gettext
+from ubuntu_sso.utils.translation import get_gettext
+
+source_path = os.path.join(os.path.dirname(__file__),
+                           os.path.pardir, os.path.pardir,
+                           'build', 'mo')
+_ = get_gettext('ubuntu-sso-client',
+                fallback_path=os.path.abspath(source_path))
 
 logger = setup_logging('ubuntu_sso.utils.ui')