Merge lp:~malizor/launchpad/fixes-bug-608631 into lp:launchpad

[Nicolas Delvaux]

Users now can use the [nnbsp] tag to input a narrow no-break space in Rosetta. This tag representation is also a workaround for some Browsers (eg. in Rekonq NNBSP is displayed as a zero-width character).

== Summary ==

Bug 608631 describe the fact that there is no easy way to input and to display narrow no-break spaces (U+202F) in Rosetta.
Narrow no-break spaces are mainly used in French before ";:!?»" chars and after "«", but they are also used in some other languages (e.g. for the short form of the Czech dates and others).

Input of NNBSP is a problem for example on MS Windows (users need to mess around in the registry) and on GNU/Linux if your keyboard layout does not provide a shortcut for it.

Display of NNBSP is a problem in some browsers.
All versions of MS Internet Explorer *on Windows XP* display a square instead of the regular character (it works on Vista and Seven, here the bug lays in the OS build-in text renderer). There is also a bug in Qt which means that Qt browsers (Konqueror, Rekonq...) display NNBSP as a 0 width char.
Basically, it displays well on Firefox, GTK based browsers (Epiphany, Midori...) and on IE (on Vista and Seven).

On top of that, it is sometime difficult to tell apart NNBSP from a white space or a no-break space. So, a helper to recognize NNBSP is necessary anyway.

== Proposed fix ==

As discussed with Данило Шеган, it seems that implementing a new tag (as already existing [tab] and [nbsp]) is the best answer to this.

== Pre-implementation notes ==

The obvious tag for NNBSP is [nnbsp], even if it may be too close to [nbsp], we agreed that this is the best solution.
(I talked about it on the ubuntu-l10n-fr mailing-list)

== Implementation details ==

Implementation was quite easy with the existing code for other tags.
There is no really new code, so this should not cause anything to break (that wouldn't have broke without this patch I mean ;-)).

== Test ==

bin/test -cvv -m lp.translations -t browser-helpers.txt

== Demo and Q/A ==

To demo and Q/A this change, do the following:

- Log on as Sample Person (<email address hidden>:test)
- Visit any translation page (e.g. https://translations.launchpad.dev/evolution/trunk/+pots/evolution-2.2-test/es/+translate )
- Input something like "Test[nnbsp]!" and/or it's escaped version "Test\[nnbsp]!" and/or the 'literal' version "Test !"
- Save your translation(s) and see the results (better here with the above example: https://translations.launchpad.dev/evolution/trunk/+pots/evolution-2.2-test/es/+filter?person=name12 )

== Launchpad lint ==

Checking for conflicts and issues in changed files.

Linting changed files:
  lib/lp/translations/browser/browser_helpers.py
  lib/lp/translations/doc/browser-helpers.txt
  lib/lp/translations/interfaces/translations.py

I fixed all warnings in these files (most of them where not related to my code)

[Данило Шеган (danilo)]

The change looks good. I am still very unhappy with the use of [nbthin]. How about we use [nnbsp] (I know why you hate it, but I find it's better than [nbthin], and others I talked to agree).

review: Needs Fixing

[Nicolas Delvaux (malizor)]

I raised the issue on ubuntu-l10n-fr and lp-l10n-fr.
We will see if someone propose something better and/or if I'm the only one that fear confusion between [nbsp] and [nnbsp].

Otherwise, I also prefer [nbthin] because the French for NNBSP is "espace fine insécable" which literally (word for word) translate to "no-break thin space".

[Nicolas Delvaux (malizor)]

The tag is now [nnbsp].
I talked about it on the ubuntu-l10n-fr mailing-list and they feel that there may not be too much confusion between [nnbsp] and [nbsp].
And NNBSP is the acronym for narrow no-break space, so [nnbsp] is simply more logical.

[Gavin Panella (allenap)]

review: Approve

[Данило Шеган (danilo)]

Good to land.

Nicolas, thanks for working on this. Once it lands this MP will be marked as 'Merged' and soon after bug will be marked as 'Fix committed' and tagged with 'qa-needstesting'. That will mean that you can go to "edge" (i.e. just use "translations.edge.launchpad.net" as the domain instead of "translations.launchpad.net" for any regular LP web page you want to try this on) and try this new feature out. Once you can confirm it works, you can tag the bug with 'qa-ok' (and remove the qa-needstesting tag at the same time).

review: Approve

[Gavin Panella (allenap)]

Nicolas, one more thing before I merge this: can you set a commit message?

Please follow the guide at https://dev.launchpad.net/PQMCommitMessages. You only need to enter the descriptive part. Don't worry about the leading parts in square [] braces; they are calculated automatically.

[Nicolas Delvaux (malizor)]

Done.

[Данило Шеган (danilo)]

Unfortunately, due to changes happening with our infrastructure, QA needed to be done on "qastaging" and not on "edge" as I suggested: sorry about the confusion. To speed deployment up, I did it this time around.

[Nicolas Delvaux (malizor)]

Ok, thank you.
I also confirm that it works on qastaging.

=== modified file 'lib/lp/translations/browser/browser_helpers.py'
--- lib/lp/translations/browser/browser_helpers.py	2010-08-20 20:31:18 +0000
+++ lib/lp/translations/browser/browser_helpers.py	2010-10-08 17:24:20 +0000
@@ -31,7 +31,9 @@
     return helpers.text_replaced(text, {'[tab]': '\t',
                                         r'\[tab]': '[tab]',
                                         '[nbsp]': u'\u00a0',
-                                        r'\[nbsp]': '[nbsp]'})
+                                        r'\[nbsp]': '[nbsp]',
+                                        '[nnbsp]': u'\u202f',
+                                        r'\[nnbsp]': '[nnbsp]'})
 
 
 def expand_rosetta_escapes(unicode_text):
@@ -39,7 +41,10 @@
     escapes = {u'\t': TranslationConstants.TAB_CHAR,
                u'[tab]': TranslationConstants.TAB_CHAR_ESCAPED,
                u'\u00a0': TranslationConstants.NO_BREAK_SPACE_CHAR,
-               u'[nbsp]': TranslationConstants.NO_BREAK_SPACE_CHAR_ESCAPED}
+               u'[nbsp]': TranslationConstants.NO_BREAK_SPACE_CHAR_ESCAPED,
+               u'\u202f': TranslationConstants.NARROW_NO_BREAK_SPACE_CHAR,
+               u'[nnbsp]':
+    TranslationConstants.NARROW_NO_BREAK_SPACE_CHAR_ESCAPED}
     return helpers.text_replaced(unicode_text, escapes)
 
 

=== modified file 'lib/lp/translations/doc/browser-helpers.txt'
--- lib/lp/translations/doc/browser-helpers.txt	2009-09-07 07:25:18 +0000
+++ lib/lp/translations/doc/browser-helpers.txt	2010-10-08 17:24:20 +0000
@@ -1,9 +1,11 @@
-= Translation message helper functions =
+Translation message helper functions
+====================================
 
 For rendering translations in the TranslationMessageView a number of
 helper functions exist. The following sections cover them in detail.
 
-== contract_rosetta_escapes ==
+contract_rosetta_escapes
+------------------------
 
     >>> from lp.translations.browser.browser_helpers import (
     ...     contract_rosetta_escapes)
@@ -45,8 +47,20 @@
     >>> contract_rosetta_escapes('foo\\[nbsp]bar')
     'foo[nbsp]bar'
 
-
-== expand_rosetta_escapes ==
+Similarly, string '[nnbsp]' gets converted to narrow no-break space
+character.
+
+    >>> contract_rosetta_escapes('foo[nnbsp]bar')
+    u'foo\u202fbar'
+
+The string '\[nnbsp]' gets converted to a literal '[nnbsp]'.
+
+    >>> contract_rosetta_escapes('foo\\[nnbsp]bar')
+    'foo[nnbsp]bar'
+
+
+expand_rosetta_escapes
+----------------------
 
     >>> from lp.translations.browser.browser_helpers import (
     ...     expand_rosetta_escapes)
@@ -93,8 +107,22 @@
     >>> expand_rosetta_escapes(u'foo[nbsp]bar')
     u'foo<code>\\[nbsp]</code>bar'
 
-
-== parse_cformat_string ==
+Similarly, narrow no-break spaces get converted to a special constant
+TranslationConstants.NARROW_NO_BREAK_SPACE_CHAR which renders as below:
+
+    >>> expand_rosetta_escapes(u'foo\u202fbar')
+    u'foo<code>[nnbsp]</code>bar'
+
+Literal occurrences of u'[nnbsp]' get escaped to a special constant
+TranslationConstants.NARROW_NO_BREAK_SPACE_CHAR_ESCAPED which renders them
+as below:
+
+    >>> expand_rosetta_escapes(u'foo[nnbsp]bar')
+    u'foo<code>\\[nnbsp]</code>bar'
+
+
+parse_cformat_string
+--------------------
 
     >>> from lp.translations.browser.browser_helpers import (
     ...     parse_cformat_string)
@@ -112,7 +140,8 @@
     UnrecognisedCFormatString: %
 
 
-== text_to_html ==
+text_to_html
+------------
 
     >>> from lp.translations.browser.browser_helpers import (
     ...     text_to_html)
@@ -178,7 +207,8 @@
     u'foo<img alt="" src="/@@/translation-newline" /><br/>\nbar'
 
 
-== convert_newlines_to_web_form ==
+convert_newlines_to_web_form
+----------------------------
 
     >>> from lp.translations.browser.browser_helpers import (
     ...     convert_newlines_to_web_form)
@@ -194,7 +224,8 @@
     u'foo\r\nbar'
 
 
-== count_lines ==
+count_lines
+-----------
 
     >>> from lp.translations.browser.browser_helpers import count_lines
     >>> count_lines("foo")
@@ -216,8 +247,8 @@
     ...     "1234566789a123456789a")
     2
     >>> count_lines(
-    ...     "123456789abc123456789abc123456789abc123456789abc123456789abc123456\n"
-    ...     "789a123456789a123456789a")
+    ...     "123456789abc123456789abc123456789abc123456789abc123456789abc"
+    ...     "123456\n789a123456789a123456789a")
     3
     >>> count_lines(
     ...     "123456789abc123456789abc123456789abc123456789abc123456789abc"

=== modified file 'lib/lp/translations/interfaces/translations.py'
--- lib/lp/translations/interfaces/translations.py	2010-08-20 20:31:18 +0000
+++ lib/lp/translations/interfaces/translations.py	2010-10-08 17:24:20 +0000
@@ -16,6 +16,7 @@
     'TranslationsBranchImportMode',
     )
 
+
 class TranslationConstants:
     """Set of constants used inside the context of translations."""
 
@@ -31,6 +32,8 @@
     TAB_CHAR_ESCAPED = '<code>' + r'\[tab]' + '</code>'
     NO_BREAK_SPACE_CHAR = '<code>[nbsp]</code>'
     NO_BREAK_SPACE_CHAR_ESCAPED = '<code>' + r'\[nbsp]' + '</code>'
+    NARROW_NO_BREAK_SPACE_CHAR = '<code>[nnbsp]</code>'
+    NARROW_NO_BREAK_SPACE_CHAR_ESCAPED = '<code>' + r'\[nnbsp]' + '</code>'
 
 
 class TranslationsBranchImportMode(DBEnumeratedType):
@@ -54,5 +57,3 @@
         Import all translation files (templates and translations)
         found in the branch.
         """)
-
-