User get wrong system language after executing oem-config, if he is a foreigner in the country he selected in timezone select stage

The report contain detailed steps on how to reproduce the problem and also the probably root of the cause, may someone on the ubiquity team have a look to it? Thanks.

@ Jerone, this impacts to Acer projects on user experience. Could you please help to raise the importance? Thanks!

Details:

1. In language selection page, user choice zh_CN.UTF-8
2. In Timezone, the timezone is changed to TW time because user lives in Taiwan.
  2.1 debian/locale is changed to zh_TW
  2.2 debian-installer/language is changed to zh_TW:zh
        debian-installer/country is changed to TW
3. set LANGUAGE, LANG by debian/locale to target system

In ubiquity-2.2.24common4/scripts/tzsetup script, it executes `localechooser` to reconfigure locale according to the user selected country, and the localechooser change language, country, locale in system by 'debian-instalelr/locale' .

This bug is fixed in r61309 version of localechooser.

>>> r61309 | fjp | 2009-11-16 20:52:12 +0800 (Mon, 16 Nov 2009) | 9 lines
>>>
>>> Allow separate preseeding of language and country
>>>
>>> Currently users can only preseed the combination of language, country
>>> and locale by preseeding debian-installer/locale. The problem with
>>> this is that with this mechanism some valid combinations cannot be
>>> preseeded.
>>>
>>> Allow users to also separately preseed the language and country templates.
>>> If *only* locale is preseeded the old functionality is preserved.

So my idea is

1. we can pressed langauge and country in `ubiquity-2.2.24common4/scripts/tzsetup` to let localechooser(>2.27) to configure correct environment.

2. if language_part of $LANGLIST and $LOCAL in locale-chooser-apply is not the same, then install language pack by LANGLIST not LOCAL.

I did a patch to fix this bug. there was a two part need to modify.

1. Upgrade d-i/localechooser to 2.27 or greater to support that pressed LOCALE, LANGUAGE, COUNTRY but not only LOCALE, can anyone help to upgrade localechooser of ubiquity.

2. Modified language-installer, a hook script executed after ubiquity with a hack in the function `run` /bin/ubiquity-dm as below

  1 Index: ubiquity-2.0.0/bin/ubiquity-dm
  2 ===================================================================
  3 --- ubiquity-2.0.0.orig/bin/ubiquity-dm-2009-10-15 09:46:12.000000000 -0400
  4 +++ ubiquity-2.0.0/bin/ubiquity-dm->>>>>2009-10-15 09:47:06.000000000 -0400
  5 @@ -211,6 +211,10 @@
  6 greeter = subprocess.Popen(program, stdin=null, stdout=logfile, stderr=logfile)
  7 ret = greeter.wait()
  8 .
  9 + if self.frontend == 'gtk_ui' and os.path.exists('/usr/bin/language-installer'):
 10 + langinstall = subprocess.Popen(['/usr/bin/language-installer'], stdin=null, stdout=logfile, stderr=logfile)
 11 + ret = langinstall.wait()
 12 +
 13 def kill_if_exists(pid, signum):
 14 try:
 15 os.kill(pid, signum)

What language-installer do is to Install languages deb by user's prefer display interface language, and it suppose to install by LOCALE variable , not by first language in LANGUAGE variable. I had done a fix for language-installer version 39, but now langauge-installer upstream moves to version 43. I will discuss with Steve.

The bug should be fixed already. Please see Gunnar Hjalmarsson explained the display language design in 553162.
https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/553162/

I don't think this is fixed yet.

I am using a slightly modified version of natty ubiquity (forked at bzr revision 4696).

I selected Chinese Simplified in oem-config Language page, then Taiwan in Time zone page.

Result was zh_TW LANG and LANGUAGE. Thus Chinese Traditional translations would display, not Chinese Simplified.

I had a proposed patch of ubiquity for this bug.

This patch must not be applied in its current form. Please read
d-i/source/README. Furthermore, it contains debugging detritus that
should have been removed before posting a patch to a bug report.

In any case, that code has changed substantially since Maverick -
patches need to be made against the most recent development version for
us to be able to look at them.

For the record, this is how gdm and language-selector address the situation where a user does not want to use the 'standard' language in a given locale. (A common example is a Swedish user who wants to see his desktop in English).

- $LANG defines the general locale, i. e. for currency, time format, paper size etc. (sv_SE.utf8 in above example).
- $LANGUAGE overrides that to define the primary/secondary language for translations (e. g. "en_US:en" in above example)
- $LC_MESSAGES is set to the most preferred language (e. g. en_US.utf8 in above example) for software which does not use $LANGUAGE, e. g. thunderbird.

This leaves the $LC_* variables free for individual further customization.

The other alternative would be to set $LANG as the language, and then set $LC_CURRENCY, $LC_PAPER etc. to the location, but that's a lot more effort and a lot harder to change.

The installer should not set $LANGUAGE from the time zone selector. This is wrong in either case, as $LANGUAGE is defined to not have any notion of a region/location at all.

So ideally the initial question about the language should set $LANGUAGE and $LC_MESSAGES, and the time zone selector should set $LANG. If $LC_MESSAGES and $LANG would match, then it could just set $LANG.

Just test Ubuntu 11.10 yesterday. It seems the issue was still there.

1. At Installation, select language: Simplified Chinese, and location: Taiwan. The assumption is many Chinese work in Taiwan , and vice versa, who usually prefer to see Simplified Chinese UI.

2. After installation, ran update manager, then updated language pack as usual.

3. I found the installed languages in language-selector was Taiwanese-Chinese-English which should be Chinese-Taiwanese-English, and the character of folder is in Traditional Chinese which should be in Simplified Chinese. Please see the attachment.

Despite the general subject, I'm going to restrict this bug to the case of those languages where the choice of location may get you a different localisation (due to different dialect, writing system, etc.): of the languages we support for installation, that's Portuguese and Chinese. There is also a problem of what should happen if (say) you select English but then say you live in Switzerland, or select Chinese but say you live in France; that is a much harder problem and I would appreciate not having the scope of this bug widened to include that. All the people who've commented on this bug so far seem to be concerned with the case of zh_CN vs. zh_TW.

To fix this, we need to change both localechooser and ubiquity. The approach I'll take is modelled on that of language-selector; in cases where they mismatch, LANG should primarily reflect the location (so that numeric, monetary, etc. locale categories are correct) while LANGUAGE, LC_MESSAGES, LC_CTYPE, and LC_COLLATE should primarily reflect the language.

This bug was fixed in the package localechooser - 2.37ubuntu2

---------------
localechooser (2.37ubuntu2) oneiric; urgency=low

  * For cases where selecting a different location may imply a different
    dialect of the language, i.e. Portuguese and Chinese, take care to set
    LANG to something reflecting the location and
    LANGUAGE/LC_MESSAGES/LC_CTYPE/LC_COLLATE to something reflecting the
    language (LP: #590108). This roughly matches the behaviour of
    language-selector.
 -- Colin Watson <email address hidden> Wed, 05 Oct 2011 23:27:33 +0100

On 2011-10-06 00:14, Colin Watson wrote:
> Despite the general subject, I'm going to restrict this bug to the case
> of those languages where the choice of location may get you a different
> localisation (due to different dialect, writing system, etc.): of the
> languages we support for installation, that's Portuguese and Chinese.
> There is also a problem of what should happen if (say) you select
> English but then say you live in Switzerland, or select Chinese but say
> you live in France; that is a much harder problem and I would appreciate
> not having the scope of this bug widened to include that.

Not sure I understand how the latter would be much harder. Isn't it basically the same thing, i.e. picking one locale for LC_MESSAGES etc. and another locale for LANG?

> To fix this, we need to change both localechooser and ubiquity. The
> approach I'll take is modelled on that of language-selector; in cases
> where they mismatch, LANG should primarily reflect the location (so that
> numeric, monetary, etc. locale categories are correct) while LANGUAGE,
> LC_MESSAGES, LC_CTYPE, and LC_COLLATE should primarily reflect the
> language.

Yes, that's how language-selector does it in Oneiric.

I for one am playing with the idea to propose that we do it the other way around for the P cycle, i.e. let LANGUAGE and LANG reflect the language and, when needed, assign to LC_TIME, LC_NUMERIC, LC_CURRENCY etc. some other locale than what's put into LANG. I can see three reasons for doing this switch:

* It's how GNOME does it in g-c-c, and considering that we are moving towards replacing the language-selector UI with "Region and Language" in g-c-c, it would eliminate one of the current differences in approach between Ubuntu and GNOME.

* There seems to be quite a few desktop apps/tools, or parts of apps, that ignore both LANGUAGE and LC_MESSAGES for the display language, and let LANG solely determine the display language. (My LANG usually contains a Swedish locale, while my display language is English, and I often see Swedish translations in dialogs and menus.)

* Some distributions may prefer the simplistic approach to equal l10n with simply picking a locale name and assigning it to LANG. If we would switch to let LANG represent the language, the LANG variable would roughly be used for the same purpose all over, which would reduce the risk for confusion with respect to locale/language settings.

If it would be decided to do this switch, I hope that it won't be too hard to make similar changes to localechooser and ubiquity.

This bug was fixed in the package ubiquity - 2.8.4

---------------
ubiquity (2.8.4) oneiric; urgency=low

  [ Evan Dandrea ]
  * Handle the keyboard query window closed callback being fired twice
    (LP: #865493).
  * Allow focusing of labels when we're in the screen reader
    acessibility profile, so that Orca can read them
    (LP: #856782, LP: #856773).
  * Make sure we account for the size of the installation and swap
    partition when calculating the bounds for the partition resizer
    (LP: #769350).

  [ Colin Watson ]
  * Don't install duplicate copies of console-setup-apply,
    localechooser-apply, and netcfg-wrapper.
  * Bump fallback Ubuntu version number in ubi-language to 11.10.
  * Update translations from Launchpad.
  * For cases where selecting a different location may imply a different
    dialect of the language, i.e. Portuguese and Chinese, take care to set
    LANG to something reflecting the location and
    LANGUAGE/LC_MESSAGES/LC_CTYPE/LC_COLLATE to something reflecting the
    language (LP: #590108). This roughly matches the behaviour of
    language-selector.
  * Automatic update of included source packages: localechooser 2.37ubuntu2,
    partman-partitioning 81ubuntu3.

  [ Stéphane Graber ]
  * Update ubiquity-dm to export self.username as SUDO_USER.
    Make gsettings work properly later on during the install.
  * Whenever we call wget, set --tries=1 instead of the default
    of 20.
  * If we think we're offline, don't try to contact geoip or run
    rdate. (LP: #868668)
 -- Colin Watson <email address hidden> Thu, 06 Oct 2011 02:20:53 +0100

It's harder because you have to actually select the location-based
locale. I picked my example quite carefully: "I speak English and I
live in Switzerland" isn't enough to isolate a single locale
representing the location, for instance, so the installer would have to
guess; depending on which one it picks, it will get different answers
for such things as LC_NUMERIC. In some cases there will be a clearly
right answer; in some cases there will not. We may even have to ask,
although I would prefer to avoid that. As far as I can see
language-selector is operating with a somewhat different set of input
data and does not attempt to solve this problem.

In the pt/zh case, the problem was that we were correctly selecting a
locale matching the location, but this was making too great a change to
the more language-oriented categories.

I don't hugely care about which way round the environment variables go.
To be honest, I think you'll probably see bugs either way.

There is further discussion of the more general case in bug 1094872.