language-options causes live CD sessions to be untranslated

Gunnar, you know probably that script best, could you maybe have a look to what changed in 19.10 to create the issue?

Well, the point with the language-options script is to only list languages where the language-packs are present. It has been that way for many cycles, and AFAIK nothing special happened in this respect in 19.10. @Alkis: Are you claiming that it worked otherwise in e.g. 18.04?

With that said, I understand that it's not an optimal approach wrt the live session for a flavor. If we are going to change this somehow (I have no idea how ATM), it should reasonably be done without changing the language-options script. The latter was written with a real install in mind at first hand.

Gunnar, yes, the live sessions are properly translated up to and including 19.04 (and of course 18.04 too).
Starting from 19.10, all child processes of lightdm have LANGUAGE=en, so the session is untranslated. But this only happens for languages that don't have the langpack installed.

Running /usr/share/language-tools/language-options in the Ubuntu 18.04 live CD does NOT show el_GR in the output though.

But if that script hasn't changed, and lightdm hasn't changed... what did change and broke this in the 19.10 live CDs? :/

I wonder if the lightdm patch changed, and it used to call locale -a, and it was changed to call the language-options script in 19.10.

In Ubuntu 20.04 live CD, in .xsession-errors, I see this line:

dbus-update-activation-environment: setting LANGUAGE=en

Maybe that's to blame here; looking more...

No, sorry, that's not it, I removed it and restarted lightdm and I still had LANGUAGE=en in the live session.

@Alkis: Since it sounds as if you have access to both a 18.04 and a 19.10 ISO, can you please check what the locale command outputs in the live session of respective release.

Hi Gunnar, here are the results:

Output of Ubuntu MATE 20.04 today's daily build:

$ env | grep LANG
LANGUAGE=en
GDM_LANG=en
LANG=el_GR.UTF-8

$ locale -a
C
C.UTF-8
de_AT.utf8
de_BE.utf8
de_CH.utf8
de_DE.utf8
de_IT.utf8
de_LI.utf8
de_LU.utf8
el_GR.utf8
en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IL
en_IL.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZM
en_ZM.utf8
en_ZW.utf8
es_AR.utf8
es_BO.utf8
es_CL.utf8
es_CO.utf8
es_CR.utf8
es_CU
es_CU.utf8
es_DO.utf8
es_EC.utf8
es_ES.utf8
es_GT.utf8
es_HN.utf8
es_MX.utf8
es_NI.utf8
es_PA.utf8
es_PE.utf8
es_PR.utf8
es_PY.utf8
es_SV.utf8
es_US.utf8
es_UY.utf8
es_VE.utf8
fr_BE.utf8
fr_CA.utf8
fr_CH.utf8
fr_FR.utf8
fr_LU.utf8
it_CH.utf8
it_IT.utf8
POSIX
pt_BR.utf8
pt_PT.utf8
ru_RU.utf8
ru_UA.utf8

$ /usr/share/language-tools/language-options
de_CH
de_DE
en
en_AU
en_CA
en_GB
en_US
es_AR
es_CL
es_CO
es_CR
es_DO
es_EC
es_ES
es_MX
es_NI
es_PA
es_PE
es_PR
es_SV
es_US
es_UY
es_VE
fr_CA
fr_FR
it_IT
pt_BR
pt_PT
ru

Output of Ubuntu MATE 18.04.3 iso:

$ env | grep LANG
LANG=el_GR.UTF-8

$ locale -a
C
C.UTF-8
de_AT.utf8
de_BE.utf8
de_CH.utf8
de_DE.utf8
de_IT.utf8
de_LI.utf8
de_LU.utf8
el_GR.utf8
en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IL
en_IL.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZM
en_ZM.utf8
en_ZW.utf8
es_AR.utf8
es_BO.utf8
es_CL.utf8
es_CO.utf8
es_CR.utf8
es_CU
es_CU.utf8
es_DO.utf8
es_EC.utf8
es_ES.utf8
es_GT.utf8
es_HN.utf8
es_MX.utf8
es_NI.utf8
es_PA.utf8
es_PE.utf8
es_PR.utf8
es_PY.utf8
es_SV.utf8
es_US.utf8
es_UY.utf8
es_VE.utf8
fr_BE.utf8
fr_CA.utf8
fr_CH.utf8
fr_FR.utf8
fr_LU.utf8
it_CH.utf8
it_IT.utf8
POSIX
pt_BR.utf8
pt_PT.utf8
ru_RU.utf8
ru_UA.utf8
zh_CN.utf8
zh_SG.utf8

$ /usr/share/language-tools/language-options
de_CH
de_DE
en
en_AU
en_CA
en_GB
en_US
es_AR
es_CL
es_CO
es_CR
es_DO
es_EC
es_ES
es_MX
es_NI
es_PA
es_PE
es_PR
es_SV
es_US
es_UY
es_VE
fr_CA
fr_FR
it_IT
pt_BR
pt_PT
ru
zh_CN

I.e. from what I can see, the basic difference is that since 19.10, something sets LANGUAGE=en.

While in the Italian session that has a langpack in the live iso, LANGUAGE=it.

It's not /etc/default/locale, in all the cases only LANG is there and not LANGUAGE.

I actually meant the locale command without options, but I still think you provided the info we need. ;)

On 2020-01-31 19:48, Alkis Georgopoulos wrote:
> It's not /etc/default/locale, in all the cases only LANG is there
> and not LANGUAGE.

LightDM does set LANGUAGE for some reason through an Ubuntu specific patch, and it might explain it. The weird thing is that LightDM has done that since long before 19.10.

Anyway, maybe we should simply stop LightDM from messing with LANGUAGE when run in a live session.

On a focal live cd, I tried to downgrade lightdm and accountsservice to their bionic versions and restart their services without rebooting, yet again the session was not translated. So I'm at a loss on which package was the one that caused the change in 19.10, maybe even glibc itself.

Nevertheless, I was able to reproduce a similar problem in a normal (non-live) focal installation, by just purging language-pack-el, or by just moving aside the directory /usr/share/locale-langpack/el and restarting accounts-daemon and lightdm.

Without the language pack, my LANGUAGE is "en" and my session is untranslated. And if I launch d-feet and check /org/freedesktop/Accounts/User1000 => org.freedesktop.Accounts.User => property Language, it says "en".

I think that is the problem, accountsservice should report that my language is either "el" or empty, but definitely not "en". Even if the langpack isn't there, LANG in /etc/default/locale is el_GR.UTF-8, and I don't have "en" defined anywhere in my system or my account; and MATE can show Greek even without a langpack.

I think then LightDM will work fine without any changes.
Does this approach sound OK?

(untagging from rls tracking since it's not impacting Ubuntu Desktop, doesn't block us to work on a fix still)

The reason that this regression doesn't affect ubuntu-desktop, might be that unlike LightDM, GDM doesn't have that Ubuntu-specific patch to use /usr/share/language-tools/language-options.

So, I subscribed "lightdm (Ubuntu)" to the affected packages, in case we just want to drop the Ubuntu-specific patch, and that way make lightdm not pass LANGUAGE when it shouldn't.

I think that this won't address the source of the regression, but it will make the problem not appear in flavors that use lightdm which should be good enough.

Yeah, I tend to think that the fix should be made in lightdm. My idea is to modify the patch, so LANGUAGE is set only if this conditional is satisfied:

    if (system ("df | grep ^/cow >/dev/null") != 0)

That way LANGUAGE should not be set in a live session AFAIU.

@Sebastien: Do you think this is a proper way to test for a live session?

Idea from https://askubuntu.com/q/442055

Gunnar, so if a MATE user doesn't have the langpack installed, he shouldn't have a translated session?
Why do we want to fix this only for Live CDs?

Because the translations of all the "main" packages (and a few others) are brought by the language packs, and also MATE users most likely use some "main" packages. He is simply supposed to have the language packs for his language installed. That's how Ubuntu including the flavors has been designed for many cycles.

Since we are close to the release of an LTS, I want to narrow the fix to only affect where it matters, i.e. the live sessions.

I mean that if, on a normal 20.04 installation, I uninstall the el langpack, then
1) If I'm using GDM, universe apps show Greek,
2) If I'm using LightDM, universe apps show English (this one regressed in 19.10).

If we're modifying LightDM, I suggest to modify LightDM to not set LANGUAGE at all, not just in live CDs.
This is what GDM does and it works fine everywhere.

Not sure why you would want to uninstall the language packs for a language you have configured. You are not supposed to do that. Please use "Language Support" to manage your languages.

This special thing in LightDM is there for a reason (even if I don't remember the reason exactly). I'm not ready to propose something which might have adverse side effects.

Attached please find a debdiff which I think fixes it in live sessions. The real test can be done only after the new version of lightdm has reached the daily build ISOs. Would appreciate your confirmation when that happens.

Btw, lightdm doesn't set LANGUAGE at all in Debian; it only sets LANG, like GDM, and everything works fine.

Gunnar, do you see any issues with just completely dropping your 04_language_handling.patch?
Or at least this line there:

+ session_set_env (session, "LANGUAGE", language);

I commented before I saw the previous reply; I'll reply to that now.

The "grep cow" approach breaks the LTSP package and other software that uses a COW root.
To check for casper, please `grep -w boot=casper /proc/cmdline` or something casper-specific.

In schools, we sometimes have students of various nationalities. They're not frequent enough to install whole langpacks for them, but they are frequent enough to configure locales for them (locale-gen).

I do not see any reason to keep the Ubuntu-specific 04_language_handling.patch, when we know that it's causing issues and we do not know its benefit. Of course it's your call. :)

On 2020-02-07 13:41, Alkis Georgopoulos wrote:
> The "grep cow" approach breaks the LTSP package and other software
> that uses a COW root.

Do those packages cause the file system /cow, listed by the df command, to be mounted? If yes, I wouldn't mind to modify the debdiff. Please let me know.

> To check for casper, please `grep -w boot=casper /proc/cmdline` or
> something casper-specific.

Thanks for the tip!

> In schools, we sometimes have students of various nationalities.
> They're not frequent enough to install whole langpacks for them, but
> they are frequent enough to configure locales for them (locale-gen).

It's unclear to me why it's more difficult to install language packs than generating locales separately.

In any case, I would prefer to consider this aspect to be beyond the scope of this bug report.

> Of course it's your call.

Yes. :)

I checked with 20.04 and it appears that recent Ubuntu live CDs don't have boot=casper anymore; they do still have initrd=/casper/initrd; but anyway, ^/cow should be fine (sorry I hadn't noticed the ^ there before).

Thank you for your work in this.

On 2020-02-07 16:41, Alkis Georgopoulos wrote:
> I checked with 20.04 and it appears that recent Ubuntu live CDs
> don't have boot=casper anymore; they do still have
> initrd=/casper/initrd;

I also saw that /proc/cmdline contains BOOT_IMAGE=/casper/..., so the approach would probably have worked with a minor modification.

> but anyway, ^/cow should be fine (sorry I hadn't noticed the ^ there
> before).

Ok, good, then I keep the debdiff for now, in the hope that some sponsor will approve and upload it soon.

> Thank you for your work in this.

Thank you for reporting it. The current behavior in live sessions of the flavors is indeed unintentional and unnecessary.

Related pull request: https://github.com/canonical/lightdm/pull/111

I tested the live session with today's daily build of Ubuntu MATE and can confirm that LANGUAGE is no longer set. Thus translated strings in universe packages are always displayed in the selected language.