Merge ~pieq/bugit/+git/qabro:fix-1806841-urwid-unicode into bugit:master

[Pierre Equoy]

[Maciej Kisielewski (kissiel)]

> By default, snap doesn't transmit locale information.
I don't think that's true. Or I'm misinterpreting your statement.

Apps in a snap are given the same locale as available in the calling environment.

Overriding locale is a very bad smell. I think your problem lies in how you use urwid. All strings inside your program should be strings. If you are forced to supply raw bytes to some function, make sure that function knows how to interpret those bytes. If it's poorly written and understands ascii only, then make sure you supply it with such (stripping, or replacing all erroneous chars).

review: Needs Fixing

[Pierre Equoy (pieq)]

Hi Maciek and thanks for your feedback!

> Apps in a snap are given the same locale as available in the calling environment.

~$ python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getlocale()
('en_US', 'UTF-8')
>>>
~$ sudo snap install qabro --devmode
[sudo] password for pierre:
qabro 0.8 from Pierre Equoy (pieq) installed
~$ snap run --shell qabro
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

pierre@tmonk:/home/pierre$ /usr/bin/python3
Python 3.5.2 (default, Nov 12 2018, 13:43:14)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getlocale()
(None, None)

both environments inside and outside of the snap contain the same LANG and the same LC_* parameters, as I explained on the snapcraft forum. [1]

According to urwid's documentation [2], urwid guesses what encoding to use based on the locale available by Python. I've run a test outside and inside the snap, and I discovered that, when outside of the snap, everything was fine (urwid.get_encoding_mode() returns "utf-8" and I can paste whatever I want in the text fields), whereas inside the snap, urwid.get_encoding_mode() returns "narrow" (which means 8bit encoding) and pasting Unicode turns it into "???".

Does the above explanation light the problem under a different light for you? Do you think it's still a problem on my side? (I'm not good at localization and encodings shenanigans :))

Thanks!

[1] https://forum.snapcraft.io/t/python-cannot-determine-the-locale-used-when-inside-a-snap/9492
[2] http://urwid.org/manual/encodings.html

[Maciej Kisielewski (kissiel)]

I followed this rabbit hole a bit.

Snap doesn't hide or change locale information.
If you'd check 'locale.getdefaultlocale()' your envvars from the host would still be there. It's that python sets them to (None, None) when it cannot load them as in your case. And it cannot load them b/c they are not included in the snap, or to be precise in the environment visible from within snap.

Unfortunately it's not as simple as adding '- locales' to the staged parts. It would be necessary to generate those from within the snap.

To properly fix this you'd need to create a launcher that ensures locale data is there (or maybe an install hook would even suffice). I know that QABro doesn't use translations, so I guess it's not worth the work ATM, so +1 on landing this with overriding to C.UTF-8.

review: Approve

diff --git a/qabro/__version__.py b/qabro/__version__.py
index 0dde18a..57cb785 100644
--- a/qabro/__version__.py
+++ b/qabro/__version__.py
@@ -1,5 +1,5 @@
 __title__ = 'qabro'
-__version__ = '0.8'
+__version__ = '0.9dev'
 __description__ = 'Tool to generate a Launchpad bug report and attach useful logs (using sosreport and the like).'
 __author__ = 'Pierre Equoy'
 __author_email__ = '[e-mail]'
diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml
index ecfa2e2..93d0d71 100644
--- a/snap/snapcraft.yaml
+++ b/snap/snapcraft.yaml
@@ -10,6 +10,8 @@ apps:
     command: qabro
     environment:
       LD_LIBRARY_PATH: $LD_LIBRARY_PATH:$SNAP/lib/fwts
+      LANG: C.UTF-8
+      LC_ALL: C.UTF-8
 
   sosreport:
     command: sosreport
@@ -59,7 +61,26 @@ parts:
       patch $SNAPCRAFT_PART_INSTALL/usr/share/sosreport/sos/policies/ubuntu.py $SNAPCRAFT_STAGE/patches/sosreport-env-path-snap.patch
 
   fwts:
-    source-tag: V18.06.02
+    plugin: autotools
+    source: git://kernel.ubuntu.com/hwe/fwts
+    source-tag: V19.01.00
+    stage-packages:
+        - libfdt1
+        - libbsd0
+        - libpci3
+    build-packages:
+        - gcc
+        - make
+        - autoconf
+        - automake
+        - libtool
+        - libjson-c-dev
+        - flex
+        - bison
+        - dh-autoreconf
+        - libglib2.0-dev
+        - libfdt-dev
+        - libbsd-dev
 
   lspci:
     plugin: nil