ubuntu-docs build failure - possible solution

@Boyuan: Do you have any thoughts on the above?

Hmm.. I saw in the buildlog from the successful build that it skipped the Chinese translation of a particular string. The reason proved to be a missing '<' character, i.e. the translated string contains an instance of '/em>' instead of '</em>'. And that's the correct behavior; it should check the Mallard syntax and skip translations with invalid syntax.

Anyway, I added the missing '<', and then ubuntu-docs built with the itstool version in focal.

So the direct reason for the build failure is a syntactically incorrect translation. OTOH, translators make mistakes, and a bad translation *should not* make the whole package build to fail.

@Shaun: Have you possibly stumbled into anything similar when building e.g. gnome-user-docs?

Uploaded a new version of ubuntu-docs, which builds on focal despite the issue discussed in this bug report.

It's segfault in libxml code, their binding doesn't seem to handle invalid files well, somewaht similar to https://gitlab.gnome.org/GNOME/libxml2/-/issues/12

@Gunnar It seems that the root cause has been found, thanks! It's well known that translation could be fragile when the build system is not well defined; all I can think of is to make the build system robust enough and yell out loudly enough whenever an error (e.g., segfault) happens.

Thanks Sebastien!

Or https://gitlab.gnome.org/GNOME/libxml2/issues/64 (which is not yet closed).

@Boyuan: Well, it does scream loudly in the sense that the build fails.

I wouldn't blame the build system in this case. It has worked in a sensible and robust way for many years. The segfault issue only happens with python3-libxml2, not with python-libxml2. And I have the impression (not yet confirmed) that the segfault only happens for XML syntax errors in Chinese translations; for similar errors in non-Chinese translations it only prints a warning and moves on (which is the desired behavior).

So in general, syntax errors in translations just cause a warning, and the original text is used. itstool only errors on syntax if you pass --strict, and yelp.m4 doesn't even give you a way to pass --strict.

There's a chance this is itstool's fault. You do still have to do memory management with libxml2's Python bindings, so stuff like double frees in application code is possible. But since this crash happens during error reporting for a language with many-byte UTF-8 chars, and since it weirdly only happens with Python 3, I suspect it's this libxml2 bug:

https://bugzilla.gnome.org/show_bug.cgi?id=789714

Thanks Shaun! It's indeed that bug. Why am I not surprised when seeing that you filed it? :)

I grabbed a Fedora patch and submitted a Debian merge request:

https://salsa.debian.org/xml-sgml-team/libxml2/-/merge_requests/3

It looks pretty spot on, don't you think? At least, with that patch applied I can build ubuntu-docs 19.10.2 successfully again. A syntax error in one of the Chinese translations is reported as warnings as expected.

This bug was fixed in the package ubuntu-docs - 20.04.1

---------------
ubuntu-docs (20.04.1) focal; urgency=medium

  * scripts/update-translations.sh:
    - Handle tarballs also from the package
  * Bump version to 20.04 / focal
  * debian/control.in, debian/control:
    - Replace Vcs-Bzr with Vcs-Git
    - Bump Standards-Version to 4.5.0
  * New languages: af, ckb
  * Translations from Launchpad:
    - Syntactically incorrect Chinese translations of two strings were
      corrected before this translation update, which works around the
      focal build failure as described in LP: #1869814.

 -- Gunnar Hjalmarsson <email address hidden> Tue, 31 Mar 2020 14:11:00 +0200

@Gunnar, looks like the patch I backported earlier in the cycle
https://launchpad.net/ubuntu/+source/libxml2/2.9.4+dfsg1-8ubuntu3

But apparently doko synced from Debian overlooking we had a delta...

@Sebastien: Indeed, it's the very same thing. Then I spent several hours on an issue that was already spotted and fixed. :(

To the sponsor:

libxml2 2.9.10+dfsg-5, including the patch, is now available in Debian unstable. So please sync libxml2.

sync'd. please close this bug once libxml2 has been accepted
https://launchpad.net/ubuntu/focal/+queue?queue_state=1

This bug was fixed in the package libxml2 - 2.9.10+dfsg-5

---------------
libxml2 (2.9.10+dfsg-5) unstable; urgency=medium

  * Team upload.

  [ Mattia Rizzolo ]
  * d/rules:
    + Drop --disable-silent-rules, already passed by dh_auto_configure.
    + Drop --parallel, now default with debhelper compat > 10.
    + Use dh_installdocs and dh_installexamples to install docs and examples.
    + Use dh_missing --fail-missing (and add the relevant d/not-installed).
    + Minimize indep build to build only the docs.
  * d/watch: fix an option to avoid a warning message.
  * d/control:
    + Move most of the build-deps to Build-Depends-Arch.
    + Use ${python:Depends} also for python-libxml2-dbg.
  * Add a lintian override for
    debian-rules-uses-supported-python-versions-without-python-all-build-depends

  [ Gunnar Hjalmarsson ]
  * d/p/python3-unicode-errors.patch:
    Fix segfault issue with itstool and py3. LP: #1869814

 -- Mattia Rizzolo <email address hidden> Fri, 10 Apr 2020 14:53:23 +0200