Can't build html with footnotes when using i18n

One problem is obviously that href names can't be translated in sphinx texts because it breaks references. :-(

Sorry:

I am trying to understand the problem:

in the phrase:

This guide .... has been `posted on GNU.org`_.

`posted on GNU.org`_. is the href name and CAN'T be translated. is it OK?
so the correct translation must be:

Esta guia..... ha sido `posted on GNU.org`_.

Yes, this should be correct "Esta guia..... ha sido `posted on GNU.org`_."

It's unfortunate we don't seem to be able to translate the link names. Once we resolved this issue, I will raise the issue with upstream. This makes it harder to get 100% translations. :-/

You can try if this works, if you check out the branch lp:ubuntu-packaging-guide and run 'make html-es'.

Ok, I will change the .po with all the href corrected asap.

I don't think that this has to do with href names. Looking around at similar errors and looking at where the build stops, it seems to me that this has something to do with the footnote in udd-merging.rst Sure enough, if you delete the foot note the build is successful.

That said, I still don't really understand the problem. But it is likely a bug in sphinx. We should be able to work-a-round it though.

Here it is. Already reported upstream:

https://bitbucket.org/birkenfeld/sphinx/issue/955/cant-build-html-with-footnotes-when-using

Hola Paco,

Por el correo que le has enviado a Daniel Holbach entiendo que te has
descargado el .po y estás corrigiéndolo. Entiendo que los problemas son de
la traducción de las referencias. Si te puedo echar una mano dime cómo.
Mientras tanto no haré nada con las traducciones para no crear conflictos,
pero me estoy instalando los paquetes que me faltaban ahora que sé como
instalar los idiomas.

Saludos

2012/10/19 Paco Molinero <email address hidden>

> Ok, I will change the .po with all the href corrected asap.
>
> --
> You received this bug notification because you are a member of Ubuntu
> Spanish Translators, which is a bug assignee.
> https://bugs.launchpad.net/bugs/1068493
>
> Title:
> Spanish translation FTBFS
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu-packaging-guide/+bug/1068493/+subscriptions
>
> --
> Lista de correo: Ubuntu-es-l10n
>
> Participa suscribiéndote y escribiendo a:
> <email address hidden>
>
> Para darte de alta, de baja o hacer ajustes a tu suscripción visita:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-es-l10n
>

OK, Maybe Adrew are rigth and the problem is not on href names.
In any case I just uploaded a new .po.es with fixed translations (avoiding translation on `... _` phrases) but in betwen the .po original was changed so I do not know if could be some collision.

The new .po.es file is up so the problem should be fixed.

Did anyone try the sphinx patch? Maybe we should upload it to our PPA?

Updated package is built in https://launchpad.net/~mitya57/+archive/test1/ and is working.

I'll upload the same version to our packaging guide PPA.

Looks like we will have to do some backporting work to get it to build for oneiric. *sigh* :-)

13.04 (raring) FTBFS too:
Running Sphinx test suite...
env python3.3 tests/run.py --verbose
env: python3.3: No such file or directory
make: *** [build-stamp] Error 127

Unfortunately raring is not easy to be fixed. Just adding python3.3 won't fix it. For now I'll go ahead with the 12.04 and 12.10 builds of sphinx which have the i18n fix included, so we can offer PPA users of those releases a nicely translated packaging guide and also offer it online on developer.u.c

Any help with fixing 13.04 and 11.10 is appreciated.

Raring issue is bug 1070336. I've spent yesterday's evening debugging
it (without any success), will continue investigation today or
tomorrow.

I completed the Spanish translation and tried to generate html
documentation after updating sphinx with the fresh version from the Ubuntu
Packaging Team's PPA, but I got exactly the same error than with the old
version.

python-sphinx package version is: 1.1.3+dfsg-4ubuntu4+ppa1~12.10

Did I make something wrong? Could anyone check it?

Thanks

2012/10/25 Daniel Holbach <email address hidden>

> Unfortunately raring is not easy to be fixed. Just adding python3.3
> won't fix it. For now I'll go ahead with the 12.04 and 12.10 builds of
> sphinx which have the i18n fix included, so we can offer PPA users of
> those releases a nicely translated packaging guide and also offer it
> online on developer.u.c
>
> Any help with fixing 13.04 and 11.10 is appreciated.
>
> --
> You received this bug notification because you are a member of Ubuntu
> Spanish Translators, which is a bug assignee.
> https://bugs.launchpad.net/bugs/1068493
>
> Title:
> Spanish translation FTBFS
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/sphinx/+bug/1068493/+subscriptions
>
> --
> Lista de correo: Ubuntu-es-l10n
>
> Participa suscribiéndote y escribiendo a:
> <email address hidden>
>
> Para darte de alta, de baja o hacer ajustes a tu suscripción visita:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-es-l10n
>

You are right. It fails to build, but I get a different error this time:

sphinx-build -Dlanguage=es -b latex -d _build/doctrees . _build/latex/es
Making output directory...
Running Sphinx v1.1.3
loading translations [es]... done
loading pickled environment... not yet created
building [latex]: all documents
updating environment: 25 added, 0 changed, 0 removed
reading sources... [ 4%] ubuntu-packaging-guide/auto-pkg-test
reading sources... [ 8%] ubuntu-packaging-guide/backports
reading sources... [ 12%] ubuntu-packaging-guide/chroots
reading sources... [ 16%] ubuntu-packaging-guide/communication
reading sources... [ 20%] ubuntu-packaging-guide/debian-dir-overview
reading sources... [ 24%] ubuntu-packaging-guide/fixing-a-bug
reading sources... [ 28%] ubuntu-packaging-guide/fixing-a-bug-example
reading sources... [ 32%] ubuntu-packaging-guide/getting-set-up

Exception occurred:
  File "/usr/lib/python2.7/dist-packages/sphinx/application.py", line 204, in build
    self.builder.build_update()
  File "/usr/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 191, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 216, in build
    purple, length):
  File "/usr/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 120, in status_iterator
    for item in iterable:
  File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 617, in update_generator
    self.read_doc(docname, app=app)
  File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 765, in read_doc
    pub.publish()
  File "/usr/lib/python2.7/dist-packages/docutils/core.py", line 212, in publish
    self.apply_transforms()
  File "/usr/lib/python2.7/dist-packages/docutils/core.py", line 193, in apply_transforms
    self.document.transformer.apply_transforms()
  File "/usr/lib/python2.7/dist-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 232, in apply
    node.children[i] = child
IndexError: list assignment index out of range
The full traceback has been saved in /tmp/sphinx-err-FcOJkS.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
Either send bugs to the mailing list at <http://groups.google.com/group/sphinx-dev/>,
or report them in the tracker at <http://bitbucket.org/birkenfeld/sphinx/issues/>. Thanks!
make[2]: *** [latexpdf-es] Fehler 1

I could generate the html-es after modifying several translation, so with current Spanish translations on Launchpad no longer FTBFS.

However there is a real problem on how Sphinx handles the translation. The problem arise whenever a segment to be translated starts or with a link and the translated version adds some text before or after the starting/ending link, respectively. For example:
  - if "`Link`_ English text" is translated into "Spanish text `Link`_ more text" it breaks Sphinx.
  - if "English text `Link`_ " is translated into "Spanish text `Link`_ more text" it breaks Sphinx.
This should be accepted by the tool, as those are perfectly normal situations (in fact, my modified translation to avoid the problem sounds less natural to me than original ones)

The reason for that problem is that the original text and the translated patch are expected to have the same number of parts, as the following debug session shows:

227
228 # copy text children
229 for i, child in enumerate(patch.children):
230 if isinstance(child, nodes.Text):
231 child.parent = node
232 -> node.children[i] = child
233
234
235 class SphinxStandaloneReader(standalone.Reader):
236 """
237 Add our own transforms.

p i
3

p patch.children
[<#text: 'La p\xe1gina '>, <reference: <#text: 'The new accoun ...'>>, <target "the new account help page": >, <#text: ' en Launchpad contiene m\xe1s informaci\xf3n sobre el proceso y par\xe1me ...'>]

p node.children
[<#text: 'La p\xe1gina '>, <target "the new account help page": >, <#text: '\non Launchpad has more information about the process and additio ...'>]

So there are 4 parts in the patch but only 3 in the original text, which lead to the above problem. For that example I had to modify the translation in the following way:

Original text:
`The new account help page <https://help.launchpad.net/YourAccount/NewAccount>`_ on Launchpad has more information about the process and additional settings you can change.

Intended translation:
La página `The new account help page <https://help.launchpad.net/YourAccount/NewAccount>`_ en Launchpad contiene más información sobre el proceso y parámetros adicionales que puede cambiar.

Modified translation:
`The new account help page <https://help.launchpad.net/YourAccount/NewAccount>`_ en Launchpad contiene más información sobre el proceso y parámetros adicionales que puede cambiar.

It seems that the issue described in comment 20 is (at least partially) caused by the patch. We have discussed this with Jakub Wilk (Debian maintainer of Sphinx) and decided to not apply the patch until that issue is resolved. I've commented on the upstream bug already.

This bug was fixed in the package sphinx - 1.1.3+dfsg-5ubuntu1

---------------
sphinx (1.1.3+dfsg-5ubuntu1) raring; urgency=low

  * Merge with Debian packaging SVN.
    - This brings in fix for LP: #1068493.
  * Remaining Ubuntu changes:
    - Switch to dh_python2.
    - debian/rules: export NO_PKG_MANGLE=1 in order to not have translations
      stripped.
    - debian/rules: Modify xvfb-run to use auto-servernum flag (fixes FTBFS).
    - debian/control: Drop the dependency on python-whoosh.
    - debian/control: Add "XS-Testsuite: autopkgtest" header.
    - debian/patches/fix_manpages_generation_with_new_docutils.diff:
      Fix FTBFS with the new python-docutils package.

sphinx (1.1.3+dfsg-6) UNRELEASED; urgency=low

  [ Jakub Wilk ]
  * DEP-8 tests: remove “Features: no-build-needed”; it's the default now.
  * Bump standards version to 3.9.4; no changes needed.

  [ Dmitry Shachnev ]
  * debian/patches/l10n_fixes.diff: fix crashes and not working external
    links in l10n mode (closes: #691719).

sphinx (1.1.3+dfsg-5) experimental; urgency=low

  [ Jakub Wilk ]
  * DEP-8 tests: use $ADTTMP.
  * dh_sphinxdoc: ignore comments when analysing HTML files (closes: #682850).
    Thanks to Dmitry Shachnev for the bug report.
  * Add dvipng to Suggests (closes: #687273). Thanks to Matthias Klose for the
    bug report.
  * Set PYTHONHASHSEED=random in debian/rules and in DEP-8 tests.
  * Backport upstream patch to fix encoding issues in test_build_html. Now
    that this is fixed, stop running Python 3 tests under LC_ALL=C.
  * Make “debian/rules binary-arch” no-op.
  * Update version number in the sphinx-autogen manpage.
  * Improve dh_sphinxdoc:
    + Fix the --tmpdir option. Thanks to Andriy Senkovych for the bug report.
    + Ignore references to JavaScript code that start with an URI scheme.
      Thanks to Dmitry Shachnev for the bug report.
    + Strip query (?...) and fragment (#...) components from JavaScript
      references. Thanks to Dmitry Shachnev for the bug report.
  * Sort stopwords in searchtools.js. Thanks to Dmitry Shachnev for the bug
    report.
  * Fix compatibility with Python 3.3. Thanks to Dmitry Shachnev for the bug
    report and hunting down the upstream patch.

  [ Dmitry Shachnev ]
  * Update Homepage field to point to http://sphinx-doc.org/.
  * Build-depend of python3-all instead of python3.
 -- Dmitry Shachnev <email address hidden> Tue, 27 Nov 2012 19:20:44 +0400

http://developer.ubuntu.com/resources/tools/packaging/ hablo Español! :-D Announce to follow in a bit.

Getting this into Ubuntu and backporting it will be a bit harder, but we should look into it.

Attached two SRU branches. I would prefer to wait for bug 1008537 to be fixed, but this has already been delayed for a long time, so we can upload it now and hope it won't fail to build.