pkgbinarymangler: Should build a tarball for pot import even when translations are not stripped

Sorry, but I don't understand this bug. Looking at the source package, I see:

  update-notifier-0.119ubuntu5/po/update-notifier.pot

This file is built at source package generation time, so it's up to date as of that point. How is dh-translations going to help here?

(And the .pot file in the source package has all of the current strings.)

If you use msgfmt --stat on that pot file, you can see that it contains 67 strings, including those you wrote about in your mail to the translators list (bug #876298).
Now let's see my translation:
https://translations.launchpad.net/ubuntu/precise/+source/update-notifier/+pots/update-notifier/hu/+translate
61 strings, all translated, very good. Minus the new ones:
https://translations.launchpad.net/ubuntu/precise/+source/update-notifier/+pots/update-notifier/hu/+translate?batch=10&show=all&search=The+following+packages+requested

This is where dh_translations would help. Only having a pot file in po/ is not enough for LP Translations to import it, a new one needs to be generated during build.

For the record, I have just uploaded the fresh template manually. For the longer term, it would make sense to merge the changes though.

On Thu, Apr 12, 2012 at 06:05:29PM -0000, Gabor Kelemen wrote:
> This is where dh_translations would help. Only having a pot file in po/
> is not enough for LP Translations to import it, a new one needs to be
> generated during build.

It does *not* need to be generated during build. In fact, generating .pot
files during build is actively harmful if you have gettext domains for
multiarch libraries.

I investigated this with Sebastien Bacher on IRC; from the build log, it
appears the problem is actually that pkgbinarymangler has update-notifier
blacklisted, which means the translations tarball (containing the .pot file)
is never generated. We don't want to directly revert that because that
would cause pkgbinarymangler to *also* strip out the .mo files from the
binary packages... so a more subtle fix is required.

In the meantime, thanks for the manual upload.

> It does *not* need to be generated during build.

So far it has been our strategy to do exactly that, and even in the case of update-notifier this should still happen. NB that this really just means "generated", not "shipped".

> In fact, generating .pot files during build is actively harmful if you have gettext domains for multiarch libraries.

How would it be harmful?

>> In fact, generating .pot files during build is actively harmful if
>> you have gettext domains for multiarch libraries.

> How would it be harmful?

Regenerating the .pot file at package build time changes timestamps, which means the .mo files shipped in the package are not identical across architectures.

Steve Langasek [2013-05-17 14:45 -0000]:
> Regenerating the .pot file at package build time changes timestamps,
> which means the .mo files shipped in the package are not identical
> across architectures.

I don't understand this. This would only happen if the package updates
its *.po files during package build, which is evil and unnecessary.
But build systems which do that would already create different *.mo
files on each build, so whether or not you update the *.pot as well
should make little difference.

In the ordinary case, the build sytem just builds *.mo from the source
*.po files, and completely ignores the *.pot. Most binary packages get
their *.mo files stripped out anyway, and for the remaining ones we
just need to make sure that we don't put *.mo files into arch specific
packages (either packages are arch:all anyway, or we put translations
into an arch:all -common binary, which is quite common for GNOME
packages).

On Mon, May 20, 2013 at 02:40:12PM -0000, Martin Pitt wrote:
> Steve Langasek [2013-05-17 14:45 -0000]:
> > Regenerating the .pot file at package build time changes timestamps,
> > which means the .mo files shipped in the package are not identical
> > across architectures.

> I don't understand this. This would only happen if the package updates
> its *.po files during package build, which is evil and unnecessary.
> But build systems which do that would already create different *.mo
> files on each build, so whether or not you update the *.pot as well
> should make little difference.

If the .pot has not changed, a .po update will be a null merge and generate
idempotent .mo output on all builds.

If the .pot has changed, you get at minimum a change to the timestamp field
in the header; which means each .mo file will be different.

Some build systems will automatically update the .po at build time if the
.pot has changed; so this is not safe behavior on the part of
pkgbinarymangler.

The canonical example for all of this is probably apt, which ships .mo files
in a library package.

> In the ordinary case, the build sytem just builds *.mo from the source
> *.po files, and completely ignores the *.pot. Most binary packages get
> their *.mo files stripped out anyway, and for the remaining ones we
> just need to make sure that we don't put *.mo files into arch specific
> packages (either packages are arch:all anyway, or we put translations
> into an arch:all -common binary, which is quite common for GNOME
> packages).

But that implies having to change the library packages which *don't* use a
-common to work around the behavior of pkgbinarymangler, which I don't think
is what we want.

Steve Langasek [2013-05-24 23:18 -0000]:
> If the .pot has not changed, a .po update will be a null merge and generate
> idempotent .mo output on all builds.

That's right, but most build systems don't automatically update .po
files (and rightly so, as that's rather pointless); also,
pkgbinarymangler works after the upstream build system, so changing
the .pot won't invoke any "make" rules any more.

> But that implies having to change the library packages which *don't* use a
> -common to work around the behavior of pkgbinarymangler, which I don't think
> is what we want.

This can only potentially affect the handful of packages which are
blacklisted from stripping (dpkg, apt, debconf-i18n, update-notifier,
and some stragglers). I don't think we ever had problems with any of
them, and together with the above (.pot build happens after upstream
build) it seems to me we are fairly safe here?