Many postinst scripts fail using either PackageKit, or QApt

So here's from today:

 Setting up kword-data (1:2.2.83-0ubuntu1) ...
 Setting up kexi (1:2.2.83-0ubuntu1) ...
 Setting up koffice-libs (1:2.2.83-0ubuntu1) ...
 Setting up kword (1:2.2.83-0ubuntu1) ...
 Setting up kspread (1:2.2.83-0ubuntu1) ...
 Processing triggers for libc-bin ...
 ldconfig deferred processing now taking place
 Log ended: 2010-11-22 07:22:14

 Log started: 2010-11-22 19:41:09
 Log ended: 2010-11-22 19:41:48

 Log started: 2010-11-22 20:27:24

This morning I installed kword, and the libc-bin postinst script caused an apparently abrupt end in the logging. The two installations I did tonight to get logs for this bug seem to have not been logged at all.

Not much to go on either way, but it should be fairly reproducible.

This happens with aptDaemon too, but I can't recall the bug number right now.
It seems to be related to the fact of not having a terminal output. But after some tests I and Matthias did we could not conclude anything yet.

Yeah, Synaptic actually has a terminal to run the APT package manager class in. The common denominator between AptDaemon, PackageKit and the QApt Worker all seem to be that they do not run in a terminal.

I have been unable to reproduce this in my tests. I've been using software-center, since that uses aptdaemon. I tried installing kword; you said "the libc-bin postinst script caused an apparently abrupt end in the logging", but that's simply the last thing that runs in that installation (libc-bin ships the ldconfig trigger, run at the end of most installations that include libraries). I also tried installing a random small unimportant package (knews - not a KDE package, despite the name) that I knew to use debconf, in case that was the determining factor, but I couldn't make it fail regardless of the setting of debconf/priority.

Could you please give me some kind of specific test sequence that I can run to reproduce this bug?

The actual problem is that the libc-bin postinst script always fails when it is triggered. The python-gmenu script does this too. Most post-installation triggers seem to fail when run via PackageKit, AptDaemon or QApt.

Package: python-gmenu
Error: subprocess installed post-installation script returned error exit status 1

I got this one by installing the "amor" package.

Hmm, and actually I can't personally get Software Center to reproduce this bug, but PackageKit and QApt-based package managers are definitely affected for me and several others.

Actually aptdaemon provides a controlling terminal for every dpkg call. If a terminal is provided by the user aptdaemon will act as a proxy.

Right, I can reproduce this with pkcon now.

As shown by strace, the failure is along the lines of this:

  write(1, "Rebuilding /usr/share/applications/bamf.index...\n", 49) = -1 EBADF (Bad file descriptor)

Not having a controlling terminal is one thing. Not even having a standard output file descriptor is quite another!

Looking earlier in the strace, apt is creating a pipe whose read end happens to land on fd 1 (it's later marked as FD_CLOEXEC, which causes dpkg to have no stdout at all, but the fact that the read end of the pipe is on fd 1 is an obvious red flag). This is probably mere chance: it's just the first available fd.

The fundamental bugs here are in PackageKit and QApt, so I'm going to reassign this bug. They should have some kind of stdout fd open when running apt functions, even if it's only a duplicate of /dev/null.

Incidentally, why does QApt have an obvious copy of code from PackageKit (compare WorkerInstallProgress::start in qapt/src/worker/workerinstallprogress.cpp with aptcc::installPackages in packagekit/backends/aptcc/apt.cpp), with no common copyright holders in the file headers? Aside from the general problems with copying code around - in this case, one of you has copied a bug from the other, so it's going to have to be fixed in both places - it seems rather naughty to copy code without preserving the required copyright notices.

I copied the code from Synaptic, actually, (PK probably did too) and I barely have a clue how it works or how to fix this. It all worked fine before natty. :(

There's another failure at least in PackageKit: it sets the locale to the result of setlocale(LC_ALL, ""), which with current libc is something like this:

  LC_CTYPE=en_GB.UTF-8;LC_NUMERIC=en_GB.UTF-8;LC_TIME=en_GB.UTF-8;LC_COLLATE=C;LC_MONETARY=en_GB.UTF-8;LC_MESSAGES=en_AG.utf8;LC_PAPER=en_GB.UTF-8;LC_NAME=en_GB.UTF-8;LC_ADDRESS=en_GB.UTF-8;LC_TELEPHONE=en_GB.UTF-8;LC_MEASUREMENT=en_GB.UTF-8;LC_IDENTIFICATION=en_GB.UTF-8

Setting LANG to that is ... not so helpful. It would be more helpful to get a single locale category. Perhaps ideally PackageKit ought to get each category separately and marshal them over to the backend; in the short term, perhaps LC_MESSAGES would be a good category to fetch, since PackageKit seems to generally use this for localisation of messages.

If it worked before natty, it was probably by luck. Running pretty much anything in Unix with a closed fd 1 will only work by luck at best.

Incidentally, I'm working on this - I have it installing packages successfully again, although I'm still investigating why /var/log/apt/term.log isn't being written properly. I'll send the changes to PackageKit upstream when I'm finished, and you can pick them up from there in cases where you've copied code.

Removing these lines seem to fix things:

// close Forked stdout and the read end of the pipe
close(1);

I wonder what it was trying to accomplish in the first place...

That's more or less what I have, although I also have a certain amount of other cleanup: it's good not to leak the pipe ends that we're not using.

term.log won't work until we set up a pty. I'll see if that's feasible.

Cool! Could you please shout if there are patches ready for PK? I'll ad them to the Debian packaging too then.
I moved the close(1) line around to "fix" this bug in APTcc, but I guess there's a better solution ;-)
Doesn't "setlocale(LC_ALL, "C")" set the locale back to English?

I sent my patches upstream for review. We should probably discuss them there rather than here if any discussion is needed. Here are archive links (the archive is a bit odd - the last three "messages" were actually attachments to the first one):

  http://lists.freedesktop.org/archives/packagekit/2011-January/006149.html
  http://lists.freedesktop.org/archives/packagekit/2011-January/006150.html
  http://lists.freedesktop.org/archives/packagekit/2011-January/006151.html
  http://lists.freedesktop.org/archives/packagekit/2011-January/006152.html

I've committed the patches upstream for QApt. Thanks for being so patient, and for fixing the code in both locations. :)

Fix committed for PackageKit in Debian. Waiting for response from Daniel to apply the patches upstream.

Thanks!

Matthias, regarding your comment on the upstream list (I have delivery disabled on the packagekit list, so didn't get a copy of your message, so this is the easiest way to reply ... sorry):

> I'll apply them to the Debian packaging, as well as 0002, but I don't know
> why the current LC_ALL approach is wrong...

I went into some detail in the patch itself, although not in the covering letter. The best way I can think of to put it is that setlocale(LC_ALL, NULL) *may* return a composite name ("category=value;category=value;..."), while setlocale(LC_anythingelse, NULL) *may not* return a composite name, and you may only give a composite name back as the value of LC_ALL, not as the value of any other locale category. PackageKit was trying to take the value of LC_ALL and set it as the value of LANG, which is not guaranteed to work.

Bumping the importance down a tad since it isn't as widespread as previously though.

Hi!
Colin, patches now committed upstream, they will therefore be in the upcoming release of PackageKit next week. I added the patches in Debian and the new Debian revision got uploaded.
Ubuntu might want to sync the package ^^ (There are also some Debian<->Ubuntu improvements, like removing the Iceweasel dependency on Ubuntu and the Firefox dep on Debian etc...) All Ubuntu changes have been merged into the Debian package.
Thanks for fixing this bug and for explaining the locale stuff to me!

sync filed on bug 707744

This bug was fixed in the package packagekit - 0.6.11-2

---------------
packagekit (0.6.11-2) unstable; urgency=low

  * Select right browser on Debian/Ubuntu
  * Reformatted control file
  * Fix crash if Section() of a package is NULL
  * Fix falure of some postinst scripts when running using APTcc (LP: #680328)
 -- Artur Rona <email address hidden> Wed, 26 Jan 2011 02:13:53 +0000

This bug was fixed in the package qapt - 1.1.0-0ubuntu2

---------------
qapt (1.1.0-0ubuntu2) natty; urgency=low

  * Add kubuntu_01_dont_close_stdout.diff from upstream to fix the failure
    of postinst scripts needing stdout. (LP: #680328)
 -- Jonathan Thomas <email address hidden> Wed, 26 Jan 2011 21:27:09 -0500