upstart unable to correctly track dovecot daemon when managesieve plugin is enabled

I am also suffering from the inability to restart dovecot (or stop it) using "sudo service dovecot restart" but unlike the original poster I do not have dovecot-postfix installed only dovecot-common and dovecot-imap.

Bug #603285 introduced the relevant change.

I have postfix and dovecot-postfix installed, but it will not update. This from the terminal:

Setting up dovecot-postfix (1:1.2.11-1ubuntu1) ...
Configuring postfix for dovecot-postfix integration: .................... done.
runlevel:/var/run/utmp: No such file or directory

I meant to add that I can't do normal updates, so this is a security issue as far as I'm concerned. I haven't found a way to not update dovecot-postfix, and it blocks all other updates. Running dpkg --configure -a just starts the dovecot-postfix update, and it just stops after the last line of my most recent post.

You can kill dovecot (sudo pkill dovecot) and it will resume package installation/configuration but dovecot will still not be running since it cannot be started with the present bug.

A reboot may be necessary -- this is how I bypassed the issue. I also found that killing dovecot's instances would not end the stop/start threads, and had to kill them also. Killing the stop thread allowed installation to proceed until dovecot was started again (which would hang); killing this start allowed the update to finish successfully. And then, just in case, I rebooted.

I had the same situation. I finish by rebooting to solve the 1st update lockout on the first occurrence. On the second, I just change conveniently the upstart script by the genuine one and the return code was OK for the update to complete.
In order to find a clean upstart-based solution, I try to switch to "expect daemon" in /etc/init/dovecot.conf to insure upstart being capable of tracking dovecot process, but I've got the same hangup problem.
Question is, is upstart capable of tracking program launched in a unprivileged way by the daemon ?
The only similar reference I'm thinking of is http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=582745 .

I did a fresh install of dovecot-postfix on Maverick, including:
dovecot-common 1:1.2.12-1ubuntu5
postfix 2.7.1-1

I had no problem restarting dovecot:
$ sudo service dovecot restart
dovecot start/running, process 2490

Or upgrading dovecot-postfix:
Unpacking replacement dovecot-postfix ...
Setting up dovecot-postfix (1:1.2.11-1ubuntu1) ...
Configuring postfix for dovecot-postfix integration: .................... done.
dovecot start/running, process 3015
 * Stopping Postfix Mail Transport Agent postfix [ OK ]
 * Starting Postfix Mail Transport Agent postfix [ OK ]

So it looks like it's not 100% reproducible, or only affects busy dovecot instances ?

The classic init script had a do_stop that would run twice in a row "start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --pidfile $PIDFILE --name ${DAEMON##*/}"... maybe that's why it succeeds in your case.

My maverick is an update of a fresh lucid 10.04.1 with dovecot+dovecot-postfix already installed (default settings).

Note for whoever would commit a fix for this, please fix the "dovoecot" typo in dovecot-common.dovecot.upstart while you are at it.

This is my proposed fix.

Fix is working in my case after a reboot. Start/stop multiple time with not hang state.

After more investigation it seems that the issue is triggered by managesieve: when managesieve is enabled dovecot forks multiple times which leads upstart to loose track of it. If managesieve is disabled upstart is correctly tracking dovecot.

The posted debdiff fixes the problem. Please upload to Maverick.

This bug was fixed in the package dovecot - 1:1.2.12-1ubuntu6

---------------
dovecot (1:1.2.12-1ubuntu6) maverick; urgency=low

  * debian/dovecot-common.dovecot.upstart: Fix for non-starting upstart job (LP: #646858)
 -- Chuck Short <email address hidden> Mon, 27 Sep 2010 11:45:50 -0400

It doesn't seem that managesieve is enabled by default and I haven't enabled it but am still subject to the bug.

I just installed a fresh 10.10 in a VM environment to check Thierry comment.
The problem by default does not occur because dovecot-postfix is now broken : if the config files are well installed for postfix, for dovecot there are created in sub directories (auth.d and conf.d) which are just plain ignored by the new default dovecot.conf. Therefore, no managesieve process nor dovecot-auth can be launched.

A patch could be to include the following lines :
!include /etc/dovecot/conf.d/*.conf
!include /etc/dovecot/auth.d/*.conf

at the end of dovecot.conf when installing dovecot-postfix.

And, not forget to include parsing of conf.d directory in the upstart /etc/init/doveconf package to look for imap4/pop3 protocols.

I continue my tests on a connected server. I had to complete the following steps to make dovecot-postfix to work :

- to work, the configuration in dovecot.conf has to be changed :
!include /etc/dovecot/conf.d/*.conf at the end of dovecot.conf
!include_try /etc/dovecot/auth.d/*.auth at the end of the auth default section of dovecot.conf

- this configuration is working because delivery in postfix is insured by calling dovecot with this precise configuration file, but this part is correctly configured by dovecot-postfix.

Excerpts from Yan Grunenberger's message of Mon Sep 27 22:40:11 UTC 2010:
> I just installed a fresh 10.10 in a VM environment to check Thierry comment.
> The problem by default does not occur because dovecot-postfix is now broken

Please file a new bug as this is not related to this specific bug.

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

Dear Mr. Jean-Baptiste,

  Thanks a lot.
  I am using squirrelmail on Ubuntu 10.10 now. Dovecot is working fine.

Regards,
Satish
(Col. SC Sood(Retd))
09216508708

________________________________
From: Jean-Baptiste Lallement <email address hidden>
To: <email address hidden>
Sent: Fri, 22 October, 2010 2:53:11 PM
Subject: [Bug 646858] Re: upstart unable to correctly track dovecot daemon when
managesieve plugin is enabled

** Tags added: maverick
** Tags removed: regression-potential

--
upstart unable to correctly track dovecot daemon when managesieve plugin is
enabled
https://bugs.launchpad.net/bugs/646858
You received this bug notification because you are a direct subscriber
of a duplicate bug (648605).

Status in “dovecot” package in Ubuntu: Fix Released
Status in “upstart” package in Ubuntu: New
Status in “dovecot” source package in Maverick: Fix Released
Status in “upstart” source package in Maverick: New

Bug description:

When the managesieve plugin is enabled it seems that dovecot forks once more
than what upstart expects. Upstart tracks the wrong pid with "expect fork" set.

============
Original description

Binary package hint: dovecot-postfix

Dovecot on Maverick is now using upstart to launch dovecot. When dovecot-postfix
is installed, the dovecot must have some component as root (for SASL
authentification purpose, in order to write to postfix chroot). Problem is, the
system can start dovecot on boot, but any attempt into using restart dovecot
will result in a hang of upstart. CTRL+C can cancel it, but then dovecot is in a
strange state, with half the processus presents in memory. This is not the
correct behavior, as we except a smooth restart of such an important service of
the mail (especially if SASL authentification of postfix is involved).

As a workaround, I replace the symlink to upstart by the old init.d original
script from ubuntu 10.04.1.
I try to figure out a clean solution, but apparently this is inherent to upstart
and explains why some package, such as apache, did not go on the upstart train
already.

To unsubscribe from this bug, go to:
https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/646858/+subscribe

Closed the upstart tasks, as this was a problem with dovecot not upstart.