dangerous action: dovecot-postfix force-installs new conf file

Hi,

Thank you for taking the time to report this bug and helping to make
Ubuntu better.

On Mon, Mar 09, 2009 at 02:57:16PM -0000, C de-Avillez wrote:
> Package Version: dovecot-postfix_1%3a1.1.11-0ubuntu2_all.deb
> Expected Results: current dovecot configuration will stay active, user will be warned to match & merge new configuration
> Actual Results: existing configuration replaced by new dovecot-postfix.cfg.
>
> I already run dovecot, and have it configured to use IMAP. After
> installing dovecot-postifx, dovecot was re-initialised, and defaulted to
> /etc/dovecot/dovecot-postfix.cfg (out of the default, and correctly
> configured my my use) /etc/dovecot/dovecot.cfg.

Technically your customized dovecot.cfg file hasn't been overwritten
since the dovecot-postfix package installs a new configuration file
(dovecot-postfix.cfg). However the new configuration file
(dovecot-postfix.cfg) takes precedence over the standard one
(dovecot.cfg) when dovecot is started from the init script. Thus the
configuration of the system has been changed.

> Immediate mayhem ensured on my email clients. Even more painful,
> Evolution immediately reconfigured all email filters to access the new
> IMAP -- which is to say, none, since I could not login to dovecot
> anymore. Uninstalling dovecot-postfix did not resolve it; uninstalling
> --purge did, followed by manually resetting all email filters on
> Evolution.
>
> Instead dovecot-postfix installation should not replace a currently in-
> use dovecot.conf, but suggest a match & merge.
>

According to the dovecot-postfix description:

  Ubuntu's mail stack provides fully operational mail server with safe
  defaults and additional options. Out of the box it supports IMAP, POP3
  and SMTP services with SASL authentication and Maildir as default
  storage engine.

  This package contains configuration file for dovecot.

What were your expectations when installing the dovecot-postfix package?
I'm not sure we can support upgrades from an existing running dovecot
system.

  status incomplete

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

The problem is that on already-configured dovecot installations there is a chance that the dovecot-postifx.conf will not work out-of-the-box. I am sure that for *new* installations of dovecot the provided d-p.conf will work fine -- but this is not to be expected for existing installs.

But ideally -- on existing & customised dovecot installs --, the new configuration should be installed as -- say -- dovecot-postfix.conf.dpkg.inst, or similar, giving time for the system maintainer to check and adjust as needed.

The newly-installed d-p.conf was broken, for me:

1. replaced my SSL certs by a default SSL cert;
2. disabled SSL client authentication;
3. reset user name formatting to the default (user@realm, instead of just username);
4. disabled MD5 authentication.

These are small changes, but the result was -- for example -- the loss of all Evolution mail filters for all clients, since nobody could connect to IMAP anymore.

If, instead, the new dovecot configuration was created *without* being set active (and a warning given that dovecot-postfix integration was not yet active), I would have had time to adjust, check & set the integration without causing any users any pain.

I have since reinstalled dovecot-postifx, but now I knew the install was going to be broken, so I took the necessary care.

If no such option (*not* to blindly replace existing dovecot configuration) can be be provided, then at least the installation procedure should provide an option to be cancelled.

Thanks for taking the time to reply.

On Mon, Mar 09, 2009 at 06:40:07PM -0000, C de-Avillez wrote:
> The problem is that on already-configured dovecot installations there is
> a chance that the dovecot-postifx.conf will not work out-of-the-box. I
> am sure that for *new* installations of dovecot the provided d-p.conf
> will work fine -- but this is not to be expected for existing installs.

All of your points are valid and your suggestions are welcomed. However
I'm trying to understand why did you install the dovecot-postfix package
in the first place? Were you expecting that it would *upgrade* your
existing system configuration and add new features to it?

  status incomplete

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

heh. Because I thought (wrongly, perhaps) that integration between Dovecot and postfix would be nice...

Of course (and no need to rub it even more on my face, it is already smeared all over ;-), I should have been more careful, and installed it first on a test machine.

I wasn't, and I didn't. And this is clear from the fact that I opened this bug <ashamed grin/>.

So, this time I 'dpkg -x' the package on a temp space, and looked at the provided configuration, and at the post-inst scripts. Then I prepared myself an adjusted dovecot-postfix.conf, installed d-p, stopped dovecot again, replaced the provided d-p with my version, and restarted -- successfully -- dovecot.

On Tue, Mar 10, 2009 at 02:23:01AM -0000, C de-Avillez wrote:
> heh. Because I thought (wrongly, perhaps) that integration between
> Dovecot and postfix would be nice...

Ok - thanks for making this clear. It seems that the description of the
package could be updated to better explain what the package is about.

I don't think supporting upgrades from an existing dovecot installation
can be implemented for the jaunty release cycle.

  status confirmed
  importance low

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

I suggest we check if dovecot.conf (minus ^protocols.* line) is different than the default one (minus ^protocols.* line) and, if it is, in preinst we copy that file into dovecot-postfix.conf and let the ucf do the rest. What do you think?

This is proposed solution:

- check if dovecot.conf is customized and, if it is, copy it to dovecot-postfix.conf
- ucf will notice diff and propose merging or overwriting to the user

This patch also includes fix for outlook SASL-AUTH.

Changelog entry:

Add SMTP-AUTH support for Outlook (login auth mechanism)

isn't quite usefull. This would be better:

dovecot-postfix: add login auth mechanism (required by Outlook)

Thanks for the debdiff. The patch looks promising:

1. The temporary directory DIR is not removed.

2. All of the ucf logic is *only* run during the first install. A dovecot-postfix package upgrade won't run the ucf logic and thus maintainer changes to dovecot-postfix.conf won't be presented to the administrator.

Right, (2) is an error from previous upload and (1) is an oversight (i didn't copy-paste it :). I'm fixing that and at the same time I would like to discuss some things.

What would user expect on purge/removal of dovecot-postfix? At the moment, if the user removes dovecot-postfix, /etc/dovecot/dovecot-postfix.conf remains and nothing changes for the user. Is that ok? Wouldn't the user expect to have dovecot-postfix integration removed?

And how about purge?

I've would also like to have postfix's master.cf and main.cf (in the state they are before installing dovecot-postfix) in /var/backups/dovecot/, so that on purge we can set up environment that was there before installation.

This patch covers all situations I could think off:
- install of dovecot-postfix on top of already working dovecot and postfix
  - removing dovecot-postfix in this case would bring back configuration of postfix from before installation and would rename dovecot-postfix.conf, so that real dovecot.conf would be used
- removing dovecot-postfix doesn't delete user made changes, but it does remove postfix-dovecot integration (postfix's configuration is reset to the state before the package was installed)
  - on reinstall of dovecot-postfix user made changes are returned and diff against default configuration is provided to the user; integration is back again
- purging dovecot-postfix removes all dovecot related configuration from dovecot-postfix package and resets postfix's configuration to the state before the package was installed
- upgrade of dovecot-postfix

This bug was fixed in the package dovecot - 1:1.1.11-0ubuntu3

---------------
dovecot (1:1.1.11-0ubuntu3) jaunty; urgency=low

  [ Ante Karamatic ]
  * Add SMTP-AUTH support for Outlook (login auth mechanism)
  * debian/dovecot-postfix.postinst:
    - merge user's custom changes from dovecot.conf (LP: #339966)

  [ Mathias Gug ]
  * properly support package upgrades.
  * handle reinstallation of package if it wasn't purged. Based on Ante's
    patch:
    + debian/dovecot-postfix.postinst, debian/dovecot-posftix.postrm:
      - rename configuration name on remove, delete on purge
    + debian/dovecot-postfix.dirs
      - create backup directory for postfix's configuration

 -- Mathias Gug <email address hidden> Tue, 31 Mar 2009 18:43:06 -0400