tomcat8 SRU for bionic uses systemd service instead of init scripts

A community user reported a startup delay for tomcat8, guessing missing entropy at the start?

This change has the potential to be a showstopper in some cases.

We're managing Tomcat on a lot of servers using Puppet. Puppet currently installs slightly modified /etc/init.d/tomcat8 and /etc/default/tomcat8 files. The changed init.d file adds support for e.g. reading a /etc/default/tomcat8.local file if it exists (in addition to /etc/default/tomcat8), making it possible to make temporary and/or local changes to certain environment variables used during Tomcat startup without having to add that change to Puppet as well. After updating to Tomcat 8.5.39-1ubuntu1~18.04 currently in proposed, this override no longer works, I guess because /etc/init.d/tomcat8 isn't used at all any more.

Making these kind of intrusive changes one year after an LTS release clearly damages the trust users have in Ubuntu LTS releases being stable across its lifetime.

Status changed to 'Confirmed' because the bug affects multiple users.

@Mathias - do you consider that a blocker that makes you re-modify the ongoing SRU in bug 1817567 ?

I don't see a SRU Template or regression potential discussion here - even thou being only released to -security would that be needed and address these concerns?

I'm flagging as regression-proposed to make clear what https://bugs.launchpad.net/ubuntu/+source/tomcat8/+bug/1819721/comments/2 is until fixed or sorted out by discussion.

Sebastian,

thank you for taking the time to report this bug and helping to make Ubuntu better.

For a systemd service you can easily override it and to modify/add options. As an example, you should be able to achieve a similar result to what you describe by using:

/etc/systemd/system/tomcat.service.d# cat override.conf
[Service]
EnvironmentFile=-/etc/default/tomcat8.local

Please see https://www.freedesktop.org/software/systemd/man/systemd.exec.html#EnvironmentFile= for a description on the expected file format.

If that solution is not enough, could you please describe why or what is it missing?

Tiago,

Thank you for helping out!

Unfortunately, that doesn't work out of the box in our case. The reason is how our tomcat8.local currently is used. We have some sane defaults in /etc/default/tomcat8 that Puppet manages, and use /etc/default/tomcat8.local to make it possible to override it locally like this:

$ cat /etc/default/tomcat8.local
JAVA_OPTS="${JAVA_OPTS} -Xmx3g"

It seems like systemd doesn't like that line at all, tomcat fails to start after creating an override.conf as you suggested:

$ sudo systemctl status tomcat8
● tomcat8.service - Apache Tomcat 8.5 Web Application Server
   Loaded: loaded (/lib/systemd/system/tomcat8.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/tomcat8.service.d
           └─override.conf
   Active: failed (Result: exit-code) since Tue 2019-04-09 07:28:51 UTC; 5s ago
     Docs: https://tomcat.apache.org/tomcat-8.5-doc/index.html
  Process: 5788 ExecStart=/bin/sh /usr/libexec/tomcat8/tomcat-start.sh (code=exited, status=1/FAILURE)
  Process: 5777 ExecStartPre=/usr/libexec/tomcat8/tomcat-update-policy.sh (code=exited, status=0/SUCCESS)
 Main PID: 5788 (code=exited, status=1/FAILURE)

apr 09 07:28:51 ubuntu1804base systemd[1]: Starting Apache Tomcat 8.5 Web Application Server...
apr 09 07:28:51 ubuntu1804base systemd[1]: Started Apache Tomcat 8.5 Web Application Server.
apr 09 07:28:51 ubuntu1804base tomcat8[5788]: Error: Could not find or load main class ${JAVA_OPTS}
apr 09 07:28:51 ubuntu1804base systemd[1]: tomcat8.service: Main process exited, code=exited, status=1/FAILURE
apr 09 07:28:51 ubuntu1804base systemd[1]: tomcat8.service: Failed with result 'exit-code'.

There are probably other ways to achieve what we currently do with tomcat8.local using systemd, but having to convert all our currently deployed /etc/default/tomcat8.local files would require a lot of manual work. In addition to that, we might have to introduce a new Puppet class for Ubuntu 18.04 + Tomcat 8 and use that one for the servers that have an updated Tomcat 8, and let the old one be used on all servers without an updated Tomcat 8.

I'm still of the opinion that this kind of change in the middle of an LTS cycle is far too intrusive.

@Sebastian (slovdahl)

Well, you have customized init.d scripts and added local features to it, which is not something that Ubuntu can know about, nor has to support. There are obvious ways how you can continue to support those things in an automated manner. For example, you can dpkg-divert systemd units to prevent them from being used and init.d scripts used instead.

Also, one can write a systemd generator that would parse the /etc/default/tomcat8 and /etc/default/tomcat8.local and produce appropriate override (i.e. /run/systemd/system/tomcat8.service to override everything, or only just /run/systemd/system/tomcat8.service.d/generator.conf)

Imho, it would be nice to continue support the older /etc/default/tomcat8 settings as much as possible.

@slovdahl can you please share your modifications to the init.d script and what sort of things you are setting in both /etc/default/tomcat8 and tomcat8.local?

That is for the future.

W.r.t. SRU, i don't know if it is ok to do init.d -> systemd switch in an SRU. And if it's ok to do it without support existing (GA-time) config files, or not.

based on discussion with Tiago this afternoon, I understand we have agreement to revert the addition of the systemd unit in this SRU in order to maximize compatibility with existing local configuration changes.

Steve, thank you, that sounds great from our perspective!

Dimitri, I would have no objections against a change like this in a new version of Ubuntu. That said, I do understand that I'm not entitled to set requirements for something I'm not paying for.

FWIW, the change we have to the Tomcat 8 init.d script is included here:

$ diff -u tomcat8 tomcat8-modified
--- tomcat8 2019-04-10 11:25:10.280954611 +0000
+++ tomcat8-modified 2019-04-10 11:28:26.551407512 +0000
@@ -79,6 +79,11 @@
  . "$DEFAULT"
 fi

+# overwrite with host specific default file if such exists
+if [ -f "$DEFAULT.local" ]; then
+ . "$DEFAULT.local"
+fi
+
 if [ ! -f "$CATALINA_HOME/bin/bootstrap.jar" ]; then
  log_failure_msg "$NAME is not installed"
  exit 1

Hello Matthias, or anyone else affected,

Accepted tomcat8 into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/tomcat8/8.5.39-1ubuntu1~18.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Matthias, or anyone else affected,

Accepted tomcat8 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/tomcat8/8.5.39-1ubuntu1~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

This bug was fixed in the package tomcat8 - 8.5.39-1ubuntu1~18.04.1

---------------
tomcat8 (8.5.39-1ubuntu1~18.04.1) bionic; urgency=medium

  [ Matthias Klose ]
  * Backport for OpenJDK 11. LP: #1817567.
    /usr/share/doc/tomcat8/NEWS.gz. LP: #1819721.

  [ Tiago Stürmer Daitx ]
  * debian/tomcat8.service: removed, use the init.d script instead.
    LP: #1819721.
  * debian/tomcat8.init, debian/logging.properties: revert back to the
    conffiles from the previous version; this allows unattended-upgrades
    to update tomcat8 even when local changes are present.
  * debian/series: no longer apply 0023-disable-shutdown-by-socket.patch
    so server.xml conffile is unmodified from previous version.

tomcat8 (8.5.39-1ubuntu1) disco; urgency=medium

  * Merge with Debian; remaining changes:
    - d/control: Break/replace tomcat8.0 binaries.

tomcat8 (8.5.39-1) experimental; urgency=medium

  * Team upload.
  * New upstream release
    - Refreshed the patches
  * Track and download the new releases from GitHub

tomcat8 (8.5.38-2ubuntu1) disco; urgency=medium

  * Merge with Debian; remaining changes:
    - d/control: Break/replace tomcat8.0 binaries.

tomcat8 (8.5.38-2) unstable; urgency=high

  * Team upload.
  * Apply upstream patch to unbreak the startup script (Closes: #922863)

tomcat8 (8.5.38-1ubuntu1) disco; urgency=medium

  * Merge with Debian unstable (LP: #1815601). Remaining changes:
    - d/control: Break/replace tomcat8.0 binaries. (LP: 1717998)
  Dropped Changes:
    - support-jre8.diff.

tomcat8 (8.5.38-1) unstable; urgency=medium

  * Team upload.
  * New upstream release
    - Refreshed the patches

tomcat8 (8.5.37-2) unstable; urgency=medium

  * Team upload.
  * No longer build the JavaEE API packages
  * Standards-Version updated to 4.3.0

tomcat8 (8.5.37-1) unstable; urgency=medium

  * Team upload.
  * New upstream release
    - Refreshed the patches

tomcat8 (8.5.35-3) unstable; urgency=medium

  * Team upload.
  * Split libservlet3.1-java into separate JavaEE API packages
    (libjsp-api-java, libel-api-java and libwebsocket-api-java)
  * Updated the version required for libtcnative-1 (>= 1.2.18)
  * Install the Russian translation added in Tomcat 8.5.33

tomcat8 (8.5.35-2) unstable; urgency=medium

  * Team upload.
  * Fixed the build failure with Easymock 4 (Closes: #913402)

tomcat8 (8.5.35-1) unstable; urgency=medium

  * Team upload.

  [ Thomas Opfer ]
  * Removed old version requirement for package ant-optional that is not
    required any more.

  [ Emmanuel Bourg ]
  * New upstream release
    - Refreshed the patches

tomcat8 (8.5.34-1ubuntu1) cosmic; urgency=medium

  * Merge from Debian unstable. Remaining changes:
    - control: Break/replace tomcat8.0 binaries. (LP: #1717998)
    - support-jre8.diff.

tomcat8 (8.5.34-1) unstable; urgency=medium

  * Team upload.
  * New upstream release
    - Refreshed the patches

tomcat8 (8.5.33-1) unstable; urgency=medium

  * Team upload.
  * New upstream version 8.5.33.
    - Tomcat compiles to Java 7 bytecode and passes release=7 to javac now.
      This ensures backwards compatibility with older JREs. (Closes: #906447)
  * Declare compliance with Debian Policy 4.2....

This bug was fixed in the package tomcat8 - 8.5.39-1ubuntu1~18.10

---------------
tomcat8 (8.5.39-1ubuntu1~18.10) cosmic; urgency=medium

  [ Matthias Klose ]
  * Backport for OpenJDK 11. LP: #1817567.
  * tomcat8 now uses systemd service instead of init scripts. See
    /usr/share/doc/tomcat8/NEWS.gz. LP: #1819721.

tomcat8 (8.5.39-1ubuntu1) disco; urgency=medium

  * Merge with Debian; remaining changes:
    - d/control: Break/replace tomcat8.0 binaries.

tomcat8 (8.5.39-1) experimental; urgency=medium

  * Team upload.
  * New upstream release
    - Refreshed the patches
  * Track and download the new releases from GitHub

tomcat8 (8.5.38-2ubuntu1) disco; urgency=medium

  * Merge with Debian; remaining changes:
    - d/control: Break/replace tomcat8.0 binaries.

tomcat8 (8.5.38-2) unstable; urgency=high

  * Team upload.
  * Apply upstream patch to unbreak the startup script (Closes: #922863)

tomcat8 (8.5.38-1ubuntu1) disco; urgency=medium

  * Merge with Debian unstable (LP: #1815601). Remaining changes:
    - d/control: Break/replace tomcat8.0 binaries. (LP: 1717998)
  Dropped Changes:
    - support-jre8.diff.

tomcat8 (8.5.38-1) unstable; urgency=medium

  * Team upload.
  * New upstream release
    - Refreshed the patches

tomcat8 (8.5.37-2) unstable; urgency=medium

  * Team upload.
  * No longer build the JavaEE API packages
  * Standards-Version updated to 4.3.0

tomcat8 (8.5.37-1) unstable; urgency=medium

  * Team upload.
  * New upstream release
    - Refreshed the patches

tomcat8 (8.5.35-3) unstable; urgency=medium

  * Team upload.
  * Split libservlet3.1-java into separate JavaEE API packages
    (libjsp-api-java, libel-api-java and libwebsocket-api-java)
  * Updated the version required for libtcnative-1 (>= 1.2.18)
  * Install the Russian translation added in Tomcat 8.5.33

tomcat8 (8.5.35-2) unstable; urgency=medium

  * Team upload.
  * Fixed the build failure with Easymock 4 (Closes: #913402)

tomcat8 (8.5.35-1) unstable; urgency=medium

  * Team upload.

  [ Thomas Opfer ]
  * Removed old version requirement for package ant-optional that is not
    required any more.

  [ Emmanuel Bourg ]
  * New upstream release
    - Refreshed the patches

 -- Tiago Stürmer Daitx <email address hidden> Wed, 10 Apr 2019 05:33:03 +0000