groovy debootstrap leaves /e/d/motd-news.wasremoved around

Bug #1895302 reported by Andreas Hasenack
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
base-files (Ubuntu)
Fix Released
High
Andreas Hasenack
Xenial
Confirmed
Undecided
Unassigned
Bionic
Confirmed
Undecided
Unassigned
Focal
Confirmed
Undecided
Unassigned

Bug Description

[Impact]
A fresh install of base-files, like done when using debootstrap, using the base-files from the -updates repository (in the case of ubuntu stable releases), will leave an empty /etc/default/motd-news.wasremoved file. This file is an artifact of the mechanism used to handle a corner case in the previous SRU where it would signal the motd-news-config package to install /etc/default/motd-news with ENABLED=0. See testcases (h) and (i) in the previous base-files SRU at https://bugs.launchpad.net/ubuntu/+source/base-files/+bug/1888575 for details. In test case (i) it was acked that the empty .wasremoved file was lying around, but its impact was deemed not relevant (see [other info] item (a)).

Another case where /etc/default/motd-news.wasremoved would be created when it shouldn't be is when you have just base-files installed (and no ubuntu-server or motd-news-config) and did a reinstall of base-files, or an upgrade. It would again touch /etc/default/motd-news.wasremoved.

The consequence of having /etc/default/motd-news.wasremoved when it's unintended is that a follow-up install of ubuntu-server, or motd-news-config for that matter, will install /etc/default/motd-news with ENABLED=0 instead of ENABLED=1.

This was the case of the groovy debootstrap which resulted in this bug being filed. While debootstrap won't mix multiple repositories (like release with updates), and thus this isn't easily a problem in released versions of ubuntu, the groovy case was the one that was doing a fresh install of base-files with the buggy touch /etc/default/motd-news.wasremoved, and a subsequent install of ubuntu-server left motd-news disabled in groovy images produced by such a method (debootstrap).

These are the scenarios I was able to come up with in which a stable release could be affected by this bug:

a) debootstrap with release and updates pocket enabled
There are no config options that I'm aware of that would tell debootstrap to use multiple pockets when creating a chroot, but let's say it was done by hacking the script or something else. It would then be the same case as groovy until this fix: subsequent installations of ubuntu-server or motd-news-config would default to having motd-news disabled

b) A system that has just base-files from the previous SRU installed, and no ubuntu-server and no motd-news-config. If base-files were updated again and without the fix presented here (let's say, another SRU instead of this one), it would create /etc/default/motd-news.wasremoved, and again, a subsequent install of ubuntu-server or motd-news-config would install motd-news in a disabled state

c) Any other case where the postinst script of base-files is run again without the fix presented here, and when there is no /etc/default/motd-news{,.dpkg*} file present.

To avoid creating /etc/default/motd-news.wasremoved when we shouldn't, the maintainer scripts were changed as follows:
- motd-news-config postinst: always remove the .wasremoved file in configure if found, regardless if /etc/default/motd-news was sed'ed or not, or if we are upgrading or on a first install
- base-files postinst: guard the creation of .wasremoved with:
  - Only during an upgrade
  - Only if ubuntu-server is installed (via a dpkg -l check)

[Test Case]
* On the system under test, remove motd-news-config and ubuntu-server if they are installed, and keep base-files from the update pocket. Something like this:
sudo apt update && sudo apt dist-upgrade -y
sudo apt purge motd-news-config ubuntu-server
apt-cache policy base-files <-- to verify it's from updates

* In this scenario, you should have no /etc/default/motd* files:
$ ll /etc/default/motd*
ls: cannot access '/etc/default/motd*': No such file or directory

* reinstall base-files:
sudo apt install --reinstall base-files

* Before this SRU, this would create /etc/default/motd-news.wasremoved:
$ ll /etc/default/motd*
-rw-r--r-- 1 root root 0 Sep 16 20:24 /etc/default/motd-news.wasremoved

With the package from proposed for this SRU installed, no such file is created:
$ ll /etc/default/motd*
ls: cannot access '/etc/default/motd*': No such file or directory

[Regression Potential]
This SRU is further changing maintainer scripts, to address an issue in the previous maintainer script. Should there be new regressions or new issues, it might get harder and harder to fix them.

I did wonder about the `dpkg -l` call in base-files' postinst. I worried about locks, or pre-dependencies, but dpkg was already being used in this script, although not with -l (list), just a version comparison. But at least it's already installed.

My other worry was with the dpkg -l output and which flags I should check to determine if ubuntu-server was installed. "^ii" doesn't work, because ubuntu-server might be being upgraded in the same apt transaction, but ^i seemed good enough. Furthermore, I changed the check for a package name that doesn't exist, to see if dpkg -l would complain and fail the whole script (which runs with set -e), but it was ok and just behaved as if the package wasn't installed, which is what we need.

[Other Info]
The previous SRU at https://bugs.launchpad.net/ubuntu/+source/base-files/+bug/1888575 and its attached merge proposals has more detailed info about the history of this motd-news split, and things that were considered.

Manually, I used debootstrap with --make-tarball, replace the base-files deb, and --unpack-tarball, to simulate debootstrap with an updated base-files, to confirm this change here would fix the debootstrap problem, and it worked. But I deemed it too complicated to describe as a testcase for this SRU. If you, dear SRU reviewer, would prefer this test to be added, please let me know.

If a user of a stable release already has an unintended /etc/default/motd-news.wasremoved file created, this update will not fix that.

[Original Description]

When debootstrapping groovy, we see an empty /etc/default/motd-news.wasremoved file.

- groovy: base-files 11ubuntu12
-rw-r--r-- 1 root root 0 set 11 10:20 /etc/default/motd-news.wasremoved

If motd-news-config is later installed, maybe via ubuntu-server, then the presence of this file will disable motd-news by default, which is unintended as it's meant to be enabled on a server.

Related branches

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I think this postinst bit never considered the fresh install case:

# special case of having /etc/default/motd-news removed by hand
# signal the motd-news-config package that this happened, so that
# it does not put back the file with default contents which would
# re-enable motd-news
motd_news_config="/etc/default/motd-news"
if [ ! -e ${motd_news_config} ]; then
  if [ ! -e ${motd_news_config}.dpkg-remove ]; then
    if [ ! -e ${motd_news_config}.dpkg-backup ]; then
      touch ${motd_news_config}.wasremoved
    fi
  fi
fi

Having it only act on upgrades might be the easier fix. I'll test.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Nor the upgrade after that, ugh. Fixing.

Changed in base-files (Ubuntu):
importance: Undecided → High
assignee: nobody → Andreas Hasenack (ahasenack)
status: New → In Progress
description: updated
Changed in base-files (Ubuntu Focal):
assignee: nobody → Andreas Hasenack (ahasenack)
status: New → In Progress
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
Changed in base-files (Ubuntu Xenial):
assignee: nobody → Andreas Hasenack (ahasenack)
Changed in base-files (Ubuntu Bionic):
assignee: nobody → Andreas Hasenack (ahasenack)
Changed in base-files (Ubuntu Xenial):
status: New → In Progress
Changed in base-files (Ubuntu Bionic):
status: New → In Progress
description: updated
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package base-files - 11ubuntu13

---------------
base-files (11ubuntu13) groovy; urgency=medium

  * Fix handling of /e/d/motd-news.wasremoved (LP: #1895302):
    - d/postinst.in: check if ubuntu-server is installed before creating
      the .wasremoved file
    - d/postinst.in: re-arrange the cascading conditionals around the
      creation of the .wasremoved file for better clarity
    - d/postinst.in: only consider creating the .wasremoved file in upgrades,
      never fresh installs like in debootstrap for example
    - d/motd-news-config.postinst: always remove /e/d/motd-news.wasremoved
      if present

 -- Andreas Hasenack <email address hidden> Wed, 16 Sep 2020 10:26:50 -0300

Changed in base-files (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote :

debootstrap never installs packages from updates: due to a longstanding design limitation, it can only install from a single pocket, so debootstrap always uses the release pocket and images are upgraded afterwards.

Given this limitation, is there actually any impact to users of stable releases that would justify SRU?

Changed in base-files (Ubuntu Focal):
status: In Progress → Incomplete
Changed in base-files (Ubuntu Xenial):
status: In Progress → Incomplete
Changed in base-files (Ubuntu Bionic):
status: In Progress → Incomplete
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I think the remaining scenario is (b) (look up in the bug description), and (c) is just a wildcard scenario ("anything else I didn't think of").

For (b), I think the more likely case would be another base-files SRU without the fix from this bug here, and where the user would have:
- base-files installed
- motd-news-config NOT installed
- ubuntu-server NOT installed

In that scenario, the base-files postinst would create /etc/default/motd-news.wasremoved, with the consequence that a follow-up install of ubuntu-server/motd-news-config would install the motd-news config with ENABLED=0 instead of ENABLED=1.

It's also possible the above would happen on a release upgrade, if only base-files is installed to begin with. That would ugrade base-files to the version in the next release, effectively behaving like just a base-files upgrade, and create the .wasremoved file as well.

Changed in base-files (Ubuntu Xenial):
status: Incomplete → In Progress
Changed in base-files (Ubuntu Bionic):
status: Incomplete → In Progress
Changed in base-files (Ubuntu Focal):
status: Incomplete → In Progress
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I retried testcase (b) in an up-to-date focal, and it still happens. It's been a long while since I touched this package and I don't remember the details anymore. Since I'm no longer working on this, I'll mark the bug status accordingly.

Changed in base-files (Ubuntu Xenial):
status: In Progress → Confirmed
Changed in base-files (Ubuntu Bionic):
status: In Progress → Confirmed
Changed in base-files (Ubuntu Focal):
status: In Progress → Confirmed
Changed in base-files (Ubuntu Xenial):
assignee: Andreas Hasenack (ahasenack) → nobody
Changed in base-files (Ubuntu Bionic):
assignee: Andreas Hasenack (ahasenack) → nobody
Changed in base-files (Ubuntu Focal):
assignee: Andreas Hasenack (ahasenack) → nobody
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Please reject the xenial upload.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.