Merge ~ahasenack/ubuntu/+source/base-files:xenial-motd-news-config-split into ubuntu/+source/base-files:ubuntu/xenial-devel

[Andreas Hasenack]

Same as https://code.launchpad.net/~ahasenack/ubuntu/+source/base-files/+git/base-files/+merge/388835, but for xenial, with the following extra details:

- backport the switch to wget, and follow-up fixes that landed in groovy
- far from just a detail: the xenial package does not use debhelper, so I had to do many things by hand in d/rules
- versions for breaks/replaces are the xenial ones:

ubuntu-server 1.361.5:
  Depends: motd-news-config

base-files 9.4ubuntu4.13:
  Breaks: ubuntu-server (<< 1.361.5)
  rm_conffile /etc/default/motd-news 9.4ubuntu4.13~ base-files

motd-news-config 9.4ubuntu4.13:
  Breaks/Replaces: base-files (<< 9.4ubuntu4.13)

PPA: https://launchpad.net/~ahasenack/+archive/ubuntu/zomg-5/

The linked bugs have the SRU template filled out.

[Andreas Hasenack (ahasenack)]

I pushed a change I found out while testing the groovy package: motd-news-config has to have priority set to optional, otherwise it will inherit base-files' which is "required" and do-release-upgrade will install it regardless of dependencies.

[Christian Ehrhardt  (paelzer)]

Change matches what we had in Groovy and soon Focal.
Additional change for priority is good as well.
Versions are good as well for an SRU and the bug LGTM as well.

The breaks and rm_conffile versions are adapted correctly, but we really need to land the meta change at the same time to avoid some updaters to remove packages or even ubuntu-server.

+1 to this in general, but I found one triviality to fix.
The changelog entries got carried over without line-wrap. So you violate the 80 chars in two lines and tools like lintian will hate it.

Fix is trivial enough, you don't need a re-review for the CL changes.

I was wondering at first why the different rm_conffile calls - but I see that adding debian/base-files.maintscript with the one line we need would have had no effect as he maintainer scripts have no #DEBHELPER# template and adding it int he SRU might change more than we want. I double checked if these three hooks should be enough - but they match the rm_conffile code well. This adds some noise, but the new lines LGTM - if your testing is good as well that should be fine.

+1 to this (condition, adapt the changelog)

review: Approve

[Andreas Hasenack (ahasenack)]

> I was wondering at first why the different rm_conffile calls - but I see that
> adding debian/base-files.maintscript with the one line we need would have had
> no effect as he maintainer scripts have no #DEBHELPER# template

Not just that, debhelper isn't used at all in d/rules. I followed the dpkg-maintscript-helper manpage which says that for rm_conffile only preinst, postinst and postrm are used

[Andreas Hasenack (ahasenack)]

changelog fixed

[Andreas Hasenack (ahasenack)]

Tagging and uploading c91565c6eb7c49f151ab7fff5a1cb9e350224e04

$ git push pkg upload/9.4ubuntu4.13
Enumerating objects: 64, done.
Counting objects: 100% (64/64), done.
Delta compression using up to 4 threads
Compressing objects: 100% (49/49), done.
Writing objects: 100% (54/54), 8.46 KiB | 2.82 MiB/s, done.
Total 54 (delta 38), reused 7 (delta 4)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/base-files
 * [new tag] upload/9.4ubuntu4.13 -> upload/9.4ubuntu4.13

$ dput ubuntu ../base-files_9.4ubuntu4.13_source.changes
Checking signature on .changes
gpg: ../base-files_9.4ubuntu4.13_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../base-files_9.4ubuntu4.13.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading base-files_9.4ubuntu4.13.dsc: done.
  Uploading base-files_9.4ubuntu4.13.tar.xz: done.
  Uploading base-files_9.4ubuntu4.13_source.buildinfo: done.
  Uploading base-files_9.4ubuntu4.13_source.changes: done.
Successfully uploaded packages.

[Andreas Hasenack (ahasenack)]

Also uploading https://code.launchpad.net/~ahasenack/ubuntu/+source/ubuntu-meta/+git/ubuntu-meta/+ref/xenial-ubuntu-server-motd-news

The seed change will also happen, but since it involves a new package, the update script from ubuntu-meta won't pick it up, so I have to add the dep manually.

Tagging and uploading d8fb428bfd5b6662e3408bdbb25121e844eb7c78 of that branch:

$ git push pkg upload/1.361.5
Enumerating objects: 13, done.
Counting objects: 100% (13/13), done.
Delta compression using up to 4 threads
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.13 KiB | 46.00 KiB/s, done.
Total 9 (delta 5), reused 0 (delta 0)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/ubuntu-meta
 * [new tag] upload/1.361.5 -> upload/1.361.5

$ dput ubuntu ../ubuntu-meta_1.361.5_source.changes
Checking signature on .changes
gpg: ../ubuntu-meta_1.361.5_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../ubuntu-meta_1.361.5.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading ubuntu-meta_1.361.5.dsc: done.
  Uploading ubuntu-meta_1.361.5.tar.xz: done.
  Uploading ubuntu-meta_1.361.5_source.buildinfo: done.
  Uploading ubuntu-meta_1.361.5_source.changes: done.
Successfully uploaded packages.

[Andreas Hasenack (ahasenack)]

Seeds changed for xenial in 2c4f57772b95223c86ac23afeb7e8034488c95c7 after ok from steve.

diff --git a/debian/changelog b/debian/changelog
index 3be5a47..cf951ad 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,38 @@
+base-files (9.4ubuntu4.13) xenial; urgency=medium
+
+  [ Andreas Hasenack ]
+  * motd/50-motd-news: don't include uptime in the user-agent string
+    (LP: #1886572)
+  * Move the /etc/default/motd-news conffile to the motd-news-config
+    package (LP: #1888575):
+    - d/postinst.in, d/postrm, d/preinst: remove /etc/default/motd-news
+      config file on base-files upgrade using dpkg-maintscript-helper
+    - d/rules: install d/preinst
+    - d/control: break on ubuntu-server << 1.361.5 to force an upgrade if
+      it is installed, which will pull motd-news-config and the conffile
+      back in
+    - d/control: new motd-news-config package, carrying the
+      configuration file for the /etc/update-motd.d/50-motd-news script.
+    - d/motd-news-config.postinst:
+      + handle the upgrade case where the motd-news config file was
+        changed while it belonged to base-files
+      + disable motd-news if the config file was removed by hand before
+        the upgrade
+    - d/postinst.in: signal the motd-news-config package if the
+      motd-news config file was removed manually before the upgrade
+    - d/conffiles: remove motd-news
+    - d/rules, d/motd-news-config.conffiles: packaging motd-news-config
+      without debhelper
+
+  [ Steve Langasek ]
+  * motd/50-motd-news: use wget instead of curl, since wget is standard but
+    curl is optional (LP: #1888572):
+    - This changes the timeout behavior slightly because wget does not have
+      an exact equivalent to curl's --max-time argument, we are using
+      --timeout instead.
+
+ -- Andreas Hasenack <[e-mail]>  Mon, 17 Aug 2020 11:19:19 -0300
+
 base-files (9.4ubuntu4.12) xenial; urgency=medium
 
   * /etc/issue, /etc/issue.net, /etc/lsb-release, /etc/os-release: Bump
diff --git a/debian/conffiles b/debian/conffiles
index 6600468..748ffb5 100644
--- a/debian/conffiles
+++ b/debian/conffiles
@@ -1,5 +1,4 @@
 /etc/debian_version
-/etc/default/motd-news
 /etc/dpkg/origins/debian
 /etc/dpkg/origins/ubuntu
 /etc/host.conf
diff --git a/debian/control b/debian/control
index 49ad4f0..a8c39a2 100644
--- a/debian/control
+++ b/debian/control
@@ -12,7 +12,7 @@ Pre-Depends: awk
 Essential: yes
 Priority: required
 Replaces: base, miscutils, dpkg (<= 1.15.0)
-Breaks: initscripts (<< 2.88dsf-13.3), sendfile (<< 2.1b.20080616-5.2~)
+Breaks: initscripts (<< 2.88dsf-13.3), sendfile (<< 2.1b.20080616-5.2~), ubuntu-server (<< 1.361.5)
 Multi-Arch: foreign
 Description: Debian base system miscellaneous files
  This package contains the basic filesystem hierarchy of a Debian system, and
@@ -25,3 +25,15 @@ Section: debian-installer
 Architecture: all
 Priority: extra
 Description: LSB release information
+
+Package: motd-news-config
+Architecture: all
+Priority: optional
+Breaks: base-files (<< 9.4ubuntu4.13)
+Replaces: base-files (<< 9.4ubuntu4.13)
+Depends: ${shlibs:Depends}, ${misc:Depends}
+Description: Configuration for motd-news shipped in base-files
+ This package contains the configuration read by the motd-news script
+ shipped in the base-files package.
+ .
+ Install this package if you want motd-news to be enabled.
diff --git a/debian/motd-news-config.conffiles b/debian/motd-news-config.conffiles
new file mode 100644
index 0000000..4109d03
--- /dev/null
+++ b/debian/motd-news-config.conffiles
@@ -0,0 +1 @@
+/etc/default/motd-news
diff --git a/debian/motd-news-config.postinst b/debian/motd-news-config.postinst
new file mode 100644
index 0000000..419d326
--- /dev/null
+++ b/debian/motd-news-config.postinst
@@ -0,0 +1,50 @@
+#!/bin/sh
+# postinst script for motd-news-config
+#
+# see: dh_installdeb(1)
+
+set -e
+
+# summary of how this script can be called:
+#        * <postinst> `configure' <most-recently-configured-version>
+#        * <old-postinst> `abort-upgrade' <new version>
+#        * <conflictor's-postinst> `abort-remove' `in-favour' <package>
+#          <new-version>
+#        * <postinst> `abort-remove'
+#        * <deconfigured's-postinst> `abort-deconfigure' `in-favour'
+#          <failed-install-package> <version> `removing'
+#          <conflicting-package> <version>
+# for details, see https://www.debian.org/doc/debian-policy/ or
+# the debian-policy package
+
+
+case "$1" in
+    configure)
+      # only run on new installs, $2 will be empty then
+      if [ -z "$2" ]; then
+          # /e/d/motd-news was moved from pkg:base-files to this package.
+          # base-files runs rm_conffile which, if it was modified, leaves
+          # /e/d/motd-news.dpkg-bak around. We want to preserve that *changed*
+          # config file in this migration, and this is something that
+          # rm_conffile is not handling.  In that case, let's put the backup
+          # file back in place
+          if [ -e /etc/default/motd-news.dpkg-bak ]; then
+              mv /etc/default/motd-news.dpkg-bak /etc/default/motd-news
+          fi
+          if [ -e /etc/default/motd-news.wasremoved ] && [ -e /etc/default/motd-news ]; then
+              sed -i -e 's/^ENABLED=1/# Changed by motd-news-config.postinst:\n# config file was manually removed - disable the service\nENABLED=0/' /etc/default/motd-news
+              rm /etc/default/motd-news.wasremoved
+          fi
+      fi
+    ;;
+
+    abort-upgrade|abort-remove|abort-deconfigure)
+    ;;
+
+    *)
+        echo "postinst called with unknown argument \`$1'" >&2
+        exit 1
+    ;;
+esac
+
+exit 0
diff --git a/debian/postinst.in b/debian/postinst.in
index db85abf..89ddd29 100644
--- a/debian/postinst.in
+++ b/debian/postinst.in
@@ -127,6 +127,19 @@ if [ "$1" = "configure" ] && [ "$2" != "" ]; then
   fi
 fi
 
+# special case of having /etc/default/motd-news removed by hand
+# signal the motd-news-config package that this happened, so that
+# it does not put back the file with default contents which would
+# re-enable motd-news
+motd_news_config="/etc/default/motd-news"
+if [ ! -e ${motd_news_config} ]; then
+  if [ ! -e ${motd_news_config}.dpkg-remove ]; then
+    if [ ! -e ${motd_news_config}.dpkg-backup ]; then
+      touch ${motd_news_config}.wasremoved
+    fi
+  fi
+fi
+
 # Manually inject expected maintainer script contents based on dh_systemd_*
 
 # This will only remove masks created by d-s-h on package removal.
@@ -147,3 +160,9 @@ if [ -d /run/systemd/system ]; then
 	systemctl --system daemon-reload >/dev/null || true
 	deb-systemd-invoke start motd-news.timer >/dev/null || true
 fi
+
+if dpkg-maintscript-helper supports rm_conffile; then
+	dpkg-maintscript-helper rm_conffile /etc/default/motd-news 9.4ubuntu4.13~ base-files -- "$@"
+fi
+
+exit 0
diff --git a/debian/postrm b/debian/postrm
index a763f0d..5f1a281 100644
--- a/debian/postrm
+++ b/debian/postrm
@@ -19,3 +19,9 @@ if [ "$1" = "purge" ]; then
         deb-systemd-helper unmask 'motd-news.timer' >/dev/null || true
     fi
 fi
+
+if dpkg-maintscript-helper supports rm_conffile; then
+	dpkg-maintscript-helper rm_conffile /etc/default/motd-news 9.4ubuntu4.13~ base-files -- "$@"
+fi
+
+exit 0
diff --git a/debian/preinst b/debian/preinst
new file mode 100644
index 0000000..6b96fc1
--- /dev/null
+++ b/debian/preinst
@@ -0,0 +1,34 @@
+#!/bin/sh
+# preinst script for base-files
+#
+# see: dh_installdeb(1)
+
+set -e
+
+# summary of how this script can be called:
+#        * <new-preinst> `install'
+#        * <new-preinst> `install' <old-version>
+#        * <new-preinst> `upgrade' <old-version>
+#        * <old-preinst> `abort-upgrade' <new-version>
+# for details, see https://www.debian.org/doc/debian-policy/ or
+# the debian-policy package
+
+
+case "$1" in
+    install|upgrade)
+    ;;
+
+    abort-upgrade)
+    ;;
+
+    *)
+        echo "preinst called with unknown argument \`$1'" >&2
+        exit 1
+    ;;
+esac
+
+if dpkg-maintscript-helper supports rm_conffile; then
+	dpkg-maintscript-helper rm_conffile /etc/default/motd-news 9.4ubuntu4.13~ base-files -- "$@"
+fi
+
+exit 0
diff --git a/debian/rules b/debian/rules
index 0ca88d0..682cbf5 100755
--- a/debian/rules
+++ b/debian/rules
@@ -25,10 +25,10 @@ build:
 clean:
 	rm -f build
 	rm -f `find . -name "*~"`
-	rm -rf debian/tmp debian/lsb-release-udeb debian/files* core debian/substvars
+	rm -rf debian/tmp debian/lsb-release-udeb debian/files* core debian/substvars debian/motd-news-config
 
 binary-arch: build
-	rm -rf debian/tmp
+	rm -rf debian/tmp debian/motd-news-config
 	install -d debian/tmp/DEBIAN $(docdir) debian/tmp/etc/dpkg/origins
 	install -p -m 644 origins/* debian/tmp/etc/dpkg/origins
 	cd debian/tmp && install -d usr/share/$(package)
@@ -42,7 +42,7 @@ binary-arch: build
 	sed -e "s/#VENDORFILE#/$(VENDORFILE)/g" debian/postinst.in \
 		> debian/tmp/DEBIAN/postinst
 	chmod 755 debian/tmp/DEBIAN/postinst
-	install -m 755 debian/prerm debian/postrm debian/tmp/DEBIAN
+	install -m 755 debian/prerm debian/postrm debian/preinst debian/tmp/DEBIAN
 	cd debian && install -m 644 conffiles tmp/DEBIAN
 	cd debian/tmp && install -d `cat ../directory-list`
 	install -p -m 644 share/* debian/tmp/usr/share/base-files
@@ -55,7 +55,6 @@ binary-arch: build
 	ln -s README $(docdir)/FAQ
 	install -p -m 644 etc/* debian/tmp/etc
 	install -p -m 755 debian/update-motd.d/* debian/tmp/etc/update-motd.d
-	install -p -m 644 debian/motd-news debian/tmp/etc/default
 	install -p -m 644 debian/motd-news.service debian/tmp/lib/systemd/system/
 	install -p -m 644 debian/motd-news.timer debian/tmp/lib/systemd/system/
 ifeq ($(DEB_HOST_GNU_SYSTEM),gnu)
@@ -87,6 +86,22 @@ endif
 		xargs -0r touch -h --date='$(BUILD_DATE)'
 	dpkg-deb --build -Zgzip debian/tmp ..
 
+	# motd-news-config packaging
+	install -d debian/motd-news-config/DEBIAN debian/motd-news-config/usr/share/doc/motd-news-config
+	install -d debian/motd-news-config/etc/default
+	install -m 0644 $(docdir)/changelog.gz $(docdir)/copyright debian/motd-news-config/usr/share/doc/motd-news-config
+	cp debian/motd-news-config.postinst debian/motd-news-config/DEBIAN/postinst
+	chmod 0755 debian/motd-news-config/DEBIAN/postinst
+	install -p -m 644 debian/motd-news debian/motd-news-config/etc/default
+	cp debian/motd-news-config.conffiles debian/motd-news-config/DEBIAN/conffiles
+	cd debian/motd-news-config && \
+		find usr -type f ! -regex "DEBIAN/.*" -print0 |\
+		LC_ALL=C sort -z | xargs -0r md5sum > DEBIAN/md5sums
+	chmod 0644 debian/motd-news-config/DEBIAN/conffiles
+	chmod 0644 debian/motd-news-config/DEBIAN/md5sums
+	dpkg-gencontrol -pmotd-news-config -Pdebian/motd-news-config
+	dpkg-deb --build -Zgzip debian/motd-news-config ..
+
 binary-indep: build
 	rm -rf debian/lsb-release-udeb
 	install -d debian/lsb-release-udeb/DEBIAN debian/lsb-release-udeb/etc
diff --git a/debian/update-motd.d/50-motd-news b/debian/update-motd.d/50-motd-news
index 08d7f0d..c9d142f 100644
--- a/debian/update-motd.d/50-motd-news
+++ b/debian/update-motd.d/50-motd-news
@@ -1,10 +1,11 @@
 #!/bin/sh
 #
 #    50-motd-news - print the live news from the Ubuntu wire
-#    Copyright (C) 2016-2017 Canonical Ltd.
+#    Copyright (C) 2016-2020 Canonical Ltd.
 #    Copyright (C) 2016-2017 Dustin Kirkland
 #
 #    Authors: Dustin Kirkland <[e-mail]>
+#             Steve Langasek <[e-mail]>
 #
 #    This program is free software; you can redistribute it and/or modify
 #    it under the terms of the GNU General Public License as published by
@@ -64,6 +65,9 @@ fi
 # If we've made it here, we've been given the --force argument,
 # probably from the systemd motd-news.service.  Let's update...
 
+# Abort early if wget is missing
+[ -x /usr/bin/wget ] || exit 0
+
 # Generate our temp files, clean up when done
 NEWS=$(mktemp) || exit 1
 ERR=$(mktemp) || exit 1
@@ -73,8 +77,8 @@ trap "rm -f $NEWS $ERR $CLOUD" HUP INT QUIT ILL TRAP KILL BUS TERM
 # Construct a user agent, similar to Firefox/Chrome/Safari/IE to
 # ensure a proper, tailored, accurate message of the day
 
-# Curl browser version, for debug purposes
-curl_ver="$(dpkg -l curl | awk '$1 == "ii" { print($3); exit(0); }')"
+# wget browser version, for debug purposes
+wget_ver="$(dpkg -l wget | awk '$1 == "ii" { print($3); exit(0); }')"
 
 # Distribution version, for messages releated to this Ubuntu release
 . /etc/lsb-release
@@ -97,12 +101,8 @@ if [ -x /usr/bin/cloud-id ]; then
     fi
 fi
 
-# Some messages may only be pertinent before or after some amount of uptime
-read up idle < /proc/uptime
-uptime="uptime/$up/$idle"
-
 # Piece together the user agent
-USER_AGENT="curl/$curl_ver $lsb $platform $cpu $uptime cloud_id/$cloud_id"
+USER_AGENT="wget/$wget_ver $lsb $platform $cpu cloud_id/$cloud_id"
 
 # Loop over any configured URLs
 for u in $URLS; do
@@ -121,7 +121,18 @@ for u in $URLS; do
 	# If we're forced, set the wait to much higher (1 minute)
 	[ "$FORCED" = "1" ] && WAIT=60
 	# Fetch and print the news motd
-	if curl --connect-timeout "$WAIT" --max-time "$WAIT" -A "$USER_AGENT" -o- "$u" >"$NEWS" 2>"$ERR"; then
+	result=0
+	not_found_is_ok=0
+	wget --timeout "$WAIT" -U "$USER_AGENT" -O- --content-on-error "$u" >"$NEWS" 2>"$ERR" || result=$?
+	# from wget's manpage: 8   Server issued an error response.
+	if [ $result -eq 8 ]; then
+		if grep -q "ERROR 404" "$ERR"; then
+			# The server's 404 document is the generic, non cloud-specific, motd-news
+			# content present in the index.txt file
+			not_found_is_ok=1
+		fi
+	fi
+	if [ $result -eq 0 ] || [ $not_found_is_ok -eq 1 ]; then
 		echo
 		# At most, 10 lines of text, remove control characters, print at most 80 characters per line
 		safe_print "$NEWS"