Merge ~ahasenack/ubuntu/+source/base-files:bionic-motd-news-config-split into ubuntu/+source/base-files:ubuntu/bionic-devel

[Andreas Hasenack]

Same as https://code.launchpad.net/~ahasenack/ubuntu/+source/base-files/+git/base-files/+merge/388835, but for bionic, with the following extra details:

- backport the switch to wget, and follow-up fixes that landed in groovy
- versions for breaks/replaces are the bionic ones:

ubuntu-server 1.417.5:
  Depends: motd-news-config

base-files 10.1ubuntu2.10:
  Breaks: ubuntu-server (<< 1.417.5)
  rm_conffile /etc/default/motd-news 10.1ubuntu2.10~ base-files

motd-news-config 10.1ubuntu2.10:
  Breaks/Replaces: base-files (<< 10.1ubuntu2.10)

PPA: https://launchpad.net/~ahasenack/+archive/ubuntu/zomg-5/

The linked bugs have the SRU template filled out.

[Andreas Hasenack (ahasenack)]

This is ready for review

[Andreas Hasenack (ahasenack)]

I pushed a change I found out while testing the groovy package: motd-news-config has to have priority set to optional, otherwise it will inherit base-files' which is "required" and do-release-upgrade will install it regardless of dependencies.

[Christian Ehrhardt  (paelzer)]

Change matches what we had in Groovy and soon Focal.
Additional change for priority is good as well.
Versions are good as well for an SRU and the bug LGTM as well.

The breaks and rm_conffile versions are adapted correctly and the maintscript lines LGTM, but we really need to land the meta change at the same time to avoid some updaters to remove packages or even ubuntu-server.

+1 to this in general, but I found one triviality to fix.
The changelog entries got carried over without line-wrap. So you violate the 80 chars in two lines and tools like lintian will hate it.

Fix is trivial enough, you don't need a re-review for the CL changes - therefore approving now.

review: Approve

[Andreas Hasenack (ahasenack)]

I'm sorry, the longest d/changelog lines I see are at 75 chars:

  * motd/50-motd-news: use wget instead of curl, since wget is standard but

and

    - This changes the timeout behavior slightly because wget does not have

I ran lintian -I --pedantic and the only changelog issues it raiesd were on older entries:

P: base-files source: file-contains-trailing-whitespace debian/changelog (line 1678)
P: base-files source: file-contains-trailing-whitespace debian/changelog (line 915)

[Andreas Hasenack (ahasenack)]

Ah, it's in the xenial MP that the lines passed 80 columns.

[Andreas Hasenack (ahasenack)]

Tagging and uploading 08744d99f2c353f8db78e777d5f79044c1a1a8c2

$ git push pkg upload/10.1ubuntu2.10
Enumerating objects: 51, done.
Counting objects: 100% (51/51), done.
Delta compression using up to 4 threads
Compressing objects: 100% (38/38), done.
Writing objects: 100% (42/42), 6.63 KiB | 3.32 MiB/s, done.
Total 42 (delta 29), reused 5 (delta 3)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/base-files
 * [new tag] upload/10.1ubuntu2.10 -> upload/10.1ubuntu2.10

$ dput ubuntu ../base-files_10.1ubuntu2.10_source.changes
Checking signature on .changes
gpg: ../base-files_10.1ubuntu2.10_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../base-files_10.1ubuntu2.10.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading base-files_10.1ubuntu2.10.dsc: done.
  Uploading base-files_10.1ubuntu2.10.tar.xz: done.
  Uploading base-files_10.1ubuntu2.10_source.buildinfo: done.
  Uploading base-files_10.1ubuntu2.10_source.changes: done.
Successfully uploaded packages.

[Andreas Hasenack (ahasenack)]

Also uploading https://code.launchpad.net/~ahasenack/ubuntu/+source/ubuntu-meta/+git/ubuntu-meta/+ref/bionic-ubuntu-server-motd-news

The seed change will also happen, but since it involves a new package, the update script from ubuntu-meta won't pick it up, so I have to add the dep manually.

Tagging and uploading 28444d68eb81af94ee88e771755991866fddf638 of that branch:
$ git push pkg upload/1.417.5
Enumerating objects: 13, done.
Counting objects: 100% (13/13), done.
Delta compression using up to 4 threads
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.13 KiB | 41.00 KiB/s, done.
Total 9 (delta 5), reused 0 (delta 0)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/ubuntu-meta
 * [new tag] upload/1.417.5 -> upload/1.417.5

$ dput ubuntu ../ubuntu-meta_1.417.5_source.changes
Checking signature on .changes
gpg: ../ubuntu-meta_1.417.5_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../ubuntu-meta_1.417.5.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading ubuntu-meta_1.417.5.dsc: done.
  Uploading ubuntu-meta_1.417.5.tar.xz: done.
  Uploading ubuntu-meta_1.417.5_source.buildinfo: done.
  Uploading ubuntu-meta_1.417.5_source.changes: done.
Successfully uploaded packages.

[Andreas Hasenack (ahasenack)]

Seeds changed for focal in 8b812c668c9cf4a6de14cbe9b38cc6c33a172735 after ok from steve.

diff --git a/debian/base-files.maintscript b/debian/base-files.maintscript
index 0b59f8f..130d339 100644
--- a/debian/base-files.maintscript
+++ b/debian/base-files.maintscript
@@ -3,3 +3,5 @@
 # Remove the short-lived (within zesty dev cycle) /etc/update-motd.d/50-news file,
 # which was renamed to /etc/update-motd.d/50-motd-news
 rm_conffile /etc/update-motd.d/50-news 9.6ubuntu11~
+
+rm_conffile /etc/default/motd-news 10.1ubuntu2.10~ base-files
diff --git a/debian/changelog b/debian/changelog
index 3b75631..3087eb8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,36 @@
+base-files (10.1ubuntu2.10) bionic; urgency=medium
+
+  [ Andreas Hasenack ]
+  * motd/50-motd-news: don't include uptime in the user-agent string
+    (LP: #1886572)
+  * Move the /etc/default/motd-news conffile to the motd-news-config
+    package (LP: #1888575):
+    - d/base-files.maintscript: remove /etc/default/motd-news config file
+      on upgrade
+    - d/control: break on ubuntu-server << 1.417.5 to force an upgrade if
+      it is installed, which will pull motd-news-config and the conffile
+      back in
+    - d/motd-news-config.postinst:
+      + handle the upgrade case where the motd-news config file was
+        changed while it belonged to base-files
+      + disable motd-news if the config file was removed by hand before
+        the upgrade
+    - d/postinst.in: signal the motd-news-config package if the
+      motd-news config file was removed manually before the upgrade
+    - d/control: new motd-news-config package, carrying the
+      configuration file for the /etc/update-motd.d/50-motd-news script.
+    - d/rules, d/motd-news-config.install: /e/d/motd-news is in the
+      motd-news-config package now
+
+  [ Steve Langasek ]
+  * motd/50-motd-news: use wget instead of curl, since wget is standard but
+    curl is optional (LP: #1888572):
+    - This changes the timeout behavior slightly because wget does not have
+      an exact equivalent to curl's --max-time argument, we are using
+      --timeout instead.
+
+ -- Andreas Hasenack <[e-mail]>  Thu, 13 Aug 2020 15:59:47 -0300
+
 base-files (10.1ubuntu2.9) bionic; urgency=medium
 
   * /etc/issue, /etc/issue.net, /etc/lsb-release, /etc/os-release: Bump
diff --git a/debian/control b/debian/control
index a22308d..dcc292c 100644
--- a/debian/control
+++ b/debian/control
@@ -14,7 +14,7 @@ Depends: ${misc:Depends}, ${shlibs:Depends}
 Essential: yes
 Priority: required
 Replaces: base, miscutils, dpkg (<= 1.15.0)
-Breaks: initscripts (<< 2.88dsf-13.3), sendfile (<< 2.1b.20080616-5.2~)
+Breaks: initscripts (<< 2.88dsf-13.3), sendfile (<< 2.1b.20080616-5.2~), ubuntu-server (<< 1.417.5)
 Multi-Arch: foreign
 Description: Debian base system miscellaneous files
  This package contains the basic filesystem hierarchy of a Debian system, and
@@ -28,3 +28,15 @@ Section: debian-installer
 Architecture: all
 Priority: extra
 Description: LSB release information
+
+Package: motd-news-config
+Architecture: all
+Priority: optional
+Breaks: base-files (<< 10.1ubuntu2.10)
+Replaces: base-files (<< 10.1ubuntu2.10)
+Depends: ${shlibs:Depends}, ${misc:Depends}
+Description: Configuration for motd-news shipped in base-files
+ This package contains the configuration read by the motd-news script
+ shipped in the base-files package.
+ .
+ Install this package if you want motd-news to be enabled.
diff --git a/debian/motd-news-config.install b/debian/motd-news-config.install
new file mode 100644
index 0000000..876b3b8
--- /dev/null
+++ b/debian/motd-news-config.install
@@ -0,0 +1 @@
+debian/motd-news etc/default
diff --git a/debian/motd-news-config.postinst b/debian/motd-news-config.postinst
new file mode 100644
index 0000000..a73427f
--- /dev/null
+++ b/debian/motd-news-config.postinst
@@ -0,0 +1,55 @@
+#!/bin/sh
+# postinst script for motd-news-config
+#
+# see: dh_installdeb(1)
+
+set -e
+
+# summary of how this script can be called:
+#        * <postinst> `configure' <most-recently-configured-version>
+#        * <old-postinst> `abort-upgrade' <new version>
+#        * <conflictor's-postinst> `abort-remove' `in-favour' <package>
+#          <new-version>
+#        * <postinst> `abort-remove'
+#        * <deconfigured's-postinst> `abort-deconfigure' `in-favour'
+#          <failed-install-package> <version> `removing'
+#          <conflicting-package> <version>
+# for details, see https://www.debian.org/doc/debian-policy/ or
+# the debian-policy package
+
+
+case "$1" in
+    configure)
+      # only run on new installs, $2 will be empty then
+      if [ -z "$2" ]; then
+          # /e/d/motd-news was moved from pkg:base-files to this package.
+          # base-files runs rm_conffile which, if it was modified, leaves
+          # /e/d/motd-news.dpkg-bak around. We want to preserve that *changed*
+          # config file in this migration, and this is something that
+          # rm_conffile is not handling.  In that case, let's put the backup
+          # file back in place
+          if [ -e /etc/default/motd-news.dpkg-bak ]; then
+              mv /etc/default/motd-news.dpkg-bak /etc/default/motd-news
+          fi
+          if [ -e /etc/default/motd-news.wasremoved ] && [ -e /etc/default/motd-news ]; then
+              sed -i -e 's/^ENABLED=1/# Changed by motd-news-config.postinst:\n# config file was manually removed - disable the service\nENABLED=0/' /etc/default/motd-news
+              rm /etc/default/motd-news.wasremoved
+          fi
+      fi
+    ;;
+
+    abort-upgrade|abort-remove|abort-deconfigure)
+    ;;
+
+    *)
+        echo "postinst called with unknown argument \`$1'" >&2
+        exit 1
+    ;;
+esac
+
+# dh_installdeb will replace this with shell code automatically
+# generated by other debhelper scripts.
+
+#DEBHELPER#
+
+exit 0
diff --git a/debian/postinst.in b/debian/postinst.in
index 69435e3..c347a62 100644
--- a/debian/postinst.in
+++ b/debian/postinst.in
@@ -125,4 +125,17 @@ if [ "$1" = "configure" ] && [ "$2" != "" ]; then
   fi
 fi
 
+# special case of having /etc/default/motd-news removed by hand
+# signal the motd-news-config package that this happened, so that
+# it does not put back the file with default contents which would
+# re-enable motd-news
+motd_news_config="/etc/default/motd-news"
+if [ ! -e ${motd_news_config} ]; then
+  if [ ! -e ${motd_news_config}.dpkg-remove ]; then
+    if [ ! -e ${motd_news_config}.dpkg-backup ]; then
+      touch ${motd_news_config}.wasremoved
+    fi
+  fi
+fi
+
 #DEBHELPER#
diff --git a/debian/rules b/debian/rules
index cb45e61..6c60a69 100755
--- a/debian/rules
+++ b/debian/rules
@@ -24,7 +24,6 @@ override_dh_auto_build: locale-check
 
 override_dh_auto_install:
 	install -p -m 644 etc/*      $(DESTDIR)/etc
-	install -p -m 644 debian/motd-news $(DESTDIR)/etc/default
 	install -p -m 644 debian/motd-news.service $(DESTDIR)/lib/systemd/system/
 	install -p -m 644 debian/motd-news.timer $(DESTDIR)/lib/systemd/system/
 	install -p -m 644 licenses/* $(DESTDIR)/usr/share/common-licenses
diff --git a/update-motd.d/50-motd-news b/update-motd.d/50-motd-news
index 08d7f0d..c9d142f 100644
--- a/update-motd.d/50-motd-news
+++ b/update-motd.d/50-motd-news
@@ -1,10 +1,11 @@
 #!/bin/sh
 #
 #    50-motd-news - print the live news from the Ubuntu wire
-#    Copyright (C) 2016-2017 Canonical Ltd.
+#    Copyright (C) 2016-2020 Canonical Ltd.
 #    Copyright (C) 2016-2017 Dustin Kirkland
 #
 #    Authors: Dustin Kirkland <[e-mail]>
+#             Steve Langasek <[e-mail]>
 #
 #    This program is free software; you can redistribute it and/or modify
 #    it under the terms of the GNU General Public License as published by
@@ -64,6 +65,9 @@ fi
 # If we've made it here, we've been given the --force argument,
 # probably from the systemd motd-news.service.  Let's update...
 
+# Abort early if wget is missing
+[ -x /usr/bin/wget ] || exit 0
+
 # Generate our temp files, clean up when done
 NEWS=$(mktemp) || exit 1
 ERR=$(mktemp) || exit 1
@@ -73,8 +77,8 @@ trap "rm -f $NEWS $ERR $CLOUD" HUP INT QUIT ILL TRAP KILL BUS TERM
 # Construct a user agent, similar to Firefox/Chrome/Safari/IE to
 # ensure a proper, tailored, accurate message of the day
 
-# Curl browser version, for debug purposes
-curl_ver="$(dpkg -l curl | awk '$1 == "ii" { print($3); exit(0); }')"
+# wget browser version, for debug purposes
+wget_ver="$(dpkg -l wget | awk '$1 == "ii" { print($3); exit(0); }')"
 
 # Distribution version, for messages releated to this Ubuntu release
 . /etc/lsb-release
@@ -97,12 +101,8 @@ if [ -x /usr/bin/cloud-id ]; then
     fi
 fi
 
-# Some messages may only be pertinent before or after some amount of uptime
-read up idle < /proc/uptime
-uptime="uptime/$up/$idle"
-
 # Piece together the user agent
-USER_AGENT="curl/$curl_ver $lsb $platform $cpu $uptime cloud_id/$cloud_id"
+USER_AGENT="wget/$wget_ver $lsb $platform $cpu cloud_id/$cloud_id"
 
 # Loop over any configured URLs
 for u in $URLS; do
@@ -121,7 +121,18 @@ for u in $URLS; do
 	# If we're forced, set the wait to much higher (1 minute)
 	[ "$FORCED" = "1" ] && WAIT=60
 	# Fetch and print the news motd
-	if curl --connect-timeout "$WAIT" --max-time "$WAIT" -A "$USER_AGENT" -o- "$u" >"$NEWS" 2>"$ERR"; then
+	result=0
+	not_found_is_ok=0
+	wget --timeout "$WAIT" -U "$USER_AGENT" -O- --content-on-error "$u" >"$NEWS" 2>"$ERR" || result=$?
+	# from wget's manpage: 8   Server issued an error response.
+	if [ $result -eq 8 ]; then
+		if grep -q "ERROR 404" "$ERR"; then
+			# The server's 404 document is the generic, non cloud-specific, motd-news
+			# content present in the index.txt file
+			not_found_is_ok=1
+		fi
+	fi
+	if [ $result -eq 0 ] || [ $not_found_is_ok -eq 1 ]; then
 		echo
 		# At most, 10 lines of text, remove control characters, print at most 80 characters per line
 		safe_print "$NEWS"