Merge lp:~vorlon/ubuntu-archive-tools/clean-NBS-for-EOL into lp:ubuntu-archive-tools

[Steve Langasek]

This is what I'm running in practice in order for us to be able to EOL disco; stashing it so we don't have to reinvent the wheel each cycle...

[lp:~vorlon/ubuntu-archive-tools/clean-NBS-for-EOL]

1308. By Steve Langasek on 2020-04-10

handle both updates and security. Ignore failures from remove-package, usually caused by it already being removed.

1309. By Steve Langasek on 2020-04-10

Don't use xargs, which will cause permanent failures.

Some of the packages in the update_output list are not in -updates or
-security at all, they're only in the release pocket; so batching them in a
remove-package command with others will cause failures no matter how many
times we rerun. We need to process the removals one-by-one instead.

[Brian Murray (brian-murray)]

What would happen if DIST was the current or active development series? Should there be a safe guard in place?

review: Needs Information

[Steve Langasek (vorlon)]

Brian, if run against the development series, it would effectively be a no-op, since there are no packages published to -updates or -security during development (and definitely not more than one kernel update, that would result in NBS binaries being left behind).

If run against a non-EOL series (in particular, a non-EOL LTS series, for which we do debian-installer SRUs), the impact would be that users who are locally using a netboot image that is not current will find that the udebs their netboot image references are no longer available in the archive. I do not think this is important to guard against, since we are unlikely to accidentally run this command against a non-EOL release, and the archive can't protect us from running it against a non-EOL release because it can only be run against a series in the archive that is still /open/, and the solution for users impacted if we do accidentally run it is to update to a current netboot image.

[Brian Murray (brian-murray)]

Okay, thanks for the additional information. This looks good to me then.

review: Approve

=== added file 'cleanup-NBS-for-EOL'
--- cleanup-NBS-for-EOL	1970-01-01 00:00:00 +0000
+++ cleanup-NBS-for-EOL	2020-04-10 17:06:36 +0000
@@ -0,0 +1,18 @@
+#!/bin/sh
+set -e
+
+DIST="$1"
+
+if [ -z "$DIST" ]; then
+	echo "Usage: $0 DIST"
+	exit 1
+fi
+wget -O - -q \
+	https://people.canonical.com/~ubuntu-archive/proposed-migration/"$DIST"/update_output.txt \
+| sed -e'1,/Removing packages left in testing/d; /^[^[:space:]]/,$d; s/^[[:space:]]*//; s/:.*//' \
+| grep -E '^linux|-di$' \
+| while read pkgs; do
+	remove-package -b -s "$DIST"-updates -m NBS -y $pkgs || true
+	remove-package -b -s "$DIST"-security -m NBS -y $pkgs || true
+done
+