Merge ~enr0n/ubuntu/+source/systemd:ubuntu-focal into ~ubuntu-core-dev/ubuntu/+source/systemd:ubuntu-focal

[Nick Rosbrook]

SRU for a couple of miscellaneous bugs.

systemd 245.4-4ubuntu3.23~ppa3 (amd64) -- Pass: https://autopkgtest.ubuntu.com/results/autopkgtest-focal-enr0n-systemd/focal/amd64/s/systemd/20231025_231321_1250f@/log.gz
systemd 245.4-4ubuntu3.23~ppa3 (arm64) -- Pass: https://autopkgtest.ubuntu.com/results/autopkgtest-focal-enr0n-systemd/focal/arm64/s/systemd/20231026_004625_23809@/log.gz
systemd 245.4-4ubuntu3.23~ppa4 (armhf) -- Pass: https://autopkgtest.ubuntu.com/results/autopkgtest-focal-enr0n-systemd/focal/armhf/s/systemd/20231031_140750_dd55a@/log.gz
systemd 245.4-4ubuntu3.23~ppa3 (ppc64el) -- Pass: https://autopkgtest.ubuntu.com/results/autopkgtest-focal-enr0n-systemd/focal/ppc64el/s/systemd/20231025_230609_2bd01@/log.gz
systemd 245.4-4ubuntu3.23~ppa3 (s390x) -- Fail: https://autopkgtest.ubuntu.com/results/autopkgtest-focal-enr0n-systemd/focal/s390x/s/systemd/20231025_233417_368b3@/log.gz (s390x PPA tests almost always fail like this)

[Lukas Märdian (slyon)]

Thank you for providing autopkgtest results and PPA test builds! Those are mostly upstream cherry-picks, the patch for Bug #1837227 is a backport that seems to match upstream's new logic AFAICT.

LGTM!

Some non-blocking remarks:

- Bug #2029352
This seems to be lacking the SRU template. Also, we should put a clear comment in the bug report as to why this is ONLY needed on Focal. It seems to be fixed upstream as of v251, so we should put an argument why we don't need it for Jammy.

- lp2029352-test-ignore-LXC-filesystem-when-checking-for-writable-loc.patch
See inline comment about double escaping. Might be worth double-checking, just to be safe.

review: Approve

[Nick Rosbrook (enr0n)]

> Thank you for providing autopkgtest results and PPA test builds! Those are
> mostly upstream cherry-picks, the patch for Bug #1837227 is a backport that
> seems to match upstream's new logic AFAICT.
>
> LGTM!
>
>
> Some non-blocking remarks:
>
> - Bug #2029352
> This seems to be lacking the SRU template. Also, we should put a clear comment
> in the bug report as to why this is ONLY needed on Focal. It seems to be fixed
> upstream as of v251, so we should put an argument why we don't need it for
> Jammy.

Thanks, added SRU template. I will comment on the bug too, but FYI the reason we don't need this is because this test case is already skipped in Jammy on armhf due to https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/tree/debian/patches/debian/UBUNTU-src-test-test-execute.c-Skip-parts-of-test-execute-in-con.patch?h=ubuntu-jammy.

>
> - lp2029352-test-ignore-LXC-filesystem-when-checking-for-writable-loc.patch
> See inline comment about double escaping. Might be worth double-checking, just
> to be safe.

Since this unit is only used in the unit test suite, and it passed autopkgtest, I think it's OK.

Thanks for reviewing!

diff --git a/debian/changelog b/debian/changelog
index e8e0e5b..d0f4adc 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,27 @@
+systemd (245.4-4ubuntu3.23) UNRELEASED; urgency=medium
+
+  [ Nick Rosbrook ]
+  * core/device: ignore DEVICE_FOUND_UDEV bit on switching root (LP: #2037281)
+    File: debian/patches/lp2037281-core-device-ignore-DEVICE_FOUND_UDEV-bit-on-switching-roo.patch
+    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=7793563bb38a84a3dc6bc0da1c08546c3b915ab8
+  * dns-query: bump CNAME_MAX to 16 (LP: #2024009)
+    File: debian/patches/lp2024009-dns-query-bump-CNAME_MAX-to-16.patch
+    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=193899d103d44c642d362e9916b14df844ec702f
+  * Fall back to kexec when no kexec binary exists (LP: #1969365)
+    File: debian/patches/lp1969365-Fall-back-to-kexec-when-no-kexec-binary-exists.patch
+    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=3934f3794427dee4e72824998dd4c6e6d5875289
+  * test: ignore LXC filesystem when checking for writable locations (LP: #2029352)
+    File: debian/patches/lp2029352-test-ignore-LXC-filesystem-when-checking-for-writable-loc.patch
+    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=70facbfbf54c4ffb31ba392dbe3fec3084fdf3bc
+
+  [ Heitor Alves de Siqueira ]
+  * core/mount: adjust deserialized state based on /proc/self/mountinfo (LP: #1837227)
+    Author: Heitor Alves de Siqueira
+    File: debian/patches/lp1837227-core-mount-adjust-deserialized-state-based-on-proc-self-m.patch
+    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=a0a749953d309f48bc45140102adf205d1071c4d
+
+ -- Nick Rosbrook <[e-mail]>  Mon, 30 Oct 2023 15:24:20 -0400
+
 systemd (245.4-4ubuntu3.22) focal; urgency=medium
 
   * resolve: fix potential memleak and use-after-free (LP: #2012943)
diff --git a/debian/patches/lp1837227-core-mount-adjust-deserialized-state-based-on-proc-self-m.patch b/debian/patches/lp1837227-core-mount-adjust-deserialized-state-based-on-proc-self-m.patch
new file mode 100644
index 0000000..43309fd
--- /dev/null
+++ b/debian/patches/lp1837227-core-mount-adjust-deserialized-state-based-on-proc-self-m.patch
@@ -0,0 +1,119 @@
+From: Yu Watanabe <[e-mail]>
+Date: Wed, 6 Jul 2022 15:15:09 +0900
+Subject: core/mount: adjust deserialized state based on /proc/self/mountinfo
+
+Origin: upstream, https://github.com/systemd/systemd/commit/01400460ae16
+Bug-Ubuntu: https://launchpad.net/bugs/1837227
+Bug: https://github.com/systemd/systemd/issues/23796
+
+Fixes #23796.
+Replaces #23803 and #23851.
+
+(modified to apply to v245.4)
+---
+ src/core/mount.c | 55 +++++++++++++++++++++++++++++++++++++++++++++----------
+ 1 file changed, 45 insertions(+), 10 deletions(-)
+
+diff --git a/src/core/mount.c b/src/core/mount.c
+index dbfadc8..bb2b37f 100644
+--- a/src/core/mount.c
++++ b/src/core/mount.c
+@@ -51,6 +51,9 @@ static const UnitActiveState state_translation_table[_MOUNT_STATE_MAX] = {
+
+ static int mount_dispatch_timer(sd_event_source *source, usec_t usec, void *userdata);
+ static int mount_dispatch_io(sd_event_source *source, int fd, uint32_t revents, void *userdata);
++static void mount_enter_dead(Mount *m, MountResult f);
++static void mount_enter_mounted(Mount *m, MountResult f);
++static void mount_cycle_clear(Mount *m);
+ static int mount_process_proc_self_mountinfo(Manager *m);
+
+ static bool MOUNT_STATE_WITH_PROCESS(MountState state) {
+@@ -691,23 +694,17 @@ static void mount_set_state(Mount *m, MountState state) {
+
+ static int mount_coldplug(Unit *u) {
+         Mount *m = MOUNT(u);
+-        MountState new_state = MOUNT_DEAD;
+         int r;
+
+         assert(m);
+         assert(m->state == MOUNT_DEAD);
+
+-        if (m->deserialized_state != m->state)
+-                new_state = m->deserialized_state;
+-        else if (m->from_proc_self_mountinfo)
+-                new_state = MOUNT_MOUNTED;
+-
+-        if (new_state == m->state)
++        if (m->deserialized_state == m->state)
+                 return 0;
+
+         if (m->control_pid > 0 &&
+             pid_is_unwaited(m->control_pid) &&
+-            MOUNT_STATE_WITH_PROCESS(new_state)) {
++            MOUNT_STATE_WITH_PROCESS(m->deserialized_state)) {
+
+                 r = unit_watch_pid(UNIT(m), m->control_pid, false);
+                 if (r < 0)
+@@ -718,15 +715,52 @@ static int mount_coldplug(Unit *u) {
+                         return r;
+         }
+
+-        if (!IN_SET(new_state, MOUNT_DEAD, MOUNT_FAILED)) {
++        if (!IN_SET(m->deserialized_state, MOUNT_DEAD, MOUNT_FAILED)) {
+                 (void) unit_setup_dynamic_creds(u);
+                 (void) unit_setup_exec_runtime(u);
+         }
+
+-        mount_set_state(m, new_state);
++        mount_set_state(m, m->deserialized_state);
+         return 0;
+ }
+
++static void mount_catchup(Unit *u) {
++        Mount *m = MOUNT(u);
++
++        assert(m);
++
++        /* Adjust the deserialized state. See comments in mount_process_proc_self_mountinfo(). */
++        if (m->from_proc_self_mountinfo)
++                switch (m->state) {
++                case MOUNT_DEAD:
++                case MOUNT_FAILED:
++                        assert(m->control_pid == 0);
++                        unit_acquire_invocation_id(u);
++                        mount_cycle_clear(m);
++                        mount_enter_mounted(m, MOUNT_SUCCESS);
++                        break;
++                case MOUNT_MOUNTING:
++                        assert(m->control_pid > 0);
++                        mount_set_state(m, MOUNT_MOUNTING_DONE);
++                        break;
++                default:
++                        break;
++                }
++        else
++                switch (m->state) {
++                case MOUNT_MOUNTING_DONE:
++                        assert(m->control_pid > 0);
++                        mount_set_state(m, MOUNT_MOUNTING);
++                        break;
++                case MOUNT_MOUNTED:
++                        assert(m->control_pid == 0);
++                        mount_enter_dead(m, MOUNT_SUCCESS);
++                        break;
++                default:
++                        break;
++                }
++}
++
+ static void mount_dump(Unit *u, FILE *f, const char *prefix) {
+         char buf[FORMAT_TIMESPAN_MAX];
+         Mount *m = MOUNT(u);
+@@ -2092,6 +2126,7 @@ const UnitVTable mount_vtable = {
+         .done = mount_done,
+
+         .coldplug = mount_coldplug,
++        .catchup = mount_catchup,
+
+         .dump = mount_dump,
+
diff --git a/debian/patches/lp1969365-Fall-back-to-kexec-when-no-kexec-binary-exists.patch b/debian/patches/lp1969365-Fall-back-to-kexec-when-no-kexec-binary-exists.patch
new file mode 100644
index 0000000..b8a691b
--- /dev/null
+++ b/debian/patches/lp1969365-Fall-back-to-kexec-when-no-kexec-binary-exists.patch
@@ -0,0 +1,28 @@
+From: Arusekk <[e-mail]>
+Date: Sun, 22 Mar 2020 10:41:18 +0100
+Subject: Fall back to kexec when no kexec binary exists
+
+Origin: upstream, https://github.com/systemd/systemd/commit/71180f8e57f8fbb55978b00a13990c79093ff7b3
+Bug-Ubuntu: https://launchpad.net/bugs/1969365
+
+This may be not a popular setup, but in case kexec resides somewhere different
+than the default location for KEXEC (`/usr/sbin/kexec`), don't just reboot and
+try doing `reboot(RB_KEXEC)` instead, just like what `kexec -e` normally does.
+---
+ src/shutdown/shutdown.c | 3 +++
+ 1 file changed, 3 insertions(+)
+
+diff --git a/src/shutdown/shutdown.c b/src/shutdown/shutdown.c
+index 271751a..5e2c963 100644
+--- a/src/shutdown/shutdown.c
++++ b/src/shutdown/shutdown.c
+@@ -549,6 +549,9 @@ int main(int argc, char *argv[]) {
+                                 /* Child */
+
+                                 execv(args[0], (char * const *) args);
++
++                                /* execv failed (kexec binary missing?), so try simply reboot(RB_KEXEC) */
++                                (void) reboot(cmd);
+                                 _exit(EXIT_FAILURE);
+                         }
+
diff --git a/debian/patches/lp2024009-dns-query-bump-CNAME_MAX-to-16.patch b/debian/patches/lp2024009-dns-query-bump-CNAME_MAX-to-16.patch
new file mode 100644
index 0000000..e974628
--- /dev/null
+++ b/debian/patches/lp2024009-dns-query-bump-CNAME_MAX-to-16.patch
@@ -0,0 +1,24 @@
+From: Nick Rosbrook <[e-mail]>
+Date: Tue, 24 Oct 2023 16:32:30 -0400
+Subject: dns-query: bump CNAME_MAX to 16
+
+Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/focal/+source/systemd/+bug/2024009
+Origin: upstream (partial), https://github.com/systemd/systemd/commit/e0ae456a55
+
+---
+ src/resolve/resolved-dns-query.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/resolve/resolved-dns-query.c b/src/resolve/resolved-dns-query.c
+index 4a41921..dfb2f50 100644
+--- a/src/resolve/resolved-dns-query.c
++++ b/src/resolve/resolved-dns-query.c
+@@ -10,7 +10,7 @@
+ #include "resolved-etc-hosts.h"
+ #include "string-util.h"
+
+-#define CNAME_MAX 8
++#define CNAME_MAX 16
+ #define QUERIES_MAX 2048
+ #define AUXILIARY_QUERIES_MAX 64
+
diff --git a/debian/patches/lp2029352-test-ignore-LXC-filesystem-when-checking-for-writable-loc.patch b/debian/patches/lp2029352-test-ignore-LXC-filesystem-when-checking-for-writable-loc.patch
new file mode 100644
index 0000000..d76ac3c
--- /dev/null
+++ b/debian/patches/lp2029352-test-ignore-LXC-filesystem-when-checking-for-writable-loc.patch
@@ -0,0 +1,29 @@
+From: Luca Boccassi <[e-mail]>
+Date: Wed, 11 May 2022 12:24:10 +0100
+Subject: test: ignore LXC filesystem when checking for writable locations
+
+Origin: upstream, https://github.com/systemd/systemd/commit/646cba5c4208c28c56dbe52d676ab1a176c69b7f
+Bug-Ubuntu: https://launchpad.net/bugs/2029352
+
+test-execute checks that only /var/lib/private/waldo is writable, but there are
+some filesystems that are always writable and excluded. Add /sys/devices/system/cpu
+which is created by lxcfs.
+
+Fixes https://github.com/systemd/systemd/issues/23263
+---
+ test/test-execute/exec-dynamicuser-statedir.service | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/test/test-execute/exec-dynamicuser-statedir.service b/test/test-execute/exec-dynamicuser-statedir.service
+index ca40934..4ec6c53 100644
+--- a/test/test-execute/exec-dynamicuser-statedir.service
++++ b/test/test-execute/exec-dynamicuser-statedir.service
+@@ -13,7 +13,7 @@ ExecStart=test -f /var/lib/private/quux/pief/yayyay
+ ExecStart=/bin/sh -x -c 'test "$$STATE_DIRECTORY" = "%S/waldo:%S/quux/pief"'
+
+ # Make sure that /var/lib/private/waldo is really the only writable directory besides the obvious candidates
+-ExecStart=sh -x -c 'test $$(find / \( -path /var/tmp -o -path /tmp -o -path /proc -o -path /dev/mqueue -o -path /dev/shm -o -path /sys/fs/bpf -o -path /dev/.lxc \) -prune -o -type d -writable -print 2>/dev/null | sort -u | tr -d '\\\\n') = /var/lib/private/quux/pief/var/lib/private/waldo'
++ExecStart=sh -x -c 'test $$(find / \\( -path /var/tmp -o -path /tmp -o -path /proc -o -path /dev/mqueue -o -path /dev/shm -o -path /sys/fs/bpf -o -path /dev/.lxc -o -path /sys/devices/system/cpu \\) -prune -o -type d -writable -print 2>/dev/null | sort -u | tr -d "\\\\n") = /var/lib/private/quux/pief/var/lib/private/waldo'
+
+ Type=oneshot
+ DynamicUser=yes
diff --git a/debian/patches/lp2037281-core-device-ignore-DEVICE_FOUND_UDEV-bit-on-switching-roo.patch b/debian/patches/lp2037281-core-device-ignore-DEVICE_FOUND_UDEV-bit-on-switching-roo.patch
new file mode 100644
index 0000000..4e0e2f1
--- /dev/null
+++ b/debian/patches/lp2037281-core-device-ignore-DEVICE_FOUND_UDEV-bit-on-switching-roo.patch
@@ -0,0 +1,116 @@
+From: Yu Watanabe <[e-mail]>
+Date: Fri, 29 Apr 2022 20:29:11 +0900
+Subject: core/device: ignore DEVICE_FOUND_UDEV bit on switching root
+
+Origin: upstream, https://github.com/systemd/systemd/commit/75d7b5989f99125e52d5c0e5656fa1cd0fae2405
+Bug-Ubuntu: https://launchpad.net/bugs/2037281
+
+The issue #12953 is caused by the following:
+On switching root,
+- deserialized_found == DEVICE_FOUND_UDEV | DEVICE_FOUND_MOUNT,
+- deserialized_state == DEVICE_PLUGGED,
+- enumerated_found == DEVICE_FOUND_MOUNT,
+On switching root, most devices are not found by the enumeration process.
+Hence, the device state is set to plugged by device_coldplug(), and then
+changed to the dead state in device_catchup(). So the corresponding
+mount point is unmounted. Later when the device is processed by udevd, it
+will be changed to plugged state again.
+
+The issue #23208 is caused by the fact that generated udev database in
+initramfs and the main system are often different.
+
+So, the two issues have the same root; we should not honor
+DEVICE_FOUND_UDEV bit in the deserialized_found on switching root.
+
+This partially reverts c6e892bc0eebe1d42c282bd2d8bae149fbeba85f.
+
+Fixes #12953 and #23208.
+Replaces #23215.
+
+Co-authored-by: Martin Wilck <[e-mail]>
+---
+ src/core/device.c | 59 +++++++++++++++++++++++++++++++++++++++++++++----------
+ 1 file changed, 49 insertions(+), 10 deletions(-)
+
+diff --git a/src/core/device.c b/src/core/device.c
+index 06bbbf8..81b3616 100644
+--- a/src/core/device.c
++++ b/src/core/device.c
+@@ -157,14 +157,57 @@ static int device_coldplug(Unit *u) {
+         assert(d->state == DEVICE_DEAD);
+
+         /* First, let's put the deserialized state and found mask into effect, if we have it. */
++        if (d->deserialized_state < 0)
++                return 0;
++
++        Manager *m = u->manager;
++        DeviceFound found = d->deserialized_found;
++        DeviceState state = d->deserialized_state;
++
++        /* On initial boot, switch-root, reload, reexecute, the following happen:
++         * 1. MANAGER_IS_RUNNING() == false
++         * 2. enumerate devices: manager_enumerate() -> device_enumerate()
++         *    Device.enumerated_found is set.
++         * 3. deserialize devices: manager_deserialize() -> device_deserialize()
++         *    Device.deserialize_state and Device.deserialized_found are set.
++         * 4. coldplug devices: manager_coldplug() -> device_coldplug()
++         *    deserialized properties are copied to the main properties.
++         * 5. MANAGER_IS_RUNNING() == true: manager_ready()
++         * 6. catchup devices: manager_catchup() -> device_catchup()
++         *    Device.enumerated_found is applied to Device.found, and state is updated based on that.
++         *
++         * Notes:
++         * - On initial boot, no udev database exists. Hence, no devices are enumerated in the step 2.
++         *   Also, there is no deserialized device. Device units are (a) generated based on dependencies of
++         *   other units, or (b) generated when uevents are received.
++         *
++         * - On switch-root, the udev databse may be cleared, except for devices with sticky bit, i.e.
++         *   OPTIONS="db_persist". Hence, almost no devices are enumerated in the step 2. However, in general,
++         *   we have several serialized devices. So, DEVICE_FOUND_UDEV bit in the deserialized_found must be
++         *   ignored, as udev rules in initramfs and the main system are often different. If the deserialized
++         *   state is DEVICE_PLUGGED, we need to downgrade it to DEVICE_TENTATIVE (or DEVICE_DEAD if nobody
++         *   sees the device). Unlike the other starting mode, Manager.honor_device_enumeration == false
++         *   (maybe, it is better to rename the flag) when device_coldplug() and device_catchup() are called.
++         *   Hence, let's conditionalize the operations by using the flag. After switch-root, systemd-udevd
++         *   will (re-)process all devices, and the Device.found and Device.state will be adjusted.
++         *
++         * - On reload or reexecute, we can trust enumerated_found, deserialized_found, and deserialized_state.
++         *   Of course, deserialized parameters may be outdated, but the unit state can be adjusted later by
++         *   device_catchup() or uevents. */
++
++        if (!m->honor_device_enumeration && !MANAGER_IS_USER(m)) {
++                found &= ~DEVICE_FOUND_UDEV; /* ignore DEVICE_FOUND_UDEV bit */
++                if (state == DEVICE_PLUGGED)
++                        state = DEVICE_TENTATIVE; /* downgrade state */
++                if (found == DEVICE_NOT_FOUND)
++                        state = DEVICE_DEAD; /* If nobody sees the device, downgrade more */
++        }
+
+-        if (d->deserialized_state < 0 ||
+-            (d->deserialized_state == d->state &&
+-             d->deserialized_found == d->found))
++        if (d->found == found && d->state == state)
+                 return 0;
+
+-        d->found = d->deserialized_found;
+-        device_set_state(d, d->deserialized_state);
++        d->found = found;
++        device_set_state(d, state);
+         return 0;
+ }
+
+@@ -666,13 +709,9 @@ static void device_found_changed(Device *d, DeviceFound previous, DeviceFound no
+ }
+
+ static void device_update_found_one(Device *d, DeviceFound found, DeviceFound mask) {
+-        Manager *m;
+-
+         assert(d);
+
+-        m = UNIT(d)->manager;
+-
+-        if (MANAGER_IS_RUNNING(m) && (m->honor_device_enumeration || MANAGER_IS_USER(m))) {
++        if (MANAGER_IS_RUNNING(UNIT(d)->manager)) {
+                 DeviceFound n, previous;
+
+                 /* When we are already running, then apply the new mask right-away, and trigger state changes
diff --git a/debian/patches/series b/debian/patches/series
index 9fa4a07..a249981 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -194,3 +194,8 @@ lp2002445-sd-netlink-restore-altname-on-error-in-rtnl_set_link_name.patch
 lp2002445-udev-attempt-device-rename-even-if-interface-is-up.patch
 lp1933090-test-seccomp-accept-ENOSYS-from-sysctl-2-too.patch
 lp2012943-resolve-fix-potential-memleak-and-use-after-free.patch
+lp2037281-core-device-ignore-DEVICE_FOUND_UDEV-bit-on-switching-roo.patch
+lp1837227-core-mount-adjust-deserialized-state-based-on-proc-self-m.patch
+lp2024009-dns-query-bump-CNAME_MAX-to-16.patch
+lp1969365-Fall-back-to-kexec-when-no-kexec-binary-exists.patch
+lp2029352-test-ignore-LXC-filesystem-when-checking-for-writable-loc.patch