With needrestart, apt-get does not respect non-interactive instruction when upgrading services

(Labelled as an issue with ubuntu-advantage-tools although it is apt-get doing this, because a) I can't report issues with apt-get here and b) the 22.04 desktop edition I was using didn't do this and the obvious difference is that the servers are signed up to the UA programme and the desktop wasn't so I am looking sternly at that as being the underlying cause.)

Thank you for the report Ian!

This appears to be about the behavior of needrestart running at the end of apt-get upgrade, so I've assigned this bug to that package.

ubuntu-advantage-tools doesn't change the needrestart configuration so I'm going to mark this as "Invalid" for ubuntu-advantage-tools. If you discover that ubuntu-advantage-tools is causing this then please do change the status back.

Hi Ian,
as Grant already mentioned this isn't ubuntu-advantage-tools doing, it is just the integration of needrestart.

But even there it isn't a bug, you are - so far - only using -y which is "assume yes" and the questions asked by needrestart are not yes/no questions. There would be more potential things that fall into a similar category like conffile prompts, debconf questions and others.

You might rightfully say: "So if -y isn't the way to make it fully non-interactive, what is it then?"

The answer is that each kind of interactions can be disabled individually.
- --assume-yes - will make apt [1] not ask the "Y/N are you sure"
- setting DEBIAN_FRONTEND=noninteractive environment [2] will make dpkg/debconf not ask you (this is the one also needrestart will follow)
- and predefining the behavior you want in regard to conffile prompts [3] will silence those like -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"

So if you run it like this it should be truly non-interactive:
$ DEBIAN_FRONTEND=noninteractive apt-get --assume-yes --allow-unauthenticated -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade

If you face issues even with that please speak up, I might have forgotten something or there is a real bug hidden somewhere. But until then I think this works as intended and I'd consider the bug task as incomplete until we know about an actual issue even when using those arguments.

[1]: https://manpages.ubuntu.com/manpages/jammy/en/man8/apt-get.8.html
[2]: https://manpages.ubuntu.com/manpages/jammy/man7/debconf.7.html
[3]: https://manpages.ubuntu.com/manpages/xenial/man1/dpkg.1.html

I had tried the first part of that rather lengthy spell and just setting the environmental variable isn't enough, so I will try the full thing.

Amusingly, needrestart does not have a -y option to say 'yes' to all questions. Instead, it has a -n option to say 'no'!

You are right that, despite the fact that it is waiting for the user to say 'yes, restart that, no, don't' via checkboxes, it isn't literally waiting for a 'yes/no' answer, but I would say that saying "run non-interactively" to the program we actually invoke is unambiguous and should apply to anything it runs as well as itself.

needrestart's man does not say whether "(a)utomatically restart" restarts everything or just the services that it thinks needs restarting, but especially if it's the latter then that's the argument that apt-get should pass to it when it calls it in non-interactive mode.

Am I going to need to repeat all this with the script's

aptitude -safe-upgrade -y

line?

Another 'interesting' difference with 22.04 on a server is that it's not uncommon for apt-get dist-upgrade to withhold some upgrades. Running aptitude like this next will do them, but it's not something I have needed to add to the script on Debian or Ubuntu 20.04 or earlier.

I've not noticed needrestart being run again, but I would quite believe that it is / should be.

Having just reported this spell to someone else - is the "--allow-unauthenticated" really necessary?

I can't remember if otherwise it would just stop (which would be fine, actually) if it detected a problem with the signing or ask if I really meant that.

Status changed to 'Confirmed' because the bug affects multiple users.

I have the same problem on Ubuntu 22.04.3 LTS.

unattended-upgrade log (mail):
----------
Unattended upgrade result: All upgrades installed

Packages that were upgraded:
 libssh-4 openssh-client openssh-server openssh-sftp-server

Package installation log:
Log started: 2023-12-20 06:38:04
Preconfiguring packages ...
Preconfiguring packages ...
Preparing to unpack .../openssh-sftp-server_1%3a8.9p1-3ubuntu0.5_amd64.deb ...
Unpacking openssh-sftp-server (1:8.9p1-3ubuntu0.5) over (1:8.9p1-3ubuntu0.4) ...
Preparing to unpack .../openssh-server_1%3a8.9p1-3ubuntu0.5_amd64.deb ...
Unpacking openssh-server (1:8.9p1-3ubuntu0.5) over (1:8.9p1-3ubuntu0.4) ...
Preparing to unpack .../openssh-client_1%3a8.9p1-3ubuntu0.5_amd64.deb ...
Unpacking openssh-client (1:8.9p1-3ubuntu0.5) over (1:8.9p1-3ubuntu0.4) ...
Setting up openssh-client (1:8.9p1-3ubuntu0.5) ...
Setting up openssh-sftp-server (1:8.9p1-3ubuntu0.5) ...
Setting up openssh-server (1:8.9p1-3ubuntu0.5) ...
rescue-ssh.target is a disabled or a static unit not running, not starting it.
ssh.socket is a disabled or a static unit not running, not starting it.
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for ufw (0.36.1-4ubuntu0.1) ...
NEEDRESTART-VER: 3.5
NEEDRESTART-KCUR: 5.15.0-91-generic
NEEDRESTART-KEXP: 5.15.0-91-generic
NEEDRESTART-KSTA: 1
Log ended: 2023-12-20 06:38:14

Log started: 2023-12-20 06:38:14
Preparing to unpack .../libssh-4_0.9.6-2ubuntu0.22.04.2_amd64.deb ...
Unpacking libssh-4:amd64 (0.9.6-2ubuntu0.22.04.2) over (0.9.6-2ubuntu0.22.04.1) ...
Setting up libssh-4:amd64 (0.9.6-2ubuntu0.22.04.2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.5) ...
NEEDRESTART-VER: 3.5
NEEDRESTART-KCUR: 5.15.0-91-generic
NEEDRESTART-KEXP: 5.15.0-91-generic
NEEDRESTART-KSTA: 1
NEEDRESTART-SVC: mw-jobqueue.service
NEEDRESTART-SVC: packagekit.service
NEEDRESTART-SVC: php8.1-fpm.service
NEEDRESTART-SVC: ttrss_backend.service
Log ended: 2023-12-20 06:38:19

Unattended-upgrades log:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=jammy, o=Ubuntu,a=jammy-security, o=UbuntuESMApps,a=jammy-apps-security, o=UbuntuESM,a=jammy-infra-security
Initial blacklist:
Initial whitelist (not strict):
Packages that will be upgraded: libssh-4 openssh-client openssh-server openssh-sftp-server
Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
All upgrades installed
----------

When I start needrestart manually afterwards:
----------
root@vps:~# needrestart
Scanning processes...
Scanning candidates...
Scanning linux images...

Running kernel seems to be up-to-date.

Restarting services...
 systemctl restart mw-jobqueue.service packagekit.service php8.1-fpm.service ttrss_backend.service

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
----------

Folks, this `needrestart` package breaks unattended `apt upgrade`, as well as when running upgrades via Ansible.

The very first thing I do when installing Ubuntu +22.04 is:

```
apt purge needrestart
apt autoremove
```

This is extremely inconvenient! Suppose you start an Ubuntu Server from a Cloud Image and run `apt install ubuntu-desktop-minimal` without first removing `needrestart`. In that case, the installation sometimes hangs because the `needrestart` screen does not appear, forcing me to open a new connection and kill the `needrestart` process hoping it'll continue the installation. NOTE: I'm unsure if this is really happening, but I see the `needrestart` process in the `ps ufxa` list, while `apt install ...` seems stuck.

Let's get rid of this bad-designed `needrestart` package by default? This is very annoying.

This bug was fixed in the package needrestart - 3.6-7ubuntu3

---------------
needrestart (3.6-7ubuntu3) noble; urgency=medium

  * debian/tests:
    - prompt-reboot: mark the tests as needing a VM since needrestart doesn't
      do kernel detection when within a container
    - *.py: migrate off some deprecated libtmux APIs
    - *.py: make tests less sensible to performance issues on the runners

 -- Simon Chopin <email address hidden> Fri, 08 Mar 2024 16:01:08 +0100

Will this fixed version also be released for Ubuntu 22.04.4 LTS?
I currently have only 3.5-5ubuntu2.1 as installed and candidate.