unattended-upgrades should tell the user (via motd) when security updates are held back

Bug #1823070 reported by Steve Langasek
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
unattended-upgrades (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Disco
Fix Released
Undecided
Unassigned

Bug Description

[Impact]

 * MOTD does not go into details about upgradable packages being security fixes or just normal updates.
 * Users should be made aware if some of the security updates could not have been applied.
 * The fix is adding a snipped to MOTD where the number of packages kept back by unattended-upgrades is shown.

[Test Case]

 * The debian/tests/upgrade-all-security is extended to check if the number of kept back packages are shown in MOTD and a new test is added (test/test_motd.py) to check if the list of kept back packages are saved properly.
 * To test the fix manually:
   1. Mark a package upgradable from the -security pocket as held, then run unattended-upgrades.
   2. Observe MOTD messate showing the number of packages being kept back.

[Regression Potential]

 * Unattended-upgrades may crash when saving kept packages and always return with failure. MOTD may hang or print error while printing the packages kept back by u-u.

 * It is not a regression, but the log referenced in MOTD does not always contain explanation why each package was kept back, unless debugging is enabled. One case where packages are not mentioned in the log is when the packages are held using 'apt-mark hold' command.

[Original Bug Text]

Currently we have the following pieces as part of the default UX on Ubuntu 18.04 and later:

 1) unattended-upgrades automatically installs security updates daily by default
 2) the motd reports the number of available updates, including security updates.

A user who knows about 1) also knows that a non-zero number of pending security updates listed in 2) is nothing to worry about.

However, unattended-upgrades will also cleverly detect when a security update cannot safely be installed non-interactively due to conffile changes on the system.

In this case, unattended-upgrades should also inform the user via the motd that these updates are not being installed. Otherwise, there's nothing to tell the user that the non-zero count of available security updates in motd is a *problem*.

Suggested wording:

 N security updates will not be automatically installed due to local changes.
 See /var/log/foo for details.

tags: added: id-5ca50c3568756c4a351b6f5f
Balint Reczey (rbalint)
Changed in unattended-upgrades (Ubuntu):
status: New → Confirmed
Revision history for this message
Balint Reczey (rbalint) wrote :
Changed in unattended-upgrades (Ubuntu):
status: Confirmed → In Progress
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.13

---------------
unattended-upgrades (1.13) unstable; urgency=medium

  [ Alban VIDAL ]
  * Update french translation.
    Signed-off-by: Alban VIDAL <email address hidden>

  [ Balint Reczey ]
  * Don't crash collecting transitive dependencies when package has no
    candidate (LP: #1825886)
  * Use mark_install_adjusted() in rewind_cache()
    The original cache had packages marked with adjustments thus rewinding
    should also do adjustments to reach the same state. Not using
    mark_install_adjusted() also crashes when apt raises error on held
    packages. (LP: #1826157)
  * test_rewind: Update test to check if adjustend rewinding took place
  * Only allow removals in valid autoremoval sets
  * Fix one more log location in the man page (Closes: #928601)
  * Factor out kernel regexp generation to functions
  * Packages including kernel version without flavor in their name may be
    kernel packages. Also add autopkgtest for checking if the generated
    patterns cover the currently running kernel. (LP: #1828200)
  * Skip upgrade-all-security test when there are no updates to test with.
    Shortly after a release there may not be security updates against the
    packages used in the chroot used for the test.
  * Test with latest stable in upgrade-all-security on testing
  * Fix testing Debian's updates in upgrade-all-security
  * Store list of kept packages and report the number of them in motd
    (LP: #1823070)
  * Mention APT's apt-daily-upgrade.service in the man page
  * 50unattended-upgrades.md5sum add MD5 of current files
  * PEP8: Fix breaking line after binary operator
  * debian/tests/control: Fix Depends: of upgrade-between-snapshots
  * debian/tests/control: Allow stderr in kernel-patterns

  [ Gordon Lack ]
  * Replace boolean Unattended-Upgrade::MailOnlyOnError2 mail reporting setting
    with multi-valued (string) Unattended-Upgrade::MailReport.

  [ Jaime Hablutzel ]
  * Error message improved on very improbable condition

  [ Clint Armstrong ]
  * Fix showing multi-line strings on Plymouth.
    When unattended-upgrades sends it's status to plymouth it sends a
    multi-line string which causes plymouth to display overlapping text,
    because plymouth only scrolls one line when the message is sent.
    (LP: 1826406)

 -- Balint Reczey <email address hidden> Mon, 08 Jul 2019 11:08:30 +0200

Changed in unattended-upgrades (Ubuntu):
status: In Progress → Fix Released
Balint Reczey (rbalint)
Changed in unattended-upgrades (Ubuntu Bionic):
status: New → Confirmed
Changed in unattended-upgrades (Ubuntu Disco):
status: New → Confirmed
Balint Reczey (rbalint)
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Steve, or anyone else affected,

Accepted unattended-upgrades into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.10ubuntu5.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in unattended-upgrades (Ubuntu Disco):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-disco
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Steve, or anyone else affected,

Accepted unattended-upgrades into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.12 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in unattended-upgrades (Ubuntu Bionic):
status: Confirmed → Fix Committed
tags: added: verification-needed-bionic
Changed in unattended-upgrades (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Steve, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Balint Reczey (rbalint) wrote :
Download full text (3.7 KiB)

root@uu-sru-dd:~# unattended-upgrade --verbose
Initial blacklist :
Initial whitelist:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=disco, o=Ubuntu,a=disco-security, o=UbuntuESM,a=disco
Packages that will be upgraded: file libidn2-0 libmagic-mgc libmagic1 libxslt1.1 python3-apport python3-problem-report
Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
(Reading database ... 28941 files and directories currently installed.)
Preparing to unpack .../libidn2-0_2.0.5-1ubuntu0.3_amd64.deb ...
Unpacking libidn2-0:amd64 (2.0.5-1ubuntu0.3) over (2.0.5-1) ...
Setting up libidn2-0:amd64 (2.0.5-1ubuntu0.3) ...
Processing triggers for libc-bin (2.29-0ubuntu2) ...
Log ended: 2019-11-01 17:08:54

Log started: 2019-11-01 17:08:55
(Reading database ... 28941 files and directories currently installed.)
Preparing to unpack .../file_1%3a5.35-4ubuntu0.1_amd64.deb ...
Unpacking file (1:5.35-4ubuntu0.1) over (1:5.35-4) ...
Preparing to unpack .../libmagic1_1%3a5.35-4ubuntu0.1_amd64.deb ...
Unpacking libmagic1:amd64 (1:5.35-4ubuntu0.1) over (1:5.35-4) ...
Preparing to unpack .../libmagic-mgc_1%3a5.35-4ubuntu0.1_amd64.deb ...
Unpacking libmagic-mgc (1:5.35-4ubuntu0.1) over (1:5.35-4) ...
Setting up libmagic-mgc (1:5.35-4ubuntu0.1) ...
Setting up libmagic1:amd64 (1:5.35-4ubuntu0.1) ...
Setting up file (1:5.35-4ubuntu0.1) ...
Processing triggers for libc-bin (2.29-0ubuntu2) ...
Processing triggers for man-db (2.8.5-2) ...
Log ended: 2019-11-01 17:08:59

Log started: 2019-11-01 17:09:00
(Reading database ... 28941 files and directories currently installed.)
Preparing to unpack .../python3-apport_2.20.10-0ubuntu27.2_all.deb ...
Unpacking python3-apport (2.20.10-0ubuntu27.2) over (2.20.10-0ubuntu27.1) ...
Setting up python3-apport (2.20.10-0ubuntu27.2) ...
Log ended: 2019-11-01 17:09:04

Log started: 2019-11-01 17:09:04
(Reading database ... 28941 files and directories currently installed.)
Preparing to unpack .../python3-problem-report_2.20.10-0ubuntu27.2_all.deb ...
Unpacking python3-problem-report (2.20.10-0ubuntu27.2) over (2.20.10-0ubuntu27.1) ...
Setting up python3-problem-report (2.20.10-0ubuntu27.2) ...
Log ended: 2019-11-01 17:09:07

Log started: 2019-11-01 17:09:08
(Reading database ... 28941 files and directories currently installed.)
Preparing to unpack .../libxslt1.1_1.1.32-2ubuntu0.2_amd64.deb ...
Unpacking libxslt1.1:amd64 (1.1.32-2ubuntu0.2) over (1.1.32-2ubuntu0.1) ...
Setting up libxslt1.1:amd64 (1.1.32-2ubuntu0.2) ...
Processing triggers for libc-bin (2.29-0ubuntu2) ...
All upgrades installed
root@uu-sru-dd:~# update-motd
Welcome to Ubuntu 19.04 (GNU/Linux 5.0.0-32-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage

  System information as of Fri Nov 1 17:09:55 UTC 2019

  System load: 1.99 Processes: 26
  Usage of /home: unknown Users logged in: 0
  Memory usage: 0% IP address for eth0: 10.84.73.43
  Swap usage: 49%

0 updates can be installed immediately.
0 of these updates are security updates.

1 updates could not be installed automatically. For ...

Read more...

tags: added: verification-done-disco
removed: verification-needed-disco
Revision history for this message
Balint Reczey (rbalint) wrote :
Download full text (3.7 KiB)

root@uu-sru-bb:~# unattended-upgrade --verbose
Initial blacklisted packages:
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Packages that will be upgraded: file libidn2-0 libmagic-mgc libmagic1 libxslt1.1 python3-apport python3-problem-report
Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
(Reading database ... 28655 files and directories currently installed.)
Preparing to unpack .../libidn2-0_2.0.4-1.1ubuntu0.2_amd64.deb ...
Unpacking libidn2-0:amd64 (2.0.4-1.1ubuntu0.2) over (2.0.4-1.1build2) ...
Setting up libidn2-0:amd64 (2.0.4-1.1ubuntu0.2) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Log ended: 2019-11-01 17:09:03

Log started: 2019-11-01 17:09:04
(Reading database ... 28655 files and directories currently installed.)
Preparing to unpack .../libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ...
Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) over (1.1.29-5ubuntu0.1) ...
Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Log ended: 2019-11-01 17:09:07

Log started: 2019-11-01 17:09:08
(Reading database ... 28655 files and directories currently installed.)
Preparing to unpack .../python3-apport_2.20.9-0ubuntu7.8_all.deb ...
Unpacking python3-apport (2.20.9-0ubuntu7.8) over (2.20.9-0ubuntu7.7) ...
Setting up python3-apport (2.20.9-0ubuntu7.8) ...
Log ended: 2019-11-01 17:09:12

Log started: 2019-11-01 17:09:12
(Reading database ... 28655 files and directories currently installed.)
Preparing to unpack .../python3-problem-report_2.20.9-0ubuntu7.8_all.deb ...
Unpacking python3-problem-report (2.20.9-0ubuntu7.8) over (2.20.9-0ubuntu7.7) ...
Setting up python3-problem-report (2.20.9-0ubuntu7.8) ...
Log ended: 2019-11-01 17:09:15

Log started: 2019-11-01 17:09:15
(Reading database ... 28655 files and directories currently installed.)
Preparing to unpack .../file_1%3a5.32-2ubuntu0.3_amd64.deb ...
Unpacking file (1:5.32-2ubuntu0.3) over (1:5.32-2ubuntu0.2) ...
Preparing to unpack .../libmagic1_1%3a5.32-2ubuntu0.3_amd64.deb ...
Unpacking libmagic1:amd64 (1:5.32-2ubuntu0.3) over (1:5.32-2ubuntu0.2) ...
Preparing to unpack .../libmagic-mgc_1%3a5.32-2ubuntu0.3_amd64.deb ...
Unpacking libmagic-mgc (1:5.32-2ubuntu0.3) over (1:5.32-2ubuntu0.2) ...
Setting up libmagic-mgc (1:5.32-2ubuntu0.3) ...
Setting up libmagic1:amd64 (1:5.32-2ubuntu0.3) ...
Setting up file (1:5.32-2ubuntu0.3) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
All upgrades installed
root@uu-sru-bb:~# update-motd
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 5.0.0-32-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage

  System information as of Fri Nov 1 17:28:59 UTC 2019

  System load: 1.65 Processes: 24
  Usage of /home: unknown Users logged in: 0
  Memory usage: 0% IP address for eth0: 10.84.73.22
  Swap usage: 48%

0 packages can be updated.
0 updates are security updates.

1 updates ...

Read more...

description: updated
Revision history for this message
Balint Reczey (rbalint) wrote :

root@uu-sru-x:~# unattended-upgrade --verbose
Initial blacklisted packages:
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial
Packages that will be upgraded:
root@uu-sru-x:~# update-motd
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 5.0.0-32-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage

0 packages can be updated.
0 updates are security updates.

1 updates could not be installed automatically. For more details,
see /var/log/unattended-upgrades/unattended-upgrades.log

root@uu-sru-x:~# dpkg -l unattended-upgrades | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==========================-============-===========================================
ii unattended-upgrades 1.1ubuntu1.18.04.7~16.04.4 all automatic installation of security upgrades

tags: added: verification-done verification-done-bionic verification-done-xenial
removed: verification-needed verification-needed-bionic verification-needed-xenial
Revision history for this message
Brian Murray (brian-murray) wrote :

"1 updates"? Having an "s" for 1 update is incorrect.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.10ubuntu5.2

---------------
unattended-upgrades (1.10ubuntu5.2) disco; urgency=medium

  * Report packages kept back by origin (LP: #1821376)
  * Store list of kept packages and report the number of them in motd
    (LP: #1823070)
  * Default to "/" as rootdir to fix saving list of kept packages.
    Thanks to Paul Wise (Closes: #932160) (LP: #1823070)
  * debian/tests/control: Mark upgrade-between-snapshots as flaky
    (Closes: #941752) (LP: #1848354)

 -- Balint Reczey <email address hidden> Fri, 18 Oct 2019 13:22:02 +0200

Changed in unattended-upgrades (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for unattended-upgrades has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.1ubuntu1.18.04.12

---------------
unattended-upgrades (1.1ubuntu1.18.04.12) bionic; urgency=medium

  * Report packages kept back by origin (LP: #1821376)
  * Store list of kept packages and report the number of them in motd
    (LP: #1823070)
  * Default to "/" as rootdir to fix saving list of kept packages.
    Thanks to Paul Wise (Closes: #932160)
  * debian/tests/control: Mark upgrade-between-snapshots as flaky
    (Closes: #941752) (LP: #1848354)

 -- Balint Reczey <email address hidden> Fri, 18 Oct 2019 13:24:28 +0200

Changed in unattended-upgrades (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.1ubuntu1.18.04.7~16.04.4

---------------
unattended-upgrades (1.1ubuntu1.18.04.7~16.04.4) xenial; urgency=medium

  * Report packages kept back by origin (LP: #1821376)
  * Store list of kept packages and report the number of them in motd
    (LP: #1823070)
  * Default to "/" as rootdir to fix saving list of kept packages.
    Thanks to Paul Wise (Closes: #932160)
  * debian/tests/control: Mark upgrade-between-snapshots as flaky
    (Closes: #941752) (LP: #1848354)

 -- Balint Reczey <email address hidden> Fri, 18 Oct 2019 13:29:41 +0200

Changed in unattended-upgrades (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.