apt-check needs to consider the pro client's esm cache for updates when esm is disabled

On a separate note: this also affects Xenial, and needs to be SRUed to Xenial - however no Xenial branch was found to send the patch to. What would be the steps to do so?

To anyone the core-dev team reading this:
the Pro Client has recently implemented exactly this feature: providing upgrade counts for installed packages. If you want, instead of bringing it from the cache as done in the MRs for this bug, we can change it to read the Pro Client data, as mentioned in the [Other info] section above.

Implementation would be more reliable and clean.

IMHO this would make update-notifier depend on u-a-t, but I don't think this is a big problem, given u-a-t is in minimal anyway.

Please let me know if you think it is ok to import and run the code from `uaclient`, and if it is fine to depend on the package for the counts.

I'd rather we have the implementation in update-manager reused by pro and not the other way around, the whole thing should be meaningful for users.

Notably apt has its own implementation of phased updates and phasing that update-manager and update -notifier also needs to switch to, but which is not yet exposed in python-apt.

We can't have multiple implementations of phased updates that each count slightly different because their languages have different RNG implementations.

> I'd rather we have the implementation in update-manager reused by pro and not the other way around, the whole thing should be meaningful for users.

I see the point, but what I took into consideration for this particular change is:

- Right now, update-notifier already depends on u-a-t implementation (not only code, but the details, like the soon-to-be-gone 'never' pinning on esm when disabled) to count and show esm updates when the services are disabled. With this change, we can make it keep depending on implementation, or depend on functionality directly, which is already implemented in u-a-t.

- The change sent in the MR keeps all logic as-is for update-notifier, including the dependency on the Pro Client for only those two things: checking if esm is enabled or not, and counting esm upgrades when the services are disabled.

- Even if u-a-t is to consume from update-notifier any information about standard/security upgrades, it should still take care of esm, as we discussed many times up to this point - Pro related things should live in the Pro client. Take the esm cache itself: it would not make much sense to create and maintain it under update-notifier, u-a-t should take care of that. So, I understand and agree that a single implementation is better and if we put esm upgrade counts to u-a-t and the other upgrade counts in update-notifier, we'll end up with both tools depending on each other to show complete and meaningful information; but it is not clear (at least to me, sorry, maybe I miss some information) that update-notifier should take this responsibility.

- I don ´t really understand how we can't treat phased updates in u-a-t for instance (which today we do not, btw, by design) - update manager has a python implementation and that's what update-notifier is relying on anyway. If/when the apt phasing implementation gets to python-apt, it will be available for u-a-t as well. But imho this is a separate issue and can be solved in a separate thread.

- Last, let's say we agree that the way u-a-t should consume information from update-notifier and/or vice-versa need to be reworked, this would require a redesign of how this should work before actually implementing something, and (you would not believe that) we have a deadline from the products side - u-a-t needs to SRU changes by January 26th, and it will impact update-notifier anyway. So, what I ask is: can we proceed with updating update-notifier now adapting to the changes in the client? and then (for next cycle, why not), I will be most happy to work on a better design and implementation of how package upgrades can be advertised.

I specifically replied to #2 regarding possible future changes, not this change itself.

I see. Sorry for being overdefensive in the answer.

My point in #2 is that, in this changeset itself, we could be importing uaclient and using functionality instead of relying on the implementation detail, before we think about a redesign.

Either way, please let me know what is the best approach and how to better sort this.

So I thought I made it clear in the previous message that this wasn't blocked on me, but apparently not, and nobody reviewed the merges it turns out, so I went ahead and approved them. Minimal changes make for good SRU.

Let's figure the rest out when there's no time pressure.

This bug was fixed in the package update-notifier - 3.192.61

---------------
update-notifier (3.192.61) lunar; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

 -- Christian Ehrhardt <email address hidden> Thu, 19 Jan 2023 14:03:22 +0100

The xenial upload in unapproved does not have the fix for this bug here, was that intentional? It just has the fix for the translation crash:

https://launchpadlibrarian.net/647743365/update-notifier_3.168.19_source.changes

Launchpad-Bugs-Fixed: 2003543
Changes:
 update-notifier (3.168.19) xenial; urgency=medium
 .
   * po/*.po: do not translate template variable needed for
     package-data-downloader script (LP: #2003543)

Hello Renan, or anyone else affected,

Accepted update-notifier into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.54.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted update-notifier into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.30.16 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted update-notifier (3.192.30.16) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

update-notifier/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

@Andeas
The xenial upload in unapproved does not have the fix for this bug here, was that intentional? It just has the fix for the translation crash.

No that is wrong - I had this in unapproved:
 update-notifier (3.168.19) xenial; urgency=medium
 .
   [ Renan Rodrigo Barbosa]
   * Rely on the Pro Client apt esm cache to check for esm updates
     (LP: #2002168)
=> On Jan 19 15:36

Then it was replaced by the current one you linked to when Sebastien sponsored the translation fix.
I have provided a new (still 3.168.19 as it was not yet accepted) into xenial-unapproved.

@Andreas
For Bionic - Now also a https://launchpadlibrarian.net/647902226/update-notifier_3.192.1.17_source.changes showed up in -unapproved which needs a similar fix on top (the regen of the autogen files was incomplete). On this upload, since tagging is used I need to bump the version.

Coordinating with Lucas to add the missing bits I've uploaded (and made available in git) => 3.192.1.18
=> https://launchpadlibrarian.net/647903734/update-notifier_3.192.1.18_source.changes

Hello Renan, or anyone else affected,

Accepted update-notifier into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.1.18 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted update-notifier into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.168.19 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

I have performed the verification for this bug's fix using the package from proposed.
I just uploaded the logs with all commands executed as the test steps suggests (Yeah I know, I could have a tarball... my bad.)

I am marking verification-done for the LTS series.

I verified the attached logs and am satisfied that they show the executed planned test case, and that the results are correct.

The package built correctly in all architectures and Ubuntu releases it was meant for.

There are no DEP8 regressions.

There is no SRU freeze ongoing at the moment.

There is a halted phasing on the previous update due to #2003543, for which this upload includes a fix.

This bug was fixed in the package update-notifier - 3.192.54.5

---------------
update-notifier (3.192.54.5) jammy; urgency=medium

  * po/*.po: do not translate template variable needed for
    package-data-downloader script (LP: #2003543)

update-notifier (3.192.54.4) jammy; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

 -- Lucas Moura <email address hidden> Fri, 20 Jan 2023 15:50:39 -0300

The verification of the Stable Release Update for update-notifier has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package update-notifier - 3.192.30.16

---------------
update-notifier (3.192.30.16) focal; urgency=medium

  * po/*.po: do not translate template variable needed for
    package-data-downloader script (LP: #2003543)

update-notifier (3.192.30.15) focal; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

 -- Lucas Moura <email address hidden> Fri, 20 Jan 2023 16:17:53 -0300

This bug was fixed in the package update-notifier - 3.192.1.18

---------------
update-notifier (3.192.1.18) bionic; urgency=medium

  * po/*.po: do no translate template variable needed for
    package-data-downloader script (LP: #2003543)
  * Bring generated autoconf files from the archive package (LP: #1999897)

update-notifier (3.192.1.16) bionic; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

 -- Lucas Moura <email address hidden> Fri, 20 Jan 2023 16:25:17 -0300

This bug was fixed in the package update-notifier - 3.168.19

---------------
update-notifier (3.168.19) xenial; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

  [ Lucas Moura ]
  * po/*.po: do not translate template variable needed for
    package-data-downloader script (LP: #2003543)

 -- Christian Ehrhardt <email address hidden> Thu, 19 Jan 2023 15:21:16 +0100