Backport recent release 10.3.5 to latest LTS

I subscribed the open-vm-tools related VMWare users.
Also I opened packaging MPs to get a second pair of eyes checking for potential drawbacks and mistakes.

The builds in the PPA [1] are complete and work in my trivial Tests.
@VMWare as usually please verify these builds across your supported matrix of hipervisors and confirm here if things are ok or not.

TL;DR of the next steps:
1. VMWare to confirm validity of the PPA for Bionic and Cosmic
2. I'll put this onto the SRU queue
3. SRU team accepts this into Bionic/Cosmic-proposed
4. VMWare to confirm the validity in Bionic/Cosmic-proposed

I'll also write a mail, just to be sure.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617

We would most likely want to make bug 1814832 part of all of this.

I learned that testing is stalled due to Chinese new year anyway, which actually is good for that.
That provides the time to get it resolved in Disco and then bundled here.

So do -not- test this PPA until I ping again after having bug 1814832 bundled.

The backports for Bionic and Cosmic are prepared and bundled the fixes for bug 1814832 as planned.
So I re-call for your usual full regression verification on the the PPA [1] before we upload this to the SRU queue.

Setting incomplete as we wait for feedback on this.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617

We did a quick sanity check for the quiescing related changes, looks good. I also asked our test team to test this (all of 10.3.5) too.

Thanks a lot Oliver already - I've heard that testing those could take a few days due to the breaks all around Asia. But I guess next week most people will be back and once this is pre-checked from the PPA we can upload to the SRU to have the final verification then.

It works well when do sanity check for updated open-vm-tools 10.3.5 from ppa:
   https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3617

Check items:
  * install / upgrade open-vm-tools and open-vm-tools-desktop(for desktop environment) from
    ppa: ci-train-ppa-service/3617
  * tools service and VGAuth service are running when install / upgrade open-vm-tools
  * tools service and VGAuth service are running after install / upgrade and reboot guestOS
  * uninstall open-vm-tools
  * sanity check for snapshot

in following VMs:
  * ubuntu-18.10-Desktop-amd64
  * ubuntu-18.10-live-server-amd64
  * ubuntu-18.04-Desktop-amd64
  * ubuntu-18.04-server-amd64

Thanks.

Thanks for the pre-check!
Ok, then all is ready to go for the real SRU where you then (once accepted) can do the real final verification.

Uploaded to Bionic/Cosmic SRU queue now.
Once the SRU Team accepts it they will call for the next (last) round of testing.

This falls under the general "hardware enablement" SRU exception for VM substrates.

An upload of open-vm-tools to cosmic-proposed has been rejected from the upload queue for the following reason: "needs a separate SRU bug for the debian/desktop.conf addition. Also, upload fails to comply with https://wiki.ubuntu.com/DebianMaintainerField".

An upload of open-vm-tools to bionic-proposed has been rejected from the upload queue for the following reason: "needs a separate SRU bug for the debian/desktop.conf addition. Also, upload fails to comply with https://wiki.ubuntu.com/DebianMaintainerField".

I had more assumed it would be #2 of the list that was polled - and due to that did it sometimes more strictly and sometimes not. I had so far never read the full official rules on the field (just got told what to do), thanks for the hint Steve.
But that is easy to prepare as a fixed update.

I'll also take an extra look at the modprobe in the desktop.conf

After improving on what we have found triggered by the first SRU review this is ready again.
Changes:
- Despite being an MRE in general all associated individual bugs have full SRU templates now.
- maintainers correctly updated
- The change to the vgauth Dependencies is now safer

That said, this is uploaded to Bionic/Cosmic unapproved again and waiting for SRU Team re-review.

Hello Christian, or anyone else affected,

Accepted open-vm-tools into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.3.5-7~ubuntu0.18.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Christian, or anyone else affected,

Accepted open-vm-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.3.5-7~ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Thanks Steve for accepting.

@Oliver/John - as usual verifying overall backport (in Bionic and Cosmic this time) is up to you, I'll also send a mail.

 It works well when do sanity check for 10.3.5 with repo “proposed“

 Check items:
 1. install / uninstall / upgrade open-vm-tools, check service
 2. reboot VM, check open-vm-tools/ VGAuth service
 3. check VGAuth basic function
      1) VGAuth service
      2) basic operation via VC (Guest OS —> View Guest User Mappings)
 4. snapshot with quiesce and revert

 in following VMs:
 1. ubuntu-18.10-Desktop-amd64
 2. ubuntu-18.10-live-server-amd64
 3. ubuntu-18.04-Desktop-amd64
 4. ubuntu-18.04-server-amd64
 5. ubuntu-18.04-live-server-amd64

Great, thanks Yuhua,
I'm marking this verified then - only the two special cases (SRM/Snapshots in extra bugs) are left to be confirmed then.

This bug was fixed in the package open-vm-tools - 2:10.3.5-7~ubuntu0.18.10.1

---------------
open-vm-tools (2:10.3.5-7~ubuntu0.18.10.1) cosmic; urgency=medium

  * Backport recent open-vm-tools (LP: #1813944)
    - also adresses handling of quiesced snapshot failures (LP: #1814832)
    - also adresses issues with resolutionKMS plugins sometimes fails to
      load at boot (LP: #1818473)

open-vm-tools (2:10.3.5-7) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [71b468f] make vgauth service execution more reliable.
    Since d3d47039 "Start vgauth before vmtoolsd" there is a potential race
    of starting vgauth so early that it might have issues. This was
    discussed back in the day in [1] to [2], but confirmed to be ok by
    VMWare.
    We were all somewhat convinced by this, but a bad feeling remained not
    only with me but also with Bernd [4].
    A recent SRU review denial made me rethink all of it and I think we can
    make it safer without thwarting the purpose of the original change.
    Note: Disambiguation of service names used below:
    vgauth - open-vm-tools.vgauth.service
    vmtoolsd - open-vm-tools.service
    fs - systemd-remount-fs.service
    tmp - systemd-tmpfiles-setup.service
    cloud-init - cloud-init-local.service
    Currently we have these dependency requirements:
    - vgauth should be before vmtoolsd
    - cloud init should be before vmtoolsd
    - cloud init has to be really early in general
      - therefore this is using DefaultDependencies=No
    That lead to this graph:
     fs / tmp -> vmtoolsd -> cloud-init
    And d3d47039 added it to be like:
     fs / tmp -> vmtoolsd -> cloud-init
                   ^
     vgauth --|
    But there is no need to have vgauth without any pre-dependencies at all.
    It is only needed to be "before" vmtoolsd, therefore we can make it:
     fs / tmp -> vgauth -> vmtoolsd -> cloud-init
    That will make execution of vgauth much less error-prone (even though I
    have no hard issue to report) while at the same time holding up all
    known required ordering constraints.
    [1]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/3
    [2]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/12
    [3]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/25
    [4]: https://github.com/bzed/pkg-open-vm-tools/pull/15#issuecomment-447237910
    Signed-off-by: Christian Ehrhardt <email address hidden>

open-vm-tools (2:10.3.5-6) unstable; urgency=medium

  * [43ec618] Correct and/or improve handling of certain quiesced
    snapshot failures.
    Thanks to Oliver Kurth (Closes: #921470)

open-vm-tools (2:10.3.5-5) unstable; urgency=medium

  * [54cce3e] Start vmtoolsd after apparmor.service.
    Github issue #17

open-vm-tools (2:10.3.5-4) unstable; urgency=medium

  [ Alf Gaida ]
  * [e13792d] udevadm trigger should not fail (Closes: #917642)

open-vm-tools (2:10.3.5-3) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [d3d4703] Start vgauth before vmtoolsd.
    VGAuthService needs to be ready when vmtoolsd runs. Certain cases - e.g.
    Site Reco...

The verification of the Stable Release Update for open-vm-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package open-vm-tools - 2:10.3.5-7~ubuntu0.18.04.1

---------------
open-vm-tools (2:10.3.5-7~ubuntu0.18.04.1) bionic; urgency=medium

  * Backport recent open-vm-tools (LP: #1813944)
    - also adresses handling of quiesced snapshot failures (LP: #1814832)
    - also adresses issues with resolutionKMS plugins sometimes fails to
      load at boot (LP: #1818473)

open-vm-tools (2:10.3.5-7) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [71b468f] make vgauth service execution more reliable.
    Since d3d47039 "Start vgauth before vmtoolsd" there is a potential race
    of starting vgauth so early that it might have issues. This was
    discussed back in the day in [1] to [2], but confirmed to be ok by
    VMWare.
    We were all somewhat convinced by this, but a bad feeling remained not
    only with me but also with Bernd [4].
    A recent SRU review denial made me rethink all of it and I think we can
    make it safer without thwarting the purpose of the original change.
    Note: Disambiguation of service names used below:
    vgauth - open-vm-tools.vgauth.service
    vmtoolsd - open-vm-tools.service
    fs - systemd-remount-fs.service
    tmp - systemd-tmpfiles-setup.service
    cloud-init - cloud-init-local.service
    Currently we have these dependency requirements:
    - vgauth should be before vmtoolsd
    - cloud init should be before vmtoolsd
    - cloud init has to be really early in general
      - therefore this is using DefaultDependencies=No
    That lead to this graph:
     fs / tmp -> vmtoolsd -> cloud-init
    And d3d47039 added it to be like:
     fs / tmp -> vmtoolsd -> cloud-init
                   ^
     vgauth --|
    But there is no need to have vgauth without any pre-dependencies at all.
    It is only needed to be "before" vmtoolsd, therefore we can make it:
     fs / tmp -> vgauth -> vmtoolsd -> cloud-init
    That will make execution of vgauth much less error-prone (even though I
    have no hard issue to report) while at the same time holding up all
    known required ordering constraints.
    [1]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/3
    [2]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/12
    [3]: https://bugs.launchpad.net/ubuntu/+source/open-vm-tools/+bug/1804287/comments/25
    [4]: https://github.com/bzed/pkg-open-vm-tools/pull/15#issuecomment-447237910
    Signed-off-by: Christian Ehrhardt <email address hidden>

open-vm-tools (2:10.3.5-6) unstable; urgency=medium

  * [43ec618] Correct and/or improve handling of certain quiesced
    snapshot failures.
    Thanks to Oliver Kurth (Closes: #921470)

open-vm-tools (2:10.3.5-5) unstable; urgency=medium

  * [54cce3e] Start vmtoolsd after apparmor.service.
    Github issue #17

open-vm-tools (2:10.3.5-4) unstable; urgency=medium

  [ Alf Gaida ]
  * [e13792d] udevadm trigger should not fail (Closes: #917642)

open-vm-tools (2:10.3.5-3) unstable; urgency=medium

  [ Christian Ehrhardt ]
  * [d3d4703] Start vgauth before vmtoolsd.
    VGAuthService needs to be ready when vmtoolsd runs. Certain cases - e.g.
    Site Reco...

open-vm-tools 11.0 will be too late this cycle (mid September), therefore in our "one MRE per cycle" approach to ensure thr latest LTS stays up to match hypervisors we will backport 10.3.10 now.

sorry - comment on wrong bug :-)