qemu machine types broken in wily and missed to be updated in xenial and yakkety

To $(clarify or argue :),

A xenial machine type should indeed be defined based on the newest
mt available in xenial, and SRU'd to xenial. A yakkety type should
probably be defined at the end of the y cycle, and obviously not
SRUd.

Thanks.

Hi Serge,
it clearly is clarify && !argue :-)
What you described is about what I had in mind.

With Yakkety incoming fast I have to check if all of it is still doable.
Along we identified in general that much of this recently degenerated a.k.a I miss you.

I took the action to actually define how we want to handle this (just as you to clarify && !argue) and you'll be CC on a mail about it shortly.

Awesome - thanks Christian.

The discussion is finished, the way to handle this is now defined at https://wiki.ubuntu.com/QemuKVMMigration

I'm prepping a ppa to run my newly created migration test suite against it as well as the qa tests that we discussed.

There is the usual last-minute struggle to get all in, so the ppa is based on the in-queue https://requests.ci-train.ubuntu.com/#/ticket/1931 that moves yakkety to 2.6.1 (which is great, thanks xnox!)

Please note that dev releases and especially wily is special.
It seems broken with the pc_compat_2_3 definition while the rest switched to NULL using the PC_COMPAT_ as well as the "15.04" string in its def we keep it as-is in Xenial being the unifying LTS. Everybody should already be off-wily anyway already.
According to the definition in the linked wiki u,v,w are dropped in yakkety as migrations (like an upgrade) have to go through Xenial.

ppa to test the fixed machine types is at https://launchpad.net/~paelzer/+archive/ubuntu/qemu-machine-type-dev

Test logs before the fix:
http://paste.ubuntu.com/23177826/ - shows the failed way back from T->X->T
http://paste.ubuntu.com/23177831/ - shows the lack of a proper default type

Test logs with the fixes in place:
http://paste.ubuntu.com/23181803/ - shows still the same issue, but following the guidance in bug 1536487 that isn't expected to work
http://paste.ubuntu.com/23181801/ - shows the fix to the expected types while not breaking any else

I asked smb to do his usual tests against that ppa as well to have some extra coverage.

I will go on to add the planned powerpc and s390x parts, drop the 1536487 change and run the suite again.

FYI - new bileto we depend on at https://requests.ci-train.ubuntu.com/#/ticket/1950

xnox got through with the 2.6.1 upload, now jgrimm is in the queue with ddw.
I rebased onto jon's work and will run my tests (now also ppc64le and s390x) against that then.

The machine types now look as they should:
While I have the BP ready for Xenial SRU here the Yakkety output now:

x86
ubuntu Ubuntu 16.10 PC (i440FX + PIIX, 1996) (alias of pc-i440fx-yakkety)
pc-i440fx-yakkety Ubuntu 16.10 PC (i440FX + PIIX, 1996) (default)
pc-i440fx-xenial Ubuntu 16.04 PC (i440FX + PIIX, 1996)
pc-i440fx-trusty Ubuntu 14.04 PC (i440FX + PIIX, 1996)
[...] upstream types
pc-q35-yakkety Ubuntu 16.04 PC (Q35 + ICH9, 2009)
pc-q35-xenial Ubuntu 16.04 PC (Q35 + ICH9, 2009)

ppc
[...]
pseries-2.1 pSeries Logical Partition (PAPR compliant)
pseries-2.2 pSeries Logical Partition (PAPR compliant)
pseries-2.3 pSeries Logical Partition (PAPR compliant)
pseries-2.4 pSeries Logical Partition (PAPR compliant)
pseries-2.5 pSeries Logical Partition (PAPR compliant)
pseries-2.6 pSeries Logical Partition (PAPR compliant)
pseries-xenial pSeries Logical Partition (PAPR compliant)
pseries pSeries Logical Partition (PAPR compliant) (alias of pseries-yakkety)
pseries-yakkety pSeries Logical Partition (PAPR compliant) (default)

s390
s390-ccw-virtio-2.4 VirtIO-ccw based S390 machine v2.4
s390-ccw-virtio-2.5 VirtIO-ccw based S390 machine v2.5
s390-ccw-virtio-2.6 VirtIO-ccw based S390 machine v2.6
s390-ccw-virtio-xenial VirtIO-ccw based S390 machine vxenial
s390-ccw-virtio VirtIO-ccw based S390 machine yakkety (alias of s390-ccw-virtio-yakkety)
s390-ccw-virtio-yakkety VirtIO-ccw based S390 machine yakkety (default)

Testing migration against the to-be uploaded content shows
- expected machine type fixed for s390x and ppc
- migration in&out of new code (upgrade phase!) works on s390x and ppc
- T->X migration continues to work on s390x and x86
- X->Y migration continues to work on s390x, x86 and ppc
- no regression on the in-release migration on s390x, ppc and x86
- T->X migration fails on ppc (was broken before, no change) tracked in 1625134
- P->T migration fails on x86 (was broken before, obviously no change since ppa holds no P/T packages) tracked in bug 1536331

Attaching these logs, QA tests still ongoing

smb tested the ppa as well in case we would have broken something in regard to xen, but as expected things were ok on that end (this was more about the 2.6.1 upload itself more than just our machine type changes).

Thanks smb!

Verification with the QA tests is hard atm as they are not yet made ready for yakkety.
They have some issues already and show transient issues.
I'm attaching QA test logs just in case.

On the good side - for latter Xenial SRU they seem to be pretty stable and good.

All tests that apply are good.
(even the pre QA tests I did on SRU Xenial worked)
The migration tests verified we fix what we wanted in this bug and don't introduce new ones.
Tracking of other issues was updated.

That said ready for upload, adding the Debdiff as FYI before uploading

uploaded and checking build and tests in the LP infra later on and/or tomorrow depending on the speed it passes through

This bug was fixed in the package qemu - 1:2.6.1+dfsg-0ubuntu3

---------------
qemu (1:2.6.1+dfsg-0ubuntu3) yakkety; urgency=medium

  * fix default ubuntu machine types. (LP: #1621042)
    - add dep3 header to d/p/ubuntu/define-ubuntu-machine-types.patch
    - remove double default and double ubuntu alias
    - drop former devel releases utopic, vivid, wily
    - add xenial and yakkety machine types
    - add q35 based ubuntu machine type starting at xenial
    - add ubuntu machine types on ppc64el and s390x starting at xenial

 -- Christian Ehrhardt <email address hidden> Mon, 19 Sep 2016 07:50:50 +0200

As bug 1490611 did not advance in front of me as expected I rebased onto what is released in xenial as of now.
I reran the tests, but as expected there was no change as the other bug touched totally different code.

That said the it is ready for the SRU Team to check now.
Attaching debdiff as reference and then uploading.

Subscribing the SRU Team to evaluate

And learned they just check the unapproved queue, unsubscribing ... :-)

Hello ChristianEhrhardt, or anyone else affected,

Accepted qemu into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.5+dfsg-5ubuntu10.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I was verifying proposed xenial.
All old issues gone - the only remaining one is a known issue with an open bug against it.

Attaching logs and changing tag.

This bug was fixed in the package qemu - 1:2.5+dfsg-5ubuntu10.5

---------------
qemu (1:2.5+dfsg-5ubuntu10.5) xenial; urgency=medium

  * fix default machine types. (LP: #1621042)
    - add dep3 header to d/p/ubuntu/define-ubuntu-machine-types.patch
    - remove double default and double ubuntu alias
    - add xenial machine type
    - add q35 based ubuntu machine type
    - add ubuntu machine types on ppc64el and s390x

 -- Christian Ehrhardt <email address hidden> Tue, 20 Sep 2016 09:38:03 +0200

The verification of the Stable Release Update for qemu has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.