[Impact]
* the machine type for wily (which we keep for migration from an early
xenials qemu) is broken >=qemu 2.11
* fix by correcting the definition of that type
[Test Case]
- Xenial / Bionic system
$ lxc launch ubuntu-daily:x x-wily --profile default --profile kvm
$ lxc launch ubuntu-daily:b b-wily --profile default --profile kvm
- set hostid to be different (as we have containers)
$ vim /etc/libvirt/libvirtd.conf
$ systemctl restart libvirtd
- exchange ssh keys
$ ssh-keygen
$ cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
(host)
$ lxc file pull --recursive x-wily/root/.ssh .
$ lxc file push --recursive .ssh b-wily/root/
$ lxc exec b-wily -- chown -R root:root /root/.ssh
- use uvtool to create the same guest on both systems (FS layout of images)
$ time uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily \
arch=amd64 label=daily release=eoan
$ uvt-kvm create --password ubuntu wilymigrate arch=amd64 release=eoan label=daily
- edit to set wily machine type (on source = Xenial)
$ virsh edit wilymigrate
- on target remove the former definition
$ virsh undefine wilymigrate
- Migrate
$ virsh migrate --unsafe --live wilymigrate qemu+ssh://10.253.194.250/system
error: internal error: qemu unexpectedly closed the monitor: 2019-05-22T13:04:19.108689Z qemu-
system-x86_64: warning: Unknown firmware file in legacy mode: etc/msr_feature_control
2019-05-22T13:04:19.151216Z qemu-system-x86_64: Configuration section missing
2019-05-22T13:04:19.151336Z qemu-system-x86_64: load of migration failed: Invalid argument
With the fix on the target the issue is gone.
[Regression Potential]
The purpose to keep these old types is only to allow people to "migrate off" the older releases as long as they are still supported. Never the less sooner or later people are strongly encouraged [1]. Given your great analysis you know that already, but others might come by this bug.
It is important to be considered when deciding on if/what to change.
No one should ever start a "new" guest of a wily type on Bionic or later.
So I'm not too concerned about the delta we introduce to people having done that. The time this took to be found confirms that even migrations from Xenial with a Wily type are rare. We can not really differ between wily types coming from e.g. Xenial (or Trusty-Mitaka) hosts and wily type migrations coming from other Bionic systems (only a problem after a fix to this bug here and if they are on a different patch level).
So as mentioned the one real case for the wily type to still exist is to migrate off of older systems, and that use-case is broken. So I'm considering the thoughts above as a known, but less important than "the main use case" regression.
[Other Info]
* n/a
---
In qemu 2.11 pc-i440fx-wily machine type is defined the following way by the ubuntu patch:
101 +static void pc_wily_machine_options(MachineClass *m)
102 +{
103 + pc_i440fx_2_4_machine_options(m);
104 + pc_i440fx_machine_options(m);
105 + m->desc = "Ubuntu 15.04 PC (i440FX + PIIX, 1996)",
106 + m->default_display = "std";
107 +}
108 +
109 +DEFINE_I440FX_MACHINE(wily, "pc-i440fx-wily", pc_compat_2_3,
110 + pc_wily_machine_options);
In qemu 2.5, pc_compat_2_3 contained the following skip flags: https://github.com/qemu/qemu/blob/v2.5.1.1/hw/i386/pc_piix.c#L304-L313 (skip configuration, skip section footers, and optional global state)
in qemu 2.11 those skips moved to pc_i440fx_2_3_machine_options:
https://github.com/qemu/qemu/blob/v2.11.2/hw/i386/pc_piix.c#L314-L320
https://github.com/qemu/qemu/blob/v2.11.2/hw/i386/pc_piix.c#L524-L529
https://github.com/qemu/qemu/blob/v2.11.2/include/hw/i386/pc.h#L573-L574
https://github.com/qemu/qemu/blob/v2.11.2/include/hw/compat.h#L193-L205
It happened in commits:
https://github.com/qemu/qemu/commit/71dd4c1a5672cafe9fb89abc83fe2a962f39ec42
https://github.com/qemu/qemu/commit/15c38503253bb9ba9b8efd17662069f69ca2b997
https://github.com/qemu/qemu/commit/5272298c48eb3a01c41a7822e6303d0a0a05f004
but pc_wily_machine_options still refers to pc_i440fx_2_4_machine_options instead of pc_i440fx_2_3_machine_options, migration config section is not being skipped on destination host and so during migration the issue happens:
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm-spice -name guest=instance-00054361,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-22-instance-00054361/master-key.aes -machine pc-i440fx-wily,accel=kvm,usb=off,dump-guest-core=off -cpu Broadwell -m 32768 -realtime mlock=off -smp 4,sockets=1,cores=4,threads=1 -uuid 660fed6d-bb56-4e15-b866-007419be4cf3 -smbios 'type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=15.1.5,serial=7074a01b-b759-4e91-978a-fde846e2ec9e,uuid=660fed6d-bb56-4e15-b866-007419be4cf3,family=Virtual Machine' -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-22-instance-00054361/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -object secret,id=virtio-disk0-secret0,data=xx,keyid=masterKey0,iv=xx,format=base64 -drive 'file=rbd:ephemeral-vms-ssd/660fed6d-bb56-4e15-b866-007419be4cf3_disk:id=nova:auth_supported=cephx\;none:mon_host=10.154.29.44\:6789\;10.154.29.60\:6789\;10.154.29.76\:6789,file.password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,cache=writeback,discard=unmap,throttling.iops-total=1000' -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -object secret,id=ide0-0-0-secret0,data=xx,keyid=masterKey0,iv=xx,format=base64 -drive 'file=rbd:ephemeral-vms-ssd/660fed6d-bb56-4e15-b866-007419be4cf3_disk.config:id=nova:auth_supported=cephx\;none:mon_host=10.154.29.44\:6789\;10.154.29.60\:6789\;10.154.29.76\:6789,file.password-secret=ide0-0-0-secret0,format=raw,if=none,id=drive-ide0-0-0,readonly=on,cache=writeback,discard=unmap,throttling.iops-total=1000' -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=55,id=hostnet0,vhost=on,vhostfd=58 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:60:27:5a:aa:ec,bus=pci.0,addr=0x3 -add-fd set=2,fd=60 -chardev file,id=charserial0,path=/dev/fdset/2,append=on -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 10.170.4.69:13 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -incoming defer -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on
2019-05-14 11:53:57.583+0000: Domain id=22 is tainted: shell-scripts
2019-05-14T11:53:57.608860Z qemu-system-x86_64: -chardev pty,id=charserial1: char device redirected to /dev/pts/14 (label charserial1)
2019-05-14T11:53:57.978684Z qemu-system-x86_64: Configuration section missing
2019-05-14T11:53:57.978786Z qemu-system-x86_64: load of migration failed: Invalid argument
2019-05-14 11:53:57.998+0000: shutting down, reason=failed
Status changed to 'Confirmed' because the bug affects multiple users.