libvirt

Bug #1719196
Comment #17

Comment 17 for bug 1719196

Revision history for this message

Christian Ehrhardt  (paelzer) wrote on 2017-10-11:

#17

Hi,
I was reading into this after being back from PTO (actually back next monday).
I was wondering as I tested (without openstack) just that last week with Dannf on the Rally).
And indeed it seems to work for me on Artful (which as we all know is =Ocata in terms of SW stack).

$ cat arm-template.xml
<domain type='kvm'>
        <os>
                <type arch='aarch64' machine='virt'>hvm</type>
                <loader readonly='yes' type='pflash'>/usr/share/AAVMF/AAVMF_CODE.fd</loader>
                <nvram template='/usr/share/AAVMF/AAVMF_CODE.fd'>/tmp/AAVMF_CODE.fd</nvram>
                <boot dev='hd'/>
        </os>
        <features>
                <acpi/>
                <apic/>
                <pae/>
        </features>
        <cpu mode='custom' match='exact'>
                <model fallback='allow'>host</model>
        </cpu>
        <devices>
                <interface type='network'>
                        <source network='default'/>
                        <model type='virtio'/>
                </interface>
                <serial type='pty'>
                        <source path='/dev/pts/3'/>
                        <target port='0'/>
                </serial>
        </devices>
</domain>

$ uvt-kvm create --template arm-template.xml --password=ubuntu artful-test4 release=artful arch=arm64 label=daily

The template is created in a way to let as much as possible for libvirt and qemu to fill in defaults. That way if one of them change we do not have to follow and adapt.
Maybe such a thing is happening to you for the more "defined" xml that openstack is sending.

From the hosts point of view all looks normal in journal, you see start and dhcp discover/offer/ack sequence:
Okt 11 10:45:43 seidel kernel: virbr0: port 2(vnet0) entered learning state
Okt 11 10:45:45 seidel kernel: virbr0: port 2(vnet0) entered forwarding state
Okt 11 10:45:45 seidel kernel: virbr0: topology change detected, propagating
Okt 11 10:46:14 seidel dnsmasq-dhcp[2610]: DHCPDISCOVER(virbr0) 52:54:00:b1:db:89
Okt 11 10:46:14 seidel dnsmasq-dhcp[2610]: DHCPOFFER(virbr0) 192.168.122.13 52:54:00:b1:db:89
Okt 11 10:46:14 seidel dnsmasq-dhcp[2610]: DHCPDISCOVER(virbr0) 52:54:00:b1:db:89
Okt 11 10:46:14 seidel dnsmasq-dhcp[2610]: DHCPOFFER(virbr0) 192.168.122.13 52:54:00:b1:db:89
Okt 11 10:46:14 seidel dnsmasq-dhcp[2610]: DHCPREQUEST(virbr0) 192.168.122.13 52:54:00:b1:db:89
Okt 11 10:46:14 seidel dnsmasq-dhcp[2610]: DHCPACK(virbr0) 192.168.122.13 52:54:00:b1:db:89 ubuntu

The guest also seems "normal" to me:
$ uvt-kvm ssh artful-test4 -- journalctl -u systemd-networkd
-- Logs begin at Wed 2017-10-11 10:46:03 UTC, end at Wed 2017-10-11 11:02:31 UTC. --
Oct 11 10:46:11 ubuntu systemd[1]: Starting Network Service...
Oct 11 10:46:11 ubuntu systemd-networkd[547]: Enumeration completed
Oct 11 10:46:11 ubuntu systemd[1]: Started Network Service.
Oct 11 10:46:11 ubuntu systemd-networkd[547]: enp1s0: IPv6 successfully enabled
Oct 11 10:46:11 ubuntu systemd-networkd[547]: enp1s0: Gained carrier
Oct 11 10:46:12 ubuntu systemd-networkd[547]: enp1s0: Gained IPv6LL
Oct 11 10:46:14 ubuntu systemd-networkd[547]: enp1s0: DHCPv4 address 192.168.122.13/24 via 192.168.122.1
Oct 11 10:46:14 ubuntu systemd-networkd[547]: Not connected to system bus, ignoring transient hostname.
Oct 11 10:46:24 ubuntu systemd-networkd[547]: enp1s0: Configured

The biggest and obviously most important difference is in the networking that is used:
    <interface type='network'>
      <mac address='52:54:00:b1:db:89'/>
      <source network='default' bridge='virbr0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>

While openstack generates a type="bridge" network where the bridge is not managed by libvirt (compared to the default net I use).
Never the less both setups create a bridge and tap the guest on it.
So this should functionally be equivalent other than the bridge setup right?

I tried to make a bridge type config matching to what I had before but close to yours:
    <interface type='bridge'>
      <mac address='52:54:00:b1:db:89'/>
      <source bridge='virbr0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='virtio-mmio'/>
    </interface>

But this again works:
ubuntu@artful-test4:~$ journalctl -u systemd-networkd --no-pager
-- Logs begin at Wed 2017-10-11 11:12:10 UTC, end at Wed 2017-10-11 11:13:00 UTC. --
Oct 11 11:12:17 artful-test4 systemd[1]: Starting Network Service...
Oct 11 11:12:17 artful-test4 systemd-networkd[604]: Enumeration completed
Oct 11 11:12:17 artful-test4 systemd[1]: Started Network Service.
Oct 11 11:12:17 artful-test4 systemd-networkd[604]: enp1s0: IPv6 successfully enabled
Oct 11 11:12:17 artful-test4 systemd-networkd[604]: enp1s0: Gained carrier
Oct 11 11:12:18 artful-test4 systemd-networkd[604]: enp1s0: Gained IPv6LL
Oct 11 11:12:20 artful-test4 systemd-networkd[604]: enp1s0: DHCPv4 address 192.168.122.14/24 via 192.168.122.1
Oct 11 11:12:20 artful-test4 systemd-networkd[604]: Not connected to system bus, ignoring transient hostname.
Oct 11 11:12:30 artful-test4 systemd-networkd[604]: enp1s0: Configured

Dumping the XML to check if it might have rewritten a lot shows me that my config is good:
    <interface type='bridge'>
      <mac address='52:54:00:b1:db:89'/>
      <source bridge='virbr0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='virtio-mmio'/>
    </interface>

So TL;DR a config almost the same as yours works.
Yet the difference is that livbirt has set up 'virbr0' in my case and I'd assume openstack did create qbrb5d335fc-46 on its own.

As next step I'd recommend iterating your config around different bridge scenarios to find what breaks it.
Then if reasonable try to exclude openstack from the equation as I shown above (or not if you think the fix needs to be in openstack).

I hope this helped to get closer to the root casue, but I thought that a working example on the new Ocata stack might be the best start with.

Note this was done on:
libvirt-daemon-system 3.6.0-1ubuntu4
qemu-kvm 1:2.10+dfsg-0ubuntu1

$ cat arm-template.xml 
<domain type='kvm'>
        <os>
                <type arch='aarch64' machine='virt'>hvm</type>
                <loader readonly='yes' type='pflash'>/usr/share/AAVMF/AAVMF_CODE.fd</loader>
                <nvram template='/usr/share/AAVMF/AAVMF_CODE.fd'>/tmp/AAVMF_CODE.fd</nvram>
                <boot dev='hd'/>
        </os>
        <features>
                <acpi/>
                <apic/>
                <pae/>
        </features>
        <cpu mode='custom' match='exact'>
                <model fallback='allow'>host</model>
        </cpu>
        <devices>
                <interface type='network'>
                        <source network='default'/>
                        <model type='virtio'/>
                </interface>
                <serial type='pty'>
                        <source path='/dev/pts/3'/>
                        <target port='0'/>
                </serial>
        </devices>
</domain>

$ uvt-kvm create --template arm-template.xml --password=ubuntu artful-test4 release=artful arch=arm64 label=daily

I tried to make a bridge type config matching to what I had before but close to yours:
    <interface type='bridge'>
      <mac address='52:54:00:b1:db:89'/>
      <source bridge='virbr0'/>
      <target dev='vnet0'/>   
      <model type='virtio'/>
      <alias name='net0'/>                
      <address type='virtio-mmio'/>
    </interface>

So TL;DR a config almost the same as yours works.
Yet the difference is that livbirt has set up 'virbr0' in my case and I'd assume openstack did create qbrb5d335fc-46 on its own.

I hope this helped to get closer to the root casue, but I thought that a working example on the new Ocata stack might be the best start with.

Note this was done on:
libvirt-daemon-system          3.6.0-1ubuntu4
qemu-kvm                       1:2.10+dfsg-0ubuntu1