Comment 30 for bug 1681839

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

For the sake of seeing it trigger at least once I redeployed a mchine with Xenial to create LVMs there on a free /dev/sdb disk as your example needs it.
# create guest
$ uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=xenial
$ uvt-kvm create xsnaptest arch=amd64 release=xenial label=daily

# create Volume
$ sudo pvcreate /dev/sdb
$ sudo vgcreate LVMpool_vg /dev/sdb
$ cat > lvmpool.xml <<EOF
<pool type="logical">
<name>LVMpool_vg</name>
<source>
<device path="/dev/sdb"/>
</source>
<target>
<path>/dev/LVMpool_vg</path>
</target>
</pool>
EOF
$ virsh pool-define lvmpool.xml
$ sudo vgcreate LVMpool_vg /dev/sdb
$ virsh pool-start LVMpool_vg
$ virsh vol-create-as LVMpool_vg lvvol1 15G

# Use volume in the guest
$ cat > lvmdisk.xml <<EOF
<disk type='block' device='disk'>
  <driver name='qemu' type='raw'/>
  <source dev='/dev/LVMpool_vg/lvvol1'/>
  <target dev='vdc' bus='virtio'/>
</disk>
EOF
$ virsh attach-device xsnaptest lvmdisk.xml

# Prep initial snapshot
virsh snapshot-create-as --domain xsnaptest --diskspec vdc,file=/var/lib/libvirt/images/xsnaptest-snapshot.qcow2,snapshot=external --disk-only --atomic

# Check snapshot being backed by lvmdisk
$ sudo qemu-img info /var/lib/libvirt/images/xsnaptest-snapshot.qcow2
image: /var/lib/libvirt/images/xsnaptest-snapshot.qcow2
file format: qcow2
virtual size: 15G (16106127360 bytes)
disk size: 196K
cluster_size: 65536
backing file: /dev/LVMpool_vg/lvvol1
backing file format: raw
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

# dump I/O onto that device from inside the guest
$ while /bin/true; do uvt-kvm ssh --insecure xsnaptest "sudo dd if=/dev/urandom of=/dev/vdc count=8192 bs=1M"; done

# Iterate on it while the disk/snapshot keeps getting dirty
$ while virsh blockcommit xsnaptest vdc --active --verbose --pivot --wait; do sudo rm /var/lib/libvirt/images/xsnaptest-snapshot.qcow2; sleep 2s; virsh snapshot-create-as --domain xsnaptest --diskspec vdc,file=/var/lib/libvirt/images/xsnaptest-snapshot.qcow2,snapshot=external --disk-only --atomic; sleep $(( RANDOM % 30 + 20 ))s; sudo ls -laFh /var/lib/libvirt/images/xsnaptest-snapshot.qcow2; done

Finally I saw it in action
Block commit: [100 %]error: failed to pivot job for disk vdc
error: block copy still active: disk 'vdc' not ready for pivot yet

I retried and this was reproducible.

I upgraded to the PPA (more about that later) and ran my loop.
It reached 100% and then got slow (I/O while doing the pivot).
I needed to either wait quite a while or slow down the ongoing I/O a bit.

I had the loop running a 10 times and with the fix it never failed again (sized between 519M and 7.1G).