[Impact]
* Libvirt does an overzealous check on concurrent hash usage which breaks
some automation like for example Terraform. Upstream moved the need to
lock up the stack where applicable and dropped the checks as they were
superfluous.
* Backport the upstream change dropping the extra checks on hash usage
[Test Case]
* Run the attached script test-lp1789659.sh
(based on ideas of https://bugzilla.redhat.com/show_bug.cgi?id=1581364#c7)
With the aforementioned patch, this error is not produced any more.
[Regression Potential]
* If the assumption that all remaining callers are safe isn't correct
there could be code using invalid hashes accessing guest information.
This needs thorough tests and regression checks as well as an extra
review if it can be applied to the libvirt 4.0 we have in Bionic.
This was done by me and also upstream did the same.
Quote from the patch: "Most important uses seem to be covered. Callers
have now a greater responsability, notably the ability to execute some
operations while iterating were reliably forbidden before are now
accepted."
It is important to see that the internal locking was inherently racy
(boolean) so we are replaceing a broken lock-ckeck with no check as
upper layers were supposed to do it correctly. That is true for what is
in the archive but there could be out-of archive things not doing so
calling directly into the python api bindings for example.
Never the less those are today just as affected by the locking being
broken, so those do not "loose" a lot.
All major exploiters we know of are good - the rest is a regression
risk we are willing to take in favor of fixing all the valid use cases
being affected by the broken lock handling.
[Other Info]
* n/a
----
In the SUSE Manager/Uyuni[1] project CIs we make use of Terraform[2], a tool to automate infrastructure deployments, together with terraform-provider-libvirt[3], a Terraform provider (plugin) that allow interaction with libvirt.
By default, this setup will issue several libvirt API calls concurrently in an hard-to-predict order, as demonstrated by logs that I am attaching.
In the "bad log" example, the following line appears:
error : virHashSearch:727 : Hash operation not allowed during iteration
According to analysis of a similar problem by Red Hat in an OpenStack scenario[4], this is has been fixed in upstream libvirt via commit 4d7384eb9ddef2008cb0cc165eb808f74bc83d6b [5].
I tested the patch and it applies cleanly to the 4.0.0 package shipping with Bionic and that successfully resolves this issue.
Please evaluate including this patch.
Thanks in advance
[1] http://uyuni-project.org/
[2] https://www.terraform.io/
[3] https://github.com/dmacvicar/terraform-provider-libvirt
[4] https://bugzilla.redhat.com/show_bug.cgi?id=1576464#c3
[5] https://github.com/libvirt/libvirt/commit/4d7384eb9ddef2008cb0cc165eb808f74bc83d6b.patch
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: libvirt-daemon 4.0.0-1ubuntu8.3
ProcVersionSignature: Ubuntu 4.15.0-33.36-generic 4.15.18
Uname: Linux 4.15.0-33-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.9-0ubuntu7.2
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Wed Aug 29 16:05:10 2018
InstallationDate: Installed on 2014-06-12 (1539 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140417)
SourcePackage: libvirt
UpgradeStatus: Upgraded to bionic on 2018-05-02 (119 days ago)
This is upstram since 4.3 and thereby fixed in Cosmic.
Haven't had the time to look in depth and consider an SRU for Bionic ...