libvirt client doesn't handle EINTR signal properly

Bug #1092826 reported by Alexander Vershilov
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
libvirt (Ubuntu)
Fix Released
Undecided
Unassigned
Precise
Fix Released
High
Serge Hallyn

Bug Description

Libvirt client network, breaks connection as poll calls are not re-entering. This causes libvirt connection die under a heavy load
and doesn't work well with RTS that use signals. It was fixed upsteam by Peter Feiner <email address hidden>:

http://libvirt.org/git/?p=libvirt.git;a=commit;h=bfa74ebe1f8a15872b7792480493515989130e34

============================
SRU Justification:
Impact: libvirt connections can die under a heavy load
Development fix: libvirt is made to handle -EINTR.
Stable fix: backported patch from upstream
Test case: Test case in comment #20
Regression potential: Checks for -EINTR are added to existing checks for -EAGAIN.
============================

Related branches

Revision history for this message
Alexander Vershilov (qnikst) wrote :
Revision history for this message
Alexander Vershilov (qnikst) wrote :

Affected version: The Precise Pangolin

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "Patch that fixes situation (from upstream)" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

tags: added: patch
Revision history for this message
Robie Basak (racb) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better.

This appears already fixed in the current development release, so I'm marking this Fix Released.

If you'd like this fixed in Precise, then I think this is a reasonable candidate for a stable release update, but we'll need a test case to reproduce the problem for verification of a fix. As this is a race condition, perhaps something automated that bombards a process that uses libvirt with signals to try and catch it in the middle of a poll would do?

Changed in libvirt (Ubuntu):
status: New → Fix Released
Revision history for this message
Alexander Vershilov (qnikst) wrote :

Here I'm attaching a sources of a program where you can see a error.

PREREQUIENCES:
ghc, cabal (or better cabal-dev), libvirt, qemu

STEPS TO REPRODUCE:
1). cabal(-dev) configure
2). cabal(-dev) build (it will download all deps and build program)
3). run ./dist/build/test/test

EXPECTED RESULTS:
program should finish with so output and success exit code

ACTUAL RESULTS
4). program failing with error (but creating domain in libvirt)

If it needed I can send executable only it will work on Precise.

Karma Dorje (taaroa)
tags: added: precise upgrade-software-version
Revision history for this message
Kirill Zaborsky (qrilka) wrote :

Looks like patch above is some page from libvirt's gitweb.
Attaching correct plaintext patch

Revision history for this message
Robie Basak (racb) wrote :

Alexander,

Thanks for your instructions. Based on this I've written a script that should test all the way through to a failure, starting from a fresh installation of 12.04. But I can't get this to work. It fails with:

+ cabal configure
Resolving dependencies...
Configuring test-0.1.0.0...
cabal: At least the following dependencies are missing:
base ==4.6.*, libvirt-hs ==0.1.*

I'm not as familiar with Haskell as I'd like, and am not sure how to proceed from here. Please could you correct my script?

Revision history for this message
Alexander Vershilov (qnikst) wrote :

Robie,

There are 2 problems:
1). test require a newer ghc (Fixed in attachement)
2). dependencies were not installed (Fixed in script attachement)

Fixes in script will install haskell packages in system, if you want to install them into sandbox you need to use cabal-dev instead of cabal (I don't know it's status in ubuntu, so I've used cabal with system-wide install)

Revision history for this message
Alexander Vershilov (qnikst) wrote :

Fixes in script:
1). use grep for checking error
2). cleanup environment

Revision history for this message
Robie Basak (racb) wrote :

I was going to prepare a debdiff and submit for sponsoring, but it looks like Kirill has already made a submission that is awaiting review and working its way up the sponsorship queue. I presume things are a bit slower than usual as everyone's on holiday.

Revision history for this message
Kirill Zaborsky (qrilka) wrote :

I'm not quite familiar with Ubuntu developement process so I'm not sure that my submission is enough for sponsoring - the only thing I've done is applying git commit Alexander referred in bug description.

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

This only applies to 12.04 precise right?

Unfortunately we'll have to wait until the currently proposed SRU candidate is cleared or rejected. Let me try and push that...

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

0.9.8-2ubuntu17.5 is in precise-updates now, so this could be pursued. The merge request has some issues that need to be addressed first though (see the merge request).

Changed in libvirt (Ubuntu Precise):
status: New → Triaged
Revision history for this message
Kirill Zaborsky (qrilka) wrote :

I can not fork libvirt precise-updates because of #1099882 which has no reaction from Ubuntu team

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

How about just attaching a debdiff?

Revision history for this message
Kirill Zaborsky (qrilka) wrote :

Here is a debdiff

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

(Due to upcoming 12.04.2 release on Feb 14, I've marked this to work on it on Feb 15)

Changed in libvirt (Ubuntu Precise):
assignee: nobody → Serge Hallyn (serge-hallyn)
importance: Undecided → Medium
status: Triaged → In Progress
importance: Medium → High
Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

@Alexander,

could you please write a concise test case in the Description for the SRU?

description: updated
description: updated
Revision history for this message
Alexander Vershilov (qnikst) wrote :

Hello, Serge.

Automatic testcase will work:

https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1092826/+attachment/3467412/+files/test-not-working.sh

1). install haskell build system: ghc, cabal
2). install libvirt and qemu
3). run a simple haskell program that creates an domain, close connection.
Expected results: test will finishes successfully
Actual results: on step 3 there will be fail as haskell RTS uses signals to switch between threads.

same can be done in pure C if you'll send signal to a thread that is waiting data from libvirt.

Revision history for this message
Chris Halse Rogers (raof) wrote : Please test proposed package

Hello Alexander, or anyone else affected,

Accepted libvirt into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/libvirt/0.9.8-2ubuntu17.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

description: updated
Changed in libvirt (Ubuntu Precise):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Kirill Zaborsky (qrilka) wrote :

Testing script above had some errors (e.g. missing dependencies etc).
I'm attaching fixed version.
Our tests show no signs the bug in 0.9.8-2ubuntu17.8

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Thanks Kirill - to be sure I understood comment #22 correctly, you are saying the *test* script in comment #20 had some errors, but (with the fixed test case) the bug in libvirt is fixed?

tags: added: verification-done
removed: verification-needed
Revision history for this message
Kirill Zaborsky (qrilka) wrote :

Yes, Serge, you are quite right.

Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libvirt - 0.9.8-2ubuntu17.8

---------------
libvirt (0.9.8-2ubuntu17.8) precise-proposed; urgency=low

  [ Adam Conrad ]
  * libvirt-bin.postinst: also put admin group members into the libvirtd
    group, to support systems installed before precise. (LP: #1124127)
  * libvirt-bin.postinst: use getent group instead of grep /etc/group

  [ Serge Hallyn ]
  * Update README.Debian:
    - we use libvirtd, not libvirt group (LP: #1095140)
    - we add users from sudo, not admin group, to libvirtd.
  * Handle two usb devices with same vendor/id (LP: #1082213)
    - ubuntu/qemu-Keep-list-of-USB-devices-attached-to-domains
    - ubuntu/usb-create-functions-to-search-usb-device
    - ubuntu/qemu-call-usb-search-function-for-hostdev

  [ Andres Lagar-Cavilla ]
  * Add RESUME event listener to qemu monitor (LP: #1097824)
    - ubuntu/handle_resume.patch

  [ Kirill Zaborsky ]
  * Add proper handling for EINTR signal (LP: #1092826)
    - ubuntu/fix-poll.patch
 -- Serge Hallyn <email address hidden> Thu, 21 Feb 2013 08:38:35 -0600

Changed in libvirt (Ubuntu Precise):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.