Error: viewer connection to hypervisor host got refused or disconnected!

Bug #837275 reported by Paul van Genderen
36
This bug affects 6 people
Affects Status Importance Assigned to Milestone
virt-manager (Ubuntu)
Fix Released
Undecided
Marc Deslauriers
Precise
Fix Released
Undecided
Marc Deslauriers
Quantal
Fix Released
Undecided
Marc Deslauriers

Bug Description

Test Case:
1- Prepare remote libvirt server with a VM that has VNC configured to listen on all interfaces (0.0.0.0)
2- Using a Precise machine, attempt to connect using virt-manager, and open the VM's display
3- This should not result in an error message saying that "0.0.0.0" cannot be reached

Everything was working fine until today, no display devices (VNC, SPICE, SDL) work. Previously, VNC (the default) worked fine (30 hours ago).
---
Architecture: amd64DistroRelease: Ubuntu 11.04InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Release amd64 (20110426)
NonfreeKernelModules: fglrx
Package: virt-manager 0.8.6-1ubuntu8.1
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_GB:en
 PATH=(custom, user)
 LANG=nl_NL.UTF-8
 LC_MESSAGES=en_GB.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.38-10.46-generic 2.6.38.7Tags: natty
Uname: Linux 2.6.38-10-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm admin cdrom dialout libvirtd lpadmin netdev plugdev sambashare

Revision history for this message
Paul van Genderen (paulvg) wrote : Dependencies.txt

apport information

tags: added: apport-collected natty
description: updated
Revision history for this message
Paul van Genderen (paulvg) wrote :

Workaround:

sudo apt-get install xtightvncviewer
xtightvncviewer localhost

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

Thanks for reporting this issue.

Are you able to reproduce it? If so, could you please give detailed steps on how to do so?

Thanks!

Changed in virt-manager (Ubuntu):
status: New → Incomplete
Revision history for this message
Nathan Bird (ecthellion) wrote :

Very odd: it worked yesterday, and not this morning. I'm on 12.04 beta so a lot of packages are getting updated but I think this might be in virt-manager somewhere.

Guessing at reproduction:
1. Install virt-manager
2. Connect to libvirt running on a different host using qemu+ssh connection. E.g. `virt-manager -c qemu+ssh://root@hypervisor/system`
3. Double click on a running VM to see its VNC connection.
4. See "Error: viewer connection to hypervisor host go refused or disconnected! Error: ssh: connect to host 127.0.0.1 port 5900: Connection refused."
5. `virt-viewer -c qemu+ssh://root@hypervisor/system my-vm` does work

I believe the *different* host is the key. Reading ~/.virt-manager/virt-manager.log from days ago and now it looks like the command it is using has changed to no longer try to connect remotely.

From a few days ago when it worked:
[Mon, 02 Apr 2012 15:46:09 virt-manager 19288] DEBUG (console:1081) Starting connect process for proto=vnc trans=ssh connhost=hypervisor connuser= connport=None gaddr=127.0.0.1 gport=5903 gsocket=None
[Mon, 02 Apr 2012 15:46:09 virt-manager 19288] DEBUG (console:109) Creating SSH tunnel: ssh hypervisor sh -c 'nc -q 2>&1 | grep "requires an argument" >/dev/null;if [ $? -eq 0 ] ; then CMD="nc -q 0 127.0.0.1 5903";else CMD="nc 127.0.0.1 5903";fi;eval "$CMD";'
[Mon, 02 Apr 2012 15:46:09 virt-manager 19288] DEBUG (console:132) Tunnel PID=18467 OUTFD=56 ERRFD=58
[Mon, 02 Apr 2012 15:46:09 virt-manager 19288] DEBUG (console:992) Viewer connected
[Mon, 02 Apr 2012 15:55:56 virt-manager 19288] DEBUG (console:150) Shutting down tunnel PID=18467 OUTFD=56 ERRFD=58
[Mon, 02 Apr 2012 15:55:56 virt-manager 19288] DEBUG (console:964) Viewer disconnected

Today:
[Wed, 04 Apr 2012 08:17:38 virt-manager 26335] DEBUG (console:1075) Starting connect process for proto=vnc trans=ssh connhost=127.0.0.1 connuser=root connport=5901 gaddr=127.0.0.1 gport=5901 gsocket=None
[Wed, 04 Apr 2012 08:17:38 virt-manager 26335] DEBUG (console:109) Creating SSH tunnel: ssh -p 5901 -l root 127.0.0.1 sh -c 'nc -q 2>&1 | grep "requires an argument" >/dev/null;if [ $? -eq 0 ] ; then CMD="nc -q 0 127.0.0.1 5901";else CMD="nc 127.0.0.1 5901";fi;eval "$CMD";'
[Wed, 04 Apr 2012 08:17:38 virt-manager 26335] DEBUG (console:132) Tunnel PID=27507 OUTFD=24 ERRFD=26
[Wed, 04 Apr 2012 08:17:38 virt-manager 26335] DEBUG (console:150) Shutting down tunnel PID=27507 OUTFD=24 ERRFD=26

In the first it is trying to connect to the hypervisor, in the first it is trying to connect to localhost (which isn't doing anything).

Revision history for this message
Nathan Bird (ecthellion) wrote :

The problem persists after I removed all of my ~/.ssh/config settings to do with the remote server to ensure that ControlMaster or any existing port forwards weren't getting in the way.

Revision history for this message
Nathan Bird (ecthellion) wrote :

Well I verified that the version i had that was working was 0.9.1-1ubuntu2 and 0.9.1-1ubuntu3 doesn't. So the bug was introduced in http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/precise/virt-manager/precise/revision/58/src/virtManager/domain.py

So this is probably different than Paul van Genderen's problem was.

Since I've been looking at this 0.9.1-1ubuntu4 has been released. While apt-get update+upgrade doesn't give it to me yet, looking at http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/precise/virt-manager/precise/revision/59?start_revid=59 doesn't look likely to fix it.

Looking at the upstream code repos from http://virt-manager.org/scmrepo.html the code there that is relevant to this has changed significantly, but introduces a new version of this bug that causes the same problem. I'm going to go try to file a bug+patch up there.

Revision history for this message
Nathan Bird (ecthellion) wrote :

> ... but introduces a new version of this bug that causes the same problem.

Patch has been accepted upstream. Re-merging from upstream into Ubuntu's repo will at least fix my instance of this problem; and as the code has been refactored somewhat along the way might fix Paul van Genderen's as well.

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

OK, so the fix_listen_address.patch now honors the "listen" attribute present in the remote libvirt xml file. Unfortunately, some server have "127.0.0.1" specified there, which results in the connection being wrong. Upstream code now special-cases "127.0.0.1", and fix_listen_address.patch now needs to do that too.

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

I have uploaded a fix for this issue in my testing PPA here:

https://launchpad.net/~mdeslaur/+archive/testing

Could you please test it in your environment once it finishes building, and comment if it works? If so, I'll upload it to precise.

Thanks!

Revision history for this message
Nathan Bird (ecthellion) wrote :

The version 0.9.1-1ubuntu5 that I pulled from Marc's ppa fixes it for me! :-)

https://launchpad.net/~mdeslaur/+archive/testing

Changed in virt-manager (Ubuntu):
status: Incomplete → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package virt-manager - 0.9.1-1ubuntu5

---------------
virt-manager (0.9.1-1ubuntu5) precise; urgency=low

  * debian/patches/fix_listen_address.patch: updated to prevent regression
    when the remote server has a listen attribute that specifies
    "127.0.0.1". Upstream code which has been completely refactored
    essentially does this now too. (LP: #837275)
 -- Marc Deslauriers <email address hidden> Fri, 06 Apr 2012 11:13:49 -0400

Changed in virt-manager (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Sven Arnold (sven-internetallee) wrote :

Unfortunately in my environment this fix is not sufficient

Server:
Ubuntu 10.04
qemu is listening on all public interfaces: (vnc_listen = "0.0.0.0")

Client:
Ubuntu 12.04
virt-manager

Calling virt-viewer directy is working, but from virt-manager the connection is refused.
In this case the .virt-manager/virt-manager.log file states:

DEBUG (console:1075) Starting connect process for proto=vnc trans=tls connhost=0.0.0.0 connuser= connport=5904 gaddr=0.0.0.0 gport=5904 gsocket=None
DEBUG (console:958) Viewer disconnected

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

Ah, yes, we're also missing a check for 0.0.0.0...I'll SRU a fix.

Changed in virt-manager (Ubuntu):
assignee: nobody → Marc Deslauriers (mdeslaur)
status: Fix Released → Confirmed
Changed in virt-manager (Ubuntu Precise):
status: New → Confirmed
assignee: nobody → Marc Deslauriers (mdeslaur)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package virt-manager - 0.9.1-3ubuntu1

---------------
virt-manager (0.9.1-3ubuntu1) quantal; urgency=low

  * Merge from debian unstable. Remaining changes:
    - debian/control, debian/rules: Build using dh_python2
    - debian/control: Depend on python-appindicator for appindicator
      support.
    - Add a /usr/share/pixmaps/virt-manager-icon.svg symlink to link icon to
      where the Application Indicator can find it.
    - debian/patches/more_helpful_error_message.patch: explain to the user
      why he can't connect to qemu:///system and what he can do fix it.
    - debian/control: drop python-spice-client-gtk to Suggests as it is in
      universe.
    - debian/patches/use_ubuntu_package_names.patch: Suggest installing the
      packages that are actually available in Ubuntu.
    - debian/rules: Set qemu user to libvirt-qemu so appropriate
      permissions get set.
    - debian/rules: Set Ubuntu as the preferred distro so we appear first
      in the list.
    - debian/rules: disable TUI for now, since the required dependencies
      are not available (Newt Syrup).
    - debian/rules: Drop patchsys-quilt include since dpkg-source does the
      quilt dance for us.
    - Removed python-ipy dependency as it is in universe:
      - debian/control: remove python-ipy from Depends
      - debian/series: disable 0002-Use-IPy-from-python-ipy.patch patch so
        we use the one that's included in the virt-manager source.
      - debian/rules: don't delete the IPy file.
    - debian/patches/fork_before_gtk_import.patch: work around global menu
      and appindicator not working correctly by forking before the gtk
      import.
    - debian/patches/fix_accidental_recursion.patch: fix accidental recursion
      when reporting grab keys.
    - debian/patches/fix_listen_address.patch: fix connecting when graphics
      configuration uses graphics attribute.
    - debian/patches/fix_dbus_deprecation_warning.patch: fix deprecation
      warning by switching to new way of setting up the event loop.
  * Removed patches:
    - debian/patches/fix_error_reporting.patch: has equivalent Debian
      patch.
  * Dropped changes:
    - debian/control: lower libvirt-bin from Recommends to Suggests: we
      depend on python-libvirt which in turn pulls in libvirt-bin anyway.
  * debian/patches/fix_listen_address.patch: updated again to prevent
    regression when the remote server has a listen attribute that specifies
    "0.0.0.0". (LP: #837275)
  * debian/patches/more_helpful_error_message.patch: updated for new
    version and to fix error dialog. (LP: #998724)

virt-manager (0.9.1-3) unstable; urgency=low

  * [a1de813] Add presubj file to (hopefully) increase the bug report quality.

virt-manager (0.9.1-2) unstable; urgency=low

  * [d0d2014] manager: Fix error reporting when can't connect remotely.
    Thanks to Cole Robinson (Closes: #659860)
 -- Marc Deslauriers <email address hidden> Thu, 24 May 2012 11:13:42 -0400

Changed in virt-manager (Ubuntu Quantal):
status: Confirmed → Fix Released
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

SRU Request:

Impact: Users who remotely connect to a server that has "0.0.0.0" as the vnc listen interface cannot use the remote console. This is a regression from previous ubuntu releases.

Development fix: This was fixed in Quantal by adding "0.0.0.0" to the list of exceptions in the fix_listen_address.patch patch.

Stable fix: A minimal debdiff was prepared to fix Precise in the same fashion.

Test Case:
1- Prepare remote libvirt server with a VM that has VNC configured to listen on all interfaces (0.0.0.0)
2- Using a Precise machine, attempt to connect using virt-manager, and open the VM's display
3- This should not result in an error message saying that "0.0.0.0" cannot be reached

Regression potential:
If there is something wrong with this simple fix, it would only affect remote libvirt users.

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :
description: updated
Revision history for this message
Clint Byrum (clint-fewbar) wrote : Please test proposed package

Hello Paul, or anyone else affected,

Accepted virt-manager into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in virt-manager (Ubuntu Precise):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package virt-manager - 0.9.1-1ubuntu5.1

---------------
virt-manager (0.9.1-1ubuntu5.1) precise-proposed; urgency=low

  * debian/patches/fix_listen_address.patch: updated again to prevent
    regression when the remote server has a listen attribute that specifies
    "0.0.0.0". (LP: #837275)
  * debian/patches/more_helpful_error_message.patch: remove superfluous
    line and extra comma to fix error dialog. (LP: #998724)
 -- Marc Deslauriers <email address hidden> Thu, 24 May 2012 15:35:48 -0400

Changed in virt-manager (Ubuntu Precise):
status: Fix Committed → Fix Released
description: updated
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.