vinagre fails to connect to any VNC server

This bug is reminiscent of bug #206227, by the way, in case there's any relation.

In my case, tunnelling through ssh doesn't work either. vinagre won't connect to anything. vncviewer connects just fine. I've tried connecting via vinagre on two separate VMs as well as on my main PC.

I can confirm this. I can not connect to a local server or even through an ssh tunnel. I'm on a regular installation.
Xvncviewer works in both cases.

I have two boxes, both running Meerkat amd64. Both boxes have the same problem as described above when trying to connect to the other box. The error message pops up and it does not connect.

Then I noticed this: I turn remote desktop sharing off by going to System -> Preferences -> Remote Desktop and un-checking the box to share my desktop. Then I open System -> Administration -> System Monitor and select the Processes tab. Vino-server is running at 100% CPU utilization when it should be turned off.

@Evan: the vino-server 100% CPU bug is #514458.

ie bug #514458 (https://bugs.launchpad.net/ubuntu/+source/vino/+bug/514458) - I thought it would create a link for me automatically in the comment above.

The problem actually lies in gtk-vnc. Running vinagre with the --gtk-vnc-debug flag shows:

(vinagre:23947): gvnc-DEBUG: vncconnection.c Protocol initialization
(vinagre:23947): gvnc-DEBUG: vncconnection.c Server version: 3.7
(vinagre:23947): gvnc-DEBUG: vncconnection.c Using version: 3.7
(vinagre:23947): gvnc-DEBUG: vncconnection.c Possible auth 18
(vinagre:23947): gvnc-DEBUG: vncconnection.c Possible auth 1
(vinagre:23947): gvnc-DEBUG: vncconnection.c Emit main context 10
(vinagre:23947): gvnc-DEBUG: vncconnection.c Thinking about auth type 18
(vinagre:23947): gvnc-DEBUG: vncconnection.c Decided on auth type 18
(vinagre:23947): gvnc-DEBUG: vncconnection.c Waiting for auth type
(vinagre:23947): gvnc-DEBUG: vncconnection.c Choose auth 18
(vinagre:23947): gvnc-DEBUG: vncconnection.c Do TLS handshake
(vinagre:23947): gvnc-DEBUG: vncconnection.c Handshake was blocking
...
(vinagre:23947): gvnc-DEBUG: vncconnection.c Completed TLS setup
(vinagre:23947): gvnc-DEBUG: vncconnection.c Got 1 subauths
(vinagre:23947): gvnc-DEBUG: vncconnection.c Got 1 subauths
(vinagre:23947): gvnc-DEBUG: vncconnection.c Possible sub-auth 1
(vinagre:23947): gvnc-DEBUG: vncconnection.c Emit main context 10
(vinagre:23947): gvnc-DEBUG: vncconnection.c Thinking about auth type 1
(vinagre:23947): gvnc-DEBUG: vncconnection.c Auth failed

"emit main context 10" means it is doing VNC_AUTH_CHOOSE_TYPE. It calls this twice, but instead should be calling VNC_AUTH_CHOOSE_TYPE followed by VNC_AUTH_CHOOSE_SUBTYPE like it does in Lucid (ie the second call should be saying "emit main context 12").

The following hack in vncconnection.c works around the problem and allows connection:

gboolean vnc_connection_set_auth_type(VncConnection *conn, unsigned int type)
{
 VncConnectionPrivate *priv = conn->priv;

        VNC_DEBUG("Thinking about auth type %u", type);

        // We are incorrectly called twice, once for auth and then again for subtype auth.
        // Because the auth_type is set (eg to TLS), the connection immediately fails.
        // This hack works around the problem:
        if (priv->auth_type != VNC_CONNECTION_AUTH_INVALID) {
            return vnc_connection_set_auth_subtype(conn, type);
        }

        if (priv->auth_type != VNC_CONNECTION_AUTH_INVALID) {

The attached patch fixes the problem, which is twofold:

a) vncconnection.c#vnc_connection_perform_auth_tls incorrectly emits a signal to set the primary authentication type instead of the subtype

b) vncdisplay.c#on_auth_choose_subtype incorrectly calls vnc_connection_set_auth_type instead of vnc_connection_set_auth_subtype.

This was fixed in gtk-vnc upstream 5 days ago: http://git.gnome.org/browse/gtk-vnc/commit/?id=6800dd5d7f665a3587f5f6eb4b44efee361d142c

@John: fwiw, I already tried applying that patch to the the ubuntu gtk-vnc, but it still wouldn't connect. I assume it requires other changes from git to be present to work properly (for instance, I had to manually apply the last change in the patch because it failed to apply).

But are you saying we'll get an update from git for gtk-vnc before final release? Isn't everything frozen now, so this won't happen because it would risk introducing new bugs? Not that it really matters because vinagre/vnc simply doesn't work in Maverick.

Yeah, I have the same problem, that patch doesn't fix it for me either.

@David: Did you try the git patch or the one I posted?

@Rocko: I tried the git patch. Just tried your patch and it works great as far as I can tell, thanks!

Is this going to be fixed for 10.10?

I can confirm vinagre won't connect to two 10.04 machines and an OS X machine on my LAN. Additionally it won't connection to the localhost (which is the only 10.10 machine I've got handy).

I hope this'll be tidied up before the final release as it's a bit of a show stopper for anyone who supports other people remotely.

I should hope it would be finished since vinagre is part of the standard desktop, and it currently doesn't work!

No guarantees this can make the 10.10 release at this point, but if not, definitely something we can get out in a 10.10 SRU.

"No guarantees"? Why would ubuntu include an application that cannot do it's most basic thing in a final release? That's just a waste of space on the CD. I would really hope that at least the basics work after installing ubuntu.

@waspinator: Before asking such questions, I recommend you first search on how many packages we ship, how many bugs we are currently working on, how many developers we have working on bugs, and how much Ubuntu costs. A little patience and understanding goes much farther, than the attitude you are displaying by your comment.

@Williamson: I'm sorry if I sounded ungrateful for the hard work of all those involved in Ubuntu, because that's the exact opposite of what I am. I am very grateful, but obvious show stopping bugs like this, prevent Ubuntu from looking professional and showcasing all the other great things that do work. I think we can agree that it is better to have fewer features that work, than many features which do not. If there aren't enough developers to support the 'extra' apps like vinagre, maybe the Ubuntu team shouldn't commit to having them as part of the default package set, and focus on a smaller, but polished selection.

@Robbie: To me, having a package installed by default in a distribution is an implicit endorsement for the package, and a guarantee that the developers have tested the package, and it works for most cases.

I am extremely grateful for everything that Ubuntu is doing, and I'm trying to give back a bit, by running beta builds and reporting bugs. However, I do agree with @waspinator's last point -- if a package is known to be completely broken, it should be pulled out. Broken auto-installed applications ruin the image of both Ubuntu and all Linux-based distributions.

This bug has been reported a month ago, is a clear show-stopper for the app, and has a patch that seems to get the package in a better state (clearly broken -> maybe broken in a subtle way). If nothing is done about it by the time 10.10 ships, I will consider that the time I'm spending testing and filing bugs is wasted.

As suggested above, it looks like this is really a gtk-vnc issue, so I'm adding a bug task for it. This is also confirmed by the fact I could apply the git patch and it seems to work; plus it applies fine once refreshed. See related branch.

Merged and sponsored, works fine! I'll close the unrelated vinagre task then.

This bug was fixed in the package gtk-vnc - 0.4.1-3ubuntu2

---------------
gtk-vnc (0.4.1-3ubuntu2) maverick; urgency=low

  [ Mathieu Trudel-Lapierre ]
  * Cherry-pick git commit 6800dd5 to fix authentication issues (LP: #634571)
    - update debian/patches/series
    - add debian/patches/lp634571_auth_subtype_logic_fix.patch

  [ Martin Pitt ]
  * debian/control: Drop Vcs-Bzr:. The old debian/ only branch is out of date,
    switching to lp:ubuntu/gtk-vnc now.
 -- Mathieu Trudel-Lapierre <email address hidden> Mon, 04 Oct 2010 08:13:08 +0200

I confirm that the current packages in maverick solve the issue without any apparent side effects. Not that further confirmation was needed, but since I'm the reporter and haven't said anything since I opened the bug...

Yup, the fix works. Thank you!!

- installed "fresch" Maverick 10.10 desktop i386 today from scratch
- trying outgoing VNC and SSH links from Maverick to Lucid (working) machine. It still fails to connect, returning "Connection to host xxx was closed" message
- backward connection from 10.04 Lucid computer works as a SSH conole link but VNC does not work

this is a serious issue...

I have 3 upgrade install of 10.10 (from 10.04). None can use vinagre. All are showing the same thing. Closing with the above message.