Open VM Tools FTBS on Maverick 2.6.35

Patch to allow vmxnet to compile. vmxnet still doesn't appear to *work* (AMD64), but I don't believe that is the fault of this patch. Both vmxnet and vmxnet3 load, but neither provide a usable network connection when restarting the network layer. The same is true of the official vmxnet3 driver provided by VMware Tools (installed for testing in a different VM). And at least this patch keeps vmxnet from blocking the install.

Second patch to fix vsock. Corrects all references of somevar->sk_compat_sleep to compat_sk_sleep(somevar)

Also reported at http://communities.vmware.com/thread/273917

Noel: There was a recent new upstream release of open-vm-tools (open-vm-tools-2010.06.16-268169, available at http://sourceforge.net/projects/open-vm-tools/files/) that Debian (and therefore Ubuntu) hasn't picked up yet. I was hoping to pull a package together for testing, but have you checked to see if that fixes the FTBFS?

Not, yet, Evan. I was just going to post that there is a new upstream version ( http://sourceforge.net/mailarchive/message.php?msg_name=4C1BE732.1090903%40vmware.com), but haven't pulled and checked it yet. Doing so now.

The new build still have the same failure in vmxnet:

Using 2.6.x kernel build system.
make[2]: Entering directory `/home/noel/open-vm-tools-2010.06.16-268169/modules/linux/vmxnet'
make -C /lib/modules/2.6.35-5-generic/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
   MODULEBUILDDIR=/home/noel/open-vm-tools-2010.06.16-268169/modules/linux modules
make[3]: Entering directory `/usr/src/linux-headers-2.6.35-5-generic'
  CC [M] /home/noel/open-vm-tools-2010.06.16-268169/modules/linux/vmxnet/vmxnet.o
/home/noel/open-vm-tools-2010.06.16-268169/modules/linux/vmxnet/vmxnet.c: In function ‘vmxnet_load_multicast’:
/home/noel/open-vm-tools-2010.06.16-268169/modules/linux/vmxnet/vmxnet.c:2954: error: ‘struct net_device’ has no member named ‘mc_list’
/home/noel/open-vm-tools-2010.06.16-268169/modules/linux/vmxnet/vmxnet.c:2964: error: ‘struct net_device’ has no member named ‘mc_count’
/home/noel/open-vm-tools-2010.06.16-268169/modules/linux/vmxnet/vmxnet.c:2965: error: dereferencing pointer to incomplete type
/home/noel/open-vm-tools-2010.06.16-268169/modules/linux/vmxnet/vmxnet.c:2966: error: dereferencing pointer to incomplete type

I do need to correct, as an aside, that vmxnet3 is not coming from open-vm-tools. It is now provided with the kernel (still does not work, though).

Yes, the new tools appear to build and work, with the vmxnet patch (mind you, neither vmxnet driver is providing a connection, so we use e1000).

vmsync 3992 0
vsock 44947 0
vmhgfs 60011 0
vmci 47215 2 vsock,vmhgfs
vmware_balloon 4799 0

Hopefully debian will pick up the upstream release, since the current one in Maverick is *not* Maverick compatible.

Ignore the noise regarding vmxnet3: http://sanbarrow.com/vmxnet3.html It is working fine after changing the virtual h/w manually.

And similarly for vmxnet, you may need to manually change the hardware to test the vmxnet driver. The default is e1000, but after changing that, I can verify that the fixed vmxnet driver is working.

So we can get the latest upstream code, apply my patch for vmxnet (already submitted upstream), and that should allow this bug to be closed.

Note that it's not impossible for Ubuntu to pull the newer upstream release of open-vm-tools without Debian doing so first, it's just not how packages typically get maintained. If the upstream version solved the bug, I'd have no qualms pulling it in in advance of Debian.

I'd prefer to wait for the upstream project to comment on the bug before I go applying patches myself, as doing so makes me a bit uncomfortable. In particular, the struct netdev_hw_addr you use appears to have been introduced in 2.6.31 (and it looks like netdev_for_each_mc_addr was introduced later), so this patch would break compatibility with older kernels.

Since open-vm-tools appear to currently work all the way back to 2.6.9, I suspect the upstream project won't want to break compatibility that sharply. You might be able to short-circuit a code review round-trip with upstream by adjusting the patch to conditionalize based on what features are available on the current kernel (which surely open-vm-tools already does, I just don't know how)

If upstream turns out to be slow to respond, I'd consider applying the patch independently. That being said, packages do get backported, and people do run userspaces against older kernels, so I'd be less apprehensive about applying the patch without seeing it incorporated into upstream if it were updated to work on all previously supported kernel versions.

In any case, thanks for your work on this. I'll try to keep an eye on the upstream bug report as well as Debian

I agree, Evan. Anyone who is running into the problem now can grab my patches, which apply to either the current or newer upstream drops. And I also agree that I've done a quick, version specific, patch without doing the extra #if defined work that should be done in the "real" code.

f there's a real need, I'll put in the extra time, otherwise we'll just wait on upstream, which has some other issues to address as well for Maverick.

I have taken the liberty to rebuild open-vm-tools package available in Maverick, this time I included the patches available in this report. Launchpad's Build was successful and the building of the modules using module assistant was also successful. I've added (or copied, from one of my other PPAs, to be more accurate) packages to my regular PPA. https://launchpad.net/~n-muench/+archive/ppa

Also Debian released their build of the newest upstream open-vm-tools (available via GIT). I did 2 builds with their release, one without the patches and one with the patches. The first one failed. However I did a raw compile (without pbuilder) and I've noticed that the vsock patch doesn't work with the new release. The second build (with the vmxnet patch and using pbuilder), successfully built on my Maverick VM. The install also went successfully with one issue. The following error occurred during install:

Setting up open-vm-tools (2010.06.16-268169-1~ppa1) ...
Installing new version of config file /etc/vmware-tools/poweron-vm-default ...
Installing new version of config file /etc/vmware-tools/poweroff-vm-default ...
Installing new version of config file /etc/vmware-tools/suspend-vm-default ...
Installing new version of config file /etc/vmware-tools/resume-vm-default ...
update-initramfs: deferring update (trigger activated)
 * Loading open-vm-tools modules
FATAL: Error inserting vmhgfs (/lib/modules/2.6.35-6-generic/misc/vmhgfs.ko): Invalid argument
                                                                         [ OK ]
 * Starting open-vm daemon vmtoolsd [ OK ]

I've added an updated patch for vsock. Almost everything from the old vsock patch was added in the new release (the June 16th one). The only needed to add "compat_" to section that says "sk_sleep(sk)" The VMXNET patch is still needed.

So Noel, I needed your input on this new vsock patch.

I'm about to do a test build (using pbuilder) on my Maverick virtual machine.

My Build went successfully with some issues.

I manually installed the deb files from the build, but I got errors saying I had better modules installed (or something). I completely uninstalled the modules, rebooted, and manually installed everything again, it went fine.

So because of that error I'm going to create a 5th PPA for myself, and upload the new packaging to that. Doing that will be easier to manage with Terminal.

Any news on that latest PPA Nate?Or on an Open VM Tools upgrade? Using GRUB_GFXPAYLOAD_LINUX=text in my GRUB Configuration now, which sort of works. Started a thread here http://ubuntuforums.org/showthread.php?t=1531620 .

One thing I've noticed with the new upstream release, it's that even with the config coming from Debian, the existing vmxnet, and my adjusted vsock patch. It appears we lose the vmci/vmhgfs modules which means no shared folders. Which I want to make a note of.

After you install open-vm-tools on Ubuntu, compile the modules, and reboot. There is still a problem that need to be done manually, and that's sharing folders. With the regular VMware Tools (the ones that come with Workstation, Fusion, etc.), this is done automatically. But with open-vm-tools, to share folders, a command (mount) needs to be run (as sudo) every time you boot. I've made a fix, which enters the command to mount at boot up. I think one of the Ubuntu (or Debian) developer(s) should put this script (in some shape or form) into the config.

First, I'll tell you how the script functions. What it does is check to see if the "hgfs" folder exists in the "mnt" folder, if it does, create the directory, then run the mount command. If the directory doesn't exist, then just run the mount command. I've noticed that if you already have a sharing enable before installing open-vm-tools (and compiling the modules), you need to reboot twice (once to make the hgfs directory, and another time to mount the shared folder), this is where my script can be perfected.

Now, here's how to run it. Put the "MountSharedFolders" script on your Desktop, Home folder (it doesn't matter, just do it as a regular user). Next, make it executable by running "chmod +x MountSharedFolders" Now, with sudo, move the script to the "/etc/init.d" folder. And finally, in order to make it run during boot run "update-rc.d MountSharedFolders defaults"

Like I said, the new upstream version, for some strange reason, loses the ability to access shared folders. The script attached should be used on the upstream available on Maverick (the April release). I think it should be added to the Ubuntu/Debian config, since shared folders aren't mount by default.

Nate: Hardcoding this to /mnt/hgfs seems wrong. /mnt is not generally used by packages. However, I have zero knowledge of this package :)

This bug was fixed in the package open-vm-tools - 2010.04.25-253928-2+ubuntu2

---------------
open-vm-tools (2010.04.25-253928-2+ubuntu2) maverick; urgency=low

  * Add 03-vmxnet.patch and 04-vsock.patch.
    Fixes building on Maverick kernels (LP: #598542)
 -- Nate Muench <email address hidden> Mon, 19 Jul 2010 18:01:17 -0500