FFE: open-vm-tools kernel module failed to build

(Marking confirmed due to several dupes.)

Thanks for taking the time to report this bug and helping to make Ubuntu better. It looks like the kernel module source needs an update. We will pursue that and keep you uptodate.

Thanks again.

All right, modules/linux/shared/vmciGuestKernelAPI.h appears to be moved to modules/linux/shared/vmciKernelAPI.h, but noone told:

./vmci/vmciProcess.c:#include "vmciGuestKernelAPI.h"
./vmci/vmciEvent.c:# include "vmciGuestKernelAPI.h"
./vmci/vmciDatagram.h:#include "vmciGuestKernelAPI.h"
./vmci/vmciUtil.h:#include "vmciGuestKernelAPI.h"
./vmci/vmciEvent.h:#include "vmciGuestKernelAPI.h"
./vmci/vmciQueuePairInt.h:#include "vmciGuestKernelAPI.h"
./shared/vmciGuestKernelAPI.h: * vmciGuestKernelAPI.h --
./vsock/linux/af_vsock.h:# include "vmciGuestKernelAPI.h"

Disregard comment #3, I was grepping through debian source.

Looks like the kernel module source is just a touch out of date. Doing a
 'sudo dkms --verbose build -m open-vm-tools -v 2010.06.16'
shows first that it defines file_operations->ioctl which is now replaced
by unlocked_ioctl. Fixing that shows more problems.

Note that the debian unstable source has the same problem, so a sync won't fix this. Upstream however should work.

Proposed source for package based on the upstream git tree at tag 2011.02.23

The packages in this tarfile appear to correctly build the kernel modules. Please give them a try by doing:

   tar zxf open-vm-tools-2011.02.23-debs.tar.gz
   sudo dpkg -i open-vm-tools_2011.02.23-368700-0ubuntu1_amd64.deb
   sudo dpkg -i open-vm-toolbox_2011.02.23-368700-0ubuntu1_amd64.deb
   sudo dpkg -i open-vm-dkms_2011.02.23-368700-0ubuntu1_all.deb

This is working for me now. Thanks for the update!

The VM lives on a windows system. Next time I can stand to boot the wretched thing, I will try the tar.

I am having a hard time trying to compile this for i586.
Is there a simple recipe to to that, otherwise could you provide the binary packages for i586?

I am also not an amd64. Options ?

Here are the 32-bit debs for testing.

So far so good.

Drivers are now enabled, I can work with VMware Toolbox without a problem.

Package installs, but may not work. I cannot set shared folders to host disk (widows system). I have not used vmware before, the vmware player may be crippled, so this feature does not work.

diff of debian/, based on r21 of Serge's branch... lp:~serge-hallyn/ubuntu/natty/open-vm-tools/merge-2011.02.23-368700

This is the result of 'git log --pretty=oneline' for the upstream version delta.

This is the result of 'fakeroot debian/rules binary > ../buildlog 2>&1'

Based on the git log, I'm spotting some API changes in this version, and rather extensive changes.
What level of testing has been done on it so far? What is the regression risk like?

Any update on this?

Quoting Kate Stewart (<email address hidden>):
> Any update on this?

The kernel modules in open-vm-dkms are to be installed in the
vmware guests, not the hosts. So our previous concern about
testing the kernel modules should be far far lessesened.

Nevertheless, I ran ltp on monday with and with the patches -
applied on a host, not guests - and ltp did, by the time it
got to rename014, cause a hang in xfs. I think a mutex must
have been left locked, bc basically any subsequent task closing
an fd would hang.

When not running ltp, the host would run fine with the modules
loaded.

Since these modules are to be installed in the guest, not
the host, and since ltp is a very stressful test, I would
maintain we are better off with this package, which at least
allows the open-vm-tools to install.

I did try to install vmware-player in order to test in a guest,
but that (which requires a download from vmware.com, not our
package) wouldn't work. However, earlier comments in this
bug claimed success with this package.

Quoting Kate Stewart (<email address hidden>):
> Any update on this?

Can any one else who has tested this package comment here in detail
about how they tested and whether it failed?

Serge,
My environment is VMware Workstation v7.1.3 on Windows 7, 64-bit, Host
Guest is Ubuntu 11.04 beta: Linux ubuntu 2.6.38-7-generic #39-Ubuntu SMP Fri Mar 25 21:24:57 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

VMware Tools form VMware are currently installed. Version: 7.1.3 build-324285

I follow your guidance from 3/3 in order:
   tar zxf open-vm-tools-2011.02.23-debs.tar.gz
   sudo dpkg -i open-vm-tools_2011.02.23-368700-0ubuntu1_amd64.deb
   sudo dpkg -i open-vm-toolbox_2011.02.23-368700-0ubuntu1_amd64.deb
   sudo dpkg -i open-vm-dkms_2011.02.23-368700-0ubuntu1_all.deb

On first pass, I got errors in open-vm-tools set up saying vmhgfs and vmsync were missing:
Preparing to replace open-vm-tools 2010.06.16-268169-3ubuntu1 (using open-vm-tools_2011.02.23-368700-0ubuntu1_amd64.deb) ...
 * Stopping open-vm guest daemon vmtoolsd [ OK ]
 * Removing open-vm-tools modules FATAL: Module vmhgfs not found.
FATAL: Module vmsync not found.
                                                                         [ OK ]
Unpacking replacement open-vm-tools ...
Setting up open-vm-tools (2011.02.23-368700-0ubuntu1) ...
Installing new version of config file /etc/vmware-tools/vm-support ...
update-initramfs: deferring update (trigger activated)
 * Loading open-vm-tools modules FATAL: Module vmhgfs not found.
FATAL: Module vmsync not found.
                                                                         [ OK ]
 * Starting open-vm daemon vmtoolsd [ OK ]
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-2.6.38-7-generic
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Processing triggers for man-db ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place

Kind of strange the the install continued and updated initrd.img.

Anyway, I kept going to here through to dkms:
Preparing to replace open-vm-dkms 2010.06.16-268169-3ubuntu1 (using open-vm-dkms_2011.02.23-368700-0ubuntu1_all.deb) ...

------------------------------
Deleting module version: 2010.06.16
completely from the DKMS tree.
------------------------------
Done.
Unpacking replacement open-vm-dkms ...
Setting up open-vm-dkms (2011.02.23-368700-0ubuntu1) ...
Loading new open-vm-tools-2011.02.23 DKMS files...
Building only for 2.6.38-7-generic
Building for architecture x86_64
Building initial module for 2.6.38-7-generic
Done.

vmblock.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/2.6.38-7-generic/updates/dkms/

vmci.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/2.6.38-7-generic/updates/dkms/

vmhgfs.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/2.6.38-7-generic/updates/dkms/

vmsync.ko:
Running module ver...

following reboot:
/var/log/dmesg.0 has:
vmblock: disagrees about version of symbol module_layout

Clipboard no longer works.

Quoting Tom Weiss (<email address hidden>):
> following reboot:
> /var/log/dmesg.0 has:
> vmblock: disagrees about version of symbol module_layout
>
> Clipboard no longer works.

Thanks, Tom. I've got a VMware key now so will test it myself and
try to get something working.

Ok, a few observations now that I'm able to test on vmware:

I've created a new package based on the 3.28 release (from two days ago). What I did:

   1. run vmware-uninstall-tools
       (@Tom Weiss - you need to do this first, that's why you got the errors about vmhgfs and vmsync not being found. I get that even when I install the current open-vm-tools package)
   2. dpkg -i open-vm-source_* open-vm-toolbox* open-vm-tools_*
   3. dpkg -i open-vm-dkms*
   4. reboot

At this point, clipboard did not work for me. I had to manually

   5. vmware-user-suid-wrapper

after which clipboard worked. This however did complain because vmblock was not loaded. I see bug 332323, so figure I'll fix this in vmware-user-suid-wrapper, but for now I just did:

   5a. modprobe vmblock
   5b vmware-user-suid-wrapper

vmware-user-suid-wrapper is not actually installed setuid-root, I'll fix that. WHen I fix that on the guest, then drag and drop start working!

I suppose the packaging should somehow run vmware-user-suid-wrapper on login. How to do that may be question for desktop folks.

The vmhgfs does not work for me. I don't know if that's depracated.

New proposed package is at http://people.canonical.com/~serge/open-vm-tools_2001.03.28-package.tgz.

I can put it in launchpad if preferred. With this version, after running vmware-user-suid-wrapper I can drag and drop to the host, and clipboard works.

The 64-bit packages are at http://people.canonical.com/~serge/open-vm-tools_2011.03.28-64bit-debs.tgz, in case anyone wants to test. (I"ll try to generate 32-bit ones next)

The 32-bit packages are in http://people.canonical.com/~serge/open-vm-tools_2011.03.28-32bit-debs.tgz.

Thanks for the additional testing Tom, and updates Serge.

FFE Approved.

This bug was fixed in the package open-vm-tools - 2011.03.28-387002-0ubuntu1

---------------
open-vm-tools (2011.03.28-387002-0ubuntu1) natty; urgency=low

  * Merge latest upstream git tag. This has the unlocked_ioctl change
    needed to fix dkms build failures (LP: #727342)
  * Changes in debian/rules:
    - work around a bug in toolbox/Makefile, where install-exec-hook is
      not happening. This needs to get fixed the right way.
    - don't install 'vmware-user' which seems to no longer exist
    - move /etc/xdg into open-vm-toolbox (which should be done using .install)
  * debian/open-vm-tools.init: add 'modprobe [-r] vmblock'. (LP: #332323)
  * debian/rules and debian/open-vm-toolbox.lintian-overrides:
    - Make vmware-user-suid-wrapper suid-root (LP: #332323)
 -- Serge Hallyn <email address hidden> Thu, 31 Mar 2011 14:20:05 -0500

Uninstall VMware Workstation 7.1.3's VMware Tools package.
open-vm-tools came down with latest 11.04 updates!
Needed to reboot.
Current Ubuntu vm-tools configuration:
+-+-+-+-+-+-+-+-
Package: open-vm-tools
Versions:
2011.03.28-387002-0ubuntu1 (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_natty_multiverse_binary-amd64_Packages) (/var/lib/dpkg/status)

Reverse Depends:
  open-vm-tools-dbg,open-vm-tools 2011.03.28-387002-0ubuntu1
  open-vm-toolbox,open-vm-tools 2011.03.28-387002-0ubuntu1
  open-vm-source,open-vm-tools
  open-vm-dkms,open-vm-tools
+-+-+-+-+-+-+-+-

Cut-n-paste working.

Now running VMware Workstation 7.1.4 build-385536
Host OS: Windows 7, 64-bit 6.1.7601, Service Pack 1

root@ubuntu:/var/log# lsmod | grep vm
vmblock 18671 0
vmsync 13024 0
vmhgfs 63369 0
vmw_balloon 12841 0
vmci 82515 1 vmhgfs
vmxnet 26761 0

This also seems to happen on ubuntu 10.10. or am i mistaken ? Package version is the same, and no updates available yet..
Can't even remove it...
-------------------------------------------[snip]-------------------------------
Removing old open-vm-tools-2010.06.16 DKMS files...

------------------------------
Deleting module version: 2010.06.16
completely from the DKMS tree.
------------------------------
Done.
Loading new open-vm-tools-2010.06.16 DKMS files...
Building for 2.6.32-32-generic-pae and 2.6.35-28-generic
Building for architecture i686
Building initial module for 2.6.32-32-generic-pae

Error! Build of vmxnet.ko failed for: 2.6.32-32-generic-pae (i686)
Consult the make.log in the build directory
/var/lib/dkms/open-vm-tools/2010.06.16/build/ for more information.
dpkg: error processing open-vm-dkms (--configure):
 subprocess installed post-installation script returned error exit status 7
Setting up libcurl3 (7.21.0-1ubuntu1.1) ...
No apport report written because MaxReports is reached already
                                                              Setting up curl (7.21.0-1ubuntu1.1) ...
Setting up libcurl3-gnutls (7.21.0-1ubuntu1.1) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
Errors were encountered while processing:
 open-vm-dkms
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@nas2:~#