kernel crash on symlink chased from NFS to failing automount

This has been tested on a default lucid 64-bit install. We do not believe the bug existed in 32-bit dapper.

Thomas, can you provide (sanitized) dmesg output from the machine that crash?

BTW, I marked this as a security vulnerability not because of any need for
privacy, but because the consequence (in many environments) is that simply
creating a symlink enables a DOS attack.

On Thu, Dec 2, 2010 at 7:34 AM, Jamie Strandboge <email address hidden> wrote:

> ** This bug is no longer flagged as a security vulnerability
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/683938
>
> Title:
> kernel crash on symlink chased from NFS to failing automount
>
> Status in Ubuntu:
> New
>
> Bug description:
> Create an automount indirect map entry to a nfs server that will deny the
> mount with a permission denied error.
> Create a symlink on some mounted NFS partition pointing at the name of that
> automount indirect map entry.
> Chase the symlink with ls, etc.
> Notice that the automounter tries and fails to mount the partition.
> (visible with automount -d -f, say)
>
> In a few minutes, depending on system activity, the kernel will crash with
> the symptoms of a memory corruption error.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+bug/683938/+subscribe
>

I'm at a meeting as I type this, but as soon as I'm back to my desk I'll get
some examples. Note that the kernel traces are just of clear random memory
faults at various places.

Thomas

On Wed, Dec 1, 2010 at 7:40 PM, Etienne Goyer
<email address hidden>wrote:

> Thomas, can you provide (sanitized) dmesg output from the machine that
> crash?
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/683938
>
> Title:
> kernel crash on symlink chased from NFS to failing automount
>
> Status in Ubuntu:
> New
>
> Bug description:
> Create an automount indirect map entry to a nfs server that will deny the
> mount with a permission denied error.
> Create a symlink on some mounted NFS partition pointing at the name of that
> automount indirect map entry.
> Chase the symlink with ls, etc.
> Notice that the automounter tries and fails to mount the partition.
> (visible with automount -d -f, say)
>
> In a few minutes, depending on system activity, the kernel will crash with
> the symptoms of a memory corruption error.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+bug/683938/+subscribe
>

Version info is here.

This bug did not happen for us with 32-bit dapper, but does occur (as reported) for 64-bit lucid.

$ uname -a
Linux tbushnell-test 2.6.32-26-generic #48-Ubuntu SMP Wed Nov 24 10:14:11 UTC 2010 x86_64 GNU/Linux
$ dpkg -i linux*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-====================================-===============================================-============================================
un linux-doc-2.6.32 <none> (no description available)
ii linux-firmware 1.34.1 Firmware for Linux kernel drivers
ii linux-generic 2.6.32.26.28 Complete Generic Linux kernel
un linux-headers <none> (no description available)
un linux-headers-2.6 <none> (no description available)
ii linux-headers-2.6.32-21 2.6.32-21.32 Header files related to Linux kernel version
ii linux-headers-2.6.32-21-generic 2.6.32-21.32 Linux kernel headers for version 2.6.32 on x
ii linux-headers-2.6.32-26 2.6.32-26.48 Header files related to Linux kernel version
ii linux-headers-2.6.32-26-generic 2.6.32-26.48 Linux kernel headers for version 2.6.32 on x
ii linux-headers-generic 2.6.32.26.28 Generic Linux kernel headers
un linux-image <none> (no description available)
un linux-image-2.6 <none> (no description available)
ii linux-image-2.6.32-26-generic 2.6.32-26.48 Linux kernel image for version 2.6.32 on x86
ii linux-image-generic 2.6.32.26.28 Generic Linux kernel image
un linux-initramfs-tool <none> (no description available)
un linux-kernel-headers <none> (no description available)
un linux-kernel-log-daemon <none> (no description available)
ii linux-libc-dev 2.6.32-26.48 Linux Kernel Headers for development
un linux-restricted-common <none> (no description available)
ii linux-sound-base 1.0.22.1+dfsg-0ubuntu3 base package for ALSA and OSS sound systems
un linux-source-2.6.32 <none> ...

I am unable to provoke the bug with linux-image-2.6.32-0206322612-generic_2.6.32-0206322612.201012010905_amd64.deb as found in http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.32.26.12-lucid/. This suggests strongly the problem is in an Ubuntu patch, or a failure from Ubuntu to incorporate an upstream patch. (As I look at my own diff of our kernel source against upstream, I see a lot of cases where Ubuntu has failed to incorporate obvious upstream fixes, which is sad.)

I may have spoken too soon, I think it might happen with the mainline. Unfortunately, I have installed a generic 2.6.36 as well, as Etienne Goyer suggested, and now I can't boot the machine. When I try to get to grub with ESC, it never happens, and the 2.6.36 kernel tries to start and fails (ahci controller reset failed, and more), then with modprobe problems, and I end up in busybox.

So, since grub is broken, I need to reinstall and it will be a while before I can verify. However, to repeat, I have one data point which suggests that the problem may have occurred on mainline and I missed it. I will let you know once my reinstall is over. ::sigh::

Thomas, to get the GRUB prompt with GRUB2, you need to press and hold Shift after the BIOS POST. The window of opportunity for doing that is rather short (default 1 second, IIRC).

I would rather there never be magical keystrokes, especially when Ubuntu's own docs say ESC. Well, the machine is now almost finished being re-installed.

I was asked which version of NFS; sorry for not specifying that in the original report. All the servers involved are nfs v3.

I was also asked what the client side of the nfs looks like.

Make some directories:
# mkdir /dir /auto

Create the automount master:
# echo "/auto /etc/auto.auto" > /etc/auto.master

Create the automount map:
# echo "foo -tcp server-2:/foo" > /etc/auto.auto

Start the automounter in the foreground so you can watch it
# /etc/init.d/autofs stop
# automount -d -f

Have a server server-1 exporting dir. Then mount it:
# mount server-1:/dir /test

Create a symlink to the automount filesystem
# ln -s /auto/foo /test/foo

Make sure that server-1 exports the filesystem fine. Make sure that server-2 bounces the mount attempt with a permission denied error.

Then
# ls /test/foo/
# cd /test/foo
# echo /test/foo/*

and other such things. wait a bit.

Confirmed; I was able to reproduce the bug with the instructions given above. The client machine hard-freeze, and I get a stack trace on the console.

For the record, you do not need two NFS server to reproduce the bug. You can have one, exporting two different directory. My /etc/exports on the NFS server:

/srv/data1 *(rw,sec=krb5)
/srv/data2 *(rw,no_root_squash)

In the above, /srv/data1 is the directory the client try to automount. You do not actually need to configure/setup Kerberos; I specify sec=krb5 just so that the mount fail with permission denied. With /srv/data2, I use the no_root_squash option so that I can create the symlink from the client as root.

Even in a simpler setup it seem. I just could see the crash when the symlink was placed into the rootfs. So it seems the working nfs mount is not needed in the equation. Just autofs4 and the automounter failing on a symlink. Etienne/Thomas can you confirm this?

Etienne: I'm not surprised that one server is sufficient. Our setup happens to have two, and I didn't want to introduce an extra variable that might matter.

Stefan: we tried to make the bug happen with a local symlink, and it didn't, but we did not try very hard: we were mostly focused on simply getting ourselves confident that we could get a reproducible case. I'll work harder to confirm your observation today.

Ok, and I just got the same happening on the latest Natty kernel (it took somewhat longer, I nearly thought it would be working) which is as upstream as we can get. So I would open an upstream bugzilla which could get us some help from there.

Yep, confirmed. The symlink does not need to be in an NFS-mounted file system. Just trying to defer a symlink on an automounted file system, which mount is denied, seems to be sufficient.

It seems like it took longer to trigger the crash this time, but that is quite subjective. I am going to try again and see if it does indeed take longer to crash when the symlink is on a local file system.

After some further testing, it does not seems to crash appreciably faster whether the symlink is on an NFS-mounted file system or not. It's rather random.

Upstream bug created and linked to this bug report.

I can confirm that it doesn't matter for me either whether the symlink is on an NFS-mounted filesystem.

An interesting (possible) data point. The bug does not happen if the automount fails because the target directory doesn't exist, or because the server hostname is wrong (*). It seems that if you have an automount which fails for such a reason, and then you switch to the permission-denied mount (on the same name) you also don't get the bug.

(*): This is quite odd, btw. If the hostname doesn't exist, then the automounter doesn't even attempt the mount command, but if the directory is wrong, then it does attempt the mount command, and seemingly gets exactly the same error from the server, but it doesn't crash.

I should say, that this is "seems", not "sure", because of the slight indeterminacy of the bug itself.

So there is this freakish part of this bug which I had not considered before. Looking at the autofs4 code in the kernel, and the code in the automounter, there is no difference at all between the case where the directory doesn't exist on the server or the case where the mount security options are wrong, but only the latter produces the bug.

(Check the automount code. In spawn.c we find that spawn_mount is what calls the mount program, via do_spawn, and the return value is just the exit status of the program, which all ends up in the same packet sent to the kernel saying just that it failed, and the ioctl in the kernel turns that into ENOENT and the resulting paths are thus identical.)

Back in the day, mount.nfs always contacted the server itself to get a file handle for the root, and so errors like this were reflected really without entering the kernel at all. But that is not so. If you look at mount.nfs (from the nfs-utils source package), you find that mount.c does not behave the old way anymore. If the kernel version is > 2.6.22, then we get a new interface. Instead of using the nfsmount function, which did the old thing, we get the nfsmount_string function, which ends up passing everything to mount.

That means that the initial handshake with the server, which used to be in user space in the mount program, has (since 2.6.22) been moved into the kernel. I suspect the problem can be found there. Note that this is consistent with this bug appearing in lucid, but not existing in dapper, where we used a 2.6.18 kernel.

My comment which says "It seems that if you have an automount which fails for such a reason, and then you switch to the permission-denied mount (on the same name) you also don't get the bug." is incorrect. I have produced such a crash.

After much source poking, I thought, hey, if this really depends on whether the mount is denied b/c of security mismatch rather than just a missing directory, the code paths made me convinced that the symlink origin couldn't matter. And indeed, it doesn't. Merely repeatedly trying to cd into the automount generates a crash.

So, just to be entirely clear, you can reproduce the bug in the following two cases:

- The mount is denied;

- The target of the symlink does not exist.

Inspired by the last comments from Thomas, I went back and tried to just do what the automounter is doing
- create dir
- try to mount nfs
- remove dir

and, guess what, it crashes. So I looked at the wrong suspect. It is not autofs, it is the nfs mount itself.

It seems that even the create/remove part is not necessary. Repeating the mount requests eventually results in a crash. I played around with the RPC and NFS debugging options to see what is going on. Comparing the first and last one that did not crash seems to show no difference (beside of some unix credentials that are released at different times but that has happened in between without crashing).
Given that the crash was a hard lockup again, I am not sure the data of the last try is complete.

Comparing the case of trying to mount a directory which is not exported and the one crashing, there is a difference. In the first case the error is returned by the server. While in the case we test, the request is seen as success but then the authentication flavours do not match and the client does an explicit umount request (probably the same happens when the authentication methods are supported but the authentication fails).

Stefan says "While in the case we test, the request is seen as success but then the authentication flavours do not match and the client does an explicit umount request (probably the same happens when the authentication methods are supported but the authentication fails)."

I spent a lot of time staring at this yesterday myself. The umount doesn't do much: it doesn't involve any local state manipulation. I also chased the aliases for the filehandle and such on the mount, and I didn't see any leaks of the pointer to something that might manipulate it after it gets freed (which happens in all the failing mount cases).

However, if repeated mounts fail, it is odd that the original manifestation of this bug is that a *single* chase through the symlink is sufficient to produce an (eventual) crash. I wonder if a single failing mount is really sufficient to produce a crash. That would be odd, because you'd think that would happen a lot.

So here is my current theory: what I see in the code is that there only seems to be one caller to nfs_umount (in mount_clnt.c) and this is done from super.c when nfs_try_mount checks the returned authentication modes and finds that server and client support not a common method.

This reuses the same data structure that has been prepared for the mount call. Though it seems at this point everything is ok. And locally it seems that some of the failed mounts are not causing a crash while other cases do. And the calling sequence seems the same in all cases.

One thing I saw was that this umount is claimed to be optional and is done as a UDP call which does not expect any data back. Now I checked the actual packets sent around with wireshark and to me it seems that there actually is a reply of some sort. After the RPC request is reported done successfully, there is an ICMP packet coming in to the same port, stating the destination port is unreachable.

The umount call itself creates a RPC client, sends the message and immediately tears down the client. But now I am wondering what would happen if this ICMP packet arrives before the client is completely torn down.

As one quick test, I compiled a kernel with the nfs_umount call commented out (as it is claimed to be optional anyway and we never have a nfs connection set up). This kernel seems to survive the mount loop for quite a while now. But I want to leave it running for a bit longer to be more confident of this result.

If this turns out to be stable, I would update the upstream bug with that information. Currently I only got a 2.6.32-generic 64bit debug kernel, but if someone wants to try, I am currently uploading to http://people.canonical.com/~smb/lp683938.

What lies behind the comment on the umount call is this: the NFS protocol requests that clients tell servers when they unmount partitions, so that servers can stop reporting it in tools like showmount. The client doesn't care at all if the server ever gets it, and even if you do an unmount RPC, the server is not allowed to stop handling the filehandles, which are (officially) permanent.

So since the client doesn't care beans about the umount call, or even if it works or gets blackholed, the code in mount_clnt.c does minimal retransmits and short timeouts. When it says that it doesn't expect data back, that doesn't mean that the RPC has no return (the server does follow the usual sunrpc semantics and does send back a reply) but the client doesn't care what the return is in any way.

I've been busy this morning with other responsibilities, but I can report now that I agree that no automount involvement is necessary; merely repeating the mount request eventually provokes the failure.

Removing the umount attempt seems to make the problem go away. Not a fix, but it's nice to have a workaround.

So thanks to upstream, we got a patch (I am duplicating this here). It was in a completely different corner. Instead of a race condition, the incorrect number of commands provided seems to have a hidden bad effect within the rpc code (I have not looked but from the patch description, some memory allocations are done for each command). The code put 2 as the number of commands, but the array had four entries and command index 0 and 2 being not used. The UMNT command has the index number 3, so when accessing internal arrays with that index the code accesses memory outside of the allocated range.

I am currently compiling stock Lucid kernels with just this patch applied. I will upload them as soon as those are ready to the same location I put the other debug kernels.

Awesomeness! I'll give it a try today.

This is not quite right. The second one should use ARRAY_SIZE(mnt3_procedures). The value happens to be the same, but still...let's not make a new bug if someone implements another part of the protocol someday. :)

Yes, I was told by someone else reviewing the patch when I submitted it to be included as SRU. :) But better mention it twice than have it missed. As Chuck noted it is in his version now and it should be the version to go upstream. I must admit that I did not look well enough when I saw how simple the change looked and when it actually and finally resolved the crashes,

This bug was fixed in the package linux - 2.6.37-9.22

---------------
linux (2.6.37-9.22) natty; urgency=low

  [ Andy Whitcroft ]

  * rebase to v2.6.35-rc5
  * [Config] updateconfigs following rebase to v2.6.37-rc5
  * (no-up) add support for installed header files to ubuntu directory
    - LP: #684666
  * ubuntu: AUFS -- include the aufs_types.h file in linux-libc-headers
    - LP: #684666
  * ubuntu: dm-raid4-5 -- follow changes to bio flags
  * ubuntu: dm-raid4-5 -- re-enable
  * ubuntu: omnibook -- update BOM
  * ubuntu: ndiswrapper -- update BOM to match actual version
  * ubuntu: ndiswrapper -- follow removal of the BKL and locked ioctl
  * ubuntu: ndiswrapper -- re-enable
  * ubuntu: iscsitarget -- re-instate copy_io_context
  * ubuntu: iscsitarget -- follow changes to semaphore initialisation
  * ubuntu: iscsitarget -- convert NIPQUAD to %pI4
  * ubuntu: iscsitarget -- re-enable

  [ Kees Cook ]

  * [Config] update config for CONFIG_DEBUG_SET_MODULE_RONX

  [ Manoj Iyer ]

  * SAUCE: Enable jack sense for Thinkpad Edge 13
    - LP: #685015

  [ Tim Gardner ]

  * [Config] CONFIG_9P_FSCACHE=y,CONFIG_9P_FS_POSIX_ACL=y
  * [Config] CONFIG_CRYPTO_CRC32C=y
    - LP: #681819
  * [Config] CONFIG_9P_FSCACHE=n
  * [Config] Add nfsd modules to -virtual flavour
    - LP: #688070

  [ Upstream Kernel Changes ]

  * Revert "Staging: zram: work around oops due to startup ordering snafu"
  * NFS: Fix panic after nfs_umount()
    - LP: #683938
  * x86: Add NX protection for kernel data
  * x86: Add RO/NX protection for loadable kernel modules
  * x86: Resume trampoline must be executable
  * x86: RO/NX protection for loadable kernel, jump_table fix

  [ Upstream Kernel Changes ]

  * rebase to v2.6.37-rc5
 -- Andy Whitcroft <email address hidden> Thu, 09 Dec 2010 18:15:35 +0000

Accepted linux into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Accepted into maverick-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed' to 'verification-done'.

If verification is not done by one week from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Because we run a modified kernel, it would be inconvenient to need to
install a vanilla system in order to verify the fix there. Etienne has/had a
vanilla setup reproducing the bug. I can confirm that the patch successfully
solves the problem for our kernel.

Thomas

On Thu, Jan 13, 2011 at 8:34 AM, Steve Conklin <email address hidden>wrote:

> This bug is awaiting verification that the kernel in -proposed solves
> the problem. Please test the kernel and update this bug with the
> results. If the problem is solved, change the tag 'verification-needed'
> to 'verification-done'.
>
> If verification is not done by one week from today, this fix will be
> dropped from the source code, and this bug will be closed.
>
> See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how
> to enable and use -proposed. Thank you!
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/683938
>
> Title:
> kernel crash on symlink chased from NFS to failing automount
>
> Status in The Linux Kernel:
> Unknown
> Status in “linux” package in Ubuntu:
> Fix Released
> Status in “linux” source package in Lucid:
> Fix Committed
> Status in “linux” source package in Maverick:
> Fix Committed
> Status in “linux” source package in Natty:
> Fix Released
>
> Bug description:
> SRU justification:
>
> Impact: When trying to mount an export where server and client have no
> common authentication method, the client will abort the mount by
> sending an advisory unmount message to the server. A bug in the RPC
> client setup causes the sunrpc code to access memory outside an
> allocated array, which will sooner or later cause the kernel to crash.
>
> Fix: Patch from upstream (about to be submitted and targeted for
> stable too) changes the setup to use the actual array size instead of
> a manually entered number.
>
> Testcase:
>
> Server exports a mount with an authentication method the client does not
> support, eg.:
> [/etc/exports] /srv/foo *(rw,sec=krb5)
>
> Client tries to mount this directory with no special authentication
> method:
> while true; do mount <server>:/srv/foo /mnt; sync; sleep 1; done
>
> ---
>
> Create an automount indirect map entry to a nfs server that will deny the
> mount with a permission denied error.
> Create a symlink on some mounted NFS partition pointing at the name of
> that automount indirect map entry.
> Chase the symlink with ls, etc.
> Notice that the automounter tries and fails to mount the partition.
> (visible with automount -d -f, say)
>
> In a few minutes, depending on system activity, the kernel will crash
> with the symptoms of a memory corruption error.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/linux/+bug/683938/+subscribe
>

This bug was fixed in the package linux - 2.6.32-28.55

---------------
linux (2.6.32-28.55) lucid-proposed; urgency=low

  * Another version bump because of abi check failure
  * Tracking Bug
    - LP: #699885

linux (2.6.32-28.54) lucid-proposed; urgency=low

  * Another version bump because of upload failure

linux (2.6.32-28.53) lucid-proposed; urgency=low

  * Another version bump because of upload failure

linux (2.6.32-28.52) lucid-proposed; urgency=low

  [ Steve Conklin ]

  * (removed old tracking bug link)

linux (2.6.32-28.51) lucid-proposed; urgency=low

  [ Steve Conklin ]

  * bumped version due to build fail

linux (2.6.32-28.50) lucid-proposed; urgency=low

  [ Tim Gardner ]

  * SAUCE: Change nodelayacct boot parameter polarity.
    - LP: #493156
  * [Config] CONFIG_TASK_DELAY_ACCT=y
    - LP: #493156

  [ Upstream Kernel Changes ]

  * ipc: initialize structure memory to zero for compat functions
  * tcp: Increase TCP_MAXSEG socket option minimum.
    - CVE-2010-4165
  * perf_events: Fix perf_counter_mmap() hook in mprotect()
    - CVE-2010-4169
  * af_unix: limit unix_tot_inflight
    - CVE-2010-4249
  * AppArmor: fix the upper bound check for the next/check table
    - LP: #581525
  * NFS: Fix panic after nfs_umount()
    - LP: #683938
  * block: Ensure physical block size is unsigned int
    - LP: #688669
  * block: limit vec count in bio_kmalloc() and bio_alloc_map_data()
    - LP: #688669
  * block: take care not to overflow when calculating total iov length
    - LP: #688669
  * block: check for proper length of iov entries in blk_rq_map_user_iov()
    - LP: #688669
  * jme: Fix PHY power-off error
    - LP: #688669
  * irda: Fix parameter extraction stack overflow
    - LP: #688669
  * irda: Fix heap memory corruption in iriap.c
    - LP: #688669
  * i2c-pca-platform: Change device name of request_irq
    - LP: #688669
  * microblaze: Fix build with make 3.82
    - LP: #688669
  * Staging: asus_oled: fix up some sysfs attribute permissions
    - LP: #688669
  * Staging: asus_oled: fix up my fixup for some sysfs attribute
    permissions
    - LP: #688669
  * Staging: line6: fix up some sysfs attribute permissions
    - LP: #688669
  * hpet: fix unwanted interrupt due to stale irq status bit
    - LP: #688669
  * hpet: unmap unused I/O space
    - LP: #688669
  * olpc_battery: Fix endian neutral breakage for s16 values
    - LP: #688669
  * percpu: fix list_head init bug in __percpu_counter_init()
    - LP: #688669
  * um: remove PAGE_SIZE alignment in linker script causing kernel
    segfault.
    - LP: #688669
  * um: fix global timer issue when using CONFIG_NO_HZ
    - LP: #688669
  * numa: fix slab_node(MPOL_BIND)
    - LP: #688669
  * hwmon: (lm85) Fix ADT7468 frequency table
    - LP: #688669
  * mm: fix return value of scan_lru_pages in memory unplug
    - LP: #688669
  * mm: fix is_mem_section_removable() page_order BUG_ON check
    - LP: #688669
  * ssb: b43-pci-bridge: Add new vendor for BCM4318
    - LP: #688669
  * sgi-xpc: XPC fails to discover partitions with all nasids above 128
    - LP: #688669
  * xen: ensure that all event channels start off bound to VCPU 0
    - LP: #688669
  * xen: don't bother to...

This bug was fixed in the package linux - 2.6.35-25.44

---------------
linux (2.6.35-25.44) maverick-proposed; urgency=low

  [ Upstream Kernel Changes ]

  * Revert "drm/radeon/kms: properly compute group_size on 6xx/7xx"
    - LP: #703553

linux (2.6.35-25.43) maverick-proposed; urgency=low

  [ Brad Figg ]

  - LP: #697948

  [ Andy Whitcroft ]

  * [Config] add vmware-balloon driver to -virtual flavour
    - LP: #592039

  [ Manoj Iyer ]

  * SAUCE: Enable jack sense for Thinkpad Edge 13
    - LP: #685015

  [ Robert Hooker ]

  * Revert "(pre-stable): input: Support Clickpad devices in ClickZone
    mode"
    - LP: #669399

  [ Stefan Bader ]

  * Set virtual flavour maximum of domain visible memory to 70G
    - LP: #667796

  [ Takashi Iwai ]

  * SAUCE: input: Support Clickpad devices in ClickZone mode
    - LP: #516329

  [ Tim Gardner ]

  * [Config] Add nfsd modules to -virtual flavour
    - LP: #688070
  * [Config] Added autofs4.ko to -virtual flavour
    - LP: #692917

  [ Upstream Kernel Changes ]

  * intel_idle: delete substates DEBUG modparam
    - LP: #684888
  * intel_idle: delete power_policy modparam, and choose substate functions
    - LP: #684888
  * intel_idle: add support for Westmere-EX
    - LP: #684888
  * intel_idle: recognize Lincroft Atom Processor
    - LP: #684888
  * x86, mwait: Move mwait constants to a common header file
    - LP: #684888
  * intel_idle: Change mode 755 => 644
    - LP: #684888
  * intel_idle: add missing __percpu markup
    - LP: #684888
  * cpuidle: extend cpuidle and menu governor to handle dynamic states
    - LP: #684888
  * intel_idle: Voluntary leave_mm before entering deeper
    - LP: #684888
  * intel_idle: enable Atom C6
    - LP: #684888
  * intel_idle: simplify test for leave_mm()
    - LP: #684888
  * intel_idle: delete bogus data from cpuidle_state.power_usage
    - LP: #684888
  * intel_idle: add initial Sandy Bridge support
    - LP: #684888
  * intel_idle: do not use the LAPIC timer for ATOM C2
    - LP: #684888
  * staging: usbip: Notify usb core of port status changes
    - LP: #686158
  * staging: usbip: Process event flags without delay
    - LP: #686158
  * Staging: phison: fix problem caused by libata change
    - LP: #686158
  * perf_events: Fix bogus AMD64 generic TLB events
    - LP: #686158
  * perf_events: Fix bogus context time tracking
    - LP: #686158
  * powerpc/perf: Fix sampling enable for PPC970
    - LP: #686158
  * pcmcia: synclink_cs: fix information leak to userland
    - LP: #686158
  * sched: Drop all load weight manipulation for RT tasks
    - LP: #686158
  * sched: Fix string comparison in /proc/sched_features
    - LP: #686158
  * bluetooth: Fix missing NULL check
    - LP: #686158
  * futex: Fix errors in nested key ref-counting
    - LP: #686158
  * cifs: fix broken oplock handling
    - LP: #686158
  * libahci: fix result_tf handling after an ATA PIO data-in command
    - LP: #686158
  * mm, x86: Saving vmcore with non-lazy freeing of vmas
    - LP: #686158
  * x86, cpu: Fix renamed, not-yet-shipping AMD CPUID feature bit
    - LP: #686158
  * x86, kexec: Make sure to stop all CPUs before exiting the kernel
    - LP: #686158
  * x86, olpc: Don...