2.6.24 reports invalid storage size in /sys [Sony Walkman NWZ-S618F doesn't mount in Hardy]

I experienced the same behaviour under Hardy with Sony Walkman NWZ-A818, which is expected to be mounted as a USB mass storage device. Worked in Gutsy, now broken in Hardy.

For some reason, GParted sees the device as being 1.82 GB, and you can mount it in read-only mode.

Note that the dmesg output looks similar under Gutsy and Hardy, but the volume get mounted only under Gutsy.

Output of fdisk -l:

$ fdisk -l /dev/sdb
Note: sector size is 2048 (not 512)

Disk /dev/sdb: 7843 MB, 7843348480 bytes
1 heads, 62 sectors/track, 61770 cylinders
Units = cylinders of 62 * 2048 = 126976 bytes
Disk identifier: 0x00000000

   Device Boot Start End Blocks Id System
/dev/sdb1 1 69273667 4294967294 ff BBT
Partition 1 does not end on cylinder boundary.

I have a Sony Walkman A818 with the same symptoms -- in Gutsy it auto-mounts as expected, but since upgrading to Hardy, it fails to do so. I had the same type of output in my dmesg, again under both Gutsy and Hardy.

I can add one extra bit of info: manually mounting and unmounting the device using "pmount" and "pumount" (or mount/umount as root) works as expected.

I have the same problem with my Sony Walkman NWZ-818. Mounts in all other OS's except Hardy. dsmsg | tail gives me
[ 1414.282359] attempt to access beyond end of device
[ 1414.282361] sdb: rw=0, want=4294967232, limit=15319040
[ 1414.282367] attempt to access beyond end of device
[ 1414.282370] sdb: rw=0, want=4294967284, limit=15319040
[ 1414.282372] attempt to access beyond end of device
[ 1414.282374] sdb: rw=0, want=4294967288, limit=15319040
[ 1414.282380] attempt to access beyond end of device
[ 1414.282382] sdb: rw=0, want=4294967292, limit=15319040
[ 1414.282387] attempt to access beyond end of device
[ 1414.282389] sdb: rw=0, want=4294967292, limit=15319040

I think we should mark this bug as affecting hal and/or gnome-volume-manager.

This affects nautilus, as it is now in share of mounting volumes.

This affects nautilus, as it is now in charge of mounting volumes, afaik.

not a nautilus bug no, nautilus use what gvfs is listing which relies on the driver, etc to detect the device correctly

Here is what syslog appends to /var/log/messages, with hald --verbose=yes --use-syslog.

Maybe a hint :

http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.24.4

commit 90fbe8b1fa081a99616ccd7760f5877fcafa35ef
Author: Alan Stern <email address hidden>
Date: Fri Feb 22 17:03:25 2008 -0500

    usb-storage: don't access beyond the end of the sg buffer

    This patch (as1038) fixes a bug in usb_stor_access_xfer_buf() and
    usb_stor_set_xfer_buf() (the bug was originally found by Boaz
    Harrosh): The routine must not attempt to write beyond the end of a
    scatter-gather list or beyond the number of bytes requested.

    This is the minimal 2.6.24 equivalent to as1035 +
    as1037 (7084191d53b224b953c8e1db525ea6c31aca5fc7 "USB:
    usb-storage: don't access beyond the end of the sg buffer" +
    6d512a80c26d87f8599057c86dc920fbfe0aa3aa "usb-storage: update earlier
    scatter-gather bug fix"). Mark Glines has confirmed that it fixes
    his problem.

    Signed-off-by: Alan Stern <email address hidden>
    Cc: Mark Glines <email address hidden>
    Cc: Boaz Harrosh <email address hidden>
    Signed-off-by: Chris Wright <email address hidden>
    Signed-off-by: Greg Kroah-Hartman <email address hidden>

I just gave a try with gutsy kernel (2.6.22) : A818 doesn't automount. So it's not related to the brand new hardy kernel.

Thank you for taking the time to report this bug and helping to make Ubuntu better. This bug did not have a package associated with it, which is important for ensuring that it gets looked at by the proper developers. You can learn more about finding the right package at https://wiki.ubuntu.com/Bugs/FindRightPackage . I have classified this bug as a bug in linux the kernel package for Hardy Heron.

I can also confirm this regression. Sony A818 walkman worked perfectly in Gutsy, but is not auto-mounting in Hardy. This is a shame, because this is a drag-and-drop player, so is perfect for Linux.

Can you please try the following:

 * Plug in the device
 * Find out the device node, as it appears in "dmesg" output (such as /dev/sdb1)
 * Do "lshal > /tmp/hal.txt" in a terminal, and attach it to this bug report.
 * Copy&paste the output of "gnome-mount -vbd /dev/sdb1" (or whatever your device node is) here.

This shuold tell us whether it's a bug/overzealous check in hal or gnome-mount. However, those partitions seem to be broken either way.

Device node : dmesg output

[ 3790.861961] scsi 6:0:0:0: Direct-Access SONY WALKMAN 1.00 PQ: 0 ANSI: 0 CCS
[ 3791.879644] ready
[ 3791.880267] sd 6:0:0:0: [sdb] 3829760 2048-byte hardware sectors (7843 MB)
[ 3791.881385] sd 6:0:0:0: [sdb] Write Protect is off
[ 3791.881392] sd 6:0:0:0: [sdb] Mode Sense: 00 2a 00 00
[ 3791.881396] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[ 3791.883511] sd 6:0:0:0: [sdb] 3829760 2048-byte hardware sectors (7843 MB)
[ 3791.884504] sd 6:0:0:0: [sdb] Write Protect is off
[ 3791.884510] sd 6:0:0:0: [sdb] Mode Sense: 00 2a 00 00
[ 3791.884514] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[ 3791.884520] sdb: sdb1
[ 3791.889895] sdb: p1 exceeds device capacity
[ 3791.896922] sd 6:0:0:0: [sdb] Attached SCSI removable disk
[ 3791.896994] sd 6:0:0:0: Attached scsi generic sg2 type 0
[ 3792.030963] attempt to access beyond end of device
[ 3792.030973] sdb: rw=0, want=4294967044, limit=15319040

OK, it's sdb1

Output of lshal is attached.
$ LANGUAGE=en_US gnome-mount -vbd /dev/sdb1
gnome-mount 0.8
** Message: Given device '/dev/sdb1' is not a volume or a drive.

So, to sum up, in Gutsy, the player was recognized in HAL and listed as containing a volume, which was automounted. In Hardy, the player is still recognized by HAL but no volume is detected, and of course it can't get automounted.

However, it is still possible to mount /dev/sdb1 either with mount or pmount.

Sony does probably not implement mass-storage pretty accurately, giving us the messages shown in dmesg.

BUT, what we are looking for is what changed between Gutsy and Hardy that prevents volume detection in HAL.

Hi,

for me it's sdb1
Disque /dev/sdb: 7843 Mo, 7843348480 octets
1 heads, 62 sectors/track, 61770 cylinders
Units = cylindres of 62 * 2048 = 126976 bytes
Identifiant disque: 0x00000000

Périphérique Amorce Début Fin Blocs Id Système
/dev/sdb1 1 69273667 4294967294 ff BBT
La partition 1 ne se termine pas sur une frontière de cylindre.
Note: taille de secteur est 2048 (et non pas 512)

Disque /dev/sdb1: 2199.0 Go, 2199023253504 octets
1 heads, 62 sectors/track, 17318416 cylinders
Units = cylindres of 62 * 2048 = 126976 bytes
Identifiant disque: 0x00000000

Périphérique Amorce Début Fin Blocs Id Système
/dev/sdb1p1 1 69273667 4294967294 ff BBT

gnome-mount -vbd /dev/sdb1 give this output :
laurent@laurent-laptop:~$ gnome-mount -vbd /dev/sdb1
gnome-mount 0.8
** Message: Le périphérique « /dev/sdb1 » indiqué n'est ni un volume ni un lecteur.
laurent@laurent-laptop:~$

And I attach hal.txt

OK, thanks. if /dev/sdb1 does not even exist, there is nothing we can do to get it automounted.

My dmesg output was much the same as the above, though my device appeared as sdc1. gnome-mount produces the same result:

lsd@cletus:~$ gnome-mount -vbd /dev/sdc1
gnome-mount 0.8
** Message: Given device '/dev/sdc1' is not a volume or a drive.

The device definitely does exist though:

lsd@cletus:~$ ls -la /dev/sdc1
brw-rw---- 1 root disk 8, 33 2008-04-16 06:28 /dev/sdc1

Mounting it manually using "mount" or "pmount" works exactly as expected.

Seeing this same behaviour using Kubuntu Hardy with all updates to 20th April 2008. Just connected a Sony NWZ -A828 and get the "attempt to access beyond end of device" and "p1 exceeds device capacity" messages.

It might be worth noting that the A828, A818 and S618 support MTP as well as mass-storage modes. Is there a way to force the kernel to pick them up as MTP devices rather than mass storage?

My A818 is already supported as a MTP device in Hardy. Just try to plug it in and launch Rhythmbox.

However, I would feel very angry to see that Ubuntu used to support it as a mass-storage device, but would prefer to use a proprietary drm-friendly protocol with the following drawbacks :
- it would not be possible to feed the device with a simple drag & drop
- it would be mandatory to use a special software (e.g. supporting MTP, like Rhythmbox) to load music on the player
- it would be impossible to load videos nor photos on the device, as Rhythmbox does not support video or photographic content

The only question to ask is : what has changed in HAL fdi rules (or in whatever underlying layer located between the kernel and HAL) between Gutsy and Hardy that just prevent HAL from exposing the device's FAT32 volume to other programs.

gnome-mount - doesn't see the volume because it is not exposed by HAL
HAL - detects the device, probe for a volume, find it, but does not create volume information in his tree (see following debug log)
Kernel - see the volume and is able to manually mount it using mount or pmount

Apr 8 14:11:39 thinkpad hald-addon-storage: [13675]: 14:11:39.102 [D] addon-storage.c:748: **************************************************
Apr 8 14:11:39 thinkpad hald-addon-storage: [13675]: 14:11:39.102 [D] addon-storage.c:749: Doing addon-storage for /dev/sdb (bus usb) (drive_type disk) (udi /org/freedesktop/Hal/devices/storage_serial_Sony_WALKMAN_492477603979)
Apr 8 14:11:39 thinkpad hald-addon-storage: [13675]: 14:11:39.102 [D] addon-storage.c:750: **************************************************
Apr 8 14:11:41 thinkpad orage: polling /dev/sdb (every 2 sec): [13675]: 14:11:41.003 [I] addon-storage.c:355: Checking whether device /dev/sdb is locked on HAL
Apr 8 14:11:41 thinkpad orage: polling /dev/scd0 (every 2 sec): [5329]: 14:11:41.003 [I] addon-storage.c:355: Checking whether device /dev/scd0 is locked on HAL
Apr 8 14:11:41 thinkpad orage: polling /dev/scd0 (every 2 sec): [5329]: 14:11:41.004 [I] addon-storage.c:363: ... device /dev/scd0 is not locked on HAL
Apr 8 14:11:41 thinkpad orage: polling /dev/sdb (every 2 sec): [13675]: 14:11:41.004 [I] addon-storage.c:363: ... device /dev/sdb is not locked on HAL
Apr 8 14:11:41 thinkpad orage: polling /dev/sdb (every 2 sec): [13675]: 14:11:41.006 [D] addon-storage.c:542: Media insertion detected on /dev/sdb
Apr 8 14:11:41 thinkpad hald-probe-storage: [13682]: 14:11:41.010 [D] probe-storage.c:155: Doing probe-storage for /dev/sdb (bus usb) (drive_type disk) (udi=/org/freedesktop/Hal/devices/storage_serial_Sony_WALKMAN_492477603979) (--only-check-for-fs==1)
Apr 8 14:11:41 thinkpad hald-probe-storage: [13682]: 14:11:41.010 [D] probe-storage.c:407: Checking for file system on /dev/sdb
Apr 8 14:11:41 thinkpad hald-probe-storage: [13682]: 14:11:41.010 [D] probe-storage.c:413: Doing open ("/dev/sdb", O_RDONLY)
Apr 8 14:11:41 thinkpad hald-probe-storage: [13682]: 14:11:41.012 [D] probe-storage.c:421: Returned from open(2)
Apr 8 14:11:41 thinkpad hald-probe-storage: [13682]: 14:11:41.012 [D] probe-storage.c:437: look for existing partitions for sdb
Apr 8 14:11:41 thi...

Some fresh news from this bug :

It seems as if the system was not telling HAL the size of sdb1.

18:05:19.024 [I] blockdev.c:874: block_add: sysfs_path=/sys/block/sdb/sdb1 dev=/dev/sdb1 is_part=1, parent=0x08100160
18:05:19.024 [I] blockdev.c:1448: Ignoring hotplug event - cannot read 'size'
18:05:19.024 [W] blockdev.c:1481: Not adding device object

Complete log :

18:05:18.966 [I] osspec.c:241: SEQNUM=2994, ACTION=add, SUBSYSTEM=block, DEVPATH=/sys/block/sdb, DEVNAME=/dev/sdb, IFINDEX=0
18:05:18.967 [I] osspec.c:966: hal_util_find_known_parent: '/sys/block/sdb'->'/sys/devices/pci0000:00/0000:00:1d.7/usb5/5-5/5-5:1.0/host9/target9:0:0/9:0:0:0'
18:05:18.967 [I] blockdev.c:874: block_add: sysfs_path=/sys/block/sdb dev=/dev/sdb is_part=0, parent=0x080ffc40
18:05:18.967 [I] blockdev.c:1254: parent_bus is scsi
18:05:18.967 [I] blockdev.c:502: Probing storage device /dev/sdb
woohoo
[6741]: 18:05:18.970 [D] probe-storage.c:155: Doing probe-storage for /dev/sdb (bus usb) (drive_type disk) (udi=/org/freedesktop/Hal/devices/temp/136) (--only-check-for-fs==0)
[6741]: 18:05:18.970 [D] probe-storage.c:407: Checking for file system on /dev/sdb
[6741]: 18:05:18.970 [D] probe-storage.c:413: Doing open ("/dev/sdb", O_RDONLY)
[6741]: 18:05:18.974 [D] probe-storage.c:421: Returned from open(2)
[6741]: 18:05:18.974 [D] probe-storage.c:437: look for existing partitions for sdb
[6741]: 18:05:18.974 [D] probe-storage.c:447: partition sdb1 found, skip probing for filesystem
[6741]: 18:05:18.978 [I] partutil.c:875: MSDOS partition table detected
18:05:18.980 [I] hald_dbus.c:1366: storage.removable.media_available -> True
18:05:18.980 [I] hald_dbus.c:1350: storage.removable.media_size -> 7843348480
18:05:18.980 [I] hald_dbus.c:1334: storage.partitioning_scheme -> mbr
18:05:18.981 [I] osspec.c:241: SEQNUM=3000, ACTION=add, SUBSYSTEM=kernel, DEVPATH=/sys/kernel/uids/65534, DEVNAME=, IFINDEX=0
18:05:18.981 [I] blockdev.c:387: entering; exit_type=0, return_code=0
18:05:18.987 [I] blockdev.c:141: Add callouts completed udi=/org/freedesktop/Hal/devices/storage_serial_Sony_WALKMAN_492477603979
18:05:18.987 [D] device_store.c:516: adding 0x8100160 to (linux.sysfs_path,/sys/block/sdb)
18:05:18.987 [I] hald.c:108: Added device to GDL; udi=/org/freedesktop/Hal/devices/storage_serial_Sony_WALKMAN_492477603979
18:05:19.024 [I] hald_runner.c:659: running_processes 0x80e6508, num = 4
18:05:19.024 [I] hald.c:120: Started addon hald-addon-storage for udi /org/freedesktop/Hal/devices/storage_serial_Sony_WALKMAN_492477603979
18:05:19.024 [I] device.c:4505: add_dev: subsys=kernel sysfs_path=/sys/kernel/uids/65534 dev= parent_dev=0x00000000
18:05:19.024 [I] osspec.c:241: SEQNUM=2995, ACTION=add, SUBSYSTEM=block, DEVPATH=/sys/block/sdb/sdb1, DEVNAME=/dev/sdb1, IFINDEX=0
18:05:19.024 [I] osspec.c:966: hal_util_find_known_parent: '/sys/block/sdb/sdb1'->'/sys/block/sdb'
18:05:19.024 [I] blockdev.c:874: block_add: sysfs_path=/sys/block/sdb/sdb1 dev=/dev/sdb1 is_part=1, parent=0x08100160
18:05:19.024 [I] blockdev.c:1448: Ignoring hotplug event - cannot read 'size'
18:05:19.024 [W] blockdev.c:1481: Not adding device object
18:05:19.025 [D] hald_dbus.c:3294: udi=/org/freedesktop/Hal/devices/sto...

I get roughly the same output in daemon.log as Benjamin:

Apr 27 16:56:46 hostname hald[4243]: 16:56:46.111 [I] blockdev.c:874: block_add: sysfs_path=/sys/block/sdc/sdc1 dev=/dev/sdc1 is_part=1, parent=0x080cf560
Apr 27 16:56:46 hostname hald[4243]: 16:56:46.112 [I] blockdev.c:1448: Ignoring hotplug event - cannot read 'size'
Apr 27 16:56:46 hostname hald[4243]: 16:56:46.112 [W] blockdev.c:1481: Not adding device object

Not sure why it's failing to read the size though:

root@hostname:/etc/init.d# cat /sys/block/sdc/sdc1/size
4294967292
root@hostname:/etc/init.d#

I also tried repartitioning the 828 using cfdisk, but the kernel just doesn't seem to like the apparent geometry of the device. I somehow managed to get it recognised with a single FAT32 partition, though it showed up as 30GB! The walkman itself wouldn't boot with this, so in the end I restored the original disk image.

I'm having the same problem with a Panasonic Photo Camera (Lumix FX-12), anyone found a solution ?

Confirming this with Sony-Ericsson P1i smartphone. Manually mounting it works as expected, although the read rate seems to be slower than it was on Gutsy (750KB/s?)... could be unrelated.

I am having the same problem with my Sony NWZ-A816 since upgrading to Hardy, worked fine on Gusty. Hopefully a fix is on the way but until then here is a way to at least let you put some new music on ur player

mount -t vfat -o iocharset=utf8,umask=0000,user /dev/sdd1 /media/sony

Obviously replace the last part with your device name/Mounting folder

Hope this helps

Route490

Same problem, is not a very big deal since I can manage pics and video anyway mounting it with gparted or manually as suggested above

Instead for music I use to tag album art with easytag setting id3 to version 2.3 then easily access mp3 via mtp with rhythmbox and with gnomad2 i can quickly create playlists.
I hope one day there will be a standalone software that could do everything at the same time (maybe amarok, but I hate running KDE apps on gnome)

I already solved the problem with my camera. I had some lines in /etc/fstab that generates the problem when ubuntu try to automaticaly mount my device. Many thanks to all, and check your fstab, maybe...

I finally found a possible reason of the bug we are going through.

First of all, the size of the volume reported by the kernel through sysfs hasn't changed since Gutsy, nor do the permissions :
gutsy:~$ ls -l /sys/block/sdb/size /sys/block/sdb/sdb1/size
-r--r--r-- 1 root root 4096 2008-05-06 18:05 /sys/block/sdb/sdb1/size
-r--r--r-- 1 root root 4096 2008-05-06 18:05 /sys/block/sdb/size
gutsy:~$ cat /sys/block/sdb/size /sys/block/sdb/sdb1/size
15319040
4294967292

hardy:~$ ls -l /sys/block/sdb/size /sys/block/sdb/sdb1/size
-r--r--r-- 1 root root 4096 2008-05-06 18:21 /sys/block/sdb/sdb1/size
-r--r--r-- 1 root root 4096 2008-05-06 18:22 /sys/block/sdb/size
hardy:~$ cat /sys/block/sdb/size /sys/block/sdb/sdb1/size
15319040
4294967292

I then investigated for possible change in hal code between Gutsy (0.5.9.1) and Hardy (0.5.11-rc2). Here's what I found :
- the 'Ignoring hotplug event - cannot read 'size'' message is sent from a function in hald/linux/blockdev.c called hotplug_event_begin_add_blockdev(), because the call to hal_util_set_int_from_file (d, "volume.num_blocks", sysfs_path_real, "size", 0) returned FALSE
- looking at this hal_util_set_int_from_file() function in hald/util.c, we see that it makes use of the function hal_util_get_int_from_file() in the same source file.

In this function, there is a major change between 0.5.9.1 and 0.5.11-rc2 : the error code returned by strtol is handled.

0.5.9.1:
/* TODO: handle error condition */
*result = strtol (buf, NULL, base);
ret = TRUE;

0.5.11-rc2:
errno = 0;
_result = strtol (buf, NULL, base);
if (errno == 0) {
  ret = TRUE;
  *result = _result;
}

strtol() is used to convert a string (the size read from sysfs) to a long int. If the value read from the string exceeds 2147483647 (*which is true in our case*), strtol() returns LONG_MAX and sets errno to ERANGE.

To sum up:

in Gutsy, the size exposed by sysfs already exceeded the limit, but, as there was no control on errno in hal, the volume size was set to 2147483647. This can be confirmed by running hal-device under Gutsy :
gutsy:~$ hal-device
0: udi = '/org/freedesktop/Hal/devices/volume_uuid_47EA_53F8'
...
  volume.num_blocks = 2147483647 (0x7fffffff) (int)
...

Now, in Hardy, the errno is correctly handled, but as a side effect it prevents hal from detecting the volume on the device.

A possible fix for this would be to remove the errno checking or to use an unsigned long int and strtoul() to read the size, as it would perfectly fit within the limit. Moreover, it rarely happens to have a negative size, so why use a signed long int?

To finish with, I would insist on the fact that if the regression (to the user's point of view) is related to a change in hal's source code, hal developers have done their job by checking against errno.

The real question would be: why does the kernel report such a funny size through sysfs? Is it related to the support of the mass-storage protocol by the device, or to the implementation of USB mass-storage in the kernel?

The problem is not HAL and is way to handle errno, but because sysfs give this absurd size to the volume.

I add a hal task as well, since there might actually be storage devices which have more than 2 billion blocks (in a few years, perhaps). hal shuold use strtoull() instead of strtol().

I recompiled hal-0.5.11rc2 after replacing strtol() with strtoul() in hald/util.c, and it does the trick.

Nice work Benjamin. What options did you use for the configure? (More specifically I guess I'm asking how to determine which options were used in the official Ubuntu package).

Btw, rather than modifying the call to strtol() in util.c, would it not be better to switch the call in blockdev.c to:

                if (!hal_util_set_uint64_from_file (d, "volume.num_blocks", sysfs_path_real, "size", 0)) {

And then in util.c change hal_util_set_uint64_from_file to use strtoll() instead of strtoul():

        _result = strtoul (buf, NULL, base);

Whilst this is still just masking the underlying problem with the kernel not picking up the correct number of blocks for the device, it would seem a more "compatible" change.

Sorry, that should have read "to use strtoul() instead of strtoll()".

TheFluffyOne,

Here is what I did to recompile a HAL package with the modification :

# get the source package
apt-get source hal
# install build dependencies
sudo apt-get build-dep hal
cd hal-0.5.11~rc2/
# use strtoul() instead of strtol() in util.c
sed -i '201s/strtol/strtoul/' hald/util.c
# recompile HAL and build a brand new package
dpkg-buildpackage -us -uc -rfakeroot
cd ..
# install the package
sudo dpkg -i hal_0.5.11~rc2-1ubuntu8.1_i386.deb

To get the A818 recognized by Rhythmbox as a UMS digital audio player, you'll also have to put a FDI in /etc/hal/fdi/information/. You can find a sample one at http://www.delagoutte.net/wordpress/wp-content/uploads/20-sony-a818.fdi

Corresponding process for french readers :
http://www.delagoutte.net/122-les-walkmans-sony-sous-ubuntu-hardy/

Fantastic, thanks Benjamin. I used two sed commands to implement the change as I described above:

sed -i '255s/strtoll/strtoul/' hald/util.c
sed -i '1439s/_set_int_/_set_uint64_/' hald/linux/blockdev.c

FDI file for A828 attached.

It was necessary to manually add the Walkman as a UMS device in Amarok.

I should note that for some reason apt-get source gave me hal_0.5.11~rc2-1ubuntu7 instead of hal_0.5.11~rc2-1ubuntu8.

this do the trik, but rest the problem of the wrong filesize given by syfs.
dmesg output this:

[40235.800344] usb 1-2: new high speed USB device using ehci_hcd and address 6
[40235.936718] usb 1-2: configuration #1 chosen from 1 choice
[40235.937371] scsi5 : SCSI emulation for USB Mass Storage devices
[40235.937946] usb-storage: device found at 6
[40235.937951] usb-storage: waiting for device to settle before scanning
[40240.926459] usb-storage: device scan complete
[40240.927453] scsi 5:0:0:0: Direct-Access SONY WALKMAN 1.00 PQ: 0 ANSI: 0 CCS
[40241.936684] ready
[40241.937310] sd 5:0:0:0: [sdc] 899072 2048-byte hardware sectors (1841 MB)
[40242.038491] sd 5:0:0:0: [sdc] Write Protect is off
[40242.038497] sd 5:0:0:0: [sdc] Mode Sense: 00 2a 00 00
[40242.038503] sd 5:0:0:0: [sdc] Assuming drive cache: write through
[40242.040601] sd 5:0:0:0: [sdc] 899072 2048-byte hardware sectors (1841 MB)
[40242.041606] sd 5:0:0:0: [sdc] Write Protect is off
[40242.041612] sd 5:0:0:0: [sdc] Mode Sense: 00 2a 00 00
[40242.041617] sd 5:0:0:0: [sdc] Assuming drive cache: write through
[40242.041624] sdc: sdc1
[40242.044356] sdc: p1 exceeds device capacity
[40242.044904] sd 5:0:0:0: [sdc] Attached SCSI removable disk
[40242.044975] sd 5:0:0:0: Attached scsi generic sg4 type 0
[40242.585345] attempt to access beyond end of device
[40242.585354] sdc: rw=0, want=4294967044, limit=3596288
[40242.585359] Buffer I/O error on device sdc1, logical block 1073741760
[40242.585364] attempt to access beyond end of device
[40242.585368] sdc: rw=0, want=4294967048, limit=3596288
[40242.585372] Buffer I/O error on device sdc1, logical block 1073741761
[40242.585378] attempt to access beyond end of device
[40242.585382] sdc: rw=0, want=4294967044, limit=3596288
[40242.585385] Buffer I/O error on device sdc1, logical block 1073741760
[40242.585389] attempt to access beyond end of device
[40242.585393] sdc: rw=0, want=4294967048, limit=3596288
[40242.585396] Buffer I/O error on device sdc1, logical block 1073741761
[40242.585415] attempt to access beyond end of device
[40242.585421] sdc: rw=0, want=4294967276, limit=3596288
[40242.585425] Buffer I/O error on device sdc1, logical block 1073741818
[40242.585429] attempt to access beyond end of device
[40242.585433] sdc: rw=0, want=4294967280, limit=3596288
[40242.585437] Buffer I/O error on device sdc1, logical block 1073741819
[40242.585443] attempt to access beyond end of device
[40242.585447] sdc: rw=0, want=4294967276, limit=3596288
[40242.585450] Buffer I/O error on device sdc1, logical block 1073741818
[40242.585455] attempt to access beyond end of device
[40242.585458] sdc: rw=0, want=4294967280, limit=3596288
[40242.585461] Buffer I/O error on device sdc1, logical block 1073741819
[40242.585548] attempt to access beyond end of device
[40242.585552] sdc: rw=0, want=4294967292, limit=3596288
[40242.585556] Buffer I/O error on device sdc1, logical block 1073741822
[40242.585562] attempt to access beyond end of device
[40242.585565] sdc: rw=0, want=4294967292, limit=3596288
[40242.585568] Buffer I/O error on device sdc1, logical block 1073741822
[40242.585582] attempt to access beyond end of device
[40242....

any news?
my sony nwz doesnt mount either.
same kernel log

I'm about to try this trick as well... Same problem here, but with a Polaroid Camera using 2.6.24-19-generic

Fwiw, an OpenSUSE 11 RC1 machine running kernel 2.6.25 (specifically 2.6.25.4-8-pae) reports the same (incorrect) size in /sys.

Same problem here, can't mount Sony NWZ-A816 on Hardy!

Used to work like a charm in previous ubuntu versions......

Just wanted to chime in that my NWZ-A816 also stopped working going from 7.10 to 8.04. The blue "Connecting" graphic freezes. I have no knowledge to add, but hopefully the more people that report that more emphasis the bug gets.

Just figured out that in Hardy 64 bit on a fresh install this MP3 player does mount properly. I can't use 64 bit at the moment for various reasons, but then I did reinstall (fresh) Hardy 32 and decided to test without updates like I did i 64 and the player failed to mount. Just wanted to add that difference.

"...but hopefully the more people that report that more emphasis the bug gets."

Same Problem with the NWZ- A816 in Hardy, it works perfect with Gutsy.

Thank you very much Benjamin.

You have made my day. Your solution worked like a charm.

I have the same problem with Sony NWZ-615 and hardy. (good work with gutsy).
I confirm this bug.
Thank you Benjamin for your job !

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/209483/comments/42

That solution worked perfectly - thanks Benjamin!

This bug was fixed in the package hal - 0.5.11-3~ubuntu2

---------------
hal (0.5.11-3~ubuntu2) intrepid; urgency=low

  * Drop 96_uinput_device_support.patch: It is a bad security design (user
    processes should not have direct access to the hardware, such a thing
    belongs into an unix_chkpwd-like wrapper), was intended as a hack for
    Hardy, and not accepted upstream. [UbuntuSpec:intrepid-device-permissions]
  * 02_long_storage_size.patch: Use uint64 instead of int for
    volume.num_blocks, as it can get larger than 2G. (LP: #209483)

 -- Martin Pitt <email address hidden> Mon, 21 Jul 2008 15:49:17 +0200

that's fixed for intrepid, but is it planned for hardy too?

I'm happy to apply this patch to Hardy as well. However, I'd welcome some feedback whether the intrepid patch actually works, since I used a slightly different one (while I see why "sed -i '255s/strtoll/strtoul/' hald/util.c" worked, it is the logically wrong solution.)

Martin, I'm interested to know more about your thoughts on the change from strtoul from strtoll. The function name indicates that it sets a uint64, but would strtoll not set a signed 64-bit int? That was the logic behind changing it to use strtoul, so I'm keen to know if I've completely misunderstood something!

Still having trouble with to and from: should read "... the change *to* strtoul from strtoll." :)

Hi Gordon,

Gordon Mckeown [2008-07-22 8:33 -0000]:
> Martin, I'm interested to know more about your thoughts on the change
> from strtoul from strtoll. The function name indicates that it sets a
> uint64, but would strtoll not set a signed 64-bit int? That was the
> logic behind changing it to use strtoul, so I'm keen to know if I've
> completely misunderstood something!

No, that is of course correct, but is independent from the fix for
this bug (and, TBH, I think it's pretty irrelevant, too, given the
size of 64 bit integers it hardly matters...)

The original workaround proposed here was to use a 64 bit strtoll in
the 32 bit hal_util_set_int_from_file() function, which is wrong. My
patch actually uses hal_util_set_uint64_from_file() for
volume.num_blocks.

Ah OK, I was confused because you referred to the "sed -i '255s/strtoll/strtoul/'" in my post (which as you say is not really necessary) when I think you were talking about the "sed -i '201s/strtoll/strtoul/'" which modified the 32-bit version of the call. I now understand, thanks!

Okay, I've patched hal according to Benjamin's suggestion, and still no love.

Also, I'm not able to manually mount the device (it worked under Gutsy and on my other Gentoo machine, but not under Hardy). There is no device /dev/sdd1, despite whatever the dmesg reports. Dmesg follows.

---
[16776.995015] usb-storage: device found at 9
[16776.995019] usb-storage: waiting for device to settle before scanning
[16777.208386] usb 1-2.4: new full speed USB device using ohci_hcd and address 10
[16777.322272] usb 1-2.4: configuration #1 chosen from 1 choice
[16777.331181] usblp0: USB Bidirectional printer dev 10 if 0 alt 0 proto 2 vid 0x04E8 pid 0x326C
[16781.978544] usb-storage: device scan complete
[16781.983815] scsi 10:0:0:0: Direct-Access MATSHITA DMC-FZ50 0100 PQ: 0 ANSI: 2
[16782.007208] sd 10:0:0:0: [sdd] 3910655 512-byte hardware sectors (2002 MB)
[16782.013462] sd 10:0:0:0: [sdd] Write Protect is off
[16782.013468] sd 10:0:0:0: [sdd] Mode Sense: 04 00 00 00
[16782.013471] sd 10:0:0:0: [sdd] Assuming drive cache: write through
[16782.026961] sd 10:0:0:0: [sdd] 3910655 512-byte hardware sectors (2002 MB)
[16782.033248] sd 10:0:0:0: [sdd] Write Protect is off
[16782.033254] sd 10:0:0:0: [sdd] Mode Sense: 04 00 00 00
[16782.033256] sd 10:0:0:0: [sdd] Assuming drive cache: write through
[16782.033263] sdd: sdd1
[16782.045835] sdd: p1 exceeds device capacity
[16782.045898] sd 10:0:0:0: [sdd] Attached SCSI removable disk
[16782.045940] sd 10:0:0:0: Attached scsi generic sg5 type 0
[16782.594223] attempt to access beyond end of device
[16782.594229] sdd: rw=0, want=3910656, limit=3910655
[16782.594232] printk: 6 messages suppressed.
[16782.594234] Buffer I/O error on device sdd1, logical block 3910526
[16782.646233] attempt to access beyond end of device
[16782.646237] sdd: rw=0, want=3910656, limit=3910655
[16782.646239] Buffer I/O error on device sdd1, logical block 3910526
[16782.646254] attempt to access beyond end of device
[16782.646256] sdd: rw=0, want=3910656, limit=3910655
[16782.646258] Buffer I/O error on device sdd1, logical block 3910526
[16782.646264] attempt to access beyond end of device
[16782.646266] sdd: rw=0, want=3910656, limit=3910655
[16782.646268] Buffer I/O error on device sdd1, logical block 3910526
[16782.646272] attempt to access beyond end of device
[16782.646274] sdd: rw=0, want=3910656, limit=3910655
[16782.646276] Buffer I/O error on device sdd1, logical block 3910526
[16782.646281] attempt to access beyond end of device
[16782.646283] sdd: rw=0, want=3910656, limit=3910655
[16782.646285] Buffer I/O error on device sdd1, logical block 3910526
[16782.646290] attempt to access beyond end of device
[16782.646292] sdd: rw=0, want=3910656, limit=3910655
[16782.646294] Buffer I/O error on device sdd1, logical block 3910526
[16782.701970] attempt to access beyond end of device
[16782.701976] sdd: rw=0, want=3910656, limit=3910655
[16782.701978] Buffer I/O error on device sdd1, logical block 3910526
[16782.701985] attempt to access beyond end of device
[16782.701987] sdd: rw=0, want=3910656, limit=3910655
[16782.701989] Buffer I/O error on device sdd1, logical block 3910526
[16783.635125] attempt to access ...

So, there's a workaround for hal that worked for some but not for others. What about the kernel side of the issue, is there a workaround/patch worth trying?

I need to know what to suggest to a friend that's facing this problem, but I don't have her media player to test it myself. If one was to revert to gutsy's kernel, and remain in hardy's version of hal, would the problem go away?

The Ubuntu Kernel Team is planning to move to the 2.6.27 kernel for the upcoming Intrepid Ibex 8.10 release. As a result, the kernel team would appreciate it if you could please test this newer 2.6.27 Ubuntu kernel. There are one of two ways you should be able to test:

1) If you are comfortable installing packages on your own, the linux-image-2.6.27-* package is currently available for you to install and test.

--or--

2) The upcoming Alpha5 for Intrepid Ibex 8.10 will contain this newer 2.6.27 Ubuntu kernel. Alpha5 is set to be released Thursday Sept 4. Please watch http://www.ubuntu.com/testing for Alpha5 to be announced. You should then be able to test via a LiveCD.

Please let us know immediately if this newer 2.6.27 kernel resolves the bug reported here or if the issue remains. More importantly, please open a new bug report for each new bug/regression introduced by the 2.6.27 kernel and tag the bug report with 'linux-2.6.27'. Also, please specifically note if the issue does or does not appear in the 2.6.26 kernel. Thanks again, we really appreicate your help and feedback.

New kernel, 2.6.27.1 from intrepid (i just downloaded the deb file) works fine and solve my problem.
Now I have to find out any bug, but almost all my peripherals works fine.
Thanks.

Have just installed the generic 2.6.27 kernel from http://kernel.ubuntu.com/pub/next/2.6.27-rc3/hardy/ and a quick test suggests that things have improved.

The difference seems to be that the Walkman is treated as an single unpartitioned block now rather than as a device with a partition on it.

I now can see size information in /sys/block/sdc/size where before it was in /sys/block/sdc/sdc1/size, and the size appears to be correct. Running fdisk -l still shows an apparent /dev/sdc1, and this has the invalid size information as before. My assumption, therefore, is that the kernel is now ignoring what I guess isn't a real partition.

Just tried with the kernel 2.6.27 from Intrepid 8.10 alpha5 and my Sony NWZ-A818 automount again. So for me it fixes the bug.

My Sony Walkman NWZ-S618F works with the new kernel 2.6.27 in Intrepid alpha5. Thanks to everyone who helped out with this bug!

On Intrepid (2.6.27-3-generic, x86_64) I cannot mount the Memory Stick of my SE// K800i via USB. With Hardy i386 it works.

dmesg says:

[ 804.525061] scsi3 : SCSI emulation for USB Mass Storage devices
[ 804.525196] usbcore: registered new interface driver usb-storage
[ 804.525202] USB Mass Storage support registered.
[ 804.525426] usb-storage: device found at 3
[ 804.525429] usb-storage: waiting for device to settle before scanning
[ 809.525746] usb-storage: device scan complete
[ 809.525750] usb-storage: device scan complete
[ 809.528606] scsi 2:0:0:0: Direct-Access Sony Eri Memory Stick 0000 PQ: 0 ANSI: 0
[ 809.528613] scsi 3:0:0:0: Direct-Access Sony Eri Memory Stick 0000 PQ: 0 ANSI: 0
[ 809.559163] sd 3:0:0:0: [sdb] 958999 512-byte hardware sectors (491 MB)
[ 809.566798] sd 3:0:0:0: [sdb] Test WP failed, assume Write Enabled
[ 809.566809] sd 3:0:0:0: [sdb] Assuming drive cache: write through
[ 809.577244] sd 3:0:0:0: [sdb] 958999 512-byte hardware sectors (491 MB)
[ 809.585148] sd 3:0:0:0: [sdb] Test WP failed, assume Write Enabled
[ 809.585158] sd 3:0:0:0: [sdb] Assuming drive cache: write through
[ 809.585663] sdb: sdb1
[ 809.595581] sdb: p1 exceeds device capacity
[ 809.597642] sd 3:0:0:0: [sdb] Attached SCSI removable disk
[ 809.598089] sd 3:0:0:0: Attached scsi generic sg2 type 0
[ 809.603656] sd 2:0:0:0: [sdc] 168000 512-byte hardware sectors (86 MB)
[ 809.620569] sd 2:0:0:0: [sdc] Test WP failed, assume Write Enabled
[ 809.620579] sd 2:0:0:0: [sdc] Assuming drive cache: write through
[ 809.628548] sd 2:0:0:0: [sdc] 168000 512-byte hardware sectors (86 MB)
[ 809.635608] sd 2:0:0:0: [sdc] Test WP failed, assume Write Enabled
[ 809.635617] sd 2:0:0:0: [sdc] Assuming drive cache: write through
[ 809.635623] sdc: sdc1
[ 809.647978] sd 2:0:0:0: [sdc] Attached SCSI removable disk
[ 809.648915] sd 2:0:0:0: Attached scsi generic sg3 type 0

/dev/sdb1 (which should represent the fat partition of the memory stick) doesn't exist

additional note: my hal version is 0.5.11-3~ubuntu8

tested with different kernel: 2.6.26-1-rt

dmesg says the following

[ 116.464315] sd 2:0:0:0: [sdb] Attached SCSI removable disk
[ 116.464370] sd 2:0:0:0: Attached scsi generic sg2 type 0
[ 116.469265] sd 3:0:0:0: [sdc] 958999 512-byte hardware sectors (491 MB)
[ 116.476269] sd 3:0:0:0: [sdc] Test WP failed, assume Write Enabled
[ 116.476279] sd 3:0:0:0: [sdc] Assuming drive cache: write through
[ 116.485256] sd 3:0:0:0: [sdc] 958999 512-byte hardware sectors (491 MB)
[ 116.492254] sd 3:0:0:0: [sdc] Test WP failed, assume Write Enabled
[ 116.492259] sd 3:0:0:0: [sdc] Assuming drive cache: write through
[ 116.492262] sdc: sdc1
[ 116.501294] sdc: p1 exceeds device capacity
[ 116.503321] sd 3:0:0:0: [sdc] Attached SCSI removable disk
[ 116.503380] sd 3:0:0:0: Attached scsi generic sg3 type 0
[ 117.337938] attempt to access beyond end of device
[ 117.337950] sdc: rw=0, want=959338, limit=958999
[ 117.337956] Buffer I/O error on device sdc1, logical block 958848
[ 117.337962] attempt to access beyond end of device
[ 117.337966] sdc: rw=0, want=959339, limit=958999
[ 117.337969] Buffer I/O error on device sdc1, logical block 958849
[ 117.337974] attempt to access beyond end of device
[ 117.337977] sdc: rw=0, want=959340, limit=958999
[ 117.337981] Buffer I/O error on device sdc1, logical block 958850
...

but it works well

I think since your problem is a regression against .27, you should
file it as a seperate bug against linux-2.6.27. This bug was fixed by
the new kernel, so it's not exactly the same.
Cheers!

hi, first of all sorry for my sloppy English because that's not my native language.

At first I had the problem I couldn't mount my mp3 615f on hardy heron so I moved to intrepid ibex.
It worked well for 2 weeks then stupidly I downloaded the last updated and now it doesn't work:
 Dbus error org.gtk. private.remotevolumemonitor.notfound
I read about the fact that i cannot go back in time before the mistake, so, I'm hesitating for desinstall and re-install everything.

thanks for your help,

Salut,

J'avais le même problème avec Hardy heron mon mp3 nwz s 615f ne marchait pas donc j'ai installé intrepid ibex. Pendant une semaine tout à bien fonctionné mais depuis ce matin l'icône du walkman apparaît mais c'est comme s'il n'y avait rien dessus et je suis même plus autorisé à démonter le volume.message Dbus error org.gtk. private.remotevolumemonitor.notfound
Je me demande si cela ne vient pas de la dérniére mise à jour. A ce que j'ai lu sur le forum il semble que l'on ne peut pas revenir en arriére comme j'aimerai le faire.

Je serais ravi de connaître votre avis par avance merci.

up

Per a decision made by the Ubuntu Kernel Team, bugs will longer be assigned to the ubuntu-kernel-team in Launchpad as part of the bug triage process. The ubuntu-kernel-team is being unassigned from this bug report. Refer to https://wiki.ubuntu.com/KernelTeamBugPolicies for more information. Thanks.

For further information I have had the same problem mounting a sony walkman NWZA818, and also the same problem mounting it in SUse Linux 11.0 (kernel 2.6.25.18-0.2-pae) but it works fine in 11.1 (kernel 2.6.27.7-9-pae)

My Ubuntu machine is now running 8.10 (kernel 2.6.27-8-generic) and the sony mounts fine.

I am fascinated by the idea that the kernel team doesn't need to know about kernel bugs, BTW.

OK, I have a Sony A616 (8Gb) and a Sony A739F (16Gb). Both exhibit this problem.

The problem and the fix are both described in the comments above. However, there's been a bit of too-ing and fro-ing so I thought I would just summarize and write down the EXACT sequence of commands required to fix this bug under Hardy Heron 8.10. Particularly, I'd clarify what needs fixing and what doesn't.

# uname -a
Linux parker 2.6.24-22-generic #1 SMP Mon Nov 24 18:32:42 UTC 2008 i686 GNU/Linux

First up, the clarifications. There is only one single line of code which needs to be fixed. In blockdev.c the call to get the number of blocks must use the 64 bit function hal_util_set_uint64_from_file and not the 32 bit version. This is because the device is (presumably) a character device, so 1 block = 1 byte. The "gint" type (aka G_TYPE_INT) is a (signed) integer which is -2G to + 2G. So any Sony 4G or over will blow the limit.

To be more clear. There is nothing wrong at all with the util.c function. It correctly uses strtoul as it should, since it is returning an unsigned integer. Repeat. The strtoul/strtoll changes suggested are completely bogus. Don't change the strtoul/strtoll code in util.c. It is working properly. It is only blockdev.c which is at fault.

So, assuming that you have logged in as root (add sudo to every line if not). The following should work, assuming that you have the source repositories enabled. To do this, start up Adept Manager, choose "Manage Repositories" and check that "Sources" is turned on. Then you can build the module as follows.

# apt-get build-dep hal
# apt-get install fakeroot
# apt-get source hal

# cd hal-0.5.11~rc2
# cp hald/util.c hald/util.c.orig
# cp hald/linux/blockdev.c hald/linux/blockdev.c.orig

# sed -i '1439s/_set_int_/_set_uint64_/' hald/linux/blockdev.c
# diff hald/linux/blockdev.c hald/linux/blockdev.c.orig

# dpkg-buildpackage -us -uc -rfakeroot
# cd ..
# dpkg -i hal_0.5.11~rc2-1ubuntu8.1_i386.deb

Note that the "sed" command targets the specific line number 1439 in the file blockdev.c. If you are using a different version of the hal source (i.e. not 5.11 rc 2) then it is very possible that the line you need to change is not number 1439.

In case you need to find the new line number, the old code is:

                if (!hal_util_set_int_from_file (d, "volume.num_blocks", sysfs_path_real, "size", 0)) {
                        HAL_INFO (("Ignoring hotplug event - cannot read 'size'"));
                        goto error;
                }

The new code is:

                if (!hal_util_set_uint64_from_file (d, "volume.num_blocks", sysfs_path_real, "size", 0)) {
                        HAL_INFO (("Ignoring hotplug event - cannot read 'size'"));
                        goto error;
                }

The final "dpkg -i" will install the new module and will restart the HAL daemon. You do not need to reboot your machine. Simply plug in the device and you should immediately see:

[619137.132740] usb 5-8: new high speed USB device using ehci_hcd and address 5
[619137.265725] usb 5-8: configuration #1 chosen from 1 choice
[619137.269593] scsi5 : SCSI emulation for USB Mass Storage devices
[61...

This got stalled long enough, I'll prepare an SRU now.

Package uploaded, waiting for SRU processing by Steve or Colin.

Accepted hal into hardy-proposed; please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Oh no. Again in Jaunty the Sony NWZ-A818 ist mounted as "Sony Corp. Walkman".
But in the filebrowser there is no Music in it :(

This is a different bug: 330383.

Anyone who could test the current hal in hardy-proposed, that it doesn't cause regressions?

This bug was fixed in the package hal - 0.5.11~rc2-1ubuntu8.3

---------------
hal (0.5.11~rc2-1ubuntu8.3) hardy-proposed; urgency=low

  * Add 00upstream-long_storage_size.patch: Fix mounting of devices with a
    very large number of reported blocks by making it a 64 bit property.
    Cherrypicked from upstream. (LP: #209483)

 -- Martin Pitt <email address hidden> Thu, 16 Apr 2009 13:43:19 +0200

Just like to say this fix never made it onto my Dell mini-10v running 8.04 & updating from the official repositories. I'v recently dumped that & installed 10.04 UNR - and it all works fine, wit cameras & walkman & Sansa clip.

Thanks.
uname -r
2.6.32-25-generic