ecryptfs corrupts files over 4GB size on i686

Bug #1243636 reported by Lars Düsing
36
This bug affects 5 people
Affects Status Importance Assigned to Milestone
eCryptfs
Fix Released
Critical
Tyler Hicks
linux (Ubuntu)
Fix Released
Critical
Tyler Hicks

Bug Description

[SRU Justification]

Commit 24d15266bd86b7961f309a962fa3aa177a78c49f introduced a data corruption
regression on 32 bit architectures when writing past the 4 GB.

[Impact]

32 bit users experience corruption of large files.

[Fix]

A cast is needed when shifting the page's index. Colin and I independently
identified the problem. It is a simple fix that has been merged upstream:

http://git.kernel.org/linus/43b7c6c6a4e3916edd186ceb61be0c67d1e0969e

[Test Case]

Inside of an eCryptfs mount on an i686 Ubuntu install, create a file containing
4 GB + 1 page worth (4096 bytes) of zeros. Then inspect the file for non-zero
bytes.

$ rm zeros
$ dd if=/dev/zero of=zeros bs=4096 count=$((4*1024*1024*1024/4096+4096))
1052672+0 records in
1052672+0 records out
4311744512 bytes (4.3 GB) copied, 226.133 s, 19.1 MB/s
$ hexdump -C zeros
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
101000000

The hexdump output should show all zeros. A non patched kernel will show
non-zero bytes.

[Original Bug Report]

on extracting files with extracted size >4 GB files are getting currupted.
interestingly file gets currupted in the very moment the file size gets more than 4GB.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: linux-image-3.11.0-12-generic 3.11.0-12.19
ProcVersionSignature: Ubuntu 3.11.0-12.19-generic 3.11.3
Uname: Linux 3.11.0-12-generic i686
ApportVersion: 2.12.5-0ubuntu2
Architecture: i386
Date: Wed Oct 23 12:11:43 2013
EcryptfsInUse: Yes
InstallationDate: Installed on 2013-07-20 (94 days ago)
InstallationMedia: Ubuntu 13.04 "Raring Ringtail" - Release i386 (20130424)
MarkForUpload: True
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.11.0-12-generic root=UUID=e97431f7-60b7-4fbe-b22f-5ca3304f2d50 ro quiet splash vt.handoff=7
SourcePackage: linux
UpgradeStatus: Upgraded to saucy on 2013-09-08 (45 days ago)

Revision history for this message
Lars Düsing (lars.duesing) wrote :
Revision history for this message
Brad Figg (brad-figg) wrote : Status changed to Confirmed

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed
Revision history for this message
Tyler Hicks (tyhicks) wrote : Re: ecryptfs currupts files over 4GB size

I can reproduce this in saucy and on mainline (3.12.0-031200rc6-generic). It is specific to i686.

My guess is that a size_t is being used when a loff_t is needed. I'm looking into it now.

Changed in linux (Ubuntu):
assignee: nobody → Tyler Hicks (tyhicks)
importance: Undecided → Critical
Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.12 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12-rc6-saucy/

Changed in linux (Ubuntu):
importance: Critical → Medium
status: Confirmed → Incomplete
importance: Medium → High
Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

Disregard my comment in #4. Commented at almost the same time :-)

Tyler Hicks (tyhicks)
tags: added: kernel-bug-exists-upstream
Changed in ecryptfs:
status: New → Triaged
Changed in linux (Ubuntu):
status: Incomplete → Triaged
Changed in ecryptfs:
importance: Undecided → Critical
assignee: nobody → Tyler Hicks (tyhicks)
Changed in linux (Ubuntu):
importance: High → Critical
tags: added: kernel-da-key
tags: added: kernel-key
removed: kernel-da-key
Tyler Hicks (tyhicks)
summary: - ecryptfs currupts files over 4GB size
+ ecryptfs currupts files over 4GB size on i686
Changed in ecryptfs:
status: Triaged → In Progress
Changed in linux (Ubuntu):
status: Triaged → In Progress
Revision history for this message
Colin Ian King (colin-king) wrote : Re: ecryptfs currupts files over 4GB size on i686

Tyler, the attached diff helps me read/write files over 4GB. I think some of the changes may be redundant, so I'm checking to see what is the minimal set of changes required.

Revision history for this message
Colin Ian King (colin-king) wrote :

Tyler, reduced it down to a one line fix. Tested with a 0.5GB, 2GB, 4GB, 8GB, 32GB test file with a sanity check on each byte. Patch attached below. Not yet tested with ecryptfs tests, but it looks like the root issue.

tags: added: tasty
tags: added: patch
Revision history for this message
Tyler Hicks (tyhicks) wrote :

Thanks Colin!

As I mentioned in IRC, I came up with the same patch last night and successfully ran tests against it overnight. We'll go with your patch since it already has a commit message. I targeted it for 3.11+ stable and will push it to Linus today. I'll also get the Ubuntu SRU prepared.

Tyler Hicks (tyhicks)
description: updated
Revision history for this message
Lars Düsing (lars.duesing) wrote :

Patch works for me on 3.12-rc6.

Thanks Colin and Tyler!

summary: - ecryptfs currupts files over 4GB size on i686
+ ecryptfs corrupts files over 4GB size on i686
Tyler Hicks (tyhicks)
description: updated
Changed in ecryptfs:
status: In Progress → Fix Committed
Revision history for this message
striscio (gianpaolo-racca) wrote :

Any update on this? I would like to use encrypted home, but I need to use big files for virtualbox

Revision history for this message
Lars Düsing (lars.duesing) wrote :

striscio, problem has been fixed on mainline-kernel 3.12, if you have to, just update like described in: http://ubuntuhandbook.org/index.php/2013/11/linux-kernel-3-12-released-install-ubuntu-or-linux-mint/
update to saucy-kernel will be as soon as possible.

tags: removed: kernel-key
Brad Figg (brad-figg)
Changed in linux (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-saucy' to 'verification-done-saucy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-saucy
Revision history for this message
Lars Düsing (lars.duesing) wrote :

-proposed works for me. Tested against NUL-File and multiple VirtualBox-images.

tags: added: verification-done-saucy
removed: verification-needed-saucy
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (5.6 KiB)

This bug was fixed in the package linux - 3.11.0-14.21

---------------
linux (3.11.0-14.21) saucy; urgency=low

  [Brad Figg]

  * Release Tracking Bug
    - LP: #1250540

  [ Anthony Wong ]

  * SAUCE: Work around broken ACPI backlight on Dell Inspiron 5537
    - LP: #1231305

  [ Colin Ian King ]

  * SAUCE: eCryptfs: fix 32 bit corruption issue
    - LP: #1243636

  [ Ming Lei ]

  * SAUCE: ext4: fix performance regression in ext4_writepages
    - LP: #1242812

  [ Upstream Kernel Changes ]

  * Revert "bridge: only expire the mdb entry when query is received"
    - LP: #1249081
  * ext4: fix performance regression in writeback of random writes
    - LP: #1242812
  * be2net: pass if_id for v1 and V2 versions of TX_CREATE cmd
    - LP: #1234019
  * tcp: TSO packets automatic sizing
    - LP: #1249081
  * tcp: TSQ can use a dynamic limit
    - LP: #1249081
  * tcp: must unclone packets before mangling them
    - LP: #1249081
  * tcp: do not forget FIN in tcp_shifted_skb()
    - LP: #1249081
  * tcp: fix incorrect ca_state in tail loss probe
    - LP: #1249081
  * net: do not call sock_put() on TIMEWAIT sockets
    - LP: #1249081
  * batman-adv: set up network coding packet handlers during module init
    - LP: #1249081
  * l2tp: fix kernel panic when using IPv4-mapped IPv6 addresses
    - LP: #1249081
  * l2tp: Fix build warning with ipv6 disabled.
    - LP: #1249081
  * net: mv643xx_eth: update statistics timer from timer context only
    - LP: #1249081
  * net: mv643xx_eth: fix orphaned statistics timer crash
    - LP: #1249081
  * net: heap overflow in __audit_sockaddr()
    - LP: #1249081
  * sit: amend "allow to use rtnl ops on fb tunnel"
    - LP: #1249081
  * proc connector: fix info leaks
    - LP: #1249081
  * ipv4: fix ineffective source address selection
    - LP: #1249081
  * can: dev: fix nlmsg size calculation in can_get_size()
    - LP: #1249081
  * net: secure_seq: Fix warning when CONFIG_IPV6 and CONFIG_INET are not
    selected
    - LP: #1249081
  * xen-netback: Don't destroy the netdev until the vif is shut down
    - LP: #1249081
  * net/mlx4_en: Rename name of mlx4_en_rx_alloc members
    - LP: #1249081
  * net/mlx4_en: Fix pages never dma unmapped on rx
    - LP: #1249081
  * net: vlan: fix nlmsg size calculation in vlan_get_size()
    - LP: #1249081
  * bridge: update mdb expiration timer upon reports.
    - LP: #1249081
  * vti: get rid of nf mark rule in prerouting
    - LP: #1249081
  * l2tp: must disable bh before calling l2tp_xmit_skb()
    - LP: #1249081
  * netem: update backlog after drop
    - LP: #1249081
  * netem: free skb's in tree on reset
    - LP: #1249081
  * farsync: fix info leak in ioctl
    - LP: #1249081
  * unix_diag: fix info leak
    - LP: #1249081
  * connector: use nlmsg_len() to check message length
    - LP: #1249081
  * bnx2x: record rx queue for LRO packets
    - LP: #1249081
  * virtio-net: don't respond to cpu hotplug notifier if we're not ready
    - LP: #1249081
  * virtio-net: refill only when device is up during setting queues
    - LP: #1249081
  * bridge: Correctly clamp MAX forward_delay when enabling STP
    - LP: #1249081
  * net: dst: provide accessor function to dst->xfrm
 ...

Read more...

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Changed in ecryptfs:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.