ureadahead generating oom messages during boot.

Bug #600359 reported by Tobin Davis
72
This bug affects 10 people
Affects Status Importance Assigned to Milestone
ureadahead (Ubuntu)
Fix Released
High
Scott James Remnant (Canonical)
Maverick
Fix Released
High
Scott James Remnant (Canonical)

Bug Description

Binary package hint: ureadahead

Several oom errors were detected in the dmesg logs after booting the preinstalled image on beagleboard (arm). See attached log output.

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: ureadahead 0.100.0-5
ProcVersionSignature: User Name 2.6.35-6.8-omap 2.6.35-rc3
Uname: Linux 2.6.35-6-omap armv7l
Architecture: armel
Date: Wed Jun 30 12:17:56 2010
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: ureadahead

Related branches

Revision history for this message
Tobin Davis (gruemaster) wrote :
Revision history for this message
Tobin Davis (gruemaster) wrote :

Tried to collect logs with apport-cli. Apparently they didn't get added.

Oliver Grawert (ogra)
Changed in ureadahead (Ubuntu Maverick):
milestone: none → maverick-alpha-3
importance: Undecided → Medium
status: New → Confirmed
Changed in ureadahead (Ubuntu Maverick):
assignee: nobody → Canonical Foundations Team (canonical-foundations)
Revision history for this message
Martin Pitt (pitti) wrote :

Is that the same problem as in bug 501715? Does it still happen on current maverick?

Revision history for this message
Stenten (stenten) wrote :

>
> Is that the same problem as in bug 501715? Does it still happen on
> current maverick?

I suspect not. That bug was fixed in 0.100.0-4.1.2, and the reporter is
running 0.100.0-5 (at least at the time they reported).

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

I'm still getting lots of OOM errors while booting latest Maverick image.

I just created a basic image with rootstock, using linux-image-2.6.35-14-omap, and I'm still getting the OOM errors.

Please check the attached file for the dmesg.

Packages:
 * linux-image-2.6.35-14-omap 2.6.35-14.19
 * ureadahead 0.100.0-6

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

Just running ureadahead at the console is enough to get the OOM:

root@beagle-maverick:~# free
             total used free shared buffers cached
Mem: 108912 17392 91520 0 56 2464
-/+ buffers/cache: 14872 94040
Swap: 0 0 0
root@beagle-maverick:~# ureadahead --verbose
/var/lib/ureadahead/pack: No such file or directory
[ 993.027557] Out of memory: kill process 678 (rsyslogd) score 8291 or a child
[ 993.039855] Killed process 678 (rsyslogd) vsz:33164kB, anon-rss:448kB, file-rss:0kB
[ 993.168151] Out of memory: kill process 691 (bash) score 1524 or a child
[ 993.180114] Killed process 700 (su) vsz:3968kB, anon-rss:252kB, file-rss:0kB
[ 993.202178] Out of memory: kill process 691 (bash) score 1027 or a child
[ 993.214141] Killed process 691 (bash) vsz:4108kB, anon-rss:360kB, file-rss:0kB
[ 993.449462] Out of memory: kill process 701 (bash) score 91 or a child
[ 993.461303] Killed process 721 (ureadahead) vsz:3504kB, anon-rss:156kB, file-rss:0kB

I'm testing at beagle R4, but even in a system with this much of memory ureadahead shouldn't eat all the memory.

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

Sorry, beagle B5 (will test on a C4 beagle tomorrow).

Revision history for this message
Martin Pitt (pitti) wrote :

Release noted for alpha-3.

Changed in ureadahead (Ubuntu Maverick):
milestone: maverick-alpha-3 → ubuntu-10.10-beta
Revision history for this message
Martin Pitt (pitti) wrote :

Comment 6 was on a board with a mere 108 MB RAM. Do we even officially support that? If so, I think ureadahead should just silently exit on systems with < 256 MB RAM, since there won't be enough space to fit everything at once anyway?

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

@Martin Pitt: we should support systems with at least 256MB RAM but there will be people who will try on older beagleboards, that only has 128MB RAM. Disable it with older systems would be enough (silently exit).

I'm currently working on this and will test with newer and older beagleboard revisions.

Changed in ureadahead (Ubuntu Maverick):
assignee: Canonical Foundations Team (canonical-foundations) → Ricardo Salveti (rsalveti)
status: Confirmed → In Progress
Revision history for this message
Paul Larson (pwlars) wrote :

I'm afraid 256MB is still insufficient. I see this during boot on my Beagle c4 with 256M

tags: added: iso-testing
Revision history for this message
Ricardo Salveti (rsalveti) wrote :

Added branch with the proposed fix.

The OOM happens because ureadahead sets the tracing buffer to 128MB, and at a system with small amount of memory (Beagleboard) this can cause the boot to call the OOM, killing ureadahead and plymouthd.

For the fix I decided to change only the initscript, instead of adding this restriction to the ureadahead code, because then the user can still run and test it if needed.

I also tried different memory thresholds, but didn't change the boot speed when comparing different bootcharts for BeagleBoard, so I just decided to avoid running it in this case.

Revision history for this message
Scott James Remnant (Canonical) (canonical-scott) wrote :

The OOM killer taking out ureadahead is not a problem; ureadahead will have had at least some chance to do some good work.

ureadahead using excessive memory during tracing due to the size of the tracing buffer is a different problem for which a fix has recently been uploaded, and a further fix is in the works.

So NAK on this patch

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

It's is a problem as it's showing this ugly message at the boot, never generates the pack and it usually takes other process with it (before OOM kills ureadahead).

The problem that was fixed regarding the size of tracing buffer is not related with this bug, it's just writing back the old values after running ureadahead.

And I did run ureadahead with different buffer sizes with bootchart (with values that it could generate the pack file), and the result is irrelevant, ureadahead doesn't make a difference here, and that's why we should just skip running it this way.

If you have a ureadahead rewrite that could fix this bug than OK, but I would say it would be good to test it first on this kind of systems, and testing with bootchart to see if it's relevant or not.

Oliver Grawert (ogra)
Changed in ureadahead (Ubuntu Maverick):
milestone: ubuntu-10.10-beta → ubuntu-10.10
Revision history for this message
Zygmunt Krynicki (zyga) wrote :

I experienced this issue on Linaro Beta Release Candidate despite having 256MB (Beagle Board C4) and 256MB swap (as a file on MMC).

Revision history for this message
Dave Martin (dave-martin-arm) wrote :

Ditto, but with C3 (256MB) and 512MB swapfile.

Scott James Remnant wrote on 2010-08-24:
> The OOM killer taking out ureadahead is not a problem;
> ureadahead will have had at least some chance to do some
> good work.

I'd be concerned about this: the OOM killer applies heuristics only and can never guarantee to kill the process that is responsible for low-memory conditions, so if OOM runs at all, system robustness is no longer guaranteed. This may lead to unpredictable and nasty behaviours when the platform is no longer freshly installed, extra packages and daemons have been added etc....

Relying on the OOM killer behaviour for correct operation of the system feels unsafe to me.

Revision history for this message
Jamie Bennett (jamiebennett) wrote :

Any progress on this bug?

Revision history for this message
trunet (wsartori) wrote :

I'm having the same problems using an ALIX 2D13 board(x86). I removed ureadahead package and the problem goes away.

Revision history for this message
Robbie Williamson (robbiew) wrote :

We are working on it. According to Scott, the issue is overhead balance. Right now it either uses too much CPU during boot or too much disk space after, but the good news is that he thinks he has solved it.

Changed in ureadahead (Ubuntu Maverick):
assignee: Ricardo Salveti (rsalveti) → Canonical Foundations Team (canonical-foundations)
Revision history for this message
Jamie Bennett (jamiebennett) wrote :

Great news Robbie. Any ETA on this considering the pending Final Freeze?

Revision history for this message
Robbie Williamson (robbiew) wrote :

@Jamie: sorry, not yet.

Revision history for this message
Robbie Williamson (robbiew) wrote :

Raising to "high" because of the impact to usability, and for tighter release tracking

Changed in ureadahead (Ubuntu Maverick):
importance: Medium → High
Revision history for this message
Phillip Susi (psusi) wrote :

It seems to me that ureadahead should absolutely not be run on a system with only 128 mb of ram, and probably not 256 either. A typical boot loads over 100 mb of data. With only 128 to start with, and even with 256 after some is used on the trace buffer itself, portions of the buffer cache will already be discarded by the time ureadahead checks it, so it will at best, miss a good portion of the files accessed during the boot. Those that it does catch will be the ones accessed later in the boot. During subsequent boots, this will cause ureadahead to fill the cache with files that won't be needed until later during the boot, which will be discarded to make room for the earlier files anyhow, thus making it a waste of time.

Revision history for this message
Robbie Williamson (robbiew) wrote :

A fix is in the works and should arrive next week that will lower the amount of memory ureadahead reserves, which should resolve this issue.

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

At least with current ureadahead, even when lowering the amount of memory it doesn't change the booting time for Beagle (using SD card). Hopefully this newer version also improves how ureadahead uses the data.

Revision history for this message
Robbie Williamson (robbiew) wrote :

not sure about that, in that case it might be worth not using it at all on machines with 128mb ram or lower....but I'll let Scott make that decision ;)

Revision history for this message
Dave Martin (dave-martin-arm) wrote :

> It seems to me that ureadahead should absolutely not be run on a system with only 128 mb of ram
> and probably not 256 either

Should ureadahead compare the amount of data it will try to load with the RAM size? Hard-coding a threshold like 128MB or 256MB seems like a bad idea, because this won't be robust against future changes to the footprint of the base system.

Changed in ureadahead (Ubuntu Maverick):
assignee: Canonical Foundations Team (canonical-foundations) → Scott James Remnant (scott)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ureadahead - 0.100.0-8

---------------
ureadahead (0.100.0-8) maverick; urgency=low

  * Decrease the buffer size to just 8MB, after much testing we don't
    need much more than this since it will be limited by the size of the
    page cache anyway.

    This is in lieu of a new version of ureadahead for Maverick, which
    while work is ongoing, isn't ready for shipping at this time.
    LP: #600359.
 -- Scott James Remnant <email address hidden> Mon, 20 Sep 2010 18:34:31 +0100

Changed in ureadahead (Ubuntu Maverick):
status: In Progress → Fix Released
Revision history for this message
Ayan George (ayan) wrote : new ladie profiles

Alla 23 y.o. invites you for chat (on-line now)
(Luba 25 y.o.) New reply for you.
(Lili, 23 y.o.) - Could I be your Girlfriend? :)
Vika (23 y.o.) - Are you ready to Marry Me?
Lyba 25 y.o. - I search for Real boyfriend
(Yanina, 25 y.o.), I search for beloved man, are you?
Yulia 21 y.o. , new message for you
you have got new reply from Vera 21 y.o.
you have new message from Maria 23 y.o.
Zhenya 23 y.o. -waiting for your reply

http://sonya201010.com.ua/?message_from=Lidiya

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.