Liferea stalling, uses excessive number of fsyncs

strace -etrace=fsync -o liferea_session.strace liferea

I've tried the current upstream stable version 1.4.21b with even worse results. strace reports 1749 fsyncs on first startup and shutdown after all feeds are initially loaded.

Unsuccessfully digging around for a workaround. I found that sqlite3 hard-codes the "safety_level" to full-sync and liferea keeps that setting at its defaults, i.e. doesn't make use of pragma synchronous.
There seems to be no remedy apart from patching the source.

This sounds like the known bug in xulrunner: https://bugzilla.mozilla.org/show_bug.cgi?id=421482

The issue is unfixed AFAIK (long tread, hard to keep track of the status), but one of the recommendations is to set preference 'toolkit.storage.synchronous' to 0. This is DANGEROUS, as it could result in the loss of your bookmarks and history and such, but in the case of an RSS reader, do you really care? The Liferea profile is independent of Firefox's.

I'm giving it a shot on my system. My initial impressions is "HOLY CRAP THAT STARTED IN LESS THAN 2 MINUTES!!!" It actually look all of 3 seconds.

Yes, this bug was a PITA with firefox 3.0. I've been running firefox profiles in a ramdisk ever since.
How that relates to liferea, other than both using sqlite, I dont know. Liferea calls sqlite functions on its own and as far as I could see not through xulrunner. 'toolkit.storage.synchronous=0' does have some limited effect on firefox, but how could it possibly affect liferea? Where did you set it?

Tried liferea again in Jaunty 64bit Alpha2, this time with reiserfs for a change, but there is no improvement. Still unusably slow to start and scores of fsyncs.

Meanwhile my upstream bug report has been closed without resolution.
http://sourceforge.net/support/tracker.php?aid=2216604

Same here on jaunty. liferea has become excessively slow, and stracing it also pointed to excessive fsync calls.

Being unhappy with the increasing fsync frequency of applications in general and misguided calls for even more fsyncs for ext4, I've finally given up and disabled fsync() system wide. After all that time liferea is now usable once again :)

marking this as "triaged" here, even though it's unclear what a possible solution would be...
importance=high

Same problem for me!

I can confirm that I am having this same issue.

Liferea: 1.4.26
Jaunty
Libsqlite 3.6.10

Laptop Specs: HP dv5242ea | Ubuntu 8.10 | Core Duo T2050 1.60GHz | 2048 MB RAM | 320GB SATA | DVD-RAM Matshita UJ 840S | nVidia G72M [GeForce Go 7400] 256mb | Intel PRO/Wireless 3945ABG

look https://bugs.launchpad.net/ubuntu/+source/liferea/+bug/333718

A temporary fix was posted here: http://ubuntuforums.org/showthread.php?p=7162556#8

No improvement in Karmic ~Alpha2, liferea 1.4.26-0ubuntu1, sqlite3 3.6.14.2-1. Starting liferea took >30min until I finally lost patience and killed it. That's with an empty ~/.liferea_1.4 folder and ext4 on a (admittedly rather slowish) SSD.

Here is the output of (full output attached):
$ rm -rf ~/.liferea_1.4/
$ time strace -e fsync,fdatasync -Ff -tT liferea 2>&1 | tee tmp/strace_karmic_lifrea_1.4.26-0ubuntu1.txt
...
[pid 24864] 15:39:59 fdatasync(12) = 0 <0.147859>
[pid 24864] 15:40:00 fdatasync(13) = 0 <0.000753>
[pid 24864] 15:40:00 fdatasync(12) = 0 <0.336274>
[pid 24864] 15:40:00 fdatasync(11) = 0 <0.691270>
...
[pid 24864] 16:10:36 fdatasync(15) = 0 <0.001026>
[pid 24864] 16:10:36 fdatasync(12) = 0 <0.335882>
[pid 24864] 16:10:36 fdatasync(11) = 0 <1.354669>
[pid 24864] 16:10:37 fdatasync(12 <unfinished ...>
[pid 25245] 16:10:38 +++ killed by SIGKILL +++
...
real 30m39.157s
user 0m5.044s
sys 0m6.816s

2838 fdatasyncs in total

Disabling fsync() doesn't help anymore in Karmic because Sqlite3 seems to have switched to fdatasync() by default. Disabling fdatasync does the trick for me this time, liferea starts almost instantly again.

The hack in
https://bugs.launchpad.net/ubuntu/+source/liferea/+bug/333718/comments/10
would need to be switched to fdatasync to make it work again, e.g.:

**** libfsync.c ****
int fdatasync (int fd) {
 return 0;
}
******************

I'll attach the one I'm using too, see Rodrigos instructions linked above on how to build it.

Switching to liferea 1.6.0 from https://launchpad.net/~liferea/+archive/ppa solved hang issues for me on Ubuntu 9.04 (2.6.29 kernel from http://kernel.ubuntu.com/~kernel-ppa/mainline/) and ext4.

Thanks for the PPA. Liferea-1.6.0-rc5 does a little better but it still takes 15min for the update of the initial feeds. I've built it from source too for good measure with the same result. I realize though that my SSD is probably close to the worst case and HDs would fare better.

My favorite solution of the day is now this patch for 1.4.26-0ubuntu1 that adds an environment variable for the sqlite3 sync behaviour. Possible values are 0..3, with 0 for no sync. See http://www.sqlite.org/pragma.html#pragma_synchronous.
Basically running liferea like this disables syncing:

LIFEREA_SYNCHRONOUS=0 liferea

I've created a slightly-modified version of marmuta's patch which changes the default synchronization mode to 0 (no sync), rather than preserving the existing default (2, full sync). Either patch will allow you to use an environment variable to set a specific synchronization mode, but by changing the default, this patch gives you the fast performance (and poor robustness!) of nosync without having to set any environment variables.

If that's what you want, use this patch instead.

Here is an updated quilt patch for Karmic's current liferea 1.6.0~rc6-1ubuntu1. Only line numbers changed, otherwise it applied fine. Before there were ~1400 fdatasyncs with a fresh profile, after none (assuming LIFEREA_SYNCHRONOUS set to 0).

Norman: Ultimately your patch is what I want too, but I felt it was more benign to stick with whatever liferea developers intended as a default. My hope is that this would allow Ubuntu devs/MOTUs to maybe, possibly, some day pick it up and add it to the package without risk for unaffected liferea users.

To still get the synchronization mode default to 0, you can for example add
export LIFEREA_SYNCHRONOUS=0
to ~/.profile, at least that is what I'm doing currently.

marmuta wrote:
> Here is an updated quilt patch for Karmic's current liferea
> 1.6.0~rc6-1ubuntu1.

Is it still as bad with 1.6.0? Performance fixes will come in 1.8, but I'm
interested to know if it's equally bad with 1.6 or anything has changed.

Emilio, I do see quite an improvement as 1.6 is something in the ballpark of 40% faster for me. Ext4/fdatasync(?) help too as the rest of the system stays fairly responsive.

Unfortunately it still means counting startup and feed update times in minutes on my setup. Have a look at the time stamps of the attachment. The main window showed up roughly 2min in and then it stayed mostly grey (compiz) for another 16min until all feeds where updated and drive activity went down. Add 20s for my reaction time to close it and the total is

real 18m53.812s
user 0m0.952s
sys 0m1.104s
1344 fdatasyncs
~39MiB written to the drive (/proc/diskstat)

With the patch and LIFEREA_SYNCHRONOUS=0:

real 0m37.556s
user 0m0.704s
sys 0m0.772s
0 fdatasyncs, a handful of fsyncs
~1.49 MiB written to the drive (/proc/diskstat)

This is with a fresh profile, all the default feeds and 270 new messages.
And again, this drive is slow, particularly for random-like I/O. I'm sure the majority of disks does better.

This is what I did:
rm -rf .liferea_1.4 .liferea_1.6
a) unset LIFEREA_SYNCHRONOUS
b) export LIFEREA_SYNCHRONOUS=0
sync
time strace -e fsync,fdatasync -Ff -tT liferea 2>&1 | tee /tmp/strace_liferea_1.6.0~rc6-1ubuntu1.txt

Where "/tmp" is a ram disk to take logging out of the picture.

marmuta wrote:
> Emilio, I do see quite an improvement as 1.6 is something in the
> ballpark of 40% faster for me. Ext4/fdatasync(?) help too as the rest of
> the system stays fairly responsive.

Good to hear :)

> Unfortunately it still means counting startup and feed update times in
> minutes on my setup.

This is definitely unacceptable and we will work on it for the next release.

* marmuta <email address hidden>:
> Emilio, I do see quite an improvement as 1.6 is something in the
> ballpark of 40% faster for me. Ext4/fdatasync(?) help too as the rest of
> the system stays fairly responsive.

Same here. It IS definitely faster. Not really fast, but faster.

Liferea still takes minutes to complete any feed update, and disables my system in the interim. This has essentially made liferea unusable.

Do I need to do the traces as previous?

tarek : )

Oh, to add to this, it's in the latest Karmic with liferea version 1.6.0

I've been having this issue as well, for the longest time.

In Jaunty, the workaround in post #12 worked perfectly, so I forgot about the issue. In Karmic, however, that workaround is no longer applicable.

The latest version is indeed much faster, but that doesn't mean much, as it's still unusable. Version 1.4.* with the workaround on post #12 or in Intrepid manages feeds in a matter of seconds, or instantly. Any Liferea version still included with Jaunty/Karmic by default makes even the simplest actions take minutes and turns the system unresponsive.

Is there any updated workaround for Karmic's release? There are very few (if any) alternatives to Liferea for GNOME and as it stands, it's very frustrating to use.

Tarek: I don't believe there is a need for more traces until a version >1.6 comes out.

perfectska04: Karmic's sqlite has switched from fsync() to fdatasync(). You can either update the workaround in #12 according to #14 or try the patch in #22.

I tried #12 in unadulterated form in Karmic latest, and that didn't worked.

I also tried modifying libfsync.so to:

int fdatasync (int fd) {
 return 0;
}

and that didn't work either.

Any ideas?

tarek : )

Oh, this DID work. Put is the file I used in its entirety for completeness:

****
#include <stdio.h>
int fsync(int i)
{
   return 0;
}

int fdatasync(int i)
{
   return 0;
}

****

Probably stdio.h isn't necessary.. Finally, things are back to normal! yay!

tarek : )

The new liferea does not have an editable /usr/bin/liferea file, any indications as to where I should add the last step for the workaround? (adding "export LD_PRELOAD=/usr/src/libfsync/libfsync.so" to said file)

@perfectska04

Just rename liferea to something else and call it through a script

# mv /usr/bin/liferea /usr/bin/liferea.real
# cat <<EOF >/usr/bin/liferea
#!/bin/sh
export LD_PRELOAD=/usr/local/lib/libfsync.so
/usr/bin/liferea.real $@
EOF
# chmod +x /usr/bin/liferea

@Maykel Moya
Thanks! After combining your instructions with post #30 and post #14, Liferea is now working as it should be. Globally updating feeds and "Mark all as read" now work instantly even on my netbook, as opposed to waiting for minutes.

@perfectska04

Please, would you mind explaining, really noobish friendly, what you did?

I tried to combine #32 and the fix explained in #12 with no success. But I must admit that I probably made something wrong.

Thanks

@Nicolás Schubert
Sure, I did as follows:

1. Enter the following in a terminal:
sudo mkdir /usr/src/libfsync
sudo gedit /usr/src/libfsync/libfsync.c

2. Copy the following inside the Gedit window that pops up and save+close afterwards:
#include <stdio.h>
int fsync(int i)
{
   return 0;
}

int fdatasync(int i)
{
   return 0;
}

3. Type the following into a terminal:
cd /usr/src/libfsync
gcc -Wall libfsync.c -o libfsync.so -shared -fPIC -Wl,-soname,libfsync.so
mv /usr/bin/liferea /usr/bin/liferea.real
sudo gedit /usr/bin/liferea

4. Type the following inside the Gedit window that pops up and save+close afterwards:
export LD_PRELOAD=/usr/src/libfsync/libfsync.so

That is all I had to do in order to get Karmic's Liferea to work as it normally should. Everything is mixed together from instructions posted in earlier comments.

Sorry, for step 4, you should put the following in the Gedit window:
export LD_PRELOAD=/usr/src/libfsync/libfsync.so
/usr/bin/liferea.real

And then give the new file permissions to run by running the following in a terminal:
sudo chmod +x /usr/bin/liferea

You might also need to add "sudo" to the second and third command lines in step 3, as they need administrator privileges.

Agh, I'm probably still half asleep. I apologize, for step 4 you also need an extra line at the beginning, here's what it should be:
!/bin/sh
export LD_PRELOAD=/usr/src/libfsync/libfsync.so
/usr/bin/liferea.real

@perfectska04

Thank you very much. It worked great!

I'm a bit dismayed that karmic shipped with a completely broken version of liferea. Is there any attempt to repair this?

Also, while the fix works for the first few minutes for me on the AMD64 version, it doesn't seem to last, and the system slows to a crawl again by the second sync.

tarek : )

i took the workaround and integrated it into the latest packages from the liferea unstable PPA. you can grab those packages here: https://launchpad.net/~bojo42/+archive/ppa/+sourcepub/893166/+listing-archive-extra

Does anyone now if the current upstream version 1.6.1 shows the same behaviour?

I did some quick straces with karmic's 1.6.0-1ubuntu2 vs. 1.6.1 and unstable 1.7.2 from the web site.

$ grep -c fdatasync *.txt
1.6.0-1ubuntu2.txt:1264
1.6.1.txt:1004
1.7.2.txt:996

$ grep -c fsync *.txt
1.6.0-1ubuntu2.txt:4
1.6.1.txt:3
1.7.2.txt:0

There is a slight downward trend, but most of the change is probably due to differing default feeds starting with 1.6.1. So apparently not much has changed just yet. Emilio mentioned that performance fixes will come with 1.8 anyway.

just for the record, in the workaround startup script the last line should be:
/usr/bin/liferea.real $*
otherwise parameters won't find there way to the binary and stuff like liferea-add-feed won't work.
if fixed the package in my ppa accordingly: https://launchpad.net/~bojo42/+archive/ppa/+sourcepub/896473/+listing-archive-extra

i redid the packaging of the fsync fix for version 1.7.3 and also created a seperate PPA for it:
https://launchpad.net/~bojo42/+archive/liferea

On 30/01/10 11:21, bojo42 wrote:
> i redid the packaging of the fsync fix for version 1.7.3 and also created a seperate PPA for it:
> https://launchpad.net/~bojo42/+archive/liferea

1.7.3 should be a bit better performance-wise, can you try it without this hack
and see how it performs?

@Emilio: sure i tried it before, but atleast on EXT4 and eCryptfs my HDD still makes those ugly sounds. could be a bit better than 1.7.2 but not really usable on my configuration. sry ;(

On 30/01/10 12:04, bojo42 wrote:
> @Emilio: sure i tried it before, but atleast on EXT4 and eCryptfs my HDD
> still makes those ugly sounds. could be a bit better than 1.7.2 but not
> really usable on my configuration. sry ;(

Oh right, we're still not good on ext4. On ext3 it should be much better.

any news for Lucid Lynx?

for EXT4 users: i repackaged 1.7.4 with the fsync fix for lucid https://launchpad.net/~bojo42/+archive/liferea

On 12/04/10 16:07, bojo42 wrote:
> for EXT4 users: i repackaged 1.7.4 with the fsync fix for lucid
> https://launchpad.net/~bojo42/+archive/liferea

People have reported that 1.7 works much better performance-wise. Do you still
need the hack with it? Have you tried without it?

@Emilio: this time you got me ;) i haven't tested it without the hack as the changes doesn't seem that big. i did so now and result is much better as my HDD keeps playing cool while updating the feeds, but i am already on lucid and this probably a result of changes in linux 2.6.32 regarding EXT4. i test it the next days if i there are really no drawbacks, but so far it's looking promising.

@all: would be great if someone could test 1.7.4 on karmic with EXT4 and the default 2.6.31 kernel.

On 13/04/10 10:22, bojo42 wrote:
> @Emilio: this time you got me ;) i haven't tested it without the hack as
> the changes doesn't seem that big. i did so now and result is much
> better as my HDD keeps playing cool while updating the feeds, but i am
> already on lucid and this probably a result of changes in linux 2.6.32
> regarding EXT4. i test it the next days if i there are really no
> drawbacks, but so far it's looking promising.

You can test Liferea 1.6 with that kernel to see if it's the changes in the
kernel or in Liferea (or in both).

At least yesterday, the performance in Lucid was still bad. I will post the exact versions when I get back home.

a few days of testing passed and the "vanilla" 1.7.4 from the unstable PPA is mostly usable on stock lucid (amd64). but i noticed that on high CPU load and low I/O (kernel compilation) the "vanilla" one gets unresponsive and updating feeds gets really slow. in the same situation the "hacked" build from my PPA stays responsive and update speed is okay. i don't understand the differnce as HDD usage is really low, but seems like high CPU load alone triggers a penalty regarding disc access with "vanilla". so i am staying with the hack ;(

> for EXT4 users: i repackaged 1.7.4 with the fsync fix for lucid https://launchpad.net/~bojo42/+archive/liferea

This version is much faster on lucid (ext4)

Disabling barriers on my ext4 partition seems to help.

in /etc/fstab:
UUID=9b920777-5aca-40f8-afa5-631a04057401 / ext4 defaults,noatime,nodiratime,barrier=0 1 1
UUID=27941cae-e541-407e-9572-f0adc25ba175 /home ext4 defaults,noatime,nodiratime,barrier=0 1 2

I added "barrier=0" as option and rebooted.

Thanks, Ralf, incredibly much faster after uninstalling my old liferea and get that version 1.7.4.
Installing...
liferea-data_1.7.4-2~llfsyncfix1_all.deb
liferea_1.7.4-2~llfsyncfix1_i386.deb
liferea-dbg_1.7.4-2~llfsyncfix1_i386.deb
...did the trick, now it works like a Ferrari :)

This patch solves the problem. Users should be aware that disabling fsync compromises durability (i.e. system crashes can corrupt your database).
The relevant liferea bug is https://sourceforge.net/tracker/index.php?func=detail&aid=2216604&group_id=87005&atid=581684

I think there is some development going on to circumvent this in future releases by making writes asynchronous.

I have forwarded the most recent patches upstream. If any other patches are created, could you make sure that you inform the original developers by posting the patch to their bug tracker too, otherwise the author of the program may not see your patch. Thanks!

Upstream doesn't like the patch. They prefer the fsyncs since that provides database consistency. As said in my previous post, some work is done in another direction, living with fsyncs.

If you don't want them and are ok with the consequences of disabling fsync, you can apply the patch. Upstream won't.

Apparently fixed upstream in the yet-unreleased 1.7.5. At least, I found this in the changelog:

* Improve the UI responsiveness by using sqlite3async.
   (patch by Wictor Lund)

So what is the status for natty? I recently switched from a patched version to the "original" natty version and OH BOY, what a PITA. Slow as hell.

Yes, slow as hell. One solution is to use:

eatmydata liferea

which works like a charm; but unfortunately if you start liferea from the indicator menu (in unity), it won't use "eatmydata liferea" but only "liferea" - and that's slow as hell.

I used patched version of liferea since it was uploaded, and I haven't faced any serious issues even when killing liferea process during feeds update. Definitely vote for "speed" over "persistence".

I totally understand why liferea developers don't want to remove fsyncs. But on the other side is one of the best RSS aggregator taking several seconds to just mark folder read. When I see HDD led glowing bright my first thought is "why liferea is updating feeds now?".
Maybe it's possible to provide liferea "add-on" in USC which will cause it to run with fsyncs disabled by default? With proper warnings in place.

Regards.

P.S. Thanks for the tip about "eatmydata".

Vadim, with "killing liferea process during feeds update" you are not testing the problematic case - for that fsync is not required.

"unplug the power cable of your computer during feed update" is the problematic case (or kernel crashes).

Same here on Ubuntu Natty Beta 2, with Liferea 1.6.4. It is slow e.g. when you mark multiple feeds as read.
Sorry, but for me it is unuseable in this state, I can't wait several seconds/minutes to mark feeds as read.

To reproduce this behaviour just install liferea, run it from me menu, wait until all feeds are received and finally mark all as read.

I've packaged up the fsync() and fsyncdata() functions above, written a little wrapper which runs a command with it (such as liferea) and then syncs ONCE when the application closes. Again, this is NOT the solution, but a lot of people might find this useful until liferea / sqlite is fixed.

https://projects.kintassa.com/projects/nosync

Installation: make && sudo make install. Running liferea through it: "nosync liferea".

Benchmarks:

liferea --debug-performance:

PERF: default_source_source_import took 3,482s

Perception of performance: liferea grinds the disk for a few seconds (on a 12GB desktop with no swap enabled!), then grinds some more even after the window appears.

nosync liferea --debug-performance:

PERF: default_source_source_import took 0,017s

Perception of performance: liferea comes up IMMEDIATELY, and is very responsive.

Ahh, I just saw the eatmydata comment above. Didn't know about that utility, so I'll withdraw this rather than duplicating efforts :)

Liferea is still unbearably slow on Precise - 12.04. Using eatmydata makes it usable, but this is not a solution.

After using the fsync disabled versions from my PPA for over two years now on regular basis i finally managed to crash my database. As Adrian pointed out in Comment #65 unplugging the power cord is quite effective ;)

All in all i think it still is a good tradeoff for personal use to run without the recommended safety from upstream, but you BETTER RUN some BACKUPS and be AWARE of POWER LOSSES while UPDATING your feeds. I'm quite happy i could restore a week old backup.

To spread the good news: Liferea 1.8.3 is our holy grail as it really fixes this bug.

We already have it in Debian and what's left to close this bug is to merge it to Ubuntu. If you like to help on that go to Bug #935147 and at least state that it affects you.

For anyone who likes to experience the wonders of 1.8.3 grab my proposed package for the merge from:
https://launchpad.net/~bojo42/+archive/liferea

This bug was fixed in the package liferea - 1.8.3-0.1ubuntu1

---------------
liferea (1.8.3-0.1ubuntu1) precise; urgency=low

  * New upstream release (LP: #290666, #371754, #741543, #716688)
  * Merge from Debian unstable (LP: #935147), remaining changes:
  * debian/patches:
    - drop gtk-status-icon.patch & notification-append as in upstream
    - drop fix_systray_behavior as mostly upstreamed and rest seems unused
    - 01_ubuntu_feedlists: update & rename, move planets to "Open Source"
    - add_X-Ubuntu-Gettext-Domain: rebase
    - libunity.patch: rebase, apply before indicator patch (liferea_shell.c)
    - libindicate_increase_version.patch: exclude from libindicate.patch
    - deactivate libindicate.patch, seems partly upstreamed and needs rework
  * debian/control: libindicate-dev, libindicate-gtk-dev & libunity-dev
  * debian/liferea.indicate & liferea.install: ship indicator desktop file
  * debian/rules: enable libindicate

liferea (1.8.3-0.1) unstable; urgency=low

   [ Bojo42 ]
   * Non-maintainer upload.
   * New upstream release (Closes: #502307, #623619, #631778, #651913)
   * debian/patches:
     - drop libnotify0.7 as in upstream
     - debian-example-feeds: update, move planets from "News" to "Open Source"
     - www-browser: update due to new file location
     - libtool-dont-rearange-as-needed: rebase
   * debian/control:
     - update Standards-Version
     - remove obsolete Build-Depends:
       - quilt not needed for "3.0 (quilt)" source format
       - libnm-glib-dev & libdbus-glib-1-dev: upstream switched to GDBus
       - liblua5.1-0-dev: LUA scripting support got dropped
     - new Build-Depends on libunique-dev, libjson-glib-dev & dh_autoreconf
     - update version dependencies
   * debian/rules: run dh_autoreconf & update translations
   * debian/liferea.install: nothing to ship from /usr/lib/liferea

   [ Rodrigo Gallardo ]
   * Lintian love:
     - debian/control: switch from Conflicts to Breaks
     - debian/rules: redo build targets

   [ Moray Allan ]
   * debian/copyright: update to include additional copyright owners.
   * debian/patches/www-browser: also set default external browser.
 -- bojo42 <email address hidden> Thu, 29 Mar 2012 14:17:21 +0200

Okay, I'm on fresh 13.10, just installed Liferea - slow performance and high HDD disk usage during updating is back.