Huge memory usage

Could you attach files nuvolaplayer.ps and nuvolaplayer.pmap produced by following commands while Nuvola Player is playing?

ps auxw | egrep "RSS|nuvolaplayer" > nuvolaplayer.ps
pmap `pidof nuvolaplayer` > nuvolaplayer.pmap

And also nuvolaplayer.smem:

sudo apt-get install smem
smem -P nuvolaplayer > nuvolaplayer.smem

I have two machines with this issue. I had previously posted about the "work" machine, which seems to suffer from the problem worse. Attached files are for my "home" machine. I'll grab info from the work machine when I'm back at work on Monday.

Thanks for your data. Your pmem files revealed about 2000-3000 memory allocations not present in my own data. Some of them are really huge (e.g. single block of 34204K memory). It is necessary to identify where these allocations come from. I must admin I've never tried memory debugging, so I have to study this topic first.

Here's the promised entries from the work machine. I'm not sure why it has even higher memory usage from the home machine, as both are running the same Debian release, with similar packages installed.

Could you intstall packages "nuvolaplayer-dbg valgrind" and provide nuvolaplayer.massif generated by following command?

G_DEBUG=gc-friendly G_SLICE=always-malloc,debug-blocks \
MALLOC_CHECK_=2 valgrind --tool=massif \
--massif-out-file=nuvolaplayer.massif --max-snapshots=200 \
/usr/lib/nuvolaplayer/nuvolaplayer -D

Info (and console output) attached. Note that nuvolaplayer segfaulted before loading the google music library, much less playing anything! Tried this a couple times, and it was repeatable.

I launched with your command and after watching it trying to load for 30 minutes (w/ valgrind at 100% CPU) I killed the window and it didn't dump the nuvolaplayer.massif file.

Last log lines:

<normal startup debug>
    Nuvola Debug servicesmanager.vala:288: Found service Deezer at /usr/share/nuvolaplayer/services/deezer, version 1.5
< ~10 minutes passes >
*** NSPlugin Viewer *** ERROR: rpc_end_sync called when not in sync!

I'll try to give it another run while I'm not at my computer later.

On Mon, Feb 3, 2014 at 7:17 PM, Matthew Gabeler-Lee <email address hidden> wrote:
> Info (and console output) attached. Note that nuvolaplayer segfaulted
> before loading the google music library, much less playing anything!
> Tried this a couple times, and it was repeatable.

It's ok. Nuvola Player currently crashes under valgrind because of bug #1263705,

> ** Attachment added: "nuvolaplayer.work.massif.tar.gz"

Could you process the log with ms_print? (ms_print on my system cannot
parse your file produced by newer version of valgrind)

 ms_print nuvolaplayer.massif.log > nuvolaplayer.massif.report

actually, it looks like I buggered up creating that tarball and it didn't actually have the massif data. Here's one that actually does, with the ms_print formatted version too, just in case.

I gave it another shot. This time I left it alone for more than 3 hours. The GTK GUI loaded but nothing inside the webkit frame and the GTK menu wasn't responding to clicks or anything.

Here's the console output:

$ G_DEBUG=gc-friendly G_SLICE=always-malloc,debug-blocks MALLOC_CHECK_=2 valgrind --tool=massif --massif-out-file=nuvolaplayer.massif --max-snapshots=200 /usr/lib/nuvolaplayer/nuvolaplayer -D
==23997== Massif, a heap profiler
==23997== Copyright (C) 2003-2012, and GNU GPL'd, by Nicholas Nethercote
==23997== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==23997== Command: /usr/lib/nuvolaplayer/nuvolaplayer -D
==23997==
Gtk-Message: Failed to load module "overlay-scrollbar"
    Nuvola Info nuvolaplayer.vala:142: Welcome to Nuvola Player, version: 2.3.0
    Nuvola Info nuvolaplayer.vala:146: Revision: 790, <email address hidden>
    Nuvola Info nuvolaplayer.vala:147: Report any issues/bugs you might find to http://nuvolaplayer.fenryxo.cz/support/bug_reporting.html
    Nuvola Debug nuvolaplayer.vala:148: command: /usr/lib/nuvolaplayer/nuvolaplayer -D
    Nuvola Debug nuvolaplayer.vala:150: Enabled features: Unity Quicklist, optimization of SVG images, Last.fm scrobbling, Notifications, debug symbols
    Nuvola Debug nuvolaplayer.vala:151: Disabled features: experimental features, debug memory usage
       Gtk Debug Connecting to session manager
    Nuvola Debug nuvolaplayer.vala:233: Starting new instance
   Diorite Debug Libsoup version: 2.42.2
    Nuvola Debug nuvola-formatsupport.vala:97: Unable to init GStreamer 1.2.0, maybe already initialized
   Diorite Debug Max data cache size: 500
   Diorite Debug Setting proxy (auto): dynamic resolver
 <unknown> Debug NP_Initialize
 <unknown> Debug NP_Initialize succeeded
 <unknown> Debug NP_Initialize
 <unknown> Debug NP_Initialize succeeded
No bp log location saved, using default.
[000:000] Cpu: 6.58.9, x4, 2101Mhz, 7687MB
[000:024] Computer model: Not available
[000:029] Browser XEmbed support present: 1
[000:033] Browser toolkit is Gtk2.
[000:041] Using Gtk2 toolkit
No bp log location saved, using default.
[000:001] Cpu: 6.58.9, x4, 2101Mhz, 7687MB
[000:010] Computer model: Not available
 <unknown> Debug NP_Initialize
 <unknown> Debug NP_Initialize succeeded
GnomeShell Debug plugin loaded
Gtk-Message: Failed to load module "overlay-scrollbar"

(npviewer.bin:24033): Gtk-WARNING **: Unable to locate theme engine in module_path: "adwaita",

(npviewer.bin:24033): Gtk-WARNING **: Unable to locate theme engine in module_path: "adwaita",
[001:275] Warning(optionsfile.cc:47): Load: Could not open file, err=2
[001:280] No bp log location saved, using default.
[001:340] Cpu: 6.58.9, x4, 2101Mhz, 7687MB
[001:341] Computer model: Not available
[001:341] Browser XEmbed support present: 1
[001:341] Browser toolkit is Gtk2.
[001:342] Using Gtk2 toolkit
[001:125] Warning(optionsfile.cc:47): Load: Could not open file, err=2
[001:128] No bp log location saved, using default.
[001:179] Cpu: 6.58.9, x4, 2101Mhz, 7687MB
[001:180] Computer model: Not available
 <unknown> Debug NP_Initialize
 <un...

Same problem here on Arch Linux - Kernel 3.12.9-2-ARCH - KDE 4.12.1 - Nuvola 2.3.0

800 MB of RAM usage at start, way over 1 GB of RAM while playing.

Just putting this up if it may help at all as I experience this same bug:

ps auxw | egrep "RSS|nuvola" output:

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
breaker 14772 8.0 46.0 4775544 1802380 ? Sl 20:42 1:04 /usr/lib/nuvolaplayer/nuvolaplayer
breaker 14817 5.3 2.0 253528 79668 ? Sl 20:42 0:42 /usr/lib/nspluginwrapper/i386/linux/npviewer.bin --plugin /opt/nuvolaplayer/flash/orig/libflashplayer.so --connection /org/wrapper/NSPlugins/libflashplayer.so/14772-2/1117437047
breaker 15440 0.0 0.0 6032 980 pts/0 S+ 20:55 0:00 egrep RSS|nuvola

I am running Arch with Nuvola from the AUR, so 2.3.0. So far no problems other than the memory bug.

Same thing here. Arch Linux, latest Nuvola.

When it starts with Grooveshark, the memory is ~300MB. Then when I switch to Google Music, it climbs to 800MB, and by the time I click PLAY on a song, it is at 1.1GB, which then increases to ~1.5GB over the first several seconds of playback. It then settles and slowly but steadily rises to about 2GB, depending how long I keep it open. Stopping the playback has no effect on it.

However, if I don't switch to Google Music and stay on Grooveshark playing various songs, the memory usage stays between 120 and 180MB.

It will be necessary to repeat valgrind massif diagnostics with Nuvola Player 2.3.1. NP 2.3.0 crashes too early :-(

Also, please specify whether you use Flash-based or HTML5-based playback.

Flash-based. HTML5 doesn't seem to produce sound for me, besides it looks ugly :)

Did you mean I should run the massif diagnostics on my Nuvola?

On Fri, Mar 7, 2014 at 9:53 PM, lockheed <email address hidden> wrote:
> Flash-based. HTML5 doesn't seem to produce sound for me, besides it
> looks ugly :)

Google Play Music interface is same for both Flash-based and
HTML5-based playback.

> Did you mean I should run the massif diagnostics on my Nuvola?

Not with NP 2.3.0, because it crashed under valgrind. I hope I will
release NP 2.3.1 with a fix this week and I'll post a request for data
then.

Hello, as a regular user and patron this is my biggest issue with Nuvola. I'm a developer and I like to crunch some code by listening music. Then again, my applications developed are quite resource hug (mainly in memory) so every MB of memory that non development application use matters (because when I hit the swap, my machine starts to crumble), and I hate restarting the system. Currently Nuvola 2.3.0 at me uses 1.2GB of memory, tried on both Linux Mint 15 and Ubuntu 14.04. I hate using Google Music from the browser, so I'm really interested in resolving this. If there's anything I can assist with let me know. As for Nuvola 3.0, what I would love is to make it as resource low as possible. Let me configure to use only what I absolutely need. From my point of view the app should be able to fit inside 100-200 MB, given that it's a native app; and the only real feature I'm interested in is to stream music from Google Music, with the system integration of it's controlling (Play/Pause/Next - media keys keyboard support).

Other than this thanks for making this project, I love it!77//

On Sat, Mar 8, 2014 at 1:58 PM, Yeti <email address hidden> wrote:
> Currently Nuvola 2.3.0 at
> me uses 1.2GB of memory, tried on both Linux Mint 15 and Ubuntu 14.04.
> I hate using Google Music from the browser, so I'm really interested in
> resolving this. If there's anything I can assist with let me know.

The biggest problem for now is that I cannot reproduce the issue and
have no idea where the extra crazy memory allocation comes from :-(
Therefore, I will need some memory debugging data from users that
experience the issue.

> As
> for Nuvola 3.0, what I would love is to make it as resource low as
> possible. Let me configure to use only what I absolutely need.

Nuvola 3.0 will have all "extensions" built-in like NP 2.x, but NP 3.1
should have real extensions as loadable modules.

> From my
> point of view the app should be able to fit inside 100-200 MB, given
> that it's a native app;

Although Nuvola Player is a native app, it runs web apps that are
generally more memory hungry, so minimal memory usage will be always
around 200 MB. If you want memory usage bellow 200 MB, you should use
a completely native app instead (e.g. Rhythmbox).

I'm also affected by the memory bug (archlinux 64bit, gnome 3.10).

I haven't run any memory tools (besides taking a look at the memory map feature offered by system monitor, which revealed a single map w/ 1gb).

Running the above posted valgrind command yields in a segfault after a few seconds:

G_DEBUG=gc-friendly G_SLICE=always-malloc,debug-blocks \
MALLOC_CHECK_=2 valgrind --tool=massif \
--massif-out-file=nuvolaplayer.massif --max-snapshots=200 \
./nuvolaplayer -D
[...]
==5636==
==5636== Process terminating with default action of signal 11 (SIGSEGV)
==5636== Access not within mapped region at address 0xBBADBEEF
==5636== at 0x9BFA7F7: WTFCrash (in /usr/lib/libjavascriptcoregtk-3.0.so.0.15.10)
==5636== by 0x9A3509C: ??? (in /usr/lib/libjavascriptcoregtk-3.0.so.0.15.10)
==5636== by 0x9A366FD: ??? (in /usr/lib/libjavascriptcoregtk-3.0.so.0.15.10)
==5636== by 0x3979D065: ???
==5636== by 0x9A0FE6A: JSC::JITCode::execute(JSC::JSStack*, JSC::ExecState*, JSC::VM*) (in /usr/lib/libjavascriptcoregtk-3.0.so.0.15.10)
==5636== by 0x99F4D9C: JSC::Interpreter::execute(JSC::ProgramExecutable*, JSC::ExecState*, JSC::JSObject*) (in /usr/lib/libjavascriptcoregtk-3.0.so.0.15.10)
==5636== by 0x9AFFA3A: JSC::evaluate(JSC::ExecState*, JSC::SourceCode const&, JSC::JSValue, JSC::JSValue*) (in /usr/lib/libjavascriptcoregtk-3.0.so.0.15.10)
==5636== by 0x6D72DA6: ??? (in /usr/lib/libwebkitgtk-3.0.so.0.19.13)
==5636== by 0x6D73072: ??? (in /usr/lib/libwebkitgtk-3.0.so.0.19.13)
==5636== by 0x6F2E26C: ??? (in /usr/lib/libwebkitgtk-3.0.so.0.19.13)
==5636== by 0x6F2F60E: ??? (in /usr/lib/libwebkitgtk-3.0.so.0.19.13)
==5636== by 0x70D6ABA: ??? (in /usr/lib/libwebkitgtk-3.0.so.0.19.13)
==5636== If you believe this happened as a result of a stack
==5636== overflow in your program's main thread (unlikely but
==5636== possible), you can try to increase the size of the
==5636== main thread stack using the --main-stacksize= flag.
==5636== The main thread stack size used in this run was 8388608.
==5636==
Killed

Nonetheless I've attached the massif file. I have no experience w/ C/VALA memory debugging, just did this stuff for Java Applications, but if you give some hints of what to do...

> Running the above posted valgrind command yields in a segfault after a
> few seconds:

It will be necessary to repeat valgrind massif diagnostics with Nuvola
Player 2.3.1. NP 2.3.0 crashes too early :-(

Something else I've noticed (not tested 2.3.1 tho) that may be related to this:

The loading speed of the google music service is way faster in debian wheezy 32bit (vbox) (14sec) then in my primary arch linux 64bit environment (43sec). Loading speed = time opening the app and the orange progress bar is shown.

Anyone else with memory problems experiencing the same? Otherwise I'll fill another report.

Nuvola Player 2.3.1 has been released, so it's time for another debugging.

In terminal:

sudo apt-get install nuvolaplayer-dbg libwebkitgtk-3.0-0-dbg \
libjavascriptcoregtk-3.0-0-dbg libgtk-3-0-dbg valgrind smem

cd
# replace <your_nicname>
mkdir <your_nickname>
cd <your_nickname>

# Launch Google Play Music and start playback.
# Wait until a half of the song is played.
ps auxw | egrep "RSS|nuvolaplayer" > nuvolaplayer.ps
pmap `pidof nuvolaplayer` > nuvolaplayer.pmap
smem -P nuvolaplayer > nuvolaplayer.smem

# Quit Nuvola Player completely. (Menu Application -> Quit)

# time for coffee, will take long time to launch and the app will be slow
G_DEBUG=gc-friendly G_SLICE=always-malloc,resident-modules valgrind \
--tool=massif --massif-out-file=nuvolaplayer.massif --max-snapshots=200 \
--smc-check=all build/nuvolaplayer -D
# Start Google Play service and try to play a song.
# Wait until a half of the song is played.
# Quit the app.
# Create report
ms_print nuvolaplayer.massif.log > nuvolaplayer.massif.report

# Create a tar.gz archive and attach it.
# Specify your distribution.

> The loading speed of the google music service is way faster in debian wheezy 32bit (vbox) (14sec) then in my primary arch linux 64bit environment (43sec). Loading speed = time opening the app and the orange progress bar is shown.

It's necessary to load a lot of 32bit libraries for the wrapped Flash plugin in the 64bit environment, so the loading speed might be slower. Also, different versions WebKit libraries might have different speed.

I've attached the reports from my arch linux 64bit system. I used the standard AUR build from the provided 2.3.1 tar.gz from launchpad. However, it may be that I would need to recompile w/ debug flags. For this it would be convenient that you push the 2.3.1 changes to trunk :)

Re Loading Time:
Do you really think that this makes up 30 sec of loading (laptop has SSD)? I'll have a look at a 64 bit wheezy install tonight.

Oh and the commands

G_DEBUG=gc-friendly G_SLICE=always-malloc,resident-modules valgrind \
--tool=massif --massif-out-file=nuvolaplayer.massif --max-snapshots=200 \
--smc-check=all build/nuvolaplayer -D

should be

G_DEBUG=gc-friendly G_SLICE=always-malloc,resident-modules valgrind \
--tool=massif --massif-out-file=nuvolaplayer.massif --max-snapshots=200 \
--smc-check=all nuvolaplayer -D

and

ms_print nuvolaplayer.massif.log > nuvolaplayer.massif.report

should be

ms_print nuvolaplayer.massif > nuvolaplayer.massif.report

> I've attached the reports from my arch linux 64bit system. I used the
> standard AUR build from the provided 2.3.1 tar.gz from launchpad.
> However, it may be that I would need to recompile w/ debug flags. For
> this it would be convenient that you push the 2.3.1 changes to trunk :)

All changes in 2.3.1 (except for updates of translations) are already in trunk.

> Re Loading Time:
> Do you really think that this makes up 30 sec of loading (laptop has SSD)? I'll have a look at a 64 bit wheezy install tonight.

You're right, my wheezy amd64 installation shows up Google Play in a few second.

Updated instructions:

sudo apt-get install nuvolaplayer-dbg libwebkitgtk-3.0-0-dbg \
libjavascriptcoregtk-3.0-0-dbg libgtk-3-0-dbg valgrind smem

cd
# replace <your_nicname>
mkdir <your_nickname>
cd <your_nickname>

# Launch Google Play Music and start playback.
# Wait until a half of the song is played.
ps auxw | egrep "RSS|nuvolaplayer" > nuvolaplayer.ps
pmap `pidof nuvolaplayer` > nuvolaplayer.pmap
smem -P nuvolaplayer > nuvolaplayer.smem

# Quit Nuvola Player completely. (Menu Application -> Quit)

# time for coffee, will take long time to launch and the app will be slow
G_DEBUG=gc-friendly G_SLICE=always-malloc,resident-modules valgrind \
--tool=massif --massif-out-file=nuvolaplayer.massif --max-snapshots=200 \
--smc-check=all nuvolaplayer -D
# Start Google Play service and try to play a song.
# Wait until a half of the song is played.
# Quit the app.
# Create report
ms_print nuvolaplayer.massif > nuvolaplayer.massif.report

# Create a tar.gz archive and attach it.
# Specify your distribution.

Martin Pöhlmann, could you build Nuvola Player from the latest trunk (revision 813) and report memory usage with different cache models?

$ export NUVOLA_CACHE_MODEL="DOCUMENT_VIEWER"
$ LD_LIBRARY_PATH=build build/nuvolaplayer -D
$ ps auxw | egrep "RSS|nuvolaplayer" > nuvolaplayer.ps

$ export NUVOLA_CACHE_MODEL="DOCUMENT_BROWSER"
$ LD_LIBRARY_PATH=build build/nuvolaplayer -D
$ ps auxw | egrep "RSS|nuvolaplayer" > nuvolaplayer.ps

$ export NUVOLA_CACHE_MODEL="WEB_BROWSER"
$ LD_LIBRARY_PATH=build build/nuvolaplayer -D
$ ps auxw | egrep "RSS|nuvolaplayer" > nuvolaplayer.ps

$ export NUVOLA_CACHE_MODEL=""
$ LD_LIBRARY_PATH=build build/nuvolaplayer -D
$ ps auxw | egrep "RSS|nuvolaplayer" > nuvolaplayer.ps

Here are the results:

* DOCUMENT_VIEWER: ~1400 MB
* DOCUMENT_BROWSER: ~205 MB
* WEB_BROWSER: ~205 MB
* DEFAULT: ~195 MB

I can also confirm that the above described long loading times do just happen with DOCUMENT_VIEWER cache model.

Thanks!

In addition to reducing the RAM usage, this greatly improved my loading time too.

Thanks a lot!