U1 music store plugin severely delays banshee startup

Subscribing Rodney, who maintains the U1 music store extension..

  subscribe dobey

--
Kind regards,
Loong Jin

OK. The problem was that I had checked the option to copy music to my library when importing. I unchecked that option and cleared out all the duplicate files in the file system as well as in my music library. Now startup is much faster. I guess the base bug is invalid, though this is still pretty broken behaviour. Seems like a problem in Banshee where it shouldn't import a file multiple times.

There seem to be a few things that can cause slowdown here. Having copy_on_import enabled is one of them, and I think it is also the cause of all the dupes and other similar problems. I wonder if we should perhaps patch this option out, as it is one of the worst misfeatures a music player could have.

There are also some slowdowns when we try to talk to ubuntuone-syncdaemon over dbus, if syncdaemon isn't already running, or it fails to start. This seems to be because much of the C library API we're using to talk to it, is synchronous, and DBus stuff doesn't deal well when things don't reply immediately. I will try to fix some of this, but it may not be entirely doable for 11.04. :-/

shall we add a task to libubuntuone or it's ok here?

I would suggest changing it so that the copy-on-import feature always excludes the "Purchased from Ubuntu One" folder. I often use the copy-on-import feature since it automatically renames the files and sets up the directories cleanly with no input from myself.

> I wonder if we should perhaps patch [copy_on_import] out, as it is one of the worst misfeatures a music player could have.

Why do you consider this a misfeature? copy_on_import + automatically renaming files and folder is a pretty critical combination of settings for people who don't want to manually manage the folder structure of thousands (or even tens of thousands) of media files. Crippling a Banshee feature to workaround an issue that is unique to the U1MS extension doesn't seem like the right approach.

A couple related questions:

Why doesn't U1MS store downloaded music files in the folder that has already been set aside for music files (~/Music)? Ideally, the extension should probably follow the same file naming policies that the user defines within Banshee.

If this problem only happens when copy_on_import is turned on, does that mean that the U1MS extension is triggering an import of the entire ~/.ubuntuone directory every time Banshee starts?

I agree that the copy-on-import is a critical feature to remain. I have 100GB+ of music and don't want to have to manually manage the directory structure.

This can be very easily rectified by patching the copy-to-import function to specifically exclude the "Purchased from Ubuntu One" folder.

Copy on Import is a misfeature because I shouldn't have to have 2 copies of every song I import, and it causes the import to lose information about the file being imported; primarily, the piece of information about where it came from (which is why you end up with N+1 songs from the "Purchased from Ubuntu One" folder, every time you restart banshee. The issue is not unique to U1MS. It is simply the only thing exposing the problem right now. Banshee features are already crippled, which is why we have to rescan the Ubuntu One purchases at every startup in the first place. If Banshee supported multiple library folders as Rhythmbox does, this wouldn't be an issue. But rather than having music files in arbitrary locations on disk, it has a copy-on-import misfeature.

Why does it matter to you where the files are on disk or what directory names they are in or what the files are named? Are you importing lots of music which has broken/bad/missing tags? The only thing I see copy-on-import being useful for, is to import from external/removable media.

On 22/07/2011 20:51, Rodney Dawes wrote:
> Copy on Import is a misfeature because I shouldn't have to have 2 copies
> of every song I import, and it causes the import to lose information
> about the file being imported; primarily, the piece of information about
> where it came from (which is why you end up with N+1 songs from the
> "Purchased from Ubuntu One" folder, every time you restart banshee. The
> issue is not unique to U1MS. It is simply the only thing exposing the
> problem right now. Banshee features are already crippled, which is why
> we have to rescan the Ubuntu One purchases at every startup in the first
> place. If Banshee supported multiple library folders as Rhythmbox does,
> this wouldn't be an issue. But rather than having music files in
> arbitrary locations on disk, it has a copy-on-import misfeature.
>
> Why does it matter to you where the files are on disk or what directory
> names they are in or what the files are named? Are you importing lots of
> music which has broken/bad/missing tags? The only thing I see copy-on-
> import being useful for, is to import from external/removable media.
>

Actually it does. I like all my files in ~/Music, and Banshee creates the folder
structure and renames the files accordingly using tag information for me. All I
do is import them from wherever I got them, whether I downloaded them into
~/Desktop or got them from some external media.

--
Kind regards,
Loong Jin

Rodney wrote:
"Copy on Import is a misfeature because I shouldn't have to have 2 copies of every song I import, and it causes the import to lose information about the file being imported; primarily, the piece of information about where it came from (which is why you end up with N+1 songs from the "Purchased from Ubuntu One" folder, every time you restart banshee."

What you're describing makes sense, but IMHO you're saying what's wrong with the current implementation and what we got to fix, but nothing about if the idea of a "copy/move-on-import" feature is a good one or not. And I strongly agree it is a great feature to save your filesystem from clutter and I wouldn't use banshee without it.

My quick-fix for this problem would be:
1) turn off copy on import/rescanning of the U1 music folder
2) symlink the U1 music folder into ~/Music

On Sat, 2011-07-23 at 05:03 +0000, Chow Loong Jin wrote:
> Actually it does. I like all my files in ~/Music, and Banshee creates the folder
> structure and renames the files accordingly using tag information for me. All I
> do is import them from wherever I got them, whether I downloaded them into
> ~/Desktop or got them from some external media.

This is exactly why it is a misfeature. It does nothing to make the
program itself any better in any measurable way. It is there to satisfy
this odd corner case. And you didn't answer my question exactly. I asked
*why* it matters where they are, not whether or not you like them to be
in a certain place on disk. :)

On Sat, 2011-07-23 at 07:04 +0000, Jan Girlich wrote:
> What you're describing makes sense, but IMHO you're saying what's
> wrong
> with the current implementation and what we got to fix, but nothing
> about if the idea of a "copy/move-on-import" feature is a good one or
> not. And I strongly agree it is a great feature to save your
> filesystem
> from clutter and I wouldn't use banshee without it.

My calling it a misfeature seems to me to pretty clearly state that I
think it is a bad idea.

> My quick-fix for this problem would be:
> 1) turn off copy on import/rescanning of the U1 music folder
> 2) symlink the U1 music folder into ~/Music

The way Banshee works, does not make this an easy problem to solve,
short of simply patching out the copy-on-import misfeature. We must
re-scan the U1MS purchases on start to ensure the songs are imported
into the library. AFAICT, there is no [easy] way to tell Banshee to NOT
do the copy-on-import thing, when importing a track from the API.

I don't see how we can release Banshee in Oneiric with this kind of performance problem.

If this bug is not fixed, we'll have to disable the plugin by default.

OK, it seems the plugin is going to be disabled per the ubuntuone-installer plan in bug #833824

  https://code.launchpad.net/~dobey/ubuntu/oneiric/banshee/suggest-u1ms/+merge/74618

But still it remains that this bug will exist when the plugin is enabled via the installer (which will be on the images). dobey tells me that this will not be fixed for release. What should we do here? I guess a release note at the least.

There is a new build of libubuntuone in ppa:ubuntuone/nightlies now, which should help with this issue a bit. If people could install it and please let me know if there are any problems, that would be great. Thanks.

Much better for me, thanks

This "feels" much faster than before with the daily build.

although, I think it hangs again probably after 10 seconds when the timer fires.

You can test that by opening banshee, and immediately start music playing before the 10s timeout. If it's going to hang, the music should pause/stutter/something while it's rescanning, but it didn't seem to do this for me when I tested.

On Fri, Sep 09, 2011 at 08:05:55PM -0000, Rodney Dawes wrote:
> You can test that by opening banshee, and immediately start music
> playing before the 10s timeout. If it's going to hang, the music should
> pause/stutter/something while it's rescanning, but it didn't seem to do
> this for me when I tested.

The music does not hang, but the UI does. I'm sure it's the U1
extension, because I started banshee without it, began to play a song
and then enabled the extension. Exactly 10 seconds after I enabled the
extension, the UI hung for 25 seconds (presumably while it was waiting
for the DBUS timeout).

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]
PhD student [ <email address hidden> ]

my issue is slightly different (syncdaemon cannot be started at all vs. time taken scanning songs); I filed it a few days ago as bug #845031

This bug was fixed in the package libubuntuone - 0.11.0-0ubuntu1

---------------
libubuntuone (0.11.0-0ubuntu1) oneiric; urgency=low

  * New upstream release.
    - Dely initial rescan of purchased music folder for 10s (LP: #788532)
      This enables banshee to start faster
 -- Rodney Dawes <email address hidden> Tue, 13 Sep 2011 15:30:33 -0400