Merge lp:~jamesh/mediascanner/albumart-download into lp:mediascanner

[James Henstridge]

Download album art to the media art cache during scanning rather than storing remote URLs in the index.

Instead of just adding album art URLs to the index from the metadata resolver, actively download the art to the media art cache.

When reading metadata from the index later, add any art found in the cache to the results.

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:398
http://jenkins.qa.ubuntu.com/job/mediascanner-ci/70/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-amd64-ci/70/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-armhf-ci/70/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-i386-ci/70/console

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/mediascanner-ci/70/rebuild

review: Needs Fixing (continuous-integration)

[Paweł Stołowski (stolowski)]

Please add libsoup to debian/control

review: Needs Fixing

[Paweł Stołowski (stolowski)]

Also it fails for me on tests when building the deb.

[Paweł Stołowski (stolowski)]

I got quite a few warnings about getting image/png instead of image/jpeg. Looking at the code we ignore such album art? Can we handle it?

review: Needs Information

[lp:~jamesh/mediascanner/albumart-download]

399. By James Henstridge on 2013-10-08

Separate out loading of the Grilo plugin from getting the source in
MediaArtDownloader, and fix the FileSystemScanner tests to set up the
lastfm plugin.

400. By James Henstridge on 2013-10-08

Add libsoup as a Build-Depend.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:400
http://jenkins.qa.ubuntu.com/job/mediascanner-ci/72/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-amd64-ci/72
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-armhf-ci/72
        deb: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-armhf-ci/72/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-i386-ci/72

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/mediascanner-ci/72/rebuild

review: Approve (continuous-integration)

[lp:~jamesh/mediascanner/albumart-download]

401. By James Henstridge on 2013-10-09

Merge lp:~jpakkane/mediascanner/quickfixes

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:401
http://jenkins.qa.ubuntu.com/job/mediascanner-ci/74/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-amd64-ci/74
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-armhf-ci/74
        deb: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-armhf-ci/74/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mediascanner-saucy-i386-ci/74

Click here to trigger a rebuild:
http://10.97.0.26:8080/job/mediascanner-ci/74/rebuild

review: Approve (continuous-integration)

[Paweł Stołowski (stolowski)]

Looks good to me, functional tests confirm it works. I'm slightly concerned about the image/png case mentioned in the comment, but don't want to block on that; to my understanding it should work as long as mediascanner returns "" to the shell; if it's a problem, then it will be a minor fix for later.

review: Approve

=== modified file 'CMakeLists.txt'
--- CMakeLists.txt	2013-08-29 08:00:35 +0000
+++ CMakeLists.txt	2013-10-09 07:38:10 +0000
@@ -105,6 +105,7 @@
                   gstreamer-pbutils-1.0>=0.11.93
                   gudev-1.0>=175
                   liblucene++>=3.0.3.4
+                  libsoup-2.4
 )
 
 # http://public.kitware.com/Bug/view.php?id=14381

=== modified file 'debian/control'
--- debian/control	2013-08-28 13:33:08 +0000
+++ debian/control	2013-10-09 07:38:10 +0000
@@ -20,6 +20,7 @@
                libgtest-dev,
                libgudev-1.0-dev,
                liblucene++-dev (>= 3.0.4),
+               libsoup2.4-dev,
                python,
 Standards-Version: 3.9.4
 Homepage: https://launchpad.net/mediascanner

=== modified file 'mediascanner.gschema.xml.in'
--- mediascanner.gschema.xml.in	2013-08-23 02:58:42 +0000
+++ mediascanner.gschema.xml.in	2013-10-09 07:38:10 +0000
@@ -76,7 +76,7 @@
        dictionary of strings describing configuration parameters like API keys.
       </description>
 
-      <default>[("grl-tmdb", "grl-tmdb", {"api-key": "@TMDB_APIKEY@"}), ("grl-lastfm-albumart", "grl-lastfm-albumart", {})]</default>
+      <default>[("grl-tmdb", "grl-tmdb", {"api-key": "@TMDB_APIKEY@"})]</default>
     </key>
   </schema>
 </schemalist>

=== modified file 'src/mediascanner-service/main.cpp'
--- src/mediascanner-service/main.cpp	2013-09-11 09:49:12 +0000
+++ src/mediascanner-service/main.cpp	2013-10-09 07:38:10 +0000
@@ -43,6 +43,7 @@
 #include "mediascanner/glibutils.h"
 #include "mediascanner/locale.h"
 #include "mediascanner/logging.h"
+#include "mediascanner/mediaartdownloader.h"
 #include "mediascanner/mediaroot.h"
 #include "mediascanner/metadataresolver.h"
 #include "mediascanner/property.h"
@@ -881,6 +882,9 @@
     if (not LoadMetadataSources())
         return false;
 
+    // Set up the media art downloader
+    MediaArtDownloader::LoadGriloPlugin();
+
     // Setup the scanner itself.
     notifier_.reset(new CommitNotifier(this));
     task_manager_.reset(new TaskManager("media scanner service"));

=== modified file 'src/mediascanner/CMakeLists.txt'
--- src/mediascanner/CMakeLists.txt	2013-08-27 14:41:14 +0000
+++ src/mediascanner/CMakeLists.txt	2013-10-09 07:38:10 +0000
@@ -38,6 +38,7 @@
     logging.cpp
     logging.h
     mediaartcache.cpp
+    mediaartdownloader.cpp
     mediaindex.cpp
     mediaroot.cpp
     mediautils.cpp

=== modified file 'src/mediascanner/declarations.h'
--- src/mediascanner/declarations.h	2013-08-23 09:45:11 +0000
+++ src/mediascanner/declarations.h	2013-10-09 07:38:10 +0000
@@ -112,6 +112,7 @@
 
 class CommitPolicy;
 class Filter;
+class MediaArtDownloader;
 class MediaIndex;
 class MediaInfo;
 class MediaRoot;
@@ -127,6 +128,7 @@
 class Wrapper;
 
 typedef std::shared_ptr<CommitPolicy> CommitPolicyPtr;
+typedef std::shared_ptr<MediaArtDownloader> MediaArtDownloaderPtr;
 typedef std::shared_ptr<MediaRootManager> MediaRootManagerPtr;
 typedef std::shared_ptr<MetadataResolver> MetadataResolverPtr;
 typedef std::shared_ptr<TaskManager> TaskManagerPtr;

=== modified file 'src/mediascanner/filesystemscanner.cpp'
--- src/mediascanner/filesystemscanner.cpp	2013-09-09 11:49:53 +0000
+++ src/mediascanner/filesystemscanner.cpp	2013-10-09 07:38:10 +0000
@@ -32,6 +32,7 @@
 #include "mediascanner/logging.h"
 #include "mediascanner/mediaroot.h"
 #include "mediascanner/metadataresolver.h"
+#include "mediascanner/mediaartdownloader.h"
 #include "mediascanner/taskfacades.h"
 #include "mediascanner/taskmanager.h"
 #include "mediascanner/writablemediaindex.h"
@@ -58,6 +59,7 @@
             TaskManagerPtr index_task_manager,
             TaskFacadePtr index_task_facade)
         : metadata_resolver_(resolver)
+        , art_downloader_(new MediaArtDownloader())
         , file_task_manager_(new TaskManager("file system scanner"))
         , index_task_manager_(index_task_manager)
         , index_task_facade_(index_task_facade)
@@ -116,6 +118,7 @@
     }
 
     MetadataResolverPtr metadata_resolver_;
+    MediaArtDownloaderPtr art_downloader_;
     TaskManagerPtr file_task_manager_;
     TaskManagerPtr index_task_manager_;
     TaskFacadePtr index_task_facade_;
@@ -153,6 +156,7 @@
     }
 
     FileSystemWalkerPtr walker(new FileSystemWalker(root, metadata_resolver_,
+                                                    art_downloader_,
                                                     file_task_manager_,
                                                     index_task_manager_,
                                                     index_task_facade_));
@@ -348,6 +352,9 @@
         if (metadata_resolver_)
             metadata_resolver_->WaitForFinished();
 
+        if (art_downloader_)
+            art_downloader_->WaitForFinished();
+
         BOOST_ASSERT(is_idle());
 
         Idle::AddOnce(std::bind(&Private::NotifyScanningFinished,

=== modified file 'src/mediascanner/filesystemwalker.cpp'
--- src/mediascanner/filesystemwalker.cpp	2013-09-17 10:48:31 +0000
+++ src/mediascanner/filesystemwalker.cpp	2013-10-09 07:38:10 +0000
@@ -45,6 +45,7 @@
 #include "mediascanner/glibutils.h"
 #include "mediascanner/locale.h"
 #include "mediascanner/logging.h"
+#include "mediascanner/mediaartdownloader.h"
 #include "mediascanner/mediaroot.h"
 #include "mediascanner/mediautils.h"
 #include "mediascanner/metadataresolver.h"
@@ -97,6 +98,7 @@
     Private(size_t group_id,
             const MediaRoot &media_root,
             MetadataResolverPtr resolver,
+            MediaArtDownloaderPtr art_downloader,
             TaskManagerPtr file_task_manager,
             TaskManagerPtr index_task_manager,
             TaskFacadePtr index_task_facade,
@@ -189,6 +191,7 @@
     const ErrorFunction report_fatal_error_;
     const Wrapper<GCancellable> cancellable_;
     const MetadataResolverPtr metadata_resolver_;
+    const MediaArtDownloaderPtr art_downloader_;
     const TaskManagerPtr file_task_manager_;
     const TaskManagerPtr index_task_manager_;
     const TaskFacadePtr index_task_facade_;
@@ -220,6 +223,7 @@
 FileSystemWalker::Private::Private(size_t group_id,
                                    const MediaRoot &media_root,
                                    MetadataResolverPtr resolver,
+                                   MediaArtDownloaderPtr art_downloader,
                                    TaskManagerPtr file_task_manager,
                                    TaskManagerPtr index_task_manager,
                                    TaskFacadePtr index_task_facade,
@@ -227,6 +231,7 @@
     : report_fatal_error_(report_fatal_error)
     , cancellable_(take(g_cancellable_new()))
     , metadata_resolver_(resolver)
+    , art_downloader_(art_downloader)
     , file_task_manager_(file_task_manager)
     , index_task_manager_(index_task_manager)
     , index_task_facade_(index_task_facade)
@@ -248,12 +253,13 @@
 
 FileSystemWalker::FileSystemWalker(const MediaRoot &media_root,
                                    MetadataResolverPtr resolver,
+                                   MediaArtDownloaderPtr art_downloader,
                                    TaskManagerPtr file_task_manager,
                                    TaskManagerPtr index_task_manager,
                                    TaskFacadePtr index_task_facade)
     : d(new Private(reinterpret_cast<size_t>(this),
-                    media_root, resolver, file_task_manager,
-                    index_task_manager, index_task_facade,
+                    media_root, resolver, art_downloader,
+                    file_task_manager, index_task_manager, index_task_facade,
                     d->bind_report_error_literal())) {
 }
 
@@ -926,6 +932,15 @@
         metadata.add_single(schema::kTitle.bind_value(w_title));
     }
 
+    // If the media has artist and album, then resolve artwork
+    if (not metadata.first(schema::kArtist).empty() &&
+        not metadata.first(schema::kAlbum).empty()) {
+        std::string artist = FromUnicode(metadata.first(schema::kArtist));
+        std::string album = FromUnicode(metadata.first(schema::kAlbum));
+
+        art_downloader_->resolve(artist, album);
+    }
+
     StoreMediaInfo(w_url, metadata);
 
     if ((details & ScanMetadata) && metadata_resolver_)

=== modified file 'src/mediascanner/filesystemwalker.h'
--- src/mediascanner/filesystemwalker.h	2013-08-23 09:45:11 +0000
+++ src/mediascanner/filesystemwalker.h	2013-10-09 07:38:10 +0000
@@ -39,6 +39,7 @@
 
     FileSystemWalker(const MediaRoot &media_root,
                      MetadataResolverPtr resolver,
+                     MediaArtDownloaderPtr art_downloader,
                      TaskManagerPtr file_task_manager,
                      TaskManagerPtr index_task_manager,
                      TaskFacadePtr index_task_facade);

=== modified file 'src/mediascanner/glibutils.h'
--- src/mediascanner/glibutils.h	2013-09-10 14:02:38 +0000
+++ src/mediascanner/glibutils.h	2013-10-09 07:38:10 +0000
@@ -26,6 +26,7 @@
 #include <grilo.h>
 #include <gst/pbutils/gstdiscoverer.h>
 #include <gudev/gudev.h>
+#include <libsoup/soup.h>
 
 // Boost C++
 #include <boost/date_time/posix_time/posix_time_duration.hpp>
@@ -350,6 +351,11 @@
 template<> struct CopyHelper<GUdevDevice> :
         public ObjectCopyHelper<GUdevDevice> { };
 
+template<> struct CopyHelper<SoupMessage> :
+        public ObjectCopyHelper<SoupMessage> { };
+template<> struct CopyHelper<SoupSession> :
+        public ObjectCopyHelper<SoupSession> { };
+
 template<> struct CopyHelper<char> {
     static char *Copy(char *p) { return g_strdup(p); }
     static void Free(char *p) { g_free(p); }

=== modified file 'src/mediascanner/mediaartcache.cpp'
--- src/mediascanner/mediaartcache.cpp	2013-08-29 11:28:09 +0000
+++ src/mediascanner/mediaartcache.cpp	2013-10-09 07:38:10 +0000
@@ -85,7 +85,7 @@
 }
 
 void MediaArtCache::add_art(const std::string &artist, const std::string &album,
-        char *data, unsigned int datalen) {
+        const char *data, unsigned int datalen) {
     string abs_fname = get_full_filename(artist, album);
     GError *err = nullptr;
     if(!g_file_set_contents(abs_fname.c_str(), data, datalen, &err)) {
@@ -108,6 +108,30 @@
     return "";
 }
 
+string MediaArtCache::get_art_uri(const string &artist, const string &album) const
+{
+    std::string filename = get_art_file(artist, album);
+
+    if (filename == "") {
+        return "";
+    }
+
+    GError *err = nullptr;
+    char *c_uri = g_filename_to_uri(filename.c_str(), "", &err);
+    if (!c_uri) {
+        string e("Could not convert file name ");
+        e += filename;
+        e += " to URI: ";
+        e += err->message;
+        g_error_free(err);
+        throw runtime_error(e);
+    }
+
+    std::string uri = c_uri;
+    g_free(c_uri);
+    return uri;
+}
+
 std::string MediaArtCache::get_full_filename(const std::string &artist, const std::string & album) const {
     return root_dir + "/" + compute_base_name(artist, album);
 }

=== modified file 'src/mediascanner/mediaartcache.h'
--- src/mediascanner/mediaartcache.h	2013-08-29 11:28:09 +0000
+++ src/mediascanner/mediaartcache.h	2013-10-09 07:38:10 +0000
@@ -47,8 +47,9 @@
     MediaArtCache();
     bool has_art(const std::string &artist, const std::string &album) const;
     void add_art(const std::string &artist, const std::string &album,
-            char *data, unsigned int datalen);
+            const char *data, unsigned int datalen);
     std::string get_art_file(const std::string &artist, const std::string &album) const;
+    std::string get_art_uri(const std::string &artist, const std::string &album) const;
     void clear() const;
     void prune();
     std::string get_cache_dir() const { return root_dir; }

=== added file 'src/mediascanner/mediaartdownloader.cpp'
--- src/mediascanner/mediaartdownloader.cpp	1970-01-01 00:00:00 +0000
+++ src/mediascanner/mediaartdownloader.cpp	2013-10-09 07:38:10 +0000
@@ -0,0 +1,297 @@
+/*
+ * This file is part of the Ubuntu TV Media Scanner
+ * Copyright (C) 2012-2013 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License version 3 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Contact: Jim Hodapp <[e-mail]>
+ * Authored by: James Henstridge <[e-mail]>
+ */
+
+#include "mediascanner/logging.h"
+#include "mediascanner/glibutils.h"
+#include "mediascanner/mediaartcache.h"
+#include "mediascanner/mediaartdownloader.h"
+
+#define ALBUM_ART_PLUGIN "grl-lastfm-albumart"
+#define ALBUM_ART_SOURCE "grl-lastfm-albumart"
+
+namespace mediascanner {
+
+static const logging::Domain kInfo("info/mediaartdownloader", logging::info());
+static const logging::Domain kWarning("warning/mediaartdownloader", logging::warning());
+
+class MediaArtDownloader::Private {
+public:
+    struct ResolveData {
+        ResolveData(Private *d,
+                    const std::string &artist,
+                    const std::string &album)
+            : d(d)
+            , artist(artist)
+            , album(album) {
+        }
+
+        Private *const d;
+        const std::string artist;
+        const std::string album;
+
+        guint operation_id; // Grilo operation
+        Wrapper<SoupMessage> message;
+    };
+    typedef std::shared_ptr<ResolveData> ResolveDataPtr;
+
+    Private() {
+        g_mutex_init(&mutex);
+        g_cond_init(&cond);
+        setup_resolver();
+        setup_soup_session();
+    }
+
+    ~Private() {
+        cancel();
+    }
+
+    bool is_idle() const {
+        return jobs.empty();
+    }
+
+    GMutex mutex;
+    GCond cond;
+    MediaArtCache cache;
+    Wrapper<GrlSource> art_resolver;
+    Wrapper<SoupSession> session;
+
+    std::vector<ResolveDataPtr> jobs;
+
+    void cancel();
+    void setup_resolver();
+    void setup_soup_session();
+
+    void add_job(ResolveDataPtr data);
+    void complete(ResolveData *data);
+
+    static void OnResolveThumbnail(GrlSource *source, unsigned operation_id,
+                                   GrlMedia *media, void *user_data,
+                                   const GError *error);
+    static void OnResponseReceived(SoupSession *session, SoupMessage *message,
+                                   void *user_data);
+};
+
+MediaArtDownloader::MediaArtDownloader()
+    : d(new Private) {
+}
+
+MediaArtDownloader::~MediaArtDownloader() {
+    delete d;
+}
+
+void MediaArtDownloader::LoadGriloPlugin() {
+    const Wrapper<GrlRegistry> registry = wrap(grl_registry_get_default());
+    Wrapper<GError> error;
+
+    if (not grl_registry_load_plugin_by_id(
+            registry.get(), ALBUM_ART_PLUGIN, error.out_param())) {
+        const std::string error_message = to_string(error);
+        kWarning("Could not load album art plugin: {1}") % error_message;
+        return;
+    }
+}
+
+void MediaArtDownloader::Private::setup_resolver() {
+    const Wrapper<GrlRegistry> registry = wrap(grl_registry_get_default());
+
+    art_resolver = wrap(grl_registry_lookup_source(
+                            registry.get(), ALBUM_ART_SOURCE));
+    if (not art_resolver) {
+        kWarning("Could not load album art source");
+        return;
+    }
+}
+
+void MediaArtDownloader::Private::setup_soup_session() {
+    session = wrap(soup_session_async_new_with_options(
+                       SOUP_SESSION_SSL_USE_SYSTEM_CA_FILE, TRUE,
+                       SOUP_SESSION_SSL_STRICT, TRUE,
+                       SOUP_SESSION_USER_AGENT, "Ubuntu-Media-Scanner/1.0",
+                       NULL));
+    soup_session_add_feature_by_type(
+        session.get(), SOUP_TYPE_PROXY_RESOLVER_DEFAULT);
+}
+
+void MediaArtDownloader::Private::add_job(ResolveDataPtr data)
+{
+    jobs.push_back(data);
+}
+
+void MediaArtDownloader::Private::complete(ResolveData *data)
+{
+    for (auto it = jobs.begin(); it != jobs.end(); it++) {
+        if (it->get() == data) {
+            jobs.erase(it);
+            break;
+        }
+    }
+    if (is_idle())
+        g_cond_signal(&cond);
+}
+
+void MediaArtDownloader::resolve(const std::string &artist,
+                                 const std::string &album) {
+    // If we couldn't configure either of these objects, then we won't
+    // have any luck resolving the album art.
+    if (not d->art_resolver || not d->session)
+        return;
+
+    kInfo("Resolving art for {1} - {2}") % artist % album;
+
+    if (d->cache.has_art(artist, album)) {
+        return;
+    }
+
+    g_mutex_lock(&d->mutex);
+
+    /* First check whether there is any queued requests for this art */
+    for (auto it = d->jobs.begin(); it != d->jobs.end(); it++) {
+        if ((*it)->artist == artist && (*it)->album == album) {
+            kInfo("Art for {1} - {2} already being resolved") % artist % album;
+            g_mutex_unlock(&d->mutex);
+            return;
+        }
+    }
+
+
+    const Private::ResolveDataPtr data(
+        new Private::ResolveData(d, artist, album));
+    d->add_job(data);
+
+    Wrapper<GrlMedia> media = take(grl_media_audio_new());
+    grl_media_audio_set_artist(GRL_MEDIA_AUDIO(media.get()), artist.c_str());
+    grl_media_audio_set_album(GRL_MEDIA_AUDIO(media.get()), album.c_str());
+    Wrapper<GList> resolve_keys = take(grl_metadata_key_list_new(
+        GRL_METADATA_KEY_THUMBNAIL,
+        GRL_METADATA_KEY_INVALID));
+    Wrapper<GrlOperationOptions> options =
+        take(grl_operation_options_new(nullptr));
+
+    data->operation_id = grl_source_resolve(
+        d->art_resolver.get(), media.get(), resolve_keys,
+        options.get(), &Private::OnResolveThumbnail, data.get());
+    g_mutex_unlock(&d->mutex);
+}
+
+void MediaArtDownloader::Private::OnResolveThumbnail(
+    GrlSource *source, unsigned operation_id, GrlMedia *media,
+    void *user_data, const GError *error) {
+    ResolveData *data = static_cast<ResolveData*>(user_data);
+    Private *const d = data->d;
+
+    g_mutex_lock(&d->mutex);
+    data->operation_id = 0;
+    if (error != NULL) {
+        kWarning("Error determining album art for {1} - {2}: {3}")
+            % data->artist % data->album % error->message;
+        d->complete(data);
+        g_mutex_unlock(&d->mutex);
+        return;
+    }
+
+    const char *thumbnail = grl_media_get_thumbnail(media);
+    if (thumbnail == NULL) {
+        kInfo("No thumbnail found for {1} - {2}") % data->artist % data->album;
+        d->complete(data);
+        g_mutex_unlock(&d->mutex);
+        return;
+    }
+
+    kInfo("Thumbnail for {1} - {2}: {3}") % data->artist % data->album % thumbnail;
+
+    data->message = take(soup_message_new("GET", thumbnail));
+    // queue_message() takes ownership of the message passed to it.
+    soup_session_queue_message(d->session.get(), data->message.dup(),
+                               &OnResponseReceived, data);
+    g_mutex_unlock(&d->mutex);
+}
+
+void MediaArtDownloader::Private::OnResponseReceived(
+    SoupSession *session, SoupMessage *message, void *user_data) {
+    // If we have been cancelled, then our data pointer has been
+    // cleaned up.
+    if (message->status_code == SOUP_STATUS_CANCELLED)
+        return;
+
+    ResolveData *data = static_cast<ResolveData*>(user_data);
+    Private *const d = data->d;
+
+    g_mutex_lock(&d->mutex);
+    data->message = NULL;
+
+    if (message->status_code != SOUP_STATUS_OK) {
+        kWarning("Failed to retrieve album art for {1} - {2}")
+            % data->artist % data->album;
+        d->complete(data);
+        g_mutex_unlock(&d->mutex);
+        return;
+    }
+
+    std::string content_type = soup_message_headers_get_content_type(
+        message->response_headers, NULL);
+    if (content_type != "image/jpeg") {
+        kWarning("Expected album art for {1} - {2} to be image/jpeg, but got {3}")
+            % data->artist % data->album % content_type;
+        d->complete(data);
+        g_mutex_unlock(&d->mutex);
+        return;
+    }
+
+    kInfo("Storing album art for {1} - {2}") % data->artist % data->album;
+    d->cache.add_art(
+        data->artist, data->album,
+        message->response_body->data, message->response_body->length);
+    d->complete(data);
+    g_mutex_unlock(&d->mutex);
+}
+
+bool MediaArtDownloader::is_idle() const {
+    return d->is_idle();
+}
+
+void MediaArtDownloader::WaitForFinished() {
+    g_mutex_lock(&d->mutex);
+    kInfo("Waiting for pending album art downloads.");
+    while (not is_idle()) {
+        g_cond_wait(&d->cond, &d->mutex);
+    }
+    kInfo("Album art downloads complete.");
+    g_mutex_unlock(&d->mutex);
+}
+
+void MediaArtDownloader::Private::cancel() {
+    g_mutex_lock(&mutex);
+
+    for (const auto &data: jobs) {
+        if (data->operation_id != 0) {
+            grl_operation_cancel(data->operation_id);
+        }
+        data->operation_id = 0;
+        if (data->message) {
+            soup_session_cancel_message(
+                session.get(), data->message.get(), SOUP_STATUS_CANCELLED);
+        }
+        data->message = NULL;
+    }
+
+    g_mutex_unlock(&mutex);
+}
+
+}

=== added file 'src/mediascanner/mediaartdownloader.h'
--- src/mediascanner/mediaartdownloader.h	1970-01-01 00:00:00 +0000
+++ src/mediascanner/mediaartdownloader.h	2013-10-09 07:38:10 +0000
@@ -0,0 +1,50 @@
+/*
+ * This file is part of the Ubuntu TV Media Scanner
+ * Copyright (C) 2012-2013 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License version 3 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Contact: Jim Hodapp <[e-mail]>
+ * Authored by: James Henstridge <[e-mail]>
+ */
+
+#ifndef MEDIAARTDOWNLOADER_H
+#define MEDIAARTDOWNLOADER_H
+
+#include <string>
+
+namespace mediascanner {
+
+class MediaArtDownloader {
+    class Private;
+    friend class Private;
+private:
+
+public:
+    MediaArtDownloader();
+    ~MediaArtDownloader();
+
+    static void LoadGriloPlugin();
+
+    void resolve(const std::string &artist, const std::string &album);
+
+    bool is_idle() const;
+    void WaitForFinished();
+
+private:
+    Private *const d;
+};
+
+}
+
+#endif

=== modified file 'src/mediascanner/mediaindex.cpp'
--- src/mediascanner/mediaindex.cpp	2013-10-02 12:27:35 +0000
+++ src/mediascanner/mediaindex.cpp	2013-10-09 07:38:10 +0000
@@ -60,6 +60,7 @@
 #include "mediascanner/filter.h"
 #include "mediascanner/locale.h"
 #include "mediascanner/logging.h"
+#include "mediascanner/mediaartcache.h"
 #include "mediascanner/mediaroot.h"
 #include "mediascanner/mediautils.h"
 #include "mediascanner/glibutils.h"
@@ -182,6 +183,8 @@
     typedef std::map<MediaRoot, Lucene::IndexReaderPtr> MediaReaderMap;
     MediaReaderMap media_readers_;
 
+    MediaArtCache art_cache_;
+
 private:
     Lucene::IndexReaderPtr index_reader_;
     Lucene::IndexSearcherPtr searcher_;
@@ -644,6 +647,21 @@
         }
     }
 
+    // If we have an artist and album name, check to see if there is
+    // art in the cache.
+    if (not metadata.first(schema::kArtist).empty() &&
+        not metadata.first(schema::kAlbum).empty()) {
+        std::string artist = FromUnicode(metadata.first(schema::kArtist));
+        std::string album = FromUnicode(metadata.first(schema::kAlbum));
+        std::string cover = d->art_cache_.get_art_uri(artist, album);
+
+        g_warning("Found cover art: %s", cover.c_str());
+        if (not cover.empty()) {
+            metadata.add_single(
+                std::make_pair(schema::kCover, ToUnicode(cover)));
+        }
+    }
+
     return metadata;
 }
 
@@ -702,9 +720,11 @@
 
 class MediaIndex::Collector : public Lucene::Collector {
 public:
-    Collector(const ItemVistor &visit_item, Lucene::SearcherPtr searcher,
-              Lucene::QueryPtr query, int32_t offset, int32_t limit)
-        : visit_item_(visit_item)
+    Collector(MediaIndex *index, const ItemVistor &visit_item,
+              Lucene::SearcherPtr searcher, Lucene::QueryPtr query,
+              int32_t offset, int32_t limit)
+        : index_(index)
+        , visit_item_(visit_item)
         , searcher_(searcher)
         , query_(query)
         , offset_(offset)
@@ -729,7 +749,7 @@
         Lucene::DocumentPtr document = reader_->document(doc);
 
         if (document)
-            visit_item_(ExtractProperties(document), limit_);
+            visit_item_(index_->ExtractProperties(document), limit_);
 
         if (limit_ > 0)
             --limit_;
@@ -744,6 +764,7 @@
     }
 
 private:
+    MediaIndex *const index_;
     ItemVistor visit_item_;
     Lucene::IndexReaderPtr reader_;
     Lucene::SearcherPtr searcher_;
@@ -837,7 +858,8 @@
     if (limit < 0) {
         /* If no limit is specified, return results unsorted */
         const Lucene::CollectorPtr media_collector =
-            newLucene<Collector>(visit_item, searcher, query, offset, limit);
+            newLucene<Collector>(this, visit_item, searcher, query,
+                                 offset, limit);
         searcher->search(query, media_collector);
     } else {
         const auto collector = Lucene::TopScoreDocCollector::create(

=== modified file 'src/mediascanner/mediaindex.h'
--- src/mediascanner/mediaindex.h	2013-09-10 14:02:38 +0000
+++ src/mediascanner/mediaindex.h	2013-10-09 07:38:10 +0000
@@ -208,9 +208,9 @@
 
     static Property::Set GetFields(Lucene::DocumentPtr document);
 
-    static MediaInfo ExtractProperties(Lucene::DocumentPtr document);
-    static MediaInfo ExtractProperties(Lucene::DocumentPtr document,
-                                       const Property::Set &fields);
+    MediaInfo ExtractProperties(Lucene::DocumentPtr document);
+    MediaInfo ExtractProperties(Lucene::DocumentPtr document,
+                                const Property::Set &fields);
 
     bool Lookup(const std::wstring &url, const Property &key,
                 Property::Value *value);

=== modified file 'src/mediascanner/metadataresolver.cpp'
--- src/mediascanner/metadataresolver.cpp	2013-09-17 10:48:31 +0000
+++ src/mediascanner/metadataresolver.cpp	2013-10-09 07:38:10 +0000
@@ -277,6 +277,9 @@
     const size_t removals = pending_pushes_.erase(push_id);
     BOOST_ASSERT(removals == 1);
 
+    if (is_idle())
+        g_cond_signal(&cond_);
+
     g_mutex_unlock(&mutex_);
 }
 

=== modified file 'src/mediascanner/propertyschema.cpp'
--- src/mediascanner/propertyschema.cpp	2013-09-09 12:17:56 +0000
+++ src/mediascanner/propertyschema.cpp	2013-10-09 07:38:10 +0000
@@ -220,6 +220,21 @@
                       merge_nothing) {
 }
 
+static bool get_network_available(const GstDiscovererInfo *media) {
+    GNetworkMonitor *monitor = g_network_monitor_get_default();
+    return g_network_monitor_get_network_available(monitor);
+}
+
+NetworkAvailable::NetworkAvailable()
+    : NumericProperty(L"network-available",
+                      define("lucene-network-available",
+                             N_("Network available (Lucene)"),
+                             N_("...")),
+                      Property::Category::Generic,
+                      Property::MergeReplace,
+                      bind_attr(get_network_available)) {
+}
+
 // File Properties /////////////////////////////////////////////////////////////
 
 Url::Url()
@@ -1320,6 +1335,7 @@
 const MaximumVideoBitRate kMaximumVideoBitRate;
 const MeteringMode kMeteringMode;
 const MimeType kMimeType;
+const NetworkAvailable kNetworkAvailable;
 const Organization kOrganization;
 const OriginalTitle kOriginalTitle;
 const Performer kPerformer;

=== modified file 'src/mediascanner/propertyschema.h'
--- src/mediascanner/propertyschema.h	2013-08-06 06:36:26 +0000
+++ src/mediascanner/propertyschema.h	2013-10-09 07:38:10 +0000
@@ -108,6 +108,9 @@
 struct Favourite : public NumericProperty<bool> { Favourite(); }
 extern const kFavourite;
 
+struct NetworkAvailable : public NumericProperty<bool> { NetworkAvailable(); }
+extern const kNetworkAvailable;
+
 // File Properties /////////////////////////////////////////////////////////////
 
 // Local URL of the media, taken from File System

=== modified file 'tests/auto/filesystemscannertest.cpp'
--- tests/auto/filesystemscannertest.cpp	2013-09-11 09:49:12 +0000
+++ tests/auto/filesystemscannertest.cpp	2013-10-09 07:38:10 +0000
@@ -341,6 +341,7 @@
 
     AddGlobalTestEnvironment(new LuceneEnvironment(kMediaIndexPath));
     AddGlobalTestEnvironment(new TheMovieDbEnvironment(REQUEST_FILE));
+    AddGlobalTestEnvironment(new LastFmEnvironment(REQUEST_FILE));
 }
 
 } // namespace filesystemscannertest

=== modified file 'tests/auto/loggingtest.cpp'
--- tests/auto/loggingtest.cpp	2013-09-11 09:49:12 +0000
+++ tests/auto/loggingtest.cpp	2013-10-09 07:38:10 +0000
@@ -205,7 +205,7 @@
 } // namespace mediascanner
 
 int main(int argc, char *argv[]) {
-    ::setenv("MEDIASCANNER_DEBUG", 0, true);
+    ::setenv("MEDIASCANNER_DEBUG", "0", true);
     mediascanner::InitTests(&argc, argv);
     return RUN_ALL_TESTS();
 }

=== modified file 'tests/auto/propertytest.cpp'
--- tests/auto/propertytest.cpp	2013-08-23 07:49:06 +0000
+++ tests/auto/propertytest.cpp	2013-10-09 07:38:10 +0000
@@ -71,7 +71,8 @@
                                         &schema::kCertification,
                                         &schema::kComment,
                                         &schema::kKeyword,
-                                        &schema::kLanguage));
+                                        &schema::kLanguage,
+                                        &schema::kNetworkAvailable));
 
 INSTANTIATE_TEST_CASE_P(FileProperties, PropertyTest,
                         testing::Values(&schema::kMimeType,