Merge lp:~stolowski/unity-scope-click/departments into lp:unity-scope-click/devel

[Paweł Stołowski]

Add support for departments and highlights.

[dobey (dobey)]

426 +std::map<std::string, std::string> Index::build_headers(const std::string& language)
438 + {"Accept-Language", language},
486 + QSharedPointer<click::web::Response> response(client->call(
487 + get_base_url() + click::BOOTSTRAP_PATH, "GET", false, build_headers("en"), "", params)); //TODO: language

You don't need to do all this work to pass the language around or set Accept-Language manually. It is already added automatically in web::Client::cal().

review: Needs Fixing

[dobey (dobey)]

Can you also add a commit with --fixes=lp:1209224 to link that bug?

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:254
http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-ci/124/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-amd64-ci/99/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-armhf-ci/98/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-i386-ci/98/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-team-unity-scope-click-devel-ci/124/rebuild

review: Needs Fixing (continuous-integration)

[dobey (dobey)]

This conflicts with the changes in https://code.launchpad.net/~stolowski/unity-scope-click/update-for-api-0.5.0/+merge/223549.

review: Needs Fixing

[Paweł Stołowski (stolowski)]

That should have been fixed now.

[lp:~stolowski/unity-scope-click/departments]

257. By Paweł Stołowski on 2014-06-18

Merged devel.

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:255
http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-ci/128/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-amd64-ci/103/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-armhf-ci/102/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-i386-ci/102/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-team-unity-scope-click-devel-ci/128/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:257
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~stolowski/unity-scope-click/departments/+merge/223629/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-ci/129/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-amd64-ci/104/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-armhf-ci/103/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-i386-ci/103/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-team-unity-scope-click-devel-ci/129/rebuild

review: Needs Fixing (continuous-integration)

[Alejandro J. Cura (alecu)]

On one hand, I like this branch.
I've run it on my desktop against testing, and it's showing the highlights as expected, and seems to be working fine otherwise, so kudos for that.

On the other, I'm a bit disappointed with the length, and a bit more that the unit testing for it seems lacking. Here's a list of what I've found is missing:
- tests for methods in departments.cpp (eg: from_json_root_node)
- tests for failures in departments json parsing
- tests for DepartmentLookup::get_department_info
- tests for parsing Highlights, and for failing to parse them
- tests for Index::bootstrap and Index::departments
- test_query has fixes for existing "search" tests, but it has no new tests for Highlights, Departments

Also, I think Index::bootstrap() could be rewritten as just a call to Index::departments(get_base_url() + click::BOOTSTRAP_PATH, .....)

Before approving I'd like to have jenkins build a proper package to test on the device.
After that, and since we are so close to feature freeze, I'll propose to land this, and to open a bug for the missing tests.

review: Needs Fixing

[lp:~stolowski/unity-scope-click/departments]

258. By Paweł Stołowski on 2014-06-23

Reuse code between bootstrap and departments call.

259. By Paweł Stołowski on 2014-06-23

Cleanups. Check department href in tests.

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:259
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~stolowski/unity-scope-click/departments/+merge/223629/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-ci/139/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-amd64-ci/114/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-armhf-ci/113/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-i386-ci/113/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-team-unity-scope-click-devel-ci/139/rebuild

review: Needs Fixing (continuous-integration)

[lp:~stolowski/unity-scope-click/departments]

260. By Paweł Stołowski on 2014-06-23

Test get_department_info().

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:260
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~stolowski/unity-scope-click/departments/+merge/223629/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-ci/140/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-amd64-ci/115/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-armhf-ci/114/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-i386-ci/114/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-team-unity-scope-click-devel-ci/140/rebuild

review: Needs Fixing (continuous-integration)

[lp:~stolowski/unity-scope-click/departments]

261. By Paweł Stołowski on 2014-06-23

Added tests for broken departments json.

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:261
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~stolowski/unity-scope-click/departments/+merge/223629/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-ci/141/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-amd64-ci/116/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-armhf-ci/115/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-team-unity-scope-click-devel-utopic-i386-ci/115/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-team-unity-scope-click-devel-ci/141/rebuild

review: Needs Fixing (continuous-integration)

[lp:~stolowski/unity-scope-click/departments]

262. By Paweł Stołowski on 2014-06-23

Don't pass departments with search.

263. By Paweł Stołowski on 2014-06-23

Test for invalid highlights. Cleanups.

264. By Paweł Stołowski on 2014-06-23

Reuse push_package().

265. By Paweł Stołowski on 2014-06-23

Fixed comment.

266. By Paweł Stołowski on 2014-06-23

Test bootstrap call.

267. By Paweł Stołowski on 2014-06-23

Merged devel.

Unmerged revisions

=== modified file 'libclickscope/click/CMakeLists.txt'
--- libclickscope/click/CMakeLists.txt	2014-06-20 20:00:53 +0000
+++ libclickscope/click/CMakeLists.txt	2014-06-23 10:26:18 +0000
@@ -12,6 +12,9 @@
 add_library(${SCOPE_LIB_NAME} STATIC
   configuration.cpp
   download-manager.cpp
+  department-lookup.cpp
+  departments.cpp
+  highlights.cpp
   index.cpp
   interface.cpp
   key_file_locator.cpp

=== added file 'libclickscope/click/department-lookup.cpp'
--- libclickscope/click/department-lookup.cpp	1970-01-01 00:00:00 +0000
+++ libclickscope/click/department-lookup.cpp	2014-06-23 10:26:18 +0000
@@ -0,0 +1,84 @@
+/*
+ * Copyright (C) 2014 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * In addition, as a special exception, the copyright holders give
+ * permission to link the code of portions of this program with the
+ * OpenSSL library under certain conditions as described in each
+ * individual source file, and distribute linked combinations
+ * including the two.
+ * You must obey the GNU General Public License in all respects
+ * for all of the code used other than OpenSSL.  If you modify
+ * file(s) with this exception, you may extend this exception to your
+ * version of the file(s), but you are not obligated to do so.  If you
+ * do not wish to do so, delete this exception statement from your
+ * version.  If you delete this exception statement from all source
+ * files in the program, then also delete it here.
+ */
+
+#include "department-lookup.h"
+
+namespace click
+{
+
+DepartmentLookup::DepartmentLookup()
+{
+}
+
+void DepartmentLookup::rebuild(const Department::SPtr& dept)
+{
+    departments[dept->id()] = dept;
+    for (auto const& subdep: dept->sub_departments())
+    {
+        parent_lut[subdep->id()] = dept;
+        rebuild(subdep);
+    }
+}
+
+void DepartmentLookup::rebuild(const std::list<Department::SPtr>& root_departments)
+{
+    parent_lut.clear();
+    departments.clear();
+    for (auto const& dep: root_departments)
+    {
+        rebuild(dep);
+    }
+}
+
+Department::SPtr DepartmentLookup::get_parent(const std::string& department_id) const
+{
+    auto it = parent_lut.find(department_id);
+    if (it != parent_lut.end())
+    {
+        return it->second;
+    }
+    return Department::SPtr(nullptr);
+}
+
+Department::SPtr DepartmentLookup::get_department_info(const std::string& department_id) const
+{
+    auto it = departments.find(department_id);
+    if (it != departments.end())
+    {
+        return it->second;
+    }
+    return nullptr;
+}
+
+int DepartmentLookup::size() const
+{
+    return parent_lut.size();
+}
+
+}

=== added file 'libclickscope/click/department-lookup.h'
--- libclickscope/click/department-lookup.h	1970-01-01 00:00:00 +0000
+++ libclickscope/click/department-lookup.h	2014-06-23 10:26:18 +0000
@@ -0,0 +1,58 @@
+/*
+ * Copyright (C) 2014 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * In addition, as a special exception, the copyright holders give
+ * permission to link the code of portions of this program with the
+ * OpenSSL library under certain conditions as described in each
+ * individual source file, and distribute linked combinations
+ * including the two.
+ * You must obey the GNU General Public License in all respects
+ * for all of the code used other than OpenSSL.  If you modify
+ * file(s) with this exception, you may extend this exception to your
+ * version of the file(s), but you are not obligated to do so.  If you
+ * do not wish to do so, delete this exception statement from your
+ * version.  If you delete this exception statement from all source
+ * files in the program, then also delete it here.
+ */
+
+#ifndef CLICK_DEPARTMENT_LOOKUP_H
+#define CLICK_DEPARTMENT_LOOKUP_H
+
+#include "departments.h"
+#include <string>
+#include <memory>
+#include <map>
+
+namespace click
+{
+
+class DepartmentLookup
+{
+    public:
+        DepartmentLookup();
+        void rebuild(const std::list<Department::SPtr>& root_departments);
+        Department::SPtr get_parent(const std::string& department_id) const;
+        Department::SPtr get_department_info(const std::string& department_id) const;
+        int size() const;
+
+    private:
+        void rebuild(const Department::SPtr& dept);
+        std::map<std::string, Department::SPtr> parent_lut;
+        std::map<std::string, Department::SPtr> departments;
+};
+
+}
+
+#endif

=== added file 'libclickscope/click/departments.cpp'
--- libclickscope/click/departments.cpp	1970-01-01 00:00:00 +0000
+++ libclickscope/click/departments.cpp	2014-06-23 10:26:18 +0000
@@ -0,0 +1,173 @@
+/*
+ * Copyright (C) 2014 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * In addition, as a special exception, the copyright holders give
+ * permission to link the code of portions of this program with the
+ * OpenSSL library under certain conditions as described in each
+ * individual source file, and distribute linked combinations
+ * including the two.
+ * You must obey the GNU General Public License in all respects
+ * for all of the code used other than OpenSSL.  If you modify
+ * file(s) with this exception, you may extend this exception to your
+ * version of the file(s), but you are not obligated to do so.  If you
+ * do not wish to do so, delete this exception statement from your
+ * version.  If you delete this exception statement from all source
+ * files in the program, then also delete it here.
+ */
+
+#include "departments.h"
+#include <iostream>
+
+namespace click
+{
+
+Department::Department(const std::string& id, const std::string &name, const std::string& href, bool has_children)
+    : id_(id),
+      name_(name),
+      href_(href),
+      has_children_flag_(has_children)
+{
+}
+
+std::string Department::id() const
+{
+    return id_;
+}
+
+std::string Department::name() const
+{
+    return name_;
+}
+
+std::string Department::href() const
+{
+    return href_;
+}
+
+bool Department::has_children_flag() const
+{
+    return has_children_flag_;
+}
+
+void Department::set_subdepartments(const std::list<Department::SPtr>& deps)
+{
+    sub_departments_ = deps;
+}
+
+std::list<Department::SPtr> Department::sub_departments() const
+{
+    return sub_departments_;
+}
+
+Json::Value Department::check_mandatory_attribute(const Json::Value& item, const std::string& name, Json::ValueType valtype)
+{
+    if (!item.isMember(name)) {
+        throw std::runtime_error("Missing '" + name + "' node");
+    }
+    auto const val = item[name];
+    if (val.type() != valtype) {
+        throw std::runtime_error("Invalid type of '" + name + "' node");
+    }
+    return val;
+}
+
+std::list<Department::SPtr> Department::from_json_node(const Json::Value& node)
+{
+    std::list<Department::SPtr> deps;
+
+    for (uint i = 0; i < node.size(); i++)
+    {
+        try
+        {
+            auto const item = node[i];
+
+            auto name = check_mandatory_attribute(item, Department::JsonKeys::name, Json::ValueType::stringValue).asString();
+            const bool has_children = item.isMember(Department::JsonKeys::has_children) ? item[Department::JsonKeys::has_children].asBool() : false;
+
+            auto const links = check_mandatory_attribute(item, Department::JsonKeys::links, Json::ValueType::objectValue);
+            auto const self = check_mandatory_attribute(links, Department::JsonKeys::self, Json::ValueType::objectValue);
+            auto const href = check_mandatory_attribute(self, Department::JsonKeys::href, Json::ValueType::stringValue).asString();
+
+            auto dep = std::make_shared<Department>(name, name, href, has_children); //FIXME: id
+            if (item.isObject() && item.isMember(Department::JsonKeys::embedded))
+            {
+                auto const emb = item[Department::JsonKeys::embedded];
+                if (emb.isObject() && emb.isMember(Department::JsonKeys::department))
+                {
+                    auto const ditem = emb[Department::JsonKeys::department];
+                    auto const subdeps = from_json_node(ditem);
+                    dep->set_subdepartments(subdeps);
+                }
+            }
+            deps.push_back(dep);
+        }
+        catch (const std::runtime_error& e)
+        {
+            std::cerr << "Invalid department #" << i << std::endl;
+        }
+    }
+
+    return deps;
+}
+
+std::list<Department::SPtr> Department::from_json_root_node(const Json::Value& root)
+{
+    if (root.isObject() && root.isMember(Department::JsonKeys::embedded))
+    {
+        auto const emb = root[Department::JsonKeys::embedded];
+        if (emb.isObject() && emb.isMember(Department::JsonKeys::department))
+        {
+            auto const ditem = emb[Department::JsonKeys::department];
+            return from_json_node(ditem);
+        }
+    }
+
+    return std::list<Department::SPtr>();
+}
+
+std::list<Department::SPtr> Department::from_json(const std::string& json)
+{
+    Json::Reader reader;
+    Json::Value root;
+
+    try
+    {
+        if (!reader.parse(json, root)) {
+            throw std::runtime_error(reader.getFormattedErrorMessages());
+        }
+
+        if (root.isObject() && root.isMember(Department::JsonKeys::embedded))
+        {
+            auto const emb = root[Department::JsonKeys::embedded];
+            if (emb.isObject() && emb.isMember(Department::JsonKeys::department))
+            {
+                auto const ditem = emb[Department::JsonKeys::department];
+                return from_json_node(ditem);
+            }
+        }
+    }
+    catch (const std::exception& e)
+    {
+        std::cerr << "Error parsing departments: " << e.what() << std::endl;
+    }
+    catch (...)
+    {
+        std::cerr << "Unknown error when parsing departments" << std::endl;
+    }
+
+    return std::list<Department::SPtr>();
+}
+
+}

=== added file 'libclickscope/click/departments.h'
--- libclickscope/click/departments.h	1970-01-01 00:00:00 +0000
+++ libclickscope/click/departments.h	2014-06-23 10:26:18 +0000
@@ -0,0 +1,83 @@
+/*
+ * Copyright (C) 2014 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * In addition, as a special exception, the copyright holders give
+ * permission to link the code of portions of this program with the
+ * OpenSSL library under certain conditions as described in each
+ * individual source file, and distribute linked combinations
+ * including the two.
+ * You must obey the GNU General Public License in all respects
+ * for all of the code used other than OpenSSL.  If you modify
+ * file(s) with this exception, you may extend this exception to your
+ * version of the file(s), but you are not obligated to do so.  If you
+ * do not wish to do so, delete this exception statement from your
+ * version.  If you delete this exception statement from all source
+ * files in the program, then also delete it here.
+ */
+
+#ifndef CLICK_DEPARTMENTS_H
+#define CLICK_DEPARTMENTS_H
+
+#include <string>
+#include <list>
+#include <memory>
+#include <json/json.h>
+
+namespace click
+{
+
+class Department
+{
+    public:
+        typedef std::shared_ptr<Department> SPtr;
+        typedef std::shared_ptr<Department const> SCPtr;
+
+        struct JsonKeys
+        {
+            JsonKeys() = delete;
+            constexpr static const char* name {"name"};
+            constexpr static const char* embedded {"_embedded"};
+            constexpr static const char* department {"clickindex:department"};
+            constexpr static const char* has_children {"has_children"};
+            constexpr static const char* links {"_links"};
+            constexpr static const char* self {"self"};
+            constexpr static const char* href {"href"};
+        };
+
+        Department(const std::string &id, const std::string &name, const std::string& href, bool has_children);
+        std::string id() const;
+        std::string name() const;
+        std::string href() const;
+        bool has_children_flag() const;
+        void set_subdepartments(const std::list<Department::SPtr>& deps);
+        std::list<Department::SPtr> sub_departments() const;
+        static std::list<Department::SPtr> from_json(const std::string& json);
+        static std::list<Department::SPtr> from_json_root_node(const Json::Value& val);
+
+    private:
+        static std::list<Department::SPtr> from_json_node(const Json::Value& val);
+        static Json::Value check_mandatory_attribute(const Json::Value& item, const std::string& name, Json::ValueType valtype);
+        std::string id_;
+        std::string name_;
+        std::string href_;
+        bool has_children_flag_;
+        std::list<Department::SPtr> sub_departments_;
+};
+
+typedef std::list<Department::SPtr> DepartmentList;
+
+}
+
+#endif

=== added file 'libclickscope/click/highlights.cpp'
--- libclickscope/click/highlights.cpp	1970-01-01 00:00:00 +0000
+++ libclickscope/click/highlights.cpp	2014-06-23 10:26:18 +0000
@@ -0,0 +1,95 @@
+/*
+ * Copyright (C) 2014 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * In addition, as a special exception, the copyright holders give
+ * permission to link the code of portions of this program with the
+ * OpenSSL library under certain conditions as described in each
+ * individual source file, and distribute linked combinations
+ * including the two.
+ * You must obey the GNU General Public License in all respects
+ * for all of the code used other than OpenSSL.  If you modify
+ * file(s) with this exception, you may extend this exception to your
+ * version of the file(s), but you are not obligated to do so.  If you
+ * do not wish to do so, delete this exception statement from your
+ * version.  If you delete this exception statement from all source
+ * files in the program, then also delete it here.
+ */
+
+#include "highlights.h"
+#include <iostream>
+
+namespace click
+{
+
+Highlight::Highlight(const std::string& name)
+    : name_(name)
+{
+}
+
+Highlight::Highlight(const std::string& name, const Packages& pkgs)
+    : name_(name),
+      packages_(pkgs)
+{
+}
+
+void Highlight::add_package(const Package& pkg)
+{
+    packages_.push_back(pkg);
+}
+
+std::string Highlight::name() const
+{
+    return name_;
+}
+
+Packages Highlight::packages() const
+{
+    return packages_;
+}
+
+std::list<Highlight> Highlight::from_json_node(const Json::Value& node)
+{
+    std::list<Highlight> highlights;
+
+    for (uint i = 0; i < node.size(); i++)
+    {
+        auto const item = node[i];
+        if (item.isObject() && item.isMember(Highlight::JsonKeys::name))
+        {
+            auto name = item[Highlight::JsonKeys::name].asString();
+            auto pkgs = package_list_from_json_node(item);
+            highlights.push_back(Highlight(name, pkgs));
+        }
+    }
+
+    return highlights;
+}
+
+std::list<Highlight> Highlight::from_json_root_node(const Json::Value& root)
+{
+    if (root.isObject() && root.isMember(Highlight::JsonKeys::embedded))
+    {
+        auto const emb = root[Highlight::JsonKeys::embedded];
+        if (emb.isObject() && emb.isMember(Highlight::JsonKeys::highlight))
+        {
+            auto const hl = emb[Highlight::JsonKeys::highlight];
+            return from_json_node(hl);
+        }
+    }
+
+    return std::list<Highlight>();
+}
+
+}

=== added file 'libclickscope/click/highlights.h'
--- libclickscope/click/highlights.h	1970-01-01 00:00:00 +0000
+++ libclickscope/click/highlights.h	2014-06-23 10:26:18 +0000
@@ -0,0 +1,72 @@
+/*
+ * Copyright (C) 2014 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * In addition, as a special exception, the copyright holders give
+ * permission to link the code of portions of this program with the
+ * OpenSSL library under certain conditions as described in each
+ * individual source file, and distribute linked combinations
+ * including the two.
+ * You must obey the GNU General Public License in all respects
+ * for all of the code used other than OpenSSL.  If you modify
+ * file(s) with this exception, you may extend this exception to your
+ * version of the file(s), but you are not obligated to do so.  If you
+ * do not wish to do so, delete this exception statement from your
+ * version.  If you delete this exception statement from all source
+ * files in the program, then also delete it here.
+ */
+
+#ifndef CLICK_HIGHLIGHTS_H
+#define CLICK_HIGHLIGHTS_H
+
+#include <string>
+#include <list>
+#include <json/json.h>
+#include <click/package.h>
+
+namespace click
+{
+
+class Highlight
+{
+public:
+    struct JsonKeys
+    {
+        JsonKeys() = delete;
+        constexpr static const char* name {"name"};
+        constexpr static const char* embedded {"_embedded"};
+        constexpr static const char* highlight {"clickindex:highlight"};
+    };
+
+    Highlight(const std::string& name);
+    Highlight(const std::string& name, const Packages& pkgs);
+    void add_package(const Package& pkg);
+
+    std::string name() const;
+    Packages packages() const;
+
+    static std::list<Highlight> from_json_root_node(const Json::Value& val);
+
+private:
+    static std::list<Highlight> from_json_node(const Json::Value& val);
+
+    std::string name_;
+    Packages packages_;
+};
+
+typedef std::list<Highlight> HighlightList;
+
+}
+
+#endif

=== modified file 'libclickscope/click/index.cpp'
--- libclickscope/click/index.cpp	2014-06-12 21:05:25 +0000
+++ libclickscope/click/index.cpp	2014-06-23 10:26:18 +0000
@@ -88,11 +88,15 @@
 
 }
 
-std::string Index::build_index_query(const std::string& query)
+std::string Index::build_index_query(const std::string& query, const std::string& department)
 {
     std::stringstream result;
 
     result << query;
+    if (!department.empty()) {
+        result << ",department:" << department;
+    }
+
     return result.str();
 }
 
@@ -113,7 +117,7 @@
 click::web::Cancellable Index::search (const std::string& query, std::function<void(click::Packages)> callback)
 {
     click::web::CallParams params;
-    const std::string built_query(build_index_query(query));
+    const std::string built_query(build_index_query(query, ""));
     params.add(click::QUERY_ARGNAME, built_query.c_str());
     QSharedPointer<click::web::Response> response(client->call(
         get_base_url() + click::SEARCH_PATH, "GET", false, build_headers(), "", params));
@@ -139,6 +143,40 @@
     return click::web::Cancellable(response);
 }
 
+click::web::Cancellable Index::bootstrap(std::function<void(const click::DepartmentList&, const click::HighlightList&, Error, int)> callback)
+{
+    return departments(get_base_url() + click::BOOTSTRAP_PATH, callback);
+}
+
+click::web::Cancellable Index::departments(const std::string& department_href, std::function<void(const DepartmentList&, const HighlightList&, Error, int)> callback)
+{
+    click::web::CallParams params;
+    QSharedPointer<click::web::Response> response(client->call(
+        department_href, "GET", false, build_headers(), "", params));
+
+    QObject::connect(response.data(), &click::web::Response::finished, [=](QString reply) {
+            qDebug() << "departments request finished";
+            Json::Reader reader;
+            Json::Value root;
+
+            click::DepartmentList depts;
+            click::HighlightList highlights;
+            if (reader.parse(reply.toUtf8().constData(), root)) {
+                depts = Department::from_json_root_node(root);
+                highlights = Highlight::from_json_root_node(root);
+            }
+            callback(depts, highlights, click::Index::Error::NoError, 0);
+        });
+    QObject::connect(response.data(), &click::web::Response::error, [=](QString /*description*/, int error_code) {
+            qWarning() << "departments call failed due to network error";
+            const click::DepartmentList depts;
+            const click::HighlightList highlights;
+            qDebug() << "departments: calling callback";
+            callback(depts, highlights, click::Index::Error::NetworkError, error_code);
+        });
+    return click::web::Cancellable(response);
+}
+
 click::web::Cancellable Index::get_details (const std::string& package_name, std::function<void(PackageDetails, click::Index::Error)> callback)
 {
     QSharedPointer<click::web::Response> response = client->call

=== modified file 'libclickscope/click/index.h'
--- libclickscope/click/index.h	2014-06-12 21:05:25 +0000
+++ libclickscope/click/index.h	2014-06-23 10:26:18 +0000
@@ -38,6 +38,8 @@
 #include <click/webclient.h>
 
 #include "package.h"
+#include <click/departments.h>
+#include <click/highlights.h>
 
 
 namespace click {
@@ -47,6 +49,7 @@
 const std::string SEARCH_BASE_URL_ENVVAR = "U1_SEARCH_BASE_URL";
 const std::string SEARCH_BASE_URL = "https://search.apps.ubuntu.com/";
 const std::string SEARCH_PATH = "api/v1/search";
+const std::string BOOTSTRAP_PATH = "api/v1";
 const std::string SUPPORTED_FRAMEWORKS = "framework:ubuntu-sdk-13.10";
 const std::string QUERY_ARGNAME = "q";
 const std::string ARCHITECTURE = "architecture:";
@@ -64,7 +67,7 @@
 protected:
     QSharedPointer<web::Client> client;
     QSharedPointer<Configuration> configuration;
-    virtual std::string build_index_query(const std::string& query);
+    virtual std::string build_index_query(const std::string& query, const std::string& department);
     virtual std::map<std::string, std::string> build_headers();
 
 public:
@@ -73,6 +76,8 @@
           const QSharedPointer<Configuration> configuration=QSharedPointer<Configuration>(new Configuration()));
     virtual click::web::Cancellable search (const std::string& query, std::function<void(Packages)> callback);
     virtual click::web::Cancellable get_details(const std::string& package_name, std::function<void(PackageDetails, Error)> callback);
+    virtual click::web::Cancellable bootstrap(std::function<void(const DepartmentList&, const HighlightList&, Error, int)> callback);
+    virtual click::web::Cancellable departments(const std::string& department_href, std::function<void(const DepartmentList&, const HighlightList&, Error, int)> callback);
     virtual ~Index();
 
     static std::string get_base_url ();

=== modified file 'libclickscope/click/package.h'
--- libclickscope/click/package.h	2014-06-13 22:42:02 +0000
+++ libclickscope/click/package.h	2014-06-23 10:26:18 +0000
@@ -34,6 +34,7 @@
 #include <string>
 #include <unordered_set>
 #include <vector>
+#include <functional>
 
 #include <json/json.h>
 

=== modified file 'libclickscope/click/webclient.cpp'
--- libclickscope/click/webclient.cpp	2014-05-23 18:45:01 +0000
+++ libclickscope/click/webclient.cpp	2014-06-23 10:26:18 +0000
@@ -176,7 +176,8 @@
 {
     auto message = reply->errorString() + QString(" (%1)").arg(network_error);
     qWarning() << "Network error:" << message << "\n" << reply->readAll();
-    emit error(message);
+    int error_code = reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();
+    emit error(message, error_code);
 }
 
 void click::web::Response::abort()

=== modified file 'libclickscope/click/webclient.h'
--- libclickscope/click/webclient.h	2014-05-23 18:45:01 +0000
+++ libclickscope/click/webclient.h	2014-06-23 10:26:18 +0000
@@ -84,7 +84,7 @@
 
 signals:
     void finished(QByteArray result);
-    void error(QString description);
+    void error(QString description, int error_code);
 
 private:
     QSharedPointer<click::network::Reply> reply;

=== modified file 'libclickscope/tests/CMakeLists.txt'
--- libclickscope/tests/CMakeLists.txt	2014-05-26 14:27:31 +0000
+++ libclickscope/tests/CMakeLists.txt	2014-06-23 10:26:18 +0000
@@ -28,6 +28,8 @@
   test_reviews.cpp
   test_smartconnect.cpp
   test_webclient.cpp
+  test_bootstrap.cpp
+  test_departments.cpp
 
   ${CMAKE_CURRENT_BINARY_DIR}/test_data.cpp
 )

=== modified file 'libclickscope/tests/fake_json.h'
--- libclickscope/tests/fake_json.h	2014-06-13 23:18:41 +0000
+++ libclickscope/tests/fake_json.h	2014-06-23 10:26:18 +0000
@@ -150,6 +150,277 @@
     }
 )foo";
 
+const std::string FAKE_JSON_BOOTSTRAP = R"(
+      {
+          "_embedded": {
+              "clickindex:department": [
+                {
+                    "has_children": false,
+                    "_links": {
+                        "self": {
+                            "href": "https://search.apps.staging.ubuntu.com/api/v1/departments/fake-subdepartment"}
+                    },
+                    "name": "Fake Subdepartment", "slug": "fake-subdepartment"}
+              ],
+              "clickindex:highlight": [
+              {
+                  "_embedded": {
+                      "clickindex:package": [
+                       {
+                            "publisher": "Awesome Widget Company",
+                            "name": "org.example.awesomelauncher",
+                            "title": "Awesome Launcher",
+                            "price": 1.99,
+                            "_links": {
+                                "self": {
+                                    "href": "https://search.apps.staging.ubuntu.com/api/v1/package/org.example.awesomelauncher"}
+                             },
+                             "icon": "http://example.org/media/org.example.awesomelauncher/icons/icon16.png"
+                        },
+                      {
+                          "publisher": "Awesome Widget Company",
+                          "name": "org.example.awesomewidget",
+                          "title": "Awesome Widget", "price": 1.99,
+                          "_links": {
+                              "self": {
+                                  "href": "https://search.apps.staging.ubuntu.com/api/v1/package/org.example.awesomewidget"
+                              }
+                           },
+                          "icon": "http://example.org/media/org.example.awesomewidget/icons/icon16.png"}
+                    ]
+                  },
+                  "_links": {
+                      "self": {
+                          "href": "https://search.apps.staging.ubuntu.com/api/v1/highlights/top-apps"
+                      }
+                  },
+                  "name": "Top Apps", "slug": "top-apps"
+              },
+              {
+                  "_embedded": {
+                      "clickindex:package": [
+                      {
+                          "publisher": "Awesome Widget Company",
+                          "name": "org.example.awesomelauncher",
+                          "title": "Awesome Launcher",
+                          "price": 1.99,
+                          "_links": {
+                              "self": {
+                                  "href": "https://search.apps.staging.ubuntu.com/api/v1/package/org.example.awesomelauncher"
+                              }
+                          },
+                          "icon": "http://example.org/media/org.example.awesomelauncher/icons/icon16.png"
+                      },
+                      {
+                          "publisher": "Awesome Widget Company",
+                          "name": "org.example.awesomewidget",
+                          "title": "Awesome Widget",
+                          "price": 1.99,
+                          "_links": {
+                              "self": {
+                                  "href": "https://search.apps.staging.ubuntu.com/api/v1/package/org.example.awesomewidget"
+                              }
+                          },
+                          "icon": "http://example.org/media/org.example.awesomewidget/icons/icon16.png"
+                      }
+                      ]
+                  },
+                  "_links": {
+                      "self": {
+                          "href": "https://search.apps.staging.ubuntu.com/api/v1/highlights/most-purchased"
+                      }
+                  },
+                  "name": "Most Purchased",
+                  "slug": "most-purchased"
+              },
+              {
+                  "_embedded": {
+                      "clickindex:package": [
+                      {
+                          "publisher": "Awesome Widget Company",
+                          "name": "org.example.awesomelauncher",
+                          "title": "Awesome Launcher",
+                          "price": 1.99,
+                          "_links": {
+                              "self": {
+                                  "href": "https://search.apps.staging.ubuntu.com/api/v1/package/org.example.awesomelauncher"
+                              }
+                          },
+                          "icon": "http://example.org/media/org.example.awesomelauncher/icons/icon16.png"
+                      },
+                      {
+                          "publisher": "Awesome Widget Company",
+                          "name": "org.example.awesomewidget",
+                          "title": "Awesome Widget",
+                          "price": 1.99,
+                          "_links": {
+                              "self": {
+                                  "href": "https://search.apps.staging.ubuntu.com/api/v1/package/org.example.awesomewidget"
+                              }
+                          },
+                          "icon": "http://example.org/media/org.example.awesomewidget/icons/icon16.png"
+                      }
+                      ]
+                  },
+                  "_links": {
+                      "self": {
+                          "href": "https://search.apps.staging.ubuntu.com/api/v1/highlights/new-releases"
+                      }
+                  },
+                  "name": "New Releases",
+                  "slug": "new-releases"
+              }
+            ]
+          }, "has_children": true,
+              "_links": {
+                  "curies": [
+                  {
+                      "href": "https://search.apps.staging.ubuntu.com/docs/v1/relations.html{#rel}",
+                      "name": "clickindex", "templated": true
+                  }
+                  ],
+                      "self": {
+                          "href": "https://search.apps.staging.ubuntu.com/api/v1/departments/fake-department-with-subdepartments"
+                      },
+                      "collection": {
+                          "href": "https://search.apps.staging.ubuntu.com/api/v1/departments"
+                      }
+              },
+              "name": "Fake Department With Subdepartments",
+              "slug": "fake-department-with-subdepartments"
+    })";
+
+const std::string FAKE_JSON_DEPARTMENTS_ONLY = R"(
+  {
+    "_links": {
+        "self": {
+            "href": "https://search.apps.ubuntu.com/api/v1/departments"
+        },
+        "curies": [
+            {
+                "name": "clickindex",
+                "href": "https://search.apps.ubuntu.com/docs/v1/relations.html{#rel}",
+                "templated": true
+            }
+        ]
+    },
+    "_embedded": {
+        "clickindex:department": [
+            {
+                "name": "Games",
+                "_links": {
+                    "self": {
+                        "href": "https://search.apps.ubuntu.com/api/v1/departments/Games"
+                    }
+                },
+                "_embedded": {
+                    "clickindex:department": [
+                        {
+                            "name": "Board Games",
+                            "_links": {
+                                "self": {
+                                    "href": "https://search.apps.ubuntu.com/api/v1/departments/Games/Board+Games"
+                                }
+                            }
+                        }
+                    ]
+                }
+            },
+            {
+                "name": "Graphics",
+                "_links": {
+                    "self": {
+                        "href": "https://search.apps.ubuntu.com/api/v1/departments/Graphics"
+                    }
+                },
+                "_embedded": {
+                    "clickindex:department": [
+                        {
+                            "name": "Drawing",
+                            "_links": {
+                                "self": {
+                                    "href": "https://search.apps.ubuntu.com/api/v1/departments/Graphics/Drawing"
+                                }
+                            }
+                        }
+                    ]
+                }
+            },
+            {
+                "name": "Internet",
+                "_links": {
+                    "self": {
+                        "href": "https://search.apps.ubuntu.com/api/v1/departments/Internet"
+                    }
+                },
+                "_embedded": {
+                    "clickindex:department": [
+                        {
+                            "name": "Chat",
+                            "_links": {
+                                "self": {
+                                    "href": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Chat"
+                                }
+                            }
+                        },
+                        {
+                            "name": "Mail",
+                            "_links": {
+                                "self": {
+                                    "href": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Mail"
+                                }
+                            }
+                        },
+                        {
+                            "name": "Web Browsers",
+                            "_links": {
+                                "self": {
+                                    "href": "https://search.apps.ubuntu.com/api/v1/departments/Internet/Web+Browsers"
+                                }
+                            }
+                        }
+                    ]
+                }
+            }
+        ]
+    }
+})";
+
+const std::string FAKE_JSON_BROKEN_DEPARTMENTS = R"(
+  {
+    "_links": {
+        "self": {
+            "href": "https://search.apps.ubuntu.com/api/v1/departments"
+        },
+        "curies": [
+            {
+                "name": "clickindex",
+                "href": "https://search.apps.ubuntu.com/docs/v1/relations.html{#rel}",
+                "templated": true
+            }
+        ]
+    },
+    "_embedded": {
+        "clickindex:department": [
+            {
+                "name": "Games",
+                "_links": {
+                    "self": {
+                        "href": "https://search.apps.ubuntu.com/api/v1/departments/Games"
+                    }
+                },
+                "_embedded": {
+                    "clickindex:department": [
+                        {
+                            "name": "Broken department"
+                        }
+                    ]
+                }
+            }
+        ]
+    }
+  })";
+
 const std::string FAKE_JSON_MANIFEST_REMOVABLE = R"foo(
     {
         "_removable": 1,

=== modified file 'libclickscope/tests/mock_network_access_manager.h'
--- libclickscope/tests/mock_network_access_manager.h	2014-05-20 19:33:41 +0000
+++ libclickscope/tests/mock_network_access_manager.h	2014-06-23 10:26:18 +0000
@@ -48,6 +48,7 @@
     {
         // Set a default value for QByteArray-returning mocked methods.
         ::testing::DefaultValue<QByteArray>::Set(QByteArray(""));
+        ON_CALL(*this, attribute(::testing::_)).WillByDefault(::testing::Return(0));
     }
 
     MOCK_METHOD0(abort, void());

=== modified file 'libclickscope/tests/mock_webclient.h'
--- libclickscope/tests/mock_webclient.h	2014-05-23 18:45:01 +0000
+++ libclickscope/tests/mock_webclient.h	2014-06-23 10:26:18 +0000
@@ -86,7 +86,7 @@
         const click::web::CallParams& params));
     QSharedPointer<click::web::Response> call(
         const std::string& iri,
-        const click::web::CallParams& params=click::web::CallParams()) {
+        const click::web::CallParams& params=click::web::CallParams()) override {
         return callImpl(iri, "GET", false,
                         std::map<std::string, std::string>(), "", params);
     }
@@ -96,7 +96,7 @@
         bool sign = false,
         const std::map<std::string, std::string>& headers = std::map<std::string, std::string>(),
         const std::string& data = "",
-        const click::web::CallParams& params=click::web::CallParams()) {
+        const click::web::CallParams& params=click::web::CallParams()) override {
         return callImpl(iri, method, sign, headers, data, params);
     }
 };

=== added file 'libclickscope/tests/test_bootstrap.cpp'
--- libclickscope/tests/test_bootstrap.cpp	1970-01-01 00:00:00 +0000
+++ libclickscope/tests/test_bootstrap.cpp	2014-06-23 10:26:18 +0000
@@ -0,0 +1,73 @@
+/*
+ * Copyright (C) 2014 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * In addition, as a special exception, the copyright holders give
+ * permission to link the code of portions of this program with the
+ * OpenSSL library under certain conditions as described in each
+ * individual source file, and distribute linked combinations
+ * including the two.
+ * You must obey the GNU General Public License in all respects
+ * for all of the code used other than OpenSSL.  If you modify
+ * file(s) with this exception, you may extend this exception to your
+ * version of the file(s), but you are not obligated to do so.  If you
+ * do not wish to do so, delete this exception statement from your
+ * version.  If you delete this exception statement from all source
+ * files in the program, then also delete it here.
+ */
+
+#include <gtest/gtest.h>
+#include "fake_json.h"
+#include <json/reader.h>
+#include <json/value.h>
+#include <click/highlights.h>
+#include <click/departments.h>
+
+class BootstrapTest: public ::testing::Test
+{
+    protected:
+        void SetUp() override
+        {
+        }
+};
+
+TEST_F(BootstrapTest, testParsing)
+{
+    Json::Reader reader;
+    Json::Value root;
+
+    EXPECT_TRUE(reader.parse(FAKE_JSON_BOOTSTRAP, root));
+
+    {
+        auto highlights = click::Highlight::from_json_root_node(root);
+        EXPECT_EQ(3u, highlights.size());
+        auto it = highlights.begin();
+        EXPECT_EQ("Top Apps", it->name());
+        EXPECT_EQ(2u, it->packages().size());
+        ++it;
+        EXPECT_EQ("Most Purchased", it->name());
+        EXPECT_EQ(2u, it->packages().size());
+        ++it;
+        EXPECT_EQ("New Releases", it->name());
+        EXPECT_EQ(2u, it->packages().size());
+    }
+    {
+        auto depts = click::Department::from_json_root_node(root);
+        EXPECT_EQ(1u, depts.size());
+        auto it = depts.begin();
+        EXPECT_EQ("Fake Subdepartment", (*it)->name());
+        EXPECT_FALSE((*it)->has_children_flag());
+        EXPECT_EQ("https://search.apps.staging.ubuntu.com/api/v1/departments/fake-subdepartment", (*it)->href());
+    }
+}

=== added file 'libclickscope/tests/test_departments.cpp'
--- libclickscope/tests/test_departments.cpp	1970-01-01 00:00:00 +0000
+++ libclickscope/tests/test_departments.cpp	2014-06-23 10:26:18 +0000
@@ -0,0 +1,151 @@
+/*
+ * Copyright (C) 2014 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3, as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranties of
+ * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ * PURPOSE.  See the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * In addition, as a special exception, the copyright holders give
+ * permission to link the code of portions of this program with the
+ * OpenSSL library under certain conditions as described in each
+ * individual source file, and distribute linked combinations
+ * including the two.
+ * You must obey the GNU General Public License in all respects
+ * for all of the code used other than OpenSSL.  If you modify
+ * file(s) with this exception, you may extend this exception to your
+ * version of the file(s), but you are not obligated to do so.  If you
+ * do not wish to do so, delete this exception statement from your
+ * version.  If you delete this exception statement from all source
+ * files in the program, then also delete it here.
+ */
+
+#include <gtest/gtest.h>
+#include "fake_json.h"
+#include <click/departments.h>
+#include <click/department-lookup.h>
+
+class DepartmentsTest : public ::testing::Test
+{
+    protected:
+        void SetUp() override
+        {
+        }
+};
+
+TEST_F(DepartmentsTest, testParsing)
+{
+    const std::string jsonstr(FAKE_JSON_DEPARTMENTS_ONLY);
+    auto depts = click::Department::from_json(jsonstr);
+    EXPECT_EQ(3u, depts.size());
+    auto it = depts.cbegin();
+    {
+        auto dep = *it;
+        EXPECT_EQ("Games", dep->id());
+        EXPECT_EQ("Games", dep->name());
+        EXPECT_EQ("https://search.apps.ubuntu.com/api/v1/departments/Games", dep->href());
+        EXPECT_FALSE(dep->has_children_flag());
+        auto subdepts = dep->sub_departments();
+        EXPECT_EQ(1u, subdepts.size());
+        auto sit = subdepts.cbegin();
+        EXPECT_EQ("Board Games", (*sit)->name());
+    }
+    {
+        ++it;
+        auto dep = *it;
+        EXPECT_EQ("Graphics", dep->id());
+        EXPECT_EQ("Graphics", dep->name());
+        EXPECT_EQ("https://search.apps.ubuntu.com/api/v1/departments/Graphics", dep->href());
+        EXPECT_FALSE(dep->has_children_flag());
+        auto subdepts = dep->sub_departments();
+        EXPECT_EQ(1u, subdepts.size());
+        auto sit = subdepts.cbegin();
+        EXPECT_EQ("Drawing", (*sit)->name());
+    }
+    {
+        ++it;
+        auto dep = *it;
+        EXPECT_EQ("Internet", dep->id());
+        EXPECT_EQ("Internet", dep->name());
+        EXPECT_EQ("https://search.apps.ubuntu.com/api/v1/departments/Internet", dep->href());
+        EXPECT_FALSE(dep->has_children_flag());
+        auto subdepts = dep->sub_departments();
+        EXPECT_EQ(3u, subdepts.size());
+        auto sit = subdepts.cbegin();
+        auto subdep = *sit;
+        EXPECT_EQ("Chat", subdep->name());
+        EXPECT_EQ("https://search.apps.ubuntu.com/api/v1/departments/Internet/Chat", subdep->href());
+        subdep = *(++sit);
+        EXPECT_EQ("Mail", subdep->name());
+        EXPECT_EQ("https://search.apps.ubuntu.com/api/v1/departments/Internet/Mail", subdep->href());
+        subdep = *(++sit);
+        EXPECT_EQ("Web Browsers", subdep->name());
+        EXPECT_EQ("https://search.apps.ubuntu.com/api/v1/departments/Internet/Web+Browsers", subdep->href());
+    }
+}
+
+TEST_F(DepartmentsTest, testParsingErrors)
+{
+    // invalid json
+    {
+        const std::string jsonstr("{{{");
+        auto depts = click::Department::from_json(jsonstr);
+        EXPECT_EQ(0, depts.size());
+    }
+    // one of the departments is invalid
+    {
+        const std::string jsonstr(FAKE_JSON_BROKEN_DEPARTMENTS);
+        auto depts = click::Department::from_json(jsonstr);
+        EXPECT_EQ(1, depts.size());
+    }
+}
+
+TEST_F(DepartmentsTest, testLookup)
+{
+    auto dep_games = std::make_shared<click::Department>("games", "Games", "http://foobar.com/", false);
+    auto dep_rpg = std::make_shared<click::Department>("rpg", "RPG", "http://ubuntu.com/", false);
+    auto dep_strategy = std::make_shared<click::Department>("strategy", "Strategy", "", false);
+    const std::list<click::Department::SPtr> departments {dep_rpg, dep_strategy};
+    dep_games->set_subdepartments(departments);
+
+    const std::list<click::Department::SPtr> root {dep_games};
+    click::DepartmentLookup lut;
+    lut.rebuild(root);
+
+    {
+        EXPECT_EQ(2u, lut.size());
+        EXPECT_EQ("games", lut.get_parent("strategy")->id());
+        EXPECT_EQ("games", lut.get_parent("rpg")->id());
+        EXPECT_EQ(nullptr, lut.get_parent("games"));
+    }
+    {
+        auto info = lut.get_department_info("games");
+        EXPECT_EQ("games", info->id());
+        EXPECT_EQ("Games", info->name());
+        EXPECT_EQ("http://foobar.com/", info->href());
+        EXPECT_EQ(false, info->has_children_flag());
+
+        auto sub = info->sub_departments();
+        EXPECT_EQ(2u, sub.size());
+
+        auto it = sub.begin();
+        EXPECT_EQ("rpg", (*it)->id());
+        EXPECT_EQ("RPG", (*it)->name());
+        EXPECT_EQ("http://ubuntu.com/", (*it)->href());
+        ++it;
+        EXPECT_EQ("strategy", (*it)->id());
+        EXPECT_EQ("Strategy", (*it)->name());
+    }
+    {
+        lut.rebuild(root);
+        EXPECT_EQ(2u, lut.size());
+    }
+}
+

=== modified file 'libclickscope/tests/test_index.cpp'
--- libclickscope/tests/test_index.cpp	2014-06-12 21:05:25 +0000
+++ libclickscope/tests/test_index.cpp	2014-06-23 10:26:18 +0000
@@ -52,8 +52,8 @@
         click::Index(client, configuration)
     {
     }
-    MOCK_METHOD1(build_index_query, std::string(const std::string&));
     MOCK_METHOD0(build_headers, std::map<std::string, std::string>());
+    MOCK_METHOD2(build_index_query, std::string(const std::string&, const std::string&));
 };
 
 class MockConfiguration : public click::Configuration {
@@ -116,7 +116,7 @@
             .Times(1)
             .WillOnce(Return(response));
 
-    EXPECT_CALL(*indexPtr, build_index_query(FAKE_QUERY))
+    EXPECT_CALL(*indexPtr, build_index_query(FAKE_QUERY, ""))
             .Times(1)
             .WillOnce(Return(FAKE_BUILT_QUERY));
 

=== modified file 'scope/clickstore/store-query.cpp'
--- scope/clickstore/store-query.cpp	2014-06-18 16:23:50 +0000
+++ scope/clickstore/store-query.cpp	2014-06-23 10:26:18 +0000
@@ -30,6 +30,7 @@
 #include <click/application.h>
 #include <click/interface.h>
 #include "store-query.h"
+#include "store-scope.h"
 #include <click/qtbridge.h>
 
 #include <click/key_file_locator.h>
@@ -37,6 +38,7 @@
 #include <unity/scopes/Annotation.h>
 #include <unity/scopes/CategoryRenderer.h>
 #include <unity/scopes/CategorisedResult.h>
+#include <unity/scopes/Department.h>
 #include <unity/scopes/CannedQuery.h>
 #include <unity/scopes/SearchReply.h>
 #include <unity/scopes/SearchMetadata.h>
@@ -93,19 +95,37 @@
 
 struct click::Query::Private
 {
+<<<<<<< TREE
     Private(click::Index& index, const scopes::SearchMetadata& metadata)
         : index(index),
+=======
+    Private(click::Index& index, click::DepartmentLookup& depts,
+            click::HighlightList& highlights, const scopes::SearchMetadata& metadata)
+        : index(index),
+          department_lookup(depts),
+          highlights(highlights),
+>>>>>>> MERGE-SOURCE
           meta(metadata)
     {
     }
     click::Index& index;
+    click::DepartmentLookup& department_lookup;
+    click::HighlightList& highlights;
     scopes::SearchMetadata meta;
     click::web::Cancellable search_operation;
 };
 
+<<<<<<< TREE
 click::Query::Query(unity::scopes::CannedQuery const& query, click::Index& index, scopes::SearchMetadata const& metadata)
     : unity::scopes::SearchQueryBase(query, metadata),
       impl(new Private(index, metadata))
+=======
+click::Query::Query(unity::scopes::CannedQuery const& query, click::Index& index, click::DepartmentLookup& depts,
+        click::HighlightList& highlights,
+        scopes::SearchMetadata const& metadata)
+    : unity::scopes::SearchQueryBase(query, metadata),
+      impl(new Private(index, depts, highlights, metadata))
+>>>>>>> MERGE-SOURCE
 {
 }
 
@@ -154,6 +174,166 @@
     });
 }
 
+//
+// creates department menu with narrowed-down list of subdepartments of current department, as
+// returned by server call
+void click::Query::populate_departments(const click::DepartmentList& subdepts, const std::string& current_dep_id, unity::scopes::Department::SPtr &root)
+{
+    unity::scopes::DepartmentList departments;
+
+    // create a list of subdepartments of current department
+    foreach (auto d, subdepts)
+    {
+        unity::scopes::Department::SPtr department = unity::scopes::Department::create(d->id(), query(), d->name());
+        if (d->has_children_flag())
+        {
+            department->set_has_subdepartments();
+        }
+        departments.push_back(department);
+    }
+
+    if (current_dep_id != "")
+    {
+        auto curr_dpt = impl->department_lookup.get_department_info(current_dep_id);
+        if (curr_dpt != nullptr)
+        {
+            unity::scopes::Department::SPtr current = unity::scopes::Department::create(current_dep_id, query(), curr_dpt->name());
+            if (departments.size() > 0) // this may be a leaf department
+            {
+                current->set_subdepartments(departments);
+            }
+
+            auto parent_info = impl->department_lookup.get_parent(current_dep_id);
+            if (parent_info != nullptr)
+            {
+                root = unity::scopes::Department::create(parent_info->id(), query(), parent_info->name());
+                root->set_subdepartments({current});
+                return;
+            }
+            else
+            {
+                root = unity::scopes::Department::create("", query(), _("All departments"));
+                root->set_subdepartments({current});
+                return;
+            }
+        }
+        else
+        {
+            qWarning() << "Unknown department:" << QString::fromStdString(current_dep_id);
+        }
+    }
+
+    root = unity::scopes::Department::create("", query(), _("All departments"));
+    root->set_subdepartments(departments);
+}
+
+void click::Query::push_package(const scopes::SearchReplyProxy& searchReply, scopes::Category::SCPtr category, const PackageSet &installedPackages, const Package& pkg)
+{
+    qDebug() << "pushing result" << QString::fromStdString(pkg.name);
+    try {
+        scopes::CategorisedResult res(category);
+        res.set_title(pkg.title);
+        res.set_art(pkg.icon_url);
+        res.set_uri(pkg.url);
+        res[click::Query::ResultKeys::NAME] = pkg.name;
+        auto installed = installedPackages.find(pkg);
+        if (installed != installedPackages.end()) {
+            res[click::Query::ResultKeys::INSTALLED] = true;
+            res["subtitle"] = _("✔ INSTALLED");
+            res[click::Query::ResultKeys::VERSION] = installed->version;
+        } else {
+            res[click::Query::ResultKeys::INSTALLED] = false;
+            // TODO: get the real price from the webservice (upcoming branch)
+            res["subtitle"] = _("FREE");
+        }
+
+        this->push_result(searchReply, res);
+    } catch(const std::exception& e){
+        qDebug() << "PackageDetails::loadJson: Exception thrown while decoding JSON: " << e.what() ;
+    } catch(...){
+        qDebug() << "no reason to catch";
+    }
+}
+
+void click::Query::push_highlights(const scopes::SearchReplyProxy& searchReply, const HighlightList& highlights, const PackageSet &locallyInstalledApps)
+{
+    std::string categoryTemplate = CATEGORY_APPS_DISPLAY; //FIXME
+    scopes::CategoryRenderer renderer(categoryTemplate);
+
+    for (auto const& hl: highlights)
+    {
+        auto category = register_category(searchReply, hl.name(), hl.name(), "", renderer); //FIXME: highlight slug
+        for (auto const& pkg: hl.packages())
+        {
+            push_package(searchReply, category, locallyInstalledApps, pkg);
+        }
+    }
+    qDebug() << "Highlights pushed";
+}
+
+void click::Query::push_departments(const scopes::SearchReplyProxy& searchReply, const scopes::Department::SCPtr& root)
+{
+    if (root != nullptr)
+    {
+        try
+        {
+            qDebug() << "pushing departments";
+            searchReply->register_departments(root);
+        }
+        catch (const std::exception& e)
+        {
+            qWarning() << "Failed to register departments for query " << QString::fromStdString(query().query_string()) <<
+                ", current department " << QString::fromStdString(query().department_id()) << ": " << e.what();
+        }
+    }
+    else
+    {
+        qWarning() << "No departments data for query " << QString::fromStdString(query().query_string()) <<
+            "', current department " << QString::fromStdString(query().department_id());
+    }
+}
+
+//
+// push highlights and departments
+// use cached highlights for root department, otherwise run an async job for highlights of current department.
+void click::Query::add_highlights(scopes::SearchReplyProxy const& searchReply, const PackageSet& locallyInstalledApps)
+{
+    auto curdep = impl->department_lookup.get_department_info(query().department_id());
+    assert(curdep);
+    auto subdepts = curdep->sub_departments();
+    if (query().department_id() == "") // top-level departments
+    {
+        unity::scopes::Department::SPtr root;
+        populate_departments(subdepts, query().department_id(), root);
+        push_departments(searchReply, root);
+
+        qDebug() << "pushing cached highlights";
+        push_highlights(searchReply, impl->highlights, locallyInstalledApps);
+        this->finished(searchReply); //FIXME: this shouldn't be needed
+    }
+    else
+    {
+        qDebug() << "starting departments call for department" << QString::fromStdString(curdep->id()) << ", href" << QString::fromStdString(curdep->href());
+        impl->search_operation = impl->index.departments(curdep->href(), [this, locallyInstalledApps, searchReply](const DepartmentList& depts,
+                    const HighlightList& highlights, Index::Error error, int)
+                {
+                    if (error == click::Index::Error::NoError)
+                    {
+                        qDebug() << "departments call completed";
+                        unity::scopes::Department::SPtr root;
+                        populate_departments(depts, query().department_id(), root);
+                        push_departments(searchReply, root);
+                        push_highlights(searchReply, highlights, locallyInstalledApps);
+                    }
+                    else
+                    {
+                        qWarning() << "departments call failed";
+                    }
+                    this->finished(searchReply); //FIXME: this shouldn't be needed
+                });
+        }
+}
+
 void click::Query::add_available_apps(scopes::SearchReplyProxy const& searchReply,
                                       const PackageSet& installedPackages,
                                       const std::string& categoryTemplate)
@@ -161,12 +341,14 @@
     scopes::CategoryRenderer categoryRenderer(categoryTemplate);
     auto category = register_category(searchReply, "appstore", _("Available"), "", categoryRenderer);
 
+    assert(searchReply);
+
     run_under_qt([=]()
     {
             auto search_cb = [this, searchReply, category, installedPackages](Packages packages) {
                 qDebug("search callback");
 
-                // handle packages data
+                // handle packages data; FIXME: use push_package()
                 foreach (auto p, packages) {
                     qDebug() << "pushing result" << QString::fromStdString(p.name);
                     try {
@@ -194,11 +376,66 @@
                     }
                 }
                 qDebug() << "search completed";
+<<<<<<< TREE
                 this->finished(searchReply);
         };
 
             qDebug() << "starting search of" << QString::fromStdString(query().query_string());
             impl->search_operation = impl->index.search(query().query_string(), search_cb);
+=======
+                this->finished(searchReply); //FIXME: this shouldn't be needed
+            };
+
+            // this is the case when we do bootstrap for the first time, or it failed last time
+            if (impl->department_lookup.size() == 0 && !click::Scope::use_old_api())
+            {
+                qDebug() << "performing bootstrap request";
+                impl->search_operation = impl->index.bootstrap([this, search_cb, searchReply, installedPackages](const DepartmentList& deps, const
+                            HighlightList& highlights, click::Index::Error error, int error_code) {
+                if (error == click::Index::Error::NoError)
+                {
+                    qDebug() << "bootstrap request completed";
+                    auto root = std::make_shared<click::Department>("", "All Departments", "", true);
+                    root->set_subdepartments(deps);
+                    DepartmentList rdeps { root };
+                    impl->department_lookup.rebuild(rdeps);
+                    impl->highlights = highlights;
+                    qDebug() << "Total number of departments:" << impl->department_lookup.size() << ", highlights:" << highlights.size();
+                }
+                else
+                {
+                    qWarning() << "bootstrap request failed";
+                    if (error_code == 405) // method not allowed
+                    {
+                        qDebug() << "bootstrap not available, using old API";
+                        click::Scope::set_use_old_api();
+                    }
+                }
+
+                if (query().query_string().empty() && !click::Scope::use_old_api())
+                {
+                    add_highlights(searchReply, installedPackages);
+                }
+                else
+                {
+                    qDebug() << "starting search of" << QString::fromStdString(query().query_string());
+                    impl->search_operation = impl->index.search(query().query_string(), search_cb);
+                }
+            });
+        }
+        else
+        {
+            if (query().query_string().empty() && !click::Scope::use_old_api())
+            {
+                add_highlights(searchReply, installedPackages);
+            }
+            else // normal search
+            {
+                qDebug() << "starting search of" << QString::fromStdString(query().query_string());
+                impl->search_operation = impl->index.search(query().query_string(), search_cb);
+            }
+        }
+>>>>>>> MERGE-SOURCE
     });
 }
 

=== modified file 'scope/clickstore/store-query.h'
--- scope/clickstore/store-query.h	2014-06-16 14:09:21 +0000
+++ scope/clickstore/store-query.h	2014-06-23 10:26:18 +0000
@@ -32,10 +32,16 @@
 
 
 #include <unity/scopes/SearchQueryBase.h>
+#include <unity/scopes/Department.h>
 
 namespace scopes = unity::scopes;
 
 #include <QSharedPointer>
+#include <set>
+
+#include <click/department-lookup.h>
+#include <click/package.h>
+#include <click/highlights.h>
 #include <click/interface.h>
 
 namespace click
@@ -43,6 +49,7 @@
 
 class Application;
 class Index;
+class DepartmentLookup;
 
 class Query : public scopes::SearchQueryBase
 {
@@ -69,7 +76,8 @@
         constexpr static const char* VERSION{"version"};
     };
 
-    Query(unity::scopes::CannedQuery const& query, click::Index& index, scopes::SearchMetadata const& metadata);
+    Query(unity::scopes::CannedQuery const& query, click::Index& index, click::DepartmentLookup& dept_lookup, click::HighlightList& highlights,
+            scopes::SearchMetadata const& metadata);
     virtual ~Query();
 
     virtual void cancelled() override;
@@ -77,8 +85,10 @@
     virtual void run(scopes::SearchReplyProxy const& reply) override;
 
 protected:
-    virtual void add_available_apps(const scopes::SearchReplyProxy &searchReply,
-                                    const PackageSet &installedPackages, const std::string &category);
+    virtual void populate_departments(const click::DepartmentList& depts, const std::string& current_department_id, unity::scopes::Department::SPtr &root);
+    virtual void push_departments(const scopes::SearchReplyProxy& searchReply, const scopes::Department::SCPtr& root);
+    virtual void add_highlights(scopes::SearchReplyProxy const& searchReply, const PackageSet& installedPackages);
+    virtual void add_available_apps(const scopes::SearchReplyProxy &searchReply, const PackageSet &installedPackages, const std::string &category);
     virtual click::Interface& clickInterfaceInstance();
     virtual PackageSet get_installed_packages();
     virtual bool push_result(const scopes::SearchReplyProxy &searchReply, scopes::CategorisedResult const& res);
@@ -88,6 +98,9 @@
                                                std::string const& title,
                                                std::string const& icon,
                                                scopes::CategoryRenderer const& renderer_template);
+    virtual void push_package(const scopes::SearchReplyProxy& searchReply, scopes::Category::SCPtr category, const PackageSet &locallyInstalledApps,
+            const click::Package& pkg);
+    virtual void push_highlights(const scopes::SearchReplyProxy& searchReply, const HighlightList& highlights, const PackageSet &locallyInstalledApps);
     virtual void run_under_qt(const std::function<void()> &task);
 
 private:

=== modified file 'scope/clickstore/store-scope.cpp'
--- scope/clickstore/store-scope.cpp	2014-06-19 17:05:34 +0000
+++ scope/clickstore/store-scope.cpp	2014-06-23 10:26:18 +0000
@@ -28,6 +28,7 @@
  */
 
 #include <click/qtbridge.h>
+#include <click/department-lookup.h>
 #include "store-scope.h"
 #include "store-query.h"
 #include <click/preview.h>
@@ -42,19 +43,36 @@
 
 #include <logging.h>
 
+bool click::Scope::old_api = false;
 
 click::Scope::Scope()
 {
     nam.reset(new click::network::AccessManager());
     client.reset(new click::web::Client(nam));
     index.reset(new click::Index(client));
+    depts.reset(new click::DepartmentLookup());
+    highlights.reset(new click::HighlightList());
 }
 
 click::Scope::~Scope()
 {
 }
 
-void click::Scope::start(std::string const&, scopes::RegistryProxy const&)
+<<<<<<< TREE
+void click::Scope::start(std::string const&, scopes::RegistryProxy const&)
+=======
+void click::Scope::set_use_old_api()
+{
+    old_api = true;
+}
+
+bool click::Scope::use_old_api()
+{
+    return old_api;
+}
+
+void click::Scope::start(std::string const&, scopes::RegistryProxy const&)
+>>>>>>> MERGE-SOURCE
 {
     setlocale(LC_ALL, "");
     // FIXME: This is wrong, but needed for json-cpp workaround.
@@ -68,7 +86,6 @@
     static const int zero = 0;
     auto emptyCb = [this]()
     {
-
     };
 
     qt::core::world::build_and_run(zero, nullptr, emptyCb);
@@ -81,7 +98,7 @@
 
 scopes::SearchQueryBase::UPtr click::Scope::search(unity::scopes::CannedQuery const& q, scopes::SearchMetadata const& metadata)
 {
-    return scopes::SearchQueryBase::UPtr(new click::Query(q, *index, metadata));
+    return scopes::SearchQueryBase::UPtr(new click::Query(q, *index, *depts, *highlights, metadata));
 }
 
 
@@ -91,10 +108,21 @@
     return scopes::PreviewQueryBase::UPtr{new click::Preview(result, metadata, client, nam)};
 }
 
+<<<<<<< TREE
 unity::scopes::ActivationQueryBase::UPtr click::Scope::perform_action(unity::scopes::Result const& result, unity::scopes::ActionMetadata const& metadata, std::string const& /* widget_id */, std::string const& action_id)
+=======
+unity::scopes::ActivationQueryBase::UPtr click::Scope::perform_action(unity::scopes::Result const& result, unity::scopes::ActionMetadata const& metadata,
+        std::string const& widget_id, std::string const& _action_id)
+>>>>>>> MERGE-SOURCE
 {
+<<<<<<< TREE
     auto activation = new ScopeActivation(result, metadata);
     qDebug() << "perform_action called with action_id" << QString().fromStdString(action_id);
+=======
+    std::string action_id = _action_id;
+    qDebug() << "perform_action called with widget_id" << QString::fromStdString(widget_id) << "and action_id:" << QString::fromStdString(action_id);
+    auto activation = new ScopeActivation(result, metadata);
+>>>>>>> MERGE-SOURCE
 
     // if the purchase is completed, do the install
     if (action_id == "purchaseCompleted") {

=== modified file 'scope/clickstore/store-scope.h'
--- scope/clickstore/store-scope.h	2014-06-18 14:42:47 +0000
+++ scope/clickstore/store-scope.h	2014-06-23 10:26:18 +0000
@@ -30,6 +30,7 @@
 #ifndef CLICK_SCOPE_H
 #define CLICK_SCOPE_H
 
+#include <memory>
 #include <click/network_access_manager.h>
 #include <click/webclient.h>
 
@@ -43,6 +44,9 @@
 
 namespace click
 {
+
+class DepartmentLookup;
+
 class Scope : public scopes::ScopeBase
 {
 public:
@@ -60,12 +64,18 @@
 
     virtual unity::scopes::ActivationQueryBase::UPtr perform_action(unity::scopes::Result const& result, unity::scopes::ActionMetadata const& metadata, std::string const& widget_id, std::string const& action_id) override;
 
+    static void set_use_old_api();
+    static bool use_old_api();
+
 private:
     QSharedPointer<click::network::AccessManager> nam;
     QSharedPointer<click::web::Client> client;
     QSharedPointer<click::Index> index;
+    std::shared_ptr<click::DepartmentLookup> depts;
+    std::shared_ptr<click::HighlightList> highlights;
 
     std::string installApplication(unity::scopes::Result const& result);
+    static bool old_api;
 };
 }
 #endif // CLICK_SCOPE_H

=== modified file 'scope/tests/integration/webclient_integration.cpp'
--- scope/tests/integration/webclient_integration.cpp	2014-06-12 21:05:25 +0000
+++ scope/tests/integration/webclient_integration.cpp	2014-06-23 10:26:18 +0000
@@ -30,6 +30,7 @@
 #include <click/network_access_manager.h>
 #include <click/webclient.h>
 #include <click/index.h>
+#include <click/departments.h>
 
 #include <QCoreApplication>
 #include <QDebug>

=== modified file 'scope/tests/test_query.cpp'
--- scope/tests/test_query.cpp	2014-06-12 22:17:42 +0000
+++ scope/tests/test_query.cpp	2014-06-23 10:26:18 +0000
@@ -45,6 +45,7 @@
 #include <unity/scopes/CannedQuery.h>
 #include <unity/scopes/ScopeBase.h>
 #include <unity/scopes/SearchReply.h>
+#include <unity/scopes/testing/MockSearchReply.h>
 
 using namespace ::testing;
 using namespace click;
@@ -57,10 +58,17 @@
 
 class MockIndex : public click::Index {
     click::Packages packages;
+    click::DepartmentList departments;
+    click::DepartmentList bootstrap_departments;
+    click::HighlightList bootstrap_highlights;
 public:
-    MockIndex(click::Packages packages = click::Packages())
+    MockIndex(click::Packages packages = click::Packages(),
+            click::DepartmentList departments = click::DepartmentList(),
+            click::DepartmentList boot_departments = click::DepartmentList())
         : Index(QSharedPointer<click::web::Client>()),
-          packages(packages)
+          packages(packages),
+          departments(departments),
+          bootstrap_departments(boot_departments)
     {
 
     }
@@ -72,15 +80,21 @@
         return click::web::Cancellable();
     }
 
-    MOCK_METHOD2(do_search,
-                 void(const std::string&,
-                      std::function<void(click::Packages)>));
+    click::web::Cancellable bootstrap(std::function<void(const click::DepartmentList&, const click::HighlightList&, Error, int)> callback) override
+    {
+        callback(bootstrap_departments, bootstrap_highlights, click::Index::Error::NoError, 0);
+        return click::web::Cancellable();
+    }
+
+    MOCK_METHOD2(do_search, void(const std::string&, std::function<void(click::Packages)>));
 };
 
 class MockQueryBase : public click::Query {
 public:
     MockQueryBase(const unity::scopes::CannedQuery& query, click::Index& index,
-                  scopes::SearchMetadata const& metadata) : click::Query(query, index, metadata)
+                  click::DepartmentLookup& depts,
+                  click::HighlightList& highlights,
+                  scopes::SearchMetadata const& metadata) : click::Query(query, index, depts, highlights, metadata)
     {
 
     }
@@ -94,7 +108,9 @@
 class MockQuery : public MockQueryBase {
 public:
     MockQuery(const unity::scopes::CannedQuery& query, click::Index& index,
-              scopes::SearchMetadata const& metadata) : MockQueryBase(query, index, metadata)
+              click::DepartmentLookup& depts,
+              click::HighlightList& highlights,
+              scopes::SearchMetadata const& metadata) : MockQueryBase(query, index, depts, highlights, metadata)
     {
 
     }
@@ -118,7 +134,9 @@
 class MockQueryRun : public MockQueryBase {
 public:
     MockQueryRun(const unity::scopes::CannedQuery& query, click::Index& index,
-                 scopes::SearchMetadata const& metadata) : MockQueryBase(query, index, metadata)
+                 click::DepartmentLookup& depts,
+                 click::HighlightList& highlights, 
+                 scopes::SearchMetadata const& metadata) : MockQueryBase(query, index, depts, highlights, metadata)
     {
 
     }
@@ -147,12 +165,16 @@
 TEST(QueryTest, testAddAvailableAppsCallsClickIndex)
 {
     MockIndex mock_index;
+    click::DepartmentLookup dept_lookup;
+    click::HighlightList highlights;
     scopes::SearchMetadata metadata("en_EN", "phone");
     PackageSet no_installed_packages;
     const unity::scopes::CannedQuery query("foo.scope", FAKE_QUERY, "");
-    MockQuery q(query, mock_index, metadata);
+    MockQuery q(query, mock_index, dept_lookup, highlights, metadata);
     EXPECT_CALL(mock_index, do_search(FAKE_QUERY, _)).Times(1);
-    scopes::SearchReplyProxy reply;
+
+    scopes::testing::MockSearchReply mock_reply;
+    scopes::SearchReplyProxy reply(&mock_reply, [](unity::scopes::SearchReply*){});
 
     scopes::CategoryRenderer renderer("{}");
     auto ptrCat = std::make_shared<FakeCategory>("id", "", "", renderer);
@@ -166,17 +188,21 @@
         {"name", "title", 0.0, "icon", "uri"}
     };
     MockIndex mock_index(packages);
+    click::DepartmentLookup dept_lookup;
+    click::HighlightList highlights;
     scopes::SearchMetadata metadata("en_EN", "phone");
     PackageSet no_installed_packages;
     const unity::scopes::CannedQuery query("foo.scope", FAKE_QUERY, "");
-    MockQuery q(query, mock_index, metadata);
+    MockQuery q(query, mock_index, dept_lookup, highlights, metadata);
     EXPECT_CALL(mock_index, do_search(FAKE_QUERY, _));
 
     scopes::CategoryRenderer renderer("{}");
     auto ptrCat = std::make_shared<FakeCategory>("id", "", "", renderer);
     EXPECT_CALL(q, register_category(_, _, _, _, _)).WillOnce(Return(ptrCat));
 
-    scopes::SearchReplyProxy reply;
+    scopes::testing::MockSearchReply mock_reply;
+    scopes::SearchReplyProxy reply(&mock_reply, [](unity::scopes::SearchReply*){});
+
     auto expected_title = packages.front().title;
     EXPECT_CALL(q, push_result(_, Property(&scopes::CategorisedResult::title, expected_title)));
     q.wrap_add_available_apps(reply, no_installed_packages, FAKE_CATEGORY_TEMPLATE);
@@ -188,17 +214,20 @@
         {"name", "title", 0.0, "icon", "uri"}
     };
     MockIndex mock_index(packages);
+    click::DepartmentLookup dept_lookup;
+    click::HighlightList highlights;
     scopes::SearchMetadata metadata("en_EN", "phone");
     PackageSet no_installed_packages;
     const unity::scopes::CannedQuery query("foo.scope", FAKE_QUERY, "");
-    MockQuery q(query, mock_index, metadata);
+    MockQuery q(query, mock_index, dept_lookup, highlights, metadata);
     EXPECT_CALL(mock_index, do_search(FAKE_QUERY, _));
 
     scopes::CategoryRenderer renderer("{}");
     auto ptrCat = std::make_shared<FakeCategory>("id", "", "", renderer);
     EXPECT_CALL(q, register_category(_, _, _, _, _)).WillOnce(Return(ptrCat));
 
-    scopes::SearchReplyProxy reply;
+    scopes::testing::MockSearchReply mock_reply;
+    scopes::SearchReplyProxy reply(&mock_reply, [](unity::scopes::SearchReply*){});
     EXPECT_CALL(q, finished(_));
     q.wrap_add_available_apps(reply, no_installed_packages, FAKE_CATEGORY_TEMPLATE);
 }
@@ -209,10 +238,12 @@
         {"name", "title", 0.0, "icon", "uri"}
     };
     MockIndex mock_index(packages);
+    click::DepartmentLookup dept_lookup;
+    click::HighlightList highlights;
     scopes::SearchMetadata metadata("en_EN", "phone");
     PackageSet no_installed_packages;
     const unity::scopes::CannedQuery query("foo.scope", FAKE_QUERY, "");
-    MockQueryRun q(query, mock_index, metadata);
+    MockQueryRun q(query, mock_index, dept_lookup, highlights, metadata);
     auto reply = scopes::SearchReplyProxy();
     EXPECT_CALL(q, get_installed_packages()).WillOnce(Return(no_installed_packages));
     EXPECT_CALL(q, add_available_apps(reply, no_installed_packages, _));
@@ -234,15 +265,18 @@
     PackageSet one_installed_package {
         {"org.example.app2", "0.2"}
     };
+    click::DepartmentLookup dept_lookup;
+    click::HighlightList highlights;
     const unity::scopes::CannedQuery query("foo.scope", FAKE_QUERY, "");
-    MockQuery q(query, mock_index, metadata);
+    MockQuery q(query, mock_index, dept_lookup, highlights, metadata);
     EXPECT_CALL(mock_index, do_search(FAKE_QUERY, _));
 
     scopes::CategoryRenderer renderer("{}");
     auto ptrCat = std::make_shared<FakeCategory>("id", "", "", renderer);
     EXPECT_CALL(q, register_category(_, _, _, _, _)).WillOnce(Return(ptrCat));
 
-    scopes::SearchReplyProxy reply;
+    scopes::testing::MockSearchReply mock_reply;
+    scopes::SearchReplyProxy reply(&mock_reply, [](unity::scopes::SearchReply*){});
     auto expected_name1 = packages.front().name;
     EXPECT_CALL(q, push_result(_, HasPackageName(expected_name1)));
     auto expected_name2 = packages.back().name;
@@ -261,15 +295,18 @@
     PackageSet one_installed_package {
         {"org.example.app2", "0.2"}
     };
+    click::DepartmentLookup dept_lookup;
+    click::HighlightList highlights;
     const unity::scopes::CannedQuery query("foo.scope", FAKE_QUERY, "");
-    MockQuery q(query, mock_index, metadata);
+    MockQuery q(query, mock_index, dept_lookup, highlights, metadata);
     EXPECT_CALL(mock_index, do_search(FAKE_QUERY, _));
 
     scopes::CategoryRenderer renderer("{}");
     auto ptrCat = std::make_shared<FakeCategory>("id", "", "", renderer);
     EXPECT_CALL(q, register_category(_, _, _, _, _)).WillOnce(Return(ptrCat));
 
-    scopes::SearchReplyProxy reply;
+    scopes::testing::MockSearchReply mock_reply;
+    scopes::SearchReplyProxy reply(&mock_reply, [](unity::scopes::SearchReply*){});
     EXPECT_CALL(q, push_result(_, IsInstalled(true)));
     EXPECT_CALL(q, push_result(_, IsInstalled(false)));
     q.wrap_add_available_apps(reply, one_installed_package, FAKE_CATEGORY_TEMPLATE);
@@ -288,8 +325,10 @@
     };
     MockIndex mock_index(uninstalled_packages);
     scopes::SearchMetadata metadata("en_EN", "phone");
+    click::DepartmentLookup dept_lookup;
+    click::HighlightList highlights;
     const unity::scopes::CannedQuery query("foo.scope", FAKE_QUERY, "");
-    MockQuery q(query, mock_index, metadata);
+    MockQuery q(query, mock_index, dept_lookup, highlights, metadata);
     PackageSet installed_packages{{"package_1", "0.1"}};
 
     FakeInterface fake_interface;