Merge lp:~sil/component-store/community-components into lp:component-store

[Stuart Langridge]

The community store.

Reorganise the existing trunk to put the app stuff into an "app" folder.
Rewrite "ucs" so that it knows about community and curated components, and about submitting community components.
Make "ucs install" correctly install both pure QML and binary compiled components in an appropriate place in the current project, and integrate those components with the build system for this project, automatically. (Work in progress; done for pure QML projects; partially done for qmake projects; not done for cmake projects.)
Implement a community server which receives "ucs submit" requests and updates the master list of all components (in lp:~ubuntu-touch-community-dev/component-store/community-components-json).
Add documentation for how to create a community component and submit it.

[lp:~sil/component-store/community-components]

53. By Stuart Langridge on 2015-01-04

merge lp:~nik90/component-store/fix-gallerysrc

54. By Stuart Langridge on 2015-01-04

Add version requirements to everything

55. By Stuart Langridge on 2015-01-04

https for the community server, if you please

56. By Stuart Langridge on 2015-01-04

Correctly show errors

57. By Stuart Langridge on 2015-01-04

FIx eror messages, properly this time

58. By Stuart Langridge on 2015-01-04

Get and cache curated component metadata and use it for searching and display

59. By Stuart Langridge on 2015-01-04

merged lp:~nik90/component-store/rename-app-folder

60. By Stuart Langridge on 2015-01-04

allow forcing refresh on search

61. By Stuart Langridge on 2015-01-04

Handle adding pure QML components to cmake-based projects

62. By Stuart Langridge on 2015-01-04

Add decent logging to server, including by gmail email

63. By Stuart Langridge on 2015-01-05

Add note clarifying that compoennts with with qml and qmllib should work

[Nekhelesh Ramananthan (nik90)]

TODO: Just before merging this branch, we can remove ucs.sh (legacy script).

[Nekhelesh Ramananthan (nik90)]

Things that needs fixing,

1. Remove legacy ucs.sh script

2. Remove support for qmake and cmake projects. What we have currently has 2 main issues which are, "installing to the wrong directory" and "invalid CMakeLists.txt". The app project structure is not constant and this is more prevalent in cmake projects where qml code usually goes into app/* and c++ components go into backend/* or src/*. Adding ubuntu_component_store to the root folder might not be appreciated by developers.

for now just adding the necessary components into a folder called ubuntu_component_store and requiring app developers to do the rest of the bootstrapping would suffice IMO. We can build on top of that once we get help from the community and the SDK devs.

3. Merge lp:~nik90/component-store/fix-debian-packaging into this MP. Although before doing that please check the debian/control file to see if it has all the necessary dependency for ucs. If a dependency is missing, please let me know and I will add it.

After this MP is merged I will update the online docs and release a update in the PPA.

[Nekhelesh Ramananthan (nik90)]

review: Needs Fixing

[lp:~sil/component-store/community-components]

64. By Stuart Langridge on 2015-04-13

Remove attempts to add installed components to the project infrastructure for qmake and cmake projects; instead, print a message explaining what to do. DWIMming this is pretty difficult, especially given that I don't really understand either qmake or cmake, and so what we do is punt, explain the situation, and assume that anyone who has chosen to use cmake or qmake in a project knows something about C++ and compilation steps and therefore can add it themselves.

65. By Stuart Langridge on 2015-04-13

Remove legacy ucs.sh script

[Stuart Langridge (sil)]

New versions pushed which remove attempts to insert components into the build process for [cq]make projects.

[lp:~sil/component-store/community-components]

66. By Stuart Langridge on 2015-04-13

merge lp:~nik90/component-store/fix-debian-packaging

[Nekhelesh Ramananthan (nik90)]

Lets get this awesomeness merged and out to the world!

review: Approve

=== modified file '.bzrignore'
--- .bzrignore	2014-11-09 00:04:53 +0000
+++ .bzrignore	2015-04-13 19:51:02 +0000
@@ -3,3 +3,7 @@
 docs/_build
 docs/*.save
 
+venv
+../logs
+../logs/server.log
+gmailconf.json

=== added directory 'community-server'
=== added file 'community-server/requirements.txt'
--- community-server/requirements.txt	1970-01-01 00:00:00 +0000
+++ community-server/requirements.txt	2015-04-13 19:51:02 +0000
@@ -0,0 +1,7 @@
+Flask==0.10.1
+Jinja2==2.7.3
+MarkupSafe==0.23
+Werkzeug==0.9.6
+argparse==1.2.1
+itsdangerous==0.24
+wsgiref==0.1.2

=== added file 'community-server/server.py'
--- community-server/server.py	1970-01-01 00:00:00 +0000
+++ community-server/server.py	2015-04-13 19:51:02 +0000
@@ -0,0 +1,329 @@
+import re, time, json, codecs, os, fcntl
+from bzrlib.plugin import load_plugins
+load_plugins()
+from bzrlib.branch import Branch
+from bzrlib import workingtree
+from flask import Flask, request, Response
+app = Flask(__name__)
+
+import logging
+from logging import StreamHandler, Formatter
+from logging.handlers import RotatingFileHandler 
+
+file_handler = StreamHandler()
+file_handler.setLevel(logging.DEBUG)
+file_handler.setFormatter(Formatter(
+    '%(asctime)s %(levelname)s: %(message)s '
+    '[in %(pathname)s:%(funcName)s:%(lineno)d]'
+))
+app.logger.addHandler(file_handler)
+
+log_file_handler = RotatingFileHandler(
+    os.path.join(os.path.dirname(__file__), "..", "logs", "server.log"),
+    maxBytes=100000)
+log_file_handler.setLevel(logging.DEBUG)
+log_file_handler.setFormatter(Formatter(
+    '%(asctime)s %(levelname)s: %(message)s '
+    '[in %(pathname)s:%(funcName)s:%(lineno)d]'
+))
+app.logger.addHandler(log_file_handler)
+
+# http://mynthon.net/howto/-/python/python%20-%20logging.SMTPHandler-how-to-use-gmail-smtp-server.txt
+class TlsSMTPHandler(logging.handlers.SMTPHandler):
+    def emit(self, record):
+        """
+        Emit a record.
+ 
+        Format the record and send it to the specified addressees.
+        """
+        try:
+            import smtplib
+            import string # for tls add this line
+            try:
+                from email.utils import formatdate
+            except ImportError:
+                formatdate = self.date_time
+            port = self.mailport
+            if not port:
+                port = smtplib.SMTP_PORT
+            smtp = smtplib.SMTP(self.mailhost, port)
+            msg = self.format(record)
+            msg = "From: %s\r\nTo: %s\r\nSubject: %s\r\nDate: %s\r\n\r\n%s" % (
+                            self.fromaddr,
+                            string.join(self.toaddrs, ","),
+                            self.getSubject(record),
+                            formatdate(), msg)
+            if self.username:
+                smtp.ehlo() # for tls add this line
+                smtp.starttls() # for tls add this line
+                smtp.ehlo() # for tls add this line
+                smtp.login(self.username, self.password)
+            smtp.sendmail(self.fromaddr, self.toaddrs, msg)
+            smtp.quit()
+        except (KeyboardInterrupt, SystemExit):
+            raise
+        except:
+            self.handleError(record)
+
+logger = logging.getLogger()
+
+try:
+    fp = open("gmailconf.json")
+    maildata = json.load(fp)
+    fp.close()
+    if "mailfrom" not in maildata:
+        raise Exception("bad mail data, so don't add a mail log")
+    gm = TlsSMTPHandler(
+        ("smtp.gmail.com", 587), 
+        maildata["mailfrom"],
+        [maildata["mailto"]],
+        "Server error with ucs submit",
+        (maildata["username"], maildata["password"])
+    )
+    gm.setFormatter(Formatter('''
+    Message type:       %(levelname)s
+    Location:           %(pathname)s:%(lineno)d
+    Module:             %(module)s
+    Function:           %(funcName)s
+    Time:               %(asctime)s
+
+    Message:
+
+    %(message)s
+    '''))
+    gm.setLevel(logging.ERROR)
+    app.logger.addHandler(gm)
+except:
+    pass
+
+# location of master manifest. Note: this MUST point at community_components.json in a
+#  bzr branch which was fetched with "bzr checkout", NOT "bzr branch", so that a 
+#  "bzr commit" writes the contents back to Launchpad with appropriate authentication.
+CCJSON = os.path.join(os.path.dirname(__file__), "..", "..", "community-components-json", "community_components.json")
+
+# https://github.com/kennethreitz/flask-sslify/blob/master/flask_sslify.py
+from flask import request, redirect, current_app
+YEAR_IN_SECS = 31536000
+class SSLify(object):
+    """Secures your Flask App."""
+
+    def __init__(self, app=None, age=YEAR_IN_SECS, subdomains=False, permanent=False):
+        self.hsts_age = age
+        self.hsts_include_subdomains = subdomains
+        self.permanent = permanent
+
+        if app is not None:
+            self.init_app(app)
+
+    def init_app(self, app):
+        """Configures the configured Flask app to enforce SSL."""
+
+        app.before_request(self.redirect_to_ssl)
+        app.after_request(self.set_hsts_header)
+
+    @property
+    def hsts_header(self):
+        """Returns the proper HSTS policy."""
+        hsts_policy = 'max-age={0}'.format(self.hsts_age)
+
+        if self.hsts_include_subdomains:
+            hsts_policy += '; includeSubDomains'
+
+        return hsts_policy
+
+    def redirect_to_ssl(self):
+        """Redirect incoming requests to HTTPS."""
+        # Should we redirect?
+        criteria = [
+            request.is_secure,
+            current_app.debug,
+            request.headers.get('X-Forwarded-Proto', 'http') == 'https'
+        ]
+
+        if not any(criteria):
+            if request.url.startswith('http://'):
+                url = request.url.replace('http://', 'https://', 1)
+                code = 302
+                if self.permanent:
+                    code = 301
+                r = redirect(url, code=code)
+
+                return r
+
+    def set_hsts_header(self, response):
+        """Adds HSTS header to each response."""
+        if request.is_secure:
+            response.headers.setdefault('Strict-Transport-Security', self.hsts_header)
+        return response
+
+#https://github.com/derpston/python-simpleflock/blob/master/src/simpleflock.py
+class SimpleFlock:
+   """Provides the simplest possible interface to flock-based file locking. 
+      Intended for use with the `with` syntax. It will create/truncate/delete 
+      the lock file as necessary."""
+
+   def __init__(self, path, timeout = None):
+      self._path = path
+      self._timeout = timeout
+      self._fd = None
+
+   def __enter__(self):
+      self._fd = os.open(self._path, os.O_CREAT)
+      start_lock_search = time.time()
+      while True:
+         try:
+            fcntl.flock(self._fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
+            # Lock acquired!
+            return
+         except IOError, ex:
+            if ex.errno != errno.EAGAIN: # Resource temporarily unavailable
+               raise
+            elif self._timeout is not None and time.time() > (start_lock_search + self._timeout):
+               # Exceeded the user-specified timeout.
+               raise
+         
+         # TODO It would be nice to avoid an arbitrary sleep here, but spinning
+         # without a delay is also undesirable.
+         time.sleep(0.1)
+
+   def __exit__(self, *args):
+      fcntl.flock(self._fd, fcntl.LOCK_UN)
+      os.close(self._fd)
+      self._fd = None
+
+      # Try to remove the lock file, but don't try too hard because it is
+      # unnecessary. This is mostly to help the user see whether a lock
+      # exists by examining the filesystem.
+      try:
+         os.unlink(self._path)
+      except:
+         pass
+
+
+def process_lpurl(lpurl, username):
+    try:
+        manifest_file_name = "ubuntu_component_store.json"
+        yield "ok:Checking Launchpad branch %s\n" % (lpurl,)
+        try:
+            b = Branch.open(lpurl)
+        except:
+            msg = "error:Unable to read LP branch %s\n" % (lpurl,)
+            app.logger.warn(msg)
+            yield msg
+            return
+        tree = b.basis_tree()
+        manifest_file_id = tree.path2id(manifest_file_name)
+        if not manifest_file_id:
+            msg = "error:Unable to find the component manifest %s/%s.\n" % (lpurl, manifest_file_name)
+            app.logger.warn(msg)
+            yield msg
+            return
+        try:
+            manifest_data = tree.get_file_text(manifest_file_id)
+        except:
+            msg = "error:Unable to read the component manifest %s/%s\n" % (lpurl, manifest_file_name)
+            app.logger.warn(msg)
+            yield msg
+            return
+        try:
+            manifest = json.loads(manifest_data)
+        except:
+            msg = "error:Component manifest %s/%s seems corrupt\n" % (lpurl, manifest_file_name)
+            app.logger.warn(msg)
+            yield msg
+            return
+        if "name" not in manifest:
+            msg = "error:Component manifest %s/%s does not specify a component 'name' field\n" % (lpurl, manifest_file_name)
+            app.logger.warn(msg)
+            yield msg
+            return
+        if not re.match("^[A-Za-z0-9_]+$", manifest["name"]):
+            msg = "error:Component manifest %s/%s specifies an invalid 'name' field (it must match [A-Za-z0-9_]+)\n" % (lpurl, manifest_file_name)
+            app.logger.warn(msg)
+            yield msg
+            return
+        if "version" not in manifest:
+            msg = "error:Component manifest %s/%s does not specify a component 'version' field\n" % (lpurl, manifest_file_name)
+            app.logger.warn(msg)
+            yield msg
+            return
+        if not re.match("^[0-9]+\.[0-9]+$", manifest["version"]):
+            msg = "error:Component manifest %s/%s specifies an invalid 'version' field (it must be a number such as 1.0)\n" % (lpurl, manifest_file_name)
+            app.logger.warn(msg)
+            yield msg
+            return
+        has_binary = False; has_qml = False
+        if tree.path2id("qmllib"):
+            has_binary = True
+        if tree.path2id("qml"):
+            has_qml = True
+        if not has_binary and not has_qml:
+            msg = "error:Component has neither 'qmllib' nor 'qml' directories\n"
+            app.logger.warn(msg)
+            yield msg
+            return
+        yield "ok:Checks passed OK\n"
+        pkg_data = {
+            "name": "%s/%s" % (username, manifest["name"]),
+            "revno": b.revno(),
+            "description": manifest.get("description"),
+            "lpurl": lpurl,
+            "type": [],
+            "version": manifest["version"]
+        }
+        if has_binary: pkg_data["type"].append("binary")
+        if has_qml: pkg_data["type"].append("qml")
+        yield "ok:Calculated package summary data\n"
+
+        # take an exclusive lock on the lockfile to make sure we don't collide with another worker
+        with SimpleFlock("lockfile"):
+            try:
+                fp = codecs.open(CCJSON, encoding="utf8")
+                comps = json.load(fp)
+                fp.close()
+            except:
+                msg = "error:Unable to save summary data\n"
+                app.logger.warn(msg)
+                yield msg
+                return
+            if "components" not in comps:
+                yield "Master component list is corrupt\n"
+                return
+            if pkg_data["name"] in comps["components"]:
+                change_verb = "updated"
+            else:
+                change_verb = "added"
+            comps["components"][pkg_data["name"]] = pkg_data
+            fp = codecs.open(CCJSON, encoding="utf8", mode="w")
+            json.dump(comps, fp, indent=2)
+            fp.close()
+            yield "ok:Updating master record\n"
+            wt, relpath = workingtree.WorkingTree.open_containing(CCJSON)
+            commsg = "Server %s component %s %s" % (change_verb, pkg_data["name"], time.asctime(),)
+            wt.commit(message=commsg, specific_files=[os.path.split(CCJSON)[1]])
+        yield "ok:Component %s %s successfully\n" % (pkg_data["name"], change_verb)
+    except:
+        app.logger.exception("Submit error on url %s" % (lpurl,))
+        msg = "error:Server error\n"
+        yield msg
+
[e-mail]("/submit", methods=["POST"])
+def submit():
+    lpurl = request.form.get("lpurl")
+    if not lpurl:
+        logging.warn("bad request")
+        return "error:No lpurl specified\n", 400
+    m = re.match("^lp:~(?P<username>[^/]+)/(?P<project>[^/]+)/(?P<branch>[^/]+)$", lpurl)
+    if not m:
+        return "error:Malformed lp URL (should be lp:~username/project/branch, not lp:project)\n", 400
+    return Response(process_lpurl(lpurl, m.groupdict()["username"]), mimetype="text/plain")
+
[e-mail]("/")
+def root():
+    return "The Ubuntu community component store server."
+
+SSLify(app)
+
+if __name__ == "__main__":
+    app.debug = True
+    app.run()
\ No newline at end of file

=== added directory 'curated-store'
=== renamed directory 'ComponentStore' => 'curated-store/ComponentStore'
=== renamed file 'ComponentStore.apparmor' => 'curated-store/ComponentStore.apparmor'
=== renamed file 'ComponentStore.desktop' => 'curated-store/ComponentStore.desktop'
=== renamed file 'ComponentStore.png' => 'curated-store/ComponentStore.png'
=== renamed file 'ComponentStore.qmlproject' => 'curated-store/ComponentStore.qmlproject'
=== renamed directory 'GallerySRC' => 'curated-store/GallerySRC'
=== renamed file 'main.qml' => 'curated-store/main.qml'
=== renamed file 'manifest.json' => 'curated-store/manifest.json'
=== renamed directory 'tests' => 'curated-store/tests'
=== added file 'docs/contribute-community.rst'
--- docs/contribute-community.rst	1970-01-01 00:00:00 +0000
+++ docs/contribute-community.rst	2015-04-13 19:51:02 +0000
@@ -0,0 +1,144 @@
+Contributing a component to the Community store
+===============================================
+
+To publish an Ubuntu SDK component to the Community store, you will need a 
+`Launchpad account <https://help.launchpad.net/YourAccount/NewAccount>`_, and you will need to be 
+familiar with `using bzr to push code to Launchpad <https://help.launchpad.net/Code/UploadingABranch>`_.
+
+Community components can be pure QML, or they can be compiled binary components. 
+
+Publishing a pure QML component
+-------------------------------
+
+Create a branch on Launchpad (in any project of your choice) which is named how you plan to name 
+the component, with a top-level *qml* folder, and put your QML file and any required assets for 
+your component in that qml folder. Add an `ubuntu_component_store.json`_ file. Then, submit the 
+component to the community store with *ucs submit lp:~username/project/branch*.
+
+So, your branch should look like: ::
+
+    /qml/
+         MyComponent.qml
+         required_image.png
+         included_script.js
+    /ubuntu_component_store.json
+
+If your component does not have any external assets, it is fine to have a branch 
+with *qml/MyComponent.qml* and nothing else in it. Your branch may also contain any other files 
+you choose outside the *qml* top-level folder; these files are not installed with your component, 
+but may provide useful guidance, READMEs, or other code for people who want to make changes to 
+the component and contribute them back to you.
+
+The name of the component itself is defined by `ubuntu_component_store.json`_; the QML filename 
+is what names the QML Item that it provides. So, if you are Launchpad user *sil*, and 
+you push your component to *lp:~sil/SomeProject/mything*, it defines its own name 
+in `ubuntu_component_store.json`_ as *UsefulComponent*, and it 
+contains *qml/RedRectangle.qml*, an app developer will use it like this: ::
+
+    import QtQuick 2.0
+    import ubuntu_component_store.sil.UsefulComponent 1.0
+    ....
+    MainView {
+        RedRectangle {
+            ....
+        }
+    }
+
+Most components should have component name and QML file name be the same thing to avoid confusion.
+
+Do not publish two unrelated QML components in one UCS component; publish them separately, so 
+they can be used separately.
+
+.. note:: if you do not want to create a whole Launchpad project just for this component, you can push to a Launchpad "junk" branch: *lp:~username/+junk/somename*
+
+Once your branch is created, publish it to the Community store with ::
+
+    $ ucs submit lp:~username/project/somename
+    (submitting to community repository)
+    Checking Launchpad branch lp:~username/project/somename
+    Checks passed OK
+    Calculated package summary data
+    Updating master record
+    Component username/ComponentName updated successfully
+
+It should then be visible to *ucs search*.
+
+Publishing a compiled component
+-------------------------------
+
+Publishing a compiled component is a little more complicated, for CPU architecture reasons. Your component must 
+be `an Extension Plugin <http://doc.qt.io/qt-5/qtqml-tutorials-extending-qml-example.html#chapter-6-writing-an-extension-plugin>`_, "a 
+plugin library to make it available to the QML engine as a new QML import module". Creating such a plugin is 
+currently beyond the scope of this document. (We are hoping to provide an Ubuntu SDK IDE template 
+for creating such components with the proper filesystem layout; before then, the "App with QML Extension 
+Library" option creates an appropriate type of component.)
+
+You *must* compile your component for three different architectures: ARM, x86, and amd64 (for Ubuntu phones, 
+the desktop emulator, and Ubuntu desktops). Once you have compiled it as such, you will have 
+three different *libMyComponent.so* files.
+
+Assemble a Launchpad branch with a top-level *qmllib* folder, and in it put a folder for each architecture, 
+named for the GNU architecture triplet, and then the *.so* file within. So: ::
+
+    /qmllib
+        /x86_64-linux-gnu
+                         /libMyComponent.so
+        /i386-linux-gnu
+                         /libMyComponent.so
+        /arm-linux-gnueabihf
+                         /libMyComponent.so
+    /ubuntu_component_store.json
+
+Add an `ubuntu_component_store.json`_ file to the root of the branch.
+
+Your branch may contain any other files of your choice outside the */qmllib* folder; in particular, it 
+should contain the source code for the plugin so others can build it themselves if they choose!
+
+The name of the component itself is defined by `ubuntu_component_store.json`_; your component is expected to 
+use `qmlRegisterType <http://doc.qt.io/qt-5/qqmlextensionplugin.html#details>`_ to provide QML Item types. 
+So, if you are Launchpad user *sil*, and you push your component to *lp:~sil/SomeProject/Whatever*, 
+it defines its name in `ubuntu_component_store.json`_ as *SomeComponent*, and it registers 
+a *Triangle* type, an app developer will use it like this: ::
+
+    import QtQuick 2.0
+    import ubuntu_component_store.sil.SomeComponent 1.0
+    ....
+    MainView {
+        Triangle {
+            ....
+        }
+    }
+
+Do not publish two unrelated components in one UCS component; publish them separately, so they can 
+be used separately.
+
+.. note:: if you do not want to create a whole Launchpad project just for this component, you can push to a Launchpad "junk" branch: *lp:~username/+junk/somename*
+
+Once your branch is created, publish it to the Community store with ::
+
+    $ ucs submit lp:~username/project/somename
+    (submitting to community repository)
+    Checking Launchpad branch lp:~username/project/somename
+    Checks passed OK
+    Calculated package summary data
+    Updating master record
+    Component username/ComponentName updated successfully
+
+It should then be visible to *ucs search*.
+
+A component can contain both *qml* and *qmllib* folders and so contain both QML parts and binary parts; both will be installed when a developer uses *ucs install* to install your component.
+
+ubuntu_component_store.json
+===========================
+
+A community component must contain a file *ubuntu_component_store.json* describing metadata about the component.
+It must be valid JSON, with keys *name*, *version*, and *description*: ::
+
+    {
+        "name": "GenericPodcastApp",
+        "version": "1.0",
+        "description": "A component which manages a podcast RSS feed, with playback and display of episodes"
+    }
+
+Other keys may be added in future. Note that the public name of the component will be launchpadusername/name, 
+where *name* is the name taken from *ubuntu_component_store.json*. The branch name in Launchpad is ignored.

=== added file 'docs/contribute-curated.rst'
--- docs/contribute-curated.rst	1970-01-01 00:00:00 +0000
+++ docs/contribute-curated.rst	2015-04-13 19:51:02 +0000
@@ -0,0 +1,120 @@
+Contributing a component to the Curated store
+=============================================
+
+Ubuntu Component Store is intended to be a community project where app developers are
+encouraged to contribute any component that they think would be useful to other developers.
+To contribute a component to UCS, we recommend first joining the team by applying `here`_.
+This would then make it possible for you to maintain your component by pushing bug fixes and
+improvements as and when you choose.
+
+By moderating the members of the team and components in UCS, we can promise a well tested
+component with a stable API. We recognize that this can impede the rate at which new components
+can be added to UCS, but we feel that this is an acceptable drawback to ensure good quality
+components to our userbase.
+
+Let's go through the steps required to upload a new component to the store.
+
+Getting Started
+---------------
+
+You can technically contribute new components to the store without being a member of the team by
+requesting someone from the team to maintain your component. However we highly recommend that you
+join the team since that would allow you to push bug fixes and improvements to your component
+quickly. You can apply to become a member by applying `here`_. The approval should be done within a
+day or two.
+
+UCS is hosted on launchpad and requires bazaar (bzr) to grab or push the code. As such you would
+need to have an account on launchpad and be familiar with bzr. You can grab the code by, ::
+
+    bzr branch lp:component-store
+    
+UCS by itself is an ordinary QML project. You should be able to open it using qtcreator like any other
+project. Run the UCS Gallery app by either pressing the green button in qtcreator or via the command line
+as shown below, ::
+
+    qmlscene main.qml
+
+This should open the UCS Gallery app as shown below. It provides a visual overview of all the components
+in the store.
+
+.. image:: _images/gallery.png
+   :align: center
+   :scale: 90 % 
+   
+.. note:: At the time of writing this documentation, you would need a 14.10 desktop to run the component store gallery. You can run the gallery app on the phone or emulator using a 14.04 desktop, however the gallery app doesn't converge well on the phone yet. This should be fixed soon.
+   
+Adding a new component to the store
+-----------------------------------
+
+Adding a new component to the store involves 3 main steps which are,
+
+1. Adding a new component to the ComponentStore folder
+2. Updating the gallery app to showcase your new component
+3. Updating the documentation
+
+Adding a new component to the ComponentStore Folder
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The components are stored in their own folders in the ComponentStore folder. Let's assume for illustration purposes, 
+we are trying to add a new component *MyNewComponent* to UCS. Let's grab UCS first and then add the new component. ::
+
+    bzr branch lp:component-store MyNewComponentBranch
+
+Let create a new folder for our component in the ComponentStore folder, ::
+
+    cd MyNewComponentBranch/ComponentStore
+    mkdir MyNewComponent && bzr add MyNewComponent
+    cd MyNewComponent
+    
+Inside your *MyNewComponent* folder, add your generic component files. Everything that is required to use your component
+must be included in this folder. Let's now add these files to the source control bzr and commit them. ::
+
+    bzr add *
+    bzr commit -m "Added my new component"
+
+Updating the gallery app
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Now that we have our component ready and added, let's update the gallery app to show that. Create a new file in the GallerySRC
+folder called MyNewComponentWidget.qml. This file is rather easy to fill out. Just copy the code from the other widget template
+files. It basically involves filling in information like the author, license, contact and a description of your component which
+is rather simple. Once completed, add MyNewComponentWidget.qml to the version control. ::
+
+    cd GallerySRC
+    bzr add MyNewComponentWidget.qml
+    bzr commit -m "Showcase new component in the gallery"
+
+Next, open WidgetsModel.qml and add your component to the list. That's it! Run the gallery app and see if your component shows up
+as expected.
+
+Updating the documentation
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This is one of the most important steps and benefits of adding your component to UCS. By providing a well written and clear documentation
+you make it easier for other app developers to use your component in their app. All the documentation is hosted in the docs folder. Create
+a new documentation file in the _components folder and fill in the necessary documentation. Use the existing documentation files to help you
+with setting it up. 
+
+Once done, add your component to the list in index.rst. Now let's test if the documentation looks good. To build the documentation, you
+need sphinx and pip packages to be installed. Let's install that for the first time. ::
+
+    sudo apt-get install pip
+    sudo pip install sphinx sphinx-autobuild
+    
+Building your documentation is now really simple. Once the build process is complete, open _build/index.html in your browser to see your documentation. ::
+
+    make html
+
+Once again, let's add this to the version control. ::
+
+    cd docs/_components
+    bzr add MyNewComponent.rst
+    bzr commit -m "Added documentation for the new component"
+    
+That was it! Your component is ready. Let's push this online to UCS. ::
+
+    bzr push lp:~launchpad-id/component-store/MyNewComponentBranch
+    
+From there on, it is just a matter of reviewing the code (for new components, we try to ensure everything is in order) and then merging to trunk.
+
+.. _here: https://launchpad.net/~ubuntu-touch-community-dev

=== modified file 'docs/contribute.rst'
--- docs/contribute.rst	2014-11-08 21:43:43 +0000
+++ docs/contribute.rst	2015-04-13 19:51:02 +0000
@@ -1,120 +1,13 @@
 Contributing to UCS
 ===================
 
-Ubuntu Component Store is intended to be a community project where app developers are
-encouraged to contribute any component that they think would be useful to other developers.
-To contribute a component to UCS, we recommend first joining the team by applying `here`_.
-This would then make it possible for you to maintain your component by pushing bug fixes and
-improvements as and when you choose.
-
-By moderating the members of the team and components in UCS, we can promise a well tested
-component with a stable API. We recognize that this can impede the rate at which new components
-can be added to UCS, but we feel that this is an acceptable drawback to ensure good quality
-components to our userbase.
-
-Let's go through the steps required to upload a new component to the store.
-
-Getting Started
----------------
-
-You can technically contribute new components to the store without being a member of the team by
-requesting someone from the team to maintain your component. However we highly recommend that you
-join the team since that would allow you to push bug fixes and improvements to your component
-quickly. You can apply to become a member by applying `here`_. The approval should be done within a
-day or two.
-
-UCS is hosted on launchpad and requires bazaar (bzr) to grab or push the code. As such you would
-need to have an account on launchpad and be familiar with bzr. You can grab the code by, ::
-
-    bzr branch lp:component-store
-    
-UCS by itself is an ordinary QML project. You should be able to open it using qtcreator like any other
-project. Run the UCS Gallery app by either pressing the green button in qtcreator or via the command line
-as shown below, ::
-
-    qmlscene main.qml
-
-This should open the UCS Gallery app as shown below. It provides a visual overview of all the components
-in the store.
-
-.. image:: _images/gallery.png
-   :align: center
-   :scale: 90 % 
-   
-.. note:: At the time of writing this documentation, you would need a 14.10 desktop to run the component store gallery. You can run the gallery app on the phone or emulator using a 14.04 desktop, however the gallery app doesn't converge well on the phone yet. This should be fixed soon.
-   
-Adding a new component to the store
------------------------------------
-
-Adding a new component to the store involves 3 main steps which are,
-
-1. Adding a new component to the ComponentStore folder
-2. Updating the gallery app to showcase your new component
-3. Updating the documentation
-
-Adding a new component to the ComponentStore Folder
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The components are stored in their own folders in the ComponentStore folder. Let's assume for illustration purposes, 
-we are trying to add a new component *MyNewComponent* to UCS. Let's grab UCS first and then add the new component. ::
-
-    bzr branch lp:component-store MyNewComponentBranch
-
-Let create a new folder for our component in the ComponentStore folder, ::
-
-    cd MyNewComponentBranch/ComponentStore
-    mkdir MyNewComponent && bzr add MyNewComponent
-    cd MyNewComponent
-    
-Inside your *MyNewComponent* folder, add your generic component files. Everything that is required to use your component
-must be included in this folder. Let's now add these files to the source control bzr and commit them. ::
-
-    bzr add *
-    bzr commit -m "Added my new component"
-
-Updating the gallery app
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-Now that we have our component ready and added, let's update the gallery app to show that. Create a new file in the GallerySRC
-folder called MyNewComponentWidget.qml. This file is rather easy to fill out. Just copy the code from the other widget template
-files. It basically involves filling in information like the author, license, contact and a description of your component which
-is rather simple. Once completed, add MyNewComponentWidget.qml to the version control. ::
-
-    cd GallerySRC
-    bzr add MyNewComponentWidget.qml
-    bzr commit -m "Showcase new component in the gallery"
-
-Next, open WidgetsModel.qml and add your component to the list. That's it! Run the gallery app and see if your component shows up
-as expected.
-
-Updating the documentation
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-This is one of the most important steps and benefits of adding your component to UCS. By providing a well written and clear documentation
-you make it easier for other app developers to use your component in their app. All the documentation is hosted in the docs folder. Create
-a new documentation file in the _components folder and fill in the necessary documentation. Use the existing documentation files to help you
-with setting it up. 
-
-Once done, add your component to the list in index.rst. Now let's test if the documentation looks good. To build the documentation, you
-need sphinx and pip packages to be installed. Let's install that for the first time. ::
-
-    sudo apt-get install pip
-    sudo pip install sphinx sphinx-autobuild
-    
-Building your documentation is now really simple. Once the build process is complete, open _build/index.html in your browser to see your documentation. ::
-
-    make html
-
-Once again, let's add this to the version control. ::
-
-    cd docs/_components
-    bzr add MyNewComponent.rst
-    bzr commit -m "Added documentation for the new component"
-    
-That was it! Your component is ready. Let's push this online to UCS. ::
-
-    bzr push lp:~launchpad-id/component-store/MyNewComponentBranch
-    
-From there on, it is just a matter of reviewing the code (for new components, we try to ensure everything is in order) and then merging to trunk.
-
-.. _here: https://launchpad.net/~ubuntu-touch-community-dev
+There are two sorts of components in the Ubuntu Component Store: *Curated* components and *Community* components. As a  developer, you can publish your component to either. Components in the Curated store have higher requirements to ensure that they are high-quality; anyone can publish to the Community store without approval.
+
+Contributing a component
+------------------------
+
+.. toctree::
+   :maxdepth: 1
+
+   contribute-curated
+   contribute-community

=== modified file 'docs/index.rst'
--- docs/index.rst	2015-03-03 14:06:38 +0000
+++ docs/index.rst	2015-04-13 19:51:02 +0000
@@ -6,7 +6,7 @@
 Welcome to the Ubuntu Component Store
 ==================================================
 
-The Ubuntu Component Store (UCS) is intended to be a library hosting a collection of well tested and trusted components created by community developers. 
+The Ubuntu Component Store (UCS) is intended to be a library hosting a collection of components created by community developers for building Ubuntu SDK apps. 
 UCS will enable new developers by providing plug-and-play components/modules and thereby preventing the need to start from scratch.
 
 Purpose
@@ -26,11 +26,13 @@
 How will this work?
 -------------------
 
-A developer who just wants to use these components will be able to grab the components easily and keep them updated by simply running the ucs program. On the 
-other hand, if a developer wants to contribute his component, then he/she will be required to join the team and maintain his component. This way, any bug fixes 
-or improvements to the components can be done quickly by the original author of that component.
- 
-You can find the installation guide at :doc:`install`
+A developer who just wants to use these components will be able to grab the components easily and keep them updated by simply running the ucs program. 
+
+You can find the installation guide, to install *ucs* so that you can use it to add components to your Ubuntu SDK apps, at :doc:`install`.
+
+A component developer can create a component and contribute it to the Ubuntu Component Store. There are two types of components: *Curated* and *Community*. A component developer can publish their component to the Community store by themselves, with no oversight. Publishing to the Curated store requires approval that the curated component is well-constructed: it must have good documentation and tests, and the component developer will join the store team and maintail their component. This leads to the Curated components being superior.
+
+If you want to contribute a component, you can find the contribution guide at :doc:`contribute`.
 
 Main Contents
 -------------

=== modified file 'docs/install.rst'
--- docs/install.rst	2014-11-12 12:03:47 +0000
+++ docs/install.rst	2015-04-13 19:51:02 +0000
@@ -1,27 +1,54 @@
 Installation Guide
 ==================
 
-UCS components should be bundled inside the application's click package. Importing
-the UCS components in your app is as simple as installing the *ucs* package and running
-it from your application's components folder. To install *ucs*, we need to first add a PPA ::
+Installing UCS itself
+---------------------
+
+Before using *ucs* to install components, you first need to install *ucs* itself!
+
+To install *ucs*, we need to add a PPA ::
 
     sudo add-apt-repository ppa:ubuntu-touch-community-dev/ppa
     sudo apt-get update
     sudo apt-get install ucs
-    
+
+Finding Components
+------------------
+
+Use the *search* command to search for available components by name or description.::
+
+    $ ucs search state
+    Matching curated components:
+                          EmptyState  
+    Matching community components:
+                        (no matches)
+    $ ucs search podcast
+    Matching curated components:
+                        (no matches)
+    Matching community components:
+               sil/GenericPodcastApp  A component which manages a podcast RSS feed, with playback and display of episodes
+                       
+
+You can list all components in the store, both curated and community components, with *ucs search*.
+
 Installing Components
 ---------------------
 
 You can install components individually using the *install* command, ::
 
-    ucs install EmptyState PageWithBottomEdge
+    ucs install EmptyState
+    ucs install PageWithBottomEdge
+    ucs install sil/GenericPodcastApp
 
 .. note:: Remember to run the *ucs* command from inside your application's component folder!   
-.. note:: You can install all components in one go by ``ucs install``
 
 You can then use the components by importing it in your qml files by, ::
 
-    import "path/to/components/folder"
+    import ubuntu_component_store.Curated.EmptyState 1.0
+    import ubuntu_component_store.Curated.PageWithBottomEdge 1.0
+    import ubuntu_component_store.sil.GenericPodcastApp 1.0
+
+.. note:: Community components are named as *developer/ComponentName*. Curated components are installed with just a ComponentName (like *ucs install EmptyState*), but when importing them in QML you must call them *ubuntu_component_store.Curated.EmptyState* as above.
 
 Updating Components
 -------------------
@@ -29,7 +56,7 @@
 You can also update the UCS components using the *ucs* script. It will automatically fetch the latest
 upstream code. ::
 
-    ucs update EmptyState PageWithBottomEdge
+    ucs update EmptyState
     
 .. note:: Remember to run the *ucs* command from inside your application's component folder!
   

=== modified file 'script/debian/changelog'
--- script/debian/changelog	2014-11-09 15:37:52 +0000
+++ script/debian/changelog	2015-04-13 19:51:02 +0000
@@ -1,3 +1,15 @@
+ucs (0.2) trusty; urgency=medium
+
+  [Stuart Langridge]
+  * Added community channel which allows developers to submit components
+    easily to ucs.
+  * Converted ucs script to python
+
+  [Nekhelesh Ramananthan]
+  * Fixed UCS Gallery app and debian packaging
+
+ -- Nekhelesh Ramananthan <[e-mail]>  Tue, 06 Jan 2015 15:57:14 +0100
+
 ucs (0.1.2) trusty; urgency=medium
 
   * Improved install and update command to use less network bandwidth by only

=== modified file 'script/debian/control'
--- script/debian/control	2014-11-08 22:37:17 +0000
+++ script/debian/control	2015-04-13 19:51:02 +0000
@@ -10,7 +10,8 @@
 Package: ucs
 Architecture: all
 Depends: ${misc:Depends},
-	 bzr
+	 bzr,
+	 python-bzrlib
 Description: Ubuntu Component Store
  Ubuntu Component Store is a library hosting a collection of component and
  modules that app developers can use as plug-and-play components instead of

=== added file 'script/ucs'
--- script/ucs	1970-01-01 00:00:00 +0000
+++ script/ucs	2015-04-13 19:51:02 +0000
@@ -0,0 +1,531 @@
+#!/usr/bin/python
+# has to be Python2 because there's no python3 bzrlib
+
+import sys, argparse, os, time, codecs, json, urllib, urllib2, re, subprocess
+from gi.repository import GLib
+from bzrlib.plugin import load_plugins
+load_plugins()
+from bzrlib.branch import Branch
+
+class Project(object):
+    def __init__(self, rootfolder):
+        self.rootfolder = rootfolder
+
+    def add_to_import_path(self, component_type):
+        pass
+
+    def copy_branch_folder_contents_to(self, branch, tree, folder, destfolder):
+        # Ensure destfolder exists
+        try:
+            os.makedirs(destfolder)
+        except OSError, e:
+            if e.errno == 17:
+                pass
+            else:
+                raise
+        # Copy immediate file children of folder to it (not child folders or their contents)
+        copied_files = []
+        for path, versioned, kind, file_id, inventory_entry in tree.list_files(from_dir=folder, recursive=False):
+            if kind == "file":
+                fp = open(os.path.join(destfolder, path), "wb")
+                fp.write(tree.get_file_text(file_id))
+                fp.close()
+                copied_files.append(path)
+        return copied_files
+
+    def install_component_from_branch_folder(self, branch, tree, folder, component, component_type):
+        if component_type == "qml":
+            destfolder = os.path.join(self.rootfolder, "ubuntu_component_store", 
+                component["name"] + "." + component["version"])
+            copied_files = self.copy_branch_folder_contents_to(branch, tree, folder, destfolder)
+            # create qml qmldir file
+            qmlfiles = []
+            for f in copied_files:
+                if f.endswith(".qml"):
+                    qmlfiles.append("%s %s %s" % (os.path.splitext(f)[0], component["version"], f))
+            if qmlfiles:
+                username, cname = component["name"].split("/")
+                cname = cname[0].upper() + cname[1:]
+                qmldir = ["module ubuntu_component_store.%s.%s" % (username, cname)]
+                qmldir += qmlfiles
+                qmldirfn = os.path.join(destfolder, "qmldir")
+                fp = codecs.open(qmldirfn, mode="w", encoding="utf8")
+                fp.write("\n".join(qmldir))
+                fp.write("\n")
+                fp.close()
+        elif component_type == "binary":
+            fid = tree.path2id(folder)
+            if not fid:
+                raise Exception("Component has no '%s' folder" % (folder,))
+            arches = []
+            for path, versioned, kind, file_id, inventory_entry in tree.list_files(from_dir=folder, recursive=False):
+                if kind == "directory":
+                    arches.append(path)
+            for arch in arches:
+                destfolder = os.path.join(self.rootfolder, "lib", arch, "ubuntu_component_store", 
+                    component["name"] + "." + component["version"])
+                copied_files = self.copy_branch_folder_contents_to(branch, tree, folder + "/" + arch, destfolder)
+                # create binary qmldir file
+                plugin_names = []
+                for f in copied_files:
+                    m = re.match("^lib(?P<libname>[^z.]+)\.so", f)
+                    if m:
+                        plugin_names.append(m.groupdict()["libname"])
+                if plugin_names:
+                    username, cname = component["name"].split("/")
+                    cname = cname[0].upper() + cname[1:]
+                    qmldir = ["module ubuntu_component_store.%s.%s" % (username, cname)]
+                    qmldir += ["plugin %s" % x for x in plugin_names]
+                    qmldirfn = os.path.join(destfolder, "qmldir")
+                    fp = codecs.open(qmldirfn, mode="w", encoding="utf8")
+                    fp.write("\n".join(qmldir))
+                    fp.write("\n")
+                    fp.close()
+        else:
+            raise Exception("Unknown component type '%s'" % (component_type,))
+
+    def add_to_import_path(self, component_type):
+        raise NotImplementedError
+
+    def add_to_project_specific_copy_path(self, folder):
+        raise NotImplementedError
+
+    def add_to_project_specific_import_path(self, folder):
+        raise NotImplementedError
+
+
+class PureQMLProject(Project):
+    def add_to_project_specific_import_path(self, folder):
+        "Add the passed folder to root/projname.qmlproject:importPaths"
+        projname = os.path.split(os.path.abspath(self.rootfolder))[1]
+        qmlprojfile = os.path.join(self.rootfolder, "%s.qmlproject" % projname)
+        if not os.path.exists(qmlprojfile):
+            raise Exception("Couldn't add import path to %s" % (qmlprojfile,))
+        fp = codecs.open(qmlprojfile, encoding="utf8")
+        data = fp.read()
+        fp.close()
+        ipre = "importPaths\s*:\s*\[(?P<paths>[^]]*)\]"
+        ip = re.search(ipre, data)
+        if not ip:
+            raise Exception("Couldn't add import path to %s" % (qmlprojfile,))
+        pathsline = ip.groupdict()["paths"]
+        paths = [re.sub('^"|"$', '', x.strip()) for x in pathsline.split(",")]
+        if folder in paths:
+            # this folder is already on the importPaths, so bail
+            print "done already"
+            return
+        pathsline += ', "%s"' % (folder,)
+        newpathline = "importPaths: [%s]" % pathsline
+        data = re.sub(ipre, newpathline, data)
+        fp = codecs.open(qmlprojfile, encoding="utf8", mode="w")
+        fp.write(data)
+        fp.close()
+
+    def add_to_project_specific_copy_path(self, folder):
+        "QML projects don't copy anything anywhere; they build in place. So this is not relevant."
+        pass
+
+    def add_to_import_path(self, component_type):
+        if component_type == "binary":
+            # since this is a binary component, add the part for our arch to the Qt Creator import path
+            # FIXME it would be nice to not have to shell out for this
+            ourarch = subprocess.check_output(["dpkg-architecture", "-qDEB_BUILD_GNU_TYPE"]).strip()
+            self.add_to_project_specific_import_path("./lib/%s" % (ourarch,))
+            # FIXME may need an add_to_project_specific_copy_path here too
+        elif component_type == "qml":
+            # qml projects build in place anyway
+            pass
+
+
+class QMakeProject(Project):
+    def add_to_import_path(self, component_type):
+        if component_type == "binary":
+            print ("This is a qmake project.\n"
+                "You will need to ensure that all the ./lib/(arch)/ paths are added to "
+                "your import path when running the project, and that they are included "
+                "in the build process so they are copied into the build folder.")
+        elif component_type == "qml":
+            print ("This is a qmake project.\n"
+                "You will need to ensure that the ubuntu_component_store folder is "
+                "added to your import path when running the project, and that it is "
+                "included in the build process so it is copied into the build folder.")
+
+class CMakeProject(QMakeProject):
+    def add_to_import_path(self, component_type):
+        if component_type == "binary":
+            print ("This is a cmake project.\n"
+                "You will need to ensure that all the ./lib/(arch)/ paths are added to "
+                "your import path when running the project, and that they are included "
+                "in the build process so they are copied into the build folder.")
+        elif component_type == "qml":
+            print ("This is a cmake project.\n"
+                "You will need to ensure that the ubuntu_component_store folder is "
+                "added to your import path when running the project, and that it is "
+                "included in the build process so it is copied into the build folder.")
+
+def get_project():
+    """Identify the current project type and return an appropriate wrapper.
+
+    Pure QML projects are in a folder Foo which has a Foo.qmlproject file in it.
+    qMake-based projects are in a folder Foo which has a Foo.pro file in it.
+    CMake-based projects are in a folder Foo which has CMakeLists.txt and that
+        file contains a project(Foo ...) line in it.
+    """
+
+    def get_class_for_folder(folder):
+        afolder = os.path.abspath(folder)
+        basename = os.path.split(afolder)[1]
+        cmakelistname = os.path.join(afolder, "CMakeLists.txt")
+        if os.path.exists(os.path.join(afolder, "%s.%s" % (basename, "qmlproject"))):
+            return PureQMLProject, afolder
+        elif os.path.exists(os.path.join(afolder, "%s.%s" % (basename, "pro"))):
+            return QMakeProject, afolder
+        elif os.path.exists(cmakelistname):
+            fp = codecs.open(cmakelistname, encoding="utf8")
+            lines = fp.read().split("\n")
+            fp.close()
+            for line in lines:
+                m = re.match("^\s*project\s*\(\s*(?P<project>[^\s]+)((\s+[^\s]+)*\s*)\s*$", line)
+                if m:
+                    if m.groupdict()["project"] == basename:
+                        return CMakeProject, afolder
+        elif os.path.abspath(afolder) == "/":
+            # got to the root and found nothing
+            return None, None
+        return get_class_for_folder(os.path.join(afolder, ".."))
+
+    project_class, root_folder = get_class_for_folder(os.curdir)
+    if not project_class:
+        raise Exception("Not in an Ubuntu SDK app: %s" % (os.curdir))
+    return project_class(root_folder)
+
+class Repository(object):
+    def install_component_from_branch_folder(self, branch, tree, folder, component_type, component):
+        """ Download a thing from an LP folder and install it.
+
+        To install a component we need to:
+        1.  fetch it from LP
+        2.  put it in the correct place. 
+            For pure QML components, the component will be one or more QML files, which
+            are in the branch folder that we are passed, and they get put in: 
+                <project root>/ubuntu_component_store/username/componentname/
+            For binary components, the component will be a libFoo.so file for each arch,
+            and the branch folder we are passed should contain files <arch>/libFoo.so
+            for each arch. We then set up:
+                <project root>/lib/$arch/ubuntu_component_store/username/componentname/
+            for each arch and copy the appropriate libFoo.so into each.
+        3.  Set up qmldir files.
+            Pure QML components get a qmldir file 
+                <project root>/ubuntu_component_store/username/componentname/qmldir
+            with content
+                module RedRectangle
+                RedRectangle 1.0 RedRectangle.qml
+            Binary components (libFoo.so, in component myname/BarPlugin) get a qmldir file
+                <project root>/lib/$arch/ubuntu_component_store/myname/BarPlugin/qmldir
+            with content
+                module ubuntu_component_store.myname.BarPlugin
+                plugin Foo
+        4.  Make sure the components are on the import path.
+            Pure QML components do this automatically; they are on the import path because
+            their folder (ubuntu_component_store/username/componentname/) is in the project
+            root, and that's on the import path. Binary components do not, because the
+            <project root>/lib/$arch folder is *not* on the import path for running in Qt Creator.
+            (It *is* on the import path when packaged as a click and installed on Ubuntu;
+            the click-apparmor wrapper /usr/bin/aa-exec-click sets it.) So it's important to
+            ensure when installing a binary component that it's on the import path. This happens
+            in different ways depending on the type of project:
+
+            Pure QML projects
+            These have a projectname.qmlproject file in the project root, with an importPaths key. We add
+            "./lib/$arch" to it, for the arch that this machine is currently using. This is a 
+            bit dodgy (adding arch-specific paths), but this importPaths key *already* contains
+            arch-specific paths (frex, "/usr/lib/x86_64-linux-gnu/qt5/qml") and so adding a project-specific
+            arch-specific path can't make that any worse.
+
+            CMake-based projects
+            FIXME: dunno yet
+
+            qMake-based projects
+            FIXME: dunno yet
+
+        """
+        project = get_project()
+        project.install_component_from_branch_folder(branch, tree, folder, component, component_type)
+        project.add_to_import_path(component_type)
+
+    def search_component_list(self, query, md):
+        matches = [{"componentname": x[0], "lcn": x[0].lower(), 
+            "description": x[1].get("description", "(none)"),
+            "lcd": x[1].get("description", "").lower()}
+            for x in md.items()]
+        for word in query:
+            word = word.lower()
+            matches2 = []
+            for pkg in matches:
+                if word in pkg["lcn"] or word in pkg["lcd"]:
+                    matches2.append(pkg)
+            matches = matches2
+        return matches
+
+class CuratedRepository(Repository):
+    UCS_BRANCH = "lp:~ubuntu-touch-community-dev/component-store/trunk.14.10"
+    COMPONENTS_ROOT = "curated-store/ComponentStore"
+
+    def __init__(self, args):
+        self.__refresh = args.refresh
+
+    def __get_branch(self):
+        b = Branch.open(self.UCS_BRANCH)
+        tree = b.basis_tree()
+        return b, tree
+
+    def __update_metadata_if_required(self):
+        cache_dir = GLib.get_user_cache_dir()
+        my_cache_dir = os.path.join(cache_dir, "ubuntu_component_store")
+        try:
+            os.makedirs(my_cache_dir)
+        except OSError, e:
+            if e.errno != 17:
+                raise
+
+        cached_curated_file = os.path.join(my_cache_dir, "ucs-curated-components-metadata.json")
+
+        branch, tree = self.__get_branch()
+        fetch_packages = False
+        try:
+            fp = codecs.open(cached_curated_file, encoding="utf8")
+            component_metadata = json.load(fp)
+            if branch.revno() > component_metadata.get("cached_revno", 0):
+                fetch_packages = True
+        except:
+            fetch_packages = True
+
+        if self.__refresh:
+            fetch_packages = True
+            self.__refresh = False
+
+        if fetch_packages:
+            component_metadata = {"cached_revno": branch.revno(), "components": {}}
+            cns = [path
+                for (path, versioned, kind, file_id, inventory_entry) 
+                in tree.list_files(from_dir=self.COMPONENTS_ROOT)
+                if kind == "directory"
+            ]
+            for c in cns:
+                try:
+                    mdpath = "%s/%s/%s" % (self.COMPONENTS_ROOT, c, "ubuntu_component_store.json")
+                    file_id = tree.path2id(mdpath)
+                    mdjson = tree.get_file_text(file_id)
+                    md = json.loads(mdjson)
+                    component_metadata["components"][c] = md
+                except:
+                    print "(unable to get metadata for curated component %s)" % (c,)
+            fp = codecs.open(cached_curated_file, mode="w", encoding="utf8")
+            json.dump(component_metadata, fp)
+            fp.close()
+        return component_metadata
+
+    def install(self, cn):
+        print("(installing {0} from curated repository)".format(cn))
+        component_metadata = self.__update_metadata_if_required()
+        component = component_metadata["components"].get(cn)
+        if not component:
+            raise Exception("Unable to find curated component '%s'." % (cn,))
+        b, tree = self.__get_branch()
+        component_path = "/".join([self.COMPONENTS_ROOT, cn])
+        file_id = tree.path2id(component_path)
+        if not file_id: # should not happen because it's in the metadata
+            raise Exception("Unable to find curated component '%s'." % (cn,))
+        # Curated components are all QML-only
+        component["name"] = "Curated/%s" % (component["name"],)
+        self.install_component_from_branch_folder(b, tree, component_path, "qml", component)
+
+    def update(self, cn):
+        raise NotImplementedError
+
+    def search(self, query):
+        component_metadata = self.__update_metadata_if_required()
+        return self.search_component_list(query, component_metadata.get("components", {}))
+
+class CommunityRepository(Repository):
+    PACKAGES_FILE_BRANCH = "lp:~ubuntu-touch-community-dev/component-store/community-components-json"
+    PACKAGES_FILE_NAME = "community_components.json"
+
+    def __init__(self, args):
+        self.__refresh = args.refresh
+
+    def __update_packages_file_if_required(self):
+        cache_dir = GLib.get_user_cache_dir()
+        my_cache_dir = os.path.join(cache_dir, "ubuntu_component_store")
+        try:
+            os.makedirs(my_cache_dir)
+        except OSError, e:
+            if e.errno != 17:
+                raise
+
+        cached_packages_file = os.path.join(my_cache_dir, "ucs-packages-cache.json")
+
+        fetch_packages = False
+        try:
+            age = time.time() - os.stat(cached_packages_file).st_mtime
+            if age > 3600: fetch_packages = True
+        except OSError:
+            fetch_packages = True
+        if self.__refresh:
+            fetch_packages = True
+            self.__refresh = False
+
+        if fetch_packages:
+            packages_data = self.__fetch_packages_file(cached_packages_file)
+        else:
+            fp = codecs.open(cached_packages_file, encoding="utf8")
+            try:
+                packages_data = json.load(fp)
+            except:
+                raise Exception("Local package cache seems corrupt")
+            fp.close()
+        return packages_data
+
+    def __fetch_packages_file(self, cached_packages_file):
+        print "(getting packages file)"
+        b = Branch.open(self.PACKAGES_FILE_BRANCH)
+        tree = b.basis_tree()
+        file_id = tree.path2id(self.PACKAGES_FILE_NAME)
+        if not file_id:
+            raise Exception("Unable to find the packages file.")
+        try:
+            packages_data = tree.get_file_text(file_id)
+        except:
+            raise Exception("Unable to read the packages file.")
+        try:
+            packages_data = json.loads(packages_data)
+        except:
+            raise Exception("Master packages file seems corrupt")
+        fp = codecs.open(cached_packages_file, mode="w", encoding="utf8")
+        json.dump(packages_data, fp)
+        fp.close()
+        return packages_data
+
+    def install(self, cn):
+        packages_data = self.__update_packages_file_if_required()
+        print("(installing {0} from community repository)".format(cn))
+        pkg = packages_data.get("components", {}).get(cn)
+        if not pkg:
+            print("Package {0} not found".format(cn))
+            return
+        print("OK installing.")
+        b = Branch.open(pkg["lpurl"])
+        tree = b.repository.revision_tree(b.get_rev_id(pkg["revno"]))
+        if "qml" in pkg["type"]:
+            self.install_component_from_branch_folder(b, tree, "qml", "qml", pkg)
+        if "binary" in pkg["type"]:
+            self.install_component_from_branch_folder(b, tree, "qmllib", "binary", pkg)
+
+    def update(self, cn):
+        raise NotImplementedError
+
+    def search(self, query):
+        packages_data = self.__update_packages_file_if_required()
+        return self.search_component_list(query, packages_data.get("components", {}))
+
+    def submit(self, lpurl):
+
+        def chunk_read(response):
+            read = ""
+
+            while 1:
+                chunk = response.read(1)
+                read += chunk
+
+                if "\n" in read:
+                    lines = read.split("\n")
+                    read = lines.pop(-1)
+                    lines = [x.split(":", 1) for x in lines]
+                    if lines:
+                        for x in lines:
+                            if x[0] != "ok":
+                                print "Error: %s" % x[1]
+                                return False
+                        print "\n".join([x[1] for x in lines])
+
+                if not chunk:
+                    print read
+                    if not read.startswith("ok:"): return False
+                    return True
+            return True
+
+        try:
+            response = urllib2.urlopen('https://sil.pythonanywhere.com/submit', data=urllib.urlencode({"lpurl": lpurl}));
+        except urllib2.HTTPError, e:
+            print("Error while submitting:")
+            print(e.read())
+            return
+        ok = chunk_read(response)
+        if not ok: sys.exit(1)
+
+def get_repository_from_component_name(args):
+    if "/" in args.componentname:
+        return CommunityRepository(args)
+    else:
+        return CuratedRepository(args)
+
+def cmd_install(args):
+    get_repository_from_component_name(args).install(args.componentname)
+def cmd_update(args):
+    get_repository_from_component_name(args).update(args.componentname)
+def cmd_search(args):
+    print("Matching curated components")
+    print("===========================")
+    cu = CuratedRepository(args)
+    results = cu.search(args.query)
+    for result in results:
+        print("  {componentname:>30}  {description}".format(**result))
+    if not results:
+        print("  {0:>30}".format("(no matches)"))
+    print
+    print("Matching community components")
+    print("=============================")
+    co = CommunityRepository(args)
+    results = co.search(args.query)
+    for result in results:
+        print("  {componentname:>30}  {description}".format(**result))
+    if not results:
+        print("  {0:>30}".format("(no matches)"))
+
+def cmd_submit(args):
+    print("(submitting to community repository)")
+    args.refresh = False
+    co = CommunityRepository(args)
+    co.submit(args.launchpadBranchURL)
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Install and manage third-party components into your Ubuntu SDK app")
+    subparsers = parser.add_subparsers(dest="subparser_name")
+    parser_install = subparsers.add_parser("install", help="Install a component into this app")
+    parser_install.add_argument("componentname", type=str, 
+        help="ComponentName for curated components, developer/ComponentName for community components")
+    parser_install.add_argument("--refresh", action="store_true", help="Refresh the packages list")
+    parser_install.set_defaults(func=cmd_install)
+    parser_update = subparsers.add_parser("update", help="Update an installed component in this app")
+    parser_update.add_argument("componentname", type=str, 
+        help="ComponentName for curated components, developer/ComponentName for community components")
+    parser_update.add_argument("--refresh", action="store_true", help="Refresh the packages list")
+    parser_update.set_defaults(func=cmd_update)
+    parser_search = subparsers.add_parser("search", help="Search names and descriptions of existing components")
+    parser_search.add_argument("query", type=str, nargs="*", help="Search terms")
+    parser_search.add_argument("--refresh", action="store_true", help="Refresh the packages list")
+    parser_search.set_defaults(func=cmd_search)
+    parser_submit = subparsers.add_parser("submit", help="Publish a component to the community store")
+    parser_submit.add_argument("launchpadBranchURL", type=str, help="lp:~username/project/branch")
+    parser_submit.set_defaults(func=cmd_submit)
+    parser_help = subparsers.add_parser("help", help="Help on subcommands")
+    parser_help.add_argument("subcommand", nargs="?", type=str)
+
+    args = parser.parse_args()
+    if args.subparser_name == "help":
+        if args.subcommand:
+            parser.parse_args([args.subcommand, "-h"])
+        else:
+            parser.parse_args(["-h"])
+    else:
+        args.func(args)

=== removed file 'script/ucs'
--- script/ucs	2014-11-09 15:36:15 +0000
+++ script/ucs	1970-01-01 00:00:00 +0000
@@ -1,115 +0,0 @@
-#!/bin/bash
-
-# Branch specs
-PWD=$(pwd)
-ARGS=("$@")
-UCS_BRANCH=lp:component-store
-UCS_FILEPATH=lp:~ubuntu-touch-community-dev/component-store/trunk.14.10/ComponentStore
-BOOTSTRAP_FOLDER=${PWD}/ComponentStoreSetup
-
-# Color Output
-RC='\e[0;31m'
-GC='\e[0;32m'
-NC='\e[0m'
-
-show_usage() {
-	echo "Usage:"
-	echo "  ucs <command> [options]"
-	echo ""
-	echo "Commands:"
-	echo -e "  install	\tInstall Components"
-	echo -e "  update 	\tUpdate Components"
-	echo -e "  help		\tdisplay this help and exit"
-	echo -e "  version	\toutput version information and exit"
-}
-
-install_prerequisites() {
-	if [ $(dpkg-query -W -f='${Status}' bzr 2>/dev/null | grep -c "ok installed") -eq 0 ]; then
-		echo "bzr is not installed...installing it now.."
-		sudo apt-get install -y bzr --no-install-recommends;
-	fi
-}
-
-if [ $# -lt 1 ]
-then
-	show_usage
-	exit 1
-fi
-
-#CASE: HELP ARGUMENT
-if [[ $1 == "help" ]]; then
-	show_usage
-	exit 0
-
-#CASE: ABOUT ARGUMENT
-elif [[ $1 == "version"  ]]; then
-	echo "Ubuntu Component Store (UCS) v0.1.2"
-	echo "Copyright (C) 2014 Ubuntu Touch Community Dev Team"
-	echo "License GPLv3: GNU version 3 <http://gnu.org/licenses/gpl.html>."
-	echo "This is free software: you are free to change and redistribute it."
-	echo "There is NO WARRANTY, to the extent permitted by law."
-	echo
-	echo "Written by Ubuntu Touch Community Dev"
-	exit 0
-
-#CASE: INSTALL ARGUMENT
-elif [[ $1 == "install" ]]; then
-	install_prerequisites
-	if [ $# -gt 1 ]; then
-		for (( i=1; i<$#; i++ ))
-		do
-			echo -n "[$i/`expr $# - 1`] Installing ${ARGS[$i]}..."
-			componentfilelist=(`bzr ls -q $UCS_FILEPATH/${ARGS[$i]}`)
-			componentfilelistcount=${#componentfilelist[@]}
-			if (( componentfilelistcount == 0 )); then
-				echo -e "${RC}FAILED${NC} (Invalid component name)"
-			else
-				for (( j=0; j<componentfilelistcount; j++ ))
-				do
-					filename=$(basename ${componentfilelist[j]})
-					bzr cat -q ${componentfilelist[j]} > ${PWD}/${filename}
-				done
-				echo -e "${GC}DONE${NC}"
-			fi
-		done
-	else
-		echo -n "Installing all components..."
-		componentlist=(`bzr ls -q $UCS_FILEPATH`)
-		componentlistcount=${#componentlist[@]}
-		for (( i=0; i<componentlistcount; i++ ))
-		do
-			componentfilelist=(`bzr ls -q ${componentlist[i]}`)
-			componentfilelistcount=${#componentfilelist[@]}
-			for (( j=0; j<componentfilelistcount; j++ ))
-			do
-				bzr cat -q ${componentlist[i]}${componentfilelist[j]} > ${PWD}/${componentfilelist[j]}
-			done
-		done
-		echo -e "${GC}DONE${NC}"
-	fi
-	echo -e "${GC}Installation Complete!${NC}"
-
-#CASE: UPDATE ARGUMENT
-elif [[ $1 == "update" ]]; then
-	if [ $# -gt 1 ]; then
-		for (( i=1; i<$#; i++ ))
-		do
-			echo -n "[$i/`expr $# - 1`] Updating ${ARGS[$i]}..."
-			componentfilelist=(`bzr ls -q $UCS_FILEPATH/${ARGS[$i]}`)
-			componentfilelistcount=${#componentfilelist[@]}
-			if (( componentfilelistcount == 0 )); then
-				echo -e "${RC}FAILED${NC} (Invalid component name)"
-			else
-				for (( j=0; j<componentfilelistcount; j++ ))
-				do
-					filename=$(basename ${componentfilelist[j]})
-					bzr cat -q ${componentfilelist[j]} > ${PWD}/${filename}
-				done
-				echo -e "${GC}DONE${NC}"
-			fi
-		done
-	echo -e "${GC}Updating Components Complete!${NC}"
-	else
-		echo -e "${RC}Error updating components! Please specify the component name you want to update. Eg. ucs update EmptyState${NC}"
-	fi
-fi