Merge lp:~sil/component-store/community-components into lp:component-store

Proposed by Stuart Langridge on 2015-01-04
Status: Merged
Merged at revision: 49
Proposed branch: lp:~sil/component-store/community-components
Merge into: lp:component-store
Diff against target: 1575 lines (+671/-247)
11 files modified
.bzrignore (+4/-0)
community-server/requirements.txt (+7/-0)
community-server/server.py (+329/-0)
docs/contribute-community.rst (+144/-0)
docs/contribute-curated.rst (+120/-0)
docs/contribute.rst (+10/-117)
docs/index.rst (+8/-6)
docs/install.rst (+35/-8)
script/debian/changelog (+12/-0)
script/debian/control (+2/-1)
script/ucs (+0/-115)
To merge this branch: bzr merge lp:~sil/component-store/community-components
Reviewer Review Type Date Requested Status
Nekhelesh Ramananthan 2015-01-04 Approve on 2015-04-13
Review via email: mp+245510@code.launchpad.net

Description of the change

The community store.

Reorganise the existing trunk to put the app stuff into an "app" folder.
Rewrite "ucs" so that it knows about community and curated components, and about submitting community components.
Make "ucs install" correctly install both pure QML and binary compiled components in an appropriate place in the current project, and integrate those components with the build system for this project, automatically. (Work in progress; done for pure QML projects; partially done for qmake projects; not done for cmake projects.)
Implement a community server which receives "ucs submit" requests and updates the master list of all components (in lp:~ubuntu-touch-community-dev/component-store/community-components-json).
Add documentation for how to create a community component and submit it.

To post a comment you must log in.
53. By Stuart Langridge on 2015-01-04

merge lp:~nik90/component-store/fix-gallerysrc

54. By Stuart Langridge on 2015-01-04

Add version requirements to everything

55. By Stuart Langridge on 2015-01-04

https for the community server, if you please

56. By Stuart Langridge on 2015-01-04

Correctly show errors

57. By Stuart Langridge on 2015-01-04

FIx eror messages, properly this time

58. By Stuart Langridge on 2015-01-04

Get and cache curated component metadata and use it for searching and display

59. By Stuart Langridge on 2015-01-04

merged lp:~nik90/component-store/rename-app-folder

60. By Stuart Langridge on 2015-01-04

allow forcing refresh on search

61. By Stuart Langridge on 2015-01-04

Handle adding pure QML components to cmake-based projects

62. By Stuart Langridge on 2015-01-04

Add decent logging to server, including by gmail email

63. By Stuart Langridge on 2015-01-05

Add note clarifying that compoennts with with qml and qmllib should work

Nekhelesh Ramananthan (nik90) wrote :

TODO: Just before merging this branch, we can remove ucs.sh (legacy script).

Nekhelesh Ramananthan (nik90) wrote :

Things that needs fixing,

1. Remove legacy ucs.sh script

2. Remove support for qmake and cmake projects. What we have currently has 2 main issues which are, "installing to the wrong directory" and "invalid CMakeLists.txt". The app project structure is not constant and this is more prevalent in cmake projects where qml code usually goes into app/* and c++ components go into backend/* or src/*. Adding ubuntu_component_store to the root folder might not be appreciated by developers.

for now just adding the necessary components into a folder called ubuntu_component_store and requiring app developers to do the rest of the bootstrapping would suffice IMO. We can build on top of that once we get help from the community and the SDK devs.

3. Merge lp:~nik90/component-store/fix-debian-packaging into this MP. Although before doing that please check the debian/control file to see if it has all the necessary dependency for ucs. If a dependency is missing, please let me know and I will add it.

After this MP is merged I will update the online docs and release a update in the PPA.

review: Needs Fixing
64. By Stuart Langridge on 2015-04-13

Remove attempts to add installed components to the project infrastructure for qmake and cmake projects; instead, print a message explaining what to do. DWIMming this is pretty difficult, especially given that I don't really understand either qmake or cmake, and so what we do is punt, explain the situation, and assume that anyone who has chosen to use cmake or qmake in a project knows something about C++ and compilation steps and therefore can add it themselves.

65. By Stuart Langridge on 2015-04-13

Remove legacy ucs.sh script

Stuart Langridge (sil) wrote :

New versions pushed which remove attempts to insert components into the build process for [cq]make projects.

66. By Stuart Langridge on 2015-04-13

merge lp:~nik90/component-store/fix-debian-packaging

Nekhelesh Ramananthan (nik90) wrote :

Lets get this awesomeness merged and out to the world!

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2014-11-09 00:04:53 +0000
3+++ .bzrignore 2015-04-13 19:51:02 +0000
4@@ -3,3 +3,7 @@
5 docs/_build
6 docs/*.save
7
8+venv
9+../logs
10+../logs/server.log
11+gmailconf.json
12
13=== added directory 'community-server'
14=== added file 'community-server/requirements.txt'
15--- community-server/requirements.txt 1970-01-01 00:00:00 +0000
16+++ community-server/requirements.txt 2015-04-13 19:51:02 +0000
17@@ -0,0 +1,7 @@
18+Flask==0.10.1
19+Jinja2==2.7.3
20+MarkupSafe==0.23
21+Werkzeug==0.9.6
22+argparse==1.2.1
23+itsdangerous==0.24
24+wsgiref==0.1.2
25
26=== added file 'community-server/server.py'
27--- community-server/server.py 1970-01-01 00:00:00 +0000
28+++ community-server/server.py 2015-04-13 19:51:02 +0000
29@@ -0,0 +1,329 @@
30+import re, time, json, codecs, os, fcntl
31+from bzrlib.plugin import load_plugins
32+load_plugins()
33+from bzrlib.branch import Branch
34+from bzrlib import workingtree
35+from flask import Flask, request, Response
36+app = Flask(__name__)
37+
38+import logging
39+from logging import StreamHandler, Formatter
40+from logging.handlers import RotatingFileHandler
41+
42+file_handler = StreamHandler()
43+file_handler.setLevel(logging.DEBUG)
44+file_handler.setFormatter(Formatter(
45+ '%(asctime)s %(levelname)s: %(message)s '
46+ '[in %(pathname)s:%(funcName)s:%(lineno)d]'
47+))
48+app.logger.addHandler(file_handler)
49+
50+log_file_handler = RotatingFileHandler(
51+ os.path.join(os.path.dirname(__file__), "..", "logs", "server.log"),
52+ maxBytes=100000)
53+log_file_handler.setLevel(logging.DEBUG)
54+log_file_handler.setFormatter(Formatter(
55+ '%(asctime)s %(levelname)s: %(message)s '
56+ '[in %(pathname)s:%(funcName)s:%(lineno)d]'
57+))
58+app.logger.addHandler(log_file_handler)
59+
60+# http://mynthon.net/howto/-/python/python%20-%20logging.SMTPHandler-how-to-use-gmail-smtp-server.txt
61+class TlsSMTPHandler(logging.handlers.SMTPHandler):
62+ def emit(self, record):
63+ """
64+ Emit a record.
65+
66+ Format the record and send it to the specified addressees.
67+ """
68+ try:
69+ import smtplib
70+ import string # for tls add this line
71+ try:
72+ from email.utils import formatdate
73+ except ImportError:
74+ formatdate = self.date_time
75+ port = self.mailport
76+ if not port:
77+ port = smtplib.SMTP_PORT
78+ smtp = smtplib.SMTP(self.mailhost, port)
79+ msg = self.format(record)
80+ msg = "From: %s\r\nTo: %s\r\nSubject: %s\r\nDate: %s\r\n\r\n%s" % (
81+ self.fromaddr,
82+ string.join(self.toaddrs, ","),
83+ self.getSubject(record),
84+ formatdate(), msg)
85+ if self.username:
86+ smtp.ehlo() # for tls add this line
87+ smtp.starttls() # for tls add this line
88+ smtp.ehlo() # for tls add this line
89+ smtp.login(self.username, self.password)
90+ smtp.sendmail(self.fromaddr, self.toaddrs, msg)
91+ smtp.quit()
92+ except (KeyboardInterrupt, SystemExit):
93+ raise
94+ except:
95+ self.handleError(record)
96+
97+logger = logging.getLogger()
98+
99+try:
100+ fp = open("gmailconf.json")
101+ maildata = json.load(fp)
102+ fp.close()
103+ if "mailfrom" not in maildata:
104+ raise Exception("bad mail data, so don't add a mail log")
105+ gm = TlsSMTPHandler(
106+ ("smtp.gmail.com", 587),
107+ maildata["mailfrom"],
108+ [maildata["mailto"]],
109+ "Server error with ucs submit",
110+ (maildata["username"], maildata["password"])
111+ )
112+ gm.setFormatter(Formatter('''
113+ Message type: %(levelname)s
114+ Location: %(pathname)s:%(lineno)d
115+ Module: %(module)s
116+ Function: %(funcName)s
117+ Time: %(asctime)s
118+
119+ Message:
120+
121+ %(message)s
122+ '''))
123+ gm.setLevel(logging.ERROR)
124+ app.logger.addHandler(gm)
125+except:
126+ pass
127+
128+# location of master manifest. Note: this MUST point at community_components.json in a
129+# bzr branch which was fetched with "bzr checkout", NOT "bzr branch", so that a
130+# "bzr commit" writes the contents back to Launchpad with appropriate authentication.
131+CCJSON = os.path.join(os.path.dirname(__file__), "..", "..", "community-components-json", "community_components.json")
132+
133+# https://github.com/kennethreitz/flask-sslify/blob/master/flask_sslify.py
134+from flask import request, redirect, current_app
135+YEAR_IN_SECS = 31536000
136+class SSLify(object):
137+ """Secures your Flask App."""
138+
139+ def __init__(self, app=None, age=YEAR_IN_SECS, subdomains=False, permanent=False):
140+ self.hsts_age = age
141+ self.hsts_include_subdomains = subdomains
142+ self.permanent = permanent
143+
144+ if app is not None:
145+ self.init_app(app)
146+
147+ def init_app(self, app):
148+ """Configures the configured Flask app to enforce SSL."""
149+
150+ app.before_request(self.redirect_to_ssl)
151+ app.after_request(self.set_hsts_header)
152+
153+ @property
154+ def hsts_header(self):
155+ """Returns the proper HSTS policy."""
156+ hsts_policy = 'max-age={0}'.format(self.hsts_age)
157+
158+ if self.hsts_include_subdomains:
159+ hsts_policy += '; includeSubDomains'
160+
161+ return hsts_policy
162+
163+ def redirect_to_ssl(self):
164+ """Redirect incoming requests to HTTPS."""
165+ # Should we redirect?
166+ criteria = [
167+ request.is_secure,
168+ current_app.debug,
169+ request.headers.get('X-Forwarded-Proto', 'http') == 'https'
170+ ]
171+
172+ if not any(criteria):
173+ if request.url.startswith('http://'):
174+ url = request.url.replace('http://', 'https://', 1)
175+ code = 302
176+ if self.permanent:
177+ code = 301
178+ r = redirect(url, code=code)
179+
180+ return r
181+
182+ def set_hsts_header(self, response):
183+ """Adds HSTS header to each response."""
184+ if request.is_secure:
185+ response.headers.setdefault('Strict-Transport-Security', self.hsts_header)
186+ return response
187+
188+#https://github.com/derpston/python-simpleflock/blob/master/src/simpleflock.py
189+class SimpleFlock:
190+ """Provides the simplest possible interface to flock-based file locking.
191+ Intended for use with the `with` syntax. It will create/truncate/delete
192+ the lock file as necessary."""
193+
194+ def __init__(self, path, timeout = None):
195+ self._path = path
196+ self._timeout = timeout
197+ self._fd = None
198+
199+ def __enter__(self):
200+ self._fd = os.open(self._path, os.O_CREAT)
201+ start_lock_search = time.time()
202+ while True:
203+ try:
204+ fcntl.flock(self._fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
205+ # Lock acquired!
206+ return
207+ except IOError, ex:
208+ if ex.errno != errno.EAGAIN: # Resource temporarily unavailable
209+ raise
210+ elif self._timeout is not None and time.time() > (start_lock_search + self._timeout):
211+ # Exceeded the user-specified timeout.
212+ raise
213+
214+ # TODO It would be nice to avoid an arbitrary sleep here, but spinning
215+ # without a delay is also undesirable.
216+ time.sleep(0.1)
217+
218+ def __exit__(self, *args):
219+ fcntl.flock(self._fd, fcntl.LOCK_UN)
220+ os.close(self._fd)
221+ self._fd = None
222+
223+ # Try to remove the lock file, but don't try too hard because it is
224+ # unnecessary. This is mostly to help the user see whether a lock
225+ # exists by examining the filesystem.
226+ try:
227+ os.unlink(self._path)
228+ except:
229+ pass
230+
231+
232+def process_lpurl(lpurl, username):
233+ try:
234+ manifest_file_name = "ubuntu_component_store.json"
235+ yield "ok:Checking Launchpad branch %s\n" % (lpurl,)
236+ try:
237+ b = Branch.open(lpurl)
238+ except:
239+ msg = "error:Unable to read LP branch %s\n" % (lpurl,)
240+ app.logger.warn(msg)
241+ yield msg
242+ return
243+ tree = b.basis_tree()
244+ manifest_file_id = tree.path2id(manifest_file_name)
245+ if not manifest_file_id:
246+ msg = "error:Unable to find the component manifest %s/%s.\n" % (lpurl, manifest_file_name)
247+ app.logger.warn(msg)
248+ yield msg
249+ return
250+ try:
251+ manifest_data = tree.get_file_text(manifest_file_id)
252+ except:
253+ msg = "error:Unable to read the component manifest %s/%s\n" % (lpurl, manifest_file_name)
254+ app.logger.warn(msg)
255+ yield msg
256+ return
257+ try:
258+ manifest = json.loads(manifest_data)
259+ except:
260+ msg = "error:Component manifest %s/%s seems corrupt\n" % (lpurl, manifest_file_name)
261+ app.logger.warn(msg)
262+ yield msg
263+ return
264+ if "name" not in manifest:
265+ msg = "error:Component manifest %s/%s does not specify a component 'name' field\n" % (lpurl, manifest_file_name)
266+ app.logger.warn(msg)
267+ yield msg
268+ return
269+ if not re.match("^[A-Za-z0-9_]+$", manifest["name"]):
270+ msg = "error:Component manifest %s/%s specifies an invalid 'name' field (it must match [A-Za-z0-9_]+)\n" % (lpurl, manifest_file_name)
271+ app.logger.warn(msg)
272+ yield msg
273+ return
274+ if "version" not in manifest:
275+ msg = "error:Component manifest %s/%s does not specify a component 'version' field\n" % (lpurl, manifest_file_name)
276+ app.logger.warn(msg)
277+ yield msg
278+ return
279+ if not re.match("^[0-9]+\.[0-9]+$", manifest["version"]):
280+ msg = "error:Component manifest %s/%s specifies an invalid 'version' field (it must be a number such as 1.0)\n" % (lpurl, manifest_file_name)
281+ app.logger.warn(msg)
282+ yield msg
283+ return
284+ has_binary = False; has_qml = False
285+ if tree.path2id("qmllib"):
286+ has_binary = True
287+ if tree.path2id("qml"):
288+ has_qml = True
289+ if not has_binary and not has_qml:
290+ msg = "error:Component has neither 'qmllib' nor 'qml' directories\n"
291+ app.logger.warn(msg)
292+ yield msg
293+ return
294+ yield "ok:Checks passed OK\n"
295+ pkg_data = {
296+ "name": "%s/%s" % (username, manifest["name"]),
297+ "revno": b.revno(),
298+ "description": manifest.get("description"),
299+ "lpurl": lpurl,
300+ "type": [],
301+ "version": manifest["version"]
302+ }
303+ if has_binary: pkg_data["type"].append("binary")
304+ if has_qml: pkg_data["type"].append("qml")
305+ yield "ok:Calculated package summary data\n"
306+
307+ # take an exclusive lock on the lockfile to make sure we don't collide with another worker
308+ with SimpleFlock("lockfile"):
309+ try:
310+ fp = codecs.open(CCJSON, encoding="utf8")
311+ comps = json.load(fp)
312+ fp.close()
313+ except:
314+ msg = "error:Unable to save summary data\n"
315+ app.logger.warn(msg)
316+ yield msg
317+ return
318+ if "components" not in comps:
319+ yield "Master component list is corrupt\n"
320+ return
321+ if pkg_data["name"] in comps["components"]:
322+ change_verb = "updated"
323+ else:
324+ change_verb = "added"
325+ comps["components"][pkg_data["name"]] = pkg_data
326+ fp = codecs.open(CCJSON, encoding="utf8", mode="w")
327+ json.dump(comps, fp, indent=2)
328+ fp.close()
329+ yield "ok:Updating master record\n"
330+ wt, relpath = workingtree.WorkingTree.open_containing(CCJSON)
331+ commsg = "Server %s component %s %s" % (change_verb, pkg_data["name"], time.asctime(),)
332+ wt.commit(message=commsg, specific_files=[os.path.split(CCJSON)[1]])
333+ yield "ok:Component %s %s successfully\n" % (pkg_data["name"], change_verb)
334+ except:
335+ app.logger.exception("Submit error on url %s" % (lpurl,))
336+ msg = "error:Server error\n"
337+ yield msg
338+
339+@app.route("/submit", methods=["POST"])
340+def submit():
341+ lpurl = request.form.get("lpurl")
342+ if not lpurl:
343+ logging.warn("bad request")
344+ return "error:No lpurl specified\n", 400
345+ m = re.match("^lp:~(?P<username>[^/]+)/(?P<project>[^/]+)/(?P<branch>[^/]+)$", lpurl)
346+ if not m:
347+ return "error:Malformed lp URL (should be lp:~username/project/branch, not lp:project)\n", 400
348+ return Response(process_lpurl(lpurl, m.groupdict()["username"]), mimetype="text/plain")
349+
350+@app.route("/")
351+def root():
352+ return "The Ubuntu community component store server."
353+
354+SSLify(app)
355+
356+if __name__ == "__main__":
357+ app.debug = True
358+ app.run()
359\ No newline at end of file
360
361=== added directory 'curated-store'
362=== renamed directory 'ComponentStore' => 'curated-store/ComponentStore'
363=== renamed file 'ComponentStore.apparmor' => 'curated-store/ComponentStore.apparmor'
364=== renamed file 'ComponentStore.desktop' => 'curated-store/ComponentStore.desktop'
365=== renamed file 'ComponentStore.png' => 'curated-store/ComponentStore.png'
366=== renamed file 'ComponentStore.qmlproject' => 'curated-store/ComponentStore.qmlproject'
367=== renamed directory 'GallerySRC' => 'curated-store/GallerySRC'
368=== renamed file 'main.qml' => 'curated-store/main.qml'
369=== renamed file 'manifest.json' => 'curated-store/manifest.json'
370=== renamed directory 'tests' => 'curated-store/tests'
371=== added file 'docs/contribute-community.rst'
372--- docs/contribute-community.rst 1970-01-01 00:00:00 +0000
373+++ docs/contribute-community.rst 2015-04-13 19:51:02 +0000
374@@ -0,0 +1,144 @@
375+Contributing a component to the Community store
376+===============================================
377+
378+To publish an Ubuntu SDK component to the Community store, you will need a
379+`Launchpad account <https://help.launchpad.net/YourAccount/NewAccount>`_, and you will need to be
380+familiar with `using bzr to push code to Launchpad <https://help.launchpad.net/Code/UploadingABranch>`_.
381+
382+Community components can be pure QML, or they can be compiled binary components.
383+
384+Publishing a pure QML component
385+-------------------------------
386+
387+Create a branch on Launchpad (in any project of your choice) which is named how you plan to name
388+the component, with a top-level *qml* folder, and put your QML file and any required assets for
389+your component in that qml folder. Add an `ubuntu_component_store.json`_ file. Then, submit the
390+component to the community store with *ucs submit lp:~username/project/branch*.
391+
392+So, your branch should look like: ::
393+
394+ /qml/
395+ MyComponent.qml
396+ required_image.png
397+ included_script.js
398+ /ubuntu_component_store.json
399+
400+If your component does not have any external assets, it is fine to have a branch
401+with *qml/MyComponent.qml* and nothing else in it. Your branch may also contain any other files
402+you choose outside the *qml* top-level folder; these files are not installed with your component,
403+but may provide useful guidance, READMEs, or other code for people who want to make changes to
404+the component and contribute them back to you.
405+
406+The name of the component itself is defined by `ubuntu_component_store.json`_; the QML filename
407+is what names the QML Item that it provides. So, if you are Launchpad user *sil*, and
408+you push your component to *lp:~sil/SomeProject/mything*, it defines its own name
409+in `ubuntu_component_store.json`_ as *UsefulComponent*, and it
410+contains *qml/RedRectangle.qml*, an app developer will use it like this: ::
411+
412+ import QtQuick 2.0
413+ import ubuntu_component_store.sil.UsefulComponent 1.0
414+ ....
415+ MainView {
416+ RedRectangle {
417+ ....
418+ }
419+ }
420+
421+Most components should have component name and QML file name be the same thing to avoid confusion.
422+
423+Do not publish two unrelated QML components in one UCS component; publish them separately, so
424+they can be used separately.
425+
426+.. note:: if you do not want to create a whole Launchpad project just for this component, you can push to a Launchpad "junk" branch: *lp:~username/+junk/somename*
427+
428+Once your branch is created, publish it to the Community store with ::
429+
430+ $ ucs submit lp:~username/project/somename
431+ (submitting to community repository)
432+ Checking Launchpad branch lp:~username/project/somename
433+ Checks passed OK
434+ Calculated package summary data
435+ Updating master record
436+ Component username/ComponentName updated successfully
437+
438+It should then be visible to *ucs search*.
439+
440+Publishing a compiled component
441+-------------------------------
442+
443+Publishing a compiled component is a little more complicated, for CPU architecture reasons. Your component must
444+be `an Extension Plugin <http://doc.qt.io/qt-5/qtqml-tutorials-extending-qml-example.html#chapter-6-writing-an-extension-plugin>`_, "a
445+plugin library to make it available to the QML engine as a new QML import module". Creating such a plugin is
446+currently beyond the scope of this document. (We are hoping to provide an Ubuntu SDK IDE template
447+for creating such components with the proper filesystem layout; before then, the "App with QML Extension
448+Library" option creates an appropriate type of component.)
449+
450+You *must* compile your component for three different architectures: ARM, x86, and amd64 (for Ubuntu phones,
451+the desktop emulator, and Ubuntu desktops). Once you have compiled it as such, you will have
452+three different *libMyComponent.so* files.
453+
454+Assemble a Launchpad branch with a top-level *qmllib* folder, and in it put a folder for each architecture,
455+named for the GNU architecture triplet, and then the *.so* file within. So: ::
456+
457+ /qmllib
458+ /x86_64-linux-gnu
459+ /libMyComponent.so
460+ /i386-linux-gnu
461+ /libMyComponent.so
462+ /arm-linux-gnueabihf
463+ /libMyComponent.so
464+ /ubuntu_component_store.json
465+
466+Add an `ubuntu_component_store.json`_ file to the root of the branch.
467+
468+Your branch may contain any other files of your choice outside the */qmllib* folder; in particular, it
469+should contain the source code for the plugin so others can build it themselves if they choose!
470+
471+The name of the component itself is defined by `ubuntu_component_store.json`_; your component is expected to
472+use `qmlRegisterType <http://doc.qt.io/qt-5/qqmlextensionplugin.html#details>`_ to provide QML Item types.
473+So, if you are Launchpad user *sil*, and you push your component to *lp:~sil/SomeProject/Whatever*,
474+it defines its name in `ubuntu_component_store.json`_ as *SomeComponent*, and it registers
475+a *Triangle* type, an app developer will use it like this: ::
476+
477+ import QtQuick 2.0
478+ import ubuntu_component_store.sil.SomeComponent 1.0
479+ ....
480+ MainView {
481+ Triangle {
482+ ....
483+ }
484+ }
485+
486+Do not publish two unrelated components in one UCS component; publish them separately, so they can
487+be used separately.
488+
489+.. note:: if you do not want to create a whole Launchpad project just for this component, you can push to a Launchpad "junk" branch: *lp:~username/+junk/somename*
490+
491+Once your branch is created, publish it to the Community store with ::
492+
493+ $ ucs submit lp:~username/project/somename
494+ (submitting to community repository)
495+ Checking Launchpad branch lp:~username/project/somename
496+ Checks passed OK
497+ Calculated package summary data
498+ Updating master record
499+ Component username/ComponentName updated successfully
500+
501+It should then be visible to *ucs search*.
502+
503+A component can contain both *qml* and *qmllib* folders and so contain both QML parts and binary parts; both will be installed when a developer uses *ucs install* to install your component.
504+
505+ubuntu_component_store.json
506+===========================
507+
508+A community component must contain a file *ubuntu_component_store.json* describing metadata about the component.
509+It must be valid JSON, with keys *name*, *version*, and *description*: ::
510+
511+ {
512+ "name": "GenericPodcastApp",
513+ "version": "1.0",
514+ "description": "A component which manages a podcast RSS feed, with playback and display of episodes"
515+ }
516+
517+Other keys may be added in future. Note that the public name of the component will be launchpadusername/name,
518+where *name* is the name taken from *ubuntu_component_store.json*. The branch name in Launchpad is ignored.
519
520=== added file 'docs/contribute-curated.rst'
521--- docs/contribute-curated.rst 1970-01-01 00:00:00 +0000
522+++ docs/contribute-curated.rst 2015-04-13 19:51:02 +0000
523@@ -0,0 +1,120 @@
524+Contributing a component to the Curated store
525+=============================================
526+
527+Ubuntu Component Store is intended to be a community project where app developers are
528+encouraged to contribute any component that they think would be useful to other developers.
529+To contribute a component to UCS, we recommend first joining the team by applying `here`_.
530+This would then make it possible for you to maintain your component by pushing bug fixes and
531+improvements as and when you choose.
532+
533+By moderating the members of the team and components in UCS, we can promise a well tested
534+component with a stable API. We recognize that this can impede the rate at which new components
535+can be added to UCS, but we feel that this is an acceptable drawback to ensure good quality
536+components to our userbase.
537+
538+Let's go through the steps required to upload a new component to the store.
539+
540+Getting Started
541+---------------
542+
543+You can technically contribute new components to the store without being a member of the team by
544+requesting someone from the team to maintain your component. However we highly recommend that you
545+join the team since that would allow you to push bug fixes and improvements to your component
546+quickly. You can apply to become a member by applying `here`_. The approval should be done within a
547+day or two.
548+
549+UCS is hosted on launchpad and requires bazaar (bzr) to grab or push the code. As such you would
550+need to have an account on launchpad and be familiar with bzr. You can grab the code by, ::
551+
552+ bzr branch lp:component-store
553+
554+UCS by itself is an ordinary QML project. You should be able to open it using qtcreator like any other
555+project. Run the UCS Gallery app by either pressing the green button in qtcreator or via the command line
556+as shown below, ::
557+
558+ qmlscene main.qml
559+
560+This should open the UCS Gallery app as shown below. It provides a visual overview of all the components
561+in the store.
562+
563+.. image:: _images/gallery.png
564+ :align: center
565+ :scale: 90 %
566+
567+.. note:: At the time of writing this documentation, you would need a 14.10 desktop to run the component store gallery. You can run the gallery app on the phone or emulator using a 14.04 desktop, however the gallery app doesn't converge well on the phone yet. This should be fixed soon.
568+
569+Adding a new component to the store
570+-----------------------------------
571+
572+Adding a new component to the store involves 3 main steps which are,
573+
574+1. Adding a new component to the ComponentStore folder
575+2. Updating the gallery app to showcase your new component
576+3. Updating the documentation
577+
578+Adding a new component to the ComponentStore Folder
579+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
580+
581+The components are stored in their own folders in the ComponentStore folder. Let's assume for illustration purposes,
582+we are trying to add a new component *MyNewComponent* to UCS. Let's grab UCS first and then add the new component. ::
583+
584+ bzr branch lp:component-store MyNewComponentBranch
585+
586+Let create a new folder for our component in the ComponentStore folder, ::
587+
588+ cd MyNewComponentBranch/ComponentStore
589+ mkdir MyNewComponent && bzr add MyNewComponent
590+ cd MyNewComponent
591+
592+Inside your *MyNewComponent* folder, add your generic component files. Everything that is required to use your component
593+must be included in this folder. Let's now add these files to the source control bzr and commit them. ::
594+
595+ bzr add *
596+ bzr commit -m "Added my new component"
597+
598+Updating the gallery app
599+^^^^^^^^^^^^^^^^^^^^^^^^
600+
601+Now that we have our component ready and added, let's update the gallery app to show that. Create a new file in the GallerySRC
602+folder called MyNewComponentWidget.qml. This file is rather easy to fill out. Just copy the code from the other widget template
603+files. It basically involves filling in information like the author, license, contact and a description of your component which
604+is rather simple. Once completed, add MyNewComponentWidget.qml to the version control. ::
605+
606+ cd GallerySRC
607+ bzr add MyNewComponentWidget.qml
608+ bzr commit -m "Showcase new component in the gallery"
609+
610+Next, open WidgetsModel.qml and add your component to the list. That's it! Run the gallery app and see if your component shows up
611+as expected.
612+
613+Updating the documentation
614+^^^^^^^^^^^^^^^^^^^^^^^^^^
615+
616+This is one of the most important steps and benefits of adding your component to UCS. By providing a well written and clear documentation
617+you make it easier for other app developers to use your component in their app. All the documentation is hosted in the docs folder. Create
618+a new documentation file in the _components folder and fill in the necessary documentation. Use the existing documentation files to help you
619+with setting it up.
620+
621+Once done, add your component to the list in index.rst. Now let's test if the documentation looks good. To build the documentation, you
622+need sphinx and pip packages to be installed. Let's install that for the first time. ::
623+
624+ sudo apt-get install pip
625+ sudo pip install sphinx sphinx-autobuild
626+
627+Building your documentation is now really simple. Once the build process is complete, open _build/index.html in your browser to see your documentation. ::
628+
629+ make html
630+
631+Once again, let's add this to the version control. ::
632+
633+ cd docs/_components
634+ bzr add MyNewComponent.rst
635+ bzr commit -m "Added documentation for the new component"
636+
637+That was it! Your component is ready. Let's push this online to UCS. ::
638+
639+ bzr push lp:~launchpad-id/component-store/MyNewComponentBranch
640+
641+From there on, it is just a matter of reviewing the code (for new components, we try to ensure everything is in order) and then merging to trunk.
642+
643+.. _here: https://launchpad.net/~ubuntu-touch-community-dev
644
645=== modified file 'docs/contribute.rst'
646--- docs/contribute.rst 2014-11-08 21:43:43 +0000
647+++ docs/contribute.rst 2015-04-13 19:51:02 +0000
648@@ -1,120 +1,13 @@
649 Contributing to UCS
650 ===================
651
652-Ubuntu Component Store is intended to be a community project where app developers are
653-encouraged to contribute any component that they think would be useful to other developers.
654-To contribute a component to UCS, we recommend first joining the team by applying `here`_.
655-This would then make it possible for you to maintain your component by pushing bug fixes and
656-improvements as and when you choose.
657-
658-By moderating the members of the team and components in UCS, we can promise a well tested
659-component with a stable API. We recognize that this can impede the rate at which new components
660-can be added to UCS, but we feel that this is an acceptable drawback to ensure good quality
661-components to our userbase.
662-
663-Let's go through the steps required to upload a new component to the store.
664-
665-Getting Started
666----------------
667-
668-You can technically contribute new components to the store without being a member of the team by
669-requesting someone from the team to maintain your component. However we highly recommend that you
670-join the team since that would allow you to push bug fixes and improvements to your component
671-quickly. You can apply to become a member by applying `here`_. The approval should be done within a
672-day or two.
673-
674-UCS is hosted on launchpad and requires bazaar (bzr) to grab or push the code. As such you would
675-need to have an account on launchpad and be familiar with bzr. You can grab the code by, ::
676-
677- bzr branch lp:component-store
678-
679-UCS by itself is an ordinary QML project. You should be able to open it using qtcreator like any other
680-project. Run the UCS Gallery app by either pressing the green button in qtcreator or via the command line
681-as shown below, ::
682-
683- qmlscene main.qml
684-
685-This should open the UCS Gallery app as shown below. It provides a visual overview of all the components
686-in the store.
687-
688-.. image:: _images/gallery.png
689- :align: center
690- :scale: 90 %
691-
692-.. note:: At the time of writing this documentation, you would need a 14.10 desktop to run the component store gallery. You can run the gallery app on the phone or emulator using a 14.04 desktop, however the gallery app doesn't converge well on the phone yet. This should be fixed soon.
693-
694-Adding a new component to the store
695------------------------------------
696-
697-Adding a new component to the store involves 3 main steps which are,
698-
699-1. Adding a new component to the ComponentStore folder
700-2. Updating the gallery app to showcase your new component
701-3. Updating the documentation
702-
703-Adding a new component to the ComponentStore Folder
704-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
705-
706-The components are stored in their own folders in the ComponentStore folder. Let's assume for illustration purposes,
707-we are trying to add a new component *MyNewComponent* to UCS. Let's grab UCS first and then add the new component. ::
708-
709- bzr branch lp:component-store MyNewComponentBranch
710-
711-Let create a new folder for our component in the ComponentStore folder, ::
712-
713- cd MyNewComponentBranch/ComponentStore
714- mkdir MyNewComponent && bzr add MyNewComponent
715- cd MyNewComponent
716-
717-Inside your *MyNewComponent* folder, add your generic component files. Everything that is required to use your component
718-must be included in this folder. Let's now add these files to the source control bzr and commit them. ::
719-
720- bzr add *
721- bzr commit -m "Added my new component"
722-
723-Updating the gallery app
724-^^^^^^^^^^^^^^^^^^^^^^^^
725-
726-Now that we have our component ready and added, let's update the gallery app to show that. Create a new file in the GallerySRC
727-folder called MyNewComponentWidget.qml. This file is rather easy to fill out. Just copy the code from the other widget template
728-files. It basically involves filling in information like the author, license, contact and a description of your component which
729-is rather simple. Once completed, add MyNewComponentWidget.qml to the version control. ::
730-
731- cd GallerySRC
732- bzr add MyNewComponentWidget.qml
733- bzr commit -m "Showcase new component in the gallery"
734-
735-Next, open WidgetsModel.qml and add your component to the list. That's it! Run the gallery app and see if your component shows up
736-as expected.
737-
738-Updating the documentation
739-^^^^^^^^^^^^^^^^^^^^^^^^^^
740-
741-This is one of the most important steps and benefits of adding your component to UCS. By providing a well written and clear documentation
742-you make it easier for other app developers to use your component in their app. All the documentation is hosted in the docs folder. Create
743-a new documentation file in the _components folder and fill in the necessary documentation. Use the existing documentation files to help you
744-with setting it up.
745-
746-Once done, add your component to the list in index.rst. Now let's test if the documentation looks good. To build the documentation, you
747-need sphinx and pip packages to be installed. Let's install that for the first time. ::
748-
749- sudo apt-get install pip
750- sudo pip install sphinx sphinx-autobuild
751-
752-Building your documentation is now really simple. Once the build process is complete, open _build/index.html in your browser to see your documentation. ::
753-
754- make html
755-
756-Once again, let's add this to the version control. ::
757-
758- cd docs/_components
759- bzr add MyNewComponent.rst
760- bzr commit -m "Added documentation for the new component"
761-
762-That was it! Your component is ready. Let's push this online to UCS. ::
763-
764- bzr push lp:~launchpad-id/component-store/MyNewComponentBranch
765-
766-From there on, it is just a matter of reviewing the code (for new components, we try to ensure everything is in order) and then merging to trunk.
767-
768-.. _here: https://launchpad.net/~ubuntu-touch-community-dev
769+There are two sorts of components in the Ubuntu Component Store: *Curated* components and *Community* components. As a developer, you can publish your component to either. Components in the Curated store have higher requirements to ensure that they are high-quality; anyone can publish to the Community store without approval.
770+
771+Contributing a component
772+------------------------
773+
774+.. toctree::
775+ :maxdepth: 1
776+
777+ contribute-curated
778+ contribute-community
779
780=== modified file 'docs/index.rst'
781--- docs/index.rst 2015-03-03 14:06:38 +0000
782+++ docs/index.rst 2015-04-13 19:51:02 +0000
783@@ -6,7 +6,7 @@
784 Welcome to the Ubuntu Component Store
785 ==================================================
786
787-The Ubuntu Component Store (UCS) is intended to be a library hosting a collection of well tested and trusted components created by community developers.
788+The Ubuntu Component Store (UCS) is intended to be a library hosting a collection of components created by community developers for building Ubuntu SDK apps.
789 UCS will enable new developers by providing plug-and-play components/modules and thereby preventing the need to start from scratch.
790
791 Purpose
792@@ -26,11 +26,13 @@
793 How will this work?
794 -------------------
795
796-A developer who just wants to use these components will be able to grab the components easily and keep them updated by simply running the ucs program. On the
797-other hand, if a developer wants to contribute his component, then he/she will be required to join the team and maintain his component. This way, any bug fixes
798-or improvements to the components can be done quickly by the original author of that component.
799-
800-You can find the installation guide at :doc:`install`
801+A developer who just wants to use these components will be able to grab the components easily and keep them updated by simply running the ucs program.
802+
803+You can find the installation guide, to install *ucs* so that you can use it to add components to your Ubuntu SDK apps, at :doc:`install`.
804+
805+A component developer can create a component and contribute it to the Ubuntu Component Store. There are two types of components: *Curated* and *Community*. A component developer can publish their component to the Community store by themselves, with no oversight. Publishing to the Curated store requires approval that the curated component is well-constructed: it must have good documentation and tests, and the component developer will join the store team and maintail their component. This leads to the Curated components being superior.
806+
807+If you want to contribute a component, you can find the contribution guide at :doc:`contribute`.
808
809 Main Contents
810 -------------
811
812=== modified file 'docs/install.rst'
813--- docs/install.rst 2014-11-12 12:03:47 +0000
814+++ docs/install.rst 2015-04-13 19:51:02 +0000
815@@ -1,27 +1,54 @@
816 Installation Guide
817 ==================
818
819-UCS components should be bundled inside the application's click package. Importing
820-the UCS components in your app is as simple as installing the *ucs* package and running
821-it from your application's components folder. To install *ucs*, we need to first add a PPA ::
822+Installing UCS itself
823+---------------------
824+
825+Before using *ucs* to install components, you first need to install *ucs* itself!
826+
827+To install *ucs*, we need to add a PPA ::
828
829 sudo add-apt-repository ppa:ubuntu-touch-community-dev/ppa
830 sudo apt-get update
831 sudo apt-get install ucs
832-
833+
834+Finding Components
835+------------------
836+
837+Use the *search* command to search for available components by name or description.::
838+
839+ $ ucs search state
840+ Matching curated components:
841+ EmptyState
842+ Matching community components:
843+ (no matches)
844+ $ ucs search podcast
845+ Matching curated components:
846+ (no matches)
847+ Matching community components:
848+ sil/GenericPodcastApp A component which manages a podcast RSS feed, with playback and display of episodes
849+
850+
851+You can list all components in the store, both curated and community components, with *ucs search*.
852+
853 Installing Components
854 ---------------------
855
856 You can install components individually using the *install* command, ::
857
858- ucs install EmptyState PageWithBottomEdge
859+ ucs install EmptyState
860+ ucs install PageWithBottomEdge
861+ ucs install sil/GenericPodcastApp
862
863 .. note:: Remember to run the *ucs* command from inside your application's component folder!
864-.. note:: You can install all components in one go by ``ucs install``
865
866 You can then use the components by importing it in your qml files by, ::
867
868- import "path/to/components/folder"
869+ import ubuntu_component_store.Curated.EmptyState 1.0
870+ import ubuntu_component_store.Curated.PageWithBottomEdge 1.0
871+ import ubuntu_component_store.sil.GenericPodcastApp 1.0
872+
873+.. note:: Community components are named as *developer/ComponentName*. Curated components are installed with just a ComponentName (like *ucs install EmptyState*), but when importing them in QML you must call them *ubuntu_component_store.Curated.EmptyState* as above.
874
875 Updating Components
876 -------------------
877@@ -29,7 +56,7 @@
878 You can also update the UCS components using the *ucs* script. It will automatically fetch the latest
879 upstream code. ::
880
881- ucs update EmptyState PageWithBottomEdge
882+ ucs update EmptyState
883
884 .. note:: Remember to run the *ucs* command from inside your application's component folder!
885
886
887=== modified file 'script/debian/changelog'
888--- script/debian/changelog 2014-11-09 15:37:52 +0000
889+++ script/debian/changelog 2015-04-13 19:51:02 +0000
890@@ -1,3 +1,15 @@
891+ucs (0.2) trusty; urgency=medium
892+
893+ [Stuart Langridge]
894+ * Added community channel which allows developers to submit components
895+ easily to ucs.
896+ * Converted ucs script to python
897+
898+ [Nekhelesh Ramananthan]
899+ * Fixed UCS Gallery app and debian packaging
900+
901+ -- Nekhelesh Ramananthan <krnekhelesh@gmail.com> Tue, 06 Jan 2015 15:57:14 +0100
902+
903 ucs (0.1.2) trusty; urgency=medium
904
905 * Improved install and update command to use less network bandwidth by only
906
907=== modified file 'script/debian/control'
908--- script/debian/control 2014-11-08 22:37:17 +0000
909+++ script/debian/control 2015-04-13 19:51:02 +0000
910@@ -10,7 +10,8 @@
911 Package: ucs
912 Architecture: all
913 Depends: ${misc:Depends},
914- bzr
915+ bzr,
916+ python-bzrlib
917 Description: Ubuntu Component Store
918 Ubuntu Component Store is a library hosting a collection of component and
919 modules that app developers can use as plug-and-play components instead of
920
921=== added file 'script/ucs'
922--- script/ucs 1970-01-01 00:00:00 +0000
923+++ script/ucs 2015-04-13 19:51:02 +0000
924@@ -0,0 +1,531 @@
925+#!/usr/bin/python
926+# has to be Python2 because there's no python3 bzrlib
927+
928+import sys, argparse, os, time, codecs, json, urllib, urllib2, re, subprocess
929+from gi.repository import GLib
930+from bzrlib.plugin import load_plugins
931+load_plugins()
932+from bzrlib.branch import Branch
933+
934+class Project(object):
935+ def __init__(self, rootfolder):
936+ self.rootfolder = rootfolder
937+
938+ def add_to_import_path(self, component_type):
939+ pass
940+
941+ def copy_branch_folder_contents_to(self, branch, tree, folder, destfolder):
942+ # Ensure destfolder exists
943+ try:
944+ os.makedirs(destfolder)
945+ except OSError, e:
946+ if e.errno == 17:
947+ pass
948+ else:
949+ raise
950+ # Copy immediate file children of folder to it (not child folders or their contents)
951+ copied_files = []
952+ for path, versioned, kind, file_id, inventory_entry in tree.list_files(from_dir=folder, recursive=False):
953+ if kind == "file":
954+ fp = open(os.path.join(destfolder, path), "wb")
955+ fp.write(tree.get_file_text(file_id))
956+ fp.close()
957+ copied_files.append(path)
958+ return copied_files
959+
960+ def install_component_from_branch_folder(self, branch, tree, folder, component, component_type):
961+ if component_type == "qml":
962+ destfolder = os.path.join(self.rootfolder, "ubuntu_component_store",
963+ component["name"] + "." + component["version"])
964+ copied_files = self.copy_branch_folder_contents_to(branch, tree, folder, destfolder)
965+ # create qml qmldir file
966+ qmlfiles = []
967+ for f in copied_files:
968+ if f.endswith(".qml"):
969+ qmlfiles.append("%s %s %s" % (os.path.splitext(f)[0], component["version"], f))
970+ if qmlfiles:
971+ username, cname = component["name"].split("/")
972+ cname = cname[0].upper() + cname[1:]
973+ qmldir = ["module ubuntu_component_store.%s.%s" % (username, cname)]
974+ qmldir += qmlfiles
975+ qmldirfn = os.path.join(destfolder, "qmldir")
976+ fp = codecs.open(qmldirfn, mode="w", encoding="utf8")
977+ fp.write("\n".join(qmldir))
978+ fp.write("\n")
979+ fp.close()
980+ elif component_type == "binary":
981+ fid = tree.path2id(folder)
982+ if not fid:
983+ raise Exception("Component has no '%s' folder" % (folder,))
984+ arches = []
985+ for path, versioned, kind, file_id, inventory_entry in tree.list_files(from_dir=folder, recursive=False):
986+ if kind == "directory":
987+ arches.append(path)
988+ for arch in arches:
989+ destfolder = os.path.join(self.rootfolder, "lib", arch, "ubuntu_component_store",
990+ component["name"] + "." + component["version"])
991+ copied_files = self.copy_branch_folder_contents_to(branch, tree, folder + "/" + arch, destfolder)
992+ # create binary qmldir file
993+ plugin_names = []
994+ for f in copied_files:
995+ m = re.match("^lib(?P<libname>[^z.]+)\.so", f)
996+ if m:
997+ plugin_names.append(m.groupdict()["libname"])
998+ if plugin_names:
999+ username, cname = component["name"].split("/")
1000+ cname = cname[0].upper() + cname[1:]
1001+ qmldir = ["module ubuntu_component_store.%s.%s" % (username, cname)]
1002+ qmldir += ["plugin %s" % x for x in plugin_names]
1003+ qmldirfn = os.path.join(destfolder, "qmldir")
1004+ fp = codecs.open(qmldirfn, mode="w", encoding="utf8")
1005+ fp.write("\n".join(qmldir))
1006+ fp.write("\n")
1007+ fp.close()
1008+ else:
1009+ raise Exception("Unknown component type '%s'" % (component_type,))
1010+
1011+ def add_to_import_path(self, component_type):
1012+ raise NotImplementedError
1013+
1014+ def add_to_project_specific_copy_path(self, folder):
1015+ raise NotImplementedError
1016+
1017+ def add_to_project_specific_import_path(self, folder):
1018+ raise NotImplementedError
1019+
1020+
1021+class PureQMLProject(Project):
1022+ def add_to_project_specific_import_path(self, folder):
1023+ "Add the passed folder to root/projname.qmlproject:importPaths"
1024+ projname = os.path.split(os.path.abspath(self.rootfolder))[1]
1025+ qmlprojfile = os.path.join(self.rootfolder, "%s.qmlproject" % projname)
1026+ if not os.path.exists(qmlprojfile):
1027+ raise Exception("Couldn't add import path to %s" % (qmlprojfile,))
1028+ fp = codecs.open(qmlprojfile, encoding="utf8")
1029+ data = fp.read()
1030+ fp.close()
1031+ ipre = "importPaths\s*:\s*\[(?P<paths>[^]]*)\]"
1032+ ip = re.search(ipre, data)
1033+ if not ip:
1034+ raise Exception("Couldn't add import path to %s" % (qmlprojfile,))
1035+ pathsline = ip.groupdict()["paths"]
1036+ paths = [re.sub('^"|"$', '', x.strip()) for x in pathsline.split(",")]
1037+ if folder in paths:
1038+ # this folder is already on the importPaths, so bail
1039+ print "done already"
1040+ return
1041+ pathsline += ', "%s"' % (folder,)
1042+ newpathline = "importPaths: [%s]" % pathsline
1043+ data = re.sub(ipre, newpathline, data)
1044+ fp = codecs.open(qmlprojfile, encoding="utf8", mode="w")
1045+ fp.write(data)
1046+ fp.close()
1047+
1048+ def add_to_project_specific_copy_path(self, folder):
1049+ "QML projects don't copy anything anywhere; they build in place. So this is not relevant."
1050+ pass
1051+
1052+ def add_to_import_path(self, component_type):
1053+ if component_type == "binary":
1054+ # since this is a binary component, add the part for our arch to the Qt Creator import path
1055+ # FIXME it would be nice to not have to shell out for this
1056+ ourarch = subprocess.check_output(["dpkg-architecture", "-qDEB_BUILD_GNU_TYPE"]).strip()
1057+ self.add_to_project_specific_import_path("./lib/%s" % (ourarch,))
1058+ # FIXME may need an add_to_project_specific_copy_path here too
1059+ elif component_type == "qml":
1060+ # qml projects build in place anyway
1061+ pass
1062+
1063+
1064+class QMakeProject(Project):
1065+ def add_to_import_path(self, component_type):
1066+ if component_type == "binary":
1067+ print ("This is a qmake project.\n"
1068+ "You will need to ensure that all the ./lib/(arch)/ paths are added to "
1069+ "your import path when running the project, and that they are included "
1070+ "in the build process so they are copied into the build folder.")
1071+ elif component_type == "qml":
1072+ print ("This is a qmake project.\n"
1073+ "You will need to ensure that the ubuntu_component_store folder is "
1074+ "added to your import path when running the project, and that it is "
1075+ "included in the build process so it is copied into the build folder.")
1076+
1077+class CMakeProject(QMakeProject):
1078+ def add_to_import_path(self, component_type):
1079+ if component_type == "binary":
1080+ print ("This is a cmake project.\n"
1081+ "You will need to ensure that all the ./lib/(arch)/ paths are added to "
1082+ "your import path when running the project, and that they are included "
1083+ "in the build process so they are copied into the build folder.")
1084+ elif component_type == "qml":
1085+ print ("This is a cmake project.\n"
1086+ "You will need to ensure that the ubuntu_component_store folder is "
1087+ "added to your import path when running the project, and that it is "
1088+ "included in the build process so it is copied into the build folder.")
1089+
1090+def get_project():
1091+ """Identify the current project type and return an appropriate wrapper.
1092+
1093+ Pure QML projects are in a folder Foo which has a Foo.qmlproject file in it.
1094+ qMake-based projects are in a folder Foo which has a Foo.pro file in it.
1095+ CMake-based projects are in a folder Foo which has CMakeLists.txt and that
1096+ file contains a project(Foo ...) line in it.
1097+ """
1098+
1099+ def get_class_for_folder(folder):
1100+ afolder = os.path.abspath(folder)
1101+ basename = os.path.split(afolder)[1]
1102+ cmakelistname = os.path.join(afolder, "CMakeLists.txt")
1103+ if os.path.exists(os.path.join(afolder, "%s.%s" % (basename, "qmlproject"))):
1104+ return PureQMLProject, afolder
1105+ elif os.path.exists(os.path.join(afolder, "%s.%s" % (basename, "pro"))):
1106+ return QMakeProject, afolder
1107+ elif os.path.exists(cmakelistname):
1108+ fp = codecs.open(cmakelistname, encoding="utf8")
1109+ lines = fp.read().split("\n")
1110+ fp.close()
1111+ for line in lines:
1112+ m = re.match("^\s*project\s*\(\s*(?P<project>[^\s]+)((\s+[^\s]+)*\s*)\s*$", line)
1113+ if m:
1114+ if m.groupdict()["project"] == basename:
1115+ return CMakeProject, afolder
1116+ elif os.path.abspath(afolder) == "/":
1117+ # got to the root and found nothing
1118+ return None, None
1119+ return get_class_for_folder(os.path.join(afolder, ".."))
1120+
1121+ project_class, root_folder = get_class_for_folder(os.curdir)
1122+ if not project_class:
1123+ raise Exception("Not in an Ubuntu SDK app: %s" % (os.curdir))
1124+ return project_class(root_folder)
1125+
1126+class Repository(object):
1127+ def install_component_from_branch_folder(self, branch, tree, folder, component_type, component):
1128+ """ Download a thing from an LP folder and install it.
1129+
1130+ To install a component we need to:
1131+ 1. fetch it from LP
1132+ 2. put it in the correct place.
1133+ For pure QML components, the component will be one or more QML files, which
1134+ are in the branch folder that we are passed, and they get put in:
1135+ <project root>/ubuntu_component_store/username/componentname/
1136+ For binary components, the component will be a libFoo.so file for each arch,
1137+ and the branch folder we are passed should contain files <arch>/libFoo.so
1138+ for each arch. We then set up:
1139+ <project root>/lib/$arch/ubuntu_component_store/username/componentname/
1140+ for each arch and copy the appropriate libFoo.so into each.
1141+ 3. Set up qmldir files.
1142+ Pure QML components get a qmldir file
1143+ <project root>/ubuntu_component_store/username/componentname/qmldir
1144+ with content
1145+ module RedRectangle
1146+ RedRectangle 1.0 RedRectangle.qml
1147+ Binary components (libFoo.so, in component myname/BarPlugin) get a qmldir file
1148+ <project root>/lib/$arch/ubuntu_component_store/myname/BarPlugin/qmldir
1149+ with content
1150+ module ubuntu_component_store.myname.BarPlugin
1151+ plugin Foo
1152+ 4. Make sure the components are on the import path.
1153+ Pure QML components do this automatically; they are on the import path because
1154+ their folder (ubuntu_component_store/username/componentname/) is in the project
1155+ root, and that's on the import path. Binary components do not, because the
1156+ <project root>/lib/$arch folder is *not* on the import path for running in Qt Creator.
1157+ (It *is* on the import path when packaged as a click and installed on Ubuntu;
1158+ the click-apparmor wrapper /usr/bin/aa-exec-click sets it.) So it's important to
1159+ ensure when installing a binary component that it's on the import path. This happens
1160+ in different ways depending on the type of project:
1161+
1162+ Pure QML projects
1163+ These have a projectname.qmlproject file in the project root, with an importPaths key. We add
1164+ "./lib/$arch" to it, for the arch that this machine is currently using. This is a
1165+ bit dodgy (adding arch-specific paths), but this importPaths key *already* contains
1166+ arch-specific paths (frex, "/usr/lib/x86_64-linux-gnu/qt5/qml") and so adding a project-specific
1167+ arch-specific path can't make that any worse.
1168+
1169+ CMake-based projects
1170+ FIXME: dunno yet
1171+
1172+ qMake-based projects
1173+ FIXME: dunno yet
1174+
1175+ """
1176+ project = get_project()
1177+ project.install_component_from_branch_folder(branch, tree, folder, component, component_type)
1178+ project.add_to_import_path(component_type)
1179+
1180+ def search_component_list(self, query, md):
1181+ matches = [{"componentname": x[0], "lcn": x[0].lower(),
1182+ "description": x[1].get("description", "(none)"),
1183+ "lcd": x[1].get("description", "").lower()}
1184+ for x in md.items()]
1185+ for word in query:
1186+ word = word.lower()
1187+ matches2 = []
1188+ for pkg in matches:
1189+ if word in pkg["lcn"] or word in pkg["lcd"]:
1190+ matches2.append(pkg)
1191+ matches = matches2
1192+ return matches
1193+
1194+class CuratedRepository(Repository):
1195+ UCS_BRANCH = "lp:~ubuntu-touch-community-dev/component-store/trunk.14.10"
1196+ COMPONENTS_ROOT = "curated-store/ComponentStore"
1197+
1198+ def __init__(self, args):
1199+ self.__refresh = args.refresh
1200+
1201+ def __get_branch(self):
1202+ b = Branch.open(self.UCS_BRANCH)
1203+ tree = b.basis_tree()
1204+ return b, tree
1205+
1206+ def __update_metadata_if_required(self):
1207+ cache_dir = GLib.get_user_cache_dir()
1208+ my_cache_dir = os.path.join(cache_dir, "ubuntu_component_store")
1209+ try:
1210+ os.makedirs(my_cache_dir)
1211+ except OSError, e:
1212+ if e.errno != 17:
1213+ raise
1214+
1215+ cached_curated_file = os.path.join(my_cache_dir, "ucs-curated-components-metadata.json")
1216+
1217+ branch, tree = self.__get_branch()
1218+ fetch_packages = False
1219+ try:
1220+ fp = codecs.open(cached_curated_file, encoding="utf8")
1221+ component_metadata = json.load(fp)
1222+ if branch.revno() > component_metadata.get("cached_revno", 0):
1223+ fetch_packages = True
1224+ except:
1225+ fetch_packages = True
1226+
1227+ if self.__refresh:
1228+ fetch_packages = True
1229+ self.__refresh = False
1230+
1231+ if fetch_packages:
1232+ component_metadata = {"cached_revno": branch.revno(), "components": {}}
1233+ cns = [path
1234+ for (path, versioned, kind, file_id, inventory_entry)
1235+ in tree.list_files(from_dir=self.COMPONENTS_ROOT)
1236+ if kind == "directory"
1237+ ]
1238+ for c in cns:
1239+ try:
1240+ mdpath = "%s/%s/%s" % (self.COMPONENTS_ROOT, c, "ubuntu_component_store.json")
1241+ file_id = tree.path2id(mdpath)
1242+ mdjson = tree.get_file_text(file_id)
1243+ md = json.loads(mdjson)
1244+ component_metadata["components"][c] = md
1245+ except:
1246+ print "(unable to get metadata for curated component %s)" % (c,)
1247+ fp = codecs.open(cached_curated_file, mode="w", encoding="utf8")
1248+ json.dump(component_metadata, fp)
1249+ fp.close()
1250+ return component_metadata
1251+
1252+ def install(self, cn):
1253+ print("(installing {0} from curated repository)".format(cn))
1254+ component_metadata = self.__update_metadata_if_required()
1255+ component = component_metadata["components"].get(cn)
1256+ if not component:
1257+ raise Exception("Unable to find curated component '%s'." % (cn,))
1258+ b, tree = self.__get_branch()
1259+ component_path = "/".join([self.COMPONENTS_ROOT, cn])
1260+ file_id = tree.path2id(component_path)
1261+ if not file_id: # should not happen because it's in the metadata
1262+ raise Exception("Unable to find curated component '%s'." % (cn,))
1263+ # Curated components are all QML-only
1264+ component["name"] = "Curated/%s" % (component["name"],)
1265+ self.install_component_from_branch_folder(b, tree, component_path, "qml", component)
1266+
1267+ def update(self, cn):
1268+ raise NotImplementedError
1269+
1270+ def search(self, query):
1271+ component_metadata = self.__update_metadata_if_required()
1272+ return self.search_component_list(query, component_metadata.get("components", {}))
1273+
1274+class CommunityRepository(Repository):
1275+ PACKAGES_FILE_BRANCH = "lp:~ubuntu-touch-community-dev/component-store/community-components-json"
1276+ PACKAGES_FILE_NAME = "community_components.json"
1277+
1278+ def __init__(self, args):
1279+ self.__refresh = args.refresh
1280+
1281+ def __update_packages_file_if_required(self):
1282+ cache_dir = GLib.get_user_cache_dir()
1283+ my_cache_dir = os.path.join(cache_dir, "ubuntu_component_store")
1284+ try:
1285+ os.makedirs(my_cache_dir)
1286+ except OSError, e:
1287+ if e.errno != 17:
1288+ raise
1289+
1290+ cached_packages_file = os.path.join(my_cache_dir, "ucs-packages-cache.json")
1291+
1292+ fetch_packages = False
1293+ try:
1294+ age = time.time() - os.stat(cached_packages_file).st_mtime
1295+ if age > 3600: fetch_packages = True
1296+ except OSError:
1297+ fetch_packages = True
1298+ if self.__refresh:
1299+ fetch_packages = True
1300+ self.__refresh = False
1301+
1302+ if fetch_packages:
1303+ packages_data = self.__fetch_packages_file(cached_packages_file)
1304+ else:
1305+ fp = codecs.open(cached_packages_file, encoding="utf8")
1306+ try:
1307+ packages_data = json.load(fp)
1308+ except:
1309+ raise Exception("Local package cache seems corrupt")
1310+ fp.close()
1311+ return packages_data
1312+
1313+ def __fetch_packages_file(self, cached_packages_file):
1314+ print "(getting packages file)"
1315+ b = Branch.open(self.PACKAGES_FILE_BRANCH)
1316+ tree = b.basis_tree()
1317+ file_id = tree.path2id(self.PACKAGES_FILE_NAME)
1318+ if not file_id:
1319+ raise Exception("Unable to find the packages file.")
1320+ try:
1321+ packages_data = tree.get_file_text(file_id)
1322+ except:
1323+ raise Exception("Unable to read the packages file.")
1324+ try:
1325+ packages_data = json.loads(packages_data)
1326+ except:
1327+ raise Exception("Master packages file seems corrupt")
1328+ fp = codecs.open(cached_packages_file, mode="w", encoding="utf8")
1329+ json.dump(packages_data, fp)
1330+ fp.close()
1331+ return packages_data
1332+
1333+ def install(self, cn):
1334+ packages_data = self.__update_packages_file_if_required()
1335+ print("(installing {0} from community repository)".format(cn))
1336+ pkg = packages_data.get("components", {}).get(cn)
1337+ if not pkg:
1338+ print("Package {0} not found".format(cn))
1339+ return
1340+ print("OK installing.")
1341+ b = Branch.open(pkg["lpurl"])
1342+ tree = b.repository.revision_tree(b.get_rev_id(pkg["revno"]))
1343+ if "qml" in pkg["type"]:
1344+ self.install_component_from_branch_folder(b, tree, "qml", "qml", pkg)
1345+ if "binary" in pkg["type"]:
1346+ self.install_component_from_branch_folder(b, tree, "qmllib", "binary", pkg)
1347+
1348+ def update(self, cn):
1349+ raise NotImplementedError
1350+
1351+ def search(self, query):
1352+ packages_data = self.__update_packages_file_if_required()
1353+ return self.search_component_list(query, packages_data.get("components", {}))
1354+
1355+ def submit(self, lpurl):
1356+
1357+ def chunk_read(response):
1358+ read = ""
1359+
1360+ while 1:
1361+ chunk = response.read(1)
1362+ read += chunk
1363+
1364+ if "\n" in read:
1365+ lines = read.split("\n")
1366+ read = lines.pop(-1)
1367+ lines = [x.split(":", 1) for x in lines]
1368+ if lines:
1369+ for x in lines:
1370+ if x[0] != "ok":
1371+ print "Error: %s" % x[1]
1372+ return False
1373+ print "\n".join([x[1] for x in lines])
1374+
1375+ if not chunk:
1376+ print read
1377+ if not read.startswith("ok:"): return False
1378+ return True
1379+ return True
1380+
1381+ try:
1382+ response = urllib2.urlopen('https://sil.pythonanywhere.com/submit', data=urllib.urlencode({"lpurl": lpurl}));
1383+ except urllib2.HTTPError, e:
1384+ print("Error while submitting:")
1385+ print(e.read())
1386+ return
1387+ ok = chunk_read(response)
1388+ if not ok: sys.exit(1)
1389+
1390+def get_repository_from_component_name(args):
1391+ if "/" in args.componentname:
1392+ return CommunityRepository(args)
1393+ else:
1394+ return CuratedRepository(args)
1395+
1396+def cmd_install(args):
1397+ get_repository_from_component_name(args).install(args.componentname)
1398+def cmd_update(args):
1399+ get_repository_from_component_name(args).update(args.componentname)
1400+def cmd_search(args):
1401+ print("Matching curated components")
1402+ print("===========================")
1403+ cu = CuratedRepository(args)
1404+ results = cu.search(args.query)
1405+ for result in results:
1406+ print(" {componentname:>30} {description}".format(**result))
1407+ if not results:
1408+ print(" {0:>30}".format("(no matches)"))
1409+ print
1410+ print("Matching community components")
1411+ print("=============================")
1412+ co = CommunityRepository(args)
1413+ results = co.search(args.query)
1414+ for result in results:
1415+ print(" {componentname:>30} {description}".format(**result))
1416+ if not results:
1417+ print(" {0:>30}".format("(no matches)"))
1418+
1419+def cmd_submit(args):
1420+ print("(submitting to community repository)")
1421+ args.refresh = False
1422+ co = CommunityRepository(args)
1423+ co.submit(args.launchpadBranchURL)
1424+
1425+if __name__ == "__main__":
1426+ parser = argparse.ArgumentParser(description="Install and manage third-party components into your Ubuntu SDK app")
1427+ subparsers = parser.add_subparsers(dest="subparser_name")
1428+ parser_install = subparsers.add_parser("install", help="Install a component into this app")
1429+ parser_install.add_argument("componentname", type=str,
1430+ help="ComponentName for curated components, developer/ComponentName for community components")
1431+ parser_install.add_argument("--refresh", action="store_true", help="Refresh the packages list")
1432+ parser_install.set_defaults(func=cmd_install)
1433+ parser_update = subparsers.add_parser("update", help="Update an installed component in this app")
1434+ parser_update.add_argument("componentname", type=str,
1435+ help="ComponentName for curated components, developer/ComponentName for community components")
1436+ parser_update.add_argument("--refresh", action="store_true", help="Refresh the packages list")
1437+ parser_update.set_defaults(func=cmd_update)
1438+ parser_search = subparsers.add_parser("search", help="Search names and descriptions of existing components")
1439+ parser_search.add_argument("query", type=str, nargs="*", help="Search terms")
1440+ parser_search.add_argument("--refresh", action="store_true", help="Refresh the packages list")
1441+ parser_search.set_defaults(func=cmd_search)
1442+ parser_submit = subparsers.add_parser("submit", help="Publish a component to the community store")
1443+ parser_submit.add_argument("launchpadBranchURL", type=str, help="lp:~username/project/branch")
1444+ parser_submit.set_defaults(func=cmd_submit)
1445+ parser_help = subparsers.add_parser("help", help="Help on subcommands")
1446+ parser_help.add_argument("subcommand", nargs="?", type=str)
1447+
1448+ args = parser.parse_args()
1449+ if args.subparser_name == "help":
1450+ if args.subcommand:
1451+ parser.parse_args([args.subcommand, "-h"])
1452+ else:
1453+ parser.parse_args(["-h"])
1454+ else:
1455+ args.func(args)
1456
1457=== removed file 'script/ucs'
1458--- script/ucs 2014-11-09 15:36:15 +0000
1459+++ script/ucs 1970-01-01 00:00:00 +0000
1460@@ -1,115 +0,0 @@
1461-#!/bin/bash
1462-
1463-# Branch specs
1464-PWD=$(pwd)
1465-ARGS=("$@")
1466-UCS_BRANCH=lp:component-store
1467-UCS_FILEPATH=lp:~ubuntu-touch-community-dev/component-store/trunk.14.10/ComponentStore
1468-BOOTSTRAP_FOLDER=${PWD}/ComponentStoreSetup
1469-
1470-# Color Output
1471-RC='\e[0;31m'
1472-GC='\e[0;32m'
1473-NC='\e[0m'
1474-
1475-show_usage() {
1476- echo "Usage:"
1477- echo " ucs <command> [options]"
1478- echo ""
1479- echo "Commands:"
1480- echo -e " install \tInstall Components"
1481- echo -e " update \tUpdate Components"
1482- echo -e " help \tdisplay this help and exit"
1483- echo -e " version \toutput version information and exit"
1484-}
1485-
1486-install_prerequisites() {
1487- if [ $(dpkg-query -W -f='${Status}' bzr 2>/dev/null | grep -c "ok installed") -eq 0 ]; then
1488- echo "bzr is not installed...installing it now.."
1489- sudo apt-get install -y bzr --no-install-recommends;
1490- fi
1491-}
1492-
1493-if [ $# -lt 1 ]
1494-then
1495- show_usage
1496- exit 1
1497-fi
1498-
1499-#CASE: HELP ARGUMENT
1500-if [[ $1 == "help" ]]; then
1501- show_usage
1502- exit 0
1503-
1504-#CASE: ABOUT ARGUMENT
1505-elif [[ $1 == "version" ]]; then
1506- echo "Ubuntu Component Store (UCS) v0.1.2"
1507- echo "Copyright (C) 2014 Ubuntu Touch Community Dev Team"
1508- echo "License GPLv3: GNU version 3 <http://gnu.org/licenses/gpl.html>."
1509- echo "This is free software: you are free to change and redistribute it."
1510- echo "There is NO WARRANTY, to the extent permitted by law."
1511- echo
1512- echo "Written by Ubuntu Touch Community Dev"
1513- exit 0
1514-
1515-#CASE: INSTALL ARGUMENT
1516-elif [[ $1 == "install" ]]; then
1517- install_prerequisites
1518- if [ $# -gt 1 ]; then
1519- for (( i=1; i<$#; i++ ))
1520- do
1521- echo -n "[$i/`expr $# - 1`] Installing ${ARGS[$i]}..."
1522- componentfilelist=(`bzr ls -q $UCS_FILEPATH/${ARGS[$i]}`)
1523- componentfilelistcount=${#componentfilelist[@]}
1524- if (( componentfilelistcount == 0 )); then
1525- echo -e "${RC}FAILED${NC} (Invalid component name)"
1526- else
1527- for (( j=0; j<componentfilelistcount; j++ ))
1528- do
1529- filename=$(basename ${componentfilelist[j]})
1530- bzr cat -q ${componentfilelist[j]} > ${PWD}/${filename}
1531- done
1532- echo -e "${GC}DONE${NC}"
1533- fi
1534- done
1535- else
1536- echo -n "Installing all components..."
1537- componentlist=(`bzr ls -q $UCS_FILEPATH`)
1538- componentlistcount=${#componentlist[@]}
1539- for (( i=0; i<componentlistcount; i++ ))
1540- do
1541- componentfilelist=(`bzr ls -q ${componentlist[i]}`)
1542- componentfilelistcount=${#componentfilelist[@]}
1543- for (( j=0; j<componentfilelistcount; j++ ))
1544- do
1545- bzr cat -q ${componentlist[i]}${componentfilelist[j]} > ${PWD}/${componentfilelist[j]}
1546- done
1547- done
1548- echo -e "${GC}DONE${NC}"
1549- fi
1550- echo -e "${GC}Installation Complete!${NC}"
1551-
1552-#CASE: UPDATE ARGUMENT
1553-elif [[ $1 == "update" ]]; then
1554- if [ $# -gt 1 ]; then
1555- for (( i=1; i<$#; i++ ))
1556- do
1557- echo -n "[$i/`expr $# - 1`] Updating ${ARGS[$i]}..."
1558- componentfilelist=(`bzr ls -q $UCS_FILEPATH/${ARGS[$i]}`)
1559- componentfilelistcount=${#componentfilelist[@]}
1560- if (( componentfilelistcount == 0 )); then
1561- echo -e "${RC}FAILED${NC} (Invalid component name)"
1562- else
1563- for (( j=0; j<componentfilelistcount; j++ ))
1564- do
1565- filename=$(basename ${componentfilelist[j]})
1566- bzr cat -q ${componentfilelist[j]} > ${PWD}/${filename}
1567- done
1568- echo -e "${GC}DONE${NC}"
1569- fi
1570- done
1571- echo -e "${GC}Updating Components Complete!${NC}"
1572- else
1573- echo -e "${RC}Error updating components! Please specify the component name you want to update. Eg. ucs update EmptyState${NC}"
1574- fi
1575-fi

Subscribers

People subscribed via source and target branches