Merge lp:~mvo/click/repository into lp:click/devel
- repository
- Merge into devel
Status: | Needs review |
---|---|
Proposed branch: | lp:~mvo/click/repository |
Merge into: | lp:click/devel |
Diff against target: |
546 lines (+416/-16) 8 files modified
click/commands/__init__.py (+2/-0) click/commands/info.py (+58/-5) click/commands/search.py (+46/-0) click/commands/update.py (+78/-0) click/install.py (+7/-6) click/repository.py (+220/-0) click/tests/test_install.py (+2/-2) debian/control (+3/-3) |
To merge this branch: | bzr merge lp:~mvo/click/repository |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
PS Jenkins bot (community) | continuous-integration | Approve | |
Colin Watson | Approve | ||
Review via email: mp+237604@code.launchpad.net |
Commit message
Add support for remote repositories to click.
Description of the change
This branch adds a new "repository" concept for packages coming from a remote location. It also adds "click search", "click update", "click info --remote" to query the repository.
At this point its mostly up for review to get feedback on the approach used and if the CLI interface fits the vision of click. But once the acquire branch lands and my sso branch gets into better shape this will allow managing clicks on a server image from the commandline easily.
Feedback welcome!
PS Jenkins bot (ps-jenkins) wrote : | # |
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:529
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:530
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 528. By Michael Vogt
-
debian/control: add python3-pycurl
- 529. By Michael Vogt
-
click/commands/
update. py: add missing file - 530. By Michael Vogt
-
pep8 fixes
- 531. By Michael Vogt
-
merge lp:click/devel
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:531
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Colin Watson (cjwatson) wrote : | # |
I'm happy with the general direction, yes, thanks. Some specific comments / nitpicking below.
Presumably this could ultimately replace the current app updates handling on the phone as well, perhaps by way of exposing some of this via D-Bus?
- 532. By Michael Vogt
-
use the bulk interface that click-update-
manager is using to find available updates - 533. By Michael Vogt
-
address review comments by Colin (thanks!)
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:532
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Michael Vogt (mvo) wrote : | # |
> I'm happy with the general direction, yes, thanks. Some specific comments /
> nitpicking below.
>
> Presumably this could ultimately replace the current app updates handling on
> the phone as well, perhaps by way of exposing some of this via D-Bus?
Thanks a bunch! I replied to your comments inline, the ones I did not reply to are fixed in my branch now.
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:533
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 534. By Michael Vogt
-
click/repositor
y.py: fix crash
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:534
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 535. By Michael Vogt
-
add docstrings/cleanup to ClickRepository
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:535
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 536. By Michael Vogt
-
click/commands/
update. py. use prettytable to generate table output - 537. By Michael Vogt
-
click/commands/
update. py: add --machine-readable
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:537
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 538. By Michael Vogt
-
click/repositor
y.py: add description property to a click repository - 539. By Michael Vogt
-
click/commands/
info.py: always search local/remote click names in click info
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:539
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 540. By Michael Vogt
-
merged from lp:click/devel
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:540
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 541. By Michael Vogt
-
click/repositor
y.py: do not hit the network on update if nothing is installed
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:541
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 542. By Michael Vogt
-
add new Repository.
credentails property
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:542
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 543. By Michael Vogt
-
click/commands/
update. py: support "click update click1 click2" style invocation
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:543
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 544. By Michael Vogt
-
click/commands/
update. py: add --user/--all-users - 545. By Michael Vogt
-
click/commands/
info.py: fix pep8 failure
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:544
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 546. By Michael Vogt
-
add missing python3-prettytable build dependency
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:546
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 547. By Michael Vogt
-
set the right search headers
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:547
http://
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
- 548. By Michael Vogt
-
fix tests
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:548
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 549. By Michael Vogt
-
click/repositor
y.py: use application/ hal+json instead of application/json
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:549
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 550. By Michael Vogt
-
click/repositor
y.py: the ClickRepository AppsUbuntuCom. details( ) call needs to send framework info in the headers too) - 551. By Michael Vogt
-
click/repositor
y.py: fix headers in get_upgradable - 552. By Michael Vogt
-
revert r551 - the myapps.
developer. u.c API breaks with that - 553. By Michael Vogt
-
merged lp:click/devel
- 554. By Michael Vogt
-
click/commands/
info.py: add info -v to address Colins review comment - 555. By Michael Vogt
-
click/repositor
y.py: use the bulk uri - 556. By Michael Vogt
-
click/repositor
y.py: silly typo
Unmerged revisions
- 556. By Michael Vogt
-
click/repositor
y.py: silly typo - 555. By Michael Vogt
-
click/repositor
y.py: use the bulk uri - 554. By Michael Vogt
-
click/commands/
info.py: add info -v to address Colins review comment - 553. By Michael Vogt
-
merged lp:click/devel
- 552. By Michael Vogt
-
revert r551 - the myapps.
developer. u.c API breaks with that - 551. By Michael Vogt
-
click/repositor
y.py: fix headers in get_upgradable - 550. By Michael Vogt
-
click/repositor
y.py: the ClickRepository AppsUbuntuCom. details( ) call needs to send framework info in the headers too) - 549. By Michael Vogt
-
click/repositor
y.py: use application/ hal+json instead of application/json - 548. By Michael Vogt
-
fix tests
- 547. By Michael Vogt
-
set the right search headers
Preview Diff
1 | === modified file 'click/commands/__init__.py' | |||
2 | --- click/commands/__init__.py 2014-05-20 09:03:05 +0000 | |||
3 | +++ click/commands/__init__.py 2015-02-24 16:14:00 +0000 | |||
4 | @@ -31,6 +31,8 @@ | |||
5 | 31 | "list", | 31 | "list", |
6 | 32 | "pkgdir", | 32 | "pkgdir", |
7 | 33 | "register", | 33 | "register", |
8 | 34 | "search", | ||
9 | 35 | "update", | ||
10 | 34 | "unregister", | 36 | "unregister", |
11 | 35 | "verify", | 37 | "verify", |
12 | 36 | ) | 38 | ) |
13 | 37 | 39 | ||
14 | === modified file 'click/commands/info.py' | |||
15 | --- click/commands/info.py 2014-10-07 08:56:03 +0000 | |||
16 | +++ click/commands/info.py 2015-02-24 16:14:00 +0000 | |||
17 | @@ -28,6 +28,10 @@ | |||
18 | 28 | 28 | ||
19 | 29 | from click.install import DebFile | 29 | from click.install import DebFile |
20 | 30 | from click.json_helpers import json_object_to_python | 30 | from click.json_helpers import json_object_to_python |
21 | 31 | from click.repository import ( | ||
22 | 32 | get_repository, | ||
23 | 33 | ClickRepositoryError, | ||
24 | 34 | ) | ||
25 | 31 | 35 | ||
26 | 32 | 36 | ||
27 | 33 | def _load_manifest(manifest_file): | 37 | def _load_manifest(manifest_file): |
28 | @@ -73,14 +77,63 @@ | |||
29 | 73 | "--user", metavar="USER", | 77 | "--user", metavar="USER", |
30 | 74 | help="look up PACKAGE-NAME for USER (if you have permission; " | 78 | help="look up PACKAGE-NAME for USER (if you have permission; " |
31 | 75 | "default: current user)") | 79 | "default: current user)") |
32 | 80 | parser.add_option( | ||
33 | 81 | "--repository", default=False, action="store_true", | ||
34 | 82 | help="Search in remote repository only") | ||
35 | 83 | parser.add_option( | ||
36 | 84 | "--no-repository", default=False, action="store_true", | ||
37 | 85 | help="Never search in a remote repository") | ||
38 | 86 | parser.add_option( | ||
39 | 87 | "-v", "--verbose", default=False, action="store_true", | ||
40 | 88 | help="Show verbose information") | ||
41 | 76 | options, args = parser.parse_args(argv) | 89 | options, args = parser.parse_args(argv) |
42 | 90 | pkgname = args[0] | ||
43 | 77 | if len(args) < 1: | 91 | if len(args) < 1: |
44 | 78 | parser.error("need file name") | 92 | parser.error("need file name") |
50 | 79 | try: | 93 | |
51 | 80 | manifest = get_manifest(options, args[0]) | 94 | # see if its available locally: pkgname, click-path, path-to-unpacked-click |
52 | 81 | except Exception as e: | 95 | manifest = None |
53 | 82 | print(e, file=sys.stderr) | 96 | if not options.repository: |
54 | 83 | return 1 | 97 | try: |
55 | 98 | manifest = get_manifest(options, pkgname) | ||
56 | 99 | except Exception as e: | ||
57 | 100 | pass | ||
58 | 101 | |||
59 | 102 | # now check remote | ||
60 | 103 | if manifest is None and not options.no_repository: | ||
61 | 104 | repo = get_repository() | ||
62 | 105 | try: | ||
63 | 106 | details = repo.details(pkgname) | ||
64 | 107 | if options.verbose: | ||
65 | 108 | # all information from the store, lots of stuff | ||
66 | 109 | manifest = details | ||
67 | 110 | else: | ||
68 | 111 | # by default show only a subset of all the store information | ||
69 | 112 | # by default, it contains a lot of additional info | ||
70 | 113 | # like hashes | ||
71 | 114 | manifest = {k: details.get(k) | ||
72 | 115 | for k in ('name', | ||
73 | 116 | 'title', | ||
74 | 117 | 'description', | ||
75 | 118 | 'binary_filesize', | ||
76 | 119 | 'publisher', | ||
77 | 120 | 'framework', | ||
78 | 121 | 'version', | ||
79 | 122 | 'last_updated', | ||
80 | 123 | 'architecture', | ||
81 | 124 | 'department', | ||
82 | 125 | 'changelog', | ||
83 | 126 | 'ratings_average', | ||
84 | 127 | 'price', | ||
85 | 128 | 'keywords', | ||
86 | 129 | 'license', | ||
87 | 130 | )} | ||
88 | 131 | manifest["_repository"] = repo.description | ||
89 | 132 | except ClickRepositoryError as e: | ||
90 | 133 | print( | ||
91 | 134 | "Can not get details for %s:%s" % (pkgname, e), file=sys.stderr) | ||
92 | 135 | return 1 | ||
93 | 136 | |||
94 | 84 | json.dump( | 137 | json.dump( |
95 | 85 | manifest, sys.stdout, ensure_ascii=False, sort_keys=True, indent=4, | 138 | manifest, sys.stdout, ensure_ascii=False, sort_keys=True, indent=4, |
96 | 86 | separators=(",", ": ")) | 139 | separators=(",", ": ")) |
97 | 87 | 140 | ||
98 | === added file 'click/commands/search.py' | |||
99 | --- click/commands/search.py 1970-01-01 00:00:00 +0000 | |||
100 | +++ click/commands/search.py 2015-02-24 16:14:00 +0000 | |||
101 | @@ -0,0 +1,46 @@ | |||
102 | 1 | # Copyright (C) 2014 Canonical Ltd. | ||
103 | 2 | # Author: Michael Vogt | ||
104 | 3 | |||
105 | 4 | # This program is free software: you can redistribute it and/or modify | ||
106 | 5 | # it under the terms of the GNU General Public License as published by | ||
107 | 6 | # the Free Software Foundation; version 3 of the License. | ||
108 | 7 | # | ||
109 | 8 | # This program is distributed in the hope that it will be useful, | ||
110 | 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
111 | 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
112 | 11 | # GNU General Public License for more details. | ||
113 | 12 | # | ||
114 | 13 | # You should have received a copy of the GNU General Public License | ||
115 | 14 | # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
116 | 15 | |||
117 | 16 | """Search for Click packages""" | ||
118 | 17 | |||
119 | 18 | from __future__ import print_function | ||
120 | 19 | |||
121 | 20 | from optparse import OptionParser | ||
122 | 21 | import sys | ||
123 | 22 | from textwrap import dedent | ||
124 | 23 | |||
125 | 24 | from click.repository import ( | ||
126 | 25 | get_repository, | ||
127 | 26 | ClickRepositoryError, | ||
128 | 27 | ) | ||
129 | 28 | |||
130 | 29 | |||
131 | 30 | def run(argv): | ||
132 | 31 | parser = OptionParser(dedent("""\ | ||
133 | 32 | %prog search search-term(s) | ||
134 | 33 | |||
135 | 34 | """)) | ||
136 | 35 | options, args = parser.parse_args(argv) | ||
137 | 36 | if len(args) < 1: | ||
138 | 37 | parser.error("need package file name") | ||
139 | 38 | repo = get_repository() | ||
140 | 39 | try: | ||
141 | 40 | result = repo.search(args[0]) | ||
142 | 41 | except ClickRepositoryError as e: | ||
143 | 42 | print("Search failed %s" % e, file=sys.stderr) | ||
144 | 43 | return 1 | ||
145 | 44 | for app in result: | ||
146 | 45 | print("%s - %s" % (app["name"], app["title"])) | ||
147 | 46 | return 0 | ||
148 | 0 | 47 | ||
149 | === added file 'click/commands/update.py' | |||
150 | --- click/commands/update.py 1970-01-01 00:00:00 +0000 | |||
151 | +++ click/commands/update.py 2015-02-24 16:14:00 +0000 | |||
152 | @@ -0,0 +1,78 @@ | |||
153 | 1 | # Copyright (C) 2014 Canonical Ltd. | ||
154 | 2 | # Author: Michael Vogt | ||
155 | 3 | |||
156 | 4 | # This program is free software: you can redistribute it and/or modify | ||
157 | 5 | # it under the terms of the GNU General Public License as published by | ||
158 | 6 | # the Free Software Foundation; version 3 of the License. | ||
159 | 7 | # | ||
160 | 8 | # This program is distributed in the hope that it will be useful, | ||
161 | 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
162 | 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
163 | 11 | # GNU General Public License for more details. | ||
164 | 12 | # | ||
165 | 13 | # You should have received a copy of the GNU General Public License | ||
166 | 14 | # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
167 | 15 | |||
168 | 16 | """Update Click packages""" | ||
169 | 17 | |||
170 | 18 | from __future__ import print_function | ||
171 | 19 | |||
172 | 20 | from optparse import OptionParser | ||
173 | 21 | import sys | ||
174 | 22 | from textwrap import dedent | ||
175 | 23 | |||
176 | 24 | from prettytable import PrettyTable, PLAIN_COLUMNS | ||
177 | 25 | |||
178 | 26 | from click.repository import ( | ||
179 | 27 | get_repository, | ||
180 | 28 | ClickRepositoryError, | ||
181 | 29 | ) | ||
182 | 30 | |||
183 | 31 | |||
184 | 32 | def run(argv): | ||
185 | 33 | parser = OptionParser(dedent("""\ | ||
186 | 34 | %prog update | ||
187 | 35 | |||
188 | 36 | """)) | ||
189 | 37 | parser.add_option( | ||
190 | 38 | "--list", default=False, action="store_true", | ||
191 | 39 | help="Just list the update") | ||
192 | 40 | parser.add_option( | ||
193 | 41 | "--machine-readable", default=False, action="store_true", | ||
194 | 42 | help="output in machine readable form") | ||
195 | 43 | parser.add_option( | ||
196 | 44 | "--user", metavar="USER", help="register package for USER") | ||
197 | 45 | parser.add_option( | ||
198 | 46 | "--all-users", default=False, action="store_true", | ||
199 | 47 | help="register package for all users") | ||
200 | 48 | options, args = parser.parse_args(argv) | ||
201 | 49 | repo = get_repository() | ||
202 | 50 | try: | ||
203 | 51 | all_updates = repo.get_upgradable() | ||
204 | 52 | except ClickRepositoryError as e: | ||
205 | 53 | print("Update failed: %s" % e, file=sys.stderr) | ||
206 | 54 | return 1 | ||
207 | 55 | if not all_updates: | ||
208 | 56 | return 0 | ||
209 | 57 | |||
210 | 58 | if len(args) > 0: | ||
211 | 59 | updates = [up for up in all_updates if up[0]["name"] in args] | ||
212 | 60 | else: | ||
213 | 61 | updates = all_updates | ||
214 | 62 | |||
215 | 63 | if options.machine_readable: | ||
216 | 64 | for (current, new) in updates: | ||
217 | 65 | print("%s|%s|%s" % ( | ||
218 | 66 | new["name"], current["version"], new["version"])) | ||
219 | 67 | elif options.list: | ||
220 | 68 | t = PrettyTable(["Part","Installed", "Available"]) | ||
221 | 69 | t.set_style(PLAIN_COLUMNS) | ||
222 | 70 | t.align["Part"] = "l" | ||
223 | 71 | for (current, new) in updates: | ||
224 | 72 | t.add_row((new["name"], current["version"], new["version"])) | ||
225 | 73 | print(t) | ||
226 | 74 | else: | ||
227 | 75 | for (current, new) in updates: | ||
228 | 76 | print("downloading %s" % new["download_url"]) | ||
229 | 77 | |||
230 | 78 | return 0 | ||
231 | 0 | 79 | ||
232 | === modified file 'click/install.py' | |||
233 | --- click/install.py 2014-12-03 12:42:21 +0000 | |||
234 | +++ click/install.py 2015-02-24 16:14:00 +0000 | |||
235 | @@ -76,6 +76,12 @@ | |||
236 | 76 | apt_pkg.init_system() | 76 | apt_pkg.init_system() |
237 | 77 | 77 | ||
238 | 78 | 78 | ||
239 | 79 | def _dpkg_architecture(): | ||
240 | 80 | return subprocess.check_output( | ||
241 | 81 | ["dpkg", "--print-architecture"], | ||
242 | 82 | universal_newlines=True).rstrip("\n") | ||
243 | 83 | |||
244 | 84 | |||
245 | 79 | class DebsigVerifyError(Exception): | 85 | class DebsigVerifyError(Exception): |
246 | 80 | pass | 86 | pass |
247 | 81 | 87 | ||
248 | @@ -145,11 +151,6 @@ | |||
249 | 145 | return os.path.abspath(preload) | 151 | return os.path.abspath(preload) |
250 | 146 | return preload_path | 152 | return preload_path |
251 | 147 | 153 | ||
252 | 148 | def _dpkg_architecture(self): | ||
253 | 149 | return subprocess.check_output( | ||
254 | 150 | ["dpkg", "--print-architecture"], | ||
255 | 151 | universal_newlines=True).rstrip("\n") | ||
256 | 152 | |||
257 | 153 | def extract(self, path, target): | 154 | def extract(self, path, target): |
258 | 154 | command = ["dpkg-deb", "-R", path, target] | 155 | command = ["dpkg-deb", "-R", path, target] |
259 | 155 | with open(path, "rb") as fd: | 156 | with open(path, "rb") as fd: |
260 | @@ -263,7 +264,7 @@ | |||
261 | 263 | if check_arch: | 264 | if check_arch: |
262 | 264 | architecture = manifest.get("architecture", "all") | 265 | architecture = manifest.get("architecture", "all") |
263 | 265 | if architecture != "all": | 266 | if architecture != "all": |
265 | 266 | dpkg_architecture = self._dpkg_architecture() | 267 | dpkg_architecture = _dpkg_architecture() |
266 | 267 | if isinstance(architecture, list): | 268 | if isinstance(architecture, list): |
267 | 268 | if dpkg_architecture not in architecture: | 269 | if dpkg_architecture not in architecture: |
268 | 269 | raise ClickInstallerAuditError( | 270 | raise ClickInstallerAuditError( |
269 | 270 | 271 | ||
270 | === added file 'click/repository.py' | |||
271 | --- click/repository.py 1970-01-01 00:00:00 +0000 | |||
272 | +++ click/repository.py 2015-02-24 16:14:00 +0000 | |||
273 | @@ -0,0 +1,220 @@ | |||
274 | 1 | # Copyright (C) 2014 Canonical Ltd. | ||
275 | 2 | # Author: Michael Vogt | ||
276 | 3 | |||
277 | 4 | # This program is free software: you can redistribute it and/or modify | ||
278 | 5 | # it under the terms of the GNU General Public License as published by | ||
279 | 6 | # the Free Software Foundation; version 3 of the License. | ||
280 | 7 | # | ||
281 | 8 | # This program is distributed in the hope that it will be useful, | ||
282 | 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
283 | 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
284 | 11 | # GNU General Public License for more details. | ||
285 | 12 | # | ||
286 | 13 | # You should have received a copy of the GNU General Public License | ||
287 | 14 | # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
288 | 15 | |||
289 | 16 | """Search Click packages.""" | ||
290 | 17 | |||
291 | 18 | from __future__ import print_function | ||
292 | 19 | |||
293 | 20 | __metaclass__ = type | ||
294 | 21 | __all__ = [ | ||
295 | 22 | 'ClickRepository', | ||
296 | 23 | 'ClickRepositoryError', | ||
297 | 24 | ] | ||
298 | 25 | |||
299 | 26 | from io import BytesIO, StringIO | ||
300 | 27 | import json | ||
301 | 28 | import platform | ||
302 | 29 | import os | ||
303 | 30 | |||
304 | 31 | import apt_pkg | ||
305 | 32 | from gi.repository import Click | ||
306 | 33 | import pycurl | ||
307 | 34 | |||
308 | 35 | from six.moves.urllib.parse import quote as url_quote | ||
309 | 36 | |||
310 | 37 | from click.json_helpers import json_array_to_python | ||
311 | 38 | import click.install | ||
312 | 39 | |||
313 | 40 | class ClickRepositoryError(Exception): | ||
314 | 41 | pass | ||
315 | 42 | |||
316 | 43 | |||
317 | 44 | class HttpResponse: | ||
318 | 45 | """Http Response with status and data members""" | ||
319 | 46 | |||
320 | 47 | def __init__(self, status, data): | ||
321 | 48 | self.status = status | ||
322 | 49 | self.data = data | ||
323 | 50 | |||
324 | 51 | def __str__(self): | ||
325 | 52 | return self.data | ||
326 | 53 | |||
327 | 54 | |||
328 | 55 | # similar to httplib2.Http().request() | ||
329 | 56 | def http_request(url, data=None, headers=None): | ||
330 | 57 | """Send a http request to the given url and return (response, data) | ||
331 | 58 | |||
332 | 59 | param url The url to send the request to | ||
333 | 60 | param data (Optional) data to POST to the url | ||
334 | 61 | param headers (Optional) headers to include with the request | ||
335 | 62 | return A tuple of (response, body) | ||
336 | 63 | rtype (HttpResponse, bytes) | ||
337 | 64 | """ | ||
338 | 65 | def write_header(line): | ||
339 | 66 | header.write(line.decode("utf-8")) | ||
340 | 67 | return len(line) | ||
341 | 68 | def write_body(data): | ||
342 | 69 | body.write(data) | ||
343 | 70 | return len(data) | ||
344 | 71 | def post_data(num): | ||
345 | 72 | return data_stream.read(num) | ||
346 | 73 | header = StringIO() | ||
347 | 74 | body = BytesIO() | ||
348 | 75 | curl = pycurl.Curl() | ||
349 | 76 | curl.setopt(curl.URL, url) | ||
350 | 77 | if os.environ.get("CLICK_DEBUG_HTTP", ""): | ||
351 | 78 | curl.setopt(curl.VERBOSE, 1) | ||
352 | 79 | curl.setopt(pycurl.WRITEFUNCTION, write_body) | ||
353 | 80 | curl.setopt(pycurl.HEADERFUNCTION, write_header) | ||
354 | 81 | if data is not None: | ||
355 | 82 | data_stream = BytesIO(data) | ||
356 | 83 | curl.setopt(pycurl.READFUNCTION, post_data) | ||
357 | 84 | curl.setopt(pycurl.POSTFIELDSIZE, len(data)) | ||
358 | 85 | curl.setopt(pycurl.POST, 1) | ||
359 | 86 | if headers is not None: | ||
360 | 87 | header_list = [] | ||
361 | 88 | for k, v in headers.items(): | ||
362 | 89 | header_list.append("%s: %s" % (k, v)) | ||
363 | 90 | curl.setopt(pycurl.HTTPHEADER, header_list) | ||
364 | 91 | # ssl: no need to set extra options, pycurl has sensible defaults | ||
365 | 92 | curl.perform() | ||
366 | 93 | resp = HttpResponse(curl.getinfo(pycurl.HTTP_CODE), header.getvalue()) | ||
367 | 94 | return resp, body.getvalue() | ||
368 | 95 | |||
369 | 96 | |||
370 | 97 | def get_repository(): | ||
371 | 98 | """Factory to get a click repository for the current distro""" | ||
372 | 99 | distro = platform.dist()[0] | ||
373 | 100 | if distro.startswith('Ubuntu'): | ||
374 | 101 | return ClickRepositoryAppsUbuntuCom() | ||
375 | 102 | raise ClickRepositoryError("No repository for distribution '%s'" % distro) | ||
376 | 103 | |||
377 | 104 | |||
378 | 105 | class ClickRepository: | ||
379 | 106 | |||
380 | 107 | def search(self, search_string): | ||
381 | 108 | """Search the repository for the given search string | ||
382 | 109 | |||
383 | 110 | Returns a list of click manifest from the repository | ||
384 | 111 | """ | ||
385 | 112 | raise NotImplementedError() | ||
386 | 113 | |||
387 | 114 | def get_upgradable(self): | ||
388 | 115 | """Get all click packages where a newer version is available | ||
389 | 116 | |||
390 | 117 | Returns a list of (old_manifest, new_manifest) pairs | ||
391 | 118 | """ | ||
392 | 119 | raise NotImplementedError() | ||
393 | 120 | |||
394 | 121 | def details(self, name): | ||
395 | 122 | """Get the details for the given click name | ||
396 | 123 | |||
397 | 124 | Returns a click manifest | ||
398 | 125 | """ | ||
399 | 126 | raise NotImplementedError() | ||
400 | 127 | |||
401 | 128 | @property | ||
402 | 129 | def description(self): | ||
403 | 130 | """The description of the repository as a URL""" | ||
404 | 131 | raise NotImplementedError() | ||
405 | 132 | |||
406 | 133 | @property | ||
407 | 134 | def credentials(self): | ||
408 | 135 | """The credentials for the repository (or None)""" | ||
409 | 136 | return None | ||
410 | 137 | |||
411 | 138 | |||
412 | 139 | def get_store_headers(): | ||
413 | 140 | frameworks = ["%s-%s" % (f.get_base_name(), f.get_base_version()) | ||
414 | 141 | for f in Click.Framework.get_frameworks()] | ||
415 | 142 | headers = { | ||
416 | 143 | "Accept": "application/hal+json", | ||
417 | 144 | "X-Ubuntu-Frameworks": ",".join(frameworks), | ||
418 | 145 | "X-Ubuntu-Architecture": click.install._dpkg_architecture(), | ||
419 | 146 | } | ||
420 | 147 | return headers | ||
421 | 148 | |||
422 | 149 | |||
423 | 150 | class ClickRepositoryAppsUbuntuCom(ClickRepository): | ||
424 | 151 | """Interface to click repository from apps.ubuntu.com""" | ||
425 | 152 | |||
426 | 153 | SEARCH_URI = "https://search.apps.ubuntu.com/api/v1/search?q=%s" | ||
427 | 154 | DETAILS_URI = "https://search.apps.ubuntu.com/api/v1/package/%s" | ||
428 | 155 | BULK_APPS_URI = "https://search.apps.ubuntu.com/api/v1/click-metadata" | ||
429 | 156 | |||
430 | 157 | @property | ||
431 | 158 | def credentials(self): | ||
432 | 159 | return None | ||
433 | 160 | |||
434 | 161 | @property | ||
435 | 162 | def description(self): | ||
436 | 163 | return "https://myapps.developer.ubuntu.com/dev/click-apps/" | ||
437 | 164 | |||
438 | 165 | def search(self, search_term): | ||
439 | 166 | result = [] | ||
440 | 167 | url = self.SEARCH_URI % url_quote(search_term) | ||
441 | 168 | if not self.credentials: | ||
442 | 169 | url += ",allow_unauthenticated:True" | ||
443 | 170 | resp, raw_content = http_request(url, headers=get_store_headers()) | ||
444 | 171 | content = raw_content.decode("utf-8") | ||
445 | 172 | if resp.status != 200: | ||
446 | 173 | raise ClickRepositoryError(content) | ||
447 | 174 | data = json.loads(content) | ||
448 | 175 | if not "_embedded" in data: | ||
449 | 176 | return result | ||
450 | 177 | for item in data["_embedded"]["clickindex:package"]: | ||
451 | 178 | result.append(item) | ||
452 | 179 | return result | ||
453 | 180 | |||
454 | 181 | def details(self, name): | ||
455 | 182 | resp, raw_content = http_request( | ||
456 | 183 | self.DETAILS_URI % url_quote(name), headers=get_store_headers()) | ||
457 | 184 | content = raw_content.decode("utf-8") | ||
458 | 185 | if resp.status != 200: | ||
459 | 186 | raise ClickRepositoryError(content) | ||
460 | 187 | return json.loads(content) | ||
461 | 188 | |||
462 | 189 | def get_upgradable(self): | ||
463 | 190 | result = [] | ||
464 | 191 | db = Click.DB() | ||
465 | 192 | db.read(db_dir=None) | ||
466 | 193 | registry = Click.User.for_user(db, name=None) | ||
467 | 194 | # build the dict of installed apps for the POST | ||
468 | 195 | current_apps = {} | ||
469 | 196 | for app in json_array_to_python(registry.get_manifests()): | ||
470 | 197 | current_apps[app["name"]] = app | ||
471 | 198 | # if we have nothing installed, do not query the server | ||
472 | 199 | if not current_apps: | ||
473 | 200 | return result | ||
474 | 201 | json_post_data = json.dumps({ | ||
475 | 202 | "name": list(current_apps.keys()), | ||
476 | 203 | }).encode("utf-8") | ||
477 | 204 | # query the API | ||
478 | 205 | resp, raw_content = http_request( | ||
479 | 206 | self.BULK_APPS_URI, | ||
480 | 207 | json_post_data, | ||
481 | 208 | headers={"content-type": "application/json"}) | ||
482 | 209 | content = raw_content.decode("utf-8") | ||
483 | 210 | if resp.status != 200: | ||
484 | 211 | raise ClickRepositoryError( | ||
485 | 212 | "Failed to get upgradable packages: %s" % resp.content) | ||
486 | 213 | # return a list of (old, new) pairs to the caller | ||
487 | 214 | server_apps = json.loads(content) | ||
488 | 215 | result = [] | ||
489 | 216 | for new in server_apps: | ||
490 | 217 | current = current_apps[new["name"]] | ||
491 | 218 | if apt_pkg.version_compare(current["version"], new["version"]) < 0: | ||
492 | 219 | result.append( (current, new) ) | ||
493 | 220 | return result | ||
494 | 0 | 221 | ||
495 | === modified file 'click/tests/test_install.py' | |||
496 | --- click/tests/test_install.py 2014-12-03 12:42:21 +0000 | |||
497 | +++ click/tests/test_install.py 2015-02-24 16:14:00 +0000 | |||
498 | @@ -605,7 +605,7 @@ | |||
499 | 605 | os.path.exists(ClickInstaller(None)._preload_path()), | 605 | os.path.exists(ClickInstaller(None)._preload_path()), |
500 | 606 | "preload bits not built; installing packages will fail") | 606 | "preload bits not built; installing packages will fail") |
501 | 607 | @mock.patch("gi.repository.Click.package_install_hooks") | 607 | @mock.patch("gi.repository.Click.package_install_hooks") |
503 | 608 | @mock.patch("click.install.ClickInstaller._dpkg_architecture") | 608 | @mock.patch("click.install._dpkg_architecture") |
504 | 609 | def test_single_architecture(self, mock_dpkg_architecture, | 609 | def test_single_architecture(self, mock_dpkg_architecture, |
505 | 610 | mock_package_install_hooks): | 610 | mock_package_install_hooks): |
506 | 611 | with self.run_in_subprocess( | 611 | with self.run_in_subprocess( |
507 | @@ -642,7 +642,7 @@ | |||
508 | 642 | os.path.exists(ClickInstaller(None)._preload_path()), | 642 | os.path.exists(ClickInstaller(None)._preload_path()), |
509 | 643 | "preload bits not built; installing packages will fail") | 643 | "preload bits not built; installing packages will fail") |
510 | 644 | @mock.patch("gi.repository.Click.package_install_hooks") | 644 | @mock.patch("gi.repository.Click.package_install_hooks") |
512 | 645 | @mock.patch("click.install.ClickInstaller._dpkg_architecture") | 645 | @mock.patch("click.install._dpkg_architecture") |
513 | 646 | def test_multiple_architectures(self, mock_dpkg_architecture, | 646 | def test_multiple_architectures(self, mock_dpkg_architecture, |
514 | 647 | mock_package_install_hooks): | 647 | mock_package_install_hooks): |
515 | 648 | with self.run_in_subprocess( | 648 | with self.run_in_subprocess( |
516 | 649 | 649 | ||
517 | === modified file 'debian/control' | |||
518 | --- debian/control 2014-10-10 07:10:23 +0000 | |||
519 | +++ debian/control 2015-02-24 16:14:00 +0000 | |||
520 | @@ -3,7 +3,7 @@ | |||
521 | 3 | Priority: optional | 3 | Priority: optional |
522 | 4 | Maintainer: Colin Watson <cjwatson@ubuntu.com> | 4 | Maintainer: Colin Watson <cjwatson@ubuntu.com> |
523 | 5 | Standards-Version: 3.9.5 | 5 | Standards-Version: 3.9.5 |
525 | 6 | Build-Depends: debhelper (>= 9~), dh-autoreconf, intltool, python3:any (>= 3.2), python3-all:any, python3-setuptools, python3-apt, python3-debian, python3-gi, python3:any (>= 3.3) | python3-mock, pep8, python3-pep8, pyflakes, python3-sphinx, pkg-config, valac, gobject-introspection (>= 0.6.7), libgirepository1.0-dev (>= 0.6.7), libglib2.0-dev (>= 2.34), gir1.2-glib-2.0, libjson-glib-dev (>= 0.10), libgee-0.8-dev, libpackagekit-glib2-dev (>= 0.7.2), python3-coverage, python3-six, dh-systemd (>= 1.3) | 6 | Build-Depends: debhelper (>= 9~), dh-autoreconf, intltool, python3:any (>= 3.2), python3-all:any, python3-setuptools, python3-apt, python3-debian, python3-gi, python3:any (>= 3.3) | python3-mock, pep8, python3-pep8, pyflakes, python3-sphinx, pkg-config, valac, gobject-introspection (>= 0.6.7), libgirepository1.0-dev (>= 0.6.7), libglib2.0-dev (>= 2.34), gir1.2-glib-2.0, libjson-glib-dev (>= 0.10), libgee-0.8-dev, libpackagekit-glib2-dev (>= 0.7.2), python3-coverage, python3-six, dh-systemd (>= 1.3), python3-pycurl, python3-prettytable |
526 | 7 | Vcs-Bzr: https://code.launchpad.net/~ubuntu-managed-branches/click/click | 7 | Vcs-Bzr: https://code.launchpad.net/~ubuntu-managed-branches/click/click |
527 | 8 | Vcs-Browser: http://bazaar.launchpad.net/~ubuntu-managed-branches/click/click/files | 8 | Vcs-Browser: http://bazaar.launchpad.net/~ubuntu-managed-branches/click/click/files |
528 | 9 | X-Auto-Uploader: no-rewrite-version | 9 | X-Auto-Uploader: no-rewrite-version |
529 | @@ -14,7 +14,7 @@ | |||
530 | 14 | Package: click | 14 | Package: click |
531 | 15 | Architecture: any | 15 | Architecture: any |
532 | 16 | Pre-Depends: ${misc:Pre-Depends} | 16 | Pre-Depends: ${misc:Pre-Depends} |
534 | 17 | Depends: ${shlibs:Depends}, ${misc:Depends}, ${python3:Depends}, python3-click (= ${binary:Version}), adduser | 17 | Depends: ${shlibs:Depends}, ${misc:Depends}, ${python3:Depends}, python3-click (= ${binary:Version}), adduser, python3-prettytable |
535 | 18 | Recommends: click-apparmor | 18 | Recommends: click-apparmor |
536 | 19 | Suggests: click-reviewers-tools (>= 0.9), ubuntu-app-launch-tools | upstart-app-launch-tools | 19 | Suggests: click-reviewers-tools (>= 0.9), ubuntu-app-launch-tools | upstart-app-launch-tools |
537 | 20 | Conflicts: click-package | 20 | Conflicts: click-package |
538 | @@ -40,7 +40,7 @@ | |||
539 | 40 | Package: python3-click | 40 | Package: python3-click |
540 | 41 | Section: python | 41 | Section: python |
541 | 42 | Architecture: any | 42 | Architecture: any |
543 | 43 | Depends: ${misc:Depends}, ${python3:Depends}, gir1.2-click-0.4 (= ${binary:Version}), gir1.2-glib-2.0, python3-apt, python3-debian, python3-gi | 43 | Depends: ${misc:Depends}, ${python3:Depends}, gir1.2-click-0.4 (= ${binary:Version}), gir1.2-glib-2.0, python3-apt, python3-debian, python3-gi, python3-pycurl |
544 | 44 | Conflicts: python3-click-package | 44 | Conflicts: python3-click-package |
545 | 45 | Replaces: python3-click-package | 45 | Replaces: python3-click-package |
546 | 46 | Provides: python3-click-package | 46 | Provides: python3-click-package |
FAILED: Continuous integration, rev:527 /code.launchpad .net/~mvo/ click/repositor y/+merge/ 237604/ +edit-commit- message
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http:// jenkins. qa.ubuntu. com/job/ click-devel- ci/93/ jenkins. qa.ubuntu. com/job/ click-devel- utopic- amd64-ci/ 95/console jenkins. qa.ubuntu. com/job/ click-devel- utopic- armhf-ci/ 93/console jenkins. qa.ubuntu. com/job/ click-devel- utopic- i386-ci/ 93/console
Executed test runs:
FAILURE: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild: s-jenkins. ubuntu- ci:8080/ job/click- devel-ci/ 93/rebuild
http://