Merge lp:~flacoste/launchpad/bug-801233 into lp:launchpad
- bug-801233
- Merge into devel
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Francis J. Lacoste | ||||
Approved revision: | no longer in the source branch. | ||||
Merged at revision: | 13501 | ||||
Proposed branch: | lp:~flacoste/launchpad/bug-801233 | ||||
Merge into: | lp:launchpad | ||||
Diff against target: |
1343 lines (+617/-83) 25 files modified
lib/canonical/launchpad/configure.zcml (+1/-0) lib/lp/app/browser/launchpad.py (+5/-1) lib/lp/buildmaster/configure.zcml (+1/-0) lib/lp/buildmaster/interfaces/builder.py (+36/-19) lib/lp/buildmaster/model/builder.py (+13/-10) lib/lp/buildmaster/security.py (+19/-0) lib/lp/buildmaster/tests/test_webservice.py (+68/-0) lib/lp/services/webservice/configure.zcml (+15/-0) lib/lp/services/webservice/json.py (+24/-0) lib/lp/services/webservice/tests/test_json.py (+30/-0) lib/lp/services/webservice/wadl-to-refhtml.xsl (+126/-6) lib/lp/soyuz/browser/configure.zcml (+7/-5) lib/lp/soyuz/browser/processor.py (+8/-14) lib/lp/soyuz/browser/tests/test_processor.py (+41/-0) lib/lp/soyuz/configure.zcml (+7/-0) lib/lp/soyuz/interfaces/processor.py (+34/-4) lib/lp/soyuz/interfaces/webservice.py (+2/-6) lib/lp/soyuz/model/processor.py (+21/-0) lib/lp/soyuz/security.py (+26/-0) lib/lp/soyuz/tests/test_packagecloner.py (+1/-4) lib/lp/soyuz/tests/test_processor.py (+90/-7) lib/lp/testing/__init__.py (+11/-0) lib/lp/testing/factory.py (+6/-3) lib/lp/testing/publication.py (+11/-3) lib/lp/testing/tests/test_publication.py (+14/-1) |
||||
To merge this branch: | bzr merge lp:~flacoste/launchpad/bug-801233 | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Brad Crittenden (community) | code | Approve | |
Review via email: mp+67353@code.launchpad.net |
Commit message
[r=bac][bug=801233] Export IBuilderSet.
Description of the change
= Summary =
This is a massive Yak shaving branch! The goal is to write an API script which
will allow copy rebuilds to be manually allocated a number of builders. To do
this, we need to find dynamically the number of available builders for a
particular processor. That information isn't currently available over the
API.
== Proposed fix ==
It exports IBuilder.processor, IBuilderSet.
IBuilderSet.
shaving along the way. An important one worth mentioning here: I changed the
URL of IProcessor. They were living under their family and their DB id was
part of th URL. Since IProcessor name are globally unique, I added a
/+processors collection and their URL point their using their name. That will
make it easier to get processor by crafting the URL.
== Pre-implementation notes ==
No pre-implementation done.
== Implementation details ==
Here are the numerous yak shaved along the way:
* Removed getBuildersByArch() which wasn't used anywhere.
* getBuildQueueSize() returns a dict containing datetime.timedelta object.
simplejson didn't know how to serialize those, so I hooked up a very
simple one which simply str() it. That was added into
lp.services.
* Collections check that launchpad.View is available on the entry. So I had
to define adapters for IBuilder, IProcessor and IProcessorFamily.
* Since there is no web page for IProcessor nor IProcessorFamily, I changed
pulish_web_link to False.
Everything else should be straightforward, let me know if you have any
questions.
== Tests ==
./bin/test -vvt
'lp.soyuz.
== Demo and Q/A ==
This will be QA-ed by checking the exported attribute and methods on
qastaging.
= Launchpad lint =
Checking for conflicts and issues in changed files.
Linting changed files:
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/canonical
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
lib/lp/
./lib/lp/
174: Line exceeds 78 characters.
261: Line exceeds 78 characters.
219: W291 trailing whitespace
284: E301 expected 1 blank line, found 0
302: E301 expected 1 blank line, found 0
373: E301 expected 1 blank line, found 0
517: E301 expected 1 blank line, found 0
519: E301 expected 1 blank line, found 0
608: E301 expected 1 blank line, found 0
683: E301 expected 1 blank line, found 0
686: E301 expected 1 blank line, found 0
./lib/lp/
48: Line exceeds 78 characters.
55: Line exceeds 78 characters.
57: Line exceeds 78 characters.
123: Line exceeds 78 characters.
88: E202 whitespace before ')'
123: E501 line too long (123 characters)
123: E202 whitespace before ')'
./lib/lp/
87: E301 expected 1 blank line, found 0
./lib/lp/
18: E302 expected 2 blank lines, found 1
22: E302 expected 2 blank lines, found 1
./lib/lp/
36: E302 expected 2 blank lines, found 1
./lib/lp/
27: E303 too many blank lines (2)
30: W291 trailing whitespace
33: W391 blank line at end of file
./lib/lp/
49: E301 expected 1 blank line, found 0
58: W291 trailing whitespace
79: E301 expected 1 blank line, found 0
101: E301 expected 1 blank line, found 0
./lib/lp/
139: Line exceeds 78 characters.
210: Line exceeds 78 characters.
249: W293 blank line contains whitespace
256: W293 blank line contains whitespace
./lib/lp/
13: E302 expected 2 blank lines, found 1
41: W391 blank line at end of file
I'll fix the white-space lint post-review.
Francis J. Lacoste (flacoste) wrote : | # |
Brad Crittenden (bac) wrote : | # |
Hi Francis,
This is a really good follow up branch to the changes I made. Thanks for making the processor traversal by name.
I only found a few things to note:
* As we discussed on IRC the docstring for getBuildQueueSizes make the wadl generator unhappy.
* In lib/lp/
* In lib/lp/
* Some of the lint is real, including new code with super long lines such as in test_default_
* in ProcessorSet getAll and getByName why don't you use the storeOf(self)?
Francis J. Lacoste (flacoste) wrote : | # |
On 11-07-11 03:43 PM, Brad Crittenden wrote:
> Review: Approve code
> Hi Francis,
>
> This is a really good follow up branch to the changes I made. Thanks for making the processor traversal by name.
>
> I only found a few things to note:
>
> * As we discussed on IRC the docstring for getBuildQueueSizes make the wadl generator unhappy.
Yeah, this was a simple matter of using the rst literal block directive
to make it happy. (::)
This introduces a few other drive-bys:
* Turned out that our WADL to HTML conversion excludes the RST
parameters table (because we don't want the :param name: documentation
to be redundant or misleading with the web-service specific
documentation.) So I added support for the :return: (and :raise:)
parameters for custom methods (when appropriate). This will make the
return value of a couple other JSON-returning method documentated.
* Got bored and tired of seeing so many Unkonwn URL entry messages, so I
fixed a bunch. Not all of them are fixed. Will eventually chase the
others, but it's still an improvement!
>
> * In lib/lp/
Done.
>
> * In lib/lp/
>
Done.
> * Some of the lint is real, including new code with super long lines such as in test_default_
Branch is now lint-free!
>
> * in ProcessorSet getAll and getByName why don't you use the storeOf(self)?
Because self isn't a DB instance (and thus doesn't have a store).
(That's usually the case in *Set object.)
Thanks for the review!
--
Francis J. Lacoste
<email address hidden>
Preview Diff
1 | === modified file 'lib/canonical/launchpad/configure.zcml' |
2 | --- lib/canonical/launchpad/configure.zcml 2011-02-23 21:15:23 +0000 |
3 | +++ lib/canonical/launchpad/configure.zcml 2011-07-21 19:34:45 +0000 |
4 | @@ -27,6 +27,7 @@ |
5 | <include package="lp.coop.answersbugs" /> |
6 | <include package="lp.code" /> |
7 | <include package="lp.soyuz" /> |
8 | + <include package="lp.services.webservice" /> |
9 | <include package="lp.translations" /> |
10 | <include package="lp.testopenid" /> |
11 | <include package="lp.blueprints" /> |
12 | |
13 | === modified file 'lib/lp/app/browser/launchpad.py' |
14 | --- lib/lp/app/browser/launchpad.py 2011-06-30 11:28:59 +0000 |
15 | +++ lib/lp/app/browser/launchpad.py 2011-07-21 19:34:45 +0000 |
16 | @@ -146,7 +146,10 @@ |
17 | from lp.services.worlddata.interfaces.language import ILanguageSet |
18 | from lp.soyuz.interfaces.binarypackagename import IBinaryPackageNameSet |
19 | from lp.soyuz.interfaces.packageset import IPackagesetSet |
20 | -from lp.soyuz.interfaces.processor import IProcessorFamilySet |
21 | +from lp.soyuz.interfaces.processor import ( |
22 | + IProcessorFamilySet, |
23 | + IProcessorSet, |
24 | + ) |
25 | from lp.testopenid.interfaces.server import ITestOpenIDApplication |
26 | from lp.translations.interfaces.translationgroup import ITranslationGroupSet |
27 | from lp.translations.interfaces.translationimportqueue import ( |
28 | @@ -617,6 +620,7 @@ |
29 | 'people': IPersonSet, |
30 | 'pillars': IPillarNameSet, |
31 | '+processor-families': IProcessorFamilySet, |
32 | + '+processors': IProcessorSet, |
33 | 'projects': IProductSet, |
34 | 'projectgroups': IProjectGroupSet, |
35 | 'sourcepackagenames': ISourcePackageNameSet, |
36 | |
37 | === modified file 'lib/lp/buildmaster/configure.zcml' |
38 | --- lib/lp/buildmaster/configure.zcml 2010-11-19 13:25:25 +0000 |
39 | +++ lib/lp/buildmaster/configure.zcml 2011-07-21 19:34:45 +0000 |
40 | @@ -10,6 +10,7 @@ |
41 | xmlns:xmlrpc="http://namespaces.zope.org/xmlrpc" |
42 | i18n_domain="launchpad"> |
43 | <include package=".browser"/> |
44 | + <authorizations module=".security" /> |
45 | |
46 | <!-- Builder --> |
47 | <class |
48 | |
49 | === modified file 'lib/lp/buildmaster/interfaces/builder.py' |
50 | --- lib/lp/buildmaster/interfaces/builder.py 2011-02-24 15:30:54 +0000 |
51 | +++ lib/lp/buildmaster/interfaces/builder.py 2011-07-21 19:34:45 +0000 |
52 | @@ -27,6 +27,12 @@ |
53 | exported, |
54 | operation_parameters, |
55 | operation_returns_entry, |
56 | + operation_returns_collection_of, |
57 | + operation_for_version, |
58 | + ) |
59 | +from lazr.restful.fields import ( |
60 | + Reference, |
61 | + ReferenceChoice, |
62 | ) |
63 | from zope.interface import ( |
64 | Attribute, |
65 | @@ -34,7 +40,6 @@ |
66 | ) |
67 | from zope.schema import ( |
68 | Bool, |
69 | - Choice, |
70 | Field, |
71 | Int, |
72 | Text, |
73 | @@ -45,6 +50,7 @@ |
74 | from lp.app.validators.name import name_validator |
75 | from lp.app.validators.url import builder_url_validator |
76 | from lp.registry.interfaces.role import IHasOwner |
77 | +from lp.soyuz.interfaces.processor import IProcessor |
78 | from lp.services.fields import ( |
79 | Description, |
80 | PersonChoice, |
81 | @@ -106,10 +112,12 @@ |
82 | |
83 | id = Attribute("Builder identifier") |
84 | |
85 | - processor = Choice( |
86 | + processor = exported(ReferenceChoice( |
87 | title=_('Processor'), required=True, vocabulary='Processor', |
88 | + schema=IProcessor, |
89 | description=_('Build Slave Processor, used to identify ' |
90 | - 'which jobs can be built by this device.')) |
91 | + 'which jobs can be built by this device.')), |
92 | + as_of='devel', readonly=True) |
93 | |
94 | owner = exported(PersonChoice( |
95 | title=_('Owner'), required=True, vocabulary='ValidOwner', |
96 | @@ -128,7 +136,8 @@ |
97 | |
98 | title = exported(Title( |
99 | title=_('Title'), required=True, |
100 | - description=_('The builder slave title. Should be just a few words.'))) |
101 | + description=_( |
102 | + 'The builder slave title. Should be just a few words.'))) |
103 | |
104 | description = exported(Description( |
105 | title=_('Description'), required=False, |
106 | @@ -199,8 +208,8 @@ |
107 | :param file_sha1: The file's sha1, which is how the file is addressed |
108 | in the slave XMLRPC protocol. Specially, the file_sha1 'buildlog' |
109 | will cause the build log to be retrieved and gzipped. |
110 | - :param filename: The name of the file to be given to the librarian file |
111 | - alias. |
112 | + :param filename: The name of the file to be given to the librarian |
113 | + file alias. |
114 | :param private: True if the build is for a private archive. |
115 | :return: A Deferred that calls back with a librarian file alias. |
116 | """ |
117 | @@ -238,14 +247,14 @@ |
118 | |
119 | def updateStatus(logger=None): |
120 | """Update the builder's status by probing it. |
121 | - |
122 | + |
123 | :return: A Deferred that fires when the dialog with the slave is |
124 | finished. It does not have a return value. |
125 | """ |
126 | |
127 | def cleanSlave(): |
128 | """Clean any temporary files from the slave. |
129 | - |
130 | + |
131 | :return: A Deferred that fires when the dialog with the slave is |
132 | finished. It does not have a return value. |
133 | """ |
134 | @@ -385,26 +394,34 @@ |
135 | def getBuilders(): |
136 | """Return all active configured builders.""" |
137 | |
138 | - def getBuildersByArch(arch): |
139 | - """Return all configured builders for a given DistroArchSeries.""" |
140 | - |
141 | + @export_read_operation() |
142 | + @operation_for_version('devel') |
143 | def getBuildQueueSizes(): |
144 | """Return the number of pending builds for each processor. |
145 | |
146 | :return: a dict of tuples with the queue size and duration for |
147 | - each processor and virtualisation. For example: |
148 | - { |
149 | - 'virt': { |
150 | - '386': (1, datetime.timedelta(0, 60)), |
151 | - 'amd64': (2, datetime.timedelta(0, 30)), |
152 | - }, |
153 | - 'nonvirt':... |
154 | - } |
155 | + each processor and virtualisation. For example:: |
156 | + |
157 | + { |
158 | + 'virt': { |
159 | + '386': (1, datetime.timedelta(0, 60)), |
160 | + 'amd64': (2, datetime.timedelta(0, 30)), |
161 | + }, |
162 | + 'nonvirt':... |
163 | + } |
164 | |
165 | The tuple contains the size of the queue, as an integer, |
166 | and the sum of the jobs 'estimated_duration' in queue, |
167 | as a timedelta or None for empty queues. |
168 | """ |
169 | |
170 | + @operation_parameters( |
171 | + processor=Reference( |
172 | + title=_("Processor"), required=True, schema=IProcessor), |
173 | + virtualized=Bool( |
174 | + title=_("Virtualized"), required=False, default=True)) |
175 | + @operation_returns_collection_of(IBuilder) |
176 | + @export_read_operation() |
177 | + @operation_for_version('devel') |
178 | def getBuildersForQueue(processor, virtualized): |
179 | """Return all builders for given processor/virtualization setting.""" |
180 | |
181 | === modified file 'lib/lp/buildmaster/model/builder.py' |
182 | --- lib/lp/buildmaster/model/builder.py 2011-02-01 18:14:57 +0000 |
183 | +++ lib/lp/buildmaster/model/builder.py 2011-07-21 19:34:45 +0000 |
184 | @@ -171,7 +171,8 @@ |
185 | |
186 | :param builder_url: The URL of the slave buildd machine, |
187 | e.g. http://localhost:8221 |
188 | - :param vm_host: If the slave is virtual, specify its host machine here. |
189 | + :param vm_host: If the slave is virtual, specify its host machine |
190 | + here. |
191 | :param reactor: Used by tests to override the Twisted reactor. |
192 | :param proxy: Used By tests to override the xmlrpc.Proxy. |
193 | """ |
194 | @@ -216,7 +217,7 @@ |
195 | def getFile(self, sha_sum, file_to_write): |
196 | """Fetch a file from the builder. |
197 | |
198 | - :param sha_sum: The sha of the file (which is also its name on the |
199 | + :param sha_sum: The sha of the file (which is also its name on the |
200 | builder) |
201 | :param file_to_write: A file name or file-like object to write |
202 | the file to |
203 | @@ -258,7 +259,8 @@ |
204 | resume_command = config.builddmaster.vm_resume_command % { |
205 | 'vm_host': self._vm_host} |
206 | # Twisted API requires string but the configuration provides unicode. |
207 | - resume_argv = [term.encode('utf-8') for term in resume_command.split()] |
208 | + resume_argv = [ |
209 | + term.encode('utf-8') for term in resume_command.split()] |
210 | d = defer.Deferred() |
211 | p = ProcessWithTimeout( |
212 | d, config.builddmaster.socket_timeout, clock=clock) |
213 | @@ -281,6 +283,7 @@ |
214 | def sendFileToSlave(self, sha1, url, username="", password=""): |
215 | """Helper to send the file at 'url' with 'sha1' to this builder.""" |
216 | d = self.ensurepresent(sha1, url, username, password) |
217 | + |
218 | def check_present((present, info)): |
219 | if not present: |
220 | raise CannotFetchFile(url, info) |
221 | @@ -299,6 +302,7 @@ |
222 | """ |
223 | d = self._with_timeout(self._server.callRemote( |
224 | 'build', buildid, builder_type, chroot_sha1, filemap, args)) |
225 | + |
226 | def got_fault(failure): |
227 | failure.trap(xmlrpclib.Fault) |
228 | raise BuildSlaveFailure(failure.value) |
229 | @@ -370,6 +374,7 @@ |
230 | d = builder.cleanSlave() |
231 | else: |
232 | d = builder.requestAbort() |
233 | + |
234 | def log_rescue(ignored): |
235 | if logger: |
236 | logger.info( |
237 | @@ -514,8 +519,10 @@ |
238 | logger.info("Resuming %s (%s)" % (self.name, self.url)) |
239 | |
240 | d = self.slave.resume() |
241 | + |
242 | def got_resume_ok((stdout, stderr, returncode)): |
243 | return stdout, stderr |
244 | + |
245 | def got_resume_bad(failure): |
246 | stdout, stderr, code = failure.value |
247 | raise CannotResumeHost( |
248 | @@ -605,6 +612,7 @@ |
249 | def slaveStatus(self): |
250 | """See IBuilder.""" |
251 | d = self.slave.status() |
252 | + |
253 | def got_status(status_sentence): |
254 | status = {'builder_status': status_sentence[0]} |
255 | |
256 | @@ -680,9 +688,11 @@ |
257 | if not self.builderok: |
258 | return defer.succeed(False) |
259 | d = self.slaveStatusSentence() |
260 | + |
261 | def catch_fault(failure): |
262 | failure.trap(xmlrpclib.Fault, socket.error) |
263 | return False |
264 | + |
265 | def check_available(status): |
266 | return status[0] == BuilderStatus.IDLE |
267 | return d.addCallbacks(check_available, catch_fault) |
268 | @@ -896,13 +906,6 @@ |
269 | return Builder.selectBy( |
270 | active=True, orderBy=['virtualized', 'processor', 'name']) |
271 | |
272 | - def getBuildersByArch(self, arch): |
273 | - """See IBuilderSet.""" |
274 | - return Builder.select('builder.processor = processor.id ' |
275 | - 'AND processor.family = %d' |
276 | - % arch.processorfamily.id, |
277 | - clauseTables=("Processor",)) |
278 | - |
279 | def getBuildQueueSizes(self): |
280 | """See `IBuilderSet`.""" |
281 | store = getUtility(IStoreSelector).get(MAIN_STORE, SLAVE_FLAVOR) |
282 | |
283 | === added file 'lib/lp/buildmaster/security.py' |
284 | --- lib/lp/buildmaster/security.py 1970-01-01 00:00:00 +0000 |
285 | +++ lib/lp/buildmaster/security.py 2011-07-21 19:34:45 +0000 |
286 | @@ -0,0 +1,19 @@ |
287 | +# Copyright 2011 Canonical Ltd. This software is licensed under the |
288 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
289 | + |
290 | +"""Security adapters for the buildmaster package.""" |
291 | + |
292 | +__metaclass__ = type |
293 | +__all__ = [ |
294 | + 'ViewBuilder', |
295 | + ] |
296 | + |
297 | +from lp.app.security import AnonymousAuthorization |
298 | +from lp.buildmaster.interfaces.builder import ( |
299 | + IBuilder, |
300 | + ) |
301 | + |
302 | + |
303 | +class ViewBuilder(AnonymousAuthorization): |
304 | + """Anyone can view a `IBuilder`.""" |
305 | + usedfor = IBuilder |
306 | |
307 | === added file 'lib/lp/buildmaster/tests/test_webservice.py' |
308 | --- lib/lp/buildmaster/tests/test_webservice.py 1970-01-01 00:00:00 +0000 |
309 | +++ lib/lp/buildmaster/tests/test_webservice.py 2011-07-21 19:34:45 +0000 |
310 | @@ -0,0 +1,68 @@ |
311 | +# Copyright 2011 Canonical Ltd. This software is licensed under the |
312 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
313 | + |
314 | +"""Tests for the builders webservice .""" |
315 | + |
316 | +__metaclass__ = type |
317 | + |
318 | +from canonical.testing.layers import DatabaseFunctionalLayer |
319 | +from canonical.launchpad.testing.pages import LaunchpadWebServiceCaller |
320 | +from lp.testing import ( |
321 | + api_url, |
322 | + logout, |
323 | + TestCaseWithFactory, |
324 | + ) |
325 | + |
326 | + |
327 | +class TestBuildersCollection(TestCaseWithFactory): |
328 | + layer = DatabaseFunctionalLayer |
329 | + |
330 | + def setUp(self): |
331 | + super(TestBuildersCollection, self).setUp() |
332 | + self.webservice = LaunchpadWebServiceCaller() |
333 | + |
334 | + def test_getBuildQueueSizes(self): |
335 | + logout() |
336 | + results = self.webservice.named_get( |
337 | + '/builders', 'getBuildQueueSizes', api_version='devel') |
338 | + self.assertEquals( |
339 | + ['nonvirt', 'virt'], sorted(results.jsonBody().keys())) |
340 | + |
341 | + def test_getBuildersForQueue(self): |
342 | + g1 = self.factory.makeProcessorFamily('g1').processors[0] |
343 | + quantum = self.factory.makeProcessorFamily('quantum').processors[0] |
344 | + self.factory.makeBuilder( |
345 | + processor=quantum, name='quantum_builder1') |
346 | + self.factory.makeBuilder( |
347 | + processor=quantum, name='quantum_builder2') |
348 | + self.factory.makeBuilder( |
349 | + processor=quantum, name='quantum_builder3', virtualized=False) |
350 | + self.factory.makeBuilder( |
351 | + processor=g1, name='g1_builder', virtualized=False) |
352 | + |
353 | + logout() |
354 | + results = self.webservice.named_get( |
355 | + '/builders', 'getBuildersForQueue', |
356 | + processor=api_url(quantum), virtualized=True, |
357 | + api_version='devel').jsonBody() |
358 | + self.assertEquals( |
359 | + ['quantum_builder1', 'quantum_builder2'], |
360 | + sorted(builder['name'] for builder in results['entries'])) |
361 | + |
362 | + |
363 | +class TestBuilderEntry(TestCaseWithFactory): |
364 | + layer = DatabaseFunctionalLayer |
365 | + |
366 | + def setUp(self): |
367 | + super(TestBuilderEntry, self).setUp() |
368 | + self.webservice = LaunchpadWebServiceCaller() |
369 | + |
370 | + def test_exports_processor(self): |
371 | + processor_family = self.factory.makeProcessorFamily('s1') |
372 | + builder = self.factory.makeBuilder( |
373 | + processor=processor_family.processors[0]) |
374 | + |
375 | + logout() |
376 | + entry = self.webservice.get( |
377 | + api_url(builder), api_version='devel').jsonBody() |
378 | + self.assertEndsWith(entry['processor_link'], '/+processors/s1') |
379 | |
380 | === added file 'lib/lp/services/webservice/configure.zcml' |
381 | --- lib/lp/services/webservice/configure.zcml 1970-01-01 00:00:00 +0000 |
382 | +++ lib/lp/services/webservice/configure.zcml 2011-07-21 19:34:45 +0000 |
383 | @@ -0,0 +1,15 @@ |
384 | +<!-- Copyright 2011 Canonical Ltd. This software is licensed under the |
385 | + GNU Affero General Public License version 3 (see the file LICENSE). |
386 | +--> |
387 | + |
388 | +<configure |
389 | + xmlns="http://namespaces.zope.org/zope" |
390 | + xmlns:i18n="http://namespaces.zope.org/i18n" |
391 | + i18n_domain="launchpad"> |
392 | + |
393 | + <adapter |
394 | + provides="lazr.restful.interfaces.IJSONPublishable" |
395 | + for="zope.interface.common.idatetime.ITimeDelta" |
396 | + factory="lp.services.webservice.json.StrJSONSerializer" |
397 | + permission="zope.Public"/> |
398 | +</configure> |
399 | |
400 | === added file 'lib/lp/services/webservice/json.py' |
401 | --- lib/lp/services/webservice/json.py 1970-01-01 00:00:00 +0000 |
402 | +++ lib/lp/services/webservice/json.py 2011-07-21 19:34:45 +0000 |
403 | @@ -0,0 +1,24 @@ |
404 | +# Copyright 2011 Canonical Ltd. This software is licensed under the |
405 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
406 | + |
407 | +"""Additional JSON serializer for the web service.""" |
408 | + |
409 | +__metaclass__ = type |
410 | +__all__ = [ |
411 | + 'StrJSONSerializer', |
412 | + ] |
413 | + |
414 | + |
415 | +from lazr.restful.interfaces import IJSONPublishable |
416 | +from zope.interface import implements |
417 | + |
418 | + |
419 | +class StrJSONSerializer: |
420 | + """Simple JSON serializer that simply str() it's context. """ |
421 | + implements(IJSONPublishable) |
422 | + |
423 | + def __init__(self, context): |
424 | + self.context = context |
425 | + |
426 | + def toDataForJSON(self, media_type): |
427 | + return str(self.context) |
428 | |
429 | === added directory 'lib/lp/services/webservice/tests' |
430 | === added file 'lib/lp/services/webservice/tests/__init__.py' |
431 | === added file 'lib/lp/services/webservice/tests/test_json.py' |
432 | --- lib/lp/services/webservice/tests/test_json.py 1970-01-01 00:00:00 +0000 |
433 | +++ lib/lp/services/webservice/tests/test_json.py 2011-07-21 19:34:45 +0000 |
434 | @@ -0,0 +1,30 @@ |
435 | +# Copyright 2011 Canonical Ltd. This software is licensed under the |
436 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
437 | + |
438 | +"""Tests for the JSON serializer.""" |
439 | + |
440 | +__metaclass__ = type |
441 | + |
442 | +from datetime import timedelta |
443 | + |
444 | +from canonical.testing.layers import FunctionalLayer |
445 | +from lazr.restful.interfaces import IJSONPublishable |
446 | +from lp.services.webservice.json import StrJSONSerializer |
447 | +from lp.testing import TestCase |
448 | + |
449 | + |
450 | +class TestStrJSONSerializer(TestCase): |
451 | + layer = FunctionalLayer |
452 | + |
453 | + def test_toDataForJSON(self): |
454 | + serializer = StrJSONSerializer( |
455 | + timedelta(days=2, hours=2, seconds=5)) |
456 | + self.assertEquals( |
457 | + '2 days, 2:00:05', |
458 | + serializer.toDataForJSON('application/json')) |
459 | + |
460 | + def test_timedelta_users_StrJSONSerializer(self): |
461 | + delta = timedelta(seconds=5) |
462 | + serializer = IJSONPublishable(delta) |
463 | + self.assertEquals('0:00:05', |
464 | + serializer.toDataForJSON('application/json')) |
465 | |
466 | === modified file 'lib/lp/services/webservice/wadl-to-refhtml.xsl' |
467 | --- lib/lp/services/webservice/wadl-to-refhtml.xsl 2011-01-13 19:45:32 +0000 |
468 | +++ lib/lp/services/webservice/wadl-to-refhtml.xsl 2011-07-21 19:34:45 +0000 |
469 | @@ -165,9 +165,16 @@ |
470 | <xsl:call-template name="resource-uri-doc"> |
471 | <xsl:with-param name="url"> |
472 | <xsl:choose> |
473 | - <xsl:when test="@id = 'has_milestones' |
474 | - or @id = 'bug_target' |
475 | - or @id = 'has_bugs'"> |
476 | + <xsl:when test=" |
477 | + @id = 'bug_link_target' |
478 | + or @id = 'bug_target' |
479 | + or @id = 'has_bugs' |
480 | + or @id = 'has_milestones' |
481 | + or @id = 'object_with_translation_imports' |
482 | + or @id = 'question_target' |
483 | + or @id = 'specification_target' |
484 | + or @id = 'structural_subscription_target' |
485 | + "> |
486 | <em>depends on the underlying entry</em> |
487 | </xsl:when> |
488 | <xsl:otherwise> |
489 | @@ -264,10 +271,18 @@ |
490 | <xsl:text>/+build/</xsl:text> |
491 | <var><id></var> |
492 | </xsl:when> |
493 | + <xsl:when test="@id = 'builder'"> |
494 | + <xsl:text>/builders/</xsl:text> |
495 | + <var><builder.name></var> |
496 | + </xsl:when> |
497 | <xsl:when test="@id = 'cve'"> |
498 | <xsl:text>/bugs/cve/</xsl:text> |
499 | <var><sequence></var> |
500 | </xsl:when> |
501 | + <xsl:when test="@id = 'country'"> |
502 | + <xsl:text>/+countries/</xsl:text> |
503 | + <var><iso3166code2></var> |
504 | + </xsl:when> |
505 | <xsl:when test="@id = 'distribution_source_package'"> |
506 | <xsl:text>/</xsl:text> |
507 | <var><distribution.name></var> |
508 | @@ -294,6 +309,15 @@ |
509 | <xsl:text>/+email/</xsl:text> |
510 | <var><email></var> |
511 | </xsl:when> |
512 | + <xsl:when test="@id = 'gpg_key'"> |
513 | + <xsl:text>/</xsl:text> |
514 | + <var><person.name></var> |
515 | + <xsl:text>/+gpg-keys/</xsl:text> |
516 | + <var><keyid></var> |
517 | + </xsl:when> |
518 | + <xsl:when test="@id = 'hwdb'"> |
519 | + <xsl:text>/+hwdb</xsl:text> |
520 | + </xsl:when> |
521 | <xsl:when test="@id = 'h_w_device'"> |
522 | <xsl:text>/+hwdb/+device/</xsl:text> |
523 | <var><id></var> |
524 | @@ -360,6 +384,17 @@ |
525 | <xsl:text>/~</xsl:text> |
526 | <var><name></var> |
527 | </xsl:when> |
528 | + <xsl:when test="@id = 'pillars'"> |
529 | + <xsl:text>/pillars</xsl:text> |
530 | + </xsl:when> |
531 | + <xsl:when test="@id = 'processor'"> |
532 | + <xsl:text>/+processors/</xsl:text> |
533 | + <var><processor.name></var> |
534 | + </xsl:when> |
535 | + <xsl:when test="@id = 'processor_family'"> |
536 | + <xsl:text>/+processor-families/</xsl:text> |
537 | + <var><processor_family.name></var> |
538 | + </xsl:when> |
539 | <xsl:when test="@id = 'product_release'"> |
540 | <xsl:text>/</xsl:text> |
541 | <var><product.name></var> |
542 | @@ -398,6 +433,20 @@ |
543 | <xsl:text>/</xsl:text> |
544 | <var><name></var> |
545 | </xsl:when> |
546 | + <xsl:when test="@id = 'question'"> |
547 | + <xsl:text>/</xsl:text> |
548 | + <var><target.name></var> |
549 | + <xsl:text>/+question/</xsl:text> |
550 | + <var ><question.id></var> |
551 | + </xsl:when> |
552 | + <xsl:when test="@id = 'question_message'"> |
553 | + <xsl:text>/</xsl:text> |
554 | + <var><target.name></var> |
555 | + <xsl:text>/+question/</xsl:text> |
556 | + <var ><question.id></var> |
557 | + <xsl:text>/messages/</xsl:text> |
558 | + <var ><message.index></var> |
559 | + </xsl:when> |
560 | <xsl:when test="@id = 'source_package'"> |
561 | <xsl:text>/</xsl:text> |
562 | <var><distribution.name></var> |
563 | @@ -414,6 +463,34 @@ |
564 | <xsl:text>/+sourcepub/</xsl:text> |
565 | <var><id></var> |
566 | </xsl:when> |
567 | + <xsl:when test="@id = 'specification'"> |
568 | + <xsl:text>/</xsl:text> |
569 | + <var><target.name></var> |
570 | + <xsl:text>/+spec/</xsl:text> |
571 | + <var ><specification.name></var> |
572 | + </xsl:when> |
573 | + <xsl:when test="@id = 'specification_branch'"> |
574 | + <xsl:text>/</xsl:text> |
575 | + <var><target.name></var> |
576 | + <xsl:text>/+spec/</xsl:text> |
577 | + <var ><specification.name></var> |
578 | + <xsl:text>/+branch/</xsl:text> |
579 | + <var ><branch.unique_name[1:]></var> |
580 | + </xsl:when> |
581 | + <xsl:when test="@id = 'specification_subscription'"> |
582 | + <xsl:text>/</xsl:text> |
583 | + <var><target.name></var> |
584 | + <xsl:text>/+spec/</xsl:text> |
585 | + <var ><specification.name></var> |
586 | + <xsl:text>/+subscription/</xsl:text> |
587 | + <var ><person.name></var> |
588 | + </xsl:when> |
589 | + <xsl:when test="@id = 'ssh_key'"> |
590 | + <xsl:text>/</xsl:text> |
591 | + <var><person.name></var> |
592 | + <xsl:text>/+ssh-keys/</xsl:text> |
593 | + <var><keyid></var> |
594 | + </xsl:when> |
595 | <xsl:when test="@id = 'team_membership'"> |
596 | <xsl:text>/~</xsl:text> |
597 | <var><team.name></var> |
598 | @@ -864,6 +941,12 @@ |
599 | <xsl:apply-templates select="wadl:doc"/> |
600 | <xsl:apply-templates select="wadl:request"/> |
601 | <xsl:apply-templates select="wadl:response"/> |
602 | + <xsl:if test="not(wadl:response)"> |
603 | + <xsl:apply-templates select="wadl:doc//html:th[ |
604 | + node() = 'return:' |
605 | + ]"/> |
606 | + </xsl:if> |
607 | + <xsl:call-template name="error-documentation"/> |
608 | </xsl:when> |
609 | <xsl:otherwise> |
610 | <p><em>Missing documentation.</em></p> |
611 | @@ -910,6 +993,39 @@ |
612 | </p> |
613 | </xsl:template> |
614 | |
615 | + <!-- Documentation of the custom method return type. --> |
616 | + <xsl:template match="wadl:doc//html:th[node() = 'return:']"> |
617 | + <h6>Response (application/json)</h6> |
618 | + <xsl:choose> |
619 | + <xsl:when test="following-sibling::html:td/text()"> |
620 | + <p><xsl:apply-templates select="following-sibling::html:td" |
621 | + mode="copy"/></p> |
622 | + </xsl:when> |
623 | + <xsl:otherwise> |
624 | + <xsl:apply-templates select="following-sibling::html:td" |
625 | + mode="copy"/> |
626 | + </xsl:otherwise> |
627 | + </xsl:choose> |
628 | + </xsl:template> |
629 | + |
630 | + <!-- Documentation of the error raised by the operation. --> |
631 | + <xsl:template name="error-documentation"> |
632 | + <xsl:if test="wadl:doc//html:th[node() = 'raise:']"> |
633 | + <h6>Errors</h6> |
634 | + <ul> |
635 | + <xsl:apply-templates |
636 | + select="wadl:doc//html:th[node() = 'raise:']"/> |
637 | + </ul> |
638 | + </xsl:if> |
639 | + </xsl:template> |
640 | + |
641 | + <xsl:template match="wadl:doc//html:th[node() = 'raise:']"> |
642 | + <li> |
643 | + <xsl:apply-templates select="following-sibling::html:td" |
644 | + mode="copy"/> |
645 | + </li> |
646 | + </xsl:template> |
647 | + |
648 | <!-- Documentation for request parameter. --> |
649 | <xsl:template match="wadl:param"> |
650 | <tr> |
651 | @@ -1032,11 +1148,15 @@ |
652 | mode="representation-type"/> |
653 | </xsl:template> |
654 | |
655 | - <!-- Omit docutils parameter lists in methods since they are redundant |
656 | - or misleading with the one we give. --> |
657 | + <!-- Omit docutils parameter table in methods. The parameter names |
658 | + description is either redundant or misleading with the one we |
659 | + give. |
660 | + |
661 | + We process the return and raise parameters separately. |
662 | + --> |
663 | <xsl:template match="wadl:method//html:table[ |
664 | contains(@class, 'field-list')]" |
665 | - mode="copy"/> |
666 | + mode="copy" /> |
667 | |
668 | <!-- Output the mediaType attribute of a representation --> |
669 | <xsl:template match="wadl:representation[@mediaType]" |
670 | |
671 | === modified file 'lib/lp/soyuz/browser/configure.zcml' |
672 | --- lib/lp/soyuz/browser/configure.zcml 2011-06-23 13:42:38 +0000 |
673 | +++ lib/lp/soyuz/browser/configure.zcml 2011-07-21 19:34:45 +0000 |
674 | @@ -32,18 +32,20 @@ |
675 | path_expression="string:+binarypub" |
676 | attribute_to_parent="archive" |
677 | urldata="lp.soyuz.browser.publishing.BinaryPublicationURL"/> |
678 | - <browser:url |
679 | - for="lp.soyuz.interfaces.processor.IProcessorFamilySet" |
680 | + <browser:url for="lp.soyuz.interfaces.processor.IProcessorFamilySet" |
681 | path_expression="string:+processor-families" |
682 | parent_utility="canonical.launchpad.webapp.interfaces.ILaunchpadRoot"/> |
683 | <browser:url |
684 | for="lp.soyuz.interfaces.processor.IProcessorFamily" |
685 | path_expression="string:${name}" |
686 | parent_utility="lp.soyuz.interfaces.processor.IProcessorFamilySet" /> |
687 | + <browser:url for="lp.soyuz.interfaces.processor.IProcessorSet" |
688 | + path_expression="string:+processors" |
689 | + parent_utility="canonical.launchpad.webapp.interfaces.ILaunchpadRoot"/> |
690 | <browser:url |
691 | for="lp.soyuz.interfaces.processor.IProcessor" |
692 | - path_expression="string:${id}" |
693 | - attribute_to_parent="family" /> |
694 | + path_expression="string:${name}" |
695 | + parent_utility="lp.soyuz.interfaces.processor.IProcessorSet" /> |
696 | </facet> |
697 | <browser:navigation |
698 | module="lp.soyuz.browser.binarypackagerelease" |
699 | @@ -233,7 +235,7 @@ |
700 | <browser:navigation |
701 | module="lp.soyuz.browser.processor" |
702 | classes=" |
703 | - ProcessorFamilySetNavigation ProcessorFamilyNavigation"/> |
704 | + ProcessorFamilySetNavigation ProcessorSetNavigation"/> |
705 | <browser:url |
706 | for="lp.soyuz.interfaces.archive.IPPA" |
707 | path_expression="string:+archive" |
708 | |
709 | === modified file 'lib/lp/soyuz/browser/processor.py' |
710 | --- lib/lp/soyuz/browser/processor.py 2011-06-23 16:11:42 +0000 |
711 | +++ lib/lp/soyuz/browser/processor.py 2011-07-21 19:34:45 +0000 |
712 | @@ -8,15 +8,15 @@ |
713 | |
714 | __all__ = [ |
715 | 'ProcessorFamilySetNavigation', |
716 | - 'ProcessorFamilyNavigation', |
717 | + 'ProcessorSetNavigation', |
718 | ] |
719 | |
720 | |
721 | from canonical.launchpad.webapp import Navigation |
722 | from lp.app.errors import NotFoundError |
723 | from lp.soyuz.interfaces.processor import ( |
724 | - IProcessorFamily, |
725 | IProcessorFamilySet, |
726 | + IProcessorSet, |
727 | ) |
728 | |
729 | |
730 | @@ -32,15 +32,9 @@ |
731 | return family |
732 | |
733 | |
734 | -class ProcessorFamilyNavigation(Navigation): |
735 | - """IProcessorFamily navigation.""" |
736 | - |
737 | - usedfor = IProcessorFamily |
738 | - |
739 | - def traverse(self, id_): |
740 | - id_ = int(id_) |
741 | - processors = self.processors |
742 | - for p in processors: |
743 | - if p.id == id_: |
744 | - return p |
745 | - raise NotFoundError(id_) |
746 | +class ProcessorSetNavigation(Navigation): |
747 | + """IProcessorFamilySet navigation.""" |
748 | + usedfor = IProcessorSet |
749 | + |
750 | + def traverse(self, name): |
751 | + return self.context.getByName(name) |
752 | |
753 | === added file 'lib/lp/soyuz/browser/tests/test_processor.py' |
754 | --- lib/lp/soyuz/browser/tests/test_processor.py 1970-01-01 00:00:00 +0000 |
755 | +++ lib/lp/soyuz/browser/tests/test_processor.py 2011-07-21 19:34:45 +0000 |
756 | @@ -0,0 +1,41 @@ |
757 | +# Copyright 2011 Canonical Ltd. This software is licensed under the |
758 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
759 | + |
760 | +"""Tests for process navigation.""" |
761 | + |
762 | +__metaclass__ = type |
763 | + |
764 | +from canonical.testing.layers import DatabaseFunctionalLayer |
765 | +from canonical.launchpad.webapp.publisher import canonical_url |
766 | +from lp.testing import TestCaseWithFactory |
767 | +from lp.testing.publication import test_traverse |
768 | + |
769 | + |
770 | +class TestProcessorNavigation(TestCaseWithFactory): |
771 | + layer = DatabaseFunctionalLayer |
772 | + |
773 | + def test_processor_family_url(self): |
774 | + family = self.factory.makeProcessorFamily('quantum') |
775 | + self.assertEquals( |
776 | + '/+processor-families/quantum', |
777 | + canonical_url(family, force_local_path=True)) |
778 | + |
779 | + def test_processor_url(self): |
780 | + family = self.factory.makeProcessorFamily('quantum') |
781 | + quantum = family.processors[0] |
782 | + self.assertEquals( |
783 | + '/+processors/quantum', |
784 | + canonical_url(quantum, force_local_path=True)) |
785 | + |
786 | + def test_processor_family_navigation(self): |
787 | + family = self.factory.makeProcessorFamily('quantum') |
788 | + obj, view, request = test_traverse( |
789 | + 'http://api.launchpad.dev/devel/+processor-families/quantum') |
790 | + self.assertEquals(family, obj) |
791 | + |
792 | + def test_processor_navigation(self): |
793 | + family = self.factory.makeProcessorFamily('quantum') |
794 | + obj, view, request = test_traverse( |
795 | + 'http://api.launchpad.dev/' |
796 | + 'devel/+processors/quantum') |
797 | + self.assertEquals(family.processors[0], obj) |
798 | |
799 | === modified file 'lib/lp/soyuz/configure.zcml' |
800 | --- lib/lp/soyuz/configure.zcml 2011-07-12 14:23:40 +0000 |
801 | +++ lib/lp/soyuz/configure.zcml 2011-07-21 19:34:45 +0000 |
802 | @@ -11,6 +11,7 @@ |
803 | i18n_domain="launchpad"> |
804 | <include |
805 | package=".browser"/> |
806 | + <authorizations module=".security" /> |
807 | |
808 | <!-- PackageCloner --> |
809 | |
810 | @@ -375,6 +376,12 @@ |
811 | <allow |
812 | interface="lp.soyuz.interfaces.processor.IProcessorFamilySet"/> |
813 | </securedutility> |
814 | + <securedutility |
815 | + class="lp.soyuz.model.processor.ProcessorSet" |
816 | + provides="lp.soyuz.interfaces.processor.IProcessorSet"> |
817 | + <allow |
818 | + interface="lp.soyuz.interfaces.processor.IProcessorSet"/> |
819 | + </securedutility> |
820 | <adapter |
821 | for="lp.soyuz.interfaces.distroarchseriesbinarypackagerelease.IDistroArchSeriesBinaryPackageRelease" |
822 | provides="canonical.launchpad.webapp.interfaces.IBreadcrumb" |
823 | |
824 | === modified file 'lib/lp/soyuz/interfaces/processor.py' |
825 | --- lib/lp/soyuz/interfaces/processor.py 2011-06-23 16:11:42 +0000 |
826 | +++ lib/lp/soyuz/interfaces/processor.py 2011-07-21 19:34:45 +0000 |
827 | @@ -11,6 +11,8 @@ |
828 | 'IProcessor', |
829 | 'IProcessorFamily', |
830 | 'IProcessorFamilySet', |
831 | + 'IProcessorSet', |
832 | + 'ProcessorNotFound', |
833 | ] |
834 | |
835 | from zope.interface import ( |
836 | @@ -38,6 +40,12 @@ |
837 | CollectionField, |
838 | Reference, |
839 | ) |
840 | +from lp.app.errors import NameLookupFailed |
841 | + |
842 | + |
843 | +class ProcessorNotFound(NameLookupFailed): |
844 | + """Exception raised when a processor name isn't found.""" |
845 | + _message_prefix = 'No such processor' |
846 | |
847 | |
848 | class IProcessor(Interface): |
849 | @@ -49,7 +57,7 @@ |
850 | # the WADL generation work it must be back-dated to the earliest version. |
851 | # Note that individual attributes and methods can and must truthfully set |
852 | # 'devel' as their version. |
853 | - export_as_webservice_entry(publish_web_link=True, as_of='beta') |
854 | + export_as_webservice_entry(publish_web_link=False, as_of='beta') |
855 | id = Attribute("The Processor ID") |
856 | family = exported( |
857 | Reference( |
858 | @@ -84,7 +92,7 @@ |
859 | # 'devel' as their version. |
860 | export_as_webservice_entry( |
861 | plural_name='processor_families', |
862 | - publish_web_link=True, |
863 | + publish_web_link=False, |
864 | as_of='beta') |
865 | |
866 | id = Attribute("The ProcessorFamily ID") |
867 | @@ -123,14 +131,36 @@ |
868 | """ |
869 | |
870 | |
871 | +class IProcessorSet(Interface): |
872 | + """Operations related to Processor instances.""" |
873 | + export_as_webservice_collection(IProcessor) |
874 | + |
875 | + @operation_parameters( |
876 | + name=TextLine(required=True)) |
877 | + @operation_returns_entry(IProcessor) |
878 | + @export_read_operation() |
879 | + @operation_for_version('devel') |
880 | + def getByName(name): |
881 | + """Return the IProcessor instance with the matching name. |
882 | + |
883 | + :param name: The name to look for. |
884 | + :raise ProcessorNotFound: if there is no processor with that name. |
885 | + :return: A `IProcessor` instance if found |
886 | + """ |
887 | + |
888 | + @collection_default_content() |
889 | + def getAll(): |
890 | + """Return all the `IProcessor` known to Launchpad.""" |
891 | + |
892 | + |
893 | class IProcessorFamilySet(Interface): |
894 | """Operations related to ProcessorFamily instances.""" |
895 | |
896 | - export_as_webservice_collection(Interface) |
897 | + export_as_webservice_collection(IProcessorFamily) |
898 | |
899 | @operation_parameters( |
900 | name=TextLine(required=True)) |
901 | - @operation_returns_entry(Interface) |
902 | + @operation_returns_entry(IProcessorFamily) |
903 | @export_read_operation() |
904 | @operation_for_version('devel') |
905 | def getByName(name): |
906 | |
907 | === modified file 'lib/lp/soyuz/interfaces/webservice.py' |
908 | --- lib/lp/soyuz/interfaces/webservice.py 2011-06-23 18:26:26 +0000 |
909 | +++ lib/lp/soyuz/interfaces/webservice.py 2011-07-21 19:34:45 +0000 |
910 | @@ -35,6 +35,7 @@ |
911 | 'IProcessor', |
912 | 'IProcessorFamily', |
913 | 'IProcessorFamilySet', |
914 | + 'IProcessorSet', |
915 | 'ISourcePackagePublishingHistory', |
916 | 'IncompatibleArguments', |
917 | 'InsufficientUploadRights', |
918 | @@ -96,6 +97,7 @@ |
919 | IProcessor, |
920 | IProcessorFamily, |
921 | IProcessorFamilySet, |
922 | + IProcessorSet, |
923 | ) |
924 | from lp.soyuz.interfaces.publishing import ( |
925 | IBinaryPackagePublishingHistory, |
926 | @@ -105,7 +107,6 @@ |
927 | |
928 | from canonical.launchpad.components.apihelpers import ( |
929 | patch_collection_property, |
930 | - patch_entry_return_type, |
931 | patch_plain_parameter_type, |
932 | patch_reference_property, |
933 | ) |
934 | @@ -115,15 +116,10 @@ |
935 | from canonical.launchpad.interfaces import _schema_circular_imports |
936 | _schema_circular_imports |
937 | |
938 | -from lazr.restful.declarations import LAZR_WEBSERVICE_EXPORTED |
939 | -IProcessorFamilySet.queryTaggedValue( |
940 | - LAZR_WEBSERVICE_EXPORTED)['collection_entry_schema'] = IProcessorFamily |
941 | - |
942 | # IProcessor |
943 | patch_reference_property( |
944 | IProcessor, 'family', IProcessorFamily) |
945 | |
946 | -patch_entry_return_type(IProcessorFamilySet, 'getByName', IProcessorFamily) |
947 | patch_collection_property( |
948 | IArchive, 'enabled_restricted_families', IProcessorFamily) |
949 | patch_plain_parameter_type( |
950 | |
951 | === modified file 'lib/lp/soyuz/model/processor.py' |
952 | --- lib/lp/soyuz/model/processor.py 2010-11-12 02:15:28 +0000 |
953 | +++ lib/lp/soyuz/model/processor.py 2011-07-21 19:34:45 +0000 |
954 | @@ -25,6 +25,8 @@ |
955 | IProcessor, |
956 | IProcessorFamily, |
957 | IProcessorFamilySet, |
958 | + IProcessorSet, |
959 | + ProcessorNotFound, |
960 | ) |
961 | |
962 | |
963 | @@ -42,6 +44,24 @@ |
964 | return "<Processor %r>" % self.title |
965 | |
966 | |
967 | +class ProcessorSet: |
968 | + """See `IProcessorSet`.""" |
969 | + implements(IProcessorSet) |
970 | + |
971 | + def getByName(self, name): |
972 | + """See `IProcessorSet`.""" |
973 | + store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR) |
974 | + processor = store.find(Processor, Processor.name == name).one() |
975 | + if processor is None: |
976 | + raise ProcessorNotFound(name) |
977 | + return processor |
978 | + |
979 | + def getAll(self): |
980 | + """See `IProcessorSet`.""" |
981 | + store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR) |
982 | + return store.find(Processor) |
983 | + |
984 | + |
985 | class ProcessorFamily(SQLBase): |
986 | implements(IProcessorFamily) |
987 | _table = 'ProcessorFamily' |
988 | @@ -64,6 +84,7 @@ |
989 | |
990 | class ProcessorFamilySet: |
991 | implements(IProcessorFamilySet) |
992 | + |
993 | def getByName(self, name): |
994 | """Please see `IProcessorFamilySet`.""" |
995 | # Please note that ProcessorFamily.name is unique i.e. the database |
996 | |
997 | === added file 'lib/lp/soyuz/security.py' |
998 | --- lib/lp/soyuz/security.py 1970-01-01 00:00:00 +0000 |
999 | +++ lib/lp/soyuz/security.py 2011-07-21 19:34:45 +0000 |
1000 | @@ -0,0 +1,26 @@ |
1001 | +# Copyright 2011 Canonical Ltd. This software is licensed under the |
1002 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
1003 | + |
1004 | +"""Security adapters for the soyuz module.""" |
1005 | + |
1006 | +__metaclass__ = type |
1007 | +__all__ = [ |
1008 | + 'ViewProcessor', |
1009 | + 'ViewProcessorFamily', |
1010 | + ] |
1011 | + |
1012 | +from lp.app.security import AnonymousAuthorization |
1013 | +from lp.soyuz.interfaces.processor import ( |
1014 | + IProcessor, |
1015 | + IProcessorFamily, |
1016 | + ) |
1017 | + |
1018 | + |
1019 | +class ViewProcessor(AnonymousAuthorization): |
1020 | + """Anyone can view an `IProcessor`.""" |
1021 | + usedfor = IProcessor |
1022 | + |
1023 | + |
1024 | +class ViewProcessorFamily(AnonymousAuthorization): |
1025 | + """Anyone can view an `IProcessorFamily`.""" |
1026 | + usedfor = IProcessorFamily |
1027 | |
1028 | === modified file 'lib/lp/soyuz/tests/test_packagecloner.py' |
1029 | --- lib/lp/soyuz/tests/test_packagecloner.py 2011-06-23 09:31:34 +0000 |
1030 | +++ lib/lp/soyuz/tests/test_packagecloner.py 2011-07-21 19:34:45 +0000 |
1031 | @@ -344,7 +344,6 @@ |
1032 | # This is a processor family without a DAS in the source, so |
1033 | # we expect no builds. |
1034 | family = self.factory.makeProcessorFamily(name="armel") |
1035 | - self.factory.makeProcessor(family=family, name="armel") |
1036 | proc_families = [family] |
1037 | copy_archive, distroseries = self.makeCopyArchive( |
1038 | [package_info], proc_families=proc_families) |
1039 | @@ -357,7 +356,6 @@ |
1040 | # One of these processor families has a DAS in the source, so |
1041 | # we expect one set of builds |
1042 | family = self.factory.makeProcessorFamily(name="armel") |
1043 | - self.factory.makeProcessor(family=family, name="armel") |
1044 | proc_families = [family, ProcessorFamilySet().getByName("x86")] |
1045 | copy_archive, distroseries = self.makeCopyArchive( |
1046 | [package_info], proc_families=proc_families) |
1047 | @@ -399,7 +397,6 @@ |
1048 | copy_archive, distroseries, proc_families=proc_families) |
1049 | self.checkBuilds(copy_archive, [package_info, package_info]) |
1050 | |
1051 | - |
1052 | def diffArchives(self, target_archive, target_distroseries, |
1053 | source_archive=None, source_distroseries=None): |
1054 | """Run a packageSetDiff of two archives.""" |
1055 | @@ -423,6 +420,7 @@ |
1056 | expected_changed_tuples = [(e.name, e.version) |
1057 | for e in expected_changed] |
1058 | expected_new_tuples = [(e.name, e.version) for e in expected_new] |
1059 | + |
1060 | def get_tuples(source_keys): |
1061 | tuples = [] |
1062 | for source_key in source_keys: |
1063 | @@ -503,7 +501,6 @@ |
1064 | [package_infos[0]], [package_infos[1]], diff, |
1065 | distroseries.distribution.main_archive) |
1066 | |
1067 | - |
1068 | def mergeCopy(self, target_archive, target_distroseries, |
1069 | source_archive=None, source_distroseries=None): |
1070 | if source_distroseries is None: |
1071 | |
1072 | === modified file 'lib/lp/soyuz/tests/test_processor.py' |
1073 | --- lib/lp/soyuz/tests/test_processor.py 2010-10-04 19:50:45 +0000 |
1074 | +++ lib/lp/soyuz/tests/test_processor.py 2011-07-21 19:34:45 +0000 |
1075 | @@ -5,13 +5,29 @@ |
1076 | |
1077 | from zope.component import getUtility |
1078 | |
1079 | -from canonical.testing.layers import LaunchpadZopelessLayer |
1080 | +from canonical.launchpad.webapp.interfaces import ( |
1081 | + DEFAULT_FLAVOR, |
1082 | + IStoreSelector, |
1083 | + MAIN_STORE, |
1084 | + ) |
1085 | +from canonical.launchpad.testing.pages import LaunchpadWebServiceCaller |
1086 | +from canonical.testing.layers import ( |
1087 | + DatabaseFunctionalLayer, |
1088 | + LaunchpadZopelessLayer, |
1089 | + ) |
1090 | + |
1091 | from lp.soyuz.interfaces.processor import ( |
1092 | IProcessor, |
1093 | IProcessorFamily, |
1094 | IProcessorFamilySet, |
1095 | - ) |
1096 | -from lp.testing import TestCaseWithFactory |
1097 | + IProcessorSet, |
1098 | + ProcessorNotFound, |
1099 | + ) |
1100 | +from lp.testing import ( |
1101 | + ExpectedException, |
1102 | + logout, |
1103 | + TestCaseWithFactory, |
1104 | + ) |
1105 | |
1106 | |
1107 | class ProcessorFamilyTests(TestCaseWithFactory): |
1108 | @@ -29,16 +45,83 @@ |
1109 | """Test adding a new Processor to a ProcessorFamily.""" |
1110 | family = getUtility(IProcessorFamilySet).new("avr", "Atmel AVR", |
1111 | "The Modified Harvard architecture 8-bit RISC processors.") |
1112 | - proc = family.addProcessor("avr2001", "The 2001 AVR", "Fast as light.") |
1113 | + proc = family.addProcessor( |
1114 | + "avr2001", "The 2001 AVR", "Fast as light.") |
1115 | self.assertProvides(proc, IProcessor) |
1116 | self.assertEquals(family, proc.family) |
1117 | |
1118 | def test_get_restricted(self): |
1119 | """Test retrieving all restricted processors.""" |
1120 | family_set = getUtility(IProcessorFamilySet) |
1121 | - normal_family = getUtility(IProcessorFamilySet).new("avr", "Atmel AVR", |
1122 | + normal_family = getUtility(IProcessorFamilySet).new( |
1123 | + "avr", "Atmel AVR", |
1124 | "The Modified Harvard architecture 8-bit RISC processors.") |
1125 | - restricted_family = getUtility(IProcessorFamilySet).new("5051", "5051", |
1126 | - "Another small processor family", restricted=True) |
1127 | + restricted_family = getUtility(IProcessorFamilySet).new( |
1128 | + "5051", "5051", "Another small processor family", |
1129 | + restricted=True) |
1130 | self.assertFalse(normal_family in family_set.getRestricted()) |
1131 | self.assertTrue(restricted_family in family_set.getRestricted()) |
1132 | + |
1133 | + |
1134 | +class ProcessorSetTests(TestCaseWithFactory): |
1135 | + layer = DatabaseFunctionalLayer |
1136 | + |
1137 | + def test_getByName(self): |
1138 | + processor_set = getUtility(IProcessorSet) |
1139 | + q1 = self.factory.makeProcessorFamily(name='q1') |
1140 | + self.assertEquals(q1.processors[0], processor_set.getByName('q1')) |
1141 | + |
1142 | + def test_getByName_not_found(self): |
1143 | + processor_set = getUtility(IProcessorSet) |
1144 | + with ExpectedException(ProcessorNotFound, 'No such processor.*'): |
1145 | + processor_set.getByName('q1') |
1146 | + |
1147 | + def test_getAll(self): |
1148 | + processor_set = getUtility(IProcessorSet) |
1149 | + # Make it easy to filter out sample data |
1150 | + store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR) |
1151 | + store.execute("UPDATE Processor SET name = 'sample_data_' || name") |
1152 | + self.factory.makeProcessorFamily(name='q1') |
1153 | + self.factory.makeProcessorFamily(name='i686') |
1154 | + self.factory.makeProcessorFamily(name='g4') |
1155 | + self.assertEquals( |
1156 | + ['g4', 'i686', 'q1'], |
1157 | + sorted( |
1158 | + processor.name for processor in processor_set.getAll() |
1159 | + if not processor.name.startswith('sample_data_'))) |
1160 | + |
1161 | + |
1162 | +class ProcessorSetWebServiceTests(TestCaseWithFactory): |
1163 | + layer = DatabaseFunctionalLayer |
1164 | + |
1165 | + def setUp(self): |
1166 | + super(ProcessorSetWebServiceTests, self).setUp() |
1167 | + self.webservice = LaunchpadWebServiceCaller() |
1168 | + |
1169 | + def test_getByName(self): |
1170 | + self.factory.makeProcessorFamily(name='transmeta') |
1171 | + logout() |
1172 | + |
1173 | + processor = self.webservice.named_get( |
1174 | + '/+processors', 'getByName', name='transmeta', |
1175 | + api_version='devel', |
1176 | + ).jsonBody() |
1177 | + self.assertEquals('transmeta', processor['name']) |
1178 | + |
1179 | + def test_default_collection(self): |
1180 | + # Make it easy to filter out sample data |
1181 | + store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR) |
1182 | + store.execute("UPDATE Processor SET name = 'sample_data_' || name") |
1183 | + self.factory.makeProcessorFamily(name='q1') |
1184 | + self.factory.makeProcessorFamily(name='i686') |
1185 | + self.factory.makeProcessorFamily(name='g4') |
1186 | + |
1187 | + logout() |
1188 | + |
1189 | + collection = self.webservice.get( |
1190 | + '/+processors?ws.size=10', api_version='devel').jsonBody() |
1191 | + self.assertEquals( |
1192 | + ['g4', 'i686', 'q1'], |
1193 | + sorted( |
1194 | + processor['name'] for processor in collection['entries'] |
1195 | + if not processor['name'].startswith('sample_data_'))) |
1196 | |
1197 | === modified file 'lib/lp/testing/__init__.py' |
1198 | --- lib/lp/testing/__init__.py 2011-07-18 14:07:48 +0000 |
1199 | +++ lib/lp/testing/__init__.py 2011-07-21 19:34:45 +0000 |
1200 | @@ -618,6 +618,17 @@ |
1201 | self._unfoldEmailHeader(expected), |
1202 | self._unfoldEmailHeader(observed)) |
1203 | |
1204 | + def assertStartsWith(self, s, prefix): |
1205 | + if not s.startswith(prefix): |
1206 | + raise AssertionError( |
1207 | + 'string %r does not start with %r' % (s, prefix)) |
1208 | + |
1209 | + def assertEndsWith(self, s, suffix): |
1210 | + """Asserts that s ends with suffix.""" |
1211 | + if not s.endswith(suffix): |
1212 | + raise AssertionError( |
1213 | + 'string %r does not end with %r' % (s, suffix)) |
1214 | + |
1215 | |
1216 | class TestCaseWithFactory(TestCase): |
1217 | |
1218 | |
1219 | === modified file 'lib/lp/testing/factory.py' |
1220 | --- lib/lp/testing/factory.py 2011-07-19 13:51:36 +0000 |
1221 | +++ lib/lp/testing/factory.py 2011-07-21 19:34:45 +0000 |
1222 | @@ -894,6 +894,9 @@ |
1223 | restricted=False): |
1224 | """Create a new processor family. |
1225 | |
1226 | + A default processor for the family will be created with the |
1227 | + same name as the family. |
1228 | + |
1229 | :param name: Name of the family (e.g. x86) |
1230 | :param title: Optional title of the family |
1231 | :param description: Optional extended description |
1232 | @@ -906,11 +909,11 @@ |
1233 | description = "Description of the %s processor family" % name |
1234 | if title is None: |
1235 | title = "%s and compatible processors." % name |
1236 | - family = getUtility(IProcessorFamilySet).new(name, title, description, |
1237 | - restricted=restricted) |
1238 | + family = getUtility(IProcessorFamilySet).new( |
1239 | + name, title, description, restricted=restricted) |
1240 | # Make sure there's at least one processor in the family, so that |
1241 | # other things can have a default processor. |
1242 | - self.makeProcessor(family=family) |
1243 | + self.makeProcessor(name=name, family=family) |
1244 | return family |
1245 | |
1246 | def makeProductRelease(self, milestone=None, product=None, |
1247 | |
1248 | === modified file 'lib/lp/testing/publication.py' |
1249 | --- lib/lp/testing/publication.py 2010-10-03 15:30:06 +0000 |
1250 | +++ lib/lp/testing/publication.py 2011-07-21 19:34:45 +0000 |
1251 | @@ -20,6 +20,7 @@ |
1252 | from zope.interface import providedBy |
1253 | from zope.publisher.interfaces.browser import IDefaultSkin |
1254 | from zope.security.management import restoreInteraction |
1255 | +from zope.security.proxy import removeSecurityProxy |
1256 | |
1257 | from canonical.launchpad.interfaces.launchpad import IOpenLaunchBag |
1258 | import canonical.launchpad.layers as layers |
1259 | @@ -30,6 +31,7 @@ |
1260 | ) |
1261 | from canonical.launchpad.webapp.servers import ProtocolErrorPublication |
1262 | |
1263 | + |
1264 | # Defines an helper function that returns the appropriate |
1265 | # IRequest and IPublication. |
1266 | def get_request_and_publication(host='localhost', port=None, |
1267 | @@ -119,9 +121,15 @@ |
1268 | getUtility(IOpenLaunchBag).clear() |
1269 | app = publication.getApplication(request) |
1270 | view = request.traverse(app) |
1271 | - # Since the last traversed object is the view, the second last should be |
1272 | - # the object that the view is on. |
1273 | - obj = request.traversed_objects[-2] |
1274 | + # Find the object from the view instead on relying that it stays |
1275 | + # in the traversed_objects stack. That doesn't apply to the web |
1276 | + # service for example. |
1277 | + try: |
1278 | + obj = removeSecurityProxy(view).context |
1279 | + except AttributeError: |
1280 | + # But sometime the view didn't store the context... |
1281 | + # Use the last traversed object in these cases. |
1282 | + obj = request.traversed_objects[-2] |
1283 | |
1284 | restoreInteraction() |
1285 | |
1286 | |
1287 | === modified file 'lib/lp/testing/tests/test_publication.py' |
1288 | --- lib/lp/testing/tests/test_publication.py 2010-10-04 19:50:45 +0000 |
1289 | +++ lib/lp/testing/tests/test_publication.py 2011-07-21 19:34:45 +0000 |
1290 | @@ -23,6 +23,7 @@ |
1291 | from canonical.launchpad.webapp.publisher import get_current_browser_request |
1292 | from canonical.launchpad.webapp.servers import LaunchpadTestRequest |
1293 | from canonical.testing.layers import DatabaseFunctionalLayer |
1294 | +from lazr.restful import EntryResource |
1295 | from lp.testing import ( |
1296 | ANONYMOUS, |
1297 | login, |
1298 | @@ -45,15 +46,17 @@ |
1299 | """ |
1300 | # This method is completely out of control. Thanks, Zope. |
1301 | name = '+' + self.factory.getUniqueString() |
1302 | + |
1303 | class new_class(simple): |
1304 | def __init__(self, context, request): |
1305 | + self.context = context |
1306 | view_callable() |
1307 | required = {} |
1308 | for n in ('browserDefault', '__call__', 'publishTraverse'): |
1309 | required[n] = CheckerPublic |
1310 | defineChecker(new_class, Checker(required)) |
1311 | getSiteManager().registerAdapter( |
1312 | - new_class, (ILaunchpadRoot, IDefaultBrowserLayer), Interface, |
1313 | + new_class, (ILaunchpadRoot, IDefaultBrowserLayer), Interface, |
1314 | name) |
1315 | self.addCleanup( |
1316 | getSiteManager().unregisterAdapter, new_class, |
1317 | @@ -74,6 +77,7 @@ |
1318 | # traversal in the sense of get_current_browser_request. |
1319 | login(ANONYMOUS) |
1320 | requests = [] |
1321 | + |
1322 | def record_current_request(): |
1323 | requests.append(get_current_browser_request()) |
1324 | context, view, request = test_traverse( |
1325 | @@ -96,9 +100,18 @@ |
1326 | person = self.factory.makePerson() |
1327 | login_person(person) |
1328 | users = [] |
1329 | + |
1330 | def record_user(): |
1331 | users.append(getUtility(ILaunchBag).user) |
1332 | context, view, request = test_traverse( |
1333 | self.registerViewCallable(record_user)) |
1334 | self.assertEqual(1, len(users)) |
1335 | self.assertEqual(person, users[0]) |
1336 | + |
1337 | + def test_webservice_traverse(self): |
1338 | + login(ANONYMOUS) |
1339 | + product = self.factory.makeProduct() |
1340 | + context, view, request = test_traverse( |
1341 | + 'http://api.launchpad.dev/devel/' + product.name) |
1342 | + self.assertEqual(product, context) |
1343 | + self.assertIsInstance(view, EntryResource) |
Another peculiar I forgot about: the test_traverse() helper wasn't working for API requests, I fixed. that.