Merge lp:~flacoste/launchpad/bug-801233 into lp:launchpad

[Francis J. Lacoste]

[r=bac][bug=801233] Export IBuilderSet.getBuildQueueSize(), getBuildersForQueue() and IBuilder.architecture over the API. Change the URL to IProcessor to be more user-friendly. Bonus: add proper URL entry to API documentation for many entry types. More bonus: Include :return: and :raise: docutils documentation for custom methods in the webservice documentation.

= Summary =

This is a massive Yak shaving branch! The goal is to write an API script which
will allow copy rebuilds to be manually allocated a number of builders. To do
this, we need to find dynamically the number of available builders for a
particular processor. That information isn't currently available over the
API.

== Proposed fix ==

It exports IBuilder.processor, IBuilderSet.getBuidQueueSizes() and
IBuilderSet.getBuilderrsForQueue(). Along with numerous other yak that needed
shaving along the way. An important one worth mentioning here: I changed the
URL of IProcessor. They were living under their family and their DB id was
part of th URL. Since IProcessor name are globally unique, I added a
/+processors collection and their URL point their using their name. That will
make it easier to get processor by crafting the URL.

== Pre-implementation notes ==

No pre-implementation done.

== Implementation details ==

Here are the numerous yak shaved along the way:

 * Removed getBuildersByArch() which wasn't used anywhere.
 * getBuildQueueSize() returns a dict containing datetime.timedelta object.
 simplejson didn't know how to serialize those, so I hooked up a very
 simple one which simply str() it. That was added into
 lp.services.webservice.json and hooked through zcml.
 * Collections check that launchpad.View is available on the entry. So I had
 to define adapters for IBuilder, IProcessor and IProcessorFamily.
 * Since there is no web page for IProcessor nor IProcessorFamily, I changed
 pulish_web_link to False.

Everything else should be straightforward, let me know if you have any
questions.

== Tests ==

 ./bin/test -vvt
'lp.soyuz.tests.test_processor|lp.soyuz.browser.tests.test_processor|lp.buildmaster.tests.test_webservice'

== Demo and Q/A ==

This will be QA-ed by checking the exported attribute and methods on
qastaging.

= Launchpad lint =

Checking for conflicts and issues in changed files.

Linting changed files:
  lib/lp/buildmaster/model/builder.py
  lib/lp/soyuz/tests/test_processor.py
  lib/lp/services/webservice/tests/__init__.py
  lib/lp/buildmaster/security.py
  lib/lp/soyuz/model/processor.py
  lib/lp/soyuz/interfaces/processor.py
  lib/lp/soyuz/security.py
  lib/lp/app/browser/launchpad.py
  lib/lp/testing/publication.py
  lib/lp/soyuz/configure.zcml
  lib/lp/testing/factory.py
  lib/lp/testing/__init__.py
  lib/lp/soyuz/interfaces/webservice.py
  lib/lp/buildmaster/configure.zcml
  lib/lp/soyuz/browser/configure.zcml
  lib/lp/services/webservice/tests/test_json.py
  lib/lp/testing/tests/test_publication.py
  lib/canonical/launchpad/configure.zcml
  lib/lp/services/webservice/json.py
  lib/lp/soyuz/browser/processor.py
  lib/lp/services/webservice/configure.zcml
  lib/lp/buildmaster/tests/test_webservice.py
  lib/lp/buildmaster/interfaces/builder.py
  lib/lp/soyuz/browser/tests/test_processor.py

./lib/lp/buildmaster/model/builder.py
     174: Line exceeds 78 characters.
     261: Line exceeds 78 characters.
     219: W291 trailing whitespace
     284: E301 expected 1 blank line, found 0
     302: E301 expected 1 blank line, found 0
     373: E301 expected 1 blank line, found 0
     517: E301 expected 1 blank line, found 0
     519: E301 expected 1 blank line, found 0
     608: E301 expected 1 blank line, found 0
     683: E301 expected 1 blank line, found 0
     686: E301 expected 1 blank line, found 0
./lib/lp/soyuz/tests/test_processor.py
      48: Line exceeds 78 characters.
      55: Line exceeds 78 characters.
      57: Line exceeds 78 characters.
     123: Line exceeds 78 characters.
      88: E202 whitespace before ')'
     123: E501 line too long (123 characters)
     123: E202 whitespace before ')'
./lib/lp/soyuz/model/processor.py
      87: E301 expected 1 blank line, found 0
./lib/lp/soyuz/security.py
      18: E302 expected 2 blank lines, found 1
      22: E302 expected 2 blank lines, found 1
./lib/lp/testing/publication.py
      36: E302 expected 2 blank lines, found 1
./lib/lp/services/webservice/tests/test_json.py
      27: E303 too many blank lines (2)
      30: W291 trailing whitespace
      33: W391 blank line at end of file
./lib/lp/testing/tests/test_publication.py
      49: E301 expected 1 blank line, found 0
      58: W291 trailing whitespace
      79: E301 expected 1 blank line, found 0
     101: E301 expected 1 blank line, found 0
./lib/lp/buildmaster/interfaces/builder.py
     139: Line exceeds 78 characters.
     210: Line exceeds 78 characters.
     249: W293 blank line contains whitespace
     256: W293 blank line contains whitespace
./lib/lp/soyuz/browser/tests/test_processor.py
      13: E302 expected 2 blank lines, found 1
      41: W391 blank line at end of file

I'll fix the white-space lint post-review.

[Francis J. Lacoste (flacoste)]

Another peculiar I forgot about: the test_traverse() helper wasn't working for API requests, I fixed. that.

[Brad Crittenden (bac)]

Hi Francis,

This is a really good follow up branch to the changes I made. Thanks for making the processor traversal by name.

I only found a few things to note:

* As we discussed on IRC the docstring for getBuildQueueSizes make the wadl generator unhappy.

* In lib/lp/services/webservice/json.py you should add StrJSONSerializer to __all__.

* In lib/lp/testing/factory.py, s/same name than the family./same name as the family./

* Some of the lint is real, including new code with super long lines such as in test_default_collection.

* in ProcessorSet getAll and getByName why don't you use the storeOf(self)?

review: Approve (code)

[Francis J. Lacoste (flacoste)]

On 11-07-11 03:43 PM, Brad Crittenden wrote:
> Review: Approve code
> Hi Francis,
>
> This is a really good follow up branch to the changes I made. Thanks for making the processor traversal by name.
>
> I only found a few things to note:
>
> * As we discussed on IRC the docstring for getBuildQueueSizes make the wadl generator unhappy.

Yeah, this was a simple matter of using the rst literal block directive
to make it happy. (::)

This introduces a few other drive-bys:

* Turned out that our WADL to HTML conversion excludes the RST
parameters table (because we don't want the :param name: documentation
to be redundant or misleading with the web-service specific
documentation.) So I added support for the :return: (and :raise:)
parameters for custom methods (when appropriate). This will make the
return value of a couple other JSON-returning method documentated.

* Got bored and tired of seeing so many Unkonwn URL entry messages, so I
fixed a bunch. Not all of them are fixed. Will eventually chase the
others, but it's still an improvement!

>
> * In lib/lp/services/webservice/json.py you should add StrJSONSerializer to __all__.

Done.

>
> * In lib/lp/testing/factory.py, s/same name than the family./same name as the family./
>

Done.

> * Some of the lint is real, including new code with super long lines such as in test_default_collection.

Branch is now lint-free!

>
> * in ProcessorSet getAll and getByName why don't you use the storeOf(self)?

Because self isn't a DB instance (and thus doesn't have a store).
(That's usually the case in *Set object.)

Thanks for the review!
--
Francis J. Lacoste
<email address hidden>

=== modified file 'lib/canonical/launchpad/configure.zcml'
--- lib/canonical/launchpad/configure.zcml	2011-02-23 21:15:23 +0000
+++ lib/canonical/launchpad/configure.zcml	2011-07-21 19:34:45 +0000
@@ -27,6 +27,7 @@
   <include package="lp.coop.answersbugs" />
   <include package="lp.code" />
   <include package="lp.soyuz" />
+  <include package="lp.services.webservice" />
   <include package="lp.translations" />
   <include package="lp.testopenid" />
   <include package="lp.blueprints" />
 
=== modified file 'lib/lp/app/browser/launchpad.py'
--- lib/lp/app/browser/launchpad.py	2011-06-30 11:28:59 +0000
+++ lib/lp/app/browser/launchpad.py	2011-07-21 19:34:45 +0000
@@ -146,7 +146,10 @@
 from lp.services.worlddata.interfaces.language import ILanguageSet
 from lp.soyuz.interfaces.binarypackagename import IBinaryPackageNameSet
 from lp.soyuz.interfaces.packageset import IPackagesetSet
-from lp.soyuz.interfaces.processor import IProcessorFamilySet
+from lp.soyuz.interfaces.processor import (
+    IProcessorFamilySet,
+    IProcessorSet,
+    )
 from lp.testopenid.interfaces.server import ITestOpenIDApplication
 from lp.translations.interfaces.translationgroup import ITranslationGroupSet
 from lp.translations.interfaces.translationimportqueue import (
@@ -617,6 +620,7 @@
         'people': IPersonSet,
         'pillars': IPillarNameSet,
         '+processor-families': IProcessorFamilySet,
+        '+processors': IProcessorSet,
         'projects': IProductSet,
         'projectgroups': IProjectGroupSet,
         'sourcepackagenames': ISourcePackageNameSet,
 
=== modified file 'lib/lp/buildmaster/configure.zcml'
--- lib/lp/buildmaster/configure.zcml	2010-11-19 13:25:25 +0000
+++ lib/lp/buildmaster/configure.zcml	2011-07-21 19:34:45 +0000
@@ -10,6 +10,7 @@
     xmlns:xmlrpc="http://namespaces.zope.org/xmlrpc"
     i18n_domain="launchpad">
     <include package=".browser"/>
+    <authorizations module=".security" />
 
     <!-- Builder -->
     <class
 
=== modified file 'lib/lp/buildmaster/interfaces/builder.py'
--- lib/lp/buildmaster/interfaces/builder.py	2011-02-24 15:30:54 +0000
+++ lib/lp/buildmaster/interfaces/builder.py	2011-07-21 19:34:45 +0000
@@ -27,6 +27,12 @@
     exported,
     operation_parameters,
     operation_returns_entry,
+    operation_returns_collection_of,
+    operation_for_version,
+    )
+from lazr.restful.fields import (
+    Reference,
+    ReferenceChoice,
     )
 from zope.interface import (
     Attribute,
@@ -34,7 +40,6 @@
     )
 from zope.schema import (
     Bool,
-    Choice,
     Field,
     Int,
     Text,
@@ -45,6 +50,7 @@
 from lp.app.validators.name import name_validator
 from lp.app.validators.url import builder_url_validator
 from lp.registry.interfaces.role import IHasOwner
+from lp.soyuz.interfaces.processor import IProcessor
 from lp.services.fields import (
     Description,
     PersonChoice,
@@ -106,10 +112,12 @@
 
     id = Attribute("Builder identifier")
 
-    processor = Choice(
+    processor = exported(ReferenceChoice(
         title=_('Processor'), required=True, vocabulary='Processor',
+        schema=IProcessor,
         description=_('Build Slave Processor, used to identify '
-                      'which jobs can be built by this device.'))
+                      'which jobs can be built by this device.')),
+        as_of='devel', readonly=True)
 
     owner = exported(PersonChoice(
         title=_('Owner'), required=True, vocabulary='ValidOwner',
@@ -128,7 +136,8 @@
 
     title = exported(Title(
         title=_('Title'), required=True,
-        description=_('The builder slave title. Should be just a few words.')))
+        description=_(
+            'The builder slave title. Should be just a few words.')))
 
     description = exported(Description(
         title=_('Description'), required=False,
@@ -199,8 +208,8 @@
         :param file_sha1: The file's sha1, which is how the file is addressed
             in the slave XMLRPC protocol. Specially, the file_sha1 'buildlog'
             will cause the build log to be retrieved and gzipped.
-        :param filename: The name of the file to be given to the librarian file
-            alias.
+        :param filename: The name of the file to be given to the librarian
+            file alias.
         :param private: True if the build is for a private archive.
         :return: A Deferred that calls back with a librarian file alias.
         """
@@ -238,14 +247,14 @@
 
     def updateStatus(logger=None):
         """Update the builder's status by probing it.
-        
+
         :return: A Deferred that fires when the dialog with the slave is
             finished.  It does not have a return value.
         """
 
     def cleanSlave():
         """Clean any temporary files from the slave.
-        
+
         :return: A Deferred that fires when the dialog with the slave is
             finished.  It does not have a return value.
         """
@@ -385,26 +394,34 @@
     def getBuilders():
         """Return all active configured builders."""
 
-    def getBuildersByArch(arch):
-        """Return all configured builders for a given DistroArchSeries."""
-
+    @export_read_operation()
+    @operation_for_version('devel')
     def getBuildQueueSizes():
         """Return the number of pending builds for each processor.
 
         :return: a dict of tuples with the queue size and duration for
-            each processor and virtualisation. For example:
-            {
-                'virt': {
-                            '386': (1, datetime.timedelta(0, 60)),
-                            'amd64': (2, datetime.timedelta(0, 30)),
-                        },
-                'nonvirt':...
-            }
+            each processor and virtualisation. For example::
+
+                {
+                    'virt': {
+                                '386': (1, datetime.timedelta(0, 60)),
+                                'amd64': (2, datetime.timedelta(0, 30)),
+                            },
+                    'nonvirt':...
+                }
 
             The tuple contains the size of the queue, as an integer,
             and the sum of the jobs 'estimated_duration' in queue,
             as a timedelta or None for empty queues.
         """
 
+    @operation_parameters(
+        processor=Reference(
+            title=_("Processor"), required=True, schema=IProcessor),
+        virtualized=Bool(
+            title=_("Virtualized"), required=False, default=True))
+    @operation_returns_collection_of(IBuilder)
+    @export_read_operation()
+    @operation_for_version('devel')
     def getBuildersForQueue(processor, virtualized):
         """Return all builders for given processor/virtualization setting."""
 
=== modified file 'lib/lp/buildmaster/model/builder.py'
--- lib/lp/buildmaster/model/builder.py	2011-02-01 18:14:57 +0000
+++ lib/lp/buildmaster/model/builder.py	2011-07-21 19:34:45 +0000
@@ -171,7 +171,8 @@
 
         :param builder_url: The URL of the slave buildd machine,
             e.g. http://localhost:8221
-        :param vm_host: If the slave is virtual, specify its host machine here.
+        :param vm_host: If the slave is virtual, specify its host machine
+            here.
         :param reactor: Used by tests to override the Twisted reactor.
         :param proxy: Used By tests to override the xmlrpc.Proxy.
         """
@@ -216,7 +217,7 @@
     def getFile(self, sha_sum, file_to_write):
         """Fetch a file from the builder.
 
-        :param sha_sum: The sha of the file (which is also its name on the 
+        :param sha_sum: The sha of the file (which is also its name on the
             builder)
         :param file_to_write: A file name or file-like object to write
             the file to
@@ -258,7 +259,8 @@
         resume_command = config.builddmaster.vm_resume_command % {
             'vm_host': self._vm_host}
         # Twisted API requires string but the configuration provides unicode.
-        resume_argv = [term.encode('utf-8') for term in resume_command.split()]
+        resume_argv = [
+            term.encode('utf-8') for term in resume_command.split()]
         d = defer.Deferred()
         p = ProcessWithTimeout(
             d, config.builddmaster.socket_timeout, clock=clock)
@@ -281,6 +283,7 @@
     def sendFileToSlave(self, sha1, url, username="", password=""):
         """Helper to send the file at 'url' with 'sha1' to this builder."""
         d = self.ensurepresent(sha1, url, username, password)
+
         def check_present((present, info)):
             if not present:
                 raise CannotFetchFile(url, info)
@@ -299,6 +302,7 @@
         """
         d = self._with_timeout(self._server.callRemote(
             'build', buildid, builder_type, chroot_sha1, filemap, args))
+
         def got_fault(failure):
             failure.trap(xmlrpclib.Fault)
             raise BuildSlaveFailure(failure.value)
@@ -370,6 +374,7 @@
                 d = builder.cleanSlave()
             else:
                 d = builder.requestAbort()
+
             def log_rescue(ignored):
                 if logger:
                     logger.info(
@@ -514,8 +519,10 @@
         logger.info("Resuming %s (%s)" % (self.name, self.url))
 
         d = self.slave.resume()
+
         def got_resume_ok((stdout, stderr, returncode)):
             return stdout, stderr
+
         def got_resume_bad(failure):
             stdout, stderr, code = failure.value
             raise CannotResumeHost(
@@ -605,6 +612,7 @@
     def slaveStatus(self):
         """See IBuilder."""
         d = self.slave.status()
+
         def got_status(status_sentence):
             status = {'builder_status': status_sentence[0]}
 
@@ -680,9 +688,11 @@
         if not self.builderok:
             return defer.succeed(False)
         d = self.slaveStatusSentence()
+
         def catch_fault(failure):
             failure.trap(xmlrpclib.Fault, socket.error)
             return False
+
         def check_available(status):
             return status[0] == BuilderStatus.IDLE
         return d.addCallbacks(check_available, catch_fault)
@@ -896,13 +906,6 @@
         return Builder.selectBy(
             active=True, orderBy=['virtualized', 'processor', 'name'])
 
-    def getBuildersByArch(self, arch):
-        """See IBuilderSet."""
-        return Builder.select('builder.processor = processor.id '
-                              'AND processor.family = %d'
-                              % arch.processorfamily.id,
-                              clauseTables=("Processor",))
-
     def getBuildQueueSizes(self):
         """See `IBuilderSet`."""
         store = getUtility(IStoreSelector).get(MAIN_STORE, SLAVE_FLAVOR)
 
=== added file 'lib/lp/buildmaster/security.py'
--- lib/lp/buildmaster/security.py	1970-01-01 00:00:00 +0000
+++ lib/lp/buildmaster/security.py	2011-07-21 19:34:45 +0000
@@ -0,0 +1,19 @@
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Security adapters for the buildmaster package."""
+
+__metaclass__ = type
+__all__ = [
+    'ViewBuilder',
+    ]
+
+from lp.app.security import AnonymousAuthorization
+from lp.buildmaster.interfaces.builder import (
+    IBuilder,
+    )
+
+
+class ViewBuilder(AnonymousAuthorization):
+    """Anyone can view a `IBuilder`."""
+    usedfor = IBuilder
 
=== added file 'lib/lp/buildmaster/tests/test_webservice.py'
--- lib/lp/buildmaster/tests/test_webservice.py	1970-01-01 00:00:00 +0000
+++ lib/lp/buildmaster/tests/test_webservice.py	2011-07-21 19:34:45 +0000
@@ -0,0 +1,68 @@
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for the builders webservice ."""
+
+__metaclass__ = type
+
+from canonical.testing.layers import DatabaseFunctionalLayer
+from canonical.launchpad.testing.pages import LaunchpadWebServiceCaller
+from lp.testing import (
+    api_url,
+    logout,
+    TestCaseWithFactory,
+    )
+
+
+class TestBuildersCollection(TestCaseWithFactory):
+    layer = DatabaseFunctionalLayer
+
+    def setUp(self):
+        super(TestBuildersCollection, self).setUp()
+        self.webservice = LaunchpadWebServiceCaller()
+
+    def test_getBuildQueueSizes(self):
+        logout()
+        results = self.webservice.named_get(
+            '/builders', 'getBuildQueueSizes', api_version='devel')
+        self.assertEquals(
+            ['nonvirt', 'virt'], sorted(results.jsonBody().keys()))
+
+    def test_getBuildersForQueue(self):
+        g1 = self.factory.makeProcessorFamily('g1').processors[0]
+        quantum = self.factory.makeProcessorFamily('quantum').processors[0]
+        self.factory.makeBuilder(
+            processor=quantum, name='quantum_builder1')
+        self.factory.makeBuilder(
+            processor=quantum, name='quantum_builder2')
+        self.factory.makeBuilder(
+            processor=quantum, name='quantum_builder3', virtualized=False)
+        self.factory.makeBuilder(
+            processor=g1, name='g1_builder', virtualized=False)
+
+        logout()
+        results = self.webservice.named_get(
+            '/builders', 'getBuildersForQueue',
+            processor=api_url(quantum), virtualized=True,
+            api_version='devel').jsonBody()
+        self.assertEquals(
+            ['quantum_builder1', 'quantum_builder2'],
+            sorted(builder['name'] for builder in results['entries']))
+
+
+class TestBuilderEntry(TestCaseWithFactory):
+    layer = DatabaseFunctionalLayer
+
+    def setUp(self):
+        super(TestBuilderEntry, self).setUp()
+        self.webservice = LaunchpadWebServiceCaller()
+
+    def test_exports_processor(self):
+        processor_family = self.factory.makeProcessorFamily('s1')
+        builder = self.factory.makeBuilder(
+            processor=processor_family.processors[0])
+
+        logout()
+        entry = self.webservice.get(
+            api_url(builder), api_version='devel').jsonBody()
+        self.assertEndsWith(entry['processor_link'], '/+processors/s1')
 
=== added file 'lib/lp/services/webservice/configure.zcml'
--- lib/lp/services/webservice/configure.zcml	1970-01-01 00:00:00 +0000
+++ lib/lp/services/webservice/configure.zcml	2011-07-21 19:34:45 +0000
@@ -0,0 +1,15 @@
+<!-- Copyright 2011 Canonical Ltd.  This software is licensed under the
+     GNU Affero General Public License version 3 (see the file LICENSE).
+-->
+
+<configure
+    xmlns="http://namespaces.zope.org/zope"
+    xmlns:i18n="http://namespaces.zope.org/i18n"
+    i18n_domain="launchpad">
+
+    <adapter
+            provides="lazr.restful.interfaces.IJSONPublishable"
+            for="zope.interface.common.idatetime.ITimeDelta"
+            factory="lp.services.webservice.json.StrJSONSerializer"
+            permission="zope.Public"/>
+</configure>
 
=== added file 'lib/lp/services/webservice/json.py'
--- lib/lp/services/webservice/json.py	1970-01-01 00:00:00 +0000
+++ lib/lp/services/webservice/json.py	2011-07-21 19:34:45 +0000
@@ -0,0 +1,24 @@
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Additional JSON serializer for the web service."""
+
+__metaclass__ = type
+__all__ = [
+    'StrJSONSerializer',
+    ]
+
+
+from lazr.restful.interfaces import IJSONPublishable
+from zope.interface import implements
+
+
+class StrJSONSerializer:
+    """Simple JSON serializer that simply str() it's context. """
+    implements(IJSONPublishable)
+
+    def __init__(self, context):
+        self.context = context
+
+    def toDataForJSON(self, media_type):
+        return str(self.context)
 
=== added directory 'lib/lp/services/webservice/tests'
=== added file 'lib/lp/services/webservice/tests/__init__.py'
=== added file 'lib/lp/services/webservice/tests/test_json.py'
--- lib/lp/services/webservice/tests/test_json.py	1970-01-01 00:00:00 +0000
+++ lib/lp/services/webservice/tests/test_json.py	2011-07-21 19:34:45 +0000
@@ -0,0 +1,30 @@
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for the JSON serializer."""
+
+__metaclass__ = type
+
+from datetime import timedelta
+
+from canonical.testing.layers import FunctionalLayer
+from lazr.restful.interfaces import IJSONPublishable
+from lp.services.webservice.json import StrJSONSerializer
+from lp.testing import TestCase
+
+
+class TestStrJSONSerializer(TestCase):
+    layer = FunctionalLayer
+
+    def test_toDataForJSON(self):
+        serializer = StrJSONSerializer(
+            timedelta(days=2, hours=2, seconds=5))
+        self.assertEquals(
+            '2 days, 2:00:05',
+            serializer.toDataForJSON('application/json'))
+
+    def test_timedelta_users_StrJSONSerializer(self):
+        delta = timedelta(seconds=5)
+        serializer = IJSONPublishable(delta)
+        self.assertEquals('0:00:05',
+            serializer.toDataForJSON('application/json'))
 
=== modified file 'lib/lp/services/webservice/wadl-to-refhtml.xsl'
--- lib/lp/services/webservice/wadl-to-refhtml.xsl	2011-01-13 19:45:32 +0000
+++ lib/lp/services/webservice/wadl-to-refhtml.xsl	2011-07-21 19:34:45 +0000
@@ -165,9 +165,16 @@
         <xsl:call-template name="resource-uri-doc">
             <xsl:with-param name="url">
                 <xsl:choose>
-                    <xsl:when test="@id = 'has_milestones'
-                                    or @id = 'bug_target'
-                                    or @id = 'has_bugs'">
+                    <xsl:when test="
+                        @id = 'bug_link_target'
+                        or @id = 'bug_target'
+                        or @id = 'has_bugs'
+                        or @id = 'has_milestones'
+                        or @id = 'object_with_translation_imports'
+                        or @id = 'question_target'
+                        or @id = 'specification_target'
+                        or @id = 'structural_subscription_target'
+                        ">
                         <em>depends on the underlying entry</em>
                     </xsl:when>
                     <xsl:otherwise>
@@ -264,10 +271,18 @@
                 <xsl:text>/+build/</xsl:text>
                 <var>&lt;id&gt;</var>
             </xsl:when>
+            <xsl:when test="@id = 'builder'">
+                <xsl:text>/builders/</xsl:text>
+                <var>&lt;builder.name&gt;</var>
+            </xsl:when>
             <xsl:when test="@id = 'cve'">
                 <xsl:text>/bugs/cve/</xsl:text>
                 <var>&lt;sequence&gt;</var>
             </xsl:when>
+            <xsl:when test="@id = 'country'">
+                <xsl:text>/+countries/</xsl:text>
+                <var>&lt;iso3166code2&gt;</var>
+            </xsl:when>
             <xsl:when test="@id = 'distribution_source_package'">
                 <xsl:text>/</xsl:text>
                 <var>&lt;distribution.name&gt;</var>
@@ -294,6 +309,15 @@
                 <xsl:text>/+email/</xsl:text>
                 <var>&lt;email&gt;</var>
             </xsl:when>
+            <xsl:when test="@id = 'gpg_key'">
+                <xsl:text>/</xsl:text>
+                <var>&lt;person.name&gt;</var>
+                <xsl:text>/+gpg-keys/</xsl:text>
+                <var>&lt;keyid&gt;</var>
+            </xsl:when>
+            <xsl:when test="@id = 'hwdb'">
+                <xsl:text>/+hwdb</xsl:text>
+            </xsl:when>
             <xsl:when test="@id = 'h_w_device'">
                 <xsl:text>/+hwdb/+device/</xsl:text>
                 <var>&lt;id&gt;</var>
@@ -360,6 +384,17 @@
                 <xsl:text>/~</xsl:text>
                 <var>&lt;name&gt;</var>
             </xsl:when>
+            <xsl:when test="@id = 'pillars'">
+                <xsl:text>/pillars</xsl:text>
+            </xsl:when>
+            <xsl:when test="@id = 'processor'">
+                <xsl:text>/+processors/</xsl:text>
+                <var>&lt;processor.name&gt;</var>
+            </xsl:when>
+            <xsl:when test="@id = 'processor_family'">
+                <xsl:text>/+processor-families/</xsl:text>
+                <var>&lt;processor_family.name&gt;</var>
+            </xsl:when>
             <xsl:when test="@id = 'product_release'">
                 <xsl:text>/</xsl:text>
                 <var>&lt;product.name&gt;</var>
@@ -398,6 +433,20 @@
                 <xsl:text>/</xsl:text>
                 <var>&lt;name&gt;</var>
             </xsl:when>
+            <xsl:when test="@id = 'question'">
+                <xsl:text>/</xsl:text>
+                <var>&lt;target.name&gt;</var>
+                <xsl:text>/+question/</xsl:text>
+                <var >&lt;question.id&gt;</var>
+            </xsl:when>
+            <xsl:when test="@id = 'question_message'">
+                <xsl:text>/</xsl:text>
+                <var>&lt;target.name&gt;</var>
+                <xsl:text>/+question/</xsl:text>
+                <var >&lt;question.id&gt;</var>
+                <xsl:text>/messages/</xsl:text>
+                <var >&lt;message.index&gt;</var>
+            </xsl:when>
             <xsl:when test="@id = 'source_package'">
                 <xsl:text>/</xsl:text>
                 <var>&lt;distribution.name&gt;</var>
@@ -414,6 +463,34 @@
                 <xsl:text>/+sourcepub/</xsl:text>
                 <var>&lt;id&gt;</var>
             </xsl:when>
+            <xsl:when test="@id = 'specification'">
+                <xsl:text>/</xsl:text>
+                <var>&lt;target.name&gt;</var>
+                <xsl:text>/+spec/</xsl:text>
+                <var >&lt;specification.name&gt;</var>
+            </xsl:when>
+            <xsl:when test="@id = 'specification_branch'">
+                <xsl:text>/</xsl:text>
+                <var>&lt;target.name&gt;</var>
+                <xsl:text>/+spec/</xsl:text>
+                <var >&lt;specification.name&gt;</var>
+                <xsl:text>/+branch/</xsl:text>
+                <var >&lt;branch.unique_name[1:]&gt;</var>
+            </xsl:when>
+            <xsl:when test="@id = 'specification_subscription'">
+                <xsl:text>/</xsl:text>
+                <var>&lt;target.name&gt;</var>
+                <xsl:text>/+spec/</xsl:text>
+                <var >&lt;specification.name&gt;</var>
+                <xsl:text>/+subscription/</xsl:text>
+                <var >&lt;person.name&gt;</var>
+            </xsl:when>
+            <xsl:when test="@id = 'ssh_key'">
+                <xsl:text>/</xsl:text>
+                <var>&lt;person.name&gt;</var>
+                <xsl:text>/+ssh-keys/</xsl:text>
+                <var>&lt;keyid&gt;</var>
+            </xsl:when>
             <xsl:when test="@id = 'team_membership'">
                 <xsl:text>/~</xsl:text>
                 <var>&lt;team.name&gt;</var>
@@ -864,6 +941,12 @@
                     <xsl:apply-templates select="wadl:doc"/>
                     <xsl:apply-templates select="wadl:request"/>
                     <xsl:apply-templates select="wadl:response"/>
+                    <xsl:if test="not(wadl:response)">
+                        <xsl:apply-templates select="wadl:doc//html:th[
+                            node() = 'return:'
+                            ]"/>
+                    </xsl:if>
+                    <xsl:call-template name="error-documentation"/>
                 </xsl:when>
                 <xsl:otherwise>
                     <p><em>Missing documentation.</em></p>
@@ -910,6 +993,39 @@
         </p>
     </xsl:template>
 
+    <!-- Documentation of the custom method return type. -->
+    <xsl:template match="wadl:doc//html:th[node() = 'return:']">
+        <h6>Response (application/json)</h6>
+        <xsl:choose>
+            <xsl:when test="following-sibling::html:td/text()">
+                <p><xsl:apply-templates select="following-sibling::html:td"
+                      mode="copy"/></p>
+            </xsl:when>
+            <xsl:otherwise>
+                <xsl:apply-templates select="following-sibling::html:td"
+                      mode="copy"/>
+            </xsl:otherwise>
+        </xsl:choose>
+    </xsl:template>
+
+    <!-- Documentation of the error raised by the operation. -->
+    <xsl:template name="error-documentation">
+        <xsl:if test="wadl:doc//html:th[node() = 'raise:']">
+            <h6>Errors</h6>
+            <ul>
+                <xsl:apply-templates
+                    select="wadl:doc//html:th[node() = 'raise:']"/>
+            </ul>
+        </xsl:if>
+    </xsl:template>
+
+    <xsl:template match="wadl:doc//html:th[node() = 'raise:']">
+        <li>
+            <xsl:apply-templates select="following-sibling::html:td"
+                mode="copy"/>
+        </li>
+    </xsl:template>
+
     <!-- Documentation for request parameter. -->
     <xsl:template match="wadl:param">
         <tr>
@@ -1032,11 +1148,15 @@
              mode="representation-type"/>
     </xsl:template>
 
-    <!-- Omit docutils parameter lists in methods since they are redundant
-    or misleading with the one we give. -->
+    <!-- Omit docutils parameter table in methods. The parameter names
+         description is either redundant or misleading with the one we
+         give.
+
+         We process the return and raise parameters separately.
+      -->
     <xsl:template match="wadl:method//html:table[
         contains(@class, 'field-list')]"
-        mode="copy"/>
+        mode="copy" />
 
     <!-- Output the mediaType attribute of a representation -->
     <xsl:template match="wadl:representation[@mediaType]"
 
=== modified file 'lib/lp/soyuz/browser/configure.zcml'
--- lib/lp/soyuz/browser/configure.zcml	2011-06-23 13:42:38 +0000
+++ lib/lp/soyuz/browser/configure.zcml	2011-07-21 19:34:45 +0000
@@ -32,18 +32,20 @@
             path_expression="string:+binarypub"
             attribute_to_parent="archive"
             urldata="lp.soyuz.browser.publishing.BinaryPublicationURL"/>
-        <browser:url
-	    for="lp.soyuz.interfaces.processor.IProcessorFamilySet"
+        <browser:url for="lp.soyuz.interfaces.processor.IProcessorFamilySet"
             path_expression="string:+processor-families"
             parent_utility="canonical.launchpad.webapp.interfaces.ILaunchpadRoot"/>
         <browser:url
 	    for="lp.soyuz.interfaces.processor.IProcessorFamily"
             path_expression="string:${name}"
             parent_utility="lp.soyuz.interfaces.processor.IProcessorFamilySet" />
+        <browser:url for="lp.soyuz.interfaces.processor.IProcessorSet"
+            path_expression="string:+processors"
+            parent_utility="canonical.launchpad.webapp.interfaces.ILaunchpadRoot"/>
         <browser:url
 	    for="lp.soyuz.interfaces.processor.IProcessor"
-            path_expression="string:${id}"
-            attribute_to_parent="family" />
+            path_expression="string:${name}"
+            parent_utility="lp.soyuz.interfaces.processor.IProcessorSet" />
     </facet>
     <browser:navigation
         module="lp.soyuz.browser.binarypackagerelease"
@@ -233,7 +235,7 @@
     <browser:navigation
         module="lp.soyuz.browser.processor"
 	classes="
-	    ProcessorFamilySetNavigation ProcessorFamilyNavigation"/>
+	    ProcessorFamilySetNavigation ProcessorSetNavigation"/>
     <browser:url
         for="lp.soyuz.interfaces.archive.IPPA"
         path_expression="string:+archive"
 
=== modified file 'lib/lp/soyuz/browser/processor.py'
--- lib/lp/soyuz/browser/processor.py	2011-06-23 16:11:42 +0000
+++ lib/lp/soyuz/browser/processor.py	2011-07-21 19:34:45 +0000
@@ -8,15 +8,15 @@
 
 __all__ = [
     'ProcessorFamilySetNavigation',
-    'ProcessorFamilyNavigation',
+    'ProcessorSetNavigation',
     ]
 
 
 from canonical.launchpad.webapp import Navigation
 from lp.app.errors import NotFoundError
 from lp.soyuz.interfaces.processor import (
-    IProcessorFamily,
     IProcessorFamilySet,
+    IProcessorSet,
     )
 
 
@@ -32,15 +32,9 @@
         return family
 
 
-class ProcessorFamilyNavigation(Navigation):
-    """IProcessorFamily navigation."""
-
-    usedfor = IProcessorFamily
-
-    def traverse(self, id_):
-        id_ = int(id_)
-        processors = self.processors
-        for p in processors:
-            if p.id == id_:
-                return p
-        raise NotFoundError(id_)
+class ProcessorSetNavigation(Navigation):
+    """IProcessorFamilySet navigation."""
+    usedfor = IProcessorSet
+
+    def traverse(self, name):
+        return self.context.getByName(name)
 
=== added file 'lib/lp/soyuz/browser/tests/test_processor.py'
--- lib/lp/soyuz/browser/tests/test_processor.py	1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/browser/tests/test_processor.py	2011-07-21 19:34:45 +0000
@@ -0,0 +1,41 @@
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for process navigation."""
+
+__metaclass__ = type
+
+from canonical.testing.layers import DatabaseFunctionalLayer
+from canonical.launchpad.webapp.publisher import canonical_url
+from lp.testing import TestCaseWithFactory
+from lp.testing.publication import test_traverse
+
+
+class TestProcessorNavigation(TestCaseWithFactory):
+    layer = DatabaseFunctionalLayer
+
+    def test_processor_family_url(self):
+        family = self.factory.makeProcessorFamily('quantum')
+        self.assertEquals(
+            '/+processor-families/quantum',
+            canonical_url(family, force_local_path=True))
+
+    def test_processor_url(self):
+        family = self.factory.makeProcessorFamily('quantum')
+        quantum = family.processors[0]
+        self.assertEquals(
+            '/+processors/quantum',
+            canonical_url(quantum, force_local_path=True))
+
+    def test_processor_family_navigation(self):
+        family = self.factory.makeProcessorFamily('quantum')
+        obj, view, request = test_traverse(
+            'http://api.launchpad.dev/devel/+processor-families/quantum')
+        self.assertEquals(family, obj)
+
+    def test_processor_navigation(self):
+        family = self.factory.makeProcessorFamily('quantum')
+        obj, view, request = test_traverse(
+            'http://api.launchpad.dev/'
+            'devel/+processors/quantum')
+        self.assertEquals(family.processors[0], obj)
 
=== modified file 'lib/lp/soyuz/configure.zcml'
--- lib/lp/soyuz/configure.zcml	2011-07-12 14:23:40 +0000
+++ lib/lp/soyuz/configure.zcml	2011-07-21 19:34:45 +0000
@@ -11,6 +11,7 @@
     i18n_domain="launchpad">
     <include
         package=".browser"/>
+    <authorizations module=".security" />
 
     <!-- PackageCloner -->
 
@@ -375,6 +376,12 @@
         <allow
             interface="lp.soyuz.interfaces.processor.IProcessorFamilySet"/>
     </securedutility>
+    <securedutility
+        class="lp.soyuz.model.processor.ProcessorSet"
+        provides="lp.soyuz.interfaces.processor.IProcessorSet">
+        <allow
+            interface="lp.soyuz.interfaces.processor.IProcessorSet"/>
+    </securedutility>
     <adapter
         for="lp.soyuz.interfaces.distroarchseriesbinarypackagerelease.IDistroArchSeriesBinaryPackageRelease"
         provides="canonical.launchpad.webapp.interfaces.IBreadcrumb"
 
=== modified file 'lib/lp/soyuz/interfaces/processor.py'
--- lib/lp/soyuz/interfaces/processor.py	2011-06-23 16:11:42 +0000
+++ lib/lp/soyuz/interfaces/processor.py	2011-07-21 19:34:45 +0000
@@ -11,6 +11,8 @@
     'IProcessor',
     'IProcessorFamily',
     'IProcessorFamilySet',
+    'IProcessorSet',
+    'ProcessorNotFound',
     ]
 
 from zope.interface import (
@@ -38,6 +40,12 @@
     CollectionField,
     Reference,
     )
+from lp.app.errors import NameLookupFailed
+
+
+class ProcessorNotFound(NameLookupFailed):
+    """Exception raised when a processor name isn't found."""
+    _message_prefix = 'No such processor'
 
 
 class IProcessor(Interface):
@@ -49,7 +57,7 @@
     # the WADL generation work it must be back-dated to the earliest version.
     # Note that individual attributes and methods can and must truthfully set
     # 'devel' as their version.
-    export_as_webservice_entry(publish_web_link=True, as_of='beta')
+    export_as_webservice_entry(publish_web_link=False, as_of='beta')
     id = Attribute("The Processor ID")
     family = exported(
         Reference(
@@ -84,7 +92,7 @@
     # 'devel' as their version.
     export_as_webservice_entry(
         plural_name='processor_families',
-        publish_web_link=True,
+        publish_web_link=False,
         as_of='beta')
 
     id = Attribute("The ProcessorFamily ID")
@@ -123,14 +131,36 @@
         """
 
 
+class IProcessorSet(Interface):
+    """Operations related to Processor instances."""
+    export_as_webservice_collection(IProcessor)
+
+    @operation_parameters(
+        name=TextLine(required=True))
+    @operation_returns_entry(IProcessor)
+    @export_read_operation()
+    @operation_for_version('devel')
+    def getByName(name):
+        """Return the IProcessor instance with the matching name.
+
+        :param name: The name to look for.
+        :raise ProcessorNotFound: if there is no processor with that name.
+        :return: A `IProcessor` instance if found
+        """
+
+    @collection_default_content()
+    def getAll():
+        """Return all the `IProcessor` known to Launchpad."""
+
+
 class IProcessorFamilySet(Interface):
     """Operations related to ProcessorFamily instances."""
 
-    export_as_webservice_collection(Interface)
+    export_as_webservice_collection(IProcessorFamily)
 
     @operation_parameters(
         name=TextLine(required=True))
-    @operation_returns_entry(Interface)
+    @operation_returns_entry(IProcessorFamily)
     @export_read_operation()
     @operation_for_version('devel')
     def getByName(name):
 
=== modified file 'lib/lp/soyuz/interfaces/webservice.py'
--- lib/lp/soyuz/interfaces/webservice.py	2011-06-23 18:26:26 +0000
+++ lib/lp/soyuz/interfaces/webservice.py	2011-07-21 19:34:45 +0000
@@ -35,6 +35,7 @@
     'IProcessor',
     'IProcessorFamily',
     'IProcessorFamilySet',
+    'IProcessorSet',
     'ISourcePackagePublishingHistory',
     'IncompatibleArguments',
     'InsufficientUploadRights',
@@ -96,6 +97,7 @@
     IProcessor,
     IProcessorFamily,
     IProcessorFamilySet,
+    IProcessorSet,
     )
 from lp.soyuz.interfaces.publishing import (
     IBinaryPackagePublishingHistory,
@@ -105,7 +107,6 @@
 
 from canonical.launchpad.components.apihelpers import (
     patch_collection_property,
-    patch_entry_return_type,
     patch_plain_parameter_type,
     patch_reference_property,
     )
@@ -115,15 +116,10 @@
 from canonical.launchpad.interfaces import _schema_circular_imports
 _schema_circular_imports
 
-from lazr.restful.declarations import LAZR_WEBSERVICE_EXPORTED
-IProcessorFamilySet.queryTaggedValue(
-    LAZR_WEBSERVICE_EXPORTED)['collection_entry_schema'] = IProcessorFamily
-
 # IProcessor
 patch_reference_property(
     IProcessor, 'family', IProcessorFamily)
 
-patch_entry_return_type(IProcessorFamilySet, 'getByName', IProcessorFamily)
 patch_collection_property(
     IArchive, 'enabled_restricted_families', IProcessorFamily)
 patch_plain_parameter_type(
 
=== modified file 'lib/lp/soyuz/model/processor.py'
--- lib/lp/soyuz/model/processor.py	2010-11-12 02:15:28 +0000
+++ lib/lp/soyuz/model/processor.py	2011-07-21 19:34:45 +0000
@@ -25,6 +25,8 @@
     IProcessor,
     IProcessorFamily,
     IProcessorFamilySet,
+    IProcessorSet,
+    ProcessorNotFound,
     )
 
 
@@ -42,6 +44,24 @@
         return "<Processor %r>" % self.title
 
 
+class ProcessorSet:
+    """See `IProcessorSet`."""
+    implements(IProcessorSet)
+
+    def getByName(self, name):
+        """See `IProcessorSet`."""
+        store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
+        processor = store.find(Processor, Processor.name == name).one()
+        if processor is None:
+            raise ProcessorNotFound(name)
+        return processor
+
+    def getAll(self):
+        """See `IProcessorSet`."""
+        store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
+        return store.find(Processor)
+
+
 class ProcessorFamily(SQLBase):
     implements(IProcessorFamily)
     _table = 'ProcessorFamily'
@@ -64,6 +84,7 @@
 
 class ProcessorFamilySet:
     implements(IProcessorFamilySet)
+
     def getByName(self, name):
         """Please see `IProcessorFamilySet`."""
         # Please note that ProcessorFamily.name is unique i.e. the database
 
=== added file 'lib/lp/soyuz/security.py'
--- lib/lp/soyuz/security.py	1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/security.py	2011-07-21 19:34:45 +0000
@@ -0,0 +1,26 @@
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Security adapters for the soyuz module."""
+
+__metaclass__ = type
+__all__ = [
+    'ViewProcessor',
+    'ViewProcessorFamily',
+    ]
+
+from lp.app.security import AnonymousAuthorization
+from lp.soyuz.interfaces.processor import (
+    IProcessor,
+    IProcessorFamily,
+    )
+
+
+class ViewProcessor(AnonymousAuthorization):
+    """Anyone can view an `IProcessor`."""
+    usedfor = IProcessor
+
+
+class ViewProcessorFamily(AnonymousAuthorization):
+    """Anyone can view an `IProcessorFamily`."""
+    usedfor = IProcessorFamily
 
=== modified file 'lib/lp/soyuz/tests/test_packagecloner.py'
--- lib/lp/soyuz/tests/test_packagecloner.py	2011-06-23 09:31:34 +0000
+++ lib/lp/soyuz/tests/test_packagecloner.py	2011-07-21 19:34:45 +0000
@@ -344,7 +344,6 @@
         # This is a processor family without a DAS in the source, so
         # we expect no builds.
         family = self.factory.makeProcessorFamily(name="armel")
-        self.factory.makeProcessor(family=family, name="armel")
         proc_families = [family]
         copy_archive, distroseries = self.makeCopyArchive(
             [package_info], proc_families=proc_families)
@@ -357,7 +356,6 @@
         # One of these processor families has a DAS in the source, so
         # we expect one set of builds
         family = self.factory.makeProcessorFamily(name="armel")
-        self.factory.makeProcessor(family=family, name="armel")
         proc_families = [family, ProcessorFamilySet().getByName("x86")]
         copy_archive, distroseries = self.makeCopyArchive(
             [package_info], proc_families=proc_families)
@@ -399,7 +397,6 @@
             copy_archive, distroseries, proc_families=proc_families)
         self.checkBuilds(copy_archive, [package_info, package_info])
 
-
     def diffArchives(self, target_archive, target_distroseries,
                      source_archive=None, source_distroseries=None):
         """Run a packageSetDiff of two archives."""
@@ -423,6 +420,7 @@
         expected_changed_tuples = [(e.name, e.version)
                                    for e in expected_changed]
         expected_new_tuples = [(e.name, e.version) for e in expected_new]
+
         def get_tuples(source_keys):
             tuples = []
             for source_key in source_keys:
@@ -503,7 +501,6 @@
             [package_infos[0]], [package_infos[1]], diff,
             distroseries.distribution.main_archive)
 
-
     def mergeCopy(self, target_archive, target_distroseries,
                   source_archive=None, source_distroseries=None):
         if source_distroseries is None:
 
=== modified file 'lib/lp/soyuz/tests/test_processor.py'
--- lib/lp/soyuz/tests/test_processor.py	2010-10-04 19:50:45 +0000
+++ lib/lp/soyuz/tests/test_processor.py	2011-07-21 19:34:45 +0000
@@ -5,13 +5,29 @@
 
 from zope.component import getUtility
 
-from canonical.testing.layers import LaunchpadZopelessLayer
+from canonical.launchpad.webapp.interfaces import (
+    DEFAULT_FLAVOR,
+    IStoreSelector,
+    MAIN_STORE,
+    )
+from canonical.launchpad.testing.pages import LaunchpadWebServiceCaller
+from canonical.testing.layers import (
+    DatabaseFunctionalLayer,
+    LaunchpadZopelessLayer,
+    )
+
 from lp.soyuz.interfaces.processor import (
     IProcessor,
     IProcessorFamily,
     IProcessorFamilySet,
-    )
-from lp.testing import TestCaseWithFactory
+    IProcessorSet,
+    ProcessorNotFound,
+    )
+from lp.testing import (
+    ExpectedException,
+    logout,
+    TestCaseWithFactory,
+    )
 
 
 class ProcessorFamilyTests(TestCaseWithFactory):
@@ -29,16 +45,83 @@
         """Test adding a new Processor to a ProcessorFamily."""
         family = getUtility(IProcessorFamilySet).new("avr", "Atmel AVR",
             "The Modified Harvard architecture 8-bit RISC processors.")
-        proc = family.addProcessor("avr2001", "The 2001 AVR", "Fast as light.")
+        proc = family.addProcessor(
+            "avr2001", "The 2001 AVR", "Fast as light.")
         self.assertProvides(proc, IProcessor)
         self.assertEquals(family, proc.family)
 
     def test_get_restricted(self):
         """Test retrieving all restricted processors."""
         family_set = getUtility(IProcessorFamilySet)
-        normal_family = getUtility(IProcessorFamilySet).new("avr", "Atmel AVR",
+        normal_family = getUtility(IProcessorFamilySet).new(
+            "avr", "Atmel AVR",
             "The Modified Harvard architecture 8-bit RISC processors.")
-        restricted_family = getUtility(IProcessorFamilySet).new("5051", "5051",
-            "Another small processor family", restricted=True)
+        restricted_family = getUtility(IProcessorFamilySet).new(
+            "5051", "5051", "Another small processor family",
+            restricted=True)
         self.assertFalse(normal_family in family_set.getRestricted())
         self.assertTrue(restricted_family in family_set.getRestricted())
+
+
+class ProcessorSetTests(TestCaseWithFactory):
+    layer = DatabaseFunctionalLayer
+
+    def test_getByName(self):
+        processor_set = getUtility(IProcessorSet)
+        q1 = self.factory.makeProcessorFamily(name='q1')
+        self.assertEquals(q1.processors[0], processor_set.getByName('q1'))
+
+    def test_getByName_not_found(self):
+        processor_set = getUtility(IProcessorSet)
+        with ExpectedException(ProcessorNotFound, 'No such processor.*'):
+            processor_set.getByName('q1')
+
+    def test_getAll(self):
+        processor_set = getUtility(IProcessorSet)
+        # Make it easy to filter out sample data
+        store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
+        store.execute("UPDATE Processor SET name = 'sample_data_' || name")
+        self.factory.makeProcessorFamily(name='q1')
+        self.factory.makeProcessorFamily(name='i686')
+        self.factory.makeProcessorFamily(name='g4')
+        self.assertEquals(
+            ['g4', 'i686', 'q1'],
+            sorted(
+            processor.name for processor in processor_set.getAll()
+            if not processor.name.startswith('sample_data_')))
+
+
+class ProcessorSetWebServiceTests(TestCaseWithFactory):
+    layer = DatabaseFunctionalLayer
+
+    def setUp(self):
+        super(ProcessorSetWebServiceTests, self).setUp()
+        self.webservice = LaunchpadWebServiceCaller()
+
+    def test_getByName(self):
+        self.factory.makeProcessorFamily(name='transmeta')
+        logout()
+
+        processor = self.webservice.named_get(
+            '/+processors', 'getByName', name='transmeta',
+            api_version='devel',
+            ).jsonBody()
+        self.assertEquals('transmeta', processor['name'])
+
+    def test_default_collection(self):
+        # Make it easy to filter out sample data
+        store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
+        store.execute("UPDATE Processor SET name = 'sample_data_' || name")
+        self.factory.makeProcessorFamily(name='q1')
+        self.factory.makeProcessorFamily(name='i686')
+        self.factory.makeProcessorFamily(name='g4')
+
+        logout()
+
+        collection = self.webservice.get(
+            '/+processors?ws.size=10', api_version='devel').jsonBody()
+        self.assertEquals(
+            ['g4', 'i686', 'q1'],
+            sorted(
+            processor['name'] for processor in collection['entries']
+            if not processor['name'].startswith('sample_data_')))
 
=== modified file 'lib/lp/testing/__init__.py'
--- lib/lp/testing/__init__.py	2011-07-18 14:07:48 +0000
+++ lib/lp/testing/__init__.py	2011-07-21 19:34:45 +0000
@@ -618,6 +618,17 @@
             self._unfoldEmailHeader(expected),
             self._unfoldEmailHeader(observed))
 
+    def assertStartsWith(self, s, prefix):
+        if not s.startswith(prefix):
+            raise AssertionError(
+                'string %r does not start with %r' % (s, prefix))
+
+    def assertEndsWith(self, s, suffix):
+        """Asserts that s ends with suffix."""
+        if not s.endswith(suffix):
+            raise AssertionError(
+                'string %r does not end with %r' % (s, suffix))
+
 
 class TestCaseWithFactory(TestCase):
 
 
=== modified file 'lib/lp/testing/factory.py'
--- lib/lp/testing/factory.py	2011-07-19 13:51:36 +0000
+++ lib/lp/testing/factory.py	2011-07-21 19:34:45 +0000
@@ -894,6 +894,9 @@
                             restricted=False):
         """Create a new processor family.
 
+        A default processor for the family will be created with the
+        same name as the family.
+
         :param name: Name of the family (e.g. x86)
         :param title: Optional title of the family
         :param description: Optional extended description
@@ -906,11 +909,11 @@
             description = "Description of the %s processor family" % name
         if title is None:
             title = "%s and compatible processors." % name
-        family = getUtility(IProcessorFamilySet).new(name, title, description,
-            restricted=restricted)
+        family = getUtility(IProcessorFamilySet).new(
+            name, title, description, restricted=restricted)
         # Make sure there's at least one processor in the family, so that
         # other things can have a default processor.
-        self.makeProcessor(family=family)
+        self.makeProcessor(name=name, family=family)
         return family
 
     def makeProductRelease(self, milestone=None, product=None,
 
=== modified file 'lib/lp/testing/publication.py'
--- lib/lp/testing/publication.py	2010-10-03 15:30:06 +0000
+++ lib/lp/testing/publication.py	2011-07-21 19:34:45 +0000
@@ -20,6 +20,7 @@
 from zope.interface import providedBy
 from zope.publisher.interfaces.browser import IDefaultSkin
 from zope.security.management import restoreInteraction
+from zope.security.proxy import removeSecurityProxy
 
 from canonical.launchpad.interfaces.launchpad import IOpenLaunchBag
 import canonical.launchpad.layers as layers
@@ -30,6 +31,7 @@
     )
 from canonical.launchpad.webapp.servers import ProtocolErrorPublication
 
+
 # Defines an helper function that returns the appropriate
 # IRequest and IPublication.
 def get_request_and_publication(host='localhost', port=None,
@@ -119,9 +121,15 @@
     getUtility(IOpenLaunchBag).clear()
     app = publication.getApplication(request)
     view = request.traverse(app)
-    # Since the last traversed object is the view, the second last should be
-    # the object that the view is on.
-    obj = request.traversed_objects[-2]
+    # Find the object from the view instead on relying that it stays
+    # in the traversed_objects stack. That doesn't apply to the web
+    # service for example.
+    try:
+        obj = removeSecurityProxy(view).context
+    except AttributeError:
+        # But sometime the view didn't store the context...
+        # Use the last traversed object in these cases.
+        obj = request.traversed_objects[-2]
 
     restoreInteraction()
 
 
=== modified file 'lib/lp/testing/tests/test_publication.py'
--- lib/lp/testing/tests/test_publication.py	2010-10-04 19:50:45 +0000
+++ lib/lp/testing/tests/test_publication.py	2011-07-21 19:34:45 +0000
@@ -23,6 +23,7 @@
 from canonical.launchpad.webapp.publisher import get_current_browser_request
 from canonical.launchpad.webapp.servers import LaunchpadTestRequest
 from canonical.testing.layers import DatabaseFunctionalLayer
+from lazr.restful import EntryResource
 from lp.testing import (
     ANONYMOUS,
     login,
@@ -45,15 +46,17 @@
         """
         # This method is completely out of control.  Thanks, Zope.
         name = '+' + self.factory.getUniqueString()
+
         class new_class(simple):
             def __init__(self, context, request):
+                self.context = context
                 view_callable()
         required = {}
         for n in ('browserDefault', '__call__', 'publishTraverse'):
             required[n] = CheckerPublic
         defineChecker(new_class, Checker(required))
         getSiteManager().registerAdapter(
-            new_class, (ILaunchpadRoot, IDefaultBrowserLayer), Interface, 
+            new_class, (ILaunchpadRoot, IDefaultBrowserLayer), Interface,
             name)
         self.addCleanup(
             getSiteManager().unregisterAdapter, new_class,
@@ -74,6 +77,7 @@
         # traversal in the sense of get_current_browser_request.
         login(ANONYMOUS)
         requests = []
+
         def record_current_request():
             requests.append(get_current_browser_request())
         context, view, request = test_traverse(
@@ -96,9 +100,18 @@
         person = self.factory.makePerson()
         login_person(person)
         users = []
+
         def record_user():
             users.append(getUtility(ILaunchBag).user)
         context, view, request = test_traverse(
             self.registerViewCallable(record_user))
         self.assertEqual(1, len(users))
         self.assertEqual(person, users[0])
+
+    def test_webservice_traverse(self):
+        login(ANONYMOUS)
+        product = self.factory.makeProduct()
+        context, view, request = test_traverse(
+            'http://api.launchpad.dev/devel/' + product.name)
+        self.assertEqual(product, context)
+        self.assertIsInstance(view, EntryResource)