Merge lp:~mithrandi/txfluiddb/object-operations into lp:~txfluiddb-maint/txfluiddb/trunk-obsolete-1.6

[Tristan Seligmann]

[Tristan Seligmann (mithrandi)]

Implement operations on objects.

[Jonathan Jacobs (jjacobs)]

  1. `Endpoint.submit`'s parameters need documentation, specifically the "data" parameter probably needs some careful and clear documentation.

  2. The `agent` parameter to `getPage` should probably include a version.

  3. There are some klaxons going off in `Object.get`. You test for the "valueType" key, and then read "valueEncoding", test it against a value and then create a blob with "valueType". It looks like "valueEncoding" was a brainfart. Also, why not use the "encoding" value (that you tested) as the parameter to `decode`? See point 5.

  4. `Object.query`'s return value is not documented.

  5. The description of the return value for `Object.getTags` is slightly confusing.

  6. There isn't any test coverage for the potential exception / errback in `Object.get`.

  7. `ObjectTests.test_getInteger` incorrectly documents the type as "integer", you probably meant "int".

  8. In "ObjectTests", `test_setTag`, `test_setBlob`, `test_createObjectWithAbout` have no docs.

  9. There are a handful of lines longer than 80 columns.

review: Needs Fixing

[Jonathan Jacobs (jjacobs)]

> parameter to `decode`? See point 5.

I meant point 6.

[lp:~mithrandi/txfluiddb/object-operations]

13. By Tristan Seligmann <tristan@viridian> on 2009-08-21

Improve Endpoint.submit documentation.

14. By Tristan Seligmann <tristan@viridian> on 2009-08-21

Document Object.query return value.

15. By Tristan Seligmann <tristan@viridian> on 2009-08-21

Fix test docstring.

16. By Tristan Seligmann <tristan@viridian> on 2009-08-22

Tests for encoding stuff.

17. By Tristan Seligmann on 2009-08-22

Test unsupported encodings.

18. By Tristan Seligmann on 2009-08-22

Split long lines.

[Tristan Seligmann (mithrandi)]

I believe I've addressed all of these points except for point 8; I'm not sure how to write a useful docstring for those tests. Any suggestions?

[Tristan Seligmann (mithrandi)]

Testing something.

review: Needs Resubmitting

[Jonathan Jacobs (jjacobs)]

> I believe I've addressed all of these points except for point 8; I'm not sure
> how to write a useful docstring for those tests. Any suggestions?

My only suggestion is the obvious one, attempt to concisely explain the desired behaviour of the code. What is supposed to happen? etc.

I noticed that several other tests are missing docstrings too.

[lp:~mithrandi/txfluiddb/object-operations]

19. By Tristan Seligmann on 2009-08-23

Docstrings.

=== modified file 'LICENSE'
--- LICENSE	2009-08-17 23:05:23 +0000
+++ LICENSE	2009-08-21 00:05:49 +0000
@@ -1,4 +1,4 @@
-Copyright (c) 2009 Tristan Seligmann
+Copyright © 2009 Tristan Seligmann
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the
 
=== modified file 'txfluiddb/client.py'
--- txfluiddb/client.py	2009-08-19 20:33:58 +0000
+++ txfluiddb/client.py	2009-08-21 00:42:26 +0000
@@ -15,7 +15,7 @@
     @type components: C{list} of C{unicode}
     @ivar components: A list of path components for this identifier.
 
-    @type collectionName: C{str}
+    @type collectionName: C{unicode}
     @cvar collectionName: The top-level path component under which these
         identifiers exist.
 
@@ -66,13 +66,19 @@
         return u'/'.join(self.components)
 
 
-    def getURL(self, endpoint, prefix=None):
+    def getURL(self, endpoint, prefix=None, suffix=[]):
         """
         Get the URL for this path as accessed through the given endpoint.
 
         @type  endpoint: L{Endpoint}
         @param endpoint: The endpoint to operate through.
 
+        @type  prefix: C{list} of C{unicode}
+        @param prefix: A list of components to prepend before this path.
+
+        @type  suffix: C{list} of C{unicode}
+        @param suffix: A list of components to append after this path.
+
         @rtype: C{str}
         """
         if self.collectionName is None:
@@ -83,12 +89,11 @@
             components.append(self.collectionName)
         else:
             components.extend(prefix)
-
-        for component in self.components:
-            encoded = component.encode('utf-8')
-            quoted = quote(encoded, safe='')
-            components.append(quoted)
-
+        components.extend(self.components)
+        components.extend(suffix)
+
+        components = [quote(component.encode('utf-8'), safe='')
+                      for component in components]
         return endpoint.getRootURL() + '/'.join(components)
 
 
@@ -97,7 +102,7 @@
     """
     Representation of a FluidDB namespace.
     """
-    collectionName = 'namespaces'
+    collectionName = u'namespaces'
 
 
     def child(self, name):
@@ -141,7 +146,7 @@
         @rtype:  C{Deferred} -> C{unicode}
         @return: The description.
         """
-        url = self.getURL(endpoint) + '?' + urlencode({'returnDescription': 'true'})
+        url = self.getURL(endpoint) + '?returnDescription=true'
         d = endpoint.submit(url=url, method='GET')
         return d.addCallback(lambda res: res[u'description'])
 
@@ -179,7 +184,7 @@
                 namespaces.append(self.child(name))
             return namespaces
 
-        url = self.getURL(endpoint) + '?' + urlencode({'returnNamespaces': 'true'})
+        url = self.getURL(endpoint) + '?returnNamespaces=true'
         d = endpoint.submit(url=url, method='GET')
         return d.addCallback(_parseResponse)
 
@@ -200,7 +205,7 @@
                 namespaces.append(self.tag(name))
             return namespaces
 
-        url = self.getURL(endpoint) + '?' + urlencode({'returnTags': 'true'})
+        url = self.getURL(endpoint) + '?returnTags=true'
         d = endpoint.submit(url=url, method='GET')
         return d.addCallback(_parseResponse)
 
@@ -273,7 +278,7 @@
     """
     Representation of a FluidDB tag.
     """
-    collectionName = 'tags'
+    collectionName = u'tags'
 
 
     def getDescription(self, endpoint):
@@ -286,7 +291,7 @@
         @rtype:  C{Deferred} -> C{unicode}
         @return: The description.
         """
-        url = self.getURL(endpoint) + '?' + urlencode({'returnDescription': 'true'})
+        url = self.getURL(endpoint) + '?returnDescription=true'
         d = endpoint.submit(url=url, method='GET')
         return d.addCallback(lambda res: res[u'description'])
 
@@ -356,12 +361,15 @@
         return getPage(*a, **kw)
 
 
-    def submit(self, url, method, data=None):
+    def submit(self, url, method, data=None, headers=None):
         """
         Submit a request through this endpoint.
         """
-        headers = {'Accept': 'application/json'}
-        if data is not None:
+        if headers is None:
+            headers = {}
+
+        headers['Accept'] = 'application/json'
+        if data is not None and not isinstance(data, str):
             data = json.dumps(data)
             headers['Content-Type'] = 'application/json'
 
@@ -369,5 +377,202 @@
             if res:
                 return json.loads(unicode(res, 'utf-8'))
 
-        d = self.getPage(url=url, method=method, postdata=data, headers=headers, agent='txFluidDB')
+        d = self.getPage(url=url,
+                         method=method,
+                         postdata=data,
+                         headers=headers,
+                         agent='txFluidDB')
         return d.addCallback(_parse)
+
+
+
+class Object(_HasPath):
+    """
+    A FluidDB object.
+
+    @type uuid: C{unicode}
+    @ivar uuid: The UUID of the object.
+    """
+    collectionName = u'objects'
+
+
+    def __init__(self, uuid):
+        self.uuid = uuid
+
+
+    @property
+    def components(self):
+        """
+        Our only path component is the object UUID.
+        """
+        return [self.uuid]
+
+
+    @classmethod
+    def create(cls, endpoint, about=None):
+        """
+        Create a new object.
+
+        @type  endpoint: L{Endpoint}
+        @param endpoint: The endpoint to operate through.
+
+        @type  about: C{unicode} or C{None}
+        @param about: The value for the about tag, if desired.
+
+        @rtype:  C{Deferred} -> L{Object}
+        @return: The newly created object.
+        """
+        url = endpoint.getRootURL() + 'objects'
+        data = {}
+        if about is not None:
+            data[u'about'] = about
+
+        def _parseResponse(response):
+            return Object(response[u'id'])
+
+        d = endpoint.submit(url=url, method='POST', data=data)
+        return d.addCallback(_parseResponse)
+
+
+    @classmethod
+    def query(cls, endpoint, query):
+        """
+        Search for objects that match a query.
+
+        @type  endpoint: L{Endpoint}
+        @param endpoint: The endpoint to operate through.
+
+        @type  query: C{unicode}
+        @param query: A query string to search on.
+        """
+        qs = '?' + urlencode({'query': query.encode('utf-8')})
+        url = endpoint.getRootURL() + 'objects' + qs
+
+        def _parseResponse(response):
+            return [Object(uuid) for uuid in response['ids']]
+
+        d = endpoint.submit(url=url, method='GET')
+        return d.addCallback(_parseResponse)
+
+
+    def getTags(self, endpoint):
+        """
+        Get the visible tags on this object.
+
+        @type  endpoint: L{Endpoint}
+        @param endpoint: The endpoint to operate through.
+
+        @rtype:  C{Deferred} -> (C{unicode}, C{list} of L{Tag})
+        @return: A tuple of the about tag value and the list of tags.
+        """
+        def _parseResponse(response):
+            about = response[u'about']
+            tags = [Tag(*path.split(u'/')) for path in response[u'tagPaths']]
+            return (about, tags)
+
+        url = self.getURL(endpoint)
+        d = endpoint.submit(url=url,
+                            method='GET',
+                            data={u'showAbout': True})
+        return d.addCallback(_parseResponse)
+
+
+    def get(self, endpoint, tag):
+        """
+        Get the value of a tag on this object.
+
+        @type  endpoint: L{Endpoint}
+        @param endpoint: The endpoint to operate through.
+
+        @type  tag: L{Tag}
+        @param tag: The tag to retrieve.
+
+        @rtype:  Varies depending on what value is stored.
+        @return: The stored value.
+        """
+        def _parseResponse(response):
+            value = response[u'value']
+            if u'valueType' in response:
+                encoding = response[u'valueEncoding']
+                if encoding != u'base-64':
+                    raise ValueError('Unsupported encoding received: %r' % encoding)
+                value = value.decode('base-64')
+                return Blob(response[u'valueType'], value)
+            return value
+
+        url = self.getURL(endpoint, suffix=tag.components) + '?format=json'
+        d = endpoint.submit(url=url,
+                            method='GET',
+                            headers={'Accept-Encoding': 'base-64'})
+        return d.addCallback(_parseResponse)
+
+
+    def set(self, endpoint, tag, value):
+        """
+        Set the value of a tag on this object.
+
+        @type  endpoint: L{Endpoint}
+        @param endpoint: The endpoint to operate through.
+
+        @type  tag: L{Tag}
+        @param tag: The tag to set or replace.
+
+        @type  value: Any JSON-encodable value
+        @param value: The value to store.
+        """
+        url = self.getURL(endpoint, suffix=tag.components) + '?format=json'
+        return endpoint.submit(url=url,
+                               method='PUT',
+                               data={u'value': value})
+
+
+    def setBlob(self, endpoint, tag, value):
+        """
+        Set the value of a tag on this object to a blob.
+
+        @type  endpoint: L{Endpoint}
+        @param endpoint: The endpoint to operate through.
+
+        @type  tag: L{Tag}
+        @param tag: The tag to set or replace.
+
+        @type  value: L{Blob}
+        @param value: The value to store.
+        """
+        url = self.getURL(endpoint, suffix=tag.components)
+        return endpoint.submit(url=url,
+                               method='PUT',
+                               headers={'Content-Type': value.contentType},
+                               data=value.data)
+
+
+    def delete(self, endpoint, tag):
+        """
+        Delete a tag from this object.
+
+        Note that FluidDB does not support deleting objects themselves.
+
+        @type  endpoint: L{Endpoint}
+        @param endpoint: The endpoint to operate through.
+
+        @type  tag: L{Tag}
+        @param tag: The tag to retrieve.
+        """
+        url = self.getURL(endpoint, suffix=tag.components)
+        return endpoint.submit(url=url, method='DELETE')
+
+
+
+class Blob(object):
+    """
+    A binary blob with a MIME content type.
+
+    @type contentType: C{unicode}
+    @ivar contentType: The MIME content-type of the data.
+
+    @type data: C{str}
+    @ivar data: The actual data.
+    """
+    def __init__(self, contentType, data):
+        self.contentType = contentType
+        self.data = data
 
=== modified file 'txfluiddb/test/test_client.py'
--- txfluiddb/test/test_client.py	2009-08-19 20:33:58 +0000
+++ txfluiddb/test/test_client.py	2009-08-21 00:42:26 +0000
@@ -4,7 +4,7 @@
 import simplejson as json
 
 from txfluiddb.errors import InvalidName
-from txfluiddb.client import Namespace, Tag, Endpoint, _HasPath
+from txfluiddb.client import Namespace, Tag, Endpoint, _HasPath, Object, Blob
 
 
 
@@ -44,6 +44,28 @@
                          'http://fluiddb.test.url/tests/foo')
 
 
+    def test_getURLPrefix(self):
+        """
+        Passing prefix to getURL prepends the given components to the path.
+        """
+        endpoint = Endpoint('http://fluiddb.test.url/')
+        path = _HasPath(u'foo')
+        path.collectionName = 'tests'
+        self.assertEqual(path.getURL(endpoint, prefix=[u'quux', u'42']),
+                         'http://fluiddb.test.url/quux/42/foo')
+
+
+    def test_getURLSuffix(self):
+        """
+        Passing suffix to getURL appends the given components to the path.
+        """
+        endpoint = Endpoint('http://fluiddb.test.url/')
+        path = _HasPath(u'foo')
+        path.collectionName = 'tests'
+        self.assertEqual(path.getURL(endpoint, suffix=[u'quux', u'42']),
+                         'http://fluiddb.test.url/tests/foo/quux/42')
+
+
     def test_urlEncoding(self):
         """
         URL encoding is done according to IRI rules.
@@ -374,3 +396,199 @@
         endpoint = Endpoint('http://fluiddb.test.url/', '20090817')
         self.assertEqual(endpoint.getRootURL(),
                          'http://fluiddb.test.url/20090817/')
+
+
+
+class ObjectTests(TestCase):
+    """
+    Tests for L{Object}.
+    """
+    def setUp(self):
+        self.endpoint = MockEndpoint('http://fluiddb.test.url/')
+
+
+    def test_getTags(self):
+        """
+        The getTags method returns the about value and the list of tags.
+        """
+        response = {u'about': u'Testing object',
+                    u'tagPaths': [u'test/tag1', u'test/tag2']}
+        self.endpoint.response = json.dumps(response)
+
+        def _gotResponse(response):
+            about, tags = response
+            self.assertEqual(about, u'Testing object')
+            self.assertEqual(len(tags), 2)
+            tag1, tag2 = tags
+            self.assertIsInstance(tag1, Tag)
+            self.assertEqual(tag1.getPath(), u'test/tag1')
+            self.assertIsInstance(tag2, Tag)
+            self.assertEqual(tag2.getPath(), u'test/tag2')
+
+        obj = Object(u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx')
+        d = obj.getTags(self.endpoint).addCallback(_gotResponse)
+        self.assertEqual(self.endpoint.method, 'GET')
+        self.assertEqual(
+            self.endpoint.url,
+            'http://fluiddb.test.url/objects/xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx')
+        self.assertEqual(json.loads(self.endpoint.data),
+                         {u'showAbout': True})
+        return d
+
+
+    def makeGetRequest(self, response):
+        """
+        Utility wrapper for testing L{Object.get} operations.
+        """
+        self.endpoint.response = json.dumps(response)
+        obj = Object(u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx')
+        tag = Namespace(u'test').child(u'tag')
+        d = obj.get(self.endpoint, tag)
+        self.assertEqual(self.endpoint.method, 'GET')
+        self.assertEqual(self.endpoint.data, None)
+        self.assertEqual(
+            self.endpoint.url,
+            'http://fluiddb.test.url/objects/xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx/test/tag?format=json')
+        return d
+
+
+    def test_getString(self):
+        """
+        Retrieving a JSON string value results in a C{unicode} object.
+        """
+        response = {u'value': u'Data goes where?'}
+        def _gotResponse(response):
+            self.assertIsInstance(response, unicode)
+            self.assertEqual(response, u'Data goes where?')
+        return self.makeGetRequest(response).addCallback(_gotResponse)
+
+
+    def test_getInteger(self):
+        """
+        Retrieving a JSON integer value results in an C{integer} object.
+        """
+        response = {u'value': 42}
+        def _gotResponse(response):
+            self.assertEqual(response, 42)
+        return self.makeGetRequest(response).addCallback(_gotResponse)
+
+
+    def test_getBlob(self):
+        """
+        Retrieving a binary value results in a C{Blob} object.
+        """
+        response = {u'value': u'PHA+Zm9vPC9wPg==',
+                    u'valueEncoding': u'base-64',
+                    u'valueType': u'text/html'}
+        def _gotResponse(response):
+            self.assertEqual(self.endpoint.headers['Accept-Encoding'], 'base-64')
+            self.assertEqual(response.contentType, u'text/html')
+            self.assertEqual(response.data, '<p>foo</p>')
+        return self.makeGetRequest(response).addCallback(_gotResponse)
+
+
+    def test_deleteTag(self):
+        """
+        Deleting a tag returns nothing if successful.
+        """
+        self.endpoint.response = ''
+
+        obj = Object(u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx')
+        tag = Namespace(u'test').child(u'tag')
+        d = obj.delete(self.endpoint, tag)
+        self.assertEqual(self.endpoint.method, 'DELETE')
+        self.assertEqual(
+            self.endpoint.url,
+            'http://fluiddb.test.url/objects/xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx/test/tag')
+        self.assertEqual(self.endpoint.data, None)
+        return d
+
+
+    def test_setTag(self):
+        self.endpoint.response = ''
+
+        obj = Object(u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx')
+        tag = Namespace(u'test').child(u'tag')
+        d = obj.set(self.endpoint, tag, 42)
+        self.assertEqual(self.endpoint.method, 'PUT')
+        self.assertEqual(
+            self.endpoint.url,
+            'http://fluiddb.test.url/objects/xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx/test/tag?format=json')
+        self.assertEqual(json.loads(self.endpoint.data),
+                         {u'value': 42})
+        return d
+
+
+    def test_setBlob(self):
+        self.endpoint.response = ''
+
+        obj = Object(u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx')
+        tag = Namespace(u'test').child(u'tag')
+        blob = Blob(u'text/html', '<p>foo</p>')
+        d = obj.setBlob(self.endpoint, tag, blob)
+        self.assertEqual(self.endpoint.method, 'PUT')
+        self.assertEqual(
+            self.endpoint.url,
+            'http://fluiddb.test.url/objects/xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx/test/tag')
+        self.assertEqual(self.endpoint.data, '<p>foo</p>')
+        self.assertEqual(self.endpoint.headers['Content-Type'], 'text/html')
+        return d
+
+
+    def test_createObject(self):
+        """
+        Creating an object returns the newly created object.
+        """
+        response = {u'URI': u'http://fluiddb.test.url/objects/xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx',
+                    u'id': u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'}
+        self.endpoint.response = json.dumps(response)
+
+        def _gotResponse(obj):
+            self.assertIsInstance(obj, Object)
+            self.assertEqual(obj.uuid, u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx')
+
+        d = Object.create(self.endpoint).addCallback(_gotResponse)
+        self.assertEqual(self.endpoint.method, 'POST')
+        self.assertEqual(self.endpoint.url, 'http://fluiddb.test.url/objects')
+        self.assertEqual(json.loads(self.endpoint.data), {})
+        return d
+
+
+    def test_createObjectWithAbout(self):
+        response = {u'URI': u'http://fluiddb.test.url/objects/xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx',
+                    u'id': u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'}
+        self.endpoint.response = json.dumps(response)
+
+        def _gotResponse(obj):
+            self.assertIsInstance(obj, Object)
+            self.assertEqual(obj.uuid, u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx')
+
+        d = Object.create(self.endpoint, u'New object').addCallback(_gotResponse)
+        self.assertEqual(self.endpoint.method, 'POST')
+        self.assertEqual(self.endpoint.url, 'http://fluiddb.test.url/objects')
+        self.assertEqual(json.loads(self.endpoint.data),
+                         {u'about': u'New object'})
+        return d
+
+
+    def test_query(self):
+        """
+        Querying returns all objects matching the query.
+        """
+        response = {u'ids': [u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxx1',
+                             u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxx2']}
+        self.endpoint.response = json.dumps(response)
+
+        def _gotResponse(objs):
+            self.assertEqual(len(objs), 2)
+            obj1, obj2 = objs
+            self.assertIsInstance(obj1, Object)
+            self.assertEqual(obj1.uuid, u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxx1')
+            self.assertIsInstance(obj2, Object)
+            self.assertEqual(obj2.uuid, u'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxx2')
+
+        d = Object.query(self.endpoint, u'has fluiddb/about').addCallback(_gotResponse)
+        self.assertEqual(self.endpoint.method, 'GET')
+        self.assertEqual(self.endpoint.url, 'http://fluiddb.test.url/objects?query=has+fluiddb%2Fabout')
+        self.assertEqual(self.endpoint.data, None)
+        return d