Merge lp:~oubiwann/txaws/484858-s3-scripts into lp:txaws

[Duncan McGreggor]

[Duncan McGreggor (oubiwann)]

This branch depends upon two other branches:
 * 474353-storage-ec2-symmetry
 * 475571-s3-error-wrapper

These scripts serve dual purpose:
 1. convenience for users wanting to perform actions against S3-compliant storage servers, and
 2. examples for how to use the API.

For the latter purpose, code in the scripts has not been abstracted out for maximal reuse. Instead, each script presents a complete picture to an API user on how to use the given method to accomplish a common task. This is done at the cost of code redundancy.

Note that the DELETE operations currently return an exit code of 1 when it should be 0. This will be addressed by bug #486363 (Successful delete operations return a 204 No Content "error").

[Duncan McGreggor (oubiwann)]

Update:

bug #486363 (Successful delete operations return a 204 No Content "error") is currently in review.

[Duncan McGreggor (oubiwann)]

Landscape trunk has been tested against this branch.

[Duncan McGreggor (oubiwann)]

Robert made this comment in the merge proposal of a child branch:

> There is a bunch of duplication in the example scripts. I'd like to see that removed. Perhaps:
> - give them a if __name__ == guard
> - move the error/return etc callback support into script.py

There is a bunch of duplication... for now, that is intentional, as it offers an example of API usage: all a potential user/developer has to do is read the bin scripts to see how to use the API.

I did think about moving redundant code into a txaws.script module, but I chose in favor of clarity, sacrificing with redundancy.

If you don't this is a valid concern, I'm willing to be convinced :-)

review: Approve

=== modified file 'LICENSE'
--- LICENSE	2008-07-06 22:51:54 +0000
+++ LICENSE	2009-11-28 01:10:25 +0000
@@ -1,3 +1,8 @@
+Copyright (C) 2008 Tristan Seligmann <[e-mail]>
+Copyright (C) 2009 Robert Collins <[e-mail]>
+Copyright (C) 2009 Canonical Ltd
+Copyright (C) 2009 Duncan McGreggor <[e-mail]>
+
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the
 "Software"), to deal in the Software without restriction, including
@@ -11,8 +16,8 @@
 
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
-LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
-OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
-WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

=== modified file 'README'
--- README	2009-08-19 20:55:49 +0000
+++ README	2009-11-28 01:10:25 +0000
@@ -14,3 +14,9 @@
 * The txaws python package. (No installer at the moment)
 
 * bin/aws-status, a GUI status program for aws resources.
+
+License
+-------
+
+txAWS is open source software, MIT License. See the LICENSE file for more
+details.

=== added file 'bin/txaws-create-bucket'
--- bin/txaws-create-bucket	1970-01-01 00:00:00 +0000
+++ bin/txaws-create-bucket	2009-11-28 01:10:25 +0000
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+"""
+%prog [options]
+"""
+
+import sys
+
+from txaws.credentials import AWSCredentials
+from txaws.script import parse_options
+from txaws.service import AWSServiceRegion
+from txaws.util import reactor
+
+
+def printResults(results):
+    return 0
+
+
+def printError(error):
+    print error.value
+    return 1
+
+
+def finish(return_code):
+    reactor.stop(exitStatus=return_code)
+
+
+options, args = parse_options(__doc__.strip())
+if options.bucket is None:
+    print "Error Message: A bucket name is required."
+    sys.exit(1)
+creds = AWSCredentials(options.access_key, options.secret_key)
+region = AWSServiceRegion(
+    creds=creds, region=options.region, s3_endpoint=options.url)
+client = region.get_s3_client()
+
+d = client.create_bucket(options.bucket)
+d.addCallback(printResults)
+d.addErrback(printError)
+d.addCallback(finish)
+# We use a custom reactor so that we can return the exit status from
+# reactor.run().
+sys.exit(reactor.run())

=== added file 'bin/txaws-delete-bucket'
--- bin/txaws-delete-bucket	1970-01-01 00:00:00 +0000
+++ bin/txaws-delete-bucket	2009-11-28 01:10:25 +0000
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+"""
+%prog [options]
+"""
+
+import sys
+
+from txaws.credentials import AWSCredentials
+from txaws.script import parse_options
+from txaws.service import AWSServiceRegion
+from txaws.util import reactor
+
+
+def printResults(results):
+    return 0
+
+
+def printError(error):
+    print error.value
+    return 1
+
+
+def finish(return_code):
+    reactor.stop(exitStatus=return_code)
+
+
+options, args = parse_options(__doc__.strip())
+if options.bucket is None:
+    print "Error Message: A bucket name is required."
+    sys.exit(1)
+creds = AWSCredentials(options.access_key, options.secret_key)
+region = AWSServiceRegion(
+    creds=creds, region=options.region, s3_endpoint=options.url)
+client = region.get_s3_client()
+
+d = client.delete_bucket(options.bucket)
+d.addCallback(printResults)
+d.addErrback(printError)
+d.addCallback(finish)
+# We use a custom reactor so that we can return the exit status from
+# reactor.run().
+sys.exit(reactor.run())

=== added file 'bin/txaws-delete-object'
--- bin/txaws-delete-object	1970-01-01 00:00:00 +0000
+++ bin/txaws-delete-object	2009-11-28 01:10:25 +0000
@@ -0,0 +1,46 @@
+#!/usr/bin/env python
+"""
+%prog [options]
+"""
+
+import sys
+
+from txaws.credentials import AWSCredentials
+from txaws.script import parse_options
+from txaws.service import AWSServiceRegion
+from txaws.util import reactor
+
+
+def printResults(results):
+    print results
+    return 0
+
+
+def printError(error):
+    print error.value
+    return 1
+
+
+def finish(return_code):
+    reactor.stop(exitStatus=return_code)
+
+
+options, args = parse_options(__doc__.strip())
+if options.bucket is None:
+    print "Error Message: A bucket name is required."
+    sys.exit(1)
+if options.object_name is None:
+    print "Error Message: An object name is required."
+    sys.exit(1)
+creds = AWSCredentials(options.access_key, options.secret_key)
+region = AWSServiceRegion(
+    creds=creds, region=options.region, s3_endpoint=options.url)
+client = region.get_s3_client()
+
+d = client.delete_object(options.bucket, options.object_name)
+d.addCallback(printResults)
+d.addErrback(printError)
+d.addCallback(finish)
+# We use a custom reactor so that we can return the exit status from
+# reactor.run().
+sys.exit(reactor.run())

=== added file 'bin/txaws-get-object'
--- bin/txaws-get-object	1970-01-01 00:00:00 +0000
+++ bin/txaws-get-object	2009-11-28 01:10:25 +0000
@@ -0,0 +1,46 @@
+#!/usr/bin/env python
+"""
+%prog [options]
+"""
+
+import sys
+
+from txaws.credentials import AWSCredentials
+from txaws.script import parse_options
+from txaws.service import AWSServiceRegion
+from txaws.util import reactor
+
+
+def printResults(results):
+    print results
+    return 0
+
+
+def printError(error):
+    print error.value
+    return 1
+
+
+def finish(return_code):
+    reactor.stop(exitStatus=return_code)
+
+
+options, args = parse_options(__doc__.strip())
+if options.bucket is None:
+    print "Error Message: A bucket name is required."
+    sys.exit(1)
+if options.object_name is None:
+    print "Error Message: An object name is required."
+    sys.exit(1)
+creds = AWSCredentials(options.access_key, options.secret_key)
+region = AWSServiceRegion(
+    creds=creds, region=options.region, s3_endpoint=options.url)
+client = region.get_s3_client()
+
+d = client.get_object(options.bucket, options.object_name)
+d.addCallback(printResults)
+d.addErrback(printError)
+d.addCallback(finish)
+# We use a custom reactor so that we can return the exit status from
+# reactor.run().
+sys.exit(reactor.run())

=== added file 'bin/txaws-head-object'
--- bin/txaws-head-object	1970-01-01 00:00:00 +0000
+++ bin/txaws-head-object	2009-11-28 01:10:25 +0000
@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+"""
+%prog [options]
+"""
+
+import sys
+from pprint import pprint
+
+from txaws.credentials import AWSCredentials
+from txaws.script import parse_options
+from txaws.service import AWSServiceRegion
+from txaws.util import reactor
+
+
+def printResults(results):
+    pprint(results)
+    return 0
+
+
+def printError(error):
+    print error.value
+    return 1
+
+
+def finish(return_code):
+    reactor.stop(exitStatus=return_code)
+
+
+options, args = parse_options(__doc__.strip())
+if options.bucket is None:
+    print "Error Message: A bucket name is required."
+    sys.exit(1)
+if options.object_name is None:
+    print "Error Message: An object name is required."
+    sys.exit(1)
+creds = AWSCredentials(options.access_key, options.secret_key)
+region = AWSServiceRegion(
+    creds=creds, region=options.region, s3_endpoint=options.url)
+client = region.get_s3_client()
+
+d = client.head_object(options.bucket, options.object_name)
+d.addCallback(printResults)
+d.addErrback(printError)
+d.addCallback(finish)
+# We use a custom reactor so that we can return the exit status from
+# reactor.run().
+sys.exit(reactor.run())

=== added file 'bin/txaws-list-buckets'
--- bin/txaws-list-buckets	1970-01-01 00:00:00 +0000
+++ bin/txaws-list-buckets	2009-11-28 01:10:25 +0000
@@ -0,0 +1,43 @@
+#!/usr/bin/env python
+"""
+%prog [options]
+"""
+
+import sys
+
+from txaws.credentials import AWSCredentials
+from txaws.script import parse_options
+from txaws.service import AWSServiceRegion
+from txaws.util import reactor
+
+
+def printResults(results):
+    print "\nBuckets:"
+    for bucket in results:
+        print "\t%s (created on %s)" % (bucket.name, bucket.creation_date)
+    print "Total buckets: %s\n" % len(list(results))
+    return 0
+
+
+def printError(error):
+    print error.value
+    return 1
+
+
+def finish(return_code):
+    reactor.stop(exitStatus=return_code)
+
+
+options, args = parse_options(__doc__.strip())
+creds = AWSCredentials(options.access_key, options.secret_key)
+region = AWSServiceRegion(
+    creds=creds, region=options.region, s3_endpoint=options.url)
+client = region.get_s3_client()
+
+d = client.list_buckets()
+d.addCallback(printResults)
+d.addErrback(printError)
+d.addCallback(finish)
+# We use a custom reactor so that we can return the exit status from
+# reactor.run().
+sys.exit(reactor.run())

=== added file 'bin/txaws-put-object'
--- bin/txaws-put-object	1970-01-01 00:00:00 +0000
+++ bin/txaws-put-object	2009-11-28 01:10:25 +0000
@@ -0,0 +1,56 @@
+#!/usr/bin/env python
+"""
+%prog [options]
+"""
+
+import os
+import sys
+
+from txaws.credentials import AWSCredentials
+from txaws.script import parse_options
+from txaws.service import AWSServiceRegion
+from txaws.util import reactor
+
+
+def printResults(results):
+    return 0
+
+
+def printError(error):
+    print error.value
+    return 1
+
+
+def finish(return_code):
+    reactor.stop(exitStatus=return_code)
+
+
+options, args = parse_options(__doc__.strip())
+if options.bucket is None:
+    print "Error Message: A bucket name is required."
+    sys.exit(1)
+filename = options.object_filename
+if filename:
+    options.object_name = os.path.basename(filename)
+    try:
+        options.object_data = open(filename).read()
+    except Exception, error:
+        print error
+        sys.exit(1)
+elif options.object_name is None:
+    print "Error Message: An object name is required."
+    sys.exit(1)
+creds = AWSCredentials(options.access_key, options.secret_key)
+region = AWSServiceRegion(
+    creds=creds, region=options.region, s3_endpoint=options.url)
+client = region.get_s3_client()
+
+d = client.put_object(
+    options.bucket, options.object_name, options.object_data,
+    options.content_type)
+d.addCallback(printResults)
+d.addErrback(printError)
+d.addCallback(finish)
+# We use a custom reactor so that we can return the exit status from
+# reactor.run().
+sys.exit(reactor.run())

=== modified file 'txaws/client/base.py'
--- txaws/client/base.py	2009-11-28 01:10:25 +0000
+++ txaws/client/base.py	2009-11-28 01:10:26 +0000
@@ -1,11 +1,50 @@
+from xml.parsers.expat import ExpatError
+
 from twisted.internet import reactor, ssl
+from twisted.web import http
 from twisted.web.client import HTTPClientFactory
+from twisted.web.error import Error as TwistedWebError
 
 from txaws.util import parse
 from txaws.credentials import AWSCredentials
+from txaws.exception import AWSResponseParseError
 from txaws.service import AWSServiceEndpoint
 
 
+def error_wrapper(error, errorClass):
+    """
+    We want to see all error messages from cloud services. Amazon's EC2 says
+    that their errors are accompanied either by a 400-series or 500-series HTTP
+    response code. As such, the first thing we want to do is check to see if
+    the error is in that range. If it is, we then need to see if the error
+    message is an EC2 one.
+
+    In the event that an error is not a Twisted web error nor an EC2 one, the
+    original exception is raised.
+    """
+    http_status = 0
+    if error.check(TwistedWebError):
+        xml_payload = error.value.response
+        if error.value.status:
+            http_status = int(error.value.status)
+    else:
+        error.raiseException()
+    if http_status >= 400:
+        if not xml_payload:
+            error.raiseException()
+        try:
+            fallback_error = errorClass(
+                xml_payload, error.value.status, error.value.message,
+                error.value.response)
+        except (ExpatError, AWSResponseParseError):
+            error_message = http.RESPONSES.get(http_status)
+            fallback_error = TwistedWebError(
+                http_status, error_message, error.value.response)
+        raise fallback_error
+    else:
+        error.raiseException()
+
+
 class BaseClient(object):
     """Create an AWS client.
 

=== modified file 'txaws/ec2/client.py'
--- txaws/ec2/client.py	2009-11-28 01:10:25 +0000
+++ txaws/ec2/client.py	2009-11-28 01:10:26 +0000
@@ -8,16 +8,11 @@
 from datetime import datetime
 from urllib import quote
 from base64 import b64encode
-from xml.parsers.expat import ExpatError
-
-from twisted.web import http
-from twisted.web.error import Error as TwistedWebError
 
 from txaws import version
-from txaws.client.base import BaseClient, BaseQuery
+from txaws.client.base import BaseClient, BaseQuery, error_wrapper
 from txaws.ec2 import model
 from txaws.ec2.exception import EC2Error
-from txaws.exception import AWSResponseParseError
 from txaws.util import iso8601time, XML
 
 
@@ -25,34 +20,7 @@
 
 
 def ec2_error_wrapper(error):
-    """
-    We want to see all error messages from cloud services. Amazon's EC2 says
-    that their errors are accompanied either by a 400-series or 500-series HTTP
-    response code. As such, the first thing we want to do is check to see if
-    the error is in that range. If it is, we then need to see if the error
-    message is an EC2 one.
-
-    In the event that an error is not a Twisted web error nor an EC2 one, the
-    original exception is raised.
-    """
-    http_status = 0
-    if error.check(TwistedWebError):
-        xml_payload = error.value.response
-        if error.value.status:
-            http_status = int(error.value.status)
-    else:
-        error.raiseException()
-    if http_status >= 400:
-        try:
-            fallback_error = EC2Error(xml_payload, error.value.status,
-                                 error.value.message, error.value.response)
-        except (ExpatError, AWSResponseParseError):
-            error_message = http.RESPONSES.get(http_status)
-            fallback_error = TwistedWebError(http_status, error_message,
-                                        error.value.response)
-        raise fallback_error
-    else:
-        error.raiseException()
+    error_wrapper(error, EC2Error)
 
 
 class EC2Client(BaseClient):
@@ -848,5 +816,4 @@
         url = "%s?%s" % (self.endpoint.get_uri(),
                          self.get_canonical_query_params())
         d = self.get_page(url, method=self.endpoint.method)
-        d.addErrback(ec2_error_wrapper)
-        return d
+        return d.addErrback(ec2_error_wrapper)

=== modified file 'txaws/ec2/exception.py'
--- txaws/ec2/exception.py	2009-11-28 01:10:25 +0000
+++ txaws/ec2/exception.py	2009-11-28 01:10:26 +0000
@@ -1,39 +1,14 @@
 # Copyright (c) 2009 Canonical Ltd <[e-mail]>
 # Licenced under the txaws licence available at /LICENSE in the txaws source.
 
-from txaws.exception import AWSError, AWSResponseParseError
-from txaws.util import XML
+from txaws.exception import AWSError
 
 
 class EC2Error(AWSError):
     """
     A error class providing custom methods on EC2 errors.
     """
-    def __init__(self, xml_bytes, status=None, message=None, response=None):
-        super(AWSError, self).__init__(status, message, response)
-        if not xml_bytes:
-            raise ValueError("XML cannot be empty.")
-        self.original = xml_bytes
-        self.errors = []
-        self.request_id = ""
-        self.host_id = ""
-        self.parse()
-
-    def __str__(self):
-        return self._get_error_message_string()
-
-    def __repr__(self):
-        return "<%s object with %s>" % (
-            self.__class__.__name__, self._get_error_code_string())
-
-    def _set_request_id(self, tree):
-        request_id_node = tree.find(".//RequestID")
-        if hasattr(request_id_node, "text"):
-            text = request_id_node.text
-            if text:
-                self.request_id = text
-
-    def _set_400_errors(self, tree):
+    def _set_400_error(self, tree):
         errors_node = tree.find(".//Errors")
         if errors_node:
             for error in errors_node:
@@ -41,84 +16,5 @@
                 if data:
                     self.errors.append(data)
 
-    def _set_host_id(self, tree):
-        host_id = tree.find(".//HostID")
-        if hasattr(host_id, "text"):
-            text = host_id.text
-            if text:
-                self.host_id = text
-
-    def _set_500_error(self, tree):
-        self._set_request_id(tree)
-        self._set_host_id(tree)
-        data = self._node_to_dict(tree)
-        if data:
-            self.errors.append(data)
-
-    def _get_error_code_string(self):
-        count = len(self.errors)
-        error_code = self.get_error_codes()
-        if count > 1:
-            return "Error count: %s" % error_code
-        else:
-            return "Error code: %s" % error_code
-
-    def _get_error_message_string(self):
-        count = len(self.errors)
-        error_message = self.get_error_messages()
-        if count > 1:
-            return "%s." % error_message
-        else:
-            return "Error Message: %s" % error_message
-
-    def _node_to_dict(self, node):
-        data = {}
-        for child in node:
-            if child.tag and child.text:
-                data[child.tag] = child.text
-        return data
-
-    def _check_for_html(self, tree):
-        if tree.tag == "html":
-            message = "Could not parse HTML in the response."
-            raise AWSResponseParseError(message)
-
-    def parse(self, xml_bytes=""):
-        if not xml_bytes:
-            xml_bytes = self.original
-        self.original = xml_bytes
-        tree = XML(xml_bytes.strip())
-        self._check_for_html(tree)
-        self._set_request_id(tree)
-        if self.status:
-            status = int(self.status)
-        else:
-            status = 400
-        if status >= 500:
-            self._set_500_error(tree)
-        else:
-            self._set_400_errors(tree)
-
-    def has_error(self, errorString):
-        for error in self.errors:
-            if errorString in error.values():
-                return True
-        return False
-
-    def get_error_codes(self):
-        count = len(self.errors)
-        if count > 1:
-            return count
-        elif count == 0:
-            return
-        else:
-            return self.errors[0]["Code"]
-
-    def get_error_messages(self):
-        count = len(self.errors)
-        if count > 1:
-            return "Multiple EC2 Errors"
-        elif count == 0:
-            return "Empty error list"
-        else:
-            return self.errors[0]["Message"]
+
+

=== modified file 'txaws/ec2/tests/test_exception.py'
--- txaws/ec2/tests/test_exception.py	2009-11-28 01:10:25 +0000
+++ txaws/ec2/tests/test_exception.py	2009-11-28 01:10:26 +0000
@@ -4,7 +4,6 @@
 from twisted.trial.unittest import TestCase
 
 from txaws.ec2.exception import EC2Error
-from txaws.exception import AWSResponseParseError
 from txaws.testing import payload
 from txaws.util import XML
 
@@ -14,77 +13,14 @@
 
 class EC2ErrorTestCase(TestCase):
 
-    def test_creation(self):
-        error = EC2Error("<dummy1 />", 400, "Not Found", "<dummy2 />")
-        self.assertEquals(error.status, 400)
-        self.assertEquals(error.response, "<dummy2 />")
-        self.assertEquals(error.original, "<dummy1 />")
-        self.assertEquals(error.errors, [])
-        self.assertEquals(error.request_id, "")
-
-    def test_node_to_dict(self):
-        xml = "<parent><child1>text1</child1><child2>text2</child2></parent>"
-        error = EC2Error("<dummy />")
-        data = error._node_to_dict(XML(xml))
-        self.assertEquals(data, {"child1": "text1", "child2": "text2"})
-
-    def test_set_request_id(self):
-        xml = "<a><b /><RequestID>%s</RequestID></a>" % REQUEST_ID
-        error = EC2Error("<dummy />")
-        error._set_request_id(XML(xml))
-        self.assertEquals(error.request_id, REQUEST_ID)
-
-    def test_set_400_errors(self):
+    def test_set_400_error(self):
         errorsXML = "<Error><Code>1</Code><Message>2</Message></Error>"
         xml = "<a><Errors>%s</Errors><b /></a>" % errorsXML
         error = EC2Error("<dummy />")
-        error._set_400_errors(XML(xml))
+        error._set_400_error(XML(xml))
         self.assertEquals(error.errors[0]["Code"], "1")
         self.assertEquals(error.errors[0]["Message"], "2")
 
-    def test_set_host_id(self):
-        host_id = "ASD@#FDG$E%FG"
-        xml = "<a><b /><HostID>%s</HostID></a>" % host_id
-        error = EC2Error("<dummy />")
-        error._set_host_id(XML(xml))
-        self.assertEquals(error.host_id, host_id)
-
-    def test_set_500_error(self):
-        xml = "<Error><Code>500</Code><Message>Oops</Message></Error>"
-        error = EC2Error("<dummy />")
-        error._set_500_error(XML(xml))
-        self.assertEquals(error.errors[0]["Code"], "500")
-        self.assertEquals(error.errors[0]["Message"], "Oops")
-
-    def test_set_empty_errors(self):
-        xml = "<a><Errors /><b /></a>"
-        error = EC2Error("<dummy />")
-        error._set_400_errors(XML(xml))
-        self.assertEquals(error.errors, [])
-
-    def test_set_empty_error(self):
-        xml = "<a><Errors><Error /><Error /></Errors><b /></a>"
-        error = EC2Error("<dummy />")
-        error._set_400_errors(XML(xml))
-        self.assertEquals(error.errors, [])
-
-    def test_parse_without_xml(self):
-        xml = "<dummy />"
-        error = EC2Error(xml)
-        error.parse()
-        self.assertEquals(error.original, xml)
-
-    def test_parse_with_xml(self):
-        xml1 = "<dummy1 />"
-        xml2 = "<dummy2 />"
-        error = EC2Error(xml1)
-        error.parse(xml2)
-        self.assertEquals(error.original, xml2)
-
-    def test_parse_html(self):
-        xml = "<html><body>a page</body></html>"
-        self.assertRaises(AWSResponseParseError, EC2Error, xml)
-
     def test_has_error(self):
         errorsXML = "<Error><Code>Code1</Code><Message>2</Message></Error>"
         xml = "<a><Errors>%s</Errors><b /></a>" % errorsXML
@@ -99,69 +35,6 @@
         error = EC2Error(payload.sample_ec2_error_messages)
         self.assertEquals(len(error.errors), 2)
 
-    def test_empty_xml(self):
-        self.assertRaises(ValueError, EC2Error, "")
-
-    def test_no_request_id(self):
-        errors = "<Errors><Error><Code /><Message /></Error></Errors>"
-        xml = "<Response>%s<RequestID /></Response>" % errors
-        error = EC2Error(xml)
-        self.assertEquals(error.request_id, "")
-
-    def test_no_request_id_node(self):
-        errors = "<Errors><Error><Code /><Message /></Error></Errors>"
-        xml = "<Response>%s</Response>" % errors
-        error = EC2Error(xml)
-        self.assertEquals(error.request_id, "")
-
-    def test_no_errors_node(self):
-        xml = "<Response><RequestID /></Response>"
-        error = EC2Error(xml)
-        self.assertEquals(error.errors, [])
-
-    def test_no_error_node(self):
-        xml = "<Response><Errors /><RequestID /></Response>"
-        error = EC2Error(xml)
-        self.assertEquals(error.errors, [])
-
-    def test_no_error_code_node(self):
-        errors = "<Errors><Error><Message /></Error></Errors>"
-        xml = "<Response>%s<RequestID /></Response>" % errors
-        error = EC2Error(xml)
-        self.assertEquals(error.errors, [])
-
-    def test_no_error_message_node(self):
-        errors = "<Errors><Error><Code /></Error></Errors>"
-        xml = "<Response>%s<RequestID /></Response>" % errors
-        error = EC2Error(xml)
-        self.assertEquals(error.errors, [])
-
-    def test_single_get_error_codes(self):
-        error = EC2Error(payload.sample_ec2_error_message)
-        self.assertEquals(error.get_error_codes(), "Error.Code")
-
-    def test_multiple_get_error_codes(self):
-        error = EC2Error(payload.sample_ec2_error_messages)
-        self.assertEquals(error.get_error_codes(), 2)
-
-    def test_zero_get_error_codes(self):
-        xml = "<Response><RequestID /></Response>"
-        error = EC2Error(xml)
-        self.assertEquals(error.get_error_codes(), None)
-
-    def test_single_get_error_messages(self):
-        error = EC2Error(payload.sample_ec2_error_message)
-        self.assertEquals(error.get_error_messages(), "Message for Error.Code")
-
-    def test_multiple_get_error_messages(self):
-        error = EC2Error(payload.sample_ec2_error_messages)
-        self.assertEquals(error.get_error_messages(), "Multiple EC2 Errors")
-
-    def test_zero_get_error_messages(self):
-        xml = "<Response><RequestID /></Response>"
-        error = EC2Error(xml)
-        self.assertEquals(error.get_error_messages(), "Empty error list")
-
     def test_single_error_str(self):
         error = EC2Error(payload.sample_ec2_error_message)
         self.assertEquals(str(error), "Error Message: Message for Error.Code")

=== modified file 'txaws/exception.py'
--- txaws/exception.py	2009-10-28 17:43:23 +0000
+++ txaws/exception.py	2009-11-28 01:10:25 +0000
@@ -3,11 +3,124 @@
 
 from twisted.web.error import Error
 
+from txaws.util import XML
+
 
 class AWSError(Error):
     """
     A base class for txAWS errors.
     """
+    def __init__(self, xml_bytes, status=None, message=None, response=None):
+        super(AWSError, self).__init__(status, message, response)
+        if not xml_bytes:
+            raise ValueError("XML cannot be empty.")
+        self.original = xml_bytes
+        self.errors = []
+        self.request_id = ""
+        self.host_id = ""
+        self.parse()
+
+    def __str__(self):
+        return self._get_error_message_string()
+
+    def __repr__(self):
+        return "<%s object with %s>" % (
+            self.__class__.__name__, self._get_error_code_string())
+
+    def _set_request_id(self, tree):
+        request_id_node = tree.find(".//RequestID")
+        if hasattr(request_id_node, "text"):
+            text = request_id_node.text
+            if text:
+                self.request_id = text
+
+    def _set_host_id(self, tree):
+        host_id = tree.find(".//HostID")
+        if hasattr(host_id, "text"):
+            text = host_id.text
+            if text:
+                self.host_id = text
+
+    def _get_error_code_string(self):
+        count = len(self.errors)
+        error_code = self.get_error_codes()
+        if count > 1:
+            return "Error count: %s" % error_code
+        else:
+            return "Error code: %s" % error_code
+
+    def _get_error_message_string(self):
+        count = len(self.errors)
+        error_message = self.get_error_messages()
+        if count > 1:
+            return "%s." % error_message
+        else:
+            return "Error Message: %s" % error_message
+
+    def _node_to_dict(self, node):
+        data = {}
+        for child in node:
+            if child.tag and child.text:
+                data[child.tag] = child.text
+        return data
+
+    def _check_for_html(self, tree):
+        if tree.tag == "html":
+            message = "Could not parse HTML in the response."
+            raise AWSResponseParseError(message)
+
+    def _set_400_error(self, tree):
+        """
+        This method needs to be implemented by subclasses.
+        """
+
+    def _set_500_error(self, tree):
+        self._set_request_id(tree)
+        self._set_host_id(tree)
+        data = self._node_to_dict(tree)
+        if data:
+            self.errors.append(data)
+
+    def parse(self, xml_bytes=""):
+        if not xml_bytes:
+            xml_bytes = self.original
+        self.original = xml_bytes
+        tree = XML(xml_bytes.strip())
+        self._check_for_html(tree)
+        self._set_request_id(tree)
+        if self.status:
+            status = int(self.status)
+        else:
+            status = 400
+        if status >= 500:
+            self._set_500_error(tree)
+        else:
+            self._set_400_error(tree)
+
+    def has_error(self, errorString):
+        for error in self.errors:
+            if errorString in error.values():
+                return True
+        return False
+
+    def get_error_codes(self):
+        count = len(self.errors)
+        if count > 1:
+            return count
+        elif count == 0:
+            return
+        else:
+            return self.errors[0]["Code"]
+
+    def get_error_messages(self):
+        count = len(self.errors)
+        if count > 1:
+            return "Multiple EC2 Errors"
+        elif count == 0:
+            return "Empty error list"
+        else:
+            return self.errors[0]["Message"]
+
 
 
 class AWSResponseParseError(Exception):

=== added file 'txaws/meta.py'
--- txaws/meta.py	1970-01-01 00:00:00 +0000
+++ txaws/meta.py	2009-11-28 01:10:25 +0000
@@ -0,0 +1,10 @@
+display_name = "txAWS"
+library_name = "txaws"
+author = "txAWS Deelopers"
+author_email = "[e-mail]"
+license = "MIT"
+url = "http://launchpad.net/txaws"
+description = """
+Twisted-based Asynchronous Libraries for Amazon Web Services
+"""
+

=== modified file 'txaws/s3/client.py'
--- txaws/s3/client.py	2009-11-28 01:10:25 +0000
+++ txaws/s3/client.py	2009-11-28 01:10:26 +0000
@@ -17,12 +17,17 @@
 
 from epsilon.extime import Time
 
-from txaws.client.base import BaseClient, BaseQuery
+from txaws.client.base import BaseClient, BaseQuery, error_wrapper
 from txaws.s3 import model
+from txaws.s3.exception import S3Error
 from txaws.service import AWSServiceEndpoint, S3_ENDPOINT
 from txaws.util import XML, calculate_md5
 
 
+def s3_error_wrapper(error):
+    error_wrapper(error, S3Error)
+
+
 class URLContext(object):
     """
     The hosts and the paths that form an S3 endpoint change depending upon the
@@ -190,6 +195,10 @@
         self.endpoint.set_method(self.action)
 
     def set_content_type(self):
+        """
+        Set the content type based on the file extension used in the object
+        name.
+        """
         if self.object_name and not self.content_type:
             # XXX nothing is currently done with the encoding... we may
             # need to in the future
@@ -197,6 +206,9 @@
                 self.object_name, strict=False)
 
     def get_headers(self):
+        """
+        Build the list of headers needed in order to perform S3 operations.
+        """
         headers = {"Content-Length": len(self.data),
                    "Content-MD5": calculate_md5(self.data),
                    "Date": self.date}
@@ -214,6 +226,9 @@
         return headers
 
     def get_canonicalized_amz_headers(self, headers):
+        """
+        Get the headers defined by Amazon S3.
+        """
         headers = [
             (name.lower(), value) for name, value in headers.iteritems()
             if name.lower().startswith("x-amz-")]
@@ -224,6 +239,9 @@
         return "".join("%s:%s\n" % (name, value) for name, value in headers)
 
     def get_canonicalized_resource(self):
+        """
+        Get an S3 resource path.
+        """
         resource = "/"
         if self.bucket:
             resource += self.bucket
@@ -232,7 +250,7 @@
         return resource
 
     def sign(self, headers):
-
+        """Sign this query using its built in credentials."""
         text = (self.action + "\n" +
                 headers.get("Content-MD5", "") + "\n" +
                 headers.get("Content-Type", "") + "\n" +
@@ -242,14 +260,14 @@
         return self.creds.sign(text, hash_type="sha1")
 
     def submit(self, url_context=None):
+        """Submit this query.
+
+        @return: A deferred from get_page
+        """
         if not url_context:
             url_context = URLContext(
                 self.endpoint, self.bucket, self.object_name)
         d = self.get_page(
             url_context.get_url(), method=self.action, postdata=self.data,
             headers=self.get_headers())
-        # XXX - we need an error wrapper like we have for ec2... but let's
-        # wait until the new error-wrapper branch has landed, and possibly
-        # generalize a base class for all clients.
-        #d.addErrback(s3_error_wrapper)
-        return d
+        return d.addErrback(s3_error_wrapper)

=== added file 'txaws/s3/exception.py'
--- txaws/s3/exception.py	1970-01-01 00:00:00 +0000
+++ txaws/s3/exception.py	2009-11-28 01:10:26 +0000
@@ -0,0 +1,21 @@
+# Copyright (c) 2009 Canonical Ltd <[e-mail]>
+# Licenced under the txaws licence available at /LICENSE in the txaws source.
+
+from txaws.exception import AWSError
+
+
+class S3Error(AWSError):
+    """
+    A error class providing custom methods on S3 errors.
+    """
+    def _set_400_error(self, tree):
+        if tree.tag.lower() == "error":
+            data = self._node_to_dict(tree)
+            if data:
+                self.errors.append(data)
+
+    def get_error_code(self, *args, **kwargs):
+        return super(S3Error, self).get_error_codes(*args, **kwargs)
+
+    def get_error_message(self, *args, **kwargs):
+        return super(S3Error, self).get_error_messages(*args, **kwargs)

=== added file 'txaws/s3/tests/test_exception.py'
--- txaws/s3/tests/test_exception.py	1970-01-01 00:00:00 +0000
+++ txaws/s3/tests/test_exception.py	2009-11-28 01:10:26 +0000
@@ -0,0 +1,62 @@
+# Copyright (c) 2009 Canonical Ltd <[e-mail]>
+# Licenced under the txaws licence available at /LICENSE in the txaws source.
+
+from twisted.trial.unittest import TestCase
+
+from txaws.s3.exception import S3Error
+from txaws.testing import payload
+from txaws.util import XML
+
+
+REQUEST_ID = "0ef9fc37-6230-4d81-b2e6-1b36277d4247"
+
+
+class S3ErrorTestCase(TestCase):
+
+    def test_set_400_error(self):
+        xml = "<Error><Code>1</Code><Message>2</Message></Error>"
+        error = S3Error("<dummy />")
+        error._set_400_error(XML(xml))
+        self.assertEquals(error.errors[0]["Code"], "1")
+        self.assertEquals(error.errors[0]["Message"], "2")
+
+    def test_get_error_code(self):
+        error = S3Error(payload.sample_s3_invalid_access_key_result)
+        self.assertEquals(error.get_error_code(), "InvalidAccessKeyId")
+
+    def test_get_error_message(self):
+        error = S3Error(payload.sample_s3_invalid_access_key_result)
+        self.assertEquals(
+            error.get_error_message(),
+            ("The AWS Access Key Id you provided does not exist in our "
+             "records."))
+
+    def test_error_count(self):
+        error = S3Error(payload.sample_s3_invalid_access_key_result)
+        self.assertEquals(len(error.errors), 1)
+
+    def test_error_repr(self):
+        error = S3Error(payload.sample_s3_invalid_access_key_result)
+        self.assertEquals(
+            repr(error),
+            "<S3Error object with Error code: InvalidAccessKeyId>")
+
+    def test_signature_mismatch_result(self):
+        error = S3Error(payload.sample_s3_signature_mismatch)
+        self.assertEquals(
+            error.get_error_messages(), 
+            ("The request signature we calculated does not match the "
+             "signature you provided. Check your key and signing method."))
+
+    def test_invalid_access_key_result(self):
+        error = S3Error(payload.sample_s3_invalid_access_key_result)
+        self.assertEquals(
+            error.get_error_messages(),
+            ("The AWS Access Key Id you provided does not exist in our "
+             "records."))
+
+    def test_internal_error_result(self):
+        error = S3Error(payload.sample_server_internal_error_result)
+        self.assertEquals(
+            error.get_error_messages(),
+            "We encountered an internal error. Please try again.")

=== added file 'txaws/script.py'
--- txaws/script.py	1970-01-01 00:00:00 +0000
+++ txaws/script.py	2009-11-28 01:10:25 +0000
@@ -0,0 +1,42 @@
+from optparse import OptionParser
+
+from txaws import meta
+from txaws import version
+
+
+# XXX Once we start adding script that require conflicting options, we'll need
+# multiple parsers and option dispatching...
+def parse_options(usage):
+    parser = OptionParser(usage, version="%s %s" % (
+        meta.display_name, version.txaws))
+    parser.add_option(
+        "-a", "--access-key", dest="access_key", help="access key ID")
+    parser.add_option(
+        "-s", "--secret-key", dest="secret_key", help="access secret key")
+    parser.add_option(
+        "-r", "--region", dest="region", help="US or EU (valid for AWS only)")
+    parser.add_option(
+        "-U", "--url", dest="url", help="service URL/endpoint")
+    parser.add_option(
+        "-b", "--bucket", dest="bucket", help="name of the bucket")
+    parser.add_option(
+        "-o", "--object-name", dest="object_name", help="name of the object")
+    parser.add_option(
+        "-d", "--object-data", dest="object_data",
+        help="content data of the object")
+    parser.add_option(
+        "--object-file", dest="object_filename",
+        help=("the path to the file that will be saved as an object; if "
+               "provided, the --object-name and --object-data options are "
+               "not necessary"))
+    parser.add_option(
+        "-c", "--content-type", dest="content_type",
+        help="content type of the object")
+    options, args = parser.parse_args()
+    if not (options.access_key and options.secret_key):
+        parser.error(
+            "both the access key ID and the secret key must be supplied")
+    region = options.region
+    if region and region.upper() not in ["US", "EU"]:
+        parser.error("region must be one of 'US' or 'EU'")
+    return (options, args)

=== modified file 'txaws/testing/payload.py'
--- txaws/testing/payload.py	2009-11-28 01:10:25 +0000
+++ txaws/testing/payload.py	2009-11-28 01:10:26 +0000
@@ -656,6 +656,31 @@
 """
 
 
+sample_restricted_resource_result = """\
+<?xml version="1.0"?>
+<Response>
+  <Errors>
+    <Error>
+      <Code>AuthFailure</Code>
+      <Message>Unauthorized attempt to access restricted resource</Message>
+    </Error>
+  </Errors>
+  <RequestID>a99e832e-e6e0-416a-9a35-81798ea521b4</RequestID>
+</Response>
+"""
+
+
+sample_server_internal_error_result = """\
+<?xml version="1.0" encoding="UTF-8"?>
+<Error>
+  <Code>InternalError</Code>
+  <Message>We encountered an internal error. Please try again.</Message>
+  <RequestID>A2A7E5395E27DFBB</RequestID>
+  <HostID>f691zulHNsUqonsZkjhILnvWwD3ZnmOM4ObM1wXTc6xuS3GzPmjArp8QC/sGsn6K</HostID>
+</Error>
+"""
+
+
 sample_list_buckets_result = """\
 <?xml version="1.0" encoding="UTF-8"?>
 <ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/%s/">
@@ -692,26 +717,13 @@
 """
 
 
-sample_server_internal_error_result = """\
+sample_s3_invalid_access_key_result = """\
 <?xml version="1.0" encoding="UTF-8"?>
 <Error>
-  <Code>InternalError</Code>
-  <Message>We encountered an internal error. Please try again.</Message>
-  <RequestID>A2A7E5395E27DFBB</RequestID>
-  <HostID>f691zulHNsUqonsZkjhILnvWwD3ZnmOM4ObM1wXTc6xuS3GzPmjArp8QC/sGsn6K</HostID>
+  <Code>InvalidAccessKeyId</Code>
+  <Message>The AWS Access Key Id you provided does not exist in our records.</Message>
+  <RequestId>0223AD81A94821CE</RequestId>
+  <HostId>HAw5g9P1VkN8ztgLKFTK20CY5LmCfTwXcSths1O7UQV6NuJx2P4tmFnpuOsziwOE</HostId>
+  <AWSAccessKeyId>SOMEKEYID</AWSAccessKeyId>
 </Error>
 """
-
-
-sample_restricted_resource_result = """\
-<?xml version="1.0"?>
-<Response>
-  <Errors>
-    <Error>
-      <Code>AuthFailure</Code>
-      <Message>Unauthorized attempt to access restricted resource</Message>
-    </Error>
-  </Errors>
-  <RequestID>a99e832e-e6e0-416a-9a35-81798ea521b4</RequestID>
-</Response>
-"""

=== added file 'txaws/tests/test_exception.py'
--- txaws/tests/test_exception.py	1970-01-01 00:00:00 +0000
+++ txaws/tests/test_exception.py	2009-11-28 01:10:26 +0000
@@ -0,0 +1,114 @@
+# Copyright (c) 2009 Canonical Ltd <[e-mail]>
+# Licenced under the txaws licence available at /LICENSE in the txaws source.
+
+from twisted.trial.unittest import TestCase
+
+from txaws.exception import AWSError
+from txaws.exception import AWSResponseParseError
+from txaws.util import XML
+
+
+REQUEST_ID = "0ef9fc37-6230-4d81-b2e6-1b36277d4247"
+
+
+class AWSErrorTestCase(TestCase):
+
+    def test_creation(self):
+        error = AWSError("<dummy1 />", 500, "Server Error", "<dummy2 />")
+        self.assertEquals(error.status, 500)
+        self.assertEquals(error.response, "<dummy2 />")
+        self.assertEquals(error.original, "<dummy1 />")
+        self.assertEquals(error.errors, [])
+        self.assertEquals(error.request_id, "")
+
+    def test_node_to_dict(self):
+        xml = "<parent><child1>text1</child1><child2>text2</child2></parent>"
+        error = AWSError("<dummy />")
+        data = error._node_to_dict(XML(xml))
+        self.assertEquals(data, {"child1": "text1", "child2": "text2"})
+
+    def test_set_request_id(self):
+        xml = "<a><b /><RequestID>%s</RequestID></a>" % REQUEST_ID
+        error = AWSError("<dummy />")
+        error._set_request_id(XML(xml))
+        self.assertEquals(error.request_id, REQUEST_ID)
+
+    def test_set_host_id(self):
+        host_id = "ASD@#FDG$E%FG"
+        xml = "<a><b /><HostID>%s</HostID></a>" % host_id
+        error = AWSError("<dummy />")
+        error._set_host_id(XML(xml))
+        self.assertEquals(error.host_id, host_id)
+
+    def test_set_empty_errors(self):
+        xml = "<a><Errors /><b /></a>"
+        error = AWSError("<dummy />")
+        error._set_500_error(XML(xml))
+        self.assertEquals(error.errors, [])
+
+    def test_set_empty_error(self):
+        xml = "<a><Errors><Error /><Error /></Errors><b /></a>"
+        error = AWSError("<dummy />")
+        error._set_500_error(XML(xml))
+        self.assertEquals(error.errors, [])
+
+    def test_parse_without_xml(self):
+        xml = "<dummy />"
+        error = AWSError(xml)
+        error.parse()
+        self.assertEquals(error.original, xml)
+
+    def test_parse_with_xml(self):
+        xml1 = "<dummy1 />"
+        xml2 = "<dummy2 />"
+        error = AWSError(xml1)
+        error.parse(xml2)
+        self.assertEquals(error.original, xml2)
+
+    def test_parse_html(self):
+        xml = "<html><body>a page</body></html>"
+        self.assertRaises(AWSResponseParseError, AWSError, xml)
+
+    def test_empty_xml(self):
+        self.assertRaises(ValueError, AWSError, "")
+
+    def test_no_request_id(self):
+        errors = "<Errors><Error><Code /><Message /></Error></Errors>"
+        xml = "<Response>%s<RequestID /></Response>" % errors
+        error = AWSError(xml)
+        self.assertEquals(error.request_id, "")
+
+    def test_no_request_id_node(self):
+        errors = "<Errors><Error><Code /><Message /></Error></Errors>"
+        xml = "<Response>%s</Response>" % errors
+        error = AWSError(xml)
+        self.assertEquals(error.request_id, "")
+
+    def test_no_errors_node(self):
+        xml = "<Response><RequestID /></Response>"
+        error = AWSError(xml)
+        self.assertEquals(error.errors, [])
+
+    def test_no_error_node(self):
+        xml = "<Response><Errors /><RequestID /></Response>"
+        error = AWSError(xml)
+        self.assertEquals(error.errors, [])
+
+    def test_no_error_code_node(self):
+        errors = "<Errors><Error><Message /></Error></Errors>"
+        xml = "<Response>%s<RequestID /></Response>" % errors
+        error = AWSError(xml)
+        self.assertEquals(error.errors, [])
+
+    def test_no_error_message_node(self):
+        errors = "<Errors><Error><Code /></Error></Errors>"
+        xml = "<Response>%s<RequestID /></Response>" % errors
+        error = AWSError(xml)
+        self.assertEquals(error.errors, [])
+
+    def test_set_500_error(self):
+        xml = "<Error><Code>500</Code><Message>Oops</Message></Error>"
+        error = AWSError("<dummy />")
+        error._set_500_error(XML(xml))
+        self.assertEquals(error.errors[0]["Code"], "500")
+        self.assertEquals(error.errors[0]["Message"], "Oops")

=== modified file 'txaws/util.py'
--- txaws/util.py	2009-11-28 01:10:25 +0000
+++ txaws/util.py	2009-11-28 01:10:26 +0000
@@ -94,3 +94,33 @@
     if path == "":
         path = "/"
     return (str(scheme), str(host), port, str(path))
+
+
+def get_exitcode_reactor():
+    """
+    This is only neccesary until a fix like the one outlined here is
+    implemented for Twisted:
+        http://twistedmatrix.com/trac/ticket/2182
+    """
+    from twisted.internet.main import installReactor
+    from twisted.internet.selectreactor import SelectReactor
+
+    class ExitCodeReactor(SelectReactor):
+
+        def stop(self, exitStatus=0):
+            super(ExitCodeReactor, self).stop()
+            self.exitStatus = exitStatus
+
+        def run(self, *args, **kwargs):
+            super(ExitCodeReactor, self).run(*args, **kwargs)
+            return self.exitStatus
+
+    reactor = ExitCodeReactor()
+    installReactor(reactor)
+    return reactor
+
+
+try:
+    reactor = get_exitcode_reactor()
+except:
+    from twisted.internet import reactor