Merge lp:~fwereade/pyjuju/webdav-auth-alternative into lp:pyjuju

[William Reade]

I think this is a slightly cleaner approach.

The big drawback of this branch (from a certain perspective) is that unrelated tests are mocking out get_page_auth (rather than setting up a local server and running the full interaction, as is done in tests for get_page_auth itself, and those for FileStorage which use it directly). IMO, mocking at this level is justifiable(-shading-to-desirable), simply because trying to run a server with (almost) every Orchestra test has a disproportionately obfuscating effect on those tests (changes to orchestra-server and/or storage-url in config, determined at runtime, plus additional complexity on every url check).

The big benefit is that authentication itself is much more cleanly separated from the rest of the code, and it will be much easier to fix the authentication in isolation if/when we need to deal with other mechanisms and/or implementations.

[lp:~fwereade/pyjuju/webdav-auth-alternative]

338. By William Reade on 2011-10-11

merge trunk

[Kapil Thangavelu (hazmat)]

much nicer, and full of awesome, +1

[0] has this been tested against a live apache install?

review: Approve

[William Reade (fwereade)]

[0]

Yep.

[lp:~fwereade/pyjuju/webdav-auth-alternative]

339. By William Reade on 2011-10-13

merge trunk

340. By William Reade on 2011-10-14

merge trunk

[Kapil Thangavelu (hazmat)]

ne minor i noticed in a second pass.

[0]

66 + realm = info["realm"]
67 + nonce = info["nonce"]
68 + algorithm = info.get("algorithm", "MD5")
69 + if algorithm != "MD5":
70 + raise ProviderError("Unsupported digest algorithm: %s" % algorithm)
71 + qop = info["qop"]

Seems like all the info retrieval, should be behind a keyerror catch, broken
implementations abound in external systems.

[lp:~fwereade/pyjuju/webdav-auth-alternative]

341. By William Reade on 2011-10-14

hazmat's followup review point; imports in test_files

342. By William Reade on 2011-10-14

merge trunk

[Benjamin Saller (bcsaller)]

This branch looks good, thanks. +1

[1] trivial: wondering about the style of

+ d.addErrback(_convert_404, url)
+ d.addErrback(_convert_unknown_twisted_error, "GET", url)
+ d.addErrback(convert_unknown_error)

which seems to me a way of building a switch out of private functions where evolution here would require adding another function and more errbacks. Unless I misread wouldn't it be more maintainable to put the error handling switch in a single errback so that any changes only needed to occur in the errback and not in any of the places it registered as well (as they'd already have the orig function). get_page_auth already has the HTTP method which is the only part of the errorback chain thats somewhat modular and could return the defer with the errbacks already bound as expected.

Feel free to disagree, but this is a case where modularity only asks us to repeat ourselves.

review: Approve

[William Reade (fwereade)]

> Feel free to disagree, but this is a case where modularity only asks us to
> repeat ourselves.

I can see that _convert_unknown_twisted_error and convert_unknown_error are used (almost) identically in .get() and .put(), and it would make sense for _c_u_t_e to just call c_u_e itself. However, the important bits are the 404 and 401 handlers respectively, and they have to run before we fall back to the generic error handling; we could pass, say, a dict of codes to handlers, but that feels like a step back.

I don't think it's appropriate to put the 401 fallback handler in get_page_auth, simply because it then makes get_page_auth work on a different level to getPage; IMO, it's cognitively useful to have getPage (which it's reasonable to assume familiarity with) next to, and used the same as, the unfamiliar get_page_auth. It also fels slightly icky to handle 401s and 404s at different levels of the program.

[Kapil Thangavelu (hazmat)]

pep8 trivial, need some spaces
+ self.quietLoss=True

[lp:~fwereade/pyjuju/webdav-auth-alternative]

343. By William Reade on 2011-10-17

address review points

=== modified file 'juju/environment/config.py'
--- juju/environment/config.py	2011-10-06 22:24:21 +0000
+++ juju/environment/config.py	2011-10-17 07:46:17 +0000
@@ -53,9 +53,12 @@
                               "acquired-mgmt-class": String(),
                               "available-mgmt-class": String(),
                               "storage-url": String(),
+                              "storage-user": String(),
+                              "storage-pass": String(),
                               "placement": String(),
                               "default-series": String()},
-                             optional=["storage-url", "placement"]),
+                             optional=["storage-url", "storage-user",
+                                       "storage-pass", "placement"]),
         "local": KeyDict({"admin-secret": String(),
                         "data-dir": String(),
                         "placement": Constant("local"),

=== modified file 'juju/providers/orchestra/__init__.py'
--- juju/providers/orchestra/__init__.py	2011-09-23 20:35:02 +0000
+++ juju/providers/orchestra/__init__.py	2011-10-17 07:46:17 +0000
@@ -19,6 +19,7 @@
     def __init__(self, environment_name, config):
         super(MachineProvider, self).__init__(environment_name, config)
         self.cobbler = CobblerClient(config)
+        self._storage = FileStorage(config)
 
     @property
     def provider_type(self):
@@ -26,10 +27,7 @@
 
     def get_file_storage(self):
         """Return a WebDAV-backed FileStorage abstraction."""
-        if "storage-url" not in self.config:
-            return FileStorage(
-                "http://%(orchestra-server)s/webdav" % self.config)
-        return FileStorage(self.config["storage-url"])
+        return self._storage
 
     def start_machine(self, machine_data, master=False):
         """Start an Orchestra machine.

=== added file 'juju/providers/orchestra/digestauth.py'
--- juju/providers/orchestra/digestauth.py	1970-01-01 00:00:00 +0000
+++ juju/providers/orchestra/digestauth.py	2011-10-17 07:46:17 +0000
@@ -0,0 +1,105 @@
+from hashlib import md5
+from urllib2 import parse_http_list, parse_keqv_list
+from uuid import uuid4
+
+from twisted.internet import reactor
+from twisted.internet.ssl import ClientContextFactory
+from twisted.python.failure import Failure
+from twisted.web.client import HTTPClientFactory, HTTPPageGetter
+from twisted.web.error import Error
+
+from juju.errors import ProviderError
+
+
+def _parse_auth_info(auth_info):
+    method, info_str = auth_info.split(' ', 1)
+    if method != "Digest":
+        raise ProviderError("Unknown authentication method: %s" % method)
+    items = parse_http_list(info_str)
+    info = parse_keqv_list(items)
+
+    try:
+        qop = info["qop"]
+        realm = info["realm"]
+        nonce = info["nonce"]
+    except KeyError as e:
+        raise ProviderError(
+            "Authentication request missing required key: %s" % e)
+    algorithm = info.get("algorithm", "MD5")
+    if algorithm != "MD5":
+        raise ProviderError("Unsupported digest algorithm: %s" % algorithm)
+    if "auth" not in qop.split(","):
+        raise ProviderError("Unsupported quality-of-protection: %s" % qop)
+    return realm, nonce, "auth", algorithm
+
+
+def _digest_squish(*fields):
+    return md5(":".join(fields)).hexdigest()
+
+
+class DigestAuthenticator(object):
+
+    def __init__(self, username, password):
+        self._user = username
+        self._pass = password
+        self._nonce_count = 0
+
+    def authenticate(self, method, url, auth_info):
+        realm, nonce, qop, algorithm = _parse_auth_info(auth_info)
+        ha1 = _digest_squish(self._user, realm, self._pass)
+        ha2 = _digest_squish(method, url)
+        cnonce = str(uuid4())
+        self._nonce_count += 1
+        nc = "%08x" % self._nonce_count
+        response = _digest_squish(ha1, nonce, nc, cnonce, qop, ha2)
+        return (
+            'Digest username="%s", realm="%s", nonce="%s", uri="%s", '
+            'algorithm="%s", response="%s", qop="%s", nc="%s", cnonce="%s"'
+            % (self._user, realm, nonce, url, algorithm, response, qop,
+               nc, cnonce))
+
+
+def _connect(factory):
+    if factory.scheme == 'https':
+        reactor.connectSSL(
+            factory.host, factory.port, factory, ClientContextFactory())
+    else:
+        reactor.connectTCP(factory.host, factory.port, factory)
+
+
+class _AuthPageGetter(HTTPPageGetter):
+
+    handleStatus_204 = lambda self: self.handleStatus_200()
+
+    def handleStatus_401(self):
+        if not self.factory.authenticated:
+            (auth_info,) = self.headers["www-authenticate"]
+            self.factory.authenticate(auth_info)
+            _connect(self.factory)
+        else:
+            self.handleStatusDefault()
+            self.factory.noPage(Failure(Error(self.status, self.message)))
+        self.quietLoss = True
+        self.transport.loseConnection()
+
+
+class _AuthClientFactory(HTTPClientFactory):
+
+    protocol = _AuthPageGetter
+    authenticated = False
+
+    def __init__(self, url, authenticator, **kwargs):
+        HTTPClientFactory.__init__(self, url, **kwargs)
+        self._authenticator = authenticator
+
+    def authenticate(self, auth_info):
+        self.headers["authorization"] = self._authenticator.authenticate(
+            self.method, self.url, auth_info)
+        self.authenticated = True
+
+
+def get_page_auth(url, authenticator, method="GET", postdata=None):
+    factory = _AuthClientFactory(
+        url, authenticator, method=method, postdata=postdata)
+    _connect(factory)
+    return factory.deferred

=== modified file 'juju/providers/orchestra/files.py'
--- juju/providers/orchestra/files.py	2011-09-15 18:50:23 +0000
+++ juju/providers/orchestra/files.py	2011-10-17 07:46:17 +0000
@@ -5,14 +5,32 @@
 from twisted.web.client import getPage
 from twisted.web.error import Error
 
-from juju.errors import FileNotFound
+from juju.errors import FileNotFound, ProviderError
+from juju.providers.common.utils import convert_unknown_error
+from juju.providers.orchestra.digestauth import (
+    DigestAuthenticator, get_page_auth)
+
+
+def _convert_error(failure, method, url, errors):
+    if failure.check(Error):
+        status = failure.value.status
+        error = errors.get(int(status))
+        if error:
+            raise error
+        raise ProviderError(
+            "Unexpected HTTP %s trying to %s %s" % (status, method, url))
+    return convert_unknown_error(failure)
 
 
 class FileStorage(object):
     """A WebDAV-backed :class:`FileStorage` abstraction"""
 
-    def __init__(self, base_url):
-        self._base_url = base_url
+    def __init__(self, config):
+        fallback_url = "http://%(orchestra-server)s/webdav" % config
+        self._base_url = config.get("storage-url", fallback_url)
+        self._auth = DigestAuthenticator(
+            config.get("storage-user", config["orchestra-user"]),
+            config.get("storage-pass", config["orchestra-pass"]))
 
     def get_url(self, name):
         """Return a URL that can be used to access a stored file.
@@ -44,17 +62,11 @@
         url = self.get_url(name)
         d = getPage(url)
         d.addCallback(StringIO)
-
-        def convert_404(failure):
-            failure.trap(Error)
-            if failure.value.status == "404":
-                raise FileNotFound(url)
-            return failure
-        d.addErrback(convert_404)
+        d.addErrback(_convert_error, "GET", url, {404: FileNotFound(url)})
         return d
 
-    def put(self, remote_path, file_object):
-        """Upload a file to S3.
+    def put(self, name, file_object):
+        """Upload a file to WebDAV.
 
         :param unicode remote_path: path on which to store the content
 
@@ -62,16 +74,11 @@
 
         :rtype: :class:`twisted.internet.defer.Deferred`
         """
-        url = self.get_url(remote_path)
-        data = file_object.read()
-        d = getPage(url, method="PUT", postdata=data)
+        url = self.get_url(name)
+        postdata = file_object.read()
+        d = get_page_auth(url, self._auth, method="PUT", postdata=postdata)
         d.addCallback(lambda _: True)
-
-        def accept_204(failure):
-            # NOTE: webdav returns 204s when we overwrite
-            failure.trap(Error)
-            if failure.value.status == "204":
-                return True
-            return failure
-        d.addErrback(accept_204)
+        d.addErrback(_convert_error, "PUT", url, {401: ProviderError(
+            "The supplied storage credentials were not accepted by the "
+            "server")})
         return d

=== modified file 'juju/providers/orchestra/tests/common.py'
--- juju/providers/orchestra/tests/common.py	2011-09-29 05:35:04 +0000
+++ juju/providers/orchestra/tests/common.py	2011-10-17 07:46:17 +0000
@@ -4,11 +4,11 @@
 from xmlrpclib import Fault
 from yaml import dump, load
 
-from twisted.internet.defer import fail, succeed
+from twisted.internet.defer import fail, succeed, inlineCallbacks, returnValue
 from twisted.web.error import Error
 from twisted.web.xmlrpc import Proxy
 
-from juju.lib.mocker import MATCH
+from juju.lib.mocker import ANY, MATCH
 from juju.providers.orchestra import MachineProvider
 
 DATA_DIR = os.path.join(os.path.abspath(os.path.dirname(__file__)), "data")
@@ -42,16 +42,45 @@
         Proxy_m("http://somewhe.re/cobbler_api")
         self.mocker.result(self.proxy_m)
         self.getPage = self.mocker.replace("twisted.web.client.getPage")
+        self.get_page_auth = self.mocker.replace(
+            "juju.providers.orchestra.digestauth.get_page_auth")
+
+    def mock_fs_get(self, url, code, content=None):
+        self.getPage(url)
+        if code == 200:
+            self.mocker.result(succeed(content))
+        else:
+            self.mocker.result(fail(Error(str(code))))
+
+    def mock_fs_put(self, url, expect, code=201):
+        # NOTE: in some respects, it would be better to simulate the complete
+        # interaction with the webdav provider; the factors that work against
+        # doing so are:
+        # 1) authentication is tested on DigestAuthenticator, on get_page_auth,
+        #    and again on FileStorage; even if it were easy to do so, testing
+        #    the same paths at yet another level starts to feel somewhat
+        #    superfluous.
+        # 2) it's not *easy* to do so: we'd have a unique storage URL base for
+        #    every test, which involves a custom config dict for every test,
+        #    and unique URLs to check for every test that hits webdav, and
+        #    another base class to setUp/tearDown. that's not to say it's
+        #    *hard* to do so, but it's time-consuming and costs more complexity
+        #    -- in a large number of the orchestra tests -- than is warranted
+        #    by whatever additional verification it might allow for.
+        self.get_page_auth(url, ANY, method="PUT", postdata=expect)
+        if code in (201, 204):
+            self.mocker.result(succeed(""))
+        else:
+            self.mocker.result(fail(Error(str(code))))
 
     def mock_find_zookeepers(self, existing=None):
-        self.getPage("http://somewhe.re/webdav/provider-state")
-
+        url = "http://somewhe.re/webdav/provider-state"
         if existing is None:
-            self.mocker.result(fail(Error("404")))
+            self.mock_fs_get(url, 404)
         else:
             uid, name = existing
-            self.mocker.result(succeed(dump(
-                {"zookeeper-instances": [uid]})))
+            content = dump({"zookeeper-instances": [uid]})
+            self.mock_fs_get(url, 200, content)
             self.mock_describe_systems(succeed([{
                 "uid": uid, "name": name, "mgmt_classes": ["acquired"]}]))
 

=== added file 'juju/providers/orchestra/tests/data/server.crt'
--- juju/providers/orchestra/tests/data/server.crt	1970-01-01 00:00:00 +0000
+++ juju/providers/orchestra/tests/data/server.crt	2011-10-17 07:46:17 +0000
@@ -0,0 +1,12 @@
+-----BEGIN CERTIFICATE-----
+MIIBtzCCASACCQCBVxLrMqKj9TANBgkqhkiG9w0BAQUFADAfMR0wGwYDVQQDExRl
+bnNlbWJsZSB0ZXN0IHNlcnZlcjAgFw0xMTA5MDcxMzIwMDBaGA8yMTExMDgxNDEz
+MjAwMFowHzEdMBsGA1UEAxMUZW5zZW1ibGUgdGVzdCBzZXJ2ZXIwgZ8wDQYJKoZI
+hvcNAQEBBQADgY0AMIGJAoGBANUSvEXDA8HTbFVGi+v4akPHjOHicp8nYZK2m6wJ
+R/jV0ACha6s10YMnT0pA+6Dpqms+isw51nP7tbnnRFvZL4qbLKTgO6Y2/mehb9k0
+368Tesw0+xKU9ORBX78hlSU6YCfvxX6VxRDWocczCnTWuw2SDPf88kLFQTfzULEA
+6+fRAgMBAAEwDQYJKoZIhvcNAQEFBQADgYEARrXUZSdVsljB5lj2T1IqXQosPR+6
+ndMAS9w05Qe4Jmo5wKrJwH7zwgki/ByS3yqTk9ADPoqZy0MU9WG1io3Z/3MFkg8Y
+ywTdzm1FxJNWD9tTPlwQFR+BlBxpVLszPadwaZnPkYZ4a+WR5R1bohlGElvFZX9I
+2ipfQu0TKxTdGuc=
+-----END CERTIFICATE-----

=== added file 'juju/providers/orchestra/tests/data/server.key'
--- juju/providers/orchestra/tests/data/server.key	1970-01-01 00:00:00 +0000
+++ juju/providers/orchestra/tests/data/server.key	2011-10-17 07:46:17 +0000
@@ -0,0 +1,15 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIICWwIBAAKBgQDVErxFwwPB02xVRovr+GpDx4zh4nKfJ2GStpusCUf41dAAoWur
+NdGDJ09KQPug6aprPorMOdZz+7W550Rb2S+Kmyyk4DumNv5noW/ZNN+vE3rMNPsS
+lPTkQV+/IZUlOmAn78V+lcUQ1qHHMwp01rsNkgz3/PJCxUE381CxAOvn0QIDAQAB
+AoGAWMHpM5Y85mzP3+X3O2DLw1hI03+lB6878gWna06icIGAmAKl+zf8AopJeUEA
+kNNFbk8rOk+Nidr8pGg2DZy3NF678Vy03x8lycoU6ndI6XTev4j4sZa7IMnDThrv
+ldpXZsJfHL9CJXLSqFKZu1OFHxBZwIE794Yeh42QfxV99cECQQD7aE/ZYK583x/7
+6rauUWU15xNNnOaGpqRNE2T3ZnU67xHlMCWa365MCYX6o2bNGi102LUsPxkgkm33
+G7KAiQFNAkEA2Pcn+OGIjLpCSLJw18TbfzO7iDR5tIoec0QQXP3OP+iBQ24CPJW/
+EUBqlMSyd00ORigP//T78AT4KHekT5O+lQJAaSXdj5siH1PquqAWO54LaJn2ttVS
+jSqROTNNXTPbAAURRPv4HmhDK8Yn5QYGbu3t6Rrh21mglsDngRxycdPbWQJALMx7
+uGv5IfWjkhcmLac8Gzu3URxktN6AAxTevBS77X44ko+4boIM/abrWuRyZSfH9rx2
+8UbIbnrYMqLhjnzXMQJAacGuIeKxpiuwC9eJ6Iz+Vmh6bAbhrlfQgiPR2XZIEv8z
+FqEl7L3i6K8l8tuLQFRtQuu1L9YGWvV+g7pDMaR/Bw==
+-----END RSA PRIVATE KEY-----

=== modified file 'juju/providers/orchestra/tests/test_bootstrap.py'
--- juju/providers/orchestra/tests/test_bootstrap.py	2011-09-30 04:52:20 +0000
+++ juju/providers/orchestra/tests/test_bootstrap.py	2011-10-17 07:46:17 +0000
@@ -12,15 +12,12 @@
 class OrchestraBootstrapTest(TestCase, OrchestraTestMixin):
 
     def mock_verify(self):
-        self.getPage("http://somewhe.re/webdav/bootstrap-verify",
-                     method="PUT", postdata="storage is writable")
-        self.mocker.result(succeed(True))
+        self.mock_fs_put("http://somewhe.re/webdav/bootstrap-verify",
+                         "storage is writable")
 
     def mock_save_state(self):
         data = dump({"zookeeper-instances": ["winston-uid"]})
-        self.getPage("http://somewhe.re/webdav/provider-state",
-                     method="PUT", postdata=data)
-        self.mocker.result(succeed(None))
+        self.mock_fs_put("http://somewhe.re/webdav/provider-state", data)
 
     def test_already_bootstrapped(self):
         self.setup_mocks()

=== modified file 'juju/providers/orchestra/tests/test_connect.py'
--- juju/providers/orchestra/tests/test_connect.py	2011-09-15 18:50:23 +0000
+++ juju/providers/orchestra/tests/test_connect.py	2011-10-17 07:46:17 +0000
@@ -20,9 +20,9 @@
 
     def mock_connect(self, share, result):
         self.setup_mocks()
-        self.getPage("http://somewhe.re/webdav/provider-state")
-        self.mocker.result(succeed(dump(
-            {"zookeeper-instances": ["foo"]})))
+        self.mock_fs_get(
+            "http://somewhe.re/webdav/provider-state", 200, dump(
+                {"zookeeper-instances": ["foo"]}))
         self.proxy_m.callRemote("get_systems")
         self.mocker.result(succeed([{"uid": "foo",
                                      "name": "foo.example.com",

=== added file 'juju/providers/orchestra/tests/test_digestauth.py'
--- juju/providers/orchestra/tests/test_digestauth.py	1970-01-01 00:00:00 +0000
+++ juju/providers/orchestra/tests/test_digestauth.py	2011-10-17 07:46:17 +0000
@@ -0,0 +1,302 @@
+import os
+
+from twisted.internet import reactor
+from twisted.internet.defer import inlineCallbacks
+from twisted.internet.ssl import DefaultOpenSSLContextFactory
+from twisted.protocols.policies import WrappingFactory
+from twisted.web.error import Error
+from twisted.web.resource import Resource
+from twisted.web.server import Site
+
+from juju.errors import ProviderError
+from juju.lib.testing import TestCase
+from juju.providers.orchestra.digestauth import (
+    DigestAuthenticator, get_page_auth)
+
+
+def data_file(name):
+    return os.path.join(os.path.dirname(__file__), "data", name)
+
+
+class DigestAuthenticatorTest(TestCase):
+
+    def setUp(self):
+        super(DigestAuthenticatorTest, self).setUp()
+        self.uuid4_m = self.mocker.replace("uuid.uuid4")
+        self.auth = DigestAuthenticator("jiminy", "cricket")
+
+    def challenge(self, method="Digest", **kwargs):
+        kwargs.setdefault("qop", "auth")
+        details = ", ".join("%s=%s" % item for item in kwargs.items())
+        return ("%s realm=loathing, nonce=blah, %s" % (method, details))
+
+    def default_response(self):
+        return ('Digest username="jiminy", realm="loathing", nonce="blah", '
+                'uri="http://somewhe.re/", algorithm="MD5", '
+                'response="8891952040a96ba62cce585972bd3360", qop="auth", '
+                'nc="00000001", cnonce="twiddle"')
+
+    def assert_response(self, challenge, response):
+        self.assertEquals(
+            self.auth.authenticate("METH", "http://somewhe.re/", challenge),
+            response)
+
+    def assert_error(self, challenge, err_type, err_message):
+        error = self.assertRaises(
+            err_type,
+            self.auth.authenticate, "METH", "http://somewhe.re/", challenge)
+        self.assertEquals(str(error), err_message)
+
+    def assert_missing_key(self, challenge, key):
+        self.mocker.replay()
+        message = "Authentication request missing required key: %r" % key
+        self.assert_error(challenge, ProviderError, message)
+
+    def test_normal(self):
+        self.uuid4_m()
+        self.mocker.result("twiddle")
+        self.mocker.replay()
+        self.assert_response(self.challenge(), self.default_response())
+
+    def test_nc_increments(self):
+        self.uuid4_m()
+        self.mocker.result("twiddle")
+        self.uuid4_m()
+        self.mocker.result("twaddle")
+        self.mocker.replay()
+        self.assert_response(self.challenge(), self.default_response())
+        self.assert_response(
+            self.challenge(),
+            'Digest username="jiminy", realm="loathing", nonce="blah", '
+            'uri="http://somewhe.re/", algorithm="MD5", '
+            'response="c12e9ec2e0569d896b2f26b91c0e74c3", qop="auth", '
+            'nc="00000002", cnonce="twaddle"')
+
+    def test_qop_choices(self):
+        self.uuid4_m()
+        self.mocker.result("twiddle")
+        self.mocker.replay()
+        self.assert_response(self.challenge(qop='"auth,auth-int"'),
+                             self.default_response())
+
+    def test_specify_algorithm(self):
+        self.uuid4_m()
+        self.mocker.result("twiddle")
+        self.mocker.replay()
+        self.assert_response(self.challenge(algorithm="MD5"),
+                             self.default_response())
+
+    def test_bad_method(self):
+        self.mocker.replay()
+        self.assert_error(self.challenge("Masticate"), ProviderError,
+                          "Unknown authentication method: Masticate")
+
+    def test_bad_algorithm(self):
+        self.mocker.replay()
+        self.assert_error(self.challenge(algorithm="ROT13"), ProviderError,
+                          "Unsupported digest algorithm: ROT13")
+
+    def test_bad_qop(self):
+        self.mocker.replay()
+        self.assert_error(self.challenge(qop="auth-int"), ProviderError,
+                          "Unsupported quality-of-protection: auth-int")
+
+    def test_missing_keys(self):
+        self.assert_missing_key("Digest realm=x, nonce=y", "qop")
+        self.assert_missing_key("Digest realm=x, qop=y", "nonce")
+        self.assert_missing_key("Digest qop=x, nonce=y", "realm")
+
+class PlainResource(Resource):
+
+    def __init__(self, test, method, content, expect_content=None, status=200):
+        Resource.__init__(self)
+        self._test = test
+        self._method = method
+        self._content = content
+        self._expect_content = expect_content
+        self._status = status
+
+    def render(self, request):
+        self._test.assertEquals(request.method, self._method)
+        if self._expect_content:
+            self._test.assertEquals(
+                request.content.read(), self._expect_content)
+        request.setResponseCode(self._status)
+        return self._content
+
+
+class AuthResource(Resource):
+
+    def __init__(self, test, method, content, challenge, check_response,
+                 expect_content=None, status=200):
+        Resource.__init__(self)
+        self._test = test
+        self._method = method
+        self._content = content
+        self._challenge = challenge
+        self._check_response = check_response
+        self._expect_content = expect_content
+        self._status = status
+        self._rendered = False
+
+    def render(self, request):
+        if self._expect_content:
+            self._test.assertEquals(
+                request.content.read(), self._expect_content)
+        if not self._rendered:
+            self._rendered = True
+            request.setResponseCode(401)
+            request.setHeader("www-authenticate", self._challenge)
+            return ""
+        else:
+            self._check_response(request.getHeader("authorization"))
+            request.setResponseCode(self._status)
+            return self._content
+
+
+class GetPageAuthTestCase(TestCase):
+
+    scheme = "http"
+
+    def _listen(self, site):
+        if self.scheme == "http":
+            return reactor.listenTCP(0, site, interface="127.0.0.1")
+        elif self.scheme == "https":
+            sslFactory = DefaultOpenSSLContextFactory(
+                data_file("server.key"), data_file("server.crt"))
+            return reactor.listenSSL(
+                0, site, sslFactory, interface="127.0.0.1")
+        else:
+            self.fail("unknown scheme: %s" % self.scheme)
+
+    def setUp(self):
+        super(GetPageAuthTestCase, self).setUp()
+        self.root = Resource()
+        self.wrapper = WrappingFactory(Site(self.root, timeout=None))
+        self.port = self._listen(self.wrapper)
+        self.portno = self.port.getHost().port
+
+    def tearDown(self):
+        super(GetPageAuthTestCase, self).tearDown()
+        if self.wrapper.protocols.keys():
+            print "CONNECTION(S) ALIVE! buggy test?"
+        return self.port.stopListening()
+
+    def get_url(self, path):
+        return "%s/%s" % (self.get_base_url(), path)
+
+    def get_base_url(self):
+        return "%s://127.0.0.1:%s" % (self.scheme, self.portno)
+
+    def add_plain(self, path, method, content,
+                  expect_content=None, status=200):
+        self.root.putChild(path, PlainResource(
+            self, method, content,
+            expect_content=expect_content, status=status))
+
+    def add_auth(self, path, method, content, challenge, check_response,
+                 expect_content=None, status=200):
+        self.root.putChild(path, AuthResource(
+            self, method, content, challenge, check_response,
+            expect_content=expect_content, status=status))
+
+
+class GetPageAuthTestsMixin(object):
+
+    def setup_mock(self):
+        self.uuid4_m = self.mocker.replace("uuid.uuid4")
+
+    @inlineCallbacks
+    def test_404(self):
+        # verify that usual mechanism is in place
+        d = get_page_auth(self.get_url("missing"), None)
+        error = yield self.assertFailure(d, Error)
+        self.assertEquals(error.status, "404")
+
+    def test_default_method(self):
+        self.add_plain("blah", "GET", "cheese")
+        d = get_page_auth(self.get_url("blah"), None)
+        d.addCallback(self.assertEquals, "cheese")
+        return d
+
+    def test_other_method(self):
+        self.add_plain("blob", "TWIDDLE", "pickle")
+        d = get_page_auth(self.get_url("blob"), None, method="TWIDDLE")
+        d.addCallback(self.assertEquals, "pickle")
+        return d
+
+    def test_authenticate(self):
+        self.setup_mock()
+        self.uuid4_m()
+        self.mocker.result("baguette")
+        self.mocker.replay()
+
+        url = self.get_url("blip")
+        auth = DigestAuthenticator("sandwich", "earl")
+
+        def check(response):
+            self.assertTrue(response.startswith(
+                'Digest username="sandwich", realm="x", nonce="y", uri="%s"'
+                % url))
+            self.assertIn(
+                'qop="auth", nc="00000001", cnonce="baguette"', response)
+
+        self.add_auth(
+            "blip", "PROD", "ham", "Digest realm=x, nonce=y, qop=auth", check)
+        d = get_page_auth(url, auth, method="PROD")
+        d.addCallback(self.assertEquals, "ham")
+        return d
+
+    def test_authenticate_postdata_201(self):
+        self.setup_mock()
+        self.uuid4_m()
+        self.mocker.result("ciabatta")
+        self.mocker.replay()
+
+        url = self.get_url("blam")
+        auth = DigestAuthenticator("pizza", "principessa")
+
+        def check(response):
+            self.assertTrue(response.startswith(
+                'Digest username="pizza", realm="a", nonce="b", uri="%s"'
+                % url))
+            self.assertIn(
+                'qop="auth", nc="00000001", cnonce="ciabatta"', response)
+
+        self.add_auth(
+            "blam", "FLING", "tomato", "Digest realm=a, nonce=b, qop=auth",
+            check, expect_content="oven", status=201)
+        d = get_page_auth(url, auth, method="FLING", postdata="oven")
+        d.addCallback(self.assertEquals, "tomato")
+        return d
+
+    def test_authenticate_postdata_204(self):
+        self.setup_mock()
+        self.uuid4_m()
+        self.mocker.result("focaccia")
+        self.mocker.replay()
+
+        url = self.get_url("blur")
+        auth = DigestAuthenticator("quesadilla", "king")
+
+        def check(response):
+            self.assertTrue(response.startswith(
+                'Digest username="quesadilla", realm="p", nonce="q", uri="%s"'
+                % url))
+            self.assertIn(
+                'qop="auth", nc="00000001", cnonce="focaccia"', response)
+
+        self.add_auth(
+            "blur", "HURL", "", "Digest realm=p, nonce=q, qop=auth", check,
+            expect_content="bbq", status=204)
+        d = get_page_auth(url, auth, method="HURL", postdata="bbq")
+        d.addCallback(self.assertEquals, "")
+        return d
+
+
+class GetPageAuthHttpTest(GetPageAuthTestCase, GetPageAuthTestsMixin):
+   scheme = "http"
+
+
+class GetPageAuthHttpsTest(GetPageAuthTestCase, GetPageAuthTestsMixin):
+   scheme = "https"

=== modified file 'juju/providers/orchestra/tests/test_files.py'
--- juju/providers/orchestra/tests/test_files.py	2011-09-15 18:50:23 +0000
+++ juju/providers/orchestra/tests/test_files.py	2011-10-17 07:46:17 +0000
@@ -3,144 +3,203 @@
 from twisted.internet.defer import fail, succeed
 from twisted.web.error import Error
 
-from juju.errors import FileNotFound
+from juju.errors import FileNotFound, ProviderError, ProviderInteractionError
 from juju.lib.testing import TestCase
 from juju.providers.orchestra import MachineProvider
 
+from .test_digestauth import GetPageAuthTestCase
+
+
+class SomeError(Exception):
+    pass
+
 
 def get_file_storage(custom_config=None):
     config = {"orchestra-server": "somewhereel.se",
-              "orchestra-user": "user",
-              "orchestra-pass": "pass",
+              "orchestra-user": "fallback-user",
+              "orchestra-pass": "fallback-pass",
               "acquired-mgmt-class": "acquired",
               "available-mgmt-class": "available"}
     if custom_config is None:
-        config["storage-url"] = "http://somewhe.re/webdav"
+        config["storage-url"] = "http://somewhe.re"
+        config["storage-user"] = "user"
+        config["storage-pass"] = "pass"
     else:
         config.update(custom_config)
     provider = MachineProvider("blah", config)
     return provider.get_file_storage()
 
 
-class FileStorageTest(TestCase):
-
-    def test_get_works_no_storage_url(self):
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhereel.se/webdav/rubber/chicken")
-        self.mocker.result(succeed("pulley"))
-        self.mocker.replay()
-
-        fs = get_file_storage({})
-        d = fs.get("rubber/chicken")
-
-        def verify(result):
-            self.assertEquals(result.read(), "pulley")
-        d.addCallback(verify)
-        return d
-
-    def test_get_works(self):
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/rubber/chicken")
-        self.mocker.result(succeed("pulley"))
-        self.mocker.replay()
-
-        fs = get_file_storage()
-        d = fs.get("rubber/chicken")
-
-        def verify(result):
-            self.assertEquals(result.read(), "pulley")
-        d.addCallback(verify)
-        return d
-
-    def test_get_fails(self):
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/rubber/chicken")
-        self.mocker.result(fail(Error("404")))
-        self.mocker.replay()
-
-        fs = get_file_storage()
-        d = fs.get("rubber/chicken")
-        self.assertFailure(d, FileNotFound)
-        return d
-
-    def test_get_errors(self):
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/rubber/chicken")
-        self.mocker.result(fail(Error("500")))
-        self.mocker.replay()
-
-        fs = get_file_storage()
-        d = fs.get("rubber/chicken")
-        self.assertFailure(d, Error)
-        return d
+class FileStorageGetTest(TestCase):
+
+    def setUp(self):
+        self.uuid4_m = self.mocker.replace("uuid.uuid4")
+        self.getPage = self.mocker.replace("twisted.web.client.getPage")
 
     def test_get_url(self):
-        fs = get_file_storage()
-        url = fs.get_url("rubber/chicken")
-        self.assertEqual(url, "http://somewhe.re/webdav/rubber/chicken")
-
-    def test_get_url_unicode(self):
-        fs = get_file_storage({"storage-url": u"http://\u2666.co.\u2660"})
-        url = fs.get_url(u"rubber/\u2665/chicken")
-        self.assertEqual(
-            url, "http://xn--h6h.co.xn--b6h/rubber/%E2%99%A5/chicken")
-        self.assertInstance(url, str)
-
-    def test_put_works(self):
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/rubber/chicken",
-                method="PUT", postdata="pulley")
-        self.mocker.result(succeed(None))
-        self.mocker.replay()
-
-        fs = get_file_storage()
-        d = fs.put("rubber/chicken", StringIO("pulley"))
-
-        def verify(result):
-            self.assertEquals(result, True)
-        d.addCallback(verify)
-        return d
-
-    def test_put_works_no_storage_url(self):
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhereel.se/webdav/rubber/chicken",
-                method="PUT", postdata="pulley")
-        self.mocker.result(succeed(None))
-        self.mocker.replay()
-
+        self.mocker.replay()
+        fs = get_file_storage()
+        self.assertEquals(fs.get_url("angry/birds"),
+                          "http://somewhe.re/angry/birds")
+
+    def test_get_url_fallback(self):
+        self.mocker.replay()
         fs = get_file_storage({})
-        d = fs.put("rubber/chicken", StringIO("pulley"))
-
-        def verify(result):
-            self.assertEquals(result, True)
-        d.addCallback(verify)
-        return d
-
-    def test_put_handles_204(self):
-        """If we're overwriting instead of creating, we get 204 instead of 200
-        """
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/rubber/chicken",
-                method="PUT", postdata="pulley")
-        self.mocker.result(fail(Error("204")))
-        self.mocker.replay()
-
-        fs = get_file_storage()
-        d = fs.put("rubber/chicken", StringIO("pulley"))
-
-        def verify(result):
-            self.assertEquals(result, True)
-        d.addCallback(verify)
-        return d
-
-    def test_put_errors(self):
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/rubber/chicken",
-                method="PUT", postdata="pulley")
-        self.mocker.result(fail(Error("500")))
-        self.mocker.replay()
-
-        fs = get_file_storage()
-        d = fs.put("rubber/chicken", StringIO("pulley"))
-        self.assertFailure(d, Error)
+        self.assertEquals(fs.get_url("angry/birds"),
+                          "http://somewhereel.se/webdav/angry/birds")
+
+    def test_get(self):
+        self.getPage("http://somewhe.re/rubber/chicken")
+        self.mocker.result(succeed("pulley"))
+        self.mocker.replay()
+
+        fs = get_file_storage()
+        d = fs.get("rubber/chicken")
+
+        def verify(result):
+            self.assertEquals(result.read(), "pulley")
+        d.addCallback(verify)
+        return d
+
+    def check_get_error(self, result, err_type, err_message):
+        self.getPage("http://somewhe.re/rubber/chicken")
+        self.mocker.result(result)
+        self.mocker.replay()
+
+        fs = get_file_storage()
+        d = fs.get("rubber/chicken")
+        self.assertFailure(d, err_type)
+
+        def verify(error):
+            self.assertEquals(str(error), err_message)
+        d.addCallback(verify)
+        return d
+
+    def test_get_error(self):
+        return self.check_get_error(
+            fail(SomeError("pow!")),
+            ProviderInteractionError,
+            "Unexpected SomeError interacting with provider: pow!")
+
+    def test_get_404(self):
+        return self.check_get_error(
+            fail(Error("404")),
+            FileNotFound,
+            "File was not found: 'http://somewhe.re/rubber/chicken'")
+
+    def test_get_bad_code(self):
+        return self.check_get_error(
+            fail(Error("999")),
+            ProviderError,
+            "Unexpected HTTP 999 trying to GET "
+            "http://somewhe.re/rubber/chicken")
+
+
+class FileStoragePutTest(GetPageAuthTestCase):
+
+    def setup_mock(self):
+        self.uuid4_m = self.mocker.replace("uuid.uuid4")
+
+    def get_file_storage(self, with_user=True):
+        storage_url = self.get_base_url()
+        custom_config = {"storage-url": storage_url}
+        if with_user:
+            custom_config["storage-user"] = "user"
+            custom_config["storage-pass"] = "pass"
+        return get_file_storage(custom_config)
+
+    def test_no_auth_error(self):
+        self.add_plain("peregrine", "PUT", "", "croissant", 999)
+        fs = self.get_file_storage()
+        d = fs.put("peregrine", StringIO("croissant"))
+        self.assertFailure(d, ProviderError)
+
+        def verify(error):
+            self.assertIn("Unexpected HTTP 999 trying to PUT ", str(error))
+        d.addCallback(verify)
+        return d
+
+    def test_no_auth_201(self):
+        self.add_plain("peregrine", "PUT", "", "croissant", 201)
+        fs = self.get_file_storage()
+        d = fs.put("peregrine", StringIO("croissant"))
+        d.addCallback(self.assertEquals, True)
+        return d
+
+    def test_no_auth_204(self):
+        self.add_plain("peregrine", "PUT", "", "croissant", 204)
+        fs = self.get_file_storage()
+        d = fs.put("peregrine", StringIO("croissant"))
+        d.addCallback(self.assertEquals, True)
+        return d
+
+    def auth_common(self, username, status, with_user=True):
+        self.setup_mock()
+        self.uuid4_m()
+        self.mocker.result("dinner")
+        self.mocker.replay()
+
+        url = self.get_url("possum")
+
+        def check(response):
+            self.assertTrue(response.startswith(
+                'Digest username="%s", realm="sparta", nonce="meh", uri="%s"'
+                % (username, url)))
+            self.assertIn(
+                'qop="auth", nc="00000001", cnonce="dinner"', response)
+        self.add_auth(
+            "possum", "PUT", "", "Digest realm=sparta, nonce=meh, qop=auth",
+            check, expect_content="canabalt", status=status)
+
+        fs = self.get_file_storage(with_user)
+        return fs.put("possum", StringIO("canabalt"))
+
+    def test_auth_error(self):
+        d = self.auth_common("user", 808)
+        self.assertFailure(d, ProviderError)
+
+        def verify(error):
+            self.assertIn("Unexpected HTTP 808 trying to PUT", str(error))
+        d.addCallback(verify)
+        return d
+
+    def test_auth_bad_credentials(self):
+        d = self.auth_common("user", 401)
+        self.assertFailure(d, ProviderError)
+
+        def verify(error):
+            self.assertEquals(
+                str(error),
+                "The supplied storage credentials were not accepted by the "
+                "server")
+        d.addCallback(verify)
+        return d
+
+    def test_auth_201(self):
+        d = self.auth_common("user", 201)
+        d.addCallback(self.assertEquals, True)
+        return d
+
+    def test_auth_204(self):
+        d = self.auth_common("user", 204)
+        d.addCallback(self.assertEquals, True)
+        return d
+
+    def test_auth_fallback_error(self):
+        d = self.auth_common("fallback-user", 747, False)
+        self.assertFailure(d, ProviderError)
+
+        def verify(error):
+            self.assertIn("Unexpected HTTP 747 trying to PUT", str(error))
+        d.addCallback(verify)
+        return d
+
+    def test_auth_fallback_201(self):
+        d = self.auth_common("fallback-user", 201, False)
+        d.addCallback(self.assertEquals, True)
+        return d
+
+    def test_auth_fallback_204(self):
+        d = self.auth_common("fallback-user", 204, False)
+        d.addCallback(self.assertEquals, True)
         return d

=== modified file 'juju/providers/orchestra/tests/test_findzookeepers.py'
--- juju/providers/orchestra/tests/test_findzookeepers.py	2011-09-15 18:50:23 +0000
+++ juju/providers/orchestra/tests/test_findzookeepers.py	2011-10-17 07:46:17 +0000
@@ -1,14 +1,14 @@
 from yaml import dump
 
-from twisted.internet.defer import fail, succeed
-from twisted.web.error import Error
-from twisted.web.xmlrpc import Proxy
+from twisted.internet.defer import succeed
 
 from juju.errors import EnvironmentNotFound
 from juju.lib.testing import TestCase
 from juju.providers.orchestra import MachineProvider
 from juju.providers.orchestra.machine import OrchestraMachine
 
+from .common import OrchestraTestMixin
+
 CONFIG = {"orchestra-server": "somewhe.re",
           "storage-url": "http://somewhe.re/webdav",
           "orchestra-user": "user",
@@ -17,15 +17,14 @@
           "available-mgmt-class": "available"}
 
 
-class FindZookeepersTest(TestCase):
+class FindZookeepersTest(TestCase, OrchestraTestMixin):
 
     def get_provider(self):
         return MachineProvider("tetrascape", CONFIG)
 
-    def mock_load_state(self, result):
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/provider-state")
-        self.mocker.result(result)
+    def mock_load_state(self, code, content):
+        self.mock_fs_get(
+            "http://somewhe.re/webdav/provider-state", code, content)
 
     def assert_no_environment(self):
         provider = self.get_provider()
@@ -33,41 +32,38 @@
         self.assertFailure(d, EnvironmentNotFound)
         return d
 
-    def verify_no_environment(self, load_result):
-        self.mock_load_state(load_result)
+    def verify_no_environment(self, code, content):
+        self.mock_load_state(code, content)
         self.mocker.replay()
         return self.assert_no_environment()
 
     def test_no_state(self):
-        self.verify_no_environment(fail(Error("404")))
+        self.setup_mocks()
+        self.verify_no_environment(404, None)
 
     def test_empty_state(self):
-        self.verify_no_environment(succeed(dump([])))
+        self.setup_mocks()
+        self.verify_no_environment(200, dump([]))
 
     def test_no_hosts(self):
-        self.verify_no_environment(succeed(dump({"abc": 123})))
+        self.setup_mocks()
+        self.verify_no_environment(200, dump({"abc": 123}))
 
     def test_bad_instance(self):
-        self.mock_load_state(succeed(dump({"zookeeper-instances": ["foo"]})))
-        proxy_m = self.mocker.mock(Proxy)
-        Proxy_m = self.mocker.replace(Proxy, spec=None)
-        Proxy_m("http://somewhe.re/cobbler_api")
-        self.mocker.result(proxy_m)
-        proxy_m.callRemote("get_systems")
+        self.setup_mocks()
+        self.mock_load_state(200, dump({"zookeeper-instances": ["foo"]}))
+        self.proxy_m.callRemote("get_systems")
         self.mocker.result(succeed([]))
         self.mocker.replay()
 
         return self.assert_no_environment()
 
     def test_eventual_success(self):
-        self.mock_load_state(succeed(dump({
-            "zookeeper-instances": ["bad", "foo", "missing", "bar"]})))
-        proxy_m = self.mocker.mock(Proxy)
-        Proxy_m = self.mocker.replace(Proxy, spec=None)
-        Proxy_m("http://somewhe.re/cobbler_api")
-        self.mocker.result(proxy_m)
+        self.setup_mocks()
+        self.mock_load_state(200, dump({
+            "zookeeper-instances": ["bad", "foo", "missing", "bar"]}))
         for _ in range(4):
-            proxy_m.callRemote("get_systems")
+            self.proxy_m.callRemote("get_systems")
             self.mocker.result(succeed([
                 {"uid": "bad", "mgmt_classes": ["whatever"]},
                 {"uid": "foo", "mgmt_classes": ["acquired"], "name": "foo"},

=== modified file 'juju/providers/orchestra/tests/test_shutdown.py'
--- juju/providers/orchestra/tests/test_shutdown.py	2011-09-15 18:50:23 +0000
+++ juju/providers/orchestra/tests/test_shutdown.py	2011-10-17 07:46:17 +0000
@@ -169,16 +169,12 @@
 
     def test_destroy_environment(self):
         self.setup_mocks()
-        self.getPage("http://somewhe.re/webdav/provider-state",
-                     method="PUT", postdata="{}\n")
-        self.mocker.result(succeed(True))
+        self.mock_fs_put("http://somewhe.re/webdav/provider-state", "{}\n")
         return self.check_shutdown_all("destroy_environment")
 
     def test_destroy_environment_no_machines(self):
         self.setup_mocks()
-        self.getPage("http://somewhe.re/webdav/provider-state",
-                     method="PUT", postdata="{}\n")
-        self.mocker.result(succeed(True))
+        self.mock_fs_put("http://somewhe.re/webdav/provider-state", "{}\n")
         self.mock_get_systems()
         self.mocker.replay()
 
@@ -189,7 +185,6 @@
 
     def test_destroy_environment_unwritable(self):
         self.setup_mocks()
-        self.getPage("http://somewhe.re/webdav/provider-state",
-                     method="PUT", postdata="{}\n")
-        self.mocker.result(fail(SomeError()))
+        self.mock_fs_put(
+            "http://somewhe.re/webdav/provider-state", "{}\n", 500)
         return self.check_shutdown_all("destroy_environment")

=== modified file 'juju/providers/orchestra/tests/test_state.py'
--- juju/providers/orchestra/tests/test_state.py	2011-09-15 18:50:23 +0000
+++ juju/providers/orchestra/tests/test_state.py	2011-10-17 07:46:17 +0000
@@ -1,10 +1,10 @@
 from yaml import dump
 
-from twisted.internet.defer import succeed
-
 from juju.lib.testing import TestCase
 from juju.providers.orchestra import MachineProvider
 
+from .common import OrchestraTestMixin
+
 
 def get_provider():
     config = {"orchestra-server": "somewhe.re",
@@ -15,14 +15,13 @@
     return MachineProvider("tetrascape", config)
 
 
-class StateTest(TestCase):
+class StateTest(TestCase, OrchestraTestMixin):
 
     def test_save(self):
+        self.setup_mocks()
         state = {"foo": "blah blah"}
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/provider-state",
-                method="PUT", postdata=dump(state))
-        self.mocker.result(succeed(None))
+        self.mock_fs_put(
+            "http://somewhe.re/webdav/provider-state", dump(state))
         self.mocker.replay()
 
         provider = get_provider()
@@ -34,10 +33,10 @@
         return d
 
     def test_load(self):
+        self.setup_mocks()
         expect_state = {"foo": "blah blah"}
-        getPage = self.mocker.replace("twisted.web.client.getPage")
-        getPage("http://somewhe.re/webdav/provider-state")
-        self.mocker.result(succeed(dump(expect_state)))
+        self.mock_fs_get(
+            "http://somewhe.re/webdav/provider-state", 200, dump(expect_state))
         self.mocker.replay()
 
         provider = get_provider()