Merge lp:~khussein/swift/authn into lp:~hudson-openstack/swift/trunk

[Khaled Hussein]

Implemented the proposed protocol to handle authentication in OpenStack services.

[Mike Barton (redbo)]

> self.response_status = status

self.response_status is storing per-request info in a shared object. That could be wrong under concurrency.

> ''.join(response_itr)
> return [resp.read()]
> response.content_length = sum(map(len, response.app_iter))

Storing many GB of data in RAM is a bad idea.

> usersConfig.readfp(open('/etc/openstack/users.ini'))

You shouldn't re-read and parse the basic auth credentials file on every request. The local imports there are kind of ugly too.

> def validateCreds(self, username, password):

only major pep8 violation I see is those camel caps.

> - 'auth=swift.auth.server:app_factory',
> + #'auth=swift.auth.server:app_factory',
> + 'auth=swift.auth.basicauth:app_factory',

Was leaving it this way a mistake? The basic auth isn't super useful right now, and removing the devauth filter completely breaks our dev and testing environments.

review: Needs Fixing

[gholt (gholt)]

We meant for this to be a work-in-progress; sorry.

[gholt (gholt)]

Oh, and these weren't the droids you were looking for anyhow. Move along; move along.

[Khaled Hussein (khussein)]

The proxy-server config now would be [auth1 devauth papiauth]

[lp:~khussein/swift/authn]

152. By Khaled Hussein on 2011-01-10

Fixed unit tests

153. By Khaled Hussein on 2011-01-10

Merged trunk

154. By Khaled Hussein on 2011-01-10

setup.py config changes

155. By Khaled Hussein on 2011-01-11

fixed functional tests

156. By Khaled Hussein on 2011-01-11

fixing conflicts

157. By Khaled Hussein on 2011-01-11

Refactored and Added unit tests

158. By Khaled Hussein on 2011-01-11

PEP8 Stuff :)

159. By Khaled Hussein on 2011-01-11

saio documentation changes

160. By Khaled Hussein on 2011-01-12

documentation change

161. By Khaled Hussein on 2011-01-12

merge trunk

Unmerged revisions

161. By Khaled Hussein on 2011-01-12

merge trunk

160. By Khaled Hussein on 2011-01-12

documentation change

159. By Khaled Hussein on 2011-01-11

saio documentation changes

158. By Khaled Hussein on 2011-01-11

PEP8 Stuff :)

157. By Khaled Hussein on 2011-01-11

Refactored and Added unit tests

156. By Khaled Hussein on 2011-01-11

fixing conflicts

155. By Khaled Hussein on 2011-01-11

fixed functional tests

154. By Khaled Hussein on 2011-01-10

setup.py config changes

153. By Khaled Hussein on 2011-01-10

Merged trunk

152. By Khaled Hussein on 2011-01-10

Fixed unit tests

=== modified file 'setup.py'
--- setup.py	2011-01-04 23:34:43 +0000
+++ setup.py	2011-01-07 16:38:27 +0000
@@ -89,7 +89,9 @@
             'auth=swift.auth.server:app_factory',
             ],
         'paste.filter_factory': [
-            'auth=swift.common.middleware.auth:filter_factory',
+            'auth=swift.common.middleware.auth1:filter_factory',
+            'devauth=swift.common.middleware.devauth:filter_factory',
+            'papiauth=swift.common.middleware.papiauth:filter_factory',
             'healthcheck=swift.common.middleware.healthcheck:filter_factory',
             'memcache=swift.common.middleware.memcache:filter_factory',
             'ratelimit=swift.common.middleware.ratelimit:filter_factory',

=== added file 'swift/common/middleware/auth1.py'
--- swift/common/middleware/auth1.py	1970-01-01 00:00:00 +0000
+++ swift/common/middleware/auth1.py	2011-01-07 16:38:27 +0000
@@ -0,0 +1,147 @@
+# Copyright (c) 2010 OpenStack, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from time import time
+
+from eventlet.timeout import Timeout
+from webob.exc import HTTPUnauthorized, HTTPNotFound
+
+from swift.common.bufferedhttp import http_connect_raw as http_connect
+from swift.common.utils import cache_from_env, split_path, TRUE_VALUES
+
+class Auth1(object):
+    """Auth Middleware that uses the dev auth server."""
+
+    def __init__(self, app, conf):
+        self.app = app
+        self.conf = conf
+        self.reseller_prefix = conf.get('reseller_prefix', 'AUTH').strip()
+        if self.reseller_prefix and self.reseller_prefix[-1] != '_':
+            self.reseller_prefix += '_'
+        self.auth_host = conf.get('ip', '127.0.0.1')
+        self.auth_port = int(conf.get('port', 11000))
+        self.ssl = conf.get('ssl', 'false').lower() in TRUE_VALUES
+        self.timeout = int(conf.get('node_timeout', 10))
+        self.delegated = int(conf.get('delegated', 0))
+
+    def __call__(self, env, start_response):
+        """
+        Accepts a standard WSGI application call and authenticates the request.
+        For an authenticated request, REMOTE_USER will be set to a comma
+        separated list of the user's groups. It'll also set X-Authorization
+        header to 'Proxy [Username]'. If it is running in a delegated mode, it
+        sets the X-Identity-Status header to 'Confirmed' if the token is valid,
+        or 'Indeterminate' if the token doesn't exist.
+
+        With a non-empty reseller prefix, acts as the definitive auth service
+        for just tokens and accounts that begin with that prefix, but will deny
+        requests outside this prefix if no other auth middleware overrides it.
+
+        With an empty reseller prefix, acts as the definitive auth service only
+        for tokens that validate to a non-empty set of groups. For all other
+        requests, acts as the fallback auth service when no other auth
+        middleware overrides it.
+        """
+
+        def custom_start_response(status, headers, exc_info=None):
+            if self.delegated:
+                headers.append(('WWW-Authenticate', "Basic realm='API Realm'"))
+            return start_response(status, headers, exc_info)
+
+        token = env.get('HTTP_X_AUTH_TOKEN', env.get('HTTP_X_STORAGE_TOKEN'))
+        if token and token.startswith(self.reseller_prefix):
+            # Note: Empty reseller_prefix will match all tokens.
+            # Attempt to auth my token with my auth server
+            user, groups = \
+                self.get_groups(token, memcache_client=cache_from_env(env))
+            if groups:
+                env['SWIFT_GROUPS'] = groups
+                env['HTTP_X_AUTHORIZATION'] = "Proxy " + user
+                if self.delegated:
+                    env['HTTP_X_IDENTITY_STATUS'] = "Confirmed"
+                # We know the proxy logs the token, so we augment it just a bit
+                # to also log the authenticated user.
+                env['HTTP_X_AUTH_TOKEN'] = '%s,%s' % (user, token)
+            else:
+                # Unauthorized token
+                return HTTPUnauthorized()(env, custom_start_response)
+        else:
+            env['HTTP_X_AUTHORIZATION'] = "Proxy"
+            if not token and self.delegated:
+                env['HTTP_X_IDENTITY_STATUS'] = "Indeterminate"
+
+            if self.reseller_prefix:
+                # With a non-empty reseller_prefix, I would like to be called
+                # back for anonymous access to accounts I know I'm the
+                # definitive auth for.
+                try:
+                    version, rest = split_path(env.get('PATH_INFO', ''),
+                                               1, 2, True)
+                except ValueError:
+                    return HTTPNotFound()(env, custom_start_response)
+
+        env['HTTP_AUTHORIZATION'] = "Basic dTpw"
+        return self.app(env, custom_start_response)
+        
+    def get_groups(self, token, memcache_client=None):
+        """
+        Get user and groups info for the given token.
+
+        If memcache_client is set, token credentials will be cached
+        appropriately.
+
+        With a cache miss, or no memcache_client, the configurated external
+        authentication server will be queried for the group information.
+
+        :param token: Token to validate and return a group string for.
+        :param memcache_client: Memcached client to use for caching token
+                                credentials; None if no caching is desired.
+        :returns: None if the token is invalid or a string for the username and
+                  a string containing a comma separated list of groups the 
+                  authenticated user is a member of. The first group in the 
+                  list is also considered a unique identifier for that user.
+        """
+        groups = None
+        key = '%s/token/%s' % (self.reseller_prefix, token)
+        cached_auth_data = memcache_client and memcache_client.get(key)
+        if cached_auth_data:
+            start, expiration, groups = cached_auth_data
+            if time() - start > expiration:
+                groups = None
+        if not groups:
+            with Timeout(self.timeout):
+                conn = http_connect(self.auth_host, self.auth_port, 'GET',
+                                    '/token/%s' % token, ssl=self.ssl)
+                resp = conn.getresponse()
+                resp.read()
+                conn.close()
+            if resp.status // 100 != 2:
+                return None
+            expiration = float(resp.getheader('x-auth-ttl'))
+            groups = resp.getheader('x-auth-groups')
+            if memcache_client:
+                memcache_client.set(key, (time(), expiration, groups),
+                                    timeout=expiration)
+        user = groups and groups.split(',', 1)[0] or ''
+        return user, groups
+
+def filter_factory(global_conf, **local_conf):
+    """Returns a WSGI filter app for use with paste.deploy."""
+    conf = global_conf.copy()
+    conf.update(local_conf)
+
+    def auth_filter(app):
+        return Auth1(app, conf)
+    return auth_filter

=== renamed file 'swift/common/middleware/auth.py' => 'swift/common/middleware/devauth.py'
--- swift/common/middleware/auth.py	2011-01-04 23:34:43 +0000
+++ swift/common/middleware/devauth.py	2011-01-07 16:38:27 +0000
@@ -13,14 +13,10 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from time import time
-
-from eventlet.timeout import Timeout
 from webob.exc import HTTPForbidden, HTTPUnauthorized, HTTPNotFound
 
-from swift.common.bufferedhttp import http_connect_raw as http_connect
 from swift.common.middleware.acl import clean_acl, parse_acl, referrer_allowed
-from swift.common.utils import cache_from_env, split_path, TRUE_VALUES
+from swift.common.utils import split_path, TRUE_VALUES
 
 
 class DevAuth(object):
@@ -41,110 +37,49 @@
         """
         Accepts a standard WSGI application call, authenticating the request
         and installing callback hooks for authorization and ACL header
-        validation. For an authenticated request, REMOTE_USER will be set to a
-        comma separated list of the user's groups.
-
-        With a non-empty reseller prefix, acts as the definitive auth service
-        for just tokens and accounts that begin with that prefix, but will deny
-        requests outside this prefix if no other auth middleware overrides it.
-
-        With an empty reseller prefix, acts as the definitive auth service only
-        for tokens that validate to a non-empty set of groups. For all other
-        requests, acts as the fallback auth service when no other auth
-        middleware overrides it.
+        validation. 
         """
-        token = env.get('HTTP_X_AUTH_TOKEN', env.get('HTTP_X_STORAGE_TOKEN'))
-        if token and token.startswith(self.reseller_prefix):
-            # Note: Empty reseller_prefix will match all tokens.
-            # Attempt to auth my token with my auth server
-            groups = \
-                self.get_groups(token, memcache_client=cache_from_env(env))
-            if groups:
-                env['REMOTE_USER'] = groups
-                user = groups and groups.split(',', 1)[0] or ''
-                # We know the proxy logs the token, so we augment it just a bit
-                # to also log the authenticated user.
-                env['HTTP_X_AUTH_TOKEN'] = '%s,%s' % (user, token)
+        groups = None
+        if 'SWIFT_GROUPS' in env:
+            groups = env['SWIFT_GROUPS']
+            env['REMOTE_USER'] = groups
+            self.authorize = self.dev_authorize
+            env['swift.clean_acl'] = clean_acl
+        elif 'swift.authorize' not in env:
+            self.authorize = self.empty_authorize
+
+        if self.reseller_prefix:
+            # With a non-empty reseller_prefix, I would like to be called
+            # back for anonymous access to accounts I know I'm the
+            # definitive auth for.
+            try:
+                version, rest = split_path(env.get('PATH_INFO', ''),
+                                           1, 2, True)
+            except ValueError:
+                return HTTPNotFound()(env, start_response)
+            if rest and rest.startswith(self.reseller_prefix):
+                # Handle anonymous access to accounts I'm the definitive
+                # auth for.
                 env['swift.authorize'] = self.authorize
                 env['swift.clean_acl'] = clean_acl
-            else:
-                # Unauthorized token
-                if self.reseller_prefix:
-                    # Because I know I'm the definitive auth for this token, I
-                    # can deny it outright.
-                    return HTTPUnauthorized()(env, start_response)
-                # Because I'm not certain if I'm the definitive auth for empty
-                # reseller_prefixed tokens, I won't overwrite swift.authorize.
-                elif 'swift.authorize' not in env:
-                    env['swift.authorize'] = self.denied_response
-        else:
-            if self.reseller_prefix:
-                # With a non-empty reseller_prefix, I would like to be called
-                # back for anonymous access to accounts I know I'm the
-                # definitive auth for.
-                try:
-                    version, rest = split_path(env.get('PATH_INFO', ''),
-                                               1, 2, True)
-                except ValueError:
-                    return HTTPNotFound()(env, start_response)
-                if rest and rest.startswith(self.reseller_prefix):
-                    # Handle anonymous access to accounts I'm the definitive
-                    # auth for.
-                    env['swift.authorize'] = self.authorize
-                    env['swift.clean_acl'] = clean_acl
-                # Not my token, not my account, I can't authorize this request,
-                # deny all is a good idea if not already set...
-                elif 'swift.authorize' not in env:
-                    env['swift.authorize'] = self.denied_response
-            # Because I'm not certain if I'm the definitive auth for empty
-            # reseller_prefixed accounts, I won't overwrite swift.authorize.
+            # Not my token, not my account, I can't authorize this request,
+            # deny all is a good idea if not already set...
             elif 'swift.authorize' not in env:
-                env['swift.authorize'] = self.authorize
-                env['swift.clean_acl'] = clean_acl
+                env['swift.authorize'] = self.denied_response
+        # Because I'm not certain if I'm the definitive auth for empty
+        # reseller_prefixed accounts, I won't overwrite swift.authorize.
+        elif 'swift.authorize' not in env:
+            env['swift.authorize'] = self.authorize
+            env['swift.clean_acl'] = clean_acl
         return self.app(env, start_response)
 
-    def get_groups(self, token, memcache_client=None):
-        """
-        Get groups for the given token.
-
-        If memcache_client is set, token credentials will be cached
-        appropriately.
-
-        With a cache miss, or no memcache_client, the configurated external
-        authentication server will be queried for the group information.
-
-        :param token: Token to validate and return a group string for.
-        :param memcache_client: Memcached client to use for caching token
-                                credentials; None if no caching is desired.
-        :returns: None if the token is invalid or a string containing a comma
-                  separated list of groups the authenticated user is a member
-                  of. The first group in the list is also considered a unique
-                  identifier for that user.
-        """
-        groups = None
-        key = '%s/token/%s' % (self.reseller_prefix, token)
-        cached_auth_data = memcache_client and memcache_client.get(key)
-        if cached_auth_data:
-            start, expiration, groups = cached_auth_data
-            if time() - start > expiration:
-                groups = None
-        if not groups:
-            with Timeout(self.timeout):
-                conn = http_connect(self.auth_host, self.auth_port, 'GET',
-                                    '/token/%s' % token, ssl=self.ssl)
-                resp = conn.getresponse()
-                resp.read()
-                conn.close()
-            if resp.status // 100 != 2:
-                return None
-            expiration = float(resp.getheader('x-auth-ttl'))
-            groups = resp.getheader('x-auth-groups')
-            if memcache_client:
-                memcache_client.set(key, (time(), expiration, groups),
-                                    timeout=expiration)
-        return groups
-
-    def authorize(self, req):
+    def empty_authorize(self, req):
+        if 'x_identity_status' in req.headers and \
+           req.headers['x_identity_status'] == 'Indeterminate':
+            return self.denied_response(req)
+        return None
+
+    def dev_authorize(self, req):
         """
         Returns None if the request is authorized to continue or a standard
         WSGI response callable if not.
@@ -177,10 +112,13 @@
         Returns a standard WSGI response callable with the status of 403 or 401
         depending on whether the REMOTE_USER is set or not.
         """
+        headers = [('www-authenticate', 'delegated')]
         if req.remote_user:
-            return HTTPForbidden(request=req)
+            resp = HTTPForbidden(headers=headers, request=req)
         else:
-            return HTTPUnauthorized(request=req)
+            resp = HTTPUnauthorized(headers=headers, request=req)
+
+        return resp
 
 
 def filter_factory(global_conf, **local_conf):

=== added file 'swift/common/middleware/papiauth.py'
--- swift/common/middleware/papiauth.py	1970-01-01 00:00:00 +0000
+++ swift/common/middleware/papiauth.py	2011-01-07 16:38:27 +0000
@@ -0,0 +1,63 @@
+# Copyright (c) 2010 OpenStack, LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from webob.exc import HTTPUseProxy, HTTPUnauthorized
+
+from swift.common.utils import TRUE_VALUES
+
+
+class PAPIAuth(object):
+    """Auth Middleware that uses the dev auth server."""
+
+    def __init__(self, app, conf):
+        self.app = app
+        self.conf = conf
+        self.reseller_prefix = conf.get('reseller_prefix', 'AUTH').strip()
+        if self.reseller_prefix and self.reseller_prefix[-1] != '_':
+            self.reseller_prefix += '_'
+        self.auth_host = conf.get('ip', '127.0.0.1')
+        self.auth_port = int(conf.get('port', 11000))
+        self.auth_pass = conf.get('pass', 'dTpw')
+        self.ssl = conf.get('ssl', 'false').lower() in TRUE_VALUES
+        self.timeout = int(conf.get('node_timeout', 10))
+
+    def __call__(self, env, start_response):
+        # Make sure that the user has been authenticated by the Auth Service
+        if 'HTTP_X_AUTHORIZATION' not in env:
+            proxy_location = 'http://' + self.auth_host + ':' + \
+                str(self.auth_port) + '/'
+            return HTTPUseProxy(location=proxy_location)(env, start_response)
+
+        # Authenticate the Auth component itself. 
+        headers = [('www-authenticate', 'Basic realm="swift"')]
+        if 'HTTP_AUTHORIZATION' not in env:
+            return HTTPUnauthorized(headers=headers)(env, start_response)
+        else:
+            auth_type, encoded_creds = env['HTTP_AUTHORIZATION'].split(None, 1)
+            if encoded_creds != self.auth_pass:
+                return HTTPUnauthorized(headers=headers)(env, start_response)
+
+        return self.app(env, start_response)
+
+
+def filter_factory(global_conf, **local_conf):
+    """Returns a WSGI filter app for use with paste.deploy."""
+    conf = global_conf.copy()
+    conf.update(local_conf)
+
+    def auth_filter(app):
+        return PAPIAuth(app, conf)
+    return auth_filter
+