Revoke API calls non-existant method in revoke map syncronize

Bug #1289935 reported by Morgan Fainberg
38
This bug affects 5 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
Critical
Adam Young
keystone (Ubuntu)
Fix Released
Critical
Corey Bryant
Trusty
Fix Released
Critical
Corey Bryant

Bug Description

The "revoke_api" calls a non-existent method on the revoke tree object during the synchronize method. This results in a non-recoverable error in checking validity of a token if there are expired revocation events.

Code block in question:

http://git.openstack.org/cgit/openstack/keystone/tree/keystone/contrib/revoke/core.py?id=a240705b07b852616e39a2b93253f0a9a09a3ef9#n79

        with self._store.get_lock(_TREE_KEY):
            for e in self._current_events:
                if e.revoked_at < cutoff:
                    self.revoke_map.remove(e)
                    self._current_events.remove(e)
                else:
                    break

The code should call self.revoke_map.remove_event(e) not self.revoke_map.remove(e).

Example traceback:

2014-03-08 20:20:59.338 TRACE keystone.common.wsgi TypeError: object of type 'NoneType' has no len()
2014-03-08 20:20:59.338 TRACE keystone.common.wsgi
2014-03-08 20:20:59.340 INFO eventlet.wsgi.server [-] 172.16.28.1 - - [08/Mar/2014 20:20:59] "POST /v2.0/tokens HTTP/1.1" 400 239 0.004711
2014-03-08 20:20:59.351 DEBUG keystone.middleware.core [-] Auth token not in the request header. Will not build auth context. from (pid=14327) process_request /opt/stack/keystone/keystone/middleware/core.py:253
2014-03-08 20:20:59.352 DEBUG keystone.common.wsgi [-] arg_dict: {} from (pid=14327) __call__ /opt/stack/keystone/keystone/common/wsgi.py:180
2014-03-08 20:20:59.353 ERROR keystone.common.wsgi [-] object of type 'NoneType' has no len()
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi Traceback (most recent call last):
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/common/wsgi.py", line 205, in __call__
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi result = method(context, **params)
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/openstack/common/versionutils.py", line 102, in wrapped
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi return func(*args, **kwargs)
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/controllers.py", line 97, in authenticate
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi context, auth)
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/controllers.py", line 255, in _authenticate_local
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi if len(username) > CONF.max_param_size:
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi TypeError: object of type 'NoneType' has no len()
2014-03-08 20:20:59.353 TRACE keystone.common.wsgi
2014-03-08 20:20:59.355 INFO eventlet.wsgi.server [-] 172.16.28.1 - - [08/Mar/2014 20:20:59] "POST /v2.0/tokens HTTP/1.1" 400 239 0.004078
2014-03-08 20:20:59.385 DEBUG keystone.common.wsgi [-] arg_dict: {} from (pid=14327) __call__ /opt/stack/keystone/keystone/common/wsgi.py:180
2014-03-08 20:20:59.386 INFO eventlet.wsgi.server [-] 172.16.28.100 - - [08/Mar/2014 20:20:59] "GET / HTTP/1.1" 300 1103 0.001378
2014-03-08 20:20:59.401 DEBUG keystone.middleware.core [-] Auth token not in the request header. Will not build auth context. from (pid=14327) process_request /opt/stack/keystone/keystone/middleware/core.py:253
2014-03-08 20:20:59.403 DEBUG keystone.common.wsgi [-] arg_dict: {} from (pid=14327) __call__ /opt/stack/keystone/keystone/common/wsgi.py:180
2014-03-08 20:20:59.412 DEBUG keystone.notifications [-] CADF Event: {'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 'initiator': {'typeURI': 'service/security/account/user', 'host': {'agent': 'python-requests/1.2.3 CPython/2.7.5+ Linux/3.11.0-12-generic', 'address': '172.16.28.100'}, 'id': 'openstack:b0d57b38-6f65-43aa-b0ef-b807db297e5b', 'name': u'5b55216e7b1742978dca4ce4f721a6d3'}, 'target': {'typeURI': 'service/security/account/user', 'id': 'openstack:006ecd17-f59d-4bc4-9fb5-cde076e7607c'}, 'observer': {'typeURI': 'service/security', 'id': 'openstack:5b7eecb3-de9b-486c-9683-11d50d965cf8'}, 'eventType': 'activity', 'eventTime': '2014-03-08T19:20:59.412018+0000', 'action': 'authenticate', 'outcome': 'pending', 'id': 'openstack:41e1caa6-4e8d-47f9-8a87-3e5d23c2e22d'} from (pid=14327) _send_audit_notification /opt/stack/keystone/keystone/notifications.py:289
2014-03-08 20:20:59.447 DEBUG keystone.notifications [-] CADF Event: {'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 'initiator': {'typeURI': 'service/security/account/user', 'host': {'agent': 'python-requests/1.2.3 CPython/2.7.5+ Linux/3.11.0-12-generic', 'address': '172.16.28.100'}, 'id': 'openstack:b0d57b38-6f65-43aa-b0ef-b807db297e5b', 'name': u'5b55216e7b1742978dca4ce4f721a6d3'}, 'target': {'typeURI': 'service/security/account/user', 'id': 'openstack:86370275-85d2-4243-bb59-d6c9d93d329c'}, 'observer': {'typeURI': 'service/security', 'id': 'openstack:ea11d624-61f7-4dbb-a6af-0317dfeb5770'}, 'eventType': 'activity', 'eventTime': '2014-03-08T19:20:59.446496+0000', 'action': 'authenticate', 'outcome': 'success', 'id': 'openstack:5874fedc-6212-4367-a842-6ac1ac51015c'} from (pid=14327) _send_audit_notification /opt/stack/keystone/keystone/notifications.py:289
2014-03-08 20:20:59.538 INFO eventlet.wsgi.server [-] 172.16.28.100 - - [08/Mar/2014 20:20:59] "POST /v2.0/tokens HTTP/1.1" 200 9140 0.136870
2014-03-08 20:20:59.543 DEBUG keystone.middleware.core [-] RBAC: auth_context: {'project_id': u'8d9ffd4e5688425caea13f16473c3e64', 'user_id': u'5b55216e7b1742978dca4ce4f721a6d3', 'roles': [u'_member_', u'admin']} from (pid=14327) process_request /opt/stack/keystone/keystone/middleware/core.py:263
2014-03-08 20:20:59.545 DEBUG keystone.common.wsgi [-] arg_dict: {'token_id': u'd5f1e4259de4c4449dc8b4638e6ec0f7'} from (pid=14327) __call__ /opt/stack/keystone/keystone/common/wsgi.py:180
2014-03-08 20:20:59.545 DEBUG keystone.common.controller [-] RBAC: Authorizing identity:validate_token(token_id=d5f1e4259de4c4449dc8b4638e6ec0f7) from (pid=14327) _build_policy_check_credentials /opt/stack/keystone/keystone/common/controller.py:40
2014-03-08 20:20:59.546 DEBUG keystone.common.controller [-] RBAC: using auth context from the request environment from (pid=14327) _build_policy_check_credentials /opt/stack/keystone/keystone/common/controller.py:45
2014-03-08 20:20:59.546 DEBUG keystone.policy.backends.rules [-] enforce identity:validate_token: {'project_id': u'8d9ffd4e5688425caea13f16473c3e64', 'user_id': u'5b55216e7b1742978dca4ce4f721a6d3', 'roles': [u'_member_', u'admin']} from (pid=14327) enforce /opt/stack/keystone/keystone/policy/backends/rules.py:100
2014-03-08 20:20:59.547 DEBUG keystone.openstack.common.policy [-] Rule identity:validate_token will be now enforced from (pid=14327) enforce /opt/stack/keystone/keystone/openstack/common/policy.py:258
2014-03-08 20:20:59.548 DEBUG keystone.common.controller [-] RBAC: Authorization granted from (pid=14327) inner /opt/stack/keystone/keystone/common/controller.py:137
2014-03-08 20:20:59.551 DEBUG keystone.common.kvs.core [-] KVS lock acquired for: os-revoke-tree from (pid=14327) acquire /opt/stack/keystone/keystone/common/kvs/core.py:375
2014-03-08 20:20:59.552 DEBUG keystone.common.kvs.core [-] KVS lock released for: os-revoke-tree from (pid=14327) release /opt/stack/keystone/keystone/common/kvs/core.py:394
2014-03-08 20:20:59.553 ERROR keystone.common.wsgi [-] 'RevokeTree' object has no attribute 'remove'
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi Traceback (most recent call last):
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/common/wsgi.py", line 205, in __call__
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi result = method(context, **params)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/openstack/common/versionutils.py", line 102, in wrapped
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi return func(*args, **kwargs)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/common/controller.py", line 138, in inner
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi return f(self, context, *args, **kwargs)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/controllers.py", line 411, in validate_token
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi return self.token_provider_api.validate_v2_token(token_id, belongs_to)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/provider.py", line 137, in validate_v2_token
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi self.check_revocation_v2(token)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/token/provider.py", line 130, in check_revocation_v2
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi self.revoke_api.check_token(token_values)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/contrib/revoke/core.py", line 190, in check_token
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi self._cache.synchronize_revoke_map(self.driver)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi File "/opt/stack/keystone/keystone/contrib/revoke/core.py", line 79, in synchronize_revoke_map
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi self.revoke_map.remove(e)
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi AttributeError: 'RevokeTree' object has no attribute 'remove'
2014-03-08 20:20:59.553 TRACE keystone.common.wsgi

Related branches

Changed in keystone:
importance: Undecided → Critical
status: New → Triaged
assignee: nobody → Morgan Fainberg (mdrnstm)
milestone: none → icehouse-rc1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/79174

Changed in keystone:
status: Triaged → In Progress
Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → Adam Young (ayoung)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/79174
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=3035a6b394ba4b460d9ea18409fa0cb87c86d38d
Submitter: Jenkins
Branch: master

commit 3035a6b394ba4b460d9ea18409fa0cb87c86d38d
Author: Morgan Fainberg <email address hidden>
Date: Sat Mar 8 21:57:51 2014 -0800

    Call an existing method in sync cache for revoke events

    The cache used for synchronizing the revocation tree across
    green threads had an issue where it was calling a non-existant
    method ``remove`` instead of ``remove_event``. The correct method
    is now being called and an expanded test to exercise the synchronize
    method has been added.

    Change-Id: I3fe47fa51f88aab89480831b2d95746319f82ceb
    Closes-Bug: 1289935

Changed in keystone:
status: In Progress → Fix Committed
Revision history for this message
Chris J Arges (arges) wrote :

This also affects the keystone version in Trusty.

Changed in keystone (Ubuntu):
importance: Undecided → Critical
Changed in keystone (Ubuntu):
status: New → Confirmed
assignee: nobody → Corey Bryant (corey.bryant)
James Page (james-page)
Changed in keystone (Ubuntu Trusty):
status: Confirmed → In Progress
James Page (james-page)
Changed in keystone (Ubuntu Trusty):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package keystone - 1:2014.1~b3-0ubuntu3

---------------
keystone (1:2014.1~b3-0ubuntu3) trusty; urgency=medium

  * d/p/revoke-api.patch: Add upstream patch to resolve critical issue with
    token revocation (LP: #1289935).
  * d/keystone.postinst: Ensure db_sync is only run when the default sqlite
    connection is configured (LP: #1290423).
 -- Corey Bryant <email address hidden> Wed, 12 Mar 2014 23:20:05 -0500

Changed in keystone (Ubuntu Trusty):
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in keystone:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in keystone:
milestone: icehouse-rc1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.