Password logging

Bug #1004114 reported by Gabriel Hurley
56
This bug affects 7 people
Affects Status Importance Assigned to Milestone
OpenStack Dashboard (Horizon)
Fix Released
High
Gabriel Hurley
OpenStack Identity (keystone)
Fix Released
High
Dolph Mathews
OpenStack Security Notes
Fix Released
Medium
Abu Shohel Ahmed
python-keystoneclient
Fix Released
Medium
Brant Knudson

Bug Description

When the log level is set to DEBUG, keystoneclient's full-request logging mechanism kicks in, exposing plaintext passwords, etc.

This bug is mostly out of the scope of Horizon, however Horizon can also be more secure in this regard. We should make sure that wherever we *are* handling sensitive data we use Django's error report filtering mechanisms so they don't appear in tracebacks, etc. (https://docs.djangoproject.com/en/dev/howto/error-reporting/#filtering-error-reports)

Keystone may also want to look at respecting such annotations in their logging mechanism, i.e. if Django were properly annotating these data objects, keystoneclient could check for those annotations and properly sanitize the log output.

If not this exact mechanism, then something similar would be wise.

For the time being, it's also worth documenting in both projects that a log level of DEBUG will log passwords in plain text.

Tags: security
description: updated
Joseph Heck (heckj)
Changed in keystone:
status: New → Confirmed
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to horizon (master)

Fix proposed to branch: master
Review: https://review.openstack.org/7773

Changed in horizon:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to horizon (master)

Reviewed: https://review.openstack.org/7773
Committed: http://github.com/openstack/horizon/commit/f986a631a25c8fa547d07d2fae4bd2b4ac1c2b9a
Submitter: Jenkins
Branch: master

commit f986a631a25c8fa547d07d2fae4bd2b4ac1c2b9a
Author: Gabriel Hurley <email address hidden>
Date: Thu May 24 15:25:35 2012 -0700

    Make sure Horizon is treating passwords securely.

    * Applies the sensitive_post_parameters and sensitive_variables
      decorators to functions that handle sensitive data.
    * Defines a custom Exception Filter class to provide some added
      security.
    * Adds notes on logging to the docs.

    Fixes bug 1004114 for Horizon.

    Change-Id: I13ac91d91e0ed2322cc61633b02455cfed39fdcd

Changed in horizon:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in horizon:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/9935

Changed in keystone:
assignee: nobody → Dolph Mathews (dolph)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/9935
Committed: http://github.com/openstack/keystone/commit/0abf6ba254638471a367cfccef65a1b9e0a70ef2
Submitter: Jenkins
Branch: master

commit 0abf6ba254638471a367cfccef65a1b9e0a70ef2
Author: Dolph Mathews <email address hidden>
Date: Tue Jul 17 16:23:49 2012 -0500

    Debug output may include passwords (bug 1004114)

    Change-Id: If0a7704ff578162d6b7fa8b68c0e0ed37e72cb73

Changed in keystone:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in keystone:
milestone: none → folsom-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in horizon:
milestone: folsom-2 → 2012.2
Thierry Carrez (ttx)
Changed in keystone:
milestone: folsom-3 → 2012.2
Dolph Mathews (dolph)
Changed in python-keystoneclient:
status: New → Triaged
importance: Undecided → Medium
Thierry Carrez (ttx)
tags: added: security
Numero 8 (numero-8)
Changed in python-keystoneclient:
assignee: nobody → Numero 8 (numero-8)
Revision history for this message
Numero 8 (numero-8) wrote :

I'm working on it...
Review in progress.

Revision history for this message
Numero 8 (numero-8) wrote :

Still working on it. See https://review.openstack.org/#/c/32343 .

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-keystoneclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/33532

Changed in python-keystoneclient:
status: Triaged → In Progress
Revision history for this message
Numero 8 (numero-8) wrote :

Waiting for a feedback following last code review. See https://review.openstack.org/#/c/33532/

Revision history for this message
Numero 8 (numero-8) wrote :

Could any one propose a review of patch set #3 for this bug?

See here: https://review.openstack.org/#/c/33532/

Thank you.

Revision history for this message
Matthew Thode (prometheanfire) wrote :

anyone able to review this patchset? https://review.openstack.org/#/c/33532/

Changed in python-keystoneclient:
assignee: Numero 8 (numero-8) → Adam Young (ayoung)
Changed in python-keystoneclient:
assignee: Adam Young (ayoung) → Numero 8 (numero-8)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/42467

Numero 8 (numero-8)
Changed in python-keystoneclient:
assignee: Numero 8 (numero-8) → nobody
Changed in python-keystoneclient:
assignee: nobody → Sergio Cazzolato (sergio-j-cazzolato)
Changed in python-keystoneclient:
assignee: Sergio Cazzolato (sergio-j-cazzolato) → nobody
Revision history for this message
Wei Wang (damon-devops) wrote :

Numero 8's patch almost resolve this bug, maybe I can continue it.

Changed in python-keystoneclient:
assignee: nobody → Wei Wang (damon-devops)
Revision history for this message
gordon chung (chungg) wrote :

might make sense to just pull/copy mask_password code from oslo-incubator log.py to mask password... worked for me.

Revision history for this message
Wei Wang (damon-devops) wrote :

Saeme to @Dolph's commit here:https://review.openstack.org/#/c/42467/2//COMMIT_MSG

the need for patch should be re-considered.

Revision history for this message
Julie Pichon (jpichon) wrote :

It seems to me the patch is still necessary, setting a service to "debug" mode to temporarily debug an issue on a production system shouldn't give visibility into user passwords.

Revision history for this message
Doug Chivers (doug-chivers) wrote :

+1

Revision history for this message
Matt Fischer (mfisch) wrote :

This is still an issue in Icehouse.

Revision history for this message
Nathan Kinder (nkinder) wrote :

There is an outstanding patch to address this for Juno, and I think we should look at proposing it for backport (if it's not intrusive) once it is accepted:

    https://review.openstack.org/101792

Revision history for this message
Dolph Mathews (dolph) wrote :

Nathan: keystoneclient doesn't get named releases like the services do, so there wouldn't be anything to backport. You should be able to run the latest version of keystoneclient (including middleware.auth_token) with any supported service release, however.

Revision history for this message
David Stanek (dstanek) wrote :

That patch needs a little work to be approved. I'll upload a new patch to address the outstanding comments unless someone i already working on it.

Changed in python-keystoneclient:
assignee: Wei Wang (damon-devops) → David Stanek (dstanek)
Changed in python-keystoneclient:
assignee: David Stanek (dstanek) → Brant Knudson (blk-u)
Changed in python-keystoneclient:
assignee: Brant Knudson (blk-u) → Jamie Lennox (jamielennox)
Changed in python-keystoneclient:
assignee: Jamie Lennox (jamielennox) → Nathan Kinder (nkinder)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-keystoneclient (master)

Reviewed: https://review.openstack.org/101792
Committed: https://git.openstack.org/cgit/openstack/python-keystoneclient/commit/?id=0e9ecaa1547306f7af6527126fb88f8151908498
Submitter: Jenkins
Branch: master

commit 0e9ecaa1547306f7af6527126fb88f8151908498
Author: Jamie Lennox <email address hidden>
Date: Wed Jun 18 10:22:10 2014 +1000

    Don't log sensitive auth data

    Add the ability to turn off logging from the session object and then
    handle logging of auth requests within their own sections. This is a
    very simplistic ability to completely disable logging. Logging more
    filtered debugging can be added later.

    This new ability is utilized in this patch to prevent logging of
    requests that include passwords. This covers authenticate, password
    change, and user update requests that include passwords.

    SecurityImpact
    Change-Id: I3dabb94ab047e86b8730e73416c1a1c333688489
    Closes-Bug: #1004114
    Closes-Bug: #1327019

Changed in python-keystoneclient:
status: In Progress → Fix Committed
Revision history for this message
Nathan Kinder (nkinder) wrote :

We should write an OSSN for this so people are aware of the fact that passwords for users will be logged in Horizon if debug logging is enabled. Now that a keystoneclient patch has been merged, we will soon have a release that doesn't log passwords anymore. We should recommend using the newer keystoneclient as soon as it's available.

Changed in ossn:
importance: Undecided → Medium
Dolph Mathews (dolph)
Changed in python-keystoneclient:
milestone: none → 0.10.1
status: Fix Committed → Fix Released
Revision history for this message
Dolph Mathews (dolph) wrote :

keystoneclient 0.10.1 was released with the password logging fix:

  https://launchpad.net/python-keystoneclient/+milestone/0.10.1

Revision history for this message
Brant Knudson (blk-u) wrote :

I tried this with the keystone command and passwords and tokens are still being printed.

$ keystone --debug user-list
DEBUG:keystoneclient.session:REQ: curl -i -X POST http://localhost:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "demo", "passwordCredentials": {"username": "admin", "password": "mypassword"}}}'
...
RESP BODY: {"access": {"token": {"issued_at": "2014-07-28T19:08:05.637184", "expires": "2014-07-28T20:08:05Z", "id": "PKIZ_<LONG-TOKEN-IN-CLEAR>", ...
...
DEBUG:keystoneclient.session:REQ: curl -i -X GET http://192.168.122.176:35357/v2.0/users -H "User-Agent: python-keystoneclient" -H "X-Auth-Token: PKIZ_<LONG-TOKEN-IN-CLEAR>"

Changed in python-keystoneclient:
status: Fix Released → Confirmed
Revision history for this message
Brant Knudson (blk-u) wrote :

I wasn't running with the fix so was getting the passwords. With the fix the token is still printed.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-keystoneclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/110117

Changed in python-keystoneclient:
assignee: Nathan Kinder (nkinder) → Brant Knudson (blk-u)
status: Confirmed → In Progress
Revision history for this message
Dolph Mathews (dolph) wrote :

Can we just remove the curl examples? I don't understand their utility if they're not valid.

Changed in ossn:
assignee: nobody → Abu Shohel Ahmed (shohel-csdu)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-keystoneclient (master)

Reviewed: https://review.openstack.org/110117
Committed: https://git.openstack.org/cgit/openstack/python-keystoneclient/commit/?id=605577192d7158ecf40bd9a94b7cf3acc2ce1c95
Submitter: Jenkins
Branch: master

commit 605577192d7158ecf40bd9a94b7cf3acc2ce1c95
Author: Brant Knudson <email address hidden>
Date: Mon Jul 28 14:34:53 2014 -0500

    Redact tokens in request headers

    Tokens shouldn't be logged since a token could be gathered from a
    log file and used. The client was logging the X-Auth-Token and
    X-Subject-Token request headers. With this change, the X-Auth-Token
    and X-Subject-Token are shown as "TOKEN_REDACTED".

    Also, the "Authentication" header is also redacted.

    This is for security hardening.

    SecurityImpact

    Closes-Bug: #1004114
    Closes-Bug: #1327019

    Change-Id: I1edc3821ed028471102cc9b95eb9f3b54c9e2778

Changed in python-keystoneclient:
status: In Progress → Fix Committed
Changed in ossn:
status: New → In Progress
Dolph Mathews (dolph)
Changed in python-keystoneclient:
milestone: 0.10.1 → 0.11.0
Dolph Mathews (dolph)
Changed in python-keystoneclient:
status: Fix Committed → Fix Released
Revision history for this message
Nathan Kinder (nkinder) wrote :

This was published as OSSN-0024:

    https://wiki.openstack.org/wiki/OSSN/OSSN-0024

Changed in ossn:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.