system-based authorization broken in gnome-keyring: NoOptionError: No option 'consumer_key' in section: '1'

Here's the serialize code for what is stored

        parser.add_section(CREDENTIALS_FILE_VERSION)
        parser.set(CREDENTIALS_FILE_VERSION,
                   'consumer_key', self.consumer.key)
        parser.set(CREDENTIALS_FILE_VERSION,
                   'consumer_secret', self.consumer.secret)
        parser.set(CREDENTIALS_FILE_VERSION,
                   'access_token', self.access_token.key)
        parser.set(CREDENTIALS_FILE_VERSION,
                   'access_secret', self.access_token.secret)
        parser.write(writable_file)

so it executes the first line fine, but the rest don't have the desired
effect.

I'm not sure what would cause the other statements to have
no effect.

The write() must have an effect, otherwise you would get nothing,
so either there is something odd going on with the ConfigParserClass
or we are getting the equivalent of short read/write (the writeable_file
isn't a real file here, just an in memory one, so it's not that exactly)

There is some oddness with a credential_save_failed that could be
involved.

Adding an upstream task for their input.

Thanks,

James

As far as I can tell this means the second run of any lplib client will fail.

parser.write() sent to a stringio does seem to produce the right kind of thing:

'[1]\nconsumer_key = System-wide: Ubuntu (myhostname)\nconsumer_secret = \naccess_token = uuuuuuuuuuuuuuuu\naccess_secret = uuuuuuuuuuuuuuuuuuuuuuuuuuuuuu\n\n'

Apparently the Python keyring module, or something it calls, assumes passwords are a single line and it truncates them at the first \n: if I immediately try to get it back then

keyring.get_password('launchpadlib', 'System-wide: Ubuntu (grace)@https://api.launchpad.net/')
'[1]'

however a simple interactive use of the keyring module doesn't hit this:

In [4]: keyring.set_password('test', 'test', 'bite\nme')

In [5]: keyring.get_password('test', 'test')
Out[5]: 'bite\nme'

In [6]: print _5
bite
me

I know Leonard has observed some odd behaviour with python-keyring (and esp. the gnomekeyring module) some weeks ago [0], so this might be related.

[0] https://bitbucket.org/kang/python-keyring-lib/issue/40/failures-happen-at-random-points-in-the

> [0] https://bitbucket.org/kang/python-keyring-lib/issue/40/failures-
> happen-at-random-points-in-the

I wonder if, for Natty, launchpadlib should disable keyring
integration if it's really as flaky as that post and this bug
suggests.

I think one thing that's happening here is that the keyring gui strongly assumes passwords fit on a single line, and it possibly also implicitly saves them when you close the window. So if you open it up to see what launchpadlib is saving in there, it will break them.

One option would be to just use json or some other serialization that will not include newlines.

On Tue, Apr 19, 2011 at 02:10:25AM -0000, Martin Pool wrote:
> I think one thing that's happening here is that the keyring gui strongly
> assumes passwords fit on a single line, and it possibly also implicitly
> saves them when you close the window. So if you open it up to see what
> launchpadlib is saving in there, it will break them.

That's not the root of the problem I'm experiencing. I looked in seahorse
only *because* saving credentials wasn't working right. Any time I use a
launchpadlib-based tool on natty, I have to first open seahorse and delete
the previously-stored invalid credentials, then re-authenticate to openid.
The host-based credentials saving is worse than useless to me at present; at
least if it weren't saving the credentials I could skip the step of opening
seahorse to delete creds.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

No, I agree it's not the root cause. I don't know what the real root is.

I will take a quick stab at this by:

- catching and gracefully handling retrieval errors
- storing a new format that is always all on one line

I'm not sure that will fix it, but it seems connected to line breaks so it might.

Many dupes in bzr too, eg bug 762065, and it's biting me every time I use it.

My comment #7 was misguided; I'm sure seahorse isn't breaking it, but also fairly sure something else involved in the stack believes passwords don't contain newlines. Which is pretty reasonable after all.

Testing this is a bit difficult because lazr.restful installs a moderately insane pth file that means you cannot possibly override the

import sys,types,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('lazr',)); ie = os.path.exists(os.path.join(p,'__init__.py')); m = not ie and sys.modules.setdefault('lazr',types.ModuleType('lazr')); mp = (m or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p)

This problem is still ongoing in oneiric, and to top it all off, seahorse now fails to run at all for me to let me remove the broken key. This makes all launchpadlib apps completely unusable for me.

I have worked around this by deleting the whole keyring. (I actually
wasn't motivated by this bug but rather other bad craziness related to
wifi, but it apparently fixed this.) What a mess. I agree it should
still be fixed.

This and bug 872853 might be related.

Escalated by Bryce for the Ubuntu community.

PPA with the fix available at https://launchpad.net/~bac/+archive/ppa

Hi Brad,

With the package from the ppa, I got the following error:
...

Traceback (most recent call last):
  File "/home/fabrice/bin/requestsync", line 336, in <module>
    main()
  File "/home/fabrice/bin/requestsync", line 149, in main
    Launchpad.login(service=options.lpinstance)
  File "/usr/lib/python2.7/dist-packages/ubuntutools/lp/lpapicache.py", line 61, in login
    version=api_version)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/launchpad.py", line 539, in login_with
    credential_save_failed, version)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/launchpad.py", line 342, in _authorize_token_and_login
    authorization_engine.unique_consumer_id)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/credentials.py", line 287, in load
    return self.do_load(unique_key)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/credentials.py", line 339, in do_load
    credentials = Credentials.from_string(credential_string)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/credentials.py", line 102, in from_string
    value = b64decode(value)
  File "/usr/lib/python2.7/base64.py", line 76, in b64decode
    raise TypeError(msg)
TypeError: Incorrect padding

/home/fabrice/bin/requestsync is a link to /usr/bin/requestsync, so it shouldn't comes from there.

Waiting for feedback from others.

Thanks,
Fabrice

Hi Fabrice,

I changed the way the credentials string is encoded before storing in the keyring while attempting to provide backward compatibility. It looks like the substring I expected to see for an older password was not there, which would have avoided the decode attempt.

Could you use seahorse to open the keyring and look for the 'network password' entry that corresponds to 'api.launchpad.net' and look at the password? After sanitizing your credentials you can add it to the bug report or send it in an encrypted email directly to me if you prefer.

Thanks,
Brad

Hi Brad,

2011/12/1 Brad Crittenden <email address hidden>:
> Could you use seahorse to open the keyring and look for the 'network
> password' entry that corresponds to 'api.launchpad.net' and look at the
> password?  After sanitizing your credentials you can add it to the bug

I actually don't have any "network password' entry in seahorse (I
checked an old oneiric VM and it's there), so that may be the reason
why it's failing.
I tried to create it manually, but it doesn't seem to let me create a
'network credential' type entry.

and manage-credentials does not exists anymore, so I don't know how to
recreate this entry.

Thanks for your help!

Fabrice

Fabrice the issue you raise has been filed as bug 900307.

thanks!

2011/12/5 Brad Crittenden <email address hidden>:
> Fabrice the issue you raise has been filed as bug 900307.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/745801
>
> Title:
>  system-based authorization broken in gnome-keyring: NoOptionError: No
>  option 'consumer_key' in section: '1'
>
> Status in Launchpad web services client library:
>  Fix Committed
> Status in “python-launchpadlib” package in Ubuntu:
>  Triaged
> Status in “python-launchpadlib” source package in Natty:
>  Triaged
>
> Bug description:
>  Binary package hint: python-launchpadlib
>
>  Running lp:svammel to do a dry run mass bugfiling against LP, I get
>  prompted to authorize my system to connect to launchpad, which is
>  neat.  But on a subsequent invocation of the tool, it fails with:
>
>   Traceback (most recent call last):
>    File "file-failures.py", line 50, in <module>
>      init(args.serviceroot, 'testing', '~/.launchpadlib/cache/')
>    File "/home/vorlon/devel/linaro/svammel/config.py", line 96, in init
>      set_launchpad(service_root, appid, cachedir)
>    File "/home/vorlon/devel/linaro/svammel/config.py", line 72, in set_launchpad
>      lp = Launchpad.login_with(appid, root, cachedir)
>    File "/usr/lib/pymodules/python2.7/launchpadlib/launchpad.py", line 538, in login_with
>      credential_save_failed, version)
>    File "/usr/lib/pymodules/python2.7/launchpadlib/launchpad.py", line 341, in _authorize_token_and_login
>      authorization_engine.unique_consumer_id)
>    File "/usr/lib/pymodules/python2.7/launchpadlib/credentials.py", line 273, in load
>      return self.do_load(unique_key)
>    File "/usr/lib/pymodules/python2.7/launchpadlib/credentials.py", line 322, in do_load
>      return Credentials.from_string(credential_string)
>    File "/usr/lib/pymodules/python2.7/launchpadlib/credentials.py", line 89, in from_string
>      credentials.load(StringIO(value))
>    File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/authorize/oauth.py", line 165, in load
>      CREDENTIALS_FILE_VERSION, 'consumer_key')
>    File "/usr/lib/python2.7/ConfigParser.py", line 610, in get
>      raise NoOptionError(option, section)
>   ConfigParser.NoOptionError: No option 'consumer_key' in section: '1'
>
>  At James Westby's suggestion, I had a peek inside gnome-keyring with
>  seahorse and found this as the 'password' value of the 'network
>  password' token:
>
>    [1]
>
>  Presumably there should be a real password here instead. :)
>
>  People experiencing this problem can work around it by opening their
>  gnome keyring, and deleting the broken password.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/launchpadlib/+bug/745801/+subscriptions

python-launchpadlib now has a daily build at https://launchpad.net/~launchpad/+archive/ppa which includes the fix for this bug.

Hello Steve, or anyone else affected,

Accepted python-launchpadlib into oneiric-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Hello Steve, or anyone else affected,

Accepted python-launchpadlib into natty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

I have verified the natty-proposed and oneiric-proposed packages.

This bug was fixed in the package python-launchpadlib - 1.9.7-0ubuntu2.1

---------------
python-launchpadlib (1.9.7-0ubuntu2.1) natty-proposed; urgency=low

  * Add 100_base64_encode_credentials.patch cherrypicked from upstream
    tree to fix issue causing launchpadlib scripts to fail due to problems
    associated with keyring corruption. We encode launchpadlib's entries so
    the keyring won't be confused by unexpected characters.
     - Convert credentials from unicode when retrieved from the keyring.
       (LP: #877374)
     - Use base 64 encoding for credentials stored in the keyring.
       (LP: #745801)
     - Properly handle decoding base 64.
       (LP: #900307)
 -- Bryce Harrington <email address hidden> Fri, 09 Dec 2011 15:03:01 -0800

This bug was fixed in the package python-launchpadlib - 1.9.8-2ubuntu0.1

---------------
python-launchpadlib (1.9.8-2ubuntu0.1) oneiric-proposed; urgency=low

  * Add 100_base64_encode_credentials.patch cherrypicked from upstream
    tree to fix issue causing launchpadlib scripts to fail due to problems
    associated with keyring corruption. We encode launchpadlib's entries so
    the keyring won't be confused by unexpected characters.
     - Convert credentials from unicode when retrieved from the keyring.
       (LP: #877374)
     - Use base 64 encoding for credentials stored in the keyring.
       (LP: #745801)
     - Properly handle decoding base 64.
       (LP: #900307)
 -- Bryce Harrington <email address hidden> Fri, 09 Dec 2011 14:55:16 -0800