Glance installation fails if password contains '@' symbol

Bug #1695299 reported by Rahul Sharma
38
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Glance
Fix Released
High
Ian Wienand
Rocky
Fix Released
High
Ian Wienand

Bug Description

I was doing a fresh installation of devstack today and had set the admin password as "Test@321" in local.conf file. The installation started failing for Glance and when I had a look at the backtrace, it looks like it converts '@' to '%40' and starts failing.

Either this needs to be fixed or proper note needs to be added in devstack setup stating that one cannot use '@' symbol in passwords.

ubuntu@openstack:~/devstack$ glance --version
2.6.0
ubuntu@openstack:~/devstack$

CRITICAL glance [-] Unhandled error: ValueError: invalid interpolation syntax in 'mysql+pymysql://root:Test%40321@127.0.0.1/glance?charset=utf8' at position 27
ERROR glance Traceback (most recent call last):
ERROR glance File "/usr/local/bin/glance-manage", line 10, in <module>
ERROR glance sys.exit(main())
ERROR glance File "/opt/stack/glance/glance/cmd/manage.py", line 452, in main
ERROR glance return CONF.command.action_fn()
ERROR glance File "/opt/stack/glance/glance/cmd/manage.py", line 291, in sync
ERROR glance self.command_object.sync(CONF.command.version)
ERROR glance File "/opt/stack/glance/glance/cmd/manage.py", line 117, in sync
ERROR glance alembic_migrations.place_database_under_alembic_control()
ERROR glance File "/opt/stack/glance/glance/db/sqlalchemy/alembic_migrations/__init__.py", line 73, in place_database_under_alembic_control
ERROR glance a_config = get_alembic_config()
ERROR glance File "/opt/stack/glance/glance/db/sqlalchemy/alembic_migrations/__init__.py", line 37, in get_alembic_config
ERROR glance config.set_main_option('sqlalchemy.url', str(engine.url))
ERROR glance File "/usr/local/lib/python2.7/dist-packages/alembic/config.py", line 218, in set_main_option
ERROR glance self.set_section_option(self.config_ini_section, name, value)
ERROR glance File "/usr/local/lib/python2.7/dist-packages/alembic/config.py", line 245, in set_section_option
ERROR glance self.file_config.set(section, name, value)
ERROR glance File "/usr/lib/python2.7/ConfigParser.py", line 752, in set
ERROR glance "position %d" % (value, tmp_value.find('%')))
ERROR glance ValueError: invalid interpolation syntax in 'mysql+pymysql://root:Test%40321@127.0.0.1/glance?charset=utf8' at position 27
ERROR glance
+lib/glance:init_glance:1 exit_trap
+./stack.sh:exit_trap:492 local r=1
++./stack.sh:exit_trap:493 jobs -p
+./stack.sh:exit_trap:493 jobs=
+./stack.sh:exit_trap:496 [[ -n '' ]]
+./stack.sh:exit_trap:502 kill_spinner
+./stack.sh:kill_spinner:388 '[' '!' -z '' ']'
+./stack.sh:exit_trap:504 [[ 1 -ne 0 ]]
+./stack.sh:exit_trap:505 echo 'Error on exit'
Error on exit
+./stack.sh:exit_trap:506 generate-subunit 1496417170 632 fail
+./stack.sh:exit_trap:507 [[ -z /opt/stack/logs ]]
+./stack.sh:exit_trap:510 /home/ubuntu/devstack/tools/worlddump.py -d /opt/stack/logs
World dumping... see /opt/stack/logs/worlddump-2017-06-02-153642.txt for details
+./stack.sh:exit_trap:516 exit 1
ubuntu@openstack:~/devstack$

Revision history for this message
Tiago Farias (tgoboards) wrote :

same problem here when tried to install devstack and using a password with '@'.

Revision history for this message
Gleb Zimin (gzimin) wrote :

So, also when password or login have '/' symbol glance throw error.

Revision history for this message
Aizuddin Zali (mymzbe) wrote :

Seems like this hit interpolation on ConfigParser. Are you able to escape using `%%`?

One possible solution is to set:

config = ConfigParser(strict=False, interpolation=None)

Changed in glance:
assignee: nobody → Aizuddin Zali (mymzbe)
status: New → Confirmed
Revision history for this message
Aizuddin Zali (mymzbe) wrote :

correction '%@' as example to escapte @.

Aizuddin Zali (mymzbe)
Changed in devstack:
status: New → Invalid
Revision history for this message
Aizuddin Zali (mymzbe) wrote :

From https://git.launchpad.net/glance/tree/glance/db/sqlalchemy/alembic_migrations/__init__.py master bracnh line 37.

config.set_main_option('sqlalchemy.url', str(engine.url))

This indeed an expected behaviour, quoting to alembic documentation (http://alembic.zzzcomputing.com/en/latest/api/config.html).

 set_main_option(name, value)

    Set an option programmatically within the ‘main’ section.

    This overrides whatever was in the .ini file.
    Parameters:

        name – name of the value
        value¶ – the value. Note that this value is passed to ConfigParser.set, which supports variable interpolation using pyformat (e.g. %(some_value)s). A raw percent sign not part of an interpolation symbol must therefore be escaped, e.g. %%. The given value may refer to another value already in the file using the interpolation format.

Changed in glance:
status: Confirmed → Invalid
Aizuddin Zali (mymzbe)
Changed in devstack:
assignee: nobody → Aizuddin Zali (mymzbe)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to devstack (master)

Fix proposed to branch: master
Review: https://review.openstack.org/498427

Changed in devstack:
assignee: Aizuddin Zali (mymzbe) → Gleb Zimin (glathor)
status: Invalid → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/498730

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on devstack (master)

Change abandoned by Gleb Zimin (<email address hidden>) on branch: master
Review: https://review.openstack.org/498730

Revision history for this message
Ian Wienand (iwienand) wrote :

There's an unanswered question here ... how did

> "Test@321" in local.conf file.

end up being escaped to %40

> CRITICAL glance [-] Unhandled error: ValueError: invalid interpolation syntax
> in 'mysql+pymysql://root:Test%40321@127.0.0.1/glance?charset=utf8'

? afaict we don't do that anywhere

Revision history for this message
Ian Wienand (iwienand) wrote :

I've actually convinced myself this really is glance's fault. I have a suggested change incoming

Changed in glance:
status: Invalid → New
Revision history for this message
Gleb Zimin (gzimin) wrote :

So, we can delete devstack from affects.

Changed in devstack:
status: In Progress → Invalid
no longer affects: devstack
Erno Kuvaja (jokke)
Changed in glance:
status: New → In Progress
importance: Undecided → High
Revision history for this message
Brian Rosmaita (brian-rosmaita) wrote :

Unassigning, patches have been abandoned.

Changed in glance:
assignee: Aizuddin Zali (mymzbe) → nobody
Revision history for this message
Brian Rosmaita (brian-rosmaita) wrote :
no longer affects: glance/pike
no longer affects: glance/queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (master)

Reviewed: https://review.openstack.org/499410
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=f601cfccf1d8e2e314a270943d91e8aa1932f2a4
Submitter: Zuul
Branch: master

commit f601cfccf1d8e2e314a270943d91e8aa1932f2a4
Author: Ian Wienand <email address hidden>
Date: Thu Aug 31 11:06:28 2017 +1000

    Support RFC1738 quoted chars in passwords

    In the bug, a user tried setting a devstack password with a "@" in it.

    As it turns out, sqlalchmey turns the connection-string into a
    sqlalchemy.engine.url.URL object [1] which returns a RFC1738 quoted
    string.

    However, alembic's set_main_option [2] uses python
    string-interpolation which interprets '%' characters. This means you
    end up with an interpolation traceback when using any quoted character
    (':@/') in a user/password (more likely password).

    Avoid this by ensuring the URL is safe for python interpolation in
    set_main_option by replacing '%' -> '%%'.

    I convinced myself this is safe because sqlalchemy correctly parses
    the quoted and unquoted versions just the same

    ---
     >>> str(sqlalchemy.engine.url.make_url('mysql+pymysql://foo:crazy:@/pw@/moo'))
     'mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'
     >>> str(sqlalchemy.engine.url.make_url('mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'))
     'mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'
    ---

    A test is added

    [1] https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/engine/url.py
    [2] http://alembic.zzzcomputing.com/en/latest/api/config.html#alembic.config.Config.set_main_option

    Change-Id: I3ef7e3e539e35ce040573f2044ab6eb3c990200a
    Closes-Bug: #1695299

Changed in glance:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/592210

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (stable/rocky)

Reviewed: https://review.openstack.org/592210
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=626018b991080c0706df8481e37addefd10c58fa
Submitter: Zuul
Branch: stable/rocky

commit 626018b991080c0706df8481e37addefd10c58fa
Author: Ian Wienand <email address hidden>
Date: Thu Aug 31 11:06:28 2017 +1000

    Support RFC1738 quoted chars in passwords

    In the bug, a user tried setting a devstack password with a "@" in it.

    As it turns out, sqlalchmey turns the connection-string into a
    sqlalchemy.engine.url.URL object [1] which returns a RFC1738 quoted
    string.

    However, alembic's set_main_option [2] uses python
    string-interpolation which interprets '%' characters. This means you
    end up with an interpolation traceback when using any quoted character
    (':@/') in a user/password (more likely password).

    Avoid this by ensuring the URL is safe for python interpolation in
    set_main_option by replacing '%' -> '%%'.

    I convinced myself this is safe because sqlalchemy correctly parses
    the quoted and unquoted versions just the same

    ---
     >>> str(sqlalchemy.engine.url.make_url('mysql+pymysql://foo:crazy:@/pw@/moo'))
     'mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'
     >>> str(sqlalchemy.engine.url.make_url('mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'))
     'mysql+pymysql://foo:crazy%3A%40%2Fpw@/moo'
    ---

    A test is added

    [1] https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/engine/url.py
    [2] http://alembic.zzzcomputing.com/en/latest/api/config.html#alembic.config.Config.set_main_option

    Change-Id: I3ef7e3e539e35ce040573f2044ab6eb3c990200a
    Closes-Bug: #1695299
    (cherry picked from commit f601cfccf1d8e2e314a270943d91e8aa1932f2a4)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/glance 17.0.0.0rc2

This issue was fixed in the openstack/glance 17.0.0.0rc2 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/glance 18.0.0.0b1

This issue was fixed in the openstack/glance 18.0.0.0b1 development milestone.

Erno Kuvaja (jokke)
Changed in glance:
milestone: rocky-rc2 → stein-1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.