glance metadef tables need unique constraints

Bug #1468946 reported by Wayne
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Glance
Fix Released
Critical
Wayne
Nominated for Kilo by Erno Kuvaja

Bug Description

Sometime during Kilo, the unique constraints on
metadef_namespaces (namespace)
metadef_objects(namespace_id, name)
metadef_properties(namespace_id, name)
metadef_tags(namespace_id, name)
metadef_resource_types(name)
were removed.
I believe this was done erroneously to make the migrate_repo/versions/scripts match the db/sqlalchemy/models_metadef.py definitions. Unfortunately, the schema scripts were correct with the unique constraints and what should have changed was the models_metadef.py.
This bug, puts one more migrate script in place which will rename any duplicate records it finds to make them unique and then re-establishes the unique constraints. It also, fixes models_metadef.py and adds in tests to create duplicates which should result in an HTTPConflict.

Tags: metadef
Wayne (wayne-okuma)
Changed in glance:
assignee: nobody → Wayne (wayne-okuma)
Changed in glance:
status: New → In Progress
Revision history for this message
Wayne (wayne-okuma) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/#/c/195820/

Revision history for this message
Eric Brown (ericwb) wrote :

This bug actually describes the root cause of what I was running into using metadef import.

In my case, here is how to reproduce:
- run 'glance-manage db unload_metadefs'
- Log into Horizon
- import compute-libvirt.json
- then import compute-libvirt.json again
- Notice you get an error that it can't add the metadef
- Now click on Metadata Definitions again. Notice you get an error again. Metadata Definitions are no longer useable. In fact, the only way to recover is to run 'glance-manage db unload_metadefs'

Here's the traceback when this occurs: http://paste.openstack.org/show/412256/

The root cause is no unique constraint on table metadef_namespaces, column namespace

Erno Kuvaja (jokke)
Changed in glance:
importance: Undecided → High
Revision history for this message
Jeremy Stanley (fungi) wrote :

Note that this is a duplicate of bug 1479385 (or vice versa).

Revision history for this message
Erno Kuvaja (jokke) wrote :

Jeremy,

I don't know how the duplication should be dealt with while the DOS condition was not called out here, the behavior was explained already month before logging of 1479385 and the attack surface cleared by Eric's comment #2 @ 08.08. So yes, Id's say the 1479385 is the duplicate.

I will look into this how we get the backporting moving forward for Kilo as soon as possible, specially now if the master fix merges.

Changed in glance:
importance: High → Undecided
importance: Undecided → Critical
milestone: none → liberty-rc1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (master)

Reviewed: https://review.openstack.org/195820
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=5369e86e8d521115228573dd1878d764cd249e97
Submitter: Jenkins
Branch: master

commit 5369e86e8d521115228573dd1878d764cd249e97
Author: Wayne Okuma <email address hidden>
Date: Thu Jun 25 17:07:01 2015 -0700

    Glance metadef tables need unique constraints.

    Sometime during Kilo, the unique constraints on
    metadef_namespaces (namespace)
    metadef_objects(namespace_id, name)
    metadef_properties(namespace_id, name)
    metadef_tags(namespace_id, name)
    metadef_resource_types(name)
    were replaced with non-unique indices.

    I believe this was done erroneously to make the migrate_repo/versions/scripts
    match the db/sqlalchemy/models_metadef.py definitions. Unfortunately, the
    schema scripts were correct with the unique constraints and what should have
    changed was models_metadef.py.

    This bug, puts one more migrate script in place which will rename any
    duplicate records it finds to make them unique and then re-establishes
    the unique constraints while dropping the non-unique indices. It also,
    fixes models_metadef.py and adds in tests to attempt to create duplicates
    which should result in an HTTPConflict.

    Change-Id: Idf3569a27d64abea3ed6ec92fb77b36a4d6d5fd5
    Closes-Bug: 1468946

Changed in glance:
status: In Progress → Fix Committed
Revision history for this message
Nikhil Komawar (nikhil-komawar) wrote :

This was raised as a possible backport for Kilo and I see the nomination already. In response to the request made, I think making the DB migration doesn't really fit the rules of backports. The question remains is there a way to avoid the DOS? I currently can't think of one.

Thierry Carrez (ttx)
Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in glance:
milestone: liberty-rc1 → 11.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.