Use of BIGINT for image size column causes sqlite problems

Bug #966243 reported by Mark McLoughlin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Fix Released
Medium
Eoghan Glynn

Bug Description

To reproduce:

  - using the diablo branch or packages

  - configure to use sqlite

  - ensure the database doesn't exist

  - run glance-registry, observe the database is auto-created

  - shut down glance-registry again

  - version control the database (see bug #966242)

  - switch to the essex branch or packages

  - try to upgrade the database using 'glance-manage upgrade'

This is the error:

2012-03-27 14:19:07 13803 INFO [glance.registry.db.migration] Upgrading sqlite:///glance.sqlite to version latest
2012-03-27 14:19:07 13803 INFO [migrate.versioning.api] 9 -> 10...
2012-03-27 14:19:07 13803 INFO [migrate.versioning.api] done
2012-03-27 14:19:07 13803 INFO [migrate.versioning.api] 10 -> 11...
/home/markmc/git/openstack/glance/.venv/lib/python2.7/site-packages/sqlalchemy/engine/reflection.py:47: SAWarning: Did not recognize type 'BIGINT' of column 'size'
...
  File "/home/markmc/git/openstack/glance/glance/registry/db/migrate_repo/versions/011_make_mindisk_and_minram_notnull.py", line 27, in upgrade
    images.c.min_disk.alter(nullable=False)
...
  File "/home/markmc/git/openstack/glance/.venv/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 1875, in visit_null
    raise NotImplementedError("Can't generate DDL for the null type")
NotImplementedError: Can't generate DDL for the null type

The problem stems from the fact that the auto-creation of the database with register_models() - as opposed to its creation using db_sync - creates a BIGINT column. When we try and upgrade, sqlalchemy doesn't recognize the type and sets it to NullType which the DDL compiler can't handle.

One solution would be to make sure the register_models() codepath doesn't create a BigInteger column with sqlite.

Another solution would be to just auto-create the DB using db_sync().

Revision history for this message
Mark McLoughlin (markmc) wrote :
Brian Waldon (bcwaldon)
tags: added: essex-rc-potential
Changed in glance:
importance: Undecided → Medium
status: New → Confirmed
Eoghan Glynn (eglynn)
Changed in glance:
assignee: nobody → Eoghan Glynn (eglynn)
Revision history for this message
Eoghan Glynn (eglynn) wrote :

OK, finally twigged that in order to reproduce this issue, the starting point version must be 9 as opposed to zero.

Whereas the instructions as stated will cause the version field in the migrate_version table to be set to zero, as the the following commit to allow a specific version be specified has not yet been applied to the stable diablo branch (and isn't directly applicable):

  https://github.com/openstack/glance/commit/da66de06

Otherwise the Essex upgrade from version 0 fails with:

  http://paste.openstack.org/show/12345/

I've backported the "Allow specifying the current version in 'glance-manage version_control'" commit to diablo stable:

  https://review.openstack.org/6077

and with that patch in place, the bug is reproducible if the target version is explicitly set to 9:

  sudo ./tools/with_venv.sh bin/glance-manage version_control 9 --config-file ./etc/glance-registry.conf

before switching to the Essex branch.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/6101

Changed in glance:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (master)

Reviewed: https://review.openstack.org/6101
Committed: http://github.com/openstack/glance/commit/f28f16a750ad134fad2ff663680b149ce77e31c9
Submitter: Jenkins
Branch: master

commit f28f16a750ad134fad2ff663680b149ce77e31c9
Author: Eoghan Glynn <email address hidden>
Date: Mon Apr 2 18:55:35 2012 +0100

    Compile BigInteger to INTEGER for sqlite

    Fixes bug 966243

    Ensure BigInteger does not map to BIGINT for an auto-created
    sqlite registry DB, as this type is not supported by sqlite.

    Change-Id: I61f44fbe50ea406d2d593f8f53cab5da3af2222a

Changed in glance:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (stable/diablo)

Fix proposed to branch: stable/diablo
Review: https://review.openstack.org/6111

Brian Waldon (bcwaldon)
tags: removed: essex-rc-potential
Eoghan Glynn (eglynn)
Changed in glance:
milestone: none → folsom-1
Brian Waldon (bcwaldon)
tags: added: essex-rc-potential
Thierry Carrez (ttx)
Changed in glance:
milestone: folsom-1 → essex-rc3
tags: removed: essex-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/6168

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (milestone-proposed)

Reviewed: https://review.openstack.org/6168
Committed: http://github.com/openstack/glance/commit/0d4d1f7a1d5a595acf7c66c9054446140044b838
Submitter: Jenkins
Branch: milestone-proposed

commit 0d4d1f7a1d5a595acf7c66c9054446140044b838
Author: Eoghan Glynn <email address hidden>
Date: Mon Apr 2 18:55:35 2012 +0100

    Compile BigInteger to INTEGER for sqlite

    Fixes bug 966243

    Ensure BigInteger does not map to BIGINT for an auto-created
    sqlite registry DB, as this type is not supported by sqlite.

    Change-Id: I61f44fbe50ea406d2d593f8f53cab5da3af2222a

Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in glance:
milestone: essex-rc3 → 2012.1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (stable/diablo)

Reviewed: https://review.openstack.org/6111
Committed: http://github.com/openstack/glance/commit/871ea1eb38002e3279e72687bef053e2756a5497
Submitter: Jenkins
Branch: stable/diablo

commit 871ea1eb38002e3279e72687bef053e2756a5497
Author: Eoghan Glynn <email address hidden>
Date: Mon Apr 2 18:55:35 2012 +0100

    Compile BigInteger to INTEGER for sqlite

    Fixes bug 966243

    Ensure BigInteger does not map to BIGINT for an auto-created
    sqlite registry DB, as this type is not supported by sqlite.

    Change-Id: I61f44fbe50ea406d2d593f8f53cab5da3af2222a

tags: added: in-stable-diablo
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.