Juju Charms Collection
neutron-api package

Juno -> Kilo upgrade failed - Duplicate key name

Bug #1469621 reported by Ante Karamatić
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron-api (Juju Charms Collection)
Fix Released
High
James Page
Juju Charms Collection 15.07

Bug Description

I've done three Juno -> Kilo updates on three unrelated deployments (shared keystone thou), and in all three upgrades 2 out of 3 nodes (HA deployment) failed upgrading. Upgrade succeeded only on nodes that had VIP running. All other nodes failed. Running 'resolved --retry' resolved the issue on all of them.

Logs from one of the failed nodes is attached. All other nodes have same issue.

See original description
Tags: cpec

Related branches

lp:~james-page/charms/trusty/neutron-api/lp-1469621
Ante Karamatić (community): Approve
Liam Young (community): Approve
Diff: 95 lines (+45/-1)
2 files modified
hooks/neutron_api_utils.py (+2/-1)
unit_tests/test_neutron_api_utils.py (+43/-0)
lp:~james-page/charms/trusty/neutron-api/lp-1469621-stable
Liam Young: Pending requested
Diff: 95 lines (+45/-1)
2 files modified
hooks/neutron_api_utils.py (+2/-1)
unit_tests/test_neutron_api_utils.py (+43/-0)
Liam Young: Pending requested
Diff: 14789 lines (+10693/-1226)
107 files modified
.bzrignore (+2/-0)
.project (+17/-0)
.pydevproject (+9/-0)
Makefile (+21/-5)
README.md (+121/-18)
actions.yaml (+2/-0)
actions/git_reinstall.py (+45/-0)
charm-helpers-hooks.yaml (+5/-2)
charm-helpers-tests.yaml (+5/-0)
config.yaml (+269/-11)
hooks/charmhelpers/__init__.py (+38/-0)
hooks/charmhelpers/contrib/__init__.py (+15/-0)
hooks/charmhelpers/contrib/charmsupport/__init__.py (+15/-0)
hooks/charmhelpers/contrib/charmsupport/nrpe.py (+360/-0)
hooks/charmhelpers/contrib/charmsupport/volumes.py (+175/-0)
hooks/charmhelpers/contrib/hahelpers/__init__.py (+15/-0)
hooks/charmhelpers/contrib/hahelpers/apache.py (+26/-3)
hooks/charmhelpers/contrib/hahelpers/ceph.py (+0/-297)
hooks/charmhelpers/contrib/hahelpers/cluster.py (+152/-38)
hooks/charmhelpers/contrib/network/__init__.py (+15/-0)
hooks/charmhelpers/contrib/network/ip.py (+450/-0)
hooks/charmhelpers/contrib/network/ovs/__init__.py (+96/-0)
hooks/charmhelpers/contrib/openstack/__init__.py (+15/-0)
hooks/charmhelpers/contrib/openstack/alternatives.py (+16/-0)
hooks/charmhelpers/contrib/openstack/amulet/__init__.py (+15/-0)
hooks/charmhelpers/contrib/openstack/amulet/deployment.py (+146/-0)
hooks/charmhelpers/contrib/openstack/amulet/utils.py (+294/-0)
hooks/charmhelpers/contrib/openstack/context.py (+838/-210)
hooks/charmhelpers/contrib/openstack/files/__init__.py (+18/-0)
hooks/charmhelpers/contrib/openstack/files/check_haproxy.sh (+32/-0)
hooks/charmhelpers/contrib/openstack/files/check_haproxy_queue_depth.sh (+30/-0)
hooks/charmhelpers/contrib/openstack/ip.py (+151/-0)
hooks/charmhelpers/contrib/openstack/neutron.py (+156/-5)
hooks/charmhelpers/contrib/openstack/templates/__init__.py (+16/-0)
hooks/charmhelpers/contrib/openstack/templates/git.upstart (+17/-0)
hooks/charmhelpers/contrib/openstack/templates/haproxy.cfg (+30/-8)
hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend (+9/-8)
hooks/charmhelpers/contrib/openstack/templates/openstack_https_frontend.conf (+9/-8)
hooks/charmhelpers/contrib/openstack/templates/section-keystone-authtoken (+9/-0)
hooks/charmhelpers/contrib/openstack/templates/section-rabbitmq-oslo (+22/-0)
hooks/charmhelpers/contrib/openstack/templates/section-zeromq (+14/-0)
hooks/charmhelpers/contrib/openstack/templating.py (+43/-28)
hooks/charmhelpers/contrib/openstack/utils.py (+280/-86)
hooks/charmhelpers/contrib/python/__init__.py (+15/-0)
hooks/charmhelpers/contrib/python/packages.py (+96/-0)
hooks/charmhelpers/contrib/storage/__init__.py (+15/-0)
hooks/charmhelpers/contrib/storage/linux/__init__.py (+15/-0)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+160/-103)
hooks/charmhelpers/contrib/storage/linux/loopback.py (+19/-3)
hooks/charmhelpers/contrib/storage/linux/lvm.py (+18/-1)
hooks/charmhelpers/contrib/storage/linux/utils.py (+41/-6)
hooks/charmhelpers/core/__init__.py (+15/-0)
hooks/charmhelpers/core/decorators.py (+57/-0)
hooks/charmhelpers/core/fstab.py (+134/-0)
hooks/charmhelpers/core/hookenv.py (+229/-23)
hooks/charmhelpers/core/host.py (+190/-37)
hooks/charmhelpers/core/services/__init__.py (+18/-0)
hooks/charmhelpers/core/services/base.py (+329/-0)
hooks/charmhelpers/core/services/helpers.py (+267/-0)
hooks/charmhelpers/core/strutils.py (+42/-0)
hooks/charmhelpers/core/sysctl.py (+56/-0)
hooks/charmhelpers/core/templating.py (+68/-0)
hooks/charmhelpers/core/unitdata.py (+477/-0)
hooks/charmhelpers/fetch/__init__.py (+228/-97)
hooks/charmhelpers/fetch/archiveurl.py (+115/-17)
hooks/charmhelpers/fetch/bzrurl.py (+32/-3)
hooks/charmhelpers/fetch/giturl.py (+71/-0)
hooks/charmhelpers/payload/__init__.py (+16/-0)
hooks/charmhelpers/payload/execd.py (+16/-0)
hooks/neutron_api_context.py (+188/-7)
hooks/neutron_api_hooks.py (+288/-41)
hooks/neutron_api_utils.py (+282/-24)
metadata.yaml (+17/-2)
templates/git/neutron_sudoers (+4/-0)
templates/git/upstart/neutron-server.upstart (+22/-0)
templates/icehouse/ml2_conf.ini (+10/-4)
templates/icehouse/neutron.conf (+22/-2)
templates/icehouse/nsx.ini (+11/-0)
templates/juno/neutron.conf (+85/-0)
templates/kilo/ml2_conf.ini (+36/-0)
templates/kilo/neutron.conf (+91/-0)
templates/parts/section-database (+1/-0)
tests/00-setup (+11/-0)
tests/014-basic-precise-icehouse (+11/-0)
tests/015-basic-trusty-icehouse (+9/-0)
tests/016-basic-trusty-juno (+11/-0)
tests/017-basic-trusty-kilo (+11/-0)
tests/018-basic-utopic-juno (+9/-0)
tests/019-basic-vivid-kilo (+9/-0)
tests/050-basic-trusty-icehouse-git (+9/-0)
tests/051-basic-trusty-juno-git (+12/-0)
tests/README (+53/-0)
tests/basic_deployment.py (+460/-0)
tests/charmhelpers/__init__.py (+38/-0)
tests/charmhelpers/contrib/__init__.py (+15/-0)
tests/charmhelpers/contrib/amulet/__init__.py (+15/-0)
tests/charmhelpers/contrib/amulet/deployment.py (+93/-0)
tests/charmhelpers/contrib/amulet/utils.py (+323/-0)
tests/charmhelpers/contrib/openstack/__init__.py (+15/-0)
tests/charmhelpers/contrib/openstack/amulet/__init__.py (+15/-0)
tests/charmhelpers/contrib/openstack/amulet/deployment.py (+146/-0)
tests/charmhelpers/contrib/openstack/amulet/utils.py (+294/-0)
unit_tests/__init__.py (+2/-0)
unit_tests/test_actions_git_reinstall.py (+105/-0)
unit_tests/test_neutron_api_context.py (+346/-25)
unit_tests/test_neutron_api_hooks.py (+474/-55)
unit_tests/test_neutron_api_utils.py (+398/-49)
Revision history for this message
Ante Karamatić (ivoks) wrote :
description: updated
Revision history for this message
James Page (james-page) wrote :

I see the issue:

    # set CONFIGS to load templates from new release
    configs.set_release(openstack_release=new_os_rel)
    # Before kilo it's nova-cloud-controllers job
    if new_os_rel >= 'kilo':
        stamp_neutron_database(cur_os_rel)
        migrate_neutron_database()

the neutron-api charm is making no effort to only sync from the leader.

Changed in neutron-api (Juju Charms Collection):
status: New → Confirmed
importance: Undecided → High
Revision history for this message
James Page (james-page) wrote :

Running resolved --retry in turn on each unit will resolve this problem; alembic is clever enough to figure things out.

Fix proposed to ensure that db upgrade is only run on the lead unit.

Changed in neutron-api (Juju Charms Collection):
status: Confirmed → In Progress
assignee: nobody → James Page (james-page)
Revision history for this message
Ante Karamatić (ivoks) wrote :

I can confirm that lp:~james-page/charms/trusty/neutron-api/lp-1469621 branch does indeed solve the bug.

James Page (james-page)
Changed in neutron-api (Juju Charms Collection):
status: In Progress → Fix Committed
Revision history for this message
Felipe Reyes (freyes) wrote :

This fix is already available in /trunk[0] marking as Fix Released, and this landed in /next at July 3rd, before 15.07 was released, so setting 15.07 as milestone.

http://bazaar.launchpad.net/~openstack-charmers/charms/trusty/neutron-api/trunk/view/head:/hooks/neutron_api_utils.py#L376

Changed in neutron-api (Juju Charms Collection):
milestone: none → 15.07
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.