OpenStack Nova Compute Charm

os_release should detect if upgrade has occurred and reset cache

Bug #1715624 reported by Corey Bryant
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Ceilometer Agent Charm
Fix Released
Medium
Corey Bryant
OpenStack Ceilometer Agent Charm 17.08
OpenStack Ceilometer Charm
Fix Released
Medium
James Page
OpenStack Ceilometer Charm 17.08
OpenStack Cinder Charm
Fix Released
Medium
Corey Bryant
OpenStack Cinder Charm 17.08
OpenStack Dashboard Charm
Fix Released
Medium
James Page
OpenStack Dashboard Charm 17.08
OpenStack Glance Charm
Fix Released
Medium
Corey Bryant
OpenStack Glance Charm 17.08
OpenStack Heat Charm
Fix Released
Medium
James Page
OpenStack Heat Charm 17.08
OpenStack Keystone Charm
Fix Released
Medium
James Page
OpenStack Keystone Charm 17.08
OpenStack Nova Cloud Controller Charm
Fix Released
Medium
David Ames
OpenStack Nova Cloud Controller Charm 17.08
OpenStack Nova Compute Charm
Fix Released
Medium
Corey Bryant
OpenStack Nova Compute Charm 17.08

Bug Description

In upgrade testing for Pike we tripped over at least 2 scenarios so far where os_release() returns the old cached release after upgrade. For late release fixes, we've patched these issues at the calls to os_release, like this for example:

diff --git a/hooks/charmhelpers/contrib/openstack/utils.py b/hooks/charmhelpers/contrib/openstack/utils.py
index 837a167..f3dcecb 100644
--- a/hooks/charmhelpers/contrib/openstack/utils.py
+++ b/hooks/charmhelpers/contrib/openstack/utils.py
@@ -2012,7 +2012,7 @@ def enable_memcache(source=None, release=None, package=None):
     if release:
         _release = release
     else:
- _release = os_release(package, base='icehouse')
+ _release = os_release(package, base='icehouse', reset_cache=True)
     if not _release:
         _release = get_os_codename_install_source(source)

and this for example:

https://review.openstack.org/#/c/501489/

We should really fix this in os_release() though so we never have to deal with this issue again in another call to os_release. os_release should be updated to detect if an upgrade has occurred and reset it's cache after upgrade on it's own.

See original description
Corey Bryant (corey.bryant)
description: updated
description: updated
description: updated
description: updated
description: updated
Revision history for this message
Corey Bryant (corey.bryant) wrote :

another option would be to update all charms to call os_release(... , reset_cache=True) after they upgrade.

Changed in charm-nova-cloud-controller:
status: New → Triaged
Changed in charm-nova-compute:
status: New → Triaged
Changed in charm-nova-cloud-controller:
importance: Undecided → Medium
Changed in charm-nova-compute:
importance: Undecided → Medium
Changed in charm-nova-cloud-controller:
assignee: nobody → Corey Bryant (corey.bryant)
Changed in charm-nova-compute:
assignee: nobody → Corey Bryant (corey.bryant)
James Page (james-page)
no longer affects: ceilometer
Changed in charm-ceilometer:
status: New → Triaged
Changed in charm-heat:
status: New → Triaged
Changed in charm-ceilometer:
importance: Undecided → High
importance: High → Medium
Changed in charm-heat:
importance: Undecided → Medium
Changed in charm-ceilometer:
milestone: none → 17.08
Changed in charm-heat:
milestone: none → 17.08
Changed in charm-nova-cloud-controller:
milestone: none → 17.08
Changed in charm-nova-compute:
milestone: none → 17.08
James Page (james-page)
Changed in charm-keystone:
status: New → Triaged
Changed in charm-openstack-dashboard:
status: New → Triaged
importance: Undecided → Medium
Changed in charm-keystone:
importance: Undecided → Medium
milestone: none → 17.08
Changed in charm-openstack-dashboard:
milestone: none → 17.08
David Ames (thedac)
Changed in charm-nova-cloud-controller:
assignee: Corey Bryant (corey.bryant) → David Ames (thedac)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-ceilometer (master)

Reviewed: https://review.openstack.org/501785
Committed: https://git.openstack.org/cgit/openstack/charm-ceilometer/commit/?id=f778d87ca870ce31840d2c067d5f720af99bda32
Submitter: Jenkins
Branch: master

commit f778d87ca870ce31840d2c067d5f720af99bda32
Author: James Page <email address hidden>
Date: Thu Sep 7 16:54:07 2017 +0100

    Reset os_release cache during upgrade

    Ensure that the os_release cache is cleared during the openstack
    upgrade process, ensuring that package list and configuration
    options are correctly set for the new OpenStack version.

    Change-Id: Ie685854952527f59ee344bfda8e49250c248150a
    Closes-Bug: 1715624

Changed in charm-ceilometer:
status: Triaged → Fix Committed
Changed in charm-heat:
status: Triaged → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-heat (master)

Reviewed: https://review.openstack.org/501788
Committed: https://git.openstack.org/cgit/openstack/charm-heat/commit/?id=7f38a8b648d16148d106e7cb9c3ed9f0ca9bc7fa
Submitter: Jenkins
Branch: master

commit 7f38a8b648d16148d106e7cb9c3ed9f0ca9bc7fa
Author: James Page <email address hidden>
Date: Thu Sep 7 17:01:56 2017 +0100

    Use determine_packages during openstack upgrade

    This ensures that memcached is installed as part of an OpenStack
    upgrade through the Mitaka release.

    Change-Id: I2feffde8073b2415853a0950a0e37c5836e39a63
    Closes-Bug: 1715624

Changed in charm-openstack-dashboard:
status: Triaged → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-openstack-dashboard (master)

Reviewed: https://review.openstack.org/501801
Committed: https://git.openstack.org/cgit/openstack/charm-openstack-dashboard/commit/?id=efda3b9ddfcb8a249ae30cd4516feac19c5f40c9
Submitter: Jenkins
Branch: master

commit efda3b9ddfcb8a249ae30cd4516feac19c5f40c9
Author: James Page <email address hidden>
Date: Thu Sep 7 17:25:47 2017 +0100

    Fix minor openstack upgrade issues

    Ensure that any new package requirements are installed for the
    new OpenStack release version.

    Reset os_release cache during the upgrade process to pickup the
    new release.

    Change-Id: Ied85f83ad4fabf1fb946df7db94aa4c847b4c075
    Closes-Bug: 1715624

Corey Bryant (corey.bryant)
Changed in charm-ceilometer-agent:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Corey Bryant (corey.bryant)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-ceilometer-agent (master)

Fix proposed to branch: master
Review: https://review.openstack.org/501841

Changed in charm-ceilometer-agent:
status: Triaged → In Progress
Changed in charm-nova-compute:
status: Triaged → In Progress
James Page (james-page)
Changed in charm-openstack-dashboard:
assignee: nobody → James Page (james-page)
Changed in charm-keystone:
assignee: nobody → James Page (james-page)
Changed in charm-heat:
assignee: nobody → James Page (james-page)
Changed in charm-ceilometer:
assignee: nobody → James Page (james-page)
Changed in charm-ceilometer-agent:
milestone: none → 17.08
Changed in charm-keystone:
status: Triaged → In Progress
OpenStack Infra (hudson-openstack)
Changed in charm-nova-cloud-controller:
status: Triaged → In Progress
Corey Bryant (corey.bryant)
Changed in charm-cinder:
status: New → Triaged
Changed in charm-glance:
status: New → Triaged
Changed in charm-cinder:
importance: Undecided → Medium
Changed in charm-glance:
importance: Undecided → Medium
Changed in charm-cinder:
assignee: nobody → Corey Bryant (corey.bryant)
Changed in charm-glance:
assignee: nobody → Corey Bryant (corey.bryant)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/501881

Changed in charm-glance:
status: Triaged → In Progress
Changed in charm-cinder:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/501882

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-keystone (master)

Reviewed: https://review.openstack.org/501790
Committed: https://git.openstack.org/cgit/openstack/charm-keystone/commit/?id=7fb7ff9063e5b7487694902cf68e078c3780ebcf
Submitter: Jenkins
Branch: master

commit 7fb7ff9063e5b7487694902cf68e078c3780ebcf
Author: James Page <email address hidden>
Date: Thu Sep 7 17:10:15 2017 +0100

    Ensure os_release is reset during upgrades

    Reset the os_release cache during the OpenStack upgrade process,
    ensuring that any post dist-upgrade operations are made in the
    context of the new OpenStack release, not the old one.

    Change-Id: I3d3584dd8e97f85e16c38e1143f627b03fa63bd0
    Closes-Bug: 1715624

Changed in charm-keystone:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-compute (master)

Reviewed: https://review.openstack.org/501712
Committed: https://git.openstack.org/cgit/openstack/charm-nova-compute/commit/?id=fa7655baeb9dc2c0ccdd3ca4d8d1c7575eec7675
Submitter: Jenkins
Branch: master

commit fa7655baeb9dc2c0ccdd3ca4d8d1c7575eec7675
Author: Corey Bryant <email address hidden>
Date: Thu Sep 7 12:03:58 2017 +0000

    Reset os_release cached value after upgrade

    In enable_memcache, ensure that the os_release cache is reset
    after upgrade.

    Change-Id: If25be2df23b08b69ad3ee28c53d7dce1de509c6e
    Partial-Bug: 1715624

Changed in charm-nova-cloud-controller:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-nova-cloud-controller (master)

Reviewed: https://review.openstack.org/501813
Committed: https://git.openstack.org/cgit/openstack/charm-nova-cloud-controller/commit/?id=0161dde79660a248742e41e67f92ed13af561dd7
Submitter: Jenkins
Branch: master

commit 0161dde79660a248742e41e67f92ed13af561dd7
Author: David Ames <email address hidden>
Date: Thu Sep 7 09:13:12 2017 -0700

    Resolve OS upgrade bugs

    Ensure that the os_release cache is cleared during the openstack
    upgrade process, ensuring that package list and configuration
    options are correctly set for the new OpenStack version.

    Ensure that map_instances gets run on upgrade. This updates the
    nova_api.instance_mappings table with pre-existing instances.

    Change-Id: Idfcdc48c25b24d0cc9ded3eda2bc4d13d3b04f6d
    Closes-Bug: 1715624

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-glance (master)

Reviewed: https://review.openstack.org/501881
Committed: https://git.openstack.org/cgit/openstack/charm-glance/commit/?id=624c77ca594bf38437e9d8700e10db8f8e53ab4b
Submitter: Jenkins
Branch: master

commit 624c77ca594bf38437e9d8700e10db8f8e53ab4b
Author: Corey Bryant <email address hidden>
Date: Thu Sep 7 19:40:13 2017 +0000

    Reset os_release cached value after upgrade

    In enable_memcache, ensure that the os_release cache is reset
    after upgrade.

    Change-Id: Id4d2a7f2d9c83550c0f82130306a21da7073549f
    Partial-Bug: 1715624

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-cinder (master)

Reviewed: https://review.openstack.org/501882
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=0524e9e4002f666350f932ffc6d744c1131c576a
Submitter: Jenkins
Branch: master

commit 0524e9e4002f666350f932ffc6d744c1131c576a
Author: Corey Bryant <email address hidden>
Date: Thu Sep 7 19:38:23 2017 +0000

    Reset os_release cached value after upgrade

    In enable_memcache, ensure that the os_release cache is reset
    after upgrade.

    Change-Id: I728da2d1e9675baeb52d43e6bef523ffd12fde2a
    Partial-Bug: 1715624

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-ceilometer-agent (master)

Reviewed: https://review.openstack.org/501841
Committed: https://git.openstack.org/cgit/openstack/charm-ceilometer-agent/commit/?id=2c4766575866057eb191f2fe8e79e6005312c844
Submitter: Jenkins
Branch: master

commit 2c4766575866057eb191f2fe8e79e6005312c844
Author: Corey Bryant <email address hidden>
Date: Thu Sep 7 15:46:11 2017 +0000

    Reset os_release cached value after upgrade

    In enable_memcache, ensure that the os_release cache is reset
    after upgrade.

    Additionally, call apt_install(get_packages()) for upgrades
    to ensure that release-specific packages (such as memcached)
    are installed.

    Change-Id: Ia3689d2d0c549fc55f1bd533654f80c98ce5462a
    Partial-Bug: 1715624

James Page (james-page)
Changed in charm-openstack-dashboard:
status: Fix Committed → Fix Released
Changed in charm-keystone:
status: Fix Committed → Fix Released
Changed in charm-nova-cloud-controller:
status: Fix Committed → Fix Released
Changed in charm-ceilometer:
status: Fix Committed → Fix Released
Changed in charm-heat:
status: Fix Committed → Fix Released
James Page (james-page)
Changed in charm-nova-compute:
milestone: 17.08 → 17.11
Changed in charm-ceilometer-agent:
milestone: 17.08 → 17.11
James Page (james-page)
Changed in charm-nova-compute:
milestone: 17.11 → 18.02
Changed in charm-ceilometer-agent:
milestone: 17.11 → 18.02
Ryan Beisner (1chb1n)
Changed in charm-nova-compute:
milestone: 18.02 → 18.05
Changed in charm-ceilometer-agent:
milestone: 18.02 → 18.05
David Ames (thedac)
Changed in charm-nova-compute:
milestone: 18.05 → 18.08
Changed in charm-ceilometer-agent:
milestone: 18.05 → 18.08
James Page (james-page)
Changed in charm-nova-compute:
milestone: 18.08 → 18.11
Changed in charm-ceilometer-agent:
milestone: 18.08 → 18.11
David Ames (thedac)
Changed in charm-nova-compute:
milestone: 18.11 → 19.04
Changed in charm-ceilometer-agent:
milestone: 18.11 → 19.04
David Ames (thedac)
Changed in charm-nova-compute:
milestone: 19.04 → 19.07
Changed in charm-ceilometer-agent:
milestone: 19.04 → 19.07
David Ames (thedac)
Changed in charm-nova-compute:
milestone: 19.07 → 19.10
Changed in charm-ceilometer-agent:
milestone: 19.07 → 19.10
David Ames (thedac)
Changed in charm-nova-compute:
milestone: 19.10 → 20.01
Changed in charm-ceilometer-agent:
milestone: 19.10 → 20.01
Alex Kavanagh (ajkavanagh)
tags: added: openstack-upgrade
James Page (james-page)
Changed in charm-nova-compute:
milestone: 20.01 → 20.05
Changed in charm-ceilometer-agent:
milestone: 20.01 → 20.05
David Ames (thedac)
Changed in charm-nova-compute:
milestone: 20.05 → 20.08
Changed in charm-ceilometer-agent:
milestone: 20.05 → 20.08
James Page (james-page)
Changed in charm-nova-compute:
milestone: 20.08 → none
Changed in charm-ceilometer-agent:
milestone: 20.08 → none
Liam Young (gnuoy)
Changed in charm-ceilometer-agent:
status: In Progress → Fix Released
Changed in charm-cinder:
status: In Progress → Fix Released
Changed in charm-glance:
status: In Progress → Fix Released
Changed in charm-nova-compute:
status: In Progress → Fix Released
Changed in charm-ceilometer-agent:
milestone: none → 17.08
Changed in charm-cinder:
milestone: none → 17.08
Changed in charm-glance:
milestone: none → 17.08
Changed in charm-nova-compute:
milestone: none → 17.08
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.