Ubuntu Cloud Archive

UCA python-nova requires UCA python-oslo.middleware but doesn't specify in dependency

Bug #1830770 reported by Paul Goins
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Fix Released
Undecided
Chris MacNaughton
Ocata
Fix Released
Undecided
Chris MacNaughton

Bug Description

This occurred specifically on python-nova 2:15.1.5-0ubuntu1~cloud3, pulled from http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/ocata/main amd64 Packages.

Upon attempting to deploy some nova compute nodes, we ended up with several nodes which would not start up properly. /var/log/syslog had a traceback which seems to point at python-nova:

May 27 16:49:50 <HOSTNAME> systemd[1]: Starting OpenStack Compute...
May 27 16:49:50 <HOSTNAME> systemd[1]: Started OpenStack Compute.
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: Traceback (most recent call last):
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: File "/usr/bin/nova-compute", line 10, in <module>
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: sys.exit(main())
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: File "/usr/lib/python2.7/dist-packages/nova/cmd/compute.py", line 42, in main
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: config.parse_args(sys.argv)
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: File "/usr/lib/python2.7/dist-packages/nova/config.py", line 47, in parse_args
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: config.set_middleware_defaults()
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: File "/usr/lib/python2.7/dist-packages/nova/common/config.py", line 20, in set_middleware_defaults
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: cors.set_defaults(
May 27 16:49:51 <HOSTNAME> nova-compute[20093]: AttributeError: 'module' object has no attribute 'set_defaults'
May 27 16:49:51 <HOSTNAME> systemd[1]: nova-compute.service: Main process exited, code=exited, status=1/FAILURE
May 27 16:49:51 <HOSTNAME> systemd[1]: nova-compute.service: Unit entered failed state.
May 27 16:49:51 <HOSTNAME> systemd[1]: nova-compute.service: Failed with result 'exit-code'.

Upon deeper investigation, it appears that the system is running python-oslo.middleware version 3.8.0-2ubuntu1, rather than the UCA version 3.23.1-0ubuntu1.1~cloud0. As such, it's missing the set_defaults() function which is available in the UCA version.

The reason we ended up in this state: ubuntu-cloud-keyring was installed after python-nova, thus originally python-nova and python-oslo.middleware were both pulled from the distro. However, a package installed after the cloud keychain was available required an upgrade of python-nova. However, since the UCA python-nova simply requires "python-oslo.middleware (>= 3.0.0)", no update to python-oslo.middleware was pulled.

Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote :
Download full text (4.7 KiB)

I can confirm this happens:

root@test:~# apt-cache policy python-nova
python-nova:
  Installed: 2:13.1.4-0ubuntu4.5
  Candidate: 2:13.1.4-0ubuntu4.5
  Version table:
 *** 2:13.1.4-0ubuntu4.5 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
        100 /var/lib/dpkg/status
     2:13.0.0-0ubuntu2 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
root@test:~# apt-cache policy python-oslo.middleware
python-oslo.middleware:
  Installed: 3.8.0-2ubuntu1
  Candidate: 3.8.0-2ubuntu1
  Version table:
 *** 3.8.0-2ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
        100 /var/lib/dpkg/status
     3.8.0-2 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
root@test:~# add-apt-repository cloud-archive:ocata
<--snip-->
root@test:~# sudo apt install python-nova
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  python-ndg-httpsclient
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
  alembic os-brick-common python-alembic python-babel python-babel-localedata python-blinker python-ceilometerclient
  python-cffi python-glanceclient python-jwt python-keystoneauth1 python-microversion-parse python-oauthlib python-os-brick
  python-os-vif python-os-win python-os-xenapi python-oslo.concurrency python-oslo.config python-oslo.context python-oslo.db
  python-oslo.log python-oslo.messaging python-oslo.policy python-oslo.privsep python-oslo.service python-oslo.utils
  python-oslo.versionedobjects python-osprofiler python-ply python-positional python-pycparser python-pyparsing
  python-requests python-stevedore python-tenacity python-urllib3 python-webob
Suggested packages:
  python-editor python-blinker-doc python-ceilometerclient-doc python-dev python-glanceclient-doc python-keystoneauth1-doc
  python-requests-kerberos python-microversion-parse-doc python-ldap python-os-brick-doc python-os-vif-doc python-os-win-doc
  python-os-xenapi-doc python-oslo.concurrency-doc python-oslo.log-doc python-kafka python-zmq python-oslo.policy-doc
  python-oslo.privsep-doc python-oslo.service-doc python-oslo.versionedobjects-doc python-ply-doc python-positional-doc cpp
  python-pyparsing-doc python-socks python-tenacity-doc python-ntlm python-webob-doc
The following NEW packages will be installed:
  python-blinker python-ceilometerclient python-cffi python-jwt python-microversion-parse python-oauthlib python-os-vif
  python-os-xenapi python-oslo.privsep python-osprofiler python-ply python-pycparser python-tenacity
The following packages will be upgraded:
  alembic os-brick-common python-alembic python-babel python-babel-localedata python-glanceclient python-keystoneauth1
  python-nova python-os-brick python-os-win python-oslo.concurrency python-oslo.config python-oslo.context python-oslo.db
  python-oslo.log python-oslo.messagin...

Read more...

Changed in cloud-archive:
status: New → Confirmed
Chris MacNaughton (chris.macnaughton)
Changed in cloud-archive:
status: Confirmed → In Progress
assignee: nobody → Chris MacNaughton (chris.macnaughton)
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Thanks Chris, new version of nova with your fix is now building in ocata-staging.
https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/ocata-staging/+packages?field.name_filter=nova&field.status_filter=published&field.series_filter=

Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote :

I've confirmed that doing the same task on Pike doesn't exhibit this bug.

Revision history for this message
Corey Bryant (corey.bryant) wrote : Please test proposed package

Hello Paul, or anyone else affected,

Accepted nova into ocata-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:ocata-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ocata-needed to verification-ocata-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ocata-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-ocata-needed
Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote :

Using the Ocata staging PPA, the python-oslo.middleware package is properly updated:

root@test:~# apt-cache policy python-nova
python-nova:
  Installed: 2:13.1.4-0ubuntu4.5
  Candidate: 2:15.1.5-0ubuntu1~cloud6
  Version table:
     2:15.1.5-0ubuntu1~cloud6 500
        500 http://ppa.launchpad.net/ubuntu-cloud-archive/ocata-staging/ubuntu xenial/main amd64 Packages
     2:15.1.5-0ubuntu1~cloud5 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/ocata/main amd64 Packages
 *** 2:13.1.4-0ubuntu4.5 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
        100 /var/lib/dpkg/status
     2:13.0.0-0ubuntu2 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
root@test:~# apt-cache policy python-oslo.middleware
python-oslo.middleware:
  Installed: 3.8.0-2ubuntu1
  Candidate: 3.23.1-0ubuntu1.1~cloud0
  Version table:
     3.23.1-0ubuntu1.1~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/ocata/main amd64 Packages
     3.23.1-0ubuntu1.1~cloud0 500
        500 http://ppa.launchpad.net/ubuntu-cloud-archive/ocata-staging/ubuntu xenial/main amd64 Packages
 *** 3.8.0-2ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
        100 /var/lib/dpkg/status
     3.8.0-2 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
root@test:~# apt install python-nova -y
<--snip-->
root@test:~# apt-cache policy python-nova
python-nova:
  Installed: 2:15.1.5-0ubuntu1~cloud6
  Candidate: 2:15.1.5-0ubuntu1~cloud6
  Version table:
 *** 2:15.1.5-0ubuntu1~cloud6 500
        500 http://ppa.launchpad.net/ubuntu-cloud-archive/ocata-staging/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status
     2:15.1.5-0ubuntu1~cloud5 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/ocata/main amd64 Packages
     2:13.1.4-0ubuntu4.5 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     2:13.0.0-0ubuntu2 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
root@test:~# apt-cache policy python-oslo.middleware
python-oslo.middleware:
  Installed: 3.23.1-0ubuntu1.1~cloud0
  Candidate: 3.23.1-0ubuntu1.1~cloud0
  Version table:
 *** 3.23.1-0ubuntu1.1~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/ocata/main amd64 Packages
        100 /var/lib/dpkg/status
     3.23.1-0ubuntu1.1~cloud0 500
        500 http://ppa.launchpad.net/ubuntu-cloud-archive/ocata-staging/ubuntu xenial/main amd64 Packages
     3.8.0-2ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     3.8.0-2 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages

Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote :

Following the same process as above, pointing at ocata-proposed, I see the the python-nova upgrade successfully pull in the oslo.middleware package as well:

root@test:~# apt-cache policy python-nova
python-nova:
  Installed: 2:15.1.5-0ubuntu1~cloud6
  Candidate: 2:15.1.5-0ubuntu1~cloud6
  Version table:
 *** 2:15.1.5-0ubuntu1~cloud6 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-proposed/ocata/main amd64 Packages
        100 /var/lib/dpkg/status
     2:13.1.4-0ubuntu4.5 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     2:13.0.0-0ubuntu2 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
root@test:~# apt-cache policy python-oslo.middleware
python-oslo.middleware:
  Installed: 3.23.1-0ubuntu1.1~cloud0
  Candidate: 3.23.1-0ubuntu1.1~cloud0
  Version table:
 *** 3.23.1-0ubuntu1.1~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-proposed/ocata/main amd64 Packages
        100 /var/lib/dpkg/status
     3.8.0-2ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     3.8.0-2 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages

tags: added: verification-ocata-done
removed: verification-ocata-needed
Revision history for this message
Corey Bryant (corey.bryant) wrote : Update Released

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package nova - 2:15.1.5-0ubuntu1~cloud6
---------------

 nova (2:15.1.5-0ubuntu1~cloud6) xenial-ocata; urgency=medium
 .
   * d/control: Ensure that oslo.middleware dependency version is upgraded
     to proper version for ocata (LP: #1830770).

Chris MacNaughton (chris.macnaughton)
Changed in cloud-archive:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.