Running image-import call on queued image having valid container and disk formats returns 500 internal server error

Bug #1733813 reported by Abhishek Kekane
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Fix Released
High
Abhishek Kekane

Bug Description

If you run image-import api on any image which is in queued state having valid container-format and disk-format set will return 500 error as it raises IOError: [Errno 2] No such file or directory: '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be'. Also image status changes from 'queued' to 'importing'. Ideally transition from queued to importing should not be allowed and it should return HTTP 409 conflict error to the user.

Prerequisites:
1. Ensure you have latest version of python-glanceclient (version 2.8.0) installed
2. Due to isssue [1] to execute taskflow you need to modify line [2] as shown below and restart glance-api service
   - pool.spawn_n(import_task.run, task_executor)
   + import_task.run(task_executor)
   [1] https://bugs.launchpad.net/glance/+bug/1712463
   [2] https://github.com/openstack/glance/blob/master/glance/api/v2/images.py#L106

Steps to reporoduce:
1. Create an image with valid container format and disk-format
    $ glance image-create --container-format ami --disk-format ami --name cirros_image
2. Ensure image is in queued state
3. Run image-import call
   $ glance image-import <ID of image created in 1st step> --import-method glance-direct

Output:
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)

Glance API Logs:

Nov 22 09:12:57 devstack <email address hidden>[14229]: pdict['tenant'] = self.tenant
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi [None req-52ea6328-83bf-4c25-a137-a51272308be9 admin admin] Caught error: [Errno 2] No such file or directory: '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be': IOError: [Errno 2] No such file or directory: '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be'
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi Traceback (most recent call last):
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/wsgi.py", line 1222, in __call__
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi request, **action_args)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/wsgi.py", line 1261, in dispatch
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi return method(*args, **kwargs)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/utils.py", line 363, in wrapped
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/api/v2/images.py", line 107, in import_image
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi import_task.run(task_executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/domain/proxy.py", line 238, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.base.run(executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/notifier.py", line 581, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi super(TaskProxy, self).run(executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/domain/proxy.py", line 238, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.base.run(executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/domain/proxy.py", line 238, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.base.run(executor)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/domain/__init__.py", line 439, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi executor.begin_processing(self.task_id)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/taskflow_executor.py", line 143, in begin_processing
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi super(TaskExecutor, self).begin_processing(task_id)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/__init__.py", line 63, in begin_processing
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self._run(task_id, task.type)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/taskflow_executor.py", line 181, in _run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.task_repo.save(task)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.force_reraise()
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/taskflow_executor.py", line 172, in _run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi engine.run()
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi for _state in self.run_iter(timeout=timeout):
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi failure.Failure.reraise_if_any(er_failures)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi failures[0].reraise()
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi six.reraise(*self._exc_info)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi result = task.execute(**arguments)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/async/flows/api_image_import.py", line 239, in execute
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi image_import.set_image_data(image, file_path or self.uri, self.task_id)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/scripts/image_import/main.py", line 154, in set_image_data
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi "task_id": task_id})
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi self.force_reraise()
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/scripts/image_import/main.py", line 145, in set_image_data
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi data_iter = script_utils.get_image_data_iter(uri)
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/scripts/utils.py", line 139, in get_image_data_iter
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi return open(uri, "r")
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi IOError: [Errno 2] No such file or directory: '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be'
Nov 22 09:12:57 devstack <email address hidden>[14229]: ERROR glance.common.wsgi

Changed in glance:
assignee: nobody → Abhishek Kekane (abhishek-kekane)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/522703

Changed in glance:
status: New → In Progress
Changed in glance:
importance: Undecided → High
milestone: none → queens-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (master)

Reviewed: https://review.openstack.org/522703
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=a19fdb0e5f47eb5ea6f5150e056f1df26fb8b7e9
Submitter: Zuul
Branch: master

commit a19fdb0e5f47eb5ea6f5150e056f1df26fb8b7e9
Author: Abhishek Kekane <email address hidden>
Date: Fri Nov 24 04:08:13 2017 +0000

    Fix 500 from image-import on queued images

    If you run image-import api on any image which is in queued state having
    valid container-format and disk-format set will return 500 error as it
    raises IOError: [Errno 2] No such file or directory:
    '/tmp/staging/567bfb61-d9f7-47e5-aa1a-90b7797e70be'. The reason is image
    does not have data stored in staging area.

    Restricted transition from queued to importing state and returned HTTP 409
    conflict error to the user.

    Change-Id: I3d3ab283fd0faf4b63365ff2dbeb714913f02b27
    Closes-Bug: #1733813

Changed in glance:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/glance 16.0.0.0b2

This issue was fixed in the openstack/glance 16.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.