Glance

Image import fails with python 3.5, image stuck in importing state

Bug #1759510 reported by Abhishek Kekane on 2018-03-28

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Glance	Fix Released	Critical	Abhishek Kekane	Glance rocky-1 "17.0.0.0b1"
	Queens	Fix Committed	Critical	Abhishek Kekane	Glance queens-stable-1 "16.0.1"

Bug Description

New image import api with glance-direct or web-download fails to import image if cloud is using python3.5. Image stuck in importing state forever.

Steps:
1. Ensure you are running glance on python 3.5
   Add below lines in your devstack/local.conf
   WSGI_MODE=mod_wsgi
   USE_PYTHON3=True
   PYTHON3_VERSION=3.5

2. Source devstack/openrc using "$ source devstack/openrc admin admin"

3. Create image using new import api
$ glance image-create-via-import --container-format ami --disk-format ami --name cirros_image --file <file_path>

g-api logs:

Mar 28 08:36:01 ubuntu-3 glance-api[23449]: |__Flow 'api_image_import': TypeError: Unicode-objects must be encoded before hashing
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor Traceback (most recent call last):
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor result = task.execute(**arguments)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/async/flows/api_image_import.py", line 218, in execute
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor image_import.set_image_data(image, file_path or self.uri, self.task_id)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/common/scripts/image_import/main.py", line 154, in set_image_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor "task_id": task_id})
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.force_reraise()
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor six.reraise(self.type_, self.value, self.tb)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor raise value
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/common/scripts/image_import/main.py", line 146, in set_image_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor image.set_data(data_iter)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/domain/proxy.py", line 195, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.base.set_data(data, size)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/notifier.py", line 480, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor _send_notification(notify_error, 'image.upload', msg)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.force_reraise()
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor six.reraise(self.type_, self.value, self.tb)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor raise value
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/notifier.py", line 427, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.repo.set_data(data, size)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/api/policy.py", line 193, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor return self.image.set_data(*args, **kwargs)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/quota/__init__.py", line 304, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.image.set_data(data, size=size)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/opt/stack/glance/glance/location.py", line 439, in set_data
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor verifier=verifier)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/backend.py", line 453, in add_to_backend
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor verifier)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/backend.py", line 426, in store_add_to_backend
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor verifier=verifier)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/capabilities.py", line 225, in op_checker
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor return store_op_fun(store, *args, **kwargs)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/_drivers/filesystem.py", line 697, in add
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self._delete_partial(filepath, image_id)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor self.force_reraise()
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor six.reraise(self.type_, self.value, self.tb)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor raise value
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor File "/usr/local/lib/python3.5/dist-packages/glance_store/_drivers/filesystem.py", line 684, in add
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor checksum.update(buf)
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor TypeError: Unicode-objects must be encoded before hashing
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: ERROR glance.async.taskflow_executor
Mar 28 08:36:01 ubuntu-3 glance-api[23449]: DEBUG glance.async.taskflow_executor [-] Task 'api_image_import-ImportToStore-a4ef0f11-2e4f-4ed8-ac4a-cbb28293a513' (74506991-d9ed-4fdf-819a-21dcadca3c04) transitioned into state 'REVERTING' from state 'FAILURE' {{(pid=23619) _task_receiver /usr/local/lib/python3.5/dist-packages/taskflow/listeners/logging.py:194}}

See original description

Tags:

Abhishek Kekane (abhishek-kekane) on 2018-03-28

Changed in glance:
assignee:	nobody → Abhishek Kekane (abhishek-kekane)

Abhishek Kekane (abhishek-kekane) on 2018-03-28

summary:	- Image import fails with python 3.5, image stuck in uploading state + Image import fails with python 3.5, image stuck in importing state
description:	updated

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-03-29: Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/557673

Changed in glance:
status:	New → In Progress

Brian Rosmaita (brian-rosmaita) on 2018-04-02

tags:

added: queens-backport-potential

Brian Rosmaita (brian-rosmaita) on 2018-04-02

Changed in glance:
importance:	Undecided → Critical

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-04-03: Fix merged to glance (master)

Reviewed: https://review.openstack.org/557673
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=ab4139e30ca56425fb4f62ebf2d5cb48a0455952
Submitter: Zuul
Branch: master

commit ab4139e30ca56425fb4f62ebf2d5cb48a0455952
Author: Abhishek Kekane <email address hidden>
Date: Thu Mar 29 09:54:19 2018 +0000

Python 3.5: Image Import fails with Unicode Error

Newly added image import api fails for python 3.5 as it
raises Unicode error as it fails to open file in binary mode.

Added 'rb' mode while opening the file which will open the provided
file in binary mode.

Change-Id: I297a415d6fa8700c6a131e0931d11bda65265c73
Closes-Bug: #1759510

Changed in glance:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-04-03: Fix proposed to glance (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/558359

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-04-04: Fix merged to glance (stable/queens)

Reviewed: https://review.openstack.org/558359
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=0da269128ed14096315c9b9e54679fd7e85e9b9b
Submitter: Zuul
Branch: stable/queens

commit 0da269128ed14096315c9b9e54679fd7e85e9b9b
Author: Abhishek Kekane <email address hidden>
Date: Thu Mar 29 09:54:19 2018 +0000

Python 3.5: Image Import fails with Unicode Error

Newly added image import api fails for python 3.5 as it
raises Unicode error as it fails to open file in binary mode.

Added 'rb' mode while opening the file which will open the provided
file in binary mode.

    Change-Id: I297a415d6fa8700c6a131e0931d11bda65265c73
    Closes-Bug: #1759510
    (cherry picked from commit ab4139e30ca56425fb4f62ebf2d5cb48a0455952)

Brian Rosmaita (brian-rosmaita) on 2018-04-05