neutron

neutron.tests.unit.objects.test_subnet.SubnetDbObjectTestCase.test_db_obj failing with <NetworkSegment at 0x7fdf760a7310> is not bound to a Session

Bug #1770452 reported by Miguel Lavalle
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Low
Ihar Hrachyshka

Bug Description

Today I found the same failure in two unrelated patches:

http://logs.openstack.org/61/567461/1/check/openstack-tox-py27/1bbd80a/job-output.txt.gz#_2018-05-10_03_52_48_299412

http://logs.openstack.org/73/566673/2/gate/openstack-tox-cover/a8914e0/job-output.txt.gz#_2018-05-10_12_42_13_309505

The traceback looks as follows:

2018-05-10 12:42:13.309505 | ubuntu-xenial | FAIL: neutron.tests.unit.objects.test_subnet.SubnetDbObjectTestCase.test_db_obj
2018-05-10 12:42:13.309555 | ubuntu-xenial | tags: worker-1
2018-05-10 12:42:13.309605 | ubuntu-xenial | ----------------------------------------------------------------------
2018-05-10 12:42:13.309632 | ubuntu-xenial | Traceback (most recent call last):
2018-05-10 12:42:13.309666 | ubuntu-xenial | File "neutron/tests/base.py", line 140, in func
2018-05-10 12:42:13.309692 | ubuntu-xenial | return f(self, *args, **kwargs)
2018-05-10 12:42:13.309739 | ubuntu-xenial | File "neutron/tests/unit/objects/test_base.py", line 2017, in test_db_obj
2018-05-10 12:42:13.309768 | ubuntu-xenial | old_model = copy.deepcopy(obj.db_obj)
2018-05-10 12:42:13.309806 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 190, in deepcopy
2018-05-10 12:42:13.309833 | ubuntu-xenial | y = _reconstruct(x, rv, 1, memo)
2018-05-10 12:42:13.309897 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct
2018-05-10 12:42:13.309928 | ubuntu-xenial | state = deepcopy(state, memo)
2018-05-10 12:42:13.309967 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 163, in deepcopy
2018-05-10 12:42:13.310010 | ubuntu-xenial | y = copier(x, memo)
2018-05-10 12:42:13.310066 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict
2018-05-10 12:42:13.310141 | ubuntu-xenial | y[deepcopy(key, memo)] = deepcopy(value, memo)
2018-05-10 12:42:13.310187 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 190, in deepcopy
2018-05-10 12:42:13.310214 | ubuntu-xenial | y = _reconstruct(x, rv, 1, memo)
2018-05-10 12:42:13.310255 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct
2018-05-10 12:42:13.310280 | ubuntu-xenial | state = deepcopy(state, memo)
2018-05-10 12:42:13.310318 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 163, in deepcopy
2018-05-10 12:42:13.310338 | ubuntu-xenial | y = copier(x, memo)
2018-05-10 12:42:13.310380 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict
2018-05-10 12:42:13.310414 | ubuntu-xenial | y[deepcopy(key, memo)] = deepcopy(value, memo)
2018-05-10 12:42:13.310452 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 190, in deepcopy
2018-05-10 12:42:13.310479 | ubuntu-xenial | y = _reconstruct(x, rv, 1, memo)
2018-05-10 12:42:13.310519 | ubuntu-xenial | File "/usr/lib/python2.7/copy.py", line 352, in _reconstruct
2018-05-10 12:42:13.310538 | ubuntu-xenial | y.append(item)
2018-05-10 12:50:00.920204 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/collections.py", line 1044, in append
2018-05-10 12:50:00.920385 | ubuntu-xenial | item = __set(self, item, _sa_initiator)
2018-05-10 12:50:00.920620 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/collections.py", line 1016, in __set
2018-05-10 12:50:00.920754 | ubuntu-xenial | item = executor.fire_append_event(item, _sa_initiator)
2018-05-10 12:50:00.920975 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/collections.py", line 680, in fire_append_event
2018-05-10 12:50:00.921021 | ubuntu-xenial | item, initiator)
2018-05-10 12:50:00.921251 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 943, in fire_append_event
2018-05-10 12:50:00.921335 | ubuntu-xenial | state, value, initiator or self._append_token)
2018-05-10 12:50:00.921566 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 1228, in emit_backref_from_collection_append_event
2018-05-10 12:50:00.921628 | ubuntu-xenial | passive=PASSIVE_NO_FETCH)
2018-05-10 12:50:00.921845 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 624, in append
2018-05-10 12:50:00.921964 | ubuntu-xenial | self.set(state, dict_, value, initiator, passive=passive)
2018-05-10 12:50:00.922173 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 798, in set
2018-05-10 12:50:00.922227 | ubuntu-xenial | LOAD_AGAINST_COMMITTED)
2018-05-10 12:50:00.922414 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 599, in get
2018-05-10 12:50:00.922479 | ubuntu-xenial | value = self.callable_(state, passive)
2018-05-10 12:50:00.922680 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/neutron/.tox/cover/local/lib/python2.7/site-packages/sqlalchemy/orm/strategies.py", line 596, in _load_for_state
2018-05-10 12:50:00.922743 | ubuntu-xenial | (orm_util.state_str(state), self.key)
2018-05-10 12:50:00.923061 | ubuntu-xenial | sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <NetworkSegment at 0x7fdf760a7310> is not bound to a Session; lazy load operation of attribute 'network' cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)

Miguel Lavalle (minsel)
Changed in neutron:
importance: Undecided → Low
Revision history for this message
Brian Haley (brian-haley) wrote :

I noticed this in another patch as well, changing to Confirmed.

Changed in neutron:
status: New → Confirmed
Ihar Hrachyshka (ihar-hrachyshka)
tags: added: gate-failure unittest
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/568390

Changed in neutron:
assignee: nobody → Ihar Hrachyshka (ihar-hrachyshka)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/568390
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1a8a15f6309a56087702a0974041fa9850de4f62
Submitter: Zuul
Branch: master

commit 1a8a15f6309a56087702a0974041fa9850de4f62
Author: Ihar Hrachyshka <email address hidden>
Date: Mon May 14 21:57:52 2018 +0000

    objects: avoid deepcopying models in test_db_obj

    SQLAlchemy may asynchronously push models out of session cache in which
    case we may receive DetachedInstanceError.

    In the test case, instead of deepcopying models to compare, compare
    each modified attribute independently.

    This change also includes conversion from InstrumentedLists to regular
    lists when converting model attributes to object fields. The fact that
    we were returning InstrumentedLists was always an oversight but it
    revealed itself after the modification of the test case that is the
    core of this patch.

    When converting object fields to db, convert Port's distributed_binding
    None value to a empty list to reflect that the relationship of the Port
    database model is a list. It was not an issue before the patch because
    we were not comparing model attribute for equality but for in-equality
    before, and so None was always != [].

    Finally, this patch moves a bunch of TODOs to better reflect where they
    belong to.

    Closes-Bug: #1770452
    Change-Id: I42cdf540129bd4470ec1a59345db9845a6198328

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 13.0.0.0b2

This issue was fixed in the openstack/neutron 13.0.0.0b2 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/650516

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/queens)

Change abandoned by Doug Wiegley (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/650516
Reason: Needed by https://review.openstack.org/#/c/650525/. Since that won't go in, this can be skipped.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/queens)

Reviewed: https://review.opendev.org/650516
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8915febaaf956a13d4dc77a120b2608769cf394d
Submitter: Zuul
Branch: stable/queens

commit 8915febaaf956a13d4dc77a120b2608769cf394d
Author: Ihar Hrachyshka <email address hidden>
Date: Mon May 14 21:57:52 2018 +0000

    objects: avoid deepcopying models in test_db_obj

    SQLAlchemy may asynchronously push models out of session cache in which
    case we may receive DetachedInstanceError.

    In the test case, instead of deepcopying models to compare, compare
    each modified attribute independently.

    This change also includes conversion from InstrumentedLists to regular
    lists when converting model attributes to object fields. The fact that
    we were returning InstrumentedLists was always an oversight but it
    revealed itself after the modification of the test case that is the
    core of this patch.

    When converting object fields to db, convert Port's distributed_binding
    None value to a empty list to reflect that the relationship of the Port
    database model is a list. It was not an issue before the patch because
    we were not comparing model attribute for equality but for in-equality
    before, and so None was always != [].

    Finally, this patch moves a bunch of TODOs to better reflect where they
    belong to.

    Closes-Bug: #1770452
    Change-Id: I42cdf540129bd4470ec1a59345db9845a6198328
    (cherry picked from commit 1a8a15f6309a56087702a0974041fa9850de4f62)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 12.1.0

This issue was fixed in the openstack/neutron 12.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.