OpenStack Object Storage (swift)

test_connect_put_node_timeout failed in gate/check job

Bug #1302781 reported by Peter Portante on 2014-04-04

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Object Storage (swift)	Fix Released	Undecided	Unassigned	OpenStack Object Storage (swift) 2.0.0

Bug Description

See: http://logs.openstack.org/39/84339/3/check/gate-swift-python27/aa3d7a6/console.html

2014-04-04 11:34:31.587 | ======================================================================
2014-04-04 11:34:31.587 | FAIL: test_connect_put_node_timeout (test.unit.proxy.controllers.test_obj.TestObjControllerWriteAffinity)
2014-04-04 11:34:31.588 | ----------------------------------------------------------------------
2014-04-04 11:34:31.588 | Traceback (most recent call last):
2014-04-04 11:34:31.588 | File "/home/jenkins/workspace/gate-swift-python27/test/unit/proxy/controllers/test_obj.py", line 89, in test_connect_put_node_timeout
2014-04-04 11:34:31.588 | self.assertTrue(res is None)
2014-04-04 11:34:31.589 | AssertionError: False is not true
2014-04-04 11:34:31.589 | 'False is not true' = self._formatMessage('False is not true', "%s is not true" % safe_repr(False))
2014-04-04 11:34:31.589 | >> raise self.failureException('False is not true')
2014-04-04 11:34:31.590 |

It looks like the "slow_connect" option for FakeConn adds a sleep delay of 0.1, which is the same value for the connect_timeout used by the test. It is possible that the two timeouts end up having the same time value, such that in some queue the connect timeout fires after the sleep() timeout, allowing the connection to be "made". However, there is little evidence to support this.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-04-09: Fix merged to swift (master)

Reviewed: https://review.openstack.org/85457
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=9d0067a0f546ce8a0fddb257655f2944c05e35e6
Submitter: Jenkins
Branch: master

commit 9d0067a0f546ce8a0fddb257655f2944c05e35e6
Author: Peter Portante <email address hidden>
Date: Fri Apr 4 15:26:32 2014 -0400

Attempt to ensure connect always timesout

    It seems that the test_connect_put_timeout() test does not always fail
    when it is expected. Sometimes, not very often, the attempt to connect
    succeeds, resulting in a failed test.

    This might be because the fake-connection infrastructure uses a
    sleep(0.1) and the test uses a connect timeout of 0.1. There might be a
    case where the two values result in the exact time where the entries
    happen to be added in the wrong order such that the sleep() completes
    first before the connect timeout fires, where the connect completes
    successfully.

Closes bug 1302781

Change-Id: Ie23e40cf294170eccdf0713e313f9a31a92f9071

Changed in swift:
status:	New → Fix Committed

Revision history for this message

Openstack Gerrit (openstack-gerrit) wrote on 2014-04-17: Fix proposed to swift (feature/ec)

Fix proposed to branch: feature/ec
Review: https://review.openstack.org/88359

Revision history for this message

Openstack Gerrit (openstack-gerrit) wrote on 2014-04-19: Fix merged to swift (feature/ec)

Download full text (8.0 KiB)

Reviewed: https://review.openstack.org/88359
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=282548d538153fa2fae1d2a4ec19d75dc2a41028
Submitter: Jenkins
Branch: feature/ec

commit 856c15539a958dabe3b8a1f22d305048ca39de9a
Author: Paul Luse <email address hidden>
Date: Mon Apr 14 15:19:22 2014 -0700

Fix testcase test_print_db_info_metadata()

    Test compares cluster info to hardcoded expected data and wasn't
    sorting the two sets of things being compared leading to some
    sporadic unit test failures.

Change-Id: I3ef98260a62c15d06ba8cc196196d4e90abca3f0

commit 67fff5b2973f5ca12316041664eddf3e8ece45be
Author: Madhuri Kumari <email address hidden>
Date: Fri Apr 11 11:53:48 2014 +0530

Print 'Container Count' in data base info

Currently, 'Container Count' was missing in data base info.
So this patch will help printing 'Container Count' also.

Change-Id: I1ca80ee79e71b086b30fd2d1ab024ea1cfb324f5

commit deb0dfd090cee2feef664bdd4449194c274d830e
Author: Peter Portante <email address hidden>
Date: Thu Apr 10 14:36:43 2014 -0400

Add includes of referenced SAIO bin scripts

Change-Id: I6810e69a757336a3aed0a38146c27f270fe2dde1

commit 2c2ede22338d6ca9637233076157f1d26248fbf1
Author: Chuck Thier <email address hidden>
Date: Thu Apr 10 18:59:01 2014 +0000

Fix logging issue when services stop on py26

    On older versions of python 2.6, exceptions would be spewed to the error
    log whenever a service would stop. This gets magnified by the
    container-updater which seems to do it with every pass. This catches
    and squelches the error.

Change-Id: I128c09c240e768e8195af1f6fe79b10d4e432471
Closes-Bug: #1306027

commit 5ff6a4d5d6cdbf39ba9f6d5d416cdd1c6c6a52ac
Author: Peter Portante <email address hidden>
Date: Mon Apr 7 13:01:44 2014 -0400

Use eventlet instead of threading for timeout

    The only explicit use of Python threading is found in the
    testFileSizeLimit test. Using eventlet seems a bit easier to follow,
    accomplishing the same goal, and does not constrain us to a
    multi-threaded environment.

    The chunks() and timeout() module level functions are only used by one
    test each, so we just move them to those tests to indicate they are not
    used globally.

Change-Id: I50b9fb798fbfd1d552b3c3f90309f6b86da34853

commit 58fe2f256fccc14f5078e9fde9b9f7b2219a06e5
Author: anc <email address hidden>
Date: Tue Apr 8 18:44:06 2014 +0100

Add tests and comments re constraints in /info

    Add test to check that only the expected keys are
    reported by proxy in /info, and add comments to
    raise awareness that default constraints will be
    automatically published by proxy in response to /info
    requests.

Change-Id: Ia5f6339b06cdc2e1dc960d1f75562a2505530202

commit c2744caac43586b745c43c37a9c31483f7a126fc
Author: Samuel Merritt <email address hidden>
Date: Tue Apr 8 11:44:58 2014 -0700

Fix deprecation warning

Accessing BaseException.message spews a warning; we can get the same
information with str(err), which does not spew.

Change-Id: I67...

Reviewed:  https://review.openstack.org/88359
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=282548d538153fa2fae1d2a4ec19d75dc2a41028
Submitter: Jenkins
Branch:    feature/ec

commit 856c15539a958dabe3b8a1f22d305048ca39de9a
Author: Paul Luse <paul.e.luse@intel.com>
Date:   Mon Apr 14 15:19:22 2014 -0700

Fix testcase test_print_db_info_metadata()
    
    Test compares cluster info to hardcoded expected data and wasn't
    sorting the two sets of things being compared leading to some
    sporadic unit test failures.
    
    Change-Id: I3ef98260a62c15d06ba8cc196196d4e90abca3f0

commit 67fff5b2973f5ca12316041664eddf3e8ece45be
Author: Madhuri Kumari <madhuri.rai07@gmail.com>
Date:   Fri Apr 11 11:53:48 2014 +0530

Print 'Container Count' in data base info
    
    Currently, 'Container Count' was missing in data base info.
    So this patch will help printing 'Container Count' also.
    
    Change-Id: I1ca80ee79e71b086b30fd2d1ab024ea1cfb324f5

commit deb0dfd090cee2feef664bdd4449194c274d830e
Author: Peter Portante <peter.portante@redhat.com>
Date:   Thu Apr 10 14:36:43 2014 -0400

Add includes of referenced SAIO bin scripts
    
    Change-Id: I6810e69a757336a3aed0a38146c27f270fe2dde1

commit 2c2ede22338d6ca9637233076157f1d26248fbf1
Author: Chuck Thier <cthier@gmail.com>
Date:   Thu Apr 10 18:59:01 2014 +0000

Fix logging issue when services stop on py26
    
    On older versions of python 2.6, exceptions would be spewed to the error
    log whenever a service would stop.  This gets magnified by the
    container-updater which seems to do it with every pass.  This catches
    and squelches the error.
    
    Change-Id: I128c09c240e768e8195af1f6fe79b10d4e432471
    Closes-Bug: #1306027

commit 5ff6a4d5d6cdbf39ba9f6d5d416cdd1c6c6a52ac
Author: Peter Portante <peter.portante@redhat.com>
Date:   Mon Apr 7 13:01:44 2014 -0400

Use eventlet instead of threading for timeout
    
    The only explicit use of Python threading is found in the
    testFileSizeLimit test. Using eventlet seems a bit easier to follow,
    accomplishing the same goal, and does not constrain us to a
    multi-threaded environment.
    
    The chunks() and timeout() module level functions are only used by one
    test each, so we just move them to those tests to indicate they are not
    used globally.
    
    Change-Id: I50b9fb798fbfd1d552b3c3f90309f6b86da34853

commit 58fe2f256fccc14f5078e9fde9b9f7b2219a06e5
Author: anc <alistair.coles@hp.com>
Date:   Tue Apr 8 18:44:06 2014 +0100

Add tests and comments re constraints in /info
    
    Add test to check that only the expected keys are
    reported by proxy in /info, and add comments to
    raise awareness that default constraints will be
    automatically published by proxy in response to /info
    requests.
    
    Change-Id: Ia5f6339b06cdc2e1dc960d1f75562a2505530202

commit c2744caac43586b745c43c37a9c31483f7a126fc
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Apr 8 11:44:58 2014 -0700

Fix deprecation warning
    
    Accessing BaseException.message spews a warning; we can get the same
    information with str(err), which does not spew.
    
    Change-Id: I67648d53d25522be074a78c44b3ce97dc27d1f07

commit d47a582e749d3a61071ff249175d67e37bef7e6c
Author: John Dickinson <me@not.mn>
Date:   Tue Apr 8 10:26:40 2014 -0700

update setup.py with pbr version
    
    This file is normally updated in the standard "update from
    global requirements" patch, but since those contain other
    changes we don't want, the setup.py file never gets updated.
    
    This patch updates setup.py to bring it in line with the
    standard global requirements way of doing things without also
    updating [test-]requirements.txt
    
    Change-Id: Id0e4f8e17dd4c714b4d2dd75985431a9e12e6a4f

commit 677ee3c931b5d307e44416b431207495435bf402
Author: anc <alistair.coles@hp.com>
Date:   Mon Apr 7 18:42:50 2014 +0100

Add missing constraints to /info
    
    Two of the default constraints, max_header_size and
    max_meta_overall_size, don't get registered for the
    /info response by the proxy server.
    
    Rather than adding them individually to the proxy's
    register_swift_info call, this patch proposes to
    register all the constraints for /info at once
    using the constraints.EFFECTIVE_CONSTRAINTS dict.
    Any future additions to default constraints will
    then be automatically included in /info.
    
    Change-Id: I5c10d4c8eda90ba94745b6f89df85aafbb50f8ef

commit d32dc8d49c19de8b1264cae23b82b3ffd9707454
Author: Greg Lange <greglange@gmail.com>
Date:   Wed Mar 26 22:55:55 2014 +0000

Unify backend logging
    
    Make account, object, and container servers construct log lines using the
    same utility function so they will produce identically formatted lines.
    
    This change reorders the fields logged for the account server.
    
    This change also adds the "additional info" field to the two servers that
    didn't log that field.  This makes the log lines identical across all 3
    servers.  If people don't like that, I can take that out.  I think it makes
    the documentation, parsing of the log lines, and the code a tad cleaner.
    
    DocImpact
    
    Change-Id: I268dc0df9dd07afa5382592a28ea37b96c6c2f44
    Closes-Bug: 1280955

commit b67a4b993815f1830b59054ae0d08a468044a53e
Author: Gil Vernik <gilv@il.ibm.com>
Date:   Mon Apr 7 08:44:54 2014 +0300

Support for content-length in the upload object method for
    internal client.
    
    Internal client uses 'chunked' transfer encoding in it's
    upload_object method. If content-length is known in advance,
    internal_client should allowed to use content length in the
    upload_object method and not the chunked method.
    
    Change-Id: I68d2ebde78e01fa16b7187a2f045ea20f4310722

commit b9b5fef89af51c66905de33e2436c063f4b09d36
Author: James Page <james.page@ubuntu.com>
Date:   Sat Apr 5 09:38:12 2014 +0100

Set permissions on generated ring files
    
    The use of NamedTemporaryFile creates rings with permissions 0600;
    however most installs probably generate the rings as root but the
    swift-proxy runs as user swift.
    
    Set the permissions on the generated ring to 0644 prior to rename so
    that the swift user can read the rings.
    
    Change-Id: Ia511931f471c5c9840012c3a75b89c1f35b1b245
    Closes-Bug: #1302700

commit 9d0067a0f546ce8a0fddb257655f2944c05e35e6
Author: Peter Portante <peter.portante@redhat.com>
Date:   Fri Apr 4 15:26:32 2014 -0400

Attempt to ensure connect always timesout
    
    It seems that the test_connect_put_timeout() test does not always fail
    when it is expected. Sometimes, not very often, the attempt to connect
    succeeds, resulting in a failed test.
    
    This might be because the fake-connection infrastructure uses a
    sleep(0.1) and the test uses a connect timeout of 0.1. There might be a
    case where the two values result in the exact time where the entries
    happen to be added in the wrong order such that the sleep() completes
    first before the connect timeout fires, where the connect completes
    successfully.
    
    Closes bug 1302781
    
    Change-Id: Ie23e40cf294170eccdf0713e313f9a31a92f9071

commit f63b37572df9188f2fc753c06f32488a2b2f59a9
Author: Peter Portante <peter.portante@redhat.com>
Date:   Tue Apr 1 22:56:43 2014 -0400

Update callback with proper bytes transferred
    
    The first set of bytes transferred was being dropped.
    
    Change-Id: I8e055190d04af1718c5e88bf5e8c44fde6794c74

commit 62a1e7e0593723ff19a83c1533b02f6f444f6942
Author: Samuel Merritt <sam@swiftstack.com>
Date:   Tue Mar 4 11:52:48 2014 -0800

Support If-[Un]Modified-Since for object HEAD
    
    We already supported it for object GET requests, but not for
    HEAD. This lets clients keep metadata up-to-date without having to
    either fetch the whole object when it's changed or do their own date
    parsing. They can just treat Last-Modified as opaque and update their
    idea of metadata when they get a 200.
    
    Change-Id: Iff25d8989a93d651fd2c327e1e58036e79e1bde1

Thierry Carrez (ttx) on 2014-06-21

Changed in swift:
status:	Fix Committed → Fix Released
milestone:	none → 2.0.0

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.