OpenStack Compute (nova)

NoMoreFloatingIps with stack information responsed

Bug #1103324 reported by Zhou ShaoYu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Undecided
Davanum Srinivas (DIMS)
OpenStack Compute (nova) 2013.1 "grizzly"
Folsom
Fix Released
Medium
Vish Ishaya
OpenStack Compute (nova) 2012.2.4

Bug Description

Allocate a floating IP when no floating IPs available, although HTTP code is correct,
exception message with stack information is responsed which may be not friendly.

{
    "itemNotFound": {
        "code": 404,
        "message": "NoMoreFloatingIps_Remote: Zero floating ips available.\nTraceback (most recent call last):\n\n File \"path/nova/openstack/common/rpc/amqp.py\", line 275, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n\n File \"path/nova/openstack/common/rpc/dispatcher.py\", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n\n File \"path/nova/network/manager.py\", line 279, in wrapped\n return func(self, context, *args, **kwargs)\n\n File \"path/nova/network/manager.py\", line 481, in allocate_floating_ip\n QUOTAS.rollback(context, reservations)\n\n File \"/usr/lib/python2.7/contextlib.py\", line 24, in __exit__\n self.gen.next()\n\n File \"path/nova/network/manager.py\", line 468, in allocate_floating_ip\n pool)\n\n File \"path/nova/db/api.py\", line 277, in floating_ip_allocate_address\n return IMPL.floating_ip_allocate_address(context, project_id, pool)\n\n File \"path/nova/db/sqlalchemy/api.py\", line 131, in wrapper\n return f(*args, **kwargs)\n\n File \"path/nova/db/sqlalchemy/api.py\", line 744, in floating_ip_allocate_address\n raise exception.NoMoreFloatingIps()\n\nNoMoreFloatingIps: Zero floating ips available.\n"
    }
}

The problem also meet when FloatingIpLimitExceeded raised.

{
    "overLimit": {
        "code": 413,
        "message": "FloatingIpLimitExceeded_Remote: Maximum number of floating ips exceeded\nTraceback (most recent call last):\n\n File \"path/nova/openstack/common/rpc/amqp.py\", line 275, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n\n File \"path/nova/openstack/common/rpc/dispatcher.py\", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n\n File \"path/nova/network/manager.py\", line 279, in wrapped\n return func(self, context, *args, **kwargs)\n\n File \"path/nova/network/manager.py\", line 463, in allocate_floating_ip\n raise exception.FloatingIpLimitExceeded()\n\nFloatingIpLimitExceeded: Maximum number of floating ips exceeded\n",
        "retryAfter": 0
    }
}

Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote :

Can you please add the logs from the nova-api log? i did manage to figure out a way to recreate the 2nd problem, but not the first, that would help.

thanks,
dims

Changed in nova:
status: New → Incomplete
Revision history for this message
Zhou ShaoYu (hzzhoushaoyu) wrote :
Download full text (7.9 KiB)

Try to set quota of floating IP as 0 and allocate a floating IP for user.

2013-01-25 09:43:42 INFO nova.api.openstack.wsgi [req-35cbe242-3345-4fb5-9a41-a32fea528282 f2665c1140c54a03a98110cb86262ec3 dc32392af0ae4a098fb7235760077fa6] POST http://localhost:8774/v1.1/dc32392af0ae4a098fb7235760077fa6/os-floating-ips
2013-01-25 09:43:42 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous call on network ... from (pid=25430) multicall path/nova/openstack/common/rpc/amqp.py:351
2013-01-25 09:43:42 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is 6f7f1740b68d40a0a94dde47eca33c62 from (pid=25430) multicall path/nova/openstack/common/rpc/amqp.py:354
2013-01-25 09:43:42 ERROR nova.api.openstack [req-35cbe242-3345-4fb5-9a41-a32fea528282 f2665c1140c54a03a98110cb86262ec3 dc32392af0ae4a098fb7235760077fa6] Caught error: Maximum number of floating ips exceeded
Traceback (most recent call last):

  File "path/nova/openstack/common/rpc/amqp.py", line 275, in _process_data
    rval = self.proxy.dispatch(ctxt, version, method, **args)

  File "path/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch
    return getattr(proxyobj, method)(ctxt, **kwargs)

  File "path/nova/network/manager.py", line 279, in wrapped
    return func(self, context, *args, **kwargs)

  File "path/nova/network/manager.py", line 463, in allocate_floating_ip
    raise exception.FloatingIpLimitExceeded()

FloatingIpLimitExceeded: Maximum number of floating ips exceeded
2013-01-25 09:43:42 TRACE nova.api.openstack Traceback (most recent call last):
2013-01-25 09:43:42 TRACE nova.api.openstack File "path/nova/api/openstack/__init__.py", line 78, in __call__
2013-01-25 09:43:42 TRACE nova.api.openstack return req.get_response(self.application)
2013-01-25 09:43:42 TRACE nova.api.openstack File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1053, in get_response
2013-01-25 09:43:42 TRACE nova.api.openstack application, catch_exc_info=False)
2013-01-25 09:43:42 TRACE nova.api.openstack File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1022, in call_application
2013-01-25 09:43:42 TRACE nova.api.openstack app_iter = application(self.environ, start_response)
2013-01-25 09:43:42 TRACE nova.api.openstack File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
2013-01-25 09:43:42 TRACE nova.api.openstack return resp(environ, start_response)
2013-01-25 09:43:42 TRACE nova.api.openstack File "/usr/local/lib/python2.7/dist-packages/keystone/middleware/auth_token.py", line 176, in __call__
2013-01-25 09:43:42 TRACE nova.api.openstack return self.app(env, start_response)
2013-01-25 09:43:42 TRACE nova.api.openstack File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
2013-01-25 09:43:42 TRACE nova.api.openstack return resp(environ, start_response)
2013-01-25 09:43:42 TRACE nova.api.openstack File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
2013-01-25 09:43:42 TRACE nova.api.openstack return resp(environ, start_response)
2013-01-25 09:43:42 TRACE nova.api.openstack File "/usr/local/lib/python2.7/dist-packages/webob/dec.p...

Read more...

Davanum Srinivas (DIMS) (dims-v)
Changed in nova:
assignee: nobody → Davanum Srinivas (DIMS) (dims-v)
status: Incomplete → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/20502

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
Sean Dague (sdague) wrote :

So we should also fix the stack traces, as they never should have been past.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/20502
Committed: http://github.com/openstack/nova/commit/fa52cb09b2270876c9d9a03106f961be6c9db834
Submitter: Jenkins
Branch: master

commit fa52cb09b2270876c9d9a03106f961be6c9db834
Author: Davanum Srinivas <email address hidden>
Date: Fri Jan 25 13:07:34 2013 -0500

    Strip out Traceback from HTTP response

    We should not let stack traces leak to the REST API users

    Fixes LP# 1103324
    Fixes LP# 1092610

    Change-Id: Ic1208bfeb7d44e672f8b8d3a4004a9802c14e7b9

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-3
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/folsom)

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/24471

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/24475

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/folsom)

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/24476

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/24475
Committed: http://github.com/openstack/nova/commit/d63bd8d692a9eb16caa818101cdbb3358f6b4681
Submitter: Jenkins
Branch: master

commit d63bd8d692a9eb16caa818101cdbb3358f6b4681
Author: Vishvananda Ishaya <email address hidden>
Date: Thu Mar 14 13:47:42 2013 -0700

    Don't include traceback when wrapping exceptions

    The fix in fa52cb09b2270876c9d9a03106f961be6c9db834 strips tracebacks
    from exceptions when returning them to the user, but it still spams
    the log with a long traceback. We shouldn't be including the traceback
    when we wrap the exception in the first place. Instead we just
    include the message.

    It also updates the error code to 409 for device in use since this
    is actually a conflict.

    Fix for:

      bug 1155315
      bug 1103324
      bug 1092610

    Change-Id: I95019a3022eb52e0335c455009c13fe229475d03

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/folsom)

Reviewed: https://review.openstack.org/24476
Committed: http://github.com/openstack/nova/commit/524a5a3dc89331649bf89ab442d0295f45dcae3f
Submitter: Jenkins
Branch: stable/folsom

commit 524a5a3dc89331649bf89ab442d0295f45dcae3f
Author: Vishvananda Ishaya <email address hidden>
Date: Thu Mar 14 13:47:42 2013 -0700

    Don't include traceback when wrapping exceptions

    The fix in fa52cb09b2270876c9d9a03106f961be6c9db834 strips tracebacks
    from exceptions when returning them to the user, but it still spams
    the log with a long traceback. We shouldn't be including the traceback
    when we wrap the exception in the first place. Instead we just
    include the message.

    It also updates the error code to 409 for device in use since this
    is actually a conflict.

    Fix for:

      bug 1155315
      bug 1103324
      bug 1092610

    Change-Id: I95019a3022eb52e0335c455009c13fe229475d03
    (cherry picked from commit d63bd8d692a9eb16caa818101cdbb3358f6b4681)

Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-3 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.