performance degradation in agent<->server port wiring process

Fix proposed to branch: master
Review: https://review.openstack.org/434677

Fix proposed to branch: master
Review: https://review.openstack.org/434678

Fix proposed to branch: master
Review: https://review.openstack.org/434679

Fix proposed to branch: master
Review: https://review.openstack.org/434680

Fix proposed to branch: master
Review: https://review.openstack.org/434681

I have a string of patches that will help address this increasing in complexity. We will have to decide what is safe enough to back-port for each one.

Fix proposed to branch: master
Review: https://review.openstack.org/434682

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/435633

Reviewed: https://review.openstack.org/434677
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=840e04b6f122f022f67b600ed2f96ae5e92eabe0
Submitter: Jenkins
Branch: master

commit 840e04b6f122f022f67b600ed2f96ae5e92eabe0
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 20:49:45 2017 -0800

    Skip native DHCP notifications on status change

    On profiling the get_devices_details communications between
    the agent and the server, a significant amount of time
    (60% in my dev env) is being spent in the AFTER_UPDATE events
    for the port updates resulting from the port status changes.

    One of the major offenders is the native DHCP agent notifier.
    On each port update it ends up retrieving the network for the
    port, the DHCP agents for the network, and the segments.

    This patch addresses this particular issue by adding logic to
    skip a DHCP notification if the only thing that changed on the
    port was the status. The DHCP agent doesn't do anything based on
    the status field so there is no need to update it when this is
    the only change.

    Change-Id: I948132924ec5021a9db78cf17efbba96b2500e8e
    Partial-Bug: #1665215

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/435706

Reviewed: https://review.openstack.org/435706
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3d272b2201392799c31f4219f214c47ba56c51ca
Submitter: Jenkins
Branch: stable/newton

commit 3d272b2201392799c31f4219f214c47ba56c51ca
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 20:49:45 2017 -0800

    Skip native DHCP notifications on status change

    On profiling the get_devices_details communications between
    the agent and the server, a significant amount of time
    (60% in my dev env) is being spent in the AFTER_UPDATE events
    for the port updates resulting from the port status changes.

    One of the major offenders is the native DHCP agent notifier.
    On each port update it ends up retrieving the network for the
    port, the DHCP agents for the network, and the segments.

    This patch addresses this particular issue by adding logic to
    skip a DHCP notification if the only thing that changed on the
    port was the status. The DHCP agent doesn't do anything based on
    the status field so there is no need to update it when this is
    the only change.

    Change-Id: I948132924ec5021a9db78cf17efbba96b2500e8e
    Partial-Bug: #1665215
    (cherry picked from commit 840e04b6f122f022f67b600ed2f96ae5e92eabe0)

Reviewed: https://review.openstack.org/435633
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=958388fff0ae69ffb5b95639801c339744706c8e
Submitter: Jenkins
Branch: stable/ocata

commit 958388fff0ae69ffb5b95639801c339744706c8e
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 20:49:45 2017 -0800

    Skip native DHCP notifications on status change

    On profiling the get_devices_details communications between
    the agent and the server, a significant amount of time
    (60% in my dev env) is being spent in the AFTER_UPDATE events
    for the port updates resulting from the port status changes.

    One of the major offenders is the native DHCP agent notifier.
    On each port update it ends up retrieving the network for the
    port, the DHCP agents for the network, and the segments.

    This patch addresses this particular issue by adding logic to
    skip a DHCP notification if the only thing that changed on the
    port was the status. The DHCP agent doesn't do anything based on
    the status field so there is no need to update it when this is
    the only change.

    Change-Id: I948132924ec5021a9db78cf17efbba96b2500e8e
    Partial-Bug: #1665215
    (cherry picked from commit 840e04b6f122f022f67b600ed2f96ae5e92eabe0)

Reviewed: https://review.openstack.org/434678
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4134f882cc695a4e1efa22502bc5dddc31a9865f
Submitter: Jenkins
Branch: master

commit 4134f882cc695a4e1efa22502bc5dddc31a9865f
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 21:13:18 2017 -0800

    Make ML2 OVO push notification asynchronous

    The OVO push notification logic was blocking the AFTER_UPDATE
    event for all core resources. This isn't problematic for individual
    HTTP API calls. However, the agent current updates the status of
    every port it wires two times, including on agent restarts.

    In order to drastically reduce the impact of this notifier, this
    patch moves it into a spawned eventlet coroutine so it doesn't
    block the main AFTER_UPDATE events. A semaphore is used to prevent
    multiple coroutines from trying to hit the database at once in the
    background.

    This doesn't change the semantics of the notifier since its goal
    is always to send the latest events.

    Partial-Bug: #1665215
    Change-Id: Ic259bad6f65f4bc45f5b900e0979c9a91865089b

Reviewed: https://review.openstack.org/434679
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2c01ad9c564dcc294fb954a62d86d1e334f52178
Submitter: Jenkins
Branch: master

commit 2c01ad9c564dcc294fb954a62d86d1e334f52178
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 21:36:12 2017 -0800

    Avoid segment DB lookup in _expand_segment on port

    The port context already has a network attached to it that contains
    all of the segments for that network. In all cases the segments the
    port wants to lookup should be present here so we can just iterate
    over them without a separate DB query.

    Partial-Bug: #1665215
    Change-Id: I9014c4e401134f4db908ed989e0095d0ca95b7c9

Reviewed: https://review.openstack.org/434680
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=60edb4c9519543a04d5ccc73c8b5371123fefcd8
Submitter: Jenkins
Branch: master

commit 60edb4c9519543a04d5ccc73c8b5371123fefcd8
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 21:40:07 2017 -0800

    Allow no network to be passed into PortContext

    This allows a PortContext to be constructed without a network
    passed in for cases like update_port_status that don't examine
    bindings, segments, or any other network properties.

    In the event that a mechanism driver is loaded that does reference
    the 'network' property for these occasions, the context will look
    up the network then in a late binding fashion.

    This will improve the performance of the update_port_status call
    from the provisioning blocks callback, which doesn't provide a
    cached network to update_port_status.

    Partial-Bug: #1665215
    Change-Id: I498791614fd456ab67c453cad691f7658d107123

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/439810

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/439811

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/439813

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/439814

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/439817

Reviewed: https://review.openstack.org/439813
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ad6983141d7f6acc6df7802ac1906c0bcde0a5f7
Submitter: Jenkins
Branch: stable/newton

commit ad6983141d7f6acc6df7802ac1906c0bcde0a5f7
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 21:36:12 2017 -0800

    Avoid segment DB lookup in _expand_segment on port

    The port context already has a network attached to it that contains
    all of the segments for that network. In all cases the segments the
    port wants to lookup should be present here so we can just iterate
    over them without a separate DB query.

    Conflicts:
     neutron/plugins/ml2/driver_context.py

    Partial-Bug: #1665215
    Change-Id: I9014c4e401134f4db908ed989e0095d0ca95b7c9
    (cherry picked from commit 2c01ad9c564dcc294fb954a62d86d1e334f52178)

Reviewed: https://review.openstack.org/439817
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7502c26d385072c0e30a1849a579c44b37bbaf31
Submitter: Jenkins
Branch: stable/newton

commit 7502c26d385072c0e30a1849a579c44b37bbaf31
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 21:40:07 2017 -0800

    Allow no network to be passed into PortContext

    This allows a PortContext to be constructed without a network
    passed in for cases like update_port_status that don't examine
    bindings, segments, or any other network properties.

    In the event that a mechanism driver is loaded that does reference
    the 'network' property for these occasions, the context will look
    up the network then in a late binding fashion.

    This will improve the performance of the update_port_status call
    from the provisioning blocks callback, which doesn't provide a
    cached network to update_port_status.

    Conflicts:
     neutron/plugins/ml2/plugin.py

    Partial-Bug: #1665215
    Change-Id: I498791614fd456ab67c453cad691f7658d107123
    (cherry picked from commit 60edb4c9519543a04d5ccc73c8b5371123fefcd8)

Reviewed: https://review.openstack.org/439814
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=bd7902f965848b36ef485c60f3ad51aa9420aae6
Submitter: Jenkins
Branch: stable/ocata

commit bd7902f965848b36ef485c60f3ad51aa9420aae6
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 21:40:07 2017 -0800

    Allow no network to be passed into PortContext

    This allows a PortContext to be constructed without a network
    passed in for cases like update_port_status that don't examine
    bindings, segments, or any other network properties.

    In the event that a mechanism driver is loaded that does reference
    the 'network' property for these occasions, the context will look
    up the network then in a late binding fashion.

    This will improve the performance of the update_port_status call
    from the provisioning blocks callback, which doesn't provide a
    cached network to update_port_status.

    Partial-Bug: #1665215
    Change-Id: I498791614fd456ab67c453cad691f7658d107123
    (cherry picked from commit 60edb4c9519543a04d5ccc73c8b5371123fefcd8)

Reviewed: https://review.openstack.org/439810
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f465aa84238cb60d8740a5e286bdcd39eb962f93
Submitter: Jenkins
Branch: stable/ocata

commit f465aa84238cb60d8740a5e286bdcd39eb962f93
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 21:13:18 2017 -0800

    Make ML2 OVO push notification asynchronous

    The OVO push notification logic was blocking the AFTER_UPDATE
    event for all core resources. This isn't problematic for individual
    HTTP API calls. However, the agent current updates the status of
    every port it wires two times, including on agent restarts.

    In order to drastically reduce the impact of this notifier, this
    patch moves it into a spawned eventlet coroutine so it doesn't
    block the main AFTER_UPDATE events. A semaphore is used to prevent
    multiple coroutines from trying to hit the database at once in the
    background.

    This doesn't change the semantics of the notifier since its goal
    is always to send the latest events.

    Partial-Bug: #1665215
    Change-Id: Ic259bad6f65f4bc45f5b900e0979c9a91865089b
    (cherry picked from commit 4134f882cc695a4e1efa22502bc5dddc31a9865f)

Reviewed: https://review.openstack.org/439811
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1960f384aacbdb540a140210f06ea1098f9b023d
Submitter: Jenkins
Branch: stable/ocata

commit 1960f384aacbdb540a140210f06ea1098f9b023d
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 21:36:12 2017 -0800

    Avoid segment DB lookup in _expand_segment on port

    The port context already has a network attached to it that contains
    all of the segments for that network. In all cases the segments the
    port wants to lookup should be present here so we can just iterate
    over them without a separate DB query.

    Partial-Bug: #1665215
    Change-Id: I9014c4e401134f4db908ed989e0095d0ca95b7c9
    (cherry picked from commit 2c01ad9c564dcc294fb954a62d86d1e334f52178)

Fix proposed to branch: master
Review: https://review.openstack.org/454190

Fix proposed to branch: master
Review: https://review.openstack.org/454191

Reviewed: https://review.openstack.org/454190
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=604e598a7d43b8b3a260969de84af737a4988c4e
Submitter: Jenkins
Branch: master

commit 604e598a7d43b8b3a260969de84af737a4988c4e
Author: Kevin Benton <email address hidden>
Date: Thu Apr 6 05:01:30 2017 -0700

    Allow offloading lookups in driver contexts

    This allows segments looked up ahead of time to be passed
    into NetworkContext objects and NetworkContext objects to
    be passed into PortContext objects. This allows us to avoid
    doing segments lookups for every PortContext construction
    when handling a bunch of ports (e.g. in RPC handler).

    Change-Id: Ib4c43a7894fe1285ecf4bdf9af5e5f1b93b0b39b
    Partial-Bug: #1665215

Reviewed: https://review.openstack.org/454191
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=323eb7f2e146ecc2f7c25d59e57d1e1e006cb71f
Submitter: Jenkins
Branch: master

commit 323eb7f2e146ecc2f7c25d59e57d1e1e006cb71f
Author: Kevin Benton <email address hidden>
Date: Thu Apr 6 05:42:25 2017 -0700

    Add some bulk lookup methods to ML2 for RPC handling

    This adds three methods to make working with bulk port
    DB lookups easier in ML2:

    * partial_port_ids_to_full_ids: takes short port IDs map to
      full port IDs. This will allow us to eliminate many LIKE
      queries and do one just once for all ports on an RPC call.
    * get_port_db_objects: Takes a list of port IDs and returns
      a map to port DB objects. This allows us to get access to
      sqla obejcts for a bunch of ports without a custom
      session.query call.
    * get_network_contexts: Takes a list of network_ids and does
      a bulk construction of NetworkContext objects and returns
      them as a map of network_id to NetworkContext to avoid
      expensive net lookups when constructing lots of PortContext
      objects.

    Partial-Bug: #1665215
    Change-Id: I330eefbf429bd62f2a7e8ebadf7037da15c86815

Fix proposed to branch: master
Review: https://review.openstack.org/460718

Reviewed: https://review.openstack.org/434681
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=529da4e583aefe7640d8559987282f223f3ad06b
Submitter: Jenkins
Branch: master

commit 529da4e583aefe7640d8559987282f223f3ad06b
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 22:27:53 2017 -0800

    Bulk up port context retrieval

    With the switch to subquery relationships, individual get_port calls
    can get expensive with large numbers of ports
    (100ms per port in my dev environment). This patch bulks up the
    retrieval of the port contexts so one set of queries covers all
    of the devices in an RPC call.

    Partial-Bug: #1665215
    Change-Id: I63757e143b23c24c349be98dc5a09115b8709a25

Fix proposed to branch: master
Review: https://review.openstack.org/464904

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/465180

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/465182

Reviewed: https://review.openstack.org/460718
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a04332ca4a3182d0fcfd2c20fbe0bc1dd0fdfe14
Submitter: Jenkins
Branch: master

commit a04332ca4a3182d0fcfd2c20fbe0bc1dd0fdfe14
Author: Kevin Benton <email address hidden>
Date: Thu Apr 27 11:56:56 2017 -0700

    Separate port status update from getting details

    This is a small refactor to separate updating the port status
    from the method retrieving device details. This is in preparation
    for patch I99c2b77b35e6eabb6e4f633c4e8e2533594c6b55 which will allow
    updating ports' status in bulk.

    Change-Id: Ifa78f6911cfbbd3e982807122bddc898613f323f
    Partial-Bug: #1665215

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/466394

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/466395

Reviewed: https://review.openstack.org/465180
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a2ae48c2cee71d8a27f44c5d5cfbfbdacc1fee00
Submitter: Jenkins
Branch: stable/ocata

commit a2ae48c2cee71d8a27f44c5d5cfbfbdacc1fee00
Author: Kevin Benton <email address hidden>
Date: Thu Apr 6 05:01:30 2017 -0700

    Allow offloading lookups in driver contexts

    This allows segments looked up ahead of time to be passed
    into NetworkContext objects and NetworkContext objects to
    be passed into PortContext objects. This allows us to avoid
    doing segments lookups for every PortContext construction
    when handling a bunch of ports (e.g. in RPC handler).

    Conflicts:
     neutron/plugins/ml2/driver_context.py

    Change-Id: Ib4c43a7894fe1285ecf4bdf9af5e5f1b93b0b39b
    Partial-Bug: #1665215
    (cherry picked from commit 604e598a7d43b8b3a260969de84af737a4988c4e)

Reviewed: https://review.openstack.org/464904
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9f55d77016083ffb9c4d9cd3da5bb95de82186f7
Submitter: Jenkins
Branch: master

commit 9f55d77016083ffb9c4d9cd3da5bb95de82186f7
Author: Kevin Benton <email address hidden>
Date: Mon May 15 19:08:05 2017 -0700

    Notify L2pop driver from update_device_(up|down)

    This patch calls update_port_up and update_port_down inside
    of the l2pop driver directly from update_device_up and
    update_device_down in the ML2 RPC layer. This allows the L2pop
    driver to setup forwarding entries completely independent of
    the port status update events.

    This will allow L2pop to function without requiring the ports to
    transition from ACTIVE->BUILD->ACTIVE every time the agent requests
    device details.

    This will unblock the push notifications work and will additionally
    enable us to remove the update to BUILD status as part of a performance
    improvement backport for bug #1665215.

    Partial-Bug: #1665215
    Partially-Implements: blueprint push-notifications
    Change-Id: Icd4cd4e3f735e88299e86468380c5f786e7628fe

Reviewed: https://review.openstack.org/466394
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ce0e4b25d5cbabaff147cc7212fe033a95233a54
Submitter: Jenkins
Branch: stable/ocata

commit ce0e4b25d5cbabaff147cc7212fe033a95233a54
Author: Kevin Benton <email address hidden>
Date: Thu Apr 6 05:42:25 2017 -0700

    Add some bulk lookup methods to ML2 for RPC handling

    This adds three methods to make working with bulk port
    DB lookups easier in ML2:

    * partial_port_ids_to_full_ids: takes short port IDs map to
      full port IDs. This will allow us to eliminate many LIKE
      queries and do one just once for all ports on an RPC call.
    * get_port_db_objects: Takes a list of port IDs and returns
      a map to port DB objects. This allows us to get access to
      sqla obejcts for a bunch of ports without a custom
      session.query call.
    * get_network_contexts: Takes a list of network_ids and does
      a bulk construction of NetworkContext objects and returns
      them as a map of network_id to NetworkContext to avoid
      expensive net lookups when constructing lots of PortContext
      objects.

    Conflicts:
     neutron/plugins/ml2/db.py
     neutron/plugins/ml2/plugin.py

    Partial-Bug: #1665215
    Change-Id: I330eefbf429bd62f2a7e8ebadf7037da15c86815
    (cherry picked from commit 323eb7f2e146ecc2f7c25d59e57d1e1e006cb71f)

Reviewed: https://review.openstack.org/434682
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1be00e8239db3db6e962b30a6b3c2b90144e19b8
Submitter: Jenkins
Branch: master

commit 1be00e8239db3db6e962b30a6b3c2b90144e19b8
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 23:05:42 2017 -0800

    Bulk up port status updating in ML2 RPC

    This eliminates the last of the bottlenecks in
    get_devices_details_list_and_failed_devices by making the status
    updates use a bulk data retrieval as well.

    The last remaining thing that will impact performance is the status
    update back to ACTIVE on removal of the provisioning blocks. However,
    that will require a much larger refactor since it is callback driven
    at the individual port level.

    Elimination of the L2pop driver will ultimately solve this completely
    since we won't need to cycle the port status anymore on every single
    agent restart.

    Closes-Bug: #1665215
    Change-Id: I99c2b77b35e6eabb6e4f633c4e8e2533594c6b55

Reviewed: https://review.openstack.org/465182
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=bd7055daf23f03fe2e939e6d585885341422978c
Submitter: Jenkins
Branch: stable/newton

commit bd7055daf23f03fe2e939e6d585885341422978c
Author: Kevin Benton <email address hidden>
Date: Thu Apr 6 05:01:30 2017 -0700

    Allow offloading lookups in driver contexts

    This allows segments looked up ahead of time to be passed
    into NetworkContext objects and NetworkContext objects to
    be passed into PortContext objects. This allows us to avoid
    doing segments lookups for every PortContext construction
    when handling a bunch of ports (e.g. in RPC handler).

    Conflicts:
     neutron/plugins/ml2/driver_context.py

    Change-Id: Ib4c43a7894fe1285ecf4bdf9af5e5f1b93b0b39b
    Partial-Bug: #1665215
    (cherry picked from commit 604e598a7d43b8b3a260969de84af737a4988c4e)
    (cherry picked from commit a2ae48c2cee71d8a27f44c5d5cfbfbdacc1fee00)

Reviewed: https://review.openstack.org/466395
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2a32ae927147ebb07a188b569c264bca80bbb116
Submitter: Jenkins
Branch: stable/ocata

commit 2a32ae927147ebb07a188b569c264bca80bbb116
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 22:27:53 2017 -0800

    Bulk up port context retrieval

    With the switch to subquery relationships, individual get_port calls
    can get expensive with large numbers of ports
    (100ms per port in my dev environment). This patch bulks up the
    retrieval of the port contexts so one set of queries covers all
    of the devices in an RPC call.

    Partial-Bug: #1665215
    Change-Id: I63757e143b23c24c349be98dc5a09115b8709a25
    (cherry picked from commit 529da4e583aefe7640d8559987282f223f3ad06b)

This issue was fixed in the openstack/neutron 11.0.0.0b2 development milestone.

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/643141

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/643143

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/643144

Reviewed: https://review.opendev.org/643141
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=912b25f3d89b2f599253ce86e542884a58c708df
Submitter: Zuul
Branch: stable/ocata

commit 912b25f3d89b2f599253ce86e542884a58c708df
Author: Kevin Benton <email address hidden>
Date: Thu Apr 27 11:56:56 2017 -0700

    Separate port status update from getting details

    This is a small refactor to separate updating the port status
    from the method retrieving device details. This is in preparation
    for patch I99c2b77b35e6eabb6e4f633c4e8e2533594c6b55 which will allow
    updating ports' status in bulk.

    Change-Id: Ifa78f6911cfbbd3e982807122bddc898613f323f
    Partial-Bug: #1665215
    (cherry picked from commit a04332ca4a3182d0fcfd2c20fbe0bc1dd0fdfe14)

Reviewed: https://review.opendev.org/643143
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e374e0e5e57d26ca69e512190e9a9269c00b603c
Submitter: Zuul
Branch: stable/ocata

commit e374e0e5e57d26ca69e512190e9a9269c00b603c
Author: Kevin Benton <email address hidden>
Date: Wed Feb 15 23:05:42 2017 -0800

    Bulk up port status updating in ML2 RPC

    This eliminates the last of the bottlenecks in
    get_devices_details_list_and_failed_devices by making the status
    updates use a bulk data retrieval as well.

    The last remaining thing that will impact performance is the status
    update back to ACTIVE on removal of the provisioning blocks. However,
    that will require a much larger refactor since it is callback driven
    at the individual port level.

    Elimination of the L2pop driver will ultimately solve this completely
    since we won't need to cycle the port status anymore on every single
    agent restart.

    Closes-Bug: #1665215
    Change-Id: I99c2b77b35e6eabb6e4f633c4e8e2533594c6b55
    (cherry picked from commit 1be00e8239db3db6e962b30a6b3c2b90144e19b8)

Reviewed: https://review.opendev.org/643144
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7b960e57ce2525904a4b4d0f0a9e89f2c56fb6cf
Submitter: Zuul
Branch: stable/ocata

commit 7b960e57ce2525904a4b4d0f0a9e89f2c56fb6cf
Author: Kevin Benton <email address hidden>
Date: Mon May 15 19:08:05 2017 -0700

    Notify L2pop driver from update_device_(up|down)

    This patch calls update_port_up and update_port_down inside
    of the l2pop driver directly from update_device_up and
    update_device_down in the ML2 RPC layer. This allows the L2pop
    driver to setup forwarding entries completely independent of
    the port status update events.

    This will allow L2pop to function without requiring the ports to
    transition from ACTIVE->BUILD->ACTIVE every time the agent requests
    device details.

    This will unblock the push notifications work and will additionally
    enable us to remove the update to BUILD status as part of a performance
    improvement backport for bug #1665215.

    Partial-Bug: #1665215
    Partially-Implements: blueprint push-notifications
    Change-Id: Icd4cd4e3f735e88299e86468380c5f786e7628fe
    (cherry picked from commit 9f55d77016083ffb9c4d9cd3da5bb95de82186f7)

This issue was fixed in the openstack/neutron ocata-eol release.