Mir

Merge lp:~vanvugt/mir/double into lp:mir

double
Merge into development-branch

Proposed by Daniel van Vugt on 2014-05-29

Status:

Superseded

Proposed branch:

lp:~vanvugt/mir/double

Merge into:

lp:mir

Prerequisite:

lp:~vanvugt/mir/unrestrict-bufferstream

Diff against target:

458 lines (+133/-72)

5 files modified

src/server/compositor/buffer_queue.cpp (+57/-14)
src/server/compositor/buffer_queue.h (+3/-0)
tests/integration-tests/compositor/test_buffer_stream.cpp (+57/-47)
tests/integration-tests/test_surface_first_frame_sync.cpp (+1/-1)
tests/unit-tests/compositor/test_buffer_queue.cpp (+15/-10)

To merge this branch:

bzr merge lp:~vanvugt/mir/double

Medium

Fix Released

Link a bug report

Reviewer	Review Type	Date Requested	Status
PS Jenkins bot (community)	continuous-integration		Needs Fixing on 2014-05-30
Daniel van Vugt			Needs Fixing on 2014-05-30
Mir development team		2014-05-30	Pending
Review via email: mp+221431@code.launchpad.net

Commit message

Scale BufferQueue length dynamically according to demand, up to a maximum
of nbuffers. Under normal synchronous rendering conditions clients will
now typically only get two unique buffers. So expect visible input lag to be
at least halved. And of course resource usage will be 33% less.

Description of the change

Notes:
* Bug 1308843 is tricking the new algorithm into using 3 buffers when two would be safe. That will be solved separately, soon.
* I would like to continue improving this, but may run out of time soon. So am proposing it slightly early.

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-05-29:

FAILED: Continuous integration, rev:1663
http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-ci/1776/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-utopic-i386-build/400
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-clang-utopic-amd64-build/401
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-utopic-touch/399/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-utopic-amd64-ci/295
        deb: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-utopic-amd64-ci/295/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-utopic-armhf-ci/294
        deb: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-utopic-armhf-ci/294/artifact/work/output/*zip*/output.zip
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/1106/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/mir-team-mir-development-branch-ci/1776/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Daniel van Vugt (vanvugt) wrote on 2014-05-30:

[ FAILED ] BufferQueueTest.slow_client_framerate_matches_compositor

Needs work. But I'm undecided as to whether BufferQueue should be fixed or the test case itself. Loosening the test case such that it can pass without re-introducing either of the bugs it is designed to protected against seems possible. But undesirable.

review: Needs Fixing

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-05-30:

FAILED: Continuous integration, rev:1665
http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-ci/1784/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-utopic-i386-build/412
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-clang-utopic-amd64-build/413/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-utopic-touch/411/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-utopic-amd64-ci/303/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-utopic-armhf-ci/302/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/1142/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/mir-team-mir-development-branch-ci/1784/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Alberto Aguirre (albaguirre) wrote on 2014-06-27:

superseded by https://code.launchpad.net/~mir-team/mir/enable-dynamic-buffer-queue/+merge/224742

lp:~vanvugt/mir/double updated on 2014-11-23

1666. By Daniel van Vugt on 2014-07-04

Merge latest development-branch and fix conflicts (I think)

1667. By Daniel van Vugt on 2014-07-04

Start fixing failing/hanging tests since the merge

1668. By Daniel van Vugt on 2014-07-04

Disable hanging integration test

1669. By Daniel van Vugt on 2014-07-04

Remove FIXME for bugs now fixed.

1670. By Daniel van Vugt on 2014-07-04

Add debug output

1671. By Daniel van Vugt on 2014-07-04

More comments

1672. By Daniel van Vugt on 2014-07-04

Fix test case: framedropping_clients_get_max_2_buffers

1673. By Daniel van Vugt on 2014-07-04

Fix test case: compositor_acquires_resized_frames

1674. By Daniel van Vugt on 2014-07-04

Fix hanging test case: uncomposited_client_swaps_when_policy_triggered

1675. By Daniel van Vugt on 2014-07-07

Merge latest development-branch

1676. By Daniel van Vugt on 2014-07-07

Shrink the diff

1677. By Daniel van Vugt on 2014-07-07

Merge latest development-branch

1678. By Daniel van Vugt on 2014-07-09

Merge latest development-branch

1679. By Daniel van Vugt on 2014-07-09

Reenable debug output

1680. By Daniel van Vugt on 2014-07-09

Replace magic number 300 with a constant. I actually think this is less
readable defining constants so far away from the single location where
they're used. But it will generate less argument this way.

1681. By Daniel van Vugt on 2014-07-09

Tentatively drop bypass detection, and all associated logic. It seems bypass
doesn't need triple buffers any more.

1682. By Daniel van Vugt on 2014-07-09

Merge lp:~vanvugt/mir/unrestrict-bypass

1683. By Daniel van Vugt on 2014-07-09

Merge latest development branch via lp:~vanvugt/mir/unrestrict-bypass
to get fixes for spurious test failures.

1684. By Daniel van Vugt on 2014-07-11

Merge latest development-branch and fix a conflict.

1685. By Daniel van Vugt on 2014-07-11

Ultra-simplified the min_buffers decision.

1686. By Daniel van Vugt on 2014-07-11

Remove the excess/shrinkage delay algorithm. No delay is required at all any
more.

1687. By Daniel van Vugt on 2014-07-11

Remove debug output

1688. By Daniel van Vugt on 2014-07-11

Fix failing test: BufferQueueTest.clients_can_have_multiple_pending_completions

1689. By Daniel van Vugt on 2014-07-11

Fix failing test: BufferQueueTest.throws_on_out_of_order_client_release

1690. By Daniel van Vugt on 2014-07-11

Fix test case: BufferQueueTest.compositor_acquires_frames_in_order

1691. By Daniel van Vugt on 2014-07-11

Fix test case: BufferQueueTest.buffers_are_not_lost

1692. By Daniel van Vugt on 2014-07-11

Fix test case: BufferQueueTest.async_client_cycles_through_all_buffers

1693. By Daniel van Vugt on 2014-07-11

Fixed test case:
BufferQueueTest.partially_composited_client_swaps_when_policy_triggered

1694. By Daniel van Vugt on 2014-07-14

Merge latest development-branch

1695. By Daniel van Vugt on 2014-07-14

Merge latest development-branch

1696. By Daniel van Vugt on 2014-07-15

Merge latest development-branch

1697. By Daniel van Vugt on 2014-07-15

Re-enable the remaining failing test

1698. By Daniel van Vugt on 2014-07-15

Fix failing test: BufferStreamTest.blocked_client_is_released_on_timeout

1699. By Daniel van Vugt on 2014-07-15

Shrink the diff

1700. By Daniel van Vugt on 2014-07-15

Shrink the diff

1701. By Daniel van Vugt on 2014-07-15

Restore test case BufferQueueTest.framedropping_clients_get_all_buffers
properly. Also shrink the diff.

1702. By Daniel van Vugt on 2014-07-15

Shrink the diff

1703. By Daniel van Vugt on 2014-07-15

Clean up comments

1704. By Daniel van Vugt on 2014-07-15

Rename nbuffers to max_buffers to avoid confusion.

1705. By Daniel van Vugt on 2014-07-15

Add new test: BufferQueueTest.queue_size_scales_on_demand

1706. By Daniel van Vugt on 2014-07-16

Merge latest development-branch

1707. By Daniel van Vugt on 2014-07-16

Shrink the buffers vector from any location, not just the back.

1708. By Daniel van Vugt on 2014-07-16

Shrink the buffer queue immediately when framedropping is toggled. This
provides much better determinism for tests.

1709. By Daniel van Vugt on 2014-07-16

BufferQueueTest.slow_client_framerate_matches_compositor: Loosen thresholds
so it's less likely to fail spuriously under the strain of valgrind-on-armhf.

1710. By Daniel van Vugt on 2014-07-17

Merge latest development-branch

1711. By Daniel van Vugt on 2014-07-21

Merge latest development-branch

1712. By Daniel van Vugt on 2014-07-22

Merge latest development-branch

1713. By Daniel van Vugt on 2014-07-22

Rename min_buffers to ideal_buffers

1714. By Daniel van Vugt on 2014-07-22

Add a "min_buffers" constant in preparation for future configurability.

1715. By Daniel van Vugt on 2014-07-22

Fix failing tests (bad initalizer of min_buffers)

1716. By Daniel van Vugt on 2014-07-22

Revert evil fudges to TEST_F(BufferQueueTest, slow_client_framerate_matches_compositor)

Let's see if we can make it pass again properly.

1717. By Daniel van Vugt on 2014-07-22

Add detection of missed frame deadlines and auto-expand the buffer queue when
it happens too much.

1718. By Daniel van Vugt on 2014-07-22

Remember to reset extra_buffers when frame dropping

1719. By Daniel van Vugt on 2014-07-22

Tidy up

1720. By Daniel van Vugt on 2014-07-22

Make the resize delay configurable so slow_client_framerate_matches_compositor
can pass again reliably without cheating.

1721. By Daniel van Vugt on 2014-07-22

Fix test: BufferQueueTest.synchronous_clients_only_get_two_real_buffers

1722. By Daniel van Vugt on 2014-07-22

Fix unreliable test: queue_size_scales_instantly_on_framedropping

1723. By Daniel van Vugt on 2014-07-22

More rigorous auto-expansion testing

1724. By Daniel van Vugt on 2014-07-22

Avoid ping-pong queue resizing. If a slow client needs more buffers, giving
it an extra one will make it look fast. But that doesn't make it a fast client.
So avoid shrinking the buffer queue based on performance observations alone.

1725. By Daniel van Vugt on 2014-07-22

More intelligent hysteresis. Ensure missed_frames can come down if we're
mostly keeping up.

1726. By Daniel van Vugt on 2014-07-23

Merge latest development-branch

1727. By Daniel van Vugt on 2014-07-24

Merge latest development-branch

1728. By Daniel van Vugt on 2014-07-25

Merge latest development-branch

1729. By Daniel van Vugt on 2014-07-28

Merge latest development-branch and fix conflicts.

1730. By Daniel van Vugt on 2014-07-28

Merge latest development-branch

1731. By Daniel van Vugt on 2014-07-28

Fix failing test case, merged from devel:
gives_compositor_the_newest_buffer_after_dropping_old_buffers

1732. By Daniel van Vugt on 2014-07-28

Make tests which require 3 buffers more consistent

1733. By Daniel van Vugt on 2014-07-28

Fix failing test: TEST_F(StaleFrames, are_dropped_when_restarting_compositor)
It was designed to assume three unique buffers available by default. Remove
that assumption.

1734. By Daniel van Vugt on 2014-07-28

Lower the resize delay to 100 frames so slow clients only take ~1.5s to
be detected as slow and change to triple buffers.

1735. By Daniel van Vugt on 2014-07-29

Merge latest development-branch

1736. By Daniel van Vugt on 2014-07-29

Merge latest development-branch

1737. By Daniel van Vugt on 2014-07-29

Add unit test: BufferQueueTest.switch_to_triple_buffers_is_permanent

This verifies we don't accidentally ping-pong between 2 and 3 buffers.

1738. By Daniel van Vugt on 2014-07-30

Merge latest development-branch

1739. By Daniel van Vugt on 2014-07-30

Add a regression test for the idle-vs-slow client problem Alexandros
described. Presently failing.

1740. By Daniel van Vugt on 2014-07-30

Move the "missed frames" decision to a point where it won't be skewed by
multiple compositors (multi-monitor).

1741. By Daniel van Vugt on 2014-07-30

Prototype a detection of "idle" clients. Other questionable tests still
failing.

1742. By Daniel van Vugt on 2014-07-30

Simplify

1743. By Daniel van Vugt on 2014-07-30

Unmerged revisions

1799. By Daniel van Vugt on 2014-11-23: Even simpler
1798. By Daniel van Vugt on 2014-11-23: A simpler fix for the crashing test -- don't try to release the same
old_buffer multiple times.
1797. By Daniel van Vugt on 2014-11-23: Prototype fix for the crashing integration test
1796. By Daniel van Vugt on 2014-11-23: Merge latest trunk
1795. By Daniel van Vugt on 2014-11-14: Merge latest trunk
1794. By Daniel van Vugt on 2014-11-13: Merge latest trunk
1793. By Daniel van Vugt on 2014-11-12: Merge latest trunk
1792. By Daniel van Vugt on 2014-11-11: Merge latest trunk
1791. By Daniel van Vugt on 2014-11-10: Merge latest trunk
1790. By Daniel van Vugt on 2014-11-07: Merge latest trunk

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk

Subscribers

People subscribed via source and target branches

to all changes:

Daniel van Vugt

Emanuele Antonio Faraone

Gerry Boland

Mir development team

 === modified file 'src/server/compositor/buffer_queue.cpp'
 --- src/server/compositor/buffer_queue.cpp	2014-05-29 15:51:54 +0000
 +++ src/server/compositor/buffer_queue.cpp	2014-05-30 13:43:29 +0000
@@ -95,6 +95,8 @@
      std::shared_ptr<graphics::GraphicBufferAllocator> const& gralloc,
      graphics::BufferProperties const& props)
      : nbuffers{nbuffers},
++      excess{0},
++      overlapping_compositors{false},
        frame_dropping_enabled{false},
        the_properties{props},
        gralloc{gralloc}
@@ -143,12 +145,8 @@
          return;
+     }
--    /* No empty buffers, attempt allocating more
--     * TODO: need a good heuristic to switch
--     * between double-buffering to n-buffering
--     */
      int const allocated_buffers = buffers.size();
--    if (allocated_buffers < nbuffers)
++    if (allocated_buffers < min_buffers())
+     {
          auto const& buffer = gralloc->alloc_buffer(the_properties);
          buffers.push_back(buffer);
@@ -213,9 +211,18 @@
          current_buffer_users.push_back(user_id);
          current_compositor_buffer = pop(ready_to_composite_queue);
+     }
++    else if (current_buffer_users.empty())
++    {   // current_buffer_users and ready_to_composite_queue both empty
++        current_buffer_users.push_back(user_id);
++    }
      buffers_sent_to_compositor.push_back(current_compositor_buffer);
++    // Detect bypassing compositors (those that need to hold multiple different
++    // buffers simultaneously).
++    overlapping_compositors =
++        buffers_sent_to_compositor.size() > current_buffer_users.size();
++
      std::shared_ptr<mg::Buffer> const acquired_buffer =
          buffer_for(current_compositor_buffer, buffers);
@@ -337,14 +344,7 @@
  int mc::BufferQueue::buffers_free_for_client() const
+ {
      std::lock_guard<decltype(guard)> lock(guard);
--    int ret = 1;
--    if (nbuffers > 1)
--    {
--        int nfree = free_buffers.size();
--        int future_growth = nbuffers - buffers.size();
--        ret = nfree + future_growth;
--    }
--    return ret;
++    return nbuffers > 1 ? free_buffers.size() : 1;
+ }
  void mc::BufferQueue::give_buffer_to_client(
@@ -400,8 +400,51 @@
      mg::Buffer* buffer,
      std::unique_lock<std::mutex> lock)
+ {
--    if (!pending_client_notifications.empty())
++    int used_buffers = buffers.size() - free_buffers.size();
++
++    // To avoid reallocating buffers too often (which may be slow), only drop
++    // a buffer after it's continually been in excess for a long time...
++
++    if (used_buffers > min_buffers())
++        ++excess;
++    else
++        excess = 0;
++
++    // If too many frames have had excess buffers then start dropping them now
++    if (excess > 300 && buffers.back().get() == buffer && nbuffers > 1)
++    {
++        buffers.pop_back();
++        excess = 0;
++    }
++    else if (!pending_client_notifications.empty())
          give_buffer_to_client(buffer, std::move(lock));
      else
          free_buffers.push_back(buffer);
+ }
++
++/**
++ * Measure the actual number of buffers we presently need concurrently
++ * to avoid starving any compositors of fresh frames or starving clients of
++ * any buffers at all. Typically in a multi-buffer client min_buffers will
++ * be 2. However when DRM bypass is active or frame dropping enabled, the
++ * result of min_buffers could be 3. If you had bypass and frame dropping
++ * enabled simultaneously then required_buffers could reach 4 (LP: #1317403)
++ */
++int mc::BufferQueue::min_buffers() const
++{
++    if (nbuffers <= 1)
++        return nbuffers;
++
++    // else for multi-buffering with exclusivity guarantees:
++    int client_demand = buffers_owned_by_client.size() +
++                        pending_client_notifications.size();
++    // FIXME: LP: #1308844 / LP: #1308843 is inflating compositor_demand
++    int compositor_demand = overlapping_compositors ? 2 : 1;
++    int min_compositors = std::max(1, compositor_demand);
++    int min_clients = std::max(1, client_demand);
++    int min_free = frame_dropping_enabled ? 1 : 0;
++    int required_buffers = min_compositors + min_clients + min_free;
++
++    // FIXME: Sometimes required_buffers > nbuffers (LP: #1317403)
++    return std::min(nbuffers, required_buffers);
++}
 === modified file 'src/server/compositor/buffer_queue.h'
 --- src/server/compositor/buffer_queue.h	2014-05-29 15:51:54 +0000
 +++ src/server/compositor/buffer_queue.h	2014-05-30 13:43:29 +0000
@@ -65,6 +65,7 @@
      bool is_a_current_buffer_user(void const* user_id) const;
      void release(graphics::Buffer* buffer,
          std::unique_lock<std::mutex> lock);
++    int min_buffers() const;
      mutable std::mutex guard;
@@ -81,6 +82,8 @@
      std::deque<Callback> pending_client_notifications;
      int nbuffers;
++    int excess;
++    bool overlapping_compositors;
      bool frame_dropping_enabled;
      graphics::BufferProperties the_properties;
 === modified file 'tests/integration-tests/compositor/test_buffer_stream.cpp'
 --- tests/integration-tests/compositor/test_buffer_stream.cpp	2014-05-30 13:43:28 +0000
 +++ tests/integration-tests/compositor/test_buffer_stream.cpp	2014-05-30 13:43:29 +0000
@@ -45,15 +45,13 @@
+ {
      using mc::BufferStreamSurfaces::BufferStreamSurfaces;
--    // Convenient function to allow tests to be written in linear style
--    void swap_client_buffers_blocking(mg::Buffer*& buffer)
++    // Convenient functions to allow tests to be written in linear style
++    mg::Buffer* acquire_client_buffer_blocking()
+     {
          std::mutex mutex;
          std::condition_variable cv;
          bool done = false;
--
--        if (buffer)
--            release_client_buffer(buffer);
++        mg::Buffer* buffer = nullptr;
          acquire_client_buffer(
              [&](mg::Buffer* new_buffer)
@@ -67,6 +65,16 @@
          std::unique_lock<decltype(mutex)> lock(mutex);
          cv.wait(lock, [&]{ return done; });
++
++        return buffer;
++    }
++
++    void swap_client_buffers_blocking(mg::Buffer*& buffer)
++    {
++        if (buffer)
++            release_client_buffer(buffer);
++
++        buffer = acquire_client_buffer_blocking();
+     }
  };
@@ -106,12 +114,9 @@
  TEST_F(BufferStreamTest, gives_same_back_buffer_until_more_available)
+ {
--    ASSERT_THAT(buffers_free_for_client(), Ge(2)); // else we will hang
--
--    mg::Buffer* client1{nullptr};
--    buffer_stream.swap_client_buffers_blocking(client1);
++    mg::Buffer* client1 = buffer_stream.acquire_client_buffer_blocking();
      auto client1_id = client1->id();
--    buffer_stream.swap_client_buffers_blocking(client1);
++    buffer_stream.release_client_buffer(client1);
      auto comp1 = buffer_stream.lock_compositor_buffer(nullptr);
      auto comp2 = buffer_stream.lock_compositor_buffer(nullptr);
@@ -121,10 +126,14 @@
      comp1.reset();
--    buffer_stream.swap_client_buffers_blocking(client1);
++    mg::Buffer* client2 = buffer_stream.acquire_client_buffer_blocking();
++    auto client2_id = client2->id();
++    buffer_stream.release_client_buffer(client2);
++
      auto comp3 = buffer_stream.lock_compositor_buffer(nullptr);
      EXPECT_NE(client1_id, comp3->id());
++    EXPECT_EQ(client2_id, comp3->id());
      comp2.reset();
      auto comp3_id = comp3->id();
@@ -155,16 +164,10 @@
  TEST_F(BufferStreamTest, gives_different_back_buffer_asap)
+ {
--    ASSERT_THAT(buffers_free_for_client(), Ge(2)); // else we will hang
--
--    mg::Buffer* client_buffer{nullptr};
--    buffer_stream.swap_client_buffers_blocking(client_buffer);
--
--    ASSERT_THAT(nbuffers, Gt(1));
--    buffer_stream.swap_client_buffers_blocking(client_buffer);
++    buffer_stream.release_client_buffer(buffer_stream.acquire_client_buffer_blocking());
      auto comp1 = buffer_stream.lock_compositor_buffer(nullptr);
--    buffer_stream.swap_client_buffers_blocking(client_buffer);
++    buffer_stream.release_client_buffer(buffer_stream.acquire_client_buffer_blocking());
      auto comp2 = buffer_stream.lock_compositor_buffer(nullptr);
      EXPECT_NE(comp1->id(), comp2->id());
@@ -175,13 +178,13 @@
  TEST_F(BufferStreamTest, resize_affects_client_buffers_immediately)
+ {
--    ASSERT_THAT(buffers_free_for_client(), Ge(2)); // else we will hang
--
      auto old_size = buffer_stream.stream_size();
--    mg::Buffer* client{nullptr};
--    buffer_stream.swap_client_buffers_blocking(client);
++    mg::Buffer* client = buffer_stream.acquire_client_buffer_blocking();
      EXPECT_EQ(old_size, client->size());
++    buffer_stream.release_client_buffer(client);
++
++    buffer_stream.lock_compositor_buffer(this);
      geom::Size const new_size
+     {
@@ -191,28 +194,28 @@
      buffer_stream.resize(new_size);
      EXPECT_EQ(new_size, buffer_stream.stream_size());
--    buffer_stream.swap_client_buffers_blocking(client);
++    client = buffer_stream.acquire_client_buffer_blocking();
      EXPECT_EQ(new_size, client->size());
++    buffer_stream.release_client_buffer(client);
++
++    buffer_stream.lock_compositor_buffer(this);
      buffer_stream.resize(old_size);
      EXPECT_EQ(old_size, buffer_stream.stream_size());
--    /* Release a buffer so client can acquire another */
--    auto comp = buffer_stream.lock_compositor_buffer(nullptr);
--    comp.reset();
++    buffer_stream.lock_compositor_buffer(this);
--    buffer_stream.swap_client_buffers_blocking(client);
++    client = buffer_stream.acquire_client_buffer_blocking();
      EXPECT_EQ(old_size, client->size());
++    buffer_stream.release_client_buffer(client);
+ }
  TEST_F(BufferStreamTest, compositor_gets_resized_buffers)
+ {
--    ASSERT_THAT(buffers_free_for_client(), Ge(2)); // else we will hang
--
      auto old_size = buffer_stream.stream_size();
--    mg::Buffer* client{nullptr};
--    buffer_stream.swap_client_buffers_blocking(client);
++    mg::Buffer* client = buffer_stream.acquire_client_buffer_blocking();
++    buffer_stream.release_client_buffer(client);
      geom::Size const new_size
+     {
@@ -222,47 +225,54 @@
      buffer_stream.resize(new_size);
      EXPECT_EQ(new_size, buffer_stream.stream_size());
--    buffer_stream.swap_client_buffers_blocking(client);
--
      auto comp1 = buffer_stream.lock_compositor_buffer(nullptr);
++
++    client = buffer_stream.acquire_client_buffer_blocking();
++    buffer_stream.release_client_buffer(client);
++
      EXPECT_EQ(old_size, comp1->size());
      comp1.reset();
--    buffer_stream.swap_client_buffers_blocking(client);
--
      auto comp2 = buffer_stream.lock_compositor_buffer(nullptr);
++
++    client = buffer_stream.acquire_client_buffer_blocking();
++    buffer_stream.release_client_buffer(client);
++
      EXPECT_EQ(new_size, comp2->size());
      comp2.reset();
--    buffer_stream.swap_client_buffers_blocking(client);
--
      auto comp3 = buffer_stream.lock_compositor_buffer(nullptr);
++
++    client = buffer_stream.acquire_client_buffer_blocking();
++    buffer_stream.release_client_buffer(client);
++
      EXPECT_EQ(new_size, comp3->size());
      comp3.reset();
      buffer_stream.resize(old_size);
      EXPECT_EQ(old_size, buffer_stream.stream_size());
++    auto comp4 = buffer_stream.lock_compositor_buffer(nullptr);
++
      // No client frames "drawn" since resize(old_size), so compositor gets new_size
--    buffer_stream.swap_client_buffers_blocking(client);
--    auto comp4 = buffer_stream.lock_compositor_buffer(nullptr);
++    client = buffer_stream.acquire_client_buffer_blocking();
++    buffer_stream.release_client_buffer(client);
      EXPECT_EQ(new_size, comp4->size());
      comp4.reset();
++    auto comp5 = buffer_stream.lock_compositor_buffer(nullptr);
++
      // Generate a new frame, which should be back to old_size now
--    buffer_stream.swap_client_buffers_blocking(client);
--    auto comp5 = buffer_stream.lock_compositor_buffer(nullptr);
++    client = buffer_stream.acquire_client_buffer_blocking();
++    buffer_stream.release_client_buffer(client);
      EXPECT_EQ(old_size, comp5->size());
      comp5.reset();
+ }
  TEST_F(BufferStreamTest, can_get_partly_released_back_buffer)
+ {
--    ASSERT_THAT(buffers_free_for_client(), Ge(2)); // else we will hang
--
--    mg::Buffer* client{nullptr};
--    buffer_stream.swap_client_buffers_blocking(client);
--    buffer_stream.swap_client_buffers_blocking(client);
++    mg::Buffer* client = buffer_stream.acquire_client_buffer_blocking();
++    buffer_stream.release_client_buffer(client);
      int a, b, c;
      auto comp1 = buffer_stream.lock_compositor_buffer(&a);
 === modified file 'tests/integration-tests/test_surface_first_frame_sync.cpp'
 --- tests/integration-tests/test_surface_first_frame_sync.cpp	2014-05-29 15:51:54 +0000
 +++ tests/integration-tests/test_surface_first_frame_sync.cpp	2014-05-30 13:43:29 +0000
@@ -114,7 +114,7 @@
+     {
          for (auto const& r : renderables)
+         {
--            (void)r;
++            r->buffer(); // We need to consume a buffer to unblock client tests
              while (write(render_operations_fd, "a", 1) != 1) continue;
+         }
+     }
 === modified file 'tests/unit-tests/compositor/test_buffer_queue.cpp'
 --- tests/unit-tests/compositor/test_buffer_queue.cpp	2014-05-29 15:51:54 +0000
 +++ tests/unit-tests/compositor/test_buffer_queue.cpp	2014-05-30 13:43:29 +0000
@@ -305,10 +305,7 @@
+     }
      auto handle1 = client_acquire_async(q);
--    ASSERT_THAT(handle1->has_acquired_buffer(), Eq(false));
--
      auto handle2 = client_acquire_async(q);
--    ASSERT_THAT(handle1->has_acquired_buffer(), Eq(false));
      for (int i = 0; i < nbuffers + 1; ++i)
          q.compositor_release(q.compositor_acquire(this));
@@ -377,7 +374,6 @@
      handle->release_buffer();
      handle = client_acquire_async(q);
--    ASSERT_THAT(handle->has_acquired_buffer(), Eq(true));
      handle->release_buffer();
      auto comp_buffer = q.compositor_acquire(this);
@@ -623,13 +619,13 @@
      handle->release_buffer();
      handle = client_acquire_async(q);
--    ASSERT_THAT(handle->has_acquired_buffer(), Eq(true));
      // in the original bug, compositor would be given the wrong buffer here
      auto compositor_buffer = q.compositor_acquire(this);
      EXPECT_THAT(compositor_buffer->id(), Eq(first_ready_buffer_id));
++    ASSERT_THAT(handle->has_acquired_buffer(), Eq(true));
      handle->release_buffer();
      q.compositor_release(compositor_buffer);
+ }
@@ -800,7 +796,7 @@
+     }
+ }
--TEST_F(BufferQueueTest, framedropping_clients_get_all_buffers)
++TEST_F(BufferQueueTest, framedropping_clients_get_multiple_buffers)
+ {
      for (int nbuffers = 2; nbuffers <= max_nbuffers_to_test; ++nbuffers)
+     {
@@ -808,7 +804,9 @@
          q.allow_framedropping(true);
          int const nframes = 100;
--        int max_ownable_buffers = nbuffers - 1;
++        // Dynamic queue scaling sensibly limits the framedropping client to
++        // two non-overlapping buffers, before overwriting old ones.
++        int max_ownable_buffers = nbuffers <= 2 ? 1 : 2;
          std::unordered_set<uint32_t> ids_acquired;
          for (int i = 0; i < nframes; ++i)
+         {
@@ -995,6 +993,7 @@
+     {
          mc::BufferQueue q(nbuffers, allocator, basic_properties);
          mg::BufferID history[5];
++        std::shared_ptr<AcquireWaitHandle> producing[5];
          const int width0 = 123;
          const int height0 = 456;
@@ -1015,9 +1014,16 @@
              auto handle = client_acquire_async(q);
              ASSERT_THAT(handle->has_acquired_buffer(), Eq(true));
              history[produce] = handle->id();
++            producing[produce] = handle;
              auto buffer = handle->buffer();
              ASSERT_THAT(buffer->size(), Eq(new_size));
--            handle->release_buffer();
++        }
++
++        // Overlap all the client_acquires asyncronously. It's the only way
++        // the new dynamic queue scaling will let a client hold that many...
++        for (int complete = 0; complete < nbuffers_to_use; ++complete)
++        {
++            producing[complete]->release_buffer();
+         }
          width = width0;
@@ -1328,8 +1334,7 @@
+     }
+ }
--/* FIXME (enabling this optimization breaks timing tests) */
--TEST_F(BufferQueueTest, DISABLED_synchronous_clients_only_get_two_real_buffers)
++TEST_F(BufferQueueTest, synchronous_clients_only_get_two_real_buffers)
+ {
      for (int nbuffers = 2; nbuffers <= max_nbuffers_to_test; ++nbuffers)
+     {