[regression] titlebar in "canonical" example WM + KeyRepeatDispatcher causes deadlock

Introducing the KeyRepeatDispatcher means that input events can now occur on two threads: the "Input dispatch" thread and the GLibMainLoop thread. This leads to lock contention in EventFilterChainDispatcher which isn't necessarily fatal - but I think we'd be better using a single thread for dispatching input (to impose an order on events).

What is fatal is that this breaks the (dubious) assumption in CanonicalWindowManagerPolicyCopy that we can wait for swap_buffers() while handling an input event. This leads to a deadlock if an input event is on the GLibMainLoop thread (in the original description this is in EventFilterChainDispatcher, but that just prevents it occurring in BasicWindowManagerCopy).

Original text:

I caused this while testing a branch (lp:~alan-griffiths/mir/msh_SystemCompositorWindowManager), but I don't think the problem originates there:

$ sudo bin/mir_demo_server --file /tmp/mir_socket --vt 2 --arw-file --window-manager system-compositor

$ bin/mir_demo_server --host /tmp/mir_socket --window-manager canonical

$ bin/mir_demo_client_multiwin & bin/mir_demo_client_eglplasma &

After a few Alt+Tab focus switches things lock up in the nested server. Attaching with GDB gets the following stack traces:

The event loop (via) KeyRepeatDispatcher is waiting for a lock in EventFilterChainDispatcher::handle().

#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1 0x00007ffff6960cfd in __GI___pthread_mutex_lock (mutex=0x7ca060) at ../nptl/pthread_mutex_lock.c:80
#2 0x00007ffff76e7815 in __gthread_mutex_lock (__mutex=0x7ca060)
    at /usr/include/x86_64-linux-gnu/c++/4.9/bits/gthr-default.h:748
#3 0x00007ffff76e8738 in std::mutex::lock (this=0x7ca060) at /usr/include/c++/4.9/mutex:135
#4 0x00007ffff76e8b2e in std::lock_guard<std::mutex>::lock_guard (this=0x7fffffffdbb0, __m=...)
    at /usr/include/c++/4.9/mutex:377
#5 0x00007ffff775d1db in mir::input::EventFilterChainDispatcher::handle (this=0x7ca050, event=...)
    at /home/alan/display_server/mir3/src/server/input/event_filter_chain_dispatcher.cpp:34
#6 0x00007ffff775d510 in mir::input::EventFilterChainDispatcher::dispatch (this=0x7ca050, event=...)
    at /home/alan/display_server/mir3/src/server/input/event_filter_chain_dispatcher.cpp:67
#7 0x00007ffff7796e2e in mir::input::KeyRepeatDispatcher::<lambda()>::operator()(void) (__closure=0x7fffe000a0a0)
    at /home/alan/display_server/mir3/src/server/input/key_repeat_dispatcher.cpp:117
#8 0x00007ffff7797512 in std::_Function_handler<void(), mir::input::KeyRepeatDispatcher::handle_key_input(MirInputDeviceId, const MirKeyboardEvent*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/4.9/functional:2039
#9 0x00007ffff76ee54c in std::function<void ()>::operator()() const (this=0x7fffe0000e68)
    at /usr/include/c++/4.9/functional:2439
#10 0x00007ffff773b626 in mir::BasicCallback::operator() (this=0x7fffe0000e60)
    at /home/alan/display_server/mir3/src/server/basic_callback.cpp:28
#11 0x00007ffff773b111 in mir::LockableCallbackWrapper::operator() (this=0x7fffe00036f0)
    at /home/alan/display_server/mir3/src/server/lockable_callback_wrapper.cpp:41
#12 0x00007ffff770face in mir::detail::TimerGSource::dispatch (source=0x949320)
    at /home/alan/display_server/mir3/src/server/glib_main_loop_sources.cpp:299
#13 0x00007ffff5443c3d in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#14 0x00007ffff5443f20 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#15 0x00007ffff5443fcc in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#16 0x00007ffff7705f91 in mir::GLibMainLoop::run (this=0x6f6e70)
---Type <return> to continue, or q <return> to quit---

The lock is held by the window manager (which is trying to paint a titlebar).

#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007ffff64f7d1c in std::condition_variable::wait(std::unique_lock<std::mutex>&) ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#2 0x00000000004b270b in std::condition_variable::wait<(anonymous namespace)::swap_buffers(const std::shared_ptr<mir::scene::Surface>&, mir::graphics::Buffer*&)::<lambda()> >(std::unique_lock<std::mutex> &, (anonymous namespace)::<lambda()>) (
    this=0x7fffee2f21b0, __lock=..., __p=...) at /usr/include/c++/4.9/condition_variable:98
#3 0x00000000004aeaa4 in (anonymous namespace)::swap_buffers (surface=std::shared_ptr (count 4, weak 4) 0x7fffc4006750,
    surface_buffer=@0x7fffc4006fc0: 0x7fffc40113f0)
    at /home/alan/display_server/mir3/examples/server_example_canonical_window_manager.cpp:277
#4 0x00000000004aecea in (anonymous namespace)::paint_titlebar (titlebar=std::shared_ptr (count 4, weak 4) 0x7fffc4006750,
    titlebar_info=..., intensity=255) at /home/alan/display_server/mir3/examples/server_example_canonical_window_manager.cpp:297
#5 0x00000000004b0f45 in mir::examples::CanonicalWindowManagerPolicyCopy::select_active_surface (this=0x944cd8,
    surface=std::shared_ptr (count 3, weak 5) 0x7fffc4004fd0)
    at /home/alan/display_server/mir3/examples/server_example_canonical_window_manager.cpp:742
#6 0x00000000004b042a in mir::examples::CanonicalWindowManagerPolicyCopy::handle_keyboard_event (this=0x944cd8,
    event=0x7fffe0000910) at /home/alan/display_server/mir3/examples/server_example_canonical_window_manager.cpp:577
#7 0x00000000004a0c5e in mir::examples::BasicWindowManagerCopy<mir::examples::CanonicalWindowManagerPolicyCopy, mir::examples::CanonicalSessionInfoCopy, mir::examples::CanonicalSurfaceInfoCopy>::handle_keyboard_event (this=0x944cc0, event=0x7fffe0000910)
    at /home/alan/display_server/mir3/examples/server_example_basic_window_manager.h:190
#8 0x00007ffff78beccb in mir::shell::AbstractShell::handle (this=0x944bf0, event=...)
    at /home/alan/display_server/mir3/src/server/shell/abstract_shell.cpp:245
#9 0x00007ffff775d27c in mir::input::EventFilterChainDispatcher::handle (this=0x7ca050, event=...)
    at /home/alan/display_server/mir3/src/server/input/event_filter_chain_dispatcher.cpp:45
#10 0x00007ffff775d510 in mir::input::EventFilterChainDispatcher::dispatch (this=0x7ca050, event=...)
    at /home/alan/display_server/mir3/src/server/input/event_filter_chain_dispatcher.cpp:67
#11 0x00007ffff7796d22 in mir::input::KeyRepeatDispatcher::dispatch (this=0x7ca0d0, event=...)
    at /home/alan/display_server/mir3/src/server/input/key_repeat_dispatcher.cpp:63
---Type <return> to continue, or q <return> to quit---

I guess the titlebar paint doesn't complete because the event loop is waiting.