Mir

Merge lp:~alan-griffiths/mir/fix-1535894 into lp:mir

Proposed by Alan Griffiths
Status: Rejected
Rejected by: Alan Griffiths
Proposed branch: lp:~alan-griffiths/mir/fix-1535894
Merge into: lp:mir
Diff against target: 112 lines (+38/-1)
6 files modified
include/server/mir/compositor/scene.h (+2/-0)
src/server/compositor/multi_threaded_compositor.cpp (+2/-1)
src/server/scene/surface_stack.cpp (+25/-0)
src/server/scene/surface_stack.h (+1/-0)
tests/include/mir/test/doubles/mock_scene.h (+4/-0)
tests/include/mir/test/doubles/stub_scene.h (+4/-0)
To merge this branch: bzr merge lp:~alan-griffiths/mir/fix-1535894
Reviewer Review Type Date Requested Status
Kevin DuBois (community) Approve
Daniel van Vugt Needs Fixing
PS Jenkins bot (community) continuous-integration Approve
Mir CI Bot continuous-integration Needs Fixing
Review via email: mp+285058@code.launchpad.net

Commit message

compositor, scene: Filter Scene::frames_pending() by output view area

Description of the change

compositor, scene: Filter Scene::frames_pending() by output view area

We recently landed a heuristic to not schedule composition for an output if a posted buffer doesn't intersect it. This was less effective than intended as these buffers were still counted at the end of a composition pass when deciding whether to do an additional composition pass.

To post a comment you must log in.
Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

FAILED: Continuous integration, rev:3295
https://mir-jenkins.ubuntu.com/job/mir-ci/230/
Executed test runs:
    None: https://mir-jenkins.ubuntu.com/job/generic-update-mp/230/console

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/230/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

DEBUG: jenkins job parameter [use_description_for_commit]: not found
DEBUG: Unable to get "use_description_for_commit" configuration for mir-ci
DEBUG: Job is configured to fallback to description: False

/me dons SEP glasses

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Alexandros Frantzis (afrantzis) wrote :

> /me dons SEP glasses

I will check whether that message should worry us, but the real problem seems to be:

21: [ RUN ] MultiThreadedCompositor.schedules_enough_frames
21: /��BUILDDIR��/mir-0.20.0+xenial20bzr3295/tests/unit-tests/compositor/test_multi_threaded_compositor.cpp:608: Failure
21: Expected: (retry) < (max_retries), actual: 100 vs 100
21: [ FAILED ] MultiThreadedCompositor.schedules_enough_frames (1200 ms)

For example in: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-advanced/arch=i386,compiler=gcc,platform=mesa,release=xenial/20/consoleFull

lp:~alan-griffiths/mir/fix-1535894 updated
3296. By Alan Griffiths

I wonder why that passes locally

Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

FAILED: Continuous integration, rev:3296
https://mir-jenkins.ubuntu.com/job/mir-ci/233/
Executed test runs:
    None: https://mir-jenkins.ubuntu.com/job/generic-update-mp/233/console

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/233/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Alexandros Frantzis (afrantzis) wrote :

Not sure if this is related to changes in this MP:

https://mir-jenkins.ubuntu.com/job/build-2-binpkg-advanced/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/23/consoleFull

11: [ RUN ] AndroidInputReceiverSetup.slow_raw_input_doesnt_cause_frameskipping
11: /��BUILDDIR��/mir-0.20.0+vivid23bzr3296/tests/unit-tests/client/input/test_android_input_receiver.cpp:276: Failure
11: Expected: (duration) < (one_frame), actual: 8-byte object <05-B9 96-01 00-00 00-00> vs 8-byte object <2A-50 FE-00 00-00 00-00>
11: [ FAILED ] AndroidInputReceiverSetup.slow_raw_input_doesnt_cause_frameskipping (380 ms)

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> Not sure if this is related to changes in this MP:
>
> https://mir-jenkins.ubuntu.com/job/build-2-binpkg-advanced/arch=i386,compiler=
> gcc,platform=android,release=vivid+overlay/23/consoleFull
>
> 11: [ RUN ]
> AndroidInputReceiverSetup.slow_raw_input_doesnt_cause_frameskipping
> 11: /��BUILDDIR��/mir-0.20.0+vivid23bzr3296/tests/unit-
> tests/client/input/test_android_input_receiver.cpp:276: Failure
> 11: Expected: (duration) < (one_frame), actual: 8-byte object <05-B9 96-01
> 00-00 00-00> vs 8-byte object <2A-50 FE-00 00-00 00-00>
> 11: [ FAILED ]
> AndroidInputReceiverSetup.slow_raw_input_doesnt_cause_frameskipping (380 ms)

I don't see how - AFAICS the test doesn't contain a MultiThreadedCompositor.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

Ah. It's lp:1394369

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:3296
http://jenkins.qa.ubuntu.com/job/mir-ci/6209/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-vivid-i386-build/5806
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-clang-vivid-amd64-build/4713
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-vivid-touch/5762
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-xenial-touch/389
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-xenial-amd64-ci/533
        deb: http://jenkins.qa.ubuntu.com/job/mir-xenial-amd64-ci/533/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-xenial-i386-ci/533
        deb: http://jenkins.qa.ubuntu.com/job/mir-xenial-i386-ci/533/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-vivid-armhf/5759
        deb: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-vivid-armhf/5759/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-runner-touch/8169
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/27328
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-xenial-armhf/385
        deb: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-xenial-armhf/385/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-runner-xenial-touch/239
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/27336

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/mir-ci/6209/rebuild

review: Approve (continuous-integration)
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Like the fix that came before it, this sounds like a workaround rather than a fix.

You are addressing the multi-monitor performance problem by only attempting to redraw one monitor per frame. That sounds like an admission of failure and that we can't render multi-monitor at full frame rate on Android. So I'd prefer to see that problem fixed and full frame rate on both displays simultaneously. Although we might be hitting the physical performance limits of the device, in which case changes like this is all we'd have left. Not yet proven though, and that would be a bit pessimistic.

I actually don't think we're hitting the device performance limitation as much as trying to run sub-optimal code on it:
https://bugs.launchpad.net/ubuntu/+source/unity8/+bug/1494795/comments/4

review: Abstain
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Then again, if we don't intend to try and animate both screens at once then that's probably OK...

I'll stop short of approving though, because there's an additional problem with this branch that will break free-form rendering like Compiz or mir_proving_server -- Some pixels are painted outside of the surface itself (like shadows or other effects like minimizing animation). As such, your view area test will be wrong, and will fail to update the secondary screen when it needs updating.

Worse again, this change will fail completely under full screen transformations like desktop zoom. And again will fail to refresh displays correctly, appearing to freeze instead.

I actually disapprove, but will abstain because can see it will help pocket desktop as it stands right now.

review: Abstain
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Oh I forgot, we already have this feature... :)

void mc::DefaultDisplayBufferCompositor::composite(mc::SceneElementSequence&& scene_elements)
{
    report->began_frame(this);

    auto const& view_area = display_buffer.view_area();
    auto const& occlusions = mc::filter_occlusions_from(scene_elements, view_area);

Using this existing implementation would be preferable. It's just harder to modify now that we have display groups with the swap buffers call outside of the compositor. But not impossible. Plus doing the fix in the compositor would address my concerns for the future too.

All we need to do is return some result to the caller to tell them whether we have repainted (empty renderable_list), and so whether they should swap buffers.

Admittedly DefaultDisplayBufferCompositor is only used in USC and probably not in Unity8 any more. But we could do a similar fix in QtMir to address Unity8.

Fixing the problem in the compositor is ideal, because that's the only place where you might know about the additional rendering that's outside the expected confines of the surface rectangle.

It's not just other shells that don't exist yet which need this, but Unity8 needs it right now. Consider Unity8 on desktop with two monitors -- It does external rendering like minimize animations, shadows, and the whole launcher, that surface_stack.cpp doesn't know about and will get wrong using your proposed implementation.

review: Needs Fixing
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> Fixing the problem in the compositor is ideal, because that's the only place
> where you might know about the additional rendering that's outside the
> expected confines of the surface rectangle.

I assume you mean "Fixing the problem in the DisplayBufferCompositor" (not the Compositor).

Shifting the logic from this MP and from -c 3274 to the DefaultDisplayBufferCompositor (with appropriate wiring) would address your concerns with both?

I recall that we had similar logic once in an earlier version, with the DisplayBufferCompositor returning a measure of uncomposited buffers. Can you recall why that was removed - we don't want to repeat the mistakes of the past.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> Admittedly DefaultDisplayBufferCompositor is only used in USC and probably not
> in Unity8 any more. But we could do a similar fix in QtMir to address Unity8.
...
> It's not just other shells that don't exist yet which need this, but Unity8
> needs it right now. Consider Unity8 on desktop with two monitors -- It does
> external rendering like minimize animations, shadows, and the whole launcher,
> that surface_stack.cpp doesn't know about and will get wrong using your
> proposed implementation.

This highlights a serious failing in our customization points.

Unity8 uses QtCompositor from QtMir, not MultiThreadedCompositor from Mir - and [Default]DisplayBufferCompositor is only used by the latter.

So the difference between this approach and your suggested alternative makes no difference to Unity8.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> Oh I forgot, we already have this feature... :)
>
> void mc::DefaultDisplayBufferCompositor::composite(mc::SceneElementSequence&&
> scene_elements)
> {
> report->began_frame(this);
>
> auto const& view_area = display_buffer.view_area();
> auto const& occlusions = mc::filter_occlusions_from(scene_elements,
> view_area);
>
> Using this existing implementation would be preferable. It's just harder to
> modify now that we have display groups with the swap buffers call outside of
> the compositor. But not impossible. Plus doing the fix in the compositor would
> address my concerns for the future too.
>
> All we need to do is return some result to the caller to tell them whether we
> have repainted (empty renderable_list), and so whether they should swap
> buffers.

Having thought about implementing this suggestion I don't think it works. (At least, not simply.)

The criteria for repainting is more complex that "!renderable_list.empty()". For example, when the last surface is removed from the view area the current proposal repaints, but, as I understand your suggestion it would not.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

Another important case that isn't easily addressed by code in the context of DefaultDisplayBufferCompositor is when all the surfaces overlapping the view_area already have their latest buffer rendered. (This cannot be deduced from the SceneElement interface.)

Revision history for this message
Kevin DuBois (kdub) wrote :

fix looks alright to me, so +1

Curious as to why "void frames_pending(CompositorID, Rectangle)" was added, as opposed to improving "void frames_pending(CompositorID)" to take an additional parameter. Could also use a test.

review: Approve
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

> I recall that we had similar logic once in an earlier version, with the
> DisplayBufferCompositor returning a measure of uncomposited buffers. Can you
> recall why that was removed - we don't want to repeat the mistakes of the
> past.

Almost correct. What we had in an earlier version was a bool returned, which at the time indicated the DisplayBufferCompositor wanted to be scheduled for the following frame too. That's the Compiz way of doing things and it was removed (by kdub?) because it became dead code -- its first purpose of tracking pending buffers per surface was superseded, and its second purpose for compositor-based animations has not yet ever been implemented (I still want to though).

But that just highlights a bool isn't really enough, and is now confusing. The DisplayBufferCompositor might need to communicate more detailed information back to its caller than just a bool.

Yes indeed r3274 was flawed for the same reason and needs to be moved into DisplayBufferCompositor. Keeping in mind Unity8 doesn't use that and some equivalent enhancement is required to QtMir too.

Also, here's a manual test case showing this branch causes visible regressions:
  1. mir_proving_server --display-config sidebyside # needs 2 or more monitors
  2. mir_demo_client_egltriangle
  3. Move the window to the right half of the left monitor near the edge, but not overlapping.
  4. Zoom in with Super+mousewheel and move the mouse such that the zoomed window now appears on both monitors.
Using lp:mir - Triangle keeps animating smoothly.
Using this branch - Triangle freezes and stutters on the second monitor.

The same problem extends to regular viewing without any zoom. If you had a launcher notification animating on one screen, but no major window changes then the launcher animation would never be visible. Instead the screen with the launcher would appear to be frozen and idle, erroneously hiding the notification.

Someone might make the argument that in the Unity8 architecture this change only applies to system compositors and will work for Unity8. However that means we're now saying Mir multimonitor only works in a nested architectures where each system compositor surface fits one physical displays. That's not an acceptable limitation we should choose to introduce. Mir should still support non-nested configurations.

Unmerged revisions

3296. By Alan Griffiths

I wonder why that passes locally

3295. By Alan Griffiths

Filter Scene::frames_pending() by output area

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'include/server/mir/compositor/scene.h'
2--- include/server/mir/compositor/scene.h 2015-02-22 07:46:25 +0000
3+++ include/server/mir/compositor/scene.h 2016-02-04 15:01:17 +0000
4@@ -26,6 +26,7 @@
5
6 namespace mir
7 {
8+namespace geometry { class Rectangle; }
9 namespace scene
10 {
11 class Observer;
12@@ -64,6 +65,7 @@
13 * scene.
14 */
15 virtual int frames_pending(CompositorID id) const = 0;
16+ virtual int frames_pending(CompositorID id, geometry::Rectangle const& view) const = 0;
17
18 virtual void register_compositor(CompositorID id) = 0;
19 virtual void unregister_compositor(CompositorID id) = 0;
20
21=== modified file 'src/server/compositor/multi_threaded_compositor.cpp'
22--- src/server/compositor/multi_threaded_compositor.cpp 2016-02-01 09:42:10 +0000
23+++ src/server/compositor/multi_threaded_compositor.cpp 2016-02-04 15:01:17 +0000
24@@ -164,8 +164,9 @@
25 int pending = 0;
26 for (auto& compositor : compositors)
27 {
28+ auto const buffer = std::get<0>(compositor);
29 auto const comp_id = std::get<1>(compositor).get();
30- int pend = scene->frames_pending(comp_id);
31+ int pend = scene->frames_pending(comp_id, buffer->view_area());
32 if (pend > pending)
33 pending = pend;
34 }
35
36=== modified file 'src/server/scene/surface_stack.cpp'
37--- src/server/scene/surface_stack.cpp 2016-01-29 08:18:22 +0000
38+++ src/server/scene/surface_stack.cpp 2016-02-04 15:01:17 +0000
39@@ -178,6 +178,31 @@
40 return result;
41 }
42
43+int ms::SurfaceStack::frames_pending(mc::CompositorID id, geom::Rectangle const& view) const
44+{
45+ RecursiveReadLock lg(guard);
46+
47+ int result = scene_changed ? 1 : 0;
48+ for (auto const& surface : surfaces)
49+ {
50+ if (surface->visible() && view.overlaps({surface->top_left(), surface->size()}))
51+ {
52+ auto const tracker = rendering_trackers.find(surface.get());
53+ if (tracker != rendering_trackers.end() && tracker->second->is_exposed_in(id))
54+ {
55+ // Note that we ask the surface and not a Renderable.
56+ // This is because we don't want to waste time and resources
57+ // on a snapshot till we're sure we need it...
58+ int ready = surface->buffers_ready_for_compositor(id);
59+ if (ready > result)
60+ result = ready;
61+ }
62+ }
63+ }
64+ return result;
65+}
66+
67+
68 void ms::SurfaceStack::register_compositor(mc::CompositorID cid)
69 {
70 RecursiveWriteLock lg(guard);
71
72=== modified file 'src/server/scene/surface_stack.h'
73--- src/server/scene/surface_stack.h 2016-01-29 08:18:22 +0000
74+++ src/server/scene/surface_stack.h 2016-02-04 15:01:17 +0000
75@@ -75,6 +75,7 @@
76 // From Scene
77 compositor::SceneElementSequence scene_elements_for(compositor::CompositorID id) override;
78 int frames_pending(compositor::CompositorID) const override;
79+ int frames_pending(compositor::CompositorID, geometry::Rectangle const&) const override;
80 void register_compositor(compositor::CompositorID id) override;
81 void unregister_compositor(compositor::CompositorID id) override;
82
83
84=== modified file 'tests/include/mir/test/doubles/mock_scene.h'
85--- tests/include/mir/test/doubles/mock_scene.h 2015-02-22 07:46:25 +0000
86+++ tests/include/mir/test/doubles/mock_scene.h 2016-02-04 15:01:17 +0000
87@@ -42,6 +42,10 @@
88
89 MOCK_METHOD1(scene_elements_for, compositor::SceneElementSequence(compositor::CompositorID));
90 MOCK_CONST_METHOD1(frames_pending, int(compositor::CompositorID));
91+
92+ int frames_pending(compositor::CompositorID id, geometry::Rectangle const&) const override
93+ { return frames_pending(id); }
94+
95 MOCK_METHOD1(register_compositor, void(compositor::CompositorID));
96 MOCK_METHOD1(unregister_compositor, void(compositor::CompositorID));
97
98
99=== modified file 'tests/include/mir/test/doubles/stub_scene.h'
100--- tests/include/mir/test/doubles/stub_scene.h 2015-02-22 07:46:25 +0000
101+++ tests/include/mir/test/doubles/stub_scene.h 2016-02-04 15:01:17 +0000
102@@ -40,6 +40,10 @@
103 {
104 return 0;
105 }
106+ int frames_pending(compositor::CompositorID id, geometry::Rectangle const&) const override
107+ {
108+ return frames_pending(id);
109+ }
110 void register_compositor(compositor::CompositorID) override
111 {
112 }

Subscribers

People subscribed via source and target branches