Merge lp:~alan-griffiths/mir/fix-1516670 into lp:mir
- fix-1516670
- Merge into development-branch
Status: | Merged |
---|---|
Approved by: | Alan Griffiths |
Approved revision: | no longer in the source branch. |
Merged at revision: | 3131 |
Proposed branch: | lp:~alan-griffiths/mir/fix-1516670 |
Merge into: | lp:mir |
Diff against target: |
192 lines (+78/-37) 3 files modified
src/server/graphics/nested/display.cpp (+67/-30) src/server/graphics/nested/display.h (+2/-0) tests/acceptance-tests/test_nested_mir.cpp (+9/-7) |
To merge this branch: | bzr merge lp:~alan-griffiths/mir/fix-1516670 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
PS Jenkins bot (community) | continuous-integration | Approve | |
Alexandros Frantzis (community) | Needs Information | ||
Andreas Pokorny (community) | Approve | ||
Alberto Aguirre | Pending | ||
Review via email:
|
This proposal supersedes a proposal from 2015-11-17.
Commit message
nested: For clone mode use a fullscreen surface for the output with the largest intersection with the bounding rectangle
Description of the change
nested: For clone mode use a fullscreen surface for the output with the largest intersection with the bounding rectangle
The previous implementation attempted to cover the bounding rectangle of all outputs with a surface that was fullscreen on the first - which led to inconsistent states.
The surface creation logic has also been reworked to reuse existing surfaces where possible, which makes the server interaction less racy (I suspect there are some races still, but this makes plugging and replugging a lot more stable on the phone).
I also added some information to the surface name (which can be helpful when debugging window management in the host server).
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Alan Griffiths (alan-griffiths) wrote : Posted in a previous version of this proposal | # |
Having a stack of overlapping fullscreen surfaces in the host server introduces some visual artifacts (and inefficiencies).
So this is clearly not the ideal solution - just less broken than what we currently have.
There are a number of approaches for a better solution:
1. use non-fullscreen surfaces in nested mode and either:
1.1. display only the intersection of all outputs
1.2. display the bounding rectangle of all outputs
2. use only fullscreen surfaces and either:
2.1. select the "smallest" display
2.2. select the "largest" display
I'm not yet convinced by any of the above options, but favour of sticking with "fullscreen only" when nesting.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Alberto Aguirre (albaguirre) wrote : Posted in a previous version of this proposal | # |
Are the visual artifacts on phone/desktop/both?
And what are the cases where outputs overlap?
We could also potentially run into max texture size issues for high res displays if we attempt to do 1.2.
In any case, a fullscreen surface per output makes sense to me.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Alan Griffiths (alan-griffiths) wrote : Posted in a previous version of this proposal | # |
> Are the visual artifacts on phone/desktop/both?
Both, but easier to see on phone.
> And what are the cases where outputs overlap?
When displays of different sizes are "cloned".
So, with an external monitor attached to a mako:
$ sudo stop lightdm
$ bin/mir_demo_server --window-manager system-compositor&
$ bin/mir_
On the external monitor the fullscreen for the phone output is overlayed[1] on top of the fullscreen for the external monitor and the edge is visible as the flying surfaces it (as the rendering is not quite in sync.)
> We could also potentially run into max texture size issues for high res
> displays if we attempt to do 1.2.
Yep.
> In any case, a fullscreen surface per output makes sense to me.
[1] I'm not sure if this ordering is guaranteed, but was consistent in my limited testing.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal | # |
PASSED: Continuous integration, rev:3107
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
FAILURE: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
SUCCESS: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:3116
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
FAILURE: http://
FAILURE: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
FAILURE: http://
SUCCESS: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:3117
http://
Executed test runs:
SUCCESS: http://
FAILURE: http://
SUCCESS: http://
FAILURE: http://
SUCCESS: http://
deb: http://
FAILURE: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
SUCCESS: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:3118
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
FAILURE: http://
FAILURE: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
FAILURE: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andreas Pokorny (andreas-pokorny) wrote : | # |
hm I think this is an improvement but in clone mode we should use the smallest screen size for each surface and have the output scale.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:3119
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
FAILURE: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
SUCCESS: http://
FAILURE: http://
Click here to trigger a rebuild:
http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andreas Pokorny (andreas-pokorny) wrote : | # |
Ok overall after looking and testing (on kms).. our default display configuration for cloned seems to be wrong. It should figure out a suitable intersection and try to select suitable modes on each output, instead of just aligning top_left. If that would be in place it would not matter if we pick the maximum overlap or intersection. (The former might be a better solution then).
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Alexandros Frantzis (afrantzis) wrote : | # |
Note that following scenario is still not working for me (but I am not sure if this MP was supposed to fix this):
Internal monitor 1280x800
External monitor 1920x1080
In different shells:
$ sudo bin/mir_demo_server --arw-file -f /tmp/mir_host
$ bin/mir_demo_server --host-socket /tmp/mir_host -f /tmp/mir_nested --vt 1
$ bin/mir_
Should start in clone mode, press 'h', press 'c' again. After last 'c' we should expect to go back to the original setup but we get into a strange setup instead. There seem to be two "framebuffer" surfaces, the first sized for the internal screen and the second for the external. On the external screen the two FB surfaces overlap, with the internal one being in front and hiding the top left part of the external. The client surface is only drawn on the external FB surface.
Also note that both the old and the new code don't handle properly cases where one output doesn't fully contain the other (e.g. 100x100+0+0 100x100+50+50). It's not a common case but we should (eventually) either support it or explicitly prohibit it.
> area.intersecti
Since area is the bounding rectangle of all outputs, isn't this expression equivalent to just output.extents()?
@Andreas
> hm I think this is an improvement but in clone mode we should use the smallest screen size
> for each surface and have the output scale.
...
> It should figure out a suitable intersection and try to select suitable modes on each output,
> instead of just aligning top_left.
Fair enough, we should probably rename the current clone policy to '(max)overlap' and implement a new 'clone' one. In any case there are both valid policies and should be properly supported. Plus there are cases where we can't 'clone' because there aren't any matching modes, so we need to fall back to 'overlap' anyway.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
PS Jenkins bot (ps-jenkins) : | # |
Preview Diff
1 | === modified file 'src/server/graphics/nested/display.cpp' | |||
2 | --- src/server/graphics/nested/display.cpp 2015-11-18 12:38:30 +0000 | |||
3 | +++ src/server/graphics/nested/display.cpp 2015-11-25 23:12:50 +0000 | |||
4 | @@ -34,6 +34,7 @@ | |||
5 | 34 | 34 | ||
6 | 35 | #include <boost/throw_exception.hpp> | 35 | #include <boost/throw_exception.hpp> |
7 | 36 | #include <stdexcept> | 36 | #include <stdexcept> |
8 | 37 | #include <sstream> | ||
9 | 37 | 38 | ||
10 | 38 | namespace mi = mir::input; | 39 | namespace mi = mir::input; |
11 | 39 | namespace mg = mir::graphics; | 40 | namespace mg = mir::graphics; |
12 | @@ -147,6 +148,11 @@ | |||
13 | 147 | return std::chrono::milliseconds::zero(); | 148 | return std::chrono::milliseconds::zero(); |
14 | 148 | } | 149 | } |
15 | 149 | 150 | ||
16 | 151 | geom::Rectangle mgn::detail::DisplaySyncGroup::view_area() const | ||
17 | 152 | { | ||
18 | 153 | return output->view_area(); | ||
19 | 154 | } | ||
20 | 155 | |||
21 | 150 | mgn::Display::Display( | 156 | mgn::Display::Display( |
22 | 151 | std::shared_ptr<mg::Platform> const& platform, | 157 | std::shared_ptr<mg::Platform> const& platform, |
23 | 152 | std::shared_ptr<HostConnection> const& connection, | 158 | std::shared_ptr<HostConnection> const& connection, |
24 | @@ -203,11 +209,16 @@ | |||
25 | 203 | void mgn::Display::configure(mg::DisplayConfiguration const& configuration) | 209 | void mgn::Display::configure(mg::DisplayConfiguration const& configuration) |
26 | 204 | { | 210 | { |
27 | 205 | std::lock_guard<std::mutex> lock(configuration_mutex); | 211 | std::lock_guard<std::mutex> lock(configuration_mutex); |
33 | 206 | if (*current_configuration != configuration) | 212 | apply_to_connection(configuration); |
34 | 207 | { | 213 | create_surfaces(configuration); |
35 | 208 | apply_to_connection(configuration); | 214 | } |
36 | 209 | create_surfaces(configuration); | 215 | |
37 | 210 | } | 216 | namespace |
38 | 217 | { | ||
39 | 218 | long area_of(geom::Rectangle const& rect) | ||
40 | 219 | { | ||
41 | 220 | return static_cast<long>(rect.size.width.as_int())*rect.size.height.as_int(); | ||
42 | 221 | } | ||
43 | 211 | } | 222 | } |
44 | 212 | 223 | ||
45 | 213 | void mgn::Display::create_surfaces(mg::DisplayConfiguration const& configuration) | 224 | void mgn::Display::create_surfaces(mg::DisplayConfiguration const& configuration) |
46 | @@ -223,35 +234,61 @@ | |||
47 | 223 | unique_outputs.for_each_group( | 234 | unique_outputs.for_each_group( |
48 | 224 | [&](mg::OverlappingOutputGroup const& group) | 235 | [&](mg::OverlappingOutputGroup const& group) |
49 | 225 | { | 236 | { |
53 | 226 | bool have_output_for_group = false; | 237 | geometry::Rectangle const area = group.bounding_rectangle(); |
54 | 227 | geometry::Rectangle const& area = group.bounding_rectangle(); | 238 | |
55 | 228 | group.for_each_output([&](mg::DisplayConfigurationOutput output) | 239 | long max_overlap_area = 0; |
56 | 240 | mg::DisplayConfigurationOutput best_output; | ||
57 | 241 | |||
58 | 242 | group.for_each_output([&](mg::DisplayConfigurationOutput const& output) | ||
59 | 229 | { | 243 | { |
61 | 230 | if (!have_output_for_group) | 244 | if (area_of(area.intersection_with(output.extents())) > max_overlap_area) |
62 | 231 | { | 245 | { |
84 | 232 | auto const& egl_config_format = output.current_format; | 246 | max_overlap_area = area_of(area.intersection_with(output.extents())); |
85 | 233 | 247 | best_output = output; | |
65 | 234 | complete_display_initialization(egl_config_format); | ||
66 | 235 | |||
67 | 236 | auto const host_surface = connection->create_surface( | ||
68 | 237 | area.size.width.as_int(), | ||
69 | 238 | area.size.height.as_int(), | ||
70 | 239 | egl_config_format, | ||
71 | 240 | "Mir nested display", | ||
72 | 241 | mir_buffer_usage_hardware, | ||
73 | 242 | static_cast<uint32_t>(output.id.as_value())); | ||
74 | 243 | |||
75 | 244 | result[output.id] = std::make_shared<mgn::detail::DisplaySyncGroup>( | ||
76 | 245 | std::make_shared<mgn::detail::DisplayBuffer>( | ||
77 | 246 | egl_display, | ||
78 | 247 | host_surface, | ||
79 | 248 | area, | ||
80 | 249 | dispatcher, | ||
81 | 250 | cursor_listener, | ||
82 | 251 | output.current_format)); | ||
83 | 252 | have_output_for_group = true; | ||
86 | 253 | } | 248 | } |
87 | 254 | }); | 249 | }); |
88 | 250 | |||
89 | 251 | auto const& egl_config_format = best_output.current_format; | ||
90 | 252 | geometry::Rectangle const& extents = best_output.extents(); | ||
91 | 253 | |||
92 | 254 | auto& display_buffer = result[best_output.id]; | ||
93 | 255 | |||
94 | 256 | { | ||
95 | 257 | std::unique_lock<std::mutex> lock(outputs_mutex); | ||
96 | 258 | display_buffer = outputs[best_output.id]; | ||
97 | 259 | } | ||
98 | 260 | |||
99 | 261 | if (display_buffer) | ||
100 | 262 | { | ||
101 | 263 | if (display_buffer->view_area() != extents) | ||
102 | 264 | display_buffer.reset(); | ||
103 | 265 | } | ||
104 | 266 | |||
105 | 267 | if (!display_buffer) | ||
106 | 268 | { | ||
107 | 269 | complete_display_initialization(egl_config_format); | ||
108 | 270 | |||
109 | 271 | std::ostringstream surface_title; | ||
110 | 272 | |||
111 | 273 | surface_title << "Mir nested display for output #" << best_output.id.as_value(); | ||
112 | 274 | |||
113 | 275 | auto const host_surface = connection->create_surface( | ||
114 | 276 | extents.size.width.as_int(), | ||
115 | 277 | extents.size.height.as_int(), | ||
116 | 278 | egl_config_format, | ||
117 | 279 | surface_title.str().c_str(), | ||
118 | 280 | mir_buffer_usage_hardware, | ||
119 | 281 | static_cast<uint32_t>(best_output.id.as_value())); | ||
120 | 282 | |||
121 | 283 | display_buffer = std::make_shared<mgn::detail::DisplaySyncGroup>( | ||
122 | 284 | std::make_shared<mgn::detail::DisplayBuffer>( | ||
123 | 285 | egl_display, | ||
124 | 286 | host_surface, | ||
125 | 287 | extents, | ||
126 | 288 | dispatcher, | ||
127 | 289 | cursor_listener, | ||
128 | 290 | best_output.current_format)); | ||
129 | 291 | } | ||
130 | 255 | }); | 292 | }); |
131 | 256 | 293 | ||
132 | 257 | { | 294 | { |
133 | 258 | 295 | ||
134 | === modified file 'src/server/graphics/nested/display.h' | |||
135 | --- src/server/graphics/nested/display.h 2015-10-29 02:15:05 +0000 | |||
136 | +++ src/server/graphics/nested/display.h 2015-11-25 23:12:50 +0000 | |||
137 | @@ -100,6 +100,8 @@ | |||
138 | 100 | void for_each_display_buffer(std::function<void(graphics::DisplayBuffer&)> const&) override; | 100 | void for_each_display_buffer(std::function<void(graphics::DisplayBuffer&)> const&) override; |
139 | 101 | void post() override; | 101 | void post() override; |
140 | 102 | std::chrono::milliseconds recommended_sleep() const override; | 102 | std::chrono::milliseconds recommended_sleep() const override; |
141 | 103 | |||
142 | 104 | geometry::Rectangle view_area() const; | ||
143 | 103 | private: | 105 | private: |
144 | 104 | std::shared_ptr<detail::DisplayBuffer> const output; | 106 | std::shared_ptr<detail::DisplayBuffer> const output; |
145 | 105 | }; | 107 | }; |
146 | 106 | 108 | ||
147 | === modified file 'tests/acceptance-tests/test_nested_mir.cpp' | |||
148 | --- tests/acceptance-tests/test_nested_mir.cpp 2015-11-20 10:46:36 +0000 | |||
149 | +++ tests/acceptance-tests/test_nested_mir.cpp 2015-11-25 23:12:50 +0000 | |||
150 | @@ -111,7 +111,7 @@ | |||
151 | 111 | std::vector<geom::Rectangle> const display_geometry | 111 | std::vector<geom::Rectangle> const display_geometry |
152 | 112 | { | 112 | { |
153 | 113 | {{ 0, 0}, { 640, 480}}, | 113 | {{ 0, 0}, { 640, 480}}, |
155 | 114 | {{480, 0}, {1920, 1080}} | 114 | {{640, 0}, {1920, 1080}} |
156 | 115 | }; | 115 | }; |
157 | 116 | 116 | ||
158 | 117 | std::chrono::seconds const timeout{10}; | 117 | std::chrono::seconds const timeout{10}; |
159 | @@ -410,12 +410,13 @@ | |||
160 | 410 | // would be included in one of the later counts and cause a test failure. | 410 | // would be included in one of the later counts and cause a test failure. |
161 | 411 | Mock::VerifyAndClearExpectations(mock_session_mediator_report.get()); | 411 | Mock::VerifyAndClearExpectations(mock_session_mediator_report.get()); |
162 | 412 | 412 | ||
164 | 413 | // One post when surface drawn | 413 | // One post on each output when surface drawn |
165 | 414 | { | 414 | { |
166 | 415 | mt::WaitCondition wait; | 415 | mt::WaitCondition wait; |
167 | 416 | 416 | ||
170 | 417 | EXPECT_CALL(*mock_session_mediator_report, session_submit_buffer_called(_)).Times(1) | 417 | EXPECT_CALL(*mock_session_mediator_report, session_submit_buffer_called(_)).Times(2) |
171 | 418 | .WillOnce(InvokeWithoutArgs([&] { wait.wake_up_everyone(); })); | 418 | .WillOnce(InvokeWithoutArgs([]{})) |
172 | 419 | .WillOnce(InvokeWithoutArgs([&] { wait.wake_up_everyone(); })); | ||
173 | 419 | 420 | ||
174 | 420 | mir_buffer_stream_swap_buffers_sync(mir_surface_get_buffer_stream(surface)); | 421 | mir_buffer_stream_swap_buffers_sync(mir_surface_get_buffer_stream(surface)); |
175 | 421 | 422 | ||
176 | @@ -423,12 +424,13 @@ | |||
177 | 423 | Mock::VerifyAndClearExpectations(mock_session_mediator_report.get()); | 424 | Mock::VerifyAndClearExpectations(mock_session_mediator_report.get()); |
178 | 424 | } | 425 | } |
179 | 425 | 426 | ||
181 | 426 | // One post when surface released | 427 | // One post on each output when surface released |
182 | 427 | { | 428 | { |
183 | 428 | mt::WaitCondition wait; | 429 | mt::WaitCondition wait; |
184 | 429 | 430 | ||
187 | 430 | EXPECT_CALL(*mock_session_mediator_report, session_submit_buffer_called(_)).Times(1) | 431 | EXPECT_CALL(*mock_session_mediator_report, session_submit_buffer_called(_)).Times(2) |
188 | 431 | .WillOnce(InvokeWithoutArgs([&] { wait.wake_up_everyone(); })); | 432 | .WillOnce(InvokeWithoutArgs([]{})) |
189 | 433 | .WillOnce(InvokeWithoutArgs([&] { wait.wake_up_everyone(); })); | ||
190 | 432 | 434 | ||
191 | 433 | mir_surface_release_sync(surface); | 435 | mir_surface_release_sync(surface); |
192 | 434 | mir_connection_release(connection); | 436 | mir_connection_release(connection); |
FAILED: Continuous integration, rev:3106 jenkins. qa.ubuntu. com/job/ mir-ci/ 5567/ jenkins. qa.ubuntu. com/job/ mir-android- vivid-i386- build/4895 jenkins. qa.ubuntu. com/job/ mir-clang- vivid-amd64- build/3802 jenkins. qa.ubuntu. com/job/ mir-mediumtests -vivid- touch/4838/ console jenkins. qa.ubuntu. com/job/ mir-mediumtests -wily-touch/ 795/console jenkins. qa.ubuntu. com/job/ mir-wily- amd64-ci/ 1721 jenkins. qa.ubuntu. com/job/ mir-wily- amd64-ci/ 1721/artifact/ work/output/ *zip*/output. zip jenkins. qa.ubuntu. com/job/ mir-wily- i386-ci/ 795 jenkins. qa.ubuntu. com/job/ mir-wily- i386-ci/ 795/artifact/ work/output/ *zip*/output. zip jenkins. qa.ubuntu. com/job/ mir-mediumtests -builder- vivid-armhf/ 4839 jenkins. qa.ubuntu. com/job/ mir-mediumtests -builder- vivid-armhf/ 4839/artifact/ work/output/ *zip*/output. zip jenkins. qa.ubuntu. com/job/ mir-mediumtests -runner- touch/7416/ console s-jenkins. ubuntu- ci:8080/ job/touch- flash-device/ 25178 jenkins. qa.ubuntu. com/job/ mir-mediumtests -builder- wily-armhf/ 796/console
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
FAILURE: http://
FAILURE: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
SUCCESS: http://
deb: http://
FAILURE: http://
SUCCESS: http://
FAILURE: http://
Click here to trigger a rebuild: s-jenkins. ubuntu- ci:8080/ job/mir- ci/5567/ rebuild
http://