Mir

Merge lp:~alan-griffiths/mir/fix-1516670 into lp:mir

Proposed by Alan Griffiths
Status: Merged
Approved by: Alan Griffiths
Approved revision: no longer in the source branch.
Merged at revision: 3131
Proposed branch: lp:~alan-griffiths/mir/fix-1516670
Merge into: lp:mir
Diff against target: 192 lines (+78/-37)
3 files modified
src/server/graphics/nested/display.cpp (+67/-30)
src/server/graphics/nested/display.h (+2/-0)
tests/acceptance-tests/test_nested_mir.cpp (+9/-7)
To merge this branch: bzr merge lp:~alan-griffiths/mir/fix-1516670
Reviewer Review Type Date Requested Status
PS Jenkins bot (community) continuous-integration Approve
Alexandros Frantzis (community) Needs Information
Andreas Pokorny (community) Approve
Alberto Aguirre Pending
Review via email: mp+278184@code.launchpad.net

This proposal supersedes a proposal from 2015-11-17.

Commit message

nested: For clone mode use a fullscreen surface for the output with the largest intersection with the bounding rectangle

Description of the change

nested: For clone mode use a fullscreen surface for the output with the largest intersection with the bounding rectangle

The previous implementation attempted to cover the bounding rectangle of all outputs with a surface that was fullscreen on the first - which led to inconsistent states.

The surface creation logic has also been reworked to reuse existing surfaces where possible, which makes the server interaction less racy (I suspect there are some races still, but this makes plugging and replugging a lot more stable on the phone).

I also added some information to the surface name (which can be helpful when debugging window management in the host server).

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote : Posted in a previous version of this proposal

Having a stack of overlapping fullscreen surfaces in the host server introduces some visual artifacts (and inefficiencies).

So this is clearly not the ideal solution - just less broken than what we currently have.

There are a number of approaches for a better solution:

1. use non-fullscreen surfaces in nested mode and either:
1.1. display only the intersection of all outputs
1.2. display the bounding rectangle of all outputs

2. use only fullscreen surfaces and either:
2.1. select the "smallest" display
2.2. select the "largest" display

I'm not yet convinced by any of the above options, but favour of sticking with "fullscreen only" when nesting.

Revision history for this message
Alberto Aguirre (albaguirre) wrote : Posted in a previous version of this proposal

Are the visual artifacts on phone/desktop/both?

And what are the cases where outputs overlap?

We could also potentially run into max texture size issues for high res displays if we attempt to do 1.2.

In any case, a fullscreen surface per output makes sense to me.

review: Needs Information
Revision history for this message
Alan Griffiths (alan-griffiths) wrote : Posted in a previous version of this proposal

> Are the visual artifacts on phone/desktop/both?

Both, but easier to see on phone.

> And what are the cases where outputs overlap?

When displays of different sizes are "cloned".

So, with an external monitor attached to a mako:

$ sudo stop lightdm
$ bin/mir_demo_server --window-manager system-compositor&
$ bin/mir_demo_standalone_render_surfaces --no-file --host $XDG_RUNTIME_DIR/mir_socket --display-config clone

On the external monitor the fullscreen for the phone output is overlayed[1] on top of the fullscreen for the external monitor and the edge is visible as the flying surfaces it (as the rendering is not quite in sync.)

> We could also potentially run into max texture size issues for high res
> displays if we attempt to do 1.2.

Yep.

> In any case, a fullscreen surface per output makes sense to me.

[1] I'm not sure if this ordering is guaranteed, but was consistent in my limited testing.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Andreas Pokorny (andreas-pokorny) wrote :

hm I think this is an improvement but in clone mode we should use the smallest screen size for each surface and have the output scale.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
Andreas Pokorny (andreas-pokorny) wrote :

Ok overall after looking and testing (on kms).. our default display configuration for cloned seems to be wrong. It should figure out a suitable intersection and try to select suitable modes on each output, instead of just aligning top_left. If that would be in place it would not matter if we pick the maximum overlap or intersection. (The former might be a better solution then).

review: Approve
Revision history for this message
Alexandros Frantzis (afrantzis) wrote :

Note that following scenario is still not working for me (but I am not sure if this MP was supposed to fix this):

Internal monitor 1280x800
External monitor 1920x1080

In different shells:
$ sudo bin/mir_demo_server --arw-file -f /tmp/mir_host
$ bin/mir_demo_server --host-socket /tmp/mir_host -f /tmp/mir_nested --vt 1
$ bin/mir_demo_client_display_config -m /tmp/mir_nested

Should start in clone mode, press 'h', press 'c' again. After last 'c' we should expect to go back to the original setup but we get into a strange setup instead. There seem to be two "framebuffer" surfaces, the first sized for the internal screen and the second for the external. On the external screen the two FB surfaces overlap, with the internal one being in front and hiding the top left part of the external. The client surface is only drawn on the external FB surface.

Also note that both the old and the new code don't handle properly cases where one output doesn't fully contain the other (e.g. 100x100+0+0 100x100+50+50). It's not a common case but we should (eventually) either support it or explicitly prohibit it.

> area.intersection_with(output.extents()))

Since area is the bounding rectangle of all outputs, isn't this expression equivalent to just output.extents()?

@Andreas
> hm I think this is an improvement but in clone mode we should use the smallest screen size
> for each surface and have the output scale.
...
> It should figure out a suitable intersection and try to select suitable modes on each output,
> instead of just aligning top_left.

Fair enough, we should probably rename the current clone policy to '(max)overlap' and implement a new 'clone' one. In any case there are both valid policies and should be properly supported. Plus there are cases where we can't 'clone' because there aren't any matching modes, so we need to fall back to 'overlap' anyway.

review: Needs Information
Revision history for this message
PS Jenkins bot (ps-jenkins) :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'src/server/graphics/nested/display.cpp'
--- src/server/graphics/nested/display.cpp 2015-11-18 12:38:30 +0000
+++ src/server/graphics/nested/display.cpp 2015-11-25 23:12:50 +0000
@@ -34,6 +34,7 @@
3434
35#include <boost/throw_exception.hpp>35#include <boost/throw_exception.hpp>
36#include <stdexcept>36#include <stdexcept>
37#include <sstream>
3738
38namespace mi = mir::input;39namespace mi = mir::input;
39namespace mg = mir::graphics;40namespace mg = mir::graphics;
@@ -147,6 +148,11 @@
147 return std::chrono::milliseconds::zero();148 return std::chrono::milliseconds::zero();
148}149}
149150
151geom::Rectangle mgn::detail::DisplaySyncGroup::view_area() const
152{
153 return output->view_area();
154}
155
150mgn::Display::Display(156mgn::Display::Display(
151 std::shared_ptr<mg::Platform> const& platform,157 std::shared_ptr<mg::Platform> const& platform,
152 std::shared_ptr<HostConnection> const& connection,158 std::shared_ptr<HostConnection> const& connection,
@@ -203,11 +209,16 @@
203void mgn::Display::configure(mg::DisplayConfiguration const& configuration)209void mgn::Display::configure(mg::DisplayConfiguration const& configuration)
204{210{
205 std::lock_guard<std::mutex> lock(configuration_mutex);211 std::lock_guard<std::mutex> lock(configuration_mutex);
206 if (*current_configuration != configuration)212 apply_to_connection(configuration);
207 {213 create_surfaces(configuration);
208 apply_to_connection(configuration);214}
209 create_surfaces(configuration);215
210 }216namespace
217{
218long area_of(geom::Rectangle const& rect)
219{
220 return static_cast<long>(rect.size.width.as_int())*rect.size.height.as_int();
221}
211}222}
212223
213void mgn::Display::create_surfaces(mg::DisplayConfiguration const& configuration)224void mgn::Display::create_surfaces(mg::DisplayConfiguration const& configuration)
@@ -223,35 +234,61 @@
223 unique_outputs.for_each_group(234 unique_outputs.for_each_group(
224 [&](mg::OverlappingOutputGroup const& group)235 [&](mg::OverlappingOutputGroup const& group)
225 {236 {
226 bool have_output_for_group = false;237 geometry::Rectangle const area = group.bounding_rectangle();
227 geometry::Rectangle const& area = group.bounding_rectangle();238
228 group.for_each_output([&](mg::DisplayConfigurationOutput output)239 long max_overlap_area = 0;
240 mg::DisplayConfigurationOutput best_output;
241
242 group.for_each_output([&](mg::DisplayConfigurationOutput const& output)
229 {243 {
230 if (!have_output_for_group)244 if (area_of(area.intersection_with(output.extents())) > max_overlap_area)
231 {245 {
232 auto const& egl_config_format = output.current_format;246 max_overlap_area = area_of(area.intersection_with(output.extents()));
233247 best_output = output;
234 complete_display_initialization(egl_config_format);
235
236 auto const host_surface = connection->create_surface(
237 area.size.width.as_int(),
238 area.size.height.as_int(),
239 egl_config_format,
240 "Mir nested display",
241 mir_buffer_usage_hardware,
242 static_cast<uint32_t>(output.id.as_value()));
243
244 result[output.id] = std::make_shared<mgn::detail::DisplaySyncGroup>(
245 std::make_shared<mgn::detail::DisplayBuffer>(
246 egl_display,
247 host_surface,
248 area,
249 dispatcher,
250 cursor_listener,
251 output.current_format));
252 have_output_for_group = true;
253 }248 }
254 });249 });
250
251 auto const& egl_config_format = best_output.current_format;
252 geometry::Rectangle const& extents = best_output.extents();
253
254 auto& display_buffer = result[best_output.id];
255
256 {
257 std::unique_lock<std::mutex> lock(outputs_mutex);
258 display_buffer = outputs[best_output.id];
259 }
260
261 if (display_buffer)
262 {
263 if (display_buffer->view_area() != extents)
264 display_buffer.reset();
265 }
266
267 if (!display_buffer)
268 {
269 complete_display_initialization(egl_config_format);
270
271 std::ostringstream surface_title;
272
273 surface_title << "Mir nested display for output #" << best_output.id.as_value();
274
275 auto const host_surface = connection->create_surface(
276 extents.size.width.as_int(),
277 extents.size.height.as_int(),
278 egl_config_format,
279 surface_title.str().c_str(),
280 mir_buffer_usage_hardware,
281 static_cast<uint32_t>(best_output.id.as_value()));
282
283 display_buffer = std::make_shared<mgn::detail::DisplaySyncGroup>(
284 std::make_shared<mgn::detail::DisplayBuffer>(
285 egl_display,
286 host_surface,
287 extents,
288 dispatcher,
289 cursor_listener,
290 best_output.current_format));
291 }
255 });292 });
256293
257 {294 {
258295
=== modified file 'src/server/graphics/nested/display.h'
--- src/server/graphics/nested/display.h 2015-10-29 02:15:05 +0000
+++ src/server/graphics/nested/display.h 2015-11-25 23:12:50 +0000
@@ -100,6 +100,8 @@
100 void for_each_display_buffer(std::function<void(graphics::DisplayBuffer&)> const&) override;100 void for_each_display_buffer(std::function<void(graphics::DisplayBuffer&)> const&) override;
101 void post() override;101 void post() override;
102 std::chrono::milliseconds recommended_sleep() const override;102 std::chrono::milliseconds recommended_sleep() const override;
103
104 geometry::Rectangle view_area() const;
103private:105private:
104 std::shared_ptr<detail::DisplayBuffer> const output;106 std::shared_ptr<detail::DisplayBuffer> const output;
105};107};
106108
=== modified file 'tests/acceptance-tests/test_nested_mir.cpp'
--- tests/acceptance-tests/test_nested_mir.cpp 2015-11-20 10:46:36 +0000
+++ tests/acceptance-tests/test_nested_mir.cpp 2015-11-25 23:12:50 +0000
@@ -111,7 +111,7 @@
111std::vector<geom::Rectangle> const display_geometry111std::vector<geom::Rectangle> const display_geometry
112{112{
113 {{ 0, 0}, { 640, 480}},113 {{ 0, 0}, { 640, 480}},
114 {{480, 0}, {1920, 1080}}114 {{640, 0}, {1920, 1080}}
115};115};
116116
117std::chrono::seconds const timeout{10};117std::chrono::seconds const timeout{10};
@@ -410,12 +410,13 @@
410 // would be included in one of the later counts and cause a test failure.410 // would be included in one of the later counts and cause a test failure.
411 Mock::VerifyAndClearExpectations(mock_session_mediator_report.get());411 Mock::VerifyAndClearExpectations(mock_session_mediator_report.get());
412412
413 // One post when surface drawn413 // One post on each output when surface drawn
414 {414 {
415 mt::WaitCondition wait;415 mt::WaitCondition wait;
416416
417 EXPECT_CALL(*mock_session_mediator_report, session_submit_buffer_called(_)).Times(1)417 EXPECT_CALL(*mock_session_mediator_report, session_submit_buffer_called(_)).Times(2)
418 .WillOnce(InvokeWithoutArgs([&] { wait.wake_up_everyone(); }));418 .WillOnce(InvokeWithoutArgs([]{}))
419 .WillOnce(InvokeWithoutArgs([&] { wait.wake_up_everyone(); }));
419420
420 mir_buffer_stream_swap_buffers_sync(mir_surface_get_buffer_stream(surface));421 mir_buffer_stream_swap_buffers_sync(mir_surface_get_buffer_stream(surface));
421422
@@ -423,12 +424,13 @@
423 Mock::VerifyAndClearExpectations(mock_session_mediator_report.get());424 Mock::VerifyAndClearExpectations(mock_session_mediator_report.get());
424 }425 }
425426
426 // One post when surface released427 // One post on each output when surface released
427 {428 {
428 mt::WaitCondition wait;429 mt::WaitCondition wait;
429430
430 EXPECT_CALL(*mock_session_mediator_report, session_submit_buffer_called(_)).Times(1)431 EXPECT_CALL(*mock_session_mediator_report, session_submit_buffer_called(_)).Times(2)
431 .WillOnce(InvokeWithoutArgs([&] { wait.wake_up_everyone(); }));432 .WillOnce(InvokeWithoutArgs([]{}))
433 .WillOnce(InvokeWithoutArgs([&] { wait.wake_up_everyone(); }));
432434
433 mir_surface_release_sync(surface);435 mir_surface_release_sync(surface);
434 mir_connection_release(connection);436 mir_connection_release(connection);

Subscribers

People subscribed via source and target branches