Merge lp:~dandrader/qtmir/promptBeforeSurfaceDraws into lp:qtmir

Proposed by Daniel d'Andrada
Status: Merged
Approved by: Nick Dedekind
Approved revision: 486
Merged at revision: 485
Proposed branch: lp:~dandrader/qtmir/promptBeforeSurfaceDraws
Merge into: lp:qtmir
Diff against target: 138 lines (+78/-2)
5 files modified
src/modules/Unity/Application/mirsurfacelistmodel.cpp (+5/-0)
src/modules/Unity/Application/mirsurfacelistmodel.h (+1/-0)
src/modules/Unity/Application/session.cpp (+23/-2)
src/modules/Unity/Application/session.h (+6/-0)
tests/modules/SessionManager/session_test.cpp (+43/-0)
To merge this branch: bzr merge lp:~dandrader/qtmir/promptBeforeSurfaceDraws
Reviewer Review Type Date Requested Status
Nick Dedekind (community) Approve
Unity8 CI Bot (community) continuous-integration Approve
Review via email: mp+294297@code.launchpad.net

Commit message

Session: Add a blank surface to the public list if it already has child prompt surfaces

A prompt session may come in too early, before its parent session got a chance
to draw its surface's first frame.

Normally that wouldn't be a problem and we could withhold that parent surface until
it's finally drawn to. But unfortunately the application process might be blocked,
unable to draw anything, until its child prompt session gets dismissed.

Because of that we have no option but to expose this blank surface to shell so that
it can display it along with the prompt surface on top of it, so that the user can
interact with it right away and finally unblock the application.

Description of the change

* Are there any related MPs required for this MP to build/function as expected? Please list.
No.

* Did you perform an exploratory manual test run of your code change and any related functionality?
Yes.

* If you changed the packaging (debian), did you subscribe the ubuntu-unity team to this MP?
Not applicable.

To post a comment you must log in.
Revision history for this message
Unity8 CI Bot (unity8-ci-bot) wrote :
review: Needs Fixing (continuous-integration)
486. By Daniel d'Andrada

Fix MirSurfaceListModel clean up

Revision history for this message
Unity8 CI Bot (unity8-ci-bot) wrote :

PASSED: Continuous integration, rev:486
https://unity8-jenkins.ubuntu.com/job/lp-qtmir-ci/212/
Executed test runs:
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-0-fetch/1575
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/1532
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial+overlay/1532
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=amd64,release=vivid+overlay/1532
        deb: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=amd64,release=vivid+overlay/1532/artifact/output/*zip*/output.zip
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1532
        deb: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=amd64,release=xenial+overlay/1532/artifact/output/*zip*/output.zip
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=armhf,release=vivid+overlay/1532
        deb: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=armhf,release=vivid+overlay/1532/artifact/output/*zip*/output.zip
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1532
        deb: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=armhf,release=xenial+overlay/1532/artifact/output/*zip*/output.zip
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=i386,release=vivid+overlay/1532
        deb: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=i386,release=vivid+overlay/1532/artifact/output/*zip*/output.zip
    SUCCESS: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=i386,release=xenial+overlay/1532
        deb: https://unity8-jenkins.ubuntu.com/job/build-2-binpkg/arch=i386,release=xenial+overlay/1532/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://unity8-jenkins.ubuntu.com/job/lp-qtmir-ci/212/rebuild

review: Approve (continuous-integration)
Revision history for this message
Nick Dedekind (nick-dedekind) wrote :

This seems to be a cheeky hack around the fact that the surface hasn't had it's first frame drawn and is therefore not "visible". After hacking in the fact that at the moment the prompts are a child of a session, not a surface.

What happens if the surface hadn't been drawn, we create the prompt and append to the blank surface and then suddenly the surface is drawn. The prompt will still be childed to the blank surface which I think is wrong.

I think the fact that the surface "isn't ready" to be shown should be dealt with in unity8. Maybe use the MirSurface::visible property?

bool MirSurface::visible() {
    return m_firstFrameDrawn &&
        m_surface->query(mir_surface_attrib_visibility) == mir_surface_visibility_exposed;
}

SurfaceContainer {
    MirSurfaceItem {
        visible: surface.visible
    }
}

+ add in checks for visibility in animations.

IMO it's all just a bit wrong at the moment.

review: Needs Fixing
Revision history for this message
Daniel d'Andrada (dandrader) wrote :

On 11/05/2016 06:17, Nick Dedekind wrote:
> The prompt will still be childed to the blank surface which I think is wrong.

So you think a prompt surface should be treated as a top-level
application surface? Or is it the problem just that the parent surface
is still blank?

Revision history for this message
Daniel d'Andrada (dandrader) wrote :

On 11/05/2016 06:17, Nick Dedekind wrote:
> bool MirSurface::visible() {
> return m_firstFrameDrawn &&
> m_surface->query(mir_surface_attrib_visibility) == mir_surface_visibility_exposed;
> }
>
> SurfaceContainer {
> MirSurfaceItem {
> visible: surface.visible
> }
> }
MirSurfaceItem already sets surface visibility based on its own visible
property. And here you make the latter also set the former. Like A sets
B and B sets A, a cyclic data-flow/control.

Besides MirSurface::visible is already used to convey surface
exposure/occlusion information. Here you want to overload that property
to include another meaning. I don't think the two combine well.

Revision history for this message
Gerry Boland (gerboland) wrote :

I suspect we should abandon the "only notify qml of new surface after its first frame" thing - and instead have a flag on the MirSurface{,Item} saying "notDrawnToYet" - instead of this blank surface idea.

Like Nick, I'm not a big fan of the blank surface idea either, but I see why you did it. It is the least complex change to fix the issue.

But the "only notify qml of new surface after its first frame" policy is not a Mir policy, just a QtMir policy, and it seems to hurt us here.

Dropping that policy causes us new problems: how does QML draw a MirSurfaceItem which is backed by a mirSurface which was never drawn to. I suppose we do as you do here: make it blank. unity8 will need work to reflect QtMir's behaviour change too.

This what I envision as the proper fix is a much more involved change, not so easily done if it needs to land quickly. We might need to land this (with a FIXME), but we should investigate a proper fix afterwards.

Revision history for this message
Nick Dedekind (nick-dedekind) wrote :

> On 11/05/2016 06:17, Nick Dedekind wrote:
> > The prompt will still be childed to the blank surface which I think is
> wrong.
>
> So you think a prompt surface should be treated as a top-level
> application surface? Or is it the problem just that the parent surface
> is still blank?

The Mir implementation of prompts has not got surfaces in mind, they're properties of a session. We've just forced it to be a child the first surface.
I'm not saying the current mir impl is correct; and we'll have trouble as soon as 2 top level surfaces require prompts at the same time...

Sorry, I misunderstood the code, but it seems that you might be prepending the surface twice. Once if you add a prompt when it's blank, and again when it's drawn it's first frame.

Revision history for this message
Nick Dedekind (nick-dedekind) wrote :

> On 11/05/2016 06:17, Nick Dedekind wrote:
> > bool MirSurface::visible() {
> > return m_firstFrameDrawn &&
> > m_surface->query(mir_surface_attrib_visibility) ==
> mir_surface_visibility_exposed;
> > }
> >
> > SurfaceContainer {
> > MirSurfaceItem {
> > visible: surface.visible
> > }
> > }
> MirSurfaceItem already sets surface visibility based on its own visible
> property. And here you make the latter also set the former. Like A sets
> B and B sets A, a cyclic data-flow/control.

Sure. opacity, occluded by spinner, or whatever to not show the actual surface.

>
> Besides MirSurface::visible is already used to convey surface
> exposure/occlusion information. Here you want to overload that property
> to include another meaning. I don't think the two combine well.

Well, it's up to us what a exposed property means. Whether it's "occluded" or "should be visible". A separate property would be better but require API.

Revision history for this message
Daniel d'Andrada (dandrader) wrote :

Will start working on a overhauling of surface.drawnFirstFrame usage and policy.
If people get anxious they can ship this fix/work-around before the afore-mentioned overhauling is ready.

Revision history for this message
Daniel d'Andrada (dandrader) wrote :

On 11/05/2016 09:54, Nick Dedekind wrote:
> Sorry, I misunderstood the code, but it seems that you might be prepending the surface twice. Once if you add a prompt when it's blank, and again when it's drawn it's first frame.
prompt session surfaces are added to its parent session's
promptSurfaceList only in Session::addChildSession. And that's called
only once per prompt session.

Once the once-blank has drawn its first frame it's moved to the
Session's surface list. But that move doesn't affect the surfaces own
promptSurfaceList.

Revision history for this message
Nick Dedekind (nick-dedekind) wrote :

ok, looks good. (for now ;)

review: Approve
Revision history for this message
Daniel d'Andrada (dandrader) wrote :

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'src/modules/Unity/Application/mirsurfacelistmodel.cpp'
--- src/modules/Unity/Application/mirsurfacelistmodel.cpp 2016-04-05 18:58:38 +0000
+++ src/modules/Unity/Application/mirsurfacelistmodel.cpp 2016-05-10 21:06:43 +0000
@@ -28,6 +28,11 @@
28{28{
29}29}
3030
31MirSurfaceListModel::~MirSurfaceListModel()
32{
33 Q_EMIT destroyed(this); // Early warning, while MirSurfaceListModel methods can still be accessed.
34}
35
31int MirSurfaceListModel::rowCount(const QModelIndex &parent) const36int MirSurfaceListModel::rowCount(const QModelIndex &parent) const
32{37{
33 return !parent.isValid() ? m_surfaceList.size() : 0;38 return !parent.isValid() ? m_surfaceList.size() : 0;
3439
=== modified file 'src/modules/Unity/Application/mirsurfacelistmodel.h'
--- src/modules/Unity/Application/mirsurfacelistmodel.h 2016-04-05 18:58:38 +0000
+++ src/modules/Unity/Application/mirsurfacelistmodel.h 2016-05-10 21:06:43 +0000
@@ -41,6 +41,7 @@
41 Q_OBJECT41 Q_OBJECT
42public:42public:
43 explicit MirSurfaceListModel(QObject *parent = 0);43 explicit MirSurfaceListModel(QObject *parent = 0);
44 virtual ~MirSurfaceListModel();
4445
45 Q_INVOKABLE unity::shell::application::MirSurfaceInterface *get(int index) override;46 Q_INVOKABLE unity::shell::application::MirSurfaceInterface *get(int index) override;
46 const unity::shell::application::MirSurfaceInterface *get(int index) const;47 const unity::shell::application::MirSurfaceInterface *get(int index) const;
4748
=== modified file 'src/modules/Unity/Application/session.cpp'
--- src/modules/Unity/Application/session.cpp 2016-04-06 16:52:49 +0000
+++ src/modules/Unity/Application/session.cpp 2016-05-10 21:06:43 +0000
@@ -199,8 +199,17 @@
199 if (newSurface->isFirstFrameDrawn()) {199 if (newSurface->isFirstFrameDrawn()) {
200 prependSurface(newSurface);200 prependSurface(newSurface);
201 } else {201 } else {
202 connect(newSurface, &MirSurfaceInterface::firstFrameDrawn,202 m_blankSurfaces.append(newSurface);
203 this, [this, newSurface]() { this->prependSurface(newSurface); });203 connect(newSurface, &QObject::destroyed, this, [this, newSurface]()
204 {
205 this->m_blankSurfaces.removeAll(newSurface);
206 });
207 connect(newSurface, &MirSurfaceInterface::firstFrameDrawn, this, [this, newSurface]()
208 {
209 this->m_blankSurfaces.removeAll(newSurface);
210 newSurface->disconnect(this);
211 this->prependSurface(newSurface);
212 });
204 }213 }
205}214}
206215
@@ -389,6 +398,18 @@
389 // we assume that the top-most surface is the one that caused the prompt session to show up.398 // we assume that the top-most surface is the one that caused the prompt session to show up.
390 auto promptSurfaceList = static_cast<MirSurfaceListModel*>(m_surfaceList.get(0)->promptSurfaceList());399 auto promptSurfaceList = static_cast<MirSurfaceListModel*>(m_surfaceList.get(0)->promptSurfaceList());
391 promptSurfaceList->addSurfaceList(session->surfaceList());400 promptSurfaceList->addSurfaceList(session->surfaceList());
401 } else if (m_blankSurfaces.count() > 0) {
402 // Prompt session came in too early.
403 // Parent session might be blocked until the prompt session is dismissed.
404 // So we cannot wait anymore and must put that blank surface on the surface list so that the
405 // user can see the prompt surface on top of that it and interact.
406 auto blankSurface = m_blankSurfaces.takeFirst();
407
408 auto promptSurfaceList = static_cast<MirSurfaceListModel*>(blankSurface->promptSurfaceList());
409 promptSurfaceList->addSurfaceList(session->surfaceList());
410
411 blankSurface->disconnect(this);
412 prependSurface(blankSurface);
392 }413 }
393}414}
394415
395416
=== modified file 'src/modules/Unity/Application/session.h'
--- src/modules/Unity/Application/session.h 2016-03-28 18:02:26 +0000
+++ src/modules/Unity/Application/session.h 2016-05-10 21:06:43 +0000
@@ -107,6 +107,12 @@
107 std::shared_ptr<mir::scene::Session> m_session;107 std::shared_ptr<mir::scene::Session> m_session;
108 Application* m_application;108 Application* m_application;
109 MirSurfaceListModel m_surfaceList;109 MirSurfaceListModel m_surfaceList;
110
111 // Registered surfaces that haven't yet drawn their first frames.
112 // Once they do, they get moved to m_surfaceList.
113 // So this list is just a temporary holder and most of the time it's gonna be empty.
114 QList<MirSurfaceInterface*> m_blankSurfaces;
115
110 SessionModel* m_children;116 SessionModel* m_children;
111 bool m_fullscreen;117 bool m_fullscreen;
112 State m_state;118 State m_state;
113119
=== modified file 'tests/modules/SessionManager/session_test.cpp'
--- tests/modules/SessionManager/session_test.cpp 2016-03-28 18:02:26 +0000
+++ tests/modules/SessionManager/session_test.cpp 2016-05-10 21:06:43 +0000
@@ -311,3 +311,46 @@
311 EXPECT_EQ(Session::Stopped, session->state());311 EXPECT_EQ(Session::Stopped, session->state());
312 EXPECT_FALSE(session->m_suspendTimer->isRunning());312 EXPECT_FALSE(session->m_suspendTimer->isRunning());
313}313}
314
315/*
316 Regression test for https://bugs.launchpad.net/ubuntu/+source/qtmir/+bug/1578665
317
318 A child prompt session was arriving before the application surface got a chance to draw
319 its first frame. And on top of that, the application was blocked from drawing anything
320 until that prompt session gone away.
321 */
322TEST_F(SessionTests, ChildSessionArrivesBeforeSurfaceDrawnFirstFrame)
323{
324 using namespace testing;
325
326 const QString appId("test-app");
327 const pid_t procId = 5551;
328
329 auto mirSession = std::make_shared<MockSession>(appId.toStdString(), procId);
330
331 auto session = std::make_shared<qtmir::Session>(mirSession, promptSessionManager);
332
333 FakeMirSurface *blankSurface = new FakeMirSurface;
334 session->registerSurface(blankSurface);
335
336 // Surface doesn't appear in this list as it has not drawn its first frame yet
337 EXPECT_EQ(0, session->surfaceList()->count());
338
339 Session* childSession = new Session(mirSession, promptSessionManager);
340 session->addChildSession(childSession);
341
342 FakeMirSurface *promptSurface = new FakeMirSurface;
343 childSession->registerSurface(promptSurface);
344 promptSurface->drawFirstFrame();
345
346 // Even though blankSurface still hasn't drawn its first frame (ie, it's still blank),
347 // it should now appear in this list as it now has a child prompt surface, which the
348 // user must interact with.
349 auto sessionSurfaceList = static_cast<MirSurfaceListModel*>(session->surfaceList());
350 EXPECT_EQ(1, sessionSurfaceList->count());
351 EXPECT_TRUE(sessionSurfaceList->contains(blankSurface));
352
353 auto promptSurfaceList = static_cast<MirSurfaceListModel*>(blankSurface->promptSurfaceList());
354 EXPECT_EQ(1, promptSurfaceList->count());
355 EXPECT_TRUE(promptSurfaceList->contains(promptSurface));
356}

Subscribers

People subscribed via source and target branches