Merge lp:~vanvugt/compiz/fix-731685-trunk into lp:~compiz/compiz/ubuntu

[Daniel van Vugt]

Fix dock shadow artifacts with rounded window decorations. (LP: #731685)

Removes the assumption that all window decorations are rectangular when overlapping the shadow of a dock like gnome-panel. This false assumption was added to compiz 0.9.4 apparently for performance reasons, however it causes visible artifacts where the shadow should be visible.

The patch contained in this proposal was developed and tested on natty, however it appears to be compatible with oneiric also (thankfully).

Fixing bug 731685 is particularly useful because it opens the door for us to be able to eliminate and simplify the redundant shadowing code in unity (panel). This should allow us to fix other outstanding unity shadow bugs quite easily.

[Sam Spilsbury (smspillaz)]

Indeed this assumption (window is not transformed) is in place.

However, I'd prefer not to render shadows on top of the desktop window. I think that there's a better way to handle this and ensure that we're not rendering the decoration twice. It might be worth clipping the shadow based on the frame region rather than the border rect itself.

[Daniel van Vugt (vanvugt)]

The code is not fresh in my mind, however I think you will find that:

* This fix is much smaller, simpler to maintain and probably faster than the clipping/stencilling you talk about.
* This fix potentially supports smoothed window decorations, but clipping/stencilling would not.
* This fix works right now. No other working fix exists AFAIK :)

[Daniel van Vugt (vanvugt)]

Also, I don't think this proposal is "rendering the decoration twice".

It avoids rendering all dock shadows until it draws the desktop window. Only then does it render the dock shadow for the first time. You might have to apply the patch and look at the resulting DecorWindow::glDraw() to see why.

[Sam Spilsbury (smspillaz)]

On Thu, Sep 1, 2011 at 12:09 PM, Daniel van Vugt <email address hidden> wrote:
> The code is not fresh in my mind, however I think you will find that:
>
> * This fix is much smaller, simpler to maintain and probably faster than the clipping/stencilling you talk about.
> * This fix potentially supports smoothed window decorations, but clipping/stencilling would not.
> * This fix works right now. No other working fix exists AFAIK :)

Yeah, I agree with all this. In that case, I've changed my mind about
this and will be happy to take it, but I'll note here that decorations
really ought to be moved into their own independently paintable
objects in the paint stack, rather than hooking the paint function for
the desktop window (the primary problem with this approach is that a
plugin can inhibit the painting of a desktop window and then the panel
will have no shadow, which doesn't make any sense to me). However,
since we don't have the infrastructure in core to do that yet, we'll
have to do it this way for now.

>
> --
> https://code.launchpad.net/~vanvugt/compiz/fix-731685-trunk/+merge/70836
> You are requested to review the proposed merge of lp:~vanvugt/compiz/fix-731685-trunk into lp:compiz.
>

--
Sam Spilsbury

[Daniel van Vugt (vanvugt)]

Trying to resubmit the proposal
from: ~vanvugt/compiz-core/fix-731685
to: lp:compiz-core

But Launchpad is just timing out every time. Please wait :(

Unmerged revisions

639. By Daniel van Vugt on 2011-08-09

Fix dock shadow artifacts with rounded window decorations. (LP: #731685)

638. By Sam Spilsbury on 2011-08-01

Remove symbols that we're not shipping anymore

637. By Didier Roche-Tolomelli on 2011-08-01

* New upstream release
* debian/compiz-gnome.install:
  - install gsettings schemas

=== added file 'debian/patches/fix_dock_shadow_artifacts.patch'
--- debian/patches/fix_dock_shadow_artifacts.patch	1970-01-01 00:00:00 +0000
+++ debian/patches/fix_dock_shadow_artifacts.patch	2011-08-09 09:33:27 +0000
@@ -0,0 +1,116 @@
+Description: Fix dock shadow artifacts with rounded window decorations.
+ Removes the assumption that all window decorations are rectangular when
+ overlapping the shadow of a dock like gnome-panel. This false assumption
+ was added to compiz 0.9.4 apparently for performance reasons, however it
+ causes visible artifacts where the shadow should be visible.
+Bug-Ubuntu: https://launchpad.net/bugs/731685
+Author: Daniel van Vugt <[e-mail]>
+Forwarded: no
+
+=== modified file 'plugins/decor/src/decor.cpp'
+--- old/plugins/decor/src/decor.cpp	2011-06-01 10:23:26 +0000
++++ new/plugins/decor/src/decor.cpp	2011-08-09 08:59:21 +0000
+@@ -79,34 +79,8 @@
+ {
+     shadowRegion = CompRegion (window->outputRect ());
+ 
+-    if (window->type () == CompWindowTypeDockMask)
+-    {
+-        /* windows above this one in the stack should
+-         * clip the shadow */
+-
+-        CompWindowList::iterator it = std::find (screen->windows ().begin (),
+-                                                 screen->windows ().end (),
+-                                                 window);
+-
+-        for (it--; it != screen->windows ().end (); it--)
+-        {
+-            CompRegion inter;
+-
+-            if (!(*it)->isViewable ())
+-                continue;
+-
+-            if ((*it)->type () & CompWindowTypeDesktopMask)
+-                continue;
+-
+-	    inter = shadowRegion.intersected ((*it)->borderRect ());
+-
+-            if (!inter.isEmpty ())
+-		shadowRegion = shadowRegion.subtracted (inter);
+-
+-        }
+-    }
+-    else if (window->type () == CompWindowTypeDropdownMenuMask ||
+-             window->type () == CompWindowTypePopupMenuMask)
++    if (window->type () == CompWindowTypeDropdownMenuMask ||
++        window->type () == CompWindowTypePopupMenuMask)
+     {
+         /* Other transient menus should clip
+          * this menu's shadows, also the panel
+@@ -175,6 +149,34 @@
+ 
+     status = gWindow->glDraw (transform, attrib, region, mask);
+ 
++    /* Don't render dock decorations (shadows) on just any old window */
++    if (!(window->type () & CompWindowTypeDockMask))
++    {
++	glDecorate (transform, attrib, region, mask);
++	/* Render dock decorations (shadows) on desktop windows only */
++	if (window->type () & CompWindowTypeDesktopMask)
++	{
++	    foreach (CompWindow *w, dScreen->cScreen->getWindowPaintList ())
++	    {
++		if ((w->type () & CompWindowTypeDockMask) &&
++		    !(w->destroyed () || w->invisible ()))
++		{
++		    DecorWindow *d = DecorWindow::get (w);
++		    d->glDecorate (transform, attrib, region, mask);
++		}
++	    }
++	}
++    }
++
++    return status;
++}
++
++void
++DecorWindow::glDecorate (const GLMatrix     &transform,
++		         GLFragment::Attrib &attrib,
++		         const CompRegion   &region,
++		         unsigned int       mask)
++{
+     const CompRegion reg = (mask & PAINT_WINDOW_TRANSFORMED_MASK) ?
+ 			   infiniteRegion : shadowRegion.intersected (region);
+ 
+@@ -213,7 +215,7 @@
+ 	if (gWindow->textures ().empty ())
+ 	    gWindow->bind ();
+ 	if (gWindow->textures ().empty ())
+-	    return status;
++	    return;
+ 
+ 	if (gWindow->textures ().size () == 1)
+ 	{
+@@ -240,8 +242,6 @@
+ 	    }
+ 	}
+     }
+-
+-    return status;
+ }
+ 
+ static bool bindFailed;
+
+=== modified file 'plugins/decor/src/decor.h'
+--- old/plugins/decor/src/decor.h	2011-02-24 17:31:29 +0000
++++ new/plugins/decor/src/decor.h	2011-08-09 08:59:21 +0000
+@@ -182,6 +182,8 @@
+ 
+ 	bool glDraw (const GLMatrix &, GLFragment::Attrib &,
+ 		     const CompRegion &, unsigned int);
++	void glDecorate (const GLMatrix &, GLFragment::Attrib &,
++		         const CompRegion &, unsigned int);
+ 
+ 	void windowNotify (CompWindowNotify n);
+ 
+

=== modified file 'debian/patches/series'
--- debian/patches/series	2011-07-19 05:18:01 +0000
+++ debian/patches/series	2011-08-09 09:33:27 +0000
@@ -1,1 +1,2 @@
 01_don_t_init_a11y.patch
+fix_dock_shadow_artifacts.patch