Duplicate core libraries linked into individual plugins

I think you misunderstand dynamic loading. There is only one address space -- that of the compiz process. All plugins share the same address space.

And if a dynamic library contains static data then that's perfectly safe because the dynamic loader always ensures only one copy of an SO ever gets loaded. It is just reference counted.

"perfectly safe" on Linux I mean. I have seen dynamic loader bugs on AIX and HP-UX in the past. But I digress.

It still seems like in this case, multiple copies of the static data are being pulled in, due to -Bsymbolic-functions. Reopening.

It sounds like someone has made a linkage mistake, probably including an object or a static library (archive) in multiple different targets which eventually get dynamically loaded into the same process.

Which data are being duplicated? It's probably easy to pinpoint the cause...

These look suspicious and will probably cause libplace.so to contain duplicate symbols from the archives already built into libcompiz_core.so ...

TARGET_LINK_LIBRARIES(
  compiz_place_constrain_to_workarea
  compiz_rect <-----
  compiz_point <-----
  compiz_window_geometry <-----
  compiz_window_geometry_saver <-----

  ${GLIBMM_LIBRARIES}
)

TARGET_LINK_LIBRARIES(
  compiz_place_screen_size_change
  compiz_place_constrain_to_workarea
  compiz_rect <-----
  compiz_point <-----
  compiz_window_geometry <-----
  compiz_window_geometry_saver <-----

  ${GLIBMM_LIBRARIES}
)

The above targets should refer to compiz_core, instead of compiz_rect, compiz_point, compiz_window_geometry or compiz_window_geometry_saver.

Obviously plugins should only ever link to compiz_core and their own private libraries. Otherwise you will create duplicate symbols within the plugin.

Correction:
Obviously plugins should only ever link to dynamic libraries and/or their own private static libraries. Otherwise you will create duplicate symbols within the plugin.

OK, I can now see CompPoint, CompTimer, CompRect classes etc being duplicated in the plugin binaries.

It looks like the cause is:

target_link_libraries (
    compiz_core

    ${COMPIZ_LIBRARIES}

    m
    pthread
    dl

    -Wl,-whole-archive
    compiz_string
    compiz_timer
    compiz_logmessage
    compiz_pluginclasshandler
    compiz_point
    compiz_rect
    compiz_window_geometry
    compiz_window_geometry_saver
    compiz_window_extents
    compiz_window_constrainment
    -Wl,-no-whole-archive
# ${CORE_MOD_LIBRARIES}
)

What we actually want is for compiz_* to be added to compiz_core as objects, not as dependent libraries that plugins then mistakenly link in. This is easy to do in regular Make, but I'm not sure how to do it in CMake. In regular Make you just treat *.a as *.o, but CMake annoyingly hides object linkage from us. Maybe it will let us add_library(compiz_core SHARED *.a)

On Fri, 27 Jan 2012, Daniel van Vugt wrote:

> It looks like the cause is:
>
> target_link_libraries (
> compiz_core
>
> ${COMPIZ_LIBRARIES}
>
> m
> pthread
> dl
>
> -Wl,-whole-archive
> compiz_string
> compiz_timer
> compiz_logmessage
> compiz_pluginclasshandler
> compiz_point
> compiz_rect
> compiz_window_geometry
> compiz_window_geometry_saver
> compiz_window_extents
> compiz_window_constrainment
> -Wl,-no-whole-archive
> # ${CORE_MOD_LIBRARIES}
> )
>
> What we actually want is for compiz_* to be added to compiz_core as
> objects, not as dependent libraries that plugins then mistakenly link
> in. This is easy to do in regular Make, but I'm not sure how to do it in
> CMake. In regular Make you just treat *.a as *.o, but CMake annoyingly
> hides object linkage from us. Maybe it will let us
> add_library(compiz_core SHARED *.a)
>

Well, the plugins are currently linking to libcompiz_core directly (see
cmake/CompizPlugin.cmake) . This would cause all of the symbols to be
defined as linked in there.

However, even in this case, if you were linking to libcompiz_core from a
plugin, then you would expect that the library itself is refcounted like
you said earlier. So the static data wouldn't be shared. Indeed, this is
what happens when you disable -Bsymbolic-functions in the cflags. (at
least for now, I've worked around this by doing this in the packaging, but
we can't do it for too long because of ubuntu policy:
https://lists.ubuntu.com/archives/ubuntu-devel/2008-May/025367.html).
Right now it seems though, all static data in libcompiz_core is in fact
loaded again for every plugin that is loaded (checking the addresses of
common singletons etc shows this).

Perhaps as you said, the static libraries are being linked statically in
the plugins instead of being self-contained into libcompiz_core. So they
are duplicated.

The solution could be to make all the static libraries shared ones and
just install them, but this feels messy and will increase load times.

(of course, the libraries themselves are needed for making tests)

objdump -S on the plugins could also reveal what is going on too.

(forgive my lack of knowledge on this topic)

> ** Description changed:
>
> - Plugins get a copy of the core classes CompPoint, CompRect, CompTimer
> - etc code duplicated inside them. Obviously this should not happen. The
> - only copy of these classes should be in compiz_core.
> + Plugins get a copy of the core classes CompPoint, CompRect, CompTimer,
> + compiz::window::* duplicated inside them. Obviously this should not
> + happen. The only copy of these classes should be in compiz_core.
>
> TESTCASE:
> nm -C lib/compiz/libresize.so | grep ' T '
>
> - You will see a lot of symbols that should be in libcompiz_core.so only
> - (type 'U').
> + You will see a lot of type-T symbols that should only be in
> + libcompiz_core.so. In plugins, those symbols should be type 'U' for
> + undefined.
>
> --
> You received this bug notification because you are a member of Compiz
> Maintainers, which is the registrant for Compiz Core.
> https://bugs.launchpad.net/bugs/922199
>
> Title:
> Duplicate core libr...

I might add here - I tried simply removing the linking between plugins and core. But this doesn't work, as libcompiz_core will still be duplicated, but only with -Bsymbolic-functions.

(A small note, removing the direct link makes the symbols show up as U in nm -C on the plugins)

Don't panic, I'm very experienced at solving dynamic linkage conflicts like this (sadly).

We don't need to compromise and make yet more dynamic libraries. We also should NOT use -Bsymbolic*, because it's only a workaround telling the dynamic loader which copy of the symbols to use (while there still exists redundant copies of everything). A proper fix will ensure there is only ever one copy of each compiz symbol, and that is always in compiz_core.

All we need to do is find a way to tell CMake that "compiz_core" CONTAINS the compiz_* archives, while also telling it not to carry those target dependencies on to other components (plugins) that use compiz_core. It's that simple, and I suspect CMake can do it (somehow).

Good to know :)

I did a little more trial-and-error investigation into this last night, since I was suspecting a linker bug might be at hand here (as it makes no sense that static libraries linked into a shared library which is linked to a binary which dlopens other libraries should cause multiple copies of the static data in the shared library to appear in the libraries).

/usr/bin/c++ -fPIC -fPIC -Wall -Wno-deprecated-declarations -Werror -fPIC -Wall -Wno-deprecated-declarations -Werror -fPIC -Wall -Wno-deprecated-declarations -Werror -fPIC -Wall -Wno-deprecated-declarations -Werror -fPIC -Wall -Wno-deprecated-declarations -Werror -O2 -g -DQT_NO_DEBUG -Wl,-Bsymbolic-functions -shared -Wl,-soname,libdecor.so -o libdecor.so CMakeFiles/decor.dir/src/decor.cpp.o CMakeFiles/decor.dir/__/__/generated/decor_options.cpp.o -L/home/smspillaz/Applications/Compiz/lib/compiz -L/home/smspillaz/Applications/Compiz/lib -lXdamage -lXfixes -lXcomposite -lX11-xcb -lX11 -lxcb -lXrandr -lXinerama -lXext -lICE -lSM -lxslt -lxml2 -lgio-2.0 -lglibmm-2.4 -lgobject-2.0 -lsigc-2.0 -lglib-2.0 -lstartup-notification-1 -lboost_serialization-mt -lboost_serialization-mt -lboost_serialization-mt ../composite/libcomposite.so ../opengl/libopengl.so ../../libdecoration/libdecoration.so.0.0.0 ../composite/libcomposite.so -lXdamage -lXfixes -lXcomposite -lX11-xcb -lxcb -lXrandr -lXinerama -lXext -lICE -lSM -lxslt -lxml2 -lgio-2.0 -lglibmm-2.4 -lgobject-2.0 -lsigc-2.0 -lglib-2.0 -lstartup-notification-1 -lboost_serialization-mt -lGL -lXrender -lX11 -lm -Wl,-rpath,::::::::::::::::::::::::::::::::::::::::::::::
make[2]: Leaving directory `/media/d1dddb1a-729f-40ef-9725-dc2a9ad56031/smspillaz/Source/Compiz/dev/dev/merges/compiz/core/compiz-core/build'

Would seem to indicate that those dependencies aren't being pulled in.

Incidentally, if I create another shared library built from some other static libraries, this bug doesn't happen anymore. No idea why.

Branch that shows this inconsistency: lp:~smspillaz/compiz-core/compiz-core.linker-debugging .

Correction: lp:~smspillaz/+junk/compiz-core.linker-debugging

(I need to stop url hacking)

Fix released in Compiz 0.9.7.0 methinks.