Merge lp:~michihenning/thumbnailer/disable-tests into lp:thumbnailer

[Michi Henning]

Disabled QDBus-related tests on xenial and yakkety for the time being due to problems with Qt 5.6. See
    https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561
    https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930

Disabled QDBus-related tests on xenial and yakkety for the time being due to problems with Qt 5.6. See
    https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561
    https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930

[Timo Jyrinki (timo-jyrinki)]

Looks good otherwise, but why on xenial, as they pass there?

review: Needs Information

[Michi Henning (michihenning)]

Timo: They don't pass on xenial. Had a failure yesterday on xenial too.

[Timo Jyrinki (timo-jyrinki)]

Ok, but just flakiness? Trainguards and core-devs can retry them easily. I mean, they are possible to get to pass on xenial+overlay like https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/1969/+packages (succeeded on first try) but always failing on yakkety so there's a difference.

But as you wish, just commented when noticed.

[Michi Henning (michihenning)]

This bug has "race condition" written all over it. We've seen this fail on yakkity, and once on xenial on powerpc (where it has never failed in yakkity so far).

I'm not really interested in trying to keep these tests running just in case that they make it through by accident, only to fail again at some other inconvenient point in time.

We'll make it a priority to try and re-enable the tests ASAP. But, for now, and to unblock unity8, I want to be sure that there are no more glitches or surprises.

[James Henstridge (jamesh)]

Yep. The common factor is Qt 5.6, which is shared by xenial+overlay and yakkety.

review: Approve

[Michi Henning (michihenning)]

Re-approving because James is on leave.

review: Approve

=== modified file 'CMakeLists.txt'
--- CMakeLists.txt	2016-07-18 14:55:34 +0000
+++ CMakeLists.txt	2016-09-26 02:39:33 +0000
@@ -82,6 +82,28 @@
     add_definitions(-DSLOW_TESTS=0)
 endif()
 
+# TODO: Hack to allow disabling tests that need QDBus on xenial and yakkety,
+#       due to broken Qt 5.6. Re-enable tests ASAP!
+option(skip-dbus-tests "Skip DBus tests" ON)
+if (${skip-dbus-tests})
+    execute_process(COMMAND lsb_release -s -c OUTPUT_VARIABLE distro OUTPUT_STRIP_TRAILING_WHITESPACE)
+    execute_process(COMMAND uname -m OUTPUT_VARIABLE arch OUTPUT_STRIP_TRAILING_WHITESPACE)
+    message("DISTRO: ${distro}")
+    message("ARCH:   ${arch}")
+    if (
+            (${distro} STREQUAL "xenial" OR ${distro} STREQUAL "yakkety")
+            AND
+            (${arch} STREQUAL "aarch64" OR ${arch} STREQUAL "ppc" OR ${arch} STREQUAL "ppc64le")
+       )
+        message(WARNING "WARNING: Disabling DBus-related tests for ${distro}
+         See https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561
+             https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930")
+        add_definitions(-DSKIP_DBUS_TESTS=1)
+        add_definitions(-DDISTRO=\"${distro}\")
+        add_definitions(-DARCH=\"${arch}\")
+    endif()
+endif()
+
 # Definitions for testing with valgrind.
 
 configure_file(CTestCustom.cmake.in CTestCustom.cmake) # Tests in CTestCustom.cmake are skipped for valgrind

=== modified file 'debian/changelog'
--- debian/changelog	2016-09-26 02:39:33 +0000
+++ debian/changelog	2016-09-26 02:39:33 +0000
@@ -1,38 +1,25 @@
-<<<<<<< TREE
-thumbnailer (2.4+16.10.20160825-0ubuntu1) yakkety; urgency=medium
-
-  * Changed python-tornado dependency to python-tornado <!nocheck> for cross-builds.
-  * Relaxed copyright for trace.cpp and trace.h to LGPL.
-
- -- Michi Henning <[e-mail]>  Thu, 25 Aug 2016 06:59:02 +0000
-
-thumbnailer (2.4+16.10.20160719-0ubuntu2) yakkety; urgency=medium
-
-  * No-change rebuild for boost soname change.
-
- -- Matthias Klose <[e-mail]>  Thu, 04 Aug 2016 08:32:55 +0000
-
-=======
-thumbnailer (2.4+16.10.20160825-0ubuntu1) yakkety; urgency=medium
-
-  * Changed python-tornado dependency to python-tornado <!nocheck> for cross-builds.
-  * Relaxed copyright for trace.cpp and trace.h to LGPL.
-
- -- Michi Henning <[e-mail]>  Thu, 25 Aug 2016 06:59:02 +0000
-
-thumbnailer (2.4+16.10.20160719-0ubuntu2) yakkety; urgency=medium
-
-  * No-change rebuild for boost soname change.
-
- -- Matthias Klose <[e-mail]>  Thu, 04 Aug 2016 08:32:55 +0000
-
-thumbnailer (2.4) UNRELEASED; urgency=medium
-
-  * Changed python-tornado dependency to python-tornado <!nocheck> for cross-builds.
-
- -- Michi Henning <[e-mail]>  Mon, 25 Jul 2016 10:56:23 +1000
-
->>>>>>> MERGE-SOURCE
+thumbnailer (2.4+16.10.20160825-0ubuntu2) UNRELEASED; urgency=medium
+
+  * Disabled QDBus-related tests on xenial and yakkety for the time being due
+    to problems with Qt 5.6. See
+      https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561
+      https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930
+
+ -- Michi Henning <[e-mail]>  Fri, 23 Sep 2016 14:08:49 +1000
+
+thumbnailer (2.4+16.10.20160825-0ubuntu1) yakkety; urgency=medium
+
+  * Changed python-tornado dependency to python-tornado <!nocheck> for cross-builds.
+  * Relaxed copyright for trace.cpp and trace.h to LGPL.
+
+ -- Michi Henning <[e-mail]>  Thu, 25 Aug 2016 06:59:02 +0000
+
+thumbnailer (2.4+16.10.20160719-0ubuntu2) yakkety; urgency=medium
+
+  * No-change rebuild for boost soname change.
+
+ -- Matthias Klose <[e-mail]>  Thu, 04 Aug 2016 08:32:55 +0000
+
 thumbnailer (2.4+16.10.20160719-0ubuntu1) yakkety; urgency=medium
 
   [ Michi Henning ]

=== modified file 'doc/libthumbnailer-qt/examples/qt_example_test.cpp'
--- doc/libthumbnailer-qt/examples/qt_example_test.cpp	2015-12-12 02:06:41 +0000
+++ doc/libthumbnailer-qt/examples/qt_example_test.cpp	2016-09-26 02:39:33 +0000
@@ -178,6 +178,13 @@
 
 int main(int argc, char** argv)
 {
+#if defined(SKIP_DBUS_TESTS)
+    // TODO: Re-enable this ASAP!
+    cerr << "WARNING: Skipping tests on " << DISTRO << endl;
+    cerr << "         See https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561" << endl;
+    cerr << "             https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930" << endl;
+    return 0;
+#endif
     QCoreApplication app(argc, argv);
     setenv("GSETTINGS_BACKEND", "memory", true);
     setenv("GSETTINGS_SCHEMA_DIR", GSETTINGS_SCHEMA_DIR, true);

=== modified file 'tests/dbus/dbus_test.cpp'
--- tests/dbus/dbus_test.cpp	2016-01-04 06:09:41 +0000
+++ tests/dbus/dbus_test.cpp	2016-09-26 02:39:33 +0000
@@ -610,6 +610,13 @@
 
 int main(int argc, char** argv)
 {
+#if defined(SKIP_DBUS_TESTS)
+    // TODO: Re-enable this ASAP!
+    cerr << "WARNING: Skipping tests on " << DISTRO << " " << ARCH << endl;
+    cerr << "         See https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561" << endl;
+    cerr << "             https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930" << endl;
+    return 0;
+#endif
     QCoreApplication app(argc, argv);
     qRegisterMetaType<QProcess::ExitStatus>("QProcess::ExitStatus");  // Avoid noise from signal spy.
     qDBusRegisterMetaType<unity::thumbnailer::service::AllStats>();

=== modified file 'tests/image-provider/image-provider_test.cpp'
--- tests/image-provider/image-provider_test.cpp	2016-01-12 07:21:18 +0000
+++ tests/image-provider/image-provider_test.cpp	2016-09-26 02:39:33 +0000
@@ -204,6 +204,13 @@
 
 int main(int argc, char **argv)
 {
+#if defined(SKIP_DBUS_TESTS)
+    // TODO: Re-enable this ASAP!
+    cerr << "WARNING: Skipping tests on " << DISTRO << " " << ARCH << endl;
+    cerr << "         See https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561" << endl;
+    cerr << "             https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930" << endl;
+    return 0;
+#endif
     QGuiApplication app(argc, argv);
     setenv("GSETTINGS_BACKEND", "memory", true);
     setenv("GSETTINGS_SCHEMA_DIR", GSETTINGS_SCHEMA_DIR, true);

=== modified file 'tests/libthumbnailer-qt/libthumbnailer-qt_test.cpp'
--- tests/libthumbnailer-qt/libthumbnailer-qt_test.cpp	2016-02-26 23:13:38 +0000
+++ tests/libthumbnailer-qt/libthumbnailer-qt_test.cpp	2016-09-26 02:39:33 +0000
@@ -779,6 +779,13 @@
 
 int main(int argc, char** argv)
 {
+#if defined(SKIP_DBUS_TESTS)
+    // TODO: Re-enable this ASAP!
+    cerr << "WARNING: Skipping tests on " << DISTRO << " " << ARCH << endl;
+    cerr << "         See https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561" << endl;
+    cerr << "             https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930" << endl;
+    return 0;
+#endif
     gst_init(&argc, &argv);
 
     QCoreApplication app(argc, argv);

=== modified file 'tests/qml/qml_test.cpp'
--- tests/qml/qml_test.cpp	2015-12-12 02:06:41 +0000
+++ tests/qml/qml_test.cpp	2016-09-26 02:39:33 +0000
@@ -67,6 +67,13 @@
 
 int main(int argc, char** argv)
 {
+#if defined(SKIP_DBUS_TESTS)
+    // TODO: Re-enable this ASAP!
+    qDebug() << "WARNING: Skipping tests on" << DISTRO << " " << ARCH << endl;
+    qDebug() << "         See https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561";
+    qDebug() << "             https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930";
+    return 0;
+#endif
     QGuiApplication app(argc, argv);
 
     setenv("GSETTINGS_BACKEND", "memory", true);

=== modified file 'tests/slow-vs-thumb/slow-vs-thumb_test.cpp'
--- tests/slow-vs-thumb/slow-vs-thumb_test.cpp	2016-02-02 06:29:01 +0000
+++ tests/slow-vs-thumb/slow-vs-thumb_test.cpp	2016-09-26 02:39:33 +0000
@@ -72,7 +72,7 @@
 
     QSignalSpy spy(request.get(), &ThumbnailRequest::downloadFinished);
     request->download();
-    ASSERT_TRUE(spy.wait(15000));  // Slow vs-thumb will get killed after 10 seconds.
+    ASSERT_TRUE(spy.wait(30000));  // Slow vs-thumb will get killed after 10 seconds.
 
     EXPECT_EQ("", request->thumbnail());
 }

=== modified file 'tests/stress/stress_test.cpp'
--- tests/stress/stress_test.cpp	2016-02-02 03:56:15 +0000
+++ tests/stress/stress_test.cpp	2016-09-26 02:39:33 +0000
@@ -596,6 +596,13 @@
 
 int main(int argc, char** argv)
 {
+#if defined(SKIP_DBUS_TESTS)
+    // TODO: Re-enable this ASAP!
+    cerr << "WARNING: Skipping tests on " << DISTRO << " " << ARCH << endl;
+    cerr << "         See https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1613561" << endl;
+    cerr << "             https://bugs.launchpad.net/ubuntu/+source/qtbase-opensource-src/+bug/1625930" << endl;
+    return 0;
+#endif
     gst_init(&argc, &argv);
 
     QCoreApplication app(argc, argv);