unity-scopes-shell

Merge lp:~marcustomlinson/unity-scopes-shell/relax-harness-timeouts into lp:unity-scopes-shell

  1. relax-harness-timeouts
  2. Merge into trunk
Proposed by Marcus Tomlinson
Status: Merged
Approved by: Paweł Stołowski
Approved revision: 352
Merged at revision: 340
Proposed branch: lp:~marcustomlinson/unity-scopes-shell/relax-harness-timeouts
Merge into: lp:unity-scopes-shell
Diff against target: 310 lines (+48/-27)
13 files modified
CMakeLists.txt (+1/-1)
debian/changelog (+6/-0)
debian/control (+1/-1)
debian/control.in (+1/-1)
src/scope-harness/preview/preview-widget.cpp (+1/-1)
src/scope-harness/registry/custom-registry.cpp (+15/-2)
src/scope-harness/results/result.cpp (+1/-1)
src/scope-harness/scope-harness.cpp (+2/-2)
src/scope-harness/test-utils.cpp (+8/-8)
src/scope-harness/test-utils.h (+2/-0)
src/scope-harness/view/preview-view.cpp (+1/-1)
src/scope-harness/view/results-view.cpp (+8/-8)
src/scope-harness/view/settings-view.cpp (+1/-1)
To merge this branch: bzr merge lp:~marcustomlinson/unity-scopes-shell/relax-harness-timeouts
Reviewer Review Type Date Requested Status
Paweł Stołowski (community) Approve
unity-api-1-bot continuous-integration Needs Fixing
Review via email: mp+304459@code.launchpad.net

Commit message

Increase scope-harness timeouts to fix flaky scope tests (incl. autopkg) on slow builders.

To post a comment you must log in.
Revision history for this message
unity-api-1-bot (unity-api-1-bot) wrote :

FAILED: Continuous integration, rev:352
https://jenkins.canonical.com/unity-api-1/job/lp-unity-scopes-shell-ci/1/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/611/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/617
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/440/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/440/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/440/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/440/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/440/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/440/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/440/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/440/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/440/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-unity-scopes-shell-ci/1/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Paweł Stołowski (stolowski) wrote :

Looks good, thanks for looking at it! Even if it only improves the situation and doesn't fix all the test flakiness, it's definately worth having! +1

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'CMakeLists.txt'
--- CMakeLists.txt 2016-05-31 09:31:57 +0000
+++ CMakeLists.txt 2016-09-08 08:53:48 +0000
@@ -51,7 +51,7 @@
51find_package(Qt5Test)51find_package(Qt5Test)
52find_package(Boost COMPONENTS regex REQUIRED)52find_package(Boost COMPONENTS regex REQUIRED)
5353
54pkg_check_modules(SCOPESLIB REQUIRED libunity-scopes>=1.0.5)54pkg_check_modules(SCOPESLIB REQUIRED libunity-scopes>=1.0.7)
55pkg_check_modules(SCOPES_API REQUIRED unity-shell-scopes=12)55pkg_check_modules(SCOPES_API REQUIRED unity-shell-scopes=12)
5656
57pkg_check_modules(GSETTINGSQT REQUIRED gsettings-qt)57pkg_check_modules(GSETTINGSQT REQUIRED gsettings-qt)
5858
=== modified file 'debian/changelog'
--- debian/changelog 2016-08-10 14:55:34 +0000
+++ debian/changelog 2016-09-08 08:53:48 +0000
@@ -1,3 +1,9 @@
1unity-scopes-shell (0.5.8-0ubuntu1) UNRELEASED; urgency=medium
2
3 * Increase scope-harness timeouts to assist scope testing on slow builders.
4
5 -- Marcus Tomlinson <marcus.tomlinson@canonical.com> Thu, 08 Sep 2016 10:47:19 +0200
6
1unity-scopes-shell (0.5.7+16.10.20160624.2-0ubuntu2) yakkety; urgency=medium7unity-scopes-shell (0.5.7+16.10.20160624.2-0ubuntu2) yakkety; urgency=medium
28
3 * No-change rebuild for boost soname change.9 * No-change rebuild for boost soname change.
410
=== modified file 'debian/control'
--- debian/control 2016-06-15 20:18:25 +0000
+++ debian/control 2016-09-08 08:53:48 +0000
@@ -9,7 +9,7 @@
9 libboost-python-dev,9 libboost-python-dev,
10 libboost-regex-dev,10 libboost-regex-dev,
11 libunity-api-dev (>= 7.111),11 libunity-api-dev (>= 7.111),
12 libunity-scopes-dev (>= 1.0.5~),12 libunity-scopes-dev (>= 1.0.7~),
13 libgsettings-qt-dev (>= 0.1),13 libgsettings-qt-dev (>= 0.1),
14 libqtdbustest1-dev (>= 0.2),14 libqtdbustest1-dev (>= 0.2),
15 libqtdbusmock1-dev (>= 0.2),15 libqtdbusmock1-dev (>= 0.2),
1616
=== modified file 'debian/control.in'
--- debian/control.in 2016-05-31 09:31:57 +0000
+++ debian/control.in 2016-09-08 08:53:48 +0000
@@ -9,7 +9,7 @@
9 libboost-python-dev,9 libboost-python-dev,
10 libboost-regex-dev,10 libboost-regex-dev,
11 libunity-api-dev (>= 7.111),11 libunity-api-dev (>= 7.111),
12 libunity-scopes-dev (>= 1.0.5~),12 libunity-scopes-dev (>= 1.0.7~),
13 libgsettings-qt-dev (>= 0.1),13 libgsettings-qt-dev (>= 0.1),
14 libqtdbustest1-dev (>= 0.2),14 libqtdbustest1-dev (>= 0.2),
15 libqtdbusmock1-dev (>= 0.2),15 libqtdbusmock1-dev (>= 0.2),
1616
=== modified file 'src/scope-harness/preview/preview-widget.cpp'
--- src/scope-harness/preview/preview-widget.cpp 2016-02-17 15:49:37 +0000
+++ src/scope-harness/preview/preview-widget.cpp 2016-09-08 08:53:48 +0000
@@ -160,7 +160,7 @@
160160
161 TestUtils::throwIfNot(p->m_previewModel->processingAction(), "Should be processing action");161 TestUtils::throwIfNot(p->m_previewModel->processingAction(), "Should be processing action");
162 QSignalSpy spy(p->m_previewModel, SIGNAL(processingActionChanged()));162 QSignalSpy spy(p->m_previewModel, SIGNAL(processingActionChanged()));
163 TestUtils::throwIfNot(spy.wait(), "Processing action property didn't change");163 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Processing action property didn't change");
164 TestUtils::throwIf(p->m_previewModel->processingAction(), "Should have finished processing action");164 TestUtils::throwIf(p->m_previewModel->processingAction(), "Should have finished processing action");
165165
166 view::PreviewView::SPtr previewView = p->m_previewView.lock();166 view::PreviewView::SPtr previewView = p->m_previewView.lock();
167167
=== modified file 'src/scope-harness/registry/custom-registry.cpp'
--- src/scope-harness/registry/custom-registry.cpp 2015-08-19 13:19:22 +0000
+++ src/scope-harness/registry/custom-registry.cpp 2016-09-08 08:53:48 +0000
@@ -38,6 +38,11 @@
38namespace38namespace
39{39{
4040
41const static int c_processTimeout = 60000;
42const static int c_twoWayTimeout = 60000;
43const static int c_locateTimeout = 60000;
44const static int c_registryTimeout = 60000;
45
41const static QString RUNTIME_CONFIG = R"(46const static QString RUNTIME_CONFIG = R"(
42[Runtime]47[Runtime]
43Registry.Identity = Registry48Registry.Identity = Registry
@@ -55,11 +60,15 @@
55Scope.InstallDir = %360Scope.InstallDir = %3
56Click.InstallDir = %461Click.InstallDir = %4
57OEM.InstallDir = %562OEM.InstallDir = %5
63Process.Timeout = %6
58)";64)";
5965
60const static QString MW_CONFIG = R"(66const static QString MW_CONFIG = R"(
61[Zmq]67[Zmq]
62EndpointDir = %168EndpointDir = %1
69Default.Twoway.Timeout = %2
70Locate.Timeout = %3
71Registry.Timeout = %4
63)";72)";
6473
65}74}
@@ -218,10 +227,14 @@
218 .arg(scopeRunnerBin.fileName())227 .arg(scopeRunnerBin.fileName())
219 .arg(scopeInstallDir.path())228 .arg(scopeInstallDir.path())
220 .arg(clickInstallDir.path())229 .arg(clickInstallDir.path())
221 .arg(oemInstallDir.path());230 .arg(oemInstallDir.path())
231 .arg(c_processTimeout);
222232
223 QString mwIni = MW_CONFIG233 QString mwIni = MW_CONFIG
224 .arg(endpointsDir.path());234 .arg(endpointsDir.path())
235 .arg(c_twoWayTimeout)
236 .arg(c_locateTimeout)
237 .arg(c_registryTimeout);
225238
226 runtimeConfig.write(runtimeIni.toUtf8());239 runtimeConfig.write(runtimeIni.toUtf8());
227 registryConfig.write(registryIni.toUtf8());240 registryConfig.write(registryIni.toUtf8());
228241
=== modified file 'src/scope-harness/results/result.cpp'
--- src/scope-harness/results/result.cpp 2016-02-17 15:49:37 +0000
+++ src/scope-harness/results/result.cpp 2016-09-08 08:53:48 +0000
@@ -105,7 +105,7 @@
105105
106 if (spy.empty())106 if (spy.empty())
107 {107 {
108 TestUtils::throwIfNot(spy.wait(), "Scope activation signal failed to emit");108 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Scope activation signal failed to emit");
109 }109 }
110110
111 QVariantList response = spy.front();111 QVariantList response = spy.front();
112112
=== modified file 'src/scope-harness/scope-harness.cpp'
--- src/scope-harness/scope-harness.cpp 2015-03-10 11:23:01 +0000
+++ src/scope-harness/scope-harness.cpp 2016-09-08 08:53:48 +0000
@@ -87,7 +87,7 @@
8787
88 // wait till the registry spawns88 // wait till the registry spawns
89 QSignalSpy spy(p->m_scopes.get(), SIGNAL(loadedChanged()));89 QSignalSpy spy(p->m_scopes.get(), SIGNAL(loadedChanged()));
90 TestUtils::throwIfNot(spy.wait(), "Scopes failed to initalize");90 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Scopes failed to initalize");
9191
92 TestUtils::throwIf(p->m_scopes->rowCount() == 0 || !p->m_scopes->loaded(), "No scopes loaded");92 TestUtils::throwIf(p->m_scopes->rowCount() == 0 || !p->m_scopes->loaded(), "No scopes loaded");
9393
@@ -98,7 +98,7 @@
98 QSignalSpy spy(scope.data(), SIGNAL(searchInProgressChanged()));98 QSignalSpy spy(scope.data(), SIGNAL(searchInProgressChanged()));
99 if (scope->searchInProgress())99 if (scope->searchInProgress())
100 {100 {
101 TestUtils::throwIfNot(spy.wait(), "Search progress didn't change");101 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Search progress didn't change");
102 }102 }
103 }103 }
104}104}
105105
=== modified file 'src/scope-harness/test-utils.cpp'
--- src/scope-harness/test-utils.cpp 2016-03-07 08:29:14 +0000
+++ src/scope-harness/test-utils.cpp 2016-09-08 08:53:48 +0000
@@ -90,7 +90,7 @@
90 QVERIFY(scope->searchInProgress() || spy.count() > 1);90 QVERIFY(scope->searchInProgress() || spy.count() > 1);
91 if (scope->searchInProgress()) {91 if (scope->searchInProgress()) {
92 // wait for the search to finish92 // wait for the search to finish
93 QVERIFY(spy.wait());93 QVERIFY(spy.wait(SIG_SPY_TIMEOUT));
94 }94 }
95 QCOMPARE(scope->searchInProgress(), false);95 QCOMPARE(scope->searchInProgress(), false);
96}96}
@@ -103,10 +103,10 @@
103 scope->setSearchQuery(searchString);103 scope->setSearchQuery(searchString);
104 // search should not be happening yet104 // search should not be happening yet
105 QCOMPARE(scope->searchInProgress(), false);105 QCOMPARE(scope->searchInProgress(), false);
106 QVERIFY(spy.wait());106 QVERIFY(spy.wait(SIG_SPY_TIMEOUT));
107 if (scope->searchInProgress()) {107 if (scope->searchInProgress()) {
108 // wait for the search to finish108 // wait for the search to finish
109 QVERIFY(spy.wait());109 QVERIFY(spy.wait(SIG_SPY_TIMEOUT));
110 }110 }
111 QCOMPARE(scope->searchInProgress(), false);111 QCOMPARE(scope->searchInProgress(), false);
112}112}
@@ -116,9 +116,9 @@
116 QCOMPARE(scope->searchInProgress(), false);116 QCOMPARE(scope->searchInProgress(), false);
117 // wait for the search to finish117 // wait for the search to finish
118 QSignalSpy spy(scope.data(), SIGNAL(searchInProgressChanged()));118 QSignalSpy spy(scope.data(), SIGNAL(searchInProgressChanged()));
119 QVERIFY(spy.wait());119 QVERIFY(spy.wait(SIG_SPY_TIMEOUT));
120 if(spy.size() == 1) {120 if(spy.size() == 1) {
121 QVERIFY(spy.wait());121 QVERIFY(spy.wait(SIG_SPY_TIMEOUT));
122 }122 }
123 QCOMPARE(scope->searchInProgress(), false);123 QCOMPARE(scope->searchInProgress(), false);
124}124}
@@ -127,10 +127,10 @@
127{127{
128 QCOMPARE(scope->searchInProgress(), true);128 QCOMPARE(scope->searchInProgress(), true);
129 QSignalSpy spy(scope.data(), SIGNAL(searchInProgressChanged()));129 QSignalSpy spy(scope.data(), SIGNAL(searchInProgressChanged()));
130 QVERIFY(spy.wait());130 QVERIFY(spy.wait(SIG_SPY_TIMEOUT));
131 if (scope->searchInProgress()) {131 if (scope->searchInProgress()) {
132 // wait for the search to finish132 // wait for the search to finish
133 QVERIFY(spy.wait());133 QVERIFY(spy.wait(SIG_SPY_TIMEOUT));
134 }134 }
135 QCOMPARE(scope->searchInProgress(), false);135 QCOMPARE(scope->searchInProgress(), false);
136}136}
@@ -138,7 +138,7 @@
138void TestUtils::waitForFilterStateChange(QSharedPointer<ss::ScopeInterface> scope)138void TestUtils::waitForFilterStateChange(QSharedPointer<ss::ScopeInterface> scope)
139{139{
140 QSignalSpy spy(scope->filters(), SIGNAL(filterStateChanged()));140 QSignalSpy spy(scope->filters(), SIGNAL(filterStateChanged()));
141 QVERIFY(spy.wait());141 QVERIFY(spy.wait(SIG_SPY_TIMEOUT));
142 QCOMPARE(spy.count(), 1);142 QCOMPARE(spy.count(), 1);
143}143}
144144
145145
=== modified file 'src/scope-harness/test-utils.h'
--- src/scope-harness/test-utils.h 2016-03-07 08:29:14 +0000
+++ src/scope-harness/test-utils.h 2016-09-08 08:53:48 +0000
@@ -28,6 +28,8 @@
28namespace unity {28namespace unity {
29namespace scopeharness {29namespace scopeharness {
3030
31static constexpr int SIG_SPY_TIMEOUT = 60000; // milliseconds
32
31class TestUtils33class TestUtils
32{34{
33public:35public:
3436
=== modified file 'src/scope-harness/view/preview-view.cpp'
--- src/scope-harness/view/preview-view.cpp 2016-02-17 15:49:37 +0000
+++ src/scope-harness/view/preview-view.cpp 2016-09-08 08:53:48 +0000
@@ -63,7 +63,7 @@
63 if (!previewModel->loaded())63 if (!previewModel->loaded())
64 {64 {
65 QSignalSpy spy(previewModel, SIGNAL(loadedChanged()));65 QSignalSpy spy(previewModel, SIGNAL(loadedChanged()));
66 spy.wait();66 spy.wait(SIG_SPY_TIMEOUT);
67 }67 }
6868
69 vector<preview::PreviewWidgetList> previewModels;69 vector<preview::PreviewWidgetList> previewModels;
7070
=== modified file 'src/scope-harness/view/results-view.cpp'
--- src/scope-harness/view/results-view.cpp 2016-04-21 10:00:57 +0000
+++ src/scope-harness/view/results-view.cpp 2016-09-08 08:53:48 +0000
@@ -123,7 +123,7 @@
123123
124 if (!navigationModel->loaded())124 if (!navigationModel->loaded())
125 {125 {
126 TestUtils::throwIfNot(spy.wait(), "Department model failed to load");126 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Department model failed to load");
127 }127 }
128128
129 return results::Department(internal::DepartmentArguments{navigationModel});129 return results::Department(internal::DepartmentArguments{navigationModel});
@@ -180,11 +180,11 @@
180180
181 if (!scope->searchInProgress())181 if (!scope->searchInProgress())
182 {182 {
183 spy.wait(100);183 spy.wait(2000);
184 }184 }
185 if (scope->searchInProgress())185 if (scope->searchInProgress())
186 {186 {
187 TestUtils::throwIfNot(spy.wait(), "Active scope didn't finish searching");187 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Active scope didn't finish searching");
188 }188 }
189189
190 break;190 break;
@@ -223,11 +223,11 @@
223 p->m_active_scope->setSearchQuery(searchString);223 p->m_active_scope->setSearchQuery(searchString);
224 // search should not be happening yet224 // search should not be happening yet
225 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "Search was in progress too soon");225 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "Search was in progress too soon");
226 TestUtils::throwIfNot(spy.wait(), "Search spy received no events");226 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Search spy received no events");
227 if (p->m_active_scope->searchInProgress())227 if (p->m_active_scope->searchInProgress())
228 {228 {
229 // wait for the search to finish229 // wait for the search to finish
230 TestUtils::throwIfNot(spy.wait(), "Search spy received no events");230 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Search spy received no events");
231 }231 }
232 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "Search did not complete");232 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "Search did not complete");
233}233}
@@ -246,7 +246,7 @@
246 if (p->m_active_scope->searchInProgress())246 if (p->m_active_scope->searchInProgress())
247 {247 {
248 // wait for the search to finish248 // wait for the search to finish
249 TestUtils::throwIfNot(spy.wait(), "Search spy received no events");249 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Search spy received no events");
250 }250 }
251 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "Search did not complete");251 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "Search did not complete");
252}252}
@@ -258,9 +258,9 @@
258 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "Search is already in progress");258 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "Search is already in progress");
259 // wait for the search to finish259 // wait for the search to finish
260 QSignalSpy spy(p->m_active_scope.data(), SIGNAL(searchInProgressChanged()));260 QSignalSpy spy(p->m_active_scope.data(), SIGNAL(searchInProgressChanged()));
261 TestUtils::throwIfNot(spy.wait(), "Search status didn't change");261 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Search status didn't change");
262 if(spy.size() == 1) {262 if(spy.size() == 1) {
263 TestUtils::throwIfNot(spy.wait(), "Search status didn't change");263 TestUtils::throwIfNot(spy.wait(SIG_SPY_TIMEOUT), "Search status didn't change");
264 }264 }
265 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "");265 TestUtils::throwIf(p->m_active_scope->searchInProgress(), "");
266}266}
267267
=== modified file 'src/scope-harness/view/settings-view.cpp'
--- src/scope-harness/view/settings-view.cpp 2015-06-11 16:10:09 +0000
+++ src/scope-harness/view/settings-view.cpp 2016-09-08 08:53:48 +0000
@@ -155,7 +155,7 @@
155 }155 }
156 QSignalSpy settingChangedSpy(settings, SIGNAL(settingsChanged()));156 QSignalSpy settingChangedSpy(settings, SIGNAL(settingsChanged()));
157 settings->setData(index, ng::scopeVariantToQVariant(val), ss::SettingsModelInterface::Roles::RoleValue);157 settings->setData(index, ng::scopeVariantToQVariant(val), ss::SettingsModelInterface::Roles::RoleValue);
158 TestUtils::throwIfNot(settingChangedSpy.wait(), "Settings update failed");158 TestUtils::throwIfNot(settingChangedSpy.wait(SIG_SPY_TIMEOUT), "Settings update failed");
159 TestUtils::waitForSearchFinish(p->m_scope);159 TestUtils::waitForSearchFinish(p->m_scope);
160 return;160 return;
161 }161 }

Subscribers

People subscribed via source and target branches

to all changes: