Mir

Merge lp:~afrantzis/mir/fix-1576760-increase-timeout into lp:mir

  1. fix-1576760-increase-timeout
  2. Merge into development-branch
Proposed by Alexandros Frantzis
Status: Merged
Approved by: Alexandros Frantzis
Approved revision: no longer in the source branch.
Merged at revision: 3572
Proposed branch: lp:~afrantzis/mir/fix-1576760-increase-timeout
Merge into: lp:mir
Diff against target: 263 lines (+29/-29)
1 file modified
tests/acceptance-tests/test_nested_mir.cpp (+29/-29)
To merge this branch: bzr merge lp:~afrantzis/mir/fix-1576760-increase-timeout
Reviewer Review Type Date Requested Status
Cemil Azizoglu (community) Approve
Mir CI Bot continuous-integration Approve
Review via email: mp+298753@code.launchpad.net

Commit message

tests: Increase timeouts to accommodate valgrind slowness

Description of the change

tests: Increase timeouts to accommodate valgrind slowness

Use the existing 'timeout' variable which was set in the code, but not used until now.

To post a comment you must log in.
Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

PASSED: Continuous integration, rev:3567
https://mir-jenkins.ubuntu.com/job/mir-ci/1216/
Executed test runs:
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-mir/1397
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/1448
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/1439
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial/1439
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1411/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1411/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1411/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1411/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1411/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/1216/rebuild

review: Approve (continuous-integration)
Revision history for this message
Cemil Azizoglu (cemil-azizoglu) wrote :

I suspect these timeouts are overly conservative. If these tests start failing they may take a really long time to run since the timeout value is set to 10 at the moment. We should probably change the timeout to something like 2 or 3, and the (two) instances that require more than that can be hard-coded to have 5 or 10.

review: Needs Fixing
Revision history for this message
Alexandros Frantzis (afrantzis) wrote :

> I suspect these timeouts are overly conservative

valgrind under high load (which we can't control since our CI infrastructure is shared) can be *extremely* slow.

> If these tests start failing they may take a really long time to run since the timeout value
> is set to 10 at the moment

Can't say I am really concerned with this, since it's an extremely rare event and will add in the absolute worst case 5 minutes of total time.

> We should probably change the timeout to something like 2 or 3, and the (two) instances
> that require more than that can be hard-coded to have 5 or 10.

The reason the two instances have 5 and 10 respectively (which are arbitrary values), is because they failed at some point in the past due to valgrind slowness and high CI load, so we had to increase their timeouts.

Now NestedServer.when_monitor_plugged_in_client_is_notified_of_new_display_configuration happened to fail, so we need to increase its timeout too. In the future another test will fail for the same reason.

I tried setting the timeout to 3 seconds, but I got some failures locally under valgrind + sufficient load in various NestedServer.* tests. I couldn't get failures locally with 5 seconds, so I think it's a good start, although only time will tell if it will be enough for CI.

Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

PASSED: Continuous integration, rev:3568
https://mir-jenkins.ubuntu.com/job/mir-ci/1220/
Executed test runs:
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-mir/1403
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/1454
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/1445
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial/1445
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1417/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1417/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1417/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1417/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1417/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/1220/rebuild

review: Approve (continuous-integration)
Revision history for this message
Cemil Azizoglu (cemil-azizoglu) wrote :

ok

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'tests/acceptance-tests/test_nested_mir.cpp'
2--- tests/acceptance-tests/test_nested_mir.cpp 2016-06-02 08:19:38 +0000
3+++ tests/acceptance-tests/test_nested_mir.cpp 2016-07-01 10:10:31 +0000
4@@ -122,7 +122,7 @@
5 {{640, 0}, {1920, 1080}}
6 };
7
8-std::chrono::seconds const timeout{10};
9+std::chrono::seconds const timeout{5};
10
11 // We can't rely on the test environment to have X cursors, so we provide some of our own
12 auto const cursor_names = {
13@@ -405,7 +405,7 @@
14
15 mir_wait_for(mir_connection_apply_display_config(connection, configuration));
16
17- initial_condition.wait_for(std::chrono::seconds{1});
18+ initial_condition.wait_for(timeout);
19 mir_display_config_destroy(configuration);
20
21 Mock::VerifyAndClearExpectations(&display);
22@@ -598,7 +598,7 @@
23 trigger_lifecycle_event(mir_lifecycle_state_resumed);
24 trigger_lifecycle_event(mir_lifecycle_state_will_suspend);
25
26- events_processed.wait_for(std::chrono::seconds{5});
27+ events_processed.wait_for(timeout);
28 }
29
30 TEST_F(NestedServer, client_may_connect_to_nested_server_and_create_surface)
31@@ -613,7 +613,7 @@
32 return mir_surface_get_visibility(surface) == mir_surface_visibility_exposed
33 && mir_surface_get_focus(surface) == mir_surface_focused;
34 },
35- std::chrono::seconds{10});
36+ timeout);
37
38 EXPECT_TRUE(became_exposed_and_focused);
39 }
40@@ -644,7 +644,7 @@
41
42 mir_buffer_stream_swap_buffers_sync(mir_surface_get_buffer_stream(surface));
43
44- wait.wait_for(std::chrono::seconds{1});
45+ wait.wait_for(timeout);
46 Mock::VerifyAndClearExpectations(mock_session_mediator_report.get());
47 }
48
49@@ -659,7 +659,7 @@
50 mir_surface_release_sync(surface);
51 mir_connection_release(connection);
52
53- wait.wait_for(std::chrono::seconds{1});
54+ wait.wait_for(timeout);
55 Mock::VerifyAndClearExpectations(mock_session_mediator_report.get());
56 }
57
58@@ -685,7 +685,7 @@
59 client.update_display_configuration(
60 [](MirDisplayConfiguration* config) { config->outputs->used = false; });
61
62- condition.wait_for(std::chrono::seconds{1});
63+ condition.wait_for(timeout);
64 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
65 }
66
67@@ -714,7 +714,7 @@
68
69 mir_wait_for(mir_connection_apply_display_config(client.connection, configuration));
70
71- config_reported.wait_for(std::chrono::seconds{3});
72+ config_reported.wait_for(timeout);
73 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
74 }
75
76@@ -744,7 +744,7 @@
77 mir_wait_for(mir_surface_configure_cursor(client.surface, conf));
78 mir_cursor_configuration_destroy(conf);
79
80- condition.wait_for(std::chrono::seconds{1});
81+ condition.wait_for(timeout);
82 Mock::VerifyAndClearExpectations(mock_cursor.get());
83 }
84
85@@ -756,7 +756,7 @@
86
87 mir_buffer_stream_swap_buffers_sync(client.buffer_stream);
88
89- condition.wait_for(std::chrono::seconds{1});
90+ condition.wait_for(timeout);
91 Mock::VerifyAndClearExpectations(mock_cursor.get());
92 }
93 }
94@@ -786,7 +786,7 @@
95 mir_wait_for(mir_surface_configure_cursor(client.surface, cursor));
96 mir_cursor_configuration_destroy(cursor);
97
98- condition.wait_for(std::chrono::seconds{1});
99+ condition.wait_for(timeout);
100 Mock::VerifyAndClearExpectations(mock_cursor.get());
101 }
102 }
103@@ -814,7 +814,7 @@
104 mir_wait_for(mir_surface_configure_cursor(client.surface, conf));
105 mir_cursor_configuration_destroy(conf);
106
107- condition.wait_for(std::chrono::seconds{1});
108+ condition.wait_for(timeout);
109 Mock::VerifyAndClearExpectations(mock_cursor.get());
110 }
111
112@@ -867,7 +867,7 @@
113
114 ClientWithAPaintedSurface client(nested_mir);
115
116- condition.wait_for(std::chrono::seconds{1});
117+ condition.wait_for(timeout);
118 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
119
120 EXPECT_TRUE(condition.raised());
121@@ -889,7 +889,7 @@
122
123 server.the_display_configuration_controller()->set_base_configuration(new_config);
124
125- condition.wait_for(std::chrono::seconds{1});
126+ condition.wait_for(timeout);
127 Mock::VerifyAndClearExpectations(config_policy.get());
128 EXPECT_TRUE(condition.raised());
129 }
130@@ -914,7 +914,7 @@
131 [](MirDisplayConfiguration* config) { config->outputs->used = false; });
132
133 // Wait for initial config to be applied
134- initial_condition.wait_for(std::chrono::seconds{1});
135+ initial_condition.wait_for(timeout);
136
137 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
138 ASSERT_TRUE(initial_condition.raised());
139@@ -924,7 +924,7 @@
140 .WillRepeatedly(InvokeWithoutArgs([&] { condition.raise(); }));
141 }
142
143- condition.wait_for(std::chrono::seconds{1});
144+ condition.wait_for(timeout);
145 EXPECT_TRUE(condition.raised());
146 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
147 }
148@@ -952,7 +952,7 @@
149 nested_mir.server.the_display_configuration_controller()->set_base_configuration(new_config);
150
151 // Wait for initial config to be applied
152- initial_condition.wait_for(std::chrono::seconds{1});
153+ initial_condition.wait_for(timeout);
154 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
155 ASSERT_TRUE(initial_condition.raised());
156
157@@ -960,7 +960,7 @@
158 .WillRepeatedly(InvokeWithoutArgs([&] { condition.raise(); }));
159 }
160
161- condition.wait_for(std::chrono::seconds{1});
162+ condition.wait_for(timeout);
163 EXPECT_TRUE(condition.raised());
164 }
165
166@@ -979,7 +979,7 @@
167
168 display.emit_configuration_change_event(new_config);
169
170- client_config_changed.wait_for(std::chrono::seconds{1});
171+ client_config_changed.wait_for(timeout);
172
173 ASSERT_TRUE(client_config_changed.raised());
174
175@@ -1008,7 +1008,7 @@
176 nested_mir.server.the_display_configuration_controller()->set_base_configuration(initial_config);
177
178 // Wait for initial config to be applied
179- initial_condition.wait_for(std::chrono::seconds{1});
180+ initial_condition.wait_for(timeout);
181 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
182 ASSERT_TRUE(initial_condition.raised());
183 }
184@@ -1024,7 +1024,7 @@
185
186 display.emit_configuration_change_event(expect_config);
187
188- condition.wait_for(std::chrono::seconds{1});
189+ condition.wait_for(timeout);
190 EXPECT_TRUE(condition.raised());
191 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
192 }
193@@ -1054,7 +1054,7 @@
194
195 display.emit_configuration_change_event(expect_config);
196
197- condition.wait_for(std::chrono::seconds{1});
198+ condition.wait_for(timeout);
199 EXPECT_TRUE(condition.raised());
200 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
201 }
202@@ -1074,7 +1074,7 @@
203
204 display.emit_configuration_change_event(new_config);
205
206- client_config_changed.wait_for(std::chrono::seconds{1});
207+ client_config_changed.wait_for(timeout);
208
209 ASSERT_TRUE(client_config_changed.raised());
210
211@@ -1109,7 +1109,7 @@
212 nested_mir.server.the_display_configuration_controller()->set_base_configuration(initial_config);
213
214 // Wait for initial config to be applied
215- initial_condition.wait_for(std::chrono::seconds{1});
216+ initial_condition.wait_for(timeout);
217 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
218 ASSERT_TRUE(initial_condition.raised());
219 }
220@@ -1131,7 +1131,7 @@
221
222 display.emit_configuration_change_event(new_config);
223
224- condition.wait_for(std::chrono::seconds{1});
225+ condition.wait_for(timeout);
226 EXPECT_TRUE(condition.raised());
227 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
228 }
229@@ -1167,7 +1167,7 @@
230
231 display.emit_configuration_change_event(new_config);
232
233- condition.wait_for(std::chrono::seconds{1});
234+ condition.wait_for(timeout);
235 EXPECT_TRUE(condition.raised());
236 Mock::VerifyAndClearExpectations(the_mock_display_configuration_report().get());
237 }
238@@ -1192,7 +1192,7 @@
239
240 display.emit_configuration_change_event(new_config);
241
242- condition.wait_for(std::chrono::seconds{1});
243+ condition.wait_for(timeout);
244
245 EXPECT_TRUE(condition.raised());
246
247@@ -1235,14 +1235,14 @@
248
249 display.emit_configuration_change_event(new_hw_config);
250
251- client_config_changed.wait_for(std::chrono::seconds{1});
252+ client_config_changed.wait_for(timeout);
253 if (client_config_changed.raised())
254 {
255 client.update_display_configuration(
256 [](MirDisplayConfiguration* config) { config->outputs->orientation = mir_orientation_inverted; });
257 }
258
259- host_config_change.wait_for(std::chrono::seconds{1});
260+ host_config_change.wait_for(timeout);
261
262 EXPECT_TRUE(host_config_change.raised());
263 Mock::VerifyAndClearExpectations(&display);

Subscribers

People subscribed via source and target branches