Mir

Merge lp:~vanvugt/mir/system-performance into lp:mir

Proposed by Daniel van Vugt
Status: Merged
Approved by: Daniel van Vugt
Approved revision: no longer in the source branch.
Merged at revision: 3708
Proposed branch: lp:~vanvugt/mir/system-performance
Merge into: lp:mir
Diff against target: 478 lines (+253/-164)
4 files modified
tests/performance-tests/CMakeLists.txt (+5/-0)
tests/performance-tests/system_performance_test.cpp (+191/-0)
tests/performance-tests/system_performance_test.h (+46/-0)
tests/performance-tests/test_compositor.cpp (+11/-164)
To merge this branch: bzr merge lp:~vanvugt/mir/system-performance
Reviewer Review Type Date Requested Status
Mir CI Bot continuous-integration Approve
Cemil Azizoglu (community) Approve
Review via email: mp+305923@code.launchpad.net

Commit message

Generalise CompositorTest into "SystemPerformanceTest" so we can reuse
it for non-compositor system performance testing.

To post a comment you must log in.
Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

PASSED: Continuous integration, rev:3707
https://mir-jenkins.ubuntu.com/job/mir-ci/1744/
Executed test runs:
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-mir/2184
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/2247
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/2238
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial+overlay/2238
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=yakkety/2238
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=yakkety/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=yakkety/2212/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial+overlay/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial+overlay/2212/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=yakkety/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=yakkety/2212/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/2212/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial+overlay/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial+overlay/2212/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/1744/rebuild

review: Approve (continuous-integration)
Revision history for this message
Cemil Azizoglu (cemil-azizoglu) wrote :

ok

review: Approve
Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

FAILED: Autolanding.
More details in the following jenkins job:
https://mir-jenkins.ubuntu.com/job/mir-autolanding/595/
Executed test runs:
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-mir/2189/console
    None: https://mir-jenkins.ubuntu.com/job/generic-land-mp/634/console
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/2252
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/2243
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial+overlay/2243
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=yakkety/2243
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=yakkety/2217
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=yakkety/2217/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial+overlay/2217
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial+overlay/2217/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=yakkety/2217
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=yakkety/2217/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/2217
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/2217/artifact/output/*zip*/output.zip
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial+overlay/2217/console

review: Needs Fixing (continuous-integration)
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

^^^
Bug 1523621, unrelated.

Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

FAILED: Autolanding.
More details in the following jenkins job:
https://mir-jenkins.ubuntu.com/job/mir-autolanding/596/
Executed test runs:
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-mir/2192/console
    None: https://mir-jenkins.ubuntu.com/job/generic-land-mp/635/console
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/2255
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/2246
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial+overlay/2246
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=yakkety/2246
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=yakkety/2220
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=yakkety/2220/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial+overlay/2220
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial+overlay/2220/artifact/output/*zip*/output.zip
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=yakkety/2220/console
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/2220
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/2220/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial+overlay/2220
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial+overlay/2220/artifact/output/*zip*/output.zip

review: Needs Fixing (continuous-integration)
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

^^^
Bug 1616312 now

Revision history for this message
Mir CI Bot (mir-ci-bot) :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'tests/performance-tests/CMakeLists.txt'
--- tests/performance-tests/CMakeLists.txt 2016-08-24 02:09:08 +0000
+++ tests/performance-tests/CMakeLists.txt 2016-09-16 10:11:25 +0000
@@ -27,11 +27,16 @@
27 ${GTEST_BOTH_LIBRARIES}27 ${GTEST_BOTH_LIBRARIES}
28)28)
2929
30add_library(mir_system_performance_test STATIC
31 system_performance_test.cpp
32)
33
30mir_add_wrapped_executable(mir_compositor_performance_test34mir_add_wrapped_executable(mir_compositor_performance_test
31 test_compositor.cpp35 test_compositor.cpp
32)36)
33target_link_libraries(mir_compositor_performance_test37target_link_libraries(mir_compositor_performance_test
34 ${GTEST_BOTH_LIBRARIES}38 ${GTEST_BOTH_LIBRARIES}
39 mir_system_performance_test
35)40)
3641
37mir_add_wrapped_executable(mir_client_startup_performance_test42mir_add_wrapped_executable(mir_client_startup_performance_test
3843
=== added file 'tests/performance-tests/system_performance_test.cpp'
--- tests/performance-tests/system_performance_test.cpp 1970-01-01 00:00:00 +0000
+++ tests/performance-tests/system_performance_test.cpp 2016-09-16 10:11:25 +0000
@@ -0,0 +1,191 @@
1/*
2 * Copyright © 2016 Canonical Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3,
6 * as published by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful,
9 * but WITHOUT ANY WARRANTY; without even the implied warranty of
10 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11 * GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Author: Daniel van Vugt <daniel.van.vugt@canonical.com>
17 */
18
19#include "system_performance_test.h"
20#include <cstdio>
21#include <cstdlib>
22#include <cstring>
23#include <unistd.h>
24#include <string>
25#include <thread>
26
27using namespace std::literals::chrono_literals;
28
29namespace
30{
31
32std::string mir_bin_dir()
33{
34 char path[256];
35 auto len = readlink("/proc/self/exe", path, sizeof(path)-1);
36 if (len < 0)
37 len = 0;
38 path[len] = '\0';
39 if (auto slash = strrchr(path, '/'))
40 *slash = '\0';
41 return path;
42}
43
44void kill_nicely(pid_t pid)
45{
46 if (kill(pid, SIGTERM) < 0)
47 return;
48 int const timeout = 5;
49 int status, count = 0;
50 while (0 == waitpid(pid, &status, WNOHANG) && count < timeout)
51 {
52 sleep(1);
53 ++count;
54 }
55 kill(pid, SIGKILL);
56}
57
58int exec_cmd(char const* cmd)
59{
60 auto buf = strdup(cmd);
61 size_t argc = 1;
62 char* argv[256] = {buf};
63 char *c = buf;
64 while (*c)
65 {
66 if (*c == ' ')
67 {
68 *c = '\0';
69 if (argc < (sizeof(argv)/sizeof(argv[0]) - 1))
70 {
71 argv[argc] = c + 1;
72 ++argc;
73 }
74 }
75 ++c;
76 }
77 argv[argc] = NULL;
78 return execv(argv[0], argv);
79}
80
81FILE* popen_with_pid(char const* cmd, pid_t& pid)
82{
83 int pipefd[2];
84 if (pipe(pipefd))
85 return NULL;
86
87 int const& pipe_out = pipefd[0];
88 int const& pipe_in = pipefd[1];
89
90 pid = fork();
91 if (pid < 0)
92 {
93 close(pipe_in);
94 close(pipe_out);
95 return NULL;
96 }
97 else if (pid == 0)
98 {
99 close(pipe_out);
100 dup2(pipe_in, 1); // Child stdout goes into pipe_in
101 close(pipe_in);
102 exec_cmd(cmd);
103 exit(errno);
104 }
105 else
106 {
107 close(pipe_in);
108 }
109
110 return fdopen(pipe_out, "r");
111}
112
113bool spawn_and_forget(char const* cmd)
114{
115 int pid = fork();
116 if (pid == 0)
117 {
118 // Silence stdout/stderr
119 close(1);
120 close(2);
121 exec_cmd(cmd);
122 exit(errno);
123 }
124 return (pid > 0);
125}
126
127bool spawn_and_forget(std::string const& cmd)
128{
129 return spawn_and_forget(cmd.c_str());
130}
131
132bool wait_for_file(char const* path, std::chrono::seconds timeout)
133{
134 struct stat s;
135 int count = 0, max = timeout.count();
136 int ret;
137 while ((ret = stat(path, &s)) < 0 && errno == ENOENT && count < max)
138 {
139 ++count;
140 std::this_thread::sleep_for(1s);
141 }
142 return ret == 0;
143}
144
145} // anonymous namespace
146
147namespace mir { namespace test {
148
149SystemPerformanceTest::SystemPerformanceTest() : bin_dir{mir_bin_dir()}
150{
151}
152
153void SystemPerformanceTest::SetUp(std::string const server_args)
154{
155 auto const mir_sock = "/tmp/mir_test_socket_"+std::to_string(getpid());
156 auto const server_cmd =
157 bin_dir+"/mir_demo_server -f "+mir_sock+" "+server_args;
158
159 server_output = popen_with_pid(server_cmd.c_str(), server_pid);
160 ASSERT_TRUE(server_output) << server_cmd;
161 ASSERT_TRUE(wait_for_file(mir_sock.c_str(), 5s)) << server_cmd;
162 setenv("MIR_SOCKET", mir_sock.c_str(), 1);
163}
164
165void SystemPerformanceTest::TearDown()
166{
167 kill_nicely(server_pid);
168 fclose(server_output);
169}
170
171void SystemPerformanceTest::spawn_clients(std::initializer_list<std::string> clients)
172{
173 for (auto& client : clients)
174 {
175 spawn_and_forget(bin_dir+"/"+client);
176 std::this_thread::sleep_for(100ms);
177 }
178}
179
180void SystemPerformanceTest::run_server_for(std::chrono::seconds timeout)
181{
182 pid_t pid = server_pid;
183 std::thread killer([timeout,pid]()
184 {
185 std::this_thread::sleep_for(timeout);
186 kill_nicely(pid);
187 });
188 killer.detach();
189}
190
191} } // namespace mir::test
0192
=== added file 'tests/performance-tests/system_performance_test.h'
--- tests/performance-tests/system_performance_test.h 1970-01-01 00:00:00 +0000
+++ tests/performance-tests/system_performance_test.h 2016-09-16 10:11:25 +0000
@@ -0,0 +1,46 @@
1/*
2 * Copyright © 2016 Canonical Ltd.
3 *
4 * This program is free software: you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 3,
6 * as published by the Free Software Foundation.
7 *
8 * This program is distributed in the hope that it will be useful,
9 * but WITHOUT ANY WARRANTY; without even the implied warranty of
10 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11 * GNU General Public License for more details.
12 *
13 * You should have received a copy of the GNU General Public License
14 * along with this program. If not, see <http://www.gnu.org/licenses/>.
15 *
16 * Author: Daniel van Vugt <daniel.van.vugt@canonical.com>
17 */
18
19#ifndef MIR_TEST_SYSTEM_PERFORMANCE_TEST_H_
20#define MIR_TEST_SYSTEM_PERFORMANCE_TEST_H_
21
22#include <gtest/gtest.h>
23#include <string>
24#include <chrono>
25#include <initializer_list>
26
27namespace mir { namespace test {
28
29class SystemPerformanceTest : public testing::Test
30{
31protected:
32 SystemPerformanceTest();
33 void SetUp(std::string const server_args);
34 void TearDown() override;
35 void spawn_clients(std::initializer_list<std::string> clients);
36 void run_server_for(std::chrono::seconds timeout);
37
38 FILE* server_output;
39private:
40 std::string const bin_dir;
41 pid_t server_pid = 0;
42};
43
44} } // namespace mir::test
45
46#endif // MIR_TEST_SYSTEM_PERFORMANCE_TEST_H_
047
=== modified file 'tests/performance-tests/test_compositor.cpp'
--- tests/performance-tests/test_compositor.cpp 2016-05-09 02:49:10 +0000
+++ tests/performance-tests/test_compositor.cpp 2016-09-16 10:11:25 +0000
@@ -16,174 +16,23 @@
16 * Author: Daniel van Vugt <daniel.van.vugt@canonical.com>16 * Author: Daniel van Vugt <daniel.van.vugt@canonical.com>
17 */17 */
1818
19#include <gtest/gtest.h>19#include "system_performance_test.h"
20#include <cstdio>
21#include <cstdlib>
22#include <cstring>
23#include <unistd.h>
24#include <string>
25#include <thread>
26#include <initializer_list>
2720
28using namespace std::literals::chrono_literals;21using namespace std::literals::chrono_literals;
22using namespace mir::test;
2923
30namespace24namespace
31{25{
3226struct CompositorPerformance : SystemPerformanceTest
33std::string mir_bin_dir()
34{
35 char path[256];
36 auto len = readlink("/proc/self/exe", path, sizeof(path)-1);
37 if (len < 0)
38 len = 0;
39 path[len] = '\0';
40 if (auto slash = strrchr(path, '/'))
41 *slash = '\0';
42 return path;
43}
44
45void kill_nicely(pid_t pid)
46{
47 if (kill(pid, SIGTERM) < 0)
48 return;
49 int const timeout = 5;
50 int status, count = 0;
51 while (0 == waitpid(pid, &status, WNOHANG) && count < timeout)
52 {
53 sleep(1);
54 ++count;
55 }
56 kill(pid, SIGKILL);
57}
58
59int exec_cmd(char const* cmd)
60{
61 auto buf = strdup(cmd);
62 size_t argc = 1;
63 char* argv[256] = {buf};
64 char *c = buf;
65 while (*c)
66 {
67 if (*c == ' ')
68 {
69 *c = '\0';
70 if (argc < (sizeof(argv)/sizeof(argv[0]) - 1))
71 {
72 argv[argc] = c + 1;
73 ++argc;
74 }
75 }
76 ++c;
77 }
78 argv[argc] = NULL;
79 return execv(argv[0], argv);
80}
81
82FILE* popen_with_pid(char const* cmd, pid_t& pid)
83{
84 int pipefd[2];
85 if (pipe(pipefd))
86 return NULL;
87
88 int const& pipe_out = pipefd[0];
89 int const& pipe_in = pipefd[1];
90
91 pid = fork();
92 if (pid < 0)
93 {
94 close(pipe_in);
95 close(pipe_out);
96 return NULL;
97 }
98 else if (pid == 0)
99 {
100 close(pipe_out);
101 dup2(pipe_in, 1); // Child stdout goes into pipe_in
102 close(pipe_in);
103 exec_cmd(cmd);
104 exit(errno);
105 }
106 else
107 {
108 close(pipe_in);
109 }
110
111 return fdopen(pipe_out, "r");
112}
113
114bool spawn_and_forget(char const* cmd)
115{
116 int pid = fork();
117 if (pid == 0)
118 {
119 // Silence stdout/stderr
120 close(1);
121 close(2);
122 exec_cmd(cmd);
123 exit(errno);
124 }
125 return (pid > 0);
126}
127
128bool spawn_and_forget(std::string const& cmd)
129{
130 return spawn_and_forget(cmd.c_str());
131}
132
133bool wait_for_file(char const* path, std::chrono::seconds timeout)
134{
135 struct stat s;
136 int count = 0, max = timeout.count();
137 int ret;
138 while ((ret = stat(path, &s)) < 0 && errno == ENOENT && count < max)
139 {
140 ++count;
141 std::this_thread::sleep_for(1s);
142 }
143 return ret == 0;
144}
145
146struct CompositorPerformance : testing::Test
147{27{
148 void SetUp() override28 void SetUp() override
149 {29 {
150 compositor_fps = compositor_render_time = -1.0f;30 compositor_fps = compositor_render_time = -1.0f;
15131 SystemPerformanceTest::SetUp("--compositor-report=log");
152 auto const mir_sock = "/tmp/mir_test_socket_"+std::to_string(getpid());32 }
153 auto const server_cmd =33
154 bin_dir+"/mir_demo_server --compositor-report=log -f "+mir_sock;34 void read_compositor_report()
155 35 {
156 server_output = popen_with_pid(server_cmd.c_str(), server_pid);
157 ASSERT_TRUE(server_output) << server_cmd;
158 ASSERT_TRUE(wait_for_file(mir_sock.c_str(), 5s)) << server_cmd;
159 setenv("MIR_SOCKET", mir_sock.c_str(), 1);
160 }
161
162 void TearDown() override
163 {
164 kill_nicely(server_pid);
165 fclose(server_output);
166 }
167
168 void spawn_clients(std::initializer_list<std::string> clients)
169 {
170 for (auto& client : clients)
171 {
172 spawn_and_forget(bin_dir+"/"+client);
173 std::this_thread::sleep_for(100ms);
174 }
175 }
176
177 void wait_for_server(std::chrono::seconds timeout)
178 {
179 pid_t pid = server_pid;
180 std::thread killer([timeout,pid]()
181 {
182 std::this_thread::sleep_for(timeout);
183 kill_nicely(pid);
184 });
185 killer.detach();
186
187 char line[256];36 char line[256];
188 while (fgets(line, sizeof(line), server_output))37 while (fgets(line, sizeof(line), server_output))
189 {38 {
@@ -200,12 +49,8 @@
200 }49 }
201 }50 }
20251
203 std::string const bin_dir{mir_bin_dir()};
204 pid_t server_pid = 0;
205 FILE* server_output;
206 float compositor_fps, compositor_render_time;52 float compositor_fps, compositor_render_time;
207};53};
208
209} // anonymous namespace54} // anonymous namespace
21055
211TEST_F(CompositorPerformance, regression_test_1563287)56TEST_F(CompositorPerformance, regression_test_1563287)
@@ -216,7 +61,9 @@
216 "mir_demo_client_scroll",61 "mir_demo_client_scroll",
217 "mir_demo_client_egltriangle -b0.5",62 "mir_demo_client_egltriangle -b0.5",
218 "mir_demo_client_multiwin"});63 "mir_demo_client_multiwin"});
219 wait_for_server(10s);64 run_server_for(10s);
65
66 read_compositor_report();
220 EXPECT_GE(compositor_fps, 58.0f);67 EXPECT_GE(compositor_fps, 58.0f);
221 EXPECT_LT(compositor_render_time, 17.0f);68 EXPECT_LT(compositor_render_time, 17.0f);
222}69}

Subscribers

People subscribed via source and target branches