Merge into development-branch : system-performance : Code : Mir

Status:	Merged
Approved by:	Daniel van Vugt on 2016-09-19
Approved revision:	no longer in the source branch.
Merged at revision:	3708
Proposed branch:	lp:~vanvugt/mir/system-performance
Merge into:	lp:mir
Diff against target:	478 lines (+253/-164) 4 files modified tests/performance-tests/CMakeLists.txt (+5/-0) tests/performance-tests/system_performance_test.cpp (+191/-0) tests/performance-tests/system_performance_test.h (+46/-0) tests/performance-tests/test_compositor.cpp (+11/-164)
To merge this branch:	bzr merge lp:~vanvugt/mir/system-performance
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Mir CI Bot	continuous-integration		Approve on 2016-09-19
Cemil Azizoglu (community)		2016-09-16	Approve on 2016-09-16
Review via email: mp+305923@code.launchpad.net

Revision history for this message

Mir CI Bot (mir-ci-bot) wrote on 2016-09-16:

#

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/1744/rebuild

PASSED: Continuous integration, rev:3707
https://mir-jenkins.ubuntu.com/job/mir-ci/1744/
Executed test runs:
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-mir/2184
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/2247
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/2238
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial+overlay/2238
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=yakkety/2238
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=yakkety/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=yakkety/2212/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial+overlay/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial+overlay/2212/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=yakkety/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=yakkety/2212/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/2212/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial+overlay/2212
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial+overlay/2212/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/1744/rebuild

review: Approve (continuous-integration)

Revision history for this message

Cemil Azizoglu (cemil-azizoglu) wrote on 2016-09-16:

#

ok

review: Approve

Revision history for this message

Mir CI Bot (mir-ci-bot) wrote on 2016-09-19:

#

review: Needs Fixing (continuous-integration)

Revision history for this message

Daniel van Vugt (vanvugt) wrote on 2016-09-19:

#

^^^
Bug 1523621, unrelated.

Revision history for this message

Mir CI Bot (mir-ci-bot) wrote on 2016-09-19:

#

review: Needs Fixing (continuous-integration)

Revision history for this message

Daniel van Vugt (vanvugt) wrote on 2016-09-19:

#

^^^
Bug 1616312 now

Revision history for this message

Mir CI Bot (mir-ci-bot) on 2016-09-19:

#

review: Approve (continuous-integration)

Mir

Merge lp:~vanvugt/mir/system-performance into lp:mir

Commit message

Description of the change

Preview Diff

Subscribers

 === modified file 'tests/performance-tests/CMakeLists.txt'
 --- tests/performance-tests/CMakeLists.txt	2016-08-24 02:09:08 +0000
 +++ tests/performance-tests/CMakeLists.txt	2016-09-16 10:11:25 +0000
@@ -27,11 +27,16 @@
    ${GTEST_BOTH_LIBRARIES}
+ )
++add_library(mir_system_performance_test STATIC
++  system_performance_test.cpp
++)
++
  mir_add_wrapped_executable(mir_compositor_performance_test
    test_compositor.cpp
+ )
  target_link_libraries(mir_compositor_performance_test
    ${GTEST_BOTH_LIBRARIES}
++  mir_system_performance_test
+ )
  mir_add_wrapped_executable(mir_client_startup_performance_test
 === added file 'tests/performance-tests/system_performance_test.cpp'
 --- tests/performance-tests/system_performance_test.cpp	1970-01-01 00:00:00 +0000
 +++ tests/performance-tests/system_performance_test.cpp	2016-09-16 10:11:25 +0000
@@ -0,0 +1,191 @@
++/*
++ * Copyright © 2016 Canonical Ltd.
++ *
++ * This program is free software: you can redistribute it and/or modify it
++ * under the terms of the GNU General Public License version 3,
++ * as published by the Free Software Foundation.
++ *
++ * This program is distributed in the hope that it will be useful,
++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
++ * GNU General Public License for more details.
++ *
++ * You should have received a copy of the GNU General Public License
++ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
++ *
++ * Author: Daniel van Vugt <daniel.van.vugt@canonical.com>
++ */
++
++#include "system_performance_test.h"
++#include <cstdio>
++#include <cstdlib>
++#include <cstring>
++#include <unistd.h>
++#include <string>
++#include <thread>
++
++using namespace std::literals::chrono_literals;
++
++namespace
++{
++
++std::string mir_bin_dir()
++{
++    char path[256];
++    auto len = readlink("/proc/self/exe", path, sizeof(path)-1);
++    if (len < 0)
++        len = 0;
++    path[len] = '\0';
++    if (auto slash = strrchr(path, '/'))
++        *slash = '\0';
++    return path;
++}
++
++void kill_nicely(pid_t pid)
++{
++    if (kill(pid, SIGTERM) < 0)
++        return;
++    int const timeout = 5;
++    int status, count = 0;
++    while (0 == waitpid(pid, &status, WNOHANG) && count < timeout)
++    {
++        sleep(1);
++        ++count;
++    }
++    kill(pid, SIGKILL);
++}
++
++int exec_cmd(char const* cmd)
++{
++    auto buf = strdup(cmd);
++    size_t argc = 1;
++    char* argv[256] = {buf};
++    char *c = buf;
++    while (*c)
++    {
++        if (*c == ' ')
++        {
++            *c = '\0';
++            if (argc < (sizeof(argv)/sizeof(argv[0]) - 1))
++            {
++                argv[argc] = c + 1;
++                ++argc;
++            }
++        }
++        ++c;
++    }
++    argv[argc] = NULL;
++    return execv(argv[0], argv);
++}
++
++FILE* popen_with_pid(char const* cmd, pid_t& pid)
++{
++    int pipefd[2];
++    if (pipe(pipefd))
++        return NULL;
++
++    int const& pipe_out = pipefd[0];
++    int const& pipe_in = pipefd[1];
++
++    pid = fork();
++    if (pid < 0)
++    {
++        close(pipe_in);
++        close(pipe_out);
++        return NULL;
++    }
++    else if (pid == 0)
++    {
++        close(pipe_out);
++        dup2(pipe_in, 1);  // Child stdout goes into pipe_in
++        close(pipe_in);
++        exec_cmd(cmd);
++        exit(errno);
++    }
++    else
++    {
++        close(pipe_in);
++    }
++
++    return fdopen(pipe_out, "r");
++}
++
++bool spawn_and_forget(char const* cmd)
++{
++    int pid = fork();
++    if (pid == 0)
++    {
++        // Silence stdout/stderr
++        close(1);
++        close(2);
++        exec_cmd(cmd);
++        exit(errno);
++    }
++    return (pid > 0);
++}
++
++bool spawn_and_forget(std::string const& cmd)
++{
++    return spawn_and_forget(cmd.c_str());
++}
++
++bool wait_for_file(char const* path, std::chrono::seconds timeout)
++{
++    struct stat s;
++    int count = 0, max = timeout.count();
++    int ret;
++    while ((ret = stat(path, &s)) < 0 && errno == ENOENT && count < max)
++    {
++        ++count;
++        std::this_thread::sleep_for(1s);
++    }
++    return ret == 0;
++}
++
++} // anonymous namespace
++
++namespace mir { namespace test {
++
++SystemPerformanceTest::SystemPerformanceTest() : bin_dir{mir_bin_dir()}
++{
++}
++
++void SystemPerformanceTest::SetUp(std::string const server_args)
++{
++    auto const mir_sock = "/tmp/mir_test_socket_"+std::to_string(getpid());
++    auto const server_cmd =
++        bin_dir+"/mir_demo_server -f "+mir_sock+" "+server_args;
++
++    server_output = popen_with_pid(server_cmd.c_str(), server_pid);
++    ASSERT_TRUE(server_output) << server_cmd;
++    ASSERT_TRUE(wait_for_file(mir_sock.c_str(), 5s)) << server_cmd;
++    setenv("MIR_SOCKET", mir_sock.c_str(), 1);
++}
++
++void SystemPerformanceTest::TearDown()
++{
++    kill_nicely(server_pid);
++    fclose(server_output);
++}
++
++void SystemPerformanceTest::spawn_clients(std::initializer_list<std::string> clients)
++{
++    for (auto& client : clients)
++    {
++        spawn_and_forget(bin_dir+"/"+client);
++        std::this_thread::sleep_for(100ms);
++    }
++}
++
++void SystemPerformanceTest::run_server_for(std::chrono::seconds timeout)
++{
++    pid_t pid = server_pid;
++    std::thread killer([timeout,pid]()
++    {
++        std::this_thread::sleep_for(timeout);
++        kill_nicely(pid);
++    });
++    killer.detach();
++}
++
++} } // namespace mir::test
 === added file 'tests/performance-tests/system_performance_test.h'
 --- tests/performance-tests/system_performance_test.h	1970-01-01 00:00:00 +0000
 +++ tests/performance-tests/system_performance_test.h	2016-09-16 10:11:25 +0000
@@ -0,0 +1,46 @@
++/*
++ * Copyright © 2016 Canonical Ltd.
++ *
++ * This program is free software: you can redistribute it and/or modify it
++ * under the terms of the GNU General Public License version 3,
++ * as published by the Free Software Foundation.
++ *
++ * This program is distributed in the hope that it will be useful,
++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
++ * GNU General Public License for more details.
++ *
++ * You should have received a copy of the GNU General Public License
++ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
++ *
++ * Author: Daniel van Vugt <daniel.van.vugt@canonical.com>
++ */
++
++#ifndef MIR_TEST_SYSTEM_PERFORMANCE_TEST_H_
++#define MIR_TEST_SYSTEM_PERFORMANCE_TEST_H_
++
++#include <gtest/gtest.h>
++#include <string>
++#include <chrono>
++#include <initializer_list>
++
++namespace mir { namespace test {
++
++class SystemPerformanceTest : public testing::Test
++{
++protected:
++    SystemPerformanceTest();
++    void SetUp(std::string const server_args);
++    void TearDown() override;
++    void spawn_clients(std::initializer_list<std::string> clients);
++    void run_server_for(std::chrono::seconds timeout);
++
++    FILE* server_output;
++private:
++    std::string const bin_dir;
++    pid_t server_pid = 0;
++};
++
++} } // namespace mir::test
++
++#endif // MIR_TEST_SYSTEM_PERFORMANCE_TEST_H_
 === modified file 'tests/performance-tests/test_compositor.cpp'
 --- tests/performance-tests/test_compositor.cpp	2016-05-09 02:49:10 +0000
 +++ tests/performance-tests/test_compositor.cpp	2016-09-16 10:11:25 +0000
@@ -16,174 +16,23 @@
   * Author: Daniel van Vugt <daniel.van.vugt@canonical.com>
   */
--#include <gtest/gtest.h>
--#include <cstdio>
--#include <cstdlib>
--#include <cstring>
--#include <unistd.h>
--#include <string>
--#include <thread>
--#include <initializer_list>
++#include "system_performance_test.h"
  using namespace std::literals::chrono_literals;
++using namespace mir::test;
  namespace
+ {
--
--std::string mir_bin_dir()
--{
--    char path[256];
--    auto len = readlink("/proc/self/exe", path, sizeof(path)-1);
--    if (len < 0)
--        len = 0;
--    path[len] = '\0';
--    if (auto slash = strrchr(path, '/'))
--        *slash = '\0';
--    return path;
--}
--
--void kill_nicely(pid_t pid)
--{
--    if (kill(pid, SIGTERM) < 0)
--        return;
--    int const timeout = 5;
--    int status, count = 0;
--    while (0 == waitpid(pid, &status, WNOHANG) && count < timeout)
--    {
--        sleep(1);
--        ++count;
--    }
--    kill(pid, SIGKILL);
--}
--
--int exec_cmd(char const* cmd)
--{
--    auto buf = strdup(cmd);
--    size_t argc = 1;
--    char* argv[256] = {buf};
--    char *c = buf;
--    while (*c)
--    {
--        if (*c == ' ')
--        {
--            *c = '\0';
--            if (argc < (sizeof(argv)/sizeof(argv[0]) - 1))
--            {
--                argv[argc] = c + 1;
--                ++argc;
--            }
--        }
--        ++c;
--    }
--    argv[argc] = NULL;
--    return execv(argv[0], argv);
--}
--
--FILE* popen_with_pid(char const* cmd, pid_t& pid)
--{
--    int pipefd[2];
--    if (pipe(pipefd))
--        return NULL;
--
--    int const& pipe_out = pipefd[0];
--    int const& pipe_in = pipefd[1];
--
--    pid = fork();
--    if (pid < 0)
--    {
--        close(pipe_in);
--        close(pipe_out);
--        return NULL;
--    }
--    else if (pid == 0)
--    {
--        close(pipe_out);
--        dup2(pipe_in, 1);  // Child stdout goes into pipe_in
--        close(pipe_in);
--        exec_cmd(cmd);
--        exit(errno);
--    }
--    else
--    {
--        close(pipe_in);
--    }
--
--    return fdopen(pipe_out, "r");
--}
--
--bool spawn_and_forget(char const* cmd)
--{
--    int pid = fork();
--    if (pid == 0)
--    {
--        // Silence stdout/stderr
--        close(1);
--        close(2);
--        exec_cmd(cmd);
--        exit(errno);
--    }
--    return (pid > 0);
--}
--
--bool spawn_and_forget(std::string const& cmd)
--{
--    return spawn_and_forget(cmd.c_str());
--}
--
--bool wait_for_file(char const* path, std::chrono::seconds timeout)
--{
--    struct stat s;
--    int count = 0, max = timeout.count();
--    int ret;
--    while ((ret = stat(path, &s)) < 0 && errno == ENOENT && count < max)
--    {
--        ++count;
--        std::this_thread::sleep_for(1s);
--    }
--    return ret == 0;
--}
--
--struct CompositorPerformance : testing::Test
++struct CompositorPerformance : SystemPerformanceTest
+ {
      void SetUp() override
+     {
          compositor_fps = compositor_render_time = -1.0f;
--
--        auto const mir_sock = "/tmp/mir_test_socket_"+std::to_string(getpid());
--        auto const server_cmd =
--            bin_dir+"/mir_demo_server --compositor-report=log -f "+mir_sock;
--
--        server_output = popen_with_pid(server_cmd.c_str(), server_pid);
--        ASSERT_TRUE(server_output) << server_cmd;
--        ASSERT_TRUE(wait_for_file(mir_sock.c_str(), 5s)) << server_cmd;
--        setenv("MIR_SOCKET", mir_sock.c_str(), 1);
--    }
--
--    void TearDown() override
--    {
--        kill_nicely(server_pid);
--        fclose(server_output);
--    }
--
--    void spawn_clients(std::initializer_list<std::string> clients)
--    {
--        for (auto& client : clients)
--        {
--            spawn_and_forget(bin_dir+"/"+client);
--            std::this_thread::sleep_for(100ms);
--        }
--    }
--
--    void wait_for_server(std::chrono::seconds timeout)
--    {
--        pid_t pid = server_pid;
--        std::thread killer([timeout,pid]()
--        {
--            std::this_thread::sleep_for(timeout);
--            kill_nicely(pid);
--        });
--        killer.detach();
--
++        SystemPerformanceTest::SetUp("--compositor-report=log");
++    }
++
++    void read_compositor_report()
++    {
          char line[256];
          while (fgets(line, sizeof(line), server_output))
+         {
@@ -200,12 +49,8 @@
+         }
+     }
--    std::string const bin_dir{mir_bin_dir()};
--    pid_t server_pid = 0;
--    FILE* server_output;
      float compositor_fps, compositor_render_time;
  };
--
  } // anonymous namespace
  TEST_F(CompositorPerformance, regression_test_1563287)
@@ -216,7 +61,9 @@
                     "mir_demo_client_scroll",
                     "mir_demo_client_egltriangle -b0.5",
                     "mir_demo_client_multiwin"});
--    wait_for_server(10s);
++    run_server_for(10s);
++
++    read_compositor_report();
      EXPECT_GE(compositor_fps, 58.0f);
      EXPECT_LT(compositor_render_time, 17.0f);
+ }