Merge lp:~albaguirre/mir/simple-startup-perf-test into lp:mir

[Alberto Aguirre]

Add a simple client startup performance test

Add test to ensure the required time for a client to setup a connection,
create a surface and swap a buffer does not exceed the expected threshold.

[Mir CI Bot (mir-ci-bot)]

FAILED: Continuous integration, rev:3504
https://mir-jenkins.ubuntu.com/job/mir-ci/968/
Executed test runs:
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-mir/1042/console
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/1089
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/1080
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial/1080
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1052/console
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1052/console
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1052/console
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1052/console
    FAILURE: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1052/console

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/968/rebuild

review: Needs Fixing (continuous-integration)

[Mir CI Bot (mir-ci-bot)]

PASSED: Continuous integration, rev:3505
https://mir-jenkins.ubuntu.com/job/mir-ci/969/
Executed test runs:
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-mir/1043
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/1090
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/1081
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial/1081
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1053
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1053/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1053
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1053/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1053
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1053/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1053
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1053/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1053
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1053/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/969/rebuild

review: Approve (continuous-integration)

[Daniel van Vugt (vanvugt)]

That's a good idea. Don't need to fuss over the code structure...

review: Approve

=== modified file 'debian/mir-test-tools.install'
--- debian/mir-test-tools.install	2016-05-06 08:57:57 +0000
+++ debian/mir-test-tools.install	2016-05-11 03:44:15 +0000
@@ -7,6 +7,7 @@
 usr/bin/mir_performance_tests
 usr/bin/mir_glmark2_performance_test
 usr/bin/mir_compositor_performance_test
+usr/bin/mir_client_startup_performance_test
 usr/bin/mir_privileged_tests
 usr/bin/mir_test_reload_protobuf
 usr/bin/mir_test_client_*

=== modified file 'tests/performance-tests/CMakeLists.txt'
--- tests/performance-tests/CMakeLists.txt	2016-05-06 07:34:57 +0000
+++ tests/performance-tests/CMakeLists.txt	2016-05-11 03:44:15 +0000
@@ -23,3 +23,15 @@
 target_link_libraries(mir_compositor_performance_test
   ${GTEST_BOTH_LIBRARIES}
 )
+
+mir_add_wrapped_executable(mir_client_startup_performance_test
+  test_client_startup.cpp
+)
+target_link_libraries(mir_client_startup_performance_test
+  mir-test-static
+  mir-test-framework-static
+  mir-test-doubles-static
+  mirserver
+  ${GTEST_BOTH_LIBRARIES}
+)
+

=== modified file 'tests/performance-tests/performance_tests.sh'
--- tests/performance-tests/performance_tests.sh	2016-05-06 07:34:57 +0000
+++ tests/performance-tests/performance_tests.sh	2016-05-11 03:44:15 +0000
@@ -31,7 +31,8 @@
 tests_run=0
 failures=0
 for perf_test in mir_glmark2_performance_test \
-                 mir_compositor_performance_test
+                 mir_compositor_performance_test \
+                 mir_client_startup_performance_test
 do
     title_banner "Mir Performance Tests now running: $perf_test"
     tests_run=$(($tests_run + 1))

=== added file 'tests/performance-tests/test_client_startup.cpp'
--- tests/performance-tests/test_client_startup.cpp	1970-01-01 00:00:00 +0000
+++ tests/performance-tests/test_client_startup.cpp	2016-05-11 03:44:15 +0000
@@ -0,0 +1,115 @@
+/*
+ * Copyright © 2016 Canonical Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 3,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "mir_test_framework/async_server_runner.h"
+#include "mir_toolkit/mir_client_library.h"
+
+#include "mir/test/validity_matchers.h"
+
+#include <gmock/gmock.h>
+#include <gtest/gtest.h>
+
+#include <chrono>
+
+namespace mtf = mir_test_framework;
+
+using namespace testing;
+
+namespace
+{
+struct ClientStartupPerformance : testing::Test, mtf::AsyncServerRunner
+{
+    void SetUp() override
+    {
+        start_server();
+    }
+
+    void TearDown() override
+    {
+        stop_server();
+    }
+
+    MirConnection* create_connection()
+    {
+        auto conn = mir_connect_sync(new_connection().c_str(), "Perf test");
+        if (!mir_connection_is_valid(conn))
+        {
+            std::string error_msg{"Could not create connection: "};
+            error_msg.append(mir_connection_get_error_message(conn));
+            throw std::runtime_error(error_msg);
+        }
+        return conn;
+    }
+};
+
+MirPixelFormat find_pixel_format(MirConnection* connection)
+{
+    MirPixelFormat pixel_format = mir_pixel_format_invalid;
+    unsigned int valid_formats;
+    mir_connection_get_available_surface_formats(connection, &pixel_format, 1, &valid_formats);
+    if (valid_formats < 1 || pixel_format == mir_pixel_format_invalid)
+        throw std::runtime_error("Could not find pixel format");
+    return pixel_format;
+}
+
+MirSurface* make_surface(MirConnection* connection)
+{
+    auto format = find_pixel_format(connection);
+
+    auto spec = mir_connection_create_spec_for_normal_surface(connection,
+            720, 1280, format);
+    auto surface = mir_surface_create_sync(spec);
+
+    mir_surface_spec_release(spec);
+
+    if (!mir_surface_is_valid(surface))
+    {
+        std::string error_msg{"Could not create surface: "};
+        error_msg.append(mir_surface_get_error_message(surface));
+        throw std::runtime_error(error_msg);
+    }
+    return surface;
+}
+}
+
+TEST_F(ClientStartupPerformance, create_surface_and_swap)
+{
+    using namespace std::chrono_literals;
+    auto start = std::chrono::steady_clock::now();
+
+    auto conn = create_connection();
+    auto surf = make_surface(conn);
+    auto stream  = mir_surface_get_buffer_stream(surf);
+    if (!mir_buffer_stream_is_valid(stream))
+    {
+        std::string error_msg{"Could not get buffer stream from surface: "};
+        error_msg.append(mir_buffer_stream_get_error_message(stream));
+        throw std::runtime_error(error_msg);
+    }
+
+    mir_buffer_stream_swap_buffers_sync(stream);
+
+    auto end = std::chrono::steady_clock::now();
+    auto diff = std::chrono::duration_cast<std::chrono::milliseconds>(end-start);
+
+    //NOTE: Ideally, the expected number should vary according to platform
+    auto max_expected_time = 80ms;
+    EXPECT_THAT(diff.count(), Lt(max_expected_time.count()));
+
+    mir_surface_release_sync(surf);
+    mir_connection_release(conn);
+}
+