Mir

Merge lp:~josharenson/mir/install_glmark2 into lp:mir

Proposed by Josh Arenson
Status: Superseded
Proposed branch: lp:~josharenson/mir/install_glmark2
Merge into: lp:mir
Diff against target: 157 lines (+119/-0)
4 files modified
debian/mir-test-tools.install (+1/-0)
tests/CMakeLists.txt (+5/-0)
tests/performance-tests/CMakeLists.txt (+28/-0)
tests/performance-tests/test_glmark2-es2-mir.cpp (+85/-0)
To merge this branch: bzr merge lp:~josharenson/mir/install_glmark2
Reviewer Review Type Date Requested Status
Alan Griffiths Needs Fixing
PS Jenkins bot (community) continuous-integration Needs Fixing
Alberto Aguirre (community) Needs Fixing
Robert Carr Pending
Alexandros Frantzis Pending
Mir development team Pending
Review via email: mp+218334@code.launchpad.net

This proposal supersedes a proposal from 2014-04-30.

This proposal has been superseded by a proposal from 2014-05-08.

Commit message

* Adds a 'performance test' category.
* Runs glmark2-es2-mir, storing the results to a local file for post-analysis

To post a comment you must log in.
Revision history for this message
Robert Carr (robertcarr) wrote : Posted in a previous version of this proposal

We don't need to link mir_performance_test against so much right?

review: Needs Fixing
Revision history for this message
Josh Arenson (josharenson) wrote : Posted in a previous version of this proposal

> We don't need to link mir_performance_test against so much right?
Absolutely.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote : Posted in a previous version of this proposal

Where should we source glmark2-es2-mir?

$ apt-cache search glmark2-es2-mir
$

review: Needs Information
Revision history for this message
Josh Arenson (josharenson) wrote : Posted in a previous version of this proposal

> Where should we source glmark2-es2-mir?
>
> $ apt-cache search glmark2-es2-mir
> $
glmark2-es2-mir (the version that actually runs) is currently located in ppa:mir-team/staging. I have also successfully built it from source. Currently, the Jenkins job adds the ppa, and installs the package as part of the environment set up.

Revision history for this message
Robert Carr (robertcarr) wrote : Posted in a previous version of this proposal

Why doesnt jenkins even say hello to this branch?

review: Needs Information
Revision history for this message
Josh Arenson (josharenson) wrote : Posted in a previous version of this proposal

> Why doesnt jenkins even say hello to this branch?
According to fginther, there is an outage.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal

FAILED: Continuous integration, rev:1572
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~josharenson/mir/install_glmark2/+merge/216504/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-ci/1388/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-android-trusty-i386-build/1689/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-clang-trusty-amd64-build/1687
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-trusty-touch/1262
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-trusty-amd64-ci/1120
        deb: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-trusty-amd64-ci/1120/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-trusty-armhf-ci/1125
        deb: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-trusty-armhf-ci/1125/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-trusty-armhf/1265
        deb: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-trusty-armhf/1265/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-runner-mako/1165
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/6335

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/mir-team-mir-development-branch-ci/1388/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Alexandros Frantzis (afrantzis) wrote : Posted in a previous version of this proposal

46 +find_program(GLMARK2_ES2_MIR_FOUND glmark2-es2-mir)
47 +
48 +if(NOT GLMARK2_ES2_MIR_FOUND)
49 + message("warning: glmark2-es2-mir not found, performance tests _may_ fail.")
50 +endif()

Since we are not running the program automatically when building, I am not sure how useful it is to print this message at build time.

77 +#include <stdlib.h>

We also need <stdio.h> for popen, getline etc (we shouldn't depend on other headers bringing it in).

Some style issues (see http://unity.ubuntu.com/mir/cppguide/index.html):

85 +class GLMark2Test : public ::testing::Test
86 +{
...

Mir uses 4 space indentations.

88 + enum ResultFileType {RAW, JSON};

Lowercase identifiers for enum values.

103 + while((read = getline(&line, &len, in)) != -1)
105 + if(file_type == RAW)
107 + else if(file_type == JSON)
109 + if(boost::regex_match(line, matches, re_glmark2_score))

Spaces between keyword (if,while) and opening parenthesis.

105 + if(file_type == RAW)
106 + glmark2_output << line;

Braces around statement, since the 'else' clause also has braces.

94 + const char *cmd
93 + FILE *in;
...

'*' goes with type in our C++ code and const goes after the type. So: "FILE* in" and "char const* cmd"

98 + std::ofstream glmark2_output;

No need to define all variables in the beginning of the code block if it is more convenient (or necessary for RAII purposes) not to do so. So, for example it is allowed to have "std::ofstream glmark2_output{output_filename};" after the ASSERT.

123 + glmark2_output.close();

No need to manually close the file, it's closed on std::ofstream destruction.

review: Needs Fixing
Revision history for this message
Daniel van Vugt (vanvugt) wrote : Posted in a previous version of this proposal

I have no opinion, but Jenkins won't ever land this till you add a commit message up the top.

review: Needs Fixing
Revision history for this message
Josh Arenson (josharenson) wrote : Posted in a previous version of this proposal

> I have no opinion, but Jenkins won't ever land this till you add a commit
> message up the top.
Thanks for the heads up. I'm working on a refactor and will add the message when done.

Revision history for this message
Alberto Aguirre (albaguirre) wrote : Posted in a previous version of this proposal

> > I have no opinion, but Jenkins won't ever land this till you add a commit
> > message up the top.
> Thanks for the heads up. I'm working on a refactor and will add the message
> when done.

You should mark this as "work in progress" so that others don't try to review code that is about to change.

Revision history for this message
Daniel van Vugt (vanvugt) : Posted in a previous version of this proposal
review: Abstain
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal

FAILED: Continuous integration, rev:1577
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~josharenson/mir/install_glmark2/+merge/216504/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-ci/1439/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-trusty-i386-build/1758
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-clang-trusty-amd64-build/1756
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-trusty-touch/1328
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-trusty-amd64-ci/1171
        deb: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-trusty-amd64-ci/1171/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-trusty-armhf-ci/1176
        deb: http://jenkins.qa.ubuntu.com/job/mir-team-mir-development-branch-trusty-armhf-ci/1176/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-trusty-armhf/1331
        deb: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-trusty-armhf/1331/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-runner-mako/1225
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/6538

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/mir-team-mir-development-branch-ci/1439/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Alexandros Frantzis (afrantzis) wrote : Posted in a previous version of this proposal

119 + /*HACK to wait for the server to start*/

You can override the mir::DefaultServerConfiguration::the_server_status_listener() and provide you own implementation to wait for server start.

/*HACK to stop the mir server*/

You can get the main loop object from mir::DefaultServerConfiguration::the_main_loop() and call the stop() method.

review: Needs Fixing
Revision history for this message
Alan Griffiths (alan-griffiths) wrote : Posted in a previous version of this proposal

> 119 + /*HACK to wait for the server to start*/
>
> You can override the
> mir::DefaultServerConfiguration::the_server_status_listener() and provide you
> own implementation to wait for server start.
>
> /*HACK to stop the mir server*/
>
> You can get the main loop object from
> mir::DefaultServerConfiguration::the_main_loop() and call the stop() method.

Even simpler: just use the pre-existing InProcessServer test fixture to start the server before entering the test body and close it afterwards.

review: Needs Fixing
Revision history for this message
Alexandros Frantzis (afrantzis) wrote : Posted in a previous version of this proposal

> > 119 + /*HACK to wait for the server to start*/
> >
> > You can override the
> > mir::DefaultServerConfiguration::the_server_status_listener() and provide
> you
> > own implementation to wait for server start.
> >
> > /*HACK to stop the mir server*/
> >
> > You can get the main loop object from
> > mir::DefaultServerConfiguration::the_main_loop() and call the stop() method.
>
> Even simpler: just use the pre-existing InProcessServer test fixture to start
> the server before entering the test body and close it afterwards.

Even better, +1

Revision history for this message
Alberto Aguirre (albaguirre) wrote : Posted in a previous version of this proposal

~~~
111 + FILE* in;
112 + char* line = NULL;
113 + size_t len = 0;
114 + ssize_t read;

123 + ASSERT_TRUE((in = popen(cmd, "r")));
126 + while ((read = getline(&line, &len, in)) != -1)
151 + free(line);
152 + fclose(in);
~~~

I'd like to see that in a separate class and put under mir_test as it could be useful if we need to run other
benchmark programs OR use pstreams library which already does that for you.

~~~
virtual void RunGLMark2(char const* output_filename, ResultFileType file_type)
~~~
RunGLMark2 not in the style described at http://unity.ubuntu.com/mir/cppguide/index.html#Function_Names

~~~
106 + enum ResultFileType {raw, json};
~~~

Use enum class.

review: Needs Fixing
Revision history for this message
Alberto Aguirre (albaguirre) wrote : Posted in a previous version of this proposal

Maybe this will be useful:
https://code.launchpad.net/~mir-team/mir/popen-cpp-wrapper/+merge/217831

Popen p{"glmark2-es2-mir --fullscreen"};
std::string line;
while(p.get_line(line))
{
  <do something with line>
}

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote : Posted in a previous version of this proposal

My earlier comment hasn't been addressed:

Even simpler: just use the pre-existing InProcessServer test fixture to start the server before entering the test body and close it afterwards.

review: Needs Fixing
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

The test should be supplying the connection string required by glmark by calling InProcessServer::new_connection().

I'm not sure if there's a glmark command-line parameter you can use, or if it needs:

    setenv("MIR_SERVER", mir_connect_sync(new_connection().c_str());

In the case of using the setenv() option note that it is advisable to restore the original state of any environment variables changed in tests.

review: Needs Fixing
Revision history for this message
Josh Arenson (josharenson) wrote :

> The test should be supplying the connection string required by glmark by
> calling InProcessServer::new_connection().
>
> I'm not sure if there's a glmark command-line parameter you can use, or if it
> needs:
>
> setenv("MIR_SERVER", mir_connect_sync(new_connection().c_str());
>
> In the case of using the setenv() option note that it is advisable to restore
> the original state of any environment variables changed in tests.

I agree that the variable should be restored, but GLMark2 is also calling mir_connect_sync so I'm not sure the above method will work. This becomes more of an issue if we decide to automate closed-source benchmarks.
The other option is to override the SetUp method, and unset the variable there. This requires no changes to InProcessServer, but should the point in a test where an environment variable is set really affect the outcome?

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> > The test should be supplying the connection string required by glmark by
> > calling InProcessServer::new_connection().
> >
> > I'm not sure if there's a glmark command-line parameter you can use, or if
> it
> > needs:
> >
> > setenv("MIR_SERVER", mir_connect_sync(new_connection().c_str());
> >
> > In the case of using the setenv() option note that it is advisable to
> restore
> > the original state of any environment variables changed in tests.
>
>
> I agree that the variable should be restored, but GLMark2 is also calling
> mir_connect_sync so I'm not sure the above method will work. This becomes more
> of an issue if we decide to automate closed-source benchmarks.

Sorry, I'm still being unclear.

1. I should have written: setenv("MIR_SERVER", new_connection().c_str());
2. GLMark2 should ideally be calling mir_connect_sync() with the URI returned by new_connection() - passed through a command line parameter.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

148 + char const* origenv = getenv("MIR_SERVER_NO_FILE");
149 + unsetenv("MIR_SERVER_NO_FILE");
...
153 + setenv("MIR_SERVER_NO_FILE", origenv, true);

All thse lines should go. There is no need to do *anything* with $MIR_SERVER_NO_FILE. It is unnecessary and will cause problems if there is another process exporting the default endpoint on the filesystem.

Instead, as glmark2-es2-mir doesn't have a "mir connection" parameter you should use $MIR_SERVER to specify the Mir server connection. Specifically, this works without requiring an endpoint on the filesystem:

 auto const cmd = "MIR_SERVER=" + new_connection() + " glmark2-es2-mir --fullscreen";

~~~~

109 + virtual void RunGLMark2(char const* output_filename, ResultFileType file_type)

The Mir style guideline names functions in lower_case

review: Needs Fixing
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

(Not a blocker for this MP)

93 +char const* dummy[] = {0};
94 +int argc = 0;
95 +char const** argv = dummy;

This is a bit unsatisfactory as it means that configuring the Mir server can't be done on the command line as is possible with other tests. I've raised lp:1316987 to record the problem.

Revision history for this message
Josh Arenson (josharenson) wrote :

> (Not a blocker for this MP)
>
> 93 +char const* dummy[] = {0};
> 94 +int argc = 0;
> 95 +char const** argv = dummy;
>
> This is a bit unsatisfactory as it means that configuring the Mir server can't
> be done on the command line as is possible with other tests. I've raised
> lp:1316987 to record the problem.

Added a FIXME to the code, and took ownership of the bug.

Revision history for this message
Alberto Aguirre (albaguirre) wrote :

LGTM

review: Approve
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

[ RUN ] DemoInProcessServer.client_can_connect
/tmp/buildd/mir-0.2.0bzr1589pkg0utopic193+autopilot0/tests/mir_test_framework/in_process_server.cpp:94: Failure
Failed
bind: No such file or directory
/tmp/buildd/mir-0.2.0bzr1589pkg0utopic193+autopilot0/tests/mir_test_framework/in_process_server.cpp:51: Failure
Value of: display_server
Actual: false
Expected: true
/tmp/buildd/mir-0.2.0bzr1589pkg0utopic193+autopilot0/tests/mir_test_framework/in_process_server.cpp:63: Failure
Value of: display_server
Actual: false
Expected: true
Did you override SetUp() and forget to call mtf::InProcessServer::SetUp()?
[ FAILED ] DemoInProcessServer.client_can_connect (2004 ms)

Looks worrying but unrelated: triggering a rebuild.

review: Abstain
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

Actually, this "test" desn't assert anything. AFAICS it will pass even if glmark crashes and produces no stats.

Are we planning to introduce some test requirement at some stage (when we've benchmarked on some test kit)?

review: Needs Information
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

85 +#include <iostream>

You don't use this. (But gtest pulls it in anyway so removing this line doesn't help much.)

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Alberto Aguirre (albaguirre) wrote :

You need to rebase the branch to not include
    lp:~josharenson/mir/unset_server_nofile

and remove it as pre-requisite for this MP.

review: Needs Fixing
Revision history for this message
Josh Arenson (josharenson) wrote :

> Actually, this "test" desn't assert anything. AFAICS it will pass even if
> glmark crashes and produces no stats.
>
> Are we planning to introduce some test requirement at some stage (when we've
> benchmarked on some test kit)?

Yes, I am currently profiling devices and trying to determine an appropriate threshold.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> > Actually, this "test" desn't assert anything. AFAICS it will pass even if
> > glmark crashes and produces no stats.
> >
> > Are we planning to introduce some test requirement at some stage (when we've
> > benchmarked on some test kit)?
>
> Yes, I am currently profiling devices and trying to determine an appropriate
> threshold.

That's OK for now.

As Alberto says, you need to rid this MP of the unnecessary prerequisite (and the code of its changes).

review: Needs Resubmitting
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> /usr/src/gmock/gtest/include/gtest/gtest.h:1585:28: error: comparison of integers of different signs: 'const unsigned long' and 'const int' [-Werror,-Wsign-compare]
> GTEST_IMPL_CMP_HELPER_(GT, >);
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
> /usr/src/gmock/gtest/include/gtest/gtest.h:1562:12: note: expanded from macro 'GTEST_IMPL_CMP_HELPER_'
> if (val1 op val2) {\
> ^
> /mir/tests/performance-tests/test_glmark2-es2-mir.cpp:61:9: note: in instantiation of function template specialization 'testing::internal::CmpHelperGT<unsigned long, int>' requested here
> ASSERT_GT(score.length(), 0);

clang is a bit more sensitive to signed/unsigned comparisons than g++ and std::size_t is unsigned. It works for both if you write this as:
        ASSERT_GT(score.length(), 0u);

~~~~

As you'll be editing anyway:

145 +
...
150 +

This whitespace doesn't aid readability.

review: Needs Fixing

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'debian/mir-test-tools.install'
2--- debian/mir-test-tools.install 2014-03-06 06:05:17 +0000
3+++ debian/mir-test-tools.install 2014-05-08 17:51:28 +0000
4@@ -2,5 +2,6 @@
5 usr/bin/mir_unit_tests
6 usr/bin/mir_acceptance_tests
7 usr/bin/mir_integration_tests
8+usr/bin/mir_performance_tests
9 usr/lib/*/mir/tools/libmirclientlttng.so
10 usr/lib/*/mir/tools/libmirserverlttng.so
11
12=== modified file 'tests/CMakeLists.txt'
13--- tests/CMakeLists.txt 2014-03-26 05:48:59 +0000
14+++ tests/CMakeLists.txt 2014-05-08 17:51:28 +0000
15@@ -26,6 +26,7 @@
16
17 option(MIR_BUILD_ACCEPTANCE_TESTS "Build acceptance tests" ON)
18 option(MIR_BUILD_INTEGRATION_TESTS "Build integration tests" ON)
19+option(MIR_BUILD_PERFORMANCE_TESTS "Build performance tests" ON)
20 option(MIR_BUILD_UNIT_TESTS "Build unit tests" ON)
21
22 if (MIR_BUILD_ACCEPTANCE_TESTS)
23@@ -36,6 +37,10 @@
24 add_subdirectory(integration-tests/)
25 endif (MIR_BUILD_INTEGRATION_TESTS)
26
27+if (MIR_BUILD_PERFORMANCE_TESTS)
28+ add_subdirectory(performance-tests/)
29+endif(MIR_BUILD_PERFORMANCE_TESTS)
30+
31 if (MIR_BUILD_UNIT_TESTS)
32 add_subdirectory(unit-tests/)
33 endif (MIR_BUILD_UNIT_TESTS)
34
35=== added directory 'tests/performance-tests'
36=== added file 'tests/performance-tests/CMakeLists.txt'
37--- tests/performance-tests/CMakeLists.txt 1970-01-01 00:00:00 +0000
38+++ tests/performance-tests/CMakeLists.txt 2014-05-08 17:51:28 +0000
39@@ -0,0 +1,28 @@
40+set(
41+ PERFORMANCE_TESTS_SOURCES
42+
43+ test_glmark2-es2-mir.cpp
44+)
45+
46+add_executable(
47+ mir_performance_tests
48+
49+ ${PERFORMANCE_TESTS_SOURCES}
50+)
51+
52+target_link_libraries(
53+ mir_performance_tests
54+
55+ mir-test
56+ mir-test-framework
57+
58+ ${Boost_LIBRARIES}
59+ ${GTEST_BOTH_LIBRARIES}
60+ ${GMOCK_LIBRARY}
61+ ${GMOCK_MAIN_LIBRARY}
62+)
63+
64+install(
65+ TARGETS mir_performance_tests
66+ RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
67+)
68
69=== added file 'tests/performance-tests/test_glmark2-es2-mir.cpp'
70--- tests/performance-tests/test_glmark2-es2-mir.cpp 1970-01-01 00:00:00 +0000
71+++ tests/performance-tests/test_glmark2-es2-mir.cpp 2014-05-08 17:51:28 +0000
72@@ -0,0 +1,85 @@
73+#include "mir/default_server_configuration.h"
74+#include "mir/options/default_configuration.h"
75+
76+#include "mir_test/popen.h"
77+#include "mir_test_framework/in_process_server.h"
78+
79+#include <gtest/gtest.h>
80+
81+#include <stdlib.h>
82+#include <stdio.h>
83+
84+#include <boost/regex.hpp>
85+#include <fstream>
86+#include <string>
87+
88+namespace
89+{
90+// FIXME remove this when support for passing cli options to
91+// DefaultServerConfiguration is implemented
92+char const* dummy[] = {0};
93+int argc = 0;
94+char const** argv = dummy;
95+
96+class GLMark2Test : public mir_test_framework::InProcessServer
97+{
98+public:
99+ GLMark2Test()
100+ : config(argc, argv)
101+ {
102+ }
103+
104+ mir::DefaultServerConfiguration& server_config() override {return config;};
105+
106+protected:
107+ enum ResultFileType {raw, json};
108+ virtual void run_glmark2(char const* output_filename, ResultFileType file_type)
109+ {
110+ auto const cmd = "MIR_SERVER=" + new_connection() + " glmark2-es2-mir --fullscreen";
111+ mir::test::Popen p(cmd);
112+
113+ boost::cmatch matches;
114+ boost::regex re_glmark2_score(".*glmark2\\s+Score:\\s+(\\d+).*");
115+ std::string line;
116+ std::ofstream glmark2_output;
117+ std::string score = "";
118+ glmark2_output.open(output_filename);
119+ while (p.get_line(line))
120+ {
121+ if(boost::regex_match(line.c_str(), matches, re_glmark2_score)
122+ && score.length() == 0)
123+ {
124+ score = matches[1];
125+ }
126+
127+ if (file_type == raw)
128+ {
129+ glmark2_output << line;
130+ }
131+ }
132+
133+ ASSERT_GT(score.length(), 0);
134+ EXPECT_GT(atoi(score.c_str()), 52);
135+
136+ if (file_type == json)
137+ {
138+ std::string json = "{";
139+ json += "'benchmark_name':'glmark2-es2-mir'";
140+ json += ",";
141+ json += "'score':'" + score + "'";
142+ json += "}";
143+ glmark2_output << json;
144+ }
145+
146+ }
147+
148+private:
149+ mir::DefaultServerConfiguration config;
150+
151+};
152+
153+TEST_F(GLMark2Test, benchmark_fullscreen_default)
154+{
155+ run_glmark2("glmark2_fullscreen_default.json", json);
156+}
157+}

Subscribers

People subscribed via source and target branches