Unity 8

Merge lp:~mzanetti/unity8/make-xvfbtestSomething into lp:unity8

make-xvfbtestSomething
Merge into trunk

Proposed by Michael Zanetti on 2014-03-18

Status:	Merged
Approved by:	Albert Astals Cid on 2014-04-03
Approved revision:	785
Merged at revision:	813
Proposed branch:	lp:~mzanetti/unity8/make-xvfbtestSomething
Merge into:	lp:unity8
Prerequisite:	lp:~mzanetti/unity8/make-single-testfunction
Diff against target:	208 lines (+72/-43) 7 files modified build (+1/-0) cmake/modules/QmlTest.cmake (+64/-19) tests/CMakeLists.txt (+2/-2) tests/plugins/Ubuntu/Gestures/CMakeLists.txt (+2/-11) tests/qmltests/CMakeLists.txt (+0/-1) tests/qmltests/Components/CMakeLists.txt (+2/-10) tests/qmltests/Components/tst_LazyImage.qml (+1/-0)
To merge this branch:	bzr merge lp:~mzanetti/unity8/make-xvfbtestSomething
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Albert Astals Cid (community)			Approve on 2014-04-03
PS Jenkins bot (community)	continuous-integration	2014-03-18	Needs Fixing on 2014-04-02
Michał Sawicz			Needs Fixing on 2014-03-20
Andrea Cimitan		2014-03-18	Pending
Review via email: mp+211573@code.launchpad.net

This proposal supersedes a proposal from 2014-03-18.

Commit message

add "make xvfbtestSomething" target to run qml tests in xvfb

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-03-18: Posted in a previous version of this proposal

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/2537/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Andrea Cimitan (cimi) wrote on 2014-03-18: Posted in a previous version of this proposal

I think we need to make xvfb optional in ./build, otherwise it might not work on the devices without X (like mobile)

review: Needs Fixing

Revision history for this message

Michael Zanetti (mzanetti) wrote on 2014-03-18: Posted in a previous version of this proposal

> I think we need to make xvfb optional in ./build, otherwise it might not work
> on the devices without X (like mobile)

this is only installed with ./build -s. For running on the device you should use ./run_on_device -s

Revision history for this message

Andrea Cimitan (cimi) on 2014-03-18: Posted in a previous version of this proposal

review: Abstain

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-03-18: Posted in a previous version of this proposal

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/2541/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-03-18:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/2547/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Michał Sawicz (saviq) wrote on 2014-03-20:

You need NO_ADD_TEST for non-ui tests:

11/26 Test #11: testUnityTest ............................ Passed 0.18 sec
12/26 Test #12: xvfbtestUnityTest ........................***Failed 0.01 sec
/usr/bin/env: xvfb-run: No such file or directory

review: Needs Fixing

Revision history for this message

Michał Sawicz (saviq) wrote on 2014-03-20:

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/mesa/libGL.so.1

Ah, this won't work outside of amd64... Not sure if we should try and make it work across the board or just warn about it somewhere.

review: Needs Fixing

Revision history for this message

Michał Sawicz (saviq) wrote on 2014-03-20:

set(xfvbtestCommand... could use abstracting

review: Needs Fixing

lp:~mzanetti/unity8/make-xvfbtestSomething updated on 2014-03-21

781. By Michael Zanetti on 2014-03-21: always add NO_ADD_TEST to xvfb target
782. By Michael Zanetti on 2014-03-21: abstract binary qml test targets into add_binary_qml_test macro
783. By Michael Zanetti on 2014-03-21: make it a bit more generic

Revision history for this message

Michael Zanetti (mzanetti) wrote on 2014-03-21:

> You need NO_ADD_TEST for non-ui tests:
>
> 11/26 Test #11: testUnityTest ............................ Passed 0.18
> sec
> 12/26 Test #12: xvfbtestUnityTest ........................***Failed 0.01
> sec
> /usr/bin/env: xvfb-run: No such file or directory

fixed

Revision history for this message

Michael Zanetti (mzanetti) wrote on 2014-03-21:

> LD_PRELOAD=/usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
>
> Ah, this won't work outside of amd64... Not sure if we should try and make it
> work across the board or just warn about it somewhere.

fixed. Calling gcc -dumpmachine to determine arch-triplet.

Revision history for this message

Michael Zanetti (mzanetti) wrote on 2014-03-21:

> set(xfvbtestCommand... could use abstracting

ack. abstracted into add_binary_qmltest macro

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-03-21:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/2590/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-03-21:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/2591/rebuild

review: Needs Fixing (continuous-integration)

lp:~mzanetti/unity8/make-xvfbtestSomething updated on 2014-03-25

784. By Michael Zanetti on 2014-03-25: set LD_LIBRARY_PATH conditionally, depending on the availability of gcc

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-03-25:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/2619/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Albert Astals Cid (aacid) wrote on 2014-04-02:

Text conflict in tests/qmltests/CMakeLists.txt

review: Needs Fixing

lp:~mzanetti/unity8/make-xvfbtestSomething updated on 2014-04-02

785. By Michael Zanetti on 2014-04-02: merge trunk

Revision history for this message

Michael Zanetti (mzanetti) wrote on 2014-04-02:

> Text conflict in tests/qmltests/CMakeLists.txt

fixed

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-04-02:

FAILED: Continuous integration, rev:785
http://jenkins.qa.ubuntu.com/job/unity8-ci/2711/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty/4453/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-trusty-touch/4053/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-phablet-qmluitests-trusty/1581
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity8-trusty-amd64-ci/1232
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity8-trusty-armhf-ci/1236
        deb: http://jenkins.qa.ubuntu.com/job/unity8-trusty-armhf-ci/1236/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity8-trusty-i386-ci/1232
    FAILURE: http://jenkins.qa.ubuntu.com/job/autopilot-testrunner-otto-trusty/3849/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/4559
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-amd64/4559/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/4086
        deb: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-trusty-armhf/4086/artifact/work/output/*zip*/output.zip
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-runner-mako/6307/console
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/5554

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/2711/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-04-02:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity8-ci/2712/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Albert Astals Cid (aacid) wrote on 2014-04-03:

We have now lost the hability to run all qmluitests without xvfb, previously you achieved that by doing

make qmluitests

but that now runs the xvfb variants, i wonder if it would make sense to keep the behaviour of

make qmluitests as it was

and then introduce

make xvfbqmluitests

that runs the xvfb variant

review: Needs Fixing

Revision history for this message

Michał Sawicz (saviq) wrote on 2014-04-03:

I think it's fine as is, we don't want to encourage running them outside of xvfb, other than for debugging, when you run only one test/suite at a time.

We'll be running them under xvfb during CI as well, so that we have as close an env locally.

Revision history for this message

Albert Astals Cid (aacid) wrote on 2014-04-03:

Ok :-)

* Did you perform an exploratory manual test run of the code change and any related functionality?
Yes, tests, run

* Did CI run pass? If not, please explain why.
No, some tests are broken, being fixed in a different MR

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk

Subscribers

People subscribed via source and target branches

to all changes:

Michael Zanetti

Mohammad Rakibul Hasan

Unity Team

 === modified file 'build'
 --- build	2014-04-01 22:56:02 +0000
 +++ build	2014-04-02 12:13:20 +0000
@@ -54,6 +54,7 @@
                           unity-scope-home \
                           unity-lens-applications \
                           unity-plugin-scopes \
++                         xvfb \
          || exit 5
+ }
 === modified file 'cmake/modules/QmlTest.cmake'
 --- cmake/modules/QmlTest.cmake	2014-03-28 00:32:46 +0000
 +++ cmake/modules/QmlTest.cmake	2014-04-02 12:13:20 +0000
@@ -65,6 +65,7 @@
      cmake_parse_arguments(qmltest "${options}" "" "${multi_value_keywords}" ${ARGN})
      set(qmltest_TARGET test${COMPONENT_NAME})
++    set(qmltest_xvfb_TARGET xvfbtest${COMPONENT_NAME})
      set(qmltest_FILE ${SUBPATH}/tst_${COMPONENT_NAME})
      set(qmltestrunner_imports "")
@@ -82,24 +83,69 @@
      string(TOLOWER "${CMAKE_GENERATOR}" cmake_generator_lower)
      if(cmake_generator_lower STREQUAL "unix makefiles")
--        set(qmltest_command
--            env ${qmltest_ENVIRONMENT}
--            ${qmltestrunner_exe} -input ${CMAKE_CURRENT_SOURCE_DIR}/${qmltest_FILE}.qml
--                ${qmltestrunner_imports}
--                -o ${CMAKE_BINARY_DIR}/${qmltest_TARGET}.xml,xunitxml
--                -o -,txt
--                $(FUNCTION)
--        )
++        set(function_ARGS $(FUNCTION))
      else()
--        set(qmltest_command
--            env ${qmltest_ENVIRONMENT}
--            ${qmltestrunner_exe} -input ${CMAKE_CURRENT_SOURCE_DIR}/${qmltest_FILE}.qml
--                ${qmltestrunner_imports}
--                -o ${CMAKE_BINARY_DIR}/${qmltest_TARGET}.xml,xunitxml
--                -o -,txt
--        )
--    endif()
--
++        set(function_ARGS "")
++    endif()
++
++    set(qmltest_command
++        env ${qmltest_ENVIRONMENT}
++        ${qmltestrunner_exe} -input ${CMAKE_CURRENT_SOURCE_DIR}/${qmltest_FILE}.qml
++            ${qmltestrunner_imports}
++            -o ${CMAKE_BINARY_DIR}/${qmltest_TARGET}.xml,xunitxml
++            -o -,txt
++            ${function_ARGS}
++    )
++    find_program( HAVE_GCC gcc )
++    if (NOT ${HAVE_GCC} STREQUAL "")
++        exec_program( gcc ARGS "-dumpmachine" OUTPUT_VARIABLE ARCH_TRIPLET )
++        set(LD_PRELOAD_PATH "LD_PRELOAD=/usr/lib/${ARCH_TRIPLET}/mesa/libGL.so.1")
++    endif()
++    set(qmltest_xvfb_command
++        env ${qmltest_ENVIRONMENT} ${LD_PRELOAD_PATH}
++        xvfb-run --server-args "-screen 0 1024x768x24" --auto-servernum
++        ${qmltestrunner_exe} -input ${CMAKE_CURRENT_SOURCE_DIR}/${qmltest_FILE}.qml
++        ${qmltestrunner_imports}
++            -o ${CMAKE_BINARY_DIR}/${qmltest_TARGET}.xml,xunitxml
++            -o -,txt
++            ${function_ARGS}
++    )
++
++    add_qmltest_target(${qmltest_TARGET} "${qmltest_command}" TRUE ${qmltest_NO_ADD_TEST})
++    add_qmltest_target(${qmltest_xvfb_TARGET} "${qmltest_xvfb_command}" ${qmltest_NO_TARGETS} TRUE)
++    add_manual_qml_test(${SUBPATH} ${COMPONENT_NAME} ${ARGN})
++endmacro(add_qml_test)
++
++macro(add_binary_qml_test CLASS_NAME LD_PATH DEPS)
++    set(testCommand
++          LD_LIBRARY_PATH=${LD_PATH}
++          ${CMAKE_CURRENT_BINARY_DIR}/${CLASS_NAME}TestExec
++          -o ${CMAKE_BINARY_DIR}/${CLASSNAME}Test.xml,xunitxml
++          -o -,txt)
++
++    add_qmltest_target(test${CLASS_NAME} "${testCommand}" FALSE TRUE)
++    add_dependencies(test${CLASS_NAME} ${CLASS_NAME}TestExec ${DEPS})
++
++    find_program( HAVE_GCC gcc )
++    if (NOT ${HAVE_GCC} STREQUAL "")
++        exec_program( gcc ARGS "-dumpmachine" OUTPUT_VARIABLE ARCH_TRIPLET )
++        set(LD_PRELOAD_PATH "LD_PRELOAD=/usr/lib/${ARCH_TRIPLET}/mesa/libGL.so.1")
++    endif()
++    set(xvfbtestCommand
++          ${LD_PRELOAD_PATH}
++          LD_LIBRARY_PATH=${LD_PATH}
++          xvfb-run --server-args "-screen 0 1024x768x24" --auto-servernum
++          ${CMAKE_CURRENT_BINARY_DIR}/${CLASS_NAME}TestExec
++          -o ${CMAKE_BINARY_DIR}/${CLASS_NAME}Test.xml,xunitxml
++          -o -,txt)
++
++    add_qmltest_target(xvfbtest${CLASS_NAME} "${xvfbtestCommand}" FALSE TRUE)
++    add_dependencies(qmluitests xvfbtest${CLASS_NAME})
++
++    add_manual_qml_test(. ${CLASS_NAME} IMPORT_PATHS ${CMAKE_BINARY_DIR}/plugins)
++endmacro(add_binary_qml_test)
++
++macro(add_qmltest_target qmltest_TARGET qmltest_command qmltest_NO_TARGETS qmltest_NO_ADD_TEST)
      add_custom_target(${qmltest_TARGET} ${qmltest_command})
      if(NOT "${qmltest_PROPERTIES}" STREQUAL "")
@@ -130,5 +176,4 @@
          endif()
      endif("${qmltest_NO_TARGETS}" STREQUAL "FALSE")
--    add_manual_qml_test(${SUBPATH} ${COMPONENT_NAME} ${ARGN})
--endmacro(add_qml_test)
++endmacro(add_qmltest_target)
 === modified file 'tests/CMakeLists.txt'
 --- tests/CMakeLists.txt	2013-07-04 14:47:47 +0000
 +++ tests/CMakeLists.txt	2014-04-02 12:13:20 +0000
@@ -1,7 +1,7 @@
--# QML tests that do not require graphical capabitlies.
++# QML tests that do not require graphical capabilities.
  add_custom_target(qmlunittests)
--# QML tests that require graphical capabitlies.
++# QML tests that require graphical capabilities.
  add_custom_target(qmluitests)
  add_custom_target(qmltests)
 === modified file 'tests/plugins/Ubuntu/Gestures/CMakeLists.txt'
 --- tests/plugins/Ubuntu/Gestures/CMakeLists.txt	2013-12-17 14:56:25 +0000
 +++ tests/plugins/Ubuntu/Gestures/CMakeLists.txt	2014-04-02 12:13:20 +0000
@@ -15,20 +15,11 @@
  add_definitions(-DUBUNTU_GESTURES_PLUGIN_DIR="${CMAKE_BINARY_DIR}/plugins")
  macro(add_gesture_ui_test CLASSNAME)
--    set(testCommand
--          LD_LIBRARY_PATH=${CMAKE_BINARY_DIR}/plugins/Ubuntu/Gestures
--          ${CMAKE_CURRENT_BINARY_DIR}/${CLASSNAME}TestExec
--          -o ${CMAKE_BINARY_DIR}/${CLASSNAME}Test.xml,xunitxml
--          -o -,txt)
--
--    add_custom_target(test${CLASSNAME} ${testCommand})
--    add_dependencies(qmluitests test${CLASSNAME})
--    add_dependencies(test${CLASSNAME} ${CLASSNAME}TestExec UbuntuGesturesTestQmlFiles)
--
      add_executable(${CLASSNAME}TestExec tst_${CLASSNAME}.cpp GestureTest.cpp)
      qt5_use_modules(${CLASSNAME}TestExec Test Core Qml Gui Quick)
      target_link_libraries(${CLASSNAME}TestExec UbuntuGestureQml)
--    add_manual_qml_test(. ${CLASSNAME} IMPORT_PATHS ${CMAKE_BINARY_DIR}/plugins)
++
++    add_binary_qml_test(${CLASSNAME} ${CMAKE_BINARY_DIR}/plugins/Ubuntu/Gestures UbuntuGesturesTestQmlFiles)
  endmacro(add_gesture_ui_test)
  macro(add_gesture_test CLASSNAME)
 === modified file 'tests/qmltests/CMakeLists.txt'
 --- tests/qmltests/CMakeLists.txt	2014-04-01 22:55:28 +0000
 +++ tests/qmltests/CMakeLists.txt	2014-04-02 12:13:20 +0000
@@ -21,7 +21,6 @@
  add_qml_test(. Shell IMPORT_PATHS ${CMAKE_BINARY_DIR}/plugins ${qmltest_DEFAULT_IMPORT_PATHS} ${CMAKE_BINARY_DIR}/tests/mocks
                       ENVIRONMENT "LD_LIBRARY_PATH=${CMAKE_BINARY_DIR}/tests/mocks/LightDM/single")
--add_manual_qml_test(Components DragHandle IMPORT_PATHS ${CMAKE_BINARY_DIR}/plugins)
  add_qml_test(Components Carousel)
  add_qml_test(Components DraggingArea)
  add_qml_test(Components EdgeDemoOverlay)
 === modified file 'tests/qmltests/Components/CMakeLists.txt'
 --- tests/qmltests/Components/CMakeLists.txt	2013-12-17 14:56:25 +0000
 +++ tests/qmltests/Components/CMakeLists.txt	2014-04-02 12:13:20 +0000
@@ -6,19 +6,11 @@
      ${Qt5Gui_PRIVATE_INCLUDE_DIRS}
+     )
--set(testCommand
--      LD_LIBRARY_PATH=${CMAKE_BINARY_DIR}/plugins/Ubuntu/Gestures
--      ${CMAKE_CURRENT_BINARY_DIR}/DragHandleTestExec
--      -o ${CMAKE_BINARY_DIR}/DragHandleTest.xml,xunitxml
--      -o -,txt)
--
--add_custom_target(testDragHandle ${testCommand})
--add_dependencies(qmluitests testDragHandle)
--add_dependencies(testDragHandle DragHandleTestExec)
--
  add_executable(DragHandleTestExec tst_DragHandle.cpp)
  qt5_use_modules(DragHandleTestExec Test Core Qml Gui Quick)
  target_link_libraries(DragHandleTestExec UbuntuGestureQml)
++add_binary_qml_test(DragHandle ${CMAKE_BINARY_DIR}/plugins/Ubuntu/Gestures "")
++
  add_definitions(-DUBUNTU_GESTURES_PLUGIN_DIR="${CMAKE_BINARY_DIR}/plugins")
  add_definitions(-DTEST_DIR="${CMAKE_CURRENT_LIST_DIR}")
 === modified file 'tests/qmltests/Components/tst_LazyImage.qml'
 --- tests/qmltests/Components/tst_LazyImage.qml	2014-03-21 10:48:42 +0000
 +++ tests/qmltests/Components/tst_LazyImage.qml	2014-04-02 12:13:20 +0000
@@ -152,6 +152,7 @@
          function test_lazyimage(data) {
              data.func();
++            waitForRendering(baseRect);
              if (data.transition) {
                  // wait for the transition to complete

Unity 8

Merge lp:~mzanetti/unity8/make-xvfbtestSomething into lp:unity8

Commit message

Description of the change

Preview Diff

Subscribers