Merge lp:~mir-team/mir/consolidate-crossbuild-scripts into lp:~mir-team/mir/trunk

[Kevin DuBois]

pull the scripts used to help cross compile mir into the main build tree. also, add glog dependency to the armhf dependencies, remove some boost usage
fixes: LP #1124873

pull over the script used in ~kdub/mir/ndk-rewrite to the main build.
This branch will be deprecated.

This requires you to add the phablet-team/ppa to the setup you have for retrieving the dependencies (the ndk-rewrite branch had previously stored a copy of hybris in it, a hack that is going away)

example deb line: deb [arch=armhf] http://ppa.launchpad.net/phablet-team/ppa/ubuntu quantal main

also, remove some usage of boost threads that was causing me some problems

[Kevin DuBois (kdub)]

just a test branch at the moment to check jenkins still works with the script bumps

review: Disapprove

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:474
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~mir-team/mir/consolidate-crossbuild-scripts/+merge/154762/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/mir-ci/160/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-android-raring-i386-build/77/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/161/console

Click here to trigger a rebuild:
http://s-jenkins:8080/job/mir-ci/160/rebuild

review: Needs Fixing (continuous-integration)

[Kevin DuBois (kdub)]

the failure is because this line:
deb [arch=armhf] http://ppa.launchpad.net/phablet-team/ppa/ubuntu quantal main
is needed in the builder setup's sources.list for the hybris armhf packages

[Daniel van Vugt (vanvugt)]

I know it's just a test right now. But the name "consolidate-crossbuild-scripts" implies only script changes, and maybe docs. So this change looks out of place:

=== modified file 'tests/mir_test_framework/signal_dispatcher.cpp'

review: Needs Fixing

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:474
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~mir-team/mir/consolidate-crossbuild-scripts/+merge/154762/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/mir-ci/180/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-raring-i386-build/105
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/181/console

Click here to trigger a rebuild:
http://s-jenkins:8080/job/mir-ci/180/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:475
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~mir-team/mir/consolidate-crossbuild-scripts/+merge/154762/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/mir-ci/194/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-raring-i386-build/125
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/195/console

Click here to trigger a rebuild:
http://s-jenkins:8080/job/mir-ci/194/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:476
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~mir-team/mir/consolidate-crossbuild-scripts/+merge/154762/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/mir-ci/195/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-raring-i386-build/126
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/196
        deb: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/196/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins:8080/job/mir-ci/195/rebuild

review: Needs Fixing (continuous-integration)

[Kevin DuBois (kdub)]

> just a test branch at the moment to check jenkins still works with the script
> bumps
scripts now work, changing to abstain

review: Abstain

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:476
http://jenkins.qa.ubuntu.com/job/mir-ci/198/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-raring-i386-build/129
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/199
        deb: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/199/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins:8080/job/mir-ci/198/rebuild

review: Approve (continuous-integration)

[Kevin DuBois (kdub)]

> I know it's just a test right now. But the name "consolidate-crossbuild-
> scripts" implies only script changes, and maybe docs. So this change looks out
> of place:
>
> === modified file 'tests/mir_test_framework/signal_dispatcher.cpp'

updated the descriptions to bring attention to it.

[Alan Griffiths (alan-griffiths)]

LGTM

(The cleanup of signal_dispatcher.cpp would probably be better as a separate MP, but not worth blocking.)

review: Approve

[Alexandros Frantzis (afrantzis)]

63 can look at the script and instructions in lp:~kdub/mir/ndk-rewrite as an

Should we change this to point to lp:mir, or is the information in lp:~kdub/mir/ndk-rewrite more suitable?

76 +provides a scripting example of how to cross compile. The 'setup partial-armhf-chroot.sh' will attempt to download all the arm dependencies you need. You have to have your APT sources.list files pointed at arm repositories.

Since we want .md files to be readable without preprocessing, we should wrap the lines nicely.

103 +++ setup-android-dependencies.sh
139 +++ setup-partial-armhf-chroot.sh

Perhaps tools/ would be a better place for these files?

review: Needs Fixing

[Kevin DuBois (kdub)]

> 63 can look at the script and instructions in lp:~kdub/mir/ndk-rewrite as
> an
>
> Should we change this to point to lp:mir, or is the information in
> lp:~kdub/mir/ndk-rewrite more suitable?
>
> 76 +provides a scripting example of how to cross compile. The 'setup partial-
> armhf-chroot.sh' will attempt to download all the arm dependencies you need.
> You have to have your APT sources.list files pointed at arm repositories.
>
> Since we want .md files to be readable without preprocessing, we should wrap
> the lines nicely.
>
> 103 +++ setup-android-dependencies.sh
> 139 +++ setup-partial-armhf-chroot.sh
>
> Perhaps tools/ would be a better place for these files?

should be addressed

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:477
http://jenkins.qa.ubuntu.com/job/mir-ci/206/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-raring-i386-build/140
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-clang-raring-amd64-build/23
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/207
        deb: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/207/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins:8080/job/mir-ci/206/rebuild

review: Approve (continuous-integration)

[Alexandros Frantzis (afrantzis)]

Looks good.

review: Approve

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:479
http://jenkins.qa.ubuntu.com/job/mir-ci/207/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-raring-i386-build/141
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-clang-raring-amd64-build/24
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/208
        deb: http://jenkins.qa.ubuntu.com/job/mir-quantal-amd64-ci/208/artifact/work/output/*zip*/output.zip

Click here to trigger a rebuild:
http://s-jenkins:8080/job/mir-ci/207/rebuild

review: Approve (continuous-integration)

[Daniel van Vugt (vanvugt)]

Linked bug 1124873 because it appears to be relevant.

[Daniel van Vugt (vanvugt)]

I'm not going to spend time testing with the Nexus today so can't really declare this properly reviewed.

If I was to nit-pick then I would say try to avoid using "$1" more than once in a script. It is easier for maintenance and readability to use:
  foo="$1"
and then $foo thereafter.

I will trust the existing approvals are enough for now.

review: Abstain

=== modified file 'cross-compile-chroot.sh'
--- cross-compile-chroot.sh	2013-03-14 05:33:42 +0000
+++ cross-compile-chroot.sh	2013-03-26 16:33:23 +0000
@@ -1,34 +1,22 @@
 #!/bin/bash
 # build script for Mir on android arm devices
-# test run requires package 'android-tools-adb'
-#
-# you should use this file if you have to do an integration test, or as 
-# an example of how to build. this script will do more downloading than you 
-# might want in a development workflow
 #
 set -e
 
 BUILD_DIR=build-android-arm
 
-#note:ndk-rewrite is just temporary until the real is pushed
 if [ "$MIR_NDK_PATH" = "" ]; then
-    export MIR_NDK_PATH=`pwd`/ndk-rewrite
+    export MIR_NDK_PATH=`pwd`/partial-armhf-chroot
+    if [ ! -d ${MIR_NDK_PATH} ]; then 
+        echo "no partial root specified or detected. attempting to create one"
+        pushd tools > /dev/null
+            ./setup-partial-armhf-chroot.sh ${MIR_NDK_PATH}
+        popd > /dev/null
+    fi
 fi
 
 echo "Using MIR_NDK_PATH: $MIR_NDK_PATH"
 
-if [[ -d ${MIR_NDK_PATH} ]]; then
-    pushd ${MIR_NDK_PATH}
-    bzr pull
-    popd
-else
-    bzr branch lp:~kdub/mir/ndk-rewrite ${MIR_NDK_PATH}
-fi
-
-pushd ${MIR_NDK_PATH} > /dev/null
-    ./generate-armhf-deps.sh 
-popd > /dev/null
-
 #start with a clean build every time
 rm -rf ${BUILD_DIR}
 mkdir ${BUILD_DIR}

=== modified file 'debian/control'
--- debian/control	2013-03-13 04:54:15 +0000
+++ debian/control	2013-03-26 16:33:23 +0000
@@ -16,7 +16,8 @@
                libgbm-dev [!armhf],
                libprotobuf-dev,
                pkg-config,
-               libhybris-dev [armhf]
+               libhybris-dev [armhf],
+               libgoogle-glog-dev
 Standards-Version: 3.9.3
 Homepage: https://launchpad.net/mir
 # If you aren't a member of ~mir-team but need to upload packaging changes,

=== modified file 'doc/building_source_for_android.md'
--- doc/building_source_for_android.md	2013-03-13 09:19:56 +0000
+++ doc/building_source_for_android.md	2013-03-26 16:33:23 +0000
@@ -56,10 +56,10 @@
    targeting a quantal phablet image)
 
 2. Set up a chroot with the mir dependencies installed. At the moment, you
-   can look at the script and instructions in lp:~kdub/mir/ndk-rewrite as an
+   can look at the script and instructions in lp:mir as an
    example of how to set up a partial chroot you can build mir against.
 
-3. There are a few options, but here is an example of how to build mir for android
+3. There are a few ways to do this, but here is an example of how to build mir for android
 
         $ bzr branch lp:mir
         $ mkdir mir/build; cd mir/build
@@ -67,4 +67,7 @@
         $ make
 
 N.B. The `cross-compile-android.sh` script in mir's top level directory
-provides a scripting example of how to cross compile
+provides a scripting example of how to cross compile.
+The 'setup-partial-armhf-chroot.sh' will attempt to download all the arm
+dependencies you need. You have to have your APT sources.list files pointed at
+arm repositories.

=== modified file 'tests/mir_test_framework/signal_dispatcher.cpp'
--- tests/mir_test_framework/signal_dispatcher.cpp	2012-12-21 08:16:26 +0000
+++ tests/mir_test_framework/signal_dispatcher.cpp	2013-03-26 16:33:23 +0000
@@ -18,8 +18,8 @@
 
 #include "mir_test_framework/signal_dispatcher.h"
 
-#include <boost/thread.hpp>
-
+#include <mutex>
+#include <thread>
 #include <signal.h>
 #include <sys/types.h>
 #include <sys/socket.h>
@@ -84,7 +84,7 @@
 namespace
 {
 std::shared_ptr<mtf::SignalDispatcher> instance;
-boost::once_flag init_flag;
+std::once_flag init_flag;
 
 void init()
 {
@@ -104,6 +104,7 @@
 {
     Private() : worker_thread(std::bind(&Private::worker, this))
     {
+        worker_thread.detach();
     }
 
     void worker()
@@ -123,13 +124,13 @@
         }
     }
 
-    boost::thread worker_thread;
+    std::thread worker_thread;
     mtf::SignalDispatcher::SignalType signal_channel;
 };
 
 std::shared_ptr<mtf::SignalDispatcher> mtf::SignalDispatcher::instance()
 {
-    boost::call_once(::init_flag, ::init);
+    std::call_once(::init_flag, ::init);
 
     return ::instance;
 }

=== renamed file 'install_on_android.sh' => 'tools/install_on_android.sh'
--- install_on_android.sh	2013-03-12 21:45:48 +0000
+++ tools/install_on_android.sh	2013-03-26 16:33:23 +0000
@@ -1,5 +1,7 @@
 #!/bin/bash
 # build script for Mir on android arm devices
+# script assumes that the Mir dependencies are already set up on the device.
+# if they are not set up, see the setup-android-dependencies.sh script
 # test run requires package 'android-tools-adb'
 # test also assumes that the device is rooted and accessible over the adb bridge
 #
@@ -7,7 +9,13 @@
 #
 
 set -e
-BUILD_DIR=build-android-arm
+
+if [ -z ${1} ]; then
+    BUILD_DIR=build-android-arm
+else
+    BUILD_DIR=${1}
+fi
+
 pushd ${BUILD_DIR} > /dev/null 
     #
     # Upload and run the tests!
@@ -32,10 +40,6 @@
     done
 
     echo "ubuntu_chroot shell;
-        apt-get install libprotobuf7
-                        libboost-system1.49.0
-                        libboost-program-options1.49.0
-                        libboost-thread1.49.0;
         cd ${RUN_DIR};
         export GTEST_OUTPUT=xml:./;
         export LD_LIBRARY_PATH=.;

=== added file 'tools/setup-android-dependencies.sh'
--- tools/setup-android-dependencies.sh	1970-01-01 00:00:00 +0000
+++ tools/setup-android-dependencies.sh	2013-03-26 16:33:23 +0000
@@ -0,0 +1,31 @@
+#!/bin/bash
+# build script for Mir on android arm devices
+# test run requires package 'android-tools-adb'
+# test also assumes that the device is rooted and accessible over the adb bridge
+#
+# todo: this script should become part of the 'make install'/'make test' system
+#
+
+set -e
+
+pushd /tmp > /dev/null
+    rm -f libgoogle-glog0_0.3.2-4ubuntu1_armhf.deb
+    rm -f libgflags2_2.0-1_armhf.deb 
+
+    wget "https://launchpad.net/ubuntu/+archive/primary/+files/libgoogle-glog0_0.3.2-4ubuntu1_armhf.deb"
+    wget "https://launchpad.net/ubuntu/+archive/primary/+files/libgflags2_2.0-1_armhf.deb"
+
+    adb push libgoogle-glog0_0.3.2-4ubuntu1_armhf.deb /data/ubuntu/root
+    adb push libgflags2_2.0-1_armhf.deb /data/ubuntu/root
+
+    echo "ubuntu_chroot shell;
+        cd /root;
+        dpkg -i libgflags2_2.0-1_armhf.deb libgoogle-glog0_0.3.2-4ubuntu1_armhf.deb
+        apt-get install libprotobuf7 \
+                        libboost-system1.49.0 \
+                        libboost-program-options1.49.0 \
+                        libboost-thread1.49.0;
+
+            exit;
+            exit" | adb shell
+popd > /dev/null

=== added file 'tools/setup-partial-armhf-chroot.sh'
--- tools/setup-partial-armhf-chroot.sh	1970-01-01 00:00:00 +0000
+++ tools/setup-partial-armhf-chroot.sh	2013-03-26 16:33:23 +0000
@@ -0,0 +1,99 @@
+#!/bin/bash
+
+set -e
+
+if [ -z $1 ]; then
+    echo "please supply directory to create partial chroot in. (eg, ./setup-partial-armhf-chroot.sh mychroot-dir)"
+    exit
+fi
+
+echo "creating phablet-compatible quantal armhf partial chroot for mir compiles in directory ${1}"
+
+if [ ! -d ${1} ]; then
+    mkdir -p ${1} 
+fi
+
+pushd ${1} > /dev/null
+
+    ARCHITECTURE=armhf
+
+    declare -a PACKAGES=(
+        libboost1.49-dev
+        libboost-chrono1.49-dev
+        libboost-chrono1.49-dev
+        libboost-date-time1.49-dev
+        libboost-filesystem1.49-dev
+        libboost-program-options1.49-dev
+        libprotobuf-dev
+        libboost-chrono1.49.0
+        libboost-date-time1.49.0
+        libboost-filesystem1.49.0
+        libboost-system1.49.0
+        libboost-system1.49-dev
+        libboost-thread1.49-dev
+        libboost-thread1.49.0
+        libboost-regex1.49-dev
+        libboost-regex1.49.0
+        libboost-program-options1.49.0
+        libhybris
+        libhybris-dev
+        libicu48
+        libprotobuf7
+        libprotobuf-dev
+        libegl1-mesa-dev
+        libgles2-mesa-dev
+        zlib1g)
+
+    #cleanup
+    for i in * ; do
+        if [[ -d ${i} ]]; then 
+            echo "removing directory: ./${i}"
+            rm -rf ./${i}
+        fi
+    done
+
+    for i in ${PACKAGES[@]}; do
+
+        PACKAGE_VERSION=`apt-cache show ${i}:${ARCHITECTURE} | grep Version | awk -F: '{print $NF}' | sed "s/ //g"`
+        PACKAGE_FILENAME="${i}_${PACKAGE_VERSION}_${ARCHITECTURE}.deb"
+
+        if [ ! -f ${PACKAGE_FILENAME} ]; then
+            echo "Downloading mir dependency: ${i}"
+            apt-get download "${i}:${ARCHITECTURE}"
+        else
+            echo "already downloaded: ${PACKAGE_FILENAME}"
+        fi
+
+        #quick sanity check
+        if [ ! -f ${PACKAGE_FILENAME} ]; then
+            echo "error: did not download expected file (${PACKAGE_FILENAME}. script is malformed!"; exit 1        
+        fi
+
+        echo "Extracting: ${PACKAGE_FILENAME}"
+        dpkg -x ${PACKAGE_FILENAME} . 
+    done
+
+    #glog is only available in raring for armhf. we download the raring sources and 
+    #install them manually for this quantal script
+    if [ ! -f libgoogle-glog0_0.3.2-4ubuntu1_armhf.deb ]; then
+        wget "https://launchpad.net/ubuntu/+archive/primary/+files/libgoogle-glog0_0.3.2-4ubuntu1_armhf.deb"
+    fi
+
+    if [ ! -f libgflags2_2.0-1_armhf.deb ]; then
+        wget "https://launchpad.net/ubuntu/+archive/primary/+files/libgflags2_2.0-1_armhf.deb"
+    fi
+
+    if [ ! -f libgoogle-glog-dev_0.3.2-4ubuntu1_armhf.deb ]; then
+        wget "http://launchpadlibrarian.net/134086853/libgoogle-glog-dev_0.3.2-4ubuntu1_armhf.deb"
+    fi
+    dpkg -x libgflags2_2.0-1_armhf.deb .
+    dpkg -x libgoogle-glog0_0.3.2-4ubuntu1_armhf.deb .
+    dpkg -x libgoogle-glog-dev_0.3.2-4ubuntu1_armhf.deb .
+
+    #todo: we get egl/gles headers from the mesa packages, but should be pointing at the hybris libraries
+    #just rewrite the symlinks for now
+    rm ./usr/lib/arm-linux-gnueabihf/libEGL.so 
+    rm ./usr/lib/arm-linux-gnueabihf/libGLESv2.so 
+    ln -s libhybris-egl/libEGL.so.1 ./usr/lib/arm-linux-gnueabihf/libEGL.so 
+    ln -s libhybris-egl/libGLESv2.so.2 ./usr/lib/arm-linux-gnueabihf/libGLESv2.so
+popd > /dev/null