Merge lp:~widelands-dev/widelands/macos_build_app_compiler into lp:widelands

[Toni Förster]

choose between compiler clang or gcc, specify build type: debug or release

One can choose between Clang and GCC now. The Wiki has been changed accordingly:

https://wl.widelands.org/wiki/Building_Widelands_on_macOS/

[GunChleoc (gunchleoc)]

Code LGTM :)

review: Approve

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 3796. State: failed. Details: https://travis-ci.org/widelands/widelands/builds/415851588.
Appveyor build 3595. State: failed. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_macos_build_app_compiler-3595.

[Klaus Halfmann (klaus-halfmann)]

Toni, you re doing a great job!

Just give me some time to catch up with your speed :-)

You may contact me via PM or such, so we can align on our skills.
Ill try to review this this week but will not promise anything ...

[Klaus Halfmann (klaus-halfmann)]

Mmh, this does not contain the ASAN Flags as we have them in compile.sh,
this would make sense for a debug build (yes, it _is_ slower :-)

OTOH wee need this just for the release in then End, I would guess.

I use macports so lets try this now ....

wlbuild klaus$ ./build_app.sh clang debug ../macos_build_app_compiler

   Source: ../macos_build_app_compiler
   Version: r8783
   Destination: WidelandsRelease
   Type: Debug
   macOS: 10.13
   Compiler: Apple LLVM version 9.1.0 (clang-902.0.39.2)

./build_app.sh: line 140: brew: command not found

OK so this is tied to homebrew (will not install this here)
There is a script / reciept for macports, too.
Are you interested to get into contact with the maintainer?
(I think there is some howto for macports on the homepage, too)

Code LGTM. so maybe you want to provide me with some .dmg derived from your script?

As this will not affect any build this can go in.

[Toni Förster (stonerl)]

$ ./build_app.sh clang debug ../macos_build_app_compiler

gives me this output later on

-- Using AddressSanitizer http://clang.llvm.org/docs/AddressSanitizer.html

So it is using it without explicitly declaring it. Does work with clang. gcc currently gives me headaches, though. It seems to use the clangs dylib.

Still do some testings and gonna add some stuff to the script. When done I'll give you some .dmg

[Klaus Halfmann (klaus-halfmann)]

I rember that SirVer used brew as well for R19 and I used macports
and we had some significat differences in the end, was much bigger, or such.

SirVer, Gun: do we have that Ticket still in some Archive?

Once we target R20 we should use all variants and use the smalles/fastest on.
Lets see.

[Toni Förster (stonerl)]

Here are the clang builds:

http://fosuta.org/widelands_64bit_r8786.dmg
http://fosuta.org/widelands_64bit_r8786_Debug.dmg

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 3800. State: failed. Details: https://travis-ci.org/widelands/widelands/builds/416061570.
Appveyor build 3599. State: success. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_macos_build_app_compiler-3599.

[GunChleoc (gunchleoc)]

> SirVer, Gun: do we have that Ticket still in some Archive?

There is nothing relevant with the "macos" tag in the archive. Of course, such a bug might have been fix released before I did the tag cleanup.

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 3805. State: passed. Details: https://travis-ci.org/widelands/widelands/builds/416197711.
Appveyor build 3604. State: success. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_macos_build_app_compiler-3604.

[Toni Förster (stonerl)]

@Klaus

is there something like "brew --prefix" for MacPorts? We could easily adapt the script to handle both package manager.

Also ASsam is enabled by default for debug builds in the "CMakeLists.txt" That's why I havn't had to enable it for debug.

I disabled it for GCC, though.

[Klaus Halfmann (klaus-halfmann)]

@Toni

We already have a description to build developer verions using macports:
https://wl.widelands.org/wiki/Download/

And some dsicussion around this as well: https://wl.widelands.org/forum/topic/2627/

And the corresponding Portfile, which includes some packagnh, too:

https://github.com/kencu/myports/blob/master/games/widelands-devel/Portfile

We may have to Adopt this along with your changes.

I am downloading your .dmg and will check them (running, compare binary size etc.)

In theory macports and Brew should result in the +/- same binary, but, well ....

[Klaus Halfmann (klaus-halfmann)]

Approve, checked both images, (debug build is 50M,
my develop build is 67M but all libs are staically linked)

I only tried to open a savegame. (which was from some other brach, and failed to load, well)

travis currebtly has some erro as of a broken test / savegame.
We can either merge trunk to fix this or use the force, what do you think :-)

review: Approve (dowload,short testrun)

[Toni Förster (stonerl)]

I just added a check for ccache and added gettext to $PATH to avoid force linking it (removed thsi line from the wiki). Also merged from trunk.

ASan stays disabled for GCC since it does not pass the test and throws error like this one:

    Start 1: test_economy
1/4 Test #1: test_economy .....................***Failed 1.23 sec
==3314==The following global variable is not properly aligned.
==3314==This may happen if another global with the same name
==3314==resides in another non-instrumented module.
==3314==Or the global comes from a C file built w/o -fno-common.
==3314==In either case this is likely an ODR violation bug,
==3314==but AddressSanitizer can not provide more details.
=================================================================
==3314==ERROR: AddressSanitizer: odr-violation (0x001c0000003a):
ASAN:DEADLYSIGNAL
AddressSanitizer: nested bug in the same thread, aborting.

    Start 2: test_io_filesystem
2/4 Test #2: test_io_filesystem ...............***Failed 0.47 sec
==3315==The following global variable is not properly aligned.
==3315==This may happen if another global with the same name
==3315==resides in another non-instrumented module.
==3315==Or the global comes from a C file built w/o -fno-common.
==3315==In either case this is likely an ODR violation bug,
==3315==but AddressSanitizer can not provide more details.
=================================================================
==3315==ERROR: AddressSanitizer: odr-violation (0x0001000000e3):
ASAN:DEADLYSIGNAL
AddressSanitizer: nested bug in the same thread, aborting.

    Start 3: notifications_test
3/4 Test #3: notifications_test ............... Passed 0.43 sec
    Start 4: test_scripting
4/4 Test #4: test_scripting ...................***Failed 0.26 sec
==3317==The following global variable is not properly aligned.
==3317==This may happen if another global with the same name
==3317==resides in another non-instrumented module.
==3317==Or the global comes from a C file built w/o -fno-common.
==3317==In either case this is likely an ODR violation bug,
==3317==but AddressSanitizer can not provide more details.
=================================================================
==3317==ERROR: AddressSanitizer: odr-violation (0x000100000162):
ASAN:DEADLYSIGNAL
AddressSanitizer: nested bug in the same thread, aborting.

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 3809. State: passed. Details: https://travis-ci.org/widelands/widelands/builds/416493186.
Appveyor build 3608. State: success. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_macos_build_app_compiler-3608.

[GunChleoc (gunchleoc)]

Let's have it :)

@bunnybot merge

=== modified file 'CMakeLists.txt'
--- CMakeLists.txt	2018-05-16 19:37:59 +0000
+++ CMakeLists.txt	2018-08-15 18:37:08 +0000
@@ -79,6 +79,14 @@
   endif()
 endif()
 
+# Disable no symbols warning on macOS
+if (APPLE)
+    SET(CMAKE_C_ARCHIVE_CREATE   "<CMAKE_AR> Scr <TARGET> <LINK_FLAGS> <OBJECTS>")
+    SET(CMAKE_CXX_ARCHIVE_CREATE "<CMAKE_AR> Scr <TARGET> <LINK_FLAGS> <OBJECTS>")
+    SET(CMAKE_C_ARCHIVE_FINISH   "<CMAKE_RANLIB> -no_warning_for_no_symbols -c <TARGET>")
+    SET(CMAKE_CXX_ARCHIVE_FINISH "<CMAKE_RANLIB> -no_warning_for_no_symbols -c <TARGET>")
+endif()
+
 # TODO(sirver): One day, this should be enabled. Then we have no more cycles in our dependencies....
 # set_property(GLOBAL PROPERTY GLOBAL_DEPENDS_NO_CYCLES ON)
 

=== modified file 'utils/macos/build_app.sh'
--- utils/macos/build_app.sh	2018-08-12 19:02:16 +0000
+++ utils/macos/build_app.sh	2018-08-15 18:37:08 +0000
@@ -2,9 +2,54 @@
 
 set -e
 
-if [ "$#" == "0" ]; then
-	echo "Usage: $0 <bzr_repo_directory>"
-	exit 1
+USAGE="Usage: $0 <clang|gcc> <debug|release> <bzr_repo_directory>"
+USE_ASAN="OFF"
+
+if [ ! -z "$3" ]; then
+   case "$2" in
+   debug|Debug)
+      TYPE="Debug"
+      if [ "$1" == "clang" ]; then
+         USE_ASAN="ON"
+         # Necessary to avoid linking errors later on
+         ASANLIB=$(echo "int main(void){return 0;}" | xcrun clang -fsanitize=address \
+         -xc -o/dev/null -v - 2>&1 |   tr ' ' '\n' | grep libclang_rt.asan_osx_dynamic.dylib)
+         mkdir -p "@rpath"
+         ln -fs "$ASANLIB" "@rpath/"
+      fi
+      ;;
+   release|Release)
+      TYPE="Release"
+      ;;
+   *)
+      echo $USAGE
+      exit 1
+      ;;
+   esac
+   case "$1" in
+   clang)
+      C_COMPILER="clang"
+      CXX_COMPILER="clang++"
+      COMPILER=$(clang --version | grep "clang")
+      ;;
+   gcc)
+      C_COMPILER="gcc-7"
+      CXX_COMPILER="g++-7"
+      COMPILER=$(gcc-7 --version | grep "GCC")
+      ;;
+   *)
+      echo $USAGE
+      exit 1
+      ;;
+   esac
+   if [ ! -z $(type -p ccache) ]; then
+      C_COMPILER="$(brew --prefix ccache)/libexec/$C_COMPILER"
+      CXX_COMPILER="$(brew --prefix ccache)/libexec/$CXX_COMPILER"
+   fi
+   SOURCE_DIR=$3
+else
+   echo $USAGE
+   exit 1
 fi
 
 # Check if the SDK for the minimum build target is available.
@@ -17,10 +62,9 @@
    SDK_DIRECTORY="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX$OSX_VERSION.sdk"
 fi
 
-SOURCE_DIR=$1
 REVISION=`bzr revno $SOURCE_DIR`
 DESTINATION="WidelandsRelease"
-TYPE="Release"
+
 if [[ -f $SOURCE_DIR/WL_RELEASE ]]; then
    WLVERSION="$(cat $SOURCE_DIR/WL_RELEASE)"
 else
@@ -33,6 +77,7 @@
 echo "   Destination: $DESTINATION"
 echo "   Type:        $TYPE"
 echo "   macOS:       $OSX_MIN_VERSION"
+echo "   Compiler:    $COMPILER"
 echo ""
 
 function MakeDMG {
@@ -46,7 +91,11 @@
    cp $SOURCE_DIR/COPYING  $DESTINATION/COPYING.txt
 
    echo "Creating DMG ..."
-   hdiutil create -fs HFS+ -volname "Widelands $WLVERSION" -srcfolder "$DESTINATION" "$UP/widelands_64bit_$WLVERSION.dmg"
+   if [ "$TYPE" == "Release" ]; then
+      hdiutil create -fs HFS+ -volname "Widelands $WLVERSION" -srcfolder "$DESTINATION" "$UP/widelands_64bit_$WLVERSION.dmg"
+   elif [ "$TYPE" == "Debug" ]; then
+      hdiutil create -fs HFS+ -volname "Widelands $WLVERSION" -srcfolder "$DESTINATION" "$UP/widelands_64bit_${WLVERSION}_${TYPE}.dmg"
+   fi
 }
 
 function CopyLibrary {
@@ -109,24 +158,28 @@
 }
 
 function BuildWidelands() {
-   PREFIX_PATH="$(brew --prefix libpng)"
+   PREFIX_PATH=";$(brew --prefix gettext)"
    PREFIX_PATH+=";$(brew --prefix jpeg)"
    PREFIX_PATH+=";$(brew --prefix libpng)"
    PREFIX_PATH+=";$(brew --prefix python)"
    PREFIX_PATH+=";$(brew --prefix zlib)"
    PREFIX_PATH+=";/usr/local"
    PREFIX_PATH+=";/usr/local/Homebrew"
-
+   
+   export PATH="$(brew --prefix gettext)/bin:$PATH"
    export SDL2DIR="$(brew --prefix sdl2)"
    export SDL2IMAGEDIR="$(brew --prefix sdl2_image)"
    export SDL2MIXERDIR="$(brew --prefix sdl2_mixer)"
    export SDL2TTFDIR="$(brew --prefix sdl2_ttf)"
    export BOOST_ROOT="$(brew --prefix boost)"
-   export ICU_ROOT="$(brew --prefix icu4c)"
+   
+   # Not needed for CMake 3.12 or above
+   # see cmake --help-policy CMP0074
+   #export ICU_ROOT="$(brew --prefix icu4c)"
 
    cmake $SOURCE_DIR -G Ninja \
-      -DCMAKE_C_COMPILER:FILEPATH="$(brew --prefix ccache)/libexec/gcc-7" \
-      -DCMAKE_CXX_COMPILER:FILEPATH="$(brew --prefix ccache)/libexec/g++-7" \
+      -DCMAKE_C_COMPILER:FILEPATH="$C_COMPILER" \
+      -DCMAKE_CXX_COMPILER:FILEPATH="$CXX_COMPILER" \
       -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING="$OSX_MIN_VERSION" \
       -DCMAKE_OSX_SYSROOT:PATH="$SDK_DIRECTORY" \
       -DCMAKE_INSTALL_PREFIX:PATH="$DESTINATION/Widelands.app/Contents/MacOS" \
@@ -134,7 +187,8 @@
       -DCMAKE_BUILD_TYPE:STRING="$TYPE" \
       -DGLEW_INCLUDE_DIR:PATH="$(brew --prefix glew)/include" \
       -DGLEW_LIBRARY:PATH="$(brew --prefix glew)/lib/libGLEW.dylib" \
-      -DCMAKE_PREFIX_PATH:PATH="${PREFIX_PATH}"
+      -DCMAKE_PREFIX_PATH:PATH="${PREFIX_PATH}" \
+      -DOPTION_ASAN="$USE_ASAN"
    ninja
 
    echo "Done building."