Merge lp:~ceejatec/zorba/build-doc into lp:zorba

[Chris Hillery]

Adding more clear (if still brief) documentation about building Zorba with non-core modules.

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/build-doc-2011-09-23T11-03-05.402Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job build-doc-2011-09-23T11-03-05.402Z is finished. The final status was:

All tests succeeded!

[Zorba Build Bot (zorba-buildbot)]

Voting does not meet specified criteria. Required: Approve > 0, Disapprove < 1. Got: 1 Pending.

[Chris Hillery (ceejatec)]

http://zorbatest.lambda.nu:8080/remotequeue/build-doc-2011-09-23T11-03-05.402Z/bzr/zorba/build/doc/zorba/html/build.html

This is a link to the main build.html. See my changes here:

1. Added mention of including non-core modules in build as step 2 of primary Zorba build instructions.
2. Added new page build_noncore.html which includes instructions for invoking DownloadModules and other details of building noncore modules
3. Moved all non-core module specific information (such as building C-Client, Jansson, all that) onto build_noncore.html page.
4. Slight touchups to installation page (still a mess - I filed bug 857263 about that and more)

[Cezar Andrei (cezar-andrei)]

Approved.

review: Approve

[Zorba Build Bot (zorba-buildbot)]

Validation queue starting for merge proposal.
Log at: http://zorbatest.lambda.nu:8080/remotequeue/build-doc-2011-09-23T22-46-07.005Z/log.html

[Zorba Build Bot (zorba-buildbot)]

Validation queue job build-doc-2011-09-23T22-46-07.005Z is finished. The final status was:

All tests succeeded!

=== modified file 'doc/zorba/build.dox'
--- doc/zorba/build.dox	2011-09-08 21:01:52 +0000
+++ doc/zorba/build.dox	2011-09-23 11:00:30 +0000
@@ -1,38 +1,4 @@
-/** \page build Build Instructions
-
-  <ul>
-    <li> Table of Contents
-    <ul>
-      <li> \ref requirements
-      <ul>
-        <li> \ref buildsystem
-        <li> \ref compilers
-        <li> \ref required_libs
-        <li> \ref buildingzorba
-      </ul>
-      <li> \ref buildcclient
-      <ul>
-        <li> \ref cclientunix
-        <li> \ref cclientwindows
-      </ul>
-      <li> \ref buildjansson
-      <ul>
-        <li> \ref janssonunix
-        <li> \ref janssonwindows
-      </ul> 
-      <li> \ref buildimagemagick
-      <ul>
-        <li> \ref imagemagickunix
-        <li> \ref imagemagickwindows
-      </ul>  
-      <li> \ref windows
-      <ul>
-        <li> \ref nmake
-        <li> \ref visualstudio
-        <li> \ref https_support
-      </ul>
-    </ul>
- </ul>
+/** \page build Zorba Build Instructions
 
 \section requirements Requirements
 
@@ -45,7 +11,7 @@
 \subsection compilers Compilers
 Zorba is tested with the following compilers:
 - GNU Compiler: GCC 3.4.x (32-bit & 64-bit) and GCC 4.x.x
-- Microsoft Compiler: MS VC++ 2008, and MS VC++ 2010
+- Microsoft Compiler: MS VC++ 2008, and MS VC++ 2010 (including Express)
 
 \subsection required_libs Required Libraries
 In order to build Zorba, you need the following libraries and development headers:
@@ -100,39 +66,16 @@
 - PHP (http://www.php.net/downloads.php)
 - Python (http://www.python.org/download/)
 
-\subsubsection required_non_core_modules Required for Non-Core Modules
-Zorba provides a huge variety of <a href="../../../html/modules.html">XQuery modules</a>.
-For most of the (external) modules, you need additional libraries.
-All such modules are not build if the library (and development headers) they require are not available.
-In the following, please find a list of required libraries.
-
-- data-converters (JSon, CSV, HTML)
-  - Tidy (http://tidy.sourceforge.net)
-  - Jansson (http://www.digip.org/jansson/)
-- data-formatting (XSL-FO)
-  - Java (http://java.sun.com/javase/downloads/index.jsp)
-  - Apache FOP (http://xmlgraphics.apache.org/fop/download.html)
-- email
-  - IMAP CClient (http://www.washington.edu/imap/)
-- geo
-  - GEOS version 3.2.2 or later (http://trac.osgeo.org/geos/)
-- EXPath http-client
-  - CURL 7.12 or later (http://curl.haxx.se/)
-- image
-  - ImageMagick (http://www.imagemagick.org)
-- languages (XSLT)
-  - LibXslt version 1.1.24 or later (http://xmlsoft.org/XSLT/)
-
-Please note that some of these modules depend on other modules.
-For example, the EXPath http-client module depends on Zorba's http-client module and the tidy data-converter.
-Please see our <a href="../../zorba/xqdoc/xhtml/images/modules.svg">module-interdependency graph</a> for details.
   
-\subsection buildingzorba Building Zorba
+\section buildingzorba Building Zorba
 -# Install the Zorba source distribution (see \ref installation).
   The directory in which the Zorba sources are installed
   is referred to as \c [ZORBA] in the following.
--# Change the working directory into \c [ZORBA].
--# The recommended way to build Zorba is by creating an
+-# If desired, also install or download the source for any non-core modules
+  that you would like to build along with Zorba, and ensure that you have
+  any libraries required by those modules available. See \ref build_noncore
+  for more details.
+-# Zorba requires an
    <a href="http://www.cmake.org/Wiki/CMake_FAQ#What_is_an_.22out-of-source.22_build.3F" target="_blank">out-of-source build</a>.
    We suggest to create the directory \c [ZORBA]/build
    and refer to this directory as \c [ZORBABUILD] in the following.
@@ -141,18 +84,21 @@
    <tt>cmake [ZORBA]</tt>.
    In case the \c [ZORBABUILD]
    is located directly within the \c [ZORBA] directory,
-   just type <tt>cmake ..</tt> .
+   you can just type <tt>cmake ..</tt> .
    This command should configure Zorba and prepare for the build.
    CMake will tell you if your installation is missing
    some of the required libraries or development headers.
 -# If CMake executed successfully,
-   you should be able to run \c make.
-   Running make will take some time.
-   If \c make finishes successfully,
+   you should be able to build the project. For Makefile-based builds,
+   just type \c make (or \c make -j2 to do a parallel build on a multi-core
+   machine). For IDE-based builds, open the project created in the previous
+   step and build the \c ALL target.
+   The build will take some time.
+   If it finishes successfully,
    you're ready to install and run Zorba
    (see \ref installation).
 
-\subsubsection buildoptions Build Options
+\subsection buildoptions Build Options
 
 - CMake is a meta build system,
   meaning that it is able to generate native makefiles
@@ -173,147 +119,8 @@
 - Zorba has other options as well.
   You can tweak the performance and library footprint
   by enabling or disabling various features from Zorba.
-  Details here \ref options_and_annotations .
-
-\section buildcclient Building IMAP CClient
-
-Zorba provides email support using the CClient library part of the <a href="http://www.washington.edu/imap/" target="_blank">UW IMAP toolkit</a>.
-
-\subsection cclientunix Unix/Linux/Mac OS X
-Notes:
-- There are some known issues with the CClient packages
-  that come with diffrent Linux distributions.
-- On x32 bit OpenSuse and also on x64 bit Ubuntu,
-  we noticed that the CClient shared library is broken
-  (undefined symbol: mm_dlog).
-- Due to that fact that Mark Crispin
-  (the creator of CClient library)
-  does not support CClient as a shared library but only as a static library
-  (see <a href="http://www.washington.edu/imap/IMAP-FAQs/index.html#6.3" target="_blank">FAQs shared library</a>),
-  we strongly suggest you want to get the
-  <a href="http://www.washington.edu/imap/" target="_blank">UW IMAP toolkit</a>
-  and compile it yourself.
-- On Linux 64-bit,
-  you might discover a problem with the optional package cclient.
-  E.g. on Ubuntu 64-bit, you might discover the following:
-  \code
-Linking CXX shared library libsmtp.so
-/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib/libc-client.a(osdep.o): relocation R_X86_64_32 against
-  `server_input_wait' can not be used when making a shared object; recompile with -fPIC
-/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib/libc-client.a: could not read symbols: Bad value
-  \endcode
-  To fix this,
-  you need to compile CClient yourself.
-  The commands to compile CClient correctly are:
-  \code
-cd <some_directory>
-wget ftp://ftp.cac.washington.edu/imap/imap-2007e.tar.gz
-tar -xf imap-2007e.tar.gz
-cd imap-2007e
-make slx EXTRACFLAGS="-I/usr/include/openssl -fPIC"
-  \endcode
-
-- Also please keep in mind that if SSL/TLS authentication is required
-  by the SMTP server,
-  then you first need to install
-  <a href="http://www.openssl.org/" target="_blank">OpenSSL</a>
-  and configure CClient to use it.
-- Make sure the name of the library is prefixed by \c lib
-  and suffixed with \c .a
-  (for example \c libc-client.a on Linux/Unix
-  or \c libc-client4.a on Mac OS X).
-
-Use the following extra CMake arguments when building Zorba:
-\verbatim
--D "CCLIENT_INCLUDE=path_to_imap-2007e\c-client"
--D CCLIENT_LIBRARY=path_to_imap-2007e\c-client\libc-client.a"
-\endverbatim
-
-Here are some quick suggestions to build CClient on Linux:
-- x32-bit Linux: <tt>make lnp</tt>
-- x64-bit Linux: <tt>make lnp  EXTRACFLAGS="-I/usr/include/openssl -fPIC" EXTRAAUTHENTICATORS=gss</tt> in order to build with SSL and Kerberos support.<br /></ul>
-
-For more detailed build instructions,
-please see the
-<a href="http://www.washington.edu/imap/documentation" target="_blank">UW IMAP toolkit "Server Documentation"</a>
-and the <a href="http://www.washington.edu/imap/IMAP-FAQs/index.html" target="_blank">UW IMAP FAQs</a>.
-
-\subsection cclientwindows Windows
-
-You must build the
-<a href="http://www.washington.edu/imap/" target="_blank">UW IMAP toolkit</a>.
-
-Use the following extra CMake arguments when building Zorba:
-\code
--D "CCLIENT_INCLUDE=path_to_imap-2007e\c-client"
--D "CCLIENT_LIBRARY=path_to_imap-2007e\c-client\release\cclient.lib"
-\endcode
-
-\section buildjansson Building Jansson
-
-Zorba provides Json manipulation support
-using the Jansson Library (http://www.digip.org/jansson/).
-
-\subsection janssonunix Unix/Linux/Mac OS X
-
-For Linux/Unix systems,
-download the Jansson library from http://www.digip.org/jansson/releases/.
-Then follow the instructions from
-http://www.digip.org/jansson/doc/2.0/gettingstarted.html#compiling-and-installing-jansson.
-
-The source use GNU Autotools
-(autoconf, automake, libtool),
-so compiling and installing is extremely simple:
-\code
-./configure
-make
-make check
-make install
-\endcode
-
-On Fedora starting with version 12, Jansson is also available as a package,
-so (as root) you can simply run: 
-\code
-yum install -y jansson-devel
-\endcode
-
-Ubuntu packages are available from PPA of
-https://launchpad.net/~petri/+archive/ppa
-
-\subsection janssonwindows Windows
-
-For Windows please follow these steps:
-- download Jansson from https://github.com/akheron/jansson/zipball/master
-- rename src/jansson_config.h.win32 to src/jansson_config.h
-- replace \c snprintf with \c _snprintf in \c src/dump.c lines 188 and 202
-  and \c load.c lines 96 and 108
-- replace \c json_strtoint with \c strtol in \c load.c line 481
-- add all headers and source files in src folder and create a static library
-
-There will be a new Jansson 2.0.1 release
-that will include the fixed described above.
-
-\section buildimagemagick Building ImageMagick
-
-Zorba provides image manipulation support
-using the ImageMagick Library (http://www.imagemagick.org).
-
-\subsection imagemagickunix Unix/Linux/Mac OS X
-
-For Linux/Unix systems,
-just use the package manager to install the
-magick++, magickwand, magickcore packages
-including the developer versions of these packages
-(that inlcude the header files).
-
-On OSX, use macports to install imagemagick.
-
-\subsection imagemagickwindows Windows
-
-For Windows,
-just download the ImageMagick windows binary (Q8-windows-dll) and install it.
-
-\section mac Note for Mac OS X Users 
+
+\section mac Notes for Mac OS X Users 
 
 The easiest way to install the required packages (like CMake or Xerces-C) is
 to use Macports (http://macports.org/).
@@ -353,9 +160,8 @@
   indicated in the \ref required section):
   - Libxml2 and Iconv: http://www.zlatkovic.com/libxml.en.html
   - CURL: http://curl.haxx.se/download.html
-  - Tidy: http://dev.int64.org/tidy.html
   - Libxslt: http://www.zlatkovic.com/pub/libxml/
-  - IMAP CClient: http://www.washington.edu/imap/
+
 - Zorba has in place an automatic DLL detection mechanism.
   This will try to automatically gather all the DLLs
   from the third party librbaries

=== added file 'doc/zorba/build_noncore.dox'
--- doc/zorba/build_noncore.dox	1970-01-01 00:00:00 +0000
+++ doc/zorba/build_noncore.dox	2011-09-23 11:00:30 +0000
@@ -0,0 +1,218 @@
+/** \page build_noncore Building Non-core Zorba Modules
+
+\section noncore_download Downloading source code for non-core modules
+
+Currently, the Zorba team does not provide source downloads for the
+non-core modules. However, there is a simple CMake script inside Zorba
+which will allow you to check out these modules from source control
+easily. Note that this requires having the Subversion utility available
+on your system.
+
+From your [ZORBA] directory, type
+\code
+  cmake -Doutdir=../zorba_modules -Dmodname=NAME -P modules/DownloadModules.cmake
+\endcode
+
+to download the particular module package NAME (see \ref
+non_core_modules for a complete list of module packages and their
+contents), or
+
+\code
+  cmake -Doutdir=../zorba_modules -Dallmodules=1 -P modules/DownloadModules.cmake
+\endcode
+
+to download all the modules. (On Windows, you will need to use backslashes instead of forward slashes in the above paths.)
+
+This will place the downloaded module source code in the directory
+\c ../zorba_modules, which is where the Zorba build will look for them by
+default. If you wish to download them to some other location, you may do
+so; in that case, when you configure the Zorba build with CMake, provide
+the \c -DZORBA_MODULES_DIR=/full/path/to/modules argument.
+
+\subsection noncore_dependencies Dependencies among Non-core Modules
+
+Please note that some of these modules depend on other modules.  Most
+notably, the EXPath http-client module (in the \c http-client module
+package) depends on the \c tidy module, which is in the \c data-converters module package. So, if you download the \c http-client package, you must also download the \c data-converters package (and ensure that libtidy is installed, as mentioned below).
+
+Please see our <a
+href="../../zorba/xqdoc/xhtml/images/modules.svg">module-interdependency
+graph</a> for details.
+
+
+\section noncore_requirements Non-core Module Requirements
+For many of the non-core modules, you need additional libraries.
+All such modules are not built if the library (and development headers) they require are not available.
+Here is a list of the libraries required by the current non-core modules packages:
+
+- data-converters (JSon, CSV, HTML)
+  - Tidy (http://tidy.sourceforge.net)
+  - Jansson (http://www.digip.org/jansson/)
+- data-formatting (XSL-FO)
+  - Java (http://java.sun.com/javase/downloads/index.jsp)
+  - Apache FOP (http://xmlgraphics.apache.org/fop/download.html)
+- email
+  - IMAP CClient (http://www.washington.edu/imap/)
+- geo
+  - GEOS version 3.2.2 or later (http://trac.osgeo.org/geos/)
+- EXPath http-client
+  - CURL 7.12 or later (http://curl.haxx.se/)
+- image
+  - ImageMagick (http://www.imagemagick.org)
+- languages (XSLT)
+  - LibXslt version 1.1.24 or later (http://xmlsoft.org/XSLT/)
+
+
+In many cases you can find binary packages for these dependencies, which 
+will be the quickest and easiest way to meet the requirements. We have
+provided links or instructions for downloading binary packages for these
+libraries below, along with some instructions that we have found when
+building these libraries from source.
+
+
+\subsection buildcclient Building IMAP CClient
+
+Zorba provides email support using the CClient library part of the <a href="http://www.washington.edu/imap/" target="_blank">UW IMAP toolkit</a>.
+
+\subsubsection cclientunix Unix/Linux/Mac OS X
+Notes:
+- There are some known issues with the CClient packages
+  that come with diffrent Linux distributions.
+- On x32 bit OpenSuse and also on x64 bit Ubuntu,
+  we noticed that the CClient shared library is broken
+  (undefined symbol: mm_dlog).
+- Due to that fact that Mark Crispin
+  (the creator of CClient library)
+  does not support CClient as a shared library but only as a static library
+  (see <a href="http://www.washington.edu/imap/IMAP-FAQs/index.html#6.3" target="_blank">FAQs shared library</a>),
+  we strongly suggest you want to get the
+  <a href="http://www.washington.edu/imap/" target="_blank">UW IMAP toolkit</a>
+  and compile it yourself.
+- On Linux 64-bit,
+  you might discover a problem with the optional package cclient.
+  E.g. on Ubuntu 64-bit, you might discover the following:
+  \code
+Linking CXX shared library libsmtp.so
+/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib/libc-client.a(osdep.o): relocation R_X86_64_32 against
+  `server_input_wait' can not be used when making a shared object; recompile with -fPIC
+/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib/libc-client.a: could not read symbols: Bad value
+  \endcode
+  To fix this,
+  you need to compile CClient yourself.
+  The commands to compile CClient correctly are:
+  \code
+cd <some_directory>
+wget ftp://ftp.cac.washington.edu/imap/imap-2007e.tar.gz
+tar -xf imap-2007e.tar.gz
+cd imap-2007e
+make slx EXTRACFLAGS="-I/usr/include/openssl -fPIC"
+  \endcode
+
+- Also please keep in mind that if SSL/TLS authentication is required
+  by the SMTP server,
+  then you first need to install
+  <a href="http://www.openssl.org/" target="_blank">OpenSSL</a>
+  and configure CClient to use it.
+- Make sure the name of the library is prefixed by \c lib
+  and suffixed with \c .a
+  (for example \c libc-client.a on Linux/Unix
+  or \c libc-client4.a on Mac OS X).
+
+Use the following extra CMake arguments when building Zorba:
+\verbatim
+-D "CCLIENT_INCLUDE=path_to_imap-2007e\c-client"
+-D CCLIENT_LIBRARY=path_to_imap-2007e\c-client\libc-client.a"
+\endverbatim
+
+Here are some quick suggestions to build CClient on Linux:
+- x32-bit Linux: <tt>make lnp</tt>
+- x64-bit Linux: <tt>make lnp  EXTRACFLAGS="-I/usr/include/openssl -fPIC" EXTRAAUTHENTICATORS=gss</tt> in order to build with SSL and Kerberos support.<br /></ul>
+
+For more detailed build instructions,
+please see the
+<a href="http://www.washington.edu/imap/documentation" target="_blank">UW IMAP toolkit "Server Documentation"</a>
+and the <a href="http://www.washington.edu/imap/IMAP-FAQs/index.html" target="_blank">UW IMAP FAQs</a>.
+
+\subsubsection cclientwindows Windows
+
+You must build the
+<a href="http://www.washington.edu/imap/" target="_blank">UW IMAP toolkit</a>.
+
+Use the following extra CMake arguments when building Zorba:
+\code
+-D "CCLIENT_INCLUDE=path_to_imap-2007e\c-client"
+-D "CCLIENT_LIBRARY=path_to_imap-2007e\c-client\release\cclient.lib"
+\endcode
+
+\subsection buildjansson Building Jansson
+
+Zorba provides Json manipulation support
+using the Jansson Library (http://www.digip.org/jansson/).
+
+\subsubsection janssonunix Unix/Linux/Mac OS X
+
+For Linux/Unix systems,
+download the Jansson library from http://www.digip.org/jansson/releases/.
+Then follow the instructions from
+http://www.digip.org/jansson/doc/2.0/gettingstarted.html#compiling-and-installing-jansson.
+
+The source use GNU Autotools
+(autoconf, automake, libtool),
+so compiling and installing is extremely simple:
+\code
+./configure
+make
+make check
+make install
+\endcode
+
+On Fedora starting with version 12, Jansson is also available as a package,
+so (as root) you can simply run: 
+\code
+yum install -y jansson-devel
+\endcode
+
+Ubuntu packages are available from PPA of
+https://launchpad.net/~petri/+archive/ppa
+
+\subsubsection janssonwindows Windows
+
+For Windows please follow these steps:
+- download Jansson from https://github.com/akheron/jansson/zipball/master
+- rename src/jansson_config.h.win32 to src/jansson_config.h
+- replace \c snprintf with \c _snprintf in \c src/dump.c lines 188 and 202
+  and \c load.c lines 96 and 108
+- replace \c json_strtoint with \c strtol in \c load.c line 481
+- add all headers and source files in src folder and create a static library
+
+There will be a new Jansson 2.0.1 release
+that will include the fixed described above.
+
+\subsection buildimagemagick Building ImageMagick
+
+Zorba provides image manipulation support
+using the ImageMagick Library (http://www.imagemagick.org).
+
+\subsubsection imagemagickunix Unix/Linux/Mac OS X
+
+For Linux/Unix systems,
+just use the package manager to install the
+magick++, magickwand, magickcore packages
+including the developer versions of these packages
+(that inlcude the header files).
+
+On OSX, use macports to install imagemagick.
+
+\subsubsection imagemagickwindows Windows
+
+For Windows,
+just download the ImageMagick windows binary (Q8-windows-dll) and install it.
+
+\subsection buildtidy Libtidy
+
+\subsubsection buildtidywindows Windows
+
+A binary installer for Tidy is available at: http://dev.int64.org/tidy.html
+
+
+*/

=== modified file 'doc/zorba/installation.dox'
--- doc/zorba/installation.dox	2011-07-28 14:07:29 +0000
+++ doc/zorba/installation.dox	2011-09-23 11:00:30 +0000
@@ -10,9 +10,9 @@
 in your current working directory.
 
 To build Zorba,
-please follow the build instructions (see \ref build). 
+please follow the build instructions (see \ref buildingzorba). 
 
-\subsection BinaryInstallation Installing the Binary Distribution
+\subsection Installing Installing
 
 Once you have successfully built Zorba using the "Unix Makefile" generator
 (see \ref SourceInstallation),
@@ -27,10 +27,10 @@
 For example,
 in order to install Zorba in <tt>/opt</tt>
 use <tt>cmake -D CMAKE_INSTALL_PREFIX=/opt </tt><em>zorba source directory</em>.
+(Note that while CMake-generated Makefiles support the \c DESTDIR variable,
+this will NOT work with Zorba as certain installation paths are hard-coded
+in the binary files.)
 
-However,
-CMake generated makefiles also support the make \c DESTDIR variable
-(see the <a href="http://www.cmake.org/Wiki/CMake_FAQ#Does_CMake.27s_.22make_install.22_support_DESTDIR.3F" target="_blank">CMAKE FAQ</a>).
 There is also a <tt>make uninstall</tt> command available
 (also to be called from the build directory).