Merge lp:~ted/snapcraft/pkg-config-sysroot into lp:~snappy-dev/snapcraft/core

[Ted Gould]

Ensure C library configuration tools don't use system paths

Creating a new dev box with Wily and realized that previously some deps were getting pulled from my system. Fixing them.

[Sergio Schvezov (sergiusens)]

Looks good, I'd just move the method to repo.Ubuntu.

Also a small integration or unit test would be great.

review: Needs Fixing

[Ted Gould (ted)]

Kinda a crazy test, but it does test the feature.

[Sergio Schvezov (sergiusens)]

ty vm

review: Approve

[Snappy Tarmac (snappydevtarmac)]

The attempt to merge lp:~ted/snapcraft/pkg-config-sysroot into lp:snapcraft failed. Below is the output from the failed tests.

The project has gotten complex.
Here's the list of units exceeding 10:
- ./snapcraft/cmds.py:
  284:1: 'cmd' 13

name: # the name of the snap
version: # the version of the snap
# The vendor for the snap (replace 'Vendor <email address hidden>')
vendor: Vendor <email address hidden>
summary: # 79 char long summary
description: # A longer description for the snap
icon: # A path to an icon for the package
cp --preserve=all -R zzz /tmp/tmphawmcd93/parts/copy/install/zzz
cp --preserve=all -R src /tmp/tmpiw9glgqg/parts/copy/install/dst
cp --preserve=all -R src /tmp/tmpaqsrsx5k/parts/copy/install/dir/dst

deb http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted
deb http://ports.ubuntu.com/ubuntu-ports/ trusty universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates universe
deb http://ports.ubuntu.com/ubuntu-ports/ trusty multiverse
deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates multiverse
deb http://ports.ubuntu.com/ubuntu-ports trusty-security main restricted
deb http://ports.ubuntu.com/ubuntu-ports trusty-security universe
deb http://ports.ubuntu.com/ubuntu-ports trusty-security multiverse

===============================[ Analyzing Jobs ]===============================
=============================[ Session Statistics ]=============================
This session is about 0.00% complete
Estimated duration is 97.30 for automated jobs.
Estimated duration cannot be determined for manual jobs.
Size of the desired job list: 27
Size of the effective execution plan: 27
===========================[ Running Selected Jobs ]============================
--------------[ Running job 1 / 27. Estimated time left: 0:01:37 ]--------------
--------------------------[ snapcraft/normal/no-yaml ]--------------------------
ID: 2015.com.canonical.snapcraft::snapcraft/normal/no-yaml
Category: 2013.com.canonical.plainbox::uncategorised
... 8< -------------------------------------------------------------------------
Could not find snapcraft.yaml. Are you sure you are in the right directory? To start a new project, use 'snapcraft init'
------------------------------------------------------------------------- >8 ---
Outcome: job passed
--------------[ Running job 2 / 27. Estimated time left: 0:01:37 ]--------------
------------------------[ snapcraft/normal/no-yaml-run ]------------------------
ID: 2015.com.canonical.snapcraft::snapcraft/normal/no-yaml-run
Category: 2013.com.canonical.plainbox::uncategorised
... 8< -------------------------------------------------------------------------
Could not find snapcraft.yaml. Are you sure you are in the right directory? To start a new project, use 'snapcraft init'
------------------------------------------------------------------------- >8 ---
Outcome: job passed
--------------[ Running job 3 / 27. Estimated time left: 0:01:37 ]--------------
-----------------[ snapcraft/normal/assemble-meta-externally ]------------------
ID: 2015.com.canonical.snapcraft::snapcraft/normal/assemble-meta-ex...

[Daniel Holbach (dholbach)]

2015.com.canonical.snapcraft::snapcraft/normal/build-tool-paths: fail

+ cp -rT /tmp/tarmac/branch.l0nry5/integration-tests/data/build-tools .
+ snapcraft stage
No keyring installed in /tmp/cwd-_o0ohin1.5a9a153f54a0cb00280a728f7f26acc5a87ce06f9284b77abfa4b99251a0153d/parts/make-project/ubuntu/etc/apt/trusted.gpg.d/.No keyring installed in /tmp/cwd-_o0ohin1.5a9a153f54a0cb00280a728f7f26acc5a87ce06f9284b77abfa4b99251a0153d/parts/make-project/ubuntu/etc/apt/trusted.gpg.d/.No keyring installed in /tmp/cwd-_o0ohin1.5a9a153f54a0cb00280a728f7f26acc5a87ce06f9284b77abfa4b99251a0153d/parts/make-project/ubuntu/etc/apt/trusted.gpg.d/.+ snapcraft shell pkg-config --cflags glib-2.0
++ pwd
+ grep -q /tmp/cwd-_o0ohin1.5a9a153f54a0cb00280a728f7f26acc5a87ce06f9284b77abfa4b99251a0153d
Package glib-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `glib-2.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'glib-2.0' found

review: Needs Fixing

[lp:~ted/snapcraft/pkg-config-sysroot]

188. By Ted Gould on 2015-09-23

Add 'PKG_CONFIG_PATH' and break up long lines

189. By Ted Gould on 2015-09-23

Merge trunk

190. By Ted Gould on 2015-09-23

Make consistent with single quotes

[Sergio Schvezov (sergiusens)]

\o/

review: Approve

=== added directory 'integration-tests/data/build-tools'
=== added file 'integration-tests/data/build-tools/Makefile'
--- integration-tests/data/build-tools/Makefile	1970-01-01 00:00:00 +0000
+++ integration-tests/data/build-tools/Makefile	2015-09-23 15:54:59 +0000
@@ -0,0 +1,6 @@
+
+all:
+	@echo "Makin' like a Makefile"
+
+install:
+	@echo "Installin' like a Makefile"

=== added file 'integration-tests/data/build-tools/icon.svg'
--- integration-tests/data/build-tools/icon.svg	1970-01-01 00:00:00 +0000
+++ integration-tests/data/build-tools/icon.svg	2015-09-23 15:54:59 +0000
@@ -0,0 +1,1 @@
+<svg/>

=== added file 'integration-tests/data/build-tools/snapcraft.yaml'
--- integration-tests/data/build-tools/snapcraft.yaml	1970-01-01 00:00:00 +0000
+++ integration-tests/data/build-tools/snapcraft.yaml	2015-09-23 15:54:59 +0000
@@ -0,0 +1,14 @@
+name: build-tools-test
+version: 0.1
+vendor: Napoleon Bonaparte <[e-mail]>
+summary: one line summary
+description: a longer description
+icon: icon.svg
+
+parts:
+  make-project:
+    source: .
+    stage-packages:
+      - libxml2-dev
+      - libxslt1-dev
+      - libglib2.0-dev

=== modified file 'integration-tests/units/jobs.pxu'
--- integration-tests/units/jobs.pxu	2015-09-23 07:14:21 +0000
+++ integration-tests/units/jobs.pxu	2015-09-23 15:54:59 +0000
@@ -380,3 +380,15 @@
     test -f parts/python2/install/usr/lib/python2.7/argparse.py
     test -f parts/python3/install/usr/lib/python3.4/argparse.py
 flags: simple has-leftovers
+
+id: snapcraft/normal/build-tool-paths
+plugin: shell
+estimated_duration: 30
+command:
+    set -ex
+    cp -rT $PLAINBOX_PROVIDER_DATA/build-tools .
+    ${SNAPCRAFT} stage
+    ${SNAPCRAFT} shell pkg-config --cflags glib-2.0 | grep -q `pwd`
+    ${SNAPCRAFT} shell xml2-config --cflags | grep -q `pwd`
+    ${SNAPCRAFT} shell xslt-config --cflags | grep -q `pwd`
+flags: simple has-leftovers

=== modified file 'snapcraft/__init__.py'
--- snapcraft/__init__.py	2015-09-22 23:14:22 +0000
+++ snapcraft/__init__.py	2015-09-23 15:54:59 +0000
@@ -117,6 +117,13 @@
             ubuntu = snapcraft.repo.Ubuntu(self.ubuntudir, sources=self.PLUGIN_STAGE_SOURCES)
             ubuntu.get(self.PLUGIN_STAGE_PACKAGES + part_stage_packages)
             ubuntu.unpack(self.installdir)
+            self._fixup(self.installdir)
+
+    def _fixup(self, root):
+        if os.path.isfile(os.path.join(root, 'usr', 'bin', 'xml2-config')):
+            self.run(['sed', '-i', '-e', 's|prefix=/usr|prefix={}/usr|'.format(root), os.path.join(root, 'usr', 'bin', 'xml2-config')])
+        if os.path.isfile(os.path.join(root, 'usr', 'bin', 'xslt-config')):
+            self.run(['sed', '-i', '-e', 's|prefix=/usr|prefix={}/usr|'.format(root), os.path.join(root, 'usr', 'bin', 'xslt-config')])
 
 
 def _get_source_handler(source_type, source):

=== modified file 'snapcraft/yaml.py'
--- snapcraft/yaml.py	2015-09-18 18:04:48 +0000
+++ snapcraft/yaml.py	2015-09-23 15:54:59 +0000
@@ -166,14 +166,46 @@
 
     def runtime_env(self, root):
         env = []
-        env.append('PATH="{0}/bin:{0}/usr/bin:$PATH"'.format(root))
-        env.append('LD_LIBRARY_PATH="{0}/lib:{0}/usr/lib:{0}/lib/{1}:{0}/usr/lib/{1}:$LD_LIBRARY_PATH"'.format(root, snapcraft.common.get_arch_triplet()))
+        env.append('PATH="' + ':'.join([
+            '{0}/bin',
+            '{0}/usr/bin',
+            '$PATH'
+        ]).format(root) + '"')
+        env.append('LD_LIBRARY_PATH="' + ':'.join([
+            '{0}/lib',
+            '{0}/usr/lib',
+            '{0}/lib/{1}',
+            '{0}/usr/lib/{1}',
+            '$LD_LIBRARY_PATH'
+        ]).format(root, snapcraft.common.get_arch_triplet()) + '"')
         return env
 
     def build_env(self, root):
         env = []
-        env.append('CFLAGS="-I{0}/include -I{0}/usr/include -I{0}/include/{1} -I{0}/usr/include/{1} $CFLAGS"'.format(root, snapcraft.common.get_arch_triplet()))
-        env.append('LDFLAGS="-L{0}/lib -L{0}/usr/lib -L{0}/lib/{1} -L{0}/usr/lib/{1} $LDFLAGS"'.format(root, snapcraft.common.get_arch_triplet()))
+        env.append('CFLAGS="' + ' '.join([
+            '-I{0}/include',
+            '-I{0}/usr/include',
+            '-I{0}/include/{1}',
+            '-I{0}/usr/include/{1}',
+            '$CFLAGS'
+        ]).format(root, snapcraft.common.get_arch_triplet()) + '"')
+        env.append('LDFLAGS="' + ' '.join([
+            '-L{0}/lib',
+            '-L{0}/usr/lib',
+            '-L{0}/lib/{1}',
+            '-L{0}/usr/lib/{1}',
+            '$LDFLAGS'
+        ]).format(root, snapcraft.common.get_arch_triplet()) + '"')
+        env.append('PKG_CONFIG_SYSROOT_DIR={0}'.format(root))
+        env.append('PKG_CONFIG_PATH=' + ':'.join([
+            '{0}/usr/lib/pkgconfig',
+            '{0}/usr/lib/{1}/pkgconfig',
+            '{0}/usr/share/pkgconfig',
+            '{0}/usr/local/lib/pkgconfig',
+            '{0}/usr/local/lib/{1}/pkgconfig',
+            '{0}/usr/local/share/pkgconfig',
+            '$PKG_CONFIG_PATH'
+        ]).format(root, snapcraft.common.get_arch_triplet()))
         return env
 
     def build_env_for_part(self, part):