Merge lp:~jocave/snapcraft/plainbox-provider-plugin into lp:~snappy-dev/snapcraft/core

[Jonathan Cave]

Adds a plugin for building Plainbox Providers in to snaps.

Adds a plugin for building Plainbox Providers in to snaps.

Plainbox providers contain test definitions to be used with the Plainbox test
runner itself.

This plugin takes the a provider from the source location defined and runs the
appropriate commands to
 1. run the provider build step so any binaries are created
 2. run the provider install step to stage the provider in the desired layout

[lp:~jocave/snapcraft/plainbox-provider-plugin]

134. By Jonathan Cave on 2015-08-18

Creating a plainbox-provider plugin

[Sergio Schvezov (sergiusens)]

should this depend on the python3 module (in the yaml)?

On Mon, Aug 17, 2015 at 6:18 AM, Jonathan Cave <email address hidden>
wrote:

> Jonathan Cave has proposed merging
> lp:~jocave/snapcraft/plainbox-provider-plugin into lp:snapcraft.
>
> Requested reviews:
> Snappy Developers (snappy-dev)
>
> For more details, see:
>
> https://code.launchpad.net/~jocave/snapcraft/plainbox-provider-plugin/+merge/268221
>
> Adds a plugin for building Plainbox Providers in to snaps.
>
> Plainbox providers contain test definitions to be used with the Plainbox
> test
> runner itself.
>
> This plugin takes the a provider from the source location defined and runs
> the
> appropriate commands to
> 1. run the provider build step so any binaries are created
> 2. run the provider install step to stage the provider in the desired
> layout
>
> --
> Your team Snappy Developers is requested to review the proposed merge of
> lp:~jocave/snapcraft/plainbox-provider-plugin into lp:snapcraft.
>
> Launchpad-Message-Rationale: Reviewer @snappy-dev
> Launchpad-Notification-Type: code-review
> Launchpad-Branch: ~jocave/snapcraft/plainbox-provider-plugin
> Launchpad-Project: snapcraft
>
> === added file 'plugins/plainbox-provider.yaml'
> --- plugins/plainbox-provider.yaml 1970-01-01 00:00:00 +0000
> +++ plugins/plainbox-provider.yaml 2015-08-17 13:17:58 +0000
> @@ -0,0 +1,4 @@
> +module: plainbox_provider
> +options:
> + source:
> + required: true
>
> === added file 'snapcraft/plugins/plainbox_provider.py'
> --- snapcraft/plugins/plainbox_provider.py 1970-01-01 00:00:00 +0000
> +++ snapcraft/plugins/plainbox_provider.py 2015-08-17 13:17:58 +0000
> @@ -0,0 +1,34 @@
> +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
> +#
> +# Copyright (C) 2015 Canonical Ltd
> +#
> +# This program is free software: you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License version 3 as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +
> +import snapcraft
> +
> +
> +class PlainboxProviderPlugin(snapcraft.BasePlugin):
> +
> + # note that we don't need to setup env(), python figures it out
> + # see python3.py for more details
> +
> + def pull(self):
> + return self.handle_source_options()
> +
> + def build(self):
> + return self.run(["python3", "manage.py", "build"])
> +
> + def stage(self):
> + return self.run(
> + ["python3", "manage.py", "install", "--layout=snappy",
> + "--root=%s" % self.installdir])
>
>
>

[lp:~jocave/snapcraft/plainbox-provider-plugin]

135. By Jonathan Cave on 2015-08-19

Adding requires: on python3 to yaml

[Jonathan Cave (jocave)]

Sergio - yes, thanks. Now added.

[Zygmunt Krynicki (zyga)]

Hey.

Sorry, I'm just wrapping my head around this. I think it should _not_ depend on python 3. A provider is just a collection of definitions. We need python and plainbox on the build machine but the target machine does not need python (plainbox is a framework).

[Zygmunt Krynicki (zyga)]

(and what I mean by sorry is that I'm not sure if this pulls in python to the snap (wrong) or on the host to build the snap (correct but we then also require a >= version of plainbox to boot).

[Sergio Schvezov (sergiusens)]

On Wed, Aug 19, 2015 at 8:19 AM, Zygmunt Krynicki <
<email address hidden>> wrote:

> (and what I mean by sorry is that I'm not sure if this pulls in python to
> the snap (wrong) or on the host to build the snap (correct but we then also
> require a >= version of plainbox to boot).

Hmm, maybe an example inside examples would clear things up with the
intentions of this plugin, but you need to either depend on the plugin
which would make it part of the final snap or add a build-depends to the
yaml for constructing this.

I guess in any case, for a plugin we need an E2E working example inside
'examples' to not have a half baked plugin.

[Jonathan Cave (jocave)]

Yep a request for an example is very reasonable.

Zyga - could you suggest a suitable provider to feature in the example? Or
would the integration-tests of snapcraft itself be a convenient example?!

On 19 August 2015 at 19:57, Sergio Schvezov <email address hidden>
wrote:

> On Wed, Aug 19, 2015 at 8:19 AM, Zygmunt Krynicki <
> <email address hidden>> wrote:
>
> > (and what I mean by sorry is that I'm not sure if this pulls in python to
> > the snap (wrong) or on the host to build the snap (correct but we then
> also
> > require a >= version of plainbox to boot).
>
>
> Hmm, maybe an example inside examples would clear things up with the
> intentions of this plugin, but you need to either depend on the plugin
> which would make it part of the final snap or add a build-depends to the
> yaml for constructing this.
>
> I guess in any case, for a plugin we need an E2E working example inside
> 'examples' to not have a half baked plugin.
>
> --
>
> https://code.launchpad.net/~jocave/snapcraft/plainbox-provider-plugin/+merge/268221
> You are the owner of lp:~jocave/snapcraft/plainbox-provider-plugin.
>

[Sergio Schvezov (sergiusens)]

On Wed, Aug 19, 2015 at 8:18 AM, Zygmunt Krynicki <
<email address hidden>> wrote:

> Hey.
>
> Sorry, I'm just wrapping my head around this. I think it should _not_
> depend on python 3. A provider is just a collection of definitions. We need
> python and plainbox on the build machine but the target machine does not
> need python (plainbox is a framework).

Sorry (me too ;-) ), but I have to ask. What is the reasoning for the
plainbox runtime to be a framework?

Can't snapcraft just bundle it all together into one harmonious app snap?

Do you see any potential in keeping them separate at all?

[lp:~jocave/snapcraft/plainbox-provider-plugin]

136. By Jonathan Cave on 2015-08-24

Make the python3 depends a build dependency and add plainbox.
Add an example of the plainbox provider plugin usage.

Unmerged revisions

136. By Jonathan Cave on 2015-08-24

Make the python3 depends a build dependency and add plainbox.
Add an example of the plainbox provider plugin usage.

135. By Jonathan Cave on 2015-08-19

Adding requires: on python3 to yaml

134. By Jonathan Cave on 2015-08-18

Creating a plainbox-provider plugin

=== added directory 'examples/plainbox-provider'
=== added file 'examples/plainbox-provider/integration-tests-runner'
--- examples/plainbox-provider/integration-tests-runner	1970-01-01 00:00:00 +0000
+++ examples/plainbox-provider/integration-tests-runner	2015-08-24 17:24:20 +0000
@@ -0,0 +1,7 @@
+#!/bin/sh
+# This wrapper runs plainbox with the integration-tests test plan. It lets the
+# test operator pass any additional options so that some degree of freedom over
+# result processing is retained.
+#
+# For a totally free-form use, just call plainbox directly.
+exec plainbox-wrapper run -T 2015.com.canonical.snapcraft::normal "$@"

=== added directory 'examples/plainbox-provider/meta'
=== added file 'examples/plainbox-provider/meta/package.yaml'
--- examples/plainbox-provider/meta/package.yaml	1970-01-01 00:00:00 +0000
+++ examples/plainbox-provider/meta/package.yaml	2015-08-24 17:24:20 +0000
@@ -0,0 +1,9 @@
+name: snapcraft-integration-tests-provider
+version: 1
+vendor: Jonathan Cave <[e-mail]>
+binaries:
+ - name: integration-tests-runner
+   security-template: unconfined
+ - name: plainbox
+   exec: bin/plainbox-wrapper
+   security-template: unconfined

=== added file 'examples/plainbox-provider/meta/readme.md'
--- examples/plainbox-provider/meta/readme.md	1970-01-01 00:00:00 +0000
+++ examples/plainbox-provider/meta/readme.md	2015-08-24 17:24:20 +0000
@@ -0,0 +1,1 @@
+A snap package of snapcraft integration tests plainbox provider

=== added file 'examples/plainbox-provider/plainbox-wrapper'
--- examples/plainbox-provider/plainbox-wrapper	1970-01-01 00:00:00 +0000
+++ examples/plainbox-provider/plainbox-wrapper	2015-08-24 17:24:20 +0000
@@ -0,0 +1,15 @@
+#!/bin/sh
+# This wrapper, which is installed as $SNAP_APP_PATH/bin/plainbox
+# (taking over $SNAP_APP_PATH/usr/bin/plainbox in priority) serves
+# two purposes:
+# - It sets a PROVIDERPATH that plainbox can find bundled providers:
+#   each provider is installed to /providers/$name, they are listed
+#   in a typical :-separated PATH-like variable.
+# This is largely tracked as https://bugs.launchpad.net/plainbox/+bug/1473939
+export PROVIDERPATH="$SNAP_APP_PATH/providers/plainbox-provider-snapcraft-integration-tests:"
+# - It works around a bug in snapcraft which causes scripts to refer
+#   to the wrong (outside, not bundled) interpreter.
+# https://bugs.launchpad.net/snapcraft/+bug/1486659
+# NOTE: Here python3 is already the one from the snap, because we are running
+# in altered environment and PATH is set accordingly.
+exec python3 $(which plainbox) "$@"

=== added file 'examples/plainbox-provider/snapcraft.yaml'
--- examples/plainbox-provider/snapcraft.yaml	1970-01-01 00:00:00 +0000
+++ examples/plainbox-provider/snapcraft.yaml	2015-08-24 17:24:20 +0000
@@ -0,0 +1,14 @@
+parts:
+    # Note: this does not currently pull in all the dependencies
+    # of the integration tests themselves i.e. hg, bzr, ...
+    plainbox:
+        plugin: ubuntu
+    plainbox-provider-snapcraft-integration-tests:
+        plugin: plainbox-provider
+        source: ../../integration-tests
+    launcher:
+        plugin: copy
+        files:
+            plainbox-wrapper: bin/plainbox-wrapper
+            integration-tests-runner: bin/integration-tests-runner
+snappy-metadata: meta

=== added file 'plugins/plainbox-provider.yaml'
--- plugins/plainbox-provider.yaml	1970-01-01 00:00:00 +0000
+++ plugins/plainbox-provider.yaml	2015-08-24 17:24:20 +0000
@@ -0,0 +1,5 @@
+module: plainbox_provider
+build-tools: [python3, plainbox]
+options:
+    source:
+        required: true

=== added file 'snapcraft/plugins/plainbox_provider.py'
--- snapcraft/plugins/plainbox_provider.py	1970-01-01 00:00:00 +0000
+++ snapcraft/plugins/plainbox_provider.py	2015-08-24 17:24:20 +0000
@@ -0,0 +1,31 @@
+# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
+#
+# Copyright (C) 2015 Canonical Ltd
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import snapcraft
+
+
+class PlainboxProviderPlugin(snapcraft.BasePlugin):
+
+    # note that we don't need to setup env(), python figures it out
+    # see python3.py for more details
+
+    def pull(self):
+        return self.handle_source_options()
+
+    def build(self):
+        return self.run(["python3", "manage.py", "build"]) and \
+            self.run(["python3", "manage.py", "install", "--layout=relocatable",
+                     "--root=%s" % self.installdir])