Merge lp:~kissiel/checkbox/embedded-providers into lp:checkbox

[Maciej Kisielewski]

This MR adds option to load providers supplied with checkbox-touch app.

d7fc3cf checkbox-touch: add loading providers from '$APP_DIR/providers'
f5f2ec2 plainbox:providers:special: add loading of embedded providers

[Zygmunt Krynicki (zyga)]

Hi (sigh, lost comments)

Please convert this to a PlugIn and use FsPlugInCollection instead. We have lots of goodies there and I'd like to make sure all things like that are handled via the plugin system.

The FileSystemPlugInCollection can be used as-is, just pass the correct filters to find manage.py and disable recursion. You will want to write a PlugIn subclass that knows how to load the thing from disk though.

Otherwise ok

review: Needs Fixing

[Sylvain Pineau (sylvain-pineau)]

+1 for the new plugin mechanism. thanks

review: Approve

[Daniel Manrique (roadmr)]

The attempt to merge lp:~kissiel/checkbox/embedded-providers into lp:checkbox failed. Below is the output from the failed tests.

[precise] starting container
[precise] (timing) 0.06user 0.01system 0:05.20elapsed 1%CPU (0avgtext+0avgdata 7860maxresident)k
[precise] (timing) 0inputs+32outputs (0major+6064minor)pagefaults 0swaps
[precise] provisioning container
[precise] (timing) 39.39user 10.58system 1:31.39elapsed 54%CPU (0avgtext+0avgdata 51368maxresident)k
[precise] (timing) 0inputs+16424outputs (0major+4463039minor)pagefaults 0swaps
[precise-testing] Starting tests...
Found a test script: ./checkbox-gui/requirements/container-tests-checkbox-gui-build
[precise-testing] container-tests-checkbox-gui-build: PASS
[precise-testing] (timing) 30.23user 2.06system 0:32.91elapsed 98%CPU (0avgtext+0avgdata 116392maxresident)k
[precise-testing] (timing) 0inputs+4224outputs (0major+478228minor)pagefaults 0swaps
Found a test script: ./checkbox-ng/requirements/container-tests-checkbox-ng-unit
[precise-testing] container-tests-checkbox-ng-unit: FAIL
[precise-testing] stdout: http://paste.ubuntu.com/8948332/
[precise-testing] stderr: http://paste.ubuntu.com/8948333/
[precise-testing] (timing) Command exited with non-zero status 1
[precise-testing] (timing) 0.44user 0.13system 0:00.60elapsed 95%CPU (0avgtext+0avgdata 35356maxresident)k
[precise-testing] (timing) 0inputs+2456outputs (0major+19110minor)pagefaults 0swaps
Found a test script: ./checkbox-support/requirements/container-tests-checkbox-support
[precise-testing] container-tests-checkbox-support: PASS
[precise-testing] (timing) 16.21user 0.15system 0:16.65elapsed 98%CPU (0avgtext+0avgdata 83084maxresident)k
[precise-testing] (timing) 0inputs+1096outputs (0major+31256minor)pagefaults 0swaps
Found a test script: ./checkbox-touch/requirements/container-tests-touch-unit-tests
[precise-testing] container-tests-touch-unit-tests: PASS
[precise-testing] (timing) 0.00user 0.00system 0:00.02elapsed 44%CPU (0avgtext+0avgdata 2016maxresident)k
[precise-testing] (timing) 0inputs+8outputs (0major+2344minor)pagefaults 0swaps
Found a test script: ./plainbox/plainbox/impl/providers/categories/requirements/container-tests-provider-categories
[precise-testing] container-tests-provider-categories: FAIL
[precise-testing] stdout:
[precise-testing] stderr: http://paste.ubuntu.com/8948335/
[precise-testing] (timing) Command exited with non-zero status 1
[precise-testing] (timing) 0.13user 0.03system 0:00.18elapsed 92%CPU (0avgtext+0avgdata 12584maxresident)k
[precise-testing] (timing) 0inputs+176outputs (0major+5645minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/001-container-tests-plainbox-egg-info
[precise-testing] 001-container-tests-plainbox-egg-info: PASS
[precise-testing] (timing) 0.15user 0.04system 0:00.21elapsed 92%CPU (0avgtext+0avgdata 10492maxresident)k
[precise-testing] (timing) 0inputs+88outputs (0major+4979minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox
[precise-testing] container-tests-plainbox: FAIL
[precise-testing] stdout: http://paste.ubuntu...

[lp:~kissiel/checkbox/embedded-providers]

3394. By Maciej Kisielewski on 2014-11-12

checkbox-touch: add checkbox-touch custom providers

This patch adds manage.py providers for checkbox-touch

3395. By Maciej Kisielewski on 2014-11-12

checkbox-touch: add loading providers from '$APP_DIR/providers'

This patch makes checkbox-touch load providers that were supplied in
$APP_DIR/providers.

[Maciej Kisielewski (kissiel)]

Moved embedded providers to be checkbox-touch only.

[Sylvain Pineau (sylvain-pineau)]

Now we have two python files in checkbox-touch...

But +1 for solving the dependency on python 3.3+ this way

review: Approve

=== modified file 'checkbox-touch/py/checkbox_touch.py'
--- checkbox-touch/py/checkbox_touch.py	2014-11-05 10:56:15 +0000
+++ checkbox-touch/py/checkbox_touch.py	2014-11-12 03:15:44 +0000
@@ -48,6 +48,7 @@
 from plainbox.impl.providers.special import get_categories
 from plainbox.impl.providers.special import get_stubbox
 from plainbox.impl.providers.v1 import all_providers
+from embedded_providers import EmbeddedProvider1PlugInCollection
 from plainbox.impl.runner import JobRunner
 from plainbox.impl.secure.origin import Origin
 from plainbox.impl.secure.qualifiers import FieldQualifier
@@ -684,8 +685,14 @@
                 'com.canonical.certification.checkbox-touch'))
 
     def _get_default_providers(self):
-        all_providers.load()
-        return [get_stubbox(), get_categories()]
+        providers_collection = all_providers.get_all_plugin_objects()
+        # when running on ubuntu-touch device, APP_DIR env var is present
+        # and points to touch application top directory
+        path = os.path.join(os.path.expandvars('$APP_DIR'), 'providers')
+        if os.path.exists(path):
+            embedded_providers = EmbeddedProvider1PlugInCollection(path)
+            providers_collection += embedded_providers.get_all_plugin_objects()
+        return providers_collection
 
 
 def bootstrap():

=== added file 'checkbox-touch/py/embedded_providers.py'
--- checkbox-touch/py/embedded_providers.py	1970-01-01 00:00:00 +0000
+++ checkbox-touch/py/embedded_providers.py	2014-11-12 03:15:44 +0000
@@ -0,0 +1,92 @@
+# This file is part of Checkbox.
+#
+# Copyright 2014 Canonical Ltd.
+# Written by:
+#   Zygmunt Krynicki <[e-mail]>
+#   Maciej Kisielewski <[e-mail]>
+#
+# Checkbox is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# Checkbox is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
+"""
+:mod:`embedded_providers` -- Checkbox-touch-only providers
+===============================================================
+
+This module contains custom providers used by Checkbox-touch
+"""
+from importlib.machinery import SourceFileLoader
+import logging
+import os
+
+from plainbox.abc import IProvider1, IProviderBackend1
+from plainbox.impl.secure.plugins import FsPlugInCollection
+from plainbox.impl.secure.providers.v1 import Provider1
+from plainbox.impl.secure.providers.v1 import Provider1Definition
+from plainbox.impl.secure.providers.v1 import Provider1PlugIn
+from plainbox.impl.secure.config import Unset
+
+
+class ManagePyProvider1PlugIn(Provider1PlugIn):
+    """
+    Provider1PlugIn that is built from manage.py file.
+    """
+
+    def __init__(self, filename, file_contents, *, validate=None,
+                 validation_kwargs=None, check=None, context=None):
+        """
+        Initialize plug-in and create provider from definition extracted
+        from manage.py pointed by `filename`
+        """
+
+        # override provider_manager.setup() to capture setup's parameters
+        setup_kwargs = []
+
+        def fake_setup(**kwargs):
+            setup_kwargs.append(kwargs)
+
+        from plainbox import provider_manager
+        provider_manager.setup = fake_setup
+
+        loader = SourceFileLoader('manage', filename)
+        loader.load_module()
+        location = os.path.dirname(os.path.abspath(filename))
+        if len(setup_kwargs) < 1:
+            # did not load any provider from given manage.py
+            # creating empty definition
+            definition = Provider1Definition()
+        else:
+            setup_kwargs = setup_kwargs.pop()
+            definition = Provider1Definition()
+            definition.location = location
+            definition.name = setup_kwargs.get('name', None)
+            definition.version = setup_kwargs.get('version', None)
+            definition.description = setup_kwargs.get('description', None)
+            definition.gettext_domain = setup_kwargs.get(
+                'gettext_domain', Unset)
+        self._provider = Provider1.from_definition(
+            definition, secure=False, validate=validate,
+            validation_kwargs=validation_kwargs, check=check, context=context)
+
+
+class EmbeddedProvider1PlugInCollection(FsPlugInCollection):
+    """
+    A collection of v1 provider plugins loaded from 'manage.py'  files.
+
+    """
+    def __init__(self, path, **kwargs):
+        """
+        Initiates collection with all providers loaded from manage.py files
+        found in `path` and subdirectories.
+        """
+        # get all manage.py files to load providers from
+        super().__init__(
+            [path], 'manage.py', recursive=True, load=True,
+            wrapper=ManagePyProvider1PlugIn, **kwargs)