Merge lp:~sergiusens/snapcraft/1500902 into lp:~snappy-dev/snapcraft/core

[Sergio Schvezov]

Plugin yaml's are obsolete

[Leo Arias (elopio)]

A couple of style comments inline.

This change makes sense, because now we don't split the plugins into a python and a yaml file. But I think we need to spend some time cleaning and documenting the api of the parent base plugin. Clearly mark things that are internals, constants and things that are to be extended by children. And write nice sphinxed docs.

review: Needs Fixing

[lp:~sergiusens/snapcraft/1500902]

235. By Sergio Schvezov on 2015-10-08

Making stage-packages and build-packages regular attributes while polishing the phase in the lifecycle where it is handled

236. By Sergio Schvezov on 2015-10-08

Merging trunk

237. By Sergio Schvezov on 2015-10-09

Fix issues when using global dictionaries and plugin code polish

238. By Sergio Schvezov on 2015-10-09

The name belongs to the part, the other thing is just the plugin to use to handle it

239. By Sergio Schvezov on 2015-10-09

getattr--

240. By Sergio Schvezov on 2015-10-09

contextlib++

241. By Sergio Schvezov on 2015-10-09

More name changes

242. By Sergio Schvezov on 2015-10-09

Remove more PLUGIN_BUILD_PACKAGES refs

[Sergio Schvezov (sergiusens)]

OK, except for the docs, most of this the comments have been addressed.

[lp:~sergiusens/snapcraft/1500902]

243. By Sergio Schvezov on 2015-10-09

options is here

244. By Sergio Schvezov on 2015-10-09

import jdk for maven plugin

245. By Sergio Schvezov on 2015-10-09

missing options for scons_options

246. By Sergio Schvezov on 2015-10-09

Remove _PLUGIN_BUILD_PACKAGES from CMakePlugin

247. By Sergio Schvezov on 2015-10-09

jsonschema support

[John Lenton (chipaca)]

review: Approve

[lp:~sergiusens/snapcraft/1500902]

248. By Sergio Schvezov on 2015-10-13

Removing plugins install logic from setup.py

[Leo Arias (elopio)]

This cleans up almost everything I didn't like. Let's merge!

review: Approve

[lp:~sergiusens/snapcraft/1500902]

249. By Sergio Schvezov on 2015-10-13

Build packages for plugins

=== removed directory 'plugins'
=== removed file 'plugins/ant.yaml'
--- plugins/ant.yaml	2015-10-01 08:54:15 +0000
+++ plugins/ant.yaml	1970-01-01 00:00:00 +0000
@@ -1,13 +0,0 @@
-requires:
-  - jdk
-options:
-  source:
-    required: true
-  source-type:
-  source-tag:
-  source-branch:
-  stage-packages:
-  snap:
-  stage:
-build-packages:
-  - ant

=== removed file 'plugins/autotools.yaml'
--- plugins/autotools.yaml	2015-10-01 08:54:15 +0000
+++ plugins/autotools.yaml	1970-01-01 00:00:00 +0000
@@ -1,12 +0,0 @@
-build-packages: [autoconf, automake, autopoint]
-options:
-  source:
-    required: true
-  source-type:
-  source-tag:
-  source-branch:
-  configflags:
-    required: false
-  stage-packages:
-  snap:
-  stage:

=== removed file 'plugins/cmake.yaml'
--- plugins/cmake.yaml	2015-10-01 08:54:15 +0000
+++ plugins/cmake.yaml	1970-01-01 00:00:00 +0000
@@ -1,12 +0,0 @@
-build-packages: [cmake]
-options:
-  source:
-    required: true
-  source-type:
-  source-tag:
-  source-branch:
-  configflags:
-    required: false
-  stage-packages:
-  snap:
-  stage:

=== removed file 'plugins/copy.yaml'
--- plugins/copy.yaml	2015-09-08 18:06:03 +0000
+++ plugins/copy.yaml	1970-01-01 00:00:00 +0000
@@ -1,6 +0,0 @@
-options:
-  files:
-    required: true
-  stage-packages:
-  snap:
-  stage:

=== removed file 'plugins/go.yaml'
--- plugins/go.yaml	2015-10-01 08:54:15 +0000
+++ plugins/go.yaml	1970-01-01 00:00:00 +0000
@@ -1,10 +0,0 @@
-build-packages:
-  - golang-go
-options:
-  source:
-    required: true
-  configflags:
-    required: false
-  stage-packages:
-  snap:
-  stage:

=== removed file 'plugins/jdk.yaml'
--- plugins/jdk.yaml	2015-10-08 18:08:29 +0000
+++ plugins/jdk.yaml	1970-01-01 00:00:00 +0000
@@ -1,2 +0,0 @@
-build-packages:
-  - ca-certificates-java

=== removed file 'plugins/make.yaml'
--- plugins/make.yaml	2015-10-01 08:54:15 +0000
+++ plugins/make.yaml	1970-01-01 00:00:00 +0000
@@ -1,10 +0,0 @@
-build-packages: [make]
-options:
-  source:
-    required: true
-  source-type:
-  source-tag:
-  source-branch:
-  stage-packages:
-  snap:
-  stage:

=== removed file 'plugins/maven.yaml'
--- plugins/maven.yaml	2015-10-01 08:54:15 +0000
+++ plugins/maven.yaml	1970-01-01 00:00:00 +0000
@@ -1,13 +0,0 @@
-build-packages:
-  - maven
-requires:
-  - jdk
-options:
-  source:
-    required: true
-  source-type:
-  source-tag:
-  source-branch:
-  stage-packages:
-  snap:
-  stage:

=== removed file 'plugins/python2.yaml'
--- plugins/python2.yaml	2015-10-01 08:54:15 +0000
+++ plugins/python2.yaml	1970-01-01 00:00:00 +0000
@@ -1,11 +0,0 @@
-options:
-  requirements:
-    required: false
-  source:
-    required: false
-  source-type:
-  source-tag:
-  source-branch:
-  stage-packages:
-  snap:
-  stage:

=== removed file 'plugins/python3.yaml'
--- plugins/python3.yaml	2015-10-01 08:54:15 +0000
+++ plugins/python3.yaml	1970-01-01 00:00:00 +0000
@@ -1,11 +0,0 @@
-options:
-  requirements:
-    required: false
-  source:
-    required: false
-  source-type:
-  source-tag:
-  source-branch:
-  stage-packages:
-  snap:
-  stage:

=== removed file 'plugins/qml.yaml'
=== removed file 'plugins/readline.yaml'
--- plugins/readline.yaml	2015-10-01 08:54:15 +0000
+++ plugins/readline.yaml	1970-01-01 00:00:00 +0000
@@ -1,3 +0,0 @@
-source: git://git.sv.gnu.org/readline.git
-requires:
-  - autotools

=== removed file 'plugins/scons.yaml'
--- plugins/scons.yaml	2015-10-01 08:54:15 +0000
+++ plugins/scons.yaml	1970-01-01 00:00:00 +0000
@@ -1,9 +0,0 @@
-build-packages:
-  - scons
-options:
-  source:
-    required: true
-  scons-options:
-  source-type:
-  source-tag:
-  source-branch:

=== removed file 'plugins/tar-content.yaml'
--- plugins/tar-content.yaml	2015-09-22 03:08:07 +0000
+++ plugins/tar-content.yaml	1970-01-01 00:00:00 +0000
@@ -1,6 +0,0 @@
-options:
-  source:
-    required: true
-  stage-packages:
-  snap:
-  stage:

=== modified file 'setup.py'
--- setup.py	2015-10-02 09:13:00 +0000
+++ setup.py	2015-10-13 15:27:09 +0000
@@ -37,8 +37,6 @@
       package_data={'snapcraft': ['manifest.txt']},
       scripts=['bin/snapcraft'],
       data_files=[
-          ('share/snapcraft/plugins',
-              ['plugins/' + x for x in os.listdir('plugins')]),
           ('share/snapcraft/schema',
               ['schema/' + x for x in os.listdir('schema')]),
       ],

=== modified file 'snapcraft/__init__.py'
--- snapcraft/__init__.py	2015-10-02 09:13:00 +0000
+++ snapcraft/__init__.py	2015-10-13 15:27:09 +0000
@@ -14,6 +14,7 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+import contextlib
 import logging
 import os
 import re
@@ -28,9 +29,41 @@
 
 class BasePlugin:
 
-    @property
-    def PLUGIN_STAGE_PACKAGES(self):
-        return getattr(self, '_PLUGIN_STAGE_PACKAGES', [])
+    @classmethod
+    def schema(cls):
+        '''
+        Returns a json-schema for the plugin's properties as a dictionary.
+        Of importance to plugin authors is the 'properties' keyword and
+        optionally the 'requires' keyword with a list of required
+        'properties'.
+
+        By default the the properties will be that of a standard VCS, override
+        in custom implementations if required.
+        '''
+        return {
+            '$schema': 'http://json-schema.org/draft-04/schema#',
+            'type': 'object',
+            'properties': {
+                'source': {
+                    'type': 'string',
+                },
+                'source-type': {
+                    'type': 'string',
+                    'default': '',
+                },
+                'source-branch': {
+                    'type': 'string',
+                    'default': '',
+                },
+                'source-tag': {
+                    'type:': 'string',
+                    'default': '',
+                },
+            },
+            'required': [
+                'source',
+            ]
+        }
 
     @property
     def PLUGIN_STAGE_SOURCES(self):
@@ -38,6 +71,14 @@
 
     def __init__(self, name, options):
         self.name = name
+        self.build_packages = []
+        self.stage_packages = []
+
+        with contextlib.suppress(AttributeError):
+            self.stage_packages = options.stage_packages
+        with contextlib.suppress(AttributeError):
+            self.build_packages = options.build_packages
+
         self.options = options
         self.partdir = os.path.join(os.getcwd(), "parts", self.name)
         self.sourcedir = os.path.join(os.getcwd(), "parts", self.name, "src")
@@ -121,11 +162,10 @@
         os.makedirs(d, exist_ok=True)
 
     def setup_stage_packages(self):
-        part_stage_packages = getattr(self.options, 'stage_packages', []) or []
-        if self.PLUGIN_STAGE_PACKAGES or part_stage_packages:
+        if self.stage_packages:
             ubuntu = snapcraft.repo.Ubuntu(self.ubuntudir,
                                            sources=self.PLUGIN_STAGE_SOURCES)
-            ubuntu.get(self.PLUGIN_STAGE_PACKAGES + part_stage_packages)
+            ubuntu.get(self.stage_packages)
             ubuntu.unpack(self.installdir)
             self._fixup(self.installdir)
 
@@ -143,7 +183,7 @@
 
 
 def _get_source_handler(source_type, source):
-    if source_type is None:
+    if not source_type:
         source_type = _get_source_type_from_uri(source)
 
     if source_type == 'bzr':

=== modified file 'snapcraft/cmds.py'
--- snapcraft/cmds.py	2015-10-05 16:16:23 +0000
+++ snapcraft/cmds.py	2015-10-13 15:27:09 +0000
@@ -54,9 +54,8 @@
     if args.part:
         yaml += 'parts:\n'
     for part_name in args.part:
-        part = snapcraft.plugin.load_plugin(part_name, part_name,
-                                            load_code=False)
-        yaml += '    ' + part.names()[0] + ':\n'
+        part = snapcraft.plugin.load_plugin(part_name, part_name)
+        yaml += '    ' + part.name + ':\n'
         for opt in part.config.get('options', []):
             if part.config['options'][opt].get('required', False):
                 yaml += '        ' + opt + ':\n'
@@ -241,7 +240,7 @@
     config = _load_config()
 
     for part in config.all_parts:
-        logger.info('Cleaning up for part %r', part.names()[0])
+        logger.info('Cleaning up for part %r', part.name)
         if os.path.exists(part.partdir):
             shutil.rmtree(part.partdir)
 
@@ -287,14 +286,14 @@
                              'paths in common which have different '
                              'contents:\n  %s',
                              other_part_name,
-                             part.names()[0],
+                             part.name,
                              '\n  '.join(sorted(conflict_files)))
 
                 return False
 
         # And add our files to the list
-        parts_files[part.names()[0]] = {'files': part_files,
-                                        'installdir': part.installdir}
+        parts_files[part.name] = {'files': part_files,
+                                  'installdir': part.installdir}
 
     return True
 
@@ -336,7 +335,7 @@
             force = forceAll or cmd == forceCommand
 
             if not getattr(part, cmd)(force=force):
-                logger.error('Failed doing %s for %s!', cmd, part.names()[0])
+                logger.error('Failed doing %s for %s!', cmd, part.name)
                 sys.exit(1)
 
 

=== modified file 'snapcraft/plugin.py'
--- snapcraft/plugin.py	2015-10-05 16:56:10 +0000
+++ snapcraft/plugin.py	2015-10-13 15:27:09 +0000
@@ -14,13 +14,14 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+import contextlib
 import glob
 import importlib
+import jsonschema
 import logging
 import os
 import sys
 import shutil
-import yaml
 
 import snapcraft
 from snapcraft import common
@@ -30,15 +31,6 @@
 logger = logging.getLogger(__name__)
 
 
-_BUILTIN_OPTIONS = {
-    'filesets': {},
-    'snap': [],
-    'stage': [],
-    'stage-packages': [],
-    'organize': {}
-}
-
-
 def _local_plugindir():
     return os.path.abspath(os.path.join('parts', 'plugins'))
 
@@ -49,14 +41,16 @@
 
 class PluginHandler:
 
-    def __init__(self, name, part_name, properties, load_code=True,
-                 load_config=True):
+    @property
+    def name(self):
+        return self._name
+
+    def __init__(self, plugin_name, part_name, properties):
         self.valid = False
         self.code = None
         self.config = {}
-        self.part_names = []
+        self._name = part_name
         self.deps = []
-        self.plugin_name = name
 
         parts_dir = common.get_partsdir()
         self.partdir = os.path.join(parts_dir, part_name)
@@ -69,78 +63,46 @@
         self.statefile = os.path.join(parts_dir, part_name, 'state')
 
         try:
-            if load_config:
-                self._load_config(name)
-            if load_code:
-                self._load_code(name, part_name, properties)
+            self._load_code(plugin_name, properties)
             # only set to valid if it loads without PluginError
-            self.part_names.append(part_name)
             self.valid = True
         except PluginError as e:
             logger.error(str(e))
             return
-
-    def _load_config(self, name):
-        config_path = os.path.join(common.get_plugindir(), name + ".yaml")
-        if not os.path.exists(config_path):
-            config_path = os.path.join(_local_plugindir(), name + ".yaml")
-        if not os.path.exists(config_path):
-            raise PluginError('Unknown plugin: {}'.format(name))
-        with open(config_path, 'r') as fp:
-            self.config = yaml.load(fp) or {}
-
-    def _make_options(self, name, properties):
-        class Options():
-            pass
-        options = Options()
-
-        plugin_options = self.config.get('options', {})
-        # Let's append some mandatory options, but not .update() to not lose
-        # original content
-        for key in _BUILTIN_OPTIONS:
-            if key not in plugin_options:
-                plugin_options[key] = _BUILTIN_OPTIONS[key]
-
-        for opt in plugin_options:
-            attrname = opt.replace('-', '_')
-            opt_parameters = plugin_options[opt] or {}
-            if opt in properties:
-                setattr(options, attrname, properties[opt])
-            else:
-                if opt_parameters.get('required', False):
-                    raise PluginError('Required field {} missing on part {}'.
-                                      format(opt, name))
-                setattr(options, attrname, None)
-
-        return options
-
-    def _load_code(self, name, part_name, properties):
-        options = self._make_options(name, properties)
-        module_name = name.replace('-', '_')
-
-        try:
-            module = importlib.import_module('snapcraft.plugins.' +
-                                             module_name)
-        except ImportError:
-            module = None
-
-        if not module:
-            logger.info('Searching for local plugin for %s', name)
-            sys.path = [_local_plugindir()] + sys.path
-            module = importlib.import_module(module_name)
-            sys.path.pop(0)
-
-        for prop_name in dir(module):
-            prop = getattr(module, prop_name)
-            if issubclass(prop, snapcraft.BasePlugin):
-                self.code = prop(part_name, options)
-                break
+        except jsonschema.ValidationError as e:
+            logger.error('Issues while loading properties for '
+                         '{}: {}'.format(part_name, e.message))
+            return
+
+    def _load_code(self, plugin_name, properties):
+        module_name = plugin_name.replace('-', '_')
+        module = None
+
+        with contextlib.suppress(ImportError):
+            module = _load_local('x-{}'.format(plugin_name))
+            logger.info('Loaded local plugin for %s', plugin_name)
+
+        if not module:
+            with contextlib.suppress(ImportError):
+                module = importlib.import_module(
+                    'snapcraft.plugins.{}'.format(module_name))
+
+        if not module:
+            logger.info('Searching for local plugin for %s', plugin_name)
+            with contextlib.suppress(ImportError):
+                module = _load_local(module_name)
+            if not module:
+                raise PluginError('Unknown plugin: {}'.format(plugin_name))
+
+        plugin = _get_plugin(module)
+        options = _make_options(properties, plugin.schema())
+        self.code = plugin(self.name, options)
 
     def __str__(self):
-        return self.part_names[0]
+        return self.name
 
     def __repr__(self):
-        return self.part_names[0]
+        return self.name
 
     def makedirs(self):
         dirs = [
@@ -153,11 +115,8 @@
     def is_valid(self):
         return self.valid
 
-    def names(self):
-        return self.part_names
-
     def notify_stage(self, stage, hint=''):
-        logger.info(stage + " " + self.part_names[0] + hint)
+        logger.info('%s %s %s', stage, self.name, hint)
 
     def is_dirty(self, stage):
         try:
@@ -183,23 +142,16 @@
             return True
         self.makedirs()
 
-        run_setup_stage_packages = self.code and \
-            hasattr(self.code, 'setup_stage_packages')
-        run_pull = self.code and hasattr(self.code, 'pull')
-
-        if run_setup_stage_packages or run_pull:
-            self.notify_stage("Pulling")
-
-        if run_setup_stage_packages:
-            try:
-                self.code.setup_stage_packages()
-            except repo.PackageNotFoundError as e:
-                logger.error(e.message)
-                return False
-
-        if run_pull:
-            if not getattr(self.code, 'pull')():
-                return False
+        self.notify_stage("Pulling")
+
+        try:
+            self.code.setup_stage_packages()
+        except repo.PackageNotFoundError as e:
+            logger.error(e.message)
+            return False
+
+        if not self.code.pull():
+            return False
 
         self.mark_done('pull')
         return True
@@ -208,10 +160,9 @@
         if not self.should_stage_run('build', force):
             return True
         self.makedirs()
-        if self.code and hasattr(self.code, 'build'):
-            self.notify_stage("Building")
-            if not getattr(self.code, 'build')():
-                return False
+        self.notify_stage("Building")
+        if not self.code.build():
+            return False
 
         self.mark_done('build')
         return True
@@ -284,14 +235,59 @@
         return True
 
     def env(self, root):
-        if self.code and hasattr(self.code, 'env'):
-            return getattr(self.code, 'env')(root)
-        return []
-
-
-def load_plugin(part_name, plugin_name, properties={}, load_code=True):
-    part = PluginHandler(plugin_name, part_name, properties,
-                         load_code=load_code)
+        return self.code.env(root)
+
+
+def _builtin_options():
+    return {
+        'filesets': {},
+        'snap': [],
+        'stage': [],
+        'stage-packages': [],
+        'build-packages': [],
+        'organize': {}
+    }
+
+
+def _make_options(properties, schema):
+    class Options():
+        pass
+    options = Options()
+
+    # Built in options are already validated
+    builtin_options = _builtin_options()
+    for key in builtin_options:
+        value = properties.pop(key, builtin_options[key])
+        setattr(options, key.replace('-', '_'), value)
+
+    jsonschema.validate(properties, schema)
+
+    schema_properties = schema.get('properties', {})
+    for key in schema_properties:
+        attr_name = key.replace('-', '_')
+        default_value = schema_properties[key].get('default')
+        attr_value = properties.get(key, default_value)
+        setattr(options, attr_name, attr_value)
+
+    return options
+
+
+def _get_plugin(module):
+    for attr in vars(module).values():
+        if isinstance(attr, type) and issubclass(attr, snapcraft.BasePlugin):
+            return attr
+
+
+def _load_local(module_name):
+    sys.path = [_local_plugindir()] + sys.path
+    module = importlib.import_module(module_name)
+    sys.path.pop(0)
+
+    return module
+
+
+def load_plugin(part_name, plugin_name, properties={}):
+    part = PluginHandler(plugin_name, part_name, properties)
     if not part.is_valid():
         logger.error('Could not load part %s', plugin_name)
         sys.exit(1)

=== modified file 'snapcraft/plugins/ant.py'
--- snapcraft/plugins/ant.py	2015-10-02 09:13:00 +0000
+++ snapcraft/plugins/ant.py	2015-10-13 15:27:09 +0000
@@ -21,17 +21,24 @@
 
 import snapcraft
 import snapcraft.common
+import snapcraft.plugins.jdk
 
 
 logger = logging.getLogger(__name__)
 
 
-class AntPlugin(snapcraft.BasePlugin):
+class AntPlugin(snapcraft.plugins.jdk.JdkPlugin):
+
+    def __init__(self, name, options):
+        super().__init__(name, options)
+        self.build_packages.append('ant')
 
     def pull(self):
+        super().pull()
         return self.handle_source_options()
 
     def build(self):
+        super().build()
         if not self.run(['ant']):
             return False
         files = glob.glob(os.path.join(self.builddir, 'target', '*.jar'))
@@ -44,8 +51,11 @@
             self.run(['cp', '-a'] + files + [jardir])
 
     def env(self, root):
+        env = super().env(root)
         jars = glob.glob(os.path.join(self.installdir, 'jar', '*.jar'))
-        if not jars:
-            return []
-        jars = [os.path.join(root, 'jar', os.path.basename(x)) for x in jars]
-        return ['CLASSPATH=%s:$CLASSPATH' % ':'.join(jars)]
+        if jars:
+            jars = [os.path.join(root, 'jar',
+                    os.path.basename(x)) for x in jars]
+            env.extend(
+                ['CLASSPATH=%s:$CLASSPATH' % ':'.join(jars)])
+        return env

=== modified file 'snapcraft/plugins/autotools.py'
--- snapcraft/plugins/autotools.py	2015-10-02 09:13:00 +0000
+++ snapcraft/plugins/autotools.py	2015-10-13 15:27:09 +0000
@@ -19,10 +19,29 @@
 
 
 class AutotoolsPlugin(MakePlugin):
+
+    @classmethod
+    def schema(cls):
+        schema = super().schema()
+        schema['properties']['configflags'] = {
+            'type': 'array',
+            'minitems': 1,
+            'uniqueItems': True,
+            'items': {
+                'type': 'string',
+            },
+            'default': [],
+        }
+
+        return schema
+
     def __init__(self, name, options):
         super().__init__(name, options)
-        if self.options.configflags is None:
-            self.options.configflags = []
+        self.build_packages.extend([
+            'autoconf',
+            'automake',
+            'autopoint',
+        ])
 
     def build(self):
         if not os.path.exists(os.path.join(self.builddir, "configure")):

=== modified file 'snapcraft/plugins/cmake.py'
--- snapcraft/plugins/cmake.py	2015-10-02 09:13:00 +0000
+++ snapcraft/plugins/cmake.py	2015-10-13 15:27:09 +0000
@@ -14,14 +14,29 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-from snapcraft.plugins.make import MakePlugin
-
-
-class CMakePlugin(MakePlugin):
+import snapcraft.plugins.make
+
+
+class CMakePlugin(snapcraft.plugins.make.MakePlugin):
+
+    @classmethod
+    def schema(cls):
+        schema = super().schema()
+        schema['properties']['configflags'] = {
+            'type': 'array',
+            'minitems': 1,
+            'uniqueItems': True,
+            'items': {
+                'type': 'string',
+            },
+            'default': [],
+        }
+
+        return schema
+
     def __init__(self, name, options):
         super().__init__(name, options)
-        if self.options.configflags is None:
-            self.options.configflags = []
+        self.build_packages.append('cmake')
 
     def build(self):
         return self.run(['cmake', '.', '-DCMAKE_INSTALL_PREFIX='] +

=== modified file 'snapcraft/plugins/copy.py'
--- snapcraft/plugins/copy.py	2015-10-02 09:13:00 +0000
+++ snapcraft/plugins/copy.py	2015-10-13 15:27:09 +0000
@@ -25,6 +25,19 @@
 
 class CopyPlugin(snapcraft.BasePlugin):
 
+    @classmethod
+    def schema(cls):
+        return {
+            'properties': {
+                'files': {
+                    'type': 'object',
+                },
+            },
+            'required': [
+                'files',
+            ]
+        }
+
     def build(self):
         res = True
         for src in sorted(self.options.files):

=== modified file 'snapcraft/plugins/go.py'
--- snapcraft/plugins/go.py	2015-10-02 09:13:00 +0000
+++ snapcraft/plugins/go.py	2015-10-13 15:27:09 +0000
@@ -20,8 +20,23 @@
 
 class GoPlugin(snapcraft.BasePlugin):
 
+    @classmethod
+    def schema(cls):
+        return {
+            'properties': {
+                'source': {
+                    'type': 'string',
+                },
+            },
+            'required': [
+                'source',
+            ]
+        }
+
     def __init__(self, name, options):
         super().__init__(name, options)
+        self.build_packages.append('golang-go')
+
         if self.options.source.startswith("lp:"):
             self.fullname = self.options.source.split(":~")[1]
         else:

=== modified file 'snapcraft/plugins/jdk.py'
--- snapcraft/plugins/jdk.py	2015-09-09 19:41:36 +0000
+++ snapcraft/plugins/jdk.py	2015-10-13 15:27:09 +0000
@@ -19,9 +19,9 @@
 
 class JdkPlugin(snapcraft.BasePlugin):
 
-    _PLUGIN_STAGE_PACKAGES = [
-        'default-jdk',
-    ]
+    def __init__(self, name, options):
+        super().__init__(name, options)
+        self.stage_packages.append('default-jdk')
 
     def env(self, root):
         return ['JAVA_HOME=%s/usr/lib/jvm/default-java' % root,

=== modified file 'snapcraft/plugins/make.py'
--- snapcraft/plugins/make.py	2015-10-01 08:54:15 +0000
+++ snapcraft/plugins/make.py	2015-10-13 15:27:09 +0000
@@ -19,6 +19,10 @@
 
 class MakePlugin(snapcraft.BasePlugin):
 
+    def __init__(self, name, options):
+        super().__init__(name, options)
+        self.build_packages.append('make')
+
     def pull(self):
         return self.handle_source_options()
 

=== modified file 'snapcraft/plugins/maven.py'
--- snapcraft/plugins/maven.py	2015-10-02 09:13:00 +0000
+++ snapcraft/plugins/maven.py	2015-10-13 15:27:09 +0000
@@ -20,17 +20,25 @@
 
 import snapcraft
 import snapcraft.common
+import snapcraft.plugins.jdk
 
 
 logger = logging.getLogger(__name__)
 
 
-class MavenPlugin(snapcraft.BasePlugin):
+class MavenPlugin(snapcraft.plugins.jdk.JdkPlugin):
+
+    def __init__(self, name, options):
+        super().__init__(name, options)
+        self.build_packages.append('maven')
 
     def pull(self):
+        super().pull()
         return self.handle_source_options()
 
     def build(self):
+        super().build()
+
         if not self.run(['mvn', 'package']):
             return False
         jarfiles = glob.glob(os.path.join(self.builddir, 'target', '*.jar'))

=== modified file 'snapcraft/plugins/python2.py'
--- snapcraft/plugins/python2.py	2015-10-02 09:13:00 +0000
+++ snapcraft/plugins/python2.py	2015-10-13 15:27:09 +0000
@@ -22,23 +22,30 @@
 
 class Python2Plugin(snapcraft.BasePlugin):
 
-    _PLUGIN_STAGE_PACKAGES = [
-        'python-dev',
-        'python-pkg-resources',
-        'python-setuptools',
-    ]
+    @classmethod
+    def schema(cls):
+        schema = super().schema()
+        schema['properties']['requirements'] = {
+            'type': 'string',
+        }
+        schema.pop('required')
+
+        return schema
 
     def __init__(self, name, options):
         super().__init__(name, options)
-        self.requirements = options.requirements
-        self.source = options.source
+        self.stage_packages.extend([
+            'python-dev',
+            'python-pkg-resources',
+            'python-setuptools',
+        ])
 
     def env(self, root):
         return ["PYTHONPATH=%s" % os.path.join(
             root, 'usr', 'lib', self.python_version, 'dist-packages')]
 
     def pull(self):
-        if self.source and not self.handle_source_options():
+        if self.options.source and not self.handle_source_options():
             return False
 
         return self._pip()
@@ -48,10 +55,10 @@
         if os.listdir(self.sourcedir):
             setup = os.path.join(self.sourcedir, 'setup.py')
 
-        if self.requirements:
-            requirements = os.path.join(os.getcwd(), self.requirements)
+        if self.options.requirements:
+            requirements = os.path.join(os.getcwd(), self.options.requirements)
 
-        if not os.path.exists(setup) and not self.requirements:
+        if not os.path.exists(setup) and not self.options.requirements:
             return True
 
         easy_install = os.path.join(
@@ -73,7 +80,7 @@
         pip_install = ['python2', pip2, 'install', '--target',
                        site_packages_dir]
 
-        if self.requirements and not self.run(
+        if self.options.requirements and not self.run(
                 pip_install + ['--requirement', requirements]):
             return False
 

=== modified file 'snapcraft/plugins/python3.py'
--- snapcraft/plugins/python3.py	2015-10-05 16:16:23 +0000
+++ snapcraft/plugins/python3.py	2015-10-13 15:27:09 +0000
@@ -22,23 +22,30 @@
 
 class Python3Plugin(snapcraft.BasePlugin):
 
-    _PLUGIN_STAGE_PACKAGES = [
-        'python3-dev',
-        'python3-pkg-resources',
-        'python3-setuptools',
-    ]
+    @classmethod
+    def schema(cls):
+        schema = super().schema()
+        schema['properties']['requirements'] = {
+            'type': 'string',
+        }
+        schema.pop('required')
+
+        return schema
 
     def __init__(self, name, options):
         super().__init__(name, options)
-        self.requirements = options.requirements
-        self.source = options.source
+        self.stage_packages.extend([
+            'python3-dev',
+            'python3-pkg-resources',
+            'python3-setuptools',
+        ])
 
     def env(self, root):
         return ["PYTHONPATH=%s" % os.path.join(
             root, 'usr', 'lib', self.python_version, 'dist-packages')]
 
     def pull(self):
-        if self.source and not self.handle_source_options():
+        if self.options.source and not self.handle_source_options():
             return False
 
         return self._pip()
@@ -48,10 +55,10 @@
         if os.listdir(self.sourcedir):
             setup = os.path.join(self.sourcedir, 'setup.py')
 
-        if self.requirements:
-            requirements = os.path.join(os.getcwd(), self.requirements)
+        if self.options.requirements:
+            requirements = os.path.join(os.getcwd(), self.options.requirements)
 
-        if not os.path.exists(setup) and not self.requirements:
+        if not os.path.exists(setup) and not self.options.requirements:
             return True
 
         easy_install = os.path.join(
@@ -72,7 +79,7 @@
         pip_install = ['python3', pip3, 'install', '--target',
                        site_packages_dir]
 
-        if self.requirements and not self.run(
+        if self.options.requirements and not self.run(
                 pip_install + ['--requirement', requirements]):
             return False
 

=== modified file 'snapcraft/plugins/qml.py'
--- snapcraft/plugins/qml.py	2015-10-02 09:13:00 +0000
+++ snapcraft/plugins/qml.py	2015-10-13 15:27:09 +0000
@@ -21,41 +21,47 @@
 
 class QmlPlugin(snapcraft.BasePlugin):
 
-    _PLUGIN_STAGE_PACKAGES = [
-        "qmlscene",
-        "qtdeclarative5-qtmir-plugin",
-        "mir-graphics-drivers-desktop",
-        "qtubuntu-desktop",
-        "ttf-ubuntu-font-family",
-        # if there's a metapackage for these, please swap it in here:
-        "qml-module-qt-labs-folderlistmodel",
-        "qml-module-qt-labs-settings",
-        "qml-module-qt-websockets",
-        "qml-module-qtfeedback",
-        "qml-module-qtgraphicaleffects",
-        "qml-module-qtlocation",
-        "qml-module-qtmultimedia",
-        "qml-module-qtorganizer",
-        "qml-module-qtpositioning",
-        "qml-module-qtqml-models2",
-        "qml-module-qtqml-statemachine",
-        "qml-module-qtquick-controls",
-        "qml-module-qtquick-dialogs",
-        "qml-module-qtquick-layouts",
-        "qml-module-qtquick-localstorage",
-        "qml-module-qtquick-particles2",
-        "qml-module-qtquick-privatewidgets",
-        "qml-module-qtquick-window2",
-        "qml-module-qtquick-xmllistmodel",
-        "qml-module-qtquick2",
-        "qml-module-qtsensors",
-        "qml-module-qtsysteminfo",
-        "qml-module-qttest",
-        "qml-module-qtwebkit",
-        "qml-module-ubuntu-connectivity",
-        "qml-module-ubuntu-onlineaccounts",
-        "qml-module-ubuntu-onlineaccounts-client",
-    ]
+    @classmethod
+    def schema(cls):
+        return {}
+
+    def __init__(self, name, options):
+        super().__init__(name, options)
+        self.stage_packages.extend([
+            "qmlscene",
+            "qtdeclarative5-qtmir-plugin",
+            "mir-graphics-drivers-desktop",
+            "qtubuntu-desktop",
+            "ttf-ubuntu-font-family",
+            # if there's a metapackage for these, please swap it in here:
+            "qml-module-qt-labs-folderlistmodel",
+            "qml-module-qt-labs-settings",
+            "qml-module-qt-websockets",
+            "qml-module-qtfeedback",
+            "qml-module-qtgraphicaleffects",
+            "qml-module-qtlocation",
+            "qml-module-qtmultimedia",
+            "qml-module-qtorganizer",
+            "qml-module-qtpositioning",
+            "qml-module-qtqml-models2",
+            "qml-module-qtqml-statemachine",
+            "qml-module-qtquick-controls",
+            "qml-module-qtquick-dialogs",
+            "qml-module-qtquick-layouts",
+            "qml-module-qtquick-localstorage",
+            "qml-module-qtquick-particles2",
+            "qml-module-qtquick-privatewidgets",
+            "qml-module-qtquick-window2",
+            "qml-module-qtquick-xmllistmodel",
+            "qml-module-qtquick2",
+            "qml-module-qtsensors",
+            "qml-module-qtsysteminfo",
+            "qml-module-qttest",
+            "qml-module-qtwebkit",
+            "qml-module-ubuntu-connectivity",
+            "qml-module-ubuntu-onlineaccounts",
+            "qml-module-ubuntu-onlineaccounts-client",
+        ])
 
     def snap_fileset(self):
         return ['*',

=== modified file 'snapcraft/plugins/scons.py'
--- snapcraft/plugins/scons.py	2015-10-01 08:54:15 +0000
+++ snapcraft/plugins/scons.py	2015-10-13 15:27:09 +0000
@@ -21,11 +21,24 @@
 
 class SconsPlugin(snapcraft.BasePlugin):
 
+    @classmethod
+    def schema(cls):
+        schema = super().schema()
+        schema['properties']['scons-options'] = {
+            'type': 'array',
+            'minitems': 1,
+            'uniqueItems': True,
+            'items': {
+                'type': 'string',
+            },
+            'default': []
+        }
+
+        return schema
+
     def __init__(self, name, options):
         super().__init__(name, options)
-        self.scons_options = []
-        if options.scons_options:
-            self.scons_options = options.scons_options
+        self.build_packages.append('scons')
 
     def pull(self):
         return self.handle_source_options()
@@ -33,5 +46,6 @@
     def build(self):
         env = os.environ.copy()
         env['DESTDIR'] = self.installdir
-        return (self.run(['scons', ] + self.scons_options) and
-                self.run(['scons', 'install'] + self.scons_options, env=env))
+        return (self.run(['scons', ] + self.options.scons_options) and
+                self.run(['scons', 'install'] +
+                         self.options.scons_options, env=env))

=== modified file 'snapcraft/plugins/tar_content.py'
--- snapcraft/plugins/tar_content.py	2015-08-31 13:49:40 +0000
+++ snapcraft/plugins/tar_content.py	2015-10-13 15:27:09 +0000
@@ -20,6 +20,19 @@
 
 class TarContentPlugin(snapcraft.BasePlugin):
 
+    @classmethod
+    def schema(cls):
+        return {
+            'properties': {
+                'source': {
+                    'type': 'string',
+                },
+            },
+            'required': [
+                'source',
+            ]
+        }
+
     def __init__(self, name, options):
         super().__init__(name, options)
         self.tar = snapcraft.sources.Tar(self.options.source, self.builddir)

=== modified file 'snapcraft/tests/test_cmds.py'
--- snapcraft/tests/test_cmds.py	2015-10-02 09:13:00 +0000
+++ snapcraft/tests/test_cmds.py	2015-10-13 15:27:09 +0000
@@ -39,14 +39,14 @@
         tmpdir = tmpdirObject.name
 
         part1 = mock.Mock()
-        part1.names.return_value = ['part1']
+        part1.name = 'part1'
         part1.code.options.stage = ['*']
         part1.installdir = tmpdir + '/install1'
         os.makedirs(part1.installdir + '/a')
         open(part1.installdir + '/a/1', mode='w').close()
 
         part2 = mock.Mock()
-        part2.names.return_value = ['part2']
+        part2.name = 'part2'
         part2.code.options.stage = ['*']
         part2.installdir = tmpdir + '/install2'
         os.makedirs(part2.installdir + '/a')
@@ -57,7 +57,7 @@
             f.write('a/2')
 
         part3 = mock.Mock()
-        part3.names.return_value = ['part3']
+        part3.name = 'part3'
         part3.code.options.stage = ['*']
         part3.installdir = tmpdir + '/install3'
         os.makedirs(part3.installdir + '/a')

=== modified file 'snapcraft/tests/test_plugin.py'
--- snapcraft/tests/test_plugin.py	2015-10-02 09:13:00 +0000
+++ snapcraft/tests/test_plugin.py	2015-10-13 15:27:09 +0000
@@ -28,16 +28,12 @@
     plugin,
     tests
 )
-from snapcraft.tests import mock_plugin
-
-
-def get_test_plugin(name='mock', part_name='mock-part',
-                    properties=None, load_code=False, load_config=False):
+
+
+def get_test_plugin(name='mock', part_name='mock-part', properties=None):
     if properties is None:
         properties = {}
-    return plugin.PluginHandler(
-        name, part_name, properties, load_code=load_code,
-        load_config=load_config)
+    return plugin.PluginHandler(name, part_name, properties)
 
 
 class PluginTestCase(tests.TestCase):
@@ -46,7 +42,7 @@
         fake_logger = fixtures.FakeLogger(level=logging.ERROR)
         self.useFixture(fake_logger)
 
-        get_test_plugin('test_unexisting_name', load_config=True)
+        get_test_plugin('test_unexisting_name')
 
         self.assertEqual(
             'Unknown plugin: test_unexisting_name\n', fake_logger.output)
@@ -202,24 +198,18 @@
 
                 self.assertEqual(expected, result)
 
-    def test_notify_stage_must_log_information(self):
-        fake_logger = fixtures.FakeLogger(level=logging.INFO)
-        self.useFixture(fake_logger)
-
-        p = get_test_plugin()
-        p.notify_stage('test stage')
-
-        self.assertEqual('test stage mock-part\n', fake_logger.output)
-
-    def test_non_local_plugins(self):
-        """Ensure regular plugins are loaded from snapcraft only"""
-        def mock_import_modules(module_name):
-            # called with the full snapcraft path
-            self.assertEqual(module_name, "snapcraft.plugins.mock")
-            return mock_plugin
-        with patch("importlib.import_module", side_effect=mock_import_modules):
-            plugin.PluginHandler(
-                "mock", "mock-part", {}, load_config=False, load_code=True)
+    @patch('importlib.import_module')
+    @patch('snapcraft.plugin._load_local')
+    @patch('snapcraft.plugin._get_plugin')
+    def test_non_local_plugins(self, plugin_mock,
+                               local_load_mock, import_mock):
+        mock_plugin = Mock()
+        mock_plugin.schema.return_value = {}
+        plugin_mock.return_value = mock_plugin
+        local_load_mock.side_effect = ImportError()
+        plugin.PluginHandler('mock', 'mock-part', {})
+        import_mock.assert_called_with('snapcraft.plugins.mock')
+        local_load_mock.assert_called_with('x-mock')
 
     def test_filesets_includes_without_relative_paths(self):
         with self.assertRaises(plugin.PluginError) as raised:
@@ -240,8 +230,7 @@
         self.useFixture(fake_logger)
 
         with self.assertRaises(SystemExit) as raised:
-            plugin.load_plugin(
-                'dummy-part', 'test_unexisting_name', load_code=False)
+            plugin.load_plugin('dummy-part', 'test_unexisting_name')
 
         self.assertEqual(raised.exception.code, 1, 'Wrong exit code returned.')
         self.assertEqual(

=== modified file 'snapcraft/yaml.py'
--- snapcraft/yaml.py	2015-10-05 16:56:10 +0000
+++ snapcraft/yaml.py	2015-10-13 15:27:09 +0000
@@ -129,36 +129,19 @@
 
             self.load_plugin(part_name, plugin_name, properties)
 
-        self._load_missing_part_plugins()
         self._compute_part_dependencies(after_requests)
         self.all_parts = self._sort_parts()
 
-    def _load_missing_part_plugins(self):
-        new_parts = self.all_parts.copy()
-        while new_parts:
-            part = new_parts.pop(0)
-            requires = part.config.get('requires', [])
-            for required_part in requires:
-                present = False
-                for p in self.all_parts:
-                    if required_part in p.names():
-                        present = True
-                        break
-                if not present:
-                    new_parts.append(self.load_plugin(required_part,
-                                                      required_part, {}))
-
     def _compute_part_dependencies(self, after_requests):
         '''Gather the lists of dependencies and adds to all_parts.'''
         w = snapcraft.wiki.Wiki()
 
         for part in self.all_parts:
-            dep_names = part.config.get('requires', []) + \
-                after_requests.get(part.names()[0], [])
+            dep_names = after_requests.get(part.name, [])
             for dep in dep_names:
                 found = False
                 for i in range(len(self.all_parts)):
-                    if dep in self.all_parts[i].names():
+                    if dep in self.all_parts[i].name:
                         part.deps.append(self.all_parts[i])
                         found = True
                         break
@@ -195,11 +178,10 @@
 
         return sorted_parts
 
-    def load_plugin(self, part_name, plugin_name, properties, load_code=True):
-        part = snapcraft.plugin.load_plugin(part_name, plugin_name,
-                                            properties, load_code=load_code)
+    def load_plugin(self, part_name, plugin_name, properties):
+        part = snapcraft.plugin.load_plugin(part_name, plugin_name, properties)
 
-        self.build_tools += part.config.get('build-packages', [])
+        self.build_tools += part.code.build_packages
         self.all_parts.append(part)
         return part