Merge lp:~sergiusens/snapcraft/help into lp:~snappy-dev/snapcraft/core

[Sergio Schvezov]

Adding a help command to get snapcraft help from the cli.

[John Lenton (chipaca)]

Nice work.

Loots of comments, below ... needsfixing just because they're so many.

review: Needs Fixing

[John Lenton (chipaca)]

Mucho bettero!
One that I didn't catch last time. But +1'ing it.

review: Approve

[Snappy Tarmac (snappydevtarmac)]

The attempt to merge lp:~sergiusens/snapcraft/help into lp:snapcraft failed. Below is the output from the failed tests.

snapcraft/help.py:19: 'sys' imported but unused

[Snappy Tarmac (snappydevtarmac)]

There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.

=== modified file 'snapcraft/__init__.py'
--- snapcraft/__init__.py	2015-10-26 14:02:06 +0000
+++ snapcraft/__init__.py	2015-10-27 20:21:51 +0000
@@ -14,8 +14,104 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""Plugins drive the build process for a part.
+Each part can use an individual plugin that understands how to work with
+the declared sources.
+
+These plugins have a lifecycle that consists of the following steps:
+
+    - pull
+    - build
+    - stage
+    - snap
+    - assemble
+
+# Lifecycle
+
+## Pull
+This is the first step. This is where content is downloaded, e.g. checkout a
+git repository or download a binary component like the Java SDK. Snapcraft
+will place the downloaded content for each part in that part's
+`parts/<part-name>/src` directory.
+
+## Build
+This is the step that follows pull. Each part is built in its
+`parts/part-name/build` directory and installs itself into
+`parts/part-name/install`.
+
+## Stage
+After the build step of each part, the parts are combined into a single
+directory tree that is called the "staging area". It can be found
+under the `./stage` directory.
+
+This is the area where all parts can share assets such as libraries to link
+against.
+
+## Snap
+The snap step moves the data into a `./snap` directory. It contains only
+the content that will be put into the final snap package, unlike the staging
+area which may include some development files not destined for your package.
+
+The Snappy metadata information about your project will also now be placed
+in `./snap/meta`.
+
+This `./snap` directory is useful for inspecting what is going into your
+snap and to make any final post-processing on snapcraft's output.
+
+## Assemble
+The final step builds a snap package out of the `snap` directory.
+
+# Common keywords
+
+There are common builtin keywords provided to a snapcraft plugin which can
+be used in any part irrespective of the plugin, these are
+
+    - after:
+      (list of strings)
+      Specifies any parts that should be built before this part is.  This
+      is mostly useful when a part needs a library or build tool built by
+      another part.
+      If a part listed in `after` is not defined locally, it will be
+      searched for in the wiki (https://wiki.ubuntu.com/Snappy/Wiki)
+    - stage-packages:
+      (list of strings)
+      A list of Ubuntu packages to use that are needed to support the part
+      creation.
+    - build-packages:
+      (list of strings)
+      A list of Ubuntu packages to be installed on the host to aid in
+      building the part but not going into the final snap.
+    - organize:
+      (yaml subsection)
+      A dictionary exposing replacements, the key is the internal filename
+      whilst the value is the exposed filename, filesets will refer to the
+      exposed named applied after organization is applied.
+      This can be used to avoid conflicts by renaming files or using a
+      different layout from what came out of the build, e.g.;
+      `/usr/local/share/icon.png` -> `/usr/share/icon.png`.
+    - filesets:
+      (yaml subsection)
+      A dictionary with filesets, the key being a recognizable user defined
+      string and its value a list of filenames to be included or
+      excluded. Globbing is achieved with * for either inclusions or
+      exclusion. Exclusions are denoted by an initial `-`.
+      Globbing is computed from the part's install directory in
+      `parts/<part-name>/install`.
+    - stage:
+      (list of strings)
+      A list of files from a part’s installation to expose in `stage`.
+      Rules applying to the list here are the same as those of filesets.
+      Referencing of fileset keys is done with a $ prefixing the fileset
+      key, which will expand with the value of such key.
+    - snap:
+      (list of strings)
+      A list of files from a part’s installation to expose in `snap`.
+      Rules applying to the list here are the same as those of filesets.
+      Referencing of fileset keys is done with a $ prefixing the fileset
+      key, which will expand with the value of such key.
+"""
+
 import contextlib
-import logging
 import os
 
 import snapcraft.common
@@ -23,9 +119,6 @@
 import snapcraft.repo
 
 
-logger = logging.getLogger(__name__)
-
-
 class BasePlugin:
 
     @classmethod
@@ -35,8 +128,8 @@
         optionally the 'requires' keyword with a list of required
         'properties'.
 
-        By default the the properties will be that of a standard VCS, override
-        in custom implementations if required.
+        By default the properties will be that of a standard VCS,
+        override in custom implementations if required.
         """
         return {
             '$schema': 'http://json-schema.org/draft-04/schema#',
@@ -113,8 +206,8 @@
     def build(self):
         """Build the source code retrieved from the pull phase.
 
-        The base implementation does nothing by default. Override this method
-        if you need to process the source code to make it runnable.
+        The base implementation does nothing by default. Override this
+        method if you need to process the source code to make it runnable.
         """
         pass
 
@@ -122,7 +215,8 @@
         """Return a list of files to include or exclude in the resulting snap
 
         The staging phase of a plugin's lifecycle may populate many things
-        into the staging directory in order to succeed in building a project.
+        into the staging directory in order to succeed in building a
+        project.
         During the stripping phase and in order to have a clean snap, the
         plugin can provide additional logic for stripping build components
         from the final snap and alleviate the part author from doing so for

=== added file 'snapcraft/help.py'
--- snapcraft/help.py	1970-01-01 00:00:00 +0000
+++ snapcraft/help.py	2015-10-27 20:21:51 +0000
@@ -0,0 +1,73 @@
+# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
+#
+# Copyright (C) 2015 Canonical Ltd
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import importlib
+import logging
+
+import snapcraft
+from snapcraft import sources
+from snapcraft import cmds
+
+
+logger = logging.getLogger(__name__)
+
+
+_TOPICS = {
+    'sources': sources,
+    'plugins': snapcraft,
+}
+
+
+def topic(args=None):
+    """Get help on additional topics or plugin usage.
+    snapcraft help topics    To get the list of topics
+    snapcraft help <plugin>  To get help for a specific plugin
+    snapcraft help <topic>   To get help on a speficic topic
+
+To see the list of available plugins run
+
+    snapcraft list-plugins
+"""
+    if args.topic == 'topics':
+        for key in _TOPICS:
+            print(key)
+    elif args.topic in _TOPICS:
+        _topic_help(args.topic, args.devel)
+    else:
+        _module_help(args.topic, args.devel)
+
+
+def _topic_help(module_name, devel):
+    if devel:
+        help(_TOPICS[module_name])
+    else:
+        print(_TOPICS[module_name].__doc__)
+
+
+def _module_help(module_name, devel):
+    try:
+        module = importlib.import_module(
+            'snapcraft.plugins.{}'.format(module_name))
+    except ImportError:
+        logger.error('The plugin does not exist. Use one of the following:')
+        cmds.list_plugins()
+
+    if module.__doc__ and devel:
+        help(module)
+    elif module.__doc__:
+        print(module.__doc__)
+    else:
+        print('The plugin has no documentation')

=== modified file 'snapcraft/main.py'
--- snapcraft/main.py	2015-10-26 15:54:30 +0000
+++ snapcraft/main.py	2015-10-27 20:21:51 +0000
@@ -19,6 +19,8 @@
 import sys
 
 import snapcraft.cmds
+
+from snapcraft import help
 from snapcraft import log
 
 
@@ -65,6 +67,15 @@
         help='clean up the environment (to start from scratch)')
     parser.set_defaults(func=snapcraft.cmds.clean)
 
+    parser = subparsers.add_parser(
+        'help',
+        usage=help.topic.__doc__,
+        help='obtain help for plugins and specific topics')
+    parser.set_defaults(func=help.topic)
+    parser.add_argument('topic', help='plugin name or topic to get help from')
+    parser.add_argument('--devel', action='store_true',
+                        help='show the development help')
+
     parser = subparsers.add_parser('pull', help='get sources',
                                    parents=[cmd_parser])
     parser.set_defaults(func=snapcraft.cmds.cmd)

=== modified file 'snapcraft/plugins/ant.py'
--- snapcraft/plugins/ant.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/ant.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,16 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The ant plugin is useful for ant based parts.
+
+The ant build system is commonly used to build Java projects.
+The plugin requires a build.xml in the root of the source tree.
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+"""
+
 import glob
 import logging
 import os

=== modified file 'snapcraft/plugins/autotools.py'
--- snapcraft/plugins/autotools.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/autotools.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,26 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The autotools plugin is used for autotools based parts.
+
+Autotools based projects are the ones that have the usual
+`./configure && make && make install` instruction set.
+
+The plugin tries to build using ./configure first, if it is not there
+it will run ./autogen and if autogen is not there it will run autoreconf.
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+
+In additon, this plugin uses the following plugin specific keywords:
+
+    - configflags:
+      (list of strings)
+      configure flags to pass to the build such as those shown by running
+      './configure --help'
+"""
+
 import os
 from snapcraft.plugins.make import MakePlugin
 

=== modified file 'snapcraft/plugins/catkin.py'
--- snapcraft/plugins/catkin.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/catkin.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,19 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The catkin plugin is useful for building ROS parts.
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+
+Additionally, this plugin uses the following plugin specific keywords:
+
+    - catkin-packages:
+      (list of strings)
+      List of catkin packages to build.
+"""
+
 import lxml.etree
 import os
 import tempfile

=== modified file 'snapcraft/plugins/cmake.py'
--- snapcraft/plugins/cmake.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/cmake.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,22 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The cmake plugin is useful for building cmake based parts.
+
+These are projects that have a CMakeLists.txt that drives the build.
+The plugin requires a CMakeLists.txt in the root of the source tree.
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+
+Additionally, this plugin uses the following plugin specific keywords:
+
+    - configflags:
+      (list of strings)
+      configure flags to pass to the build using the common cmake semantics.
+"""
+
 import snapcraft.plugins.make
 
 

=== modified file 'snapcraft/plugins/go.py'
--- snapcraft/plugins/go.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/go.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,19 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The go plugin can be used for go projects using `go get`.
+
+This plugin uses the common plugin keywords, for more information check the
+'plugins' topic.
+
+Additionally, this plugin uses the following plugin specific keywords:
+
+    - source:
+      (string)
+      A path to some source tree to build in the form of something `go get`
+      understands.
+"""
+
 import os
 import snapcraft
 

=== modified file 'snapcraft/plugins/make.py'
--- snapcraft/plugins/make.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/make.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,16 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The make plugin is useful for building make based parts.
+
+Make based projects are projects that have a Makefile that drives the
+build.
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+"""
+
 import snapcraft
 
 

=== modified file 'snapcraft/plugins/maven.py'
--- snapcraft/plugins/maven.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/maven.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,22 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""This plugin is useful for building parts that use maven.
+
+The maven build system is commonly used to build Java projects.
+The plugin requires a pom.xml in the root of the source tree.
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+
+Additionally, this plugin uses the following plugin specific keywords:
+
+    - maven-options:
+      (list of strings)
+      flags to pass to the build using the maven semantics for parameters.
+"""
+
 import glob
 import logging
 import os

=== modified file 'snapcraft/plugins/python2.py'
--- snapcraft/plugins/python2.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/python2.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,26 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The python2 plugin can be used for python 2 based parts.
+
+The python2 plugin can be used for python 2 projects where you would
+want to do:
+
+    - import python modules with a requirements.txt
+    - build a python project that has a setup.py
+    - install sources straight from pip
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+
+Additionally, this plugin uses the following plugin specific keywords:
+
+    - requirements:
+      (string)
+      path to a requirements.txt file
+"""
+
 import os
 import tempfile
 

=== modified file 'snapcraft/plugins/python3.py'
--- snapcraft/plugins/python3.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/python3.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,26 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The python3 plugin can be used for python 3 based parts.
+
+The python3 plugin can be used for python 3 projects where you would
+want to do:
+
+    - import python modules with a requirements.txt
+    - build a python project that has a setup.py
+    - install sources straight from pip
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+
+Additionally, this plugin uses the following plugin specific keywords:
+
+    - requirements:
+      (string)
+      path to a requirements.txt file
+"""
+
 import os
 import tempfile
 

=== modified file 'snapcraft/plugins/scons.py'
--- snapcraft/plugins/scons.py	2015-10-21 20:03:15 +0000
+++ snapcraft/plugins/scons.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,21 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""The scons plugin is useful for building parts that build with scons.
+
+These are projects that have a SConstruct that drives the build.
+
+This plugin uses the common plugin keywords as well as those for "sources".
+For more information check the 'plugins' topic for the former and the
+'sources' topic for the latter.
+
+Additionally, this plugin uses the following plugin specific keywords:
+
+    - scons-options:
+      (list of strings)
+      flags to pass to the build using the scons semantics for parameters.
+"""
+
 import os
 
 import snapcraft

=== modified file 'snapcraft/sources.py'
--- snapcraft/sources.py	2015-10-21 20:03:15 +0000
+++ snapcraft/sources.py	2015-10-27 20:21:51 +0000
@@ -14,6 +14,36 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""Common keywords for plugins that use common source options.
+
+A part that uses common source options can have these keyword entries:
+
+    - source:
+      (string)
+      A path to some source tree to build. It can be either remote or local,
+      and either a directory tree or a tarball.
+    - source-type:
+      (string)
+      In some cases the source is not enough to identify the version control
+      system or compression algorithim. This hints the system into what to
+      do, the valid values are:
+
+                   - bzr
+                   - mercurial
+                   - hg
+                   - git
+                   - tar
+
+    - source-branch:
+      (string)
+      A specific branch from the source tree. This will result in an error
+      if used with a bazaar source type.
+    - source-tag:
+      (string)
+      A specific tag from the source tree.
+"""
+
+
 import logging
 import os
 import os.path
@@ -25,7 +55,6 @@
 import snapcraft.common
 
 
-logger = logging.getLogger(__name__)
 logging.getLogger('urllib3').setLevel(logging.CRITICAL)
 
 
@@ -225,6 +254,12 @@
 
 
 def get(sourcedir, builddir, options):
+    """Populate sourcedir and builddir from parameters defined in options.
+
+    :param str sourcedir: The source directory to use.
+    :param str builddir: The build directory to use.
+    :param options: source options.
+    """
     source_type = getattr(options, 'source_type', None)
     source_tag = getattr(options, 'source_tag', None)
     source_branch = getattr(options, 'source_branch', None)