Merge lp:~james-w/pkgme/add-docs into lp:pkgme

[James Westby]

Hi,

This branch adds some skeleton docs, splitting in to two
target audiences.

I want to get the backend docs to a good state first, as we want
to attract backend authors. I made a start on that documentation, but
the main aim of this branch was to write the sphinx extension.

This extension is similar to autodoc, but it writes out information on
the bits of information that backends can provide. This means that the
documentation will always be in sync with the code.

Thanks,

James

[lp:~james-w/pkgme/add-docs]

40. By James Westby on 2010-12-15

Fix a couple of warts.

[Barry Warsaw (barry)]

Diff looks pretty good to me, though I think there are a few typos. Also, I get many failures in the test suite (I haven't tried the trunk to see if the same failures are there).

Here are just a few quick suggestions:

=== added file 'doc/backends/index.txt'
--- doc/backends/index.txt 1970-01-01 00:00:00 +0000
+++ doc/backends/index.txt 2010-12-15 23:39:37 +0000
> +pkgme itself is agnostic about the type of project that it is working on, but knows
> +how to put information about a project in to the packaging files such that result

"...into the packaging files such that the result should..."

> +should work, assuming that the information provided is good enough.
> +
> +Every time it needs a bit of information about the project, such as its name, or
> +the dependencies it requires, it asks the backend that is the best match for the
> +project.

"...it asks the matching backend for the project."

> +
> +In this manner, the backends, and so you as a backend author, don't need to know

"In this manner the backends (and you as a backend author), don't need..."

> +However, if it is asked about a Ruby on Rails project then it should report a higher
> +number, the question is just how high. Consider the fact that there may be a
> +generic Ruby backend, which would presumably be able to give some useful information
> +about this project. However, the Ruby backend shouldn't have to detect that this is a
> +Rails project, and know that there is a Rails backend, and so answer "0", as that would

s/and know/or know/

> +There are a couple of places where the backend needs to know something related to packaging.
> +
> +The first of these is in the specification of the build dependencies, where the backend
> +has to provide the list of packages that are needed. While the backend probably has
> +available the list of modules used or similar, it wouldn't be possible for pkgme to
> +translate these to a list of packages while remaining generic.

True, but I wonder if we could provide a set of conveniences for finding
packages. For example, a backend might know the name of an upstream package
it depends on, but not the binary package it must actually depend on. A
search facility (e.g. something like 'aptitude search') might be useful. Or
an API for finding the binary packages produced by a source package. Some of
that can be generalized even if the actual package dependency calculation must
be done by the backend.

> +Therefore one of the things the backend must be able to do is to go from the list of
> +modules to the list of packages that provide them, using whatever means is appropriate.
> +
> +The second area where the backend needs to know something related to packaging is specifying
> +what build system to use. pkgme leaves all the steps of actually building the package, knowing
> +what commands to run to build, what files to put where, up to debhelper. debhelper in turn

"...what files to put where, etc., up to..."

> +delegates some of this work to its buildsystems, which are analogous to pkgme's backends.
> +
> +Therefore pkgme needs to know which buildsystem to instruct debhelper to use, and for this it
> +will query the backend. This means th...

=== modified file 'README.txt'
--- README.txt	2010-12-08 02:00:29 +0000
+++ README.txt	2010-12-15 02:19:09 +0000
@@ -1,20 +1,3 @@
-A Debian packaging generation framework.
-
-..
-    This program is free software: you can redistribute it and/or modify
-    it under the terms of the GNU General Public License as published by
-    the Free Software Foundation, either version 3 of the License, or
-    (at your option) any later version.
-
-    This program is distributed in the hope that it will be useful,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU General Public License for more details.
-
-    You should have received a copy of the GNU General Public License
-    along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-
 =====
 pkgme
 =====
@@ -89,3 +72,18 @@
     :glob:
 
     *
+..
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+

=== modified file 'conf.py'
--- conf.py	2010-11-10 22:42:51 +0000
+++ conf.py	2010-12-15 02:19:09 +0000
@@ -22,7 +22,7 @@
 
 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sphinx.ext.autodoc']
+extensions = ['sphinx.ext.autodoc', 'pkgme.rstinfo']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['../../_templates']
@@ -34,7 +34,7 @@
 #source_encoding = 'utf-8'
 
 # The master toctree document.
-master_doc = 'README'
+master_doc = 'doc/index'
 
 # General information about the project.
 project = u'pkgme'

=== added directory 'doc'
=== added directory 'doc/authors'
=== added file 'doc/authors/index.txt'
--- doc/authors/index.txt	1970-01-01 00:00:00 +0000
+++ doc/authors/index.txt	2010-12-15 02:19:09 +0000
@@ -0,0 +1,10 @@
+=====
+pkgme
+=====
+
+Welcome to pkgme, a tool that will help you package your application for Debian/Ubuntu.
+
+pkgme takes care of creating the files needed for packaging, by inspecting your
+code. As long as your project follows the conventions of one of the languages/project
+types that pkgme supports pkgme will be able to generate working packaging without
+you having to deal with any of the details.

=== added directory 'doc/backends'
=== added file 'doc/backends/index.txt'
--- doc/backends/index.txt	1970-01-01 00:00:00 +0000
+++ doc/backends/index.txt	2010-12-15 02:19:09 +0000
@@ -0,0 +1,109 @@
+==============
+pkgme backends
+==============
+
+pkgme backends provide the project specific information that is needed to generate
+the packaging. They examine the project in question and provide the information to
+the best of their ability.
+
+Each backend deals with a particular language, or project type, for instance their
+could be backends for Vala, Ruby on Rails, Python Django, C with autotools, GNOME,
+KDE, CMake, etc.
+
+How they work
+-------------
+
+pkgme itself is agnostic about the type of project that it is working on, but knows
+how to put information about a project in to the packaging files such that result
+should work, assuming that the information provided is good enough.
+
+Every time it needs a bit of information about the project, such as its name, or
+the dependencies it requires, it asks the backend that is the best match for the
+project.
+
+In this manner, the backends, and so you as a backend author, don't need to know
+much about packaging, and pkgme doesn't need to know about every different sort of
+project that there is.
+
+Selecting a backend
+-------------------
+
+When pkgme is asked to create the packaging for a project, it firsts selects the
+backend that is most suited to the task. To do this it asks each backend to report
+a score of how well it thinks that it can handle that project.
+
+A Ruby on Rails backend would presumably not be able to handle a Python project
+very well, so if asked to report a score for a Python project it should return
+"0".
+
+However, if it is asked about a Ruby on Rails project then it should report a higher
+number, the question is just how high. Consider the fact that there may be a
+generic Ruby backend, which would presumably be able to give some useful information
+about this project. However, the Ruby backend shouldn't have to detect that this is a
+Rails project, and know that there is a Rails backend, and so answer "0", as that would
+be very brittle. In addition, there may be a backend that is specifically for
+Ruby on Rails, using particular conventions and addons, which would want to out-score
+the generic Rails backend when appropriate.
+
+Assigning all the scores up-front isn't a good idea though, so we use the following
+guildelines:
+
+  * 0 - no information can be provided about the project (e.g. a Ruby backend with a Python project).
+  * 10 - some information can be provided, but the backend is generic (e.g. Ruby backend).
+  * 20 - some information can be provided, and the backend is more generic than just language (e.g. Ruby on Rails backend).
+  * 30 - some information can be provided, and the backend is highly specialised.
+
+When there are many backends that are closely related we can assign scores more carefully
+to those backend, but this scheme will suffice for now.
+
+Getting the info
+----------------
+
+When pkgme needs to get a bit of information about the project, such as its name, or
+the dependencies required to build, it asks the backend that it selected. The backend
+is expected to do its best to provide that information, but not to provide it if it can't
+do so, otherwise the generated packaging wouldn't work.
+
+The backend can use any means it likes to get the information, parsing files in
+conventional locations, scanning the tree.
+
+Interface with packaging
+------------------------
+
+There are a couple of places where the backend needs to know something related to packaging.
+
+The first of these is in the specification of the build dependencies, where the backend
+has to provide the list of packages that are needed. While the backend probably has
+available the list of modules used or similar, it wouldn't be possible for pkgme to
+translate these to a list of packages while remaining generic.
+
+Therefore one of the things the backend must be able to do is to go from the list of
+modules to the list of packages that provide them, using whatever means is appropriate.
+
+The second area where the backend needs to know something related to packaging is specifying
+what build system to use. pkgme leaves all the steps of actually building the package, knowing
+what commands to run to build, what files to put where, up to debhelper. debhelper in turn
+delegates some of this work to its buildsystems, which are analogous to pkgme's backends.
+
+Therefore pkgme needs to know which buildsystem to instruct debhelper to use, and for this it
+will query the backend. This means that the backend should know which debhelper buildsystem
+is appropriate for the project. If you are implementing a backend for which there isn't yet
+a debhelper buildsystem, you will also have to implement that in order to enjoy completely
+automated packaging.
+
+.. FIXME: reference for the currently available debhelper backends.
+.. FIXME: reference for implementing a debhelper buildsystem.
+
+Implementing a backend
+----------------------
+
+.. FIXME: Explain how to implement a backend
+
+
+Information that pkgme may request
+----------------------------------
+
+.. This outputs information on the info that a backend can provide, based
+   on introspecting the code. See pkgme.rstinfo for the code that does
+   this.
+.. pkgme_info_elements::

=== added file 'doc/index.txt'
--- doc/index.txt	1970-01-01 00:00:00 +0000
+++ doc/index.txt	2010-12-15 02:19:09 +0000
@@ -0,0 +1,12 @@
+=====
+pkgme
+=====
+
+pkgme is a tool to automate the creation of Debian packaging.
+
+.. toctree::
+   :maxdepth: 1
+
+   pkgme for application authors <authors/index>
+   Adding support for your project type to pkgme <backends/index>
+   The pkgme README </README>

=== modified file 'pkgme/info_elements.py'
--- pkgme/info_elements.py	2010-12-13 01:44:04 +0000
+++ pkgme/info_elements.py	2010-12-15 02:19:09 +0000
@@ -43,7 +43,7 @@
 class PackageName(InfoElement):
 
     name = "package_name"
-    description = "The nane of the package."
+    description = "The name of the package."
     required = True
 
 
@@ -59,7 +59,10 @@
     name = "maintainer"
     description = """The maintainer of the package.
 
-    Should be in the form Name <email>.
+    Should be in the form::
+
+        Name <email>
+
     """
     required = True
 
@@ -84,11 +87,16 @@
     name = "architecture"
     description = """The architecture(s) to build the binary package for.
 
-    This can be:
-        - any: to build on every architecture (arch-dependent)
-        - all: to build once and use the result on all architecture
-          (arch-independent)
-        - a list of architectures (arch-dependent)
+    This can be one of the following.
+
+    any
+        to build on every architecture (arch-dependent).
+    all
+        to build once and use the result on all architecture
+        (arch-independent).
+    a list of architectures (arch-dependent)
+        to build on specific architectures only.
+
     """
     required = True
 

=== added file 'pkgme/rstinfo.py'
--- pkgme/rstinfo.py	1970-01-01 00:00:00 +0000
+++ pkgme/rstinfo.py	2010-12-15 02:19:09 +0000
@@ -0,0 +1,69 @@
+from docutils import nodes
+from docutils.statemachine import ViewList
+from sphinx.util.compat import Directive
+from sphinx.util.docstrings import prepare_docstring
+
+from pkgme.package_files import default_package_file_group
+
+
+class PkgmeInfoElementsDirective(Directive):
+
+    def get_elements(self):
+        return default_package_file_group.get_elements()
+
+    def split_required(self, elements):
+        required = set()
+        not_required = set()
+        for element in elements:
+            if element.required:
+                required.add(element)
+            else:
+                not_required.add(element)
+        return required, not_required
+
+    def sort_elements(self, elements):
+        new_elements = list(elements)
+        new_elements.sort(key=lambda x: x.name)
+        return new_elements
+
+    def get_node_for_element(self, element):
+        item = nodes.definition_list_item()
+        term = nodes.term()
+        term += nodes.Text(element.name)
+        definition = nodes.definition()
+        description = ViewList()
+        for i, line in enumerate(prepare_docstring(element.description)):
+            description.append(
+                line,
+                'description attribute of %s info element' % element.name,
+                i)
+        self.state.nested_parse(description, 0, definition)
+        if element.default is not None:
+            para = nodes.paragraph()
+            definition += para
+            para += nodes.Text("The default value is ")
+            para += nodes.literal(element.default, element.default)
+        item += term
+        item += definition
+        return item
+
+    def build_definition_list(self, elements):
+        node = nodes.definition_list()
+        for element in self.sort_elements(elements):
+            node += self.get_node_for_element(element)
+        return node
+
+    def run(self):
+        elements = self.get_elements()
+        required, not_required = self.split_required(elements)
+        node = nodes.paragraph()
+        node.document = self.state.document
+        node += nodes.Text("The following elements are required, and must be provided if asked for to produce any packaging.")
+        node += self.build_definition_list(required)
+        node += nodes.Text("The following elements are optional, and do not have to be provided to produce packaging.")
+        node += self.build_definition_list(not_required)
+        return [node]
+
+
+def setup(app):
+    app.add_directive('pkgme_info_elements', PkgmeInfoElementsDirective)