Merge lp:~mvo/snappy/snappy-docs-mdlint into lp:~snappy-dev/snappy/snappy-moved-to-github

[Michael Vogt]

Add simple markdown lint to ensure the tables are correctly rendered and run as part of "run-checks.sh"

This is mostly a RFC, this branch may be overkill and too simplistic at the same time (how often do you have that!).

When looking at https://code.launchpad.net/~stephen-stewart/snappy/docs-tidy-up-package-metadata/+merge/259050 I was wondering if we should have a mdlint and run it as part of run-checks. This branch does that. The code that does the lint is very simplistic though.

[Sergio Schvezov (sergiusens)]

This looks good, I wonder if we settle with markdown(django) that maybe we should move the rst doc to md as well, in a different MP that is.

I guess there is no such thing as a markdown(django) linter, this should do for this case. I added a teeny comment though.

[Sergio Schvezov (sergiusens)]

review: Approve

[Michael Vogt (mvo)]

Thanks Sergio, I think this makes sense and I added exit code handling now.

[Snappy Tarmac (snappydevtarmac)]

There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.

=== modified file 'debian/control'
--- debian/control	2015-05-08 19:42:00 +0000
+++ debian/control	2015-05-19 14:28:51 +0000
@@ -14,7 +14,9 @@
                golang-gocheck-dev,
                golang-goconfigparser-dev,
                golang-pb-dev,
-               golang-yaml.v2-dev
+               golang-yaml.v2-dev,
+               python3,
+               python3-markdown
 Standards-Version: 3.9.6
 Homepage: https://launchpad.net/snappy
 Vcs-Browser: http://bazaar.launchpad.net/~snappy-dev/snappy/trunk/files

=== modified file 'docs/hashes.md'
--- docs/hashes.md	2015-03-26 20:35:57 +0000
+++ docs/hashes.md	2015-05-19 14:28:51 +0000
@@ -6,17 +6,17 @@
 ## Format
 
 The yaml looks like this:
- * archive-sha512: the hexdigest of the data.tar.gz
+* archive-sha512: the hexdigest of the data.tar.gz
 
 Then a list of files/directories/symlinks in the tar that are
 described with:
- * name: the full path in the archive
- * mode: First char is the type "f" for file, "d" for dir, "l" for
-         symlink. Then the unix permission bits in the form
-         rwxrwxrwx.
-         E.g. a file with mode 0644 is: "frw-r--r--"
- * size: (applies only to files)
- * sha512: (applies only to files) the hexdigest of the file content
+* name: the full path in the archive
+* mode: First char is the type "f" for file, "d" for dir, "l" for
+        symlink. Then the unix permission bits in the form
+        rwxrwxrwx.
+        E.g. a file with mode 0644 is: "frw-r--r--"
+* size: (applies only to files)
+* sha512: (applies only to files) the hexdigest of the file content
 
 ## Owner
 

=== modified file 'docs/meta.md'
--- docs/meta.md	2015-05-13 20:41:52 +0000
+++ docs/meta.md	2015-05-19 14:28:51 +0000
@@ -16,70 +16,70 @@
 This file describes the snap package and is the most important file
 for a snap package. The following keys are mandatory:
 
- * `name`: the name of the snap (only `[a-z0-9][a-z0-9+.-]`)
- * `version`: the version of the snap (only `[a-zA-Z0-9.+~-]` are allowed)
- * `vendor`: the vendor of the snap
+* `name`: the name of the snap (only `[a-z0-9][a-z0-9+.-]`)
+* `version`: the version of the snap (only `[a-zA-Z0-9.+~-]` are allowed)
+* `vendor`: the vendor of the snap
 
 The following keys are optional:
 
- * `icon`: a SVG icon for the snap that is displayed in the store
- * `explicit-license-agreement`: set to `Y` if the user needs to accept a
-   special `meta/license.txt` before the snap can be installed
- * `license-version`: a string that, when it changes and
-   `explicit-license-agreement` is `Y`, prompts the user to accept the
-   license again.
- * `type`: (optional) the type of the snap, can be:
-     * `app` - the default if empty
-     * `oem` - a special snap that OEMs can use to customize snappy for
-             their hardware
-     * `framework` - a specialized snap that extends the system that other
-                   snaps may use
-
- * `architectures`: (optional) a yaml list of supported architectures
-                    `["all"]` if empty
- * `frameworks`: a list of the frameworks the snap needs as dependencies
-
- * `services`: the servies (daemons) that the snap provides
-     * `name`: (required) name of the service (only `[a-zA-Z0-9+.-]`)
-     * `description`: (required) description of the service
-     * `start`: (required) the command to start the service
-     * `stop`: (optional) the command to stop the service
-     * `stop-timeout`: (optional) the time in seconds to wait for the
-                       service to stop
-     * `poststop`: a command that runs after the service has stopped
-     * `caps`: (optional) list of additional security policies to add.
-               See `security.md` for details
-     * `security-template`: (optional) alternate security template to use
-                            instead of `default`. See `security.md` for details
-     * `security-override`: (optional) high level overrides to use when
-                            `security-template` and `caps` are not
-                            sufficient.  See security.md for details
-     * `security-policy`: (optional) hand-crafted low-level raw security
-                          policy to use instead of using default
-                          template-based  security policy. See
-                          security.md for details
-     * `ports`: (optional) define what ports the service will work
-         * `internal`: the ports the service is going to connect to
-             * `tagname`: a free form name
-                 * `port`: (optional) number/protocol, e.g. `80/tcp`
-                 * `negotiable`: (optional) Y if the app can use a different port
-         * `external`: the ports the service offer to the world
-             * `tagname`: a free form name, some names have meaning like "ui"
-                 * `port`: (optional) see above
-                 * `negotiable`: (optional) see above
-     * `bus-name`: (optional) message bus connection name for the service.
-       May only be specified for snaps of 'type: framework' (see above). See
-       frameworks.md for details.
-
- * `binaries`: the binaries (executables) that the snap provides
-     * `name`: (required) the name of the binary, the user will be able to
-               call it as $name.$pkgname (only `[a-zA-Z0-9+.-]`)
-     * `exec`: the program that gets executed (can be omited if name points
-               to a binary already)
-     * `caps`: (optional) see entry in `services` (above)
-     * `security-template`: (optional) see entry in `services` (above)
-     * `security-override`: (optional) see entry in `services` (above)
-     * `security-policy`: (optional) see entry in `services` (above)
+* `icon`: a SVG icon for the snap that is displayed in the store
+* `explicit-license-agreement`: set to `Y` if the user needs to accept a
+  special `meta/license.txt` before the snap can be installed
+* `license-version`: a string that, when it changes and
+  `explicit-license-agreement` is `Y`, prompts the user to accept the
+  license again.
+* `type`: (optional) the type of the snap, can be:
+    * `app` - the default if empty
+    * `oem` - a special snap that OEMs can use to customize snappy for
+            their hardware
+    * `framework` - a specialized snap that extends the system that other
+                  snaps may use
+
+* `architectures`: (optional) a yaml list of supported architectures
+                   `["all"]` if empty
+* `frameworks`: a list of the frameworks the snap needs as dependencies
+
+* `services`: the servies (daemons) that the snap provides
+    * `name`: (required) name of the service (only `[a-zA-Z0-9+.-]`)
+    * `description`: (required) description of the service
+    * `start`: (required) the command to start the service
+    * `stop`: (optional) the command to stop the service
+    * `stop-timeout`: (optional) the time in seconds to wait for the
+                      service to stop
+    * `poststop`: a command that runs after the service has stopped
+    * `caps`: (optional) list of additional security policies to add.
+              See `security.md` for details
+    * `security-template`: (optional) alternate security template to use
+                           instead of `default`. See `security.md` for details
+    * `security-override`: (optional) high level overrides to use when
+                           `security-template` and `caps` are not
+                           sufficient.  See security.md for details
+    * `security-policy`: (optional) hand-crafted low-level raw security
+                         policy to use instead of using default
+                         template-based  security policy. See
+                         security.md for details
+    * `ports`: (optional) define what ports the service will work
+        * `internal`: the ports the service is going to connect to
+            * `tagname`: a free form name
+                * `port`: (optional) number/protocol, e.g. `80/tcp`
+                * `negotiable`: (optional) Y if the app can use a different port
+        * `external`: the ports the service offer to the world
+            * `tagname`: a free form name, some names have meaning like "ui"
+                * `port`: (optional) see above
+                * `negotiable`: (optional) see above
+    * `bus-name`: (optional) message bus connection name for the service.
+      May only be specified for snaps of 'type: framework' (see above). See
+      frameworks.md for details.
+
+* `binaries`: the binaries (executables) that the snap provides
+    * `name`: (required) the name of the binary, the user will be able to
+              call it as $name.$pkgname (only `[a-zA-Z0-9+.-]`)
+    * `exec`: the program that gets executed (can be omited if name points
+              to a binary already)
+    * `caps`: (optional) see entry in `services` (above)
+    * `security-template`: (optional) see entry in `services` (above)
+    * `security-override`: (optional) see entry in `services` (above)
+    * `security-policy`: (optional) see entry in `services` (above)
 
 ## license.txt
 

=== modified file 'docs/package-names.md'
--- docs/package-names.md	2015-04-15 22:03:57 +0000
+++ docs/package-names.md	2015-05-19 14:28:51 +0000
@@ -135,4 +135,4 @@
 becomes the gaming engine, you will need to be pushed out to a different name.
 
 ## Open questions
- * Must finalize what the `APP_ID` is going to be (composed? no devname? etc)
+* Must finalize what the `APP_ID` is going to be (composed? no devname? etc)

=== modified file 'docs/security.md'
--- docs/security.md	2015-05-12 15:32:03 +0000
+++ docs/security.md	2015-05-19 14:28:51 +0000
@@ -81,14 +81,14 @@
   Specify `caps: []` to indicate no additional `caps`.  When `caps` and
   `security-template` are not specified, `caps` defaults to client networking.
   Not compatible with `security-override` or `security-policy`.
- * AppArmor access is deny by default and apps are restricted to their
-   app-specific directories, libraries, etc (enforcing ro, rw, etc).
-   Additional access beyond what is allowed by the declared `security-template`
-   is declared via this option
- * seccomp is deny by default. Enough safe syscalls are allowed so that apps
-   using the declared `security-template` should work. Additional access
-   beyond what is allowed by the `security-template` is declared via this
-   option
+    * AppArmor access is deny by default and apps are restricted to
+      their app-specific directories, libraries, etc (enforcing ro,
+      rw, etc).  Additional access beyond what is allowed by the
+      declared `security-template` is declared via this option
+    * seccomp is deny by default. Enough safe syscalls are allowed so
+      that apps using the declared `security-template` should
+      work. Additional access beyond what is allowed by the
+      `security-template` is declared via this option
 * `security-template`: (optional) alternate security template to use instead of
   `default`. When specified without `caps`, `caps` defaults to being empty. Not
   compatible with `security-override` or `security-policy`.
@@ -97,13 +97,13 @@
   [advanced usage](https://wiki.ubuntu.com/SecurityTeam/Specifications/SnappyConfinement)
   for details. Not compatible with `caps`, `security-template` or
   `security-policy`
- * `apparmor: path/to/security.override`
- * `seccomp: path/to/filter.override`
+    * `apparmor: path/to/security.override`
+    * `seccomp: path/to/filter.override`
 * `security-policy`: (optional) hand-crafted low-level raw security policy to
   use instead of using default template-based security policy. Not compatible
   with `caps`, `security-template` or `security-override`.
- * `apparmor: path/to/profile`
- * `seccomp: path/to/filter`
+    * `apparmor: path/to/profile`
+    * `seccomp: path/to/filter`
 
 Eg, consider the following:
 
@@ -199,22 +199,22 @@
 The following is planned:
 
 * launcher:
- * utilize syscall argument filtering
- * setup additional cgroups (tag network traffic, memory)
- * setup iptables using cgroup tags (for internal app access)
- * drop privileges to uid of service
+    * utilize syscall argument filtering
+    * setup additional cgroups (tag network traffic, memory)
+    * setup iptables using cgroup tags (for internal app access)
+    * drop privileges to uid of service
 * fine-grained network mediation via AppArmor
 * `sockets`: (optional) `AF_UNIX` abstract socket definition for coordinated
   snap communications. Abstract sockets will be namespaced and yaml is such
   that (client) apps wanting to use the socket don't have to declare anything
   extra, but they don't have access unless the (server) binary declaring the
   socket says that app is ok).
- * `names`: (optional) list of abstract socket names (`<name>_<binaryname>` is
-   prepended)
- * `allowed-clients`: `<name>.<namespace>` or `<name>.<namespace>_<binaryname>`
-   (ie, omit version and `binaryname` to allow all from snap
-   `<name>.<namespace>` or omit version to allow only `binaryname` from snap
-   `<name>`)
+    * `names`: (optional) list of abstract socket names
+      (`<name>_<binaryname>` is prepended)
+    * `allowed-clients`: `<name>.<namespace>` or
+     `<name>.<namespace>_<binaryname>` (ie, omit version and
+     `binaryname` to allow all from snap `<name>.<namespace>` or omit
+     version to allow only `binaryname` from snap `<name>`)
 
  Eg:
 

=== added file 'mdlint.py'
--- mdlint.py	1970-01-01 00:00:00 +0000
+++ mdlint.py	2015-05-19 14:28:51 +0000
@@ -0,0 +1,36 @@
+#!/usr/bin/python
+#
+# see http://daringfireball.net/projects/markdown/syntax
+# for the "canonical" reference
+#
+# We support django-markdown which uses python-markdown, see:
+# http://pythonhosted.org/Markdown/
+
+import sys
+
+
+def lint_li(fname, text):
+    """Ensure that the list-items are multiplies of 4"""
+    is_clean = True
+    for i, line in enumerate(text.splitlines()):
+        if line.lstrip().startswith("*") and line.index("*") % 4 != 0:
+            print("%s: line %i list has non-4 spaces indent" % (fname, i))
+            is_clean = False
+    return is_clean
+
+
+def lint(md_files):
+    """lint all md files"""
+    all_clean = True
+    for md in md_files:
+        with open(md) as f:
+            buf = f.read()
+            for fname, func in globals().items():
+                if fname.startswith("lint_"):
+                    all_clean &= func(md, buf)
+    return all_clean
+
+
+if __name__ == "__main__":
+    if not lint(sys.argv):
+        sys.exit(1)

=== modified file 'run-checks'
--- run-checks	2015-03-27 18:35:24 +0000
+++ run-checks	2015-05-19 14:28:51 +0000
@@ -8,6 +8,9 @@
     goctest="go test"
 fi
 
+echo Checking docs
+./mdlint.py docs/*.md
+
 echo Checking formatting
 fmt=$(gofmt -l .)