Merge lp:~dpniel/ubuntu-autopilot-tests/add_pyflakes_pep8 into lp:ubuntu-autopilot-tests

[Dan Chapman ]

Adds pyflakes and pep8 checks to debian rules for use before mergeing. Lets get some standards involved :-D

[Nicholas Skaggs (nskaggs)]

So this is intended to be manually run as part of reviews; and folks submitting merges can run beforehand to check for themselves?

[Dan Chapman  (dpniel)]

Yes indeed, i noticed jackson did all the pep-8 errors a while back but pyflakes were left untouched. So it made me think we could do with something we could manually run when doing merges. I'm not sure having a 'tests' directory in the root seems a bit out of place and confusing for someone looking at the branch. What's your thoughts?

[Nicholas Skaggs (nskaggs)]

What if we simply renamed the folder to specify what it is? Something like checkformat or ? I suppose anything but tests. If so, I'm happy to merge this. Also should we just run flake8?

[Nicholas Skaggs (nskaggs)]

review: Needs Information

[Dan Chapman  (dpniel)]

I suppose we could just run flake8 but I have no idea how to configure it, for if we need to exclude certain pyflake errors etc. I'll go take a look through the manual see how it all works

[lp:~dpniel/ubuntu-autopilot-tests/add_pyflakes_pep8]

65. By Dan Chapman  on 2014-02-27

Changed folder name as suggested

[Dan Chapman  (dpniel)]

folder named changed so good to go now :-)

[Nicholas Skaggs (nskaggs)]

review: Approve

=== added directory 'checkformat'
=== added file 'checkformat/pyflakes.exclude'
=== added file 'checkformat/run-pep8'
--- checkformat/run-pep8	1970-01-01 00:00:00 +0000
+++ checkformat/run-pep8	2014-02-27 16:29:56 +0000
@@ -0,0 +1,6 @@
+#! /bin/sh
+#
+# Test PEP-8 conformance.
+
+pep8 ubuntu_autopilot_tests \
+     xubuntu_autopilot_tests

=== added file 'checkformat/run-pyflakes'
--- checkformat/run-pyflakes	1970-01-01 00:00:00 +0000
+++ checkformat/run-pyflakes	2014-02-27 16:29:56 +0000
@@ -0,0 +1,74 @@
+#!/usr/bin/env python
+#
+# Utility script to run pyflakes with the modules we care about and
+# exclude errors we know to be fine.
+
+# Taken from Review Board, MIT license
+
+import os
+import re
+import subprocess
+import sys
+
+
+module_exclusions = [
+    'debian',
+    
+]
+
+whitelist = [
+    
+    'tests/run-pyflakes',
+]
+
+
+def scan_for_modules():
+    return [entry
+            for entry in os.listdir(os.getcwd())
+            if ((os.path.isdir(entry) or entry.endswith(".py")) and
+                entry not in module_exclusions)] + whitelist
+
+
+def main():
+    cur_dir = os.path.dirname(__file__)
+    os.chdir(os.path.join(cur_dir, ".."))
+    modules = sys.argv[1:]
+
+    if not modules:
+        # The user didn't specify anything specific. Scan for modules.
+        modules = scan_for_modules()
+
+    os.environ['PYFLAKES_NODOCTEST'] = '1'
+    p = subprocess.Popen(['pyflakes3'] + modules,
+                         stderr=subprocess.PIPE,
+                         stdout=subprocess.PIPE,
+                         close_fds=True, universal_newlines=True)
+
+    contents = p.communicate()[0].splitlines()
+
+    # Read in the exclusions file
+    exclusions = {}
+    with open(os.path.join(cur_dir, "pyflakes.exclude"), "r") as fp:
+        for line in fp.readlines():
+            if not line.startswith("#"):
+                exclusions[line.rstrip()] = 1
+
+    # Now filter thin
+    error = False
+    for line in contents:
+        if line.startswith('#'):
+            continue
+
+        line = line.rstrip()
+        test_line = re.sub(r':[0-9]+:', r':*:', line, 1)
+        test_line = re.sub(r'line [0-9]+', r'line *', test_line)
+
+        if test_line not in exclusions:
+            print line
+            error = True
+    if error:
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

=== modified file 'debian/rules'
--- debian/rules	2013-10-28 19:29:54 +0000
+++ debian/rules	2014-02-27 16:29:56 +0000
@@ -10,4 +10,17 @@
 #export DH_VERBOSE=1
 
 %:
-	dh $@ --with python2
+	dh $@ --with python3
+
+check:
+	# Sanity-check before upload.
+	find -name debian -prune -o -name \*.py -print | xargs py3compile
+	find -type f \( -name \*.pyc -o -name \*.pyo \) -print0 | xargs -0r rm -f
+	find -name __pycache__ -print0 | xargs -0r rm -rf
+	# Check the syntax of any shell scripts.
+	set -e; for x in $$(find -type f \! -name \*.po \! -name \*.pot -print0 | xargs -0 file -i | grep "text/x-shellscript" | cut -d':' -f1); do \
+		sh -n $$x; \
+	done
+	# Check the syntax of any Python scripts.
+	./checkformat/run-pyflakes
+	./checkformat/run-pep8