Merge ~powersj/cloud-init:enable-pylint into cloud-init:master

[Joshua Powers]

test: enable pylint support in tox

Why enable pylint?
pylint has the ability to catch errors that pyflakes/flake8 simply wont. pylint sets the bar higher for what can get accepted in terms of quality.

This set of commits enables pylint to run without errors. This does not add it to the list of tests when running tox as I think that is something up for discussion. Please review the configuration and the changes to the code.

There are a number of legit errors that I have updated the code to avoid those errors. However, there are two places where a variable was changes from being instantiated as None to {}. This is due to pylint stating how a None object cannot be check for in (e.g. 'my_key' in variable, if variable is None). I really, really do not want to be breaking some existing infrastructure.

pylint output:
https://paste.ubuntu.com/24224371/

tox clean:
https://paste.ubuntu.com/24224366/

Integration test using Xenial:
https://paste.ubuntu.com/24224456/

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:faf81cf7e185c44c1e2f5638e00c4a132f5b1a40
https://jenkins.ubuntu.com/server/job/cloud-init-ci/151/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/151
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/151
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/151
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/151
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/151

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/151/rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:2393e5abf47eca5a8f58e52ac17d7a37427041df
https://jenkins.ubuntu.com/server/job/cloud-init-ci/155/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/155
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/155
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/155
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/155
    SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/155

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/155/rebuild

review: Approve (continuous-integration)

diff --git a/.pylintrc b/.pylintrc
new file mode 100644
index 0000000..783c1bb
--- /dev/null
+++ b/.pylintrc
@@ -0,0 +1,39 @@
+[MASTER]
+
+# --go-faster, use multiple processes to speed up Pylint
+jobs=4
+
+
+[MESSAGES CONTROL]
+
+# Errors only
+disable=C, F, I, R, W
+
+
+[REPORTS]
+
+# Set the output format. Available formats are text, parseable, colorized, msvs
+output-format=colorized
+
+# Just the errors please, no full report
+reports=no
+
+
+[TYPECHECK]
+
+# List of module names for which member attributes should not be checked
+# (useful for modules/projects where namespaces are manipulated during runtime
+# and thus existing member attributes cannot be deduced by static analysis. It
+# supports qualified module names, as well as Unix pattern matching.
+ignored-modules=six.moves,pkg_resources
+
+# List of class names for which member attributes should not be checked (useful
+# for classes with dynamically set attributes). This supports the use of
+# qualified names.
+ignored-classes=optparse.Values,thread._local,_thread._local,Renderer
+
+# List of members which are set dynamically and missed by pylint inference
+# system, and so shouldn't trigger E1101 when accessed. Python regular
+# expressions are accepted.
+generated-members=types,http.client,command_handlers
+
diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py
index 701aaa4..692b600 100644
--- a/cloudinit/net/network_state.py
+++ b/cloudinit/net/network_state.py
@@ -214,7 +214,7 @@ class NetworkStateInterpreter(object):
         return util.yaml_dumps(self._network_state)
 
     def as_dict(self):
-        return {'version': self.version, 'config': self.config}
+        return {'version': self._version, 'config': self._config}
 
     def get_network_state(self):
         ns = self.network_state
@@ -611,7 +611,8 @@ class NetworkStateInterpreter(object):
             self.handle_vlan(vlan_cmd)
 
     def handle_wifis(self, command):
-        raise NotImplemented('NetworkState V2: Skipping wifi configuration')
+        raise NotImplementedError("NetworkState V2: "
+                                  "Skipping wifi configuration")
 
     def _v2_common(self, cfg):
         LOG.debug('v2_common: handling config:\n%s', cfg)
diff --git a/cloudinit/sources/DataSourceAltCloud.py b/cloudinit/sources/DataSourceAltCloud.py
index c2b0eac..8528fa1 100644
--- a/cloudinit/sources/DataSourceAltCloud.py
+++ b/cloudinit/sources/DataSourceAltCloud.py
@@ -201,8 +201,7 @@ class DataSourceAltCloud(sources.DataSource):
             util.logexc(LOG, 'Failed command: %s\n%s', ' '.join(cmd), _err)
             return False
         except OSError as _err:
-            util.logexc(LOG, 'Failed command: %s\n%s', ' '.join(cmd),
-                        _err.message)
+            util.logexc(LOG, 'Failed command: %s\n%s', ' '.join(cmd), _err)
             return False
 
         try:
diff --git a/cloudinit/sources/DataSourceOpenNebula.py b/cloudinit/sources/DataSourceOpenNebula.py
index 1f1baf4..cd75e6e 100644
--- a/cloudinit/sources/DataSourceOpenNebula.py
+++ b/cloudinit/sources/DataSourceOpenNebula.py
@@ -286,12 +286,12 @@ def parse_shell_config(content, keylist=None, bash=None, asuser=None,
     output = output[0:-1]  # remove trailing null
 
     # go through output.  First _start_ is for 'preset', second for 'target'.
-    # Add to target only things were changed and not in volitile
+    # Add to ret only things were changed and not in excluded.
     for line in output.split("\x00"):
         try:
             (key, val) = line.split("=", 1)
             if target is preset:
-                target[key] = val
+                preset[key] = val
             elif (key not in excluded and
                   (key in keylist_in or preset.get(key) != val)):
                 ret[key] = val
diff --git a/cloudinit/sources/__init__.py b/cloudinit/sources/__init__.py
index 1829450..5c99437 100644
--- a/cloudinit/sources/__init__.py
+++ b/cloudinit/sources/__init__.py
@@ -50,7 +50,7 @@ class DataSource(object):
         self.distro = distro
         self.paths = paths
         self.userdata = None
-        self.metadata = None
+        self.metadata = {}
         self.userdata_raw = None
         self.vendordata = None
         self.vendordata_raw = None
diff --git a/cloudinit/url_helper.py b/cloudinit/url_helper.py
index 312b046..2f6a158 100644
--- a/cloudinit/url_helper.py
+++ b/cloudinit/url_helper.py
@@ -45,7 +45,7 @@ try:
     from distutils.version import LooseVersion
     import pkg_resources
     _REQ = pkg_resources.get_distribution('requests')
-    _REQ_VER = LooseVersion(_REQ.version)
+    _REQ_VER = LooseVersion(_REQ.version)  # pylint: disable=no-member
     if _REQ_VER >= LooseVersion('0.8.8'):
         SSL_ENABLED = True
     if _REQ_VER >= LooseVersion('0.7.0') and _REQ_VER < LooseVersion('1.0.0'):
diff --git a/tox.ini b/tox.ini
index f016f20..bf9046a 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,5 @@
 [tox]
-envlist = py27, py3, flake8, xenial
+envlist = py27, py3, flake8, xenial, pylint
 recreate = True
 
 [testenv]
@@ -17,6 +17,10 @@ commands = {envpython} -m flake8 {posargs:cloudinit/ tests/ tools/}
 setenv =
     LC_ALL = en_US.utf-8
 
+[testenv:pylint]
+deps = pylint==1.6.5
+commands = {envpython} -m pylint {posargs:cloudinit}
+
 [testenv:py3]
 basepython = python3
 commands = {envpython} -m nose {posargs:--with-coverage \
@@ -88,6 +92,10 @@ deps = pycodestyle
 commands = {envpython} -m pyflakes {posargs:cloudinit/ tests/ tools/}
 deps = pyflakes
 
+[testenv:tip-pylint]
+commands = {envpython} -m pylint {posargs:cloudinit}
+deps = pylint
+
 [testenv:citest]
 basepython = python3
 commands = {envpython} -m tests.cloud_tests {posargs}