Merge lp:~harlowja/cloud-init/cloud-init-dynamic-distro-check into lp:~cloud-init-dev/cloud-init/trunk

[Joshua Harlow]

[lp:~harlowja/cloud-init/cloud-init-dynamic-distro-check]

1260. By Joshua Harlow on 2016-07-21

Use a more advanced module 'working' function
that by default performs the same distro checking
behavior (but allows modules to provide there
own if they so desire to).

[Joshua Harlow (harlowja)]

Yup, I updated it to try that approach instead; seems like its better IMHO.

[lp:~harlowja/cloud-init/cloud-init-dynamic-distro-check]

1261. By Joshua Harlow on 2016-07-22

Rearrange some of the pre_handling code.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:1261
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-dynamic-distro-check/+merge/300805/+edit-commit-message

https://server-team-jenkins.canonical.com/job/cloud-init-ci/36/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/lp-vote-on-merge/10/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/cloud-init-ci/36/rebuild

review: Needs Fixing (continuous-integration)

[Scott Moser (smoser)]

Hello,
Thank you for taking the time to contribute to cloud-init. Cloud-init has moved its revision control system to git. As a result, we are marking all bzr merge proposals as 'rejected'. If you would like to re-submit this proposal for review, please do so by following the current HACKING documentation at http://cloudinit.readthedocs.io/en/latest/topics/hacking.html .

Unmerged revisions

1261. By Joshua Harlow on 2016-07-22

Rearrange some of the pre_handling code.

1260. By Joshua Harlow on 2016-07-21

Use a more advanced module 'working' function
that by default performs the same distro checking
behavior (but allows modules to provide there
own if they so desire to).

1259. By Joshua Harlow on 2016-07-21

Allow modules to provide a 'is_usable_on' function

This function, if it exists will allow for the module
to decide if it works on the running distro, making it
a little more flexible (vs the previous distro lists that
are more static) so that the module can itself figure out
if it should be operational or not.

=== added file 'cloudinit/exceptions.py'
--- cloudinit/exceptions.py	1970-01-01 00:00:00 +0000
+++ cloudinit/exceptions.py	2016-07-22 07:30:56 +0000
@@ -0,0 +1,28 @@
+# vi: ts=4 expandtab
+#
+#    Author: Joshua Harlow <[e-mail]>
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License version 3, as
+#    published by the Free Software Foundation.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+class ModuleNotSupported(Exception):
+    """Exception raised when a module states it is not supported.
+
+    Requires that when raised that (a textual) reason that explains why
+    the module believes it is not supported is provided so that this reason
+    can be relayed to users to provide some useful information.
+    """
+
+    def __init__(self, reason):
+        super(ModuleNotSupported, self).__init__(reason)
+        self.reason = reason
 
=== modified file 'cloudinit/stages.py'
--- cloudinit/stages.py	2016-06-10 21:22:17 +0000
+++ cloudinit/stages.py	2016-07-22 07:30:56 +0000
@@ -21,6 +21,7 @@
 #    along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import copy
+import functools
 import os
 import sys
 
@@ -40,6 +41,7 @@
 from cloudinit import cloud
 from cloudinit import config
 from cloudinit import distros
+from cloudinit import exceptions
 from cloudinit import helpers
 from cloudinit import importer
 from cloudinit import log as logging
@@ -808,31 +810,47 @@
     def run_section(self, section_name):
         raw_mods = self._read_modules(section_name)
         mostly_mods = self._fixup_modules(raw_mods)
-        d_name = self.init.distro.name
+        distro = self.init.distro
+
+        def _pre_handle(mod, name, cfg, distro, log):
+            worked_distros = set(mod.distros)
+            worked_distros.update(
+                distros.Distro.expand_osfamily(mod.osfamilies))
+            if len(worked_distros) == 0 or distro.name in worked_distros:
+                return
+            reason = ("Module '%s' is not supported on distribution %s"
+                      " as it has stated it only works correctly on"
+                      " %s distributions" % (name, distro.name,
+                                             sorted(worked_distros)))
+            raise exceptions.ModuleNotSupported(reason)
 
         skipped = []
         forced = []
         overridden = self.cfg.get('unverified_modules', [])
         for (mod, name, _freq, _args) in mostly_mods:
-            worked_distros = set(mod.distros)
-            worked_distros.update(
-                distros.Distro.expand_osfamily(mod.osfamilies))
-
-            # module does not declare 'distros' or lists this distro
-            if not worked_distros or d_name in worked_distros:
-                continue
-
             if name in overridden:
                 forced.append(name)
             else:
-                skipped.append(name)
-
-        if skipped:
-            LOG.info("Skipping modules %s because they are not verified "
-                     "on distro '%s'.  To run anyway, add them to "
-                     "'unverified_modules' in config.", skipped, d_name)
+                try:
+                    pre_handle_func = getattr(mod, 'pre_handle', None)
+                    if not six.callable(pre_handle_func):
+                        # Must be an older module, or something we can't
+                        # call into, so switch to the default that verifies
+                        # the basics...
+                        pre_handle_func = functools.partial(_pre_handle, mod)
+                    pre_handle_func(name, self.cfg, distro, config.LOG)
+                except exceptions.ModuleNotSupported as e:
+                    LOG.info(
+                        "Skipping module '%s' because %s. To run anyway,"
+                        " add this module name to 'unverified_modules'"
+                        " in config.", name, e.reason)
+                    skipped.append(name)
+                except Exception:
+                    LOG.exception("Failed at determining if module '%s'"
+                                  " can be correctly ran on this system,"
+                                  " assuming by default it can be ran.")
         if forced:
-            LOG.info("running unverified_modules: %s", forced)
+            LOG.info("Running unverified modules: %s", forced)
 
         return self._run_modules(mostly_mods)