Merge lp:~craigtracey/cloud-init/update-etc-hosts into lp:~cloud-init-dev/cloud-init/trunk

[Craig Tracey]

Fix broken cc_update_etc_hosts (LP: #1100036)

Right now, all distros but ubuntu will fail to manage /etc/hosts. This
is due to the fact that the templates are named:
- hosts.ubuntu.tmpl
- hosts.redhat.tmpl

The config handler is specifically looking for a template with the
given distro name.

This change addresses this issue and is contingent upon support of
'osfamilies' as implemented in LP: #1100029
(lp:~craigtracey/cloud-init/osfamilies)

[Joshua Harlow (harlowja)]

Does distro attribute 'self.osfamily = 'redhat'' need to be hard-coded?

The nice thing about having this come from yaml (the system_info section) is that it doesn't need to be (so can be configured to be different). Do u think this should happen? Seems useful?

review: Needs Fixing

[Craig Tracey (craigtracey)]

Hey Joshua,

As I just posed in #cloud-init (you were away), while I agreed with this and filed LP #1105494, after implementing it, I wasn't really sure how this would be used. I agree that distro should be flexible (not necessarily come via introspection) and defined in yaml. I also agree that distro should handle the case where a distro that is not explicitly supported could piggyback on another (ie. centos defines distro as rhel).

However, I could not fathom a case where you would want to alter osfamily to be something different than what the distro defined its osfamily as (ie. distro: rhel, osfamily: debian). Do you have a use case in mind? I am certainly not against implementing this, I just want to be sure I understand how this could be used.

Thanks for all the reviews - much appreciated!
Craig

[Joshua Harlow (harlowja)]

All merged. Thanks for the fixes :-)

=== modified file 'cloudinit/config/__init__.py'
--- cloudinit/config/__init__.py	2012-06-29 18:20:34 +0000
+++ cloudinit/config/__init__.py	2013-01-15 21:43:21 +0000
@@ -52,5 +52,7 @@
         if freq and freq not in FREQUENCIES:
             LOG.warn("Module %s has an unknown frequency %s", mod, freq)
     if not hasattr(mod, 'distros'):
-        setattr(mod, 'distros', None)
+        setattr(mod, 'distros', [])
+    if not hasattr(mod, 'osfamilies'):
+        setattr(mod, 'osfamilies', [])
     return mod

=== modified file 'cloudinit/config/cc_update_etc_hosts.py'
--- cloudinit/config/cc_update_etc_hosts.py	2012-10-28 02:25:48 +0000
+++ cloudinit/config/cc_update_etc_hosts.py	2013-01-15 21:43:21 +0000
@@ -37,10 +37,11 @@
 
         # Render from a template file
         tpl_fn_name = cloud.get_template_filename("hosts.%s" %
-                                                  (cloud.distro.name))
+                                                  (cloud.distro.osfamily))
         if not tpl_fn_name:
             raise RuntimeError(("No hosts template could be"
-                                " found for distro %s") % (cloud.distro.name))
+                                " found for distro %s") %
+                                (cloud.distro.osfamily))
 
         templater.render_to_file(tpl_fn_name, '/etc/hosts',
                                 {'hostname': hostname, 'fqdn': fqdn})

=== modified file 'cloudinit/distros/__init__.py'
--- cloudinit/distros/__init__.py	2013-01-07 16:36:10 +0000
+++ cloudinit/distros/__init__.py	2013-01-15 21:43:21 +0000
@@ -35,6 +35,11 @@
 
 from cloudinit.distros.parsers import hosts
 
+OSFAMILIES = {
+    'debian': ['debian', 'ubuntu'],
+    'redhat': ['fedora', 'rhel']
+}
+
 LOG = logging.getLogger(__name__)
 
 
@@ -143,6 +148,16 @@
     def _select_hostname(self, hostname, fqdn):
         raise NotImplementedError()
 
+    @staticmethod
+    def expand_osfamily(family_list):
+        distros = []
+        for family in family_list:
+            if not family in OSFAMILIES:
+                raise ValueError("No distibutions found for osfamily %s"
+                                 % (family))
+            distros.extend(OSFAMILIES[family])
+        return distros
+
     def update_hostname(self, hostname, fqdn, prev_hostname_fn):
         applying_hostname = hostname
 

=== modified file 'cloudinit/distros/debian.py'
--- cloudinit/distros/debian.py	2012-11-13 23:24:53 +0000
+++ cloudinit/distros/debian.py	2013-01-15 21:43:21 +0000
@@ -48,6 +48,7 @@
         # calls from repeatly happening (when they
         # should only happen say once per instance...)
         self._runner = helpers.Runners(paths)
+        self.osfamily = 'debian'
 
     def apply_locale(self, locale, out_fn=None):
         if not out_fn:

=== modified file 'cloudinit/distros/rhel.py'
--- cloudinit/distros/rhel.py	2012-11-13 06:14:31 +0000
+++ cloudinit/distros/rhel.py	2013-01-15 21:43:21 +0000
@@ -60,6 +60,7 @@
         # calls from repeatly happening (when they
         # should only happen say once per instance...)
         self._runner = helpers.Runners(paths)
+        self.osfamily = 'redhat'
 
     def install_packages(self, pkglist):
         self.package_command('install', pkglist)

=== modified file 'cloudinit/stages.py'
--- cloudinit/stages.py	2012-12-17 13:41:11 +0000
+++ cloudinit/stages.py	2013-01-15 21:43:21 +0000
@@ -529,11 +529,16 @@
                     freq = mod.frequency
                 if not freq in FREQUENCIES:
                     freq = PER_INSTANCE
-                worked_distros = mod.distros
+
+                worked_distros = set(mod.distros)
+                worked_distros.update(
+                    distros.Distro.expand_osfamily(mod.osfamilies))
+
                 if (worked_distros and d_name not in worked_distros):
                     LOG.warn(("Module %s is verified on %s distros"
                               " but not on %s distro. It may or may not work"
-                              " correctly."), name, worked_distros, d_name)
+                              " correctly."), name, list(worked_distros),
+                              d_name)
                 # Use the configs logger and not our own
                 # TODO(harlowja): possibly check the module
                 # for having a LOG attr and just give it back

=== added file 'templates/hosts.debian.tmpl'
--- templates/hosts.debian.tmpl	1970-01-01 00:00:00 +0000
+++ templates/hosts.debian.tmpl	2013-01-15 21:43:21 +0000
@@ -0,0 +1,25 @@
+## This file (/etc/cloud/templates/hosts.tmpl) is only utilized
+## if enabled in cloud-config.  Specifically, in order to enable it
+## you need to add the following to config:
+##    manage_etc_hosts: True
+##
+## Note, double-hash commented lines will not appear in /etc/hosts
+# 
+# Your system has configured 'manage_etc_hosts' as True.
+# As a result, if you wish for changes to this file to persist
+# then you will need to either
+# a.) make changes to the master file in /etc/cloud/templates/hosts.tmpl
+# b.) change or remove the value of 'manage_etc_hosts' in
+#     /etc/cloud/cloud.cfg or cloud-config from user-data
+# 
+## The value '$hostname' will be replaced with the local-hostname
+127.0.1.1 $fqdn $hostname
+127.0.0.1 localhost
+
+# The following lines are desirable for IPv6 capable hosts
+::1 ip6-localhost ip6-loopback
+fe00::0 ip6-localnet
+ff00::0 ip6-mcastprefix
+ff02::1 ip6-allnodes
+ff02::2 ip6-allrouters
+ff02::3 ip6-allhosts

=== removed file 'templates/hosts.ubuntu.tmpl'
--- templates/hosts.ubuntu.tmpl	2012-07-09 20:41:45 +0000
+++ templates/hosts.ubuntu.tmpl	1970-01-01 00:00:00 +0000
@@ -1,25 +0,0 @@
-## This file (/etc/cloud/templates/hosts.tmpl) is only utilized
-## if enabled in cloud-config.  Specifically, in order to enable it
-## you need to add the following to config:
-##    manage_etc_hosts: True
-##
-## Note, double-hash commented lines will not appear in /etc/hosts
-# 
-# Your system has configured 'manage_etc_hosts' as True.
-# As a result, if you wish for changes to this file to persist
-# then you will need to either
-# a.) make changes to the master file in /etc/cloud/templates/hosts.tmpl
-# b.) change or remove the value of 'manage_etc_hosts' in
-#     /etc/cloud/cloud.cfg or cloud-config from user-data
-# 
-## The value '$hostname' will be replaced with the local-hostname
-127.0.1.1 $fqdn $hostname
-127.0.0.1 localhost
-
-# The following lines are desirable for IPv6 capable hosts
-::1 ip6-localhost ip6-loopback
-fe00::0 ip6-localnet
-ff00::0 ip6-mcastprefix
-ff02::1 ip6-allnodes
-ff02::2 ip6-allrouters
-ff02::3 ip6-allhosts