Merge ~sankaraditya/cloud-init:vmware-customize-utc-time into cloud-init:master

[Sankar Tanguturi]

[VMware Guest customization] Add UTC customization for guests.

In 'VMware Guest Customization' workflow, we need to set the
setting UTC=yes|no for debian guests in /etc/default/rcS file.

Added the support to read the UTC setting and make necessary
changes to /etc/default/rcS file.
Note: For now, only debian guests are customized. Will add the
support for other guests later.

Added an utility function which can be used by other modules.
Added few test cases.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:aa52f7754b96cadbec8468374bb6d79618d2b13e
https://jenkins.ubuntu.com/server/job/cloud-init-ci/376/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/376/rebuild

review: Approve (continuous-integration)

[Sankar Tanguturi (sankaraditya)]

Chad and Scott,

Can you please take a look and provide your valuable comments. This is not a high risk patch and should be simple and straight forward.

Thanks
Sankar.

[Sankar Tanguturi (sankaraditya)]

Chad and Scott,

Any update?

Thanks
Sankar.

[Chad Smith (chad.smith)]

Grabbing this today Sankar. Sorry for the delay

[Chad Smith (chad.smith)]

So, I'm a hestant on this branch because cloud-init already surfaces a cc_timezone module which allows users to set timezone. That module talks to <distro>.set_timezone which in turn calls <distro>._find_timezone_file.

Growing that functionality in vmware-specific modules seems to be duplicating the user-facing configuration that cloud-init timsurfaces. Why wouldn't vmware providie timzone: UTC as vendor data?

If you need something specific for debian, or other distros that isn't handled by the Distro class, I'd recommend extending the distro class specifically to get this functionality instead of having distro-specific logic down in cloudinit/sources/helpers/vmware/imc/guestcust_util.py.

[Scott Moser (smoser)]

marked this as work in progress. we'd like for this functionality (configuration what is stored int he hwclock) should be in common cloud-init.

Unmerged commits

aa52f77... by Sankar Tanguturi on 2017-09-29

  [VMware Guest customization] Add UTC customization for guests.

  In 'VMware Guest Customization' workflow, we need to set the
  settig UTC=yes|no for debian guests in /etc/default/rcS file.

  Added the support to read the UTC setting and make necessary
  changes to /etc/default/rcS file.
  Note: For now, only debian guests are customized. Will add the
  support for other guests later.

  Added an utility function which can be used by other modules.
  Added few test cases.

diff --git a/cloudinit/sources/DataSourceOVF.py b/cloudinit/sources/DataSourceOVF.py
index ccebf11..94a7683 100644
--- a/cloudinit/sources/DataSourceOVF.py
+++ b/cloudinit/sources/DataSourceOVF.py
@@ -34,6 +34,7 @@ from cloudinit.sources.helpers.vmware.imc.guestcust_event \
 from cloudinit.sources.helpers.vmware.imc.guestcust_state \
     import GuestCustStateEnum
 from cloudinit.sources.helpers.vmware.imc.guestcust_util import (
+    customize_utc,
     enable_nics,
     get_nics_to_enable,
     set_customization_status
@@ -142,6 +143,9 @@ class DataSourceOVF(sources.DataSource):
                     True,
                     True,
                     self.distro.osfamily)
+                LOG.debug("Customizing the UTC setting")
+                customize_utc(self._vmware_cust_conf.utc,
+                              self.distro.osfamily)
             except Exception as e:
                 LOG.exception(e)
                 set_customization_status(
diff --git a/cloudinit/sources/helpers/vmware/imc/guestcust_util.py b/cloudinit/sources/helpers/vmware/imc/guestcust_util.py
index 4407525..518d3d2 100644
--- a/cloudinit/sources/helpers/vmware/imc/guestcust_util.py
+++ b/cloudinit/sources/helpers/vmware/imc/guestcust_util.py
@@ -117,4 +117,38 @@ def enable_nics(nics):
     logger.warning("Can't connect network interfaces after %d attempts",
                    enableNicsWaitRetries)
 
+
+def customize_utc(utc, osfamily):
+    """Customizes the UTC setting for the guest OS.
+
+    @param utc: utc setting to be set. It should be either 'yes' or 'no'
+    @param osfamily: family of the guest OS.
+    """
+
+    if not osfamily or osfamily != "debian":
+        logger.info("Debian OS not detected. Skipping the customize step")
+        return
+
+    if utc != "yes" and utc != "no":
+        logger.info("Invalid UTC value: '%s' specified. Skipping.", utc)
+        return
+
+    utc_file = "/etc/default/rcS"
+    pattern = "UTC"
+    replace_with = "UTC=" + utc
+
+    lines = []
+    try:
+        if os.path.isfile(utc_file):
+            lines = util.load_file(utc_file).splitlines()
+        else:
+            util.ensure_file(utc_file)
+    except (IOError, OSError):
+        util.logexc(logger, "Error reading lines from %s", utc_file)
+        return
+
+    new_lines = util.replace_or_append_in_lines(lines, pattern, replace_with)
+
+    util.write_file(utc_file, content='\n'.join(new_lines))
+
 # vi: ts=4 expandtab
diff --git a/cloudinit/util.py b/cloudinit/util.py
index e1290aa..410387a 100644
--- a/cloudinit/util.py
+++ b/cloudinit/util.py
@@ -2541,4 +2541,55 @@ def load_shell_content(content, add_empty=False, empty_val=None):
     return data
 
 
+def get_line_without_comments(line):
+    """Get the content from beginning to the first occurence of #
+
+    @param line: content that needs to be checked.
+    """
+    if line:
+        tokens = line.split('#')
+        return tokens[0]
+    else:
+        return line
+
+
+def replace_or_append_in_lines(lines, pattern, replace_with,
+                               skip_comments=True):
+    """Checks and replaces each line with the new content if
+       if it matches with the specified pattern. If no line
+       matches with the specified pattern, then a new line
+       is added with the new content.
+
+    @param lines: An array of lines to be checked.
+    @param pattern: regex pattern to check in each line.
+    @param replace_with: New content to replace if the pattern
+                         is matched in the line.
+    @param skip_comments: Whether to skip the comments or not.
+                          If this is true, then the pattern is
+                          not checked for the comments.
+    """
+    new_lines = []
+    found = False
+
+    if not pattern or not replace_with:
+        return new_lines
+
+    for line in lines:
+        if skip_comments:
+            new_content = get_line_without_comments(line)
+        else:
+            new_content = line
+
+        if re.search(pattern, new_content, re.IGNORECASE):
+            new_lines.append(replace_with)
+            found = True
+        else:
+            new_lines.append(line)
+
+    if not found:
+        new_lines.append(replace_with)
+
+    return new_lines
+
+
 # vi: ts=4 expandtab
diff --git a/tests/unittests/test_util.py b/tests/unittests/test_util.py
index 3e4154c..9e96d6d 100644
--- a/tests/unittests/test_util.py
+++ b/tests/unittests/test_util.py
@@ -834,4 +834,63 @@ class TestLoadShellContent(helpers.TestCase):
                 ''])))
 
 
+class TestReplaceAppendLines(helpers.TestCase):
+    def test_get_line_without_comments(self):
+        """Test get_line_without_comments function."""
+        self.assertEqual("", util.get_line_without_comments(""))
+        self.assertEqual("no comment",
+                         util.get_line_without_comments("no comment"))
+        self.assertEqual("content ",
+                         util.get_line_without_comments("content # comment"))
+        self.assertEqual("",
+                         util.get_line_without_comments("#comment"))
+
+    def test_replace_append_lines(self):
+        """Test replace_or_append_in_lines function."""
+        output = util.replace_or_append_in_lines(["content"], None,
+                                                 "replace_with")
+        self.assertEqual([], output)
+
+        output = util.replace_or_append_in_lines(["content"], "pattern",
+                                                 None)
+        self.assertEqual([], output)
+
+        output = util.replace_or_append_in_lines([], "pattern", "replace")
+        self.assertEqual(["replace"], output)
+
+        input = ["#pattern", "pattern # comment", "content line"]
+        pattern = "pattern"
+        replace_with = "replaced"
+
+        skip_comments = True
+        expected_output = ["#pattern",
+                           "replaced",
+                           "content line"]
+
+        output = util.replace_or_append_in_lines(input, pattern,
+                                                 replace_with, skip_comments)
+
+        self.assertEqual(expected_output, output)
+
+        skip_comments = False
+        expected_output = ["replaced",
+                           "replaced",
+                           "content line"]
+
+        output = util.replace_or_append_in_lines(input, pattern,
+                                                 replace_with, skip_comments)
+
+        self.assertEqual(expected_output, output)
+
+        input = ["first line", "second line"]
+        pattern = "pattern"
+        replace_with = "replaced"
+        expected_output = input[:]
+        expected_output.append(replace_with)
+
+        output = util.replace_or_append_in_lines(input, pattern,
+                                                 replace_with, skip_comments)
+        self.assertEqual(expected_output, output)
+
+
 # vi: ts=4 expandtab