Merge ~mgerdts/cloud-init:lp1765801 into cloud-init:master

[Mike Gerdts]

DataSourceSmartOS: reconfigure network on each boot

In typical cases, SmartOS does not use DHCP for network configuration.
As such, if the network configuration changes that is reflected in
metadata and will be picked up during the next boot.

LP: #1765801
Joyent: OS-6902 cloud-init should be able to reconfigure network on each boot

[Chad Smith (chad.smith)]

Thanks for the branch Mike, just a request for unit test or an additional assert on the SmartOs datasource update_events attribute as well as what seems to be a git rebase required on your branch.

[Chad Smith (chad.smith)]

Hi Mike,

  Marking this 'Work in progress' for the time being so it doesn't sit in my 'needs review' column.

When you get a chance to push your changes and want a review, please just set this branch back to 'Needs review' and we'll take care of it.

thanks again,
Chad

[Mike Gerdts (mgerdts)]

I've looked high and low for what is setting ds._network_config = sources.UNSET and have come up empty. I gave up when I realized that DataSourceAzure follows a similar pattern.

I'm also not sure about what the proper way to allow an instance to say "hands off on future reboots". Is apply_network_config the right approach?

[Mike Gerdts (mgerdts)]

Upon further experimentation, I don't think that adding (per-datasource) apply_network_config is needed. Setting it can cause a previously working instance to no longer be reachable over the net.

Better is:

network: {config: disabled}

This allows a working config to persist in /etc/netplan/50-cloud-init.yaml while not making further changes on subsequent boots.

[Ryan Harper (raharper)]

I see a merge conflict in the diff below, I think a rebase should resolve this.

review: Needs Fixing

[Mike Gerdts (mgerdts)]

As I asked in IRC on September 25:

[10:27:07] <mgerdts> I rebased an old branch, made some changes, force pushed, then resubmitted a merge request. The diff shows conflicts that don't really exist, seemingly related to the 13 month old version that was previously there. Would it be easier if I just submitted a fresh merge request rather than trying to make this one happy?
[10:27:09] <mgerdts> https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/373171

I have once again rebased it and force pushed. The diff below still shows the conflict. `git show` in my workspace does not show the conflict. My question from September 25 remains.

[Mike Gerdts (mgerdts)]

I created a fresh merge proposal from a new branch. It does not show the conflicts.

https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/374989

Unmerged commits

caff922... by Mike Gerdts on 2018-07-20

DataSourceSmartOS: reconfigure network on each boot

In typical cases, SmartOS does not use DHCP for network configuration. As such,
if the network configuration changes that is reflected in metadata and will be
picked up during the next boot.

LP: #1765801
Joyent: OS-6902 cloud-init should be able to reconfigure network on each boot

diff --git a/cloudinit/sources/DataSourceSmartOS.py b/cloudinit/sources/DataSourceSmartOS.py
index 32b57cd..cf67650 100644
--- a/cloudinit/sources/DataSourceSmartOS.py
+++ b/cloudinit/sources/DataSourceSmartOS.py
@@ -1,5 +1,5 @@
 # Copyright (C) 2013 Canonical Ltd.
-# Copyright (c) 2018, Joyent, Inc.
+# Copyright 2019 Joyent, Inc.
 #
 # Author: Ben Howard <[e-mail]>
 #
@@ -34,6 +34,7 @@ from cloudinit import log as logging
 from cloudinit import serial
 from cloudinit import sources
 from cloudinit import util
+from cloudinit.event import EventType
 
 LOG = logging.getLogger(__name__)
 
@@ -178,6 +179,7 @@ class DataSourceSmartOS(sources.DataSource):
         self.metadata = {}
         self.network_data = None
         self._network_config = None
+        self.update_events['network'].add(EventType.BOOT)
 
         self.script_base_d = os.path.join(self.paths.get_cpath("scripts"))
 
@@ -319,6 +321,10 @@ class DataSourceSmartOS(sources.DataSource):
 
     @property
     def network_config(self):
+        # sources.clear_cached_data() may set _network_config to '_unset'.
+        if self._network_config == sources.UNSET:
+            self._network_config = None
+
         if self._network_config is None:
             if self.network_data is not None:
                 self._network_config = (
diff --git a/cloudinit/sources/tests/test_init.py b/cloudinit/sources/tests/test_init.py
index 4154a5f..9698261 100644
--- a/cloudinit/sources/tests/test_init.py
+++ b/cloudinit/sources/tests/test_init.py
@@ -543,11 +543,7 @@ class TestDataSource(CiTestCase):
         """update_metadata won't get_data on unsupported update events."""
         self.datasource.update_events['network'].discard(EventType.BOOT)
         self.assertEqual(
-<<<<<<< cloudinit/sources/tests/test_init.py
             {'network': set([EventType.BOOT_NEW_INSTANCE])},
-=======
-            {'network': {EventType.BOOT_NEW_INSTANCE}},
->>>>>>> cloudinit/sources/tests/test_init.py
             self.datasource.update_events)
 
         def fake_get_data():
diff --git a/tests/unittests/test_datasource/test_smartos.py b/tests/unittests/test_datasource/test_smartos.py
index 42ac697..d5b1c29 100644
--- a/tests/unittests/test_datasource/test_smartos.py
+++ b/tests/unittests/test_datasource/test_smartos.py
@@ -1,5 +1,5 @@
 # Copyright (C) 2013 Canonical Ltd.
-# Copyright (c) 2018, Joyent, Inc.
+# Copyright 2019 Joyent, Inc.
 #
 # Author: Ben Howard <[e-mail]>
 #
@@ -31,6 +31,7 @@ from cloudinit.sources.DataSourceSmartOS import (
     convert_smartos_network_data as convert_net,
     SMARTOS_ENV_KVM, SERIAL_DEVICE, get_smartos_environ,
     identify_file)
+from cloudinit.event import EventType
 
 import six
 
@@ -653,6 +654,12 @@ class TestSmartOSDataSource(FilesystemMockingTestCase):
         self.assertEqual(dsrc.device_name_to_device('FOO'),
                          mydscfg['disk_aliases']['FOO'])
 
+    def test_reconfig_network_on_boot(self):
+        # Test to ensure that network is configured from metadata on each boot
+        dsrc = self._get_ds(mockdata=MOCK_RETURNS)
+        self.assertSetEqual(set([EventType.BOOT_NEW_INSTANCE, EventType.BOOT]),
+                            dsrc.update_events['network'])
+
 
 class TestIdentifyFile(CiTestCase):
     """Test the 'identify_file' utility."""