Merge ~raharper/cloud-init:fix/add-netplan-fixed-grat-arp-spelling into cloud-init:master

[Ryan Harper]

netplan: update netplan key mappings for gratuitous-arp

Previous versions of netplan included a misspelling for the
bond parameter around gratuitous-arp. This has been fixed and released
and cloud-init needs to accept both values. This branch fixes the
key that will be rendered and transforms the previous misspelling
when capturing network_state.

LP: #1827238

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:67222ba81ed350da08cec2c3390d66040504da3a
https://jenkins.ubuntu.com/server/job/cloud-init-ci/703/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/703/rebuild

review: Approve (continuous-integration)

[Dan Watkins (oddbloke)]

Do we need to handle older versions of netplan that _don't_ understand the new key?

[Dan Watkins (oddbloke)]

This looks good to me, assuming the answer to my above question is that we don't need to support older versions of netplan.

review: Approve

[Ryan Harper (raharper)]

I need change this fix. On Xenial, it's not fixed to support both, so it only would accept the current value. I don't know if there are plans to

Since newer netplan supports both, but older only the misspelling, at least for now; I think we need to always write the misspelling until we have a way to determine whether it's safe to render the correct spelling version.

I'm aware of two images that use netplan, UbuntuCore16 and Rackspace Ubuntu images.

Let me refactor how we can accept either in the input config, but for now only render the mispelling; I think that's the safest at this point.

0ea9421... by Ryan Harper on 2019-05-03

netplan: accept the correctly spelled grat key, but write misspelling for compact reasons

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:0ea9421d31bff1b39e87cabeb0e5073122bd366e
https://jenkins.ubuntu.com/server/job/cloud-init-ci/704/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/704/rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

Minor inline suggestions, then +1 from me. Once netplan instruments support for the correct spelling of gratuitous-arp in xenial, then we can cross the bridge of removing this misspelling map.

review: Approve

[Ryan Harper (raharper)]

Thanks for the review

5524261... by Ryan Harper on 2019-05-09

Update comment to reference original bug, note release, improve code per Chad.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:552426196d3b74791597b02284d15a5a54fedf29
https://jenkins.ubuntu.com/server/job/cloud-init-ci/711/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/711/rebuild

review: Approve (continuous-integration)

diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py
index 4d19f56..3702130 100644
--- a/cloudinit/net/network_state.py
+++ b/cloudinit/net/network_state.py
@@ -707,6 +707,14 @@ class NetworkStateInterpreter(object):
             item_params = dict((key, value) for (key, value) in
                                item_cfg.items() if key not in
                                NETWORK_V2_KEY_FILTER)
+            # we accept the fixed spelling, but write the old for compatability
+            # Xenial does not have an updated netplan which supports the
+            # correct spelling.  LP: #1756701
+            params = item_params['parameters']
+            grat_value = params.pop('gratuitous-arp', None)
+            if grat_value:
+                params['gratuitious-arp'] = grat_value
+
             v1_cmd = {
                 'type': cmd_type,
                 'name': item_name,
diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
index e85e964..b936bc9 100644
--- a/tests/unittests/test_net.py
+++ b/tests/unittests/test_net.py
@@ -407,6 +407,37 @@ network:
                 - maas
 """
 
+NETPLAN_BOND_GRAT_ARP = """
+network:
+    bonds:
+        bond0:
+            interfaces:
+            - ens3
+            macaddress: 68:05:ca:64:d3:6c
+            mtu: 9000
+            parameters:
+                gratuitious-arp: 1
+        bond1:
+            interfaces:
+            - ens4
+            macaddress: 68:05:ca:64:d3:6d
+            mtu: 9000
+            parameters:
+                gratuitous-arp: 2
+    ethernets:
+        ens3:
+            dhcp4: false
+            dhcp6: false
+            match:
+                macaddress: 52:54:00:ab:cd:ef
+        ens4:
+            dhcp4: false
+            dhcp6: false
+            match:
+                macaddress: 52:54:00:11:22:ff
+    version: 2
+"""
+
 NETPLAN_DHCP_FALSE = """
 version: 2
 ethernets:
@@ -3589,6 +3620,21 @@ class TestNetplanRoundTrip(CiTestCase):
             entry['expected_netplan'].splitlines(),
             files['/etc/netplan/50-cloud-init.yaml'].splitlines())
 
+    def test_render_output_supports_both_grat_arp_spelling(self):
+        entry = {
+            'yaml': NETPLAN_BOND_GRAT_ARP,
+            'expected_netplan': NETPLAN_BOND_GRAT_ARP.replace('gratuitous',
+                                                              'gratuitious'),
+        }
+        network_config = yaml.load(entry['yaml']).get('network')
+        files = self._render_and_read(network_config=network_config)
+        print(entry['expected_netplan'])
+        print('-- expected ^ | v rendered --')
+        print(files['/etc/netplan/50-cloud-init.yaml'])
+        self.assertEqual(
+            entry['expected_netplan'].splitlines(),
+            files['/etc/netplan/50-cloud-init.yaml'].splitlines())
+
 
 class TestEniRoundTrip(CiTestCase):