Merge ~harald-jensas/cloud-init:bug/1847517 into cloud-init:master

[Harald Jensås]

net: handle openstack dhcpv6-stateless configuration

Openstack subnets can be configured to use SLAAC by setting
ipv6_address_mode=dhcpv6-stateless. When this is the case
the sysconfig interface configuration should use
IPV6_AUTOCONF=yes and not set DHCPV6C=yes.

This change sets the subnets type property to the full
network['type'] from openstack metadata.

cloudinit/net/sysconfig.py and cloudinit/net/eni.py
are updated to support new subnet types:
  - 'ipv6_dhcpv6-stateless' => IPV6_AUTOCONF=yes
  - 'ipv6_dhcpv6-stateful' => DHCPV6C=yes

Type 'dhcp6' in sysconfig is kept for backward compatibility
with any implementations that set subnet_type == 'dhcp6'.

LP: #1847517

[Scott Moser (smoser)]

thanks for the well done bug report and merge proposal!

Please add a test to the unfortunately long and difficult to follow tests/unittests/test_net.py .

Also, as this is your first submission:

To contribute, you must sign the Canonical Contributor License Agreement
(CLA) [1].

If you have already signed it as an individual, your Launchpad user will
be listed in the contributor-agreement-canonical launchpad group [2].
Unfortunately there is no easy way to check if an organization or company
you are doing work for has signed. If you are unsure or have questions,
email <email address hidden> or ping powersj in #cloud-init channel
via freenode.

For information on how to sign, please see the HACKING document [3].

Thanks again, and please feel free to reach out with any questions.

–
[1] http://www.canonical.com/contributors
[2] https://launchpad.net/~contributor-agreement-canonical/+members
[3] http://cloudinit.readthedocs.io/en/latest/topics/hacking.html

[Harald Jensås (harald-jensas)]

> thanks for the well done bug report and merge proposal!
>
> Please add a test to the unfortunately long and difficult to follow
> tests/unittests/test_net.py .
>

Thanks, I did'nt find these tests in my first iteration.
I added tests in tests/unittests/test_datasource/test_configdrive.py and tests/unittests/test_net.py.

> Also, as this is your first submission:
>
> To contribute, you must sign the Canonical Contributor License Agreement
> (CLA) [1].
>

I've submitted the form.

[Harald Jensås (harald-jensas)]

Oh, I noticed now I missed the comment in code.

I will update this to check specifically for the values we expect and raise an error for other cases.

[Ryan Harper (raharper)]

Thanks for working on this! The eni path needs fixing for stateless; see in-line comment.

review: Needs Fixing

[Harald Jensås (harald-jensas)]

> Thanks for working on this! The eni path needs fixing for stateless; see in-
> line comment.

Good catch Ryan! I've updated the patch to fix the eni path as well.

[Ryan Harper (raharper)]

Thanks! I've reworded the commit message, see above.
If you agree, feel free to add in your SoB to the commit
message box above if you like.

[Harald Jensås (harald-jensas)]

Thanks! The reworded commit message works for me.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:4cd556d88a2660dd383bbff2664f08a77f915016
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1208/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1208//rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

Thanks for this! It's ready to merge; just waiting for launchpad to add you to the CLA group and then we'll land.

[Harald Jensås (harald-jensas)]

fyi, I'm a member of the CLA group now.

[Ryan Harper (raharper)]

Thanks!

review: Approve

diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py
index b129bb6..530922b 100644
--- a/cloudinit/net/eni.py
+++ b/cloudinit/net/eni.py
@@ -411,8 +411,13 @@ class Renderer(renderer.Renderer):
                 else:
                     ipv4_subnet_mtu = subnet.get('mtu')
                 iface['inet'] = subnet_inet
-                if subnet['type'].startswith('dhcp'):
+                if (subnet['type'] == 'dhcp4' or subnet['type'] == 'dhcp6' or
+                        subnet['type'] == 'ipv6_dhcpv6-stateful'):
+                    # Configure network settings using DHCP or DHCPv6
                     iface['mode'] = 'dhcp'
+                elif subnet['type'] == 'ipv6_dhcpv6-stateless':
+                    # Configure network settings using SLAAC from RAs
+                    iface['mode'] = 'auto'
 
                 # do not emit multiple 'auto $IFACE' lines as older (precise)
                 # ifupdown complains
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index 87b548e..4e65676 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -343,10 +343,15 @@ class Renderer(renderer.Renderer):
         for i, subnet in enumerate(subnets, start=len(iface_cfg.children)):
             mtu_key = 'MTU'
             subnet_type = subnet.get('type')
-            if subnet_type == 'dhcp6':
+            if subnet_type == 'dhcp6' or subnet_type == 'ipv6_dhcpv6-stateful':
                 # TODO need to set BOOTPROTO to dhcp6 on SUSE
                 iface_cfg['IPV6INIT'] = True
+                # Configure network settings using DHCPv6
                 iface_cfg['DHCPV6C'] = True
+            elif subnet_type == 'ipv6_dhcpv6-stateless':
+                iface_cfg['IPV6INIT'] = True
+                # Configure network settings using SLAAC from RAs
+                iface_cfg['IPV6_AUTOCONF'] = True
             elif subnet_type in ['dhcp4', 'dhcp']:
                 iface_cfg['BOOTPROTO'] = 'dhcp'
             elif subnet_type == 'static':
diff --git a/cloudinit/sources/helpers/openstack.py b/cloudinit/sources/helpers/openstack.py
index 8f06911..d1c4601 100644
--- a/cloudinit/sources/helpers/openstack.py
+++ b/cloudinit/sources/helpers/openstack.py
@@ -585,7 +585,8 @@ def convert_net_json(network_json=None, known_macs=None):
             subnet = dict((k, v) for k, v in network.items()
                           if k in valid_keys['subnet'])
             if 'dhcp' in network['type']:
-                t = 'dhcp6' if network['type'].startswith('ipv6') else 'dhcp4'
+                t = (network['type'] if network['type'].startswith('ipv6')
+                     else 'dhcp4')
                 subnet.update({
                     'type': t,
                 })
diff --git a/tests/unittests/test_datasource/test_configdrive.py b/tests/unittests/test_datasource/test_configdrive.py
index 520c50f..8c788c1 100644
--- a/tests/unittests/test_datasource/test_configdrive.py
+++ b/tests/unittests/test_datasource/test_configdrive.py
@@ -499,6 +499,45 @@ class TestNetJson(CiTestCase):
                                                     known_macs=KNOWN_MACS)
         self.assertEqual(myds.network_config, network_config)
 
+    def test_network_config_conversion_dhcp6(self):
+        """Test some ipv6 input network json and check the expected
+           conversions."""
+        in_data = {
+            'links': [
+                {'vif_id': '2ecc7709-b3f7-4448-9580-e1ec32d75bbd',
+                 'ethernet_mac_address': 'fa:16:3e:69:b0:58',
+                 'type': 'ovs', 'mtu': None, 'id': 'tap2ecc7709-b3'},
+                {'vif_id': '2f88d109-5b57-40e6-af32-2472df09dc33',
+                 'ethernet_mac_address': 'fa:16:3e:d4:57:ad',
+                 'type': 'ovs', 'mtu': None, 'id': 'tap2f88d109-5b'},
+            ],
+            'networks': [
+                {'link': 'tap2ecc7709-b3', 'type': 'ipv6_dhcpv6-stateless',
+                 'network_id': '6d6357ac-0f70-4afa-8bd7-c274cc4ea235',
+                 'id': 'network0'},
+                {'link': 'tap2f88d109-5b', 'type': 'ipv6_dhcpv6-stateful',
+                 'network_id': 'd227a9b3-6960-4d94-8976-ee5788b44f54',
+                 'id': 'network1'},
+            ]
+        }
+        out_data = {
+            'version': 1,
+            'config': [
+                {'mac_address': 'fa:16:3e:69:b0:58',
+                 'mtu': None,
+                 'name': 'enp0s1',
+                 'subnets': [{'type': 'ipv6_dhcpv6-stateless'}],
+                 'type': 'physical'},
+                {'mac_address': 'fa:16:3e:d4:57:ad',
+                 'mtu': None,
+                 'name': 'enp0s2',
+                 'subnets': [{'type': 'ipv6_dhcpv6-stateful'}],
+                 'type': 'physical'}
+            ],
+        }
+        conv_data = openstack.convert_net_json(in_data, known_macs=KNOWN_MACS)
+        self.assertEqual(out_data, conv_data)
+
     def test_network_config_conversions(self):
         """Tests a bunch of input network json and checks the
            expected conversions."""
diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
index b659741..f5a9cae 100644
--- a/tests/unittests/test_net.py
+++ b/tests/unittests/test_net.py
@@ -1070,6 +1070,82 @@ NETWORK_CONFIGS = {
                 """),
         },
     },
+    'dhcpv6_stateless': {
+        'expected_eni': textwrap.dedent("""\
+        auto lo
+        iface lo inet loopback
+
+        auto iface0
+        iface iface0 inet6 auto
+    """).rstrip(' '),
+        'expected_netplan': textwrap.dedent("""
+        network:
+            version: 2
+            ethernets:
+                iface0:
+                    dhcp6: true
+    """).rstrip(' '),
+        'yaml': textwrap.dedent("""\
+        version: 1
+        config:
+          - type: 'physical'
+            name: 'iface0'
+            subnets:
+            - {'type': 'ipv6_dhcpv6-stateless'}
+    """).rstrip(' '),
+        'expected_sysconfig': {
+            'ifcfg-iface0': textwrap.dedent("""\
+            BOOTPROTO=none
+            DEVICE=iface0
+            IPV6_AUTOCONF=yes
+            IPV6INIT=yes
+            DEVICE=iface0
+            NM_CONTROLLED=no
+            ONBOOT=yes
+            STARTMODE=auto
+            TYPE=Ethernet
+            USERCTL=no
+            """),
+        },
+    },
+    'dhcpv6_stateful': {
+        'expected_eni': textwrap.dedent("""\
+        auto lo
+        iface lo inet loopback
+
+        auto iface0
+        iface iface0 inet6 dhcp
+    """).rstrip(' '),
+        'expected_netplan': textwrap.dedent("""
+        network:
+            version: 2
+            ethernets:
+                iface0:
+                    dhcp6: true
+    """).rstrip(' '),
+        'yaml': textwrap.dedent("""\
+        version: 1
+        config:
+          - type: 'physical'
+            name: 'iface0'
+            subnets:
+            - {'type': 'ipv6_dhcpv6-stateful'}
+    """).rstrip(' '),
+        'expected_sysconfig': {
+            'ifcfg-iface0': textwrap.dedent("""\
+            BOOTPROTO=none
+            DEVICE=iface0
+            DHCPV6C=yes
+            IPV6INIT=yes
+            DEVICE=iface0
+            NM_CONTROLLED=no
+            ONBOOT=yes
+            STARTMODE=auto
+            TYPE=Ethernet
+            USERCTL=no
+            """),
+        },
+    },
     'all': {
         'expected_eni': ("""\
 auto lo
@@ -2781,6 +2857,18 @@ USERCTL=no
         self._compare_files_to_expected(entry[self.expected_name], found)
         self._assert_headers(found)
 
+    def test_dhcpv6_stateless_config(self):
+        entry = NETWORK_CONFIGS['dhcpv6_stateless']
+        found = self._render_and_read(network_config=yaml.load(entry['yaml']))
+        self._compare_files_to_expected(entry[self.expected_name], found)
+        self._assert_headers(found)
+
+    def test_dhcpv6_stateful_config(self):
+        entry = NETWORK_CONFIGS['dhcpv6_stateful']
+        found = self._render_and_read(network_config=yaml.load(entry['yaml']))
+        self._compare_files_to_expected(entry[self.expected_name], found)
+        self._assert_headers(found)
+
     def test_check_ifcfg_rh(self):
         """ifcfg-rh plugin is added NetworkManager.conf if conf present."""
         render_dir = self.tmp_dir()