cloud-init

Merge ~akaris/cloud-init:bug1679817 into cloud-init:master

  1. Git
  2. lp:~akaris/cloud-init
  3. bug1679817
  4. Merge into master
Proposed by Andreas Karis
Status: Superseded
Proposed branch: ~akaris/cloud-init:bug1679817
Merge into: cloud-init:master
Diff against target: 381 lines (+158/-95)
3 files modified
cloudinit/net/sysconfig.py (+132/-32)
tests/unittests/test_distros/test_netconfig.py (+3/-5)
tests/unittests/test_net.py (+23/-58)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Needs Fixing
cloud-init Commiters Pending
Review via email: mp+323079@code.launchpad.net

This proposal supersedes a proposal from 2017-04-22.

This proposal has been superseded by a proposal from 2017-04-24.

Commit message

Fix dual stack IPv4/IPv6 configuration for RHEL

Dual stack IPv4/IPv6 configuration via config drive is broken for RHEL7.
This patch fixes several scenarios for IPv4/IPv6/dual stack with multiple IP assignment
Removes unpopular IPv4 alias files and invalid IPv6 alias files

Also fixes associated unit tests

LP: #1679817
LP: #1685534
LP: #1685532

Description of the change

Fix dual stack IPv4/IPv6 configuration for RHEL

Dual stack IPv4/IPv6 configuration via config drive is broken for RHEL7.
This patch fixes several scenarios for IPv4/IPv6/dual stack with multiple IP assignment
Removes unpopular IPv4 alias files and invalid IPv6 alias files

Also fixes associated unit tests

LP: #1679817
LP: #1685534
LP: #1685532

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote : Posted in a previous version of this proposal

FAILED: Continuous integration, rev:e43facbb2b0b0e88e16c45c8b1ed14c621bdf3ad
https://jenkins.ubuntu.com/server/job/cloud-init-ci/263/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/263/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/263/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/263/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/263/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/263/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/263/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote : Posted in a previous version of this proposal

FAILED: Continuous integration, rev:9ad5280dcc527bece2a4d024eff9f6c7a82cc4ba
https://jenkins.ubuntu.com/server/job/cloud-init-ci/266/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/266/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/266/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/266/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/266/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/266/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/266/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote : Posted in a previous version of this proposal

FAILED: Continuous integration, rev:a94ded7f67e5f5a33bbcfd4d138393e7a8e83e19
https://jenkins.ubuntu.com/server/job/cloud-init-ci/271/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/271/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/271/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/271/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/271/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/271/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/271/rebuild

review: Needs Fixing (continuous-integration)
~akaris/cloud-init:bug1679817 updated
23c7007... by Andreas Karis

cleanup for flake

46c3541... by Andreas Karis

cleanup for flake

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:a94ded7f67e5f5a33bbcfd4d138393e7a8e83e19
https://jenkins.ubuntu.com/server/job/cloud-init-ci/272/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/272/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/272/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/272/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/272/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/272/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/272/rebuild

review: Needs Fixing (continuous-integration)

Unmerged commits

46c3541... by Andreas Karis

cleanup for flake

23c7007... by Andreas Karis

cleanup for flake

a94ded7... by Andreas Karis

Moved IPv6 routes to route6-

7569e43... by Andreas Karis

added route handling for ipv4 and ipv6 routes

2d818c2... by Andreas Karis

wrapped str around netmask in cases it is integer

9ad5280... by Andreas Karis

Remove BOOTPROTO=static and make it BOOTPROTO=none as it should be

e43facb... by Andreas Karis

bug1679817, bug1685534, bug1685532

ca539b9... by Andreas Karis

fix1

6186d58... by Andreas Karis

bug1679817

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
2index 504e4d0..d44fd48 100644
3--- a/cloudinit/net/sysconfig.py
4+++ b/cloudinit/net/sysconfig.py
5@@ -59,6 +59,9 @@ class ConfigMap(object):
6 def __setitem__(self, key, value):
7 self._conf[key] = value
8
9+ def __getitem__(self, key):
10+ return self._conf[key]
11+
12 def drop(self, key):
13 self._conf.pop(key, None)
14
15@@ -83,7 +86,8 @@ class ConfigMap(object):
16 class Route(ConfigMap):
17 """Represents a route configuration."""
18
19- route_fn_tpl = '%(base)s/network-scripts/route-%(name)s'
20+ route_fn_tpl_ipv4 = '%(base)s/network-scripts/route-%(name)s'
21+ route_fn_tpl_ipv6 = '%(base)s/network-scripts/route6-%(name)s'
22
23 def __init__(self, route_name, base_sysconf_dir):
24 super(Route, self).__init__()
25@@ -102,10 +106,65 @@ class Route(ConfigMap):
26 return r
27
28 @property
29- def path(self):
30- return self.route_fn_tpl % ({'base': self._base_sysconf_dir,
31- 'name': self._route_name})
32+ def path_ipv4(self):
33+ return self.route_fn_tpl_ipv4 % ({'base': self._base_sysconf_dir,
34+ 'name': self._route_name})
35+
36+ @property
37+ def path_ipv6(self):
38+ return self.route_fn_tpl_ipv6 % ({'base': self._base_sysconf_dir,
39+ 'name': self._route_name})
40+
41+ def is_ipv6_route(self, address):
42+ return ':' in address
43
44+ def to_string_ipv4(self):
45+ buf = six.StringIO()
46+ buf.write(_make_header())
47+ if self._conf:
48+ buf.write("\n")
49+ # need to reindex IPv4 addresses
50+ # (because Route can contain a mix of IPv4 and IPv6)
51+ reindex = -1
52+ for key in sorted(self._conf.keys()):
53+ if 'ADDRESS' in key:
54+ index = key.replace('ADDRESS', '')
55+ address_value = str(self._conf[key])
56+ # if this is an IPv6 route, then ignore it
57+ if self.is_ipv6_route(address_value):
58+ continue
59+ netmask_value = str(self._conf['NETMASK' + index])
60+ gateway_value = str(self._conf['GATEWAY' + index])
61+
62+ reindex = reindex + 1
63+ buf.write("%s=%s\n" % ('ADDRESS' + str(reindex),
64+ _quote_value(address_value)))
65+ buf.write("%s=%s\n" % ('GATEWAY' + str(reindex),
66+ _quote_value(gateway_value)))
67+ buf.write("%s=%s\n" % ('NETMASK' + str(reindex),
68+ _quote_value(netmask_value)))
69+ return buf.getvalue()
70+
71+ def to_string_ipv6(self):
72+ buf = six.StringIO()
73+ buf.write(_make_header())
74+ if self._conf:
75+ buf.write("\n")
76+ for key in sorted(self._conf.keys()):
77+ if 'ADDRESS' in key:
78+ index = key.replace('ADDRESS','')
79+ address_key = key
80+ address_value = str(self._conf[address_key])
81+ if not self.is_ipv6_route(address_value):
82+ continue
83+ netmask_key = 'NETMASK' + index
84+ netmask_value = str(self._conf[netmask_key])
85+ gateway_key = 'GATEWAY' + index
86+ gateway_value = str(self._conf[gateway_key])
87+ buf.write("%s/%s via %s\n" % (address_value,
88+ netmask_value,
89+ gateway_value))
90+ return buf.getvalue()
91
92 class NetInterface(ConfigMap):
93 """Represents a sysconfig/networking-script (and its config + children)."""
94@@ -211,27 +270,73 @@ class Renderer(renderer.Renderer):
95 iface_cfg[new_key] = old_value
96
97 @classmethod
98- def _render_subnet(cls, iface_cfg, route_cfg, subnet):
99- subnet_type = subnet.get('type')
100- if subnet_type == 'dhcp6':
101- iface_cfg['DHCPV6C'] = True
102- iface_cfg['IPV6INIT'] = True
103- iface_cfg['BOOTPROTO'] = 'dhcp'
104- elif subnet_type in ['dhcp4', 'dhcp']:
105- iface_cfg['BOOTPROTO'] = 'dhcp'
106- elif subnet_type == 'static':
107- iface_cfg['BOOTPROTO'] = 'static'
108- if subnet_is_ipv6(subnet):
109- iface_cfg['IPV6ADDR'] = subnet['address']
110+ def _render_subnets(cls, iface_cfg, subnets):
111+ # setting base values
112+ iface_cfg['BOOTPROTO'] = 'none'
113+
114+ # modifying base values according to subnets
115+ for i, subnet in enumerate(subnets, start=len(iface_cfg.children)):
116+ subnet_type = subnet.get('type')
117+ if subnet_type == 'dhcp6':
118 iface_cfg['IPV6INIT'] = True
119+ iface_cfg['DHCPV6C'] = True
120+ iface_cfg['BOOTPROTO'] = 'dhcp'
121+ elif subnet_type in ['dhcp4', 'dhcp']:
122+ iface_cfg['BOOTPROTO'] = 'dhcp'
123+ elif subnet_type == 'static':
124+ # grep BOOTPROTO sysconfig.txt -A2 | head -3
125+ # BOOTPROTO=none|bootp|dhcp
126+ # 'bootp' or 'dhcp' cause a DHCP client
127+ # to run on the device. Any other
128+ # value causes any static configuration
129+ # in the file to be applied.
130+ # ==> the following should not be set to 'static'
131+ # but should remain 'none'
132+ # if iface_cfg['BOOTPROTO'] == 'none':
133+ # iface_cfg['BOOTPROTO'] = 'static'
134+ if subnet_is_ipv6(subnet):
135+ iface_cfg['IPV6INIT'] = True
136 else:
137- iface_cfg['IPADDR'] = subnet['address']
138- else:
139- raise ValueError("Unknown subnet type '%s' found"
140- " for interface '%s'" % (subnet_type,
141+ raise ValueError("Unknown subnet type '%s' found"
142+ " for interface '%s'" % (subnet_type,
143 iface_cfg.name))
144- if 'netmask' in subnet:
145- iface_cfg['NETMASK'] = subnet['netmask']
146+
147+ # set IPv4 and IPv6 static addresses
148+ ipv4_index = -1
149+ ipv6_index = -1
150+ for i, subnet in enumerate(subnets, start=len(iface_cfg.children)):
151+ subnet_type = subnet.get('type')
152+ if subnet_type == 'dhcp6':
153+ continue
154+ elif subnet_type in ['dhcp4', 'dhcp']:
155+ continue
156+ elif subnet_type == 'static':
157+ if subnet_is_ipv6(subnet):
158+ ipv6_index = ipv6_index + 1
159+ if 'netmask' in subnet and str(subnet['netmask']) != "":
160+ ipv6_cidr = subnet['address'] + '/' + str(subnet['netmask'])
161+ else:
162+ ipv6_cidr = subnet['address']
163+ if ipv6_index == 0:
164+ iface_cfg['IPV6ADDR'] = ipv6_cidr
165+ elif ipv6_index == 1:
166+ iface_cfg['IPV6ADDR_SECONDARIES'] = ipv6_cidr
167+ else:
168+ iface_cfg['IPV6ADDR_SECONDARIES'] = iface_cfg['IPV6ADDR_SECONDARIES'] + " " + ipv6_cidr
169+ else:
170+ ipv4_index = ipv4_index + 1
171+ if ipv4_index == 0:
172+ iface_cfg['IPADDR'] = subnet['address']
173+ if 'netmask' in subnet:
174+ iface_cfg['NETMASK'] = subnet['netmask']
175+ else:
176+ iface_cfg['IPADDR' + str(ipv4_index)] = subnet['address']
177+ if 'netmask' in subnet:
178+ iface_cfg['NETMASK' + str(ipv4_index)] = subnet['netmask']
179+
180+ @classmethod
181+ def _render_subnet_routes(cls, iface_cfg, route_cfg, subnets):
182+ for i, subnet in enumerate(subnets,start=len(iface_cfg.children)):
183 for route in subnet.get('routes', []):
184 if subnet.get('ipv6'):
185 gw_cfg = 'IPV6_DEFAULTGW'
186@@ -295,15 +400,9 @@ class Renderer(renderer.Renderer):
187 iface_subnets = iface.get("subnets", [])
188 iface_cfg = iface_contents[iface_name]
189 route_cfg = iface_cfg.routes
190- if len(iface_subnets) == 1:
191- cls._render_subnet(iface_cfg, route_cfg, iface_subnets[0])
192- elif len(iface_subnets) > 1:
193- for i, isubnet in enumerate(iface_subnets,
194- start=len(iface_cfg.children)):
195- iface_sub_cfg = iface_cfg.copy()
196- iface_sub_cfg.name = "%s:%s" % (iface_name, i)
197- iface_cfg.children.append(iface_sub_cfg)
198- cls._render_subnet(iface_sub_cfg, route_cfg, isubnet)
199+
200+ cls._render_subnets(iface_cfg, iface_subnets)
201+ cls._render_subnet_routes(iface_cfg, route_cfg, iface_subnets)
202
203 @classmethod
204 def _render_bond_interfaces(cls, network_state, iface_contents):
205@@ -387,7 +486,8 @@ class Renderer(renderer.Renderer):
206 if iface_cfg:
207 contents[iface_cfg.path] = iface_cfg.to_string()
208 if iface_cfg.routes:
209- contents[iface_cfg.routes.path] = iface_cfg.routes.to_string()
210+ contents[iface_cfg.routes.path_ipv4] = iface_cfg.routes.to_string_ipv4()
211+ contents[iface_cfg.routes.path_ipv6] = iface_cfg.routes.to_string_ipv6()
212 return contents
213
214 def render_network_state(self, network_state, target=None):
215diff --git a/tests/unittests/test_distros/test_netconfig.py b/tests/unittests/test_distros/test_netconfig.py
216index 8837066..6d6b985 100644
217--- a/tests/unittests/test_distros/test_netconfig.py
218+++ b/tests/unittests/test_distros/test_netconfig.py
219@@ -431,7 +431,7 @@ NETWORKING=yes
220 expected_buf = '''
221 # Created by cloud-init on instance boot automatically, do not edit.
222 #
223-BOOTPROTO=static
224+BOOTPROTO=none
225 DEVICE=eth0
226 IPADDR=192.168.1.5
227 NETMASK=255.255.255.0
228@@ -488,7 +488,6 @@ NETWORKING=yes
229 mock.patch.object(util, 'load_file', return_value=''))
230 mocks.enter_context(
231 mock.patch.object(os.path, 'isfile', return_value=False))
232-
233 rh_distro.apply_network(BASE_NET_CFG_IPV6, False)
234
235 self.assertEqual(len(write_bufs), 4)
236@@ -581,11 +580,10 @@ IPV6_AUTOCONF=no
237 expected_buf = '''
238 # Created by cloud-init on instance boot automatically, do not edit.
239 #
240-BOOTPROTO=static
241+BOOTPROTO=none
242 DEVICE=eth0
243-IPV6ADDR=2607:f0d0:1002:0011::2
244+IPV6ADDR=2607:f0d0:1002:0011::2/64
245 IPV6INIT=yes
246-NETMASK=64
247 NM_CONTROLLED=no
248 ONBOOT=yes
249 TYPE=Ethernet
250diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
251index 89e7536..bad3fb8 100644
252--- a/tests/unittests/test_net.py
253+++ b/tests/unittests/test_net.py
254@@ -136,7 +136,7 @@ OS_SAMPLES = [
255 """
256 # Created by cloud-init on instance boot automatically, do not edit.
257 #
258-BOOTPROTO=static
259+BOOTPROTO=none
260 DEFROUTE=yes
261 DEVICE=eth0
262 GATEWAY=172.19.3.254
263@@ -204,38 +204,14 @@ nameserver 172.19.0.12
264 # Created by cloud-init on instance boot automatically, do not edit.
265 #
266 BOOTPROTO=none
267-DEVICE=eth0
268-HWADDR=fa:16:3e:ed:9a:59
269-NM_CONTROLLED=no
270-ONBOOT=yes
271-TYPE=Ethernet
272-USERCTL=no
273-""".lstrip()),
274- ('etc/sysconfig/network-scripts/ifcfg-eth0:0',
275- """
276-# Created by cloud-init on instance boot automatically, do not edit.
277-#
278-BOOTPROTO=static
279 DEFROUTE=yes
280-DEVICE=eth0:0
281+DEVICE=eth0
282 GATEWAY=172.19.3.254
283 HWADDR=fa:16:3e:ed:9a:59
284 IPADDR=172.19.1.34
285+IPADDR1=10.0.0.10
286 NETMASK=255.255.252.0
287-NM_CONTROLLED=no
288-ONBOOT=yes
289-TYPE=Ethernet
290-USERCTL=no
291-""".lstrip()),
292- ('etc/sysconfig/network-scripts/ifcfg-eth0:1',
293- """
294-# Created by cloud-init on instance boot automatically, do not edit.
295-#
296-BOOTPROTO=static
297-DEVICE=eth0:1
298-HWADDR=fa:16:3e:ed:9a:59
299-IPADDR=10.0.0.10
300-NETMASK=255.255.255.0
301+NETMASK1=255.255.255.0
302 NM_CONTROLLED=no
303 ONBOOT=yes
304 TYPE=Ethernet
305@@ -264,8 +240,8 @@ nameserver 172.19.0.12
306 "gateway": "172.19.3.254",
307 }],
308 "ip_address": "172.19.1.34", "id": "network0"
309- }, {
310- "network_id": "public-ipv6",
311+ },{
312+ "network_id": "public-ipv6-a",
313 "type": "ipv6", "netmask": "",
314 "link": "tap1a81968a-79",
315 "routes": [
316@@ -276,6 +252,20 @@ nameserver 172.19.0.12
317 }
318 ],
319 "ip_address": "2001:DB8::10", "id": "network1"
320+ },{
321+ "network_id": "public-ipv6-b",
322+ "type": "ipv6", "netmask": "64",
323+ "link": "tap1a81968a-79",
324+ "routes": [
325+ ],
326+ "ip_address": "2001:DB9::10", "id": "network2"
327+ },{
328+ "network_id": "public-ipv6-c",
329+ "type": "ipv6", "netmask": "64",
330+ "link": "tap1a81968a-79",
331+ "routes": [
332+ ],
333+ "ip_address": "2001:DB10::10", "id": "network3"
334 }],
335 "links": [
336 {
337@@ -295,41 +285,16 @@ nameserver 172.19.0.12
338 # Created by cloud-init on instance boot automatically, do not edit.
339 #
340 BOOTPROTO=none
341-DEVICE=eth0
342-HWADDR=fa:16:3e:ed:9a:59
343-NM_CONTROLLED=no
344-ONBOOT=yes
345-TYPE=Ethernet
346-USERCTL=no
347-""".lstrip()),
348- ('etc/sysconfig/network-scripts/ifcfg-eth0:0',
349- """
350-# Created by cloud-init on instance boot automatically, do not edit.
351-#
352-BOOTPROTO=static
353 DEFROUTE=yes
354-DEVICE=eth0:0
355+DEVICE=eth0
356 GATEWAY=172.19.3.254
357 HWADDR=fa:16:3e:ed:9a:59
358 IPADDR=172.19.1.34
359-NETMASK=255.255.252.0
360-NM_CONTROLLED=no
361-ONBOOT=yes
362-TYPE=Ethernet
363-USERCTL=no
364-""".lstrip()),
365- ('etc/sysconfig/network-scripts/ifcfg-eth0:1',
366- """
367-# Created by cloud-init on instance boot automatically, do not edit.
368-#
369-BOOTPROTO=static
370-DEFROUTE=yes
371-DEVICE=eth0:1
372-HWADDR=fa:16:3e:ed:9a:59
373 IPV6ADDR=2001:DB8::10
374+IPV6ADDR_SECONDARIES="2001:DB9::10/64 2001:DB10::10/64"
375 IPV6INIT=yes
376 IPV6_DEFAULTGW=2001:DB8::1
377-NETMASK=
378+NETMASK=255.255.252.0
379 NM_CONTROLLED=no
380 ONBOOT=yes
381 TYPE=Ethernet

Subscribers

People subscribed via source and target branches