Merge ~raharper/cloud-init:feature/detect-netfailover into cloud-init:master

Proposed by Ryan Harper
Status: Merged
Approved by: Ryan Harper
Approved revision: 5031c2a1c15f1146a42021d357b556b8b1f1520b
Merge reported by: Server Team CI bot
Merged at revision: not available
Proposed branch: ~raharper/cloud-init:feature/detect-netfailover
Merge into: cloud-init:master
Diff against target: 790 lines (+656/-4)
5 files modified
cloudinit/net/__init__.py (+130/-3)
cloudinit/net/tests/test_init.py (+310/-0)
cloudinit/sources/DataSourceOracle.py (+61/-1)
cloudinit/sources/tests/test_oracle.py (+147/-0)
cloudinit/tests/helpers.py (+8/-0)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
Ryan Harper Approve
Chad Smith Approve
Review via email: mp+371895@code.launchpad.net

Commit message

net,Oracle: Add support for netfailover detection

Add support for detecting netfailover[1] device 3-tuple in networking
layer. In the Oracle datasource ensure that if a provided network
config, either fallback or provided config includes a netfailover master
to remove any MAC address value as this can break under 3-netdev
as the other two devices have the same MAC.

1. https://www.kernel.org/doc/html/latest/networking/net_failover.html

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:73af8a87bb5ab0e72013ed44de88dae1ac8678a2
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1080/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1080//rebuild

review: Approve (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

Nice, Ryan.

  I tried to give an alternative to allow for consolidation of all is_netfailover* functions into a single is_netfailover_device. See what you think.

Revision history for this message
Chad Smith (chad.smith) :
review: Needs Information
Revision history for this message
Ryan Harper (raharper) :
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:d09ad7e55b2bfcf1a7683529be9686cc84c01b50
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1103/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1103//rebuild

review: Approve (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

Thank you for the resolution here. I agree that the single function is a lot less readable than the separate is_netfail_* functions. LGTM!

review: Approve
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/306/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/307/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/308/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/310/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/313/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/314/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/315/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/317/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/319/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/320/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/321/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/322/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/323/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) :
review: Approve (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
Unapproved changes made after approval.
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/324/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

review: Needs Fixing (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

I've force pushed a fix for the pycodestyle change after merge, so should be good now.

Revision history for this message
Ryan Harper (raharper) wrote :

Approving

review: Approve
Revision history for this message
Server Team CI bot (server-team-bot) :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
2index ea707c0..0eb952f 100644
3--- a/cloudinit/net/__init__.py
4+++ b/cloudinit/net/__init__.py
5@@ -109,6 +109,123 @@ def is_bond(devname):
6 return os.path.exists(sys_dev_path(devname, "bonding"))
7
8
9+def is_netfailover(devname, driver=None):
10+ """ netfailover driver uses 3 nics, master, primary and standby.
11+ this returns True if the device is either the primary or standby
12+ as these devices are to be ignored.
13+ """
14+ if driver is None:
15+ driver = device_driver(devname)
16+ if is_netfail_primary(devname, driver) or is_netfail_standby(devname,
17+ driver):
18+ return True
19+ return False
20+
21+
22+def get_dev_features(devname):
23+ """ Returns a str from reading /sys/class/net/<devname>/device/features."""
24+ features = ''
25+ try:
26+ features = read_sys_net(devname, 'device/features')
27+ except Exception:
28+ pass
29+ return features
30+
31+
32+def has_netfail_standby_feature(devname):
33+ """ Return True if VIRTIO_NET_F_STANDBY bit (62) is set.
34+
35+ https://github.com/torvalds/linux/blob/ \
36+ 089cf7f6ecb266b6a4164919a2e69bd2f938374a/ \
37+ include/uapi/linux/virtio_net.h#L60
38+ """
39+ features = get_dev_features(devname)
40+ if not features or len(features) < 64:
41+ return False
42+ return features[62] == "1"
43+
44+
45+def is_netfail_master(devname, driver=None):
46+ """ A device is a "netfail master" device if:
47+
48+ - The device does NOT have the 'master' sysfs attribute
49+ - The device driver is 'virtio_net'
50+ - The device has the standby feature bit set
51+
52+ Return True if all of the above is True.
53+ """
54+ if os.path.exists(sys_dev_path(devname, path='master')):
55+ return False
56+
57+ if driver is None:
58+ driver = device_driver(devname)
59+
60+ if driver != "virtio_net":
61+ return False
62+
63+ if not has_netfail_standby_feature(devname):
64+ return False
65+
66+ return True
67+
68+
69+def is_netfail_primary(devname, driver=None):
70+ """ A device is a "netfail primary" device if:
71+
72+ - the device has a 'master' sysfs file
73+ - the device driver is not 'virtio_net'
74+ - the 'master' sysfs file points to device with virtio_net driver
75+ - the 'master' device has the 'standby' feature bit set
76+
77+ Return True if all of the above is True.
78+ """
79+ # /sys/class/net/<devname>/master -> ../../<master devname>
80+ master_sysfs_path = sys_dev_path(devname, path='master')
81+ if not os.path.exists(master_sysfs_path):
82+ return False
83+
84+ if driver is None:
85+ driver = device_driver(devname)
86+
87+ if driver == "virtio_net":
88+ return False
89+
90+ master_devname = os.path.basename(os.path.realpath(master_sysfs_path))
91+ master_driver = device_driver(master_devname)
92+ if master_driver != "virtio_net":
93+ return False
94+
95+ master_has_standby = has_netfail_standby_feature(master_devname)
96+ if not master_has_standby:
97+ return False
98+
99+ return True
100+
101+
102+def is_netfail_standby(devname, driver=None):
103+ """ A device is a "netfail standby" device if:
104+
105+ - The device has a 'master' sysfs attribute
106+ - The device driver is 'virtio_net'
107+ - The device has the standby feature bit set
108+
109+ Return True if all of the above is True.
110+ """
111+ if not os.path.exists(sys_dev_path(devname, path='master')):
112+ return False
113+
114+ if driver is None:
115+ driver = device_driver(devname)
116+
117+ if driver != "virtio_net":
118+ return False
119+
120+ if not has_netfail_standby_feature(devname):
121+ return False
122+
123+ return True
124+
125+
126 def is_renamed(devname):
127 """
128 /* interface name assignment types (sysfs name_assign_type attribute) */
129@@ -227,6 +344,9 @@ def find_fallback_nic(blacklist_drivers=None):
130 if is_bond(interface):
131 # skip any bonds
132 continue
133+ if is_netfailover(interface):
134+ # ignore netfailover primary/standby interfaces
135+ continue
136 carrier = read_sys_net_int(interface, 'carrier')
137 if carrier:
138 connected.append(interface)
139@@ -273,9 +393,14 @@ def generate_fallback_config(blacklist_drivers=None, config_driver=None):
140 if not target_name:
141 # can't read any interfaces addresses (or there are none); give up
142 return None
143- target_mac = read_sys_net_safe(target_name, 'address')
144- cfg = {'dhcp4': True, 'set-name': target_name,
145- 'match': {'macaddress': target_mac.lower()}}
146+
147+ # netfail cannot use mac for matching, they have duplicate macs
148+ if is_netfail_master(target_name):
149+ match = {'name': target_name}
150+ else:
151+ match = {
152+ 'macaddress': read_sys_net_safe(target_name, 'address').lower()}
153+ cfg = {'dhcp4': True, 'set-name': target_name, 'match': match}
154 if config_driver:
155 driver = device_driver(target_name)
156 if driver:
157@@ -661,6 +786,8 @@ def get_interfaces():
158 continue
159 if is_bond(name):
160 continue
161+ if is_netfailover(name):
162+ continue
163 mac = get_interface_mac(name)
164 # some devices may not have a mac (tun0)
165 if not mac:
166diff --git a/cloudinit/net/tests/test_init.py b/cloudinit/net/tests/test_init.py
167index d2e38f0..7259dbe 100644
168--- a/cloudinit/net/tests/test_init.py
169+++ b/cloudinit/net/tests/test_init.py
170@@ -204,6 +204,10 @@ class TestGenerateFallbackConfig(CiTestCase):
171 self.add_patch('cloudinit.net.util.is_container', 'm_is_container',
172 return_value=False)
173 self.add_patch('cloudinit.net.util.udevadm_settle', 'm_settle')
174+ self.add_patch('cloudinit.net.is_netfailover', 'm_netfail',
175+ return_value=False)
176+ self.add_patch('cloudinit.net.is_netfail_master', 'm_netfail_master',
177+ return_value=False)
178
179 def test_generate_fallback_finds_connected_eth_with_mac(self):
180 """generate_fallback_config finds any connected device with a mac."""
181@@ -268,6 +272,61 @@ class TestGenerateFallbackConfig(CiTestCase):
182 ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))
183 self.assertIsNone(net.generate_fallback_config())
184
185+ def test_generate_fallback_config_skips_netfail_devs(self):
186+ """gen_fallback_config ignores netfail primary,sby no mac on master."""
187+ mac = 'aa:bb:cc:aa:bb:cc' # netfailover devs share the same mac
188+ for iface in ['ens3', 'ens3sby', 'enP0s1f3']:
189+ write_file(os.path.join(self.sysdir, iface, 'carrier'), '1')
190+ write_file(
191+ os.path.join(self.sysdir, iface, 'addr_assign_type'), '0')
192+ write_file(
193+ os.path.join(self.sysdir, iface, 'address'), mac)
194+
195+ def is_netfail(iface, _driver=None):
196+ # ens3 is the master
197+ if iface == 'ens3':
198+ return False
199+ return True
200+ self.m_netfail.side_effect = is_netfail
201+
202+ def is_netfail_master(iface, _driver=None):
203+ # ens3 is the master
204+ if iface == 'ens3':
205+ return True
206+ return False
207+ self.m_netfail_master.side_effect = is_netfail_master
208+ expected = {
209+ 'ethernets': {
210+ 'ens3': {'dhcp4': True, 'match': {'name': 'ens3'},
211+ 'set-name': 'ens3'}},
212+ 'version': 2}
213+ result = net.generate_fallback_config()
214+ self.assertEqual(expected, result)
215+
216+
217+class TestNetFindFallBackNic(CiTestCase):
218+
219+ with_logs = True
220+
221+ def setUp(self):
222+ super(TestNetFindFallBackNic, self).setUp()
223+ sys_mock = mock.patch('cloudinit.net.get_sys_class_path')
224+ self.m_sys_path = sys_mock.start()
225+ self.sysdir = self.tmp_dir() + '/'
226+ self.m_sys_path.return_value = self.sysdir
227+ self.addCleanup(sys_mock.stop)
228+ self.add_patch('cloudinit.net.util.is_container', 'm_is_container',
229+ return_value=False)
230+ self.add_patch('cloudinit.net.util.udevadm_settle', 'm_settle')
231+
232+ def test_generate_fallback_finds_first_connected_eth_with_mac(self):
233+ """find_fallback_nic finds any connected device with a mac."""
234+ write_file(os.path.join(self.sysdir, 'eth0', 'carrier'), '1')
235+ write_file(os.path.join(self.sysdir, 'eth1', 'carrier'), '1')
236+ mac = 'aa:bb:cc:aa:bb:cc'
237+ write_file(os.path.join(self.sysdir, 'eth1', 'address'), mac)
238+ self.assertEqual('eth1', net.find_fallback_nic())
239+
240
241 class TestGetDeviceList(CiTestCase):
242
243@@ -365,6 +424,26 @@ class TestGetInterfaceMAC(CiTestCase):
244 expected = [('eth2', 'aa:bb:cc:aa:bb:cc', None, None)]
245 self.assertEqual(expected, net.get_interfaces())
246
247+ @mock.patch('cloudinit.net.is_netfailover')
248+ def test_get_interfaces_by_mac_skips_netfailvoer(self, m_netfail):
249+ """Ignore interfaces if netfailover primary or standby."""
250+ mac = 'aa:bb:cc:aa:bb:cc' # netfailover devs share the same mac
251+ for iface in ['ens3', 'ens3sby', 'enP0s1f3']:
252+ write_file(
253+ os.path.join(self.sysdir, iface, 'addr_assign_type'), '0')
254+ write_file(
255+ os.path.join(self.sysdir, iface, 'address'), mac)
256+
257+ def is_netfail(iface, _driver=None):
258+ # ens3 is the master
259+ if iface == 'ens3':
260+ return False
261+ else:
262+ return True
263+ m_netfail.side_effect = is_netfail
264+ expected = [('ens3', mac, None, None)]
265+ self.assertEqual(expected, net.get_interfaces())
266+
267
268 class TestInterfaceHasOwnMAC(CiTestCase):
269
270@@ -922,3 +1001,234 @@ class TestWaitForPhysdevs(CiTestCase):
271 self.m_get_iface_mac.return_value = {}
272 net.wait_for_physdevs(netcfg, strict=False)
273 self.assertEqual(5 * len(physdevs), self.m_udev_settle.call_count)
274+
275+
276+class TestNetFailOver(CiTestCase):
277+
278+ with_logs = True
279+
280+ def setUp(self):
281+ super(TestNetFailOver, self).setUp()
282+ self.add_patch('cloudinit.net.util', 'm_util')
283+ self.add_patch('cloudinit.net.read_sys_net', 'm_read_sys_net')
284+ self.add_patch('cloudinit.net.device_driver', 'm_device_driver')
285+
286+ def test_get_dev_features(self):
287+ devname = self.random_string()
288+ features = self.random_string()
289+ self.m_read_sys_net.return_value = features
290+
291+ self.assertEqual(features, net.get_dev_features(devname))
292+ self.assertEqual(1, self.m_read_sys_net.call_count)
293+ self.assertEqual(mock.call(devname, 'device/features'),
294+ self.m_read_sys_net.call_args_list[0])
295+
296+ def test_get_dev_features_none_returns_empty_string(self):
297+ devname = self.random_string()
298+ self.m_read_sys_net.side_effect = Exception('error')
299+ self.assertEqual('', net.get_dev_features(devname))
300+ self.assertEqual(1, self.m_read_sys_net.call_count)
301+ self.assertEqual(mock.call(devname, 'device/features'),
302+ self.m_read_sys_net.call_args_list[0])
303+
304+ @mock.patch('cloudinit.net.get_dev_features')
305+ def test_has_netfail_standby_feature(self, m_dev_features):
306+ devname = self.random_string()
307+ standby_features = ('0' * 62) + '1' + '0'
308+ m_dev_features.return_value = standby_features
309+ self.assertTrue(net.has_netfail_standby_feature(devname))
310+
311+ @mock.patch('cloudinit.net.get_dev_features')
312+ def test_has_netfail_standby_feature_short_is_false(self, m_dev_features):
313+ devname = self.random_string()
314+ standby_features = self.random_string()
315+ m_dev_features.return_value = standby_features
316+ self.assertFalse(net.has_netfail_standby_feature(devname))
317+
318+ @mock.patch('cloudinit.net.get_dev_features')
319+ def test_has_netfail_standby_feature_not_present_is_false(self,
320+ m_dev_features):
321+ devname = self.random_string()
322+ standby_features = '0' * 64
323+ m_dev_features.return_value = standby_features
324+ self.assertFalse(net.has_netfail_standby_feature(devname))
325+
326+ @mock.patch('cloudinit.net.get_dev_features')
327+ def test_has_netfail_standby_feature_no_features_is_false(self,
328+ m_dev_features):
329+ devname = self.random_string()
330+ standby_features = None
331+ m_dev_features.return_value = standby_features
332+ self.assertFalse(net.has_netfail_standby_feature(devname))
333+
334+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
335+ @mock.patch('cloudinit.net.os.path.exists')
336+ def test_is_netfail_master(self, m_exists, m_standby):
337+ devname = self.random_string()
338+ driver = 'virtio_net'
339+ m_exists.return_value = False # no master sysfs attr
340+ m_standby.return_value = True # has standby feature flag
341+ self.assertTrue(net.is_netfail_master(devname, driver))
342+
343+ @mock.patch('cloudinit.net.sys_dev_path')
344+ def test_is_netfail_master_checks_master_attr(self, m_sysdev):
345+ devname = self.random_string()
346+ driver = 'virtio_net'
347+ m_sysdev.return_value = self.random_string()
348+ self.assertFalse(net.is_netfail_master(devname, driver))
349+ self.assertEqual(1, m_sysdev.call_count)
350+ self.assertEqual(mock.call(devname, path='master'),
351+ m_sysdev.call_args_list[0])
352+
353+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
354+ @mock.patch('cloudinit.net.os.path.exists')
355+ def test_is_netfail_master_wrong_driver(self, m_exists, m_standby):
356+ devname = self.random_string()
357+ driver = self.random_string()
358+ self.assertFalse(net.is_netfail_master(devname, driver))
359+
360+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
361+ @mock.patch('cloudinit.net.os.path.exists')
362+ def test_is_netfail_master_has_master_attr(self, m_exists, m_standby):
363+ devname = self.random_string()
364+ driver = 'virtio_net'
365+ m_exists.return_value = True # has master sysfs attr
366+ self.assertFalse(net.is_netfail_master(devname, driver))
367+
368+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
369+ @mock.patch('cloudinit.net.os.path.exists')
370+ def test_is_netfail_master_no_standby_feat(self, m_exists, m_standby):
371+ devname = self.random_string()
372+ driver = 'virtio_net'
373+ m_exists.return_value = False # no master sysfs attr
374+ m_standby.return_value = False # no standby feature flag
375+ self.assertFalse(net.is_netfail_master(devname, driver))
376+
377+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
378+ @mock.patch('cloudinit.net.os.path.exists')
379+ @mock.patch('cloudinit.net.sys_dev_path')
380+ def test_is_netfail_primary(self, m_sysdev, m_exists, m_standby):
381+ devname = self.random_string()
382+ driver = self.random_string() # device not virtio_net
383+ master_devname = self.random_string()
384+ m_sysdev.return_value = "%s/%s" % (self.random_string(),
385+ master_devname)
386+ m_exists.return_value = True # has master sysfs attr
387+ self.m_device_driver.return_value = 'virtio_net' # master virtio_net
388+ m_standby.return_value = True # has standby feature flag
389+ self.assertTrue(net.is_netfail_primary(devname, driver))
390+ self.assertEqual(1, self.m_device_driver.call_count)
391+ self.assertEqual(mock.call(master_devname),
392+ self.m_device_driver.call_args_list[0])
393+ self.assertEqual(1, m_standby.call_count)
394+ self.assertEqual(mock.call(master_devname),
395+ m_standby.call_args_list[0])
396+
397+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
398+ @mock.patch('cloudinit.net.os.path.exists')
399+ @mock.patch('cloudinit.net.sys_dev_path')
400+ def test_is_netfail_primary_wrong_driver(self, m_sysdev, m_exists,
401+ m_standby):
402+ devname = self.random_string()
403+ driver = 'virtio_net'
404+ self.assertFalse(net.is_netfail_primary(devname, driver))
405+
406+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
407+ @mock.patch('cloudinit.net.os.path.exists')
408+ @mock.patch('cloudinit.net.sys_dev_path')
409+ def test_is_netfail_primary_no_master(self, m_sysdev, m_exists, m_standby):
410+ devname = self.random_string()
411+ driver = self.random_string() # device not virtio_net
412+ m_exists.return_value = False # no master sysfs attr
413+ self.assertFalse(net.is_netfail_primary(devname, driver))
414+
415+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
416+ @mock.patch('cloudinit.net.os.path.exists')
417+ @mock.patch('cloudinit.net.sys_dev_path')
418+ def test_is_netfail_primary_bad_master(self, m_sysdev, m_exists,
419+ m_standby):
420+ devname = self.random_string()
421+ driver = self.random_string() # device not virtio_net
422+ master_devname = self.random_string()
423+ m_sysdev.return_value = "%s/%s" % (self.random_string(),
424+ master_devname)
425+ m_exists.return_value = True # has master sysfs attr
426+ self.m_device_driver.return_value = 'XXXX' # master not virtio_net
427+ self.assertFalse(net.is_netfail_primary(devname, driver))
428+
429+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
430+ @mock.patch('cloudinit.net.os.path.exists')
431+ @mock.patch('cloudinit.net.sys_dev_path')
432+ def test_is_netfail_primary_no_standby(self, m_sysdev, m_exists,
433+ m_standby):
434+ devname = self.random_string()
435+ driver = self.random_string() # device not virtio_net
436+ master_devname = self.random_string()
437+ m_sysdev.return_value = "%s/%s" % (self.random_string(),
438+ master_devname)
439+ m_exists.return_value = True # has master sysfs attr
440+ self.m_device_driver.return_value = 'virtio_net' # master virtio_net
441+ m_standby.return_value = False # master has no standby feature flag
442+ self.assertFalse(net.is_netfail_primary(devname, driver))
443+
444+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
445+ @mock.patch('cloudinit.net.os.path.exists')
446+ def test_is_netfail_standby(self, m_exists, m_standby):
447+ devname = self.random_string()
448+ driver = 'virtio_net'
449+ m_exists.return_value = True # has master sysfs attr
450+ m_standby.return_value = True # has standby feature flag
451+ self.assertTrue(net.is_netfail_standby(devname, driver))
452+
453+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
454+ @mock.patch('cloudinit.net.os.path.exists')
455+ def test_is_netfail_standby_wrong_driver(self, m_exists, m_standby):
456+ devname = self.random_string()
457+ driver = self.random_string()
458+ self.assertFalse(net.is_netfail_standby(devname, driver))
459+
460+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
461+ @mock.patch('cloudinit.net.os.path.exists')
462+ def test_is_netfail_standby_no_master(self, m_exists, m_standby):
463+ devname = self.random_string()
464+ driver = 'virtio_net'
465+ m_exists.return_value = False # has master sysfs attr
466+ self.assertFalse(net.is_netfail_standby(devname, driver))
467+
468+ @mock.patch('cloudinit.net.has_netfail_standby_feature')
469+ @mock.patch('cloudinit.net.os.path.exists')
470+ def test_is_netfail_standby_no_standby_feature(self, m_exists, m_standby):
471+ devname = self.random_string()
472+ driver = 'virtio_net'
473+ m_exists.return_value = True # has master sysfs attr
474+ m_standby.return_value = False # has standby feature flag
475+ self.assertFalse(net.is_netfail_standby(devname, driver))
476+
477+ @mock.patch('cloudinit.net.is_netfail_standby')
478+ @mock.patch('cloudinit.net.is_netfail_primary')
479+ def test_is_netfailover_primary(self, m_primary, m_standby):
480+ devname = self.random_string()
481+ driver = self.random_string()
482+ m_primary.return_value = True
483+ m_standby.return_value = False
484+ self.assertTrue(net.is_netfailover(devname, driver))
485+
486+ @mock.patch('cloudinit.net.is_netfail_standby')
487+ @mock.patch('cloudinit.net.is_netfail_primary')
488+ def test_is_netfailover_standby(self, m_primary, m_standby):
489+ devname = self.random_string()
490+ driver = self.random_string()
491+ m_primary.return_value = False
492+ m_standby.return_value = True
493+ self.assertTrue(net.is_netfailover(devname, driver))
494+
495+ @mock.patch('cloudinit.net.is_netfail_standby')
496+ @mock.patch('cloudinit.net.is_netfail_primary')
497+ def test_is_netfailover_returns_false(self, m_primary, m_standby):
498+ devname = self.random_string()
499+ driver = self.random_string()
500+ m_primary.return_value = False
501+ m_standby.return_value = False
502+ self.assertFalse(net.is_netfailover(devname, driver))
503+
504+# vi: ts=4 expandtab
505diff --git a/cloudinit/sources/DataSourceOracle.py b/cloudinit/sources/DataSourceOracle.py
506index 1cb0636..eec8740 100644
507--- a/cloudinit/sources/DataSourceOracle.py
508+++ b/cloudinit/sources/DataSourceOracle.py
509@@ -16,7 +16,7 @@ Notes:
510 """
511
512 from cloudinit.url_helper import combine_url, readurl, UrlError
513-from cloudinit.net import dhcp, get_interfaces_by_mac
514+from cloudinit.net import dhcp, get_interfaces_by_mac, is_netfail_master
515 from cloudinit import net
516 from cloudinit import sources
517 from cloudinit import util
518@@ -108,6 +108,56 @@ def _add_network_config_from_opc_imds(network_config):
519 'match': {'macaddress': mac_address}}
520
521
522+def _ensure_netfailover_safe(network_config):
523+ """
524+ Search network config physical interfaces to see if any of them are
525+ a netfailover master. If found, we prevent matching by MAC as the other
526+ failover devices have the same MAC but need to be ignored.
527+
528+ Note: we rely on cloudinit.net changes which prevent netfailover devices
529+ from being present in the provided network config. For more details about
530+ netfailover devices, refer to cloudinit.net module.
531+
532+ :param network_config
533+ A v1 or v2 network config dict with the primary NIC, and possibly
534+ secondary nic configured. This dict will be mutated.
535+
536+ """
537+ # ignore anything that's not an actual network-config
538+ if 'version' not in network_config:
539+ return
540+
541+ if network_config['version'] not in [1, 2]:
542+ LOG.debug('Ignoring unknown network config version: %s',
543+ network_config['version'])
544+ return
545+
546+ mac_to_name = get_interfaces_by_mac()
547+ if network_config['version'] == 1:
548+ for cfg in [c for c in network_config['config'] if 'type' in c]:
549+ if cfg['type'] == 'physical':
550+ if 'mac_address' in cfg:
551+ mac = cfg['mac_address']
552+ cur_name = mac_to_name.get(mac)
553+ if not cur_name:
554+ continue
555+ elif is_netfail_master(cur_name):
556+ del cfg['mac_address']
557+
558+ elif network_config['version'] == 2:
559+ for _, cfg in network_config.get('ethernets', {}).items():
560+ if 'match' in cfg:
561+ macaddr = cfg.get('match', {}).get('macaddress')
562+ if macaddr:
563+ cur_name = mac_to_name.get(macaddr)
564+ if not cur_name:
565+ continue
566+ elif is_netfail_master(cur_name):
567+ del cfg['match']['macaddress']
568+ del cfg['set-name']
569+ cfg['match']['name'] = cur_name
570+
571+
572 class DataSourceOracle(sources.DataSource):
573
574 dsname = 'Oracle'
575@@ -208,9 +258,13 @@ class DataSourceOracle(sources.DataSource):
576 We nonetheless return cmdline provided config if present
577 and fallback to generate fallback."""
578 if self._network_config == sources.UNSET:
579+ # this is v1
580 self._network_config = cmdline.read_initramfs_config()
581+
582 if not self._network_config:
583+ # this is now v2
584 self._network_config = self.distro.generate_fallback_config()
585+
586 if self.ds_cfg.get('configure_secondary_nics'):
587 try:
588 # Mutate self._network_config to include secondary VNICs
589@@ -219,6 +273,12 @@ class DataSourceOracle(sources.DataSource):
590 util.logexc(
591 LOG,
592 "Failed to fetch secondary network configuration!")
593+
594+ # we need to verify that the nic selected is not a netfail over
595+ # device and, if it is a netfail master, then we need to avoid
596+ # emitting any match by mac
597+ _ensure_netfailover_safe(self._network_config)
598+
599 return self._network_config
600
601
602diff --git a/cloudinit/sources/tests/test_oracle.py b/cloudinit/sources/tests/test_oracle.py
603index 2a70bbc..85b6db9 100644
604--- a/cloudinit/sources/tests/test_oracle.py
605+++ b/cloudinit/sources/tests/test_oracle.py
606@@ -8,6 +8,7 @@ from cloudinit.tests import helpers as test_helpers
607
608 from textwrap import dedent
609 import argparse
610+import copy
611 import httpretty
612 import json
613 import mock
614@@ -586,4 +587,150 @@ class TestNetworkConfigFromOpcImds(test_helpers.CiTestCase):
615 self.assertEqual('10.0.0.231', secondary_nic_cfg['addresses'][0])
616
617
618+class TestNetworkConfigFiltersNetFailover(test_helpers.CiTestCase):
619+
620+ with_logs = True
621+
622+ def setUp(self):
623+ super(TestNetworkConfigFiltersNetFailover, self).setUp()
624+ self.add_patch(DS_PATH + '.get_interfaces_by_mac',
625+ 'm_get_interfaces_by_mac')
626+ self.add_patch(DS_PATH + '.is_netfail_master', 'm_netfail_master')
627+
628+ def test_ignore_bogus_network_config(self):
629+ netcfg = {'something': 'here'}
630+ passed_netcfg = copy.copy(netcfg)
631+ oracle._ensure_netfailover_safe(passed_netcfg)
632+ self.assertEqual(netcfg, passed_netcfg)
633+
634+ def test_ignore_network_config_unknown_versions(self):
635+ netcfg = {'something': 'here', 'version': 3}
636+ passed_netcfg = copy.copy(netcfg)
637+ oracle._ensure_netfailover_safe(passed_netcfg)
638+ self.assertEqual(netcfg, passed_netcfg)
639+
640+ def test_checks_v1_type_physical_interfaces(self):
641+ mac_addr, nic_name = '00:00:17:02:2b:b1', 'ens3'
642+ self.m_get_interfaces_by_mac.return_value = {
643+ mac_addr: nic_name,
644+ }
645+ netcfg = {'version': 1, 'config': [
646+ {'type': 'physical', 'name': nic_name, 'mac_address': mac_addr,
647+ 'subnets': [{'type': 'dhcp4'}]}]}
648+ passed_netcfg = copy.copy(netcfg)
649+ self.m_netfail_master.return_value = False
650+ oracle._ensure_netfailover_safe(passed_netcfg)
651+ self.assertEqual(netcfg, passed_netcfg)
652+ self.assertEqual([mock.call(nic_name)],
653+ self.m_netfail_master.call_args_list)
654+
655+ def test_checks_v1_skips_non_phys_interfaces(self):
656+ mac_addr, nic_name = '00:00:17:02:2b:b1', 'bond0'
657+ self.m_get_interfaces_by_mac.return_value = {
658+ mac_addr: nic_name,
659+ }
660+ netcfg = {'version': 1, 'config': [
661+ {'type': 'bond', 'name': nic_name, 'mac_address': mac_addr,
662+ 'subnets': [{'type': 'dhcp4'}]}]}
663+ passed_netcfg = copy.copy(netcfg)
664+ oracle._ensure_netfailover_safe(passed_netcfg)
665+ self.assertEqual(netcfg, passed_netcfg)
666+ self.assertEqual(0, self.m_netfail_master.call_count)
667+
668+ def test_removes_master_mac_property_v1(self):
669+ nic_master, mac_master = 'ens3', self.random_string()
670+ nic_other, mac_other = 'ens7', self.random_string()
671+ nic_extra, mac_extra = 'enp0s1f2', self.random_string()
672+ self.m_get_interfaces_by_mac.return_value = {
673+ mac_master: nic_master,
674+ mac_other: nic_other,
675+ mac_extra: nic_extra,
676+ }
677+ netcfg = {'version': 1, 'config': [
678+ {'type': 'physical', 'name': nic_master,
679+ 'mac_address': mac_master},
680+ {'type': 'physical', 'name': nic_other, 'mac_address': mac_other},
681+ {'type': 'physical', 'name': nic_extra, 'mac_address': mac_extra},
682+ ]}
683+
684+ def _is_netfail_master(iface):
685+ if iface == 'ens3':
686+ return True
687+ return False
688+ self.m_netfail_master.side_effect = _is_netfail_master
689+ expected_cfg = {'version': 1, 'config': [
690+ {'type': 'physical', 'name': nic_master},
691+ {'type': 'physical', 'name': nic_other, 'mac_address': mac_other},
692+ {'type': 'physical', 'name': nic_extra, 'mac_address': mac_extra},
693+ ]}
694+ oracle._ensure_netfailover_safe(netcfg)
695+ self.assertEqual(expected_cfg, netcfg)
696+
697+ def test_checks_v2_type_ethernet_interfaces(self):
698+ mac_addr, nic_name = '00:00:17:02:2b:b1', 'ens3'
699+ self.m_get_interfaces_by_mac.return_value = {
700+ mac_addr: nic_name,
701+ }
702+ netcfg = {'version': 2, 'ethernets': {
703+ nic_name: {'dhcp4': True, 'critical': True, 'set-name': nic_name,
704+ 'match': {'macaddress': mac_addr}}}}
705+ passed_netcfg = copy.copy(netcfg)
706+ self.m_netfail_master.return_value = False
707+ oracle._ensure_netfailover_safe(passed_netcfg)
708+ self.assertEqual(netcfg, passed_netcfg)
709+ self.assertEqual([mock.call(nic_name)],
710+ self.m_netfail_master.call_args_list)
711+
712+ def test_skips_v2_non_ethernet_interfaces(self):
713+ mac_addr, nic_name = '00:00:17:02:2b:b1', 'wlps0'
714+ self.m_get_interfaces_by_mac.return_value = {
715+ mac_addr: nic_name,
716+ }
717+ netcfg = {'version': 2, 'wifis': {
718+ nic_name: {'dhcp4': True, 'critical': True, 'set-name': nic_name,
719+ 'match': {'macaddress': mac_addr}}}}
720+ passed_netcfg = copy.copy(netcfg)
721+ oracle._ensure_netfailover_safe(passed_netcfg)
722+ self.assertEqual(netcfg, passed_netcfg)
723+ self.assertEqual(0, self.m_netfail_master.call_count)
724+
725+ def test_removes_master_mac_property_v2(self):
726+ nic_master, mac_master = 'ens3', self.random_string()
727+ nic_other, mac_other = 'ens7', self.random_string()
728+ nic_extra, mac_extra = 'enp0s1f2', self.random_string()
729+ self.m_get_interfaces_by_mac.return_value = {
730+ mac_master: nic_master,
731+ mac_other: nic_other,
732+ mac_extra: nic_extra,
733+ }
734+ netcfg = {'version': 2, 'ethernets': {
735+ nic_extra: {'dhcp4': True, 'set-name': nic_extra,
736+ 'match': {'macaddress': mac_extra}},
737+ nic_other: {'dhcp4': True, 'set-name': nic_other,
738+ 'match': {'macaddress': mac_other}},
739+ nic_master: {'dhcp4': True, 'set-name': nic_master,
740+ 'match': {'macaddress': mac_master}},
741+ }}
742+
743+ def _is_netfail_master(iface):
744+ if iface == 'ens3':
745+ return True
746+ return False
747+ self.m_netfail_master.side_effect = _is_netfail_master
748+
749+ expected_cfg = {'version': 2, 'ethernets': {
750+ nic_master: {'dhcp4': True, 'match': {'name': nic_master}},
751+ nic_extra: {'dhcp4': True, 'set-name': nic_extra,
752+ 'match': {'macaddress': mac_extra}},
753+ nic_other: {'dhcp4': True, 'set-name': nic_other,
754+ 'match': {'macaddress': mac_other}},
755+ }}
756+ oracle._ensure_netfailover_safe(netcfg)
757+ import pprint
758+ pprint.pprint(netcfg)
759+ print('---- ^^ modified ^^ ---- vv original vv ----')
760+ pprint.pprint(expected_cfg)
761+ self.assertEqual(expected_cfg, netcfg)
762+
763+
764 # vi: ts=4 expandtab
765diff --git a/cloudinit/tests/helpers.py b/cloudinit/tests/helpers.py
766index 23fddd0..4dad2af 100644
767--- a/cloudinit/tests/helpers.py
768+++ b/cloudinit/tests/helpers.py
769@@ -6,7 +6,9 @@ import functools
770 import httpretty
771 import logging
772 import os
773+import random
774 import shutil
775+import string
776 import sys
777 import tempfile
778 import time
779@@ -243,6 +245,12 @@ class CiTestCase(TestCase):
780 myds.metadata.update(metadata)
781 return cloud.Cloud(myds, self.paths, sys_cfg, mydist, None)
782
783+ @classmethod
784+ def random_string(cls, length=8):
785+ """ return a random lowercase string with default length of 8"""
786+ return ''.join(
787+ random.choice(string.ascii_lowercase) for _ in range(length))
788+
789
790 class ResourceUsingTestCase(CiTestCase):
791

Subscribers

People subscribed via source and target branches