Merge ~raharper/cloud-init:feature/detect-netfailover into cloud-init:master

Proposed by Ryan Harper
Status: Merged
Approved by: Ryan Harper
Approved revision: 5031c2a1c15f1146a42021d357b556b8b1f1520b
Merge reported by: Server Team CI bot
Merged at revision: not available
Proposed branch: ~raharper/cloud-init:feature/detect-netfailover
Merge into: cloud-init:master
Diff against target: 790 lines (+656/-4)
5 files modified
cloudinit/net/__init__.py (+130/-3)
cloudinit/net/tests/test_init.py (+310/-0)
cloudinit/sources/DataSourceOracle.py (+61/-1)
cloudinit/sources/tests/test_oracle.py (+147/-0)
cloudinit/tests/helpers.py (+8/-0)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
Ryan Harper Approve
Chad Smith Approve
Review via email: mp+371895@code.launchpad.net

Commit message

net,Oracle: Add support for netfailover detection

Add support for detecting netfailover[1] device 3-tuple in networking
layer. In the Oracle datasource ensure that if a provided network
config, either fallback or provided config includes a netfailover master
to remove any MAC address value as this can break under 3-netdev
as the other two devices have the same MAC.

1. https://www.kernel.org/doc/html/latest/networking/net_failover.html

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:73af8a87bb5ab0e72013ed44de88dae1ac8678a2
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1080/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1080//rebuild

review: Approve (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

Nice, Ryan.

  I tried to give an alternative to allow for consolidation of all is_netfailover* functions into a single is_netfailover_device. See what you think.

Revision history for this message
Chad Smith (chad.smith) :
review: Needs Information
Revision history for this message
Ryan Harper (raharper) :
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:d09ad7e55b2bfcf1a7683529be9686cc84c01b50
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1103/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1103//rebuild

review: Approve (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

Thank you for the resolution here. I agree that the single function is a lot less readable than the separate is_netfail_* functions. LGTM!

review: Approve
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/306/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/307/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/308/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/310/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/313/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/314/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/315/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/317/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/319/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/320/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/321/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/322/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/323/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) :
review: Approve (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

Autolanding: FAILED
Unapproved changes made after approval.
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/324/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

review: Needs Fixing (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

I've force pushed a fix for the pycodestyle change after merge, so should be good now.

Revision history for this message
Ryan Harper (raharper) wrote :

Approving

review: Approve
Revision history for this message
Server Team CI bot (server-team-bot) :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
index ea707c0..0eb952f 100644
--- a/cloudinit/net/__init__.py
+++ b/cloudinit/net/__init__.py
@@ -109,6 +109,123 @@ def is_bond(devname):
109 return os.path.exists(sys_dev_path(devname, "bonding"))109 return os.path.exists(sys_dev_path(devname, "bonding"))
110110
111111
112def is_netfailover(devname, driver=None):
113 """ netfailover driver uses 3 nics, master, primary and standby.
114 this returns True if the device is either the primary or standby
115 as these devices are to be ignored.
116 """
117 if driver is None:
118 driver = device_driver(devname)
119 if is_netfail_primary(devname, driver) or is_netfail_standby(devname,
120 driver):
121 return True
122 return False
123
124
125def get_dev_features(devname):
126 """ Returns a str from reading /sys/class/net/<devname>/device/features."""
127 features = ''
128 try:
129 features = read_sys_net(devname, 'device/features')
130 except Exception:
131 pass
132 return features
133
134
135def has_netfail_standby_feature(devname):
136 """ Return True if VIRTIO_NET_F_STANDBY bit (62) is set.
137
138 https://github.com/torvalds/linux/blob/ \
139 089cf7f6ecb266b6a4164919a2e69bd2f938374a/ \
140 include/uapi/linux/virtio_net.h#L60
141 """
142 features = get_dev_features(devname)
143 if not features or len(features) < 64:
144 return False
145 return features[62] == "1"
146
147
148def is_netfail_master(devname, driver=None):
149 """ A device is a "netfail master" device if:
150
151 - The device does NOT have the 'master' sysfs attribute
152 - The device driver is 'virtio_net'
153 - The device has the standby feature bit set
154
155 Return True if all of the above is True.
156 """
157 if os.path.exists(sys_dev_path(devname, path='master')):
158 return False
159
160 if driver is None:
161 driver = device_driver(devname)
162
163 if driver != "virtio_net":
164 return False
165
166 if not has_netfail_standby_feature(devname):
167 return False
168
169 return True
170
171
172def is_netfail_primary(devname, driver=None):
173 """ A device is a "netfail primary" device if:
174
175 - the device has a 'master' sysfs file
176 - the device driver is not 'virtio_net'
177 - the 'master' sysfs file points to device with virtio_net driver
178 - the 'master' device has the 'standby' feature bit set
179
180 Return True if all of the above is True.
181 """
182 # /sys/class/net/<devname>/master -> ../../<master devname>
183 master_sysfs_path = sys_dev_path(devname, path='master')
184 if not os.path.exists(master_sysfs_path):
185 return False
186
187 if driver is None:
188 driver = device_driver(devname)
189
190 if driver == "virtio_net":
191 return False
192
193 master_devname = os.path.basename(os.path.realpath(master_sysfs_path))
194 master_driver = device_driver(master_devname)
195 if master_driver != "virtio_net":
196 return False
197
198 master_has_standby = has_netfail_standby_feature(master_devname)
199 if not master_has_standby:
200 return False
201
202 return True
203
204
205def is_netfail_standby(devname, driver=None):
206 """ A device is a "netfail standby" device if:
207
208 - The device has a 'master' sysfs attribute
209 - The device driver is 'virtio_net'
210 - The device has the standby feature bit set
211
212 Return True if all of the above is True.
213 """
214 if not os.path.exists(sys_dev_path(devname, path='master')):
215 return False
216
217 if driver is None:
218 driver = device_driver(devname)
219
220 if driver != "virtio_net":
221 return False
222
223 if not has_netfail_standby_feature(devname):
224 return False
225
226 return True
227
228
112def is_renamed(devname):229def is_renamed(devname):
113 """230 """
114 /* interface name assignment types (sysfs name_assign_type attribute) */231 /* interface name assignment types (sysfs name_assign_type attribute) */
@@ -227,6 +344,9 @@ def find_fallback_nic(blacklist_drivers=None):
227 if is_bond(interface):344 if is_bond(interface):
228 # skip any bonds345 # skip any bonds
229 continue346 continue
347 if is_netfailover(interface):
348 # ignore netfailover primary/standby interfaces
349 continue
230 carrier = read_sys_net_int(interface, 'carrier')350 carrier = read_sys_net_int(interface, 'carrier')
231 if carrier:351 if carrier:
232 connected.append(interface)352 connected.append(interface)
@@ -273,9 +393,14 @@ def generate_fallback_config(blacklist_drivers=None, config_driver=None):
273 if not target_name:393 if not target_name:
274 # can't read any interfaces addresses (or there are none); give up394 # can't read any interfaces addresses (or there are none); give up
275 return None395 return None
276 target_mac = read_sys_net_safe(target_name, 'address')396
277 cfg = {'dhcp4': True, 'set-name': target_name,397 # netfail cannot use mac for matching, they have duplicate macs
278 'match': {'macaddress': target_mac.lower()}}398 if is_netfail_master(target_name):
399 match = {'name': target_name}
400 else:
401 match = {
402 'macaddress': read_sys_net_safe(target_name, 'address').lower()}
403 cfg = {'dhcp4': True, 'set-name': target_name, 'match': match}
279 if config_driver:404 if config_driver:
280 driver = device_driver(target_name)405 driver = device_driver(target_name)
281 if driver:406 if driver:
@@ -661,6 +786,8 @@ def get_interfaces():
661 continue786 continue
662 if is_bond(name):787 if is_bond(name):
663 continue788 continue
789 if is_netfailover(name):
790 continue
664 mac = get_interface_mac(name)791 mac = get_interface_mac(name)
665 # some devices may not have a mac (tun0)792 # some devices may not have a mac (tun0)
666 if not mac:793 if not mac:
diff --git a/cloudinit/net/tests/test_init.py b/cloudinit/net/tests/test_init.py
index d2e38f0..7259dbe 100644
--- a/cloudinit/net/tests/test_init.py
+++ b/cloudinit/net/tests/test_init.py
@@ -204,6 +204,10 @@ class TestGenerateFallbackConfig(CiTestCase):
204 self.add_patch('cloudinit.net.util.is_container', 'm_is_container',204 self.add_patch('cloudinit.net.util.is_container', 'm_is_container',
205 return_value=False)205 return_value=False)
206 self.add_patch('cloudinit.net.util.udevadm_settle', 'm_settle')206 self.add_patch('cloudinit.net.util.udevadm_settle', 'm_settle')
207 self.add_patch('cloudinit.net.is_netfailover', 'm_netfail',
208 return_value=False)
209 self.add_patch('cloudinit.net.is_netfail_master', 'm_netfail_master',
210 return_value=False)
207211
208 def test_generate_fallback_finds_connected_eth_with_mac(self):212 def test_generate_fallback_finds_connected_eth_with_mac(self):
209 """generate_fallback_config finds any connected device with a mac."""213 """generate_fallback_config finds any connected device with a mac."""
@@ -268,6 +272,61 @@ class TestGenerateFallbackConfig(CiTestCase):
268 ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))272 ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))
269 self.assertIsNone(net.generate_fallback_config())273 self.assertIsNone(net.generate_fallback_config())
270274
275 def test_generate_fallback_config_skips_netfail_devs(self):
276 """gen_fallback_config ignores netfail primary,sby no mac on master."""
277 mac = 'aa:bb:cc:aa:bb:cc' # netfailover devs share the same mac
278 for iface in ['ens3', 'ens3sby', 'enP0s1f3']:
279 write_file(os.path.join(self.sysdir, iface, 'carrier'), '1')
280 write_file(
281 os.path.join(self.sysdir, iface, 'addr_assign_type'), '0')
282 write_file(
283 os.path.join(self.sysdir, iface, 'address'), mac)
284
285 def is_netfail(iface, _driver=None):
286 # ens3 is the master
287 if iface == 'ens3':
288 return False
289 return True
290 self.m_netfail.side_effect = is_netfail
291
292 def is_netfail_master(iface, _driver=None):
293 # ens3 is the master
294 if iface == 'ens3':
295 return True
296 return False
297 self.m_netfail_master.side_effect = is_netfail_master
298 expected = {
299 'ethernets': {
300 'ens3': {'dhcp4': True, 'match': {'name': 'ens3'},
301 'set-name': 'ens3'}},
302 'version': 2}
303 result = net.generate_fallback_config()
304 self.assertEqual(expected, result)
305
306
307class TestNetFindFallBackNic(CiTestCase):
308
309 with_logs = True
310
311 def setUp(self):
312 super(TestNetFindFallBackNic, self).setUp()
313 sys_mock = mock.patch('cloudinit.net.get_sys_class_path')
314 self.m_sys_path = sys_mock.start()
315 self.sysdir = self.tmp_dir() + '/'
316 self.m_sys_path.return_value = self.sysdir
317 self.addCleanup(sys_mock.stop)
318 self.add_patch('cloudinit.net.util.is_container', 'm_is_container',
319 return_value=False)
320 self.add_patch('cloudinit.net.util.udevadm_settle', 'm_settle')
321
322 def test_generate_fallback_finds_first_connected_eth_with_mac(self):
323 """find_fallback_nic finds any connected device with a mac."""
324 write_file(os.path.join(self.sysdir, 'eth0', 'carrier'), '1')
325 write_file(os.path.join(self.sysdir, 'eth1', 'carrier'), '1')
326 mac = 'aa:bb:cc:aa:bb:cc'
327 write_file(os.path.join(self.sysdir, 'eth1', 'address'), mac)
328 self.assertEqual('eth1', net.find_fallback_nic())
329
271330
272class TestGetDeviceList(CiTestCase):331class TestGetDeviceList(CiTestCase):
273332
@@ -365,6 +424,26 @@ class TestGetInterfaceMAC(CiTestCase):
365 expected = [('eth2', 'aa:bb:cc:aa:bb:cc', None, None)]424 expected = [('eth2', 'aa:bb:cc:aa:bb:cc', None, None)]
366 self.assertEqual(expected, net.get_interfaces())425 self.assertEqual(expected, net.get_interfaces())
367426
427 @mock.patch('cloudinit.net.is_netfailover')
428 def test_get_interfaces_by_mac_skips_netfailvoer(self, m_netfail):
429 """Ignore interfaces if netfailover primary or standby."""
430 mac = 'aa:bb:cc:aa:bb:cc' # netfailover devs share the same mac
431 for iface in ['ens3', 'ens3sby', 'enP0s1f3']:
432 write_file(
433 os.path.join(self.sysdir, iface, 'addr_assign_type'), '0')
434 write_file(
435 os.path.join(self.sysdir, iface, 'address'), mac)
436
437 def is_netfail(iface, _driver=None):
438 # ens3 is the master
439 if iface == 'ens3':
440 return False
441 else:
442 return True
443 m_netfail.side_effect = is_netfail
444 expected = [('ens3', mac, None, None)]
445 self.assertEqual(expected, net.get_interfaces())
446
368447
369class TestInterfaceHasOwnMAC(CiTestCase):448class TestInterfaceHasOwnMAC(CiTestCase):
370449
@@ -922,3 +1001,234 @@ class TestWaitForPhysdevs(CiTestCase):
922 self.m_get_iface_mac.return_value = {}1001 self.m_get_iface_mac.return_value = {}
923 net.wait_for_physdevs(netcfg, strict=False)1002 net.wait_for_physdevs(netcfg, strict=False)
924 self.assertEqual(5 * len(physdevs), self.m_udev_settle.call_count)1003 self.assertEqual(5 * len(physdevs), self.m_udev_settle.call_count)
1004
1005
1006class TestNetFailOver(CiTestCase):
1007
1008 with_logs = True
1009
1010 def setUp(self):
1011 super(TestNetFailOver, self).setUp()
1012 self.add_patch('cloudinit.net.util', 'm_util')
1013 self.add_patch('cloudinit.net.read_sys_net', 'm_read_sys_net')
1014 self.add_patch('cloudinit.net.device_driver', 'm_device_driver')
1015
1016 def test_get_dev_features(self):
1017 devname = self.random_string()
1018 features = self.random_string()
1019 self.m_read_sys_net.return_value = features
1020
1021 self.assertEqual(features, net.get_dev_features(devname))
1022 self.assertEqual(1, self.m_read_sys_net.call_count)
1023 self.assertEqual(mock.call(devname, 'device/features'),
1024 self.m_read_sys_net.call_args_list[0])
1025
1026 def test_get_dev_features_none_returns_empty_string(self):
1027 devname = self.random_string()
1028 self.m_read_sys_net.side_effect = Exception('error')
1029 self.assertEqual('', net.get_dev_features(devname))
1030 self.assertEqual(1, self.m_read_sys_net.call_count)
1031 self.assertEqual(mock.call(devname, 'device/features'),
1032 self.m_read_sys_net.call_args_list[0])
1033
1034 @mock.patch('cloudinit.net.get_dev_features')
1035 def test_has_netfail_standby_feature(self, m_dev_features):
1036 devname = self.random_string()
1037 standby_features = ('0' * 62) + '1' + '0'
1038 m_dev_features.return_value = standby_features
1039 self.assertTrue(net.has_netfail_standby_feature(devname))
1040
1041 @mock.patch('cloudinit.net.get_dev_features')
1042 def test_has_netfail_standby_feature_short_is_false(self, m_dev_features):
1043 devname = self.random_string()
1044 standby_features = self.random_string()
1045 m_dev_features.return_value = standby_features
1046 self.assertFalse(net.has_netfail_standby_feature(devname))
1047
1048 @mock.patch('cloudinit.net.get_dev_features')
1049 def test_has_netfail_standby_feature_not_present_is_false(self,
1050 m_dev_features):
1051 devname = self.random_string()
1052 standby_features = '0' * 64
1053 m_dev_features.return_value = standby_features
1054 self.assertFalse(net.has_netfail_standby_feature(devname))
1055
1056 @mock.patch('cloudinit.net.get_dev_features')
1057 def test_has_netfail_standby_feature_no_features_is_false(self,
1058 m_dev_features):
1059 devname = self.random_string()
1060 standby_features = None
1061 m_dev_features.return_value = standby_features
1062 self.assertFalse(net.has_netfail_standby_feature(devname))
1063
1064 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1065 @mock.patch('cloudinit.net.os.path.exists')
1066 def test_is_netfail_master(self, m_exists, m_standby):
1067 devname = self.random_string()
1068 driver = 'virtio_net'
1069 m_exists.return_value = False # no master sysfs attr
1070 m_standby.return_value = True # has standby feature flag
1071 self.assertTrue(net.is_netfail_master(devname, driver))
1072
1073 @mock.patch('cloudinit.net.sys_dev_path')
1074 def test_is_netfail_master_checks_master_attr(self, m_sysdev):
1075 devname = self.random_string()
1076 driver = 'virtio_net'
1077 m_sysdev.return_value = self.random_string()
1078 self.assertFalse(net.is_netfail_master(devname, driver))
1079 self.assertEqual(1, m_sysdev.call_count)
1080 self.assertEqual(mock.call(devname, path='master'),
1081 m_sysdev.call_args_list[0])
1082
1083 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1084 @mock.patch('cloudinit.net.os.path.exists')
1085 def test_is_netfail_master_wrong_driver(self, m_exists, m_standby):
1086 devname = self.random_string()
1087 driver = self.random_string()
1088 self.assertFalse(net.is_netfail_master(devname, driver))
1089
1090 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1091 @mock.patch('cloudinit.net.os.path.exists')
1092 def test_is_netfail_master_has_master_attr(self, m_exists, m_standby):
1093 devname = self.random_string()
1094 driver = 'virtio_net'
1095 m_exists.return_value = True # has master sysfs attr
1096 self.assertFalse(net.is_netfail_master(devname, driver))
1097
1098 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1099 @mock.patch('cloudinit.net.os.path.exists')
1100 def test_is_netfail_master_no_standby_feat(self, m_exists, m_standby):
1101 devname = self.random_string()
1102 driver = 'virtio_net'
1103 m_exists.return_value = False # no master sysfs attr
1104 m_standby.return_value = False # no standby feature flag
1105 self.assertFalse(net.is_netfail_master(devname, driver))
1106
1107 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1108 @mock.patch('cloudinit.net.os.path.exists')
1109 @mock.patch('cloudinit.net.sys_dev_path')
1110 def test_is_netfail_primary(self, m_sysdev, m_exists, m_standby):
1111 devname = self.random_string()
1112 driver = self.random_string() # device not virtio_net
1113 master_devname = self.random_string()
1114 m_sysdev.return_value = "%s/%s" % (self.random_string(),
1115 master_devname)
1116 m_exists.return_value = True # has master sysfs attr
1117 self.m_device_driver.return_value = 'virtio_net' # master virtio_net
1118 m_standby.return_value = True # has standby feature flag
1119 self.assertTrue(net.is_netfail_primary(devname, driver))
1120 self.assertEqual(1, self.m_device_driver.call_count)
1121 self.assertEqual(mock.call(master_devname),
1122 self.m_device_driver.call_args_list[0])
1123 self.assertEqual(1, m_standby.call_count)
1124 self.assertEqual(mock.call(master_devname),
1125 m_standby.call_args_list[0])
1126
1127 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1128 @mock.patch('cloudinit.net.os.path.exists')
1129 @mock.patch('cloudinit.net.sys_dev_path')
1130 def test_is_netfail_primary_wrong_driver(self, m_sysdev, m_exists,
1131 m_standby):
1132 devname = self.random_string()
1133 driver = 'virtio_net'
1134 self.assertFalse(net.is_netfail_primary(devname, driver))
1135
1136 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1137 @mock.patch('cloudinit.net.os.path.exists')
1138 @mock.patch('cloudinit.net.sys_dev_path')
1139 def test_is_netfail_primary_no_master(self, m_sysdev, m_exists, m_standby):
1140 devname = self.random_string()
1141 driver = self.random_string() # device not virtio_net
1142 m_exists.return_value = False # no master sysfs attr
1143 self.assertFalse(net.is_netfail_primary(devname, driver))
1144
1145 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1146 @mock.patch('cloudinit.net.os.path.exists')
1147 @mock.patch('cloudinit.net.sys_dev_path')
1148 def test_is_netfail_primary_bad_master(self, m_sysdev, m_exists,
1149 m_standby):
1150 devname = self.random_string()
1151 driver = self.random_string() # device not virtio_net
1152 master_devname = self.random_string()
1153 m_sysdev.return_value = "%s/%s" % (self.random_string(),
1154 master_devname)
1155 m_exists.return_value = True # has master sysfs attr
1156 self.m_device_driver.return_value = 'XXXX' # master not virtio_net
1157 self.assertFalse(net.is_netfail_primary(devname, driver))
1158
1159 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1160 @mock.patch('cloudinit.net.os.path.exists')
1161 @mock.patch('cloudinit.net.sys_dev_path')
1162 def test_is_netfail_primary_no_standby(self, m_sysdev, m_exists,
1163 m_standby):
1164 devname = self.random_string()
1165 driver = self.random_string() # device not virtio_net
1166 master_devname = self.random_string()
1167 m_sysdev.return_value = "%s/%s" % (self.random_string(),
1168 master_devname)
1169 m_exists.return_value = True # has master sysfs attr
1170 self.m_device_driver.return_value = 'virtio_net' # master virtio_net
1171 m_standby.return_value = False # master has no standby feature flag
1172 self.assertFalse(net.is_netfail_primary(devname, driver))
1173
1174 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1175 @mock.patch('cloudinit.net.os.path.exists')
1176 def test_is_netfail_standby(self, m_exists, m_standby):
1177 devname = self.random_string()
1178 driver = 'virtio_net'
1179 m_exists.return_value = True # has master sysfs attr
1180 m_standby.return_value = True # has standby feature flag
1181 self.assertTrue(net.is_netfail_standby(devname, driver))
1182
1183 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1184 @mock.patch('cloudinit.net.os.path.exists')
1185 def test_is_netfail_standby_wrong_driver(self, m_exists, m_standby):
1186 devname = self.random_string()
1187 driver = self.random_string()
1188 self.assertFalse(net.is_netfail_standby(devname, driver))
1189
1190 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1191 @mock.patch('cloudinit.net.os.path.exists')
1192 def test_is_netfail_standby_no_master(self, m_exists, m_standby):
1193 devname = self.random_string()
1194 driver = 'virtio_net'
1195 m_exists.return_value = False # has master sysfs attr
1196 self.assertFalse(net.is_netfail_standby(devname, driver))
1197
1198 @mock.patch('cloudinit.net.has_netfail_standby_feature')
1199 @mock.patch('cloudinit.net.os.path.exists')
1200 def test_is_netfail_standby_no_standby_feature(self, m_exists, m_standby):
1201 devname = self.random_string()
1202 driver = 'virtio_net'
1203 m_exists.return_value = True # has master sysfs attr
1204 m_standby.return_value = False # has standby feature flag
1205 self.assertFalse(net.is_netfail_standby(devname, driver))
1206
1207 @mock.patch('cloudinit.net.is_netfail_standby')
1208 @mock.patch('cloudinit.net.is_netfail_primary')
1209 def test_is_netfailover_primary(self, m_primary, m_standby):
1210 devname = self.random_string()
1211 driver = self.random_string()
1212 m_primary.return_value = True
1213 m_standby.return_value = False
1214 self.assertTrue(net.is_netfailover(devname, driver))
1215
1216 @mock.patch('cloudinit.net.is_netfail_standby')
1217 @mock.patch('cloudinit.net.is_netfail_primary')
1218 def test_is_netfailover_standby(self, m_primary, m_standby):
1219 devname = self.random_string()
1220 driver = self.random_string()
1221 m_primary.return_value = False
1222 m_standby.return_value = True
1223 self.assertTrue(net.is_netfailover(devname, driver))
1224
1225 @mock.patch('cloudinit.net.is_netfail_standby')
1226 @mock.patch('cloudinit.net.is_netfail_primary')
1227 def test_is_netfailover_returns_false(self, m_primary, m_standby):
1228 devname = self.random_string()
1229 driver = self.random_string()
1230 m_primary.return_value = False
1231 m_standby.return_value = False
1232 self.assertFalse(net.is_netfailover(devname, driver))
1233
1234# vi: ts=4 expandtab
diff --git a/cloudinit/sources/DataSourceOracle.py b/cloudinit/sources/DataSourceOracle.py
index 1cb0636..eec8740 100644
--- a/cloudinit/sources/DataSourceOracle.py
+++ b/cloudinit/sources/DataSourceOracle.py
@@ -16,7 +16,7 @@ Notes:
16"""16"""
1717
18from cloudinit.url_helper import combine_url, readurl, UrlError18from cloudinit.url_helper import combine_url, readurl, UrlError
19from cloudinit.net import dhcp, get_interfaces_by_mac19from cloudinit.net import dhcp, get_interfaces_by_mac, is_netfail_master
20from cloudinit import net20from cloudinit import net
21from cloudinit import sources21from cloudinit import sources
22from cloudinit import util22from cloudinit import util
@@ -108,6 +108,56 @@ def _add_network_config_from_opc_imds(network_config):
108 'match': {'macaddress': mac_address}}108 'match': {'macaddress': mac_address}}
109109
110110
111def _ensure_netfailover_safe(network_config):
112 """
113 Search network config physical interfaces to see if any of them are
114 a netfailover master. If found, we prevent matching by MAC as the other
115 failover devices have the same MAC but need to be ignored.
116
117 Note: we rely on cloudinit.net changes which prevent netfailover devices
118 from being present in the provided network config. For more details about
119 netfailover devices, refer to cloudinit.net module.
120
121 :param network_config
122 A v1 or v2 network config dict with the primary NIC, and possibly
123 secondary nic configured. This dict will be mutated.
124
125 """
126 # ignore anything that's not an actual network-config
127 if 'version' not in network_config:
128 return
129
130 if network_config['version'] not in [1, 2]:
131 LOG.debug('Ignoring unknown network config version: %s',
132 network_config['version'])
133 return
134
135 mac_to_name = get_interfaces_by_mac()
136 if network_config['version'] == 1:
137 for cfg in [c for c in network_config['config'] if 'type' in c]:
138 if cfg['type'] == 'physical':
139 if 'mac_address' in cfg:
140 mac = cfg['mac_address']
141 cur_name = mac_to_name.get(mac)
142 if not cur_name:
143 continue
144 elif is_netfail_master(cur_name):
145 del cfg['mac_address']
146
147 elif network_config['version'] == 2:
148 for _, cfg in network_config.get('ethernets', {}).items():
149 if 'match' in cfg:
150 macaddr = cfg.get('match', {}).get('macaddress')
151 if macaddr:
152 cur_name = mac_to_name.get(macaddr)
153 if not cur_name:
154 continue
155 elif is_netfail_master(cur_name):
156 del cfg['match']['macaddress']
157 del cfg['set-name']
158 cfg['match']['name'] = cur_name
159
160
111class DataSourceOracle(sources.DataSource):161class DataSourceOracle(sources.DataSource):
112162
113 dsname = 'Oracle'163 dsname = 'Oracle'
@@ -208,9 +258,13 @@ class DataSourceOracle(sources.DataSource):
208 We nonetheless return cmdline provided config if present258 We nonetheless return cmdline provided config if present
209 and fallback to generate fallback."""259 and fallback to generate fallback."""
210 if self._network_config == sources.UNSET:260 if self._network_config == sources.UNSET:
261 # this is v1
211 self._network_config = cmdline.read_initramfs_config()262 self._network_config = cmdline.read_initramfs_config()
263
212 if not self._network_config:264 if not self._network_config:
265 # this is now v2
213 self._network_config = self.distro.generate_fallback_config()266 self._network_config = self.distro.generate_fallback_config()
267
214 if self.ds_cfg.get('configure_secondary_nics'):268 if self.ds_cfg.get('configure_secondary_nics'):
215 try:269 try:
216 # Mutate self._network_config to include secondary VNICs270 # Mutate self._network_config to include secondary VNICs
@@ -219,6 +273,12 @@ class DataSourceOracle(sources.DataSource):
219 util.logexc(273 util.logexc(
220 LOG,274 LOG,
221 "Failed to fetch secondary network configuration!")275 "Failed to fetch secondary network configuration!")
276
277 # we need to verify that the nic selected is not a netfail over
278 # device and, if it is a netfail master, then we need to avoid
279 # emitting any match by mac
280 _ensure_netfailover_safe(self._network_config)
281
222 return self._network_config282 return self._network_config
223283
224284
diff --git a/cloudinit/sources/tests/test_oracle.py b/cloudinit/sources/tests/test_oracle.py
index 2a70bbc..85b6db9 100644
--- a/cloudinit/sources/tests/test_oracle.py
+++ b/cloudinit/sources/tests/test_oracle.py
@@ -8,6 +8,7 @@ from cloudinit.tests import helpers as test_helpers
88
9from textwrap import dedent9from textwrap import dedent
10import argparse10import argparse
11import copy
11import httpretty12import httpretty
12import json13import json
13import mock14import mock
@@ -586,4 +587,150 @@ class TestNetworkConfigFromOpcImds(test_helpers.CiTestCase):
586 self.assertEqual('10.0.0.231', secondary_nic_cfg['addresses'][0])587 self.assertEqual('10.0.0.231', secondary_nic_cfg['addresses'][0])
587588
588589
590class TestNetworkConfigFiltersNetFailover(test_helpers.CiTestCase):
591
592 with_logs = True
593
594 def setUp(self):
595 super(TestNetworkConfigFiltersNetFailover, self).setUp()
596 self.add_patch(DS_PATH + '.get_interfaces_by_mac',
597 'm_get_interfaces_by_mac')
598 self.add_patch(DS_PATH + '.is_netfail_master', 'm_netfail_master')
599
600 def test_ignore_bogus_network_config(self):
601 netcfg = {'something': 'here'}
602 passed_netcfg = copy.copy(netcfg)
603 oracle._ensure_netfailover_safe(passed_netcfg)
604 self.assertEqual(netcfg, passed_netcfg)
605
606 def test_ignore_network_config_unknown_versions(self):
607 netcfg = {'something': 'here', 'version': 3}
608 passed_netcfg = copy.copy(netcfg)
609 oracle._ensure_netfailover_safe(passed_netcfg)
610 self.assertEqual(netcfg, passed_netcfg)
611
612 def test_checks_v1_type_physical_interfaces(self):
613 mac_addr, nic_name = '00:00:17:02:2b:b1', 'ens3'
614 self.m_get_interfaces_by_mac.return_value = {
615 mac_addr: nic_name,
616 }
617 netcfg = {'version': 1, 'config': [
618 {'type': 'physical', 'name': nic_name, 'mac_address': mac_addr,
619 'subnets': [{'type': 'dhcp4'}]}]}
620 passed_netcfg = copy.copy(netcfg)
621 self.m_netfail_master.return_value = False
622 oracle._ensure_netfailover_safe(passed_netcfg)
623 self.assertEqual(netcfg, passed_netcfg)
624 self.assertEqual([mock.call(nic_name)],
625 self.m_netfail_master.call_args_list)
626
627 def test_checks_v1_skips_non_phys_interfaces(self):
628 mac_addr, nic_name = '00:00:17:02:2b:b1', 'bond0'
629 self.m_get_interfaces_by_mac.return_value = {
630 mac_addr: nic_name,
631 }
632 netcfg = {'version': 1, 'config': [
633 {'type': 'bond', 'name': nic_name, 'mac_address': mac_addr,
634 'subnets': [{'type': 'dhcp4'}]}]}
635 passed_netcfg = copy.copy(netcfg)
636 oracle._ensure_netfailover_safe(passed_netcfg)
637 self.assertEqual(netcfg, passed_netcfg)
638 self.assertEqual(0, self.m_netfail_master.call_count)
639
640 def test_removes_master_mac_property_v1(self):
641 nic_master, mac_master = 'ens3', self.random_string()
642 nic_other, mac_other = 'ens7', self.random_string()
643 nic_extra, mac_extra = 'enp0s1f2', self.random_string()
644 self.m_get_interfaces_by_mac.return_value = {
645 mac_master: nic_master,
646 mac_other: nic_other,
647 mac_extra: nic_extra,
648 }
649 netcfg = {'version': 1, 'config': [
650 {'type': 'physical', 'name': nic_master,
651 'mac_address': mac_master},
652 {'type': 'physical', 'name': nic_other, 'mac_address': mac_other},
653 {'type': 'physical', 'name': nic_extra, 'mac_address': mac_extra},
654 ]}
655
656 def _is_netfail_master(iface):
657 if iface == 'ens3':
658 return True
659 return False
660 self.m_netfail_master.side_effect = _is_netfail_master
661 expected_cfg = {'version': 1, 'config': [
662 {'type': 'physical', 'name': nic_master},
663 {'type': 'physical', 'name': nic_other, 'mac_address': mac_other},
664 {'type': 'physical', 'name': nic_extra, 'mac_address': mac_extra},
665 ]}
666 oracle._ensure_netfailover_safe(netcfg)
667 self.assertEqual(expected_cfg, netcfg)
668
669 def test_checks_v2_type_ethernet_interfaces(self):
670 mac_addr, nic_name = '00:00:17:02:2b:b1', 'ens3'
671 self.m_get_interfaces_by_mac.return_value = {
672 mac_addr: nic_name,
673 }
674 netcfg = {'version': 2, 'ethernets': {
675 nic_name: {'dhcp4': True, 'critical': True, 'set-name': nic_name,
676 'match': {'macaddress': mac_addr}}}}
677 passed_netcfg = copy.copy(netcfg)
678 self.m_netfail_master.return_value = False
679 oracle._ensure_netfailover_safe(passed_netcfg)
680 self.assertEqual(netcfg, passed_netcfg)
681 self.assertEqual([mock.call(nic_name)],
682 self.m_netfail_master.call_args_list)
683
684 def test_skips_v2_non_ethernet_interfaces(self):
685 mac_addr, nic_name = '00:00:17:02:2b:b1', 'wlps0'
686 self.m_get_interfaces_by_mac.return_value = {
687 mac_addr: nic_name,
688 }
689 netcfg = {'version': 2, 'wifis': {
690 nic_name: {'dhcp4': True, 'critical': True, 'set-name': nic_name,
691 'match': {'macaddress': mac_addr}}}}
692 passed_netcfg = copy.copy(netcfg)
693 oracle._ensure_netfailover_safe(passed_netcfg)
694 self.assertEqual(netcfg, passed_netcfg)
695 self.assertEqual(0, self.m_netfail_master.call_count)
696
697 def test_removes_master_mac_property_v2(self):
698 nic_master, mac_master = 'ens3', self.random_string()
699 nic_other, mac_other = 'ens7', self.random_string()
700 nic_extra, mac_extra = 'enp0s1f2', self.random_string()
701 self.m_get_interfaces_by_mac.return_value = {
702 mac_master: nic_master,
703 mac_other: nic_other,
704 mac_extra: nic_extra,
705 }
706 netcfg = {'version': 2, 'ethernets': {
707 nic_extra: {'dhcp4': True, 'set-name': nic_extra,
708 'match': {'macaddress': mac_extra}},
709 nic_other: {'dhcp4': True, 'set-name': nic_other,
710 'match': {'macaddress': mac_other}},
711 nic_master: {'dhcp4': True, 'set-name': nic_master,
712 'match': {'macaddress': mac_master}},
713 }}
714
715 def _is_netfail_master(iface):
716 if iface == 'ens3':
717 return True
718 return False
719 self.m_netfail_master.side_effect = _is_netfail_master
720
721 expected_cfg = {'version': 2, 'ethernets': {
722 nic_master: {'dhcp4': True, 'match': {'name': nic_master}},
723 nic_extra: {'dhcp4': True, 'set-name': nic_extra,
724 'match': {'macaddress': mac_extra}},
725 nic_other: {'dhcp4': True, 'set-name': nic_other,
726 'match': {'macaddress': mac_other}},
727 }}
728 oracle._ensure_netfailover_safe(netcfg)
729 import pprint
730 pprint.pprint(netcfg)
731 print('---- ^^ modified ^^ ---- vv original vv ----')
732 pprint.pprint(expected_cfg)
733 self.assertEqual(expected_cfg, netcfg)
734
735
589# vi: ts=4 expandtab736# vi: ts=4 expandtab
diff --git a/cloudinit/tests/helpers.py b/cloudinit/tests/helpers.py
index 23fddd0..4dad2af 100644
--- a/cloudinit/tests/helpers.py
+++ b/cloudinit/tests/helpers.py
@@ -6,7 +6,9 @@ import functools
6import httpretty6import httpretty
7import logging7import logging
8import os8import os
9import random
9import shutil10import shutil
11import string
10import sys12import sys
11import tempfile13import tempfile
12import time14import time
@@ -243,6 +245,12 @@ class CiTestCase(TestCase):
243 myds.metadata.update(metadata)245 myds.metadata.update(metadata)
244 return cloud.Cloud(myds, self.paths, sys_cfg, mydist, None)246 return cloud.Cloud(myds, self.paths, sys_cfg, mydist, None)
245247
248 @classmethod
249 def random_string(cls, length=8):
250 """ return a random lowercase string with default length of 8"""
251 return ''.join(
252 random.choice(string.ascii_lowercase) for _ in range(length))
253
246254
247class ResourceUsingTestCase(CiTestCase):255class ResourceUsingTestCase(CiTestCase):
248256

Subscribers

People subscribed via source and target branches