Merge ~raharper/cloud-init:fix/net-get-devices-ignore-devs-with-master into cloud-init:master

[Ryan Harper]

net: add is_master check for filtering device list

Some network devices are transformed into a bond via kernel magic
and do not have the 'bonding' sysfs attribute, but like a bond they
have a duplicate MAC of other bond members. On Azure Advanced
Networking SRIOV devices are auto bonded and will have the same MAC
as the HyperV nic. We can detect this via the 'master' sysfs attribute
in the device sysfs path and this patch adds this to the list of devices
we ignore when enumerating device lists.

LP: #1844191

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:17b2058017074424fa8b87c0a1eea5547935a2a7
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1143/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1143//rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

A minor request for unit test here. otherwise +1. https://paste.ubuntu.com/p/6wMR7WGT9f/

acb65b5... by Ryan Harper on 2019-09-17

Add unittest to verify get_interfaces_by_mac skips master devs

[Ryan Harper (raharper)]

Thanks Chad, that's a great one.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:acb65b5c2344961ae04660f366aa3444b8b337cc
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1144/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1144//rebuild

review: Approve (continuous-integration)

15ffe02... by Ryan Harper on 2019-09-17

Better name is has_master since the interface has the 'master' sysfs attr file

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:15ffe027cb53d162bc1f436e0f50b8b16c796484
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1145/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1145//rebuild

review: Approve (continuous-integration)

[Dan Watkins (oddbloke)]

review: Approve

[Chad Smith (chad.smith)]

+1

[Chad Smith (chad.smith)]

review: Approve

diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
index 0eb952f..5de5c6d 100644
--- a/cloudinit/net/__init__.py
+++ b/cloudinit/net/__init__.py
@@ -109,6 +109,10 @@ def is_bond(devname):
     return os.path.exists(sys_dev_path(devname, "bonding"))
 
 
+def has_master(devname):
+    return os.path.exists(sys_dev_path(devname, path="master"))
+
+
 def is_netfailover(devname, driver=None):
     """ netfailover driver uses 3 nics, master, primary and standby.
         this returns True if the device is either the primary or standby
@@ -154,7 +158,7 @@ def is_netfail_master(devname, driver=None):
 
         Return True if all of the above is True.
     """
-    if os.path.exists(sys_dev_path(devname, path='master')):
+    if has_master(devname):
         return False
 
     if driver is None:
@@ -211,7 +215,7 @@ def is_netfail_standby(devname, driver=None):
 
         Return True if all of the above is True.
     """
-    if not os.path.exists(sys_dev_path(devname, path='master')):
+    if not has_master(devname):
         return False
 
     if driver is None:
@@ -786,6 +790,8 @@ def get_interfaces():
             continue
         if is_bond(name):
             continue
+        if has_master(name):
+            continue
         if is_netfailover(name):
             continue
         mac = get_interface_mac(name)
diff --git a/cloudinit/net/tests/test_init.py b/cloudinit/net/tests/test_init.py
index 7259dbe..999db98 100644
--- a/cloudinit/net/tests/test_init.py
+++ b/cloudinit/net/tests/test_init.py
@@ -157,6 +157,12 @@ class TestReadSysNet(CiTestCase):
         ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))
         self.assertTrue(net.is_bond('eth0'))
 
+    def test_has_master(self):
+        """has_master is True when /sys/net/devname/master exists."""
+        self.assertFalse(net.has_master('enP1s1'))
+        ensure_file(os.path.join(self.sysdir, 'enP1s1', 'master'))
+        self.assertTrue(net.has_master('enP1s1'))
+
     def test_is_vlan(self):
         """is_vlan is True when /sys/net/devname/uevent has DEVTYPE=vlan."""
         ensure_file(os.path.join(self.sysdir, 'eth0', 'uevent'))
@@ -424,6 +430,17 @@ class TestGetInterfaceMAC(CiTestCase):
         expected = [('eth2', 'aa:bb:cc:aa:bb:cc', None, None)]
         self.assertEqual(expected, net.get_interfaces())
 
+    def test_get_interfaces_by_mac_skips_master_devs(self):
+        """Ignore interfaces with a master device which would have dup mac."""
+        mac1 = mac2 = 'aa:bb:cc:aa:bb:cc'
+        write_file(os.path.join(self.sysdir, 'eth1', 'addr_assign_type'), '0')
+        write_file(os.path.join(self.sysdir, 'eth1', 'address'), mac1)
+        write_file(os.path.join(self.sysdir, 'eth1', 'master'), "blah")
+        write_file(os.path.join(self.sysdir, 'eth2', 'addr_assign_type'), '0')
+        write_file(os.path.join(self.sysdir, 'eth2', 'address'), mac2)
+        expected = [('eth2', mac2, None, None)]
+        self.assertEqual(expected, net.get_interfaces())
+
     @mock.patch('cloudinit.net.is_netfailover')
     def test_get_interfaces_by_mac_skips_netfailvoer(self, m_netfail):
         """Ignore interfaces if netfailover primary or standby."""