Merge ~chad.smith/cloud-init:bug/1781229-centos6-copr-get-linux-distro into cloud-init:master

[Chad Smith]

get_linux_distro: add support for centos6 and rawhide flavors of redhat

An empty /etc/os-release exists on some redhat images, most notably
the COPR build images of centos6 and rawhide. On platforms missing
/etc/os-release or having an empty /etc/os-release file, use
_parse_redhat_release on rhel-based images to obtain distribution and
release codename information.

LP: #1781229

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:96b18320782f2e2f75781f5dbd0a89fa3d912a35
https://jenkins.ubuntu.com/server/job/cloud-init-ci/157/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/157/rebuild

review: Needs Fixing (continuous-integration)

[Chad Smith (chad.smith)]

test run of this branch on copr: https://copr.fedorainfracloud.org/coprs/blackboxsw/cloud-init/build/776423/

kicked this off with the following:

1. git clone <thisbranch>
2. ./tools/run-container --source-package centos/7
3.in copr UI builds tab click "New Build" button
https://copr.fedorainfracloud.org/coprs/blackboxsw/cloud-init/builds/

4. click Upload tab and choose your src.rpm file that was built in step 2

dffbb57... by Chad Smith on 2018-07-11

pycodestyle fixes

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:dffbb57447888a54977c91920bdc513052c9e6b2
https://jenkins.ubuntu.com/server/job/cloud-init-ci/158/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/158/rebuild

review: Approve (continuous-integration)

[Scott Moser (smoser)]

I launched a RHEL instance on ibm softlayer in order to see what it had in /etc/redhat-release.

# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)

I'll get redhat 6 version tomorrow.

for the record, the util.get_linux_distro() did work, as it used /etc/os-release.

# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.5 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.5"
PRETTY_NAME="Red Hat"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.5:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.5
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.5"

[Scott Moser (smoser)]

Launched a redhat 6 image there, and got
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.10 (Santiago)

# cat /etc/os-release
cat: /etc/os-release: No such file or directory

[Scott Moser (smoser)]

I'm going to hit 'accept', and let this land and then i'll try to adjust to support redhat-release actually having 'Red Hat' values rather than centos.

review: Approve

diff --git a/cloudinit/tests/test_util.py b/cloudinit/tests/test_util.py
index 6a31e50..387d894 100644
--- a/cloudinit/tests/test_util.py
+++ b/cloudinit/tests/test_util.py
@@ -57,6 +57,9 @@ OS_RELEASE_CENTOS = dedent("""\
     REDHAT_SUPPORT_PRODUCT_VERSION="7"
 """)
 
+REDHAT_RELEASE_CENTOS_6 = "CentOS release 6.10 (Final)"
+REDHAT_RELEASE_CENTOS_7 = "CentOS Linux release 7.5.1804 (Core)"
+
 OS_RELEASE_DEBIAN = dedent("""\
     PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
     NAME="Debian GNU/Linux"
@@ -337,6 +340,12 @@ class TestGetLinuxDistro(CiTestCase):
         if path == '/etc/os-release':
             return 1
 
+    @classmethod
+    def redhat_release_exists(self, path):
+        """Side effect function """
+        if path == '/etc/redhat-release':
+            return 1
+
     @mock.patch('cloudinit.util.load_file')
     def test_get_linux_distro_quoted_name(self, m_os_release, m_path_exists):
         """Verify we get the correct name if the os-release file has
@@ -356,8 +365,24 @@ class TestGetLinuxDistro(CiTestCase):
         self.assertEqual(('ubuntu', '16.04', 'xenial'), dist)
 
     @mock.patch('cloudinit.util.load_file')
-    def test_get_linux_centos(self, m_os_release, m_path_exists):
-        """Verify we get the correct name and release name on CentOS."""
+    def test_get_linux_centos6(self, m_os_release, m_path_exists):
+        """Verify we get the correct name and release name on CentOS 6."""
+        m_os_release.return_value = REDHAT_RELEASE_CENTOS_6
+        m_path_exists.side_effect = TestGetLinuxDistro.redhat_release_exists
+        dist = util.get_linux_distro()
+        self.assertEqual(('centos', '6.10', 'Final'), dist)
+
+    @mock.patch('cloudinit.util.load_file')
+    def test_get_linux_centos7_redhat_release(self, m_os_release, m_exists):
+        """Verify the correct release info on CentOS 7 without os-release."""
+        m_os_release.return_value = REDHAT_RELEASE_CENTOS_7
+        m_exists.side_effect = TestGetLinuxDistro.redhat_release_exists
+        dist = util.get_linux_distro()
+        self.assertEqual(('centos', '7.5.1804', 'Core'), dist)
+
+    @mock.patch('cloudinit.util.load_file')
+    def test_get_linux_copr_centos(self, m_os_release, m_path_exists):
+        """Verify we get the correct name and release name on COPR CentOS."""
         m_os_release.return_value = OS_RELEASE_CENTOS
         m_path_exists.side_effect = TestGetLinuxDistro.os_release_exists
         dist = util.get_linux_distro()
diff --git a/cloudinit/util.py b/cloudinit/util.py
index d0b0e90..8604db5 100644
--- a/cloudinit/util.py
+++ b/cloudinit/util.py
@@ -576,12 +576,39 @@ def get_cfg_option_int(yobj, key, default=0):
     return int(get_cfg_option_str(yobj, key, default=default))
 
 
+def _parse_redhat_release(release_file=None):
+    """Return a dictionary of distro info fields from /etc/redhat-release.
+
+    Dict keys will align with /etc/os-release keys:
+        ID, VERSION_ID, VERSION_CODENAME
+    """
+
+    if not release_file:
+        release_file = '/etc/redhat-release'
+    if not os.path.exists(release_file):
+        return {}
+    redhat_release = load_file(release_file)
+    redhat_regex = (
+        r'(?P<name>\S+) (Linux )?release (?P<version>[\d\.]+) '
+        r'\((?P<codename>[^)]+)\)')
+    match = re.match(redhat_regex, redhat_release)
+    if match:
+        group = match.groupdict()
+        return {'ID': group['name'].lower(), 'VERSION_ID': group['version'],
+                'VERSION_CODENAME': group['codename']}
+    return {}
+
+
 def get_linux_distro():
     distro_name = ''
     distro_version = ''
     flavor = ''
+    os_release = {}
     if os.path.exists('/etc/os-release'):
         os_release = load_shell_content(load_file('/etc/os-release'))
+    if not os_release:
+        os_release = _parse_redhat_release()
+    if os_release:
         distro_name = os_release.get('ID', '')
         distro_version = os_release.get('VERSION_ID', '')
         if 'sles' in distro_name or 'suse' in distro_name:
@@ -594,7 +621,7 @@ def get_linux_distro():
             flavor = os_release.get('VERSION_CODENAME', '')
             if not flavor:
                 match = re.match(r'[^ ]+ \((?P<codename>[^)]+)\)',
-                                 os_release.get('VERSION'))
+                                 os_release.get('VERSION', ''))
                 if match:
                     flavor = match.groupdict()['codename']
     else: