Merge lp:~ltrager/maas/rhel_driver into lp:~maas-committers/maas/trunk

[Lee Trager]

Add RHEL driver and allow CentOS to be deployed using a custom name.

This adds the RHEL driver to MAAS. While testing I realized the CentOS driver only allowed images using the format centos<major><minor>, this branch fixes that making the RHEL and CentOS driver nearly identical. As a quick drive by I fixed 'custom' from being shown multiple times in the deploy drop down.

[Blake Rouse (blake-rouse)]

Looks good.

review: Approve

[MAAS Lander (maas-lander)]

The attempt to merge lp:~ltrager/maas/rhel_driver into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libnss-wrapper libpq-dev make nodejs-legacy npm postgresql psmisc pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
psmisc is already the newest version (22.21-2.1build1).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-0.1build2).
python-psycopg2 is already the newest version (2.6.1-1build2).
pytho...

[MAAS Lander (maas-lander)]

The attempt to merge lp:~ltrager/maas/rhel_driver into lp:maas failed. Below is the output from the failed tests.

Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease
Get:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Fetched 306 kB in 1s (261 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libnss-wrapper libpq-dev make nodejs-legacy npm postgresql psmisc pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
psmisc is already the newest version (22.21-2.1build1).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-0.1build2).
python-psycopg2 is already the newest version (2.6.1-1build2).
python-simplejson ...

=== modified file 'src/maasserver/utils/osystems.py'
--- src/maasserver/utils/osystems.py	2017-05-05 22:08:46 +0000
+++ src/maasserver/utils/osystems.py	2017-06-16 09:40:48 +0000
@@ -57,7 +57,7 @@
         (osystem['name'], osystem['title'])
         for osystem in osystems
         ]
-    return choices
+    return sorted(list(set(choices)))
 
 
 def list_all_usable_releases(osystems):

=== modified file 'src/maasserver/utils/tests/test_osystems.py'
--- src/maasserver/utils/tests/test_osystems.py	2017-05-05 22:08:46 +0000
+++ src/maasserver/utils/tests/test_osystems.py	2017-06-16 09:40:48 +0000
@@ -95,6 +95,15 @@
             [(osystem['name'], osystem['title'])],
             list_osystem_choices([osystem], include_default=False))
 
+    def test_list_osystem_choices_doesnt_duplicate(self):
+        self.assertEqual(
+            [('custom', 'Custom')],
+            list_osystem_choices(
+                [
+                    {'name': 'custom', 'title': 'Custom'},
+                    {'name': 'custom', 'title': 'Custom'},
+                ], include_default=False))
+
 
 class TestReleases(MAASServerTestCase):
 

=== modified file 'src/provisioningserver/drivers/osystem/__init__.py'
--- src/provisioningserver/drivers/osystem/__init__.py	2017-03-01 17:16:56 +0000
+++ src/provisioningserver/drivers/osystem/__init__.py	2017-06-16 09:40:48 +0000
@@ -1,4 +1,4 @@
-# Copyright 2014-2016 Canonical Ltd.  This software is licensed under the
+# Copyright 2014-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """Osystem Drivers."""
@@ -205,6 +205,7 @@
 from provisioningserver.drivers.osystem.ubuntucore import UbuntuCoreOS
 from provisioningserver.drivers.osystem.bootloader import BootLoaderOS
 from provisioningserver.drivers.osystem.centos import CentOS
+from provisioningserver.drivers.osystem.rhel import RHELOS
 from provisioningserver.drivers.osystem.custom import CustomOS
 from provisioningserver.drivers.osystem.windows import WindowsOS
 from provisioningserver.drivers.osystem.suse import SUSEOS
@@ -214,6 +215,7 @@
     UbuntuCoreOS(),
     BootLoaderOS(),
     CentOS(),
+    RHELOS(),
     CustomOS(),
     WindowsOS(),
     SUSEOS(),

=== modified file 'src/provisioningserver/drivers/osystem/centos.py'
--- src/provisioningserver/drivers/osystem/centos.py	2017-04-25 20:44:56 +0000
+++ src/provisioningserver/drivers/osystem/centos.py	2017-06-16 09:40:48 +0000
@@ -1,4 +1,4 @@
-# Copyright 2014-2015 Canonical Ltd.  This software is licensed under the
+# Copyright 2014-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """CentOS Operating System."""
@@ -14,20 +14,20 @@
     OperatingSystem,
 )
 
-
-DISTRO_SERIES_DEFAULT = 'centos65'
-
-# Regex matcher that is used to check if the release is supported.
-# It needs to match the name "centosXY". Where "X" is the major version
-# and "Y" is the minor version.
-DISTRO_MATCHER = re.compile("centos(?P<major>[0-9])(?P<minor>[0-9])?\Z")
+# Regex matcher that is used to check if the release is supported. The release
+# name just has to start with 'centos' to be supported but the major, minor,
+# and title are found if available to help format the title.
+DISTRO_MATCHER = re.compile(
+    '^centos((?P<major>[0-9])(?P<minor>[0-9])?)?([\-\.]?(?P<title>.+))?$',
+    re.I)
+DISTRO_SERIES_DEFAULT = 'centos70'
 
 
 class CentOS(OperatingSystem):
     """CentOS operating system."""
 
-    name = "centos"
-    title = "CentOS"
+    name = 'centos'
+    title = 'CentOS'
 
     def get_boot_image_purposes(self, arch, subarch, release, label):
         """Gets the purpose of each boot image."""
@@ -36,7 +36,6 @@
             ]
 
     def is_release_supported(self, release):
-        """Return True when the release is supported, False otherwise."""
         matched = DISTRO_MATCHER.match(release)
         return matched is not None
 
@@ -49,10 +48,14 @@
         """Return the title for the given release."""
         matched = DISTRO_MATCHER.match(release)
         if matched is None:
-            return None
-        matched_dict = matched.groupdict()
-        major = matched_dict['major']
-        minor = matched_dict['minor']
+            # This should never happen as is_release_supported will return
+            # false but just in case it does...
+            return "%s %s" % (self.title, release)
+
+        ret = self.title
+        major = matched.group('major')
+        minor = matched.group('minor')
+        title = matched.group('title')
         # MAAS provided images via streams are not bound to a minor
         # release version, which means we always provide the latest
         # available release from CentOS 6 and CentOS 7.  To address
@@ -65,9 +68,14 @@
         # from the stream and the minor version doesn't match to what
         # we publish. As such, we ensure that we only return minor
         # if we have any other version other that X.0, 7.0 and 6.6.
-        if minor is None or minor == '0':
-            return "CentOS %s" % major
+        if major is not None and minor is None or minor == '0':
+            ret = "%s %s" % (ret, major)
         elif major == '6' and minor == '6':
-            return "CentOS %s" % major
-        else:
-            return "CentOS %s.%s" % (major, minor)
+            ret = "%s %s" % (ret, major)
+        elif None not in (major, minor):
+            ret = "%s %s.%s" % (ret, major, minor)
+
+        if title is not None:
+            ret = "%s %s" % (ret, title)
+
+        return ret

=== added file 'src/provisioningserver/drivers/osystem/rhel.py'
--- src/provisioningserver/drivers/osystem/rhel.py	1970-01-01 00:00:00 +0000
+++ src/provisioningserver/drivers/osystem/rhel.py	2017-06-16 09:40:48 +0000
@@ -0,0 +1,67 @@
+# Copyright 2017 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""RHELOS Operating System."""
+
+__all__ = [
+    "RHELOS",
+    ]
+
+import re
+
+from provisioningserver.drivers.osystem import (
+    BOOT_IMAGE_PURPOSE,
+    OperatingSystem,
+)
+
+# Regex matcher that is used to check if the release is supported. The release
+# name just has to start with 'rhel' to be supported but the major, minor,
+# and title are found if available to help format the title.
+DISTRO_MATCHER = re.compile(
+    '^rhel((?P<major>[0-9])(?P<minor>[0-9])?)?([\-\.]?(?P<title>.+))?$',
+    re.I)
+DISTRO_SERIES_DEFAULT = 'rhel7'
+
+
+class RHELOS(OperatingSystem):
+    """RHELOS operating system."""
+
+    name = 'rhel'
+    title = 'Redhat Enterprise Linux'
+
+    def get_boot_image_purposes(self, arch, subarch, release, label):
+        """Gets the purpose of each boot image."""
+        return [
+            BOOT_IMAGE_PURPOSE.XINSTALL
+            ]
+
+    def is_release_supported(self, release):
+        matched = DISTRO_MATCHER.match(release)
+        return matched is not None
+
+    def get_default_release(self):
+        """Gets the default release to use when a release is not
+        explicit."""
+        return DISTRO_SERIES_DEFAULT
+
+    def get_release_title(self, release):
+        """Return the title for the given release."""
+        matched = DISTRO_MATCHER.match(release)
+        if matched is None:
+            # This should never happen as is_release_supported will return
+            # false but just in case it does...
+            return "%s %s" % (self.title, release)
+
+        ret = self.title
+        major = matched.group('major')
+        minor = matched.group('minor')
+        title = matched.group('title')
+        if None not in (major, minor):
+            ret = "%s %s.%s" % (ret, major, minor)
+        elif major is not None:
+            ret = "%s %s" % (ret, major)
+
+        if title is not None:
+            ret = "%s %s" % (ret, title)
+
+        return ret

=== modified file 'src/provisioningserver/drivers/osystem/tests/test_centos.py'
--- src/provisioningserver/drivers/osystem/tests/test_centos.py	2017-04-26 14:53:22 +0000
+++ src/provisioningserver/drivers/osystem/tests/test_centos.py	2017-06-16 09:40:48 +0000
@@ -1,4 +1,4 @@
-# Copyright 2014-2015 Canonical Ltd.  This software is licensed under the
+# Copyright 2014-2017 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """Tests for the CentOS module."""
@@ -41,8 +41,10 @@
             "centos7": True,
             "centos71": True,
             "cent65": False,
-            "cent": False,
-            "centos711": False,
+            "centos711": True,
+            "centos-title": True,
+            "centos71.custom": True,
+            factory.make_name("name"): False,
             }
         osystem = CentOS()
         for name, supported in name_supported.items():
@@ -62,9 +64,10 @@
             "centos7": "CentOS 7",
             "centos70": "CentOS 7",  # See LP: #1654063
             "centos71": "CentOS 7.1",
-            "cent65": None,
-            "cent": None,
-            "centos711": None,
+            "centos65": "CentOS 6.5",
+            "cent": "CentOS cent",
+            "centos711": "CentOS 7.1 1",
+            "centos71-custom": "CentOS 7.1 custom",
             }
         osystem = CentOS()
         for name, title in name_titles.items():

=== added file 'src/provisioningserver/drivers/osystem/tests/test_rhel.py'
--- src/provisioningserver/drivers/osystem/tests/test_rhel.py	1970-01-01 00:00:00 +0000
+++ src/provisioningserver/drivers/osystem/tests/test_rhel.py	2017-06-16 09:40:48 +0000
@@ -0,0 +1,73 @@
+# Copyright 2017 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for the RHEL module."""
+
+__all__ = []
+
+from itertools import product
+
+from maastesting.factory import factory
+from maastesting.testcase import MAASTestCase
+from provisioningserver.drivers.osystem.rhel import (
+    BOOT_IMAGE_PURPOSE,
+    DISTRO_SERIES_DEFAULT,
+    RHELOS,
+)
+from testtools.matchers import Equals
+
+
+class TestRHEL(MAASTestCase):
+
+    def test_get_boot_image_purposes(self):
+        osystem = RHELOS()
+        archs = [factory.make_name('arch') for _ in range(2)]
+        subarchs = [factory.make_name('subarch') for _ in range(2)]
+        releases = [factory.make_name('release') for _ in range(2)]
+        labels = [factory.make_name('label') for _ in range(2)]
+        for arch, subarch, release, label in product(
+                archs, subarchs, releases, labels):
+            expected = osystem.get_boot_image_purposes(
+                arch, subarchs, release, label)
+            self.assertIsInstance(expected, list)
+            self.assertEqual(expected, [
+                BOOT_IMAGE_PURPOSE.XINSTALL,
+                ])
+
+    def test_is_release_supported(self):
+        name_supported = {
+            "rhel6": True,
+            "rhel65": True,
+            "rhel7": True,
+            "rhel71": True,
+            "cent65": False,
+            "rhel711": True,
+            "rhel-title": True,
+            "rhel71.custom": True,
+            factory.make_name("name"): False,
+            }
+        osystem = RHELOS()
+        for name, supported in name_supported.items():
+            self.expectThat(
+                osystem.is_release_supported(name), Equals(supported))
+
+    def test_get_default_release(self):
+        osystem = RHELOS()
+        expected = osystem.get_default_release()
+        self.assertEqual(expected, DISTRO_SERIES_DEFAULT)
+
+    def test_get_release_title(self):
+        name_titles = {
+            "rhel6": "Redhat Enterprise Linux 6",
+            "rhel65": "Redhat Enterprise Linux 6.5",
+            "rhel7": "Redhat Enterprise Linux 7",
+            "rhel71": "Redhat Enterprise Linux 7.1",
+            "rhel65": "Redhat Enterprise Linux 6.5",
+            "cent": "Redhat Enterprise Linux cent",
+            "rhel711": "Redhat Enterprise Linux 7.1 1",
+            "rhel71-custom": "Redhat Enterprise Linux 7.1 custom",
+            }
+        osystem = RHELOS()
+        for name, title in name_titles.items():
+            self.expectThat(
+                osystem.get_release_title(name), Equals(title))