Merge lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1 into lp:maas/2.1

Proposed by Mike Pontillo
Status: Work in progress
Proposed branch: lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1
Merge into: lp:maas/2.1
Diff against target: 121 lines (+67/-2)
3 files modified
contrib/preseeds_v2/curtin_userdata (+16/-2)
docs/changelog.rst (+2/-0)
src/maasserver/tests/test_preseed.py (+49/-0)
To merge this branch: bzr merge lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1
Reviewer Review Type Date Requested Status
Mike Pontillo (community) Approve
Review via email: mp+318179@code.launchpad.net

Commit message

Merge revision 5756 from trunk.
Fix curtin config rendering for third-party drivers.

To post a comment you must log in.
Revision history for this message
Mike Pontillo (mpontillo) wrote :

Self-approved backport.

review: Approve
Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (2.4 MiB)

The attempt to merge lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1 into lp:maas/2.1 failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:4 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Fetched 306 kB in 0s (757 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
git is already the newest version (1:2.7.4-0ubuntu1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-...

Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (2.4 MiB)

The attempt to merge lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1 into lp:maas/2.1 failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Get:4 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Fetched 306 kB in 0s (667 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
git is already the newest version (1:2.7.4-0ubuntu1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-...

Revision history for this message
Mike Pontillo (mpontillo) wrote :

Hm, at first I thought the 1.9 lander was just being the 1.9 lander again, but it seems that the preseed code has changed significantly enough that these modifications aren't an easy backport.

With a MAAS 2.2 release imminent, that makes this lower priority for 2.1. We'll circle back if the need arises.

Unmerged revisions

5591. By Mike Pontillo

Update changelog.

5590. By Mike Pontillo

Merge revision 5756 from trunk.
Fix curtin config rendering for third-party drivers.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'contrib/preseeds_v2/curtin_userdata'
--- contrib/preseeds_v2/curtin_userdata 2016-12-08 13:45:40 +0000
+++ contrib/preseeds_v2/curtin_userdata 2017-02-24 04:33:23 +0000
@@ -6,21 +6,35 @@
6 {{endfor}}6 {{endfor}}
7{{if third_party_drivers and driver}}7{{if third_party_drivers and driver}}
8early_commands:8early_commands:
9 {{py: key_string = ''.join(['\\x%x' % x for x in map(ord, driver['key_binary'])])}}9 {{py: key_string = ''.join(['\\x%x' % x for x in driver['key_binary']])}}
10 {{if driver['key_binary'] and driver['repository'] and driver['package']}}
10 driver_00_get_key: /bin/echo -en '{{key_string}}' > /tmp/maas-{{driver['package']}}.gpg11 driver_00_get_key: /bin/echo -en '{{key_string}}' > /tmp/maas-{{driver['package']}}.gpg
11 driver_01_add_key: ["apt-key", "add", "/tmp/maas-{{driver['package']}}.gpg"]12 driver_01_add_key: ["apt-key", "add", "/tmp/maas-{{driver['package']}}.gpg"]
13 {{endif}}
14 {{if driver['repository']}}
12 driver_02_add: ["add-apt-repository", "-y", "deb {{driver['repository']}} {{node.get_distro_series()}} main"]15 driver_02_add: ["add-apt-repository", "-y", "deb {{driver['repository']}} {{node.get_distro_series()}} main"]
16 {{endif}}
17 {{if driver['package']}}
13 driver_03_update_install: ["sh", "-c", "apt-get update --quiet && apt-get --assume-yes install {{driver['package']}}"]18 driver_03_update_install: ["sh", "-c", "apt-get update --quiet && apt-get --assume-yes install {{driver['package']}}"]
14 driver_04_load: ["sh", "-c", "depmod && modprobe {{driver['module']}}"]19 {{endif}}
20 {{if driver['module']}}
21 driver_04_load: ["sh", "-c", "depmod && modprobe {{driver['module']}} || echo 'Warning: Failed to load module: {{driver['module']}}'"]
22 {{endif}}
15{{endif}}23{{endif}}
16late_commands:24late_commands:
17 maas: [wget, '--no-proxy', {{node_disable_pxe_url|escape.json}}, '--post-data', {{node_disable_pxe_data|escape.json}}, '-O', '/dev/null']25 maas: [wget, '--no-proxy', {{node_disable_pxe_url|escape.json}}, '--post-data', {{node_disable_pxe_data|escape.json}}, '-O', '/dev/null']
18{{if third_party_drivers and driver}}26{{if third_party_drivers and driver}}
27 {{if driver['key_binary'] and driver['repository'] and driver['package']}}
19 driver_00_key_get: curtin in-target -- sh -c "/bin/echo -en '{{key_string}}' > /tmp/maas-{{driver['package']}}.gpg"28 driver_00_key_get: curtin in-target -- sh -c "/bin/echo -en '{{key_string}}' > /tmp/maas-{{driver['package']}}.gpg"
20 driver_02_key_add: ["curtin", "in-target", "--", "apt-key", "add", "/tmp/maas-{{driver['package']}}.gpg"]29 driver_02_key_add: ["curtin", "in-target", "--", "apt-key", "add", "/tmp/maas-{{driver['package']}}.gpg"]
30 {{endif}}
31 {{if driver['repository']}}
21 driver_03_add: ["curtin", "in-target", "--", "add-apt-repository", "-y", "deb {{driver['repository']}} {{node.get_distro_series()}} main"]32 driver_03_add: ["curtin", "in-target", "--", "add-apt-repository", "-y", "deb {{driver['repository']}} {{node.get_distro_series()}} main"]
33 {{endif}}
22 driver_04_update_install: ["curtin", "in-target", "--", "apt-get", "update", "--quiet"]34 driver_04_update_install: ["curtin", "in-target", "--", "apt-get", "update", "--quiet"]
35 {{if driver['package']}}
23 driver_05_install: ["curtin", "in-target", "--", "apt-get", "-y", "install", "{{driver['package']}}"]36 driver_05_install: ["curtin", "in-target", "--", "apt-get", "-y", "install", "{{driver['package']}}"]
37 {{endif}}
24 driver_06_depmod: ["curtin", "in-target", "--", "depmod"]38 driver_06_depmod: ["curtin", "in-target", "--", "depmod"]
25 driver_07_update_initramfs: ["curtin", "in-target", "--", "update-initramfs", "-u"]39 driver_07_update_initramfs: ["curtin", "in-target", "--", "update-initramfs", "-u"]
26{{endif}}40{{endif}}
2741
=== modified file 'docs/changelog.rst'
--- docs/changelog.rst 2017-02-17 05:59:11 +0000
+++ docs/changelog.rst 2017-02-24 04:33:23 +0000
@@ -34,6 +34,8 @@
3434
35LP: #1665459 Fix anonymous auto-enlistment to properly detect rack-facing region IP address35LP: #1665459 Fix anonymous auto-enlistment to properly detect rack-facing region IP address
3636
37LP: #1667426 Fix curtin config rendering for custom third-party drivers.
38
3739
38Other notable changes40Other notable changes
39---------------------41---------------------
4042
=== modified file 'src/maasserver/tests/test_preseed.py'
--- src/maasserver/tests/test_preseed.py 2016-12-13 17:26:28 +0000
+++ src/maasserver/tests/test_preseed.py 2017-02-24 04:33:23 +0000
@@ -77,6 +77,7 @@
77 MAASServerTestCase,77 MAASServerTestCase,
78 MAASTransactionServerTestCase,78 MAASTransactionServerTestCase,
79)79)
80from maasserver.third_party_drivers import DriversConfig
80from maasserver.utils import absolute_reverse81from maasserver.utils import absolute_reverse
81from maasserver.utils.curtin import curtin_supports_webhook_events82from maasserver.utils.curtin import curtin_supports_webhook_events
82from maastesting.matchers import (83from maastesting.matchers import (
@@ -850,6 +851,54 @@
850 self.assertThat(mock_supports_storage, MockCalledOnceWith())851 self.assertThat(mock_supports_storage, MockCalledOnceWith())
851852
852853
854class TestRenderCurtinUserdataWithThirdPartyDrivers(
855 PreseedRPCMixin, BootImageHelperMixin, MAASServerTestCase):
856 """Ensures curtin configs for all third-party drivers can be rendered."""
857
858 # Try rendering each driver in drivers.yaml.
859 scenarios = [
860 (driver['comment'], {'driver': driver})
861 for driver in DriversConfig.load_from_cache()['drivers']
862 ]
863
864 def test_render_curtin_preseed_with_third_party_driver(self):
865 node = factory.make_Node_with_Interface_on_Subnet(
866 primary_rack=self.rpc_rack_controller)
867 Config.objects.set_config(
868 'enable_third_party_drivers', True)
869 self.configure_get_boot_images_for_node(node, 'xinstall')
870 get_third_party_driver = self.patch(
871 preseed_module, "get_third_party_driver")
872 get_third_party_driver.return_value = self.driver
873 curtin_config_text = get_curtin_config(node)
874 config = yaml.safe_load(curtin_config_text)
875 self.assertThat(
876 config['early_commands'], Contains('driver_00_get_key'))
877 self.assertThat(
878 config['early_commands'], Contains('driver_01_add_key'))
879 self.assertThat(
880 config['early_commands'], Contains('driver_02_add'))
881 self.assertThat(
882 config['early_commands'], Contains('driver_03_update_install'))
883 self.assertThat(
884 config['early_commands'], Contains('driver_04_load'))
885 self.assertThat(
886 config['late_commands'], Contains('driver_00_key_get'))
887 self.assertThat(
888 config['late_commands'], Contains('driver_02_key_add'))
889 self.assertThat(
890 config['late_commands'], Contains('driver_03_add'))
891 self.assertThat(
892 config['late_commands'], Contains('driver_04_update_install'))
893 self.assertThat(
894 config['late_commands'], Contains('driver_05_install'))
895 self.assertThat(
896 config['late_commands'], Contains('driver_06_depmod'))
897 self.assertThat(
898 config['late_commands'],
899 Contains('driver_07_update_initramfs'))
900
901
853class TestGetCurtinUserDataOS(902class TestGetCurtinUserDataOS(
854 PreseedRPCMixin, BootImageHelperMixin, MAASTransactionServerTestCase):903 PreseedRPCMixin, BootImageHelperMixin, MAASTransactionServerTestCase):
855 """Tests for `get_curtin_userdata` using os specific scenarios."""904 """Tests for `get_curtin_userdata` using os specific scenarios."""

Subscribers

People subscribed via source and target branches

to all changes: