Merge lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1 into lp:maas/2.1

Proposed by Mike Pontillo on 2017-02-24
Status: Work in progress
Proposed branch: lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1
Merge into: lp:maas/2.1
Diff against target: 121 lines (+67/-2)
3 files modified
contrib/preseeds_v2/curtin_userdata (+16/-2)
docs/changelog.rst (+2/-0)
src/maasserver/tests/test_preseed.py (+49/-0)
To merge this branch: bzr merge lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1
Reviewer Review Type Date Requested Status
Mike Pontillo (community) Approve on 2017-02-24
Review via email: mp+318179@code.launchpad.net

Commit message

Merge revision 5756 from trunk.
Fix curtin config rendering for third-party drivers.

To post a comment you must log in.
Mike Pontillo (mpontillo) wrote :

Self-approved backport.

review: Approve
MAAS Lander (maas-lander) wrote :
Download full text (2.4 MiB)

The attempt to merge lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1 into lp:maas/2.1 failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:4 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Fetched 306 kB in 0s (757 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
git is already the newest version (1:2.7.4-0ubuntu1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-...

MAAS Lander (maas-lander) wrote :
Download full text (2.4 MiB)

The attempt to merge lp:~mpontillo/maas/fix-curtin-custom-driver-preseed--bug-1667426--2.1 into lp:maas/2.1 failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Get:4 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Fetched 306 kB in 0s (667 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
git is already the newest version (1:2.7.4-0ubuntu1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-...

Mike Pontillo (mpontillo) wrote :

Hm, at first I thought the 1.9 lander was just being the 1.9 lander again, but it seems that the preseed code has changed significantly enough that these modifications aren't an easy backport.

With a MAAS 2.2 release imminent, that makes this lower priority for 2.1. We'll circle back if the need arises.

Unmerged revisions

5591. By Mike Pontillo on 2017-02-24

Update changelog.

5590. By Mike Pontillo on 2017-02-24

Merge revision 5756 from trunk.
Fix curtin config rendering for third-party drivers.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'contrib/preseeds_v2/curtin_userdata'
2--- contrib/preseeds_v2/curtin_userdata 2016-12-08 13:45:40 +0000
3+++ contrib/preseeds_v2/curtin_userdata 2017-02-24 04:33:23 +0000
4@@ -6,21 +6,35 @@
5 {{endfor}}
6 {{if third_party_drivers and driver}}
7 early_commands:
8- {{py: key_string = ''.join(['\\x%x' % x for x in map(ord, driver['key_binary'])])}}
9+ {{py: key_string = ''.join(['\\x%x' % x for x in driver['key_binary']])}}
10+ {{if driver['key_binary'] and driver['repository'] and driver['package']}}
11 driver_00_get_key: /bin/echo -en '{{key_string}}' > /tmp/maas-{{driver['package']}}.gpg
12 driver_01_add_key: ["apt-key", "add", "/tmp/maas-{{driver['package']}}.gpg"]
13+ {{endif}}
14+ {{if driver['repository']}}
15 driver_02_add: ["add-apt-repository", "-y", "deb {{driver['repository']}} {{node.get_distro_series()}} main"]
16+ {{endif}}
17+ {{if driver['package']}}
18 driver_03_update_install: ["sh", "-c", "apt-get update --quiet && apt-get --assume-yes install {{driver['package']}}"]
19- driver_04_load: ["sh", "-c", "depmod && modprobe {{driver['module']}}"]
20+ {{endif}}
21+ {{if driver['module']}}
22+ driver_04_load: ["sh", "-c", "depmod && modprobe {{driver['module']}} || echo 'Warning: Failed to load module: {{driver['module']}}'"]
23+ {{endif}}
24 {{endif}}
25 late_commands:
26 maas: [wget, '--no-proxy', {{node_disable_pxe_url|escape.json}}, '--post-data', {{node_disable_pxe_data|escape.json}}, '-O', '/dev/null']
27 {{if third_party_drivers and driver}}
28+ {{if driver['key_binary'] and driver['repository'] and driver['package']}}
29 driver_00_key_get: curtin in-target -- sh -c "/bin/echo -en '{{key_string}}' > /tmp/maas-{{driver['package']}}.gpg"
30 driver_02_key_add: ["curtin", "in-target", "--", "apt-key", "add", "/tmp/maas-{{driver['package']}}.gpg"]
31+ {{endif}}
32+ {{if driver['repository']}}
33 driver_03_add: ["curtin", "in-target", "--", "add-apt-repository", "-y", "deb {{driver['repository']}} {{node.get_distro_series()}} main"]
34+ {{endif}}
35 driver_04_update_install: ["curtin", "in-target", "--", "apt-get", "update", "--quiet"]
36+ {{if driver['package']}}
37 driver_05_install: ["curtin", "in-target", "--", "apt-get", "-y", "install", "{{driver['package']}}"]
38+ {{endif}}
39 driver_06_depmod: ["curtin", "in-target", "--", "depmod"]
40 driver_07_update_initramfs: ["curtin", "in-target", "--", "update-initramfs", "-u"]
41 {{endif}}
42
43=== modified file 'docs/changelog.rst'
44--- docs/changelog.rst 2017-02-17 05:59:11 +0000
45+++ docs/changelog.rst 2017-02-24 04:33:23 +0000
46@@ -34,6 +34,8 @@
47
48 LP: #1665459 Fix anonymous auto-enlistment to properly detect rack-facing region IP address
49
50+LP: #1667426 Fix curtin config rendering for custom third-party drivers.
51+
52
53 Other notable changes
54 ---------------------
55
56=== modified file 'src/maasserver/tests/test_preseed.py'
57--- src/maasserver/tests/test_preseed.py 2016-12-13 17:26:28 +0000
58+++ src/maasserver/tests/test_preseed.py 2017-02-24 04:33:23 +0000
59@@ -77,6 +77,7 @@
60 MAASServerTestCase,
61 MAASTransactionServerTestCase,
62 )
63+from maasserver.third_party_drivers import DriversConfig
64 from maasserver.utils import absolute_reverse
65 from maasserver.utils.curtin import curtin_supports_webhook_events
66 from maastesting.matchers import (
67@@ -850,6 +851,54 @@
68 self.assertThat(mock_supports_storage, MockCalledOnceWith())
69
70
71+class TestRenderCurtinUserdataWithThirdPartyDrivers(
72+ PreseedRPCMixin, BootImageHelperMixin, MAASServerTestCase):
73+ """Ensures curtin configs for all third-party drivers can be rendered."""
74+
75+ # Try rendering each driver in drivers.yaml.
76+ scenarios = [
77+ (driver['comment'], {'driver': driver})
78+ for driver in DriversConfig.load_from_cache()['drivers']
79+ ]
80+
81+ def test_render_curtin_preseed_with_third_party_driver(self):
82+ node = factory.make_Node_with_Interface_on_Subnet(
83+ primary_rack=self.rpc_rack_controller)
84+ Config.objects.set_config(
85+ 'enable_third_party_drivers', True)
86+ self.configure_get_boot_images_for_node(node, 'xinstall')
87+ get_third_party_driver = self.patch(
88+ preseed_module, "get_third_party_driver")
89+ get_third_party_driver.return_value = self.driver
90+ curtin_config_text = get_curtin_config(node)
91+ config = yaml.safe_load(curtin_config_text)
92+ self.assertThat(
93+ config['early_commands'], Contains('driver_00_get_key'))
94+ self.assertThat(
95+ config['early_commands'], Contains('driver_01_add_key'))
96+ self.assertThat(
97+ config['early_commands'], Contains('driver_02_add'))
98+ self.assertThat(
99+ config['early_commands'], Contains('driver_03_update_install'))
100+ self.assertThat(
101+ config['early_commands'], Contains('driver_04_load'))
102+ self.assertThat(
103+ config['late_commands'], Contains('driver_00_key_get'))
104+ self.assertThat(
105+ config['late_commands'], Contains('driver_02_key_add'))
106+ self.assertThat(
107+ config['late_commands'], Contains('driver_03_add'))
108+ self.assertThat(
109+ config['late_commands'], Contains('driver_04_update_install'))
110+ self.assertThat(
111+ config['late_commands'], Contains('driver_05_install'))
112+ self.assertThat(
113+ config['late_commands'], Contains('driver_06_depmod'))
114+ self.assertThat(
115+ config['late_commands'],
116+ Contains('driver_07_update_initramfs'))
117+
118+
119 class TestGetCurtinUserDataOS(
120 PreseedRPCMixin, BootImageHelperMixin, MAASTransactionServerTestCase):
121 """Tests for `get_curtin_userdata` using os specific scenarios."""

Subscribers

People subscribed via source and target branches

to all changes: