Merge lp:~blake-rouse/maas/fix-1487135 into lp:~maas-committers/maas/trunk

Proposed by Blake Rouse
Status: Merged
Approved by: Blake Rouse
Approved revision: no longer in the source branch.
Merged at revision: 4471
Proposed branch: lp:~blake-rouse/maas/fix-1487135
Merge into: lp:~maas-committers/maas/trunk
Diff against target: 259 lines (+130/-12)
5 files modified
src/maasserver/api/nodes.py (+27/-0)
src/maasserver/api/tests/test_doc.py (+1/-0)
src/maasserver/api/tests/test_node.py (+42/-0)
src/maasserver/preseed.py (+26/-12)
src/maasserver/tests/test_preseed.py (+34/-0)
To merge this branch: bzr merge lp:~blake-rouse/maas/fix-1487135
Reviewer Review Type Date Requested Status
Mike Pontillo (community) Approve
Jeffrey C Jones (community) Approve
Review via email: mp+276689@code.launchpad.net

Commit message

Add get_curtin_config API to return the curtin configuration that was passed when the node started deployment.

To post a comment you must log in.
Revision history for this message
Blake Rouse (blake-rouse) wrote :

I did this because I kept needing a way to get the configuration when curtin deployments fail.

Revision history for this message
Jeffrey C Jones (trapnine) wrote :

looks good, just a comment typo

review: Approve
Revision history for this message
Mike Pontillo (mpontillo) wrote :

This is great. Thanks for adding this.

review: Approve
Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (1013.7 KiB)

The attempt to merge lp:~blake-rouse/maas/fix-1487135 into lp:maas failed. Below is the output from the failed tests.

Get:1 http://security.ubuntu.com trusty-security InRelease [64.4 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Get:2 http://nova.clouds.archive.ubuntu.com trusty-updates InRelease [64.4 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:3 http://security.ubuntu.com trusty-security/main Sources [98.0 kB]
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [242 kB]
Get:5 http://security.ubuntu.com trusty-security/universe Sources [31.0 kB]
Get:6 http://security.ubuntu.com trusty-security/main amd64 Packages [361 kB]
Get:7 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [143 kB]
Get:8 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [644 kB]
Get:9 http://security.ubuntu.com trusty-security/universe amd64 Packages [117 kB]
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Get:10 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [326 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Fetched 2,092 kB in 4s (502 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 phantomjs postgresql pyflakes python-apt python-bson python-bzrlib python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-jinja2 python-jsonschema python-lxml python-mock python-netaddr python-netifaces python-nose python-oauth python-openssl python-paramiko python-pexpect python-pip python-pocket-lint python-psycopg2 python-pyinotify python-pyparsing python-seamicroclient python-simplejson python-simplestreams python-sphinx python-subunit python-tempita python-testresources python-testscenarios python-te...

Revision history for this message
Andres Rodriguez (andreserl) wrote :

should we have get_storage_config and get_network_config or something along those lines?

Revision history for this message
Blake Rouse (blake-rouse) wrote :

No because it is the same config. It is shown in the output.

Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (1013.7 KiB)

The attempt to merge lp:~blake-rouse/maas/fix-1487135 into lp:maas failed. Below is the output from the failed tests.

Get:1 http://security.ubuntu.com trusty-security InRelease [64.4 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Get:2 http://nova.clouds.archive.ubuntu.com trusty-updates InRelease [64.4 kB]
Get:3 http://security.ubuntu.com trusty-security/main Sources [98.0 kB]
Get:4 http://security.ubuntu.com trusty-security/universe Sources [31.0 kB]
Get:5 http://security.ubuntu.com trusty-security/main amd64 Packages [361 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:6 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [242 kB]
Get:7 http://security.ubuntu.com trusty-security/universe amd64 Packages [117 kB]
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Get:8 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [143 kB]
Get:9 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [644 kB]
Get:10 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [326 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Fetched 2,092 kB in 4s (493 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 phantomjs postgresql pyflakes python-apt python-bson python-bzrlib python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-jinja2 python-jsonschema python-lxml python-mock python-netaddr python-netifaces python-nose python-oauth python-openssl python-paramiko python-pexpect python-pip python-pocket-lint python-psycopg2 python-pyinotify python-pyparsing python-seamicroclient python-simplejson python-simplestreams python-sphinx python-subunit python-tempita python-testresources python-testscenarios python-te...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'src/maasserver/api/nodes.py'
--- src/maasserver/api/nodes.py 2015-11-04 23:38:04 +0000
+++ src/maasserver/api/nodes.py 2015-11-05 14:17:35 +0000
@@ -78,6 +78,7 @@
78from maasserver.models.node import RELEASABLE_STATUSES78from maasserver.models.node import RELEASABLE_STATUSES
79from maasserver.models.nodeprobeddetails import get_single_probed_details79from maasserver.models.nodeprobeddetails import get_single_probed_details
80from maasserver.node_constraint_filter_forms import AcquireNodeForm80from maasserver.node_constraint_filter_forms import AcquireNodeForm
81from maasserver.preseed import get_curtin_merged_config
81from maasserver.rpc import getClientFor82from maasserver.rpc import getClientFor
82from maasserver.storage_layouts import (83from maasserver.storage_layouts import (
83 StorageLayoutError,84 StorageLayoutError,
@@ -96,6 +97,7 @@
96from provisioningserver.rpc.cluster import PowerQuery97from provisioningserver.rpc.cluster import PowerQuery
97from provisioningserver.rpc.exceptions import NoConnectionsAvailable98from provisioningserver.rpc.exceptions import NoConnectionsAvailable
98import simplejson as json99import simplejson as json
100import yaml
99101
100# Node's fields exposed on the API.102# Node's fields exposed on the API.
101DISPLAYED_NODE_FIELDS = (103DISPLAYED_NODE_FIELDS = (
@@ -930,6 +932,10 @@
930 If the default gateways need to be specific for this node you can set932 If the default gateways need to be specific for this node you can set
931 which interface and subnet's gateway to use when this node is deployed933 which interface and subnet's gateway to use when this node is deployed
932 with the `node-interfaces set-default-gateway` API.934 with the `node-interfaces set-default-gateway` API.
935
936 Returns 404 if the node could not be found.
937 Returns 403 if the user does not have permission to clear the default
938 gateways.
933 """939 """
934 node = Node.nodes.get_node_or_404(940 node = Node.nodes.get_node_or_404(
935 system_id=system_id, user=request.user, perm=NODE_PERMISSION.ADMIN)941 system_id=system_id, user=request.user, perm=NODE_PERMISSION.ADMIN)
@@ -938,6 +944,27 @@
938 node.save()944 node.save()
939 return node945 return node
940946
947 @operation(idempotent=True)
948 def get_curtin_config(self, request, system_id):
949 """Return the rendered curtin configuration for the node.
950
951 Returns 404 if the node could not be found.
952 Returns 403 if the user does not have permission to get the curtin
953 configuration.
954 """
955 node = Node.nodes.get_node_or_404(
956 system_id=system_id, user=request.user, perm=NODE_PERMISSION.VIEW)
957 if node.status not in [
958 NODE_STATUS.DEPLOYING,
959 NODE_STATUS.DEPLOYED,
960 NODE_STATUS.FAILED_DEPLOYMENT]:
961 raise MAASAPIBadRequest(
962 "Node %s is not in a deployment state." % node.hostname)
963 return HttpResponse(
964 yaml.safe_dump(
965 get_curtin_merged_config(node), default_flow_style=False),
966 content_type='text/plain')
967
941968
942def create_node(request):969def create_node(request):
943 """Service an http request to create a node.970 """Service an http request to create a node.
944971
=== modified file 'src/maasserver/api/tests/test_doc.py'
--- src/maasserver/api/tests/test_doc.py 2015-09-15 19:07:22 +0000
+++ src/maasserver/api/tests/test_doc.py 2015-11-05 14:17:35 +0000
@@ -323,6 +323,7 @@
323 "POST set_storage_layout op=set_storage_layout restful=False",323 "POST set_storage_layout op=set_storage_layout restful=False",
324 "POST clear_default_gateways op=clear_default_gateways "324 "POST clear_default_gateways op=clear_default_gateways "
325 "restful=False",325 "restful=False",
326 "GET get_curtin_config op=get_curtin_config restful=False",
326 }327 }
327 observed_actions = {328 observed_actions = {
328 "%(method)s %(name)s op=%(op)s restful=%(restful)s" % action329 "%(method)s %(name)s op=%(op)s restful=%(restful)s" % action
329330
=== modified file 'src/maasserver/api/tests/test_node.py'
--- src/maasserver/api/tests/test_node.py 2015-10-27 20:53:16 +0000
+++ src/maasserver/api/tests/test_node.py 2015-11-05 14:17:35 +0000
@@ -24,6 +24,7 @@
24from django.core.urlresolvers import reverse24from django.core.urlresolvers import reverse
25from django.db import transaction25from django.db import transaction
26from maasserver import forms26from maasserver import forms
27from maasserver.api import nodes as nodes_module
27from maasserver.enum import (28from maasserver.enum import (
28 INTERFACE_TYPE,29 INTERFACE_TYPE,
29 IPADDRESS_TYPE,30 IPADDRESS_TYPE,
@@ -83,6 +84,7 @@
83 HasLength,84 HasLength,
84 Not,85 Not,
85)86)
87import yaml
8688
8789
88class NodeAnonAPITest(MAASServerTestCase):90class NodeAnonAPITest(MAASServerTestCase):
@@ -2186,3 +2188,43 @@
2186 node = reload_object(node)2188 node = reload_object(node)
2187 self.assertIsNone(node.gateway_link_ipv4)2189 self.assertIsNone(node.gateway_link_ipv4)
2188 self.assertIsNone(node.gateway_link_ipv6)2190 self.assertIsNone(node.gateway_link_ipv6)
2191
2192
2193class TestGetCurtinConfig(APITestCase):
2194
2195 def get_node_uri(self, node):
2196 """Get the API URI for `node`."""
2197 return reverse('node_handler', args=[node.system_id])
2198
2199 def test__500_when_node_not_in_deployment_state(self):
2200 node = factory.make_Node(
2201 owner=self.logged_in_user,
2202 status=factory.pick_enum(
2203 NODE_STATUS, but_not=[
2204 NODE_STATUS.DEPLOYING,
2205 NODE_STATUS.DEPLOYED,
2206 NODE_STATUS.FAILED_DEPLOYMENT,
2207 ]))
2208 response = self.client.get(
2209 self.get_node_uri(node), {'op': 'get_curtin_config'})
2210 self.assertEquals(
2211 httplib.BAD_REQUEST, response.status_code, response.content)
2212
2213 def test__returns_curtin_config_in_yaml(self):
2214 node = factory.make_Node(
2215 owner=self.logged_in_user, status=NODE_STATUS.DEPLOYING)
2216 fake_config = {
2217 "config": factory.make_name("config")
2218 }
2219 mock_get_curtin_merged_config = self.patch(
2220 nodes_module, "get_curtin_merged_config")
2221 mock_get_curtin_merged_config.return_value = fake_config
2222 response = self.client.get(
2223 self.get_node_uri(node), {'op': 'get_curtin_config'})
2224 self.assertEquals(
2225 httplib.OK, response.status_code, response.content)
2226 self.assertEquals(
2227 yaml.safe_dump(fake_config, default_flow_style=False),
2228 response.content)
2229 self.assertThat(
2230 mock_get_curtin_merged_config, MockCalledOnceWith(node))
21892231
=== modified file 'src/maasserver/preseed.py'
--- src/maasserver/preseed.py 2015-10-30 16:14:33 +0000
+++ src/maasserver/preseed.py 2015-11-05 14:17:35 +0000
@@ -31,6 +31,7 @@
3131
32from crochet import TimeoutError32from crochet import TimeoutError
33from curtin.commands import block_meta33from curtin.commands import block_meta
34from curtin.config import merge_config
34from curtin.pack import pack_install35from curtin.pack import pack_install
35from django.conf import settings36from django.conf import settings
36from maasserver import logger37from maasserver import logger
@@ -216,14 +217,8 @@
216 return []217 return []
217218
218219
219def get_curtin_userdata(node):220def get_curtin_yaml_config(node):
220 """Return the curtin user-data.221 """Return the curtin configration for the node."""
221
222 :param node: The node for which to generate the user-data.
223 :return: The rendered user-data string.
224 :rtype: unicode.
225 """
226 installer_url = get_curtin_installer_url(node)
227 main_config = get_curtin_config(node)222 main_config = get_curtin_config(node)
228 reporter_config = compose_curtin_maas_reporter(node)223 reporter_config = compose_curtin_maas_reporter(node)
229 swap_config = compose_curtin_swap_preseed(node)224 swap_config = compose_curtin_swap_preseed(node)
@@ -253,13 +248,32 @@
253 else:248 else:
254 storage_config = []249 storage_config = []
255250
251 return (
252 [main_config] + reporter_config + storage_config + network_config +
253 swap_config + kernel_config + verbose_config)
254
255
256def get_curtin_merged_config(node):
257 """Return the merged curtin configuration for the node."""
258 yaml_config = get_curtin_yaml_config(node)
259 config = {}
260 for cfg in yaml_config:
261 merge_config(config, yaml.load(cfg))
262 return config
263
264
265def get_curtin_userdata(node):
266 """Return the curtin user-data.
267
268 :param node: The node for which to generate the user-data.
269 :return: The rendered user-data string.
270 :rtype: unicode.
271 """
256 # Pack the curtin and the configuration into a script to execute on the272 # Pack the curtin and the configuration into a script to execute on the
257 # deploying node.273 # deploying node.
258 return pack_install(274 return pack_install(
259 configs=(275 configs=get_curtin_yaml_config(node),
260 [main_config] + reporter_config + storage_config + network_config +276 args=[get_curtin_installer_url(node)])
261 swap_config + kernel_config + verbose_config),
262 args=[installer_url])
263277
264278
265def get_curtin_image(node):279def get_curtin_image(node):
266280
=== modified file 'src/maasserver/tests/test_preseed.py'
--- src/maasserver/tests/test_preseed.py 2015-10-30 16:03:19 +0000
+++ src/maasserver/tests/test_preseed.py 2015-11-05 14:17:35 +0000
@@ -53,6 +53,7 @@
53 get_curtin_context,53 get_curtin_context,
54 get_curtin_image,54 get_curtin_image,
55 get_curtin_installer_url,55 get_curtin_installer_url,
56 get_curtin_merged_config,
56 get_curtin_userdata,57 get_curtin_userdata,
57 get_enlist_preseed,58 get_enlist_preseed,
58 get_netloc_and_path,59 get_netloc_and_path,
@@ -83,6 +84,7 @@
83 MockNotCalled,84 MockNotCalled,
84)85)
85from metadataserver.models import NodeKey86from metadataserver.models import NodeKey
87from mock import sentinel
86from provisioningserver.drivers.osystem.ubuntu import UbuntuOS88from provisioningserver.drivers.osystem.ubuntu import UbuntuOS
87from provisioningserver.rpc.exceptions import NoConnectionsAvailable89from provisioningserver.rpc.exceptions import NoConnectionsAvailable
88from provisioningserver.utils.enum import map_enum90from provisioningserver.utils.enum import map_enum
@@ -733,6 +735,38 @@
733 }, yaml.load(preseed[0]))735 }, yaml.load(preseed[0]))
734736
735737
738class TestGetCurtinMergedConfig(MAASServerTestCase):
739
740 def test__merges_configs_together(self):
741 configs = [
742 yaml.safe_dump({
743 "maas": {
744 "test": "data"
745 },
746 "override": "data",
747 }),
748 yaml.safe_dump({
749 "maas2": {
750 "test": "data2"
751 },
752 "override": "data2",
753 }),
754 ]
755 mock_yaml_config = self.patch_autospec(
756 preseed_module, "get_curtin_yaml_config")
757 mock_yaml_config.return_value = configs
758 self.assertEquals({
759 "maas": {
760 "test": "data"
761 },
762 "maas2": {
763 "test": "data2"
764 },
765 "override": "data2",
766 }, get_curtin_merged_config(sentinel.node))
767 self.assertThat(mock_yaml_config, MockCalledOnceWith(sentinel.node))
768
769
736class TestGetCurtinUserData(770class TestGetCurtinUserData(
737 PreseedRPCMixin, BootImageHelperMixin, MAASServerTestCase):771 PreseedRPCMixin, BootImageHelperMixin, MAASServerTestCase):
738 """Tests for `get_curtin_userdata`."""772 """Tests for `get_curtin_userdata`."""