Merge lp:~newell-jensen/maas/2.2-fix-1694759 into lp:maas/2.2

Proposed by Newell Jensen on 2017-06-07
Status: Merged
Approved by: Mike Pontillo on 2017-06-16
Approved revision: 6061
Merged at revision: 6062
Proposed branch: lp:~newell-jensen/maas/2.2-fix-1694759
Merge into: lp:maas/2.2
Diff against target: 235 lines (+85/-72)
2 files modified
src/provisioningserver/drivers/pod/rsd.py (+14/-10)
src/provisioningserver/drivers/pod/tests/test_rsd.py (+71/-62)
To merge this branch: bzr merge lp:~newell-jensen/maas/2.2-fix-1694759
Reviewer Review Type Date Requested Status
Newell Jensen Approve on 2017-06-07
Review via email: mp+325260@code.launchpad.net

Commit message

Backport trunk r6074: Delete newly allocated RSD node if the assembling process fails.

To post a comment you must log in.
Newell Jensen (newell-jensen) wrote :

Self review backport.

review: Approve
MAAS Lander (maas-lander) wrote :
Download full text (13.6 KiB)

The attempt to merge lp:~newell-jensen/maas/2.2-fix-1694759 into lp:maas/2.2 failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:3 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu xenial-security InRelease
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libnss-wrapper libpq-dev make nodejs-legacy npm postgresql psmisc pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
psmisc is already the newest version (22.21-2.1build1).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-0.1build2).
python-psycopg2 is already the newest version (2.6.1-1...

MAAS Lander (maas-lander) wrote :
Download full text (124.6 KiB)

The attempt to merge lp:~newell-jensen/maas/2.2-fix-1694759 into lp:maas/2.2 failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:4 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Fetched 306 kB in 0s (336 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind avahi-utils bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common isc-dhcp-server libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libnss-wrapper libpq-dev make nodejs-legacy npm postgresql psmisc pxelinux python3-all python3-apt python3-attr python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version (2.1.1+nmu1).
avahi-utils is already the newest version (0.6.32~rc+dfsg-1ubuntu2).
build-essential is already the newest version (12.1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
psmisc is already the newest version (22.21-2.1build1).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-formencode is already the newest version (1.3.0-0ubuntu5).
python-lxml is already the newest version (3.5.0-1build1).
python-netaddr is already the newest version (0.7.18-1).
python-netifaces is already the newest version (0.10.4-0.1buil...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/provisioningserver/drivers/pod/rsd.py'
2--- src/provisioningserver/drivers/pod/rsd.py 2017-05-09 19:10:12 +0000
3+++ src/provisioningserver/drivers/pod/rsd.py 2017-06-07 17:24:55 +0000
4@@ -1046,11 +1046,8 @@
5 "Unable to allocate machine with requested resources.")
6
7 @inlineCallbacks
8- def decompose(self, pod_id, context):
9- """Decompose machine."""
10- url = self.get_url(context)
11- node_id = context.get('node_id').encode('utf-8')
12- headers = self.make_auth_headers(**context)
13+ def delete_node(self, url, node_id, headers):
14+ """Delete node at node_id."""
15 # Delete machine at node_id.
16 endpoint = b"redfish/v1/Nodes/%s" % node_id
17 try:
18@@ -1060,12 +1057,17 @@
19 # XXX newell 2017-02-27 bug=1667754:
20 # Catch the 404 error when trying to decompose the
21 # resource that has already been decomposed.
22- # This is a work around and will need to be handled
23- # differently on the region so we don't try to
24- # decompose a machine multiple times.
25 if int(error.status) != HTTPStatus.NOT_FOUND:
26 raise
27
28+ @inlineCallbacks
29+ def decompose(self, pod_id, context):
30+ """Decompose machine."""
31+ url = self.get_url(context)
32+ node_id = context.get('node_id').encode('utf-8')
33+ headers = self.make_auth_headers(**context)
34+ yield self.delete_node(url, node_id, headers)
35+
36 # Retrieve pod resources.
37 discovered_pod = yield self.get_pod_resources(url, headers)
38 # Retrive pod hints.
39@@ -1114,7 +1116,8 @@
40 yield self.redfish_request(
41 b"POST", join(url, endpoint), headers)
42 elif node_state == 'Failed':
43- # Broken system.
44+ # Broken system, delete allocated node.
45+ yield self.delete_node(url, node_id, headers)
46 raise PodFatalError(
47 "Composed machine at node ID %s has a ComposedNodeState"
48 " of Failed." % node_id)
49@@ -1131,7 +1134,8 @@
50 # Check one last time if the state has became `Failed`.
51
52 if node_state == 'Failed':
53- # Broken system.
54+ # Broken system, delete allocated node.
55+ yield self.delete_node(url, node_id, headers)
56 raise PodFatalError(
57 "Composed machine at node ID %s has a ComposedNodeState"
58 " of Failed." % node_id)
59
60=== modified file 'src/provisioningserver/drivers/pod/tests/test_rsd.py'
61--- src/provisioningserver/drivers/pod/tests/test_rsd.py 2017-05-09 21:53:26 +0000
62+++ src/provisioningserver/drivers/pod/tests/test_rsd.py 2017-06-07 17:24:55 +0000
63@@ -2002,74 +2002,77 @@
64 factory.make_name('system_id'), context, request)
65
66 @inlineCallbacks
67+ def test__delete_node(self):
68+ driver = RSDPodDriver()
69+ context = make_context()
70+ url = driver.get_url(context)
71+ node_id = context.get('node_id').encode('utf-8')
72+ endpoint = b"redfish/v1/Nodes/%s" % node_id
73+ headers = driver.make_auth_headers(**context)
74+ mock_redfish_request = self.patch(driver, 'redfish_request')
75+
76+ yield driver.delete_node(url, node_id, headers)
77+ self.assertThat(mock_redfish_request, MockCalledOnceWith(
78+ b"DELETE", join(url, endpoint), headers))
79+
80+ @inlineCallbacks
81+ def test_delete_node_continues_on_404_error(self):
82+ driver = RSDPodDriver()
83+ context = make_context()
84+ url = driver.get_url(context)
85+ node_id = context.get('node_id').encode('utf-8')
86+ endpoint = b"redfish/v1/Nodes/%s" % node_id
87+ headers = driver.make_auth_headers(**context)
88+ mock_redfish_request = self.patch(driver, 'redfish_request')
89+ error = PartialDownloadError(
90+ response=json.dumps(SAMPLE_JSON_SYSTEMS).encode('utf-8'),
91+ code=HTTPStatus.NOT_FOUND)
92+ mock_redfish_request.side_effect = error
93+
94+ yield driver.delete_node(url, node_id, headers)
95+ self.assertThat(mock_redfish_request, MockCalledOnceWith(
96+ b"DELETE", join(url, endpoint), headers))
97+
98+ @inlineCallbacks
99+ def test_delete_node_raises_when_not_404_error(self):
100+ driver = RSDPodDriver()
101+ context = make_context()
102+ url = driver.get_url(context)
103+ node_id = context.get('node_id').encode('utf-8')
104+ endpoint = b"redfish/v1/Nodes/%s" % node_id
105+ headers = driver.make_auth_headers(**context)
106+ mock_redfish_request = self.patch(driver, 'redfish_request')
107+ error = PartialDownloadError(
108+ response=json.dumps(SAMPLE_JSON_SYSTEMS).encode('utf-8'),
109+ code=HTTPStatus.BAD_REQUEST)
110+ mock_redfish_request.side_effect = error
111+
112+ with ExpectedException(PartialDownloadError):
113+ yield driver.delete_node(url, node_id, headers)
114+ self.assertThat(mock_redfish_request, MockCalledOnceWith(
115+ b"DELETE", join(url, endpoint), headers))
116+
117+ @inlineCallbacks
118 def test__decompose(self):
119 driver = RSDPodDriver()
120 context = make_context()
121 url = driver.get_url(context)
122 node_id = context.get('node_id').encode('utf-8')
123 headers = driver.make_auth_headers(**context)
124- endpoint = b"redfish/v1/Nodes/%s" % node_id
125- mock_redfish_request = self.patch(driver, 'redfish_request')
126- mock_get_pod_resources = self.patch(driver, 'get_pod_resources')
127- discovered_pod = make_discovered_pod()
128- mock_get_pod_resources.return_value = discovered_pod
129- mock_get_pod_hints = self.patch(driver, 'get_pod_hints')
130-
131- yield driver.decompose(
132- factory.make_name('system_id'), context)
133- self.assertThat(mock_redfish_request, MockCalledOnceWith(
134- b"DELETE", join(url, endpoint), headers))
135- self.assertThat(mock_get_pod_resources, MockCalledOnceWith(
136- url, headers))
137- self.assertThat(mock_get_pod_hints, MockCalledOnceWith(
138- discovered_pod))
139-
140- @inlineCallbacks
141- def test_decompose_continues_on_404_error(self):
142- driver = RSDPodDriver()
143- context = make_context()
144- url = driver.get_url(context)
145- node_id = context.get('node_id').encode('utf-8')
146- headers = driver.make_auth_headers(**context)
147- endpoint = b"redfish/v1/Nodes/%s" % node_id
148- mock_redfish_request = self.patch(driver, 'redfish_request')
149- error = PartialDownloadError(
150- response=json.dumps(SAMPLE_JSON_SYSTEMS).encode('utf-8'),
151- code=HTTPStatus.NOT_FOUND)
152- mock_redfish_request.side_effect = error
153- mock_get_pod_resources = self.patch(driver, 'get_pod_resources')
154- discovered_pod = make_discovered_pod()
155- mock_get_pod_resources.return_value = discovered_pod
156- mock_get_pod_hints = self.patch(driver, 'get_pod_hints')
157-
158- yield driver.decompose(
159- factory.make_name('system_id'), context)
160- self.assertThat(mock_redfish_request, MockCalledOnceWith(
161- b"DELETE", join(url, endpoint), headers))
162- self.assertThat(mock_get_pod_resources, MockCalledOnceWith(
163- url, headers))
164- self.assertThat(mock_get_pod_hints, MockCalledOnceWith(
165- discovered_pod))
166-
167- @inlineCallbacks
168- def test_decompose_raises_when_not_404_error(self):
169- driver = RSDPodDriver()
170- context = make_context()
171- url = driver.get_url(context)
172- node_id = context.get('node_id').encode('utf-8')
173- headers = driver.make_auth_headers(**context)
174- endpoint = b"redfish/v1/Nodes/%s" % node_id
175- mock_redfish_request = self.patch(driver, 'redfish_request')
176- error = PartialDownloadError(
177- response=json.dumps(SAMPLE_JSON_SYSTEMS).encode('utf-8'),
178- code=HTTPStatus.BAD_REQUEST)
179- mock_redfish_request.side_effect = error
180-
181- with ExpectedException(PartialDownloadError):
182- yield driver.decompose(
183- factory.make_name('system_id'), context)
184- self.assertThat(mock_redfish_request, MockCalledOnceWith(
185- b"DELETE", join(url, endpoint), headers))
186+ mock_delete_node = self.patch(driver, 'delete_node')
187+ mock_get_pod_resources = self.patch(driver, 'get_pod_resources')
188+ discovered_pod = make_discovered_pod()
189+ mock_get_pod_resources.return_value = discovered_pod
190+ mock_get_pod_hints = self.patch(driver, 'get_pod_hints')
191+
192+ yield driver.decompose(
193+ factory.make_name('system_id'), context)
194+ self.assertThat(mock_delete_node, MockCalledOnceWith(
195+ url, node_id, headers))
196+ self.assertThat(mock_get_pod_resources, MockCalledOnceWith(
197+ url, headers))
198+ self.assertThat(mock_get_pod_hints, MockCalledOnceWith(
199+ discovered_pod))
200
201 @inlineCallbacks
202 def test__set_pxe_boot(self):
203@@ -2164,12 +2167,15 @@
204 mock_get_composed_node_state = self.patch(
205 driver, 'get_composed_node_state')
206 mock_get_composed_node_state.return_value = "Failed"
207+ mock_delete_node = self.patch(driver, 'delete_node')
208 mock_redfish_request = self.patch(driver, 'redfish_request')
209
210 with ExpectedException(PodFatalError):
211 yield driver.assemble_node(url, node_id, headers)
212 self.assertThat(mock_get_composed_node_state, MockCalledOnceWith(
213 url, node_id, headers))
214+ self.assertThat(mock_delete_node, MockCalledOnceWith(
215+ url, node_id, headers))
216 self.assertThat(mock_redfish_request, MockNotCalled())
217
218 @inlineCallbacks
219@@ -2183,6 +2189,7 @@
220 driver, 'get_composed_node_state')
221 mock_get_composed_node_state.side_effect = [
222 "Allocated", "Assembling", "Failed"]
223+ mock_delete_node = self.patch(driver, 'delete_node')
224 mock_redfish_request = self.patch(driver, 'redfish_request')
225 endpoint = (
226 b"redfish/v1/Nodes/%s/Actions/ComposedNode.Assemble" % node_id)
227@@ -2196,6 +2203,8 @@
228 call(url, node_id, headers)))
229 self.assertThat(mock_redfish_request, MockCalledOnceWith(
230 b"POST", uri, headers))
231+ self.assertThat(mock_delete_node, MockCalledOnceWith(
232+ url, node_id, headers))
233
234 @inlineCallbacks
235 def test_power_issues_power_reset(self):

Subscribers

People subscribed via source and target branches

to all changes: