Merge lp:~ltrager/maas/global_min_hwe_kernel into lp:~maas-committers/maas/trunk

Proposed by Lee Trager
Status: Merged
Approved by: Lee Trager
Approved revision: no longer in the source branch.
Merged at revision: 4406
Proposed branch: lp:~ltrager/maas/global_min_hwe_kernel
Merge into: lp:~maas-committers/maas/trunk
Diff against target: 367 lines (+140/-5)
12 files modified
src/maasserver/api/pxeconfig.py (+12/-1)
src/maasserver/api/tests/test_pxeconfig.py (+20/-0)
src/maasserver/forms.py (+7/-1)
src/maasserver/forms_settings.py (+39/-0)
src/maasserver/models/config.py (+1/-0)
src/maasserver/models/node.py (+6/-0)
src/maasserver/models/tests/test_node.py (+14/-0)
src/maasserver/static/js/angular/controllers/add_hardware.js (+3/-1)
src/maasserver/static/js/angular/factories/general.js (+10/-0)
src/maasserver/static/js/angular/factories/tests/test_general.js (+15/-2)
src/maasserver/tests/test_forms_commissioning.py (+8/-0)
src/maasserver/websockets/handlers/general.py (+5/-0)
To merge this branch: bzr merge lp:~ltrager/maas/global_min_hwe_kernel
Reviewer Review Type Date Requested Status
Blake Rouse (community) Approve
Andres Rodriguez (community) Needs Information
Review via email: mp+275477@code.launchpad.net

Commit message

Add global_min_hwe_kernel which is used to set the min_hwe_kernel on new nodes. min_hwe_kernel requirements are now followed during commissioning and enlistment.

Description of the change

This branch adds the global_min_hwe_kernel option to MAAS. When set new nodes, or nodes which are being recommissioned which do not have min_hwe_kernel set, will have min_hwe_kernel set to global_min_hwe_kernel automatically.

This branch also instructions MAAS to follow any requirements set by min_hwe_kernel during commissioning and enlistment. Because min_hwe_kernel is now used during enlistment and commissioning global_min_hwe_kernel is limited to the hwe kernels available with the commissioning release. The user can set min_hwe_kernel when adding a new node in the UI or command line or by setting the new global_min_hwe_kernel option.

To post a comment you must log in.
Revision history for this message
Blake Rouse (blake-rouse) wrote :

The core looks good. But needs to fix some things. See inline.

review: Needs Fixing
Revision history for this message
Lee Trager (ltrager) wrote :

Thanks for the review, I've fixed the items you marked below.

Revision history for this message
Andres Rodriguez (andreserl) wrote :

Hi Lee,

I have a couple things:

1. I think that solely using min_hwe_kernel can be very confusing to the user, which is why global_min_hwe_kernel was better. I disagree with Blake's comment here really. We can discuss this in the call.

2. see inline

review: Needs Fixing
Revision history for this message
Andres Rodriguez (andreserl) wrote :

nevermind comment 2. I'd only like to discuss the global_min_hwe_kernel, or it should at least be default_min_hwe_kernel

review: Needs Information
Revision history for this message
Blake Rouse (blake-rouse) wrote :

I am good with default_min_hwe_kernel. That will make it consistent.
"global_" was not consistent.

On Fri, Oct 23, 2015 at 11:38 AM, Andres Rodriguez <email address hidden>
wrote:

> Review: Needs Information
>
> nevermind comment 2. I'd only like to discuss the global_min_hwe_kernel,
> or it should at least be default_min_hwe_kernel
>
> Diff comments:
>
> > === modified file 'src/maasserver/api/pxeconfig.py'
> > --- src/maasserver/api/pxeconfig.py 2015-09-08 18:41:57 +0000
> > +++ src/maasserver/api/pxeconfig.py 2015-10-23 07:37:11 +0000
> > @@ -227,6 +227,10 @@
> > # we give precedence to any kernel defined in the
> subarchitecture field
> > if subarch == "generic" and node.hwe_kernel:
> > subarch = node.hwe_kernel
> > + elif(subarch == "generic"
> > + and node.get_boot_purpose() == "commissioning"
> > + and node.min_hwe_kernel):
> > + subarch = node.min_hwe_kernel
>
> nevermind this comment, I see in the code belowe this is being addressed
>
> > else:
> > nodegroup = find_nodegroup_for_pxeconfig_request(request)
> > preseed_url =
> compose_enlistment_preseed_url(nodegroup=nodegroup)
>
>
> --
>
> https://code.launchpad.net/~ltrager/maas/global_min_hwe_kernel/+merge/275477
> You are reviewing the proposed merge of
> lp:~ltrager/maas/global_min_hwe_kernel into lp:maas.
>

Revision history for this message
Blake Rouse (blake-rouse) wrote :

Looks good. Thanks for all the fixes.

review: Approve
Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (996.5 KiB)

The attempt to merge lp:~ltrager/maas/global_min_hwe_kernel into lp:maas failed. Below is the output from the failed tests.

Hit http://security.ubuntu.com trusty-security InRelease
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Get:1 http://nova.clouds.archive.ubuntu.com trusty-updates InRelease [64.4 kB]
Hit http://security.ubuntu.com trusty-security/main Sources
Hit http://security.ubuntu.com trusty-security/universe Sources
Hit http://security.ubuntu.com trusty-security/main amd64 Packages
Hit http://security.ubuntu.com trusty-security/universe amd64 Packages
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:2 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [240 kB]
Get:3 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [140 kB]
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [635 kB]
Get:5 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [323 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Fetched 1,403 kB in 3s (372 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 phantomjs postgresql pyflakes python-apt python-bson python-bzrlib python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-jinja2 python-jsonschema python-lxml python-mock python-netaddr python-netifaces python-nose python-oauth python-openssl python-paramiko python-pexpect python-pip python-pocket-lint python-psycopg2 python-pyinotify python-pyparsing python-seamicroclient python-simplejson python-simplestreams python-sphinx python-subunit python-tempita python-testresources python-testscenarios python-testtools python-twisted python-txtftp python-tz python...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/maasserver/api/pxeconfig.py'
2--- src/maasserver/api/pxeconfig.py 2015-09-08 18:41:57 +0000
3+++ src/maasserver/api/pxeconfig.py 2015-10-23 17:54:22 +0000
4@@ -227,6 +227,10 @@
5 # we give precedence to any kernel defined in the subarchitecture field
6 if subarch == "generic" and node.hwe_kernel:
7 subarch = node.hwe_kernel
8+ elif(subarch == "generic"
9+ and node.get_boot_purpose() == "commissioning"
10+ and node.min_hwe_kernel):
11+ subarch = node.min_hwe_kernel
12 else:
13 nodegroup = find_nodegroup_for_pxeconfig_request(request)
14 preseed_url = compose_enlistment_preseed_url(nodegroup=nodegroup)
15@@ -254,7 +258,14 @@
16 else:
17 arch, _ = resource.split_arch()
18
19- subarch = get_optional_param(request.GET, 'subarch', 'generic')
20+ default_min_hwe_kernel = Config.objects.get_config(
21+ 'default_min_hwe_kernel')
22+ if default_min_hwe_kernel:
23+ subarch = get_optional_param(
24+ request.GET, 'subarch', default_min_hwe_kernel)
25+ else:
26+ subarch = get_optional_param(
27+ request.GET, 'subarch', 'generic')
28
29 # If we are booting with "xinstall", then we should always return the
30 # commissioning operating system and distro_series.
31
32=== modified file 'src/maasserver/api/tests/test_pxeconfig.py'
33--- src/maasserver/api/tests/test_pxeconfig.py 2015-08-29 03:27:16 +0000
34+++ src/maasserver/api/tests/test_pxeconfig.py 2015-10-23 17:54:22 +0000
35@@ -361,6 +361,15 @@
36 (ip, hostname),
37 (json.loads(response.content)["log_host"], mock.call_args[0][0]))
38
39+ def test_pxeconfig_enlistment_checks_default_min_hwe_kernel(self):
40+ params = self.get_default_params()
41+ params['arch'] = 'armhf'
42+ Config.objects.set_config('default_min_hwe_kernel', 'hwe-v')
43+ response = self.client.get(reverse('pxeconfig'), params)
44+ self.assertEqual(
45+ "hwe-v",
46+ json.loads(response.content)["subarch"])
47+
48 def test_pxeconfig_has_preseed_url_for_known_node(self):
49 params = self.get_mac_params()
50 node = Interface.objects.get(mac_address=params['mac']).node
51@@ -631,6 +640,17 @@
52 self.assertEqual(osystem, params_out["osystem"])
53 self.assertEqual(release, params_out["release"])
54
55+ def test_pxeconfig_commissioning_node_uses_min_hwe_kernel(self):
56+ node = factory.make_Node(min_hwe_kernel="hwe-v")
57+ nic = factory.make_Interface(INTERFACE_TYPE.PHYSICAL, node=node)
58+ self.patch(Node, 'get_boot_purpose').return_value = "commissioning"
59+ params = self.get_default_params()
60+ params['mac'] = nic.mac_address
61+ response = self.client.get(reverse('pxeconfig'), params)
62+ self.assertEqual(
63+ "hwe-v",
64+ json.loads(response.content)["subarch"])
65+
66 def test_pxeconfig_returns_ubuntu_os_series_for_ubuntu_xinstall(self):
67 nodegroup = factory.make_NodeGroup()
68 ubuntu_image = make_rpc_boot_image(
69
70=== modified file 'src/maasserver/forms.py'
71--- src/maasserver/forms.py 2015-10-21 23:36:45 +0000
72+++ src/maasserver/forms.py 2015-10-23 17:54:22 +0000
73@@ -566,7 +566,11 @@
74 return boot_type
75
76 def clean_min_hwe_kernel(self):
77- return validate_min_hwe_kernel(self.cleaned_data.get('min_hwe_kernel'))
78+ min_hwe_kernel = self.cleaned_data.get('min_hwe_kernel')
79+ if self.new_node and not min_hwe_kernel:
80+ min_hwe_kernel = Config.objects.get_config(
81+ 'default_min_hwe_kernel')
82+ return validate_min_hwe_kernel(min_hwe_kernel)
83
84 def clean(self):
85 cleaned_data = super(NodeForm, self).clean()
86@@ -1306,6 +1310,8 @@
87 Form.__init__(self, *args, **kwargs)
88 self.fields['commissioning_distro_series'] = get_config_field(
89 'commissioning_distro_series')
90+ self.fields['default_min_hwe_kernel'] = get_config_field(
91+ 'default_min_hwe_kernel')
92 self._load_initials()
93
94
95
96=== modified file 'src/maasserver/forms_settings.py'
97--- src/maasserver/forms_settings.py 2015-08-18 18:02:11 +0000
98+++ src/maasserver/forms_settings.py 2015-10-23 17:54:22 +0000
99@@ -27,6 +27,7 @@
100 from django.core.exceptions import ValidationError
101 from maasserver.bootresources import IMPORT_RESOURCES_SERVICE_PERIOD
102 from maasserver.fields import IPListFormField
103+from maasserver.models import BootResource
104 from maasserver.models.config import (
105 Config,
106 DEFAULT_OS,
107@@ -39,6 +40,7 @@
108 list_all_usable_releases,
109 list_commissioning_choices,
110 list_osystem_choices,
111+ release_a_newer_than_b,
112 )
113
114
115@@ -114,6 +116,31 @@
116 return field
117
118
119+def make_default_min_hwe_kernel_field(*args, **kwargs):
120+ """Build and return the default_min_hwe_kernel field."""
121+ kernel_choices = [('', '--- No minimum kernel ---')]
122+ # Global choices are limited to the commissioning release as min_hwe_kernel
123+ # is used during commissioning.
124+ commissioning_series = Config.objects.get_config(
125+ 'commissioning_distro_series')
126+ if commissioning_series:
127+ commissioning_os_release = "ubuntu/" + commissioning_series
128+ kernel_choices += [
129+ (kernel, kernel)
130+ for kernel in BootResource.objects.get_usable_hwe_kernels(
131+ commissioning_os_release)
132+ if release_a_newer_than_b(kernel, commissioning_series)]
133+ field = forms.ChoiceField(
134+ initial=Config.objects.get_config('default_min_hwe_kernel'),
135+ choices=kernel_choices,
136+ error_messages={
137+ 'invalid_choice': compose_invalid_choice_text(
138+ 'default_min_hwe_kernel', kernel_choices)
139+ },
140+ **kwargs)
141+ return field
142+
143+
144 def make_commissioning_distro_series_field(*args, **kwargs):
145 """Build and return the commissioning_distro_series field."""
146 usable_oses = list_all_usable_osystems()
147@@ -257,6 +284,18 @@
148 # at run-time to avoid a race condition.
149 }
150 },
151+ 'default_min_hwe_kernel': {
152+ 'default': None,
153+ 'form': make_default_min_hwe_kernel_field,
154+ 'form_kwargs': {
155+ 'label': "Default Minimum Kernel Version",
156+ 'required': False,
157+ 'help_text': (
158+ "The default minimum kernel version used on all new and"
159+ " commissioned nodes."
160+ )
161+ }
162+ },
163 'default_storage_layout': {
164 'default': 'lvm',
165 'form': forms.ChoiceField,
166
167=== modified file 'src/maasserver/models/config.py'
168--- src/maasserver/models/config.py 2015-08-18 18:02:11 +0000
169+++ src/maasserver/models/config.py 2015-10-23 17:54:22 +0000
170@@ -51,6 +51,7 @@
171 'commissioning_osystem': DEFAULT_OS.name,
172 'commissioning_distro_series':
173 DEFAULT_OS.get_default_commissioning_release(),
174+ 'default_min_hwe_kernel': '',
175 'default_storage_layout': 'lvm',
176 # Network section configuration.
177 'maas_name': gethostname(),
178
179=== modified file 'src/maasserver/models/node.py'
180--- src/maasserver/models/node.py 2015-10-21 00:43:01 +0000
181+++ src/maasserver/models/node.py 2015-10-23 17:54:22 +0000
182@@ -998,6 +998,12 @@
183 old_status = self.status
184 self.status = NODE_STATUS.COMMISSIONING
185 self.owner = user
186+ # Set min_hwe_kernel to min_hwe_kernel if it isn't set incase
187+ # commissioning failed and the user set the min globally and is
188+ # retrying.
189+ if not self.min_hwe_kernel:
190+ self.min_hwe_kernel = Config.objects.get_config(
191+ 'default_min_hwe_kernel')
192 self.save()
193
194 # Prepare a transition monitor for later.
195
196=== modified file 'src/maasserver/models/tests/test_node.py'
197--- src/maasserver/models/tests/test_node.py 2015-10-21 00:43:01 +0000
198+++ src/maasserver/models/tests/test_node.py 2015-10-23 17:54:22 +0000
199@@ -1492,6 +1492,20 @@
200 post_commit_hooks.reset() # Ignore these for now.
201 self.assertThat(node_start, MockCalledOnceWith(admin, user_data))
202
203+ def test_start_commissioning_sets_min_hwe_kernel(self):
204+ node = factory.make_Node(status=NODE_STATUS.NEW)
205+ node_start = self.patch(node, '_start')
206+ node_start.side_effect = lambda user, user_data: post_commit()
207+ user_data = factory.make_string().encode('ascii')
208+ generate_user_data = self.patch(
209+ commissioning, 'generate_user_data')
210+ generate_user_data.return_value = user_data
211+ admin = factory.make_admin()
212+ Config.objects.set_config('default_min_hwe_kernel', 'hwe-v')
213+ node.start_commissioning(admin)
214+ post_commit_hooks.reset() # Ignore these for now.
215+ self.assertEqual('hwe-v', node.min_hwe_kernel)
216+
217 def test_start_commissioning_clears_node_commissioning_results(self):
218 node = factory.make_Node(status=NODE_STATUS.NEW)
219 NodeResult.objects.store_data(
220
221=== modified file 'src/maasserver/static/js/angular/controllers/add_hardware.js'
222--- src/maasserver/static/js/angular/controllers/add_hardware.js 2015-08-11 15:07:10 +0000
223+++ src/maasserver/static/js/angular/controllers/add_hardware.js 2015-10-23 17:54:22 +0000
224@@ -24,6 +24,8 @@
225 $scope.zones = ZonesManager.getItems();
226 $scope.architectures = GeneralManager.getData("architectures");
227 $scope.hwe_kernels = GeneralManager.getData("hwe_kernels");
228+ $scope.default_min_hwe_kernel = GeneralManager.getData(
229+ "default_min_hwe_kernel");
230 $scope.error = null;
231
232 // Input values.
233@@ -281,6 +283,7 @@
234 macs: [newMAC()],
235 zone: defaultZone(),
236 architecture: defaultArchitecture(),
237+ min_hwe_kernel: $scope.default_min_hwe_kernel.text,
238 power: {
239 type: null,
240 parameters: {}
241@@ -439,7 +442,6 @@
242 $scope.machine.cluster === null ||
243 $scope.machine.zone === null ||
244 $scope.machine.architecture === '' ||
245- $scope.machine.min_hwe_kernel === '' ||
246 $scope.machine.power.type === null ||
247 $scope.invalidName($scope.machine));
248 if(in_error) {
249
250=== modified file 'src/maasserver/static/js/angular/factories/general.js'
251--- src/maasserver/static/js/angular/factories/general.js 2015-10-21 19:45:50 +0000
252+++ src/maasserver/static/js/angular/factories/general.js 2015-10-23 17:54:22 +0000
253@@ -49,6 +49,16 @@
254 polling: false,
255 nextPromise: null
256 },
257+ default_min_hwe_kernel: {
258+ method: "general.default_min_hwe_kernel",
259+ data: { text: '' },
260+ loaded: false,
261+ polling: false,
262+ nextPromise: null,
263+ replaceData: function(oldData, newData) {
264+ oldData.text = newData;
265+ }
266+ },
267 osinfo: {
268 method: "general.osinfo",
269 data: {},
270
271=== modified file 'src/maasserver/static/js/angular/factories/tests/test_general.js'
272--- src/maasserver/static/js/angular/factories/tests/test_general.js 2015-10-21 19:45:50 +0000
273+++ src/maasserver/static/js/angular/factories/tests/test_general.js 2015-10-23 17:54:22 +0000
274@@ -51,7 +51,7 @@
275 it("_data has expected keys", function() {
276 expect(Object.keys(GeneralManager._data)).toEqual(
277 ["node_actions", "device_actions", "architectures", "hwe_kernels",
278- "osinfo", "bond_options", "version"]);
279+ "default_min_hwe_kernel", "osinfo", "bond_options", "version"]);
280 });
281
282 it("_data.node_actions has correct data", function() {
283@@ -90,6 +90,17 @@
284 expect(hwe_kernels.nextPromise).toBeNull();
285 });
286
287+ it("_data.default_min_hwe_kernels has correct data", function() {
288+ var default_min_hwe_kernel =
289+ GeneralManager._data.default_min_hwe_kernel;
290+ expect(default_min_hwe_kernel.method).toBe(
291+ "general.default_min_hwe_kernel");
292+ expect(default_min_hwe_kernel.data).toEqual({text: ''});
293+ expect(default_min_hwe_kernel.loaded).toBe(false);
294+ expect(default_min_hwe_kernel.polling).toBe(false);
295+ expect(default_min_hwe_kernel.nextPromise).toBeNull();
296+ });
297+
298 it("_data.osinfo has correct data", function() {
299 var osinfo = GeneralManager._data.osinfo;
300 expect(osinfo.method).toBe("general.osinfo");
301@@ -166,6 +177,7 @@
302 GeneralManager._data.device_actions.loaded = true;
303 GeneralManager._data.architectures.loaded = true;
304 GeneralManager._data.hwe_kernels.loaded = true;
305+ GeneralManager._data.default_min_hwe_kernel.loaded = true;
306 GeneralManager._data.osinfo.loaded = true;
307 GeneralManager._data.bond_options.loaded = true;
308 GeneralManager._data.version.loaded = true;
309@@ -416,7 +428,7 @@
310 spyOn(GeneralManager, "_loadData").and.returnValue(
311 $q.defer().promise);
312 GeneralManager.loadItems();
313- expect(GeneralManager._loadData.calls.count()).toBe(7);
314+ expect(GeneralManager._loadData.calls.count()).toBe(8);
315 });
316
317 it("resolve defer once all resolve", function(done) {
318@@ -427,6 +439,7 @@
319 $q.defer(),
320 $q.defer(),
321 $q.defer(),
322+ $q.defer(),
323 $q.defer()
324 ];
325 var i = 0;
326
327=== modified file 'src/maasserver/tests/test_forms_commissioning.py'
328--- src/maasserver/tests/test_forms_commissioning.py 2015-05-07 18:14:38 +0000
329+++ src/maasserver/tests/test_forms_commissioning.py 2015-10-23 17:54:22 +0000
330@@ -36,6 +36,14 @@
331 'commissioning_distro_series', field.choices),
332 field.error_messages['invalid_choice'])
333
334+ def test_commissioningform_error_msg_lists_min_hwe_kernel_choices(self):
335+ form = CommissioningForm()
336+ field = form.fields['default_min_hwe_kernel']
337+ self.assertEqual(
338+ compose_invalid_choice_text(
339+ 'default_min_hwe_kernel', field.choices),
340+ field.error_messages['invalid_choice'])
341+
342
343 class TestCommissioningScriptForm(MAASServerTestCase):
344
345
346=== modified file 'src/maasserver/websockets/handlers/general.py'
347--- src/maasserver/websockets/handlers/general.py 2015-10-21 19:45:50 +0000
348+++ src/maasserver/websockets/handlers/general.py 2015-10-23 17:54:22 +0000
349@@ -45,6 +45,7 @@
350 allowed_methods = [
351 'architectures',
352 'hwe_kernels',
353+ 'default_min_hwe_kernel',
354 'osinfo',
355 'node_actions',
356 'device_actions',
357@@ -61,6 +62,10 @@
358 """Return all supported hwe_kernels."""
359 return BootResource.objects.get_usable_hwe_kernels()
360
361+ def default_min_hwe_kernel(self, params):
362+ """Return the default_min_hwe_kernel."""
363+ return Config.objects.get_config('default_min_hwe_kernel')
364+
365 def osinfo(self, params):
366 """Return all available operating systems and releases information."""
367 osystems = list_all_usable_osystems()