Merge lp:~blake-rouse/maas/add-osystem-to-node-form-api into lp:maas

Proposed by Blake Rouse
Status: Merged
Approved by: Blake Rouse
Approved revision: 2306
Merged at revision: 2383
Proposed branch: lp:~blake-rouse/maas/add-osystem-to-node-form-api
Merge into: lp:maas
Prerequisite: lp:~blake-rouse/maas/add-osystem-to-node-migration
Diff against target: 1355 lines (+572/-114)
15 files modified
src/maasserver/api.py (+33/-9)
src/maasserver/forms.py (+173/-13)
src/maasserver/forms_settings.py (+55/-15)
src/maasserver/models/config.py (+9/-3)
src/maasserver/models/node.py (+20/-11)
src/maasserver/models/tests/test_node.py (+13/-11)
src/maasserver/preseed.py (+1/-1)
src/maasserver/testing/osystems.py (+12/-4)
src/maasserver/tests/test_api_node.py (+24/-11)
src/maasserver/tests/test_api_pxeconfig.py (+29/-4)
src/maasserver/tests/test_forms.py (+150/-4)
src/maasserver/tests/test_preseed.py (+11/-15)
src/maasserver/views/settings.py (+9/-0)
src/maasserver/views/tests/test_settings.py (+32/-12)
src/metadataserver/tests/test_api.py (+1/-1)
To merge this branch: bzr merge lp:~blake-rouse/maas/add-osystem-to-node-form-api
Reviewer Review Type Date Requested Status
Jeroen T. Vermeulen (community) Approve
Jason Hobbs (community) Approve
Review via email: mp+219521@code.launchpad.net

Commit message

Add operating system selection to NodeForm, and os selection from distro series in start api call.

Description of the change

This replaces the large merge of add-osystem-to-node, and separates it into 3 different mergers. This is the second one.

The start node API uses the same distro_series options so that the api call does not change. The osystem is selected based on the operating system that is found to contain that distro_series/release. The valid choices that get displayed in the WebUI are based on the BootImage's that are available, removing the ability for a user to select an operating system and release combination that would not boot.

Note: osystem was used throughout the code instead of os, as the python os module would conflict throughout the code base.

To post a comment you must log in.
2298. By Blake Rouse

Merge add-osystem-to-node-form-api.

Revision history for this message
Jason Hobbs (jason-hobbs) wrote :
Download full text (12.3 KiB)

I am new to a lot of this part of MAAS and as a result, this review is probably more of the trees than of the forest. Over all it looks good to me, but I do have some questions and comments.

[1] Needs make format/make lint.

[2]

@@ -403,7 +407,7 @@
             available to the nodes through the metadata service.
         :type user_data: base64-encoded unicode
         :param distro_series: If present, this parameter specifies the
- Ubuntu Release the node will use.
+ os relase the node will use.

Typo 'relase'. Maybe s/os/OS too.

[3]

-def get_boot_purpose(node):
+def get_boot_purpose(node, osystem, arch, subarch, series, label):
     """Return a suitable "purpose" for this boot, e.g. "install"."""
     # XXX: allenap bug=1031406 2012-07-31: The boot purpose is still in
     # flux. It may be that there will just be an "ephemeral" environment and
@@ -2364,7 +2368,15 @@
             if node.should_use_traditional_installer():
                 return "install"
             else:
- return "xinstall"
+ # Check that the booting operating system, actually supports
+ # fast-path installation. If it does not then, we need to
+ # return normal install.
+ osystem_obj = OperatingSystemRegistry[osystem]
+ purposes = osystem_obj.get_boot_image_purposes(
+ arch, subarch, series, label)
+ if BOOT_IMAGE_PURPOSE.XINSTALL in purposes:
+ return "xinstall"
+ return "install"

Why isn't this error state handled when we configure the node? It is
confusing to have a node configured for xinstall, but to have it really
use install behind the scenes.

[4]
+ # Get the purpose, checking that if node is using xinstall, the operating
+ # system actaully supports that mode. We pass None for the label, here
+ # because it is unknown which label will be used yet.
+ purpose = get_boot_purpose(node, osystem, arch, subarch, series, None)

Same comment as [3].

Also, don't need the comma between 'label' and 'here'.

[5]
     label = get_optional_param(request.GET, 'label', latest_label)

+ # Now that we have the correct label, lets check the boot purpose again
+ # to make sure that the boot image with that label, is the correct purpose.
+ purpose = get_boot_purpose(node, osystem, arch, subarch, series, label)

Same comment as [3] again - didn't need to grab purpose more than once
with the previous implementation.

Also, s/lets/let's and drop comma between 'label' and 'is'.

=== modified file 'src/maasserver/forms.py'
--- src/maasserver/forms.py 2014-05-14 07:12:10 +0000
+++ src/maasserver/forms.py 2014-05-14 14:40:00 +0000
@@ -71,9 +71,6 @@
     )
 from maasserver.config_forms import SKIP_CHECK_NAME
 from maasserver.enum import (
- COMMISSIONING_DISTRO_SERIES_CHOICES,
- DISTRO_SERIES,
- DISTRO_SERIES_CHOICES,
     NODE_STATUS,
     NODEGROUPINTERFACE_MANAGEMENT,
     NODEGROUPINTERFACE_MANAGEMENT_CHOICES,
@@ -86,7 +83,7 @@
 from maasserver.forms_settings import (
     CONFIG_ITEMS_KEYS,
     get_config_field,
- INVALID_DISTRO_SERIES_MESSAGE,
+ list_commisioni...

review: Needs Information
Revision history for this message
Blake Rouse (blake-rouse) wrote :
Download full text (3.1 KiB)

[1,2] Fixed

[3] Why fail a node from working? What is the downside of using install instead of xinstall, you still end up with a system that is similar after finish installation, its just speed. So if the operating system installs using fast-path great, if not then use the old way. Also since fast-path booting is only set by a tag, then it makes it even harder to catch. As you wouldn't see the error until you go to turn on the node, at which case it has already been acquired and it will fail, causing JuJu to fail.

[4,5] Comments fixed.
[5] You need to grab the purpose first only because the boot label is unknown on the first call. So first it gets the purpose without the label, then once the correct label is select. Just to check the label supports that purpose we call again. Its just a good check to make sure that the final selected boot image can handle this method.

[6] Early out used!
[6] It needs to handle two form inputs as the OS selection is used with NodeForm, and DeployForm. Didn't want to duplicate the code, when it would be the same so I split it out into a method that can handle multiple forms.

[7,8] Fixed

[9] Merge conflict resolved

[10] Its always a good check to make sure that it is not None, so I check. It really would only be a blank string, but doesn't hurt to just make sure. Being explicit.

[11] Split into two tests.

[12] This is testing that the set method is actually still being called. So if the code was to change where it wasn't this would break the API and the test would fail.

[13] This actually uses a different method on Node, release. The release call resets the osystem and distro_series. Also a good check to make sure that the release call is called.

[14] Belief that it makes it easier to see that the test is testing it multiple times.

[15] Differ in one tests that the items are there, and the other tests that the items are sorted. This is a copy of how the architecture code is tested.

[16]
Added test for maasserver.api.get_boot_purpose() on return "install" when os doesn't support "xinstall".

NodeForm.set_up_osystem_and_distro_series_fields() and maasserver.forms.clean_distro_series_field() is being tested in maasserver.tests.test_forms.TestNodeForm by the following:
  - test_contains_limited_set_of_fields
  - test_accepts_osystem
  - test_rejects_invalid_osystem
  - test_starts_with_default_osystem
  - test_accepts_osystem_distro_series
  - test_rejects_invalid_osystem_distro_series
  - test_starts_with_default_distro_series
  - test_rejects_mismatch_osystem_distro_series

maasserver.models.node.validate_osystem() and maasserver.models.node.validate_distro_series() are getting tested with the TestNodeForm and the start API call.

These are getting covered by the usage of the forms, views, and api throughout the testing code base, as you stated.
  - maasserver.forms.list_release_choices()
  - maasserver.forms.list_osystem_choices()
  - NodeForm.clean_distro_series()
  - maasserver.form_settings.list_osystem_choices()
  - maasserver.form_settings.list_release_choices()
  - maasserver.form_settings.list_commissioning_choices()
  - maasserver.models.node.get_all_releases()
  - maasserver.models.node.get_o...

Read more...

2299. By Blake Rouse

Fix formating.

2300. By Blake Rouse

Merge trunk.

2301. By Blake Rouse

Fix lint.

2302. By Blake Rouse

Add tests and some fixes from merge of trunk.

Revision history for this message
Jason Hobbs (jason-hobbs) wrote :

Thanks! A couple more comments inline below.

On Thu, May 15, 2014 at 4:22 PM, Blake Rouse <email address hidden>wrote:
>
> [3] Why fail a node from working?

I don't mean that we should fail the node instead of reverting to install,
but that we should never be able to get a Node into the state where it's
configured to xinstall with an OS that can't xinstall. Ideally we would
raise a validation error whenever someone configured a node to xinstall
with an OS/release that couldn't xinstall. Maybe it's harder than it should
be because we use a tag to indicate xinstall use.

> What is the downside of using install instead of xinstall, you still end
> up with a system that is similar after finish installation, its just speed.

What if you have a modified preseed or pxeconfig for xinstall, with stuff
that's critical in it? Or a feature that is enabled for xinstall but not
install (which will likely happen with debian-installer being deprecated).
It seems fragile to assume that they are now and will forever be equivalent
other than speed.

>
> [7,8] Fixed
>

You may want to double check the new validate_osystem() - I think you need
to invert your logic for using the early out way. If it's passing tests as
is I would also double check those :).

>
> [9] Merge conflict resolved
>
> [10] Its always a good check to make sure that it is not None, so I check.
> It really would only be a blank string, but doesn't hurt to just make sure.
> Being explicit.
>
> [11] Split into two tests.
>
> [12] This is testing that the set method is actually still being called.
> So if the code was to change where it wasn't this would break the API and
> the test would fail.
>
> [13] This actually uses a different method on Node, release. The release
> call resets the osystem and distro_series. Also a good check to make sure
> that the release call is called.
>
> [14] Belief that it makes it easier to see that the test is testing it
> multiple times.
>
> [15] Differ in one tests that the items are there, and the other tests
> that the items are sorted. This is a copy of how the architecture code is
> tested.
>

Yes, the difference just makes them more fun to refactor though :).

Jason

Revision history for this message
Jason Hobbs (jason-hobbs) :
review: Approve
2303. By Blake Rouse

Fix osystem_validation and test for bad value.

Revision history for this message
Blake Rouse (blake-rouse) wrote :

Nice catch, validate_osystem was incorrect and was not being tested for a bad value. Added a test for bad value for validate_osystem and validate_distro_series.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

I'm going to review this in detail, but time has run out for me today; I'll have to get back to it tomorrow. Smaller branches have less of this problem!

For the record, I agree with Jason that it's better to fail early than tohide potentially problematic settings. But of course that's assuming that we can reasonably catch the problem at the right stage. Once we're trying to boot a node, it's probably too late.

As for the get_boot_purpose call where the label is None, I'd suggest passing it as a keyword for clarity: label=None.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

Damn those big branches! It's good, but far too big. I'm attaching inline review notes for the first 500 lines or so, up to (but not including) src/maasserver/models/node.py. More tomorrow...

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

I just reviewed src/maasserver/models/node.py as well (again using inline comments) and I hit one sticking point: I don't think validation of a node's operating system and release against the database belongs on the model. The model can validate for basic sanity of the input string, but I feel the checks for supported systems/releases belongs at the form level.

If these were proper foreign keys then yes, clearly it's a model integrity issue. But then we'd also have model-level protection against osystems being deleted while nodes still refer to them. We would have the full stack of referential-integrity mechanisms to do the work for us, efficiently and reliably. But where possible I would avoid model validation that needs to query the database.

As it is, don't these validators break tests? I would expect factory.make_node to set a random operating system and release, and unless you surprised the caller by also creating boot images (probably breaking other tests), the validators would wreak havoc on the test suite.

Revision history for this message
Blake Rouse (blake-rouse) wrote :

osystem has validation in the model, as this field can also be set with the start API call. distro_series always had validation using the choices=, but now its just using a function to perform the validation. It gives the same result.

factory.make_node does not break tests, as the osystem field just like the distro_series field was can be blank. If the field is blank then the default is used, so no tests break.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

One of our architecture decisions is that the API should validate through the forms, so validating in the form should also cover the API.

We normally like our factories to produce randomised fields, not reliable blanks. (If a field can be blank, it can be useful to flip a coin first — heads, leave blank; tails, make up random value.)

Revision history for this message
Blake Rouse (blake-rouse) wrote :

I was just implementing it the exact way it was implemented before, I just
added a new field osystem. If you would like me to implement it
differently, then I can. Let me know what you would like me to do. I can do
this in a later branch, as this one long enough. :)

On Thu, May 22, 2014 at 10:15 AM, Jeroen T. Vermeulen <email address hidden>wrote:

> One of our architecture decisions is that the API should validate through
> the forms, so validating in the form should also cover the API.
>
> We normally like our factories to produce randomised fields, not reliable
> blanks. (If a field can be blank, it can be useful to flip a coin first —
> heads, leave blank; tails, make up random value.)
> --
>
> https://code.launchpad.net/~blake-rouse/maas/add-osystem-to-node-form-api/+merge/219521
> You are the owner of lp:~blake-rouse/maas/add-osystem-to-node-form-api.
>

Revision history for this message
Raphaël Badin (rvb) wrote :

My general take on how validation should be performed is that we should use model-level validation only to avoid inconsistencies in the DB. The rest of the validation belongs to forms. They give us a much more flexible way to validate the input from the UI and the API. More flexible because we can still manipulate objects directly in tests and migrations. This is just a matter of properly separating the M and the C (in MVC) because they are fundamentally separate steps; mingling them together reduces the flexibility of the whole model.

> osystem has validation in the model, as this field can also be set with the start API call.
> distro_series always had validation using the choices=, but now its just using a function to
> perform the validation. It gives the same result.

The result is not quite the same, no. Doing this kind of application-level integrity enforcement at the model level reduces our ability to deal gracefully with inconsistencies. Consider for instance the consequence of having an OS removed: one would be incapable of modifying the nodes referencing this OS unless he also changes the OS (because the validation would fail at the model level); if the validation happens at the form level, at least we could write a migration to deal with these nodes.

> factory.make_node does not break tests, as the osystem field just like the distro_series
> field was can be blank. If the field is blank then the default is used, so no tests break.

But then all your tests operate with a very special case (osystem=blank). It's much better to have the tests operate on data that is as similar to production data as possible. This is also why we chose to use randomness when creating test objects.

Revision history for this message
Blake Rouse (blake-rouse) wrote :

> My general take on how validation should be performed is that we should use

> The result is not quite the same, no. Doing this kind of application-level
> integrity enforcement at the model level reduces our ability to deal
> gracefully with inconsistencies. Consider for instance the consequence of
> having an OS removed: one would be incapable of modifying the nodes
> referencing this OS unless he also changes the OS (because the validation
> would fail at the model level); if the validation happens at the form level,
> at least we could write a migration to deal with these nodes.

This makes since, it would fail to save the node if an os was removed. I will refactor the code to remove the validation from the model.

> But then all your tests operate with a very special case (osystem=blank).
> It's much better to have the tests operate on data that is as similar to
> production data as possible. This is also why we chose to use randomness when
> creating test objects.

That is not a special case, it is the default case for all created nodes. Even nodes created by the user, use that value. Its the same implementation that was there for distro_series. If there was no issue the way it was before, why is there now?

Revision history for this message
Raphaël Badin (rvb) wrote :

> Its the same implementation that was there for distro_series.
> If there was no issue the way it was before, why is there now?

The `distro_series` stuff was implemented when the possible choices where not the dynamic object it is now. One could argue that even at that time it was a mistake but the point is that we shouldn't persist doing things the way `distro_series` was implemented for the many reasons stated above. This means that, ideally, we should also perform the validation for `distro_series` at the form level.

Revision history for this message
Blake Rouse (blake-rouse) wrote :

> The `distro_series` stuff was implemented when the possible choices where not
> the dynamic object it is now. One could argue that even at that time it was a
> mistake but the point is that we shouldn't persist doing things the way
> `distro_series` was implemented for the many reasons stated above. This means
> that, ideally, we should also perform the validation for `distro_series` at
> the form level.

Sorry I was talking specifically about factory.make_node. I will refactor distro_series and osystem to use form validation.

2304. By Blake Rouse

Move validate of distro_series and osystem to NodeForm for the start API call.

Revision history for this message
Blake Rouse (blake-rouse) wrote :

Removed the validation of osystem and distro_series from the NodeModel. The start API now uses the NodeForm to set the distro_series and osystem.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

I'm supposed to be reviewing the review here, I think, but things got so big that I just went back to the code! Anyway, thanks for doing this and for bearing with us when we made you jump through the additional hoops...

Notes about superficialities follow.

.

Doesn't that last line in list_commisioning_choices just return "options" as it already was? If you're trying to force eager evaluation, you could just return list(options).

.

In test_get_osystem_returns_default_osystem, you create a Node and assume that it won't have its osystem set. I think it would make sense to set a random one by default, in which case it would make more sense to set it to '' explicitly. (Yes, Django's handling of blank strings can be infuriating.) Same for test_get_distro_series_returns_default_series just below that.

.

In make_osystem_with_releases, and make_usable_osystem below that, I guess the “purpose” parameter should really be called “purposes” (plural).

.

Nice tests for list_all_usable_osystems, list_all_usable_releases, and the new form validation! Very focused, clearly named, distinct setup / execution / verification phases.

.

It'd be nice to rename _all_ mention of “series” to “releases”... maybe someday.

review: Approve
2305. By Blake Rouse

Code review fixes.

2306. By Blake Rouse

Merge trunk.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/maasserver/api.py'
2--- src/maasserver/api.py 2014-05-27 15:05:37 +0000
3+++ src/maasserver/api.py 2014-05-29 17:03:44 +0000
4@@ -253,6 +253,10 @@
5 from piston.emitters import JSONEmitter
6 from piston.handler import typemapper
7 from piston.utils import rc
8+from provisioningserver.driver import (
9+ BOOT_IMAGE_PURPOSE,
10+ OperatingSystemRegistry,
11+ )
12 from provisioningserver.kernel_opts import KernelParameters
13 from provisioningserver.power_schema import UNKNOWN_POWER_TYPE
14 import simplejson as json
15@@ -411,7 +415,7 @@
16 available to the nodes through the metadata service.
17 :type user_data: base64-encoded unicode
18 :param distro_series: If present, this parameter specifies the
19- Ubuntu Release the node will use.
20+ OS release the node will use.
21 :type distro_series: unicode
22
23 Ideally we'd have MIME multipart and content-transfer-encoding etc.
24@@ -427,7 +431,13 @@
25 node = Node.objects.get_node_or_404(
26 system_id=system_id, user=request.user,
27 perm=NODE_PERMISSION.EDIT)
28- node.set_distro_series(series=series)
29+ Form = get_node_edit_form(request.user)
30+ form = Form(instance=node)
31+ form.set_distro_series(series=series)
32+ if form.is_valid():
33+ form.save()
34+ else:
35+ raise ValidationError(form.errors)
36 nodes = Node.objects.start_nodes(
37 [system_id], request.user, user_data=user_data)
38 if len(nodes) == 0:
39@@ -2369,7 +2379,7 @@
40 context_instance=RequestContext(request))
41
42
43-def get_boot_purpose(node):
44+def get_boot_purpose(node, osystem, arch, subarch, series, label):
45 """Return a suitable "purpose" for this boot, e.g. "install"."""
46 # XXX: allenap bug=1031406 2012-07-31: The boot purpose is still in
47 # flux. It may be that there will just be an "ephemeral" environment and
48@@ -2389,7 +2399,14 @@
49 if node.should_use_traditional_installer():
50 return "install"
51 else:
52- return "xinstall"
53+ # Return normal install if the booting operating system
54+ # doesn't support xinstall.
55+ osystem_obj = OperatingSystemRegistry[osystem]
56+ purposes = osystem_obj.get_boot_image_purposes(
57+ arch, subarch, series, label)
58+ if BOOT_IMAGE_PURPOSE.XINSTALL in purposes:
59+ return "xinstall"
60+ return "install"
61 else:
62 return "local" # TODO: Investigate.
63 else:
64@@ -2457,12 +2474,12 @@
65 node = get_node_from_mac_string(request.GET.get('mac', None))
66
67 if node is None or node.status == NODE_STATUS.COMMISSIONING:
68+ osystem = Config.objects.get_config('commissioning_osystem')
69 series = Config.objects.get_config('commissioning_distro_series')
70 else:
71+ osystem = node.get_osystem()
72 series = node.get_distro_series()
73
74- purpose = get_boot_purpose(node)
75-
76 if node:
77 arch, subarch = node.architecture.split('/')
78 preseed_url = compose_preseed_url(node)
79@@ -2488,7 +2505,7 @@
80 # current series. If nothing is found, fall back to i386 like
81 # we used to. LP #1181334
82 image = BootImage.objects.get_default_arch_image_in_nodegroup(
83- nodegroup, 'ubuntu', series, purpose=purpose)
84+ nodegroup, osystem, series, purpose='commissioning')
85 if image is None:
86 arch = 'i386'
87 else:
88@@ -2496,12 +2513,16 @@
89
90 subarch = get_optional_param(request.GET, 'subarch', 'generic')
91
92+ # Get the purpose, without a selected label
93+ purpose = get_boot_purpose(
94+ node, osystem, arch, subarch, series, label=None)
95+
96 # We use as our default label the label of the most recent image for
97 # the criteria we've assembled above. If there is no latest image
98 # (which should never happen in reality but may happen in tests), we
99 # fall back to using 'no-such-image' as our default.
100 latest_image = BootImage.objects.get_latest_image(
101- nodegroup, 'ubuntu', arch, subarch, series, purpose)
102+ nodegroup, osystem, arch, subarch, series, purpose)
103 if latest_image is None:
104 # XXX 2014-03-18 gmb bug=1294131:
105 # We really ought to raise an exception here so that client
106@@ -2519,6 +2540,9 @@
107 subarch = latest_image.subarchitecture
108 label = get_optional_param(request.GET, 'label', latest_label)
109
110+ # Get the supported purpose with the boot label
111+ purpose = get_boot_purpose(node, osystem, arch, subarch, series, label)
112+
113 if node is not None:
114 # We don't care if the kernel opts is from the global setting or a tag,
115 # just get the options
116@@ -2548,7 +2572,7 @@
117 cluster_address = get_mandatory_param(request.GET, "local")
118
119 params = KernelParameters(
120- osystem='ubuntu', arch=arch, subarch=subarch, release=series,
121+ osystem=osystem, arch=arch, subarch=subarch, release=series,
122 label=label, purpose=purpose, hostname=hostname, domain=domain,
123 preseed_url=preseed_url, log_host=server_address,
124 fs_host=cluster_address, extra_opts=extra_kernel_opts)
125
126=== modified file 'src/maasserver/forms.py'
127--- src/maasserver/forms.py 2014-05-26 14:14:57 +0000
128+++ src/maasserver/forms.py 2014-05-29 17:03:44 +0000
129@@ -73,9 +73,6 @@
130 )
131 from maasserver.config_forms import SKIP_CHECK_NAME
132 from maasserver.enum import (
133- COMMISSIONING_DISTRO_SERIES_CHOICES,
134- DISTRO_SERIES,
135- DISTRO_SERIES_CHOICES,
136 NODE_STATUS,
137 NODEGROUPINTERFACE_MANAGEMENT,
138 NODEGROUPINTERFACE_MANAGEMENT_CHOICES,
139@@ -88,8 +85,8 @@
140 from maasserver.forms_settings import (
141 CONFIG_ITEMS_KEYS,
142 get_config_field,
143- INVALID_DISTRO_SERIES_MESSAGE,
144 INVALID_SETTING_MSG_TEMPLATE,
145+ list_commisioning_choices,
146 )
147 from maasserver.models import (
148 BootImage,
149@@ -117,6 +114,7 @@
150 from maasserver.utils.network import make_network
151 from metadataserver.fields import Bin
152 from metadataserver.models import CommissioningScript
153+from provisioningserver.driver import OperatingSystemRegistry
154
155 # A reusable null-option for choice fields.
156 BLANK_CHOICE = ('', '-------')
157@@ -217,6 +215,106 @@
158 return all_architectures[0]
159
160
161+def list_all_usable_osystems():
162+ """Return all operating systems that can be used for nodes.
163+
164+ These are the operating systems for which any nodegroup has the boot images
165+ required to boot the node.
166+ """
167+ # The Node edit form offers all usable operating systems as options for the
168+ # osystem field. Not all of these may be available in the node's
169+ # nodegroup, but to represent that accurately in the UI would depend on
170+ # the currently selected nodegroup. Narrowing the options down further
171+ # would have to happen browser-side.
172+ osystems = set()
173+ for nodegroup in NodeGroup.objects.all():
174+ osystems = osystems.union(
175+ BootImage.objects.get_usable_osystems(nodegroup))
176+ osystems = [OperatingSystemRegistry[osystem] for osystem in osystems]
177+ return sorted(osystems, key=lambda osystem: osystem.title)
178+
179+
180+def list_osystem_choices(osystems):
181+ """Return Django "choices" list for `osystem`."""
182+ choices = [('', 'Default OS')]
183+ choices += [
184+ (osystem.name, osystem.title)
185+ for osystem in osystems
186+ ]
187+ return choices
188+
189+
190+def list_all_usable_releases(osystems):
191+ """Return dictionary of usable `releases` for each opertaing system."""
192+ distro_series = {}
193+ nodegroups = list(NodeGroup.objects.all())
194+ for osystem in osystems:
195+ releases = set()
196+ for nodegroup in nodegroups:
197+ releases = releases.union(
198+ BootImage.objects.get_usable_releases(nodegroup, osystem.name))
199+ distro_series[osystem.name] = sorted(releases)
200+ return distro_series
201+
202+
203+def list_release_choices(releases):
204+ """Return Django "choices" list for `releases`."""
205+ choices = [('', 'Default OS Release')]
206+ for key, value in releases.items():
207+ osystem = OperatingSystemRegistry[key]
208+ options = osystem.format_release_choices(value)
209+ choices += [(
210+ '%s/' % osystem.name,
211+ 'Latest %s Release' % osystem.title
212+ )]
213+ choices += [
214+ ('%s/%s' % (osystem.name, name), title)
215+ for name, title in options
216+ ]
217+ return choices
218+
219+
220+def get_distro_series_inital(instance):
221+ """Returns the distro_series initial value for the instance."""
222+ osystem = instance.osystem
223+ series = instance.distro_series
224+ if osystem is not None and osystem != '':
225+ if series is None:
226+ series = ''
227+ return '%s/%s' % (osystem, series)
228+ return None
229+
230+
231+def clean_distro_series_field(form, field, os_field):
232+ """Cleans the distro_series field in the form. Validating that
233+ the selected operating system matches the distro_series.
234+
235+ :param form: `Form` class
236+ :param field: distro_series field name
237+ :param os_field: osystem field name
238+ :returns: clean distro_series field value
239+ """
240+ new_distro_series = form.cleaned_data.get(field)
241+ if new_distro_series is None or '/' not in new_distro_series:
242+ return new_distro_series
243+ os, release = new_distro_series.split('/', 1)
244+ if os_field in form.cleaned_data:
245+ new_os = form.cleaned_data[os_field]
246+ if os != new_os:
247+ raise ValidationError(
248+ "%s in %s does not match with "
249+ "operating system %s" % (release, field, os))
250+ return release
251+
252+
253+def get_osystem_from_release(release):
254+ """Returns the operating system that supports that release."""
255+ for _, osystem in OperatingSystemRegistry:
256+ if release in osystem.get_supported_releases():
257+ return osystem
258+ return None
259+
260+
261 class NodeForm(ModelForm):
262
263 def __init__(self, request=None, *args, **kwargs):
264@@ -229,6 +327,7 @@
265 self.fields['nodegroup'] = NodeGroupFormField(
266 required=False, empty_label="Default (master)")
267 self.set_up_architecture_field()
268+ self.set_up_osystem_and_distro_series_fields(kwargs.get('instance'))
269
270 def set_up_architecture_field(self):
271 """Create the `architecture` field.
272@@ -248,6 +347,32 @@
273 choices=choices, required=True, initial=default_arch,
274 error_messages={'invalid_choice': invalid_arch_message})
275
276+ def set_up_osystem_and_distro_series_fields(self, instance):
277+ """Create the `osystem` and `distro_series` fields.
278+
279+ This needs to be done on the fly so that we can pass a dynamic list of
280+ usable operating systems and distro_series.
281+ """
282+ osystems = list_all_usable_osystems()
283+ releases = list_all_usable_releases(osystems)
284+ os_choices = list_osystem_choices(osystems)
285+ distro_choices = list_release_choices(releases)
286+ invalid_osystem_message = compose_invalid_choice_text(
287+ 'osystem', os_choices)
288+ invalid_distro_series_message = compose_invalid_choice_text(
289+ 'distro_series', distro_choices)
290+ self.fields['osystem'] = forms.ChoiceField(
291+ label="OS", choices=os_choices, required=False, initial='',
292+ error_messages={'invalid_choice': invalid_osystem_message})
293+ self.fields['distro_series'] = forms.ChoiceField(
294+ label="Release", choices=distro_choices,
295+ required=False, initial='',
296+ error_messages={'invalid_choice': invalid_distro_series_message})
297+ if instance is not None:
298+ initial_value = get_distro_series_inital(instance)
299+ if instance is not None:
300+ self.initial['distro_series'] = initial_value
301+
302 def clean_hostname(self):
303 # Don't allow the hostname to be changed if the node is
304 # currently allocated. Juju knows the node by its old name, so
305@@ -261,6 +386,9 @@
306
307 return new_hostname
308
309+ def clean_distro_series(self):
310+ return clean_distro_series_field(self, 'distro_series', 'osystem')
311+
312 def is_valid(self):
313 is_valid = super(NodeForm, self).is_valid()
314 if len(list_all_usable_architectures()) == 0:
315@@ -269,11 +397,24 @@
316 is_valid = False
317 return is_valid
318
319- distro_series = forms.ChoiceField(
320- choices=DISTRO_SERIES_CHOICES, required=False,
321- initial=DISTRO_SERIES.default,
322- label="Release",
323- error_messages={'invalid_choice': INVALID_DISTRO_SERIES_MESSAGE})
324+ def set_distro_series(self, series=''):
325+ """Sets the osystem and distro_series, from the provided
326+ distro_series.
327+ """
328+ # This implementation is used so that current API, is not broken. This
329+ # makes the distro_series a flat namespace. The distro_series is used
330+ # to search through the supporting operating systems, to find the
331+ # correct operating system that supports this distro_series.
332+ self.is_bound = True
333+ self.data['osystem'] = ''
334+ self.data['distro_series'] = ''
335+ if series is not None and series != '':
336+ osystem = get_osystem_from_release(series)
337+ if osystem is not None:
338+ self.data['osystem'] = osystem.name
339+ self.data['distro_series'] = '%s/%s' % (osystem.name, series)
340+ else:
341+ self.data['distro_series'] = series
342
343 hostname = forms.CharField(
344 label="Host name", required=False, help_text=(
345@@ -292,6 +433,7 @@
346 fields = (
347 'hostname',
348 'architecture',
349+ 'osystem',
350 'distro_series',
351 )
352
353@@ -872,16 +1014,34 @@
354 """Settings page, Commissioning section."""
355 check_compatibility = get_config_field('check_compatibility')
356 commissioning_distro_series = forms.ChoiceField(
357- choices=COMMISSIONING_DISTRO_SERIES_CHOICES, required=False,
358- label="Default distro series used for commissioning",
359+ choices=list_commisioning_choices(), required=False,
360+ label="Default Ubuntu release used for commissioning",
361 error_messages={'invalid_choice': compose_invalid_choice_text(
362 'commissioning_distro_series',
363- COMMISSIONING_DISTRO_SERIES_CHOICES)})
364+ list_commisioning_choices())})
365+
366+
367+class DeployForm(ConfigForm):
368+ """Settings page, Deploy section."""
369+ default_osystem = get_config_field('default_osystem')
370+ default_distro_series = get_config_field('default_distro_series')
371+
372+ def _load_initials(self):
373+ super(DeployForm, self)._load_initials()
374+ initial_os = self.initial['default_osystem']
375+ initial_series = self.initial['default_distro_series']
376+ self.initial['default_distro_series'] = '%s/%s' % (
377+ initial_os,
378+ initial_series
379+ )
380+
381+ def clean_default_distro_series(self):
382+ return clean_distro_series_field(
383+ self, 'default_distro_series', 'default_osystem')
384
385
386 class UbuntuForm(ConfigForm):
387 """Settings page, Ubuntu section."""
388- default_distro_series = get_config_field('default_distro_series')
389 main_archive = get_config_field('main_archive')
390 ports_archive = get_config_field('ports_archive')
391
392
393=== modified file 'src/maasserver/forms_settings.py'
394--- src/maasserver/forms_settings.py 2014-04-10 20:21:24 +0000
395+++ src/maasserver/forms_settings.py 2014-05-29 17:03:44 +0000
396@@ -23,19 +23,41 @@
397 from socket import gethostname
398
399 from django import forms
400-from maasserver.enum import (
401- COMMISSIONING_DISTRO_SERIES_CHOICES,
402- DISTRO_SERIES,
403- DISTRO_SERIES_CHOICES,
404- )
405+from maasserver.models.config import DEFAULT_OS
406 from maasserver.utils.forms import compose_invalid_choice_text
407+from provisioningserver.driver import OperatingSystemRegistry
408
409
410 INVALID_URL_MESSAGE = "Enter a valid url (e.g. http://host.example.com)."
411
412
413-INVALID_DISTRO_SERIES_MESSAGE = compose_invalid_choice_text(
414- 'distro_series', DISTRO_SERIES_CHOICES)
415+def list_osystem_choices():
416+ osystems = [osystem for _, osystem in OperatingSystemRegistry]
417+ osystems = sorted(osystems, key=lambda osystem: osystem.title)
418+ return [
419+ (osystem.name, osystem.title)
420+ for osystem in osystems
421+ ]
422+
423+
424+def list_release_choices():
425+ osystems = [osystem for _, osystem in OperatingSystemRegistry]
426+ choices = []
427+ for osystem in osystems:
428+ supported = sorted(osystem.get_supported_releases())
429+ options = osystem.format_release_choices(supported)
430+ options = [
431+ ('%s/%s' % (osystem.name, name), title)
432+ for name, title in options
433+ ]
434+ choices += options
435+ return choices
436+
437+
438+def list_commisioning_choices():
439+ releases = DEFAULT_OS.get_supported_commissioning_releases()
440+ options = DEFAULT_OS.format_release_choices(releases)
441+ return list(options)
442
443
444 CONFIG_ITEMS = {
445@@ -139,28 +161,46 @@
446 "e.g. for ntp.ubuntu.com: '91.189.94.4'")
447 }
448 },
449+ 'default_osystem': {
450+ 'default': DEFAULT_OS.name,
451+ 'form': forms.ChoiceField,
452+ 'form_kwargs': {
453+ 'label': "Default operating system used for deployment",
454+ 'choices': list_osystem_choices(),
455+ 'required': False,
456+ 'error_messages': {
457+ 'invalid_choice': compose_invalid_choice_text(
458+ 'osystem',
459+ list_osystem_choices())},
460+ }
461+ },
462 'default_distro_series': {
463- 'default': DISTRO_SERIES.trusty,
464+ 'default': '%s/%s' % (
465+ DEFAULT_OS.name,
466+ DEFAULT_OS.get_default_release()
467+ ),
468 'form': forms.ChoiceField,
469 'form_kwargs': {
470- 'label': "Default distro series used for deployment",
471- 'choices': DISTRO_SERIES_CHOICES,
472+ 'label': "Default OS release used for deployment",
473+ 'choices': list_release_choices(),
474 'required': False,
475 'error_messages': {
476- 'invalid_choice': INVALID_DISTRO_SERIES_MESSAGE},
477+ 'invalid_choice': compose_invalid_choice_text(
478+ 'distro_series',
479+ list_release_choices())},
480 }
481 },
482 'commissioning_distro_series': {
483- 'default': DISTRO_SERIES.trusty,
484+ 'default': DEFAULT_OS.get_default_commissioning_release(),
485 'form': forms.ChoiceField,
486 'form_kwargs': {
487- 'label': "Default distro series used for commissioning",
488- 'choices': COMMISSIONING_DISTRO_SERIES_CHOICES,
489+ 'label': "Default Ubuntu release used for commissioning",
490+ 'choices': list_commisioning_choices(),
491 'required': False,
492 'error_messages': {
493 'invalid_choice': compose_invalid_choice_text(
494 'commissioning_distro_series',
495- COMMISSIONING_DISTRO_SERIES_CHOICES)},
496+ list_commisioning_choices())},
497 }
498 },
499 'enable_third_party_drivers': {
500
501=== modified file 'src/maasserver/models/config.py'
502--- src/maasserver/models/config.py 2014-04-14 21:42:00 +0000
503+++ src/maasserver/models/config.py 2014-05-29 17:03:44 +0000
504@@ -28,8 +28,11 @@
505 )
506 from django.db.models.signals import post_save
507 from maasserver import DefaultMeta
508-from maasserver.enum import DISTRO_SERIES
509 from maasserver.fields import JSONObjectField
510+from provisioningserver.driver.os_ubuntu import UbuntuOS
511+
512+
513+DEFAULT_OS = UbuntuOS()
514
515
516 def get_default_config():
517@@ -40,11 +43,14 @@
518 # Ubuntu section configuration.
519 'main_archive': 'http://archive.ubuntu.com/ubuntu',
520 'ports_archive': 'http://ports.ubuntu.com/ubuntu-ports',
521- 'commissioning_distro_series': DISTRO_SERIES.trusty,
522+ 'commissioning_osystem': DEFAULT_OS.name,
523+ 'commissioning_distro_series':
524+ DEFAULT_OS.get_default_commissioning_release(),
525 # Network section configuration.
526 'maas_name': gethostname(),
527 'enlistment_domain': b'local',
528- 'default_distro_series': DISTRO_SERIES.trusty,
529+ 'default_osystem': DEFAULT_OS.name,
530+ 'default_distro_series': DEFAULT_OS.get_default_release(),
531 'http_proxy': None,
532 'upstream_dns': None,
533 'ntp_server': '91.189.94.4', # ntp.ubuntu.com
534
535=== modified file 'src/maasserver/models/node.py'
536--- src/maasserver/models/node.py 2014-05-27 07:11:28 +0000
537+++ src/maasserver/models/node.py 2014-05-29 17:03:44 +0000
538@@ -49,8 +49,6 @@
539 logger,
540 )
541 from maasserver.enum import (
542- DISTRO_SERIES,
543- DISTRO_SERIES_CHOICES,
544 NODE_PERMISSION,
545 NODE_STATUS,
546 NODE_STATUS_CHOICES,
547@@ -72,6 +70,7 @@
548 strip_domain,
549 )
550 from piston.models import Token
551+from provisioningserver.driver import OperatingSystemRegistry
552 from provisioningserver.tasks import (
553 power_off,
554 power_on,
555@@ -486,7 +485,7 @@
556 max_length=20, blank=True, default='')
557
558 distro_series = CharField(
559- max_length=20, choices=DISTRO_SERIES_CHOICES, blank=True, default='')
560+ max_length=20, blank=True, default='')
561
562 architecture = CharField(max_length=31, blank=False)
563
564@@ -793,21 +792,30 @@
565 """The name of the queue for tasks specific to this node."""
566 return self.nodegroup.work_queue
567
568+ def get_osystem(self):
569+ """Return the operating system to install that node."""
570+ use_default_osystem = (self.osystem is None or self.osystem == '')
571+ if use_default_osystem:
572+ return Config.objects.get_config('default_osystem')
573+ else:
574+ return self.osystem
575+
576 def get_distro_series(self):
577 """Return the distro series to install that node."""
578+ use_default_osystem = (
579+ self.osystem is None or
580+ self.osystem == '')
581 use_default_distro_series = (
582- not self.distro_series or
583- self.distro_series == DISTRO_SERIES.default)
584- if use_default_distro_series:
585+ self.distro_series is None or
586+ self.distro_series == '')
587+ if use_default_osystem and use_default_distro_series:
588 return Config.objects.get_config('default_distro_series')
589+ elif use_default_distro_series:
590+ osystem = OperatingSystemRegistry[self.osystem]
591+ return osystem.get_default_release()
592 else:
593 return self.distro_series
594
595- def set_distro_series(self, series=''):
596- """Set the distro series to install that node."""
597- self.distro_series = series
598- self.save()
599-
600 def get_effective_power_parameters(self):
601 """Return effective power parameters, including any defaults."""
602 if self.power_parameters:
603@@ -860,6 +868,7 @@
604 self.token = None
605 self.agent_name = ''
606 self.set_netboot()
607+ self.osystem = ''
608 self.distro_series = ''
609 self.save()
610
611
612=== modified file 'src/maasserver/models/tests/test_node.py'
613--- src/maasserver/models/tests/test_node.py 2014-05-27 12:35:35 +0000
614+++ src/maasserver/models/tests/test_node.py 2014-05-29 17:03:44 +0000
615@@ -20,7 +20,6 @@
616 from django.core.exceptions import ValidationError
617 from maasserver.clusterrpc.power_parameters import get_power_types
618 from maasserver.enum import (
619- DISTRO_SERIES,
620 NODE_PERMISSION,
621 NODE_STATUS,
622 NODE_STATUS_CHOICES,
623@@ -270,15 +269,14 @@
624 offset += timedelta(1)
625 self.assertEqual(macs[0], node.get_primary_mac().mac_address)
626
627+ def test_get_osystem_returns_default_osystem(self):
628+ node = factory.make_node(osystem='')
629+ osystem = Config.objects.get_config('default_osystem')
630+ self.assertEqual(osystem, node.get_osystem())
631+
632 def test_get_distro_series_returns_default_series(self):
633- node = factory.make_node()
634- series = Config.objects.get_config('commissioning_distro_series')
635- self.assertEqual(series, node.get_distro_series())
636-
637- def test_set_get_distro_series_returns_series(self):
638- node = factory.make_node()
639- series = DISTRO_SERIES.quantal
640- node.set_distro_series(series)
641+ node = factory.make_node(distro_series='')
642+ series = Config.objects.get_config('default_distro_series')
643 self.assertEqual(series, node.get_distro_series())
644
645 def test_delete_node_deletes_related_mac(self):
646@@ -597,11 +595,15 @@
647 node.release()
648 self.assertTrue(node.netboot)
649
650- def test_release_clears_distro_series(self):
651+ def test_release_clears_osystem_and_distro_series(self):
652 node = factory.make_node(
653 status=NODE_STATUS.ALLOCATED, owner=factory.make_user())
654- node.set_distro_series(series=DISTRO_SERIES.quantal)
655+ osystem = factory.getRandomOS()
656+ release = factory.getRandomRelease(osystem)
657+ node.osystem = osystem.name
658+ node.distro_series = release
659 node.release()
660+ self.assertEqual("", node.osystem)
661 self.assertEqual("", node.distro_series)
662
663 def test_release_powers_off_node(self):
664
665=== modified file 'src/maasserver/preseed.py'
666--- src/maasserver/preseed.py 2014-04-23 16:09:39 +0000
667+++ src/maasserver/preseed.py 2014-05-29 17:03:44 +0000
668@@ -93,7 +93,7 @@
669
670 def get_curtin_installer_url(node):
671 """Return the URL where curtin on the node can download its installer."""
672- osystem = 'ubuntu'
673+ osystem = node.get_osystem()
674 series = node.get_distro_series()
675 cluster_host = pick_cluster_controller_address(node)
676 # XXX rvb(?): The path shouldn't be hardcoded like this, but rather synced
677
678=== modified file 'src/maasserver/testing/osystems.py'
679--- src/maasserver/testing/osystems.py 2014-05-14 15:12:48 +0000
680+++ src/maasserver/testing/osystems.py 2014-05-29 17:03:44 +0000
681@@ -25,23 +25,27 @@
682 from provisioningserver.driver import BOOT_IMAGE_PURPOSE
683
684
685-def make_osystem_with_releases(testcase, osystem_name=None, releases=None):
686+def make_osystem_with_releases(testcase, osystem_name=None, releases=None,
687+ purposes=None):
688 """Generate an arbitrary operating system.
689
690 :param osystem_name: The operating system name. Useful in cases where
691 we need to test that not supplying an os works correctly.
692 :param releases: The list of releases name. Useful in cases where
693 we need to test that not supplying a release works correctly.
694+ :param purposes: The purpose's of the boot images.
695 """
696 if osystem_name is None:
697 osystem_name = factory.make_name('os')
698 if releases is None:
699 releases = [factory.make_name('release') for _ in range(3)]
700+ if purposes is None:
701+ purposes = [BOOT_IMAGE_PURPOSE.INSTALL, BOOT_IMAGE_PURPOSE.XINSTALL]
702
703 osystem = make_osystem(
704 testcase,
705 osystem_name,
706- [BOOT_IMAGE_PURPOSE.INSTALL, BOOT_IMAGE_PURPOSE.XINSTALL])
707+ purposes)
708 if releases is not None and releases != []:
709 osystem.fake_list = releases
710 return osystem
711@@ -72,7 +76,8 @@
712 forms, 'list_all_usable_releases').return_value = distro_series
713
714
715-def make_usable_osystem(testcase, osystem_name=None, releases=None):
716+def make_usable_osystem(testcase, osystem_name=None, releases=None,
717+ purposes=None):
718 """Return arbitrary operating system, and make it "usable."
719
720 A usable operating system is one for which boot images are available.
721@@ -83,8 +88,11 @@
722 we need to test that not supplying an os works correctly.
723 :param releases: The list of releases name. Useful in cases where
724 we need to test that not supplying a release works correctly.
725+ :param purposse: The list of purposes. Useful in cases where
726+ we need to test that not supplying a purpose works correctly.
727 """
728 osystem = make_osystem_with_releases(
729- testcase, osystem_name=osystem_name, releases=releases)
730+ testcase, osystem_name=osystem_name, releases=releases,
731+ purposes=purposes)
732 patch_usable_osystems(testcase, [osystem])
733 return osystem
734
735=== modified file 'src/maasserver/tests/test_api_node.py'
736--- src/maasserver/tests/test_api_node.py 2014-04-25 18:03:38 +0000
737+++ src/maasserver/tests/test_api_node.py 2014-05-29 17:03:44 +0000
738@@ -23,7 +23,6 @@
739 import bson
740 from django.core.urlresolvers import reverse
741 from maasserver.enum import (
742- DISTRO_SERIES,
743 NODE_STATUS,
744 NODE_STATUS_CHOICES_DICT,
745 )
746@@ -40,6 +39,7 @@
747 from maasserver.testing.architecture import make_usable_architecture
748 from maasserver.testing.factory import factory
749 from maasserver.testing.oauthclient import OAuthAuthenticatedClient
750+from maasserver.testing.osystems import make_usable_osystem
751 from maasserver.testing.testcase import MAASServerTestCase
752 from maasserver.utils import map_enum
753 from metadataserver.models import (
754@@ -221,24 +221,31 @@
755 self.assertEqual(
756 node.system_id, json.loads(response.content)['system_id'])
757
758- def test_POST_start_sets_distro_series(self):
759+ def test_POST_start_sets_osystem_and_distro_series(self):
760 node = factory.make_node(
761 owner=self.logged_in_user, mac=True,
762- power_type='ether_wake')
763- distro_series = factory.getRandomEnum(DISTRO_SERIES)
764+ power_type='ether_wake',
765+ architecture=make_usable_architecture(self))
766+ osystem = make_usable_osystem(self)
767+ distro_series = factory.getRandomRelease(osystem)
768 response = self.client.post(
769- self.get_node_uri(node),
770- {'op': 'start', 'distro_series': distro_series})
771+ self.get_node_uri(node), {
772+ 'op': 'start',
773+ 'distro_series': distro_series
774+ })
775 self.assertEqual(
776 (httplib.OK, node.system_id),
777 (response.status_code, json.loads(response.content)['system_id']))
778 self.assertEqual(
779+ osystem.name, reload_object(node).osystem)
780+ self.assertEqual(
781 distro_series, reload_object(node).distro_series)
782
783 def test_POST_start_validates_distro_series(self):
784 node = factory.make_node(
785 owner=self.logged_in_user, mac=True,
786- power_type='ether_wake')
787+ power_type='ether_wake',
788+ architecture=make_usable_architecture(self))
789 invalid_distro_series = factory.getRandomString()
790 response = self.client.post(
791 self.get_node_uri(node),
792@@ -247,7 +254,8 @@
793 (
794 httplib.BAD_REQUEST,
795 {'distro_series': [
796- "Value u'%s' is not a valid choice." %
797+ "'%s' is not a valid distro_series. "
798+ "It should be one of: ''." %
799 invalid_distro_series]}
800 ),
801 (response.status_code, json.loads(response.content)))
802@@ -300,18 +308,23 @@
803 self.client.post(self.get_node_uri(node), {'op': 'release'})
804 self.assertTrue(reload_object(node).netboot)
805
806- def test_POST_release_resets_distro_series(self):
807+ def test_POST_release_resets_osystem_and_distro_series(self):
808+ osystem = factory.getRandomOS()
809+ release = factory.getRandomRelease(osystem)
810 node = factory.make_node(
811 status=NODE_STATUS.ALLOCATED, owner=self.logged_in_user,
812- distro_series=factory.getRandomEnum(DISTRO_SERIES))
813+ osystem=osystem.name, distro_series=release)
814 self.client.post(self.get_node_uri(node), {'op': 'release'})
815+ self.assertEqual('', reload_object(node).osystem)
816 self.assertEqual('', reload_object(node).distro_series)
817
818 def test_POST_release_resets_agent_name(self):
819 agent_name = factory.make_name('agent-name')
820+ osystem = factory.getRandomOS()
821+ release = factory.getRandomRelease(osystem)
822 node = factory.make_node(
823 status=NODE_STATUS.ALLOCATED, owner=self.logged_in_user,
824- distro_series=factory.getRandomEnum(DISTRO_SERIES),
825+ osystem=osystem.name, distro_series=release,
826 agent_name=agent_name)
827 self.client.post(self.get_node_uri(node), {'op': 'release'})
828 self.assertEqual('', reload_object(node).agent_name)
829
830=== modified file 'src/maasserver/tests/test_api_pxeconfig.py'
831--- src/maasserver/tests/test_api_pxeconfig.py 2014-05-20 02:58:09 +0000
832+++ src/maasserver/tests/test_api_pxeconfig.py 2014-05-29 17:03:44 +0000
833@@ -35,11 +35,13 @@
834 )
835 from maasserver.testing.architecture import make_usable_architecture
836 from maasserver.testing.factory import factory
837+from maasserver.testing.osystems import make_usable_osystem
838 from maasserver.testing.testcase import MAASServerTestCase
839 from maastesting.fakemethod import FakeMethod
840 from mock import Mock
841 from netaddr import IPNetwork
842 from provisioningserver import kernel_opts
843+from provisioningserver.driver import BOOT_IMAGE_PURPOSE
844 from provisioningserver.kernel_opts import KernelParameters
845 from testtools.matchers import (
846 Contains,
847@@ -133,7 +135,7 @@
848 self.assertEqual(value, response_dict['extra_opts'])
849
850 def test_pxeconfig_uses_present_boot_image(self):
851- osystem = 'ubuntu'
852+ osystem = Config.objects.get_config('commissioning_osystem')
853 release = Config.objects.get_config('commissioning_distro_series')
854 nodegroup = factory.make_node_group()
855 factory.make_boot_image(
856@@ -284,7 +286,8 @@
857 def test_get_boot_purpose_unknown_node(self):
858 # A node that's not yet known to MAAS is assumed to be enlisting,
859 # which uses a "commissioning" image.
860- self.assertEqual("commissioning", api.get_boot_purpose(None))
861+ self.assertEqual("commissioning", api.get_boot_purpose(
862+ None, None, None, None, None, None))
863
864 def test_get_boot_purpose_known_node(self):
865 # The following table shows the expected boot "purpose" for each set
866@@ -307,11 +310,33 @@
867 node.use_fastpath_installer()
868 for name, value in parameters.items():
869 setattr(node, name, value)
870- self.assertEqual(purpose, api.get_boot_purpose(node))
871+ osystem = node.get_osystem()
872+ series = node.get_distro_series()
873+ arch, subarch = node.architecture.split('/')
874+ self.assertEqual(
875+ purpose,
876+ api.get_boot_purpose(
877+ node, osystem, arch, subarch, series, None))
878+
879+ def test_get_boot_purpose_osystem_no_xinstall_support(self):
880+ osystem = make_usable_osystem(
881+ self, purposes=[BOOT_IMAGE_PURPOSE.INSTALL])
882+ release = factory.getRandomRelease(osystem)
883+ node = factory.make_node(
884+ status=NODE_STATUS.ALLOCATED, netboot=True,
885+ osystem=osystem.name, distro_series=release)
886+ node.use_fastpath_installer()
887+ node_os = node.get_osystem()
888+ node_series = node.get_distro_series()
889+ arch, subarch = node.architecture.split('/')
890+ self.assertEqual(
891+ 'install',
892+ api.get_boot_purpose(
893+ node, node_os, arch, subarch, node_series, None))
894
895 def test_pxeconfig_uses_boot_purpose(self):
896 fake_boot_purpose = factory.make_name("purpose")
897- self.patch(api, "get_boot_purpose", lambda node: fake_boot_purpose)
898+ self.patch(api, "get_boot_purpose").return_value = fake_boot_purpose
899 response = self.client.get(reverse('pxeconfig'),
900 self.get_default_params())
901 self.assertEqual(
902
903=== modified file 'src/maasserver/tests/test_forms.py'
904--- src/maasserver/tests/test_forms.py 2014-05-20 14:16:58 +0000
905+++ src/maasserver/tests/test_forms.py 2014-05-29 17:03:44 +0000
906@@ -53,6 +53,8 @@
907 InstanceListField,
908 INTERFACES_VALIDATION_ERROR_MESSAGE,
909 list_all_usable_architectures,
910+ list_all_usable_osystems,
911+ list_all_usable_releases,
912 MACAddressForm,
913 MAX_MESSAGES,
914 merge_error_messages,
915@@ -98,6 +100,11 @@
916 patch_usable_architectures,
917 )
918 from maasserver.testing.factory import factory
919+from maasserver.testing.osystems import (
920+ make_osystem_with_releases,
921+ make_usable_osystem,
922+ patch_usable_osystems,
923+ )
924 from maasserver.testing.testcase import MAASServerTestCase
925 from maasserver.utils import map_enum
926 from maasserver.utils.forms import compose_invalid_choice_text
927@@ -120,8 +127,8 @@
928
929 class TestHelpers(MAASServerTestCase):
930
931- def make_usable_boot_images(self, nodegroup=None, arch=None,
932- subarchitecture=None):
933+ def make_usable_boot_images(self, nodegroup=None, osystem=None,
934+ arch=None, subarchitecture=None, release=None):
935 """Create a set of boot images, so the architecture becomes "usable".
936
937 This will make the images' architecture show up in the list of usable
938@@ -131,14 +138,19 @@
939 """
940 if nodegroup is None:
941 nodegroup = factory.make_node_group()
942+ if osystem is None:
943+ osystem = factory.make_name('os')
944 if arch is None:
945 arch = factory.make_name('arch')
946 if subarchitecture is None:
947 subarchitecture = factory.make_name('subarch')
948+ if release is None:
949+ release = factory.make_name('release')
950 for purpose in ['install', 'commissioning']:
951 factory.make_boot_image(
952- nodegroup=nodegroup, architecture=arch,
953- subarchitecture=subarchitecture, purpose=purpose)
954+ nodegroup=nodegroup, osystem=osystem, architecture=arch,
955+ subarchitecture=subarchitecture, release=release,
956+ purpose=purpose)
957
958 def test_initialize_node_group_leaves_nodegroup_reference_intact(self):
959 preselected_nodegroup = factory.make_node_group()
960@@ -201,6 +213,67 @@
961 arches = [factory.make_name('arch') for _ in range(5)]
962 self.assertEqual(arches[0], pick_default_architecture(arches))
963
964+ def test_list_all_usable_osystems_combines_nodegroups(self):
965+ osystem_names = [factory.make_name('os') for _ in range(3)]
966+ expected = []
967+ for name in osystem_names:
968+ self.make_usable_boot_images(osystem=name)
969+ expected.append(make_usable_osystem(self, name))
970+ self.assertItemsEqual(expected, list_all_usable_osystems())
971+
972+ def test_list_all_usable_osystems_sorts_output(self):
973+ osystem_names = [factory.make_name('os') for _ in range(3)]
974+ expected = []
975+ for name in osystem_names:
976+ self.make_usable_boot_images(osystem=name)
977+ expected.append(make_usable_osystem(self, name))
978+ expected = sorted(expected, key=lambda osystem: osystem.title)
979+ self.assertEqual(expected, list_all_usable_osystems())
980+
981+ def test_list_all_usable_osystems_returns_no_duplicates(self):
982+ os_name = factory.make_name('os')
983+ self.make_usable_boot_images(osystem=os_name)
984+ self.make_usable_boot_images(osystem=os_name)
985+ osystem = make_usable_osystem(self, os_name)
986+ self.assertEqual(
987+ [osystem], list_all_usable_osystems())
988+
989+ def test_list_all_usable_releases_combines_nodegroups(self):
990+ expected = {}
991+ osystems = []
992+ os_names = [factory.make_name('os') for _ in range(3)]
993+ for name in os_names:
994+ releases = [factory.make_name('release') for _ in range(3)]
995+ for release in releases:
996+ self.make_usable_boot_images(osystem=name, release=release)
997+ osystems.append(
998+ make_usable_osystem(self, name, releases=releases))
999+ expected[name] = releases
1000+ self.assertItemsEqual(expected, list_all_usable_releases(osystems))
1001+
1002+ def test_list_all_usable_releases_sorts_output(self):
1003+ expected = {}
1004+ osystems = []
1005+ os_names = [factory.make_name('os') for _ in range(3)]
1006+ for name in os_names:
1007+ releases = [factory.make_name('release') for _ in range(3)]
1008+ for release in releases:
1009+ self.make_usable_boot_images(osystem=name, release=release)
1010+ osystems.append(
1011+ make_usable_osystem(self, name, releases=releases))
1012+ expected[name] = sorted(releases)
1013+ self.assertEqual(expected, list_all_usable_releases(osystems))
1014+
1015+ def test_list_all_usable_releases_returns_no_duplicates(self):
1016+ os_name = factory.make_name('os')
1017+ release = factory.make_name('release')
1018+ self.make_usable_boot_images(osystem=os_name, release=release)
1019+ self.make_usable_boot_images(osystem=os_name, release=release)
1020+ osystem = make_usable_osystem(self, os_name, releases=[release])
1021+ expected = {}
1022+ expected[os_name] = [release]
1023+ self.assertEqual(expected, list_all_usable_releases([osystem]))
1024+
1025 def test_remove_None_values_removes_None_values_in_dict(self):
1026 random_input = factory.getRandomString()
1027 self.assertEqual(
1028@@ -430,6 +503,7 @@
1029 [
1030 'hostname',
1031 'architecture',
1032+ 'osystem',
1033 'distro_series',
1034 'nodegroup',
1035 ], list(form.fields))
1036@@ -489,6 +563,77 @@
1037 [NO_ARCHITECTURES_AVAILABLE],
1038 form.errors['architecture'])
1039
1040+ def test_accepts_osystem(self):
1041+ osystem = make_usable_osystem(self)
1042+ form = NodeForm(data={
1043+ 'hostname': factory.make_name('host'),
1044+ 'architecture': make_usable_architecture(self),
1045+ 'osystem': osystem.name,
1046+ })
1047+ self.assertTrue(form.is_valid(), form._errors)
1048+
1049+ def test_rejects_invalid_osystem(self):
1050+ patch_usable_osystems(self)
1051+ form = NodeForm(data={
1052+ 'hostname': factory.make_name('host'),
1053+ 'architecture': make_usable_architecture(self),
1054+ 'osystem': factory.make_name('os'),
1055+ })
1056+ self.assertFalse(form.is_valid())
1057+ self.assertItemsEqual(['osystem'], form._errors.keys())
1058+
1059+ def test_starts_with_default_osystem(self):
1060+ osystems = [make_osystem_with_releases(self) for _ in range(5)]
1061+ patch_usable_osystems(self, osystems)
1062+ form = NodeForm()
1063+ self.assertEqual(
1064+ '',
1065+ form.fields['osystem'].initial)
1066+
1067+ def test_accepts_osystem_distro_series(self):
1068+ osystem = make_usable_osystem(self)
1069+ release = osystem.get_default_release()
1070+ form = NodeForm(data={
1071+ 'hostname': factory.make_name('host'),
1072+ 'architecture': make_usable_architecture(self),
1073+ 'osystem': osystem.name,
1074+ 'distro_series': '%s/%s' % (osystem.name, release),
1075+ })
1076+ self.assertTrue(form.is_valid(), form._errors)
1077+
1078+ def test_rejects_invalid_osystem_distro_series(self):
1079+ osystem = make_usable_osystem(self)
1080+ release = factory.make_name('release')
1081+ form = NodeForm(data={
1082+ 'hostname': factory.make_name('host'),
1083+ 'architecture': make_usable_architecture(self),
1084+ 'osystem': osystem.name,
1085+ 'distro_series': '%s/%s' % (osystem.name, release),
1086+ })
1087+ self.assertFalse(form.is_valid())
1088+ self.assertItemsEqual(['distro_series'], form._errors.keys())
1089+
1090+ def test_starts_with_default_distro_series(self):
1091+ osystems = [make_osystem_with_releases(self) for _ in range(5)]
1092+ patch_usable_osystems(self, osystems)
1093+ form = NodeForm()
1094+ self.assertEqual(
1095+ '',
1096+ form.fields['distro_series'].initial)
1097+
1098+ def test_rejects_mismatch_osystem_distro_series(self):
1099+ osystem = make_usable_osystem(self)
1100+ release = osystem.get_default_release()
1101+ invalid = factory.make_name('invalid_os')
1102+ form = NodeForm(data={
1103+ 'hostname': factory.make_name('host'),
1104+ 'architecture': make_usable_architecture(self),
1105+ 'osystem': osystem.name,
1106+ 'distro_series': '%s/%s' % (invalid, release),
1107+ })
1108+ self.assertFalse(form.is_valid())
1109+ self.assertItemsEqual(['distro_series'], form._errors.keys())
1110+
1111
1112 class TestAdminNodeForm(MAASServerTestCase):
1113
1114@@ -500,6 +645,7 @@
1115 [
1116 'hostname',
1117 'architecture',
1118+ 'osystem',
1119 'distro_series',
1120 'power_type',
1121 'power_parameters',
1122
1123=== modified file 'src/maasserver/tests/test_preseed.py'
1124--- src/maasserver/tests/test_preseed.py 2014-05-13 18:43:40 +0000
1125+++ src/maasserver/tests/test_preseed.py 2014-05-29 17:03:44 +0000
1126@@ -22,7 +22,6 @@
1127 from django.conf import settings
1128 from django.core.urlresolvers import reverse
1129 from maasserver.enum import (
1130- DISTRO_SERIES,
1131 NODE_STATUS,
1132 NODEGROUPINTERFACE_MANAGEMENT,
1133 PRESEED_TYPE,
1134@@ -590,7 +589,7 @@
1135 node = factory.make_node()
1136 arch, subarch = node.architecture.split('/')
1137 factory.make_boot_image(
1138- osystem='ubuntu',
1139+ osystem=node.get_osystem(),
1140 architecture=arch, subarchitecture=subarch,
1141 release=node.get_distro_series(), purpose='xinstall',
1142 nodegroup=node.nodegroup)
1143@@ -678,18 +677,15 @@
1144 self.assertIn('cloud-init', context['curtin_preseed'])
1145
1146 def test_get_curtin_installer_url_returns_url(self):
1147- osystem = 'ubuntu'
1148- # Exclude DISTRO_SERIES.default. It's a special value that defers
1149- # to a run-time setting which we don't provide in this test.
1150- series = factory.getRandomEnum(
1151- DISTRO_SERIES, but_not=DISTRO_SERIES.default)
1152+ osystem = factory.getRandomOS()
1153+ series = factory.getRandomRelease(osystem)
1154 architecture = make_usable_architecture(self)
1155 node = factory.make_node(
1156- architecture=architecture,
1157+ osystem=osystem.name, architecture=architecture,
1158 distro_series=series)
1159 arch, subarch = architecture.split('/')
1160 boot_image = factory.make_boot_image(
1161- osystem=osystem, architecture=arch,
1162+ osystem=osystem.name, architecture=arch,
1163 subarchitecture=subarch, release=series,
1164 purpose='xinstall', nodegroup=node.nodegroup)
1165
1166@@ -698,20 +694,20 @@
1167 [interface] = node.nodegroup.get_managed_interfaces()
1168 self.assertEqual(
1169 'http://%s/MAAS/static/images/%s/%s/%s/%s/%s/root-tgz' % (
1170- interface.ip, osystem, arch, subarch,
1171+ interface.ip, osystem.name, arch, subarch,
1172 series, boot_image.label),
1173 installer_url)
1174
1175 def test_get_curtin_installer_url_fails_if_no_boot_image(self):
1176- osystem = 'ubuntu'
1177- series = factory.getRandomEnum(
1178- DISTRO_SERIES, but_not=DISTRO_SERIES.default)
1179+ osystem = factory.getRandomOS()
1180+ series = factory.getRandomRelease(osystem)
1181 architecture = make_usable_architecture(self)
1182 node = factory.make_node(
1183+ osystem=osystem.name,
1184 architecture=architecture, distro_series=series)
1185 # Generate a boot image with a different arch/subarch.
1186 factory.make_boot_image(
1187- osystem=osystem,
1188+ osystem=osystem.name,
1189 architecture=factory.make_name('arch'),
1190 subarchitecture=factory.make_name('subarch'), release=series,
1191 purpose='xinstall', nodegroup=node.nodegroup)
1192@@ -722,7 +718,7 @@
1193 msg = (
1194 "No image could be found for the given selection: "
1195 "os=%s, arch=%s, subarch=%s, series=%s, purpose=xinstall." % (
1196- osystem,
1197+ osystem.name,
1198 arch,
1199 subarch,
1200 node.get_distro_series(),
1201
1202=== modified file 'src/maasserver/views/settings.py'
1203--- src/maasserver/views/settings.py 2014-04-10 13:43:33 +0000
1204+++ src/maasserver/views/settings.py 2014-05-29 17:03:44 +0000
1205@@ -41,6 +41,7 @@
1206 from maasserver.exceptions import CannotDeleteUserException
1207 from maasserver.forms import (
1208 CommissioningForm,
1209+ DeployForm,
1210 EditUserForm,
1211 GlobalKernelOptsForm,
1212 MAASAndNetworkForm,
1213@@ -177,6 +178,13 @@
1214 if response is not None:
1215 return response
1216
1217+ # Process the Deploy form.
1218+ deploy_form, response = process_form(
1219+ request, DeployForm, reverse('settings'), 'deploy',
1220+ "Configuration updated.")
1221+ if response is not None:
1222+ return response
1223+
1224 # Process the Ubuntu form.
1225 ubuntu_form, response = process_form(
1226 request, UbuntuForm, reverse('settings'), 'ubuntu',
1227@@ -202,6 +210,7 @@
1228 'maas_and_network_form': maas_and_network_form,
1229 'third_party_drivers_form': third_party_drivers_form,
1230 'commissioning_form': commissioning_form,
1231+ 'deploy_form': deploy_form,
1232 'ubuntu_form': ubuntu_form,
1233 'kernelopts_form': kernelopts_form,
1234 },
1235
1236=== modified file 'src/maasserver/views/tests/test_settings.py'
1237--- src/maasserver/views/tests/test_settings.py 2014-04-10 13:43:33 +0000
1238+++ src/maasserver/views/tests/test_settings.py 2014-05-29 17:03:44 +0000
1239@@ -20,14 +20,11 @@
1240 from django.contrib.auth.models import User
1241 from django.core.urlresolvers import reverse
1242 from lxml.html import fromstring
1243-from maasserver.enum import (
1244- COMMISSIONING_DISTRO_SERIES_CHOICES,
1245- DISTRO_SERIES,
1246- )
1247 from maasserver.models import (
1248 Config,
1249 UserProfile,
1250 )
1251+from maasserver.models.config import DEFAULT_OS
1252 from maasserver.testing import (
1253 extract_redirect,
1254 get_prefixed_form_data,
1255@@ -112,8 +109,7 @@
1256 def test_settings_commissioning_POST(self):
1257 self.client_log_in(as_admin=True)
1258 new_check_compatibility = factory.getRandomBoolean()
1259- new_commissioning_distro_series = factory.getRandomChoice(
1260- COMMISSIONING_DISTRO_SERIES_CHOICES)
1261+ new_commissioning = factory.getRandomCommissioningRelease(DEFAULT_OS)
1262 response = self.client.post(
1263 reverse('settings'),
1264 get_prefixed_form_data(
1265@@ -121,14 +117,14 @@
1266 data={
1267 'check_compatibility': new_check_compatibility,
1268 'commissioning_distro_series': (
1269- new_commissioning_distro_series),
1270+ new_commissioning),
1271 }))
1272
1273 self.assertEqual(httplib.FOUND, response.status_code)
1274 self.assertEqual(
1275 (
1276 new_check_compatibility,
1277- new_commissioning_distro_series,
1278+ new_commissioning,
1279 ),
1280 (
1281 Config.objects.get_config('check_compatibility'),
1282@@ -156,11 +152,38 @@
1283 Config.objects.get_config('enable_third_party_drivers'),
1284 ))
1285
1286+ def test_settings_deploy_POST(self):
1287+ self.client_log_in(as_admin=True)
1288+ new_osystem = factory.getRandomOS()
1289+ new_default_osystem = new_osystem.name
1290+ new_default_distro_series = factory.getRandomRelease(new_osystem)
1291+ response = self.client.post(
1292+ reverse('settings'),
1293+ get_prefixed_form_data(
1294+ prefix='deploy',
1295+ data={
1296+ 'default_osystem': new_default_osystem,
1297+ 'default_distro_series': '%s/%s' % (
1298+ new_default_osystem,
1299+ new_default_distro_series
1300+ ),
1301+ }))
1302+
1303+ self.assertEqual(httplib.FOUND, response.status_code, response.content)
1304+ self.assertEqual(
1305+ (
1306+ new_default_osystem,
1307+ new_default_distro_series,
1308+ ),
1309+ (
1310+ Config.objects.get_config('default_osystem'),
1311+ Config.objects.get_config('default_distro_series'),
1312+ ))
1313+
1314 def test_settings_ubuntu_POST(self):
1315 self.client_log_in(as_admin=True)
1316 new_main_archive = 'http://test.example.com/archive'
1317 new_ports_archive = 'http://test2.example.com/archive'
1318- new_default_distro_series = factory.getRandomEnum(DISTRO_SERIES)
1319 response = self.client.post(
1320 reverse('settings'),
1321 get_prefixed_form_data(
1322@@ -168,7 +191,6 @@
1323 data={
1324 'main_archive': new_main_archive,
1325 'ports_archive': new_ports_archive,
1326- 'default_distro_series': new_default_distro_series,
1327 }))
1328
1329 self.assertEqual(httplib.FOUND, response.status_code, response.content)
1330@@ -176,12 +198,10 @@
1331 (
1332 new_main_archive,
1333 new_ports_archive,
1334- new_default_distro_series,
1335 ),
1336 (
1337 Config.objects.get_config('main_archive'),
1338 Config.objects.get_config('ports_archive'),
1339- Config.objects.get_config('default_distro_series'),
1340 ))
1341
1342 def test_settings_kernelopts_POST(self):
1343
1344=== modified file 'src/metadataserver/tests/test_api.py'
1345--- src/metadataserver/tests/test_api.py 2014-04-23 16:09:39 +0000
1346+++ src/metadataserver/tests/test_api.py 2014-05-29 17:03:44 +0000
1347@@ -394,7 +394,7 @@
1348 node = factory.make_node()
1349 arch, subarch = node.architecture.split('/')
1350 factory.make_boot_image(
1351- osystem='ubuntu',
1352+ osystem=node.get_osystem(),
1353 architecture=arch, subarchitecture=subarch,
1354 release=node.get_distro_series(), purpose='xinstall',
1355 nodegroup=node.nodegroup)