Merge lp:~julian-edwards/maas/report-subarches-in-bootimage into lp:~maas-committers/maas/trunk
- report-subarches-in-bootimage
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Julian Edwards | ||||
Approved revision: | no longer in the source branch. | ||||
Merged at revision: | 2340 | ||||
Proposed branch: | lp:~julian-edwards/maas/report-subarches-in-bootimage | ||||
Merge into: | lp:~maas-committers/maas/trunk | ||||
Diff against target: |
424 lines (+275/-9) 5 files modified
src/provisioningserver/boot/tests/test_tftppath.py (+154/-1) src/provisioningserver/boot/tftppath.py (+71/-8) src/provisioningserver/import_images/boot_image_mapping.py (+27/-0) src/provisioningserver/import_images/testing/factory.py (+9/-0) src/provisioningserver/import_images/tests/test_boot_image_mapping.py (+14/-0) |
||||
To merge this branch: | bzr merge lp:~julian-edwards/maas/report-subarches-in-bootimage | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jeroen T. Vermeulen (community) | Approve | ||
Review via email: mp+219987@code.launchpad.net |
Commit message
Make list_boot_images(), which is called from the periodic Celery task, report the supported_subarches which exists in the maas.meta file as subarches in the resource data for each image. This will enable subarch fallback to the right image when looking for a boot image.
Description of the change
I am throwing this up before I've tested it on my rig so I can get some early feedback. There's probably a fair bit that needs tidying up but given I don't overlap with you lot at the moment, please forgive my indulgence :)
Julian Edwards (julian-edwards) wrote : | # |
On 20/05/14 02:49, Jeroen T. Vermeulen wrote:
> Review: Approve
>
> Comments inline. Several things need, or may need, addressing before this can land, including the docstrings.
Thanks for reviewing.
I would have liked to have split this into smaller branches for better
focused changed, but a) it evolved over time, and b) I'd be blocked on
reviews that I won't get until my night-time. So, sorry for the
larger-
> Don't forget to run "make format" when you're done
Done!
>> +
>> + def test_extract_
>> + resource = dict(
>> + subarches=
>> + other_item=
>
> Shouldn't that be a list of subarches, i.e. [factory.
No, there is only one "subarches" per resource, it's just a string.
>> + def _make_path(self):
>> + osystem = factory.
>
> The underscore doesn't do much for us in a test case... If I were you I'd just leave it out as unnecessary reading.
I honestly disagree here. I don't know if this is an unwritten rule or
not, but things starting with underscore tell me that they are exclusive
to this class (or at least *should be*). Without it, I would wonder if
make_path came from the test case base class, especially given its
factory-like nature.
>> + params = extract_
>> +
>> + self.assertEqual(
>> + [
>
> Is ordering explicitly guaranteed? If not, best just use assertItemsEqual.
Ah, good spot!
>> -def extract_
>> +def extract_
>> + """Examine the maas.meta file for any required metadata.
>> +
>> + :param metadata: contents of the maas.meta file
>> + :param params: A dict of path components for the image
>> + (see extract_
>
> I am confused. This talks about a dict of path components; the documentation for extract_
I was referring to the return value of extract_
confused you I've removed the reference entirely and directly explained
in the docstring what is required.
>> +def extract_
>> """Represent a list of TFTP path elements as a list of boot-image dicts.
>>
>> - The path must consist of a full [osystem, architecture, subarchitecture,
>> - release] that identify a kind of boot that we may need an image for.
>> + :param path: Must consist of a full [osystem, architecture,
>> + subarchitecture, release] that identify a kind of boot for which we
>
> No need to say "Must consist of" any more.
I'm not sure what you are driving at, but I've re-worded it anyway.
>
>> + may need an image.
>> + :param maas_meta: Contents of the maas.meta file. This may be an
>> + empty string.
>> +
>> + :return: A dict, which may also include additional items of meta-data
>> + that are not elements in the path, such...
Julian Edwards (julian-edwards) wrote : | # |
This tested out on my rig, so landing it. There is one more problem to fix in a subsequent branch: the pxeconfig has the path to the node's subarch instead of the image's.
MAAS Lander (maas-lander) wrote : | # |
The attempt to merge lp:~julian-edwards/maas/report-subarches-in-bootimage into lp:maas failed. Below is the output from the failed tests.
Ign http://
Get:1 http://
Get:2 http://
Ign http://
Ign http://
Hit http://
Get:3 http://
Hit http://
Get:4 http://
Get:5 http://
Hit http://
Get:6 http://
Hit http://
Get:7 http://
Hit http://
Hit http://
Hit http://
Get:8 http://
Hit http://
Hit http://
Hit http://
Ign http://
Ign http://
Ign http://
Ign http://
Get:9 http://
Get:10 http://
Get:11 http://
Get:12 http://
Hit http://
Hit http://
Ign http://
Ign http://
Fetched 410 kB in 0s (1,347 kB/s)
Reading package lists...
sudo DEBIAN_
--
Jeroen T. Vermeulen (jtv) wrote : | # |
Yes, I would feel better with the line breaks and a mention of NOQA, thanks.
Julian Edwards (julian-edwards) wrote : | # |
On 20/05/14 18:09, Jeroen T. Vermeulen wrote:
> Yes, I would feel better with the line breaks and a mention of NOQA, thanks.
>
Ok I'll do that in a followup branch.
Preview Diff
1 | === modified file 'src/provisioningserver/boot/tests/test_tftppath.py' | |||
2 | --- src/provisioningserver/boot/tests/test_tftppath.py 2014-05-02 15:13:14 +0000 | |||
3 | +++ src/provisioningserver/boot/tests/test_tftppath.py 2014-05-20 00:47:19 +0000 | |||
4 | @@ -25,6 +25,8 @@ | |||
5 | 25 | compose_image_path, | 25 | compose_image_path, |
6 | 26 | drill_down, | 26 | drill_down, |
7 | 27 | extend_path, | 27 | extend_path, |
8 | 28 | extract_image_params, | ||
9 | 29 | extract_metadata, | ||
10 | 28 | is_visible_subdir, | 30 | is_visible_subdir, |
11 | 29 | list_boot_images, | 31 | list_boot_images, |
12 | 30 | list_subdirs, | 32 | list_subdirs, |
13 | @@ -34,6 +36,14 @@ | |||
14 | 34 | OperatingSystem, | 36 | OperatingSystem, |
15 | 35 | OperatingSystemRegistry, | 37 | OperatingSystemRegistry, |
16 | 36 | ) | 38 | ) |
17 | 39 | from provisioningserver.import_images.boot_image_mapping import ( | ||
18 | 40 | BootImageMapping, | ||
19 | 41 | ) | ||
20 | 42 | from provisioningserver.import_images.helpers import ImageSpec | ||
21 | 43 | from provisioningserver.import_images.testing.factory import ( | ||
22 | 44 | make_image_spec, | ||
23 | 45 | set_resource, | ||
24 | 46 | ) | ||
25 | 37 | from provisioningserver.testing.boot_images import ( | 47 | from provisioningserver.testing.boot_images import ( |
26 | 38 | make_boot_image_storage_params, | 48 | make_boot_image_storage_params, |
27 | 39 | ) | 49 | ) |
28 | @@ -79,13 +89,15 @@ | |||
29 | 79 | ] | 89 | ] |
30 | 80 | 90 | ||
31 | 81 | 91 | ||
33 | 82 | def make_image(params, purpose): | 92 | def make_image(params, purpose, metadata=None): |
34 | 83 | """Describe an image as a dict similar to what `list_boot_images` returns. | 93 | """Describe an image as a dict similar to what `list_boot_images` returns. |
35 | 84 | 94 | ||
36 | 85 | The `params` are as returned from `make_boot_image_storage_params`. | 95 | The `params` are as returned from `make_boot_image_storage_params`. |
37 | 86 | """ | 96 | """ |
38 | 87 | image = params.copy() | 97 | image = params.copy() |
39 | 88 | image['purpose'] = purpose | 98 | image['purpose'] = purpose |
40 | 99 | if metadata is not None: | ||
41 | 100 | image.update(metadata) | ||
42 | 89 | return image | 101 | return image |
43 | 90 | 102 | ||
44 | 91 | 103 | ||
45 | @@ -133,6 +145,17 @@ | |||
46 | 133 | factory.make_file(image_dir, 'linux') | 145 | factory.make_file(image_dir, 'linux') |
47 | 134 | factory.make_file(image_dir, 'initrd.gz') | 146 | factory.make_file(image_dir, 'initrd.gz') |
48 | 135 | 147 | ||
49 | 148 | def make_meta_file(self, image_params, image_resource, tftproot): | ||
50 | 149 | image = ImageSpec( | ||
51 | 150 | arch=image_params["architecture"], | ||
52 | 151 | subarch=image_params["subarchitecture"], | ||
53 | 152 | release=image_params["release"], label=image_params["label"]) | ||
54 | 153 | mapping = BootImageMapping() | ||
55 | 154 | mapping.setdefault(image, image_resource) | ||
56 | 155 | maas_meta = mapping.dump_json() | ||
57 | 156 | with open(os.path.join(tftproot, "maas.meta"), "wb") as f: | ||
58 | 157 | f.write(maas_meta) | ||
59 | 158 | |||
60 | 136 | def test_compose_image_path_follows_storage_directory_layout(self): | 159 | def test_compose_image_path_follows_storage_directory_layout(self): |
61 | 137 | osystem = factory.make_name('osystem') | 160 | osystem = factory.make_name('osystem') |
62 | 138 | arch = factory.make_name('arch') | 161 | arch = factory.make_name('arch') |
63 | @@ -203,6 +226,21 @@ | |||
64 | 203 | ], | 226 | ], |
65 | 204 | list_boot_images(self.tftproot)) | 227 | list_boot_images(self.tftproot)) |
66 | 205 | 228 | ||
67 | 229 | def test_list_boot_images_merges_maas_meta_data(self): | ||
68 | 230 | params = make_boot_image_storage_params() | ||
69 | 231 | self.make_image_dir(params, self.tftproot) | ||
70 | 232 | # The required metadata is called "subarches" in maas.meta | ||
71 | 233 | metadata = dict(subarches=factory.make_name("subarches")) | ||
72 | 234 | self.make_meta_file(params, metadata, self.tftproot) | ||
73 | 235 | purposes = ['install', 'commissioning', 'xinstall'] | ||
74 | 236 | make_osystem(self, params['osystem'], purposes) | ||
75 | 237 | # The API requires "supported_subarches". | ||
76 | 238 | expected_metadata = dict(supported_subarches=metadata["subarches"]) | ||
77 | 239 | self.assertItemsEqual( | ||
78 | 240 | [make_image(params, purpose, expected_metadata) | ||
79 | 241 | for purpose in purposes], | ||
80 | 242 | list_boot_images(self.tftproot)) | ||
81 | 243 | |||
82 | 206 | def test_list_boot_images_empty_on_missing_osystems(self): | 244 | def test_list_boot_images_empty_on_missing_osystems(self): |
83 | 207 | params = [make_boot_image_storage_params() for counter in range(3)] | 245 | params = [make_boot_image_storage_params() for counter in range(3)] |
84 | 208 | for param in params: | 246 | for param in params: |
85 | @@ -296,3 +334,118 @@ | |||
86 | 296 | self.assertEqual( | 334 | self.assertEqual( |
87 | 297 | [[deep_dir, subdir]], | 335 | [[deep_dir, subdir]], |
88 | 298 | drill_down(base_dir, [[shallow_dir], [deep_dir]])) | 336 | drill_down(base_dir, [[shallow_dir], [deep_dir]])) |
89 | 337 | |||
90 | 338 | def test_extract_metadata(self): | ||
91 | 339 | resource = dict( | ||
92 | 340 | subarches=factory.make_name("subarch"), | ||
93 | 341 | other_item=factory.make_name("other"), | ||
94 | 342 | ) | ||
95 | 343 | image = make_image_spec() | ||
96 | 344 | mapping = set_resource(image_spec=image, resource=resource) | ||
97 | 345 | metadata = mapping.dump_json() | ||
98 | 346 | |||
99 | 347 | # Lack of consistency across maas in naming arch vs architecture | ||
100 | 348 | # and subarch vs subarchitecture means I can't just do a simple | ||
101 | 349 | # dict parameter expansion here. | ||
102 | 350 | params = { | ||
103 | 351 | "architecture": image.arch, | ||
104 | 352 | "subarchitecture": image.subarch, | ||
105 | 353 | "release": image.release, | ||
106 | 354 | "label": image.label, | ||
107 | 355 | } | ||
108 | 356 | extracted_data = extract_metadata(metadata, params) | ||
109 | 357 | |||
110 | 358 | # We only expect the supported_subarches key from the resource data. | ||
111 | 359 | expected = dict(supported_subarches=resource["subarches"]) | ||
112 | 360 | self.assertEqual(expected, extracted_data) | ||
113 | 361 | |||
114 | 362 | def _make_path(self): | ||
115 | 363 | osystem = factory.make_name("os") | ||
116 | 364 | arch = factory.make_name("arch") | ||
117 | 365 | subarch = factory.make_name("subarch") | ||
118 | 366 | release = factory.make_name("release") | ||
119 | 367 | label = factory.make_name("label") | ||
120 | 368 | path = (osystem, arch, subarch, release, label) | ||
121 | 369 | return path, osystem, arch, subarch, release, label | ||
122 | 370 | |||
123 | 371 | def _patch_osystem_registry(self, values): | ||
124 | 372 | get_item = self.patch(OperatingSystemRegistry, "get_item") | ||
125 | 373 | item_mock = Mock() | ||
126 | 374 | item_mock.get_boot_image_purposes.return_value = values | ||
127 | 375 | get_item.return_value = item_mock | ||
128 | 376 | |||
129 | 377 | def test_extract_image_params_with_no_metadata(self): | ||
130 | 378 | path, osystem, arch, subarch, release, label = self._make_path() | ||
131 | 379 | |||
132 | 380 | # Patch OperatingSystemRegistry to return a fixed list of | ||
133 | 381 | # values. | ||
134 | 382 | purpose1 = factory.make_name("purpose") | ||
135 | 383 | purpose2 = factory.make_name("purpose") | ||
136 | 384 | purposes = [purpose1, purpose2] | ||
137 | 385 | self._patch_osystem_registry(purposes) | ||
138 | 386 | |||
139 | 387 | params = extract_image_params(path, "") | ||
140 | 388 | |||
141 | 389 | self.assertItemsEqual( | ||
142 | 390 | [ | ||
143 | 391 | { | ||
144 | 392 | "osystem": osystem, | ||
145 | 393 | "architecture": arch, | ||
146 | 394 | "subarchitecture": subarch, | ||
147 | 395 | "release": release, | ||
148 | 396 | "label": label, | ||
149 | 397 | "purpose": purpose1, | ||
150 | 398 | }, | ||
151 | 399 | { | ||
152 | 400 | "osystem": osystem, | ||
153 | 401 | "architecture": arch, | ||
154 | 402 | "subarchitecture": subarch, | ||
155 | 403 | "release": release, | ||
156 | 404 | "label": label, | ||
157 | 405 | "purpose": purpose2, | ||
158 | 406 | }, | ||
159 | 407 | ], | ||
160 | 408 | params) | ||
161 | 409 | |||
162 | 410 | def test_extract_image_params_with_metadata(self): | ||
163 | 411 | path, osystem, arch, subarch, release, label = self._make_path() | ||
164 | 412 | |||
165 | 413 | # Patch OperatingSystemRegistry to return a fixed list of | ||
166 | 414 | # values. | ||
167 | 415 | purpose1 = factory.make_name("purpose") | ||
168 | 416 | purpose2 = factory.make_name("purpose") | ||
169 | 417 | purposes = [purpose1, purpose2] | ||
170 | 418 | self._patch_osystem_registry(purposes) | ||
171 | 419 | |||
172 | 420 | # Create some maas.meta content. | ||
173 | 421 | image = ImageSpec( | ||
174 | 422 | arch=arch, subarch=subarch, release=release, label=label) | ||
175 | 423 | image_resource = dict(subarches=factory.make_name("subarches")) | ||
176 | 424 | mapping = BootImageMapping() | ||
177 | 425 | mapping.setdefault(image, image_resource) | ||
178 | 426 | maas_meta = mapping.dump_json() | ||
179 | 427 | |||
180 | 428 | params = extract_image_params(path, maas_meta) | ||
181 | 429 | |||
182 | 430 | self.assertItemsEqual( | ||
183 | 431 | [ | ||
184 | 432 | { | ||
185 | 433 | "osystem": osystem, | ||
186 | 434 | "architecture": arch, | ||
187 | 435 | "subarchitecture": subarch, | ||
188 | 436 | "release": release, | ||
189 | 437 | "label": label, | ||
190 | 438 | "purpose": purpose1, | ||
191 | 439 | "supported_subarches": image_resource["subarches"], | ||
192 | 440 | }, | ||
193 | 441 | { | ||
194 | 442 | "osystem": osystem, | ||
195 | 443 | "architecture": arch, | ||
196 | 444 | "subarchitecture": subarch, | ||
197 | 445 | "release": release, | ||
198 | 446 | "label": label, | ||
199 | 447 | "purpose": purpose2, | ||
200 | 448 | "supported_subarches": image_resource["subarches"], | ||
201 | 449 | }, | ||
202 | 450 | ], | ||
203 | 451 | params) | ||
204 | 299 | 452 | ||
205 | === modified file 'src/provisioningserver/boot/tftppath.py' | |||
206 | --- src/provisioningserver/boot/tftppath.py 2014-05-19 13:16:53 +0000 | |||
207 | +++ src/provisioningserver/boot/tftppath.py 2014-05-20 00:47:19 +0000 | |||
208 | @@ -25,6 +25,10 @@ | |||
209 | 25 | import os.path | 25 | import os.path |
210 | 26 | 26 | ||
211 | 27 | from provisioningserver.driver import OperatingSystemRegistry | 27 | from provisioningserver.driver import OperatingSystemRegistry |
212 | 28 | from provisioningserver.import_images.boot_image_mapping import ( | ||
213 | 29 | BootImageMapping, | ||
214 | 30 | ) | ||
215 | 31 | from provisioningserver.import_images.helpers import ImageSpec | ||
216 | 28 | 32 | ||
217 | 29 | 33 | ||
218 | 30 | logger = getLogger(__name__) | 34 | logger = getLogger(__name__) |
219 | @@ -112,11 +116,43 @@ | |||
220 | 112 | extend_path(directory, path) for path in paths)) | 116 | extend_path(directory, path) for path in paths)) |
221 | 113 | 117 | ||
222 | 114 | 118 | ||
224 | 115 | def extract_image_params(path): | 119 | def extract_metadata(metadata, params): |
225 | 120 | """Examine the maas.meta file for any required metadata. | ||
226 | 121 | |||
227 | 122 | :param metadata: contents of the maas.meta file | ||
228 | 123 | :param params: A dict of path components for the image | ||
229 | 124 | (architecture, subarchitecture, release and label). | ||
230 | 125 | :return: a dict of name/value metadata pairs. Currently, only | ||
231 | 126 | "subarches" is extracted. | ||
232 | 127 | """ | ||
233 | 128 | mapping = BootImageMapping.load_json(metadata) | ||
234 | 129 | |||
235 | 130 | image = ImageSpec( | ||
236 | 131 | arch=params["architecture"], | ||
237 | 132 | subarch=params["subarchitecture"], | ||
238 | 133 | release=params["release"], | ||
239 | 134 | label=params["label"], | ||
240 | 135 | ) | ||
241 | 136 | try: | ||
242 | 137 | resource = mapping.mapping[image] | ||
243 | 138 | except KeyError: | ||
244 | 139 | return {} | ||
245 | 140 | |||
246 | 141 | return dict(supported_subarches=resource["subarches"]) | ||
247 | 142 | |||
248 | 143 | |||
249 | 144 | def extract_image_params(path, maas_meta): | ||
250 | 116 | """Represent a list of TFTP path elements as a list of boot-image dicts. | 145 | """Represent a list of TFTP path elements as a list of boot-image dicts. |
251 | 117 | 146 | ||
254 | 118 | The path must consist of a full [osystem, architecture, subarchitecture, | 147 | :param path: Tuple or list that consists of a full [osystem, architecture, |
255 | 119 | release] that identify a kind of boot that we may need an image for. | 148 | subarchitecture, release] that identify a kind of boot for which we |
256 | 149 | may need an image. | ||
257 | 150 | :param maas_meta: Contents of the maas.meta file. This may be an | ||
258 | 151 | empty string. | ||
259 | 152 | |||
260 | 153 | :return: A list of dicts, each of which may also include additional | ||
261 | 154 | items of meta-data that are not elements in the path, such as | ||
262 | 155 | "subarches". | ||
263 | 120 | """ | 156 | """ |
264 | 121 | osystem, arch, subarch, release, label = path | 157 | osystem, arch, subarch, release, label = path |
265 | 122 | osystem_obj = OperatingSystemRegistry.get_item(osystem, default=None) | 158 | osystem_obj = OperatingSystemRegistry.get_item(osystem, default=None) |
266 | @@ -125,13 +161,27 @@ | |||
267 | 125 | 161 | ||
268 | 126 | purposes = osystem_obj.get_boot_image_purposes( | 162 | purposes = osystem_obj.get_boot_image_purposes( |
269 | 127 | arch, subarch, release, label) | 163 | arch, subarch, release, label) |
271 | 128 | return [ | 164 | |
272 | 165 | # Expand the path into a list of dicts, one for each boot purpose. | ||
273 | 166 | params = [ | ||
274 | 129 | dict( | 167 | dict( |
275 | 130 | osystem=osystem, architecture=arch, subarchitecture=subarch, | 168 | osystem=osystem, architecture=arch, subarchitecture=subarch, |
276 | 131 | release=release, label=label, purpose=purpose) | 169 | release=release, label=label, purpose=purpose) |
277 | 132 | for purpose in purposes | 170 | for purpose in purposes |
278 | 133 | ] | 171 | ] |
279 | 134 | 172 | ||
280 | 173 | # Merge in the meta-data. | ||
281 | 174 | for image_dict in params: | ||
282 | 175 | metadata = extract_metadata(maas_meta, image_dict) | ||
283 | 176 | image_dict.update(metadata) | ||
284 | 177 | |||
285 | 178 | return params | ||
286 | 179 | |||
287 | 180 | |||
288 | 181 | def maas_meta_file_path(tftproot): | ||
289 | 182 | """Return a string containing the full path to maas.meta.""" | ||
290 | 183 | return os.path.join(tftproot, 'maas.meta') | ||
291 | 184 | |||
292 | 135 | 185 | ||
293 | 136 | def list_boot_images(tftproot): | 186 | def list_boot_images(tftproot): |
294 | 137 | """List the available boot images. | 187 | """List the available boot images. |
295 | @@ -149,9 +199,9 @@ | |||
296 | 149 | # Directory does not exist, so return empty list. | 199 | # Directory does not exist, so return empty list. |
297 | 150 | logger.warning("No boot images have been imported yet.") | 200 | logger.warning("No boot images have been imported yet.") |
298 | 151 | return [] | 201 | return [] |
302 | 152 | else: | 202 | |
303 | 153 | # Other error. Propagate. | 203 | # Other error. Propagate. |
304 | 154 | raise | 204 | raise |
305 | 155 | 205 | ||
306 | 156 | # Starting point for iteration: paths that contain only the | 206 | # Starting point for iteration: paths that contain only the |
307 | 157 | # top-level subdirectory of tftproot, i.e. the architecture name. | 207 | # top-level subdirectory of tftproot, i.e. the architecture name. |
308 | @@ -163,8 +213,21 @@ | |||
309 | 163 | for level in ['arch', 'subarch', 'release', 'label']: | 213 | for level in ['arch', 'subarch', 'release', 'label']: |
310 | 164 | paths = drill_down(tftproot, paths) | 214 | paths = drill_down(tftproot, paths) |
311 | 165 | 215 | ||
312 | 216 | # Get hold of image meta-data stored in the maas.meta file. | ||
313 | 217 | meta_file_path = maas_meta_file_path(tftproot) | ||
314 | 218 | try: | ||
315 | 219 | with open(meta_file_path, "rb") as f: | ||
316 | 220 | metadata = f.read() | ||
317 | 221 | except IOError as e: | ||
318 | 222 | if e.errno != errno.ENOENT: | ||
319 | 223 | # Unexpected error, propagate. | ||
320 | 224 | raise | ||
321 | 225 | # No meta file (yet), it means no import has run so just skip | ||
322 | 226 | # it. | ||
323 | 227 | metadata = "" | ||
324 | 228 | |||
325 | 166 | # Each path we find this way should be a boot image. | 229 | # Each path we find this way should be a boot image. |
326 | 167 | # This gets serialised to JSON, so we really have to return a list, not | 230 | # This gets serialised to JSON, so we really have to return a list, not |
327 | 168 | # just any iterable. | 231 | # just any iterable. |
328 | 169 | return list(chain.from_iterable( | 232 | return list(chain.from_iterable( |
330 | 170 | extract_image_params(path) for path in paths)) | 233 | extract_image_params(path, metadata) for path in paths)) |
331 | 171 | 234 | ||
332 | === modified file 'src/provisioningserver/import_images/boot_image_mapping.py' | |||
333 | --- src/provisioningserver/import_images/boot_image_mapping.py 2014-04-11 02:14:42 +0000 | |||
334 | +++ src/provisioningserver/import_images/boot_image_mapping.py 2014-05-20 00:47:19 +0000 | |||
335 | @@ -63,3 +63,30 @@ | |||
336 | 63 | data[arch][subarch].setdefault(release, {}) | 63 | data[arch][subarch].setdefault(release, {}) |
337 | 64 | data[arch][subarch][release][label] = resource | 64 | data[arch][subarch][release][label] = resource |
338 | 65 | return json.dumps(data, sort_keys=True) | 65 | return json.dumps(data, sort_keys=True) |
339 | 66 | |||
340 | 67 | @staticmethod | ||
341 | 68 | def load_json(json_data): | ||
342 | 69 | """Take a JSON representation and deserialize into an object. | ||
343 | 70 | |||
344 | 71 | :param json_data: string produced by dump_json(), above. | ||
345 | 72 | :return: A BootImageMapping | ||
346 | 73 | |||
347 | 74 | If the json data is invalid, an empty BootImageMapping is returned. | ||
348 | 75 | """ | ||
349 | 76 | mapping = BootImageMapping() | ||
350 | 77 | try: | ||
351 | 78 | data = json.loads(json_data) | ||
352 | 79 | except ValueError: | ||
353 | 80 | return mapping | ||
354 | 81 | |||
355 | 82 | for arch in data: | ||
356 | 83 | for subarch in data[arch]: | ||
357 | 84 | for release in data[arch][subarch]: | ||
358 | 85 | for label in data[arch][subarch][release]: | ||
359 | 86 | image = ImageSpec( | ||
360 | 87 | arch=arch, subarch=subarch, release=release, | ||
361 | 88 | label=label) | ||
362 | 89 | resource = data[arch][subarch][release][label] | ||
363 | 90 | mapping.setdefault(image, resource) | ||
364 | 91 | |||
365 | 92 | return mapping | ||
366 | 66 | 93 | ||
367 | === modified file 'src/provisioningserver/import_images/testing/factory.py' | |||
368 | --- src/provisioningserver/import_images/testing/factory.py 2014-04-08 06:53:34 +0000 | |||
369 | +++ src/provisioningserver/import_images/testing/factory.py 2014-05-20 00:47:19 +0000 | |||
370 | @@ -15,9 +15,12 @@ | |||
371 | 15 | __all__ = [ | 15 | __all__ = [ |
372 | 16 | 'make_boot_resource', | 16 | 'make_boot_resource', |
373 | 17 | 'make_image_spec', | 17 | 'make_image_spec', |
374 | 18 | 'make_maas_meta', | ||
375 | 18 | 'set_resource', | 19 | 'set_resource', |
376 | 19 | ] | 20 | ] |
377 | 20 | 21 | ||
378 | 22 | from textwrap import dedent | ||
379 | 23 | |||
380 | 21 | from maastesting.factory import factory | 24 | from maastesting.factory import factory |
381 | 22 | from provisioningserver.import_images.boot_image_mapping import ( | 25 | from provisioningserver.import_images.boot_image_mapping import ( |
382 | 23 | BootImageMapping, | 26 | BootImageMapping, |
383 | @@ -25,6 +28,12 @@ | |||
384 | 25 | from provisioningserver.import_images.helpers import ImageSpec | 28 | from provisioningserver.import_images.helpers import ImageSpec |
385 | 26 | 29 | ||
386 | 27 | 30 | ||
387 | 31 | def make_maas_meta(): | ||
388 | 32 | """Return fake maas.meta data.""" | ||
389 | 33 | return dedent("""\ | ||
390 | 34 | {"amd64": {"generic": {"precise": {"release": {"content_id": "com.ubuntu.maas:v2:download", "path": "precise/amd64/20140410/raring/generic/boot-kernel", "product_name": "com.ubuntu.maas:v2:boot:12.04:amd64:hwe-r", "subarches": "generic,hwe-p,hwe-q,hwe-r", "version_name": "20140410"}}, "trusty": {"release": {"content_id": "com.ubuntu.maas:v2:download", "path": "trusty/amd64/20140416.1/root-image.gz", "product_name": "com.ubuntu.maas:v2:boot:14.04:amd64:hwe-t", "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t", "version_name": "20140416.1"}}}, "hwe-s": {"precise": {"release": {"content_id": "com.ubuntu.maas:v2:download", "path": "precise/amd64/20140410/saucy/generic/boot-kernel", "product_name": "com.ubuntu.maas:v2:boot:12.04:amd64:hwe-s", "subarches": "generic,hwe-p,hwe-q,hwe-r,hwe-s", "version_name": "20140410"}}}}}""") # NOQA | ||
391 | 35 | |||
392 | 36 | |||
393 | 28 | def make_boot_resource(): | 37 | def make_boot_resource(): |
394 | 29 | """Create a fake resource dict.""" | 38 | """Create a fake resource dict.""" |
395 | 30 | return { | 39 | return { |
396 | 31 | 40 | ||
397 | === modified file 'src/provisioningserver/import_images/tests/test_boot_image_mapping.py' | |||
398 | --- src/provisioningserver/import_images/tests/test_boot_image_mapping.py 2014-04-11 02:13:59 +0000 | |||
399 | +++ src/provisioningserver/import_images/tests/test_boot_image_mapping.py 2014-05-20 00:47:19 +0000 | |||
400 | @@ -23,6 +23,7 @@ | |||
401 | 23 | ) | 23 | ) |
402 | 24 | from provisioningserver.import_images.testing.factory import ( | 24 | from provisioningserver.import_images.testing.factory import ( |
403 | 25 | make_image_spec, | 25 | make_image_spec, |
404 | 26 | make_maas_meta, | ||
405 | 26 | set_resource, | 27 | set_resource, |
406 | 27 | ) | 28 | ) |
407 | 28 | 29 | ||
408 | @@ -104,3 +105,16 @@ | |||
409 | 104 | }, | 105 | }, |
410 | 105 | }, | 106 | }, |
411 | 106 | json.loads(image_dict.dump_json())) | 107 | json.loads(image_dict.dump_json())) |
412 | 108 | |||
413 | 109 | def test_load_json_result_matches_dump_of_own_data(self): | ||
414 | 110 | # Loading the test data and dumping it again should result in | ||
415 | 111 | # identical test data. | ||
416 | 112 | test_meta_file_content = make_maas_meta() | ||
417 | 113 | mapping = BootImageMapping.load_json(test_meta_file_content) | ||
418 | 114 | dumped = mapping.dump_json() | ||
419 | 115 | self.assertEqual(test_meta_file_content, dumped) | ||
420 | 116 | |||
421 | 117 | def test_load_json_returns_empty_mapping_for_invalid_json(self): | ||
422 | 118 | bad_json = "" | ||
423 | 119 | mapping = BootImageMapping.load_json(bad_json) | ||
424 | 120 | self.assertEqual({}, mapping.mapping) |
Comments inline. Several things need, or may need, addressing before this can land, including the docstrings.