maas-images

Merge lp:~smoser/maas-images/trunk.lp1665376 into lp:maas-images

  1. trunk.lp1665376
  2. Merge into maas-ephemerals
Proposed by Scott Moser
Status: Merged
Merged at revision: 355
Proposed branch: lp:~smoser/maas-images/trunk.lp1665376
Merge into: lp:maas-images
Diff against target: 352 lines (+158/-68)
2 files modified
meph2/netinst.py (+6/-5)
tests/unittests/file_item_data.json (+152/-63)
To merge this branch: bzr merge lp:~smoser/maas-images/trunk.lp1665376
Reviewer Review Type Date Requested Status
Dan Watkins (community) Approve
maintainers of maas images Pending
Review via email: mp+317499@code.launchpad.net

Commit message

netinst: support d-i hwe kernels from rolling hwe

For LTS releases prior to xenial, the hardware enablement kernel was
specifically tied to a subsequent release. Thus, for trusty we had
d-i paths like:

 device-tree/apm-merlin.dtb
 wily-device-tree/apm-merlin.dtb
 xenial-device-tree/apm-merlin.dtb

 netboot/ubuntu-installer/i386/linux
 wily-netboot/ubuntu-installer/i386/linux
 xenial-netboot/ubuntu-installer/i386/linux

 generic-lpae/netboot/initrd.gz
 wily-generic-lpae/netboot/initrd.gz
 xenial-generic-lpae/netboot/initrd.gz

With xenial, the hwe kernel is not specifically tied to a release, but
rather "rolls". So we have new paths that look like this:

 hwe-device-tree/apm-merlin.dtb
 hwe-netboot/ubuntu-installer/i386/linux
 hwe-generic-lpae/netboot/initrd.gz

For those the 'kernel release' data is set to simply "hwe".

The unit test data tests/unittests/file_item_data.json is updated here
and sorted with the following:
  $ cat sort-json.py
  import json, sys
  with open(sys.argv[1], "rb") as fp:
      data = json.loads(fp.read().encode())
  with open(sys.argv[1], "wb") as fp:
      fp.write(json.dumps(data, indent=1, sort_keys=True,
                        separators=(",", ": ")).encode("utf-8") + "\n")
  $ sort-json.py tests/unittests/file_item_data.json

To post a comment you must log in.
lp:~smoser/maas-images/trunk.lp1665376 updated
355. By Scott Moser

netinst: support d-i hwe kernels from rolling hwe

For LTS releases prior to xenial, the hardware enablement kernel was
specifically tied to a subsequent release. Thus, for trusty we had
d-i paths like:

 device-tree/apm-merlin.dtb
 wily-device-tree/apm-merlin.dtb
 xenial-device-tree/apm-merlin.dtb

 netboot/ubuntu-installer/i386/linux
 wily-netboot/ubuntu-installer/i386/linux
 xenial-netboot/ubuntu-installer/i386/linux

With xenial, the hwe kernel is not specifically tied to a release, but
rather "rolls". So we have new paths that look like this:

 hwe-device-tree/apm-merlin.dtb
 hwe-netboot/ubuntu-installer/i386/linux

For those the 'kernel release' data is set to simply "hwe".

The unit test data tests/unittests/file_item_data.json is updated here
and sorted with the following:
  $ cat sort-json.py
  import json, sys
  with open(sys.argv[1], "rb") as fp:
      data = json.loads(fp.read().encode())
  with open(sys.argv[1], "wb") as fp:
      fp.write(json.dumps(data, indent=1, sort_keys=True,
                        separators=(",", ": ")).encode("utf-8") + "\n")
  $ sort-json.py tests/unittests/file_item_data.json

356. By Scott Moser

fix for paths like: hwe-generic-lpae/netboot/initrd.gz

Revision history for this message
Scott Moser (smoser) wrote :

This passes both a dry run of all supported releases:
 ./bin/netboot-mirror --dry-run -vvv out.d

and actual download of all xenial:

 ./bin/netboot-mirror -vvv --releases=xenial out.d

Revision history for this message
Dan Watkins (oddbloke) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'meph2/netinst.py'
--- meph2/netinst.py 2016-12-16 23:52:01 +0000
+++ meph2/netinst.py 2017-02-16 15:48:22 +0000
@@ -297,8 +297,9 @@
297 # /installer-ppc64el/current/images/MD5SUMS297 # /installer-ppc64el/current/images/MD5SUMS
298 # return either None (not a kernel/initrd/dtb)298 # return either None (not a kernel/initrd/dtb)
299 # or a dictionary of:299 # or a dictionary of:
300 # release-kernel, kernel-flavor, initrd-flavor, filetype, image_format300 # kernel-release, kernel-flavor, initrd-flavor, filetype, image_format
301 # release-kernel is 'wily' if trusty-hwe-w301 # kernel-release is 'wily' if trusty-hwe-w, and
302 # simply 'hwe' for xenial's hardware enablement "rolling" hwe kernel.
302 # filetype is 'dtb', 'initrd.' or 'kernel'303 # filetype is 'dtb', 'initrd.' or 'kernel'
303 # at the moment the only 'initrd-flavor' that we're supporting is netboot304 # at the moment the only 'initrd-flavor' that we're supporting is netboot
304 ftype = None305 ftype = None
@@ -332,9 +333,9 @@
332 return None333 return None
333334
334 # kernel release. all kernel release paths start with <release>-335 # kernel release. all kernel release paths start with <release>-
335 releases = REL2VER.keys()336 kernel_releases = [k for k in REL2VER.keys()] + ["hwe"]
336 kernel_release = release337 kernel_release = release
337 for rel in REL2VER.keys():338 for rel in kernel_releases:
338 if path.startswith(rel + "-"):339 if path.startswith(rel + "-"):
339 kernel_release = rel340 kernel_release = rel
340 break341 break
@@ -362,7 +363,7 @@
362 kernel_flavor = kflav363 kernel_flavor = kflav
363 break364 break
364 else:365 else:
365 for r in releases:366 for r in kernel_releases:
366 if "%s-%s" % (r, kflav) in ptoks:367 if "%s-%s" % (r, kflav) in ptoks:
367 kernel_flavor = kflav368 kernel_flavor = kflav
368 break369 break
369370
=== modified file 'tests/unittests/file_item_data.json'
--- tests/unittests/file_item_data.json 2016-08-01 17:17:09 +0000
+++ tests/unittests/file_item_data.json 2017-02-16 15:48:22 +0000
@@ -133,6 +133,7 @@
133 "kernel-flavor": "generic",133 "kernel-flavor": "generic",
134 "kernel-release": "BASE"134 "kernel-release": "BASE"
135 },135 },
136 "e500/cdrom/vmlinux": null,
136 "e500/netboot/initrd.gz": {137 "e500/netboot/initrd.gz": {
137 "ftype": "initrd",138 "ftype": "initrd",
138 "image-format": "default",139 "image-format": "default",
@@ -161,7 +162,6 @@
161 "kernel-flavor": "e500mc",162 "kernel-flavor": "e500mc",
162 "kernel-release": "BASE"163 "kernel-release": "BASE"
163 },164 },
164 "e500/cdrom/vmlinux": null,
165 "generic-lpae/netboot/initrd.gz": {165 "generic-lpae/netboot/initrd.gz": {
166 "ftype": "initrd",166 "ftype": "initrd",
167 "image-format": "default",167 "image-format": "default",
@@ -246,11 +246,11 @@
246 "generic/netboot/tegra/boot.scr-tegra-dhcp-serial": null,246 "generic/netboot/tegra/boot.scr-tegra-dhcp-serial": null,
247 "generic/netboot/tegra/boot.scr-tegra-disk-serial": null,247 "generic/netboot/tegra/boot.scr-tegra-disk-serial": null,
248 "generic/netboot/tegra/initrd.gz": {248 "generic/netboot/tegra/initrd.gz": {
249 "ftype": "initrd",249 "ftype": "initrd",
250 "image-format": "tegra",250 "image-format": "tegra",
251 "initrd-flavor": "netboot",251 "initrd-flavor": "netboot",
252 "kernel-flavor": "generic",252 "kernel-flavor": "generic",
253 "kernel-release": "BASE"253 "kernel-release": "BASE"
254 },254 },
255 "generic/netboot/tegra/tegra114-dalmore.dtb": {255 "generic/netboot/tegra/tegra114-dalmore.dtb": {
256 "ftype": "dtb",256 "ftype": "dtb",
@@ -407,11 +407,11 @@
407 "kernel-release": "BASE"407 "kernel-release": "BASE"
408 },408 },
409 "generic/netboot/tegra/vmlinuz": {409 "generic/netboot/tegra/vmlinuz": {
410 "ftype": "kernel",410 "ftype": "kernel",
411 "image-format": "tegra",411 "image-format": "tegra",
412 "initrd-flavor": "netboot",412 "initrd-flavor": "netboot",
413 "kernel-flavor": "generic",413 "kernel-flavor": "generic",
414 "kernel-release": "BASE"414 "kernel-release": "BASE"
415 },415 },
416 "generic/netboot/vmlinuz": {416 "generic/netboot/vmlinuz": {
417 "ftype": "kernel",417 "ftype": "kernel",
@@ -490,6 +490,95 @@
490 "kernel-flavor": "highbank",490 "kernel-flavor": "highbank",
491 "kernel-release": "BASE"491 "kernel-release": "BASE"
492 },492 },
493 "hwe-device-tree/README": null,
494 "hwe-device-tree/apm-merlin.dtb": {
495 "ftype": "dtb",
496 "image-format": "default",
497 "initrd-flavor": "netboot",
498 "kernel-flavor": "generic",
499 "kernel-release": "hwe"
500 },
501 "hwe-device-tree/apm-mustang.dtb": {
502 "ftype": "dtb",
503 "image-format": "xgene",
504 "initrd-flavor": "netboot",
505 "kernel-flavor": "generic",
506 "kernel-release": "hwe"
507 },
508 "hwe-generic-lpae/netboot/initrd.gz": {
509 "ftype": "initrd",
510 "image-format": "default",
511 "initrd-flavor": "netboot",
512 "kernel-flavor": "generic-lpae",
513 "kernel-release": "hwe"
514 },
515 "hwe-generic-lpae/netboot/vmlinuz": {
516 "ftype": "kernel",
517 "image-format": "default",
518 "initrd-flavor": "netboot",
519 "kernel-flavor": "generic-lpae",
520 "kernel-release": "hwe"
521 },
522 "hwe-netboot/ubuntu-installer/amd64/initrd.gz": {
523 "ftype": "initrd",
524 "image-format": "default",
525 "initrd-flavor": "netboot",
526 "kernel-flavor": "generic",
527 "kernel-release": "hwe"
528 },
529 "hwe-netboot/ubuntu-installer/amd64/linux": {
530 "ftype": "kernel",
531 "image-format": "default",
532 "initrd-flavor": "netboot",
533 "kernel-flavor": "generic",
534 "kernel-release": "hwe"
535 },
536 "hwe-netboot/ubuntu-installer/amd64/pxelinux.0": null,
537 "hwe-netboot/ubuntu-installer/arm64/bootnetaa64.efi": null,
538 "hwe-netboot/ubuntu-installer/arm64/initrd.gz": {
539 "ftype": "initrd",
540 "image-format": "default",
541 "initrd-flavor": "netboot",
542 "kernel-flavor": "generic",
543 "kernel-release": "hwe"
544 },
545 "hwe-netboot/ubuntu-installer/arm64/linux": {
546 "ftype": "kernel",
547 "image-format": "default",
548 "initrd-flavor": "netboot",
549 "kernel-flavor": "generic",
550 "kernel-release": "hwe"
551 },
552 "hwe-netboot/ubuntu-installer/i386/initrd.gz": {
553 "ftype": "initrd",
554 "image-format": "default",
555 "initrd-flavor": "netboot",
556 "kernel-flavor": "generic",
557 "kernel-release": "hwe"
558 },
559 "hwe-netboot/ubuntu-installer/i386/linux": {
560 "ftype": "kernel",
561 "image-format": "default",
562 "initrd-flavor": "netboot",
563 "kernel-flavor": "generic",
564 "kernel-release": "hwe"
565 },
566 "hwe-netboot/ubuntu-installer/i386/pxelinux.0": null,
567 "hwe-netboot/ubuntu-installer/ppc64el/fonts/unicode.pf2": null,
568 "hwe-netboot/ubuntu-installer/ppc64el/initrd.gz": {
569 "ftype": "initrd",
570 "image-format": "default",
571 "initrd-flavor": "netboot",
572 "kernel-flavor": "generic",
573 "kernel-release": "hwe"
574 },
575 "hwe-netboot/ubuntu-installer/ppc64el/vmlinux": {
576 "ftype": "kernel",
577 "image-format": "default",
578 "initrd-flavor": "netboot",
579 "kernel-flavor": "generic",
580 "kernel-release": "hwe"
581 },
493 "keystone/netboot/initrd.gz": {582 "keystone/netboot/initrd.gz": {
494 "ftype": "initrd",583 "ftype": "initrd",
495 "image-format": "default",584 "image-format": "default",
@@ -966,11 +1055,11 @@
966 "kernel-release": "utopic"1055 "kernel-release": "utopic"
967 },1056 },
968 "utopic-generic/netboot/tegra/vmlinuz": {1057 "utopic-generic/netboot/tegra/vmlinuz": {
969 "ftype": "kernel",1058 "ftype": "kernel",
970 "image-format": "tegra",1059 "image-format": "tegra",
971 "initrd-flavor": "netboot",1060 "initrd-flavor": "netboot",
972 "kernel-flavor": "generic",1061 "kernel-flavor": "generic",
973 "kernel-release": "utopic"1062 "kernel-release": "utopic"
974 },1063 },
975 "utopic-generic/netboot/vmlinuz": {1064 "utopic-generic/netboot/vmlinuz": {
976 "ftype": "kernel",1065 "ftype": "kernel",
@@ -1004,18 +1093,18 @@
1004 "kernel-release": "utopic"1093 "kernel-release": "utopic"
1005 },1094 },
1006 "utopic-generic/netboot/xgene/uImage": {1095 "utopic-generic/netboot/xgene/uImage": {
1007 "ftype": "kernel",1096 "ftype": "kernel",
1008 "image-format": "xgene",1097 "image-format": "xgene",
1009 "initrd-flavor": "netboot",1098 "initrd-flavor": "netboot",
1010 "kernel-flavor": "generic",1099 "kernel-flavor": "generic",
1011 "kernel-release": "utopic"1100 "kernel-release": "utopic"
1012 },1101 },
1013 "utopic-generic/netboot/xgene/uInitrd": {1102 "utopic-generic/netboot/xgene/uInitrd": {
1014 "ftype": "initrd",1103 "ftype": "initrd",
1015 "image-format": "xgene",1104 "image-format": "xgene",
1016 "initrd-flavor": "netboot",1105 "initrd-flavor": "netboot",
1017 "kernel-flavor": "generic",1106 "kernel-flavor": "generic",
1018 "kernel-release": "utopic"1107 "kernel-release": "utopic"
1019 },1108 },
1020 "utopic-netboot/boot.img.gz": null,1109 "utopic-netboot/boot.img.gz": null,
1021 "utopic-netboot/mini.iso": null,1110 "utopic-netboot/mini.iso": null,
@@ -1164,11 +1253,11 @@
1164 "kernel-release": "vivid"1253 "kernel-release": "vivid"
1165 },1254 },
1166 "vivid-generic/netboot/tegra/vmlinuz": {1255 "vivid-generic/netboot/tegra/vmlinuz": {
1167 "ftype": "kernel",1256 "ftype": "kernel",
1168 "image-format": "tegra",1257 "image-format": "tegra",
1169 "initrd-flavor": "netboot",1258 "initrd-flavor": "netboot",
1170 "kernel-flavor": "generic",1259 "kernel-flavor": "generic",
1171 "kernel-release": "vivid"1260 "kernel-release": "vivid"
1172 },1261 },
1173 "vivid-generic/netboot/vmlinuz": {1262 "vivid-generic/netboot/vmlinuz": {
1174 "ftype": "kernel",1263 "ftype": "kernel",
@@ -1202,18 +1291,18 @@
1202 "kernel-release": "vivid"1291 "kernel-release": "vivid"
1203 },1292 },
1204 "vivid-generic/netboot/xgene/uImage": {1293 "vivid-generic/netboot/xgene/uImage": {
1205 "ftype": "kernel",1294 "ftype": "kernel",
1206 "image-format": "xgene",1295 "image-format": "xgene",
1207 "initrd-flavor": "netboot",1296 "initrd-flavor": "netboot",
1208 "kernel-flavor": "generic",1297 "kernel-flavor": "generic",
1209 "kernel-release": "vivid"1298 "kernel-release": "vivid"
1210 },1299 },
1211 "vivid-generic/netboot/xgene/uInitrd": {1300 "vivid-generic/netboot/xgene/uInitrd": {
1212 "ftype": "initrd",1301 "ftype": "initrd",
1213 "image-format": "xgene",1302 "image-format": "xgene",
1214 "initrd-flavor": "netboot",1303 "initrd-flavor": "netboot",
1215 "kernel-flavor": "generic",1304 "kernel-flavor": "generic",
1216 "kernel-release": "vivid"1305 "kernel-release": "vivid"
1217 },1306 },
1218 "vivid-netboot/boot.img.gz": null,1307 "vivid-netboot/boot.img.gz": null,
1219 "vivid-netboot/mini.iso": null,1308 "vivid-netboot/mini.iso": null,
@@ -1327,11 +1416,11 @@
1327 "kernel-release": "wily"1416 "kernel-release": "wily"
1328 },1417 },
1329 "wily-generic/netboot/tegra/vmlinuz": {1418 "wily-generic/netboot/tegra/vmlinuz": {
1330 "ftype": "kernel",1419 "ftype": "kernel",
1331 "image-format": "tegra",1420 "image-format": "tegra",
1332 "initrd-flavor": "netboot",1421 "initrd-flavor": "netboot",
1333 "kernel-flavor": "generic",1422 "kernel-flavor": "generic",
1334 "kernel-release": "wily"1423 "kernel-release": "wily"
1335 },1424 },
1336 "wily-generic/netboot/vmlinuz": {1425 "wily-generic/netboot/vmlinuz": {
1337 "ftype": "kernel",1426 "ftype": "kernel",
@@ -1365,18 +1454,18 @@
1365 "kernel-release": "wily"1454 "kernel-release": "wily"
1366 },1455 },
1367 "wily-generic/netboot/xgene/uImage": {1456 "wily-generic/netboot/xgene/uImage": {
1368 "ftype": "kernel",1457 "ftype": "kernel",
1369 "image-format": "xgene",1458 "image-format": "xgene",
1370 "initrd-flavor": "netboot",1459 "initrd-flavor": "netboot",
1371 "kernel-flavor": "generic",1460 "kernel-flavor": "generic",
1372 "kernel-release": "wily"1461 "kernel-release": "wily"
1373 },1462 },
1374 "wily-generic/netboot/xgene/uInitrd": {1463 "wily-generic/netboot/xgene/uInitrd": {
1375 "ftype": "initrd",1464 "ftype": "initrd",
1376 "image-format": "xgene",1465 "image-format": "xgene",
1377 "initrd-flavor": "netboot",1466 "initrd-flavor": "netboot",
1378 "kernel-flavor": "generic",1467 "kernel-flavor": "generic",
1379 "kernel-release": "wily"1468 "kernel-release": "wily"
1380 },1469 },
1381 "wily-netboot/boot.img.gz": null,1470 "wily-netboot/boot.img.gz": null,
1382 "wily-netboot/mini.iso": null,1471 "wily-netboot/mini.iso": null,
@@ -1451,13 +1540,6 @@
1451 "kernel-flavor": "generic",1540 "kernel-flavor": "generic",
1452 "kernel-release": "xenial"1541 "kernel-release": "xenial"
1453 },1542 },
1454 "xenial-generic/netboot/vmlinuz": {
1455 "ftype": "kernel",
1456 "image-format": "default",
1457 "initrd-flavor": "netboot",
1458 "kernel-flavor": "generic",
1459 "kernel-release": "xenial"
1460 },
1461 "xenial-generic/netboot/ubuntu-installer/arm64/initrd.gz": {1543 "xenial-generic/netboot/ubuntu-installer/arm64/initrd.gz": {
1462 "ftype": "initrd",1544 "ftype": "initrd",
1463 "image-format": "default",1545 "image-format": "default",
@@ -1472,6 +1554,13 @@
1472 "kernel-flavor": "generic",1554 "kernel-flavor": "generic",
1473 "kernel-release": "xenial"1555 "kernel-release": "xenial"
1474 },1556 },
1557 "xenial-generic/netboot/vmlinuz": {
1558 "ftype": "kernel",
1559 "image-format": "default",
1560 "initrd-flavor": "netboot",
1561 "kernel-flavor": "generic",
1562 "kernel-release": "xenial"
1563 },
1475 "xenial-netboot/ubuntu-installer/amd64/pxelinux.0": null,1564 "xenial-netboot/ubuntu-installer/amd64/pxelinux.0": null,
1476 "xenial-powerpc64/netboot/initrd.gz": {1565 "xenial-powerpc64/netboot/initrd.gz": {
1477 "ftype": "initrd",1566 "ftype": "initrd",

Subscribers

People subscribed via source and target branches