Merge ~blake-rouse/maas:grub-http into maas:master

[Blake Rouse]

Load the kernel and initrd using HTTP when booting with UEFI.

This change needs to wait for the SRU of https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1787630 to bionic. Once the SRU is complete and new maas-images are released then this branch can land.

[Andres Rodriguez (andreserl)]

lgtm! Just one comment inline.

review: Approve

[MAAS Lander (maas-lander)]

UNIT TESTS
-b grub-http lp:~blake-rouse/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: 2fd6c05d9c4e6a1ab9b36ef382f25aa07f78c98f

review: Approve

[Blake Rouse (blake-rouse)]

This passed successfully in the CI using grub from disco.

http://10.245.136.4:8080/job/proposed-maas-images-manual/32/console

ad3a00d... by Blake Rouse on 2019-03-20

Merge the iPXE work from wpk.

dc88c70... by Blake Rouse on 2019-03-20

Cleanup the iPXE boot method.

0eae91d... by Blake Rouse on 2019-03-20

Add missing user_class to S390x boot method.

1e7461a... by Blake Rouse on 2019-03-20

Fix issue with DHCP config generation.

912dc6e... by Blake Rouse on 2019-03-20

Keep user_class as a string.

80bf3d3... by Blake Rouse on 2019-03-20

Add missing regex compile.

5098d63... by Blake Rouse on 2019-03-20

Fix the ipxe.cfg script.

b3c747d... by Blake Rouse on 2019-03-22

Merge branch 'master' into ipxe-boot

8a7b26b... by Blake Rouse on 2019-03-22

Use path_prefix_http and path_prefix_force for iPXE.

9cefcda... by Blake Rouse on 2019-03-22

Fix issues with merge and tests.

c86b63d... by Blake Rouse on 2019-03-22

Make ipxe.cfg work.

24bb7f4... by Blake Rouse on 2019-03-22

Fix some DHCP bootloader rendering issues.

aed4dcd... by Blake Rouse on 2019-03-22

Fix the ipxe template.

99d7a77... by Blake Rouse on 2019-03-22

Fix the ipxe template.

14d80c7... by Blake Rouse on 2019-03-22

Fix local boot template.

fbaffaa... by Blake Rouse on 2019-03-25

Add tests.

24622dd... by Blake Rouse on 2019-03-27

Add comment to the config.local.template.

25605f3... by Blake Rouse on 2019-03-27

Merge branch 'master' into grub-http

0748c07... by Blake Rouse on 2019-03-27

Merge branch 'ipxe-boot' into grub-http

a5012d9... by Blake Rouse on 2019-03-28

Merge branch 'master' into grub-http

[Lee Trager (ltrager)]

With the latest from in the public streams this passes CI

http://10.245.136.4:8080/job/maas-git-manual/187

review: Approve

diff --git a/src/provisioningserver/boot/__init__.py b/src/provisioningserver/boot/__init__.py
index 40be47a..d77e5b3 100644
--- a/src/provisioningserver/boot/__init__.py
+++ b/src/provisioningserver/boot/__init__.py
@@ -358,6 +358,10 @@ class BootMethod(metaclass=ABCMeta):
             return 'http://%s:5248/images/' % (
                 convert_host_to_uri_str(params.fs_host))
 
+        def fs_efihost(params):
+            return '(http,%s:5248)/images/' % (
+                convert_host_to_uri_str(params.fs_host))
+
         def image_dir(params):
             return compose_image_path(
                 params.osystem, params.arch, params.subarch,
@@ -404,6 +408,7 @@ class BootMethod(metaclass=ABCMeta):
 
         namespace = {
             "fs_host": fs_host,
+            "fs_efihost": fs_efihost,
             "initrd_path": initrd_path,
             "kernel_command": kernel_command,
             "kernel_params": kernel_params,
diff --git a/src/provisioningserver/boot/tests/test_uefi_amd64.py b/src/provisioningserver/boot/tests/test_uefi_amd64.py
index 09fdc48..0dee3f4 100644
--- a/src/provisioningserver/boot/tests/test_uefi_amd64.py
+++ b/src/provisioningserver/boot/tests/test_uefi_amd64.py
@@ -32,6 +32,7 @@ from provisioningserver.boot.uefi_amd64 import (
 from provisioningserver.tests.test_kernel_opts import make_kernel_parameters
 from provisioningserver.utils import typed
 from provisioningserver.utils.fs import tempdir
+from provisioningserver.utils.network import convert_host_to_uri_str
 from testtools.matchers import (
     ContainsAll,
     IsInstance,
@@ -77,6 +78,8 @@ class TestUEFIAMD64BootMethodRender(MAASTestCase):
         # correctly rendered.
         method = UEFIAMD64BootMethod()
         params = make_kernel_parameters(purpose="xinstall")
+        fs_host = '(http,%s:5248)/images' % (
+            convert_host_to_uri_str(params.fs_host))
         output = method.get_reader(backend=None, kernel_params=params)
         # The output is a BytesReader.
         self.assertThat(output, IsInstance(BytesReader))
@@ -96,12 +99,14 @@ class TestUEFIAMD64BootMethodRender(MAASTestCase):
                     r" \[\'MAAS\'\]\\}end_cc.*",
                     re.MULTILINE | re.DOTALL),
                 MatchesRegex(
-                    r'.*^\s+linux  %s/%s .+?$' % (
-                        re.escape(image_dir), params.kernel),
+                    r'.*^\s+linuxefi  %s/%s/%s .+?$' % (
+                        re.escape(fs_host), re.escape(image_dir),
+                        params.kernel),
                     re.MULTILINE | re.DOTALL),
                 MatchesRegex(
-                    r'.*^\s+initrd %s/%s$' % (
-                        re.escape(image_dir), params.initrd),
+                    r'.*^\s+initrdefi %s/%s/%s$' % (
+                        re.escape(fs_host), re.escape(image_dir),
+                        params.initrd),
                     re.MULTILINE | re.DOTALL)))
 
     def test_get_reader_with_extra_arguments_does_not_affect_output(self):
diff --git a/src/provisioningserver/templates/uefi/config.commissioning.template b/src/provisioningserver/templates/uefi/config.commissioning.template
index 5e3f0ed..47dbec9 100644
--- a/src/provisioningserver/templates/uefi/config.commissioning.template
+++ b/src/provisioningserver/templates/uefi/config.commissioning.template
@@ -2,7 +2,7 @@ set default="0"
 set timeout=0
 
 menuentry 'Commission' {
-    echo   'Booting under MAAS direction...'
-    linux  {{kernel_params | kernel_path }} {{kernel_params | kernel_command}} BOOTIF=01-${net_default_mac}
-    initrd {{kernel_params | initrd_path }}
+    echo      'Booting under MAAS direction...'
+    linuxefi  {{kernel_params | fs_efihost}}{{kernel_params | kernel_path }} {{kernel_params | kernel_command}} BOOTIF=01-${net_default_mac}
+    initrdefi {{kernel_params | fs_efihost}}{{kernel_params | initrd_path }}
 }
diff --git a/src/provisioningserver/templates/uefi/config.enlist.template b/src/provisioningserver/templates/uefi/config.enlist.template
index 4b496dc..edf800e 100644
--- a/src/provisioningserver/templates/uefi/config.enlist.template
+++ b/src/provisioningserver/templates/uefi/config.enlist.template
@@ -2,7 +2,7 @@ set default="0"
 set timeout=0
 
 menuentry 'Enlist' {
-    echo   'Booting under MAAS direction...'
-    linux  {{kernel_params | kernel_path }} {{kernel_params | kernel_command}} BOOTIF=01-${net_default_mac}
-    initrd {{kernel_params | initrd_path }}
+    echo      'Booting under MAAS direction...'
+    linuxefi  {{kernel_params | fs_efihost}}{{kernel_params | kernel_path }} {{kernel_params | kernel_command}} BOOTIF=01-${net_default_mac}
+    initrdefi {{kernel_params | fs_efihost}}{{kernel_params | initrd_path }}
 }
diff --git a/src/provisioningserver/templates/uefi/config.install.template b/src/provisioningserver/templates/uefi/config.install.template
index 381ae22..eea84e0 100644
--- a/src/provisioningserver/templates/uefi/config.install.template
+++ b/src/provisioningserver/templates/uefi/config.install.template
@@ -2,7 +2,7 @@ set default="0"
 set timeout=0
 
 menuentry 'Install' {
-    echo   'Booting under MAAS direction...'
-    linux  {{kernel_params | kernel_path }} {{kernel_params | kernel_command}} BOOTIF=01-${net_default_mac}
-    initrd {{kernel_params | initrd_path }}
+    echo      'Booting under MAAS direction...'
+    linuxefi  {{kernel_params | fs_efihost}}{{kernel_params | kernel_path }} {{kernel_params | kernel_command}} BOOTIF=01-${net_default_mac}
+    initrdefi {{kernel_params | fs_efihost}}{{kernel_params | initrd_path }}
 }