Merge ~tai271828/+git/maas-preseeds:mr-linux-nvidia-curtin-templating into ~ce-hyperscale/+git/maas-preseeds:main

[Taihsiang Ho]

Per our discussion in https://warthogs.atlassian.net/browse/NVDGX-108?focusedCommentId=227032 , let's apply the curtin templating workaround.

- I have deployed blanka with Jammy and focal, and I will get the corresponding linux-nvidia and linux-generic.
- I don't have a chance to try with akis to see if we have any regression yet as it was occupied by kernel testing.

[dann frazier (dannf)]

I'd suggesting cherry-picking the original versions of the first 3 commits instead of reverting revert commits, but with only that change, +1!

review: Approve

[Taihsiang Ho (tai271828)]

Rebased by following the suggestion of cherry-picking. Landing the mr.

[Taihsiang Ho (tai271828)]

Merged. Deploy to Needham MAAS and Taipei MAAS.

[Taihsiang Ho (tai271828)]

Deployed*

diff --git a/curtin_userdata_dgx b/curtin_userdata_dgx
index 7380386..d1c2a4e 100644
--- a/curtin_userdata_dgx
+++ b/curtin_userdata_dgx
@@ -97,6 +97,11 @@ debconf_selections:
 swap:
   size: 0
 
+{{if release not in ["xenial", "bionic", "focal"]}}
+kernel:
+  package: linux-nvidia
+{{endif}}
+
 late_commands:
   maas: ['wget', '--no-proxy', '{{node_disable_pxe_url}}', '--post-data', '{{node_disable_pxe_data}}', '-O', '/dev/null']
   01_maas_preseed: ['curtin', 'in-target', '--', 'git', 'clone', 'https://git.launchpad.net/~ce-hyperscale/+git/maas-preseeds', '/tmp/maas-dgx-preseed']
diff --git a/late.sh b/late.sh
index 37be944..29cfe7e 100755
--- a/late.sh
+++ b/late.sh
@@ -18,9 +18,9 @@ install_mellanox_ofed() {
             MELLANOX_OFED_VERSION=4.9-2.2.4.0
             ;;
         *)
-            # nvidia seems to make future DGX MOFED-less, and DGX OS does not
-            # install MOFED. Let's try to have the same performance
-            # characteristics for jammy and above.
+            # nvidia seems to make future DGX MOFED-less, and DGX OS starting
+            # with jammy does not install MOFED. Let's try to have the same
+            # performance characteristics for jammy and above.
             return 0
     esac
     mlnx_url="${MLNX_REPO}/${MELLANOX_OFED_VERSION}/ubuntu${ubuntu_ver}/mellanox_mlnx_ofed.list"
@@ -127,18 +127,33 @@ fi
 
 pkgs="nvidia-utils-${NVIDIA_DRIVER_VERSION}"
 pkgs="${pkgs} nvidia-kernel-source-${NVIDIA_DRIVER_VERSION}"
-ubuntu_ver="$(lsb_release -rs)"
+# For nvidia flavor it is a bit tricky since the nomenculate is not consistent
+# with generic and other flavors fully. For example,
+#    $ dpkg-query -W -f '${Package}\n' linux-nvidia*
+#    linux-nvidia
+#    linux-nvidia-headers-5.15.0-1015
+#    linux-nvidia-source-5.15.0
+#    linux-nvidia-tools
+# but the generic headers would look like
+#    linux-headers-generic
+#    linux-headers-5.15.0-25-generic
+# even more, this is the hwe flavor for example:
+#    linux-headers-generic-hwe-20.04
+#
+# So using similar pattern to generalize regex will not work.
+#
+# Lastly, brace expansion is not POSIX. Dash does not support brace expansion
+# so we do not use something like:
+#     linux-{generic,nvidia}
+# and let us use the other approaches like a for-loop instead.
+for possible_flavor_pattern in linux-generic* linux-nvidia; do
+    for metapkg in $(dpkg-query -W -f '${Package}\n' "${possible_flavor_pattern}"); do
+        flavor=${metapkg#linux-}
+        pkgs="$pkgs linux-modules-nvidia-${NVIDIA_DRIVER_VERSION}-${flavor}"
+        echo $pkgs
+    done
+done
 
-case $ubuntu_ver in
-    22.04)
-        pkgs="$pkgs linux-modules-nvidia-${NVIDIA_DRIVER_VERSION}-nvidia linux-nvidia"
-        ;;
-    *)
-        for metapkg in $(dpkg-query -W -f '${Package}\n' "linux-generic*"); do
-            flavor=${metapkg#linux-}
-            pkgs="$pkgs linux-modules-nvidia-${NVIDIA_DRIVER_VERSION}-${flavor}"
-        done
-esac
 apt install -y ${pkgs}
 install_fabric_manager
 install_mellanox_ofed