Merge ~alextu/pc-enablement/+git/oem-scripts:recovery-from-iso-inject-stella-on-somverville-img into ~oem-solutions-engineers/pc-enablement/+git/oem-scripts:master

[Alex Tu]

[Alex Tu (alextu)]

test steps:
1. install iso from http://{jenkins}/job/dell-bto-focal-fossa-edge-alloem_dev_jenkins_remote_trigger/ on VM

2. recovery-from-iso.sh --local-iso {stella iso} --ubr -t {ip of target machine}

=> the target can done recovery with target iso.

[Alex Tu (alextu)]

a test is running on http://10.101.46.50:8080/job/sanity-3-testflinger-fossa-maester-980000-0002-staging/4/console

[Alex Tu (alextu)]

new test: http://10.101.46.50:8080/job/sanity-3-testflinger-fossa-maester-980000-0002-staging/6/console

[Alex Tu (alextu)]

new test: http://10.101.46.50:8080/job/sanity-3-testflinger-fossa-maester-980000-0002-staging/7/console

[jeremyszu (os369510)]

Hi Alex,

What's different between

dell-bto-focal-fossa-edge-alloem
dell-bto-focal-fossa-edge-alloem_dev
dell-bto-focal-fossa-edge-alloem-init
dell-bto-focal-fossa-edge-alloem-init_dev
dell-bto-focal-fossa-edge-alloem_dev_jenkins_remote_trigger

?

review: Needs Information

[jeremyszu (os369510)]

I still need to try it in physical machine but I need the information from previous comment.
And one thing needs to be fixed at least.

review: Needs Fixing

[Andy Chi (andch)]

Cannot enter recovery partition.

error: file 'casper/vmlinuz' not found
error: you need to load kernel first

[Alex Tu (alextu)]

comment inline

[Alex Tu (alextu)]

just also verified on HP laptop (202103-28767) with the patch 22c3478

[Andy Chi (andch)]

Test pass with 202104-28974. One question of dell-bto-focal-fossa-edge-alloem-init-X170-20211019-16.iso, what is the expected user name of this image? I see u and ubuntu both exist. When OS startup, I observe u as login user. `user_on_target` expects "ubuntu".

[jeremyszu (os369510)]

> Hi Alex,
>
> What's different between
>
> dell-bto-focal-fossa-edge-alloem
> dell-bto-focal-fossa-edge-alloem_dev
> dell-bto-focal-fossa-edge-alloem-init
> dell-bto-focal-fossa-edge-alloem-init_dev
> dell-bto-focal-fossa-edge-alloem_dev_jenkins_remote_trigger
>
> ?

the answer is here
https://chat.canonical.com/canonical/pl/pfyw9y7yzbrcp8dcpq5a8uhddy

[Alex Tu (alextu)]

> Test pass with 202104-28974. One question of dell-bto-focal-fossa-edge-alloem-
> init-X170-20211019-16.iso, what is the expected user name of this image? I see
> u and ubuntu both exist. When OS startup, I observe u as login user.
> `user_on_target` expects "ubuntu".

`u` is used for original usb dev flow and I'm trying to make `ubuntu` as default if no one is concerning on that change.

[Andy Chi (andch)]

review: Approve

[jeremyszu (os369510)]

Hi Alex,

I can not install "dell-bto-focal-fossa-edge-alloem-X170-20211019-28.iso" on stella machine to verify your patch. IIRC, the alloem is made by you.

The error shows in dell-recovery. "this recovery media only function on Dell..."

review: Needs Information

[jeremyszu (os369510)]

Verified by

1. installing dell-bto-focal-fossa-edge-alloem-init-X172-20211029-17.iso by USB
2. recovery-from-iso.sh pc-stella-cmit-focal-amd64-X00-20211029-1725.iso, then
3. recovery-from-iso.sh pc-stella-cmit-focal-amd64-X00-20211109-1733.iso

All passed.

review: Approve

diff --git a/recovery-from-iso.sh b/recovery-from-iso.sh
index a6f4439..3b42656 100755
--- a/recovery-from-iso.sh
+++ b/recovery-from-iso.sh
@@ -4,6 +4,7 @@ set -ex
 jenkins_job_for_iso=""
 jenkins_job_build_no="lastSuccessfulBuild"
 script_on_target_machine="inject_recovery_from_iso.sh"
+additional_grub_for_ubuntu_recovery="99_ubuntu_recovery"
 user_on_target="ubuntu"
 SSH="ssh -o StrictHostKeyChecking=no"
 SCP="scp -o StrictHostKeyChecking=no"
@@ -91,6 +92,7 @@ ubiquity ubuntu-recovery/recovery_type string dev
         git clone --depth 1 -b maas-focal lp:~lyoncore-team/lyoncore/+git/somerville-maas-override maas_deps
         cd maas_deps
         git rev-parse --short HEAD
+        rm -f maas-pkgs/oem-fix-misc-cnl-maas-helper*
         cd ..
         cp -r maas_deps/maas-pkgs/*.deb .
         find .
@@ -116,6 +118,61 @@ apt-get install -fy
     # get pkgs for ssh key and skip disk checking.
     $GIT clone https://git.launchpad.net/~oem-solutions-engineers/pc-enablement/+git/oem-fix-misc-cnl-misc-for-automation --depth 1 misc_for_automation
 
+    if [ "${ubr}" == "yes" ]; then
+    mkdir -p "$temp_folder"/preseed
+    cat <<EOF1 > "$temp_folder/preseed/$additional_grub_for_ubuntu_recovery"
+#!/bin/bash -e
+source /usr/lib/grub/grub-mkconfig_lib
+cat <<EOF
+menuentry "ubuntu-recovery restore" --hotkey f9 {
+        search --no-floppy --hint '(hd0,gpt2)' --set --fs-uuid UUID_OF_RECOVERY_PARTITION
+        if [ -s /boot/grub/common.cfg ]; then
+            source /boot/grub/common.cfg
+        else
+            set options="boot=casper automatic-ubiquity noprompt quiet splash"
+        fi
+
+        if [ "\\\${grub_platform}" != "efi" ]; then
+            if [ -f \\\${prefix}/nomodesetlist.txt ]; then
+                if hwmatch \\\${prefix}/nomodesetlist.txt 3; then
+                    if [ \\\${match} != 0 ]; then
+                        set options="nomodeset \\\$options"
+                    fi
+                fi
+            fi
+        fi
+
+        #Support starting from a loopback mount (Only support ubuntu.iso for filename)
+        if [ -f /ubuntu.iso ]; then
+            loopback loop /ubuntu.iso
+            set root=(loop)
+            set options="\\\$options iso-scan/filename=/ubuntu.iso"
+        fi
+        if [ -n "\\\${lang}" ]; then
+            set options="\\\$options locale=\\\$lang"
+        fi
+
+        linux   /casper/vmlinuz ubuntu-recovery/recovery_type=hdd \\\$options
+        initrd  /casper/initrd
+}
+EOF
+EOF1
+    cat <<EOF > "$temp_folder/preseed/set_env_for_ubuntu_recovery"
+#!/bin/bash -ex
+# replace the grub entry which ubuntu_recovery expected
+recover_p=\$(lsblk -l | grep efi | cut -d ' ' -f 1 | sed 's/.$/2'/)
+UUID_OF_RECOVERY_PARTITION=\$(ls -l /dev/disk/by-uuid/ | grep \$recover_p | awk '{print \$9}')
+echo partition = \$UUID_OF_RECOVERY_PARTITION
+sed -i "s/UUID_OF_RECOVERY_PARTITION/\$UUID_OF_RECOVERY_PARTITION/" push_preseed/preseed/$additional_grub_for_ubuntu_recovery
+sudo rm -f /etc/grub.d/99_dell_recovery || true
+chmod 766 push_preseed/preseed/$additional_grub_for_ubuntu_recovery
+sudo cp push_preseed/preseed/$additional_grub_for_ubuntu_recovery /etc/grub.d/
+
+# Force changing the recovery partition label to PQSERVICE for ubuntu-recovery
+sudo fatlabel /dev/\$recover_p PQSERVICE
+EOF
+    fi
+
     return 0
 }
 push_preseed() {
@@ -152,12 +209,19 @@ inject_preseed() {
     download_preseed && \
     push_preseed
     scp -o StrictHostKeyChecking=no "$user_on_target"@"$target_ip":/cdrom/SUCCSS_push_preseed "$temp_folder" || usage
+
+    if [ "${ubr}" == "yes" ]; then
+        $SSH "$user_on_target"@"$target_ip" bash \$HOME/push_preseed/preseed/set_env_for_ubuntu_recovery || usage
+    fi
     $SSH "$user_on_target"@"$target_ip" touch /tmp/SUCCSS_inject_preseed
 }
 
 inject_recovery_iso() {
     if [ -n "$local_iso" ]; then
         img_name="$(basename "$local_iso")"
+        if [ -z "${img_name##*stella*}" ]; then
+            ubr="yes"
+        fi
         if [ "${ubr}" == "yes" ]; then
             rsync_opts="--exclude=efi --delete --temp-dir=/var/tmp/rsync"
         else
@@ -172,7 +236,8 @@ cat <<EOF > "$temp_folder/$script_on_target_machine"
 set -ex
 sudo umount /cdrom /mnt || true
 sudo mount -o loop $img_name /mnt && \
-sudo mount /dev/\$(lsblk -l | grep efi | cut -d ' ' -f 1 | sed 's/.$/2'/) /cdrom && \
+recover_p=\$(lsblk -l | grep efi | cut -d ' ' -f 1 | sed 's/.$/2'/) && \
+sudo mount /dev/\$recover_p /cdrom && \
 df | grep "cdrom\|mnt" | awk '{print \$2" "\$6}' | sort | tail -n1 | grep -q cdrom && \
 sudo mkdir -p /var/tmp/rsync && \
 sudo rsync -alv /mnt/ /cdrom/ $rsync_opts && \
@@ -188,7 +253,7 @@ EOF
         img_name="$(wget -q "$img_jenkins_out_url/" -O - | grep -o 'href=.*iso"' | awk -F/ '{print $NF}' | tr -d \")"
         pushd "$temp_folder" || usage
         wget "$img_jenkins_out_url/$img_name".md5sum
-        md5sum -c "$img_name".md5sum || wget "$img_jenkins_out_url"/"$img_name"
+        md5sum -c "$img_name".md5sum || wget "$img_jenkins_out_url"/"$img_name" 2> /dev/null
         md5sum -c "$img_name".md5sum || usage
         local_iso="$PWD/$img_name"
         popd
@@ -212,7 +277,9 @@ poll_recovery_status() {
 
 do_recovery() {
     if [ "${ubr}" == "yes" ]; then
-        echo GRUB_DEFAULT=3 | $SSH "$user_on_target"@"$target_ip" -T "sudo tee -a /etc/default/grub.d/automatic-oem-config.cfg"
+        echo GRUB_DEFAULT='"ubuntu-recovery restore"' | $SSH "$user_on_target"@"$target_ip" -T "sudo tee -a /etc/default/grub.d/automatic-oem-config.cfg"
+        echo GRUB_TIMEOUT_STYLE=menu | $SSH "$user_on_target"@"$target_ip" -T "sudo tee -a /etc/default/grub.d/automatic-oem-config.cfg"
+        echo GRUB_TIMEOUT=5 | $SSH "$user_on_target"@"$target_ip" -T "sudo tee -a /etc/default/grub.d/automatic-oem-config.cfg"
         $SSH "$user_on_target"@"$target_ip" sudo update-grub
         $SSH "$user_on_target"@"$target_ip" sudo reboot &
     else