Merge ~alextu/pc-enablement/+git/oem-scripts:rsync_iso into ~oem-solutions-engineers/pc-enablement/+git/oem-scripts:master

Proposed by Alex Tu
Status: Needs review
Proposed branch: ~alextu/pc-enablement/+git/oem-scripts:rsync_iso
Merge into: ~oem-solutions-engineers/pc-enablement/+git/oem-scripts:master
Diff against target: 197 lines (+181/-1)
2 files modified
recovery-from-iso.sh (+1/-1)
rsync_iso.sh (+180/-0)
Reviewer Review Type Date Requested Status
Yao Wei Approve
Andy Chi (community) Approve
Cyrus Lien Pending
Yuan-Chen Cheng Pending
Shih-Yuan Lee Pending
OEM Solutions Group: Engineers Pending
Review via email: mp+404595@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Alex Tu (alextu) wrote :
040f634... by Alex Tu

rsync_iso.sh: new parameger, --allusb to flash all detected usb.

Revision history for this message
Alex Tu (alextu) wrote :

a test result over VPN.
$ rsync_iso.sh --jenkins-url jenkins.canonical.pw --jenkins-job dell-bto-focal-fossa-edge-alloem-init -s
https://pastebin.canonical.com/p/8h8Zbzy3Pc/

It will popup a dialog reminding you in the end.
https://i.imgur.com/v20GtNB.png

Revision history for this message
Andy Chi (andch) wrote :

Test pass on local machine.

$ ./rsync_iso.sh ~/Downloads/dell-bto-focal-fossa-pypar-adl-X163-20211027-6.iso -f /media/andch/Dell\ Recovery\ Media/ -s

https://pastebin.canonical.com/p/88vfNxRzHM/

Can we remove set -x?

By the way, tmp.xxxx folder won't be deleted after installation usb is finished.

$ ls
tmp.Oe7dubjaNd

Revision history for this message
Alex Tu (alextu) wrote :

sure thing, let me remove the set -x and fix the un-deleted `tmp.xxxx`

7249df2... by Alex Tu

removed -x and solidly remove tmp folders.

Revision history for this message
Andy Chi (andch) wrote :

Test done.

sent 4,145,771,921 bytes received 42,432 bytes 236,903,677.31 bytes/sec
total size is 4,144,596,677 speedup is 1.00
================================================================
    dell-bto-focal-fossa-tentacool-X169-20211014-12.iso
================================================================
~/oem-scripts/tmp.AAZF4XBUX7 ~/oem-scripts
~/oem-scripts
/usr/bin/zenity
umount: /home/andch/oem-scripts/tmp.ICAjrBvoSh: no mount point specified.
umount: /home/andch/oem-scripts/tmp.AAZF4XBUX7: no mount point specified.

LGTM

review: Approve
56821cb... by Alex Tu

fix the warning during umount

Revision history for this message
OEM Taipei Bot (oem-taipei-bot) wrote :

[autopkgtest]
autopkgtest-collect-credentials PASS
autopkgtest-oem-scripts-auto PASS
pkg-somerville-meta PASS
pkg-stella-meta PASS
pkg-sutton-meta PASS
bug-bind PASS
get-private-ppa PASS
jq-lp PASS
launchpad-api PASS
lp-bug PASS
oem-meta-packages PASS
pkg-list PASS
review-merge-proposal PASS
run-autopkgtest PASS
setup-apt-dir PASS
mir-bug FAIL non-zero exit status 1
git-url-insteadof-setting FAIL non-zero exit status 128
recovery-from-iso.sh PASS
mir-bug-verification FAIL stderr: E: Unable to locate package oem-somerville-maester-meta

https://oem-share.canonical.com/partners/lyoncore/share/artifacts/oem-scripts-0.99-56821cb-in-linux-container-focal

Revision history for this message
Alex Tu (alextu) wrote :

testing for commit 215f181:

# Environment:
2 USB disks be plugged
sda 8:0 1 58.4G 0 disk
└─sda1 8:1 1 58.4G 0 part /media/alextu/C695-6B4E
sdb 8:16 1 14.5G 0 disk
└─sdb1 8:17 1 14.5G 0 part /media/alextu/CEC2-88FB

# verified command
$ ./rsync_iso.sh --jenkins-url jenkins.canonical.pw --jenkins-job dell-bto-focal-fossa-edge-alloem-init --jenkins-job-number 18 --all-usb -s 2>&1 | tee log

https://pastebin.canonical.com/p/kRc2P2x3Rg/

# result, 2 USB disks be made successfully.

Revision history for this message
Yao Wei (medicalwei) wrote :

Verified manually that by using the following command:
./rsync_iso.sh --jenkins-url jenkins.canonical.pw --jenkins-job dell-bto-focal-fossa-edge-alloem-init

The external disk I plugged in has been formatted into FAT32 and files from dell-bto-focal-fossa-edge-alloem-init-X186-20220111-27.iso has been copied over.

Other than having too much log messages (suggested by andch to remove set -x), it LGTM.

review: Approve
Revision history for this message
Alex Tu (alextu) wrote (last edit ):

+1, to remove the set -x.

2b0b120... by Alex Tu

recovery-from-iso.sh: remove set -x

Revision history for this message
OEM Taipei Bot (oem-taipei-bot) wrote :

[autopkgtest]
autopkgtest-collect-credentials PASS
autopkgtest-oem-scripts-auto PASS
pkg-somerville-meta PASS
pkg-stella-meta PASS
pkg-sutton-meta PASS
bug-bind PASS
get-private-ppa PASS
jq-lp PASS
launchpad-api PASS
lp-bug PASS
oem-meta-packages PASS
pkg-list PASS
review-merge-proposal PASS
run-autopkgtest PASS
setup-apt-dir PASS
mir-bug FAIL non-zero exit status 1
git-url-insteadof-setting FAIL non-zero exit status 128
recovery-from-iso.sh PASS
mir-bug-verification PASS

https://oem-share.canonical.com/partners/lyoncore/share/artifacts/oem-scripts/oem-scripts-0.99-2b0b120-in-linux-container-focal

Unmerged commits

2b0b120... by Alex Tu

recovery-from-iso.sh: remove set -x

040f634... by Alex Tu

rsync_iso.sh: new parameger, --allusb to flash all detected usb.

56821cb... by Alex Tu

fix the warning during umount

7249df2... by Alex Tu

removed -x and solidly remove tmp folders.

a931e27... by Alex Tu

take care the case of multiple partitions in target usb disk

045cd32... by Alex Tu

make sure the url of Jenkins is accessable

36a2a31... by Alex Tu

a handy script to create bootable usb with OEM image

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/recovery-from-iso.sh b/recovery-from-iso.sh
index 843d9cc..cb20f90 100755
--- a/recovery-from-iso.sh
+++ b/recovery-from-iso.sh
@@ -1,5 +1,5 @@
1#!/bin/bash1#!/bin/bash
2set -ex2set -e
33
4jenkins_job_for_iso=""4jenkins_job_for_iso=""
5jenkins_job_build_no="lastSuccessfulBuild"5jenkins_job_build_no="lastSuccessfulBuild"
diff --git a/rsync_iso.sh b/rsync_iso.sh
6new file mode 1007556new file mode 100755
index 0000000..6ef5f2d
--- /dev/null
+++ b/rsync_iso.sh
@@ -0,0 +1,180 @@
1#!/bin/bash
2set -ex
3CD_PATH="$PWD"
4JENKINS_B_NO="lastSuccessfulBuild"
5
6mnt_folder="$(mktemp -d -p "$PWD")"
7target_folder="$(mktemp -d -p "$PWD")"
8# $1: the target to unumount
9do_unmount() {
10 if [ -n "$1" ] && grep -q "$1" /proc/mounts; then
11 sudo umount "$1"
12 fi
13}
14clear_all() {
15 do_unmount "$mnt_folder"
16 do_unmount "$target_folder"
17 rm -rf "$mnt_folder"
18 rm -rf "$target_folder"
19}
20trap clear_all EXIT
21
22usage() {
23cat << EOF
24usage:
25$0 options --iso [ISO file] | [file.iso] [<options to get iso from jenkins artifacts>]
26$0 options [<options to get iso from jenkins artifacts>] [-s][-f]
27
28** IMPORTANT **
291. This script only works when only one USB disk plugged and it can be found in file manager,
302. USB disk will be wipe.
31**
32
33This command will create a bootable usb disk by the ISO file,
34It assume the iso file in the same folder where script be executed.
35options:
36 -h|--help Print this message
37 -f|--folder The folder you would like to copy to root of target usb disk
38 -i|--iso The iso file which be used to create a bootable USB disk.
39 -a|--allusb Flash all detected usb disk.
40 -s No interactive
41
42options to get iso from jenkins artifacts:
43 --jenkins-url The url of target Jenkins
44 --jenkins-job The job of target Jenkins
45 --jenkins-job-number The buile number of target Jenkins job
46
47example:
48 # Download iso from specific Jenkins job and build number then rsync the iso to usb disk without user interactive.
49 $ rsync_iso.sh --jenkins-url jenkins.canonical.pw --jenkins-job pc-stella-cmit-focal-amd64 --jenkins-job-number 1569 -s
50
51 # rsync local iso and dump everything in ~/path-to-the-folder to usb disk with user interactive.
52 $ rsync_iso.sh any-oem-image.iso -f ~/path-to-the-folder
53
54EOF
55exit 1
56}
57
58
59while [ $# -gt 0 ]
60do
61 case "$1" in
62 -h | --help)
63 usage 0
64 exit 0
65 ;;
66 -f |--folder)
67 shift
68 FOLDER=$1;
69 [ -z "${FOLDER##/*}" ] || [ -z "${FOLDER##~/*}" ] || FOLDER="$CD_PATH/$FOLDER"
70 if [ ! -d "$FOLDER" ];then
71 echo "not exists $FOLDER"
72 exit 1
73 fi
74 ;;
75 -i | --iso)
76 shift
77 ISO=$1
78 if [ ! -e "$ISO" ]; then
79 echo "not exists $FOLDER"
80 exit 1
81 fi
82 ;;
83 -s)
84 SILENCE="TRUE"
85 ;;
86 --jenkins-url)
87 shift
88 JENKINS_URL="$1"
89 if ! ping -c 1 -W 1 "$JENKINS_URL"; then
90 echo "[ERROR] Please make sure you can connect to Jenkins: $JENKINS_URL"
91 exit 1
92 fi
93 ;;
94 --jenkins-job)
95 shift
96 JENKINS_JOB="$1"
97 ;;
98 --jenkins-job-number)
99 shift
100 JENKINS_B_NO="$1"
101 ;;
102 -a | --all-usb)
103 FLASH_ALL_USB="true"
104 ;;
105 *)
106 if [ -z "${1##*.iso}" ]; then
107 ISO=$1
108 else
109 usage
110 fi
111 esac
112 shift
113done
114
115if [ -n "$JENKINS_URL" ] && [ -n "$JENKINS_JOB" ]; then
116 img_jenkins_out_url="ftp://$JENKINS_URL/jenkins_host/jobs/$JENKINS_JOB/builds/$JENKINS_B_NO/archive/out"
117 img_name="$(wget -q "$img_jenkins_out_url/" -O - | grep -o 'href=.*iso"' | awk -F/ '{print $NF}' | tr -d \")"
118 rm -f "$img_jenkins_out_url/$img_name".md5sum
119 wget "$img_jenkins_out_url/$img_name".md5sum
120 md5sum -c "$img_name".md5sum || wget "$img_jenkins_out_url"/"$img_name"
121 md5sum -c "$img_name".md5sum || usage
122 ISO="$img_name"
123fi
124
125[ -z "$ISO" ] && usage
126usb_mnt_list=$(mount | grep udisk | awk -F'type|on' '{print $1}' | sort)
127usb_list=$(echo "$usb_mnt_list"| sed 's/[1-9]//' | uniq)
128if [ -z "$usb_list" ]; then
129 echo "can not find any USB be insertted!"
130 usage
131fi
132
133if [ "$(echo "$usb_list" | wc -l)" -gt 1 ] && [ "$FLASH_ALL_USB" != "true" ];then
134 echo "There are more than one usb disk, please make sure only one usb disk connected to your host."
135 echo "$usb_list"
136 exit
137else
138 usb_targets=$(echo "$usb_list" | tr -d "[:blank:]")
139fi
140
141sudo mount -o loop "$ISO" "$mnt_folder"
142
143while read -r disk; do
144 do_unmount "$disk"
145done <<<"$usb_mnt_list"
146
147for usb_target in $usb_targets; do
148 echo "press any key to wipe $usb_target and create bootable usb..... "
149 [ "$SILENCE" == "TRUE" ] || read -r
150 sudo wipefs -a "$usb_target" 2>&1 || (echo "Wipefs failed"; usage)
151 sudo parted -s "$usb_target" mktable msdos || (echo "parted mktable failed."; usage)
152 sudo parted -s "$usb_target" mkpart primary 0% 100% || (echo "parted mkpart failed."; usage)
153 usb_target="${usb_target}1"
154 sleep 2
155 sudo mkfs.vfat -F 32 "$usb_target"
156
157 target_folder="$(mktemp -d -p "$PWD")"
158 sudo mount -t vfat -o gid=$UID,uid=$UID "$usb_target" "$target_folder"
159
160 echo "pless any keye to do ionice -c 3 rsync --delete -avP $mnt_folder/ $target_folder"
161 [ "$SILENCE" == "TRUE" ] || read -r
162 ionice -c 3 rsync --delete -avP "$mnt_folder"/ "$target_folder"
163 iso_name=$(grep "<iso" "$target_folder/bto.xml" | sed 's/<[^>]*>//g')
164 echo ================================================================
165 echo "$iso_name"
166 echo ================================================================
167 pushd "$target_folder" || exit 1
168 if [ -n "$FOLDER" ] && [ -d "$FOLDER" ]; then
169 echo "pless any keye to do ionice -c 3 rsync -avP $FOLDER/ ."
170 [ "$SILENCE" == "TRUE" ] || read -r
171 ionice -c 3 rsync -avP "$FOLDER"/ .
172 fi
173 popd || exit 1
174
175 do_unmount "$target_folder"
176 rm -rf "$target_folder"
177 udisksctl mount -b "$usb_target"
178done
179clear_all
180command -v zenity && zenity --timeout 600 --info --text="$0 $ISO done" &

Subscribers

People subscribed via source and target branches