Merge ~alextu/pc-enablement/+git/oem-scripts:rsync_iso into ~oem-solutions-engineers/pc-enablement/+git/oem-scripts:master

Proposed by Alex Tu
Status: Needs review
Proposed branch: ~alextu/pc-enablement/+git/oem-scripts:rsync_iso
Merge into: ~oem-solutions-engineers/pc-enablement/+git/oem-scripts:master
Diff against target: 197 lines (+181/-1)
2 files modified
recovery-from-iso.sh (+1/-1)
rsync_iso.sh (+180/-0)
Reviewer Review Type Date Requested Status
Yao Wei Approve
Andy Chi Approve
Cyrus Lien Pending
Yuan-Chen Cheng Pending
Shih-Yuan Lee Pending
OEM Solutions Group: Engineers Pending
Review via email: mp+404595@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Alex Tu (alextu) wrote :
040f634... by Alex Tu

rsync_iso.sh: new parameger, --allusb to flash all detected usb.

Revision history for this message
Alex Tu (alextu) wrote :

a test result over VPN.
$ rsync_iso.sh --jenkins-url jenkins.canonical.pw --jenkins-job dell-bto-focal-fossa-edge-alloem-init -s
https://pastebin.canonical.com/p/8h8Zbzy3Pc/

It will popup a dialog reminding you in the end.
https://i.imgur.com/v20GtNB.png

Revision history for this message
Andy Chi (andch) wrote :

Test pass on local machine.

$ ./rsync_iso.sh ~/Downloads/dell-bto-focal-fossa-pypar-adl-X163-20211027-6.iso -f /media/andch/Dell\ Recovery\ Media/ -s

https://pastebin.canonical.com/p/88vfNxRzHM/

Can we remove set -x?

By the way, tmp.xxxx folder won't be deleted after installation usb is finished.

$ ls
tmp.Oe7dubjaNd

Revision history for this message
Alex Tu (alextu) wrote :

sure thing, let me remove the set -x and fix the un-deleted `tmp.xxxx`

7249df2... by Alex Tu

removed -x and solidly remove tmp folders.

Revision history for this message
Andy Chi (andch) wrote :

Test done.

sent 4,145,771,921 bytes received 42,432 bytes 236,903,677.31 bytes/sec
total size is 4,144,596,677 speedup is 1.00
================================================================
    dell-bto-focal-fossa-tentacool-X169-20211014-12.iso
================================================================
~/oem-scripts/tmp.AAZF4XBUX7 ~/oem-scripts
~/oem-scripts
/usr/bin/zenity
umount: /home/andch/oem-scripts/tmp.ICAjrBvoSh: no mount point specified.
umount: /home/andch/oem-scripts/tmp.AAZF4XBUX7: no mount point specified.

LGTM

review: Approve
56821cb... by Alex Tu

fix the warning during umount

Revision history for this message
OEM Taipei Bot (oem-taipei-bot) wrote :

[autopkgtest]
autopkgtest-collect-credentials PASS
autopkgtest-oem-scripts-auto PASS
pkg-somerville-meta PASS
pkg-stella-meta PASS
pkg-sutton-meta PASS
bug-bind PASS
get-private-ppa PASS
jq-lp PASS
launchpad-api PASS
lp-bug PASS
oem-meta-packages PASS
pkg-list PASS
review-merge-proposal PASS
run-autopkgtest PASS
setup-apt-dir PASS
mir-bug FAIL non-zero exit status 1
git-url-insteadof-setting FAIL non-zero exit status 128
recovery-from-iso.sh PASS
mir-bug-verification FAIL stderr: E: Unable to locate package oem-somerville-maester-meta

https://oem-share.canonical.com/partners/lyoncore/share/artifacts/oem-scripts-0.99-56821cb-in-linux-container-focal

Revision history for this message
Alex Tu (alextu) wrote :

testing for commit 215f181:

# Environment:
2 USB disks be plugged
sda 8:0 1 58.4G 0 disk
└─sda1 8:1 1 58.4G 0 part /media/alextu/C695-6B4E
sdb 8:16 1 14.5G 0 disk
└─sdb1 8:17 1 14.5G 0 part /media/alextu/CEC2-88FB

# verified command
$ ./rsync_iso.sh --jenkins-url jenkins.canonical.pw --jenkins-job dell-bto-focal-fossa-edge-alloem-init --jenkins-job-number 18 --all-usb -s 2>&1 | tee log

https://pastebin.canonical.com/p/kRc2P2x3Rg/

# result, 2 USB disks be made successfully.

Revision history for this message
Yao Wei (medicalwei) wrote :

Verified manually that by using the following command:
./rsync_iso.sh --jenkins-url jenkins.canonical.pw --jenkins-job dell-bto-focal-fossa-edge-alloem-init

The external disk I plugged in has been formatted into FAT32 and files from dell-bto-focal-fossa-edge-alloem-init-X186-20220111-27.iso has been copied over.

Other than having too much log messages (suggested by andch to remove set -x), it LGTM.

review: Approve
Revision history for this message
Alex Tu (alextu) wrote (last edit ):

+1, to remove the set -x.

2b0b120... by Alex Tu

recovery-from-iso.sh: remove set -x

Revision history for this message
OEM Taipei Bot (oem-taipei-bot) wrote :

[autopkgtest]
autopkgtest-collect-credentials PASS
autopkgtest-oem-scripts-auto PASS
pkg-somerville-meta PASS
pkg-stella-meta PASS
pkg-sutton-meta PASS
bug-bind PASS
get-private-ppa PASS
jq-lp PASS
launchpad-api PASS
lp-bug PASS
oem-meta-packages PASS
pkg-list PASS
review-merge-proposal PASS
run-autopkgtest PASS
setup-apt-dir PASS
mir-bug FAIL non-zero exit status 1
git-url-insteadof-setting FAIL non-zero exit status 128
recovery-from-iso.sh PASS
mir-bug-verification PASS

https://oem-share.canonical.com/partners/lyoncore/share/artifacts/oem-scripts/oem-scripts-0.99-2b0b120-in-linux-container-focal

Unmerged commits

2b0b120... by Alex Tu

recovery-from-iso.sh: remove set -x

040f634... by Alex Tu

rsync_iso.sh: new parameger, --allusb to flash all detected usb.

56821cb... by Alex Tu

fix the warning during umount

7249df2... by Alex Tu

removed -x and solidly remove tmp folders.

a931e27... by Alex Tu

take care the case of multiple partitions in target usb disk

045cd32... by Alex Tu

make sure the url of Jenkins is accessable

36a2a31... by Alex Tu

a handy script to create bootable usb with OEM image

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/recovery-from-iso.sh b/recovery-from-iso.sh
2index 843d9cc..cb20f90 100755
3--- a/recovery-from-iso.sh
4+++ b/recovery-from-iso.sh
5@@ -1,5 +1,5 @@
6 #!/bin/bash
7-set -ex
8+set -e
9
10 jenkins_job_for_iso=""
11 jenkins_job_build_no="lastSuccessfulBuild"
12diff --git a/rsync_iso.sh b/rsync_iso.sh
13new file mode 100755
14index 0000000..6ef5f2d
15--- /dev/null
16+++ b/rsync_iso.sh
17@@ -0,0 +1,180 @@
18+#!/bin/bash
19+set -ex
20+CD_PATH="$PWD"
21+JENKINS_B_NO="lastSuccessfulBuild"
22+
23+mnt_folder="$(mktemp -d -p "$PWD")"
24+target_folder="$(mktemp -d -p "$PWD")"
25+# $1: the target to unumount
26+do_unmount() {
27+ if [ -n "$1" ] && grep -q "$1" /proc/mounts; then
28+ sudo umount "$1"
29+ fi
30+}
31+clear_all() {
32+ do_unmount "$mnt_folder"
33+ do_unmount "$target_folder"
34+ rm -rf "$mnt_folder"
35+ rm -rf "$target_folder"
36+}
37+trap clear_all EXIT
38+
39+usage() {
40+cat << EOF
41+usage:
42+$0 options --iso [ISO file] | [file.iso] [<options to get iso from jenkins artifacts>]
43+$0 options [<options to get iso from jenkins artifacts>] [-s][-f]
44+
45+** IMPORTANT **
46+1. This script only works when only one USB disk plugged and it can be found in file manager,
47+2. USB disk will be wipe.
48+**
49+
50+This command will create a bootable usb disk by the ISO file,
51+It assume the iso file in the same folder where script be executed.
52+options:
53+ -h|--help Print this message
54+ -f|--folder The folder you would like to copy to root of target usb disk
55+ -i|--iso The iso file which be used to create a bootable USB disk.
56+ -a|--allusb Flash all detected usb disk.
57+ -s No interactive
58+
59+options to get iso from jenkins artifacts:
60+ --jenkins-url The url of target Jenkins
61+ --jenkins-job The job of target Jenkins
62+ --jenkins-job-number The buile number of target Jenkins job
63+
64+example:
65+ # Download iso from specific Jenkins job and build number then rsync the iso to usb disk without user interactive.
66+ $ rsync_iso.sh --jenkins-url jenkins.canonical.pw --jenkins-job pc-stella-cmit-focal-amd64 --jenkins-job-number 1569 -s
67+
68+ # rsync local iso and dump everything in ~/path-to-the-folder to usb disk with user interactive.
69+ $ rsync_iso.sh any-oem-image.iso -f ~/path-to-the-folder
70+
71+EOF
72+exit 1
73+}
74+
75+
76+while [ $# -gt 0 ]
77+do
78+ case "$1" in
79+ -h | --help)
80+ usage 0
81+ exit 0
82+ ;;
83+ -f |--folder)
84+ shift
85+ FOLDER=$1;
86+ [ -z "${FOLDER##/*}" ] || [ -z "${FOLDER##~/*}" ] || FOLDER="$CD_PATH/$FOLDER"
87+ if [ ! -d "$FOLDER" ];then
88+ echo "not exists $FOLDER"
89+ exit 1
90+ fi
91+ ;;
92+ -i | --iso)
93+ shift
94+ ISO=$1
95+ if [ ! -e "$ISO" ]; then
96+ echo "not exists $FOLDER"
97+ exit 1
98+ fi
99+ ;;
100+ -s)
101+ SILENCE="TRUE"
102+ ;;
103+ --jenkins-url)
104+ shift
105+ JENKINS_URL="$1"
106+ if ! ping -c 1 -W 1 "$JENKINS_URL"; then
107+ echo "[ERROR] Please make sure you can connect to Jenkins: $JENKINS_URL"
108+ exit 1
109+ fi
110+ ;;
111+ --jenkins-job)
112+ shift
113+ JENKINS_JOB="$1"
114+ ;;
115+ --jenkins-job-number)
116+ shift
117+ JENKINS_B_NO="$1"
118+ ;;
119+ -a | --all-usb)
120+ FLASH_ALL_USB="true"
121+ ;;
122+ *)
123+ if [ -z "${1##*.iso}" ]; then
124+ ISO=$1
125+ else
126+ usage
127+ fi
128+ esac
129+ shift
130+done
131+
132+if [ -n "$JENKINS_URL" ] && [ -n "$JENKINS_JOB" ]; then
133+ img_jenkins_out_url="ftp://$JENKINS_URL/jenkins_host/jobs/$JENKINS_JOB/builds/$JENKINS_B_NO/archive/out"
134+ img_name="$(wget -q "$img_jenkins_out_url/" -O - | grep -o 'href=.*iso"' | awk -F/ '{print $NF}' | tr -d \")"
135+ rm -f "$img_jenkins_out_url/$img_name".md5sum
136+ wget "$img_jenkins_out_url/$img_name".md5sum
137+ md5sum -c "$img_name".md5sum || wget "$img_jenkins_out_url"/"$img_name"
138+ md5sum -c "$img_name".md5sum || usage
139+ ISO="$img_name"
140+fi
141+
142+[ -z "$ISO" ] && usage
143+usb_mnt_list=$(mount | grep udisk | awk -F'type|on' '{print $1}' | sort)
144+usb_list=$(echo "$usb_mnt_list"| sed 's/[1-9]//' | uniq)
145+if [ -z "$usb_list" ]; then
146+ echo "can not find any USB be insertted!"
147+ usage
148+fi
149+
150+if [ "$(echo "$usb_list" | wc -l)" -gt 1 ] && [ "$FLASH_ALL_USB" != "true" ];then
151+ echo "There are more than one usb disk, please make sure only one usb disk connected to your host."
152+ echo "$usb_list"
153+ exit
154+else
155+ usb_targets=$(echo "$usb_list" | tr -d "[:blank:]")
156+fi
157+
158+sudo mount -o loop "$ISO" "$mnt_folder"
159+
160+while read -r disk; do
161+ do_unmount "$disk"
162+done <<<"$usb_mnt_list"
163+
164+for usb_target in $usb_targets; do
165+ echo "press any key to wipe $usb_target and create bootable usb..... "
166+ [ "$SILENCE" == "TRUE" ] || read -r
167+ sudo wipefs -a "$usb_target" 2>&1 || (echo "Wipefs failed"; usage)
168+ sudo parted -s "$usb_target" mktable msdos || (echo "parted mktable failed."; usage)
169+ sudo parted -s "$usb_target" mkpart primary 0% 100% || (echo "parted mkpart failed."; usage)
170+ usb_target="${usb_target}1"
171+ sleep 2
172+ sudo mkfs.vfat -F 32 "$usb_target"
173+
174+ target_folder="$(mktemp -d -p "$PWD")"
175+ sudo mount -t vfat -o gid=$UID,uid=$UID "$usb_target" "$target_folder"
176+
177+ echo "pless any keye to do ionice -c 3 rsync --delete -avP $mnt_folder/ $target_folder"
178+ [ "$SILENCE" == "TRUE" ] || read -r
179+ ionice -c 3 rsync --delete -avP "$mnt_folder"/ "$target_folder"
180+ iso_name=$(grep "<iso" "$target_folder/bto.xml" | sed 's/<[^>]*>//g')
181+ echo ================================================================
182+ echo "$iso_name"
183+ echo ================================================================
184+ pushd "$target_folder" || exit 1
185+ if [ -n "$FOLDER" ] && [ -d "$FOLDER" ]; then
186+ echo "pless any keye to do ionice -c 3 rsync -avP $FOLDER/ ."
187+ [ "$SILENCE" == "TRUE" ] || read -r
188+ ionice -c 3 rsync -avP "$FOLDER"/ .
189+ fi
190+ popd || exit 1
191+
192+ do_unmount "$target_folder"
193+ rm -rf "$target_folder"
194+ udisksctl mount -b "$usb_target"
195+done
196+clear_all
197+command -v zenity && zenity --timeout 600 --info --text="$0 $ISO done" &

Subscribers

People subscribed via source and target branches