Merge lp:~pfalcon/linaro-license-protection/reshuffle-files-merge-contents into lp:~linaro-automation/linaro-license-protection/trunk

[Paul Sokolovsky]

Fix for current build breakage (in artifact transfer), more in commit message.

[Данило Шеган (danilo)]

This is definitely an improvement over the existing approach so it could land as-is.

However, it still has a race-condition that might become even more harmful since the script will succeed and perhaps "rm -rf" stuff that was pushed to snapshots only in between the "cp" evaluation and "rm" execution. I think the best thing we could do is add a simple lock file to the begining of the script, and loop for a certain time hoping that the lock is cleared, and bail out otherwise.

review: Needs Fixing

[Paul Sokolovsky (pfalcon)]

Rewrote to process one dir at time, as explicitly passed from Jenkins build.

[lp:~pfalcon/linaro-license-protection/reshuffle-files-merge-contents]

32. By Paul Sokolovsky on 2012-01-11

Rewrite to receive a directory to process explicitly from Jenkins.

This fixes race condition of removing files being transferred by concurrently
running builds in Jenkins.

33. By Paul Sokolovsky on 2012-01-11

Remove debug values.

[James Tunnicliffe (dooferlad)]

Looks good.

review: Approve

[Данило Шеган (danilo)]

Note that this approach might mean changes to the setup of the android-build-linaro and android-build-linaro-trigger users. IMO, it reduces the security of the setup, in the sense that we can influence what gets executed from a remote machine, whereas it was so far all implemented on the snapshots.l.o and we could only trigger it from the outside, without passing anything in.

That's why I suggested using a lock, but this could work as well.

=== modified file 'scripts/jenkins-post-www.sh'
--- scripts/jenkins-post-www.sh	2012-01-11 13:15:42 +0000
+++ scripts/jenkins-post-www.sh	2012-01-11 17:44:24 +0000
@@ -1,14 +1,24 @@
 #!/bin/sh
+set -x
 BASE_PATH=/home/android-build-linaro/android/.tmp
 TARGET_PATH=/srv3/snapshots.linaro.org/www/android/
-for dir in `ls $BASE_PATH`; do
-  if [ -d $BASE_PATH/$dir ]; then
-    username=`echo "$dir" | cut -d_ -f1`
-    buildname=`echo "$dir" | cut -d_ -f2-`
-    echo -n "Moving $BASE_PATH/$dir to $TARGET_PATH/~$username/$buildname... " &&
-    (mkdir -p "$TARGET_PATH/~$username/$buildname" && \
-     mv $BASE_PATH/"$dir"/* "$TARGET_PATH/~$username/$buildname/" && \
-     rmdir $BASE_PATH/"$dir" && \
+
+# Expected argument: ~username_jobname/buildno
+build_path="$1"
+
+# Paranoid security
+if echo "$build_path" | grep -q "\."; then
+    echo "No dots in build names please"
+    exit 1
+fi
+
+if [ -d $BASE_PATH/$build_path ]; then
+    job_dir=$(dirname $build_path)
+    username=`echo "$job_dir" | cut -d_ -f1`
+    jobname=`echo "$job_dir" | cut -d_ -f2-`
+    echo -n "Moving $BASE_PATH/$build_path to $TARGET_PATH/~$username/$jobname/... " &&
+    (mkdir -p "$TARGET_PATH/~$username/$jobname" && \
+     cp -a $BASE_PATH/"$build_path" "$TARGET_PATH/~$username/$jobname/" && \
+     rm -rf $BASE_PATH/"$build_path" && \
      echo "done")
-  fi
-done
+fi