Merge lp:~danilo/linaro-license-protection/bug-919165 into lp:~linaro-automation/linaro-license-protection/trunk

[Данило Шеган]

This documents the set-up of the entire click-through system, with the focus
on android-build file sync system (which is the most complicated bit).

It should help us in understanding where things need to change when something
fails.

It's missing a lot, but should be a start. I'll also add the wiki page
pointing at this README file.

[Paul Sokolovsky (pfalcon)]

line 10: "This code provides such license protection."

Probably worth to reword this to mention that this is only the core code, and it depends on integration with license-protection bits in other codebases.

line 18: "It makes have use of mod_rewrite" - typo

line 20: "Currently, all directories/files containing either 'origen' or 'snowball' in the name"

I would be more specific and say "in the URL" - and I assume that's the case, i.e. any file below dir with "origen" would be protected.

line 61: "into appropriate build directories."

As we changed it to operate on a single build dir at time, probably worth to use singular there.

One thing not mentioned is that android-build.linaro.org has exception in Apache rules to be able to: 1) mirror binary blobs for use in the build process; 2) to query file lists and lava integration data for the frontend display. (Assuming it's ok to mention that in public README, but other relevant details are mentioned already).

Otherwise, looks good from the side I'm familiar with it, and pretty comprehensive.

[Paul Sokolovsky (pfalcon)]

Ok, let me add explicit vote for this.

review: Needs Fixing

[Данило Шеган (danilo)]

Very useful input Paul, thanks. I believe I fixed all the issues you mentioned, perhaps in a slightly different way in a few cases.

[lp:~danilo/linaro-license-protection/bug-919165]

42. By Данило Шеган on 2012-02-24

Implement suggestions from Paul.

[Paul Sokolovsky (pfalcon)]

Looks good, thanks!

review: Approve

=== added file 'README'
--- README	1970-01-01 00:00:00 +0000
+++ README	2012-02-24 10:36:18 +0000
@@ -0,0 +1,84 @@
+Linaro downloads license protection
+===================================
+
+Linaro builds sometimes contain "binary blobs"—pieces of binary-only code which enable extra features like accelerated graphics or multimedia.  These pieces are distributed under a separate license, and downloading images or collections containing them requires some sort of license protection.
+
+This code provides such license protection on the hosting web server: other parts of infrastructure need to properly integrate with it (see eg. android-build.linaro.org section).  At the moment, it's implemented as a click-through license acceptance on the Apache side.
+
+To ensure and prove this keeps working, we are also using automated tests with this code.
+
+
+Setup
+-----
+
+This tree is the base for snapshots.linaro.org and releases.linaro.org www roots as served by Apache.  It makes heavy use of mod_rewrite from within the .htaccess file.
+
+Currently, all directories/files containing either 'origen' or 'snowball' in the URL path are protected with appropriate license (Samsung or ST-E) click-through.
+
+
+Technical details
+-----------------
+
+releases.linaro.org
+...................
+
+Root directory for releases.linaro.org includes .htaccess and licenses/*.
+It has mod_rewrite enabled.
+
+It allows a few hard-coded hosts to bypass the click-through protection, by their IPs:
+ * android-build.linaro.org (50.17.250.69)
+ * validation.linaro.org (213.123.120.124)
+
+Currently hosted on mombin.canonical.com.
+
+
+snapshots.linaro.org
+....................
+
+Same basic set-up as releases.linaro.org.
+
+Further, to allow android-build.linaro.org to push to snapshots.linaro.org
+www area directly, we set-up two different users on the system with SSH keys:
+
+ * android-build-linaro:
+
+   chrooted to /srv3/snapshots.linaro.org/www/android/ and allows sftp
+   access to push files over;  home directory
+
+ * android-build-linaro-trigger:
+
+   limited to executing only a single command through
+   /etc/ssh/user-authorized-keys/android-build-linaro-trigger
+
+     command="/home/android-build-linaro-trigger/scripts/trigger-android-build-linaro.sh ${SSH_ORIGINAL_COMMAND#* }"
+
+   (this passes the arguments received from the remote end as well)
+
+   This script does a sudo to 'android-build-linaro' and then runs
+
+     /home/android-build-linaro/scripts/jenkins-post-sftp.sh
+
+   script which moves files from /android/.tmp/ into appropriate
+   build directory.  It expects "build_name/build_number" as command
+   line parameters.
+
+android-build.linaro.org
+........................
+
+Runs Jenkins and uses SFTP plugin to access the above two users.  Private keys live in
+
+  /home/ubuntu/snapshots-sync/snapshots-sync — android-build-linaro
+  /home/ubuntu/snapshots-sync/snapshots-filemove — android-build-linaro-trigger
+
+To ensure serialization of steps, and allow LAVA submission, these happen as
+build steps, and not as publishing steps.
+
+
+Tests
+-----
+
+Testing infrastructure is based on 'testrepository' and requires at least
+Python and Apache2:
+
+  $ testr init
+  $ testr run