Merge lp:~sinzui/juju-release-tools/assemble-publish-purpose into lp:juju-release-tools

Proposed by Curtis Hovey
Status: Merged
Merged at revision: 60
Proposed branch: lp:~sinzui/juju-release-tools/assemble-publish-purpose
Merge into: lp:juju-release-tools
Diff against target: 513 lines (+174/-121)
2 files modified
assemble-streams.bash (+158/-101)
publish-public-tools.bash (+16/-20)
To merge this branch: bzr merge lp:~sinzui/juju-release-tools/assemble-publish-purpose
Reviewer Review Type Date Requested Status
Aaron Bentley (community) Approve
Review via email: mp+234753@code.launchpad.net

Description of the change

This branch adds support for proposed and devel purposes

./juju-release-tools/assemble-streams.bash testing 1.21-alpha2 ./new-tools
./juju-release-tools/assemble-streams.bash devel 1.21-alpha2 ./new-tools
./juju-release-tools/assemble-streams.bash proposed 1.21.0 ./new-tools
./juju-release-tools/assemble-streams.bash release 1.21.0 ./new-tools

I retracted an errant copy of 1.20.6 to my own tools like this.
./juju-release-tools/assemble-streams.bash -n -r 'juju-1.20.6*' devel IGNORE new-tools/
^ where -n means no-sync, -r means retract and IGNORE implies don't visit the archives.

Created new assemble-streams.bash based on assemble-public-tools:
1. Removed the juju stable and devel archive search because they have not been
   the source for tools for 9 months.
   Fixed https://bugs.launchpad.net/juju-release-tools/+bug/1341747
2. Documented the use of IGNORE as the release version to regenerate streams...
   ...do not try to retrieve tools and extract them when using IGNORE
   Removed the PRIVATE arg that wasn't used and overlaps with IGNORE
3. Create the whole path to tools and metadata. DEST_DIST can be
   juju-dist/ or juju-dist/purpose
4. Replaced all the early returns in functions with guards in the "main"
   calls so that is clear which functions are being used with IGNORE is
   set or GET_RELEASED_TOOL is false.
5. Removed the excludes list from s3cmd. It never worked. Fast syncing
   was achieved using a local cache. The source for the sync is determined
   by the purpose. Note the subtle rule for testing. Syncing happens
   directly in the tree that will be synced. This removed the need for
   the old new-tools/tools/releases dir. The detection of duplicate tools
   is maintained because testing is syncing from purposes, which must have
   all the stable tools released.
6. Made retract_tools() a common step so that is is well tested. Always
   retract when the purpose is testing. Also retract is RETRACT_GLOB is
   set using the -r option.
7. init_tools_maybe() started as a way to ensure Jerff can get s sane
   seed of data for devel and proposed. I used this to make my own for
   the CPCs, but I didn't need it. This function evolved to be the
   means to reset testing by first retracting the testing tools and data
   then seeding it with proposed.
8. Extracted archive_extra_ppc64_tool to make archive_tools easier to read...
   It might be deletable with the release of 1.21.0
9. Inlined the creation and cleanup to $WORK
19. Exit early when debs were search, but no new tools were found. This case
    is an error for testing, but results in a no-op for streams.canonical.com
11. Inlined JUJU_PATH, ARCH and lsb-release vars because they are now localised.
12. Moved signing key into options, factored out PRIVATE
13. Introduced purpose. DEST_DIST us made with it. Users of the script will
    have a cache for each purpose.
14. Exit early if new-tools does not exist. It is not created because the
    dir is a cache that users should not accidentally create.
15. only enter functions when we know we will use them...not more immediate
    exists because a env var is set.

Updated publish-public-tools
1. Updated the script to understand more purposes, simplified the logic
   because only "release" has a different root dir rule.
2. Removed EVENT because it was not needed when better phrasing was used.

To post a comment you must log in.
71. By Curtis Hovey

Fixed syntax errors.

72. By Curtis Hovey

Renamed retrieve_released_tools to sync_released_tools because Jerff has
redefined this function.

73. By Curtis Hovey

Delete debs without failing over.

74. By Curtis Hovey

Delete json when retracting because generate-tools appends to json; it does
not check if the tools exist.
Initialise proposed and devel streams.

75. By Curtis Hovey

Added -r option to pass a glob of tools to retract.
Allways retract all tools when the purpose is testing.

76. By Curtis Hovey

RETRACT_GLOB cannot be undefined.

Revision history for this message
Aaron Bentley (abentley) wrote :

Looks landable, but some comments below.

review: Approve
Revision history for this message
Curtis Hovey (sinzui) wrote :

I reported bug 1370246 about the magic number/validation problem

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'assemble-streams.bash'
--- assemble-streams.bash 2014-09-15 18:37:16 +0000
+++ assemble-streams.bash 2014-09-16 19:44:53 +0000
@@ -16,14 +16,15 @@
1616
17# These are the archives that are search for matching releases.17# These are the archives that are search for matching releases.
18UBUNTU_ARCH="http://archive.ubuntu.com/ubuntu/pool/universe/j/juju-core/"18UBUNTU_ARCH="http://archive.ubuntu.com/ubuntu/pool/universe/j/juju-core/"
19STABLE_ARCH="http://ppa.launchpad.net/juju/stable/ubuntu/pool/main/j/juju-core/"
20DEVEL_ARCH="http://ppa.launchpad.net/juju/devel/ubuntu/pool/main/j/juju-core/"
21ARM_ARCH="http://ports.ubuntu.com/pool/universe/j/juju-core/"19ARM_ARCH="http://ports.ubuntu.com/pool/universe/j/juju-core/"
22ALL_ARCHIVES="$UBUNTU_ARCH $STABLE_ARCH $DEVEL_ARCH $ARM_ARCH"20ALL_ARCHIVES="$UBUNTU_ARCH $ARM_ARCH"
2321
24if [ -f $JUJU_DIR/buildarchrc ]; then22if [[ -f $JUJU_DIR/buildarchrc ]]; then
25 source $JUJU_DIR/buildarchrc23 source $JUJU_DIR/buildarchrc
24 echo "Adding the build archives to list of archives to search."
26 ALL_ARCHIVES="$ALL_ARCHIVES $BUILD_STABLE_ARCH $BUILD_DEVEL_ARCH"25 ALL_ARCHIVES="$ALL_ARCHIVES $BUILD_STABLE_ARCH $BUILD_DEVEL_ARCH"
26else
27 echo "Only public archives will be searched."
27fi28fi
2829
2930
@@ -32,6 +33,8 @@
32 echo "usage: $0 $options RELEASE DESTINATION_DIRECTORY [SIGNING_KEY]"33 echo "usage: $0 $options RELEASE DESTINATION_DIRECTORY [SIGNING_KEY]"
33 echo " TEST_DEBS_DIR: The optional directory with testing debs."34 echo " TEST_DEBS_DIR: The optional directory with testing debs."
34 echo " RELEASE: The pattern (version) to match packages in the archives."35 echo " RELEASE: The pattern (version) to match packages in the archives."
36 echo " Use IGNORE when you want to regerenate metadata without"
37 echo " downloading debs and extracting new tools."
35 echo " DESTINATION_DIRECTORY: The directory to assemble the tools in."38 echo " DESTINATION_DIRECTORY: The directory to assemble the tools in."
36 echo " SIGNING_KEY: When provided, the metadata will be signed."39 echo " SIGNING_KEY: When provided, the metadata will be signed."
37 exit 140 exit 1
@@ -59,46 +62,84 @@
59 if [[ ! -d $DEST_DEBS ]]; then62 if [[ ! -d $DEST_DEBS ]]; then
60 mkdir $DEST_DEBS63 mkdir $DEST_DEBS
61 fi64 fi
62 if [[ ! -d $DEST_TOOLS ]]; then65 if [[ ! -d ${DEST_DIST}/tools/releases ]]; then
63 mkdir -p $DEST_TOOLS66 mkdir -p ${DEST_DIST}/tools/releases
64 fi67 fi
65 if [[ ! -d $DEST_DIST ]]; then68 if [[ ! -d ${DEST_DIST}/tools/streams/v1 ]]; then
66 mkdir $DEST_DIST69 mkdir -p ${DEST_DIST}/tools/streams/v1
67 fi70 fi
68}71}
6972
7073
71retrieve_released_tools() {74sync_released_tools() {
72 # Retrieve previously released tools to ensure the metadata continues75 # Retrieve previously released tools to ensure the metadata continues
73 # to work for historic releases.76 # to work for historic releases.
74 [[ $PRIVATE == "true" ]] && return 0
75 [[ $GET_RELEASED_TOOL="false" ]] && return 0
76 echo "Phase 2: Retrieving released tools."77 echo "Phase 2: Retrieving released tools."
77 # unsupported, stable, devel excludes to make sync fast.78 if [[ $PURPOSE == "release" ]]; then
78 if [[ $IS_TESTING == "true" ]]; then79 # The directory layout doesn't describe the release dir as "release".
79 excludes="--rexclude 'juju-1.1[5-7].*'"80 local source_dist="juju-dist"
81 elif [[ $PURPOSE == "testing" ]]; then
82 # The testing purpose copies from "proposed" because this stream.
83 # represents every version that can could be a stable release. Testing
84 # is volatile, it is always proposed + new tools that might be
85 # stable in the future.
86 local source_dist="juju-dist/proposed"
80 else87 else
81 excludes=""88 # The devel and proposed purposes use their own dirs for continuity.
82 fi89 local source_dist="juju-dist/$PURPOSE"
83 s3cmd -c $JUJU_DIR/s3cfg sync $excludes \90 fi
84 s3://juju-dist/tools/releases/ $DEST_TOOLS/91 s3cmd -c $JUJU_DIR/s3cfg sync \
85}92 s3://$source_dist/tools/releases/ $DEST_DIST/tools/releases/
8693}
8794
88retract_bad_tools() {95
89 echo "Phase 2.1: Retracting bad released tools."96retract_tools() {
90 bad_tools=$(find $DEST_TOOLS -name "juju-1.21-alpha1.*")97 echo "Phase 3: Reseting streams as needed."
91 for bad_tool in $bad_tools; do98 if [[ $PURPOSE == "testing" ]]; then
92 rm $bad_tool99 echo "Removing all testing tools and metadata to reset for testing."
93 done100 local RETRACT_GLOB="juju-*.tgz"
101 elif [[ -z "$RETRACT_GLOB" ]]; then
102 echo "Noting to reset"
103 return
104 fi
105 find ${DEST_DIST}/tools/releases -name "$RETRACT_GLOB" -delete
106 # juju metadata generate-tools appends to existing metadata; delete
107 # the current data to force a reset of all data, minus the deleted tools.
108 find ${DEST_DIST}/tools/streams/v1/ -type f -delete
109}
110
111
112init_tools_maybe() {
113 echo "Phase 4: Checking for $PURPOSE tools in the tree."
114 count=$(find $DESTINATION/juju-dist/tools/releases -name '*.tgz' | wc -l)
115 if [[ $((count)) < 400 ]]; then
116 echo "The tools in $DESTINATION/tools/releases looks incomplete"
117 echo "Data will be lost if metadata is regenerated."
118 exit 7
119 fi
120 count=$(find $DEST_DIST/tools/releases -name '*.tgz' | wc -l)
121 if [[ $PURPOSE == "proposed" && $((count)) == 0 ]]; then
122 echo "Seeding proposed with all release tools"
123 cp $DESTINATION/juju-dist/tools/releases/juju-*.tgz \
124 $DEST_DIST/tools/releases
125 elif [[ $PURPOSE == "devel" && $((count)) < 16 ]]; then
126 echo "Seeding devel with some release tools"
127 cp $DESTINATION/juju-dist/tools/releases/juju-1.20.5*.tgz \
128 $DEST_DIST/tools/releases
129 cp $DESTINATION/juju-dist/tools/releases/juju-1.20.7*.tgz \
130 $DEST_DIST/tools/releases
131 elif [[ $PURPOSE == "testing" && $((count)) < 16 ]]; then
132 echo "Seeding testing with all proposed tools"
133 cp $DESTINATION/juju-dist/tools/releases/juju-*.tgz \
134 $DEST_DIST/tools/releases
135 fi
94}136}
95137
96138
97retrieve_packages() {139retrieve_packages() {
98 # Retrieve the $RELEASE packages that contain jujud,140 # Retrieve the $RELEASE packages that contain jujud,
99 # or copy a locally built package.141 # or copy a locally built package.
100 [[ $PRIVATE == "true" ]] && return 0142 echo "Phase 5: Retrieving juju-core packages from archives"
101 echo "Phase 3: Retrieving juju-core packages from archives"
102 if [[ $IS_TESTING == "true" ]]; then143 if [[ $IS_TESTING == "true" ]]; then
103 for linked_file in $TEST_DEBS_DIR/juju-core_*.deb; do144 for linked_file in $TEST_DEBS_DIR/juju-core_*.deb; do
104 # We need the real file location which includes series and arch.145 # We need the real file location which includes series and arch.
@@ -168,11 +209,24 @@
168}209}
169210
170211
212archive_extra_ppc64_tool() {
213 # Hack to create ppc64 because it is not clear if juju wants
214 # this name instead of ppc64el.
215 tool="${DEST_DIST}/tools/releases/juju-${version}-${series}-ppc64.tgz"
216 if [[ ! -e $tool ]]; then
217 echo "Creating ppc64 from ppc64el: $tool"
218 tar cvfz $tool -C $change_dir jujud
219 added_tools[${#added_tools[@]}]="$tool"
220 echo "Created ${tool}."
221 fi
222}
223
224
171archive_tools() {225archive_tools() {
172 # Builds the jujud tgz for each series and arch.226 # Builds the jujud tgz for each series and arch.
173 [[ $PRIVATE == "true" ]] && return 0227 echo "Phase 6: Extracting jujud from packages and archiving tools."
174 echo "Phase 4: Extracting jujud from packages and archiving tools."
175 cd $DESTINATION228 cd $DESTINATION
229 WORK=$(mktemp -d)
176 mkdir ${WORK}/juju230 mkdir ${WORK}/juju
177 PACKAGES=$(find ${DEST_DEBS} -name "*.deb")231 PACKAGES=$(find ${DEST_DEBS} -name "*.deb")
178 for package in $PACKAGES; do232 for package in $PACKAGES; do
@@ -183,7 +237,7 @@
183 get_version $control_version237 get_version $control_version
184 get_series $control_version238 get_series $control_version
185 get_arch $control_file239 get_arch $control_file
186 tool="${DEST_TOOLS}/juju-${version}-${series}-${arch}.tgz"240 tool="${DEST_DIST}/tools/releases/juju-${version}-${series}-${arch}.tgz"
187 if [[ $arch == 'UNSUPPORTED' ]]; then241 if [[ $arch == 'UNSUPPORTED' ]]; then
188 echo "Skipping unsupported architecture $package"242 echo "Skipping unsupported architecture $package"
189 elif [[ -e $tool ]]; then243 elif [[ -e $tool ]]; then
@@ -209,28 +263,27 @@
209 tar cvfz $tool -C $change_dir jujud263 tar cvfz $tool -C $change_dir jujud
210 added_tools[${#added_tools[@]}]="$tool"264 added_tools[${#added_tools[@]}]="$tool"
211 echo "Created ${tool}."265 echo "Created ${tool}."
212 # Hack to create ppc64 because it is not clear if juju wants
213 # this name instead of ppc64el.
214 if [[ $arch == 'ppc64el' ]]; then266 if [[ $arch == 'ppc64el' ]]; then
215 tool="${DEST_TOOLS}/juju-${version}-${series}-ppc64.tgz"267 archive_extra_ppc64_tool
216 if [[ ! -e $tool ]]; then
217 echo "Creating ppc64 from ppc64el: $tool"
218 tar cvfz $tool -C $change_dir jujud
219 added_tools[${#added_tools[@]}]="$tool"
220 echo "Created ${tool}."
221 fi
222 fi268 fi
223 fi269 fi
224 rm -r ${WORK}/juju/*270 rm -r ${WORK}/juju/*
225 done271 done
226 if [[ $IS_TESTING == "true" ]]; then272 # The extracted files are no longer needed. Clean them up now.
227 set +u273 rm -r $WORK
228 if [[ -z "${added_tools[@]}" ]]; then274 # Exit early when debs were search, but no new tools were found.
229 echo "No tools were added from the built debs."275 if [[ -z "${added_tools[@]:-}" ]]; then
276 echo "No tools were added from the built debs."
277 cleanup
278 if [[ $IS_TESTING == "true" ]]; then
230 echo "The branch version may be out of date; $RELEASE is published?"279 echo "The branch version may be out of date; $RELEASE is published?"
231 exit 5280 exit 5
281 else
282 echo "No new tools were found for $RELEASE, exiting early."
283 echo "Use 'IGNORE' as the release version if you want to generate."
284 echo "streams from tools in $DEST_DIST/tools/releases"
285 exit 0
232 fi286 fi
233 set -u
234 fi287 fi
235}288}
236289
@@ -239,7 +292,9 @@
239 # Extract a juju-core that was found in the archives to run metadata.292 # Extract a juju-core that was found in the archives to run metadata.
240 # Match by release version and arch, prefer exact series, but fall back293 # Match by release version and arch, prefer exact series, but fall back
241 # to generic ubuntu.294 # to generic ubuntu.
242 echo "Phase 5.1: Using juju from a downloaded deb."295 echo "Using juju from a downloaded deb."
296 source /etc/lsb-release
297 ARCH=$(dpkg --print-architecture)
243 juju_cores=$(find $DEST_DEBS -name "juju-core_${RELEASE}*${ARCH}.deb")298 juju_cores=$(find $DEST_DEBS -name "juju-core_${RELEASE}*${ARCH}.deb")
244 juju_core=$(echo "$juju_cores" | grep $DISTRIB_RELEASE | head -1)299 juju_core=$(echo "$juju_cores" | grep $DISTRIB_RELEASE | head -1)
245 if [[ $juju_core == "" ]]; then300 if [[ $juju_core == "" ]]; then
@@ -254,8 +309,9 @@
254309
255generate_streams() {310generate_streams() {
256 # Create the streams metadata and organised the tree for later publication.311 # Create the streams metadata and organised the tree for later publication.
257 echo "Phase 5: Generating streams data."312 echo "Phase 7: Generating streams data."
258 cd $DESTINATION313 cd $DEST_DIST
314 JUJU_PATH=$(mktemp -d)
259 if [[ $RELEASE != "IGNORE" ]]; then315 if [[ $RELEASE != "IGNORE" ]]; then
260 extract_new_juju316 extract_new_juju
261 else317 else
@@ -264,16 +320,15 @@
264 fi320 fi
265 juju_version=$($JUJU_EXEC --version)321 juju_version=$($JUJU_EXEC --version)
266 echo "Using juju: $juju_version"322 echo "Using juju: $juju_version"
267 mkdir -p ${DEST_DIST}/tools/streams/v1
268 mkdir -p ${DEST_DIST}/tools/releases
269 cp $DEST_TOOLS/*tgz ${DEST_DIST}/tools/releases
270 JUJU_HOME=$JUJU_DIR PATH=$JUJU_BIN_PATH:$PATH \323 JUJU_HOME=$JUJU_DIR PATH=$JUJU_BIN_PATH:$PATH \
271 $JUJU_EXEC metadata generate-tools -d ${DEST_DIST}324 $JUJU_EXEC metadata generate-tools -d ${DEST_DIST}
325 rm -r $JUJU_PATH
272 echo "The tools are in ${DEST_DIST}."326 echo "The tools are in ${DEST_DIST}."
273}327}
274328
275329
276generate_mirrors() {330generate_mirrors() {
331 echo "Phase 8: Creating mirror josn."
277 short_now=$(date +%Y%m%d)332 short_now=$(date +%Y%m%d)
278 sed -e "s/NOW/$short_now/" ${SCRIPT_DIR}/mirrors.json.template \333 sed -e "s/NOW/$short_now/" ${SCRIPT_DIR}/mirrors.json.template \
279 > ${DEST_DIST}/tools/streams/v1/mirrors.json334 > ${DEST_DIST}/tools/streams/v1/mirrors.json
@@ -284,7 +339,7 @@
284339
285340
286sign_metadata() {341sign_metadata() {
287 echo "Phase 6: Signing metadata with $SIGNING_KEY."342 echo "Phase 9: Signing metadata with $SIGNING_KEY."
288 key_option="--default-key $SIGNING_KEY"343 key_option="--default-key $SIGNING_KEY"
289 gpg_options=""344 gpg_options=""
290 if [[ -n $SIGNING_PASSPHRASE_FILE ]]; then345 if [[ -n $SIGNING_PASSPHRASE_FILE ]]; then
@@ -310,31 +365,30 @@
310cleanup() {365cleanup() {
311 # Remove the debs and testing tools so that they are not reused in366 # Remove the debs and testing tools so that they are not reused in
312 # future runs of the script.367 # future runs of the script.
313 if [[ $PACKAGES != "" ]]; then368 find ${DEST_DEBS} -name "*.deb" -delete
314 rm ${DEST_DEBS}/*.deb
315 fi
316 if [[ $IS_TESTING == "true" ]]; then
317 for tool in "${added_tools[@]}"; do
318 rm $tool
319 done
320 fi
321 rm -r $WORK
322 rm -r $JUJU_PATH
323}369}
324370
325371
326declare -a added_tools372# Parse options and args.
327added_tools=()373RETRACT_GLOB=""
328374SIGNING_KEY=""
329IS_TESTING="false"375IS_TESTING="false"
330GET_RELEASED_TOOL="true"376GET_RELEASED_TOOL="true"
331while getopts "t:n" o; do377while getopts "r:s:t:n" o; do
332 case "${o}" in378 case "${o}" in
379 r)
380 RETRACT_GLOB=${OPTARG}
381 echo "Tools matching $RETRACT_GLOB will be removed from the data."
382 ;;
383 s)
384 SIGNING_KEY=${OPTARG}
385 echo "The streams will be signed with $SIGNING_KEY"
386 ;;
333 t)387 t)
334 TEST_DEBS_DIR=${OPTARG}388 TEST_DEBS_DIR=${OPTARG}
335 [[ -d $TEST_DEBS_DIR ]] || usage389 [[ -d $TEST_DEBS_DIR ]] || usage
336 IS_TESTING="true"390 IS_TESTING="true"
337 echo "# Assembling test tools from $TEST_DEBS_DIR"391 echo "Assembling testing tools from $TEST_DEBS_DIR"
338 ;;392 ;;
339 n)393 n)
340 GET_RELEASED_TOOL="false"394 GET_RELEASED_TOOL="false"
@@ -346,46 +400,49 @@
346 esac400 esac
347done401done
348shift $((OPTIND - 1))402shift $((OPTIND - 1))
349test $# -eq 2 || test $# -eq 3 || usage403test $# -eq 3 || usage
350404
351405PURPOSE=$1
352RELEASE=$1406if [[ ! $PURPOSE =~ ^(release|proposed|devel|testing)$ ]]; then
353DESTINATION=$(cd $2; pwd)407 echo "Invalid PURPOSE."
408 usage
409fi
410RELEASE=$2
411DESTINATION=$3
412if [[ ! -d "$3" ]]; then
413 echo "$3 is not a directory. Create it if you really mean to use it."
414 usage
415else
416 DESTINATION=$(cd $DESTINATION; pwd)
417fi
418
419
420# Configure paths, arch, and series
354DEST_DEBS="${DESTINATION}/debs"421DEST_DEBS="${DESTINATION}/debs"
355DEST_TOOLS="${DESTINATION}/tools/releases"422if [[ $PURPOSE == "release" ]]; then
356DEST_DIST="${DESTINATION}/juju-dist"423 DEST_DIST="$DESTINATION/juju-dist"
357if [[ $IS_TESTING == "true" ]]; then
358 DEST_DIST="${DESTINATION}/juju-dist-testing"
359fi
360if [[ -d $DEST_DIST ]]; then
361 rm -r $DEST_DIST
362fi
363
364SIGNING_KEY=""
365PRIVATE="false"
366EXTRA=${3:-""}
367if [[ $EXTRA == "PRIVATE" ]]; then
368 PRIVATE="true"
369 echo "Skipping release tools and packages."
370else424else
371 SIGNING_KEY=$EXTRA425 DEST_DIST="$DESTINATION/juju-dist/$PURPOSE"
372fi426fi
373427declare -a added_tools
374PACKAGES=""428added_tools=()
375WORK=$(mktemp -d)429
376JUJU_PATH=$(mktemp -d)430
377ARCH=$(dpkg --print-architecture)431# Main.
378source /etc/lsb-release
379
380check_deps432check_deps
381build_tool_tree433build_tool_tree
382retrieve_released_tools434if [[ $GET_RELEASED_TOOL == "true" ]]; then
383retract_bad_tools435 sync_released_tools
384retrieve_packages436fi
385archive_tools437retract_tools
438init_tools_maybe
439if [[ $RELEASE != "IGNORE" ]]; then
440 retrieve_packages
441 archive_tools
442fi
386generate_streams443generate_streams
387generate_mirrors
388if [[ $SIGNING_KEY != "" ]]; then444if [[ $SIGNING_KEY != "" ]]; then
445 generate_mirrors
389 sign_metadata446 sign_metadata
390fi447fi
391cleanup448cleanup
392449
=== modified file 'publish-public-tools.bash'
--- publish-public-tools.bash 2014-06-19 02:54:23 +0000
+++ publish-public-tools.bash 2014-09-16 19:44:53 +0000
@@ -12,8 +12,10 @@
1212
13usage() {13usage() {
14 echo "usage: $0 PURPOSE DIST_DIRECTORY DESTINATIONS"14 echo "usage: $0 PURPOSE DIST_DIRECTORY DESTINATIONS"
15 echo " PURPOSE: 'release' or 'testing'"15 echo " PURPOSE: 'release', 'proposed', 'devel', or 'testing'"
16 echo " release installs tools/ at the top of juju-dist/tools."16 echo " release installs tools/ at the top of juju-dist/tools."
17 echo " proposed installs tools/ at the top of juju-dist/proposed/tools."
18 echo " devel installs tools/ at the top of juju-dist/devel/tools."
17 echo " testing installs tools/ at juju-dist/testing/tools."19 echo " testing installs tools/ at juju-dist/testing/tools."
18 echo " DIST_DIRECTORY: The directory to the assembled tools."20 echo " DIST_DIRECTORY: The directory to the assembled tools."
19 echo " This is the juju-dist dir created by assemble-public-tools.bash."21 echo " This is the juju-dist dir created by assemble-public-tools.bash."
@@ -47,9 +49,9 @@
47 if [[ $PURPOSE == "release" ]]; then49 if [[ $PURPOSE == "release" ]]; then
48 local destination="s3://juju-dist/"50 local destination="s3://juju-dist/"
49 else51 else
50 local destination="s3://juju-dist/testing/"52 local destination="s3://juju-dist/$PURPOSE/"
51 fi53 fi
52 echo "Phase 1: $EVENT to AWS."54 echo "Phase 1: Publishing $PURPOSE to AWS."
53 s3cmd -c $JUJU_DIR/s3cfg sync --exclude '*mirror*' \55 s3cmd -c $JUJU_DIR/s3cfg sync --exclude '*mirror*' \
54 ${JUJU_DIST}/tools $destination56 ${JUJU_DIST}/tools $destination
55}57}
@@ -61,9 +63,9 @@
61 if [[ $PURPOSE == "release" ]]; then63 if [[ $PURPOSE == "release" ]]; then
62 local destination="tools"64 local destination="tools"
63 else65 else
64 local destination="testing/tools"66 local destination="$PURPOSE/tools"
65 fi67 fi
66 echo "Phase 2: $EVENT to canonistack."68 echo "Phase 2: Publishing $PURPOSE to canonistack."
67 source $JUJU_DIR/canonistacktoolsrc69 source $JUJU_DIR/canonistacktoolsrc
68 cd $JUJU_DIST/tools/releases/70 cd $JUJU_DIST/tools/releases/
69 ${SCRIPT_DIR}/swift_sync.py $destination/releases/ *.tgz71 ${SCRIPT_DIR}/swift_sync.py $destination/releases/ *.tgz
@@ -77,9 +79,9 @@
77 if [[ $PURPOSE == "release" ]]; then79 if [[ $PURPOSE == "release" ]]; then
78 local destination="tools"80 local destination="tools"
79 else81 else
80 local destination="testing/tools"82 local destination="$PURPOSE/tools"
81 fi83 fi
82 echo "Phase 3: $EVENT to HP Cloud."84 echo "Phase 3: Publishing $PURPOSE to HP Cloud."
83 source $JUJU_DIR/hptoolsrc85 source $JUJU_DIR/hptoolsrc
84 cd $JUJU_DIST/tools/releases/86 cd $JUJU_DIST/tools/releases/
85 ${SCRIPT_DIR}/swift_sync.py $destination/releases/ *.tgz87 ${SCRIPT_DIR}/swift_sync.py $destination/releases/ *.tgz
@@ -93,9 +95,9 @@
93 if [[ $PURPOSE == "release" ]]; then95 if [[ $PURPOSE == "release" ]]; then
94 local destination="release"96 local destination="release"
95 else97 else
96 local destination="testing"98 local destination="$PURPOSE"
97 fi99 fi
98 echo "Phase 4: $EVENT to Azure."100 echo "Phase 4: Publishing $PURPOSE to Azure."
99 source $JUJU_DIR/azuretoolsrc101 source $JUJU_DIR/azuretoolsrc
100 ${SCRIPT_DIR}/azure_publish_tools.py publish $destination ${JUJU_DIST}102 ${SCRIPT_DIR}/azure_publish_tools.py publish $destination ${JUJU_DIST}
101}103}
@@ -107,9 +109,9 @@
107 if [[ $PURPOSE == "release" ]]; then109 if [[ $PURPOSE == "release" ]]; then
108 local destination="tools"110 local destination="tools"
109 else111 else
110 local destination="testing/tools"112 local destination="$PURPOSE/tools"
111 fi113 fi
112 echo "Phase 5: $EVENT to Joyent."114 echo "Phase 5: Publishing $PURPOSE to Joyent."
113 source $JUJU_DIR/joyentrc115 source $JUJU_DIR/joyentrc
114 cd $JUJU_DIST/tools/releases/116 cd $JUJU_DIST/tools/releases/
115 ${SCRIPT_DIR}/manta_sync.py $destination/releases/ *.tgz117 ${SCRIPT_DIR}/manta_sync.py $destination/releases/ *.tgz
@@ -125,7 +127,7 @@
125 else127 else
126 local destination=$STREAMS_TESTING_DEST128 local destination=$STREAMS_TESTING_DEST
127 fi129 fi
128 echo "Phase 6: $EVENT to streams.canonical.com."130 echo "Phase 6: Publishing $PURPOSE to streams.canonical.com."
129 source $JUJU_DIR/streamsrc131 source $JUJU_DIR/streamsrc
130 rsync -avzh $JUJU_DIST/ $destination132 rsync -avzh $JUJU_DIST/ $destination
131}133}
@@ -137,7 +139,7 @@
137test $# -eq 3 || usage139test $# -eq 3 || usage
138140
139PURPOSE=$1141PURPOSE=$1
140if [[ $PURPOSE != "release" && $PURPOSE != "testing" ]]; then142if [[ ! $PURPOSE =~ ^(release|proposed|devel|testing)$ ]]; then
141 echo "Invalid PURPOSE."143 echo "Invalid PURPOSE."
142 usage144 usage
143fi145fi
@@ -154,12 +156,6 @@
154 usage156 usage
155fi157fi
156158
157if [[ $PURPOSE == "release" ]]; then
158 EVENT="Release"
159else
160 EVENT="Testing"
161fi
162
163159
164check_deps160check_deps
165publish_to_aws161publish_to_aws
@@ -168,4 +164,4 @@
168publish_to_azure164publish_to_azure
169publish_to_joyent165publish_to_joyent
170publish_to_streams166publish_to_streams
171echo "$EVENT data published to all CPCs."167echo "Published $PURPOSE data to all CPCs."

Subscribers

People subscribed via source and target branches