Merge lp:~doanac/ubuntu-test-cases/statsd into lp:ubuntu-test-cases/touch

[Andy Doan]

Gets us the ability to send some statsd metrics the mega job in staging

[lp:~doanac/ubuntu-test-cases/statsd]

98. By Paul Larson on 2013-10-25

Use bootstrap option on all images

99. By Andy Doan on 2013-10-28

change order of operations in provision.sh

100. By Andy Doan on 2013-10-28

increase timeout waiting for device in reboot-and-wait

We see places in automation where the phablet-network script
starts printing tons of:

  error: device not found

The wait-for-device is coming back too fast. ie - its not really ready
yet.

101. By Paul Larson on 2013-10-29

Modify whoopsie-upload-all in the images we provision so that whoopsie
gets restarted after .crash files are processed to work around a bug,
and use the new utah option to to force running whoopsie-upload-all
at the end of the job

[Paul Larson (pwlars)]

Looks good, just need to remember to go back and add stats for production once it goes into play :)

review: Approve

[lp:~doanac/ubuntu-test-cases/statsd]

102. By Andy Doan on 2013-10-29

add statsd support for test-execution-service

[Evan (ev)]

Shouldn't STATSD_KEY only be set if {{statsd_key}} is set?

[Andy Doan (doanac)]

On 10/29/2013 03:25 PM, Evan Dandrea wrote:
> Shouldn't STATSD_KEY only be set if {{statsd_key}} is set?

setup_jenkins.py will set it to an empty string so "[ -z $STATSD_KEY ]"
should still evaluate properly for people's development setups.

[Evan (ev)]

Will it?

+export STATSD_KEY={{statsd_key}}.{{imagetype}}

That seems to imply that it will end up as STATSD_KEY=.{{imagetype}}

[Andy Doan (doanac)]

oh - so embarrassed and have no idea how this worked at home. must have forgotten to regenerate the job.

[Andy Doan (doanac)]

its fixed in trunk now.

[Evan (ev)]

:D

Excited for this.

=== modified file 'jenkins/setup_jenkins.py'
--- jenkins/setup_jenkins.py	2013-10-24 18:41:16 +0000
+++ jenkins/setup_jenkins.py	2013-10-25 18:22:20 +0000
@@ -121,6 +121,7 @@
             'trigger_url': device['trigger_url'],
             'imagetype': config_item['image-type'],
             'image_opt': config_item.get('IMAGE_OPT', ''),
+            'statsd_key': config_item.get('statsd-key', ''),
         }
         # a hack so we can use _get_job_name
         test = testconfig.Test('', fmt='{prefix}{series}-{imagetype}-{type}')

=== modified file 'jenkins/staging.py'
--- jenkins/staging.py	2013-10-24 22:09:13 +0000
+++ jenkins/staging.py	2013-10-25 18:22:20 +0000
@@ -11,6 +11,7 @@
     {
         'image-type': 'touch',
         'node-label': 'ashes',
+        'statsd-key': 'ubuntu-ci.test-execution-service.staging',
         'devices': [
             {
                 'name': 'mako-06 || mako-07 || mako-08',

=== modified file 'jenkins/templates/touch-smoke.xml.jinja2'
--- jenkins/templates/touch-smoke.xml.jinja2	2013-10-24 19:58:18 +0000
+++ jenkins/templates/touch-smoke.xml.jinja2	2013-10-25 18:22:20 +0000
@@ -83,6 +83,7 @@
 
 export ANDROID_SERIAL={{serial}}
 export IMAGE_TYPE={{imagetype}}
+export STATSD_KEY={{statsd_key}}.{{imagetype}}
 
 {{image_opt}}
 ${BZRDIR}/scripts/run-smoke

=== modified file 'scripts/run-smoke'
--- scripts/run-smoke	2013-10-24 19:58:18 +0000
+++ scripts/run-smoke	2013-10-25 18:22:20 +0000
@@ -11,6 +11,7 @@
 ## PACKAGES - debian packages to install
 ## APPS - autopilot apps to run or ALL
 ## TESTS - UTAH tests to run or ALL
+## STATSD_KEY - If provided metrics will be sent to statsd with this prefix
 
 set -e
 
@@ -19,39 +20,75 @@
 IMAGE_TYPE="${IMAGE_TYPE-touch}"
 RESDIR="${RESDIR-$(pwd)/clientlogs}"
 
-if [ -z $INSTALL_URL ] ; then
-	if [ "$PACKAGES" = "ALL" ] ; then
-		PACKAGES="$(${BASEDIR}/jenkins/testconfig.py packages -i $IMAGE_TYPE)"
-	fi
-
-	custom_args=""
-	set +x
-	for p in $PACKAGES ; do
-		custom_args="$custom_args -p $p"
-	done
-	for p in $PPAS ; do
-		custom_args="$custom_args -P $p"
-	done
-	set -x
-	${BASEDIR}/scripts/provision.sh -i ${IMAGE_TYPE} $custom_args
-else
-	${BASEDIR}/scripts/assert-image
-fi
-
-if [ "$APPS" = "ALL" ] ; then
-	APPS="$(${BASEDIR}/jenkins/testconfig.py apps -i $IMAGE_TYPE)"
-fi
-
-rc=0
-
-[ -z "$APPS" ] || APPS=${APPS} ${BASEDIR}/scripts/run-autopilot-tests.sh || rc=1
-
-if [ "$TESTS" = "ALL" ] ; then
-	TESTS="$(${BASEDIR}/jenkins/testconfig.py utah -i $IMAGE_TYPE)"
-fi
-
-for test in $TESTS ; do
-	echo == testing $test =======================================================
-	RESDIR=${RESDIR}/$test ${BASEDIR}/scripts/jenkins.sh -a $test || rc=1
-done
-exit $rc
+add_stat() {
+	[ -z $STATSD_KEY ] || ${BASEDIR}/scripts/statsd-cli "${STATSD_KEY}.$1:$2|$3"
+}
+
+timeit() {
+	rc=0
+	start=$(date +%s.%N)
+	$* || rc=1
+	end=$(date +%s.%N)
+	diff=$(echo "($end - $start) * 1000" | bc)
+	add_stat $1 $diff ms
+	return $rc
+}
+
+gaugeit() {
+	val=$(eval "echo \$$1" | wc -w)
+	add_stat $1 $val g
+}
+
+provision() {
+	if [ -z $INSTALL_URL ] ; then
+		if [ "$PACKAGES" = "ALL" ] ; then
+			PACKAGES="$(${BASEDIR}/jenkins/testconfig.py packages -i $IMAGE_TYPE)"
+		fi
+
+		custom_args=""
+		set +x
+		gaugeit PACKAGES
+		for p in $PACKAGES ; do
+			custom_args="$custom_args -p $p"
+		done
+		for p in $PPAS ; do
+			custom_args="$custom_args -P $p"
+		done
+		set -x
+		${BASEDIR}/scripts/provision.sh -i ${IMAGE_TYPE} $custom_args
+	else
+		${BASEDIR}/scripts/assert-image
+	fi
+}
+
+test_autopilot() {
+	if [ "$APPS" = "ALL" ] ; then
+		APPS="$(${BASEDIR}/jenkins/testconfig.py apps -i $IMAGE_TYPE)"
+	fi
+	gaugeit APPS
+	[ -z "$APPS" ] || APPS=${APPS} ${BASEDIR}/scripts/run-autopilot-tests.sh || return 1
+}
+
+test_utah() {
+	if [ "$TESTS" = "ALL" ] ; then
+		TESTS="$(${BASEDIR}/jenkins/testconfig.py utah -i $IMAGE_TYPE)"
+	fi
+	gaugeit TESTS
+
+	rc=0
+	for test in $TESTS ; do
+		echo == testing $test =======================================================
+		RESDIR=${RESDIR}/$test ${BASEDIR}/scripts/jenkins.sh -a $test || rc=1
+	done
+	return $rc
+}
+
+main() {
+	rc=0
+	timeit provision
+	timeit test_autopilot || rc=1
+	timeit test_utah || rc=1
+	return $rc
+}
+
+timeit main

=== added file 'scripts/statsd-cli'
--- scripts/statsd-cli	1970-01-01 00:00:00 +0000
+++ scripts/statsd-cli	2013-10-25 18:22:20 +0000
@@ -0,0 +1,18 @@
+#!/bin/bash
+
+set -e
+
+SERVER=${SERVER-"snakefruit.canonical.com"}
+PORT=${PORT-"10041"}
+
+send_msg() {
+	exec 3<>/dev/udp/$SERVER/$PORT
+
+	echo $* >&3
+
+	exec 3<&-
+	exec 3>&-
+}
+
+[ $# -gt 0 ] || (echo "Must provide statsd formatted string" && exit 1)
+send_msg $*