Merge lp:~ben-kaehne/charms/trusty/rabbitmq-server/statsfix into lp:charms/trusty/rabbitmq-server

[Benjamin Kaehne]

Modifying the stats collections script to use a temporary staging file to write to prior to making stats available in a known file.

This is to eliminate race conditions with scripts which may be reading this file.

[Review Queue (review-queue)]

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/3531/

review: Needs Fixing (automated testing)

[Review Queue (review-queue)]

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/3481/

review: Needs Fixing (automated testing)

[Ryan Beisner (1chb1n)]

Apologies for the delay on your merge proposal review. Thank you for your work on this.

The rabbitmq-server charm development moved to a Git/Gerrit flow [1] in Feb 2016. The launchpad repos are effectively now read-only. There is a temporary sync in place from git to LP, but those LP locations will be removed as soon as deployment and test automation and tooling surrounding this charm are confirmed to no longer be dependent on the LP branches.

Please see the announcement and the development process [2], then re-propose these changes against "openstack/charm-rabbitmq-server" via the new process.

Also note that the true source of truth for this charm is in OpenStack's cgit [3], but that those changes are sync'd to github [4] in near-real-time. One can git clone either, then follow the same dev process [2] as OpenStack to propose the change.

Thanks again.

[1] https://lists.ubuntu.com/archives/juju/2016-February/006606.html
[2] http://docs.openstack.org/infra/manual/developers.html
[3] https://git.openstack.org/cgit/openstack/charm-rabbitmq-server
[4] https://github.com/openstack/charm-rabbitmq-server

review: Needs Resubmitting

[Andrew McLeod (admcleod)]

Hi Ben,

Running the tests manually (bundletester -vFl DEBUG -e AWS --no-destroy) shows that there are initial errors with make lint and make test that look like venv hasn't been set up properly:

http://pastebin.ubuntu.com/17398837/

Then I hit this error:

DEBUG:runner:2016-06-16 17:29:41,453 connect_amqp_by_unit DEBUG: Connecting to amqp on 54.171.15.113:5672 (rabbitmq-server/0) as testuser1...
DEBUG:runner:No handlers could be found for logger "pika.adapters.base_connection"
DEBUG:runner:amqp connection failed to 54.171.15.113:5672 as testuser1 (1)
DEBUG:runner:Exit Code: 1
    014-basic-precise-icehouse ERROR
DEBUG:runner:call ['/tmp/bundletester-zYF3Ob/rabbitmq-server/tests/015-basic-trusty-icehouse'] (cwd: /tmp/bundletester-zYF3Ob/rabbitmq-server)

There are quite a few test failures, really too many to detail in this report, so heres the pastebin:

http://pastebin.ubuntu.com/17426675/

review: Needs Fixing

Unmerged revisions

111. By Benjamin Kaehne on 2016-04-04

Modifying the stats collections script to use a temporary staging file to write to prior to making stats available in a known file.

This is to eliminate race conditions with scripts which may be reading this file.

=== modified file 'scripts/collect_rabbitmq_stats.sh'
--- scripts/collect_rabbitmq_stats.sh	2015-10-22 13:24:30 +0000
+++ scripts/collect_rabbitmq_stats.sh	2016-04-05 05:27:09 +0000
@@ -31,8 +31,10 @@
 fi
 DATA_DIR="/var/lib/rabbitmq/data"
 DATA_FILE="${DATA_DIR}/$(hostname -s)_queue_stats.dat"
+TMP_DATA_FILE="${DATA_DIR}/.$(hostname -s)_queue_stats.dat.tmp"
 LOG_DIR="/var/lib/rabbitmq/logs"
 RABBIT_STATS_DATA_FILE="${DATA_DIR}/$(hostname -s)_general_stats.dat"
+TMP_RABBIT_STATS_DATA_FILE="${DATA_DIR}/.$(hostname -s)_general_stats.dat.tmp"
 NOW=$(date +'%s')
 HOSTNAME=$(hostname -s)
 MNESIA_DB_SIZE=$(du -sm /var/lib/rabbitmq/mnesia | cut -f1)
@@ -43,11 +45,13 @@
 if [ ! -d $LOG_DIR ]; then
     mkdir -p $LOG_DIR
 fi
-echo "#Vhost Name Messages_ready Messages_unacknowledged Messages Consumers Memory Time" > $DATA_FILE
+echo "#Vhost Name Messages_ready Messages_unacknowledged Messages Consumers Memory Time" > $TMP_DATA_FILE
 /usr/sbin/rabbitmqctl -q list_vhosts | \
 while read VHOST; do
     /usr/sbin/rabbitmqctl -q list_queues -p $VHOST name messages_ready messages_unacknowledged messages consumers memory | \
-    awk "{print \"$VHOST \" \$0 \" $(date +'%s') \"}" >> $DATA_FILE 2>${LOG_DIR}/list_queues.log
+    awk "{print \"$VHOST \" \$0 \" $(date +'%s') \"}" >> $TMP_DATA_FILE 2>${LOG_DIR}/list_queues.log
 done
-echo "mnesia_size: ${MNESIA_DB_SIZE}@$NOW" > $RABBIT_STATS_DATA_FILE
-echo "rss_size: ${RABBIT_RSS}@$NOW" >> $RABBIT_STATS_DATA_FILE
+echo "mnesia_size: ${MNESIA_DB_SIZE}@$NOW" > $TMP_RABBIT_STATS_DATA_FILE
+echo "rss_size: ${RABBIT_RSS}@$NOW" >> $TMP_RABBIT_STATS_DATA_FILE
+mv $TMP_DATA_FILE $DATA_FILE
+mv $TMP_RABBIT_STATS_DATA_FILE $RABBIT_STATS_DATA_FILE