Merge lp:~nijaba/charm-tools/peer-scp into lp:~charmers/charm-tools/trunk

[Nick Barcet]

Add ch_peer_copy function

[lp:~nijaba/charm-tools/peer-scp]

102. By Nick Barcet on 2011-12-17

better result handling

103. By Nick Barcet on 2011-12-17

handle sudo for auto-tests (trying)

[Nick Barcet (nijaba)]

Trying to build this branch in a ppa, it seems that I have a problem with the test_peer.sh script. My current test script for ch_peer_copy requires to be executed as root so that I can start a second sshd server.

Is there a way to run a sshd server without being root (not only startiang, but able to do authentications)? I first tried to do this for a bit, but gave up.

Or is it possible to have sudo in dh_auto_test?
This is what I tried in the current branch but the build environment returned "tests/helpers/helpers.sh: 21: sudo: not found" see: https://launchpadlibrarian.net/87786598/buildlog_ubuntu-oneiric-i386.charm-tools_0.3%2B92-0~103~oneiric1_FAILEDTOBUILD.txt.gz

Mentoring welcome :)

review: Needs Information

[Kapil Thangavelu (hazmat)]

Excerpts from Nick Barcet's message of Sun Dec 18 11:48:24 UTC 2011:
> Review: Needs Information
>
> Trying to build this branch in a ppa, it seems that I have a problem with the test_peer.sh script. My current test script for ch_peer_copy requires to be executed as root so that I can start a second sshd server.
>
> Is there a way to run a sshd server without being root (not only startiang, but able to do authentications)? I first tried to do this for a bit, but gave up.
>
> Or is it possible to have sudo in dh_auto_test?
> This is what I tried in the current branch but the build environment returned "tests/helpers/helpers.sh: 21: sudo: not found" see: https://launchpadlibrarian.net/87786598/buildlog_ubuntu-oneiric-i386.charm-tools_0.3%2B92-0~103~oneiric1_FAILEDTOBUILD.txt.gz
>
> Mentoring welcome :)

an alternative to openssh for testing might be using twisted's ssh server impl.
it requres a little setup, but you can setup arbitrary users/server keys.

Some examples here.

http://twistedmatrix.com/documents/current/conch/examples/

hth,
kapil

[Nick Barcet (nijaba)]

On 12/19/2011 03:20 PM, Kapil Thangavelu wrote:
> an alternative to openssh for testing might be using twisted's ssh server impl.
> it requres a little setup, but you can setup arbitrary users/server keys.
>
> Some examples here.
>
> http://twistedmatrix.com/documents/current/conch/examples/

The problem is that after a summary search, I can't find an example
implementation that support scp nor rsync, and for just a text of a
basic charm helper, writing one seems like an overkill. Thanks for the
pointer anyway.

Nick

[lp:~nijaba/charm-tools/peer-scp]

104. By Nick Barcet on 2011-12-21

adding test-sudo_peer.sh

105. By Nick Barcet on 2011-12-21

finally able to run sshd as a normal user, thanks to Spamaps

[Nick Barcet (nijaba)]

ok, this time test_peer.sh work without being root! Thanks Clint!

[lp:~nijaba/charm-tools/peer-scp]

106. By Nick Barcet on 2011-12-21

this test is uneeded now

[Clint Byrum (clint-fewbar)]

Nick, thats great, it runs as non-root now, :)

And so commences the review.

[0] This is my bug, but you've exposed it. :) The way I've done the trap in helpers/test_net.sh is a bit unfair to other tests, and should be corrected. We should manage the EXIT trap a little more carefully then. At the bottom of each test_* script that has a trap, we need to call the registered exit trap and clear it, as the way this runs right now, the simple http server that test_net runs, keeps running after the tests run, because your new trap overrides the old one.

So, it should look something like this at the bottom of tests/helpers/test_net.sh

trap - EXIT
cleanup_temp

And at the bottom of test_peer.sh

trap - EXIT
cleanup_peer

[1] ssh-keygen can hang *forever* on stock EC2 instances. They have very poor entropy. Perhaps we should recommend haveged with charm-helpers-sh, which helps keep entropy up on instances (though i believe the entropy is of "moderate" quality). Otherwise another option is to make use of the SSH host key since the instance boot process already takes great pains to make that generation fast.

[2] I think we should consider adding a flag to make this key exchange permanent, and by default remove the key from authorized_keys after the copy. I'm not sure I really like having all nodes in a service be able to login as root to all other nodes in the service. For now, its ok, but maybe open this as a bug in charm-tools so we remember to consider it.

[3] in the tests, use 'output' not 'echo' so we get uniform output.

[4] Why the 'CHARM_HELPER_VERSION' in net.sh ?

I'd say the only blocker to merging is [0] so we don't leave a stale background python process hanging around for each test run. The rest are just nit picks, not real problems that make things bad.

review: Needs Fixing

[lp:~nijaba/charm-tools/peer-scp]

107. By Nick Barcet on 2011-12-21

Deregistering traps at end of tests

108. By Nick Barcet on 2011-12-21

changing test result to output function

109. By Nick Barcet on 2011-12-21

removing useless versioning

[Nick Barcet (nijaba)]

[0] fixed

[1] ok to recommend haveged, but since the packaging is not part of that branch, it will be another merge request

[2] I disagree. In the branch I am working on I am reusing this to allow for subsequent sync/copies within config-changed hooks, which I would not be able to do without having the keys in place since I am not in a 2 way channel anymore.

[3] fixed

[4] removed, did not make sense in the end

review: Needs Resubmitting

[Clint Byrum (clint-fewbar)]

Ok, LGTM

To follow up with [2] from above, I'd like the default to be "safer" meaning only when juju says "its ok to copy stuff" will these keys work. Another option would be to always copy as a non-root user. I agree with you that there will be times that this does not work, but I'd like users to have to choose less security. Users almost never choose more, until they're already burned.

Approved, please push this into lp:charm-tools, newbie charmer! :)

review: Approve

=== modified file 'helpers/sh/net.sh'
--- helpers/sh/net.sh	2011-12-18 21:54:05 +0000
+++ helpers/sh/net.sh	2011-12-21 19:02:23 +0000
@@ -181,3 +181,4 @@
 	echo $CHECK_IP
 	return 0
 }
+

=== modified file 'helpers/sh/peer.sh'
--- helpers/sh/peer.sh	2011-12-16 11:36:30 +0000
+++ helpers/sh/peer.sh	2011-12-21 19:02:23 +0000
@@ -83,3 +83,175 @@
 {
     echo "`ch_unit_id $JUJU_UNIT_NAME`"
 }
+
+
+##
+# ch_peer_copy [-r|--rsync][-p <port>][-o "<opt>"] sourcepath1 destpath1 [... sourcepathN destpathN]
+# ch_peer_scp [-r][-p <port>][-o "<opt>"] sourcepath1 destpath1 [... sourcepathN destpathN]
+# ch_peer_rsync [-p <port>] sourcepath1 destpath1 [... sourcepathN destpathN]
+#
+# distribute a list of file to all slave of the peer relation
+#
+# param
+#  -r            recursive scp copy (scp only, always on with rsync)
+#  -p <port>     destination port to connect to
+#  -o "<opt>"    any pathttrough options to the copy util
+#  --rsync       use rsync instead of scp
+#  sourcepath    path from which to copy (do not specify host, it will always
+#                be coming from the leader of the peer relation)
+#  destpath      path to which to copy (do not specify host, it will always
+#                be the slaves of the peer relation)
+#
+# returns
+#  1      when copy is complete on the slave side
+#  FALSE  if an error was encountered
+#
+# This executes in multiple passes between the leader and each peer
+#
+#    LEADER                      SLAVE
+#       <------------------- set scp-hostname
+# set scp-ssh-key -----------------> 
+#                            save shh-keys
+#       <------------------- set scp-ssh-key-saved
+# do file copy
+# set scp-copy-done --------------->
+#
+# This function is idempotent and should be called for each JOINED or 
+# CHANGED event for slave or leader in the peer relation exactly the same way
+
+alias ch_peer_scp=ch_peer_copy
+alias ch_peer_rsync='ch_peer_copy --rsync'
+ch_peer_copy() {
+  local USAGE="ERROR in $*
+USAGE: ch_peer_scp [-r][-p <port>][-o \"<opt>\"] sourcepath1 destpath1 [... sourcepathN destpathN]
+USAGE: ch_peer_rsync [-p <port>][-o \"<opt>\"] sourcepath1 destpath1 [... sourcepathN destpathN]"
+  local ssh_key_p="$HOME/.ssh"
+  local result=0
+  
+  if [ $# -eq 0 ]; then
+    juju-log "$USAGE"
+    juju-log "ch_peer_copy: please provide at least one argument (path)"
+    exit 1
+  fi
+
+  ## LEADER ##
+  
+  if ch_peer_i_am_leader ; then
+    juju-log "ch_peer_copy: This is our leader"
+    
+    local remote=`relation-get scp-hostname`
+    local ssh_key_saved=`relation-get scp-ssh-key-saved`
+    if [ -z $remote ]; then
+      juju-log "ch_peer_copy: We do not have a remote hostname yet"
+      remote=0
+    fi  
+  
+    local scp_options="-o StrictHostKeyChecking=no -B"
+    local rsync_options=""
+    local paths=""
+    local copy_command="scp"
+    
+    while [ "$#" -gt 0 ]; 
+    do
+      case "$1" in
+      "-r") # scp recure
+        scp_options="$scp_options -r"
+        shift
+        ;;
+      "-p") # port number
+        shift
+        scp_options="$scp_options -P $1"
+        rsync_options="$rsync_options -e 'ssh -p $1 -o StrictHostKeyChecking=no'"
+        shift
+        ;;
+      "-o") # passthrough option
+        shift
+        scp_options="$scp_options $1"
+        rsync_options="$rsync_options $1"
+        shift
+        ;;
+      "--rsync") # rsync secure (-e ssh)
+        copy_command="rsync"
+        shift
+        ;;
+      "$0")
+        shift
+        ;;
+      *) # should be a pair of file
+        if [ -e "`echo "$1" | sed 's/\*$//'`" ]; then
+          local sourcep="$1"
+          shift
+          paths="$paths $sourcep $USER@$remote:$1"
+          juju-log "ch_peer_copy: paths found: $sourcep -> $1"
+        else
+          juju-log "ch_peer_copy: unknown option, skipping: $1"
+        fi
+        shift
+        ;;
+      esac
+    done
+    if [ ! -n "$paths" ]; then
+      juju-log "$USAGE"
+      juju-log "ch_peer_copy: please provide at least one path"
+      exit 1
+    fi
+    
+    if [ -n $remote ]; then
+      # We know where to send file to
+      
+      case $ssh_key_saved in
+      1) # ssh keys have been save, let's copy
+        if [ x"$copy_command" = x"rsync" ]; then
+          scp_options=$rsync_options
+        fi
+        juju-log "ch_peer_copy: $copy_command $scp_options $paths"
+        eval "$copy_command $scp_options $paths"
+        relation-set scp-copy-done=1
+        ;;
+        
+      *) # we need to first distribute our ssh key files
+        juju-log "ch_peer_copy: distributing ssh key"
+        if [ ! -f "$ssh_key_p/id_rsa" ]; then
+          ssh-keygen -q -N '' -t rsa -b 2048 -f $ssh_key_p/id_rsa
+        fi
+        relation-set scp-ssh-key="`cat $ssh_key_p/id_rsa.pub`"
+        ;;
+        
+      esac
+    fi 
+    
+  ## REMOTE ##
+      
+  else # Not the leader
+    juju-log "ch_peer_copy: This is a slave"
+ 
+    local scp_copy_done=`relation-get scp-copy-done`
+    local scp_ssh_key="`relation-get scp-ssh-key`"
+ 
+    if [ -n "$scp_copy_done" ] && [ $scp_copy_done = 1 ]; then
+      juju-log "ch_peer_copy: copy done, thanks"
+      result=1
+    else
+      if [ -n "$scp_ssh_key" ]; then
+        juju-log "ssh key dir: $ssh_key_p"
+        mkdir -p $ssh_key_p
+        chmod 700 $ssh_key_p
+        if ! grep -q -F "$scp_ssh_key" $ssh_key_p/authorized_keys ; then
+          juju-log "ch_peer_copy: saving ssh key $scp_ssh_key"
+          echo "$scp_ssh_key" >> $ssh_key_p/authorized_keys
+          relation-set scp-ssh-key-saved=1
+        else
+          juju-log "ch_peer_copy: ssh keys already saved, thanks"
+          relation-set scp-ssh-key-saved=1
+        fi
+        chmod 600 "$ssh_key_p/authorized_keys"
+      else
+        juju-log "ch_peer_copy: ssh_keys not set yet, later"
+        relation-set scp-hostname=`unit-get private-address`
+      fi 
+    fi 
+  fi 
+  juju-log "ch_peer_copy: returning: $result"
+  echo $result
+}
+

=== modified file 'tests/helpers/helpers.sh'
--- tests/helpers/helpers.sh	2011-12-16 21:03:04 +0000
+++ tests/helpers/helpers.sh	2011-12-21 19:02:23 +0000
@@ -15,3 +15,4 @@
 for i in $test_home/test_*.sh ; do
   . $i
 done
+

=== modified file 'tests/helpers/test_net.sh'
--- tests/helpers/test_net.sh	2011-12-18 22:05:29 +0000
+++ tests/helpers/test_net.sh	2011-12-21 19:02:23 +0000
@@ -150,3 +150,6 @@
 cmp $f $temp_srv_dir/testdata.txt
 rm $f
 echo PASS
+
+trap - EXIT
+cleanup_temp

=== added file 'tests/helpers/test_peer.sh'
--- tests/helpers/test_peer.sh	1970-01-01 00:00:00 +0000
+++ tests/helpers/test_peer.sh	2011-12-21 19:02:23 +0000
@@ -0,0 +1,274 @@
+#!/bin/sh
+if [ -z "$test_home" ] ; then
+    test_home=`dirname $0`
+    test_home=`readlink -f $test_home`
+fi
+
+[ "$LIB_SOURCED" = "1" ] || . $test_home/lib.sh
+
+set -ue
+
+#mock relation-list
+alias relation-list=mock_relation_list
+mock_relation_list()
+{
+    [ -z $CH_MASTER ] && let CH_MASTER=1
+    
+    case $CH_MASTER in
+    1)
+        echo "TEST/2
+TEST/3
+TEST/4"
+        ;;
+    0)
+        echo "TEST/1
+TEST/3
+TEST/4"
+        ;;
+    esac
+        
+}
+
+#Save juju-log for debugging
+CH_TEMPLOG="/tmp/tmp-juju-log"
+echo "" > $CH_TEMPLOG
+output "creating test-log in $CH_TEMPLOG"
+alias juju-log=mock_juju_log
+mock_juju_log()
+{
+    echo "$1" >> $CH_TEMPLOG
+}
+
+#mock unit-get
+alias unit-get=mock_unit_get
+mock_unit_get()
+{
+   case $1 in
+   "private-address")
+       echo "127.0.0.1"
+       ;;
+   *)
+       echo "UNDEFINED"
+       ;;
+   esac
+}
+
+#mock relation-set/get
+alias relation-set=mock_relation_set
+alias relation-get=mock_relation_get
+CH_scp_hostname=""
+CH_scp_ssh_key_saved=""
+CH_scp_ssh_key=""
+CH_scp_copy_done=""
+mock_relation_set()
+{
+    juju-log "mock_relation_set: $1"
+    CH_varname=`echo $1 | cut -d= -f1`
+    CH_value=`echo $1 | sed 's/^[^=]*=//'`
+    case $CH_varname in
+    "scp-hostname")
+        CH_scp_hostname=$CH_value
+        ;;
+    "scp-ssh-key-saved")
+        CH_scp_ssh_key_saved=$CH_value
+        ;;
+    "scp-ssh-key")
+        CH_scp_ssh_key="$CH_value"
+        ;;
+    "scp-copy-done")
+        CH_scp_copy_done=$CH_value
+        ;;
+    *)
+        juju-log "mock_relation_set: unknow var $CH_varname"
+        ;;
+    esac
+}
+mock_relation_get()
+{
+    case $1 in
+    "scp-hostname")
+        echo $CH_scp_hostname
+        juju-log "mock_relation_get: $1 = $CH_scp_hostname"
+        ;;
+    "scp-ssh-key-saved")
+        echo $CH_scp_ssh_key_saved
+        juju-log "mock_relation_get: $1 = $CH_scp_ssh_key_saved"
+        ;;
+    "scp-ssh-key")
+        echo "$CH_scp_ssh_key"
+        juju-log "mock_relation_get: $1 = $CH_scp_ssh_key"
+        ;;
+    "scp-copy-done")
+        echo $CH_scp_copy_done
+        juju-log "mock_relation_get: $1 = $CH_scp_copy_done"
+        ;;
+    *)
+        juju-log "mock_relation_get: unknow var $1"
+        ;;
+    esac
+}
+
+
+
+# mock sshd server
+CH_TEMPDIR="/tmp/`mktemp -d "/tmp/juju-helpers-tmp.XXXXXXX"`"
+mkdir -p $CH_TEMPDIR/sourcedir/
+mkdir -p $CH_TEMPDIR/destdir/
+mkdir -p $CH_TEMPDIR/$HOME/
+[ ! `which pwgen` ] && apt-get -y install pwgen
+pwgen > $CH_TEMPDIR/sourcedir/testfile0
+pwgen > $CH_TEMPDIR/sourcedir/testfile1
+pwgen > $CH_TEMPDIR/sourcedir/testfile
+CH_portnum=28822
+ssh-keygen -t rsa -b 1024 -N "" -h -f $CH_TEMPDIR/my_host_key > /dev/null 2>&1 
+/usr/sbin/sshd -o PidFile=$CH_TEMPDIR/sshd.pid -o HostKey=$CH_TEMPDIR/my_host_key -p $CH_portnum > /dev/null 2>&1 
+cleanup_peer()
+{
+    echo "Cleaning up..."
+    kill -9 `cat $CH_TEMPDIR/sshd.pid`
+    rm -rf $CH_TEMPDIR
+    unalias juju-log
+    unalias relation-set
+    unalias relation-get
+    unalias unit-get
+    unalias relation-list
+}
+trap cleanup_peer EXIT
+
+. $HELPERS_HOME/peer.sh
+
+start_test ch_unit_id...
+JUJU_UNIT_NAME="TEST/1"
+[ ! `ch_unit_id $JUJU_UNIT_NAME` -eq 1 ] && return 1
+CH_bad="badarg"
+ch_unit_id $CH_bad > /dev/null || return 1
+output PASS
+
+start_test ch_my_unit_id...
+[ ! `ch_my_unit_id` -eq  1 ] && return 1
+output PASS
+
+start_test ch_peer_i_am_leader...
+JUJU_REMOTE_UNIT="TEST/3"
+JUJU_UNIT_NAME="TEST/2"
+CH_MASTER=1
+ch_peer_i_am_leader && return 1 || :
+JUJU_UNIT_NAME="TEST/1"
+ch_peer_i_am_leader || return 1 && :
+output PASS
+
+start_test ch_peer_leader...
+[ "`ch_peer_leader`" = "TEST/1" ] ||  return 1
+[ `ch_peer_leader --id` -eq 1 ] || return 1
+JUJU_UNIT_NAME="TEST/3"
+[ "`ch_peer_leader`" = "TEST/2" ] || return 1
+[ `ch_peer_leader --id` -eq 2 ] || return 1
+output PASS
+
+start_test "ch_peer_scp -r..."
+#save authorized keys and known_hosts before modifications
+touch $HOME/.ssh/authorized_keys
+cp $HOME/.ssh/authorized_keys $HOME/.ssh/authorized_keys_saved
+touch $HOME/.ssh/known_hosts
+cp $HOME/.ssh/known_hosts $HOME/.ssh/known_hosts_saved
+for i in 1 2 3 
+do
+    #slave relation joined
+    JUJU_UNIT_NAME="TEST/2"  
+    JUJU_REMOTE_UNIT="TEST/1"
+    CH_MASTER=0
+    ch_peer_scp -r -p $CH_portnum -o "-q" "$CH_TEMPDIR/sourcedir/*" "$CH_TEMPDIR/destdir/" > $CH_TEMPDIR/result 2> /dev/null
+    [ `cat $CH_TEMPDIR/result` = 1 ] && break
+    #master relation joined
+    JUJU_UNIT_NAME="TEST/1"  
+    JUJU_REMOTE_UNIT="TEST/2"
+    CH_MASTER=1
+    ch_peer_scp -r -p $CH_portnum -o "-q" "$CH_TEMPDIR/sourcedir/*" "$CH_TEMPDIR/destdir/" > $CH_TEMPDIR/result 2> /dev/null
+    [ `cat $CH_TEMPDIR/result` = 1 ] && break
+done
+[ ! -e $CH_TEMPDIR/destdir/ ] && output "dir not copied" && exit 1
+[ ! -e $CH_TEMPDIR/destdir/testfile0 ] && output "file1 not copied" && exit 1
+[ ! -e $CH_TEMPDIR/destdir/testfile1 ] && output "file2 not copied" && exit 1
+CH_t1=`md5sum $CH_TEMPDIR/sourcedir/testfile0 | cut -d" " -f1`
+CH_t2=`md5sum $CH_TEMPDIR/destdir/testfile0 | cut -d" " -f1`
+[ ! "$CH_t1" = "$CH_t2" ] && output "md5sum differ" && exit 1
+rm -rf $CH_TEMPDIR/destdir/*
+#restore authorized_keys & known_hosts
+mv $HOME/.ssh/authorized_keys_saved $HOME/.ssh/authorized_keys
+mv $HOME/.ssh/known_hosts_saved $HOME/.ssh/known_hosts
+output PASS
+
+start_test "ch_peer_rsync..."
+CH_scp_hostname=""
+CH_scp_ssh_key_saved=""
+CH_scp_ssh_key=""
+CH_scp_copy_done=""
+#save authorized keys and known_hosts before modifications
+touch $HOME/.ssh/authorized_keys
+cp $HOME/.ssh/authorized_keys $HOME/.ssh/authorized_keys_saved
+touch $HOME/.ssh/known_hosts
+cp $HOME/.ssh/known_hosts $HOME/.ssh/known_hosts_saved
+for i in 1 2 3 
+do
+    #slave relation joined
+    JUJU_UNIT_NAME="TEST/2"  
+    JUJU_REMOTE_UNIT="TEST/1"
+    CH_MASTER=0
+    ch_peer_rsync -p $CH_portnum -o "-azq" "$CH_TEMPDIR/sourcedir/*" "$CH_TEMPDIR/destdir/" > $CH_TEMPDIR/result 2> /dev/null
+    [ `cat $CH_TEMPDIR/result` = 1 ] && break
+    #master relation joined
+    JUJU_UNIT_NAME="TEST/1"  
+    JUJU_REMOTE_UNIT="TEST/2"
+    CH_MASTER=1
+    ch_peer_rsync -p $CH_portnum -o "-azq" "$CH_TEMPDIR/sourcedir/*" "$CH_TEMPDIR/destdir/" > $CH_TEMPDIR/result 2> /dev/null
+    [ `cat $CH_TEMPDIR/result` = 1 ] && break
+done
+[ ! -e $CH_TEMPDIR/destdir/ ] && output"dir not copied" && exit 1
+[ ! -e $CH_TEMPDIR/destdir/testfile0 ] && output "file1 not copied" && exit 1
+[ ! -e $CH_TEMPDIR/destdir/testfile1 ] && output "file2 not copied" && exit 1
+CH_t1=`md5sum $CH_TEMPDIR/sourcedir/testfile0 | cut -d" " -f1`
+CH_t2=`md5sum $CH_TEMPDIR/destdir/testfile0 | cut -d" " -f1`
+[ ! "$CH_t1" = "$CH_t2" ] && output "md5sum differ" && exit 1
+rm -rf $CH_TEMPDIR/destdir/*
+#restore authorized_keys & known_hosts
+mv $HOME/.ssh/authorized_keys_saved $HOME/.ssh/authorized_keys
+mv $HOME/.ssh/known_hosts_saved $HOME/.ssh/known_hosts
+output PASS
+
+start_test "ch_peer_scp..."
+CH_scp_hostname=""
+CH_scp_ssh_key_saved=""
+CH_scp_ssh_key=""
+CH_scp_copy_done=""
+#save authorized keys and known_hosts before modifications
+touch $HOME/.ssh/authorized_keys
+cp $HOME/.ssh/authorized_keys $HOME/.ssh/authorized_keys_saved
+touch $HOME/.ssh/known_hosts
+cp $HOME/.ssh/known_hosts $HOME/.ssh/known_hosts_saved
+for i in 1 2 3 
+do
+    #slave relation joined
+    JUJU_UNIT_NAME="TEST/2" 
+    JUJU_REMOTE_UNIT="TEST/1"
+    CH_MASTER=0
+    ch_peer_scp -p $CH_portnum -o "-q" $CH_TEMPDIR/sourcedir/testfile $CH_TEMPDIR/destdir/ > $CH_TEMPDIR/result 2> /dev/null
+    [ `cat $CH_TEMPDIR/result` = 1 ] && break
+    #master relation joined
+    JUJU_UNIT_NAME="TEST/1" 
+    JUJU_REMOTE_UNIT="TEST/2"
+    CH_MASTER=1
+    ch_peer_scp -p $CH_portnum -o "-q" $CH_TEMPDIR/sourcedir/testfile $CH_TEMPDIR/destdir/ > $CH_TEMPDIR/result 2> /dev/null
+    [ `cat $CH_TEMPDIR/result` = 1 ] && break
+done
+[ ! -e $CH_TEMPDIR/destdir/testfile ] && output"file not copied" && exit 1
+CH_t1=`md5sum $CH_TEMPDIR/sourcedir/testfile | cut -d" " -f1`
+CH_t2=`md5sum $CH_TEMPDIR/destdir/testfile | cut -d" " -f1`
+[ ! "$CH_t1" = "$CH_t2" ] && output "md5sum differ" && exit 1
+rm -rf $CH_TEMPDIR/destdir/*
+#restore authorized_keys & known_hosts
+mv $HOME/.ssh/authorized_keys_saved $HOME/.ssh/authorized_keys
+mv $HOME/.ssh/known_hosts_saved $HOME/.ssh/known_hosts
+output PASS
+
+trap - EXIT
+cleanup_peer

=== removed file 'tests/helpers/test_peer.sh'
--- tests/helpers/test_peer.sh	2011-12-16 21:03:04 +0000
+++ tests/helpers/test_peer.sh	1970-01-01 00:00:00 +0000
@@ -1,47 +0,0 @@
-#!/bin/sh
-if [ -z "$test_home" ] ; then
-    test_home=`dirname $0`
-    test_home=`readlink -f $test_home`
-fi
-
-[ "$LIB_SOURCED" = "1" ] || . $test_home/lib.sh
-
-set -ue
-
-#mock relation-list
-alias relation-list=relation_list
-relation_list()
-{
-    echo "TEST/2
-TEST/3
-TEST/4"
-}
-
-. $HELPERS_HOME/peer.sh
-
-start_test ch_unit_id...
-JUJU_UNIT_NAME="TEST/1"
-[ ! `ch_unit_id $JUJU_UNIT_NAME` -eq 1 ] && return 1
-CH_bad="badarg"
-ch_unit_id $CH_bad > /dev/null || return 1
-echo PASS
-
-start_test ch_my_unit_id...
-[ ! `ch_my_unit_id` -eq  1 ] && return 1
-echo PASS
-
-start_test ch_peer_i_am_leader...
-JUJU_REMOTE_UNIT="TEST/3"
-JUJU_UNIT_NAME="TEST/2"
-ch_peer_i_am_leader && return 1 || :
-JUJU_UNIT_NAME="TEST/1"
-ch_peer_i_am_leader || return 1 && :
-echo PASS
-
-start_test ch_peer_leader...
-[ "`ch_peer_leader`" = "TEST/1" ] ||  return 1
-[ `ch_peer_leader --id` -eq 1 ] || return 1
-JUJU_UNIT_NAME="TEST/3"
-[ "`ch_peer_leader`" = "TEST/2" ] || return 1
-[ `ch_peer_leader --id` -eq 2 ] || return 1
-echo PASS