Merge lp:~percona-core/percona-xtradb-cluster/bug-1405668-5.6 into lp:percona-xtradb-cluster

Proposed by Raghavendra D Prabhu on 2015-01-04
Status: Merged
Approved by: Alexey Kopytov on 2015-01-29
Approved revision: 950
Merge reported by: Raghavendra D Prabhu
Merged at revision: not available
Proposed branch: lp:~percona-core/percona-xtradb-cluster/bug-1405668-5.6
Merge into: lp:percona-xtradb-cluster
Diff against target: 290 lines (+64/-99)
3 files modified
percona-xtradb-cluster-tests/conf/conf20.cnf-node1 (+1/-1)
percona-xtradb-cluster-tests/conf/conf20.cnf-node2 (+1/-1)
scripts/wsrep_sst_xtrabackup-v2.sh (+62/-97)
To merge this branch: bzr merge lp:~percona-core/percona-xtradb-cluster/bug-1405668-5.6
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) 2015-01-04 Approve on 2015-01-29
Raghavendra D Prabhu (community) Approve on 2015-01-27
Review via email: mp+245519@code.launchpad.net
To post a comment you must log in.
Alexey Kopytov (akopytov) wrote :

I had to create a local clone of lp:~percona-core/percona-xtradb-cluster/bug-1405668-5.6 to see the actual changes.

This is the last revision I see in that branch:
---
revno: 949
committer: Raghavendra D Prabhu <email address hidden>
branch nick: bug-1405668-5.6
timestamp: Fri 2014-12-26 05:02:29 +0530
message:
  Fix the cpat for conf20 test
---

I'm not sure that revision is supposed to fix bug #1405668. Something probably went wrong.

review: Resubmit
Alexey Kopytov (akopytov) wrote :

Also, will there be a fix for 5.5?

This will not be fixed for 5.5.

Regarding 949, that was a fix in percona-xtradb-cluster-tests that I made.(since it is related)

Alexey Kopytov (akopytov) wrote :

I'm lost in these bugs. So this MP is supposed to fix bug #1405668. Which is not going to be fixed in 5.5.

But the revision I see in this MP doesn't mention bug #1405668. It mentions only bug #1405985. Which _is_ going to be fixed in 5.5.

Also there's a linked bug #1407599. Which is also going to be fixed in 5.5, but it's not mentioned in revision comments (only the fix itself is explained).

Can we have one revision, one MP per branch/bug rule? It gets a bit confusing and hard to track otherwise.

review: Resubmit

> Also there's a linked bug #1407599. Which is also going to be fixed in 5.5, but it's not mentioned in revision comments (only the fix itself is explained).

The commit message has been fixed to include this in bug subject. Its 5.5 MP is
here
https://code.launchpad.net/~percona-core/percona-xtradb-cluster/bug-1405668-5.5/+merge/247762

> I'm lost in these bugs. So this MP is supposed to fix bug #1405668. Which is
> not going to be fixed in 5.5.

That is correct. I have now added 5.5 to the bug, and will submit separate MP
for it.

> But the revision I see in this MP doesn't mention bug #1405668. It mentions
> only bug #1405985. Which _is_ going to be fixed in 5.5.

The revision 946 mentions it.

Anyway, same MP is used for 3 bugs since they all modify the same file (and the
only file) and are related.

review: Resubmit

Fixed.

review: Approve
Alexey Kopytov (akopytov) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'percona-xtradb-cluster-tests/conf/conf20.cnf-node1'
--- percona-xtradb-cluster-tests/conf/conf20.cnf-node1 2014-03-09 15:14:10 +0000
+++ percona-xtradb-cluster-tests/conf/conf20.cnf-node1 2015-01-27 18:38:33 +0000
@@ -15,5 +15,5 @@
15encrypt-key=6F3AD9F428143F133FD7D50D77D91EA415encrypt-key=6F3AD9F428143F133FD7D50D77D91EA4
1616
17[sst]17[sst]
18cpat='.*galera\.cache$\|.*sst_in_progress$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.borr$\|.*\.brn$'18cpat='.*galera\.cache$\|.*sst_in_progress$\|.*\.sst$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.borr$\|.*\.brn$'
19encrypt=119encrypt=1
2020
=== modified file 'percona-xtradb-cluster-tests/conf/conf20.cnf-node2'
--- percona-xtradb-cluster-tests/conf/conf20.cnf-node2 2014-03-09 15:14:10 +0000
+++ percona-xtradb-cluster-tests/conf/conf20.cnf-node2 2015-01-27 18:38:33 +0000
@@ -15,5 +15,5 @@
15encrypt-key=6F3AD9F428143F133FD7D50D77D91EA415encrypt-key=6F3AD9F428143F133FD7D50D77D91EA4
1616
17[sst]17[sst]
18cpat='.*galera\.cache$\|.*sst_in_progress$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.borr$\|.*\.brn$'18cpat='.*galera\.cache$\|.*sst_in_progress$\|.*\.sst$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.borr$\|.*\.brn$'
19encrypt=119encrypt=1
2020
=== modified file 'scripts/wsrep_sst_xtrabackup-v2.sh'
--- scripts/wsrep_sst_xtrabackup-v2.sh 2014-11-21 03:28:15 +0000
+++ scripts/wsrep_sst_xtrabackup-v2.sh 2015-01-27 18:38:33 +0000
@@ -40,13 +40,12 @@
40ttime=040ttime=0
41totime=041totime=0
42lsn=""42lsn=""
43incremental=0
44ecmd=""43ecmd=""
45rlimit=""44rlimit=""
46# Initially45# Initially
47stagemsg="${WSREP_SST_OPT_ROLE}"46stagemsg="${WSREP_SST_OPT_ROLE}"
48cpat=""47cpat=""
49speciald=048speciald=1
50ib_home_dir=""49ib_home_dir=""
51ib_log_dir=""50ib_log_dir=""
52ib_undo_dir=""51ib_undo_dir=""
@@ -302,8 +301,7 @@
302 progress=$(parse_cnf sst progress "")301 progress=$(parse_cnf sst progress "")
303 rebuild=$(parse_cnf sst rebuild 0)302 rebuild=$(parse_cnf sst rebuild 0)
304 ttime=$(parse_cnf sst time 0)303 ttime=$(parse_cnf sst time 0)
305 cpat=$(parse_cnf sst cpat '.*galera\.cache$\|.*sst_in_progress$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$')304 cpat=$(parse_cnf sst cpat '.*galera\.cache$\|.*sst_in_progress$\|.*\.sst$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$')
306 incremental=$(parse_cnf sst incremental 0)
307 ealgo=$(parse_cnf xtrabackup encrypt "")305 ealgo=$(parse_cnf xtrabackup encrypt "")
308 ekey=$(parse_cnf xtrabackup encrypt-key "")306 ekey=$(parse_cnf xtrabackup encrypt-key "")
309 ekeyfile=$(parse_cnf xtrabackup encrypt-key-file "")307 ekeyfile=$(parse_cnf xtrabackup encrypt-key-file "")
@@ -324,6 +322,11 @@
324 iapts=$(parse_cnf sst inno-apply-opts "")322 iapts=$(parse_cnf sst inno-apply-opts "")
325 impts=$(parse_cnf sst inno-move-opts "")323 impts=$(parse_cnf sst inno-move-opts "")
326 stimeout=$(parse_cnf sst sst-initial-timeout 100)324 stimeout=$(parse_cnf sst sst-initial-timeout 100)
325
326 if [[ $speciald -eq 0 ]];then
327 wsrep_log_error "sst-special-dirs equal to 0 is not supported, falling back to 1"
328 speciald=1
329 fi
327}330}
328331
329get_stream()332get_stream()
@@ -452,11 +455,7 @@
452 ss -p state listening "( sport = :$PORT )" | grep -qE 'socat|nc' && break455 ss -p state listening "( sport = :$PORT )" | grep -qE 'socat|nc' && break
453 sleep 0.2456 sleep 0.2
454 done457 done
455 if [[ $incremental -eq 1 ]];then 458 echo "ready ${ADDR}/${MODULE}//$sst_ver"
456 echo "ready ${ADDR}/${MODULE}/$lsn/$sst_ver"
457 else
458 echo "ready ${ADDR}/${MODULE}//$sst_ver"
459 fi
460}459}
461460
462check_extra()461check_extra()
@@ -571,7 +570,7 @@
571get_stream570get_stream
572get_transfer571get_transfer
573572
574if ${INNOBACKUPEX_BIN} /tmp --help | grep -q -- '--version-check'; then 573if ${INNOBACKUPEX_BIN} /tmp --help 2>/dev/null | grep -q -- '--version-check'; then
575 disver="--no-version-check"574 disver="--no-version-check"
576fi575fi
577576
@@ -622,15 +621,12 @@
622 fi621 fi
623 fi622 fi
624623
625 if [[ -n $lsn ]];then
626 INNOEXTRA+=" --incremental --incremental-lsn=$lsn "
627 fi
628624
629 check_extra625 check_extra
630626
631 wsrep_log_info "Streaming GTID file before SST"627 wsrep_log_info "Streaming GTID file before SST"
632628
633 echo "${WSREP_SST_OPT_GTID}" > "${MAGIC_FILE}"629 wsrep_log_info "${WSREP_SST_OPT_GTID}" > "${MAGIC_FILE}"
634630
635 ttcmd="$tcmd"631 ttcmd="$tcmd"
636632
@@ -712,26 +708,12 @@
712 [[ -e $SST_PROGRESS_FILE ]] && wsrep_log_info "Stale sst_in_progress file: $SST_PROGRESS_FILE"708 [[ -e $SST_PROGRESS_FILE ]] && wsrep_log_info "Stale sst_in_progress file: $SST_PROGRESS_FILE"
713 [[ -n $SST_PROGRESS_FILE ]] && touch $SST_PROGRESS_FILE709 [[ -n $SST_PROGRESS_FILE ]] && touch $SST_PROGRESS_FILE
714710
715 if [[ $speciald -eq 1 ]];then 711 ib_home_dir=$(parse_cnf mysqld innodb-data-home-dir "")
716 ib_home_dir=$(parse_cnf mysqld innodb-data-home-dir "")712 ib_log_dir=$(parse_cnf mysqld innodb-log-group-home-dir "")
717 ib_log_dir=$(parse_cnf mysqld innodb-log-group-home-dir "")713 ib_undo_dir=$(parse_cnf mysqld innodb-undo-directory "")
718 ib_undo_dir=$(parse_cnf mysqld innodb-undo-directory "")
719 if [[ -z $ib_home_dir && -z $ib_log_dir && -z $ib_undo_dir ]];then
720 speciald=0
721 fi
722 fi
723714
724 stagemsg="Joiner-Recv"715 stagemsg="Joiner-Recv"
725716
726 if [[ ! -e ${DATA}/ibdata1 ]];then
727 incremental=0
728 fi
729
730 if [[ $incremental -eq 1 ]];then
731 wsrep_log_info "Incremental SST enabled: NOT SUPPORTED yet"
732 lsn=$(grep to_lsn xtrabackup_checkpoints | cut -d= -f2 | tr -d ' ')
733 wsrep_log_info "Recovered LSN: $lsn"
734 fi
735717
736 sencrypted=1718 sencrypted=1
737 nthreads=1719 nthreads=1
@@ -760,11 +742,6 @@
760 tcmd+=" | $pcmd"742 tcmd+=" | $pcmd"
761 fi743 fi
762744
763 if [[ $incremental -eq 1 ]];then
764 BDATA=$DATA
765 DATA=$(mktemp -d)
766 MAGIC_FILE="${DATA}/${INFO_FILE}"
767 fi
768745
769 get_keys746 get_keys
770 if [[ $encrypt -eq 1 && $sencrypted -eq 1 ]];then747 if [[ $encrypt -eq 1 && $sencrypted -eq 1 ]];then
@@ -781,6 +758,13 @@
781 MAGIC_FILE="${STATDIR}/${INFO_FILE}"758 MAGIC_FILE="${STATDIR}/${INFO_FILE}"
782 recv_joiner $STATDIR "${stagemsg}-gtid" $stimeout 1759 recv_joiner $STATDIR "${stagemsg}-gtid" $stimeout 1
783760
761 if [[ -d ${DATA}/.sst ]];then
762 wsrep_log_info "WARNING: Stale temporary SST directory: ${DATA}/.sst from previous state transfer"
763 fi
764 mkdir -p ${DATA}/.sst
765 (recv_joiner $DATA/.sst "${stagemsg}-SST" 0 0) &
766 jpid=$!
767
784 if ! ps -p ${WSREP_SST_OPT_PARENT} &>/dev/null768 if ! ps -p ${WSREP_SST_OPT_PARENT} &>/dev/null
785 then769 then
786 wsrep_log_error "Parent mysqld process (PID:${WSREP_SST_OPT_PARENT}) terminated unexpectedly." 770 wsrep_log_error "Parent mysqld process (PID:${WSREP_SST_OPT_PARENT}) terminated unexpectedly."
@@ -791,48 +775,39 @@
791 then775 then
792 wsrep_log_info "Proceeding with SST"776 wsrep_log_info "Proceeding with SST"
793777
794 if [[ $speciald -eq 1 && -d ${DATA}/.sst ]];then 778
795 wsrep_log_info "WARNING: Stale temporary SST directory: ${DATA}/.sst from previous SST"779 wsrep_log_info "Cleaning the existing datadir and innodb-data/log directories"
796 fi780 find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+
797781
798 if [[ $incremental -ne 1 ]];then 782 tempdir=$(parse_cnf mysqld log-bin "")
799 if [[ $speciald -eq 1 ]];then 783 if [[ -n ${tempdir:-} ]];then
800 wsrep_log_info "Cleaning the existing datadir and innodb-data/log directories"784 binlog_dir=$(dirname $tempdir)
801 find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+785 binlog_file=$(basename $tempdir)
802 else 786 if [[ -n ${binlog_dir:-} && $binlog_dir != '.' && $binlog_dir != $DATA ]];then
803 wsrep_log_info "Cleaning the existing datadir"787 pattern="$binlog_dir/$binlog_file\.[0-9]+$"
804 find $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+788 wsrep_log_info "Cleaning the binlog directory $binlog_dir as well"
805 fi789 find $binlog_dir -maxdepth 1 -type f -regex $pattern -exec rm -fv {} 1>&2 \+ || true
806 tempdir=$(parse_cnf mysqld log-bin "")790 rm $binlog_dir/*.index || true
807 if [[ -n ${tempdir:-} ]];then791 fi
808 binlog_dir=$(dirname $tempdir)792 fi
809 binlog_file=$(basename $tempdir)793
810 if [[ -n ${binlog_dir:-} && $binlog_dir != '.' && $binlog_dir != $DATA ]];then794
811 pattern="$binlog_dir/$binlog_file\.[0-9]+$"795
812 wsrep_log_info "Cleaning the binlog directory $binlog_dir as well"796 TDATA=${DATA}
813 find $binlog_dir -maxdepth 1 -type f -regex $pattern -exec rm -fv {} 1>&2 \+ || true797 DATA="${DATA}/.sst"
814 rm $binlog_dir/*.index || true
815 fi
816 fi
817
818 else
819 wsrep_log_info "Removing existing ib_logfile files"
820 rm -f ${BDATA}/ib_logfile*
821 fi
822
823
824 if [[ $speciald -eq 1 ]];then
825 mkdir -p ${DATA}/.sst
826 TDATA=${DATA}
827 DATA="${DATA}/.sst"
828 fi
829798
830799
831 MAGIC_FILE="${DATA}/${INFO_FILE}"800 MAGIC_FILE="${DATA}/${INFO_FILE}"
832 recv_joiner $DATA "${stagemsg}-SST" 0 0801 wsrep_log_info "Waiting for SST streaming to complete!"
802 wait $jpid
833803
834 get_proc804 get_proc
835805
806 if [[ ! -s ${DATA}/xtrabackup_checkpoints ]];then
807 wsrep_log_error "xtrabackup_checkpoints missing, failed innobackupex/SST on donor"
808 exit 2
809 fi
810
836 # Rebuild indexes for compact backups811 # Rebuild indexes for compact backups
837 if grep -q 'compact = 1' ${DATA}/xtrabackup_checkpoints;then 812 if grep -q 'compact = 1' ${DATA}/xtrabackup_checkpoints;then
838 wsrep_log_info "Index compaction detected"813 wsrep_log_info "Index compaction detected"
@@ -904,11 +879,6 @@
904879
905 fi880 fi
906881
907 if [[ $incremental -eq 1 ]];then
908 # Added --ibbackup=xtrabackup_55 because it fails otherwise citing connection issues.
909 INNOAPPLY="${INNOBACKUPEX_BIN} $disver --defaults-file=${WSREP_SST_OPT_CONF} \
910 --ibbackup=xtrabackup_56 --apply-log $rebuildcmd --redo-only $BDATA --incremental-dir=${DATA} &>>${BDATA}/innobackup.prepare.log"
911 fi
912882
913 wsrep_log_info "Preparing the backup at ${DATA}"883 wsrep_log_info "Preparing the backup at ${DATA}"
914 timeit "Xtrabackup prepare stage" "$INNOAPPLY"884 timeit "Xtrabackup prepare stage" "$INNOAPPLY"
@@ -919,30 +889,25 @@
919 exit 22889 exit 22
920 fi890 fi
921891
922 if [[ $speciald -eq 1 ]];then 892 MAGIC_FILE="${TDATA}/${INFO_FILE}"
923 MAGIC_FILE="${TDATA}/${INFO_FILE}"893 set +e
924 set +e894 rm $TDATA/innobackup.prepare.log $TDATA/innobackup.move.log
925 rm $TDATA/innobackup.prepare.log $TDATA/innobackup.move.log895 set -e
926 set -e896 wsrep_log_info "Moving the backup to ${TDATA}"
927 wsrep_log_info "Moving the backup to ${TDATA}"897 timeit "Xtrabackup move stage" "$INNOMOVE"
928 timeit "Xtrabackup move stage" "$INNOMOVE"898 if [[ $? -eq 0 ]];then
929 if [[ $? -eq 0 ]];then 899 wsrep_log_info "Move successful, removing ${DATA}"
930 wsrep_log_info "Move successful, removing ${DATA}"900 rm -rf $DATA
931 rm -rf $DATA901 DATA=${TDATA}
932 DATA=${TDATA}902 else
933 else 903 wsrep_log_error "Move failed, keeping ${DATA} for further diagnosis"
934 wsrep_log_error "Move failed, keeping ${DATA} for further diagnosis"904 wsrep_log_error "Check ${DATA}/innobackup.move.log for details"
935 wsrep_log_error "Check ${DATA}/innobackup.move.log for details"
936 fi
937 fi905 fi
938906
939 if [[ $incremental -eq 1 ]];then
940 wsrep_log_info "Cleaning up ${DATA} after incremental SST"
941 [[ -d ${DATA} ]] && rm -rf ${DATA}
942 DATA=$BDATA
943 fi
944907
945 else 908 else
909 kill -KILL -$(ps -o pgid= $jpid | grep -o '[0-9]*')
910 rm -rf $DATA/.sst
946 wsrep_log_info "${IST_FILE} received from donor: Running IST"911 wsrep_log_info "${IST_FILE} received from donor: Running IST"
947 fi912 fi
948913

Subscribers

People subscribed via source and target branches

to all changes: