Merge lp:~percona-core/percona-xtradb-cluster/bug-1405668-5.6 into lp:percona-xtradb-cluster

Proposed by Raghavendra D Prabhu on 2015-01-04
Status: Merged
Approved by: Alexey Kopytov on 2015-01-29
Approved revision: 950
Merge reported by: Raghavendra D Prabhu
Merged at revision: not available
Proposed branch: lp:~percona-core/percona-xtradb-cluster/bug-1405668-5.6
Merge into: lp:percona-xtradb-cluster
Diff against target: 290 lines (+64/-99)
3 files modified
percona-xtradb-cluster-tests/conf/conf20.cnf-node1 (+1/-1)
percona-xtradb-cluster-tests/conf/conf20.cnf-node2 (+1/-1)
scripts/wsrep_sst_xtrabackup-v2.sh (+62/-97)
To merge this branch: bzr merge lp:~percona-core/percona-xtradb-cluster/bug-1405668-5.6
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) 2015-01-04 Approve on 2015-01-29
Raghavendra D Prabhu (community) Approve on 2015-01-27
Review via email: mp+245519@code.launchpad.net
To post a comment you must log in.
Alexey Kopytov (akopytov) wrote :

I had to create a local clone of lp:~percona-core/percona-xtradb-cluster/bug-1405668-5.6 to see the actual changes.

This is the last revision I see in that branch:
---
revno: 949
committer: Raghavendra D Prabhu <email address hidden>
branch nick: bug-1405668-5.6
timestamp: Fri 2014-12-26 05:02:29 +0530
message:
  Fix the cpat for conf20 test
---

I'm not sure that revision is supposed to fix bug #1405668. Something probably went wrong.

review: Resubmit
Alexey Kopytov (akopytov) wrote :

Also, will there be a fix for 5.5?

This will not be fixed for 5.5.

Regarding 949, that was a fix in percona-xtradb-cluster-tests that I made.(since it is related)

Alexey Kopytov (akopytov) wrote :

I'm lost in these bugs. So this MP is supposed to fix bug #1405668. Which is not going to be fixed in 5.5.

But the revision I see in this MP doesn't mention bug #1405668. It mentions only bug #1405985. Which _is_ going to be fixed in 5.5.

Also there's a linked bug #1407599. Which is also going to be fixed in 5.5, but it's not mentioned in revision comments (only the fix itself is explained).

Can we have one revision, one MP per branch/bug rule? It gets a bit confusing and hard to track otherwise.

review: Resubmit

> Also there's a linked bug #1407599. Which is also going to be fixed in 5.5, but it's not mentioned in revision comments (only the fix itself is explained).

The commit message has been fixed to include this in bug subject. Its 5.5 MP is
here
https://code.launchpad.net/~percona-core/percona-xtradb-cluster/bug-1405668-5.5/+merge/247762

> I'm lost in these bugs. So this MP is supposed to fix bug #1405668. Which is
> not going to be fixed in 5.5.

That is correct. I have now added 5.5 to the bug, and will submit separate MP
for it.

> But the revision I see in this MP doesn't mention bug #1405668. It mentions
> only bug #1405985. Which _is_ going to be fixed in 5.5.

The revision 946 mentions it.

Anyway, same MP is used for 3 bugs since they all modify the same file (and the
only file) and are related.

review: Resubmit

Fixed.

review: Approve
Alexey Kopytov (akopytov) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'percona-xtradb-cluster-tests/conf/conf20.cnf-node1'
2--- percona-xtradb-cluster-tests/conf/conf20.cnf-node1 2014-03-09 15:14:10 +0000
3+++ percona-xtradb-cluster-tests/conf/conf20.cnf-node1 2015-01-27 18:38:33 +0000
4@@ -15,5 +15,5 @@
5 encrypt-key=6F3AD9F428143F133FD7D50D77D91EA4
6
7 [sst]
8-cpat='.*galera\.cache$\|.*sst_in_progress$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.borr$\|.*\.brn$'
9+cpat='.*galera\.cache$\|.*sst_in_progress$\|.*\.sst$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.borr$\|.*\.brn$'
10 encrypt=1
11
12=== modified file 'percona-xtradb-cluster-tests/conf/conf20.cnf-node2'
13--- percona-xtradb-cluster-tests/conf/conf20.cnf-node2 2014-03-09 15:14:10 +0000
14+++ percona-xtradb-cluster-tests/conf/conf20.cnf-node2 2015-01-27 18:38:33 +0000
15@@ -15,5 +15,5 @@
16 encrypt-key=6F3AD9F428143F133FD7D50D77D91EA4
17
18 [sst]
19-cpat='.*galera\.cache$\|.*sst_in_progress$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.borr$\|.*\.brn$'
20+cpat='.*galera\.cache$\|.*sst_in_progress$\|.*\.sst$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$\|.*\.borr$\|.*\.brn$'
21 encrypt=1
22
23=== modified file 'scripts/wsrep_sst_xtrabackup-v2.sh'
24--- scripts/wsrep_sst_xtrabackup-v2.sh 2014-11-21 03:28:15 +0000
25+++ scripts/wsrep_sst_xtrabackup-v2.sh 2015-01-27 18:38:33 +0000
26@@ -40,13 +40,12 @@
27 ttime=0
28 totime=0
29 lsn=""
30-incremental=0
31 ecmd=""
32 rlimit=""
33 # Initially
34 stagemsg="${WSREP_SST_OPT_ROLE}"
35 cpat=""
36-speciald=0
37+speciald=1
38 ib_home_dir=""
39 ib_log_dir=""
40 ib_undo_dir=""
41@@ -302,8 +301,7 @@
42 progress=$(parse_cnf sst progress "")
43 rebuild=$(parse_cnf sst rebuild 0)
44 ttime=$(parse_cnf sst time 0)
45- cpat=$(parse_cnf sst cpat '.*galera\.cache$\|.*sst_in_progress$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$')
46- incremental=$(parse_cnf sst incremental 0)
47+ cpat=$(parse_cnf sst cpat '.*galera\.cache$\|.*sst_in_progress$\|.*\.sst$\|.*gvwstate\.dat$\|.*grastate\.dat$\|.*\.err$\|.*\.log$\|.*RPM_UPGRADE_MARKER$\|.*RPM_UPGRADE_HISTORY$')
48 ealgo=$(parse_cnf xtrabackup encrypt "")
49 ekey=$(parse_cnf xtrabackup encrypt-key "")
50 ekeyfile=$(parse_cnf xtrabackup encrypt-key-file "")
51@@ -324,6 +322,11 @@
52 iapts=$(parse_cnf sst inno-apply-opts "")
53 impts=$(parse_cnf sst inno-move-opts "")
54 stimeout=$(parse_cnf sst sst-initial-timeout 100)
55+
56+ if [[ $speciald -eq 0 ]];then
57+ wsrep_log_error "sst-special-dirs equal to 0 is not supported, falling back to 1"
58+ speciald=1
59+ fi
60 }
61
62 get_stream()
63@@ -452,11 +455,7 @@
64 ss -p state listening "( sport = :$PORT )" | grep -qE 'socat|nc' && break
65 sleep 0.2
66 done
67- if [[ $incremental -eq 1 ]];then
68- echo "ready ${ADDR}/${MODULE}/$lsn/$sst_ver"
69- else
70- echo "ready ${ADDR}/${MODULE}//$sst_ver"
71- fi
72+ echo "ready ${ADDR}/${MODULE}//$sst_ver"
73 }
74
75 check_extra()
76@@ -571,7 +570,7 @@
77 get_stream
78 get_transfer
79
80-if ${INNOBACKUPEX_BIN} /tmp --help | grep -q -- '--version-check'; then
81+if ${INNOBACKUPEX_BIN} /tmp --help 2>/dev/null | grep -q -- '--version-check'; then
82 disver="--no-version-check"
83 fi
84
85@@ -622,15 +621,12 @@
86 fi
87 fi
88
89- if [[ -n $lsn ]];then
90- INNOEXTRA+=" --incremental --incremental-lsn=$lsn "
91- fi
92
93 check_extra
94
95 wsrep_log_info "Streaming GTID file before SST"
96
97- echo "${WSREP_SST_OPT_GTID}" > "${MAGIC_FILE}"
98+ wsrep_log_info "${WSREP_SST_OPT_GTID}" > "${MAGIC_FILE}"
99
100 ttcmd="$tcmd"
101
102@@ -712,26 +708,12 @@
103 [[ -e $SST_PROGRESS_FILE ]] && wsrep_log_info "Stale sst_in_progress file: $SST_PROGRESS_FILE"
104 [[ -n $SST_PROGRESS_FILE ]] && touch $SST_PROGRESS_FILE
105
106- if [[ $speciald -eq 1 ]];then
107- ib_home_dir=$(parse_cnf mysqld innodb-data-home-dir "")
108- ib_log_dir=$(parse_cnf mysqld innodb-log-group-home-dir "")
109- ib_undo_dir=$(parse_cnf mysqld innodb-undo-directory "")
110- if [[ -z $ib_home_dir && -z $ib_log_dir && -z $ib_undo_dir ]];then
111- speciald=0
112- fi
113- fi
114+ ib_home_dir=$(parse_cnf mysqld innodb-data-home-dir "")
115+ ib_log_dir=$(parse_cnf mysqld innodb-log-group-home-dir "")
116+ ib_undo_dir=$(parse_cnf mysqld innodb-undo-directory "")
117
118 stagemsg="Joiner-Recv"
119
120- if [[ ! -e ${DATA}/ibdata1 ]];then
121- incremental=0
122- fi
123-
124- if [[ $incremental -eq 1 ]];then
125- wsrep_log_info "Incremental SST enabled: NOT SUPPORTED yet"
126- lsn=$(grep to_lsn xtrabackup_checkpoints | cut -d= -f2 | tr -d ' ')
127- wsrep_log_info "Recovered LSN: $lsn"
128- fi
129
130 sencrypted=1
131 nthreads=1
132@@ -760,11 +742,6 @@
133 tcmd+=" | $pcmd"
134 fi
135
136- if [[ $incremental -eq 1 ]];then
137- BDATA=$DATA
138- DATA=$(mktemp -d)
139- MAGIC_FILE="${DATA}/${INFO_FILE}"
140- fi
141
142 get_keys
143 if [[ $encrypt -eq 1 && $sencrypted -eq 1 ]];then
144@@ -781,6 +758,13 @@
145 MAGIC_FILE="${STATDIR}/${INFO_FILE}"
146 recv_joiner $STATDIR "${stagemsg}-gtid" $stimeout 1
147
148+ if [[ -d ${DATA}/.sst ]];then
149+ wsrep_log_info "WARNING: Stale temporary SST directory: ${DATA}/.sst from previous state transfer"
150+ fi
151+ mkdir -p ${DATA}/.sst
152+ (recv_joiner $DATA/.sst "${stagemsg}-SST" 0 0) &
153+ jpid=$!
154+
155 if ! ps -p ${WSREP_SST_OPT_PARENT} &>/dev/null
156 then
157 wsrep_log_error "Parent mysqld process (PID:${WSREP_SST_OPT_PARENT}) terminated unexpectedly."
158@@ -791,48 +775,39 @@
159 then
160 wsrep_log_info "Proceeding with SST"
161
162- if [[ $speciald -eq 1 && -d ${DATA}/.sst ]];then
163- wsrep_log_info "WARNING: Stale temporary SST directory: ${DATA}/.sst from previous SST"
164- fi
165-
166- if [[ $incremental -ne 1 ]];then
167- if [[ $speciald -eq 1 ]];then
168- wsrep_log_info "Cleaning the existing datadir and innodb-data/log directories"
169- find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+
170- else
171- wsrep_log_info "Cleaning the existing datadir"
172- find $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+
173- fi
174- tempdir=$(parse_cnf mysqld log-bin "")
175- if [[ -n ${tempdir:-} ]];then
176- binlog_dir=$(dirname $tempdir)
177- binlog_file=$(basename $tempdir)
178- if [[ -n ${binlog_dir:-} && $binlog_dir != '.' && $binlog_dir != $DATA ]];then
179- pattern="$binlog_dir/$binlog_file\.[0-9]+$"
180- wsrep_log_info "Cleaning the binlog directory $binlog_dir as well"
181- find $binlog_dir -maxdepth 1 -type f -regex $pattern -exec rm -fv {} 1>&2 \+ || true
182- rm $binlog_dir/*.index || true
183- fi
184- fi
185-
186- else
187- wsrep_log_info "Removing existing ib_logfile files"
188- rm -f ${BDATA}/ib_logfile*
189- fi
190-
191-
192- if [[ $speciald -eq 1 ]];then
193- mkdir -p ${DATA}/.sst
194- TDATA=${DATA}
195- DATA="${DATA}/.sst"
196- fi
197+
198+ wsrep_log_info "Cleaning the existing datadir and innodb-data/log directories"
199+ find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -regex $cpat -prune -o -exec rm -rfv {} 1>&2 \+
200+
201+ tempdir=$(parse_cnf mysqld log-bin "")
202+ if [[ -n ${tempdir:-} ]];then
203+ binlog_dir=$(dirname $tempdir)
204+ binlog_file=$(basename $tempdir)
205+ if [[ -n ${binlog_dir:-} && $binlog_dir != '.' && $binlog_dir != $DATA ]];then
206+ pattern="$binlog_dir/$binlog_file\.[0-9]+$"
207+ wsrep_log_info "Cleaning the binlog directory $binlog_dir as well"
208+ find $binlog_dir -maxdepth 1 -type f -regex $pattern -exec rm -fv {} 1>&2 \+ || true
209+ rm $binlog_dir/*.index || true
210+ fi
211+ fi
212+
213+
214+
215+ TDATA=${DATA}
216+ DATA="${DATA}/.sst"
217
218
219 MAGIC_FILE="${DATA}/${INFO_FILE}"
220- recv_joiner $DATA "${stagemsg}-SST" 0 0
221+ wsrep_log_info "Waiting for SST streaming to complete!"
222+ wait $jpid
223
224 get_proc
225
226+ if [[ ! -s ${DATA}/xtrabackup_checkpoints ]];then
227+ wsrep_log_error "xtrabackup_checkpoints missing, failed innobackupex/SST on donor"
228+ exit 2
229+ fi
230+
231 # Rebuild indexes for compact backups
232 if grep -q 'compact = 1' ${DATA}/xtrabackup_checkpoints;then
233 wsrep_log_info "Index compaction detected"
234@@ -904,11 +879,6 @@
235
236 fi
237
238- if [[ $incremental -eq 1 ]];then
239- # Added --ibbackup=xtrabackup_55 because it fails otherwise citing connection issues.
240- INNOAPPLY="${INNOBACKUPEX_BIN} $disver --defaults-file=${WSREP_SST_OPT_CONF} \
241- --ibbackup=xtrabackup_56 --apply-log $rebuildcmd --redo-only $BDATA --incremental-dir=${DATA} &>>${BDATA}/innobackup.prepare.log"
242- fi
243
244 wsrep_log_info "Preparing the backup at ${DATA}"
245 timeit "Xtrabackup prepare stage" "$INNOAPPLY"
246@@ -919,30 +889,25 @@
247 exit 22
248 fi
249
250- if [[ $speciald -eq 1 ]];then
251- MAGIC_FILE="${TDATA}/${INFO_FILE}"
252- set +e
253- rm $TDATA/innobackup.prepare.log $TDATA/innobackup.move.log
254- set -e
255- wsrep_log_info "Moving the backup to ${TDATA}"
256- timeit "Xtrabackup move stage" "$INNOMOVE"
257- if [[ $? -eq 0 ]];then
258- wsrep_log_info "Move successful, removing ${DATA}"
259- rm -rf $DATA
260- DATA=${TDATA}
261- else
262- wsrep_log_error "Move failed, keeping ${DATA} for further diagnosis"
263- wsrep_log_error "Check ${DATA}/innobackup.move.log for details"
264- fi
265+ MAGIC_FILE="${TDATA}/${INFO_FILE}"
266+ set +e
267+ rm $TDATA/innobackup.prepare.log $TDATA/innobackup.move.log
268+ set -e
269+ wsrep_log_info "Moving the backup to ${TDATA}"
270+ timeit "Xtrabackup move stage" "$INNOMOVE"
271+ if [[ $? -eq 0 ]];then
272+ wsrep_log_info "Move successful, removing ${DATA}"
273+ rm -rf $DATA
274+ DATA=${TDATA}
275+ else
276+ wsrep_log_error "Move failed, keeping ${DATA} for further diagnosis"
277+ wsrep_log_error "Check ${DATA}/innobackup.move.log for details"
278 fi
279
280- if [[ $incremental -eq 1 ]];then
281- wsrep_log_info "Cleaning up ${DATA} after incremental SST"
282- [[ -d ${DATA} ]] && rm -rf ${DATA}
283- DATA=$BDATA
284- fi
285
286 else
287+ kill -KILL -$(ps -o pgid= $jpid | grep -o '[0-9]*')
288+ rm -rf $DATA/.sst
289 wsrep_log_info "${IST_FILE} received from donor: Running IST"
290 fi
291

Subscribers

People subscribed via source and target branches

to all changes: