Merge lp:~stewart/drizzle/xtrabackup into lp:~drizzle-trunk/drizzle/development

Proposed by Stewart Smith
Status: Merged
Approved by: Stewart Smith
Approved revision: 2375
Merged at revision: 2259
Proposed branch: lp:~stewart/drizzle/xtrabackup
Merge into: lp:~drizzle-trunk/drizzle/development
Diff against target: 20356 lines (+19696/-13)
79 files modified
plugin/innobase/btr/btr0btr.cc (+1/-2)
plugin/innobase/buf/buf0buf.cc (+1/-1)
plugin/innobase/fil/fil0fil.cc (+2/-2)
plugin/innobase/ibuf/ibuf0ibuf.cc (+6/-0)
plugin/innobase/include/srv0srv.h (+5/-0)
plugin/innobase/include/xtrabackup_api.h (+87/-0)
plugin/innobase/log/log0log.cc (+3/-1)
plugin/innobase/log/log0recv.cc (+2/-3)
plugin/innobase/os/os0file.cc (+14/-1)
plugin/innobase/plugin.am (+14/-1)
plugin/innobase/srv/srv0srv.cc (+4/-0)
plugin/innobase/srv/srv0start.cc (+6/-2)
plugin/innobase/trx/trx0purge.cc (+3/-0)
plugin/innobase/trx/trx0rseg.cc (+2/-0)
plugin/innobase/trx/trx0sys.cc (+2/-0)
plugin/innobase/xtrabackup/BUILD.txt (+21/-0)
plugin/innobase/xtrabackup/COPYING (+351/-0)
plugin/innobase/xtrabackup/Makefile (+141/-0)
plugin/innobase/xtrabackup/doc/xtrabackup.1 (+1486/-0)
plugin/innobase/xtrabackup/innobackupex (+2543/-0)
plugin/innobase/xtrabackup/patches/innodb51_builtin.patch (+1255/-0)
plugin/innobase/xtrabackup/patches/innodb55.patch (+1100/-0)
plugin/innobase/xtrabackup/patches/tar4ibd_libtar-1.2.11.patch (+593/-0)
plugin/innobase/xtrabackup/patches/xtradb51.patch (+1181/-0)
plugin/innobase/xtrabackup/patches/xtradb55.patch (+1189/-0)
plugin/innobase/xtrabackup/test/disabled/ib_include.sh (+63/-0)
plugin/innobase/xtrabackup/test/experimental/xb_race_drop.sh (+31/-0)
plugin/innobase/xtrabackup/test/inc/common.sh (+305/-0)
plugin/innobase/xtrabackup/test/inc/incremental_sample-db/incremental_sample-schema.sql (+40/-0)
plugin/innobase/xtrabackup/test/inc/part_range_sample-db/part_range_sample-schema.sql (+54/-0)
plugin/innobase/xtrabackup/test/inc/sakila-db/sakila-data.sql (+166/-0)
plugin/innobase/xtrabackup/test/inc/sakila-db/sakila-schema.sql (+643/-0)
plugin/innobase/xtrabackup/test/run.sh (+75/-0)
plugin/innobase/xtrabackup/test/t/bug606981.sh (+59/-0)
plugin/innobase/xtrabackup/test/t/ib_csm_csv.sh (+79/-0)
plugin/innobase/xtrabackup/test/t/ib_incremental.sh (+115/-0)
plugin/innobase/xtrabackup/test/t/ib_partition.sh (+84/-0)
plugin/innobase/xtrabackup/test/t/ib_specialchar.sh (+65/-0)
plugin/innobase/xtrabackup/test/t/xb_basic.sh (+50/-0)
plugin/innobase/xtrabackup/test/t/xb_export.sh (+118/-0)
plugin/innobase/xtrabackup/test/t/xb_incremental.sh (+110/-0)
plugin/innobase/xtrabackup/test/t/xb_incremental_compressed.sh (+192/-0)
plugin/innobase/xtrabackup/test/t/xb_parallel.sh (+52/-0)
plugin/innobase/xtrabackup/test/t/xb_part_range.sh (+121/-0)
plugin/innobase/xtrabackup/test/t/xb_partial.sh (+70/-0)
plugin/innobase/xtrabackup/test/t/xb_stats.sh (+52/-0)
plugin/innobase/xtrabackup/test/t/xb_stream.sh (+52/-0)
plugin/innobase/xtrabackup/utils/build.sh (+299/-0)
plugin/innobase/xtrabackup/utils/deb/Makefile (+2/-0)
plugin/innobase/xtrabackup/utils/deb/description-pak (+1/-0)
plugin/innobase/xtrabackup/utils/debian/README.Debian (+6/-0)
plugin/innobase/xtrabackup/utils/debian/changelog (+16/-0)
plugin/innobase/xtrabackup/utils/debian/compat (+1/-0)
plugin/innobase/xtrabackup/utils/debian/control (+15/-0)
plugin/innobase/xtrabackup/utils/debian/copyright (+24/-0)
plugin/innobase/xtrabackup/utils/debian/cron.d.ex (+4/-0)
plugin/innobase/xtrabackup/utils/debian/dirs (+2/-0)
plugin/innobase/xtrabackup/utils/debian/docs (+2/-0)
plugin/innobase/xtrabackup/utils/debian/emacsen-install.ex (+45/-0)
plugin/innobase/xtrabackup/utils/debian/emacsen-remove.ex (+15/-0)
plugin/innobase/xtrabackup/utils/debian/emacsen-startup.ex (+25/-0)
plugin/innobase/xtrabackup/utils/debian/info (+1/-0)
plugin/innobase/xtrabackup/utils/debian/init.d.ex (+157/-0)
plugin/innobase/xtrabackup/utils/debian/init.d.lsb.ex (+296/-0)
plugin/innobase/xtrabackup/utils/debian/manpage.1.ex (+59/-0)
plugin/innobase/xtrabackup/utils/debian/manpage.sgml.ex (+156/-0)
plugin/innobase/xtrabackup/utils/debian/manpage.xml.ex (+291/-0)
plugin/innobase/xtrabackup/utils/debian/menu.ex (+2/-0)
plugin/innobase/xtrabackup/utils/debian/postinst.ex (+41/-0)
plugin/innobase/xtrabackup/utils/debian/postrm.ex (+39/-0)
plugin/innobase/xtrabackup/utils/debian/preinst.ex (+37/-0)
plugin/innobase/xtrabackup/utils/debian/prerm.ex (+40/-0)
plugin/innobase/xtrabackup/utils/debian/rules (+113/-0)
plugin/innobase/xtrabackup/utils/debian/watch.ex (+23/-0)
plugin/innobase/xtrabackup/utils/debian/xtrabackup.default.ex (+10/-0)
plugin/innobase/xtrabackup/utils/debian/xtrabackup.doc-base.EX (+22/-0)
plugin/innobase/xtrabackup/utils/xtrabackup.spec (+88/-0)
plugin/innobase/xtrabackup/xtrabackup.cc (+5255/-0)
po/POTFILES.in (+1/-0)
To merge this branch: bzr merge lp:~stewart/drizzle/xtrabackup
Reviewer Review Type Date Requested Status
Lee Bieber (community) Needs Fixing
Review via email: mp+55241@code.launchpad.net

Description of the change

xtrabackup port to drizzle. Produces a binary named 'drizzlebackup.innobase'

I've manually tested it a bit, but we now need to build more tests around it

To post a comment you must log in.
lp:~stewart/drizzle/xtrabackup updated
2369. By Stewart Smith

merge trunk

2370. By Stewart Smith

xtrabackup needs typedef for CHARSET_INFO, currently not changing everywhere inside xtrabackup for ease of merging from xtrabackup tree

Revision history for this message
Lee Bieber (kalebral-deactivatedaccount) wrote :
review: Needs Fixing
lp:~stewart/drizzle/xtrabackup updated
2371. By Stewart Smith

merge trunk

2372. By Stewart Smith

definition of error::level_t has moved, fix xtrabackup build

Revision history for this message
Stewart Smith (stewart) wrote :

should be fixed now - a definition moved that we needed for the skeleton errmsg_printf()

Revision history for this message
Lee Bieber (kalebral-deactivatedaccount) wrote :
review: Needs Fixing
lp:~stewart/drizzle/xtrabackup updated
2373. By Stewart Smith

since ibool is ulint, ICC gives us warnings in xtrabackup.cc when converting from ibool to bool.

2374. By Stewart Smith

merge latest xtrabackup from Percona tree

2375. By Stewart Smith

merge my patch to percona-xtrabackup for copyright headers

Revision history for this message
Stewart Smith (stewart) wrote :

should be ready again (ICC warnings and copyright header warnings fixed)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'plugin/innobase/btr/btr0btr.cc'
--- plugin/innobase/btr/btr0btr.cc 2010-12-27 18:39:11 +0000
+++ plugin/innobase/btr/btr0btr.cc 2011-03-31 06:44:38 +0000
@@ -31,6 +31,7 @@
3131
32#include "fsp0fsp.h"32#include "fsp0fsp.h"
33#include "page0page.h"33#include "page0page.h"
34#include "xtrabackup_api.h"
3435
35#ifndef UNIV_HOTBACKUP36#ifndef UNIV_HOTBACKUP
36#include "btr0cur.h"37#include "btr0cur.h"
@@ -120,7 +121,6 @@
120/**************************************************************//**121/**************************************************************//**
121Gets the root node of a tree and x-latches it.122Gets the root node of a tree and x-latches it.
122@return root page, x-latched */123@return root page, x-latched */
123static
124buf_block_t*124buf_block_t*
125btr_root_block_get(125btr_root_block_get(
126/*===============*/126/*===============*/
@@ -566,7 +566,6 @@
566/************************************************************//**566/************************************************************//**
567Returns the child page of a node pointer and x-latches it.567Returns the child page of a node pointer and x-latches it.
568@return child page, x-latched */568@return child page, x-latched */
569static
570buf_block_t*569buf_block_t*
571btr_node_ptr_get_child(570btr_node_ptr_get_child(
572/*===================*/571/*===================*/
573572
=== modified file 'plugin/innobase/buf/buf0buf.cc'
--- plugin/innobase/buf/buf0buf.cc 2011-02-22 00:20:06 +0000
+++ plugin/innobase/buf/buf0buf.cc 2011-03-31 06:44:38 +0000
@@ -4053,7 +4053,7 @@
4053 recv_recover_page(TRUE, (buf_block_t*) bpage);4053 recv_recover_page(TRUE, (buf_block_t*) bpage);
4054 }4054 }
40554055
4056 if (uncompressed && !recv_no_ibuf_operations) {4056 if (uncompressed && !recv_no_ibuf_operations && !srv_fake_write) {
4057 ibuf_merge_or_delete_for_page(4057 ibuf_merge_or_delete_for_page(
4058 (buf_block_t*) bpage, bpage->space,4058 (buf_block_t*) bpage, bpage->space,
4059 bpage->offset, buf_page_get_zip_size(bpage),4059 bpage->offset, buf_page_get_zip_size(bpage),
40604060
=== modified file 'plugin/innobase/fil/fil0fil.cc'
--- plugin/innobase/fil/fil0fil.cc 2010-12-27 18:39:11 +0000
+++ plugin/innobase/fil/fil0fil.cc 2011-03-31 06:44:38 +0000
@@ -40,6 +40,7 @@
40#include "dict0dict.h"40#include "dict0dict.h"
41#include "page0page.h"41#include "page0page.h"
42#include "page0zip.h"42#include "page0zip.h"
43#include "xtrabackup_api.h"
43#ifndef UNIV_HOTBACKUP44#ifndef UNIV_HOTBACKUP
44# include "buf0lru.h"45# include "buf0lru.h"
45# include "ibuf0ibuf.h"46# include "ibuf0ibuf.h"
@@ -297,7 +298,7 @@
297298
298/** The tablespace memory cache. This variable is NULL before the module is299/** The tablespace memory cache. This variable is NULL before the module is
299initialized. */300initialized. */
300static fil_system_t* fil_system = NULL;301fil_system_t* fil_system = NULL;
301302
302303
303/********************************************************************//**304/********************************************************************//**
@@ -3448,7 +3449,6 @@
3448idea is to read as much good data as we can and jump over bad data.3449idea is to read as much good data as we can and jump over bad data.
3449@return 0 if ok, -1 if error even after the retries, 1 if at the end3450@return 0 if ok, -1 if error even after the retries, 1 if at the end
3450of the directory */3451of the directory */
3451static
3452int3452int
3453fil_file_readdir_next_file(3453fil_file_readdir_next_file(
3454/*=======================*/3454/*=======================*/
34553455
=== modified file 'plugin/innobase/ibuf/ibuf0ibuf.cc'
--- plugin/innobase/ibuf/ibuf0ibuf.cc 2010-12-27 18:39:11 +0000
+++ plugin/innobase/ibuf/ibuf0ibuf.cc 2011-03-31 06:44:38 +0000
@@ -1125,6 +1125,9 @@
11251125
1126 ut_ad(!recv_no_ibuf_operations);1126 ut_ad(!recv_no_ibuf_operations);
11271127
1128 if (srv_fake_write)
1129 return(FALSE);
1130
1128 if (ibuf_fixed_addr_page(space, zip_size, page_no)) {1131 if (ibuf_fixed_addr_page(space, zip_size, page_no)) {
11291132
1130 return(TRUE);1133 return(TRUE);
@@ -2613,6 +2616,9 @@
2613 ulint n_bytes;2616 ulint n_bytes;
2614 ulint n_pag2;2617 ulint n_pag2;
26152618
2619 if (srv_fake_write)
2620 return(0);
2621
2616 while (sum_pages < n_pages) {2622 while (sum_pages < n_pages) {
2617 n_bytes = ibuf_contract_ext(&n_pag2, sync);2623 n_bytes = ibuf_contract_ext(&n_pag2, sync);
26182624
26192625
=== modified file 'plugin/innobase/include/srv0srv.h'
--- plugin/innobase/include/srv0srv.h 2011-03-14 05:40:28 +0000
+++ plugin/innobase/include/srv0srv.h 2011-03-31 06:44:38 +0000
@@ -222,6 +222,11 @@
222extern ulong srv_max_purge_lag;222extern ulong srv_max_purge_lag;
223223
224extern ulong srv_replication_delay;224extern ulong srv_replication_delay;
225
226extern ibool srv_read_only;
227extern ibool srv_fake_write;
228extern ibool srv_apply_log_only;
229
225/*-------------------------------------------*/230/*-------------------------------------------*/
226231
227extern ulint srv_n_rows_inserted;232extern ulint srv_n_rows_inserted;
228233
=== added file 'plugin/innobase/include/xtrabackup_api.h'
--- plugin/innobase/include/xtrabackup_api.h 1970-01-01 00:00:00 +0000
+++ plugin/innobase/include/xtrabackup_api.h 2011-03-31 06:44:38 +0000
@@ -0,0 +1,87 @@
1/*****************************************************************************
2
3Copyright (C) 1997, 2010, Innobase Oy. All Rights Reserved.
4Copyright (C) 2011 Stewart Smith
5
6This program is free software; you can redistribute it and/or modify it under
7the terms of the GNU General Public License as published by the Free Software
8Foundation; version 2 of the License.
9
10This program is distributed in the hope that it will be useful, but WITHOUT
11ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
12FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
13
14You should have received a copy of the GNU General Public License along with
15this program; if not, write to the Free Software Foundation, Inc., 51 Franklin
16St, Fifth Floor, Boston, MA 02110-1301 USA
17
18*****************************************************************************/
19
20/* this file is the internal functions that are used by xtrabackup.
21 They probably shouldn't be called anywhere else.
22 */
23
24#pragma once
25
26ulint
27recv_find_max_checkpoint(log_group_t** max_group,
28 ulint* max_field);
29
30ibool
31log_block_checksum_is_ok_or_old_format(const byte* block);
32
33ulint
34open_or_create_data_files(ibool* create_new_db,
35#ifdef UNIV_LOG_ARCHIVE
36 ulint* min_arch_log_no,/*!< out: min of archived log
37 numbers in data files */
38 ulint* max_arch_log_no,/*!< out: max of archived log
39 numbers in data files */
40#endif /* UNIV_LOG_ARCHIVE */
41 ib_uint64_t* min_flushed_lsn,/*!< out: min of flushed lsn
42 values in data files */
43 ib_uint64_t* max_flushed_lsn,/*!< out: max of flushed lsn
44 values in data files */
45 ulint* sum_of_new_sizes);/*!< out: sum of sizes of the
46 new files added */
47
48ulint
49open_or_create_log_file(
50/*====================*/
51 ibool create_new_db, /*!< in: TRUE if we should create a
52 new database */
53 ibool* log_file_created, /*!< out: TRUE if new log file
54 created */
55 ibool log_file_has_been_opened,/*!< in: TRUE if a log file has been
56 opened before: then it is an error
57 to try to create another log file */
58 ulint k, /*!< in: log group number */
59 ulint i); /*!< in: log file number in group */
60
61buf_block_t*
62btr_root_block_get(
63/*===============*/
64 dict_index_t* index, /*!< in: index tree */
65 mtr_t* mtr); /*!< in: mtr */
66
67buf_block_t*
68btr_node_ptr_get_child(
69/*===================*/
70 const rec_t* node_ptr,/*!< in: node pointer */
71 dict_index_t* index, /*!< in: index */
72 const ulint* offsets,/*!< in: array returned by rec_get_offsets() */
73 mtr_t* mtr); /*!< in: mtr */
74
75ibool
76recv_check_cp_is_consistent(
77/*========================*/
78 const byte* buf); /*!< in: buffer containing checkpoint info */
79
80int
81fil_file_readdir_next_file(
82/*=======================*/
83 ulint* err, /*!< out: this is set to DB_ERROR if an error
84 was encountered, otherwise not changed */
85 const char* dirname,/*!< in: directory name or path */
86 os_file_dir_t dir, /*!< in: directory stream */
87 os_file_stat_t* info); /*!< in/out: buffer where the info is returned */
088
=== modified file 'plugin/innobase/log/log0log.cc'
--- plugin/innobase/log/log0log.cc 2011-02-22 00:20:06 +0000
+++ plugin/innobase/log/log0log.cc 2011-03-31 06:44:38 +0000
@@ -1356,7 +1356,7 @@
1356#endif /* UNIV_DEBUG */1356#endif /* UNIV_DEBUG */
1357 ulint unlock;1357 ulint unlock;
13581358
1359 if (recv_no_ibuf_operations) {1359 if (recv_no_ibuf_operations || srv_fake_write) {
1360 /* Recovery is running and no operations on the log files are1360 /* Recovery is running and no operations on the log files are
1361 allowed yet (the variable name .._no_ibuf_.. is misleading) */1361 allowed yet (the variable name .._no_ibuf_.. is misleading) */
13621362
@@ -3116,6 +3116,7 @@
3116 for the 'very fast' shutdown, because the InnoDB layer may have3116 for the 'very fast' shutdown, because the InnoDB layer may have
3117 committed or prepared transactions and we don't want to lose them. */3117 committed or prepared transactions and we don't want to lose them. */
31183118
3119 if (! srv_apply_log_only) {
3119 if (trx_n_mysql_transactions > 03120 if (trx_n_mysql_transactions > 0
3120 || UT_LIST_GET_LEN(trx_sys->trx_list) > 0) {3121 || UT_LIST_GET_LEN(trx_sys->trx_list) > 0) {
31213122
@@ -3123,6 +3124,7 @@
31233124
3124 goto loop;3125 goto loop;
3125 }3126 }
3127 }
31263128
3127 if (srv_fast_shutdown == 2) {3129 if (srv_fast_shutdown == 2) {
3128 /* In this fastest shutdown we do not flush the buffer pool:3130 /* In this fastest shutdown we do not flush the buffer pool:
31293131
=== modified file 'plugin/innobase/log/log0recv.cc'
--- plugin/innobase/log/log0recv.cc 2011-02-22 00:20:06 +0000
+++ plugin/innobase/log/log0recv.cc 2011-03-31 06:44:38 +0000
@@ -42,6 +42,7 @@
42#include "trx0undo.h"42#include "trx0undo.h"
43#include "trx0rec.h"43#include "trx0rec.h"
44#include "fil0fil.h"44#include "fil0fil.h"
45#include "xtrabackup_api.h"
45#ifndef UNIV_HOTBACKUP46#ifndef UNIV_HOTBACKUP
46# include "buf0rea.h"47# include "buf0rea.h"
47# include "srv0srv.h"48# include "srv0srv.h"
@@ -623,7 +624,6 @@
623/***********************************************************************//**624/***********************************************************************//**
624Checks the consistency of the checkpoint info625Checks the consistency of the checkpoint info
625@return TRUE if ok */626@return TRUE if ok */
626static
627ibool627ibool
628recv_check_cp_is_consistent(628recv_check_cp_is_consistent(
629/*========================*/629/*========================*/
@@ -653,7 +653,6 @@
653/********************************************************//**653/********************************************************//**
654Looks for the maximum consistent checkpoint from the log groups.654Looks for the maximum consistent checkpoint from the log groups.
655@return error code or DB_SUCCESS */655@return error code or DB_SUCCESS */
656static
657ulint656ulint
658recv_find_max_checkpoint(657recv_find_max_checkpoint(
659/*=====================*/658/*=====================*/
@@ -822,7 +821,6 @@
822InnoDB-3.23.52 where the checksum field contains the log block number.821InnoDB-3.23.52 where the checksum field contains the log block number.
823@return TRUE if ok, or if the log block may be in the format of InnoDB822@return TRUE if ok, or if the log block may be in the format of InnoDB
824version predating 3.23.52 */823version predating 3.23.52 */
825static
826ibool824ibool
827log_block_checksum_is_ok_or_old_format(825log_block_checksum_is_ok_or_old_format(
828/*===================================*/826/*===================================*/
@@ -3260,6 +3258,7 @@
3260 that the data dictionary tables will be free of any locks.3258 that the data dictionary tables will be free of any locks.
3261 The data dictionary latch should guarantee that there is at3259 The data dictionary latch should guarantee that there is at
3262 most one data dictionary transaction active at a time. */3260 most one data dictionary transaction active at a time. */
3261 if (! srv_apply_log_only)
3263 trx_rollback_or_clean_recovered(FALSE);3262 trx_rollback_or_clean_recovered(FALSE);
3264}3263}
32653264
32663265
=== modified file 'plugin/innobase/os/os0file.cc'
--- plugin/innobase/os/os0file.cc 2011-02-07 09:18:43 +0000
+++ plugin/innobase/os/os0file.cc 2011-03-31 06:44:38 +0000
@@ -652,6 +652,10 @@
652 const char* name) /*!< in: file name */652 const char* name) /*!< in: file name */
653{653{
654 struct flock lk;654 struct flock lk;
655
656 if (srv_read_only)
657 return 0;
658
655 lk.l_type = F_WRLCK;659 lk.l_type = F_WRLCK;
656 lk.l_whence = SEEK_SET;660 lk.l_whence = SEEK_SET;
657 lk.l_start = lk.l_len = 0;661 lk.l_start = lk.l_len = 0;
@@ -1495,7 +1499,10 @@
1495 if (create_mode == OS_FILE_OPEN || create_mode == OS_FILE_OPEN_RAW1499 if (create_mode == OS_FILE_OPEN || create_mode == OS_FILE_OPEN_RAW
1496 || create_mode == OS_FILE_OPEN_RETRY) {1500 || create_mode == OS_FILE_OPEN_RETRY) {
1497 mode_str = "OPEN";1501 mode_str = "OPEN";
1498 create_flag = O_RDWR;1502 if (srv_read_only)
1503 create_flag = O_RDONLY;
1504 else
1505 create_flag = O_RDWR;
1499 } else if (create_mode == OS_FILE_CREATE) {1506 } else if (create_mode == OS_FILE_CREATE) {
1500 mode_str = "CREATE";1507 mode_str = "CREATE";
1501 create_flag = O_RDWR | O_CREAT | O_EXCL;1508 create_flag = O_RDWR | O_CREAT | O_EXCL;
@@ -2291,6 +2298,9 @@
2291 }2298 }
2292 }2299 }
22932300
2301 if (srv_fake_write)
2302 return(TRUE);
2303
2294 os_n_file_writes++;2304 os_n_file_writes++;
22952305
2296#if defined(HAVE_PWRITE) && !defined(HAVE_BROKEN_PREAD)2306#if defined(HAVE_PWRITE) && !defined(HAVE_BROKEN_PREAD)
@@ -2673,6 +2683,9 @@
2673 ut_a((offset & 0xFFFFFFFFUL) == offset);2683 ut_a((offset & 0xFFFFFFFFUL) == offset);
2674 ut_a((n & 0xFFFFFFFFUL) == n);2684 ut_a((n & 0xFFFFFFFFUL) == n);
26752685
2686 if (srv_fake_write)
2687 return(TRUE);
2688
2676 os_n_file_writes++;2689 os_n_file_writes++;
26772690
2678 ut_ad(file);2691 ut_ad(file);
26792692
=== modified file 'plugin/innobase/plugin.am'
--- plugin/innobase/plugin.am 2011-02-04 22:30:00 +0000
+++ plugin/innobase/plugin.am 2011-03-31 06:44:38 +0000
@@ -1,4 +1,4 @@
1# vim:ft=automake1 vim:ft=automake
2# Copyright (C) 2001, 2004, 2006 MySQL AB & Innobase Oy2# Copyright (C) 2001, 2004, 2006 MySQL AB & Innobase Oy
3# Copyright (C) 2010 Brian Aker, Joe Daly, Monty Taylor3# Copyright (C) 2010 Brian Aker, Joe Daly, Monty Taylor
4#4#
@@ -209,6 +209,7 @@
209 plugin/innobase/include/ut0wqueue.h \209 plugin/innobase/include/ut0wqueue.h \
210 plugin/innobase/include/ha_prototypes.h \210 plugin/innobase/include/ha_prototypes.h \
211 plugin/innobase/include/handler0alter.h \211 plugin/innobase/include/handler0alter.h \
212 plugin/innobase/include/xtrabackup_api.h \
212 plugin/innobase/mem/mem0dbg.cc \213 plugin/innobase/mem/mem0dbg.cc \
213 plugin/innobase/handler/ha_innodb.h \214 plugin/innobase/handler/ha_innodb.h \
214 plugin/innobase/handler/handler0vars.h \215 plugin/innobase/handler/handler0vars.h \
@@ -222,8 +223,20 @@
222noinst_LTLIBRARIES+= \223noinst_LTLIBRARIES+= \
223 plugin/innobase/libinnobase.la \224 plugin/innobase/libinnobase.la \
224 plugin/innobase/libpars.la225 plugin/innobase/libpars.la
226bin_PROGRAMS+= plugin/innobase/xtrabackup/drizzlebackup.innobase
225endif227endif
226228
229plugin_innobase_xtrabackup_drizzlebackup_innobase_SOURCES = plugin/innobase/xtrabackup/xtrabackup.cc
230plugin_innobase_xtrabackup_drizzlebackup_innobase_LDADD = \
231 plugin/innobase/libinnobase.la \
232 drizzled/internal/libinternal.la \
233 drizzled/libcharset.la \
234 $(BOOST_LIBS) \
235 ${LTLIBAIO} \
236 $(LIBZ)
237
238plugin_innobase_xtrabackup_drizzlebackup_innobase_CXXFLAGS=${AM_CXXFLAGS} ${INNOBASE_SKIP_WARNINGS} -I${top_builddir}/plugin/innobase/include -I$(top_srcdir)/plugin/innobase/include -DBUILD_DRIZZLE
239
227plugin_innobase_libinnobase_la_CXXFLAGS=${AM_CXXFLAGS} ${INNOBASE_SKIP_WARNINGS} -I${top_builddir}/plugin/innobase/include -I$(top_srcdir)/plugin/innobase/include -DBUILD_DRIZZLE240plugin_innobase_libinnobase_la_CXXFLAGS=${AM_CXXFLAGS} ${INNOBASE_SKIP_WARNINGS} -I${top_builddir}/plugin/innobase/include -I$(top_srcdir)/plugin/innobase/include -DBUILD_DRIZZLE
228plugin_innobase_libinnobase_la_LDDADD= ${LTLIBAIO}241plugin_innobase_libinnobase_la_LDDADD= ${LTLIBAIO}
229plugin_innobase_libinnobase_la_SOURCES= \242plugin_innobase_libinnobase_la_SOURCES= \
230243
=== modified file 'plugin/innobase/srv/srv0srv.cc'
--- plugin/innobase/srv/srv0srv.cc 2011-02-22 00:20:06 +0000
+++ plugin/innobase/srv/srv0srv.cc 2011-03-31 06:44:38 +0000
@@ -433,6 +433,10 @@
433433
434UNIV_INTERN ulong srv_replication_delay = 0;434UNIV_INTERN ulong srv_replication_delay = 0;
435435
436UNIV_INTERN ibool srv_read_only = FALSE;
437UNIV_INTERN ibool srv_fake_write = FALSE;
438UNIV_INTERN ibool srv_apply_log_only = FALSE;
439
436/*-------------------------------------------*/440/*-------------------------------------------*/
437UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;441UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;
438UNIV_INTERN ulong srv_n_free_tickets_to_enter = 500;442UNIV_INTERN ulong srv_n_free_tickets_to_enter = 500;
439443
=== modified file 'plugin/innobase/srv/srv0start.cc'
--- plugin/innobase/srv/srv0start.cc 2011-02-22 00:20:06 +0000
+++ plugin/innobase/srv/srv0start.cc 2011-03-31 06:44:38 +0000
@@ -88,6 +88,7 @@
88# include "thr0loc.h"88# include "thr0loc.h"
89# include "os0sync.h" /* for INNODB_RW_LOCKS_USE_ATOMICS */89# include "os0sync.h" /* for INNODB_RW_LOCKS_USE_ATOMICS */
90# include "zlib.h" /* for ZLIB_VERSION */90# include "zlib.h" /* for ZLIB_VERSION */
91#include "xtrabackup_api.h"
9192
92#include <errno.h>93#include <errno.h>
93#include <unistd.h>94#include <unistd.h>
@@ -555,7 +556,6 @@
555/*********************************************************************//**556/*********************************************************************//**
556Creates or opens the log files and closes them.557Creates or opens the log files and closes them.
557@return DB_SUCCESS or error code */558@return DB_SUCCESS or error code */
558static
559ulint559ulint
560open_or_create_log_file(560open_or_create_log_file(
561/*====================*/561/*====================*/
@@ -708,7 +708,6 @@
708/*********************************************************************//**708/*********************************************************************//**
709Creates or opens database data files and closes them.709Creates or opens database data files and closes them.
710@return DB_SUCCESS or error code */710@return DB_SUCCESS or error code */
711static
712ulint711ulint
713open_or_create_data_files(712open_or_create_data_files(
714/*======================*/713/*======================*/
@@ -1558,6 +1557,10 @@
1558 are initialized in trx_sys_init_at_db_start(). */1557 are initialized in trx_sys_init_at_db_start(). */
15591558
1560 recv_recovery_from_checkpoint_finish();1559 recv_recovery_from_checkpoint_finish();
1560
1561 if (srv_apply_log_only)
1562 goto skip_processes;
1563
1561 if (srv_force_recovery < SRV_FORCE_NO_IBUF_MERGE) {1564 if (srv_force_recovery < SRV_FORCE_NO_IBUF_MERGE) {
1562 /* The following call is necessary for the insert1565 /* The following call is necessary for the insert
1563 buffer to work with multiple tablespaces. We must1566 buffer to work with multiple tablespaces. We must
@@ -1824,6 +1827,7 @@
1824 ibuf_update_max_tablespace_id();1827 ibuf_update_max_tablespace_id();
1825 }1828 }
18261829
1830skip_processes:
1827 srv_file_per_table = srv_file_per_table_original_value;1831 srv_file_per_table = srv_file_per_table_original_value;
18281832
1829 srv_was_started = TRUE;1833 srv_was_started = TRUE;
18301834
=== modified file 'plugin/innobase/trx/trx0purge.cc'
--- plugin/innobase/trx/trx0purge.cc 2010-12-26 21:16:41 +0000
+++ plugin/innobase/trx/trx0purge.cc 2011-03-31 06:44:38 +0000
@@ -1100,6 +1100,9 @@
1100 /* que_thr_t* thr2; */1100 /* que_thr_t* thr2; */
1101 ulint old_pages_handled;1101 ulint old_pages_handled;
11021102
1103 if (srv_fake_write)
1104 return(0);
1105
1103 mutex_enter(&(purge_sys->mutex));1106 mutex_enter(&(purge_sys->mutex));
11041107
1105 if (purge_sys->trx->n_active_thrs > 0) {1108 if (purge_sys->trx->n_active_thrs > 0) {
11061109
=== modified file 'plugin/innobase/trx/trx0rseg.cc'
--- plugin/innobase/trx/trx0rseg.cc 2010-12-26 21:25:00 +0000
+++ plugin/innobase/trx/trx0rseg.cc 2011-03-31 06:44:38 +0000
@@ -140,9 +140,11 @@
140140
141 mutex_free(&rseg->mutex);141 mutex_free(&rseg->mutex);
142142
143 if (! srv_apply_log_only) {
143 /* There can't be any active transactions. */144 /* There can't be any active transactions. */
144 ut_a(UT_LIST_GET_LEN(rseg->update_undo_list) == 0);145 ut_a(UT_LIST_GET_LEN(rseg->update_undo_list) == 0);
145 ut_a(UT_LIST_GET_LEN(rseg->insert_undo_list) == 0);146 ut_a(UT_LIST_GET_LEN(rseg->insert_undo_list) == 0);
147 }
146148
147 undo = UT_LIST_GET_FIRST(rseg->update_undo_cached);149 undo = UT_LIST_GET_FIRST(rseg->update_undo_cached);
148150
149151
=== modified file 'plugin/innobase/trx/trx0sys.cc'
--- plugin/innobase/trx/trx0sys.cc 2011-02-22 00:20:06 +0000
+++ plugin/innobase/trx/trx0sys.cc 2011-03-31 06:44:38 +0000
@@ -1476,10 +1476,12 @@
1476 UT_LIST_REMOVE(view_list, trx_sys->view_list, prev_view);1476 UT_LIST_REMOVE(view_list, trx_sys->view_list, prev_view);
1477 }1477 }
14781478
1479 if (! srv_apply_log_only) {
1479 ut_a(UT_LIST_GET_LEN(trx_sys->trx_list) == 0);1480 ut_a(UT_LIST_GET_LEN(trx_sys->trx_list) == 0);
1480 ut_a(UT_LIST_GET_LEN(trx_sys->rseg_list) == 0);1481 ut_a(UT_LIST_GET_LEN(trx_sys->rseg_list) == 0);
1481 ut_a(UT_LIST_GET_LEN(trx_sys->view_list) == 0);1482 ut_a(UT_LIST_GET_LEN(trx_sys->view_list) == 0);
1482 ut_a(UT_LIST_GET_LEN(trx_sys->mysql_trx_list) == 0);1483 ut_a(UT_LIST_GET_LEN(trx_sys->mysql_trx_list) == 0);
1484 }
14831485
1484 mem_free(trx_sys);1486 mem_free(trx_sys);
14851487
14861488
=== added directory 'plugin/innobase/xtrabackup'
=== added file 'plugin/innobase/xtrabackup/BUILD.txt'
--- plugin/innobase/xtrabackup/BUILD.txt 1970-01-01 00:00:00 +0000
+++ plugin/innobase/xtrabackup/BUILD.txt 2011-03-31 06:44:38 +0000
@@ -0,0 +1,21 @@
1How to build XtraBackup
2=======================
3
4!!You need to build the MySQL binaries before building XtraBackup!!
5
61. Build your version of mysql
7
82. copy the xtrabackup source directory into the mysql.5x/innobase directory
9 (e.g. mysql-5.0.75/innobase/xtrabackup-0.0 )
10
113. For 5.1x you need to apply a patch.
12 From the innobase directory, apply the xtrabackup patch.
13 e.g. #patch -p2 < xtrabackup-0.0/fix_innodb_for_backup51.patch
14
15(With 5.0 run 'make clean all' from the innobase directory This is necessary
16for 5.0, because of the lack of dependencies)
17
184.Run make [5.0] [5.1] [xtradb] [plugin] in the XtraBackup directory.
19
205. enjoy... :
21
022
=== added file 'plugin/innobase/xtrabackup/COPYING'
--- plugin/innobase/xtrabackup/COPYING 1970-01-01 00:00:00 +0000
+++ plugin/innobase/xtrabackup/COPYING 2011-03-31 06:44:38 +0000
@@ -0,0 +1,351 @@
1 GNU GENERAL PUBLIC LICENSE
2 Version 2, June 1991
3
4 Copyright (C) 1989, 1991 Free Software Foundation, Inc.
5 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA
6
7 Everyone is permitted to copy and distribute verbatim copies
8 of this license document, but changing it is not allowed.
9
10Preamble
11========
12
13The licenses for most software are designed to take away your freedom
14to share and change it. By contrast, the GNU General Public License is
15intended to guarantee your freedom to share and change free
16software--to make sure the software is free for all its users. This
17General Public License applies to most of the Free Software
18Foundation's software and to any other program whose authors commit to
19using it. (Some other Free Software Foundation software is covered by
20the GNU Library General Public License instead.) You can apply it to
21your programs, too.
22
23When we speak of free software, we are referring to freedom, not price.
24Our General Public Licenses are designed to make sure that you have
25the freedom to distribute copies of free software (and charge for this
26service if you wish), that you receive source code or can get it if you
27want it, that you can change the software or use pieces of it in new
28free programs; and that you know you can do these things.
29
30To protect your rights, we need to make restrictions that forbid anyone
31to deny you these rights or to ask you to surrender the rights. These
32restrictions translate to certain responsibilities for you if you
33distribute copies of the software, or if you modify it.
34
35For example, if you distribute copies of such a program, whether gratis
36or for a fee, you must give the recipients all the rights that you
37have. You must make sure that they, too, receive or can get the source
38code. And you must show them these terms so they know their rights.
39
40We protect your rights with two steps: (1) copyright the software, and
41(2) offer you this license which gives you legal permission to copy,
42distribute and/or modify the software.
43
44Also, for each author's protection and ours, we want to make certain
45that everyone understands that there is no warranty for this free
46software. If the software is modified by someone else and passed on, we
47want its recipients to know that what they have is not the original, so
48that any problems introduced by others will not reflect on the original
49authors' reputations.
50
51Finally, any free program is threatened constantly by software patents.
52We wish to avoid the danger that redistributors of a free program will
53individually obtain patent licenses, in effect making the program
54proprietary. To prevent this, we have made it clear that any patent
55must be licensed for everyone's free use or not licensed at all.
56
57The precise terms and conditions for copying, distribution and
58modification follow.
59
60 GNU GENERAL PUBLIC LICENSE
61 TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
62 0. This License applies to any program or other work which contains a
63 notice placed by the copyright holder saying it may be distributed
64 under the terms of this General Public License. The "Program",
65 below, refers to any such program or work, and a "work based on
66 the Program" means either the Program or any derivative work under
67 copyright law: that is to say, a work containing the Program or a
68 portion of it, either verbatim or with modifications and/or
69 translated into another language. (Hereinafter, translation is
70 included without limitation in the term "modification".) Each
71 licensee is addressed as "you".
72
73 Activities other than copying, distribution and modification are
74 not covered by this License; they are outside its scope. The act
75 of running the Program is not restricted, and the output from the
76 Program is covered only if its contents constitute a work based on
77 the Program (independent of having been made by running the
78 Program). Whether that is true depends on what the Program does.
79
80 1. You may copy and distribute verbatim copies of the Program's
81 source code as you receive it, in any medium, provided that you
82 conspicuously and appropriately publish on each copy an appropriate
83 copyright notice and disclaimer of warranty; keep intact all the
84 notices that refer to this License and to the absence of any
85 warranty; and give any other recipients of the Program a copy of
86 this License along with the Program.
87
88 You may charge a fee for the physical act of transferring a copy,
89 and you may at your option offer warranty protection in exchange
90 for a fee.
91
92 2. You may modify your copy or copies of the Program or any portion
93 of it, thus forming a work based on the Program, and copy and
94 distribute such modifications or work under the terms of Section 1
95 above, provided that you also meet all of these conditions:
96
97 a. You must cause the modified files to carry prominent notices
98 stating that you changed the files and the date of any change.
99
100 b. You must cause any work that you distribute or publish, that
101 in whole or in part contains or is derived from the Program
102 or any part thereof, to be licensed as a whole at no charge
103 to all third parties under the terms of this License.
104
105 c. If the modified program normally reads commands interactively
106 when run, you must cause it, when started running for such
107 interactive use in the most ordinary way, to print or display
108 an announcement including an appropriate copyright notice and
109 a notice that there is no warranty (or else, saying that you
110 provide a warranty) and that users may redistribute the
111 program under these conditions, and telling the user how to
112 view a copy of this License. (Exception: if the Program
113 itself is interactive but does not normally print such an
114 announcement, your work based on the Program is not required
115 to print an announcement.)
116
117 These requirements apply to the modified work as a whole. If
118 identifiable sections of that work are not derived from the
119 Program, and can be reasonably considered independent and separate
120 works in themselves, then this License, and its terms, do not
121 apply to those sections when you distribute them as separate
122 works. But when you distribute the same sections as part of a
123 whole which is a work based on the Program, the distribution of
124 the whole must be on the terms of this License, whose permissions
125 for other licensees extend to the entire whole, and thus to each
126 and every part regardless of who wrote it.
127
128 Thus, it is not the intent of this section to claim rights or
129 contest your rights to work written entirely by you; rather, the
130 intent is to exercise the right to control the distribution of
131 derivative or collective works based on the Program.
132
133 In addition, mere aggregation of another work not based on the
134 Program with the Program (or with a work based on the Program) on
135 a volume of a storage or distribution medium does not bring the
136 other work under the scope of this License.
137
138 3. You may copy and distribute the Program (or a work based on it,
139 under Section 2) in object code or executable form under the terms
140 of Sections 1 and 2 above provided that you also do one of the
141 following:
142
143 a. Accompany it with the complete corresponding machine-readable
144 source code, which must be distributed under the terms of
145 Sections 1 and 2 above on a medium customarily used for
146 software interchange; or,
147
148 b. Accompany it with a written offer, valid for at least three
149 years, to give any third-party, for a charge no more than your
150 cost of physically performing source distribution, a complete
151 machine-readable copy of the corresponding source code, to be
152 distributed under the terms of Sections 1 and 2 above on a
153 medium customarily used for software interchange; or,
154
155 c. Accompany it with the information you received as to the offer
156 to distribute corresponding source code. (This alternative is
157 allowed only for noncommercial distribution and only if you
158 received the program in object code or executable form with
159 such an offer, in accord with Subsection b above.)
160
161 The source code for a work means the preferred form of the work for
162 making modifications to it. For an executable work, complete
163 source code means all the source code for all modules it contains,
164 plus any associated interface definition files, plus the scripts
165 used to control compilation and installation of the executable.
166 However, as a special exception, the source code distributed need
167 not include anything that is normally distributed (in either
168 source or binary form) with the major components (compiler,
169 kernel, and so on) of the operating system on which the executable
170 runs, unless that component itself accompanies the executable.
171
172 If distribution of executable or object code is made by offering
173 access to copy from a designated place, then offering equivalent
174 access to copy the source code from the same place counts as
175 distribution of the source code, even though third parties are not
176 compelled to copy the source along with the object code.
177
178 4. You may not copy, modify, sublicense, or distribute the Program
179 except as expressly provided under this License. Any attempt
180 otherwise to copy, modify, sublicense or distribute the Program is
181 void, and will automatically terminate your rights under this
182 License. However, parties who have received copies, or rights,
183 from you under this License will not have their licenses
184 terminated so long as such parties remain in full compliance.
185
186 5. You are not required to accept this License, since you have not
187 signed it. However, nothing else grants you permission to modify
188 or distribute the Program or its derivative works. These actions
189 are prohibited by law if you do not accept this License.
190 Therefore, by modifying or distributing the Program (or any work
191 based on the Program), you indicate your acceptance of this
192 License to do so, and all its terms and conditions for copying,
193 distributing or modifying the Program or works based on it.
194
195 6. Each time you redistribute the Program (or any work based on the
196 Program), the recipient automatically receives a license from the
197 original licensor to copy, distribute or modify the Program
198 subject to these terms and conditions. You may not impose any
199 further restrictions on the recipients' exercise of the rights
200 granted herein. You are not responsible for enforcing compliance
201 by third parties to this License.
202
203 7. If, as a consequence of a court judgment or allegation of patent
204 infringement or for any other reason (not limited to patent
205 issues), conditions are imposed on you (whether by court order,
206 agreement or otherwise) that contradict the conditions of this
207 License, they do not excuse you from the conditions of this
208 License. If you cannot distribute so as to satisfy simultaneously
209 your obligations under this License and any other pertinent
210 obligations, then as a consequence you may not distribute the
211 Program at all. For example, if a patent license would not permit
212 royalty-free redistribution of the Program by all those who
213 receive copies directly or indirectly through you, then the only
214 way you could satisfy both it and this License would be to refrain
215 entirely from distribution of the Program.
216
217 If any portion of this section is held invalid or unenforceable
218 under any particular circumstance, the balance of the section is
219 intended to apply and the section as a whole is intended to apply
220 in other circumstances.
221
222 It is not the purpose of this section to induce you to infringe any
223 patents or other property right claims or to contest validity of
224 any such claims; this section has the sole purpose of protecting
225 the integrity of the free software distribution system, which is
226 implemented by public license practices. Many people have made
227 generous contributions to the wide range of software distributed
228 through that system in reliance on consistent application of that
229 system; it is up to the author/donor to decide if he or she is
230 willing to distribute software through any other system and a
231 licensee cannot impose that choice.
232
233 This section is intended to make thoroughly clear what is believed
234 to be a consequence of the rest of this License.
235
236 8. If the distribution and/or use of the Program is restricted in
237 certain countries either by patents or by copyrighted interfaces,
238 the original copyright holder who places the Program under this
239 License may add an explicit geographical distribution limitation
240 excluding those countries, so that distribution is permitted only
241 in or among countries not thus excluded. In such case, this
242 License incorporates the limitation as if written in the body of
243 this License.
244
245 9. The Free Software Foundation may publish revised and/or new
246 versions of the General Public License from time to time. Such
247 new versions will be similar in spirit to the present version, but
248 may differ in detail to address new problems or concerns.
249
250 Each version is given a distinguishing version number. If the
251 Program specifies a version number of this License which applies
252 to it and "any later version", you have the option of following
253 the terms and conditions either of that version or of any later
254 version published by the Free Software Foundation. If the Program
255 does not specify a version number of this License, you may choose
256 any version ever published by the Free Software Foundation.
257
258 10. If you wish to incorporate parts of the Program into other free
259 programs whose distribution conditions are different, write to the
260 author to ask for permission. For software which is copyrighted
261 by the Free Software Foundation, write to the Free Software
262 Foundation; we sometimes make exceptions for this. Our decision
263 will be guided by the two goals of preserving the free status of
264 all derivatives of our free software and of promoting the sharing
265 and reuse of software generally.
266
267 NO WARRANTY
268 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO
269 WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE
270 LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
271 HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT
272 WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT
273 NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
274 FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE
275 QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
276 PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY
277 SERVICING, REPAIR OR CORRECTION.
278
279 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
280 WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY
281 MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE
282 LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL,
283 INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR
284 INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
285 DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU
286 OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY
287 OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN
288 ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
289
290 END OF TERMS AND CONDITIONS
291How to Apply These Terms to Your New Programs
292=============================================
293
294If you develop a new program, and you want it to be of the greatest
295possible use to the public, the best way to achieve this is to make it
296free software which everyone can redistribute and change under these
297terms.
298
299To do so, attach the following notices to the program. It is safest to
300attach them to the start of each source file to most effectively convey
301the exclusion of warranty; and each file should have at least the
302"copyright" line and a pointer to where the full notice is found.
303
304 ONE LINE TO GIVE THE PROGRAM'S NAME AND A BRIEF IDEA OF WHAT IT DOES.
305 Copyright (C) YYYY NAME OF AUTHOR
306
307 This program is free software; you can redistribute it and/or modify
308 it under the terms of the GNU General Public License as published by
309 the Free Software Foundation; either version 2 of the License, or
310 (at your option) any later version.
311
312 This program is distributed in the hope that it will be useful,
313 but WITHOUT ANY WARRANTY; without even the implied warranty of
314 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
315 GNU General Public License for more details.
316
317 You should have received a copy of the GNU General Public License
318 along with this program; if not, write to the Free Software
319 Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
320
321Also add information on how to contact you by electronic and paper mail.
322
323If the program is interactive, make it output a short notice like this
324when it starts in an interactive mode:
325
326 Gnomovision version 69, Copyright (C) 19YY NAME OF AUTHOR
327 Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
328 This is free software, and you are welcome to redistribute it
329 under certain conditions; type `show c' for details.
330
331The hypothetical commands `show w' and `show c' should show the
332appropriate parts of the General Public License. Of course, the
333commands you use may be called something other than `show w' and `show
334c'; they could even be mouse-clicks or menu items--whatever suits your
335program.
336
337You should also get your employer (if you work as a programmer) or your
338school, if any, to sign a "copyright disclaimer" for the program, if
339necessary. Here is a sample; alter the names:
340
341 Yoyodyne, Inc., hereby disclaims all copyright interest in the program
342 `Gnomovision' (which makes passes at compilers) written by James Hacker.
343
344 SIGNATURE OF TY COON, 1 April 1989
345 Ty Coon, President of Vice
346
347This General Public License does not permit incorporating your program
348into proprietary programs. If your program is a subroutine library,
349you may consider it more useful to permit linking proprietary
350applications with the library. If this is what you want to do, use the
351GNU Library General Public License instead of this License.
0352
=== added file 'plugin/innobase/xtrabackup/Makefile'
--- plugin/innobase/xtrabackup/Makefile 1970-01-01 00:00:00 +0000
+++ plugin/innobase/xtrabackup/Makefile 2011-03-31 06:44:38 +0000
@@ -0,0 +1,141 @@
1# Makefile to build XtraBackup for Percona Server and different versions of MySQL
2#
3# Syntax:
4# make [5.1|5.5|xtradb|xtradb55]
5#
6# Default is xtradb - to build XtraBackup for Percona Server 5.1
7# xtradb55 - Xtrabackup for Percona Server 5.5
8# 5.1 - XtraBackup for MySQL versions 5.1.* with builtin InnoDB
9# 5.5 - XtraBackup for MySQL versions 5.5.*
10
11LIBS = -lpthread
12DEFS = -DUNIV_LINUX -DMYSQL_SERVER
13
14CFLAGS += -O3 -g -pedantic -Wall -Wundef -Wshadow -fdiagnostics-show-option -fno-strict-aliasing -Wno-strict-aliasing -Wextra -Wformat -Wno-format-nonliteral -Wno-format-security -Wno-long-long -Wmissing-declarations -Wno-redundant-decls --std=gnu99
15
16TARGET=xtrabackup
17PREFIX=/usr
18BIN_DIR=$(PREFIX)/bin
19
20default: xtradb
21
22# XtraBackup for MySQL 5.1
235.1: INC = -I. -isystem.. -isystem./../include -isystem./../../include -isystem./../../../include
245.1: INNODBOBJS = ../libinnobase_a-btr0btr.o ../libinnobase_a-btr0cur.o ../libinnobase_a-btr0pcur.o \
25 ../libinnobase_a-btr0sea.o ../libinnobase_a-buf0buf.o ../libinnobase_a-buf0flu.o \
26 ../libinnobase_a-buf0lru.o ../libinnobase_a-buf0rea.o ../libinnobase_a-data0data.o \
27 ../libinnobase_a-data0type.o ../libinnobase_a-dict0boot.o ../libinnobase_a-dict0crea.o \
28 ../libinnobase_a-dict0dict.o ../libinnobase_a-dict0load.o ../libinnobase_a-dict0mem.o \
29 ../libinnobase_a-dyn0dyn.o ../libinnobase_a-eval0eval.o ../libinnobase_a-eval0proc.o \
30 ../libinnobase_a-fil0fil.o ../libinnobase_a-fsp0fsp.o ../libinnobase_a-fut0fut.o \
31 ../libinnobase_a-fut0lst.o ../libinnobase_a-ha0ha.o ../libinnobase_a-hash0hash.o \
32 ../libinnobase_a-ibuf0ibuf.o ../libinnobase_a-lock0iter.o ../libinnobase_a-lock0lock.o \
33 ../libinnobase_a-log0log.o ../libinnobase_a-log0recv.o ../libinnobase_a-mach0data.o \
34 ../libinnobase_a-mem0mem.o ../libinnobase_a-mem0pool.o ../libinnobase_a-mtr0log.o \
35 ../libinnobase_a-mtr0mtr.o ../libinnobase_a-os0file.o ../libinnobase_a-os0proc.o \
36 ../libinnobase_a-os0sync.o ../libinnobase_a-os0thread.o ../libinnobase_a-page0cur.o \
37 ../libinnobase_a-page0page.o ../libinnobase_a-lexyy.o ../libinnobase_a-pars0grm.o \
38 ../libinnobase_a-pars0opt.o ../libinnobase_a-pars0pars.o ../libinnobase_a-pars0sym.o \
39 ../libinnobase_a-que0que.o ../libinnobase_a-read0read.o ../libinnobase_a-rem0cmp.o \
40 ../libinnobase_a-rem0rec.o ../libinnobase_a-row0ins.o ../libinnobase_a-row0mysql.o \
41 ../libinnobase_a-row0purge.o ../libinnobase_a-row0row.o ../libinnobase_a-row0sel.o \
42 ../libinnobase_a-row0uins.o ../libinnobase_a-row0umod.o ../libinnobase_a-row0undo.o \
43 ../libinnobase_a-row0upd.o ../libinnobase_a-row0vers.o ../libinnobase_a-srv0que.o \
44 ../libinnobase_a-srv0srv.o ../libinnobase_a-srv0start.o ../libinnobase_a-sync0arr.o \
45 ../libinnobase_a-sync0rw.o ../libinnobase_a-sync0sync.o ../libinnobase_a-thr0loc.o \
46 ../libinnobase_a-trx0purge.o ../libinnobase_a-trx0rec.o ../libinnobase_a-trx0roll.o \
47 ../libinnobase_a-trx0rseg.o ../libinnobase_a-trx0sys.o ../libinnobase_a-trx0trx.o \
48 ../libinnobase_a-trx0undo.o ../libinnobase_a-usr0sess.o ../libinnobase_a-ut0byte.o \
49 ../libinnobase_a-ut0dbg.o ../libinnobase_a-ut0list.o ../libinnobase_a-ut0mem.o \
50 ../libinnobase_a-ut0rnd.o ../libinnobase_a-ut0ut.o ../libinnobase_a-ut0vec.o \
51 ../libinnobase_a-ut0wqueue.o
525.1: MYSQLOBJS= ../../../mysys/libmysys.a ../../../strings/libmystrings.a
535.1: TARGET := xtrabackup_51
545.1: $(TARGET)
55
56# XtraBackup for MySQL 5.5
575.5: INC = -I. -I.. -I./../include -I./../../include -I./../../../include
585.5: INNODBOBJS= ../libinnobase.a
59
605.5: MYSQLOBJS= ../../../mysys/libmysys.a ../../../strings/libstrings.a ../../../zlib/libzlib.a
61ifeq ($(shell uname -s),Linux)
625.5: LIBS += -laio
63endif
645.5: TARGET := xtrabackup_innodb55
65# In CMake server builds it is important to build with exactly the same preprocessor flags
66# as were used to build InnoDB
675.5: DEFS = $(shell grep C_DEFINES ../CMakeFiles/innobase.dir/flags.make | \
68 sed -e 's/C_DEFINES = //')
695.5: $(TARGET)
70
71# XtraBackup for XtraDB
72xtradb: INC=-I. -isystem.. -isystem./../include -isystem./../../include -isystem./../../../include
73xtradb: INNODBOBJS = ../libinnobase_a-btr0btr.o ../libinnobase_a-btr0cur.o ../libinnobase_a-btr0pcur.o \
74 ../libinnobase_a-btr0sea.o ../libinnobase_a-buf0buddy.o ../libinnobase_a-buf0buf.o \
75 ../libinnobase_a-buf0flu.o ../libinnobase_a-buf0lru.o ../libinnobase_a-buf0rea.o \
76 ../libinnobase_a-data0data.o ../libinnobase_a-data0type.o ../libinnobase_a-dict0boot.o \
77 ../libinnobase_a-dict0crea.o ../libinnobase_a-dict0dict.o ../libinnobase_a-dict0load.o \
78 ../libinnobase_a-dict0mem.o ../libinnobase_a-dyn0dyn.o ../libinnobase_a-eval0eval.o \
79 ../libinnobase_a-eval0proc.o ../libinnobase_a-fil0fil.o ../libinnobase_a-fsp0fsp.o \
80 ../libinnobase_a-fut0fut.o ../libinnobase_a-fut0lst.o ../libinnobase_a-ha0ha.o \
81 ../libinnobase_a-ha0storage.o ../libinnobase_a-hash0hash.o ../libinnobase_a-ibuf0ibuf.o \
82 ../libinnobase_a-lock0iter.o ../libinnobase_a-lock0lock.o ../libinnobase_a-log0log.o \
83 ../libinnobase_a-log0recv.o ../libinnobase_a-mach0data.o ../libinnobase_a-mem0mem.o \
84 ../libinnobase_a-mem0pool.o ../libinnobase_a-mtr0log.o ../libinnobase_a-mtr0mtr.o \
85 ../libinnobase_a-os0file.o ../libinnobase_a-os0proc.o ../libinnobase_a-os0sync.o \
86 ../libinnobase_a-os0thread.o ../libinnobase_a-page0cur.o ../libinnobase_a-page0page.o \
87 ../libinnobase_a-page0zip.o ../libinnobase_a-lexyy.o ../libinnobase_a-pars0grm.o \
88 ../libinnobase_a-pars0opt.o ../libinnobase_a-pars0pars.o ../libinnobase_a-pars0sym.o \
89 ../libinnobase_a-que0que.o ../libinnobase_a-read0read.o ../libinnobase_a-rem0cmp.o \
90 ../libinnobase_a-rem0rec.o ../libinnobase_a-row0ext.o ../libinnobase_a-row0ins.o \
91 ../libinnobase_a-row0merge.o ../libinnobase_a-row0mysql.o ../libinnobase_a-row0purge.o \
92 ../libinnobase_a-row0row.o ../libinnobase_a-row0sel.o ../libinnobase_a-row0uins.o \
93 ../libinnobase_a-row0umod.o ../libinnobase_a-row0undo.o ../libinnobase_a-row0upd.o \
94 ../libinnobase_a-row0vers.o ../libinnobase_a-srv0que.o ../libinnobase_a-srv0srv.o \
95 ../libinnobase_a-srv0start.o ../libinnobase_a-sync0arr.o ../libinnobase_a-sync0rw.o \
96 ../libinnobase_a-sync0sync.o ../libinnobase_a-thr0loc.o ../libinnobase_a-trx0purge.o \
97 ../libinnobase_a-trx0rec.o ../libinnobase_a-trx0roll.o ../libinnobase_a-trx0rseg.o \
98 ../libinnobase_a-trx0sys.o ../libinnobase_a-trx0trx.o ../libinnobase_a-trx0undo.o \
99 ../libinnobase_a-usr0sess.o ../libinnobase_a-ut0byte.o ../libinnobase_a-ut0dbg.o \
100 ../libinnobase_a-ut0list.o ../libinnobase_a-ut0mem.o ../libinnobase_a-ut0rnd.o \
101 ../libinnobase_a-ut0ut.o ../libinnobase_a-ut0vec.o ../libinnobase_a-ut0wqueue.o \
102 ../libinnobase_a-ut0rbt.o
103xtradb: MYSQLOBJS = ../../../mysys/libmysys.a ../../../strings/libmystrings.a ../../../zlib/.libs/libzlt.a
104xtradb: DEFS += -DXTRADB_BASED
105xtradb: TARGET := xtrabackup
106xtradb: $(TARGET)
107
108# XtraBackup for XtraDB 5.5
109xtradb55dbug: DBUG_LIB = ../../../dbug/libdbug.a
110xtradb55dbug: xtradb55
111xtradb55: INC=-I. -isystem.. -isystem./../include -isystem./../../include -isystem./../../../include
112xtradb55: INNODBOBJS = ../libinnobase.a
113ifeq ($(shell uname -s),Linux)
114xtradb55: LIBS += -laio
115endif
116ifeq "$(wildcard ../../../zlib/.libs/libzlt.a)" ""
117xtradb55: LIBZ= -lz
118else
119xtradb55: LIBZ= ../../../zlib/.libs/libzlt.a
120endif
121xtradb55: MYSQLOBJS = ../../../mysys/libmysys.a ../../../strings/libstrings.a $(LIBZ) $(DBUG_LIB)
122# In CMake server builds it is important to build with exactly the same preprocessor flags
123# as were used to build InnoDB
124xtradb55: DEFS = $(shell grep C_DEFINES ../CMakeFiles/innobase.dir/flags.make | \
125 sed -e 's/C_DEFINES = //')
126xtradb55: DEFS += -DXTRADB_BASED -DXTRADB55
127xtradb55: TARGET := xtrabackup_55
128xtradb55: $(TARGET)
129
130
131xtrabackup.o: xtrabackup.c
132 $(CC) $(CFLAGS) $(INC) $(DEFS) -c $*.c
133
134$(TARGET): xtrabackup.o $(INNODBOBJS) $(MYSQLOBJS)
135 $(CC) $(CFLAGS) xtrabackup.o $(INNODBOBJS) $(MYSQLOBJS) $(LIBS) -o $(TARGET)
136
137clean:
138 rm -f *.o xtrabackup_*
139install:
140 install -m 755 innobackupex-1.5.1 $(BIN_DIR)
141 install -m 755 xtrabackup_* $(BIN_DIR)
0142
=== added directory 'plugin/innobase/xtrabackup/doc'
=== added file 'plugin/innobase/xtrabackup/doc/xtrabackup.1'
--- plugin/innobase/xtrabackup/doc/xtrabackup.1 1970-01-01 00:00:00 +0000
+++ plugin/innobase/xtrabackup/doc/xtrabackup.1 2011-03-31 06:44:38 +0000
@@ -0,0 +1,1486 @@
1.TH "XtraBackup" 1 "January 2011" "" "Percona Inc"
2.SH "NAME"
3Percona XtraBackup
4.SH "DESCRIPTION"
5.P
6Percona XtraBackup is an open-source hot backup utility for MySQL that doesn't lock your database during the backup. It can back up data from InnoDB, XtraDB, and MyISAM tables on MySQL 5.0 and 5.1 servers, and it has many advanced features. Commercial support is available(http://www.percona.com/support/mysql-support-maintenance/).
7.P
8Percona XtraBackup is a combination of the xtrabackup C program, and the innobackupex Perl script. The xtrabackup program copies and manipulates InnoDB and XtraDB data files, and the Perl script enables enhanced functionality, such as interacting with a running MySQL server and backing up MyISAM tables. XtraBackup works with unmodified MySQL servers, as well as Percona Server with XtraDB. It runs on Linux, Windows, and FreeBSD.
9.P
10Summary of features:
11
12.P
13.B
14Hot backups.
15Backups are online, and queries and transactions continue to run without interruption.
16.P
17.B
18Backs up MyISAM.
19MyISAM tables are read-only while they are being backed up.
20.P
21.B
22Partial backups.
23You can select which tables and databases to back up.
24.P
25.B
26High performance.
27XtraBackup is fast and low-impact. It takes special care not to disrupt your database server. It can copy files in parallel (in multiple threads) for even higher performance.
28.P
29.B
30Remote backups.
31You can make local backups, you can stream the backups to other programs, or you can securely copy your backups to a remote host after completion.
32.P
33.B
34Compressed backups.
35You can stream a backup into a compressed format. There is a companion program tar4ibd that understands how to work with InnoDB's data format.
36.P
37.B
38Incremental (delta) backups.
39You can back up only the data pages that have changed since the last backup.
40.P
41.B
42Point-in-time recovery.
43You can recover a backup, and then use binary logs to roll forward to a point in time.
44
45.SH " Percona XtraBackup User Manual "
46
47
48This page explains how to use Percona XtraBackup. XtraBackup is really a set of three tools:
49
50.HP
51*
52.B "xtrabackup"
53- a compiled C binary, which copies only InnoDB and XtraDB data
54.HP
55*
56.B "innobackupex"
57- a wrapper script that provides functionality to backup a whole MySQL database instance with MyISAM, InnoDB, and XtraDB tables.
58.HP
59*
60.B "tar4ibd"
61- tars InnoDB data safely.
62
63It is possible to use the
64.B "xtrabackup"
65binary alone, or to use only the
66.B "innobackupex"
67wrapper script and let it execute
68.B "xtrabackup"
69for you. It might be helpful to first learn how to use
70.B "xtrabackup"
71, and then learn how to use
72.B "innobackupex"
73for convenience and added functionality.
74
75.SS " How XtraBackup Works "
76
77
78XtraBackup is based on InnoDB's crash-recovery functionality. It copies your InnoDB data files, which results in data that is internally inconsistent; but then it performs crash recovery on the files to make them a consistent, usable database again.
79
80This works because InnoDB maintains a
81.I "redo log"
82, also called the transaction log. This contains a record of every change to InnoDB's data. When InnoDB starts, it inspects the data files and the transaction log, and performs two steps. It applies committed transaction log entries to the data files, and it performs an undo operation on any transactions that modified data but did not commit.
83
84XtraBackup works by remembering the
85.I "log sequence number"
86(LSN) when it starts, and then copying away the data files. It takes some time to do this, so if the files are changing, then they reflect the state of the database at different points in time. At the same time, XtraBackup runs a background process that watches the transaction log files, and copies changes from it. XtraBackup needs to do this continually because the transaction logs are written in a round-robin fashion, and can be reused after a while. XtraBackup needs the transaction log records for every change to the data files since it began execution.
87
88The above is the
89.I "backup"
90process. Next is the
91.I "prepare"
92process. During this step, XtraBackup performs crash recovery against the copied data files, using the copied transaction log file. After this is done, the database is ready to restore and use.
93
94The above process is implemented in the
95.B "xtrabackup"
96compiled binary program. The
97.B "innobackupex"
98program adds more convenience and functionality by also permitting you to back up MyISAM tables and .frm files. It starts
99.B "xtrabackup"
100, waits until it finishes copying files, and then issues
101.B "FLUSH TABLES WITH READ LOCK"
102to prevent further changes to MySQL's data and flush all MyISAM tables to disk. It holds this lock, copies the MyISAM files, and then releases the lock.
103
104The backed-up MyISAM and InnoDB tables will eventually be consistent with each other, because after the prepare (recovery) process, InnoDB's data is rolled forward to the point at which the backup completed, not rolled back to the point at which it started. This point in time matches where the
105.B "FLUSH TABLES WITH READ LOCK"
106was taken, so the MyISAM data and the prepared InnoDB data are in sync.
107
108The
109.B "xtrabackup"
110and
111.B "innobackupex"
112tools both offer many features not mentioned in the preceding explanation. Each tool's functionality is explained in more detail on its manual page. In brief, though, the tools permit you to do operations such as streaming and incremental backups with various combinations of copying the data files, copying the log files, and applying the logs to the data.
113
114.SH " The xtrabackup Binary "
115
116
117The
118.B "xtrabackup"
119binary is a compiled C program that is linked with the InnoDB libraries and the standard MySQL client libraries. The InnoDB libraries provide functionality necessary to apply a log to data files, and the MySQL client libraries provide command-line option parsing, configuration file parsing, and so on to give the binary a familiar look and feel.
120
121The next step is to
122.B "--prepare"
123your backup so it is ready to restore.
124The
125.B "xtrabackup"
126tool runs in either
127.B "--backup"
128or
129.B "--prepare"
130mode, corresponding to the two main functions it performs. There are several variations on these functions to accomplish different tasks, and there are two less commonly used modes,
131.B "--stats"
132and
133.B "--print-param"
134. There is no need to use any wrapper script with
135.B "xtrabackup"
136if you don't want to. It is quite easy to use by itself for most purposes.
137
138This manual section explains how to use xtrabackup in detail.
139.SH " Creating a Backup "
140
141
142To create a backup, run
143.B "xtrabackup"
144with the
145.B "--backup"
146option. You also need to specify a
147.B "--target_dir"
148option, which is where the backup will be stored, and a
149.B "--datadir"
150option, which is where the MySQL data is stored. If the InnoDB data or log files aren't stored in the same directory, you might need to specify the location of those, too. If the target directory does not exist,
151.B "xtrabackup"
152creates it. If the directory does exist and is empty,
153.B "xtrabackup"
154will succeed.
155.B "xtrabackup"
156will not overwrite existing files, however; it will fail with operating system error 17, 'file exists.'
157
158The tool changes its working directory to the data directory and performs two primary tasks to complete the backup:
159
160 - It starts a log-copying thread in the background. This thread watches the InnoDB log files, and when they change, it copies the changed blocks to a file called
161.B "xtrabackup_logfile"
162in the backup target directory. This is necessary because the backup might take a long time, and the recovery process needs all of the log file entries from the beginning to the end of the backup.
163 - It copies the InnoDB data files to the target directory. This is not a simple file copy; it opens and reads the files similarly to the way InnoDB does, by reading the data dictionary and copying them a page at a time.
164
165When the data files are finished copying, xtrabackup stops the log-copying thread, and creates a files in the target directory called
166.B "xtrabackup_checkpoints"
167, which contains the type of backup performed, the log sequence number at the beginning, and the log sequence number at the end.
168
169An example command to perform a backup follows:
170
171
172.nf
173
174xtrabackup --backup --datadir=/var/lib/mysql/ --target-dir=/data/backups/mysql/
175
176.fi
177
178This takes a backup of
179.B "/var/lib/mysql"
180and stores it at
181.B "/data/backups/mysql/"
182. The
183.B "--target-dir"
184option deserves special explanation. Because the backup is performed from the data directory itself,
185.B "the target directory is relative to the data directory"
186unless you specify an absolute path. If you specify a relative path such as
187.B "--target-dir=backups"
188, for example, don't look for the backup in the directory from which you executed xtrabackup -- it will be a subdirectory of the
189.B "--datadir"
190directory instead!
191
192During the backup process, you should see a lot of output showing the data files being copied, as well as the log file thread repeatedly scanning the log files and copying from it. The last thing you should see is something like the following, where the value of the <LSN> will be a number that depends on your system:
193
194
195.nf
196
197xtrabackup: Transaction log of lsn (<SLN>) to (<LSN>) was copied.
198
199.fi
200
201After the backup is finished, the target directory will contain files such as the following, assuming you have a single InnoDB table
202.B "test.tbl1"
203and you are using
204.B "innodb_file_per_table"
205:
206
207
208.nf
209
210/data/backups/mysql/ibdata1
211/data/backups/mysql/test
212/data/backups/mysql/test/tbl1.ibd
213/data/backups/mysql/xtrabackup_checkpoints
214/data/backups/mysql/xtrabackup_logfile
215
216.fi
217
218The backup can take a long time, depending on how large the database is. It is safe to cancel at any time, because it does not modify the database.
219
220.SH " Preparing the Backup "
221
222
223After you make a backup with
224.B "--backup"
225, the next step is to
226.I "prepare"
227it. The data files are not point-in-time consistent until they've been prepared, because they were copied at different times as the program ran, and they might have been changed while this was happening. If you try to start InnoDB with these data files, it will detect corruption and crash itself to prevent you from running on damaged data. The prepare step makes the files perfectly consistent at a single instant in time, so you can run InnoDB on them.
228
229During the prepare operation, xtrabackup boots up a kind of modified InnoDB that's embedded inside it (the libraries it was linked against). The modifications are necessary to disable InnoDB's standard safety checks, such as complaining that the log file isn't the right size, which aren't appropriate for working with backups. These modifications are only for the xtrabackup binary; you don't need a modified InnoDB to use xtrabackup for your backups.
230
231The prepare step uses this "embedded InnoDB" to perform crash recovery on the copied datafiles, using the copied log file. The prepare step is very simple to use: you simply run xtrabackup with the
232.B "--prepare"
233option and tell it which directory to prepare, for example, to prepare the backup previously taken,
234
235
236.nf
237
238xtrabackup --prepare --target-dir=/data/backups/mysql/
239
240.fi
241
242When this finishes, you should see an "InnoDB shutdown" with a message such as the following, where again the value of <LSN> will depend on your system:
243
244
245.nf
246
247101107 16:40:15 InnoDB: Shutdown completed; log sequence number <LSN>
248
249.fi
250
251Your backup is now clean and consistent, and ready to
252.B "restore"
253. However, you might want to take an extra step to make restores as quick as possible. This is to prepare the backup a second time. The first time makes the data files perfectly self-consistent, but it doesn't create fresh InnoDB log files. If you restore the backup at this point and start MySQL, it will have to create new log files, which could take a little while, and you might not want to wait for that. If you run
254.B "--prepare"
255a second time, xtrabackup will create the log files for you, and output status text such as the following, which is abbreviated for clarity. The value of <SIZE> will depend on your MySQL configuration.
256
257
258.nf
259
260$ xtrabackup --prepare --target-dir=/data/backups/mysql/
261xtrabackup: This target seems to be already prepared.
262xtrabackup: notice: xtrabackup_logfile was already used to '--prepare'.
263101107 16:54:10 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
264InnoDB: Setting log file ./ib_logfile0 size to <SIZE> MB
265InnoDB: Database physically writes the file full: wait...
266101107 16:54:10 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
267InnoDB: Setting log file ./ib_logfile1 size to <SIZE> MB
268InnoDB: Database physically writes the file full: wait...
269101107 16:54:15 InnoDB: Shutdown completed; log sequence number 1284108
270
271.fi
272.SH " Restoring a Backup "
273
274
275The xtrabackup binary does not have any functionality for restoring a backup. That is up to the user to do. You might use
276.B "rsync"
277or
278.B "cp"
279to restore the files. You should check that the restored files have the correct ownership and permissions.
280
281Note that xtrabackup backs up only the InnoDB data. You must separately restore the MySQL system database, MyISAM data, table definition files (.frm files), and everything else necessary to make your database functional.
282
283.SH " How-To and Recipes "
284
285
286This page is a quick-reference list of recipes for the
287.B "xtrabackup"
288tool. It doesn't mention the
289.B "innobackupex"
290tool; that is documented separately.
291
292In all of the below examples, we assume the following:
293
294.HP
295* You are backing up a server whose data is stored in
296.B "/var/lib/mysql/"
297, which is the standard location for most distributions
298.HP
299* You are storing the backups in
300.B "/data/backups/mysql"
301
302.HP
303* You have a
304.B "my.cnf"
305file in a standard location, such as
306.B "/etc/my.cnf"
307, with at least the following contents:
308.nf
309[mysqld]
310datadir=/var/lib/mysql/
311[xtrabackup]
312target_dir=/data/backups/mysql/
313
314.fi
315
316.SS " Making a Backup "
317
318
319Making the backup copies the InnoDB data and log files to the destination. Preparing the backup makes the data files consistent and ready to use.
320
321.HP
322* Make a backup:
323.nf
324xtrabackup --backup
325.fi
326.HP
327* Prepare the backup:
328.nf
329xtrabackup --prepare
330.fi
331.HP
332* Prepare again, to create fresh InnoDB log files:
333.nf
334xtrabackup --prepare
335.fi
336
337
338.B "Success Criterion"
339
340
341.HP
342* The exit status of
343.B "xtrabackup"
344is 0.
345.HP
346* In the second
347.B "--prepare"
348step, you should see InnoDB print messages similar to "Log file ./ib_logfile0 did not exist: new to be created", followed by a line indicating the log file was created.
349
350
351.B "Relevant Configuration"
352
353
354.HP
355* You might want to set the
356.B "--use-memory"
357option to something similar to the size of your buffer pool, if you are on a dedicated server that has enough free memory.
358
359.SS " Making an Incremental Backup "
360
361
362Making an incremental backup requires a full backup as a base.
363
364.HP
365* Create a full backup as above, but do not run the
366.B "--prepare"
367command yet
368.HP
369* Suppose the full backup is on Monday
370.HP
371* Create an incremental backup on Tuesday:
372.nf
373
374 xtrabackup --backup --target-dir=/data/backups/inc/tue/ \
375 --incremental-basedir=/data/backups/mysql/
376.fi
377.HP
378* Create an incremental backup on Wednesday:
379.nf
380
381 xtrabackup --backup --target-dir=/data/backups/inc/wed/ \
382 --incremental-basedir=/data/backups/inc/tue/
383.fi
384.HP
385* Prepare the base backup from Monday:
386.nf
387
388 xtrabackup --prepare --apply-log-only --target-dir=/data/backups/mysql/
389.fi
390.HP
391* Roll Monday's data forward to the state on Tuesday:
392.nf
393
394 xtrabackup --prepare --apply-log-only --target-dir=/data/backups/mysql/ \
395 --incremental-dir=/data/backups/inc/tue/
396.fi
397.HP
398* Roll forward again to the state on Wednesday:
399.nf
400
401 xtrabackup --prepare --apply-log-only --target-dir=/data/backups/mysql/ \
402 --incremental-dir=/data/backups/inc/wed/
403.fi
404.HP
405* Follow the steps for a normal backup to complete the preparation stage so the backup is ready to restore
406
407.SS " Restoring the Backup "
408
409
410Because
411.B "xtrabackup"
412doesn't copy MyISAM files, .frm files, and the rest of the database, you might need to back those up separately. To restore the InnoDB data, simply do something like the following after preparing:
413
414
415.nf
416
417cd /data/backups/mysql/
418rsync -rvt --exclude 'xtrabackup_checkpoints' --exclude 'xtrabackup_logfile' \
419 ./ /var/lib/mysql
420chown -R mysql:mysql /var/lib/mysql/
421
422.fi
423.SH " Limitations of xtrabackup "
424
425
426The xtrabackup binary has some limitations you should be aware of to ensure that your backups go smoothly and are recoverable.
427
428.HP
429* If the xtrabackup_logfile is larger than 4GB, the
430.B "--prepare"
431step will fail on 32-bit versions of xtrabackup. This limitation also applied to some older 64-bit versions of xtrabackup (
432.B FIXME
433what versions?).
434.HP
435* xtrabackup does not currently create new InnoDB log files (ib_logfile0, etc) during the initial
436.B "--prepare"
437step. You must prepare the backup a second time to do this, if you wish.
438.HP
439* xtrabackup copies only the InnoDB data and logs. It does not copy table definition files (.frm files), MyISAM data, users, privileges, or any other portions of the overall database that lie outside of the InnoDB data. To back up this data, you need a wrapper script such as
440.B "innobackupex"
441.
442.HP
443*
444.B "xtrabackup"
445doesn't understand the very old
446.B "--set-variable"
447
448.B "my.cnf"
449syntax that MySQL uses. See
450.B "configuration"
451.
452.SH " Configuring xtrabackup "
453
454
455This page explains how to configure
456.B "xtrabackup"
457and how to configure your system to support
458.B "xtrabackup"
459correctly.
460
461.SS " Configuring xtrabackup "
462
463
464All of the
465.B "xtrabackup"
466configuration is through options, which behave exactly like standard MySQL program options: they can be specified either at the command-line, or through a file such as /etc/my.cnf.
467
468The xtrabackup binary reads the [myslqd] and [xtrabackup] sections from any configuration files, in order. That is so that it can read its options from your existing MySQL installation, such as the
469.B "datadir"
470or some of the InnoDB options. If you want to override these, just specify them in the [xtrabackup] section, and because it is read later, it will take precedence.
471
472You don't need to put any configuration in your my.cnf if you don't want to. You can simply specify the options on the command-line. Normally, the only thing you might find convenient to place in the [xtrabackup] section of your my.cnf file is the
473.B "target_dir"
474option, for example,
475
476
477.nf
478
479[xtrabackup]
480target_dir = /data/backups/mysql/
481
482.fi
483
484This manual will assume that you
485.B "do not"
486have any file-based configuration for
487.B "xtrabackup"
488, so it will always show command-line options being used explicitly. Please see the
489.B "option and variable reference"
490for details on all of the configuration options.
491
492The
493.B "xtrabackup"
494binary does not accept exactly the same syntax in the
495.B "my.cnf"
496file as the
497.B "mysqld"
498server binary does. For historical reasons, the
499.B "mysqld"
500server binary accepts parameters with a
501.B "--set-variable=<variable>=<value>"
502syntax, which
503.B "xtrabackup"
504does not understand. If your
505.B "my.cnf"
506file has such configuration directives, you should rewrite them in the
507.B "--variable=value"
508syntax.
509
510.SS " System Configuration and NFS Volumes "
511
512
513The
514.B "xtrabackup"
515tool requires no special configuration on most systems. However, the storage where the
516.B "--target-dir"
517is located must behave properly when
518.B "fsync()"
519is called. In particular, we have noticed that NFS volumes not mounted with the
520.B "sync"
521option might not really sync the data. As a result, if you back up to an NFS volume mounted with the
522.B "async"
523option, and then try to prepare the backup from a different server that also mounts that volume, the data might appear to be corrupt. You can use the
524.B "noasync"
525mount option to avoid this problem.
526.SH " The xtrabackup Option Reference "
527
528
529This page documents all of the command-line options for the
530.B "xtrabackup"
531binary.
532
533.SS " --print-defaults "
534
535
536Print the program argument list and exit. Must be given as the first option on the command-line.
537
538.SS " --no-defaults "
539
540
541Don't read default options from any option file. Must be given as the first option on the command-line.
542
543.SS " --defaults-file "
544
545
546Only read default options from the given file. Must be given as the first option on the command-line. Must be a real file; cannot be a symbolic link.
547
548.SS " --defaults-extra-file "
549
550
551Read this file after the global files are read. Must be given as the first option on the command-line.
552
553.SS " --apply-log-only "
554
555
556This option causes only the redo stage to be performed when
557.B "incremental backups"
558.
559
560.SS " --backup "
561
562
563Make a backup and place it in
564.B "--target-dir"
565. See
566.B "creating a backup"
567.
568
569.SS " --create-ib-logfile "
570
571
572This option is not currently implemented. To create the InnoDB log files, you must
573.B "prepare the backup"
574twice at present.
575
576.SS " --datadir "
577
578
579The source directory for the backup. This should be the same as the datadir for your MySQL server, so it should be read from
580.B "my.cnf"
581if that exists; otherwise you must specify it on the command line.
582
583.SS " --export "
584
585
586Create files necessary for exporting tables. See [[export and import]].
587
588.SS " --incremental-basedir "
589
590
591When creating an
592.B "incremental backup"
593, this is the directory containing the full backup that is the base dataset for the incremental backups.
594
595.SS " --incremental-dir "
596
597
598When preparing an
599.B "incremental backup"
600, this is the directory where the incremental backup is combined with the full backup to make a new full backup.
601
602.SS " --incremental-lsn "
603
604
605When creating an
606.B "incremental backup"
607, you can specify the log sequence number (LSN) in high:low format as two 32-bit integers instead of specifying
608.B "--incremental-basedir"
609.
610
611.SS " --innodb-miscellaneous "
612
613
614There is a large group of InnoDB options that are normally read from the
615.B "my.cnf"
616configuration file, so that
617.B "xtrabackup"
618boots up its embedded InnoDB in the same configuration as your current server. You normally do not need to specify these explicitly. These options have the same behavior that they have in InnoDB or XtraDB. They are as follows:
619
620.HP
621*
622.B "--innodb-adaptive-hash-index"
623
624.HP
625*
626.B "--innodb-additional-mem-pool-size"
627
628.HP
629*
630.B "--innodb-autoextend-increment"
631
632.HP
633*
634.B "--innodb-buffer-pool-size"
635
636.HP
637*
638.B "--innodb-checksums"
639
640.HP
641*
642.B "--innodb-data-file-path"
643
644.HP
645*
646.B "--innodb-data-home-dir"
647
648.HP
649*
650.B "--innodb-doublewrite-file"
651
652.HP
653*
654.B "--innodb-doublewrite"
655
656.HP
657*
658.B "--innodb-extra-undoslots"
659
660.HP
661*
662.B "--innodb-fast-checksum"
663
664.HP
665*
666.B "--innodb-file-io-threads"
667
668.HP
669*
670.B "--innodb-file-per-table"
671
672.HP
673*
674.B "--innodb-flush-log-at-trx-commit"
675
676.HP
677*
678.B "--innodb-flush-method"
679
680.HP
681*
682.B "--innodb-force-recovery"
683
684.HP
685*
686.B "--innodb-io-capacity"
687
688.HP
689*
690.B "--innodb-lock-wait-timeout"
691
692.HP
693*
694.B "--innodb-log-buffer-size"
695
696.HP
697*
698.B "--innodb-log-files-in-group"
699
700.HP
701*
702.B "--innodb-log-file-size"
703
704.HP
705*
706.B "--innodb-log-group-home-dir"
707
708.HP
709*
710.B "--innodb-max-dirty-pages-pct"
711
712.HP
713*
714.B "--innodb-open-files"
715
716.HP
717*
718.B "--innodb-page-size"
719
720.HP
721*
722.B "--innodb-read-io-threads"
723
724.HP
725*
726.B "--innodb-write-io-threads"
727
728
729.SS " --log-stream "
730
731
732Makes
733.B "xtrabackup"
734not copy data files, and output the contents of the InnoDB log files to STDOUT until the
735.B "--suspend-at-end"
736automatically.
737
738.SS " --prepare "
739
740
741Makes
742.B "xtrabackup"
743perform recovery on a backup created with
744.B "--backup"
745, so that it is ready to use. See
746.B "preparing a backup"
747.
748
749.SS " --print-param "
750
751
752Makes
753.B "xtrabackup"
754print out parameters that can be used for copying the data files back to their original locations to restore them. See
755.B "scripting with xtrabackup"
756.
757
758.SS " --stats "
759
760
761Causes
762.B "xtrabackup"
763to scan the specified data files and print out
764.B "index statistics"
765.
766
767.SS " --suspend-at-end "
768
769
770Causes
771.B "xtrabackup"
772to create a file called
773.B "xtrabackup_suspended"
774in the
775.B "scripting with xtrabackup"
776.
777
778.SS " --tables-file "
779
780
781A file containing one table name per line, in
782.B "databasename.tablename"
783format. The backup will be limited to the specified tables. See
784.B "partial backups"
785.
786
787.SS " --tables "
788
789
790A regular expression against which the full tablename, in
791.B "databasename.tablename"
792format, is matched. If the name matches, the table is backed up. See
793.B "partial backups"
794.
795
796.SS " --target-dir "
797
798
799This option specifies the destination directory for the backup. If the directory does not exist,
800.B "xtrabackup"
801creates it. If the directory does exist and is empty,
802.B "xtrabackup"
803will succeed.
804.B "xtrabackup"
805will not overwrite existing files, however; it will fail with operating system error 17, 'file exists.'
806
807Note that for
808.B "--prepare"
809, relative paths are interpreted as being relative to the current working directory.
810
811.SS " --throttle "
812
813
814This option limits
815.B "throttling a backup"
816.
817
818.SS " --tmpdir "
819
820
821This option is currently not used for anything except printing out the correct
822.B "tmpdir"
823parameter when
824.B "--print-param"
825is used.
826
827.SS " --use-memory "
828
829
830This option affects how much memory is allocated for
831.B "--stats"
832. Its purpose is similar to
833.B "innodb_buffer_pool_size"
834. It does not do the same thing as the similarly named option in Oracle's InnoDB Hot Backup tool. The default value is 100MB, and if you have enough available memory, 1GB to 2GB is a good recommended value.
835
836.SS " --parallel "
837
838
839This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer).
840
841Currently, the option only works for local backups.
842
843.SH " Exporting and Importing Tables "
844
845
846In standard InnoDB, it is not normally possible to copy tables between servers by copying the files, even with
847.B "innodb_file_per_table"
848. However, with the xtrabackup binary, you can export individual tables from any InnoDB database, and import them into Percona Server with XtraDB. (The source doesn't have to be XtraDB, but the destination does.) This functionality requires
849.B "innodb_file_per_table"
850to be used on both servers, and requires
851.B "innodb_expand_import"
852to be enabled on the destination server. It only works on individual
853.B ".ibd"
854files, and cannot export a table that is not contained in its own
855.B ".ibd"
856file.
857
858Let's see how to export and import the following table:
859
860
861.nf
862
863CREATE TABLE export_test (
864 a int(11) DEFAULT NULL
865) ENGINE=InnoDB DEFAULT CHARSET=latin1;
866
867.fi
868
869.SS " Exporting the Table "
870
871
872This table should have been created in innodb_file_per_table mode, so after taking a backup as usual with
873.B "--backup"
874, the
875.B ".ibd"
876file should exist in the target directory:
877
878
879.nf
880
881$ find /data/backups/mysql/ -name export_test.*
882/data/backups/mysql/test/export_test.ibd
883
884.fi
885
886when you prepare the backup, add the extra parameter
887.B "--export"
888to the command. If you do not have
889.B "innodb_file_per_table"
890in your my.cnf, you must add that to the command-line options. Here is an example:
891
892
893.nf
894
895xtrabackup --prepare --export --innodb-file-per-table --target-dir=/data/backups/mysql/
896
897.fi
898
899Now you should see a
900.B ".exp"
901file in the target directory:
902
903
904.nf
905
906$ find /data/backups/mysql/ -name export_test.*
907/data/backups/mysql/test/export_test.exp
908/data/backups/mysql/test/export_test.ibd
909
910.fi
911
912These two files are all you need to import the table into a server running Percona Server with XtraDB.
913
914.SS " Importing the Table "
915
916
917On the destination server running Percona Server with XtraDB, with
918.B "innodb_expand_import"
919enabled, create a table with the same structure, and then perform the following steps:
920
921 - Execute
922.B "ALTER TABLE test.export_test DISCARD TABLESPACE;"
923
924 - If you see the following message, then you must enable innodb_file_per_table and create the table again: "ERROR 1030 (HY000): Got error -1 from storage engine"
925 - Copy the exported files to the
926.B "test/"
927subdirectory of the destination server's data directory
928 - Execute
929.B "ALTER TABLE test.export_test IMPORT TABLESPACE;"
930
931
932The table should now be imported, and you should be able to SELECT from it and see the imported data.
933
934.SH " Incremental Backups "
935
936
937The
938.B "xtrabackup"
939tool supports incremental backups, which means that it can copy only the data that has changed since the last full backup. You can perform many incremental backups between each full backup, so you can set up a backup process such as a full backup once a week and an incremental backup every day, or full backups every day and incremental backups every hour.
940
941Incremental backups work because each InnoDB page (usually 16kb in size) contains a
942.I "log sequence number"
943, or LSN. The LSN is the system version number for the entire database. Each page's LSN shows how recently it was changed. An incremental backup copies each page whose LSN is newer than the previous incremental or full backup's LSN.
944
945Incremental backups do not actually compare the data files to the previous backup's data files. In fact, you can use
946.B "--incremental-lsn"
947to perform an incremental backup without even having the previous backup, if you know its LSN. Incremental backups simply read the pages and compare their LSN to the last backup's LSN. You still need a full backup to recover the incremental changes, however; without a full backup to act as a base, the incremental backups are useless.
948
949.SS " Creating an Incremental Backup "
950
951
952To make an incremental backup, begin with a full backup as usual. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup's target directory. This file contains a line showing the
953.B "to_lsn"
954, which is the database's LSN at the end of the backup. Create the full backup with a command such as the following:
955
956
957.nf
958
959xtrabackup --backup --target-dir=/data/backups/base --datadir=/var/lib/mysql/
960
961.fi
962
963
964.B "TIP:"
965If you want a
966.B "usable"
967full backup, use innobackupex since xtrabackup itself won't copy table definitions, triggers, or anything else that's not .ibd.
968
969
970If you look at the xtrabackup_checkpoints file, you should see some contents similar to the following:
971
972
973.nf
974
975backup_type = full-backuped
976from_lsn = 0
977to_lsn = 1291135
978
979.fi
980
981Now that you have a full backup, you can make an incremental backup based on it. Use a command such as the following:
982
983
984.nf
985
986xtrabackup --backup --target-dir=/data/backups/inc1 \
987 --incremental-basedir=/data/backups/base --datadir=/var/lib/mysql/
988
989.fi
990
991The
992.B "/data/backups/inc1/"
993directory should now contain delta files, such as
994.B "ibdata1.delta"
995and
996.B "test/table1.ibd.delta"
997. These represent the changes since the LSN 1291135. If you examine the
998.B "xtrabackup_checkpoints"
999file in this directory, you should see something similar to the following:
1000
1001
1002.nf
1003
1004backup_type = incremental
1005from_lsn = 1291135
1006to_lsn = 1291340
1007
1008.fi
1009
1010The meaning should be self-evident. It's now possible to use this directory as the base for yet another incremental backup:
1011
1012
1013.nf
1014
1015xtrabackup --backup --target-dir=/data/backups/inc2 \
1016 --incremental-basedir=/data/backups/inc1 --datadir=/var/lib/mysql/
1017
1018.fi
1019
1020.SS " Preparing the Backups "
1021
1022
1023The
1024.B "--prepare"
1025step for incremental backups is not the same as for normal backups. In normal backups, two types of operations are performed to make the database consistent: committed transactions are replayed from the log file against the data files, and uncommitted transactions are rolled back. For technical reasons, you must skip the rollback of uncommitted transactions when preparing a backup that will be used as the base for an incremental backup. You should use the
1026.B "--apply-log-only"
1027option to prevent the rollback phase.
1028
1029
1030.B "WARNING:"
1031If you do not use the
1032.B "--apply-log-only"
1033option to prevent the rollback phase, then your incremental backups will be useless. After transactions have been rolled back, further incremental backups cannot be applied.
1034
1035
1036Beginning with the full backup you created, you can prepare it, and then apply the incremental differences to it. Recall that you have the following backups:
1037
1038
1039.nf
1040
1041/data/backups/base
1042/data/backups/inc1
1043/data/backups/inc2
1044
1045.fi
1046
1047To prepare the base backup, you need to run
1048.B "--prepare"
1049as usual, but prevent the rollback phase:
1050
1051
1052.nf
1053
1054xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base
1055
1056.fi
1057
1058The output should end with some text such as the following:
1059
1060 101107 20:49:43 InnoDB: Shutdown completed; log sequence number 1291135
1061
1062The log sequence number should match the to_lsn of the base backup, which you saw previously.
1063
1064This backup is actually safe to restore as-is now, even though the rollback phase has been skipped. If you restore it and start MySQL, InnoDB will detect that the rollback phase was not performed, and it will do that in the background, as it usually does for a crash recovery upon start. It will notify you that the database was not shut down normally, but the recovery phase should be nearly instantaneous, because the redo log has already been applied to the data files.
1065
1066To apply the first incremental backup to the full backup, you should use the following command:
1067
1068
1069.nf
1070
1071xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \
1072 --incremental-dir=/data/backups/inc1
1073
1074.fi
1075
1076This applies the delta files to the files in /data/backups/base, which rolls them forward in time to the time of the incremental backup. It then applies the redo log as usual to the result. The final data is in /data/backups/base,
1077.B "not"
1078in the incremental directory. You should see some output such as the following:
1079
1080
1081.nf
1082
1083incremental backup from 1291135 is enabled.
1084xtrabackup: cd to /data/backups/base/
1085xtrabackup: This target seems to be already prepared.
1086xtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(1291340)
1087Applying /data/backups/inc1/ibdata1.delta ...
1088Applying /data/backups/inc1/test/table1.ibd.delta ...
1089.... snip
1090101107 20:56:30 InnoDB: Shutdown completed; log sequence number 1291340
1091
1092.fi
1093
1094Again, the LSN should match what you saw from your earlier inspection of the first incremental backup. If you restore the files from /data/backups/base, you should see the state of the database as of the first incremental backup.
1095
1096Preparing the second incremental backup is a similar process: apply the deltas to the (modified) base backup, and you will roll its data forward in time to the point of the second incremental backup:
1097
1098
1099.nf
1100
1101xtrabackup --prepare --apply-log-only --target-dir=/data/backups/base \
1102 --incremental-dir=/data/backups/inc2
1103
1104.fi
1105
1106If you wish to avoid the notice that InnoDB was not shut down normally, when you have applied the desired deltas to the base backup, you can run
1107.B "--prepare"
1108again without disabling the rollback phase.
1109.SH " Partial Backups "
1110
1111
1112xtrabackup supports taking partial backups when
1113.B "innodb_file_per_table"
1114is enabled. There are two ways to create partial backups.
1115
1116For the purposes of this manual page, we will assume that there is a database named 'test' which contains tables named 't1' and 't2'.
1117
1118.SS " Using the --tables Option "
1119
1120
1121The first method is with the
1122.B "--tables"
1123option. The option's value is a regular expression that is matched against the fully qualified tablename, including the database name, in the form
1124.B "databasename.tablename"
1125.
1126
1127To back up only tables in the 'test' database, you can use the following command:
1128
1129 xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/data/backups/ \
1130 --tables="^test[.].*"
1131
1132To back up only the table 'test.t1', you can use the following command:
1133
1134 xtrabackup --backup --datadir=/var/lib/mysql --target-dir=/data/backups/ \
1135 --tables="^test[.]t1"
1136
1137.SS " Using the --tables-file Option "
1138
1139
1140The
1141.B "--tables-file"
1142option specifies a file that can contain multiple table names, one table name per line in the file. Only the tables named in the file will be backed up. Names are matched exactly, case-sensitive, with no pattern or regular expression matching. The table names must be fully qualified, in
1143.B "databasename.tablename"
1144format.
1145
1146.SS " Preparing the Backup "
1147
1148
1149When you use the
1150.B "--prepare"
1151option on a partial backup, you will see warnings about tables that don't exist. That is because these tables exist in the data dictionary inside InnoDB, but the corresponding
1152.B ".ibd"
1153files don't exist. They were not copied into the backup directory. These tables will be removed from the data dictionary, and when you restore the backup and start InnoDB, they will no longer exist and will not cause any errors or warnings to be printed to the log file.
1154
1155An example of the error message you will see during the prepare phase follows.
1156
1157
1158.nf
1159
1160InnoDB: Reading tablespace information from the .ibd files...
1161101107 22:31:30 InnoDB: Error: table 'test1/t'
1162InnoDB: in InnoDB data dictionary has tablespace id 6,
1163InnoDB: but tablespace with that id or name does not exist. It will be removed from data dictionary.
1164
1165.fi
1166
1167.SH " Analyzing Table Statistics "
1168
1169
1170The
1171.B "xtrabackup"
1172binary can analyze InnoDB data files in read-only mode to give statistics about them. To do this, you should use the
1173.B "--stats"
1174option. You can combine this with the
1175.B "--tables"
1176option to limit the files to examine. It also uses the
1177.B "--use-memory="
1178option.
1179
1180You can perform the analysis on a running server, with some chance of errors due to the data being changed during analysis. Or, you can analyze a backup copy of the database. Either way, to use the statistics feature, you need a clean copy of the database including correctly sized log files, so you need to execute with
1181.B "--prepare"
1182twice to use this functionality on a backup.
1183
1184The result of running on a backup might look like the following:
1185
1186
1187.nf
1188
1189<INDEX STATISTICS>
1190 table: test/table1, index: PRIMARY, space id: 12, root page 3
1191 estimated statistics in dictionary:
1192 key vals: 25265338, leaf pages 497839, size pages 498304
1193 real statistics:
1194 level 2 pages: pages=1, data=5395 bytes, data/pages=32%
1195 level 1 pages: pages=415, data=6471907 bytes, data/pages=95%
1196 leaf pages: recs=25958413, pages=497839, data=7492026403 bytes, data/pages=91%
1197
1198.fi
1199
1200This can be interpreted as follows:
1201
1202.HP
1203* The first line simply shows the table and index name and its internal identifiers. If you see an index named
1204.B "GEN_CLUST_INDEX"
1205, that is the table's clustered index, automatically created because you did not explicitly create a PRIMARY KEY.
1206.HP
1207* The
1208.I "estimated statistics in dictionary"
1209information is similar to the data that's gathered through
1210.B "ANALYZE TABLE"
1211inside of InnoDB to be stored as estimated cardinality statistics and passed to the query optimizer
1212.HP
1213* The
1214.I "real statistics"
1215information is the result of scanning the data pages and computing exact information about the index.
1216.HP
1217* The
1218.B "level <X> pages:"
1219output means that the line shows information about pages at that level in the index tree. The larger <X> is, the farther it is from the leaf pages, which are level 0. The first line is the root page.
1220.HP
1221* The
1222.B "leaf pages"
1223output shows the leaf pages, of course. This is where the table's data is stored.
1224.HP
1225* The
1226.B "external pages:"
1227output (not shown) shows large external pages that hold values too long to fit in the row itself, such as long BLOB and TEXT values.
1228.HP
1229* The
1230.I "recs"
1231is the real number of records (rows) in leaf pages.
1232.HP
1233* The
1234.I "pages"
1235is the page count.
1236.HP
1237* The
1238.I "data"
1239is the total size of the data in the pages, in bytes.
1240.HP
1241* The
1242.I "data/pages"
1243is calculated as (data / (pages * PAGE_SIZE)) * 100%. It will never reach 100% because of space reserved for page headers and footers.
1244
1245
1246.SH " Throttling Backups "
1247
1248
1249Although
1250.B "xtrabackup"
1251does not block your database's operation, any backup can add load to the system being backed up. On systems that do not have much spare I/O capacity, it might be helpful to throttle the rate at which
1252.B "xtrabackup"
1253reads and writes data. You can do this with the
1254.B "--throttle"
1255option.
1256
1257In
1258.B "--backup"
1259mode, this option limits the number of pairs of read-and-write operations per second that
1260.B "xtrabackup"
1261will perform. If you are creating an
1262.B "incremental backup"
1263, then the limit is the number of read IO operations per second.
1264
1265By default, there is no throttling, and
1266.B "xtrabackup"
1267reads and writes data as quickly as it can. If you set too strict of a limit on the I/O operations, the backup might be so slow that it will never catch up with the transaction logs that InnoDB is writing, so the backup might never complete.
1268
1269.SH " Working with Binary Logs "
1270
1271
1272The
1273.B "xtrabackup"
1274binary integrates with information that InnoDB stores in its transaction log about the corresponding binary log position for committed transactions. This enables it to print out the binary log position to which a backup corresponds, so you can use it to set up new replication slaves or perform point-in-time recovery.
1275
1276.SS " Finding the Binary Log Position "
1277
1278
1279You can find the binary log position corresponding to a backup performing the
1280.B "--prepare"
1281process. If your backup is from a server with binary logging enabled,
1282.B "xtrabackup"
1283will create a file named
1284.B "xtrabackup_binlog_pos_innodb"
1285in the target directory. This file contains the binary log file name and position of the exact point in the binary log to which the prepared backup corresponds.
1286
1287You will also see output similar to the following during the prepare stage:
1288
1289
1290.nf
1291
1292InnoDB: Last MySQL binlog file position 0 3252710, file name ./mysql-bin.000001
1293... snip ...
1294[notice (again)]
1295 If you use binary log and don't use any hack of group commit,
1296 the binary log position seems to be:
1297InnoDB: Last MySQL binlog file position 0 3252710, file name ./mysql-bin.000001
1298
1299.fi
1300
1301The output should contain the same file name and position as the
1302.B "xtrabackup_binlog_pos_innodb"
1303file. The message about hacking group commit refers to an early implementation of emulated group commit in Percona Server.
1304.B FIXME
1305is this still uncertain, or do we know that it works correctly now?
1306
1307.SS " Point-In-Time Recovery "
1308
1309
1310To perform a point-in-time recovery from an
1311.B "xtrabackup"
1312backup, you should prepare and restore the backup, and then replay binary logs from the point shown in the
1313.B "xtrabackup_binlog_pos_innodb"
1314file.
1315
1316.SS " Setting Up a New Replication Slave "
1317
1318
1319To set up a new replica, you should prepare the backup, and restore it to the data directory of your new replication slave. Then in your
1320.B "CHANGE MASTER TO"
1321command, use the binary log filename and position shown in the
1322.B "xtrabackup_binlog_pos_innodb"
1323file to start replication.
1324
1325.SH " Scripting Backups With xtrabackup "
1326
1327
1328The
1329.B "xtrabackup"
1330tool has several features to enable scripts to control it while they perform related tasks. The
1331.B "innobackupex"
1332script is one example, but xtrabackup is easy to control with your own command-line scripts too.
1333
1334.SS " Suspending After Copying "
1335
1336
1337In
1338.B "backup"
1339mode, xtrabackup normally copies the log files in a background thread, copies the data files in a foreground thread, and then stops the log copying thread and finishes. If you use the
1340.B "--suspend-at-end"
1341option, instead of stopping the log thread and finishing, xtrabackup continues to copy the log files, and creates a file in the target directory called
1342.B "xtrabackup_suspended"
1343. As long as that file exists,
1344.B "xtrabackup"
1345will continue to watch the log files and copy them into the
1346.B "xtrabackup_logfile"
1347in the target directory. When the file is removed,
1348.B "xtrabackup"
1349will finish copying the log file and exit.
1350
1351This functionality is useful for coordinating the InnoDB data backups with other actions. Perhaps the most obvious is copying the table definitions (the .frm files) so that the backup can be restored. You can start
1352.B "xtrabackup"
1353in the background, wait for the
1354.B "xtrabackup_suspended"
1355file to be created, and then copy any other files you need to complete the backup. This is exactly what the
1356.B "innobackupex"
1357tool does (it also copies MyISAM data and other files).
1358
1359.SS " Generating Meta-Data "
1360
1361
1362It is a good idea for the backup to include all the information you need to restore the backup. The
1363.B "xtrabackup"
1364tool can print out the contents of a
1365.B "my.cnf"
1366file that are needed to restore the data and log files. If you add the
1367.B "--print-param"
1368option, it will print out something like the following:
1369
1370
1371.nf
1372
1373# This MySQL options file was generated by XtraBackup.
1374[mysqld]
1375datadir = /data/mysql/
1376innodb_data_home_dir = /data/innodb/
1377innodb_data_file_path = ibdata1:10M:autoextend
1378innodb_log_group_home_dir = /data/innodb-logs/
1379
1380.fi
1381
1382You can redirect this output into a file in the target directory of the backup.
1383
1384.SS " Agreeing on the Source Directory "
1385
1386
1387It's possible that the presence of a defaults file or other factors could cause
1388.B "xtrabackup"
1389to back up data from a different location than you expected. To prevent this, you can use
1390.B "--print-param"
1391to ask it where it will be copying data from. You can use the output to ensure that
1392.B "xtrabackup"
1393and your script are working on the same dataset.
1394
1395.SS " Log Streaming "
1396
1397
1398You can instruct
1399.B "xtrabackup"
1400to omit copying data files, and simply stream the log file to its standard output instead with
1401.B "--log-stream"
1402. This automatically adds the
1403.B "--suspend-at-end"
1404option. Your script can then perform tasks such as streaming remote backups by piping the log files into an SSH connection and copying the data files to another server with a tool such as
1405.B "rsync"
1406or
1407.B "tar4ibd"
1408.
1409
1410.SH " XtraBackup Exit Codes "
1411
1412
1413The
1414.B "xtrabackup"
1415binary exits with the traditional success value of 0 after a backup when no error occurs. If an error occurs during the backup, the exit value is 1.
1416
1417In certain cases, the exit value can be something other than 0 or 1, due to the command-line option code included from the MySQL libraries. An unknown command-line option, for example, will cause an exit code of 255.
1418
1419When an error happens in the
1420.B "main()"
1421function of
1422.B "xtrabackup.c"
1423, when
1424.B "xtrabackup"
1425is preparing to perform the backup, the exit code is -1. This is usually because of a missing or wrong command-line option, failure to open a file or directory that the user specified as a command-line option, or similar. This behavior is changed in xtrabackup 1.4 and later, so it always returns 0 or 1. (However, the MySQL libraries might still exit with a code of 255.)
1426
1427.SH " Implementation Details "
1428
1429
1430This page contains notes on various internal aspects of the
1431.B "xtrabackup"
1432tool's operation.
1433
1434.SS " File Permissions "
1435
1436
1437
1438.B "xtrabackup"
1439opens the source data files in read-write mode, although it does not modify the files. This means that you must run
1440.B "xtrabackup"
1441as a user who has permission to write the data files. The reason for opening the files in read-write mode is that
1442.B "xtrabackup"
1443uses the embedded InnoDB libraries to open and read the files, and InnoDB opens them in read-write mode because it normally assumes it is going to write to them.
1444
1445.SS " Tuning the OS Buffers "
1446
1447
1448Because
1449.B "xtrabackup"
1450reads large amounts of data from the filesystem, it uses
1451.B "posix_fadvise()"
1452where possible, to instruct the operating system not to try to cache the blocks it reads from disk. Without this hint, the operating system would prefer to cache the blocks, assuming that
1453.B "xtrabackup"
1454is likely to need them again, which is not the case. Caching such large files can place pressure on the operating system's virtual memory and cause other processes, such as the database server, to be swapped out. The
1455.B "xtrabackup"
1456tool avoids this with the following hint on both the source and destination files:
1457
1458 posix_fadvise(file, 0, 0, POSIX_FADV_DONTNEED)
1459
1460In addition,
1461.B "xtrabackup"
1462asks the operating system to perform more aggressive read-ahead optimizations on the source files:
1463
1464 posix_fadvise(file, 0, 0, POSIX_FADV_SEQUENTIAL)
1465
1466.SS " Copying Data Files "
1467
1468
1469When copying the data files to the target directory,
1470.B "xtrabackup"
1471reads and writes 1MB of data at a time. This is not configurable. When copying the log file,
1472.B "xtrabackup"
1473reads and writes 512 bytes at a time. This is also not possible to configure, and matches InnoDB's behavior.
1474
1475After reading from the files,
1476.B "xtrabackup"
1477iterates over the 1MB buffer a page at a time, and checks for page corruption on each page with InnoDB's
1478.B "buf_page_is_corrupted()"
1479function. If the page is corrupt, it re-reads and retries up to 10 times for each page. It skips this check on the doublewrite buffer.
1480.SH "AUTHOR"
1481Percona Inc. http://www.percona.com/
1482.SH "REPORTING BUGS"
1483Report bugs to https://bugs.launchpad.net/percona-xtrabackup/+filebug
1484.SH "LICENCE"
1485GPL version 2
1486
01487
=== added file 'plugin/innobase/xtrabackup/innobackupex'
--- plugin/innobase/xtrabackup/innobackupex 1970-01-01 00:00:00 +0000
+++ plugin/innobase/xtrabackup/innobackupex 2011-03-31 06:44:38 +0000
@@ -0,0 +1,2543 @@
1#!/usr/bin/perl -w
2#
3# A script for making backups of InnoDB and MyISAM tables, indexes and .frm
4# files.
5#
6# Copyright 2003, 2009 Innobase Oy and Percona Inc 2009-2011. All Rights Reserved.
7#
8
9use strict;
10use Getopt::Long;
11use File::Spec;
12use Pod::Usage qw(pod2usage);
13use POSIX "strftime";
14use POSIX ":sys_wait_h";
15use POSIX "tmpnam";
16use FileHandle;
17use File::Basename;
18use English qw(-no_match_vars);
19
20# version of this script
21my $innobackup_version = '1.5.1-xtrabackup';
22my $innobackup_script = basename($0);
23
24# copyright notice
25my $copyright_notice =
26"InnoDB Backup Utility v${innobackup_version}; Copyright 2003, 2009 Innobase Oy
27and Percona Inc 2009-2011. All Rights Reserved.
28
29This software is published under
30the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.
31
32";
33
34# required Perl version (5.005)
35my @required_perl_version = (5, 0, 5);
36my $required_perl_version_old_style = 5.005;
37
38# force flush after every write and print
39$| = 1;
40
41######################################################################
42# modifiable parameters
43######################################################################
44
45# maximum number of files in a database directory which are
46# separately printed when a backup is made
47my $backup_file_print_limit = 9;
48
49# timeout in seconds for a reply from mysql
50my $mysql_response_timeout = 900;
51
52# default compression level (this is an argument to ibbackup)
53my $default_compression_level = 1;
54
55# time in seconds after which a dummy query is sent to mysql server
56# in order to keep the database connection alive
57my $mysql_keep_alive_timeout = 1800;
58
59######################################################################
60# end of modifiable parameters
61######################################################################
62
63
64# command line options
65my $option_help = '';
66my $option_version = '';
67my $option_apply_log = '';
68my $option_redo_only = '';
69my $option_copy_back = '';
70my $option_include = '';
71my $option_databases = '';
72my $option_tables_file = '';
73my $option_throttle = '';
74my $option_sleep = '';
75my $option_compress = 999;
76my $option_uncompress = '';
77my $option_export = '';
78my $option_use_memory = '';
79my $option_mysql_password = '';
80my $option_mysql_user = '';
81my $option_mysql_port = '';
82my $option_mysql_socket = '';
83my $option_mysql_host = '';
84my $option_no_timestamp = '';
85my $option_slave_info = '';
86my $option_no_lock = '';
87my $option_ibbackup_binary = 'autodetect';
88
89my $option_defaults_file = '';
90my $option_incremental = '';
91my $option_incremental_basedir = '';
92my $option_incremental_dir = '';
93my $option_incremental_lsn = '';
94my $option_extra_lsndir = '';
95my $option_remote_host = '';
96my $option_stream = '';
97my $option_tmpdir = '';
98
99my $option_tar4ibd = '';
100my $option_force_tar = '';
101my $option_scp_opt = '-Cp -c arcfour';
102
103my $option_parallel = '';
104
105my $option_safe_slave_backup = '';
106my $option_safe_slave_backup_timeout = 300;
107
108# name of the my.cnf configuration file
109#my $config_file = '';
110
111# root of the backup directory
112my $backup_root = '';
113
114# backup directory pathname
115my $backup_dir = '';
116
117# name of the ibbackup suspend-at-end file
118my $suspend_file = '';
119
120# name of the temporary transaction log file during the backup
121my $tmp_logfile = '';
122
123# home directory of innoDB log files
124my $innodb_log_group_home_dir = '';
125
126# backup my.cnf file
127my $backup_config_file = '';
128
129# options from the options file
130my %config;
131
132# options from the backup options file
133#my %backup_config;
134
135# list of databases to be included in a backup
136my %databases_list;
137
138# list of tables to be included in a backup
139my %table_list;
140
141# prefix for output lines
142my $prefix = "$innobackup_script:";
143
144# process id of mysql client program (runs as a child process of this script)
145my $mysql_pid = '';
146
147# mysql server version string
148my $mysql_server_version = '';
149
150# name of the file where stderr of mysql process is directed
151my $mysql_stderr = 'stderr';
152
153# name of the file where stdout of mysql process is directed
154my $mysql_stdout = 'stdout';
155
156# name of the file where binlog position info is written
157my $binlog_info;
158
159# name of the file where slave info is written
160my $slave_info;
161
162# mysql binlog position as given by "SHOW MASTER STATUS" command
163my $mysql_binlog_position = '';
164
165# mysql master's binlog position as given by "SHOW SLAVE STATUS" command
166# run on a slave server
167my $mysql_slave_position = '';
168
169# time of the most recent mysql_check call. (value returned by time() function)
170my $mysql_last_access_time = 0;
171
172# process id of ibbackup program (runs as a child process of this script)
173my $ibbackup_pid = '';
174
175# a counter for numbering mysql connection checks
176my $hello_id = 0;
177
178# the request which has been sent to mysqld, but to which
179# mysqld has not yet replied. Empty string denotes that no
180# request has been sent to mysqld or that mysqld has replied
181# to all requests.
182my $current_mysql_request = '';
183
184# escape sequences for options files
185my %option_value_escapes = ('b' => "\b",
186 't' => "\t",
187 'n' => "\n",
188 'r' => "\r",
189 "\\" => "\\",
190 's' => ' ');
191
192# signal that is sent to child processes when they are killed
193my $kill_signal = 15;
194
195# current local time
196my $now;
197
198# incremental backup base directory
199my $incremental_basedir = '';
200
201my $src_name;
202my $dst_name;
203my $win = ($^O eq 'MSWin32' ? 1 : 0);
204my $CP_CMD = ($win eq 1 ? "copy /Y" : "cp -p");
205my $xtrabackup_binary_file = 'xtrabackup_binary';
206######################################################################
207# program execution begins here
208######################################################################
209
210# check command-line args
211check_args();
212
213# print program version and copyright
214print_version();
215
216# initialize global variables and perform some checks
217if ($option_copy_back) {
218 $option_ibbackup_binary = 'xtrabackup_51';
219} elsif ($option_apply_log) {
220 # Read XtraBackup version from backup dir
221 if (-e "$backup_dir/$xtrabackup_binary_file") {
222 # Read XtraBackup version from file
223 open XTRABACKUP_BINARY, "$backup_dir/$xtrabackup_binary_file"
224 or die "Cannot open file $backup_dir/$xtrabackup_binary_file: $!\n";
225 $option_ibbackup_binary = <XTRABACKUP_BINARY>;
226 close XTRABACKUP_BINARY;
227 }
228
229 else {
230 if( $option_ibbackup_binary eq "autodetect" ){
231 # Try to connect MySQL and get the version
232 print "option_ibbackup_binary is autodetect, trying to connect to MySQL\n";
233 my $options = get_mysql_options();
234 $mysql_pid = open(*MYSQL_WRITER, "| mysql $options >$mysql_stdout 2>$mysql_stderr ");
235 print "Connected to MySQL with pid $mysql_pid\n";
236 sleep 1;
237 if ($mysql_pid && $mysql_pid == waitpid($mysql_pid, &WNOHANG)) {
238 my $reason = `cat $mysql_stderr`;
239 $mysql_pid = '';
240 # Failed to connect to MySQL
241 die "Failed to connect to MySQL server to detect version.\nYou must set xtrabackup version to use with --ibbackup option.\nPossible values are xtrabackup_51 (for MySQL 5.0 and 5.1) or xtrabackup (for MySQL 5.1 with InnoDB plugin or Percona Server)\n";
242 }
243 else{
244 mysql_close();
245 print "Connected successfully\n";
246 $option_ibbackup_binary = set_xtrabackup_version();
247 }
248 }
249 }
250} else {
251 $option_ibbackup_binary = set_xtrabackup_version();
252}
253init();
254
255my $ibbackup_exit_code = 0;
256
257if ($option_copy_back) {
258 # copy files from backup directory back to their original locations
259 copy_back();
260} elsif ($option_apply_log) {
261 # expand data files in backup directory by applying log files to them
262 apply_log();
263} else {
264 # make a backup of InnoDB and MyISAM tables, indexes and .frm files.
265 $ibbackup_exit_code = backup();
266 open XTRABACKUP_BINARY, "> $backup_dir/$xtrabackup_binary_file"
267 or die "Cannot open file $backup_dir/$xtrabackup_binary_file: $!\n";
268 print XTRABACKUP_BINARY $option_ibbackup_binary;
269 close XTRABACKUP_BINARY;
270}
271
272$now = current_time();
273
274if ($option_stream eq 'tar') {
275 print STDERR "$prefix You must use -i (--ignore-zeros) option for extraction of the tar stream.\n";
276}
277
278if ( $ibbackup_exit_code == 0 ) {
279 # program has completed successfully
280 print STDERR "$now $prefix completed OK!\n";
281}
282else {
283 print STDERR "$now $prefix $option_ibbackup_binary failed! (exit code $ibbackup_exit_code) The backup may not be complete.\n";
284}
285
286exit $ibbackup_exit_code;
287
288######################################################################
289# end of program execution
290######################################################################
291
292
293#
294# print_version subroutine prints program version and copyright.
295#
296sub print_version {
297 printf(STDERR $copyright_notice);
298}
299
300
301#
302# usage subroutine prints instructions of how to use this program to stdout.
303#
304sub usage {
305 my $msg = shift || '';
306 pod2usage({ -msg => $msg, -verbose => 1});
307 return 0;
308}
309
310
311#
312# return current local time as string in form "070816 12:23:15"
313#
314sub current_time {
315 return strftime("%y%m%d %H:%M:%S", localtime());
316}
317
318
319#
320# Die subroutine kills all child processes and exits this process.
321# This subroutine takes the same argument as the built-in die function.
322# Parameters:
323# message string which is printed to stdout
324#
325sub Die {
326 my $message = shift;
327 my $extra_info = '';
328
329 # kill all child processes of this process
330 kill_child_processes();
331
332 if ($current_mysql_request) {
333 $extra_info = " while waiting for reply to MySQL request:" .
334 " '$current_mysql_request'";
335 }
336 die "$prefix Error: $message$extra_info";
337}
338
339
340#
341# backup subroutine makes a backup of InnoDB and MyISAM tables, indexes and
342# .frm files. It connects to the database server and runs ibbackup as a child
343# process.
344#
345sub backup {
346 my $orig_datadir = get_option(\%config, 'mysqld', 'datadir');
347
348 # check that we can connect to the database. This done by
349 # connecting, issuing a query, and closing the connection.
350 mysql_open();
351 mysql_check();
352 mysql_close();
353
354 # start ibbackup as a child process
355 start_ibbackup();
356
357 # wait for ibbackup to suspend itself
358 if (!$option_remote_host && !$option_stream) {
359 wait_for_ibbackup_suspend();
360 }
361
362 if (!$option_incremental) {
363 # connect to database
364 mysql_open();
365
366 if ( $option_safe_slave_backup ) {
367 wait_for_safe_slave();
368 }
369
370 # flush tables with read lock
371 mysql_check();
372 mysql_lockall() if !$option_no_lock;
373
374 }
375
376 # backup .frm, .MRG, .MYD, .MYI, .TRG, .TRN, .ARM, .ARZ, .CSM, .CSV and .opt files
377 backup_files();
378
379 # resume ibbackup and wait till it has finished
380 my $ibbackup_exit_code = resume_ibbackup();
381
382 if (!$option_incremental) {
383 # release read locks on all tables
384 mysql_unlockall() if !$option_no_lock;
385
386 if ( $option_safe_slave_backup ) {
387 print STDERR "$prefix: Starting slave SQL thread\n";
388 mysql_send('START SLAVE SQL_THREAD;');
389 }
390
391 # disconnect from database
392 mysql_close();
393 }
394
395 if ($option_remote_host) {
396 system("scp $option_scp_opt '$tmp_logfile' '$option_remote_host:$backup_dir/xtrabackup_logfile'")
397 and Die "Failed to scp file '$option_remote_host:$backup_dir/xtrabackup_logfile': $!";
398 unlink $tmp_logfile || Die "Failed to delete '$tmp_logfile': $!";
399
400 system("scp $option_scp_opt '$orig_datadir/xtrabackup_checkpoints' '$option_remote_host:$backup_dir/xtrabackup_checkpoints'")
401 and Die "Failed to scp file '$option_remote_host:$backup_dir/xtrabackup_checkpoints': $!";
402 unlink "$orig_datadir/xtrabackup_checkpoints" || Die "Failed to delete '$orig_datadir/xtrabackup_checkpoints': $!";
403 } elsif ($option_stream eq 'tar') {
404 system("cd $option_tmpdir; tar chf - xtrabackup_logfile")
405 and Die "Failed to stream 'xtrabackup_logfile': $!";
406 unlink $tmp_logfile || Die "Failed to delete '$tmp_logfile': $!";
407
408 system("cd $orig_datadir; tar chf - xtrabackup_checkpoints")
409 and Die "Failed to stream 'xtrabackup_checkpoints': $!";
410 unlink "$orig_datadir/xtrabackup_checkpoints" || Die "Failed to delete '$orig_datadir/xtrabackup_checkpoints': $!";
411 }
412
413 print STDERR "\n$prefix Backup created in directory '$backup_dir'\n";
414 if ($mysql_binlog_position) {
415 print STDERR "$prefix MySQL binlog position: $mysql_binlog_position\n";
416 }
417 if ($mysql_slave_position && $option_slave_info) {
418 print STDERR "$prefix MySQL slave binlog position: $mysql_slave_position\n";
419 }
420
421 return $ibbackup_exit_code;
422}
423
424
425#
426# are_equal_innodb_data_file_paths subroutine checks if the given
427# InnoDB data file option values are equal.
428# Parameters:
429# str1 InnoDB data file path option value
430# str2 InnoDB data file path option value
431# Return value:
432# 1 if values are equal
433# 0 otherwise
434#
435sub are_equal_innodb_data_file_paths {
436 my $str1 = shift;
437 my $str2 = shift;
438 my @array1 = split(/;/, $str1);
439 my @array2 = split(/;/, $str2);
440
441 if ($#array1 != $#array2) { return 0; }
442
443 for (my $i = 0; $i <= $#array1; $i++) {
444 my @def1 = split(/:/, $array1[$i]);
445 my @def2 = split(/:/, $array2[$i]);
446
447 if ($#def1 != $#def2) { return 0; }
448
449 for (my $j = 0; $j <= $#def1; $j++) {
450 if ($def1[$j] ne $def2[$j]) { return 0; }
451 }
452 }
453 return 1;
454}
455
456
457#
458# is_in_array subroutine checks if the given string is in the array.
459# Parameters:
460# str a string
461# array_ref a reference to an array of strings
462# Return value:
463# 1 if string is in the array
464# 0 otherwise
465#
466sub is_in_array {
467 my $str = shift;
468 my $array_ref = shift;
469
470 if (grep { $str eq $_ } @{$array_ref}) {
471 return 1;
472 }
473 return 0;
474}
475
476
477#
478# copy_back subroutine copies data and index files from backup directory
479# back to their original locations.
480#
481sub copy_back {
482 my $orig_datadir = get_option(\%config, 'mysqld', 'datadir');
483 my $orig_ibdata_dir =
484 get_option(\%config, 'mysqld', 'innodb_data_home_dir');
485 my $orig_innodb_data_file_path =
486 get_option(\%config, 'mysqld', 'innodb_data_file_path');
487 my $orig_iblog_dir =
488 get_option(\%config, 'mysqld', 'innodb_log_group_home_dir');
489 my $excluded_files =
490 '^(\.\.?|backup-my\.cnf|xtrabackup_logfile|mysql-std(err|out)|.*\.ibz)$';
491 my @ibdata_files;
492 my $iblog_files = '^ib_logfile.*$';
493 my $compressed_data_file = '.*\.ibz$';
494 my $file;
495 my $backup_innodb_data_file_path;
496 # check that original data directory exists
497 if (! -d $orig_datadir) {
498 Die "Original data directory '$orig_datadir' does not exist!";
499 }
500 # check that original InnoDB data directory exists
501 if (! -d $orig_ibdata_dir) {
502 Die "Original InnoDB data directory '$orig_ibdata_dir' does not exist!";
503 }
504 # check that original InnoDB log directory exists
505 if (! -d $orig_iblog_dir) {
506 Die "Original InnoDB log directory '$orig_iblog_dir' does not exist!";
507 }
508
509 # check that the original options file and the backup options file have
510 # the same value for "innodb_data_file_path" option
511 #$backup_innodb_data_file_path =
512 # get_option(\%backup_config, 'mysqld', 'innodb_data_file_path');
513 #if (!are_equal_innodb_data_file_paths($orig_innodb_data_file_path,
514 # $backup_innodb_data_file_path)
515 #) {
516 # Die "The value of 'innodb_data_file_path' option in the original "
517 # . "my.cnf file '$config_file' is different from the value "
518 # . "in the backup my.cnf file '$backup_config_file'.\n(original: "
519 # . "'$orig_innodb_data_file_path')\n"
520 # . "(backup: '$backup_innodb_data_file_path')";
521 #}
522
523 # make a list of all ibdata files in the backup directory and all
524 # directories in the backup directory under which there are ibdata files
525 foreach my $a (split(/;/, $orig_innodb_data_file_path)) {
526 my $path = (split(/:/,$a))[0];
527 my $filename = (split(/\/+/, $path))[-1];
528
529 # check that the backup data file exists
530 if (! -e "$backup_dir/$filename") {
531 if (-e "$backup_dir/${filename}.ibz") {
532 Die "Backup data file '$backup_dir/$filename' does not exist, but "
533 . "its compressed copy '${path}.ibz' exists. Check "
534 . "that you have run '$innobackup_script --apply-log --uncompress "
535 . "...' before attempting '$innobackup_script --copy-back ...' !";
536 } else {
537 Die "Backup data file '$backup_dir/$filename' does not exist.";
538 }
539 }
540
541 if (!is_in_array($filename, \@ibdata_files)) {
542 push(@ibdata_files, $filename);
543 }
544 }
545
546 # copy files from backup dir to their original locations
547
548 # copy files to original data directory
549 opendir(DIR, $backup_dir)
550 || Die "Can't open directory '$backup_dir': $!\n";
551 print STDERR "$prefix Starting to copy MyISAM tables, indexes,\n";
552 print STDERR "$prefix .MRG, .TRG, .TRN, .ARM, .ARZ, .CSM, .CSV, .opt, and .frm files\n";
553 print STDERR "$prefix in '$backup_dir'\n";
554 print STDERR "$prefix back to original data directory '$orig_datadir'\n";
555 while (defined($file = readdir(DIR))) {
556 if ($file =~ /$excluded_files/) { next; }
557 if (is_in_array($file, \@ibdata_files)) { next; }
558 if ($file =~ /$iblog_files/) { next; }
559 if (-d "$backup_dir/$file") {
560 my $subdir = "$backup_dir/$file";
561 my $file2;
562
563 print STDERR "$prefix Copying directory '$subdir'\n";
564 if (! -x "\"".escape_path("$orig_datadir/$file")."\"") {
565 system("mkdir \"".escape_path("$orig_datadir/$file")."\"")
566 and Die "Failed to create directory "
567 . "'$orig_datadir/$file' : $!";
568 }
569 opendir(SUBDIR, "$subdir")
570 || Die "Can't open directory '$subdir': $!\n";
571 while (defined($file2 = readdir(SUBDIR))) {
572 if (-d "$subdir/$file2") { next; }
573 if ($file2 =~ /$compressed_data_file/) { next; }
574 $src_name = escape_path("$subdir/$file2");
575 $dst_name = escape_path("$orig_datadir/$file");
576 system("$CP_CMD \"$src_name\" \"$dst_name\"")
577 and Die "Failed to copy file '$file': $!";
578 }
579 closedir(SUBDIR);
580 } else {
581 print STDERR "$prefix Copying file " .
582 "'$backup_dir/$file'\n";
583 $src_name = escape_path("$backup_dir/$file");
584 $dst_name = escape_path("$orig_datadir");
585 system("$CP_CMD \"$src_name\" \"$dst_name\"")
586 and Die "Failed to copy file '$file': $!";
587 }
588 }
589 closedir(DIR);
590
591 # copy InnoDB data files to original InnoDB data directory
592 print STDERR "\n$prefix Starting to copy InnoDB tables and indexes\n";
593 print STDERR "$prefix in '$backup_dir'\n";
594 print STDERR "$prefix back to original InnoDB data directory '$orig_ibdata_dir'\n";
595 foreach my $a (split(/;/, $orig_innodb_data_file_path)) {
596 # get the relative pathname of a data file
597 my $path = (split(/:/,$a))[0];
598 my $filename = (split(/\/+/, $path))[-1];
599 print STDERR "$prefix Copying file '$backup_dir/$filename'\n";
600 $src_name = escape_path("$backup_dir/$filename");
601 $dst_name = escape_path("$orig_ibdata_dir/$path");
602 system("$CP_CMD \"$src_name\" \"$dst_name\"")
603 and Die "Failed to copy file '$filename': $!";
604 }
605
606 # copy InnoDB log files to original InnoDB log directory
607 opendir(DIR, $backup_dir)
608 || Die "Can't open directory '$backup_dir': $!\n";
609 print STDERR "\n$prefix Starting to copy InnoDB log files\n";
610 print STDERR "$prefix in '$backup_dir'\n";
611 print STDERR "$prefix back to original InnoDB log directory '$orig_iblog_dir'\n";
612 while (defined($file = readdir(DIR))) {
613 if ($file =~ /$iblog_files/ && -f "$backup_dir/$file") {
614 print STDERR "$prefix Copying file '$backup_dir/$file'\n";
615 $src_name = escape_path("$backup_dir/$file");
616 $dst_name = escape_path("$orig_iblog_dir");
617 system("$CP_CMD \"$src_name\" \"$dst_name\"")
618 and Die "Failed to copy file '$file': $!";
619 }
620 }
621 closedir(DIR);
622
623 print STDERR "$prefix Finished copying back files.\n\n";
624}
625
626
627#
628# apply_log subroutine prepares a backup for starting database server
629# on the backup. It applies InnoDB log files to the InnoDB data files.
630#
631sub apply_log {
632 my $rcode;
633 my $cmdline = '';
634 my $options = '';
635
636 if ($option_defaults_file) {
637 $options = $options . " --defaults-file=\"$option_defaults_file\" ";
638 }
639
640 $options = $options . "--prepare --target-dir=$backup_dir";
641
642 if ($option_uncompress) {
643 $options = $options . ' --uncompress';
644 }
645 if ($option_export) {
646 $options = $options . ' --export';
647 }
648 if ($option_redo_only) {
649 $options = $options . ' --apply-log-only';
650 }
651 if ($option_use_memory) {
652 $options = $options . " --use-memory=$option_use_memory";
653 }
654
655 if ($option_incremental_dir) {
656 $options = $options . " --incremental-dir=$option_incremental_dir";
657 }
658
659 # run ibbackup as a child process
660 $cmdline = "$option_ibbackup_binary $options";
661 $now = current_time();
662 print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n\n";
663 $rcode = system("$cmdline");
664 if ($rcode) {
665 # failure
666 Die "\n$prefix ibbackup failed";
667 }
668
669 # We should not create ib_logfile files if we prepare for following incremental applies
670 # Also we do not prepare ib_logfile if we applied incremental changes
671 if (!( ($option_redo_only) or ($option_incremental_dir))) {
672 $now = current_time();
673 print STDERR "\n$now $prefix Restarting xtrabackup with command: $cmdline\nfor creating ib_logfile*\n\n";
674 $rcode = system("$cmdline");
675 if ($rcode) {
676 # failure
677 Die "\n$prefix xtrabackup (2nd execution) failed";
678 }
679 }
680}
681
682
683#
684# wait_for_ibbackup_suspend subroutine waits until ibbackup has suspended
685# itself.
686#
687sub wait_for_ibbackup_suspend {
688 print STDERR "$prefix Waiting for ibbackup (pid=$ibbackup_pid) to suspend\n";
689 print STDERR "$prefix Suspend file '$suspend_file'\n\n";
690 for (;;) {
691 sleep 2;
692 last if -e $suspend_file;
693
694 # check that ibbackup child process is still alive
695 if ($ibbackup_pid == waitpid($ibbackup_pid, &WNOHANG)) {
696 $ibbackup_pid = '';
697 Die "ibbackup child process has died";
698 }
699 }
700 $now = current_time();
701 print STDERR "\n$now $prefix Continuing after ibbackup has suspended\n";
702}
703
704
705#
706# resume_ibbackup subroutine signals ibbackup to complete its execution
707# by deleting the 'ibbackup_suspended' file.
708#
709sub resume_ibbackup {
710 print STDERR "$prefix Resuming ibbackup\n\n";
711 unlink $suspend_file || Die "Failed to delete '$suspend_file': $!";
712
713 # wait for ibbackup to finish
714 waitpid($ibbackup_pid, 0);
715 $ibbackup_pid = '';
716 return $CHILD_ERROR >> 8;
717}
718
719
720#
721# start_ibbackup subroutine spawns a child process running ibbackup
722# program for backing up InnoDB tables and indexes.
723#
724sub start_ibbackup {
725 my $options = '';
726 my $cmdline = '';
727 my $pid = undef;
728
729 if ($option_defaults_file) {
730 $options = $options . " --defaults-file=\"$option_defaults_file\" ";
731 }
732
733 $options = $options . "--backup --suspend-at-end";
734
735 if (!$option_remote_host && !$option_stream) {
736 $options = $options . " --target-dir=$backup_dir";
737 } else {
738 #(datadir) for 'xtrabackup_suspended' and 'xtrabackup_checkpoints'
739 $options = $options . " --log-stream --target-dir=./";
740 }
741
742 # prepare command line for running ibbackup
743 if ($option_throttle) {
744 $options = $options . " --throttle=$option_throttle";
745 }
746 if ($option_sleep) {
747 $options = $options . " --sleep=$option_sleep";
748 }
749 if ($option_compress) {
750 $options = $options . " --compress=$option_compress";
751 }
752 if ($option_use_memory) {
753 $options = $options . " --use-memory=$option_use_memory";
754 }
755 if ($option_include) {
756 $options = $options . " --tables='$option_include'";
757 }
758 if ($option_extra_lsndir) {
759 $options = $options . " --extra-lsndir='$option_extra_lsndir'";
760 }
761
762 if ($option_incremental) {
763 if($option_incremental_lsn) {
764 $options = $options . " --incremental-lsn='$option_incremental_lsn'";
765 } else {
766 $options = $options . " --incremental-basedir='$incremental_basedir'";
767 }
768 }
769
770 if ($option_tables_file) {
771 $options = $options . " --tables_file='$option_tables_file'";
772 }
773 if ($option_parallel) {
774 $options = $options. " --parallel=$option_parallel";
775 }
776 $cmdline = "$option_ibbackup_binary $options";
777
778 # run ibbackup as a child process
779 $now = current_time();
780 print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n";
781 if (defined($pid = fork)) {
782 if ($pid) {
783 # parent process
784 $ibbackup_pid = $pid;
785
786 if($option_remote_host || $option_stream) {
787 #direct copy to remote
788 my $orig_datadir = get_option(\%config, 'mysqld', 'datadir');
789 my $orig_ibdata_dir =
790 get_option(\%config, 'mysqld', 'innodb_data_home_dir');
791 my $orig_innodb_data_file_path =
792 get_option(\%config, 'mysqld', 'innodb_data_file_path');
793 my $innodb_flush_method =
794 get_option(\%config, 'mysqld', 'innodb_flush_method');
795 my $innodb_use_odirect;
796 $innodb_use_odirect = 1 if $innodb_flush_method =~ m/O_DIRECT/i;
797
798 my $subdir;
799 my @list;
800
801 if($option_remote_host) {
802 if (system("ssh $option_remote_host test -e $backup_dir/ib_logfile0")
803 == 0) {
804 print STDERR "$prefix Remove $option_remote_host:$backup_dir/ib_logfile*\n";
805 system("ssh $option_remote_host rm $backup_dir/ib_logfile\*")
806 and Die "Failed to rm file '$backup_dir/ib_logfile*': $!";
807 }
808 }
809
810 wait_for_ibbackup_suspend();
811
812 #InnoDB data files from original InnoDB data directory
813 print STDERR "\n$prefix Starting to backup InnoDB tables and indexes\n";
814 if($option_remote_host) {
815 print STDERR "$prefix to '$backup_dir'\n";
816 }
817 print STDERR "$prefix from original InnoDB data directory '$orig_ibdata_dir'\n";
818 foreach my $a (split(/;/, $orig_innodb_data_file_path)) {
819 my $path = (split(/:/,$a))[0];
820 $path=~s/([\$\\\" ])/\\$1/g;
821 if($option_remote_host) {
822 print STDERR "$prefix Backing up file '$orig_ibdata_dir/$path'\n";
823 system("scp $option_scp_opt '$orig_ibdata_dir/$path' '$option_remote_host:$backup_dir/$path'")
824 and Die "Failed to scp file '$path': $!";
825 } elsif($option_stream eq 'tar') {
826 my $ret = 0;
827 my $tarcmd;
828 print STDERR "$prefix Backing up as tar stream '$path'\n";
829 if (!$option_tar4ibd) {
830 $tarcmd = "tar chf - -b 32";
831 } else {
832 $tarcmd = "tar4ibd";
833 if ($innodb_use_odirect) {
834 $tarcmd = "$tarcmd -d";
835 }
836 $tarcmd = "$tarcmd -c";
837 }
838 $ret = system("cd $orig_ibdata_dir; $tarcmd $path") >> 8;
839 if ($ret == 1) {
840 print STDERR "$prefix If you use GNU tar, this warning can be ignored.\n";
841 } elsif ($ret != 0) {
842 print STDERR "$prefix tar returned with exit code $ret.\n";
843 if ( -e "$orig_ibdata_dir/$path" ) {
844 Die "Failed to stream '$orig_ibdata_dir/$path': $!";
845 }
846 else {
847 print STDERR "$prefix Ignoring nonexistent file '$orig_ibdata_dir/$path'.\n";
848 }
849 }
850 }
851 }
852
853 #copy *.ibd files
854 opendir(DIR, $orig_datadir)
855 || Die "Can't open directory '$orig_datadir': $!\n";
856 while (defined($subdir = readdir(DIR))) {
857 my $print_each_file = 0;
858 my $file_c;
859 my $file;
860 if ($subdir eq '.' || $subdir eq '..') { next; }
861 next unless -d "$orig_datadir/$subdir";
862 next unless check_if_required($subdir);
863
864 @list = glob("$orig_datadir/$subdir/" . '*.ibd');
865
866 $file_c = @list;
867 if ($file_c <= $backup_file_print_limit) {
868 $print_each_file = 1;
869 } else {
870 print STDERR "$prefix Backing up files " .
871 "'$orig_datadir/$subdir/*.ibd' ($file_c files)\n";
872 }
873 foreach $file (@list) {
874 next unless check_if_required($subdir, $file);
875 if($option_include) {
876 my $table_name;
877
878 $table_name = substr($file, rindex($file, '/'));
879 $table_name = substr($table_name, 1, rindex($table_name, '.') - 1);
880 $table_name = $subdir . "." . $table_name;
881
882 if (!($table_name =~ /$option_include/)) {
883 print STDERR "'$file' is skipped.\n";
884 next;
885 }
886 }
887
888 if ($print_each_file) {
889 print STDERR "$prefix Backing up file '$file'\n";
890 }
891 if($option_remote_host) {
892 if (system("ssh $option_remote_host test -e $backup_dir/$subdir")
893 != 0) {
894 system("ssh $option_remote_host mkdir $backup_dir/$subdir");
895 }
896 system("scp $option_scp_opt '$file' '$option_remote_host:$backup_dir/$subdir/'")
897 and Die "Failed to scp file '$file': $!";
898 } elsif($option_stream eq 'tar') {
899 my $ret = 0;
900 my $file_name = substr($file, rindex($file, '/') + 1);
901 $file_name=~s/([\$\\\" ])/\\$1/g;
902 if (!$option_tar4ibd) {
903 $ret = system("cd $orig_datadir; tar chf - -b 32 $subdir/$file_name") >> 8;
904 } else {
905 $ret = system("cd $orig_datadir; tar4ibd -c $subdir/$file_name") >> 8;
906 }
907 if ($ret == 1) {
908 print STDERR "$prefix If you use GNU tar, this warning can be ignored.\n";
909 } elsif ($ret != 0) {
910 print STDERR "$prefix tar returned with exit code $ret.\n";
911 if ( -e "$orig_datadir/$subdir/$file_name" ) {
912 Die "Failed to stream '$orig_datadir/$subdir/$file_name': $!";
913 }
914 else {
915 print STDERR "$prefix Ignoring nonexistent file '$orig_datadir/$subdir/$file_name'.\n";
916 }
917 }
918 }
919 }
920 }
921 closedir(DIR);
922 }
923 } else {
924 if($option_remote_host || $option_stream) {
925 open(STDOUT, "> $tmp_logfile")
926 || Die "Failed to open file '$tmp_logfile': $!"
927 }
928
929 # child process
930 exec($cmdline) || Die "Failed to exec ibbackup: $!";
931 }
932 } else {
933 Die "failed to fork ibbackup child process: $!";
934 }
935}
936
937
938#
939# get_mysql_options subroutine returns the options to mysql client program
940# as a string. The options are determined from the options given by the
941# user to innobackup.
942#
943sub get_mysql_options {
944 my $options = '';
945
946 # this option has to be first
947 if ($option_defaults_file) {
948 $options = "$options --defaults-file='$option_defaults_file'";
949 }
950
951 if ($option_mysql_password) {
952 $options = "$options --password='$option_mysql_password'";
953 }
954 if ($option_mysql_user) {
955 $options = "$options --user='$option_mysql_user'";
956 }
957 if ($option_mysql_host) {
958 $options = "$options --host='$option_mysql_host'";
959 }
960 if ($option_mysql_port) {
961 $options = "$options --port='$option_mysql_port'";
962 }
963 if ($option_mysql_socket) {
964 $options = "$options --socket='$option_mysql_socket'";
965 }
966 $options = "$options --unbuffered --";
967 return $options;
968}
969
970
971#
972# mysql_open subroutine starts mysql as a child process with
973# a pipe connection.
974#
975sub mysql_open {
976 my $options = get_mysql_options();
977 # run mysql as a child process with a pipe connection
978 $now = current_time();
979 print STDERR "$now $prefix Starting mysql with options: $options\n";
980 $mysql_pid = open(*MYSQL_WRITER, "| mysql $options >$mysql_stdout 2>$mysql_stderr ") or Die "Failed to spawn mysql child process: $!";
981 MYSQL_WRITER->autoflush(1);
982 $now = current_time();
983 print STDERR "$now $prefix Connected to database with mysql child process (pid=$mysql_pid)\n";
984
985 mysql_check();
986}
987
988
989#
990# mysql_check subroutine checks that the connection to mysql child process
991# is ok.
992#
993sub mysql_check {
994 my $mysql_pid_copy = $mysql_pid;
995
996 # send a dummy query to mysql child process
997 $hello_id++;
998 my $hello_message = "innobackup hello $hello_id";
999 print MYSQL_WRITER "select '$hello_message';\n"
1000 or Die "Connection to mysql child process failed: $!";
1001
1002 # wait for reply
1003 eval {
1004 local $SIG{ALRM} = sub { die "alarm clock restart" };
1005 my $stdout = '';
1006 my $stderr = '';
1007 alarm $mysql_response_timeout;
1008 while (index($stdout, $hello_message) < 0) {
1009 sleep 2;
1010 if ($mysql_pid && $mysql_pid == waitpid($mysql_pid, &WNOHANG)) {
1011 my $reason = `cat $mysql_stderr`;
1012 $mysql_pid = '';
1013 die "mysql child process has died: $reason";
1014 }
1015 $stdout = `cat $mysql_stdout`;
1016 $stderr = `cat $mysql_stderr | grep -v ^Warning`;
1017 if ($stderr) {
1018 # mysql has reported an error, do exit
1019 die "mysql error: $stderr";
1020 }
1021 }
1022 alarm 0;
1023 };
1024 if ($@ =~ /alarm clock restart/) {
1025 Die "Connection to mysql child process (pid=$mysql_pid_copy) timedout."
1026 . " (Time limit of $mysql_response_timeout seconds exceeded."
1027 . " You may adjust time limit by editing the value of parameter"
1028 . " \"\$mysql_response_timeout\" in this script.)";
1029 } elsif ($@) { Die $@; }
1030
1031 $mysql_last_access_time = time();
1032}
1033
1034
1035#
1036# mysql_keep_alive subroutine tries to keep connection to the mysqld database
1037# server alive by sending a dummy query when the connection has been idle
1038# for the specified time.
1039#
1040sub mysql_keep_alive {
1041 if ((time() - $mysql_last_access_time) > $mysql_keep_alive_timeout) {
1042 # too long idle, send a dummy query
1043 mysql_check();
1044 }
1045}
1046
1047
1048#
1049# mysql_send subroutine send a request string to mysql child process.
1050# This subroutine appends a newline character to the request and checks
1051# that mysqld receives the query.
1052# Parameters:
1053# request request string
1054#
1055sub mysql_send {
1056 my $request = shift;
1057
1058 $current_mysql_request = $request;
1059 print MYSQL_WRITER "$request\n";
1060 mysql_check();
1061 $current_mysql_request = '';
1062}
1063
1064
1065#
1066# mysql_close subroutine terminates mysql child process gracefully.
1067#
1068sub mysql_close {
1069 print MYSQL_WRITER "quit\n";
1070 $now = current_time();
1071 print STDERR "$now $prefix Connection to database server closed\n";
1072 $mysql_pid = '';
1073}
1074
1075
1076#
1077# write_binlog_info subroutine retrieves MySQL binlog position and
1078# saves it in a file. It also prints it to stdout.
1079#
1080sub write_binlog_info {
1081 my @lines;
1082 my @info_lines = ();
1083 my $position = '';
1084 my $filename = '';
1085
1086 # get binlog position
1087 mysql_send "SHOW MASTER STATUS;";
1088
1089 # get "show master status" output lines (2) from mysql output
1090 file_to_array($mysql_stdout, \@lines);
1091 foreach my $line (@lines) {
1092 if ($line =~ m/innobackup hello/) {
1093 # this is a hello message, ignore it
1094 } else {
1095 # this is output line from "show master status"
1096 push(@info_lines, $line);
1097 }
1098 }
1099
1100 # write binlog info file
1101 if (!defined $info_lines[1]) {
1102 $info_lines[1] = "";
1103 }
1104 if (!$option_remote_host) {
1105 open(FILE, ">$binlog_info") ||
1106 Die "Failed to open file '$binlog_info': $!";
1107 } else {
1108 open(FILE, "| ssh $option_remote_host 'cat > $binlog_info'") ||
1109 Die "Failed to open file '$option_remote_host:$binlog_info': $!";
1110 }
1111 print FILE "$info_lines[1]\n";
1112 close(FILE);
1113
1114 if ($option_stream eq 'tar') {
1115 system("cd $option_tmpdir; tar chf - xtrabackup_binlog_info")
1116 and Die "Failed to stream 'xtrabackup_binlog_info': $!";
1117 unlink $binlog_info || Die "Failed to delete '$binlog_info': $!";
1118 }
1119
1120 # get the name of the last binlog file and position in it
1121 ($filename, $position) = $info_lines[1] =~ /^\s*([^\s]+)\s+(.*)$/;
1122
1123 if (defined $filename && defined $position) {
1124 $mysql_binlog_position = "filename '$filename', position $position";
1125 } else {
1126 $mysql_binlog_position = "filename '', position ";
1127 }
1128}
1129
1130
1131#
1132# write_slave_info subroutine retrieves MySQL binlog position of the
1133# master server in a replication setup and saves it in a file. It
1134# also saves it in $msql_slave_position variable.
1135#
1136sub write_slave_info {
1137 my @lines;
1138 my @info_lines;
1139 my $position = '';
1140 my $filename = '';
1141 my $master= '';
1142
1143 # get slave status. Use single quotes here, otherwise
1144 # \G is evaluated as a control character.
1145 mysql_send 'SHOW SLAVE STATUS\G;';
1146
1147 # get output of the "show slave status" command from mysql output
1148 # and extract binlog position of the master server
1149 file_to_array($mysql_stdout, \@lines);
1150 for (@lines) {
1151 $master = $1 if /Master_Host:\s*(\S*)\s*$/;
1152 $filename = $1 if /Master_Log_File:\s*(\S*)\s*$/;
1153 $position = $1 if /Master_Log_Pos:\s*(\S*)\s*$/;
1154 }
1155
1156 # print slave status to a file
1157 if (!$option_remote_host) {
1158 open(FILE, ">$slave_info") ||
1159 Die "Failed to open file '$slave_info': $!";
1160 } else {
1161 open(FILE, "| ssh $option_remote_host 'cat > $slave_info'") ||
1162 Die "Failed to open file '$option_remote_host:$slave_info': $!";
1163 }
1164 print FILE "CHANGE MASTER TO MASTER_LOG_FILE='$filename', MASTER_LOG_POS=$position\n";
1165 close(FILE);
1166
1167 if ($option_stream eq 'tar') {
1168 system("cd $option_tmpdir; tar chf - xtrabackup_slave_info")
1169 and Die "Failed to stream 'xtrabackup_slave_info': $!";
1170 unlink $slave_info || Die "Failed to delete '$slave_info': $!";
1171 }
1172
1173 $mysql_slave_position = "master host '$master', filename '$filename', position $position";
1174}
1175
1176
1177#
1178# mysql_lockall subroutine puts a read lock on all tables in all databases.
1179#
1180sub mysql_lockall {
1181 $now = current_time();
1182 print STDERR "$now $prefix Starting to lock all tables...\n";
1183
1184 mysql_send "USE mysql;";
1185# mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;";
1186# if (compare_versions($mysql_server_version, '4.1.0') == -1) {
1187# # MySQL server version is 4.0 or older, ENGINE keyword not supported
1188# mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) TYPE=INNODB;";
1189# } else {
1190# # MySQL server version is 4.1 or newer, use ENGINE keyword
1191# mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) ENGINE=INNODB;";
1192# }
1193 mysql_send "SET AUTOCOMMIT=0;";
1194# mysql_send "INSERT INTO ibbackup_binlog_marker VALUES (1);";
1195 if (compare_versions($mysql_server_version, '4.0.22') == 0
1196 || compare_versions($mysql_server_version, '4.1.7') == 0) {
1197 # MySQL server version is 4.0.22 or 4.1.7
1198 mysql_send "COMMIT;";
1199 mysql_send "FLUSH TABLES WITH READ LOCK;";
1200 } else {
1201 # MySQL server version is other than 4.0.22 or 4.1.7
1202 mysql_send "FLUSH TABLES WITH READ LOCK;";
1203 mysql_send "COMMIT;";
1204 }
1205 write_binlog_info;
1206 write_slave_info if $option_slave_info;
1207
1208 $now = current_time();
1209 print STDERR "$now $prefix All tables locked and flushed to disk\n";
1210}
1211
1212
1213#
1214# mysql_unlockall subroutine releases read locks on all tables in all
1215# databases.
1216#
1217sub mysql_unlockall {
1218 mysql_send "UNLOCK TABLES;";
1219# mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;";
1220
1221 $now = current_time();
1222 print STDERR "$now $prefix All tables unlocked\n";
1223}
1224
1225
1226#
1227# catch_sigpipe subroutine is a signal handler for SIGPIPE.
1228#
1229sub catch_sigpipe {
1230 my $rcode;
1231
1232 if ($mysql_pid && (-1 == ($rcode = waitpid($mysql_pid, &WNOHANG))
1233 || $rcode == $mysql_pid)) {
1234 my $reason = `cat $mysql_stderr`;
1235 print STDERR "Pipe to mysql child process broken: $reason at";
1236 system("date +'%H:%M:%S'");
1237 exit(1);
1238 } else {
1239 Die "Broken pipe";
1240 }
1241}
1242
1243
1244#
1245# kill_child_processes subroutine kills all child processes of this process.
1246#
1247sub kill_child_processes {
1248 if ($ibbackup_pid) {
1249 kill($kill_signal, $ibbackup_pid);
1250 $ibbackup_pid = '';
1251 }
1252
1253 if ($mysql_pid) {
1254 kill($kill_signal, $mysql_pid);
1255 $mysql_pid = '';
1256 }
1257}
1258
1259
1260#
1261# require_external subroutine checks that an external program is runnable
1262# via the shell. This is tested by calling the program with the
1263# given arguments. It is checked that the program returns 0 and does
1264# not print anything to stderr. If this check fails, this subroutine exits.
1265# Parameters:
1266# program pathname of the program file
1267# args arguments to the program
1268# pattern a string containing a regular expression for finding
1269# the program version.
1270# this pattern should contain a subpattern enclosed
1271# in parentheses which is matched with the version.
1272# version_ref a reference to a variable where the program version
1273# string is returned. Example "2.0-beta2".
1274#
1275sub require_external {
1276 my $program = shift;
1277 my $args = shift;
1278 my $pattern = shift;
1279 my $version_ref = shift;
1280 my @lines;
1281 my $tmp_stdout = tmpnam();
1282 my $tmp_stderr = tmpnam();
1283 my $rcode;
1284 my $error;
1285
1286 $rcode = system("$program $args >$tmp_stdout 2>$tmp_stderr");
1287 if ($rcode) {
1288 $error = $!;
1289 }
1290 my $stderr = `cat $tmp_stderr | grep -v ^Warning`;
1291 unlink $tmp_stderr;
1292 if ($stderr ne '') {
1293 # failure
1294 unlink $tmp_stdout;
1295 Die "Couldn't run $program: $stderr";
1296 } elsif ($rcode) {
1297 # failure
1298 unlink $tmp_stdout;
1299 Die "Couldn't run $program: $error";
1300 }
1301
1302 # success
1303 my $stdout = `cat $tmp_stdout`;
1304 unlink $tmp_stdout;
1305 @lines = split(/\n|;/,$stdout);
1306 print STDERR "$prefix Using $lines[0]\n";
1307
1308 # get version string from the first output line of the program
1309 ${$version_ref} = '';
1310 if ($lines[0] =~ /$pattern/) {
1311 ${$version_ref} = $1;
1312 }
1313}
1314
1315
1316# compare_versions subroutine compares two GNU-style version strings.
1317# A GNU-style version string consists of three decimal numbers delimitted
1318# by dots, and optionally followed by extra attributes.
1319# Examples: "4.0.1", "4.1.1-alpha-debug".
1320# Parameters:
1321# str1 a GNU-style version string
1322# str2 a GNU-style version string
1323# Return value:
1324# -1 if str1 < str2
1325# 0 if str1 == str2
1326# 1 is str1 > str2
1327sub compare_versions {
1328 my $str1 = shift;
1329 my $str2 = shift;
1330 my $extra1 = '';
1331 my $extra2 = '';
1332 my @array1 = ();
1333 my @array2 = ();
1334 my $i;
1335
1336 # remove possible extra attributes
1337 ($str1, $extra1) = $str1 =~ /^([0-9.]*)(.*)/;
1338 ($str2, $extra2) = $str2 =~ /^([0-9.]*)(.*)/;
1339
1340 # split "dotted" decimal number string into an array
1341 @array1 = split('\.', $str1);
1342 @array2 = split('\.', $str2);
1343
1344 # compare in lexicographic order
1345 for ($i = 0; $i <= $#array1 && $i <= $#array2; $i++) {
1346 if ($array1[$i] < $array2[$i]) {
1347 return -1;
1348 } elsif ($array1[$i] > $array2[$i]) {
1349 return 1;
1350 }
1351 }
1352 if ($#array1 < $#array2) {
1353 return -1;
1354 } elsif ($#array1 > $#array2) {
1355 return 1;
1356 } else {
1357 return 0;
1358 }
1359}
1360
1361
1362#
1363# init subroutine initializes global variables and performs some checks on the
1364# system we are running on.
1365#
1366sub init {
1367 my $mysql_version = '';
1368 my $ibbackup_version = '';
1369 my $run = '';
1370
1371 # print some instructions to the user
1372 if (!$option_apply_log && !$option_copy_back) {
1373 $run = 'backup';
1374 } elsif ($option_copy_back) {
1375 $run = 'copy-back';
1376 } else {
1377 $run = 'apply-log';
1378 }
1379 print STDERR "IMPORTANT: Please check that the $run run completes successfully.\n";
1380 print STDERR " At the end of a successful $run run $innobackup_script\n";
1381 print STDERR " prints \"completed OK!\".\n\n";
1382
1383 # check that MySQL client program and InnoDB Hot Backup program
1384 # are runnable via shell
1385 if (!$option_copy_back) {
1386 # we are making a backup or applying log to backup
1387 if (!$option_apply_log) {
1388 # we are making a backup, we need mysql server
1389 my $output = '';
1390 my @lines = ();
1391
1392 # check that we have mysql client program
1393 require_external('mysql', '--version', 'Ver ([^,]+)',
1394 \$mysql_version);
1395
1396 # get mysql server version
1397 my $options = get_mysql_options();
1398 @lines = split('\n',
1399 `mysql $options -e "select \@\@version"`);
1400 $mysql_server_version = $lines[1];
1401 print STDERR "$prefix Using mysql server version $mysql_server_version\n";
1402 }
1403 #require_external($option_ibbackup_binary, '--license',
1404 # 'version (\S+)', \$ibbackup_version);
1405 print STDERR "\n";
1406
1407 if ($option_include
1408 && $ibbackup_version
1409 && $ibbackup_version le "2.0") {
1410 # --include option was given, but ibbackup is too
1411 # old to support it
1412 Die "--include option was given, but ibbackup is too old"
1413 . " to support it. You must upgrade to InnoDB Hot Backup"
1414 . " v2.0 in order to use --include option.\n";
1415 }
1416 }
1417
1418 # set signal handlers
1419 $SIG{PIPE} = \&catch_sigpipe;
1420
1421 # read MySQL options file
1422 #read_config_file($config_file, \%config);
1423 read_config_file(\%config);
1424
1425 if(!$option_tmpdir) {
1426 $option_tmpdir = get_option(\%config, 'mysqld', 'tmpdir');
1427 }
1428
1429 # get innodb log home directory from options file
1430 #$innodb_log_group_home_dir =
1431 # get_option(\%config, 'mysqld', 'innodb_log_group_home_dir');
1432
1433 if (!$option_apply_log && !$option_copy_back) {
1434 # we are making a backup, create a new backup directory
1435 if (!$option_remote_host) {
1436 $backup_dir = File::Spec->rel2abs(make_backup_dir());
1437 } else {
1438 $backup_dir = make_backup_dir();
1439 }
1440 print STDERR "$prefix Created backup directory $backup_dir\n";
1441 if (!$option_remote_host && !$option_stream) {
1442 $backup_config_file = $backup_dir . '/backup-my.cnf';
1443 $suspend_file = $backup_dir . '/xtrabackup_suspended';
1444 $mysql_stdout = $backup_dir . '/mysql-stdout';
1445 $mysql_stderr = $backup_dir . '/mysql-stderr';
1446 $binlog_info = $backup_dir . '/xtrabackup_binlog_info';
1447 $slave_info = $backup_dir . '/xtrabackup_slave_info';
1448 } else {
1449 $suspend_file = get_option(\%config, 'mysqld', 'datadir') . '/xtrabackup_suspended';
1450 $tmp_logfile = $option_tmpdir . '/xtrabackup_logfile';
1451 $mysql_stdout = $option_tmpdir . '/mysql-stdout';
1452 $mysql_stderr = $option_tmpdir . '/mysql-stderr';
1453 if ($option_stream) {
1454 $backup_config_file = $option_tmpdir . '/backup-my.cnf';
1455 $binlog_info = $option_tmpdir . '/xtrabackup_binlog_info';
1456 $slave_info = $option_tmpdir . '/xtrabackup_slave_info';
1457 } else {
1458 $backup_config_file = $backup_dir . '/backup-my.cnf';
1459 $binlog_info = $backup_dir . '/xtrabackup_binlog_info';
1460 $slave_info = $backup_dir . '/xtrabackup_slave_info';
1461 }
1462 }
1463 write_backup_config_file($backup_config_file);
1464 } elsif ($option_copy_back) {
1465 #$backup_config_file = $backup_dir . '/backup-my.cnf';
1466 #read_config_file($backup_config_file, \%backup_config);
1467 }
1468}
1469
1470
1471#
1472# write_backup_config_file subroutine creates a backup options file for
1473# ibbackup program. It writes to the file only those options that
1474# are required by ibbackup.
1475# Parameters:
1476# filename name for the created options file
1477#
1478sub write_backup_config_file {
1479 my $filename = shift;
1480 my $innodb_data_file_path =
1481 get_option(\%config, 'mysqld', 'innodb_data_file_path');
1482 my $innodb_log_files_in_group =
1483 get_option(\%config, 'mysqld', 'innodb_log_files_in_group');
1484 my $innodb_log_file_size =
1485 get_option(\%config, 'mysqld', 'innodb_log_file_size');
1486 my $root;
1487
1488 my @array = split(/;/, $innodb_data_file_path);
1489 for (my $i = 0; $i <= $#array; $i++) {
1490 my @tmp = split(/\/+/, $array[$i]);
1491 $array[$i] = $tmp[-1];
1492 }
1493 $innodb_data_file_path = join(";", @array);
1494
1495 if (!$option_remote_host) {
1496 $root = $backup_dir;
1497 open(FILE, "> $filename") || Die "Failed to open file '$filename': $!";
1498 } else {
1499 $root = `ssh $option_remote_host 'cd $backup_dir; pwd'`;
1500 open(FILE, "| ssh $option_remote_host 'cat > $filename'")
1501 || Die "Failed to open file '$option_remote_host:$filename': $!";
1502 }
1503
1504 print FILE "# This MySQL options file was generated by $innobackup_script.\n\n" .
1505 "# The MySQL server\n" .
1506 "[mysqld]\n" .
1507 "datadir=$root\n" .
1508 "innodb_data_home_dir=$root\n" .
1509 "innodb_data_file_path=$innodb_data_file_path\n" .
1510 "innodb_log_group_home_dir=$root\n" .
1511 "innodb_log_files_in_group=$innodb_log_files_in_group\n" .
1512 "innodb_log_file_size=$innodb_log_file_size\n";
1513 close(FILE);
1514
1515 if ($option_stream) {
1516 my $filename_dir = dirname($filename);
1517 my $filename_name = basename($filename);
1518 if ($option_stream eq 'tar') {
1519 system("cd $filename_dir; tar chf - $filename_name")
1520 and Die "Failed to stream '$filename_name': $!";
1521 }
1522 unlink $filename || Die "Failed to delete '$filename': $!";
1523 }
1524}
1525
1526
1527#
1528# check_args subroutine checks command line arguments. If there is a problem,
1529# this subroutine prints error message and exits.
1530#
1531sub check_args {
1532 my $i;
1533 my $rcode;
1534 my $buf;
1535 my $perl_version;
1536
1537 # check the version of the perl we are running
1538 if (!defined $^V) {
1539 # this perl is prior to 5.6.0 and uses old style version string
1540 my $required_version = $required_perl_version_old_style;
1541 if ($] lt $required_version) {
1542 print STDERR "$prefix Warning: " .
1543 "Your perl is too old! Innobackup requires\n";
1544 print STDERR "$prefix Warning: perl $required_version or newer!\n";
1545 }
1546 }
1547
1548 if (@ARGV == 0) {
1549 # no command line arguments
1550 print STDERR "$prefix You must specify the backup directory.\n";
1551 exit(1);
1552 }
1553
1554 # read command line options
1555 $rcode = GetOptions('compress:i' => \$option_compress,
1556 'help' => \$option_help,
1557 'version' => \$option_version,
1558 'throttle=i' => \$option_throttle,
1559 'sleep=i' => \$option_sleep,
1560 'apply-log' => \$option_apply_log,
1561 'redo-only' => \$option_redo_only,
1562 'copy-back' => \$option_copy_back,
1563 'include=s' => \$option_include,
1564 'databases=s' => \$option_databases,
1565 'tables-file=s', => \$option_tables_file,
1566 'use-memory=s' => \$option_use_memory,
1567 'uncompress' => \$option_uncompress,
1568 'export' => \$option_export,
1569 'password=s' => \$option_mysql_password,
1570 'user=s' => \$option_mysql_user,
1571 'host=s' => \$option_mysql_host,
1572 'port=s' => \$option_mysql_port,
1573 'slave-info' => \$option_slave_info,
1574 'socket=s' => \$option_mysql_socket,
1575 'no-timestamp' => \$option_no_timestamp,
1576 'defaults-file=s' => \$option_defaults_file,
1577 'incremental' => \$option_incremental,
1578 'incremental-basedir=s' => \$option_incremental_basedir,
1579 'incremental-lsn=s' => \$option_incremental_lsn,
1580 'incremental-dir=s' => \$option_incremental_dir,
1581 'extra-lsndir=s' => \$option_extra_lsndir,
1582 'remote-host=s' => \$option_remote_host,
1583 'stream=s' => \$option_stream,
1584 'tmpdir=s' => \$option_tmpdir,
1585 'no-lock' => \$option_no_lock,
1586 'ibbackup=s' => \$option_ibbackup_binary,
1587 'scpopt=s' => \$option_scp_opt,
1588 'force-tar', => \$option_force_tar,
1589 'parallel=i' => \$option_parallel,
1590 'safe-slave-backup' => \$option_safe_slave_backup,
1591 'safe-slave-backup-timeout' => $option_safe_slave_backup_timeout,
1592 );
1593
1594 if (!$rcode) {
1595 # failed to read options
1596 print STDERR "$prefix Bad command line arguments\n";
1597 exit(1);
1598 }
1599 if ($option_help) {
1600 # print help text and exit
1601 usage();
1602 exit(0);
1603 }
1604 if ($option_version) {
1605 # print program version and copyright
1606 print_version();
1607 exit(0);
1608 }
1609
1610 if ($option_compress == 0) {
1611 # compression level no specified, use default level
1612 $option_compress = $default_compression_level;
1613 }
1614
1615 if ($option_compress == 999) {
1616 # compress option not given in the command line
1617 $option_compress = 0;
1618 }
1619
1620 if (@ARGV < 1) {
1621 print STDERR "$prefix Missing command line argument\n";
1622 exit(1);
1623 } elsif (@ARGV > 1) {
1624 print STDERR "$prefix Too many command line arguments\n";
1625 exit(1);
1626 }
1627
1628 if ($option_stream) {
1629 if ($option_stream eq 'tar') {
1630 if ( !$option_force_tar ) {
1631 $option_tar4ibd = 'tar4ibd';
1632 }
1633 else {
1634 print STDERR "Forcing tar instead of tar4ibd\n";
1635 }
1636 } elsif ($option_stream eq 'tar4ibd') {
1637 $option_stream = 'tar';
1638 $option_tar4ibd = 'tar4ibd';
1639 } elsif ($option_stream eq 'cpio') {
1640 print STDERR "$prefix --stream=cpio is not supported yet\n";
1641 exit(1);
1642 } else {
1643 print STDERR "$prefix Unknown option --stream=$option_stream\n";
1644 exit(1);
1645 }
1646 }
1647
1648 # get options file name
1649 #$config_file = $ARGV[0];
1650
1651 if (!$option_apply_log && !$option_copy_back) {
1652 # we are making a backup, get backup root directory
1653 $backup_root = $ARGV[0];
1654 if ($option_incremental) {
1655 my @dirs = `ls -1 -t $backup_root`;
1656 my $inc_dir = $dirs[0];
1657 chomp($inc_dir);
1658 if ($option_incremental_basedir) {
1659 $incremental_basedir = $option_incremental_basedir;
1660 } else {
1661 $incremental_basedir = File::Spec->catfile($backup_root, $inc_dir);
1662 }
1663
1664 #print STDERR "--incremental_basedir=$incremental_basedir\n";
1665 #print STDERR "incremental backup is not supported for now.\n";
1666 #exit(1);
1667 }
1668 } else {
1669 # get backup directory
1670 $backup_dir = File::Spec->rel2abs($ARGV[0]);
1671 }
1672
1673 print STDERR "\n";
1674
1675 parse_databases_option_value();
1676 parse_tables_file_option_value($option_tables_file);
1677}
1678
1679
1680#
1681# make_backup_dir subroutine creates a new backup directory and returns
1682# its name.
1683#
1684sub make_backup_dir {
1685 my $dir;
1686 my $innodb_data_file_path =
1687 get_option(\%config, 'mysqld', 'innodb_data_file_path');
1688
1689 # create backup directory
1690 $dir = $backup_root;
1691 if ($option_stream) {
1692 return $dir;
1693 }
1694
1695 $dir .= '/' . strftime("%Y-%m-%d_%H-%M-%S", localtime())
1696 unless $option_no_timestamp;
1697 if (!$option_remote_host) {
1698 mkdir($dir, 0777) || Die "Failed to create backup directory $dir: $!";
1699 } else {
1700 system("ssh $option_remote_host mkdir $dir");
1701 }
1702
1703 # create subdirectories for ibdata files if needed
1704# foreach my $a (split(/;/, $innodb_data_file_path)) {
1705# my $path = (split(/:/,$a))[0];
1706# my @relative_path = split(/\/+/, $path);
1707# pop @relative_path;
1708# if (@relative_path) {
1709# # there is a non-trivial path from the backup directory
1710# # to the directory of this backup ibdata file, check
1711# # that all the directories in the path exist.
1712# create_path_if_needed($dir, \@relative_path);
1713# }
1714# }
1715
1716 return $dir;
1717}
1718
1719
1720#
1721# create_path_if_needed subroutine checks that all components
1722# in the given relative path are directories. If the
1723# directories do not exist, they are created.
1724# Parameters:
1725# root a path to the root directory of the relative pathname
1726# relative_path a relative pathname (a reference to an array of
1727# pathname components)
1728#
1729sub create_path_if_needed {
1730 my $root = shift;
1731 my $relative_path = shift;
1732 my $path;
1733
1734 $path = $root;
1735 foreach $a (@{$relative_path}) {
1736 $path = $path . "/" . $a;
1737 if (!$option_remote_host) {
1738 if (! -d $path) {
1739 # this directory does not exist, create it !
1740 mkdir($path, 0777) || Die "Failed to create backup directory: $!";
1741 }
1742 } else {
1743 if (system("ssh $option_remote_host test -d $path") != 0) {
1744 system("ssh $option_remote_host mkdir $path");
1745 }
1746 }
1747 }
1748}
1749
1750
1751#
1752# remove_from_array subroutine removes excluded element from the array.
1753# Parameters:
1754# array_ref a reference to an array of strings
1755# excluded a string to be excluded from the copy
1756#
1757sub remove_from_array {
1758 my $array_ref = shift;
1759 my $excluded = shift;
1760 my @copy = ();
1761 my $size = 0;
1762
1763 foreach my $str (@{$array_ref}) {
1764 if ($str ne $excluded) {
1765 $copy[$size] = $str;
1766 $size = $size + 1;
1767 }
1768 }
1769 @{$array_ref} = @copy;
1770}
1771
1772
1773#
1774# backup_files subroutine copies .frm, .MRG, .MYD and .MYI files to
1775# backup directory.
1776#
1777sub backup_files {
1778 my $source_dir = get_option(\%config, 'mysqld', 'datadir');
1779 my @list;
1780 my $file;
1781 my $database;
1782 my $wildcard = '*.{frm,MYD,MYI,MRG,TRG,TRN,ARM,ARZ,CSM,CSV,opt,par}';
1783
1784 opendir(DIR, $source_dir)
1785 || Die "Can't open directory '$source_dir': $!\n";
1786 $now = current_time();
1787 print STDERR "\n$now $prefix Starting to backup .frm, .MRG, .MYD, .MYI,\n";
1788 print STDERR "$prefix .TRG, .TRN, .ARM, .ARZ, .CSM, .CSV and .opt files in\n";
1789 print STDERR "$prefix subdirectories of '$source_dir'\n";
1790 # loop through all database directories
1791 while (defined($database = readdir(DIR))) {
1792 my $print_each_file = 0;
1793 my $file_c;
1794 my @scp_files;
1795 # skip files that are not database directories
1796 if ($database eq '.' || $database eq '..') { next; }
1797 next unless -d "$source_dir/$database";
1798 next unless check_if_required($database);
1799
1800 if (!$option_remote_host && !$option_stream) {
1801 if (! -e "$backup_dir/$database") {
1802 # create database directory for the backup
1803 mkdir("$backup_dir/$database", 0777)
1804 || Die "Couldn't create directory '$backup_dir/$database': $!";
1805 }
1806 } elsif ($option_remote_host) {
1807 if (system("ssh $option_remote_host test -e $backup_dir/$database")
1808 != 0) {
1809 system("ssh $option_remote_host mkdir $backup_dir/$database");
1810 }
1811 }
1812
1813 # copy files of this database
1814 opendir(DBDIR, "$source_dir/$database");
1815 @list = grep(/\.(frm)|(MYD)|(MYI)|(MRG)|(TRG)|(TRN)|(ARM)|(ARZ)|(CSM)|(CSV)|(opt)|(par)$/, readdir(DBDIR));
1816 closedir DBDIR;
1817 $file_c = @list;
1818 if ($file_c <= $backup_file_print_limit) {
1819 $print_each_file = 1;
1820 } else {
1821 print STDERR "$prefix Backing up files " .
1822 "'$source_dir/$database/$wildcard' ($file_c files)\n";
1823 }
1824 foreach $file (@list) {
1825 # copying may take a long time, so we have to prevent
1826 # mysql connection from timing out
1827 mysql_keep_alive();
1828 next unless check_if_required($database, $file);
1829
1830 if($option_include) {
1831 if (!("$database.$file" =~ /$option_include/)) {
1832 print STDERR "$database.$file is skipped because it does not match $option_include.\n";
1833 next;
1834 }
1835 }
1836
1837
1838 if ($print_each_file) {
1839 print STDERR "$prefix Backing up file '$source_dir/$database/$file'\n";
1840 }
1841 if (!$option_remote_host && !$option_stream) {
1842 $src_name = escape_path("$source_dir/$database/$file");
1843 $dst_name = escape_path("$backup_dir/$database");
1844 system("$CP_CMD \"$src_name\" \"$dst_name\"")
1845 and Die "Failed to copy file '$file': $!";
1846 } elsif ($option_remote_host) {
1847 # Queue up files for one single scp per database.
1848 push(@scp_files, "'$file'");
1849 } elsif($option_stream eq 'tar') {
1850 my $ret = 0;
1851 my $file_name = substr($file, rindex($file, '/') + 1);
1852 $file_name=~s/([\$\\\" ])/\\$1/g;
1853 $ret = system("cd $source_dir; tar cf - $database/$file_name") >> 8;
1854 if ($ret == 1) {
1855 print STDERR "$prefix If you use GNU tar, this warning can be ignored.\n";
1856 } elsif ($ret != 0) {
1857 print STDERR "$prefix tar returned with exit code $ret.\n";
1858 Die "Failed to stream '$database/$file_name': $!";
1859 }
1860 }
1861 }
1862 if ($option_remote_host and @scp_files) {
1863 my $scp_file_list = join(" ", map { "$source_dir/$database/$_" } @scp_files);
1864 system("scp $option_scp_opt $scp_file_list '$option_remote_host:$backup_dir/$database/'")
1865 and Die "Failed to execute \"scp $option_scp_opt $scp_file_list '$option_remote_host:$backup_dir/$database/'\": $!";
1866 }
1867 }
1868 closedir(DIR);
1869
1870 $now = current_time();
1871 print STDERR "$now $prefix Finished backing up .frm, .MRG, .MYD, .MYI, .TRG, .TRN, .ARM, .ARZ, .CSV, .CSM and .opt files\n\n";
1872}
1873
1874
1875#
1876# file_to_array subroutine reads the given text file into an array and
1877# stores each line as an element of the array. The end-of-line
1878# character(s) are removed from the lines stored in the array.
1879# Parameters:
1880# filename name of a text file
1881# lines_ref a reference to an array
1882#
1883sub file_to_array {
1884 my $filename = shift;
1885 my $lines_ref = shift;
1886
1887 open(FILE, $filename) || Die "can't open file '$filename': $!";
1888 @{$lines_ref} = <FILE>;
1889 close(FILE) || Die "can't close file '$filename': $!";
1890
1891 foreach my $a (@{$lines_ref}) {
1892 chomp($a);
1893 }
1894}
1895
1896
1897#
1898# unescape_string subroutine expands escape sequences found in the string and
1899# returns the expanded string. It also removes possible single or double quotes
1900# around the value.
1901# Parameters:
1902# value a string
1903# Return value:
1904# a string with expanded escape sequences
1905#
1906sub unescape_string {
1907 my $value = shift;
1908 my $result = '';
1909 my $offset = 0;
1910
1911 # remove quotes around the value if they exist
1912 if (length($value) >= 2) {
1913 if ((substr($value, 0, 1) eq "'" && substr($value, -1, 1) eq "'")
1914 || (substr($value, 0, 1) eq '"' && substr($value, -1, 1) eq '"')) {
1915 $value = substr($value, 1, -1);
1916 }
1917 }
1918
1919 # expand escape sequences
1920 while ($offset < length($value)) {
1921 my $pos = index($value, "\\", $offset);
1922 if ($pos < 0) {
1923 $pos = length($value);
1924 $result = $result . substr($value, $offset, $pos - $offset);
1925 $offset = $pos;
1926 } else {
1927 my $replacement = substr($value, $pos, 2);
1928 my $escape_code = substr($value, $pos + 1, 1);
1929 if (exists $option_value_escapes{$escape_code}) {
1930 $replacement = $option_value_escapes{$escape_code};
1931 }
1932 $result = $result
1933 . substr($value, $offset, $pos - $offset)
1934 . $replacement;
1935 $offset = $pos + 2;
1936 }
1937 }
1938
1939 return $result;
1940}
1941
1942
1943#
1944# read_config_file subroutine reads MySQL options file and
1945# returns the options in a hash containing one hash per group.
1946# Parameters:
1947# filename name of a MySQL options file
1948# groups_ref a reference to hash variable where the read
1949# options are returned
1950#
1951sub read_config_file {
1952 #my $filename = shift;
1953 my $groups_ref = shift;
1954 my @lines ;
1955 my $i;
1956 my $group;
1957 my $group_hash_ref;
1958
1959 my $cmdline = '';
1960 my $options = '';
1961
1962 if ($option_defaults_file) {
1963 $options = $options . " --defaults-file=\"$option_defaults_file\" ";
1964 }
1965
1966 $options = $options . "--print-param";
1967
1968
1969 # read file to an array, one line per element
1970 #file_to_array($filename, \@lines);
1971 $cmdline = "$option_ibbackup_binary $options";
1972 @lines = `$cmdline`;
1973
1974 # classify lines and save option values
1975 $group = 'default';
1976 $group_hash_ref = {};
1977 ${$groups_ref}{$group} = $group_hash_ref;
1978 # this pattern described an option value which may be
1979 # quoted with single or double quotes. This pattern
1980 # does not work by its own. It assumes that the first
1981 # opening parenthesis in this string is the second opening
1982 # parenthesis in the full pattern.
1983 my $value_pattern = q/((["'])([^\\\4]|(\\[^\4]))*\4)|([^\s]+)/;
1984 for ($i = 0; $i < @lines; $i++) {
1985 SWITCH: for ($lines[$i]) {
1986 # comment
1987 /^\s*(#|;)/
1988 && do { last; };
1989
1990 # group
1991 /^\s*\[(.*)\]/
1992 && do {
1993 $group = $1;
1994 if (!exists ${$groups_ref}{$group}) {
1995 $group_hash_ref = {};
1996 ${$groups_ref}{$group} = $group_hash_ref;
1997 } else {
1998 $group_hash_ref = ${$groups_ref}{$group};
1999 }
2000 last;
2001 };
2002
2003 # option
2004 /^\s*([^\s=]+)\s*(#.*)?$/
2005 && do {
2006 ${$group_hash_ref}{$1} = '';
2007 last;
2008 };
2009
2010 # set-variable = option = value
2011 /^\s*set-variable\s*=\s*([^\s=]+)\s*=\s*($value_pattern)\s*(#.*)?$/
2012 && do { ${$group_hash_ref}{$1} = unescape_string($2); last; };
2013
2014 # option = value
2015 /^\s*([^\s=]+)\s*=\s*($value_pattern)\s*(#.*)?$/
2016 && do { ${$group_hash_ref}{$1} = unescape_string($2); last; };
2017
2018 # empty line
2019 /^\s*$/
2020 && do { last; };
2021
2022 # unknown
2023 print("$prefix: Warning: Ignored unrecognized line ",
2024 $i + 1,
2025 " in options : '${lines[$i]}'\n"
2026 );
2027 }
2028 }
2029}
2030
2031
2032#
2033# get_option subroutine returns the value of given option in the config
2034# structure. If option is missing, this subroutine calls exit.
2035# Parameters:
2036# config_ref a reference to a config data
2037# group option group name
2038# option_name name of the option
2039# Return value:
2040# option value as a string
2041#
2042sub get_option {
2043 my $config_ref = shift;
2044 my $group = shift;
2045 my $option_name = shift;
2046 my $group_hash_ref;
2047
2048 if (!exists $config{$group}) {
2049 # no group
2050 print STDERR "$prefix fatal error: no '$group' group in MySQL options\n";
2051 print STDERR "$prefix fatal error: OR no 'datadir' option in group '$group' in MySQL options\n";
2052 exit(1);
2053 }
2054
2055 $group_hash_ref = ${$config_ref}{$group};
2056 if (!exists ${$group_hash_ref}{$option_name}) {
2057 # no option
2058 print STDERR "$prefix fatal error: no '$option_name' option in group '$group' in MySQL options\n";
2059 exit(1);
2060 }
2061
2062 return ${$group_hash_ref}{$option_name};
2063}
2064
2065# check_if_required subroutine returns 1 if the specified database and
2066# table needs to be backed up.
2067# Parameters:
2068# $_[0] name of database to be checked
2069# $_[1] full path of table file (This argument is optional)
2070# Return value:
2071# 1 if backup should be done and 0 if not
2072#
2073sub check_if_required {
2074 my ( $db, $table_path ) = @_;
2075 my $db_count = scalar keys %databases_list;
2076 my $tbl_count = scalar keys %table_list;
2077 my $filename;
2078 my $table;
2079
2080 if ( $db_count == 0 && $tbl_count == 0 ) {
2081 # No databases defined with --databases option, include all databases,
2082 # and no tables defined with --tables-file option, include all tables.
2083 return 1;
2084 }
2085 else {
2086 if ( $table_path ) {
2087 # get the last component in the table pathname
2088 $filename = (reverse(split(/\//, $table_path)))[0];
2089 # get name of the table by removing file suffix
2090 $table = (split(/\./, $filename))[0];
2091 }
2092 }
2093
2094 # Filter for --databases.
2095 if ( $db_count ) {
2096 if (defined $databases_list{$db}) {
2097 if (defined $table_path) {
2098 my $db_hash = $databases_list{$db};
2099 $db_count = keys %$db_hash;
2100 if ($db_count > 0 && ! defined $databases_list{$db}->{$table}) {
2101 # --databases option specified, but table is not included
2102 return 0;
2103 }
2104 }
2105 # include this database and table
2106 return 1;
2107 }
2108 else {
2109 # --databases option given, but database is not included
2110 return 0;
2111 }
2112 }
2113
2114 # Filter for --tables-file.
2115 if ( $tbl_count ) {
2116 return 0 unless exists $table_list{$db};
2117 return 0 if $table && !$table_list{$db}->{$table};
2118 }
2119
2120 return 1; # backup the table
2121}
2122
2123
2124# parse_databases_option_value subroutine parses the value of
2125# --databases option. If the option value begins with a slash
2126# it is considered a pathname and the option value is read
2127# from the file.
2128#
2129# This subroutine sets the global "databases_list" variable.
2130#
2131sub parse_databases_option_value {
2132 my $item;
2133
2134 if ($option_databases =~ /^\//) {
2135 # the value of the --databases option begins with a slash,
2136 # the option value is pathname of the file containing
2137 # list of databases
2138 if (! -f $option_databases) {
2139 Die "can't find file '$option_databases'";
2140 }
2141
2142 # read from file the value of --databases option
2143 my @lines;
2144 file_to_array($option_databases, \@lines);
2145 $option_databases = join(" ", @lines);
2146 }
2147
2148 # mark each database or database.table definition in the
2149 # global databases_list.
2150 foreach $item (split(/\s/, $option_databases)) {
2151 my $db = "";
2152 my $table = "";
2153 my %hash;
2154
2155 if ($item eq "") {
2156 # ignore empty strings
2157 next;
2158 }
2159
2160 # get database and table names
2161 if ($item =~ /(\S*)\.(\S*)/) {
2162 # item is of the form DATABASE.TABLE
2163 $db = $1;
2164 $table = $2;
2165 } else {
2166 # item is database name, table is undefined
2167 $db = $item;
2168 }
2169
2170 if (! defined $databases_list{$db}) {
2171 # create empty hash for the database
2172 $databases_list{$db} = \%hash;
2173 }
2174 if ($table ne "") {
2175 # add mapping table --> 1 to the database hash
2176 my $h = $databases_list{$db};
2177 $h->{$table} = 1;
2178 }
2179 }
2180}
2181
2182# Parse the --tables-file file to determine which InnoDB tables
2183# are backedup up. Only backedup tables have their .frm, etc.
2184# files copied.
2185sub parse_tables_file_option_value {
2186 my ( $filename ) = @_;
2187
2188 return unless $filename;
2189
2190 open my $fh, '<', $filename;
2191 if ( $fh ) {
2192 while ( my $line = <$fh> ) {
2193 chomp $line;
2194 my ( $db, $tbl ) = $line =~ m/\s*([^\.]+)\.([^\.]+)\s*/;
2195 if ( $db && $tbl ) {
2196 $table_list{$db}->{$tbl} = 1;
2197 print STDERR "$prefix $db.$tbl will be registerd to the list\n";
2198 }
2199 else {
2200 warn "$prefix Invalid line in $filename: $line";
2201 }
2202 }
2203 }
2204 else {
2205 warn "$prefix Cannot read --tables-file $filename: $OS_ERROR";
2206 }
2207
2208 return;
2209}
2210
2211sub escape_path {
2212 my $str = shift;
2213 if ($win eq 1) {
2214 $str =~ s/\//\\/g;
2215 $str =~ s/\\\\/\\/g;
2216 }
2217 else{
2218 $str =~ s/\/\//\//g;
2219 }
2220 return $str;
2221
2222}
2223
2224sub set_xtrabackup_version {
2225# Based on MySQL version choose correct binary
2226# MySQL 5.0.* - xtrabackup_51
2227# MySQL 5.1.* - xtrabackup_51
2228# MySQL 5.1.* with InnoDB plugin - xtrabackup
2229# Percona Server >= 11.0 - xtrabackup
2230# MySQL 5.5.* - xtrabackup_55
2231
2232my @lines;
2233my $var_version = '';
2234my $var_innodb_version = '';
2235my $ibbackup_binary;
2236mysql_open();
2237mysql_send "SHOW VARIABLES LIKE 'version'\\G";
2238file_to_array($mysql_stdout, \@lines);
2239for (@lines) {
2240 $var_version = $1 if /Value:\s+(\S+)/;
2241 }
2242mysql_send "SHOW VARIABLES LIKE 'innodb_version'\\G";
2243file_to_array($mysql_stdout, \@lines);
2244for (@lines) {
2245 $var_innodb_version = $1 if /Value:\s+(\S+)/;
2246 }
2247if($var_version =~ m/5\.0\.\d/){
2248 $ibbackup_binary = ($win eq 1 ? 'xtrabackup.exe' : 'xtrabackup_51');
2249}
2250if($var_version =~ m/5\.1\.\d/ and $var_innodb_version =~ m//){
2251 $ibbackup_binary = ($win eq 1 ? 'xtrabackup.exe' : 'xtrabackup_51');
2252}
2253if($var_version =~ m/5\.1\.\d/ and $var_innodb_version =~ m/1\.0\.\d+$/){
2254 $ibbackup_binary = ($win eq 1 ? 'xtrabackup.exe' : 'xtrabackup');
2255}
2256if($var_version =~ m/5\.1\.\d/ and $var_innodb_version =~ m/1\.0\.\d+-\d/){
2257 $ibbackup_binary = ($win eq 1 ? 'xtrabackup.exe' : 'xtrabackup');
2258}
2259if($var_version =~ m/5\.5\.\d/){
2260 $ibbackup_binary = ($win eq 1 ? 'xtrabackup.exe' : 'xtrabackup_55');
2261}
2262mysql_close();
2263return $ibbackup_binary;
2264}
2265
2266# Wait until it's safe to backup a slave. Returns immediately if
2267# the host isn't a slave. Currently there's only one check:
2268# Slave_open_temp_tables has to be zero. Dies on timeout.
2269sub wait_for_safe_slave {
2270 my @lines;
2271
2272 my $host_is_slave = 0;
2273 mysql_send 'SHOW SLAVE STATUS\G;';
2274 file_to_array($mysql_stdout, \@lines);
2275 foreach my $line ( @lines ) {
2276 if ( $line =~ m/Read_Master_Log_Pos/ ) {
2277 $host_is_slave = 1;
2278 last;
2279 }
2280 }
2281 if ( !$host_is_slave ) {
2282 print STDERR "$prefix: Not checking slave open temp tables for --safe-slave-backup because host is not a slave\n";
2283 return;
2284 }
2285
2286 mysql_send 'STOP SLAVE SQL_THREAD;';
2287
2288 my $open_temp_tables = get_slave_open_temp_tables();
2289 print STDERR "$prefix: Slave open temp tables: $open_temp_tables\n";
2290
2291 return if $open_temp_tables == 0;
2292
2293 my $sleep_time = 3;
2294 my $n_attempts = int($option_safe_slave_backup_timeout / $sleep_time) || 1;
2295 while ( $n_attempts-- ) {
2296 print STDERR "$prefix: Starting slave SQL thread, waiting $sleep_time seconds, then checking Slave_open_temp_tables again ($n_attempts attempts remaining)...\n";
2297
2298 mysql_send 'START SLAVE SQL_THREAD;';
2299 sleep $sleep_time;
2300 mysql_send 'STOP SLAVE SQL_THREAD;';
2301
2302 $open_temp_tables = get_slave_open_temp_tables();
2303 print STDERR "$prefix: Slave open temp tables: $open_temp_tables\n";
2304 if ( !$open_temp_tables ) {
2305 print STDERR "$prefix: Slave is safe to backup\n";
2306 return;
2307 }
2308 }
2309
2310 Die "Slave_open_temp_tables did not become zero after waiting $option_safe_slave_backup_timeout seconds";
2311}
2312
2313sub get_slave_open_temp_tables {
2314 my @lines;
2315 mysql_send 'SHOW STATUS LIKE "slave_open_temp_tables"\G;';
2316 file_to_array($mysql_stdout, \@lines);
2317 my $last_value;
2318 for my $i ( 0..$#lines ) {
2319 $last_value = $i + 1
2320 if $lines[$i] =~ m/Variable_name: Slave_open_temp_tables/i;
2321 }
2322 Die "SHOW STATUS LIKE 'slave_open_temp_tables' did not return anything"
2323 unless $last_value;
2324
2325 Die "Failed to get Slave_open_temp_tables from SHOW STATUS"
2326 unless defined $lines[$last_value];
2327
2328 my ($n) = $lines[$last_value] =~ m/(\d+)/;
2329 return $n;
2330}
2331
2332=pod
2333
2334=head1 NAME
2335
2336innobackupex - Non-blocking backup tool for InnoDB, XtraDB and HailDB databases
2337
2338=head1 SYNOPOSIS
2339
2340innobackupex [--compress[=LEVEL]] [--include=REGEXP] [--user=NAME]
2341 [--password=WORD] [--port=PORT] [--socket=SOCKET]
2342 [--no-timestamp] [--ibbackup-binary=IBBACKUP-BINARY]
2343 [--slave-info] [--stream=tar] [--force-tar]
2344 [--scpopt=OPTIONS-FOR-SCP] [--defaults-file=MY.CNF]
2345 [--databases=LIST] [--remote-host=HOSTNAME] [--no-lock]
2346 [--tmpdir=DIRECTORY] [--tables-file=FILE]
2347 [--incremental] [--incremental-basedir]
2348 [--incremental-dir] [--incremental-lsn]
2349 BACKUP-ROOT-DIR
2350
2351innobackupex --apply-log [--use-memory=MB] [--uncompress]
2352 [--defaults-file=MY.CNF]
2353 [--export] [--redo-only] [--ibbackup=IBBACKUP-BINARY]
2354 BACKUP-DIR
2355
2356innobackupex --copy-back [--defaults-file=MY.CNF] BACKUP-DIR
2357
2358=head1 DESCRIPTION
2359
2360The first command line above makes a hot backup of a MySQL database.
2361By default it creates a backup directory (named by the current date
2362and time) in the given backup root directory. With the --no-timestamp
2363option it does not create a time-stamped backup directory, but it puts
2364the backup in the given directory (which must not exist). This
2365command makes a complete backup of all MyISAM and InnoDB tables and
2366indexes in all databases or in all of the databases specified with the
2367--databases option. The created backup contains .frm, .MRG, .MYD,
2368.MYI, .TRG, .TRN, .ARM, .ARZ, .CSM, CSV, .opt, .par, and InnoDB data and log files.
2369The MY.CNF options file defines the location of the database. This command
2370connects to the MySQL server using the mysql client program, and runs
2371xtrabackup as a child process.
2372
2373The --apply-log command prepares a backup for starting a MySQL
2374server on the backup. This command recovers InnoDB data files as specified
2375in BACKUP-DIR/backup-my.cnf using BACKUP-DIR/xtrabackup_logfile,
2376and creates new InnoDB log files as specified in BACKUP-DIR/backup-my.cnf.
2377The BACKUP-DIR should be the path to a backup directory created by
2378xtrabackup. This command runs xtrabackup as a child process, but it does not
2379connect to the database server.
2380
2381The --copy-back command copies data, index, and log files
2382from the backup directory back to their original locations.
2383The MY.CNF options file defines the original location of the database.
2384The BACKUP-DIR is the path to a backup directory created by xtrabackup.
2385
2386On success the exit code innobackupex is 0. A non-zero exit code
2387indicates an error.
2388
2389
2390=head1 OPTIONS
2391
2392=over
2393
2394=item --apply-log
2395
2396Prepare a backup in BACKUP-DIR by applying the transaction log file named "xtrabackup_logfile" located in the same directory. Also, create new transaction logs. The InnoDB configuration is read from the file "backup-my.cnf".
2397
2398=item --copy-back
2399
2400Copy all the files in a previously made backup from the backup directory to their original locations.
2401
2402=item --databases=LIST
2403
2404This option specifies the list of databases that innobackupex should back up. The option accepts a string argument. The list is of the form "databasename1[.table_name1] databasename2[.table_name2] . . .". If this option is not specified, all databases containing MyISAM and InnoDB tables will be backed up. Please make sure that --databases contains all of the InnoDB databases and tables, so that all of the innodb.frm files are also backed up. In case the list is very long, this can be specified in a file, and the full path of the file can be specified instead of the list. (See option --tables-file.)
2405
2406=item --defaults-file=[MY.CNF]
2407
2408This option specifies what file to read the default MySQL options from. The option accepts a string argument. It is also passed directly to xtrabackup's --defaults-file option. See the xtrabackup documentation for details.
2409
2410=item --export
2411
2412This option is passed directly to xtrabackup's --export option. It enables exporting individual tables for import into another server. See the xtrabackup documentation for details.
2413
2414=item --extra-lsndir=DIRECTORY
2415
2416This option specifies the directory in which to save an extra copy of the "xtrabackup_checkpoints" file. The option accepts a string argument. It is passed directly to xtrabackup's --extra-lsndir option. See the xtrabackup documentation for details.
2417
2418=item --force-tar
2419
2420This option forces the use of tar when creating a streamed backup, rather than tar4ibd, which is the default.
2421
2422=item --help
2423
2424This option displays a help screen and exits.
2425
2426=item --host=HOST
2427
2428This option specifies the host to use when connecting to the database server with TCP/IP. The option accepts a string argument. It is passed to the mysql child process without alteration. See mysql --help for details.
2429
2430=item --ibbackup-binary=IBBACKUP-BINARY
2431
2432This option specifies which xtrabackup binary should be used. The option accepts a string argument. IBBACKUP-BINARY should be the command used to run XtraBackup. The option can be useful if the xtrabackup binary is not in your search path or working directory. If this option is not specified, innobackupex attempts to determine the binary to use automatically. By default, "xtrabackup" is the command used. However, when option --copy-back is specified, "xtrabackup_51" is the command used. And when option --apply-log is specified, the binary is used whose name is in the file "xtrabackup_binary" in the backup directory, if that file exists.
2433
2434=item --include=REGEXP
2435
2436This option is a regular expression to be matched against table names in databasename.tablename format. It is passed directly to xtrabackup's --tables option. See the xtrabackup documentation for details.
2437
2438=item --incremental
2439
2440This option tells xtrabackup to create an incremental backup, rather than a full one. It is passed to the xtrabackup child process. When this option is specified, either --incremental-lsn or --incremental-basedir can also be given. If neither option is given, option --incremental-basedir is passed to xtrabackup by default, set to the first timestamped backup directory in the backup base directory.
2441
2442=item --incremental-basedir=DIRECTORY
2443
2444This option specifies the directory containing the full backup that is the base dataset for the incremental backup. The option accepts a string argument. It is used with the --incremental option.
2445
2446=item --incremental-dir=DIRECTORY
2447
2448This option specifies the directory where the incremental backup will be combined with the full backup to make a new full backup. The option accepts a string argument. It is used with the --incremental option.
2449
2450=item --incremental-lsn
2451
2452This option specifies the log sequence number (LSN) to use for the incremental backup. The option accepts a string argument. It is used with the --incremental option. It is used instead of specifying --incremental-basedir. For databases created by MySQL and Percona Server 5.0-series versions, specify the LSN as two 32-bit integers in high:low format. For databases created in 5.1 and later, specify the LSN as a single 64-bit integer.
2453
2454=item --no-lock
2455
2456Use this option to disable table lock with "FLUSH TABLES WITH READ LOCK". Use it only if ALL your tables are InnoDB and you DO NOT CARE about the binary log position of the backup.
2457
2458=item --no-timestamp
2459
2460This option prevents creation of a time-stamped subdirectory of the BACKUP-ROOT-DIR given on the command line. When it is specified, the backup is done in BACKUP-ROOT-DIR instead.
2461
2462=item --parallel=NUMBER-OF-THREADS
2463
2464This option specifies the number of threads the xtrabackup child process should use to back up files concurrently. The option accepts an integer argument. It is passed directly to xtrabackup's --parallel option. See the xtrabackup documentation for details.
2465
2466
2467=item --password=WORD
2468
2469This option specifies the password to use when connecting to the database. It accepts a string argument. It is passed to the mysql child process without alteration. See mysql --help for details.
2470
2471=item --port=PORT
2472
2473This option specifies the port to use when connecting to the database server with TCP/IP. The option accepts a string argument. It is passed to the mysql child process. It is passed to the mysql child process without alteration. See mysql --help for details.
2474
2475=item --redo-only
2476
2477This option is passed directly to xtrabackup's --apply-log-only option. This forces xtrabackup to skip the "rollback" phase and do a "redo" only. This is necessary if the backup will have incremental changes applied to it later. See the xtrabackup documentation for details.
2478
2479=item --remote-host=HOSTNAME
2480
2481This option specifies the remote host on which the backup files will be created, by using an ssh connection. The option accepts a string argument.
2482
2483=item --safe-slave-backup
2484
2485Stop slave SQL thread and wait to start backup until Slave_open_temp_tables in "SHOW STATUS" is zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail if Slave_open_temp_tables does not become zero after --safe-slave-backup-timeout seconds. The slave SQL thread will be restarted when the backup finishes.
2486
2487=item --safe-slave-backup-timeout
2488
2489How many seconds --safe-slave-backup should wait for Slave_open_temp_tables to become zero. (default 300)
2490
2491=item --scpopt=SCP-OPTIONS
2492
2493This option specifies the command line options to pass to scp when the option --remost-host is specified. The option accepts a string argument. If the option is not specified, the default options are "-Cp -c arcfour".
2494
2495=item --slave-info
2496
2497This option is useful when backing up a replication slave server. It prints the binary log position and name of the master server. It also writes this information to the "xtrabackup_slave_info" file as a "CHANGE MASTER" command. A new slave for this master can be set up by starting a slave server on this backup and issuing a "CHANGE MASTER" command with the binary log position saved in the "xtrabackup_slave_info" file.
2498
2499=item --socket=SOCKET
2500
2501This option specifies the socket to use when connecting to the local database server with a UNIX domain socket. The option accepts a string argument. It is passed to the mysql child process without alteration. See mysql --help for details.
2502
2503=item --stream=[tar|. . .]
2504
2505This option specifies the format in which to do the streamed backup. The option accepts a string argument. The backup will be done to STDOUT in the specified format. Currently, the only supported format is tar. Uses tar4ibd, which is available in XtraBackup distributions.
The diff has been truncated for viewing.