Merge lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug1222062 into lp:percona-xtrabackup/2.1

Proposed by Sergei Glushchenko
Status: Merged
Approved by: Alexey Kopytov
Approved revision: no longer in the source branch.
Merged at revision: 764
Proposed branch: lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug1222062
Merge into: lp:percona-xtrabackup/2.1
Diff against target: 631 lines (+109/-41)
13 files modified
doc/source/innobackupex/innobackupex_option_reference.rst (+4/-0)
doc/source/xtrabackup_bin/xbk_option_reference.rst (+4/-0)
innobackupex.pl (+11/-1)
patches/innodb51.patch (+10/-8)
patches/innodb55.patch (+9/-7)
patches/innodb56.patch (+7/-5)
patches/xtradb51.patch (+9/-7)
patches/xtradb55.patch (+10/-8)
src/fil_cur.cc (+1/-1)
src/xtrabackup.cc (+14/-1)
test/inc/common.sh (+4/-1)
test/run.sh (+15/-2)
test/t/bug1222062.sh (+11/-0)
To merge this branch: bzr merge lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug1222062
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Approve
Review via email: mp+233639@code.launchpad.net

Description of the change

Bug 1222062: Add an option to disable opening all tablespaces on backup start

Introduce xtrabackup option --close-file-handlers to close file
handlers when they aren't needed anymore. Introduce new option -x
for run.sh to force xtrabackup options for all tests by putting them
into [xtrabackup] section of my.cnf.

Two jenkins runs with and without --close-file-handlers, ddl.sh fails as expected

http://jenkins.percona.com/view/PXB%202.1/job/percona-xtrabackup-2.1-param-new/7/
http://jenkins.percona.com/view/PXB%202.1/job/percona-xtrabackup-2.1-param-new/9/

To post a comment you must log in.
Revision history for this message
Alexey Kopytov (akopytov) wrote :

Sergei,

- I would call the option just ‘--close-files’, because ‘handlers’ looks
  confusing (and should be ‘handles’ for that matter).
- users are generally not supposed to call xtrabackup directly, but use
  innobackupex instead. Why is there no corresponding innobackupex
  option?
- please add the warning to the built-in help for xtrabackup option
- please also update the docs (with the “use at your own risk” warning )
- the ‘-i’ option to run.sh has been added to usage text, but is not
  actually handled
- XB_EXTRA_ARGS in run.sh is unused
- srv_close_file_handlers should be initialized to FALSE for consistency

review: Needs Fixing
Revision history for this message
Alexey Kopytov (akopytov) wrote :

In the warning text, I would also explain the consequences briefly, rather than refer users to a bug with a long discussion.

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

Hi Alexey,

All comments are reasonable. I just want to explain why I haven't implement option for innobackupex. We don't recommend to use this option and normally user will not use it. However, if he need it badly, then he actually nows what he is doing. So there is an option for him to add it into my.cnf. Isn't it enough?

Revision history for this message
Alexey Kopytov (akopytov) wrote :

Sure, but what if a user hits the open files limit, but still wants to create full backups with mysql.* and other MyISAM tables, buffer pool dumps, etc.?

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

He can put the option into [xtrabackup] of my.cnf, it will work file. Anyways I already added the corresponding innobackupex option.

Revision history for this message
Alexey Kopytov (akopytov) wrote :

I'd be very cautious to put such an option to my.cnf. It's too easy to forget and replicate it to another server, for example. Anyway, now the users are free to choose how they want to use it. Thanks.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'doc/source/innobackupex/innobackupex_option_reference.rst'
--- doc/source/innobackupex/innobackupex_option_reference.rst 2014-05-18 08:23:31 +0000
+++ doc/source/innobackupex/innobackupex_option_reference.rst 2014-09-10 11:23:06 +0000
@@ -14,6 +14,10 @@
1414
15 Prepare a backup in ``BACKUP-DIR`` by applying the transaction log file named :file:`xtrabackup_logfile` located in the same directory. Also, create new transaction logs. The InnoDB configuration is read from the file :file:`backup-my.cnf` created by |innobackupex| when the backup was made. innobackupex --apply-log uses InnoDB configuration from ``backup-my.cnf`` by default, or from --defaults-file, if specified. InnoDB configuration in this context means server variables that affect data format, i.e. :option:`innodb_page_size`, :option:`innodb_log_block_size`, etc. Location-related variables, like :option:`innodb_log_group_home_dir` or :option:`innodb_data_file_path` are always ignored by --apply-log, so preparing a backup always works with data files from the backup directory, rather than any external ones.15 Prepare a backup in ``BACKUP-DIR`` by applying the transaction log file named :file:`xtrabackup_logfile` located in the same directory. Also, create new transaction logs. The InnoDB configuration is read from the file :file:`backup-my.cnf` created by |innobackupex| when the backup was made. innobackupex --apply-log uses InnoDB configuration from ``backup-my.cnf`` by default, or from --defaults-file, if specified. InnoDB configuration in this context means server variables that affect data format, i.e. :option:`innodb_page_size`, :option:`innodb_log_block_size`, etc. Location-related variables, like :option:`innodb_log_group_home_dir` or :option:`innodb_data_file_path` are always ignored by --apply-log, so preparing a backup always works with data files from the backup directory, rather than any external ones.
1616
17.. option:: --close-files
18
19 Do not keep files opened. This option is passed directly to xtrabackup. When xtrabackup opens tablespace it normally doesn't close its file handle in order to handle the DDL operations correctly. However, if the number of tablespaces is really huge and can not fit into any limit, there is an option to close file handles once they are no longer accessed. Xtrabackup can produce inconsistent backups with this option enabled. Use at your own risk.
20
17.. option:: --compact21.. option:: --compact
1822
19 Create a compact backup with all secondary index pages omitted. This option is passed directly to xtrabackup. See the :program:`xtrabackup` :doc:`documentation <../xtrabackup_bin/xtrabackup_binary>` for details.23 Create a compact backup with all secondary index pages omitted. This option is passed directly to xtrabackup. See the :program:`xtrabackup` :doc:`documentation <../xtrabackup_bin/xtrabackup_binary>` for details.
2024
=== modified file 'doc/source/xtrabackup_bin/xbk_option_reference.rst'
--- doc/source/xtrabackup_bin/xbk_option_reference.rst 2014-05-18 08:23:31 +0000
+++ doc/source/xtrabackup_bin/xbk_option_reference.rst 2014-09-10 11:23:06 +0000
@@ -17,6 +17,10 @@
1717
18 Make a backup and place it in :option:`--target-dir`. See :doc:`Creating a backup <creating_a_backup>`.18 Make a backup and place it in :option:`--target-dir`. See :doc:`Creating a backup <creating_a_backup>`.
1919
20.. option:: --close-files
21
22 Do not keep files opened. When xtrabackup opens tablespace it normally doesn't close its file handle in order to handle the DDL operations correctly. However, if the number of tablespaces is really huge and can not fit into any limit, there is an option to close file handles once they are no longer accessed. Xtrabackup can produce inconsistent backups with this option enabled. Use at your own risk.
23
20.. option:: --compact 24.. option:: --compact
2125
22 Create a compact backup by skipping secondary index pages.26 Create a compact backup by skipping secondary index pages.
2327
=== modified file 'innobackupex.pl'
--- innobackupex.pl 2014-09-02 14:54:11 +0000
+++ innobackupex.pl 2014-09-10 11:23:06 +0000
@@ -135,6 +135,7 @@
135my $option_safe_slave_backup = '';135my $option_safe_slave_backup = '';
136my $option_safe_slave_backup_timeout = 300;136my $option_safe_slave_backup_timeout = 300;
137137
138my $option_close_files = '';
138my $option_compact = '';139my $option_compact = '';
139my $option_rebuild_indexes = '';140my $option_rebuild_indexes = '';
140my $option_rebuild_threads = 0;141my $option_rebuild_threads = 0;
@@ -2802,6 +2803,10 @@
2802 $options = $options . " --stream=$option_stream";2803 $options = $options . " --stream=$option_stream";
2803 }2804 }
28042805
2806 if ($option_close_files) {
2807 $options = $options . " --close-files";
2808 }
2809
2805 if ($option_compact) {2810 if ($option_compact) {
2806 $options = $options . " --compact";2811 $options = $options . " --compact";
2807 }2812 }
@@ -3664,6 +3669,7 @@
3664 'parallel=i' => \$option_parallel,3669 'parallel=i' => \$option_parallel,
3665 'safe-slave-backup' => \$option_safe_slave_backup,3670 'safe-slave-backup' => \$option_safe_slave_backup,
3666 'safe-slave-backup-timeout=i' => \$option_safe_slave_backup_timeout,3671 'safe-slave-backup-timeout=i' => \$option_safe_slave_backup_timeout,
3672 'close-files' => \$option_close_files,
3667 'compact' => \$option_compact,3673 'compact' => \$option_compact,
3668 'rebuild-indexes' => \$option_rebuild_indexes,3674 'rebuild-indexes' => \$option_rebuild_indexes,
3669 'rebuild-threads=i' => \$option_rebuild_threads,3675 'rebuild-threads=i' => \$option_rebuild_threads,
@@ -4802,7 +4808,7 @@
4802 [--tmpdir=DIRECTORY] [--tables-file=FILE]4808 [--tmpdir=DIRECTORY] [--tables-file=FILE]
4803 [--incremental] [--incremental-basedir]4809 [--incremental] [--incremental-basedir]
4804 [--incremental-dir] [--incremental-force-scan] [--incremental-lsn]4810 [--incremental-dir] [--incremental-force-scan] [--incremental-lsn]
4805 [--compact] 4811 [--close-files] [--compact]
4806 BACKUP-ROOT-DIR4812 BACKUP-ROOT-DIR
48074813
4808innobackupex --apply-log [--use-memory=B]4814innobackupex --apply-log [--use-memory=B]
@@ -4874,6 +4880,10 @@
48744880
4875Prepare a backup in BACKUP-DIR by applying the transaction log file named "xtrabackup_logfile" located in the same directory. Also, create new transaction logs. The InnoDB configuration is read from the file "backup-my.cnf".4881Prepare a backup in BACKUP-DIR by applying the transaction log file named "xtrabackup_logfile" located in the same directory. Also, create new transaction logs. The InnoDB configuration is read from the file "backup-my.cnf".
48764882
4883=item --close-files
4884
4885Do not keep files opened. This option is passed directly to xtrabackup. Use at your own risk.
4886
4877=item --compact4887=item --compact
48784888
4879Create a compact backup with all secondary index pages omitted. This option is passed directly to xtrabackup. See xtrabackup documentation for details.4889Create a compact backup with all secondary index pages omitted. This option is passed directly to xtrabackup. See xtrabackup documentation for details.
48804890
=== modified file 'patches/innodb51.patch'
--- patches/innodb51.patch 2014-02-23 01:28:49 +0000
+++ patches/innodb51.patch 2014-09-10 11:23:06 +0000
@@ -288,7 +288,7 @@
288 }288 }
289 289
290 fil_system->max_assigned_id = id;290 fil_system->max_assigned_id = id;
291@@ -1438,7 +1488,12 @@291@@ -1438,7 +1448,12 @@
292 the file yet; the following calls will open it and update the292 the file yet; the following calls will open it and update the
293 size fields */293 size fields */
294 294
@@ -487,7 +487,7 @@
487+ inconsistencies between the file name and the actual tablespace contents487+ inconsistencies between the file name and the actual tablespace contents
488+ if a DDL occurs between a fil_load_single_table_tablespaces() call and488+ if a DDL occurs between a fil_load_single_table_tablespaces() call and
489+ the actual copy operation. */489+ the actual copy operation. */
490+ if (srv_backup_mode) {490+ if (srv_backup_mode && !srv_close_files) {
491+491+
492+ fil_node_t* node;492+ fil_node_t* node;
493+ fil_space_t* space;493+ fil_space_t* space;
@@ -723,7 +723,7 @@
723 || ibuf_page(space_id, zip_size, block_offset, NULL));723 || ibuf_page(space_id, zip_size, block_offset, NULL));
724 # endif /* UNIV_LOG_DEBUG */724 # endif /* UNIV_LOG_DEBUG */
725 if (sync) {725 if (sync) {
726@@ -4361,6 +4697,16 @@726@@ -4361,6 +4597,16 @@
727 727
728 ut_ad((mode != OS_AIO_IBUF) || (space->purpose == FIL_TABLESPACE));728 ut_ad((mode != OS_AIO_IBUF) || (space->purpose == FIL_TABLESPACE));
729 729
@@ -817,7 +817,7 @@
817 817
818--- a/storage/innodb_plugin/include/srv0srv.h818--- a/storage/innodb_plugin/include/srv0srv.h
819+++ b/storage/innodb_plugin/include/srv0srv.h819+++ b/storage/innodb_plugin/include/srv0srv.h
820@@ -202,6 +202,11 @@820@@ -202,6 +202,12 @@
821 extern ulong srv_max_purge_lag;821 extern ulong srv_max_purge_lag;
822 822
823 extern ulong srv_replication_delay;823 extern ulong srv_replication_delay;
@@ -826,6 +826,7 @@
826+extern ibool srv_fake_write;826+extern ibool srv_fake_write;
827+extern ibool srv_apply_log_only;827+extern ibool srv_apply_log_only;
828+extern ibool srv_backup_mode;828+extern ibool srv_backup_mode;
829+extern ibool srv_close_files;
829 /*-------------------------------------------*/830 /*-------------------------------------------*/
830 831
831 extern ulint srv_n_rows_inserted;832 extern ulint srv_n_rows_inserted;
@@ -1217,7 +1218,7 @@
1217 1218
1218--- a/storage/innodb_plugin/srv/srv0srv.c1219--- a/storage/innodb_plugin/srv/srv0srv.c
1219+++ b/storage/innodb_plugin/srv/srv0srv.c1220+++ b/storage/innodb_plugin/srv/srv0srv.c
1220@@ -383,6 +383,10 @@1221@@ -383,6 +383,11 @@
1221 1222
1222 UNIV_INTERN ulong srv_replication_delay = 0;1223 UNIV_INTERN ulong srv_replication_delay = 0;
1223 1224
@@ -1225,10 +1226,11 @@
1225+UNIV_INTERN ibool srv_fake_write = FALSE;1226+UNIV_INTERN ibool srv_fake_write = FALSE;
1226+UNIV_INTERN ibool srv_apply_log_only = FALSE;1227+UNIV_INTERN ibool srv_apply_log_only = FALSE;
1227+UNIV_INTERN ibool srv_backup_mode = FALSE;1228+UNIV_INTERN ibool srv_backup_mode = FALSE;
1229+UNIV_INTERN ibool srv_close_files = FALSE;
1228 /*-------------------------------------------*/1230 /*-------------------------------------------*/
1229 UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;1231 UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;
1230 UNIV_INTERN ulong srv_n_free_tickets_to_enter = 500;1232 UNIV_INTERN ulong srv_n_free_tickets_to_enter = 500;
1231@@ -1011,7 +1015,7 @@1233@@ -1011,7 +1016,7 @@
1232 }1234 }
1233 1235
1234 /* Initialize some INFORMATION SCHEMA internal structures */1236 /* Initialize some INFORMATION SCHEMA internal structures */
@@ -1237,7 +1239,7 @@
1237 }1239 }
1238 1240
1239 /*********************************************************************//**1241 /*********************************************************************//**
1240@@ -1022,6 +1026,7 @@1242@@ -1022,6 +1027,7 @@
1241 /*==========*/1243 /*==========*/
1242 {1244 {
1243 os_fast_mutex_free(&srv_conc_mutex);1245 os_fast_mutex_free(&srv_conc_mutex);
@@ -1245,7 +1247,7 @@
1245 mem_free(srv_conc_slots);1247 mem_free(srv_conc_slots);
1246 srv_conc_slots = NULL;1248 srv_conc_slots = NULL;
1247 1249
1248@@ -1035,6 +1040,7 @@1250@@ -1035,6 +1041,7 @@
1249 srv_mysql_table = NULL;1251 srv_mysql_table = NULL;
1250 1252
1251 trx_i_s_cache_free(trx_i_s_cache);1253 trx_i_s_cache_free(trx_i_s_cache);
12521254
=== modified file 'patches/innodb55.patch'
--- patches/innodb55.patch 2014-02-23 01:28:49 +0000
+++ patches/innodb55.patch 2014-09-10 11:23:06 +0000
@@ -487,7 +487,7 @@
487+ inconsistencies between the file name and the actual tablespace contents487+ inconsistencies between the file name and the actual tablespace contents
488+ if a DDL occurs between a fil_load_single_table_tablespaces() call and488+ if a DDL occurs between a fil_load_single_table_tablespaces() call and
489+ the actual copy operation. */489+ the actual copy operation. */
490+ if (srv_backup_mode) {490+ if (srv_backup_mode && !srv_close_files) {
491+491+
492+ fil_node_t* node;492+ fil_node_t* node;
493+ fil_space_t* space;493+ fil_space_t* space;
@@ -784,7 +784,7 @@
784 Returns TRUE if a single-table tablespace does not exist in the memory cache,784 Returns TRUE if a single-table tablespace does not exist in the memory cache,
785--- a/storage/innobase/include/srv0srv.h785--- a/storage/innobase/include/srv0srv.h
786+++ b/storage/innobase/include/srv0srv.h786+++ b/storage/innobase/include/srv0srv.h
787@@ -218,6 +218,11 @@787@@ -218,6 +218,12 @@
788 extern ulong srv_max_purge_lag;788 extern ulong srv_max_purge_lag;
789 789
790 extern ulong srv_replication_delay;790 extern ulong srv_replication_delay;
@@ -793,6 +793,7 @@
793+extern ibool srv_fake_write;793+extern ibool srv_fake_write;
794+extern ibool srv_apply_log_only;794+extern ibool srv_apply_log_only;
795+extern ibool srv_backup_mode;795+extern ibool srv_backup_mode;
796+extern ibool srv_close_files;
796 /*-------------------------------------------*/797 /*-------------------------------------------*/
797 798
798 extern ulint srv_n_rows_inserted;799 extern ulint srv_n_rows_inserted;
@@ -1139,7 +1140,7 @@
1139 }1140 }
1140--- a/storage/innobase/srv/srv0srv.c1141--- a/storage/innobase/srv/srv0srv.c
1141+++ b/storage/innobase/srv/srv0srv.c1142+++ b/storage/innobase/srv/srv0srv.c
1142@@ -405,6 +405,10 @@1143@@ -405,6 +405,11 @@
1143 1144
1144 UNIV_INTERN ulong srv_replication_delay = 0;1145 UNIV_INTERN ulong srv_replication_delay = 0;
1145 1146
@@ -1147,10 +1148,11 @@
1147+UNIV_INTERN ibool srv_fake_write = FALSE;1148+UNIV_INTERN ibool srv_fake_write = FALSE;
1148+UNIV_INTERN ibool srv_apply_log_only = FALSE;1149+UNIV_INTERN ibool srv_apply_log_only = FALSE;
1149+UNIV_INTERN ibool srv_backup_mode = FALSE;1150+UNIV_INTERN ibool srv_backup_mode = FALSE;
1151+UNIV_INTERN ibool srv_close_files = FALSE;
1150 /*-------------------------------------------*/1152 /*-------------------------------------------*/
1151 UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;1153 UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;
1152 UNIV_INTERN ulong srv_n_free_tickets_to_enter = 500;1154 UNIV_INTERN ulong srv_n_free_tickets_to_enter = 500;
1153@@ -1084,7 +1088,7 @@1155@@ -1084,7 +1089,7 @@
1154 }1156 }
1155 1157
1156 /* Initialize some INFORMATION SCHEMA internal structures */1158 /* Initialize some INFORMATION SCHEMA internal structures */
@@ -1159,7 +1161,7 @@
1159 }1161 }
1160 1162
1161 /*********************************************************************//**1163 /*********************************************************************//**
1162@@ -1095,6 +1099,7 @@1164@@ -1095,6 +1100,7 @@
1163 /*==========*/1165 /*==========*/
1164 {1166 {
1165 os_fast_mutex_free(&srv_conc_mutex);1167 os_fast_mutex_free(&srv_conc_mutex);
@@ -1167,7 +1169,7 @@
1167 mem_free(srv_conc_slots);1169 mem_free(srv_conc_slots);
1168 srv_conc_slots = NULL;1170 srv_conc_slots = NULL;
1169 1171
1170@@ -1108,6 +1113,7 @@1172@@ -1108,6 +1114,7 @@
1171 srv_mysql_table = NULL;1173 srv_mysql_table = NULL;
1172 1174
1173 trx_i_s_cache_free(trx_i_s_cache);1175 trx_i_s_cache_free(trx_i_s_cache);
@@ -1175,7 +1177,7 @@
1175 }1177 }
1176 1178
1177 /*********************************************************************//**1179 /*********************************************************************//**
1178@@ -1728,7 +1734,7 @@1180@@ -1728,7 +1735,7 @@
1179 }1181 }
1180 1182
1181 /* Record the lock wait time for this thread */1183 /* Record the lock wait time for this thread */
11821184
=== modified file 'patches/innodb56.patch'
--- patches/innodb56.patch 2014-08-17 09:44:20 +0000
+++ patches/innodb56.patch 2014-09-10 11:23:06 +0000
@@ -567,7 +567,7 @@
567+ inconsistencies between the file name and the actual tablespace contents567+ inconsistencies between the file name and the actual tablespace contents
568+ if a DDL occurs between a fil_load_single_table_tablespaces() call and568+ if a DDL occurs between a fil_load_single_table_tablespaces() call and
569+ the actual copy operation. */569+ the actual copy operation. */
570+ if (srv_backup_mode) {570+ if (srv_backup_mode && !srv_close_files) {
571+571+
572+ fil_node_t* node;572+ fil_node_t* node;
573+ fil_space_t* space;573+ fil_space_t* space;
@@ -1261,7 +1261,7 @@
1261 extern mysql_pfs_key_t innodb_file_data_key;1261 extern mysql_pfs_key_t innodb_file_data_key;
1262--- a/storage/innobase/include/srv0srv.h1262--- a/storage/innobase/include/srv0srv.h
1263+++ b/storage/innobase/include/srv0srv.h1263+++ b/storage/innobase/include/srv0srv.h
1264@@ -353,6 +353,13 @@1264@@ -353,6 +353,14 @@
1265 extern ulong srv_max_purge_lag_delay;1265 extern ulong srv_max_purge_lag_delay;
1266 1266
1267 extern ulong srv_replication_delay;1267 extern ulong srv_replication_delay;
@@ -1271,11 +1271,12 @@
1271+extern ibool srv_apply_log_only;1271+extern ibool srv_apply_log_only;
1272+1272+
1273+extern ibool srv_backup_mode;1273+extern ibool srv_backup_mode;
1274+extern ibool srv_close_files;
1274+1275+
1275 /*-------------------------------------------*/1276 /*-------------------------------------------*/
1276 1277
1277 extern my_bool srv_print_innodb_monitor;1278 extern my_bool srv_print_innodb_monitor;
1278@@ -496,13 +503,17 @@1279@@ -496,13 +504,17 @@
1279 the reason for which is that some FS1280 the reason for which is that some FS
1280 do not flush meta-data when1281 do not flush meta-data when
1281 unbuffered IO happens */1282 unbuffered IO happens */
@@ -2259,13 +2260,14 @@
2259 /* User settable value of the number of pages that must be present2260 /* User settable value of the number of pages that must be present
2260 in the buffer cache and accessed sequentially for InnoDB to trigger a2261 in the buffer cache and accessed sequentially for InnoDB to trigger a
2261 readahead request. */2262 readahead request. */
2262@@ -349,6 +353,13 @@2263@@ -349,6 +353,14 @@
2263 2264
2264 UNIV_INTERN ulong srv_replication_delay = 0;2265 UNIV_INTERN ulong srv_replication_delay = 0;
2265 2266
2266+UNIV_INTERN ibool srv_apply_log_only = FALSE;2267+UNIV_INTERN ibool srv_apply_log_only = FALSE;
2267+2268+
2268+UNIV_INTERN ibool srv_backup_mode = FALSE;2269+UNIV_INTERN ibool srv_backup_mode = FALSE;
2270+UNIV_INTERN ibool srv_close_files = TRUE;
2269+2271+
2270+UNIV_INTERN ulint srv_log_checksum_algorithm =2272+UNIV_INTERN ulint srv_log_checksum_algorithm =
2271+ SRV_CHECKSUM_ALGORITHM_INNODB;2273+ SRV_CHECKSUM_ALGORITHM_INNODB;
@@ -2273,7 +2275,7 @@
2273 /*-------------------------------------------*/2275 /*-------------------------------------------*/
2274 UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;2276 UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;
2275 UNIV_INTERN ulong srv_spin_wait_delay = 6;2277 UNIV_INTERN ulong srv_spin_wait_delay = 6;
2276@@ -1818,7 +1829,8 @@2278@@ -1818,7 +1830,8 @@
2277 if (ret == SRV_NONE2279 if (ret == SRV_NONE
2278 && srv_shutdown_state != SRV_SHUTDOWN_NONE2280 && srv_shutdown_state != SRV_SHUTDOWN_NONE
2279 && trx_purge_state() != PURGE_STATE_DISABLED2281 && trx_purge_state() != PURGE_STATE_DISABLED
22802282
=== modified file 'patches/xtradb51.patch'
--- patches/xtradb51.patch 2014-02-23 01:28:49 +0000
+++ patches/xtradb51.patch 2014-09-10 11:23:06 +0000
@@ -380,7 +380,7 @@
380+ inconsistencies between the file name and the actual tablespace contents380+ inconsistencies between the file name and the actual tablespace contents
381+ if a DDL occurs between a fil_load_single_table_tablespaces() call and381+ if a DDL occurs between a fil_load_single_table_tablespaces() call and
382+ the actual copy operation. */382+ the actual copy operation. */
383+ if (srv_backup_mode) {383+ if (srv_backup_mode && !srv_close_files) {
384+384+
385+ fil_node_t* node;385+ fil_node_t* node;
386+ fil_space_t* space;386+ fil_space_t* space;
@@ -763,7 +763,7 @@
763 return(buf);763 return(buf);
764--- a/storage/innodb_plugin/include/srv0srv.h764--- a/storage/innodb_plugin/include/srv0srv.h
765+++ b/storage/innodb_plugin/include/srv0srv.h765+++ b/storage/innodb_plugin/include/srv0srv.h
766@@ -243,6 +243,11 @@766@@ -243,6 +243,12 @@
767 767
768 extern ulong srv_replication_delay;768 extern ulong srv_replication_delay;
769 769
@@ -771,6 +771,7 @@
771+extern ibool srv_fake_write;771+extern ibool srv_fake_write;
772+extern ibool srv_apply_log_only;772+extern ibool srv_apply_log_only;
773+extern ibool srv_backup_mode;773+extern ibool srv_backup_mode;
774+extern ibool srv_close_files;
774+775+
775 extern long long srv_ibuf_max_size;776 extern long long srv_ibuf_max_size;
776 extern ulint srv_ibuf_active_contract;777 extern ulint srv_ibuf_active_contract;
@@ -1140,7 +1141,7 @@
1140 /* prototypes for new functions added to ha_innodb.cc */1141 /* prototypes for new functions added to ha_innodb.cc */
1141 ibool innobase_get_slow_log();1142 ibool innobase_get_slow_log();
1142 1143
1143@@ -438,6 +433,11 @@1144@@ -438,6 +433,12 @@
1144 1145
1145 UNIV_INTERN ulong srv_replication_delay = 0;1146 UNIV_INTERN ulong srv_replication_delay = 0;
1146 1147
@@ -1148,11 +1149,12 @@
1148+UNIV_INTERN ibool srv_fake_write = FALSE;1149+UNIV_INTERN ibool srv_fake_write = FALSE;
1149+UNIV_INTERN ibool srv_apply_log_only = FALSE;1150+UNIV_INTERN ibool srv_apply_log_only = FALSE;
1150+UNIV_INTERN ibool srv_backup_mode = FALSE;1151+UNIV_INTERN ibool srv_backup_mode = FALSE;
1152+UNIV_INTERN ibool srv_close_files = FALSE;
1151+1153+
1152 UNIV_INTERN long long srv_ibuf_max_size = 0;1154 UNIV_INTERN long long srv_ibuf_max_size = 0;
1153 UNIV_INTERN ulint srv_ibuf_active_contract = 0; /* 0:disable 1:enable */1155 UNIV_INTERN ulint srv_ibuf_active_contract = 0; /* 0:disable 1:enable */
1154 UNIV_INTERN ulint srv_ibuf_accel_rate = 100;1156 UNIV_INTERN ulint srv_ibuf_accel_rate = 100;
1155@@ -1095,7 +1095,7 @@1157@@ -1095,7 +1096,7 @@
1156 }1158 }
1157 1159
1158 /* Initialize some INFORMATION SCHEMA internal structures */1160 /* Initialize some INFORMATION SCHEMA internal structures */
@@ -1161,7 +1163,7 @@
1161 }1163 }
1162 1164
1163 /*********************************************************************//**1165 /*********************************************************************//**
1164@@ -1106,6 +1106,7 @@1166@@ -1106,6 +1107,7 @@
1165 /*==========*/1167 /*==========*/
1166 {1168 {
1167 os_fast_mutex_free(&srv_conc_mutex);1169 os_fast_mutex_free(&srv_conc_mutex);
@@ -1169,7 +1171,7 @@
1169 mem_free(srv_conc_slots);1171 mem_free(srv_conc_slots);
1170 srv_conc_slots = NULL;1172 srv_conc_slots = NULL;
1171 1173
1172@@ -1119,6 +1120,7 @@1174@@ -1119,6 +1121,7 @@
1173 srv_mysql_table = NULL;1175 srv_mysql_table = NULL;
1174 1176
1175 trx_i_s_cache_free(trx_i_s_cache);1177 trx_i_s_cache_free(trx_i_s_cache);
@@ -1177,7 +1179,7 @@
1177 }1179 }
1178 1180
1179 /*********************************************************************//**1181 /*********************************************************************//**
1180@@ -2634,36 +2636,6 @@1182@@ -2634,36 +2637,6 @@
1181 old_sema = sema;1183 old_sema = sema;
1182 }1184 }
1183 1185
11841186
=== modified file 'patches/xtradb55.patch'
--- patches/xtradb55.patch 2014-02-23 01:28:49 +0000
+++ patches/xtradb55.patch 2014-09-10 11:23:06 +0000
@@ -382,7 +382,7 @@
382+ inconsistencies between the file name and the actual tablespace contents382+ inconsistencies between the file name and the actual tablespace contents
383+ if a DDL occurs between a fil_load_single_table_tablespaces() call and383+ if a DDL occurs between a fil_load_single_table_tablespaces() call and
384+ the actual copy operation. */384+ the actual copy operation. */
385+ if (srv_backup_mode) {385+ if (srv_backup_mode && !srv_close_files) {
386+386+
387+ fil_node_t* node;387+ fil_node_t* node;
388+ fil_space_t* space;388+ fil_space_t* space;
@@ -791,7 +791,7 @@
791 extern char srv_adaptive_flushing;791 extern char srv_adaptive_flushing;
792 792
793 /* If this flag is TRUE, then we will load the indexes' (and tables') metadata793 /* If this flag is TRUE, then we will load the indexes' (and tables') metadata
794@@ -271,6 +270,11 @@794@@ -271,6 +270,12 @@
795 extern ulint srv_expand_import;795 extern ulint srv_expand_import;
796 extern ulint srv_pass_corrupt_table;796 extern ulint srv_pass_corrupt_table;
797 797
@@ -799,6 +799,7 @@
799+extern ibool srv_fake_write;799+extern ibool srv_fake_write;
800+extern ibool srv_apply_log_only;800+extern ibool srv_apply_log_only;
801+extern ibool srv_backup_mode;801+extern ibool srv_backup_mode;
802+extern ibool srv_close_files;
802+803+
803 /* Helper macro to support srv_pass_corrupt_table checks. If 'cond' is FALSE,804 /* Helper macro to support srv_pass_corrupt_table checks. If 'cond' is FALSE,
804 execute 'code' if srv_pass_corrupt_table is non-zero, or trigger a fatal error805 execute 'code' if srv_pass_corrupt_table is non-zero, or trigger a fatal error
@@ -1193,7 +1194,7 @@
1193 1194
1194 /* Try to flush dirty pages so as to avoid IO bursts at1195 /* Try to flush dirty pages so as to avoid IO bursts at
1195 the checkpoints. */1196 the checkpoints. */
1196@@ -435,6 +428,11 @@1197@@ -435,6 +428,12 @@
1197 UNIV_INTERN ulint srv_pass_corrupt_table = 0; /* 0:disable 1:enable */1198 UNIV_INTERN ulint srv_pass_corrupt_table = 0; /* 0:disable 1:enable */
1198 1199
1199 UNIV_INTERN ulint srv_dict_size_limit = 0;1200 UNIV_INTERN ulint srv_dict_size_limit = 0;
@@ -1202,10 +1203,11 @@
1202+UNIV_INTERN ibool srv_fake_write = FALSE;1203+UNIV_INTERN ibool srv_fake_write = FALSE;
1203+UNIV_INTERN ibool srv_apply_log_only = FALSE;1204+UNIV_INTERN ibool srv_apply_log_only = FALSE;
1204+UNIV_INTERN ibool srv_backup_mode = FALSE;1205+UNIV_INTERN ibool srv_backup_mode = FALSE;
1206+UNIV_INTERN ibool srv_close_files = FALSE;
1205 /*-------------------------------------------*/1207 /*-------------------------------------------*/
1206 UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;1208 UNIV_INTERN ulong srv_n_spin_wait_rounds = 30;
1207 UNIV_INTERN ulong srv_n_free_tickets_to_enter = 500;1209 UNIV_INTERN ulong srv_n_free_tickets_to_enter = 500;
1208@@ -1185,7 +1183,7 @@1210@@ -1185,7 +1184,7 @@
1209 }1211 }
1210 1212
1211 /* Initialize some INFORMATION SCHEMA internal structures */1213 /* Initialize some INFORMATION SCHEMA internal structures */
@@ -1214,7 +1216,7 @@
1214 }1216 }
1215 1217
1216 /*********************************************************************//**1218 /*********************************************************************//**
1217@@ -1196,6 +1194,7 @@1219@@ -1196,6 +1195,7 @@
1218 /*==========*/1220 /*==========*/
1219 {1221 {
1220 os_fast_mutex_free(&srv_conc_mutex);1222 os_fast_mutex_free(&srv_conc_mutex);
@@ -1222,7 +1224,7 @@
1222 mem_free(srv_conc_slots);1224 mem_free(srv_conc_slots);
1223 srv_conc_slots = NULL;1225 srv_conc_slots = NULL;
1224 1226
1225@@ -1209,6 +1208,7 @@1227@@ -1209,6 +1209,7 @@
1226 srv_mysql_table = NULL;1228 srv_mysql_table = NULL;
1227 1229
1228 trx_i_s_cache_free(trx_i_s_cache);1230 trx_i_s_cache_free(trx_i_s_cache);
@@ -1230,7 +1232,7 @@
1230 }1232 }
1231 1233
1232 /*********************************************************************//**1234 /*********************************************************************//**
1233@@ -1952,7 +1952,7 @@1235@@ -1952,7 +1953,7 @@
1234 }1236 }
1235 1237
1236 /* Record the lock wait time for this thread */1238 /* Record the lock wait time for this thread */
@@ -1239,7 +1241,7 @@
1239 }1241 }
1240 1242
1241 if (trx->was_chosen_as_deadlock_victim) {1243 if (trx->was_chosen_as_deadlock_victim) {
1242@@ -2975,36 +2975,6 @@1244@@ -2975,36 +2976,6 @@
1243 old_sema = sema;1245 old_sema = sema;
1244 }1246 }
1245 1247
12461248
=== modified file 'src/fil_cur.cc'
--- src/fil_cur.cc 2014-03-01 13:30:44 +0000
+++ src/fil_cur.cc 2014-09-10 11:23:06 +0000
@@ -162,7 +162,7 @@
162 /* In the backup mode we should already have a tablespace handle created162 /* In the backup mode we should already have a tablespace handle created
163 by fil_load_single_table_tablespace() unless it is a system163 by fil_load_single_table_tablespace() unless it is a system
164 tablespace. Otherwise we open the file here. */164 tablespace. Otherwise we open the file here. */
165 if (cursor->is_system || !srv_backup_mode) {165 if (cursor->is_system || !srv_backup_mode || srv_close_files) {
166 node->handle =166 node->handle =
167 xb_file_create_no_error_handling(node->name,167 xb_file_create_no_error_handling(node->name,
168 OS_FILE_OPEN,168 OS_FILE_OPEN,
169169
=== modified file 'src/xtrabackup.cc'
--- src/xtrabackup.cc 2014-09-09 17:22:21 +0000
+++ src/xtrabackup.cc 2014-09-10 11:23:06 +0000
@@ -298,6 +298,7 @@
298lsn_t xtrabackup_arch_last_file_lsn = 0ULL;298lsn_t xtrabackup_arch_last_file_lsn = 0ULL;
299299
300ulong xb_open_files_limit= 0;300ulong xb_open_files_limit= 0;
301my_bool xb_close_files= FALSE;
301302
302/* Datasinks */303/* Datasinks */
303ds_ctxt_t *ds_data = NULL;304ds_ctxt_t *ds_data = NULL;
@@ -487,7 +488,8 @@
487#endif488#endif
488 OPT_XTRA_INCREMENTAL_FORCE_SCAN,489 OPT_XTRA_INCREMENTAL_FORCE_SCAN,
489 OPT_DEFAULTS_GROUP,490 OPT_DEFAULTS_GROUP,
490 OPT_OPEN_FILES_LIMIT491 OPT_OPEN_FILES_LIMIT,
492 OPT_CLOSE_FILES
491};493};
492494
493#if MYSQL_VERSION_ID >= 50600495#if MYSQL_VERSION_ID >= 50600
@@ -915,6 +917,10 @@
915 (G_PTR*) &xb_open_files_limit, (G_PTR*) &xb_open_files_limit, 0, GET_ULONG,917 (G_PTR*) &xb_open_files_limit, (G_PTR*) &xb_open_files_limit, 0, GET_ULONG,
916 REQUIRED_ARG, 0, 0, UINT_MAX, 0, 1, 0},918 REQUIRED_ARG, 0, 0, UINT_MAX, 0, 1, 0},
917919
920 {"close_files", OPT_CLOSE_FILES, "do not keep files opened. Use at your own "
921 "risk.", (G_PTR*) &xb_close_files, (G_PTR*) &xb_close_files, 0, GET_BOOL,
922 NO_ARG, 0, 0, 0, 0, 0, 0},
923
918 { 0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}924 { 0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}
919};925};
920926
@@ -3059,6 +3065,13 @@
3059 xb_set_innodb_read_only();3065 xb_set_innodb_read_only();
30603066
3061 srv_backup_mode = TRUE;3067 srv_backup_mode = TRUE;
3068 srv_close_files = xb_close_files;
3069
3070 if (srv_close_files)
3071 msg("xtrabackup: warning: close-files specified. Use it "
3072 "at your own risk. If there are DDL operations like table DROP TABLE "
3073 "or RENAME TABLE during the backup, inconsistent backup will be "
3074 "produced.\n");
30623075
3063 /* initialize components */3076 /* initialize components */
3064 if(innodb_init_param())3077 if(innodb_init_param())
30653078
=== modified file 'test/inc/common.sh'
--- test/inc/common.sh 2014-09-10 07:36:39 +0000
+++ test/inc/common.sh 2014-09-10 11:23:06 +0000
@@ -240,7 +240,7 @@
240 fi240 fi
241241
242 IB_ARGS="--defaults-file=$MYSQLD_VARDIR/my.cnf --ibbackup=$XB_BIN \242 IB_ARGS="--defaults-file=$MYSQLD_VARDIR/my.cnf --ibbackup=$XB_BIN \
243--no-version-check"243--no-version-check ${IB_EXTRA_OPTS:-}"
244 XB_ARGS="--defaults-file=$MYSQLD_VARDIR/my.cnf"244 XB_ARGS="--defaults-file=$MYSQLD_VARDIR/my.cnf"
245245
246 # Some aliases for compatibility, as tests use the following names246 # Some aliases for compatibility, as tests use the following names
@@ -303,6 +303,9 @@
303[client]303[client]
304socket=${MYSQLD_SOCKET}304socket=${MYSQLD_SOCKET}
305user=root305user=root
306
307[xtrabackup]
308${XB_EXTRA_MY_CNF_OPTS:-}
306EOF309EOF
307310
308 # Create datadir and call mysql_install_db if it doesn't exist311 # Create datadir and call mysql_install_db if it doesn't exist
309312
=== modified file 'test/run.sh'
--- test/run.sh 2014-05-12 08:40:00 +0000
+++ test/run.sh 2014-09-10 11:23:06 +0000
@@ -51,6 +51,8 @@
51 on the MySQL version.51 on the MySQL version.
52-j N Run tests in N parallel processes.52-j N Run tests in N parallel processes.
53-T seconds Test timeout (default is $TEST_TIMEOUT seconds).53-T seconds Test timeout (default is $TEST_TIMEOUT seconds).
54-x options Extra options to pass to xtrabackup
55-i options Extra options to pass to innobackupex
54EOF56EOF
55}57}
5658
@@ -263,6 +265,9 @@
263 if gnutar --version > /dev/null 2>&1265 if gnutar --version > /dev/null 2>&1
264 then266 then
265 TAR=gnutar267 TAR=gnutar
268 elif gtar --version > /dev/null 2>&1
269 then
270 TAR=gtar
266 else271 else
267 TAR=tar272 TAR=tar
268 fi273 fi
@@ -324,7 +329,7 @@
324 MYSQLD_EXTRA_ARGS=329 MYSQLD_EXTRA_ARGS=
325330
326 XB_BIN=""331 XB_BIN=""
327 IB_ARGS="--user=root --ibbackup=$XB_BIN"332 IB_ARGS="--user=root --ibbackup=$XB_BIN ${IB_EXTRA_OPTS:-}"
328 XB_ARGS="--no-defaults"333 XB_ARGS="--no-defaults"
329334
330 if [ "$XB_BUILD" != "autodetect" ]335 if [ "$XB_BUILD" != "autodetect" ]
@@ -794,7 +799,7 @@
794NWORKERS=799NWORKERS=
795DEBUG_WORKER=""800DEBUG_WORKER=""
796801
797while getopts "fgh?:t:s:d:c:j:T:" options; do802while getopts "fgh?:t:s:d:c:j:T:x:i:" options; do
798 case $options in803 case $options in
799 f ) force="yes";;804 f ) force="yes";;
800 t )805 t )
@@ -831,6 +836,14 @@
831 TEST_TIMEOUT="$OPTARG"836 TEST_TIMEOUT="$OPTARG"
832 ;;837 ;;
833838
839 x )
840 XB_EXTRA_MY_CNF_OPTS="$OPTARG"
841 ;;
842
843 i )
844 IB_EXTRA_OPTS="$OPTARG"
845 ;;
846
834 ? ) echo "Use \`$0 -h' for the list of available options."847 ? ) echo "Use \`$0 -h' for the list of available options."
835 exit -1;;848 exit -1;;
836 esac849 esac
837850
=== added file 'test/t/bug1222062.sh'
--- test/t/bug1222062.sh 1970-01-01 00:00:00 +0000
+++ test/t/bug1222062.sh 2014-09-10 11:23:06 +0000
@@ -0,0 +1,11 @@
1########################################################################
2# Bug #1222062: add option --close-files
3########################################################################
4
5start_server
6
7mkdir $topdir/backup
8
9innobackupex --close-files $topdir/backup
10
11grep "xtrabackup: warning: close-files specified" $OUTFILE

Subscribers

People subscribed via source and target branches