lp:~laurynas-biveinis/percona-xtrabackup/xb-changed-page-bitmap
- Get this branch:
- bzr branch lp:~laurynas-biveinis/percona-xtrabackup/xb-changed-page-bitmap
Branch merges
- Alexey Kopytov (community): Approve
-
Diff: 3089 lines (+2134/-172)35 files modifieddoc/source/innobackupex/how_innobackupex_works.rst (+2/-2)
doc/source/xtrabackup_bin/incremental_backups.rst (+1/-1)
doc/source/xtrabackup_bin/xbk_option_reference.rst (+4/-0)
innobackupex (+123/-66)
src/Makefile (+11/-2)
src/changed_page_bitmap.cc (+1013/-0)
src/changed_page_bitmap.h (+84/-0)
src/compact.cc (+1/-1)
src/fil_cur.cc (+23/-19)
src/fil_cur.h (+10/-5)
src/innodb_int.cc (+1/-1)
src/innodb_int.h (+2/-0)
src/read_filt.cc (+206/-0)
src/read_filt.h (+63/-0)
src/xtrabackup.cc (+118/-31)
src/xtrabackup.h (+6/-0)
test/inc/common.sh (+61/-0)
test/inc/ib_incremental_common.sh (+25/-3)
test/t/bug1007446.sh (+1/-1)
test/t/ib_incremental_bitmap.sh (+26/-0)
test/t/ib_incremental_force_full_scan.sh (+27/-0)
test/t/ib_incremental_full_scan.sh (+8/-0)
test/t/xb_incremental_bitmap_misc.sh (+76/-0)
test/t/xb_incremental_compressed.inc (+59/-32)
test/t/xb_incremental_compressed_bitmap_16kb.sh (+28/-0)
test/t/xb_incremental_compressed_bitmap_1kb.sh (+28/-0)
test/t/xb_incremental_compressed_bitmap_2kb.sh (+28/-0)
test/t/xb_incremental_compressed_bitmap_4kb.sh (+28/-0)
test/t/xb_incremental_compressed_bitmap_8kb.sh (+28/-0)
test/t/xb_incremental_compressed_full_scan_16kb.sh (+8/-0)
test/t/xb_incremental_compressed_full_scan_1kb.sh (+8/-0)
test/t/xb_incremental_compressed_full_scan_2kb.sh (+8/-0)
test/t/xb_incremental_compressed_full_scan_4kb.sh (+8/-0)
test/t/xb_incremental_compressed_full_scan_8kb.sh (+8/-0)
test/t/xb_log_overwrap.sh (+3/-8)
Related bugs
Related blueprints
Branch information
- Owner:
- Laurynas Biveinis
- Status:
- Merged
Recent revisions
- 535. By Laurynas Biveinis
-
Implement bitmap-based incremental backups.
Blueprints:
https://blueprints. launchpad. net/percona- xtrabackup/ +spec/changed- page-bmp- inc-backups
and
https://blueprints. launchpad. net/percona- xtrabackup/ +spec/multiple- bmp-file- inc-backups - innobackupex: split the xtrabackup_
suspended to three different
files depending on the context (1st suspend, 2nd suspend, log copy
end). If backing up, check if server supports changed page bitmaps
by querying for the presence of I_S.INNODB_CHANGED_ PAGES plugin. If
found, add 1st suspend to xtrabackup invocation and issue FLUSH
CHANGED_PAGE_BITMAPS during it. Factor out waitpid/sync file
presence loop out of wait_for_ibbackup_ suspend and resume_ibbackup
into new subroutine wait_for_ibbackup_ file_create. Split the log
copying finish logic out of resume_ibbackup into a new subroutine
wait_for_ibbackup_ log_copy_ finish. Add a new command line option
--incremental-force- scan that passes the same option to xtrabackup,
and skips the first suspend and FLUSH CHANGED_PAGE_BITMAP calls.- New source files changed_
page_bitmap. h and changed_ page_bitmap. cc
containing the types and functions for working with page bitmaps and
their iterators.- New source files read_filt.h and read_filt.cc containing the data
file read filter abstraction. Provide two filters: rf_pass_through
for no filtering, and rf_bitmap that uses the changed page bitmaps.- fil_cur.h, fil_cur.cc: xb_fil_cur_t: move offset field to
xb_read_filt_ctxt_ t. Add fields for the read filter and its
context: read_filter and read_filter_ctxt. xb_fil_cur_open(): new
arg read_filter. Initialize the read filter context.
xb_fil_cur_read: use the read filter to get the next read offset and
length. xb_fil_cur_close(): deinitialize the read filter.- innodb_int.h: include mysql_version.h to make version checks not
depend on mysql_version.h having been included before innodb_int.h.- innodb_int.cc: fix an unrelated warning to make it easier to ensure
that the feature does not regress in warnings: xb_space_get_by_ name:
mark fold as unused.- xtrabackup.h: declare checkpoint_
lsn_start and changed_ page_bitmap. - xtrabackup.cc: split xtrabackup_
suspended to three different files
as in innobackupex. New functions xb_make_sync_file_ name and
xtrabackup_suspend, rename xb_create_ suspend_ file to
xb_create_sync_file. Add new option --suspend-at-start to suspend
xtrabackup after the log copying thread has started.
xtrabackup_copy_datafile: set up a bitmap read filter if the bitmap
has been allocated, a pass through filter otherwise and pass it to
the file cursor.
xtrabackup_backup_ func(): in case of incremental backups attempt to
read the changed page bitmap. Write a message if the bitmap was
found, otherwise that a full scan is going to be used. Free the
bitmap in the end.
New command-line option --incremental-force-scan to use the
full-scan algorithm even in the presence of bitmap data.- test/inc/common.sh: init_server_
variables( ), reset_server_ variables:
set up server error log location in SRV_MYSQLD_ERRFILE[ ].
switch_server(): set up MySQL error log.
check_full_scan_ inc_backup( ), check_bitmap_ inc_backup( ): new
functions for grepping XtraBackup output for the incremental backup
type used. wait_for_xb_suspend( ), resume_ suspended_ xb(): new helper
functions for the testcases that use xtrabackup suspend.- Rename test test/t/
ib_incremental. sh to a test include file
test/inc/ib_ incremental_ common. sh. Pass additional args in
$mysqld_extra_args to mysqld. - Rename test test/t/
xb_incremental. sh to an include file
test/inc/xb_ incremental_ common. sh. Pass additional args in
$mysqld_extra_args to mysqld. Adjust server options to have a
minimum possible log file size (1M) and thread concurrency that can
support such size for 5.0. Change the schema of test table T2 to
have large rows. Insert enough rows into a database so that the
resulting bitmap file spans at least two bitmap pages for the T2
tablespace.- Rename test test/t/
ib_incremental. sh to an include file
test/inc/ib_ incremental_ common. sh. Pass additional args in
$mysqld_extra_args to mysqld. Tweak it to increase the testing
coverage. ADd three new testcases ib_incremental_bitmap. sh,
ib_incremental_full_ scan.sh, and ib_incremental_ force_full_ scan.sh
that use it.- Adjust test/t/
xb_incremental_ compressed. inc for bitmap backups. New
tests xb_incremental_compressed_ bitmap_ (1-16)kb. sh. Rename
the previous xb_incremental_compressed_ *.sh tests to
xb_incremental_compressed_ full_scan_ *.sh. Remove the InnoDB version
check from xb_incremental_compressed. inc as it's always passing with
the currently supported versions.- New test xb_incremental_
bitmap_ misc.sh to test diagnostics for
partially-missing bitmap data and --incremental-force-scan option. - Temporarily disable all the bitmap tests on XtraDB Cluster
configuration until a supporting version is released.- Adjust test/t/bug1007446.sh for changed suspend file names.
- Adjust test/t/
xb_log_ overwraph. sh to use
wait_for_xb_ to_suspend/ resume_ suspended_ xb. - Makefile: update dependencies.
- Updated docs.
- 534. By Alexey Kopytov
-
Bug #1166713: Unused code in innobackupex
Removed a tiny bit of unused code from innobackupex. There are certainly
other parts, but it will eventually be "fixed" when innobackupex is
merged with xtrabackup binary. - 526. By <email address hidden>
-
Merge lp:~laurynas-biveinis/percona-xtrabackup/BT-28340-bug1158154-2.1
Branch metadata
- Branch format:
- Branch format 7
- Repository format:
- Bazaar repository format 2a (needs bzr 1.16 or later)
- Stacked on:
- lp:percona-xtrabackup/2.1