Created by Laurynas Biveinis and last modified
Get this branch:
bzr branch lp:~laurynas-biveinis/percona-xtrabackup/xb-changed-page-bitmap
Only Laurynas Biveinis can upload to this branch. If you are Laurynas Biveinis please log in for upload directions.

Branch merges

Related bugs

Related blueprints

Recent revisions

535. By Laurynas Biveinis

Implement bitmap-based incremental backups.

- innobackupex: split the xtrabackup_suspended to three different
  files depending on the context (1st suspend, 2nd suspend, log copy
  end). If backing up, check if server supports changed page bitmaps
  by querying for the presence of I_S.INNODB_CHANGED_PAGES plugin. If
  found, add 1st suspend to xtrabackup invocation and issue FLUSH
  CHANGED_PAGE_BITMAPS during it. Factor out waitpid/sync file
  presence loop out of wait_for_ibbackup_suspend and resume_ibbackup
  into new subroutine wait_for_ibbackup_file_create. Split the log
  copying finish logic out of resume_ibbackup into a new subroutine
  wait_for_ibbackup_log_copy_finish. Add a new command line option
  --incremental-force-scan that passes the same option to xtrabackup,
  and skips the first suspend and FLUSH CHANGED_PAGE_BITMAP calls.

- New source files changed_page_bitmap.h and changed_page_bitmap.cc
  containing the types and functions for working with page bitmaps and
  their iterators.

- New source files read_filt.h and read_filt.cc containing the data
  file read filter abstraction. Provide two filters: rf_pass_through
  for no filtering, and rf_bitmap that uses the changed page bitmaps.

- fil_cur.h, fil_cur.cc: xb_fil_cur_t: move offset field to
  xb_read_filt_ctxt_t. Add fields for the read filter and its
  context: read_filter and read_filter_ctxt. xb_fil_cur_open(): new
  arg read_filter. Initialize the read filter context.
  xb_fil_cur_read: use the read filter to get the next read offset and
  length. xb_fil_cur_close(): deinitialize the read filter.

- innodb_int.h: include mysql_version.h to make version checks not
  depend on mysql_version.h having been included before innodb_int.h.

- innodb_int.cc: fix an unrelated warning to make it easier to ensure
  that the feature does not regress in warnings: xb_space_get_by_name:
  mark fold as unused.

- xtrabackup.h: declare checkpoint_lsn_start and changed_page_bitmap.

- xtrabackup.cc: split xtrabackup_suspended to three different files
  as in innobackupex. New functions xb_make_sync_file_name and
  xtrabackup_suspend, rename xb_create_suspend_file to
  xb_create_sync_file. Add new option --suspend-at-start to suspend
  xtrabackup after the log copying thread has started.
  xtrabackup_copy_datafile: set up a bitmap read filter if the bitmap
  has been allocated, a pass through filter otherwise and pass it to
  the file cursor.
  xtrabackup_backup_func(): in case of incremental backups attempt to
  read the changed page bitmap. Write a message if the bitmap was
  found, otherwise that a full scan is going to be used. Free the
  bitmap in the end.
  New command-line option --incremental-force-scan to use the
  full-scan algorithm even in the presence of bitmap data.

- test/inc/common.sh: init_server_variables(), reset_server_variables:
  set up server error log location in SRV_MYSQLD_ERRFILE[].
  switch_server(): set up MySQL error log.
  check_full_scan_inc_backup(), check_bitmap_inc_backup(): new
  functions for grepping XtraBackup output for the incremental backup
  type used. wait_for_xb_suspend(), resume_suspended_xb(): new helper
  functions for the testcases that use xtrabackup suspend.

- Rename test test/t/ib_incremental.sh to a test include file
  test/inc/ib_incremental_common.sh. Pass additional args in
  $mysqld_extra_args to mysqld.

- Rename test test/t/xb_incremental.sh to an include file
  test/inc/xb_incremental_common.sh. Pass additional args in
  $mysqld_extra_args to mysqld. Adjust server options to have a
  minimum possible log file size (1M) and thread concurrency that can
  support such size for 5.0. Change the schema of test table T2 to
  have large rows. Insert enough rows into a database so that the
  resulting bitmap file spans at least two bitmap pages for the T2

- Rename test test/t/ib_incremental.sh to an include file
  test/inc/ib_incremental_common.sh. Pass additional args in
  $mysqld_extra_args to mysqld. Tweak it to increase the testing
  coverage. ADd three new testcases ib_incremental_bitmap.sh,
  ib_incremental_full_scan.sh, and ib_incremental_force_full_scan.sh
  that use it.

- Adjust test/t/xb_incremental_compressed.inc for bitmap backups. New
  tests xb_incremental_compressed_bitmap_(1-16)kb.sh. Rename
  the previous xb_incremental_compressed_*.sh tests to
  xb_incremental_compressed_full_scan_*.sh. Remove the InnoDB version
  check from xb_incremental_compressed.inc as it's always passing with
  the currently supported versions.

- New test xb_incremental_bitmap_misc.sh to test diagnostics for
  partially-missing bitmap data and --incremental-force-scan option.

- Temporarily disable all the bitmap tests on XtraDB Cluster
  configuration until a supporting version is released.

- Adjust test/t/bug1007446.sh for changed suspend file names.

- Adjust test/t/xb_log_overwraph.sh to use

- Makefile: update dependencies.

- Updated docs.

534. By Alexey Kopytov

Bug #1166713: Unused code in innobackupex

Removed a tiny bit of unused code from innobackupex. There are certainly
other parts, but it will eventually be "fixed" when innobackupex is
merged with xtrabackup binary.

533. By Alexey Kopytov

Manual merge from 2.0.

532. By Alexey Kopytov

Manual merge of basic 5.6 support from 2.0.

530. By <email address hidden>

Merge lp:~percona-core/percona-xtrabackup/release-2.1.0-alpha1

529. By <email address hidden>

Merge lp:~akopytov/percona-xtrabackup/bug1162765-2.1

528. By <email address hidden>

Merge lp:~akopytov/percona-xtrabackup/bug945161-2.1

527. By <email address hidden>

Merge lp:~akopytov/percona-xtrabackup/bug1116177-2.1

526. By <email address hidden>

Merge lp:~laurynas-biveinis/percona-xtrabackup/BT-28340-bug1158154-2.1

Branch metadata

Branch format:
Branch format 7
Repository format:
Bazaar repository format 2a (needs bzr 1.16 or later)
Stacked on:
This branch contains Public information 
Everyone can see this information.