Merge lp:~sergei.glushchenko/percona-xtrabackup/BT-26901-2.1 into lp:percona-xtrabackup/2.0

[Sergei Glushchenko]

Merge fix for #1066843 from 2.0

http://jenkins.percona.com/view/XtraBackup/job/percona-xtrabackup-2.1-param/66/

[Sergei Glushchenko (sergei.glushchenko)]

#26901

[Laurynas Biveinis (laurynas-biveinis)]

There is a conflict in innobackupex.

(Just a heads-up, not a full review)

review: Needs Fixing

[Laurynas Biveinis (laurynas-biveinis)]

Ah, wrong target branch.

=== modified file 'VERSION'
--- VERSION	2012-08-06 01:11:14 +0000
+++ VERSION	2012-10-18 11:58:23 +0000
@@ -1,1 +1,1 @@
-XTRABACKUP_VERSION=2.0.3
+XTRABACKUP_VERSION=2.1.0
 
=== modified file 'doc/source/conf.py'
--- doc/source/conf.py	2012-09-21 12:32:30 +0000
+++ doc/source/conf.py	2012-10-18 11:58:23 +0000
@@ -51,9 +51,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '2.0.3'
+version = '2.1'
 # The full version, including alpha/beta/rc tags.
-release = '2.0.3'
+release = '2.1'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
@@ -222,7 +222,7 @@
 # Grouping the document tree into LaTeX files. List of tuples
 # (source start file, target name, title, author, documentclass [howto/manual]).
 latex_documents = [
-  ('index', 'PerconaXtraBackup.tex', u'Percona XtraBackup Documentation',
+  ('index', 'PerconaXtraBackup-2.1.tex', u'Percona XtraBackup Documentation',
    u'Percona Inc', 'manual'),
 ]
 
 
=== modified file 'doc/source/innobackupex/how_innobackupex_works.rst'
--- doc/source/innobackupex/how_innobackupex_works.rst	2012-09-06 05:15:27 +0000
+++ doc/source/innobackupex/how_innobackupex_works.rst	2012-10-18 11:58:23 +0000
@@ -53,10 +53,6 @@
 :file:`mysql-stdout`
   containing the ``STDOUT`` of the server.
 
-If the :option:`--remote-host` was set, |innobackupex| will test the connection to the host via :command:`ssh` and create the backup directories. Then the same process will be applied but the log will be written to a temporary file and will be copied via :command:`scp` with the options set by :option:`--scpopt` (``-Cp -c arcfour`` by default).
-
-After each copy the files will be deleted. The same rationale is for the :option:`--stream` mode.
-
 Finally, the binary log position will be printed to ``STDERR`` and |innobackupex| will exit returning 0 if all went OK.
 
 Note that the ``STDERR`` of |innobackupex| is not written in any file. You will have to redirect it to a file, e.g., ``innobackupex OPTIONS 2> backupout.log``.
 
=== modified file 'doc/source/innobackupex/innobackupex_option_reference.rst'
--- doc/source/innobackupex/innobackupex_option_reference.rst	2012-09-21 12:32:30 +0000
+++ doc/source/innobackupex/innobackupex_option_reference.rst	2012-10-18 11:58:23 +0000
@@ -127,17 +127,13 @@
 
    This option accepts a string argument that specifies the directory in which to save an extra copy of the :file:`xtrabackup_checkpoints` file. It is passed directly to |xtrabackup|'s :option:`--extra-lsndir` option. See the :program:`xtrabackup` documentation for details.
 
-.. option:: --remote-host=HOSTNAME
-
-   This option accepts a string argument that specifies the remote host on which the backup files will be created, by using an ssh connection. This option is DEPRECATED and will be removed in Percona XtraBackup 2.1. In Percona XtraBackup 2.0 and later, you should use streaming backups instead.
-
 .. option:: --stream=STREAMNAME
 
    This option accepts a string argument that specifies the format in which to do the streamed backup. The backup will be done to ``STDOUT`` in the specified format. Currently, supported formats are `tar` and `xbstream`. Uses :doc:`xbstream <../xbstream/xbstream>`, which is available in *XtraBackup* distributions. If you specify a path after this option, it will be interpreted as the value of :option:`tmpdir`.
 
 .. option:: --tmpdir=DIRECTORY
 
-   This option accepts a string argument that specifies the location where a temporary file will be stored. It should be used when :option:`--remote-host` or :option:`--stream` is specified. For these options, the transaction log will first be stored to a temporary file, before streaming or copying to a remote host. This option specifies the location where that temporary file will be stored. If the option is not specifed, the default is to use the value of ``tmpdir`` read from the server configuration.
+   This option accepts a string argument that specifies the location where a temporary file will be stored. It should be used when :option:`--stream` is specified. For these options, the transaction log will first be stored to a temporary file, before streaming. This option specifies the location where that temporary file will be stored. If the option is not specifed, the default is to use the value of ``tmpdir`` read from the server configuration.
 
 .. option:: --scpopt = '-Cp -c arcfour'
 
@@ -161,4 +157,4 @@
 
 .. option:: --rsync
 
-   Use the :program:`rsync` utility to optimize local file transfers. When this option is specified, :program:`innobackupex` uses :program:`rsync` to copy all non-InnoDB files instead of spawning a separate :program:`cp` for each file, which can be much faster for servers with a large number of databases or tables.  This option cannot be used together with :option:`--remote-host` or :option:`--stream`.
+   Use the :program:`rsync` utility to optimize local file transfers. When this option is specified, :program:`innobackupex` uses :program:`rsync` to copy all non-InnoDB files instead of spawning a separate :program:`cp` for each file, which can be much faster for servers with a large number of databases or tables.  This option cannot be used together with :option:`--stream`.
 
=== modified file 'doc/source/innobackupex/innobackupex_script.rst'
--- doc/source/innobackupex/innobackupex_script.rst	2011-07-28 05:29:04 +0000
+++ doc/source/innobackupex/innobackupex_script.rst	2012-10-18 11:58:23 +0000
@@ -46,7 +46,6 @@
    replication_ibk
    parallel_copy_ibk
    throttling_ibk
-   remote_backups_ibk
    importing_exporting_tables_ibk
    pit_recovery_ibk
 
 
=== renamed file 'doc/source/innobackupex/remote_backups_ibk.rst' => 'doc/source/innobackupex/remote_backups_ibk.rst.THIS'
--- doc/source/innobackupex/remote_backups_ibk.rst	2012-09-19 13:00:42 +0000
+++ doc/source/innobackupex/remote_backups_ibk.rst.THIS	2012-10-18 11:58:23 +0000
@@ -13,10 +13,6 @@
 Then all the log files will be written to a temporary file (you can choose where to store this file with the :option:`--tmpdir` option) and will be copied via :command:`scp`. The options for :command:`scp` can be specified with :option:`--options-scp` (``-Cp -c arcfour`` by default), for example::
 
   $ innobackupex --remote-host=REMOTEUSER@REMOTEHOST /path/IN/REMOTE/HOST/to/backup/ \
-     --tmpdir=/tmp --scpopt="-Cp -c arcfour"
-
-.. note:: 
-
- `SSH  public key authentication <http://www.petefreitag.com/item/532.cfm>`_ should be set up to avoid the login prompt when doing the backup to the remote host.
+     --tmpdir=/tmp --options-scp="-Cp -c arcfour"
 
 
 
=== modified file 'doc/xtrabackup.1'
--- doc/xtrabackup.1	2011-01-10 12:01:38 +0000
+++ doc/xtrabackup.1	2012-10-18 11:58:23 +0000
@@ -1392,21 +1392,6 @@
 .B "xtrabackup"
 and your script are working on the same dataset.
 
-.SS " Log Streaming "
-
-
-You can instruct 
-.B "xtrabackup"
-to omit copying data files, and simply stream the log file to its standard output instead with 
-.B "--log-stream"
-.  This automatically adds the 
-.B "--suspend-at-end"
-option.  Your script can then perform tasks such as streaming remote backups by piping the log files into an SSH connection and copying the data files to another server with a tool such as 
-.B "rsync"
-or 
-.B "tar4ibd"
-.
-
 .SH " XtraBackup Exit Codes "
 
 
 
=== modified file 'innobackupex'
--- innobackupex	2012-10-16 15:05:23 +0000
+++ innobackupex	2012-10-18 11:58:23 +0000
@@ -101,7 +101,6 @@
 my $option_incremental_dir = '';
 my $option_incremental_lsn = '';
 my $option_extra_lsndir = '';
-my $option_remote_host = '';
 my $option_rsync = '';
 my $option_stream = '';
 my $stream_cmd = '';
@@ -116,6 +115,8 @@
 my $option_safe_slave_backup = '';
 my $option_safe_slave_backup_timeout = 300;
 
+my $option_compact = '';
+
 # name of the my.cnf configuration file
 #my $config_file = '';
 
@@ -290,11 +291,7 @@
 } else {
     # make a backup of InnoDB and MyISAM tables, indexes and .frm files.
     $ibbackup_exit_code = backup();
-    if ($option_remote_host) {
-      open(XTRABACKUP_BINARY,
-	   "| ssh $option_ssh_opt $option_remote_host 'cat > $backup_dir/$xtrabackup_binary_file'")
-	|| die "Failed to open file '$option_remote_host:$backup_dir/$xtrabackup_binary_file': $!";
-    } elsif ($option_stream) {
+    if ($option_stream) {
       open XTRABACKUP_BINARY, "> $option_tmpdir/$xtrabackup_binary_file"
     	|| die "Cannot open file $option_tmpdir/$xtrabackup_binary_file: $!\n";
     } else {
@@ -396,9 +393,7 @@
     start_ibbackup();
 
     # wait for ibbackup to suspend itself
-    if (!$option_remote_host) {
-        wait_for_ibbackup_suspend();
-    }
+    wait_for_ibbackup_suspend();
 
     # connect to database
     mysql_open();
@@ -441,11 +436,7 @@
 
     # copy ib_lru_dump
     if (-e "$orig_datadir/ib_lru_dump") {
-        if ($option_remote_host) {
-            print STDERR "$prefix Backing up file 'ib_lru_dump'\n";
-            system("scp $option_scp_opt '$orig_datadir/ib_lru_dump' '$option_remote_host:$backup_dir/ib_lru_dump'")
-                and Die "Failed to scp file 'ib_lru_dump': $!";
-        } elsif ($option_stream) {
+        if ($option_stream) {
             print STDERR "$prefix Backing up as tar stream 'ib_lru_dump'\n";
             system("cd $orig_datadir; $stream_cmd ib_lru_dump")
                 and Die "Failed to stream 'ib_lru_dump': $!";
@@ -457,16 +448,6 @@
         }
     }
 
-    if ($option_remote_host) {
-        system("scp $option_scp_opt '$tmp_logfile' '$option_remote_host:$backup_dir/xtrabackup_logfile'")
-            and Die "Failed to scp file '$option_remote_host:$backup_dir/xtrabackup_logfile': $!";
-        unlink $tmp_logfile || Die "Failed to delete '$tmp_logfile': $!";
-
-        system("scp $option_scp_opt '$option_tmpdir/xtrabackup_checkpoints' '$option_remote_host:$backup_dir/xtrabackup_checkpoints'")
-            and Die "Failed to scp file '$option_remote_host:$backup_dir/xtrabackup_checkpoints': $!";
-        unlink "$option_tmpdir/xtrabackup_checkpoints" || Die "Failed to delete '$option_tmpdir/xtrabackup_checkpoints': $!";
-    }
-
     print STDERR "\n$prefix Backup created in directory '$backup_dir'\n";
     if ($mysql_binlog_position) {
         print STDERR "$prefix MySQL binlog position: $mysql_binlog_position\n";
@@ -719,6 +700,14 @@
     my $file;
     my $backup_innodb_data_file_path;
 
+    if (has_option(\%config, $option_defaults_group, 'innodb_doublewrite_file')) {
+        my $doublewrite_file =
+            (split(/\/+/,
+                get_option(\%config, $option_defaults_group,
+                            'innodb_doublewrite_file')))[-1];
+        $excluded_files = $excluded_files . '|' . $doublewrite_file;
+    }
+
     # check whether files should be copied or moved to dest directory
     my $move_or_copy_file = $move_flag ? \&move_file : \&copy_file;
     my $move_or_copy_dir = $move_flag ?
@@ -955,14 +944,11 @@
 
     $options = $options . "--backup --suspend-at-end";
 
-    if (!$option_remote_host && !$option_stream) {
-        $options = $options . " --target-dir=$backup_dir";
-    } else {
+    if ($option_stream) {
         #(datadir) for 'xtrabackup_suspended' and 'xtrabackup_checkpoints'
         $options = $options . " --target-dir=" . $option_tmpdir;
-	if ($option_remote_host) {
-	  $options = $options . " --log-stream";
-	}
+    } else {
+	$options = $options . " --target-dir=$backup_dir";
     }
 
     # prepare command line for running ibbackup
@@ -1003,6 +989,10 @@
     if ($option_stream) {
 	$options = $options . " --stream=$option_stream";
     }
+    if ($option_compact) {
+	$options = $options. " --compact";
+    }
+
     $cmdline = "$option_ibbackup_binary $options";
 
     # run ibbackup as a child process
@@ -1012,97 +1002,7 @@
         if ($pid) {
             # parent process
             $ibbackup_pid = $pid;
-
-            if($option_remote_host) {
-                #direct copy to remote
-                my $orig_datadir = get_option(\%config, $option_defaults_group, 'datadir');
-                my $orig_ibdata_dir =
-                    get_option(\%config, $option_defaults_group, 'innodb_data_home_dir');
-                my $orig_innodb_data_file_path =
-                    get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
-                my $innodb_flush_method = 
-                    get_option(\%config, $option_defaults_group, 'innodb_flush_method');
-                my $innodb_use_odirect;
-                $innodb_use_odirect = 1 if $innodb_flush_method =~ m/^(ALL_)?O_DIRECT$/i;
-
-                my $subdir;
-                my @list;
-
-                if (system("ssh $option_ssh_opt $option_remote_host test -e $backup_dir/ib_logfile0")
-                    == 0) {
-                    print STDERR "$prefix Remove $option_remote_host:$backup_dir/ib_logfile*\n";
-                    system("ssh $option_ssh_opt $option_remote_host rm $backup_dir/ib_logfile\*")
-                        and Die "Failed to rm file '$backup_dir/ib_logfile*': $!";
-                }
-
-                wait_for_ibbackup_suspend();
-
-                #InnoDB data files from original InnoDB data directory
-                print STDERR "\n$prefix Starting to backup InnoDB tables and indexes\n";
-                print STDERR "$prefix to '$backup_dir'\n";
-                print STDERR "$prefix from original InnoDB data directory '$orig_ibdata_dir'\n";
-                foreach my $a (split(/;/, $orig_innodb_data_file_path)) {
-                    my $path = (split(/:/,$a))[0];
-                    $path=~s/([\$\\\" ])/\\$1/g;
-                    print STDERR "$prefix Backing up file '$orig_ibdata_dir/$path'\n";
-                    system("scp $option_scp_opt '$orig_ibdata_dir/$path' '$option_remote_host:$backup_dir/$path'")
-                      and Die "Failed to scp file '$path': $!";
-                }
-
-                #copy *.ibd files
-                opendir(DIR, $orig_datadir)
-                    || Die "Can't open directory '$orig_datadir': $!\n";
-                while (defined($subdir = readdir(DIR))) {
-                    my $print_each_file = 0;
-                    my $file_c;
-                    my $file;
-                    if ($subdir eq '.' || $subdir eq '..') { next; }
-                    next unless -d "$orig_datadir/$subdir";
-                    next unless check_if_required($subdir);
-
-                    @list = glob("$orig_datadir/$subdir/" . '*.ibd');
-
-                    $file_c = @list;
-                    if ($file_c <= $backup_file_print_limit) {
-                        $print_each_file = 1;
-                    } else {
-                        print STDERR "$prefix Backing up files " .
-                            "'$orig_datadir/$subdir/*.ibd' ($file_c files)\n";
-                    }
-                    foreach $file (@list) {
-			next unless check_if_required($subdir, $file);
-                        if($option_include) {
-                            my $table_name;
-
-                            $table_name = substr($file, rindex($file, '/'));
-                            $table_name = substr($table_name, 1, rindex($table_name, '.') - 1);
-                            $table_name = $subdir . "." . $table_name;
-
-                            if (!($table_name =~ /$option_include/)) {
-                                print STDERR "'$file' is skipped.\n";
-                                next;
-                            }
-                        }
-
-                        if ($print_each_file) {
-                            print STDERR "$prefix Backing up file '$file'\n";
-                        }
-                        if (system("ssh $option_ssh_opt $option_remote_host test -e $backup_dir/$subdir")
-                            != 0) {
-                          system("ssh $option_ssh_opt $option_remote_host mkdir $backup_dir/$subdir");
-                        }
-                        system("scp $option_scp_opt '$file' '$option_remote_host:$backup_dir/$subdir/'")
-                          and Die "Failed to scp file '$file': $!";
-		      }
-                }
-                closedir(DIR);
-            }
         } else {
-            if($option_remote_host) {
-                open(STDOUT, "> $tmp_logfile")
-                || Die "Failed to open file '$tmp_logfile': $!"
-            }
-
             # child process
             exec($cmdline) || Die "Failed to exec ibbackup: $!";
         }
@@ -1299,13 +1199,7 @@
     }
 
     # write binlog info file
-    if (!$option_remote_host) {
-        open(FILE, ">$binlog_info") || 
-            Die "Failed to open file '$binlog_info': $!";
-    } else {
-        open(FILE, "| ssh $option_ssh_opt $option_remote_host 'cat > $binlog_info'") ||
-            Die "Failed to open file '$option_remote_host:$binlog_info': $!";
-    }
+    open(FILE, ">$binlog_info") || Die "Failed to open file '$binlog_info': $!";
     print FILE  "$filename\t$position\t\t\n";
     close(FILE);
 
@@ -1389,13 +1283,7 @@
     }
 
     # print slave status to a file
-    if (!$option_remote_host) {
-        open(FILE, ">$slave_info") || 
-            Die "Failed to open file '$slave_info': $!";
-    } else {
-        open(FILE, "| ssh $option_ssh_opt $option_remote_host 'cat > $slave_info'") ||
-            Die "Failed to open file '$option_remote_host:$slave_info': $!";
-    }
+    open(FILE, ">$slave_info") || Die "Failed to open file '$slave_info': $!";
     print FILE  "CHANGE MASTER TO MASTER_LOG_FILE='$filename', MASTER_LOG_POS=$position\n";
     close(FILE);
 
@@ -1669,32 +1557,20 @@
 
     if (!$option_apply_log && !$option_copy_back && !$option_move_back) {
         # we are making a backup, create a new backup directory
-        if (!$option_remote_host) {
-            $backup_dir = File::Spec->rel2abs(make_backup_dir());
-        } else {
-            $backup_dir = make_backup_dir();
-        }
+        $backup_dir = File::Spec->rel2abs(make_backup_dir());
         print STDERR "$prefix Created backup directory $backup_dir\n";
-        if (!$option_remote_host && !$option_stream) {
-        $backup_config_file = $backup_dir . '/backup-my.cnf';
-        $suspend_file = $backup_dir . '/xtrabackup_suspended';
-        $binlog_info = $backup_dir . '/xtrabackup_binlog_info';
-        $galera_info = $backup_dir . '/xtrabackup_galera_info';
-        $slave_info = $backup_dir . '/xtrabackup_slave_info';
-        } else {
-        $suspend_file = $option_tmpdir . '/xtrabackup_suspended';
-        $tmp_logfile = $option_tmpdir . '/xtrabackup_logfile';
-        if ($option_stream) {
-            $backup_config_file = $option_tmpdir . '/backup-my.cnf';
-            $binlog_info = $option_tmpdir . '/xtrabackup_binlog_info';
-            $galera_info = $option_tmpdir . '/xtrabackup_galera_info';
-            $slave_info = $option_tmpdir . '/xtrabackup_slave_info';
-        } else {
+        if (!$option_stream) {
             $backup_config_file = $backup_dir . '/backup-my.cnf';
+            $suspend_file = $backup_dir . '/xtrabackup_suspended';
             $binlog_info = $backup_dir . '/xtrabackup_binlog_info';
             $galera_info = $backup_dir . '/xtrabackup_galera_info';
             $slave_info = $backup_dir . '/xtrabackup_slave_info';
-        }
+        } else {
+            $suspend_file = $option_tmpdir . '/xtrabackup_suspended';
+            $tmp_logfile = $option_tmpdir . '/xtrabackup_logfile';
+	    $backup_config_file = $option_tmpdir . '/backup-my.cnf';
+	    $binlog_info = $option_tmpdir . '/xtrabackup_binlog_info';
+	    $slave_info = $option_tmpdir . '/xtrabackup_slave_info';
         }
         write_backup_config_file($backup_config_file);
     } elsif ($option_copy_back || $option_move_back) {
@@ -1714,12 +1590,7 @@
 sub write_backup_config_file {
     my $filename = shift;
 
-    if (!$option_remote_host) {
-        open(FILE, "> $filename") || Die "Failed to open file '$filename': $!";
-    } else {
-        open(FILE, "| ssh $option_ssh_opt $option_remote_host 'cat > $filename'")
-            || Die "Failed to open file '$option_remote_host:$filename': $!";
-    }
+    open(FILE, "> $filename") || Die "Failed to open file '$filename': $!";
 
     my @option_names = (
         "innodb_data_file_path",
@@ -1800,7 +1671,7 @@
                         'use-memory=s' => \$option_use_memory,
                         'uncompress' => \$option_uncompress,
                         'export' => \$option_export,
-                        'password=s' => \$option_mysql_password,
+                        'password:s' => \$option_mysql_password,
                         'user=s' => \$option_mysql_user,
                         'host=s' => \$option_mysql_host,
                         'port=s' => \$option_mysql_port,
@@ -1815,7 +1686,6 @@
                         'incremental-lsn=s' => \$option_incremental_lsn,
                         'incremental-dir=s' => \$option_incremental_dir,
                         'extra-lsndir=s' => \$option_extra_lsndir,
-                        'remote-host=s' => \$option_remote_host,
                         'stream=s' => \$option_stream,
 			'rsync' => \$option_rsync,
                         'tmpdir=s' => \$option_tmpdir,
@@ -1825,7 +1695,12 @@
                         'sshopt=s' => \$option_ssh_opt,
                         'parallel=i' => \$option_parallel,
                         'safe-slave-backup' => \$option_safe_slave_backup,
+<<<<<<< TREE
                         'safe-slave-backup-timeout=i' => \$option_safe_slave_backup_timeout,
+=======
+                        'safe-slave-backup-timeout' => $option_safe_slave_backup_timeout,
+                        'compact' => \$option_compact
+>>>>>>> MERGE-SOURCE
     );
                         
     if (!$rcode) {
@@ -1901,16 +1776,11 @@
 	}
     }
 
-    if ($option_rsync && ($option_remote_host || $option_stream)) {
-        print STDERR "--rsync doesn't work with --remote-host or --stream\n";
+    if ($option_rsync && $option_stream) {
+        print STDERR "--rsync doesn't work with --stream\n";
 	exit(1);
     }
 
-    if ($option_remote_host) {
-	print STDERR "\nWARNING: --remote-host is DEPRECATED and will be ",
-	"removed in Percona XtraBackup 2.1 in favor of streaming backups.\n";
-    }
-
     print STDERR "\n";
 
     parse_databases_option_value();
@@ -1935,11 +1805,7 @@
 
     $dir .= '/' . strftime("%Y-%m-%d_%H-%M-%S", localtime())
        unless $option_no_timestamp;
-    if (!$option_remote_host) {
-        mkdir($dir, 0777) || Die "Failed to create backup directory $dir: $!";
-    } else {
-        system("ssh $option_ssh_opt $option_remote_host mkdir $dir");
-    }
+    mkdir($dir, 0777) || Die "Failed to create backup directory $dir: $!";
 
     # create subdirectories for ibdata files if needed
 #    foreach my $a (split(/;/, $innodb_data_file_path)) {
@@ -1975,15 +1841,9 @@
     $path = $root;
     foreach $a (@{$relative_path}) {
         $path = $path . "/" . $a;
-        if (!$option_remote_host) {
-            if (! -d $path) {
-                # this directory does not exist, create it !
-                mkdir($path, 0777) || Die "Failed to create backup directory: $!";
-            }
-        } else {
-            if (system("ssh $option_ssh_opt $option_remote_host test -d $path") != 0) {
-                system("ssh $option_ssh_opt $option_remote_host mkdir $path");
-            }
+        if (! -d $path) {
+            # this directory does not exist, create it !
+            mkdir($path, 0777) || Die "Failed to create backup directory: $!";
         }
     }
 }
@@ -2065,17 +1925,12 @@
         next unless -d "$source_dir/$database";
 	     next unless check_if_required($database);
         
-        if (!$option_remote_host && !$option_stream) {
+        if (!$option_stream) {
             if (! -e "$backup_dir/$database") {
                 # create database directory for the backup
                 mkdir("$backup_dir/$database", 0777)
                     || Die "Couldn't create directory '$backup_dir/$database': $!";
             }
-        } elsif ($option_remote_host) {
-            if (system("ssh $option_ssh_opt $option_remote_host test -e $backup_dir/$database")
-                    != 0) {
-                system("ssh $option_ssh_opt $option_remote_host mkdir $backup_dir/$database");
-            }
         }
 
         # copy files of this database
@@ -2109,16 +1964,13 @@
 		if (!$prep_mode) {
 		    $rsync_files_hash{"$database/$file"} = 1;
 		}
-            } elsif (!$option_remote_host && !$option_stream) {
+            } elsif (!$option_stream) {
                 $src_name = escape_path("$source_dir/$database/$file");
                 $dst_name = escape_path("$backup_dir/$database");
                 # Copy the file - If we get an error and the file actually exists, die with error msg
                 copy_if_exists("$src_name", "$dst_name")
                   or Die "Failed to copy file '$file': $!";
-            } elsif ($option_remote_host) {
-                # Queue up files for one single scp per database.
-                push(@scp_files, "'$file'");
-            } elsif($option_stream) {
+            } else {
                 my $ret = 0;
                 my $file_name = substr($file, rindex($file, '/') + 1);
                 $file_name=~s/([\$\\\" ])/\\$1/g;
@@ -2138,11 +1990,6 @@
                 }
             }
         }
-        if ($option_remote_host and @scp_files) {
-          my $scp_file_list = join(" ", map { "$source_dir/$database/$_" } @scp_files);
-          system("scp $option_scp_opt $scp_file_list '$option_remote_host:$backup_dir/$database/'")
-              and Die "Failed to execute \"scp $option_scp_opt $scp_file_list '$option_remote_host:$backup_dir/$database/'\": $!";
-        }
     }
     closedir(DIR);
 
@@ -2716,12 +2563,12 @@
              [--password=WORD] [--port=PORT] [--socket=SOCKET]
              [--no-timestamp] [--ibbackup=IBBACKUP-BINARY]
              [--slave-info] [--galera-info] [--stream=tar|xbstream]
-             [--scpopt=OPTIONS-FOR-SCP]  [--sshopt=OPTIONS-FOR-SSH]
              [--defaults-file=MY.CNF] [--defaults-group=GROUP-NAME]
-             [--databases=LIST] [--remote-host=HOSTNAME] [--no-lock] 
+             [--databases=LIST] [--no-lock] 
              [--tmpdir=DIRECTORY] [--tables-file=FILE]
              [--incremental] [--incremental-basedir]
              [--incremental-dir] [--incremental-lsn]
+             [--compact]     
              BACKUP-ROOT-DIR
 
 innobackupex --apply-log [--use-memory=B] [--uncompress]
@@ -2778,6 +2625,10 @@
 
 Prepare a backup in BACKUP-DIR by applying the transaction log file named "xtrabackup_logfile" located in the same directory. Also, create new transaction logs. The InnoDB configuration is read from the file "backup-my.cnf".
 
+=item --compact
+
+Create a compact backup with all secondary index pages omitted. This option is passed directly to xtrabackup. See xtrabackup documentation for details.
+
 =item --compress
 
 This option instructs xtrabackup to compress backup copies of InnoDB
@@ -2875,10 +2726,6 @@
 
 This option is passed directly to xtrabackup's --apply-log-only option. This forces xtrabackup to skip the "rollback" phase and do a "redo" only. This is necessary if the backup will have incremental changes applied to it later. See the xtrabackup documentation for details. 
 
-=item --remote-host=HOSTNAME 
-
-This option is DEPRECATED and will be removed in Percona XtraBackup 2.1. In Percona XtraBackup 2.0 and later, you should use streaming backups instead. This option specifies the remote host on which the backup files will be created, by using an ssh connection.  The option accepts a string argument.
-
 =item --safe-slave-backup
 
 Stop slave SQL thread and wait to start backup until Slave_open_temp_tables in "SHOW STATUS" is zero. If there are no open temporary tables, the backup will take place, otherwise the SQL thread will be started and stopped until there are no open temporary tables. The backup will fail if Slave_open_temp_tables does not become zero after --safe-slave-backup-timeout seconds. The slave SQL thread will be restarted when the backup finishes.
@@ -2917,7 +2764,7 @@
 
 =item --tmpdir=DIRECTORY
 
-This option specifies the location where a temporary file will be stored.  The option accepts a string argument. It should be used when --remote-host or --stream is specified. For these options, the transaction log will first be stored to a temporary file, before streaming or copying to a remote host. This option specifies the location where that temporary file will be stored. If the option is not specifed, the default is to use the value of tmpdir read from the server configuration.
+This option specifies the location where a temporary file will be stored.  The option accepts a string argument. It should be used when --stream is specified. For these options, the transaction log will first be stored to a temporary file, before streaming. This option specifies the location where that temporary file will be stored. If the option is not specifed, the default is to use the value of tmpdir read from the server configuration.
 
 =item --use-memory=B
 
 
=== modified file 'src/Makefile'
--- src/Makefile	2012-02-10 20:05:56 +0000
+++ src/Makefile	2012-10-18 11:58:23 +0000
@@ -12,10 +12,10 @@
 LIBS += -lpthread
 DEFS = -DUNIV_LINUX -DMYSQL_SERVER
 
-CFLAGS += -DXTRABACKUP_VERSION=\"$(XTRABACKUP_VERSION)\" -pedantic -Wall -Wundef -Wshadow  -fdiagnostics-show-option  -fno-strict-aliasing -Wno-strict-aliasing -Wextra -Wformat -Wno-format-nonliteral -Wno-format-security -Wno-long-long  -Wmissing-declarations  -Wno-redundant-decls --std=gnu99
+CFLAGS += -g -DXTRABACKUP_VERSION=\"$(XTRABACKUP_VERSION)\" -pedantic -Wall -fdiagnostics-show-option  -fno-strict-aliasing -Wno-strict-aliasing -Wextra -Wformat -Wno-format-nonliteral -Wno-format-security -Wno-long-long  -Wmissing-declarations  -Wno-redundant-decls --std=gnu99 # -Wundef -Wshadow
 
 ifneq ($(DEBUG),)
-CFLAGS += -DXB_DEBUG=1
+CFLAGS += -DXB_DEBUG=1 -DUNIV_DEBUG
 endif
 
 TARGET=xtrabackup
@@ -23,7 +23,16 @@
 BIN_DIR=$(PREFIX)/bin
 
 COMMON_INC = -I. -I libarchive/libarchive -I quicklz
-XTRABACKUPOBJS = xtrabackup.o stream.o local.o compress.o xbstream_write.o \
+XTRABACKUPOBJS = xtrabackup.o \
+	ds_stream.o \
+	ds_local.o \
+	ds_compress.o \
+	ds_tmpfile.o \
+	datasink.o \
+	write_filt.o \
+	fil_cur.o \
+	innodb_int.o \
+	xbstream_write.o \
 	quicklz/quicklz.o
 XBSTREAMOBJS = xbstream.o xbstream_write.o xbstream_read.o
 
@@ -176,10 +185,10 @@
 xbstream.o xbstream_read.o: %.o: %.c
 	$(CC) $(CFLAGS) $(INC) $(DEFS) -c $< -o $@
 
-xbstream: $(XBSTREAMOBJS) $(MYSQLOBJS) local.o
+xbstream: $(XBSTREAMOBJS) $(MYSQLOBJS) ds_local.o datasink.o
 	$(CC) $(CFLAGS) $^ $(INC) $(MYSQLOBJS) $(LIBS) -o $@
 
-xtrabackup.o: xtrabackup.c xb_regex.h
+xtrabackup.o: xtrabackup.c xb_regex.h write_filt.h fil_cur.h xtrabackup.h
 
 $(TARGET): $(XTRABACKUPOBJS) $(INNODBOBJS) $(MYSQLOBJS) $(LIBARCHIVE_A)
 	$(CC) $(CFLAGS) $(XTRABACKUPOBJS) $(INNODBOBJS) $(MYSQLOBJS) $(LIBS) \
 
=== modified file 'src/common.h'
--- src/common.h	2012-02-10 20:05:56 +0000
+++ src/common.h	2012-10-18 11:58:23 +0000
@@ -24,12 +24,14 @@
 #include <my_global.h>
 #include <mysql_version.h>
 #include <fcntl.h>
+#include <stdarg.h>
+#include "innodb_int.h"
 
 #define xb_a(expr)							\
 	do {								\
 		if (!(expr)) {						\
 			msg("Assertion \"%s\" failed at %s:%lu\n",	\
-			    #expr, __FILE__, (unsigned long) __LINE__); \
+			    #expr, __FILE__, (ulint) __LINE__);		\
 			abort();					\
 		}							\
 	} while (0);
@@ -40,6 +42,8 @@
 #define xb_ad(expr)
 #endif
 
+#define XB_DELTA_INFO_SUFFIX ".meta"
+
 static inline int msg(const char *fmt, ...) ATTRIBUTE_FORMAT(printf, 1, 2);
 static inline int msg(const char *fmt, ...)
 {
@@ -61,13 +65,29 @@
 /* Use POSIX_FADV_NORMAL when available */
 
 #ifdef POSIX_FADV_NORMAL
-#define USE_POSIX_FADVISE
+# define USE_POSIX_FADVISE
+#else
+# define POSIX_FADV_NORMAL
+# define POSIX_FADV_SEQUENTIAL
+# define POSIX_FADV_DONTNEED
+# define posix_fadvise(a,b,c,d) do {} while(0)
 #endif
 
-typedef enum {
-	XB_STREAM_FMT_NONE,
-	XB_STREAM_FMT_TAR,
-	XB_STREAM_FMT_XBSTREAM
-} xb_stream_fmt_t;
+/***********************************************************************
+Computes bit shift for a given value. If the argument is not a power
+of 2, returns 0.*/
+static inline ulint
+get_bit_shift(ulint value)
+{
+    ulint shift;
+
+    if (value == 0)
+	return 0;
+
+    for (shift = 0; !(value & 1UL); shift++) {
+	value >>= 1;
+    }
+    return (value >> 1) ? 0 : shift;
+}
 
 #endif
 
=== added file 'src/datasink.c'
--- src/datasink.c	1970-01-01 00:00:00 +0000
+++ src/datasink.c	2012-10-18 11:58:23 +0000
@@ -0,0 +1,113 @@
+/******************************************************
+Copyright (c) 2011 Percona Inc.
+
+Data sink interface.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+#include <my_base.h>
+#include "common.h"
+#include "datasink.h"
+#include "ds_compress.h"
+#include "ds_stream.h"
+#include "ds_local.h"
+#include "ds_tmpfile.h"
+
+/************************************************************************
+Create a datasink of the specified type */
+ds_ctxt_t *
+ds_create(const char *root, ds_type_t type)
+{
+	datasink_t	*ds;
+	ds_ctxt_t	*ctxt;
+
+	switch (type) {
+	case DS_TYPE_LOCAL:
+		ds = &datasink_local;
+		break;
+	case DS_TYPE_STREAM:
+		ds = &datasink_stream;
+		break;
+	case DS_TYPE_COMPRESS:
+		ds = &datasink_compress;
+		break;
+	case DS_TYPE_TMPFILE:
+		ds = &datasink_tmpfile;
+		break;
+	default:
+		msg("Unknown datasink type: %d\n", type);
+		return NULL;
+	}
+
+	ctxt = ds->init(root);
+	if (ctxt != NULL) {
+		ctxt->datasink = ds;
+	} else {
+		msg("Error: failed to initialize datasink.\n");
+		exit(EXIT_FAILURE);
+	}
+
+	return ctxt;
+}
+
+/************************************************************************
+Open a datasink file */
+ds_file_t *
+ds_open(ds_ctxt_t *ctxt, const char *path, MY_STAT *stat)
+{
+	ds_file_t	*file;
+
+	file = ctxt->datasink->open(ctxt, path, stat);
+	if (file != NULL) {
+		file->datasink = ctxt->datasink;
+	}
+
+	return file;
+}
+
+/************************************************************************
+Write to a datasink file.
+@return 0 on success, 1 on error. */
+int
+ds_write(ds_file_t *file, const void *buf, size_t len)
+{
+	return file->datasink->write(file, buf, len);
+}
+
+/************************************************************************
+Close a datasink file.
+@return 0 on success, 1, on error. */
+int
+ds_close(ds_file_t *file)
+{
+	return file->datasink->close(file);
+}
+
+/************************************************************************
+Destroy a datasink handle */
+void
+ds_destroy(ds_ctxt_t *ctxt)
+{
+	ctxt->datasink->deinit(ctxt);
+}
+
+/************************************************************************
+Set the destination pipe for a datasink (only makes sense for compress and
+tmpfile). */
+void ds_set_pipe(ds_ctxt_t *ctxt, ds_ctxt_t *pipe_ctxt)
+{
+	ctxt->pipe_ctxt = pipe_ctxt;
+}
 
=== modified file 'src/datasink.h'
--- src/datasink.h	2012-02-10 20:05:56 +0000
+++ src/datasink.h	2012-10-18 11:58:23 +0000
@@ -25,24 +25,62 @@
 #include <my_dir.h>
 
 struct datasink_struct;
+typedef struct datasink_struct datasink_t;
 
-typedef struct {
-	struct datasink_struct	*datasink;
-	char 			*root;
-	void			*ptr;
+typedef struct ds_ctxt {
+	datasink_t	*datasink;
+	char 		*root;
+	void		*ptr;
+	struct ds_ctxt	*pipe_ctxt;
 } ds_ctxt_t;
 
 typedef struct {
-	void *ptr;
-	char *path;
+	void		*ptr;
+	char		*path;
+	datasink_t	*datasink;
 } ds_file_t;
 
-typedef struct datasink_struct {
+struct datasink_struct {
 	ds_ctxt_t *(*init)(const char *root);
 	ds_file_t *(*open)(ds_ctxt_t *ctxt, const char *path, MY_STAT *stat);
 	int (*write)(ds_file_t *file, const void *buf, size_t len);
 	int (*close)(ds_file_t *file);
 	void (*deinit)(ds_ctxt_t *ctxt);
-} datasink_t;
+};
+
+/* Supported datasink types */
+typedef enum {
+	DS_TYPE_LOCAL,
+	DS_TYPE_STREAM,
+	DS_TYPE_COMPRESS,
+	DS_TYPE_TMPFILE
+} ds_type_t;
+
+/************************************************************************
+Create a datasink of the specified type */
+ds_ctxt_t *ds_create(const char *root, ds_type_t type);
+
+/************************************************************************
+Open a datasink file */
+ds_file_t *ds_open(ds_ctxt_t *ctxt, const char *path, MY_STAT *stat);
+
+/************************************************************************
+Write to a datasink file.
+@return 0 on success, 1 on error. */
+int ds_write(ds_file_t *file, const void *buf, size_t len);
+
+/************************************************************************
+Close a datasink file.
+@return 0 on success, 1, on error. */
+int ds_close(ds_file_t *file);
+
+/************************************************************************
+Destroy a datasink handle */
+void ds_destroy(ds_ctxt_t *ctxt);
+
+/************************************************************************
+Set the destination pipe for a datasink (only makes sense for compress and
+tmpfile). */
+void ds_set_pipe(ds_ctxt_t *ctxt, ds_ctxt_t *pipe_ctxt);
 
 #endif /* XB_DATASINK_H */
 
=== renamed file 'src/compress.c' => 'src/ds_compress.c'
--- src/compress.c	2012-06-14 10:50:28 +0000
+++ src/ds_compress.c	2012-10-18 11:58:23 +0000
@@ -25,8 +25,8 @@
 #include <zlib.h>
 #include "common.h"
 #include "datasink.h"
-#include "stream.h"
-#include "local.h"
+#include "ds_stream.h"
+#include "ds_local.h"
 
 #define COMPRESS_CHUNK_SIZE (64 * 1024UL)
 #define MY_QLZ_COMPRESS_OVERHEAD 400
@@ -50,20 +50,16 @@
 } comp_thread_ctxt_t;
 
 typedef struct {
-	ds_ctxt_t 		*dest_ctxt;
 	comp_thread_ctxt_t	*threads;
 	uint			nthreads;
 } ds_compress_ctxt_t;
 
 typedef struct {
-	datasink_t 		*dest_ds;
 	ds_file_t		*dest_file;
 	ds_compress_ctxt_t	*comp_ctxt;
 	size_t			bytes_processed;
 } ds_compress_file_t;
 
-extern ibool 	xtrabackup_stream;
-extern uint	xtrabackup_parallel;
 extern ibool	xtrabackup_compress_threads;
 
 static ds_ctxt_t *compress_init(const char *root);
@@ -81,10 +77,8 @@
 	&compress_deinit
 };
 
-static inline int write_uint32_le(datasink_t *sink, ds_file_t *file,
-				  ulong n);
-static inline int write_uint64_le(datasink_t *sink, ds_file_t *file,
-				  ulonglong n);
+static inline int write_uint32_le(ds_file_t *file, ulong n);
+static inline int write_uint64_le(ds_file_t *file, ulonglong n);
 
 static comp_thread_ctxt_t *create_worker_threads(uint n);
 static void destroy_worker_threads(comp_thread_ctxt_t *threads, uint n);
@@ -96,25 +90,12 @@
 {
 	ds_ctxt_t		*ctxt;
 	ds_compress_ctxt_t	*compress_ctxt;
-	datasink_t		*dest_ds;
-	ds_ctxt_t		*dest_ctxt;
 	comp_thread_ctxt_t	*threads;
 
-	/* Decide whether the compressed data will be stored in local files or
-	streamed to an archive */
-	dest_ds = xtrabackup_stream ? &datasink_stream : &datasink_local;
-
-	dest_ctxt = dest_ds->init(root);
-	if (dest_ctxt == NULL) {
-		msg("compress: failed to initialize the target datasink.\n");
-		return NULL;
-	}
-
 	/* Create and initialize the worker threads */
 	threads = create_worker_threads(xtrabackup_compress_threads);
 	if (threads == NULL) {
 		msg("compress: failed to create worker threads.\n");
-		dest_ds->deinit(dest_ctxt);
 		return NULL;
 	}
 
@@ -123,12 +104,11 @@
 				       MYF(MY_FAE));
 
 	compress_ctxt = (ds_compress_ctxt_t *) (ctxt + 1);
-	compress_ctxt->dest_ctxt = dest_ctxt;
 	compress_ctxt->threads = threads;
 	compress_ctxt->nthreads = xtrabackup_compress_threads;
 
-	ctxt->datasink = &datasink_compress;
 	ctxt->ptr = compress_ctxt;
+	ctxt->root = my_strdup(root, MYF(MY_FAE));
 
 	return ctxt;
 }
@@ -138,7 +118,6 @@
 compress_open(ds_ctxt_t *ctxt, const char *path, MY_STAT *mystat)
 {
 	ds_compress_ctxt_t	*comp_ctxt;
-	datasink_t		*dest_ds;
 	ds_ctxt_t		*dest_ctxt;
  	ds_file_t		*dest_file;
 	char			new_name[FN_REFLEN];
@@ -146,21 +125,22 @@
 	ds_file_t		*file;
 	ds_compress_file_t	*comp_file;
 
+	xb_a(ctxt->pipe_ctxt != NULL);
+	dest_ctxt = ctxt->pipe_ctxt;
+
 	comp_ctxt = (ds_compress_ctxt_t *) ctxt->ptr;
-	dest_ctxt = comp_ctxt->dest_ctxt;
-	dest_ds = dest_ctxt->datasink;
 
 	/* Append the .qp extension to the filename */
 	fn_format(new_name, path, "", ".qp", MYF(MY_APPEND_EXT));
 
-	dest_file = dest_ds->open(dest_ctxt, new_name, mystat);
+	dest_file = ds_open(dest_ctxt, new_name, mystat);
 	if (dest_file == NULL) {
 		return NULL;
 	}
 
 	/* Write the qpress archive header */
-	if (dest_ds->write(dest_file, "qpress10", 8) ||
-	    write_uint64_le(dest_ds, dest_file, COMPRESS_CHUNK_SIZE)) {
+	if (ds_write(dest_file, "qpress10", 8) ||
+	    write_uint64_le(dest_file, COMPRESS_CHUNK_SIZE)) {
 		goto err;
 	}
 
@@ -171,10 +151,10 @@
 
 	/* Write the qpress file header */
 	name_len = strlen(new_name);
-	if (dest_ds->write(dest_file, "F", 1) ||
-	    write_uint32_le(dest_ds, dest_file, name_len) ||
+	if (ds_write(dest_file, "F", 1) ||
+	    write_uint32_le(dest_file, name_len) ||
 	    /* we want to write the terminating \0 as well */
-	    dest_ds->write(dest_file, new_name, name_len + 1)) {
+	    ds_write(dest_file, new_name, name_len + 1)) {
 		goto err;
 	}
 
@@ -183,7 +163,6 @@
 				       MYF(MY_FAE));
 	comp_file = (ds_compress_file_t *) (file + 1);
 	comp_file->dest_file = dest_file;
-	comp_file->dest_ds = dest_ds;
 	comp_file->comp_ctxt = comp_ctxt;
 	comp_file->bytes_processed = 0;
 
@@ -193,7 +172,7 @@
 	return file;
 
 err:
-	dest_ds->close(dest_file);
+	ds_close(dest_file);
 	return NULL;
 }
 
@@ -208,12 +187,10 @@
 	uint			nthreads;
 	uint			i;
 	const char		*ptr;
-	datasink_t		*dest_ds;
 	ds_file_t		*dest_file;
 
 	comp_file = (ds_compress_file_t *) file->ptr;
 	comp_ctxt = comp_file->comp_ctxt;
-	dest_ds = comp_file->dest_ds;
 	dest_file = comp_file->dest_file;
 
 	threads = comp_ctxt->threads;
@@ -262,8 +239,8 @@
 
 			ut_a(threads[i].to_len > 0);
 
-			if (dest_ds->write(dest_file, "NEWBNEWB", 8) ||
-			    write_uint64_le(dest_ds, dest_file,
+			if (ds_write(dest_file, "NEWBNEWB", 8) ||
+			    write_uint64_le(dest_file,
 					    comp_file->bytes_processed)) {
 				msg("compress: write to the destination stream "
 				    "failed.\n");
@@ -272,9 +249,8 @@
 
 			comp_file->bytes_processed += threads[i].from_len;
 
-			if (write_uint32_le(dest_ds, dest_file,
-					    threads[i].adler) ||
-			    dest_ds->write(dest_file, threads[i].to,
+			if (write_uint32_le(dest_file, threads[i].adler) ||
+			    ds_write(dest_file, threads[i].to,
 					   threads[i].to_len)) {
 				msg("compress: write to the destination stream "
 				    "failed.\n");
@@ -294,23 +270,21 @@
 compress_close(ds_file_t *file)
 {
 	ds_compress_file_t	*comp_file;
-	datasink_t		*dest_ds;
 	ds_file_t		*dest_file;
 
 	comp_file = (ds_compress_file_t *) file->ptr;
-	dest_ds = comp_file->dest_ds;
 	dest_file = comp_file->dest_file;
 
 	/* Write the qpress file trailer */
-	dest_ds->write(dest_file, "ENDSENDS", 8);
+	ds_write(dest_file, "ENDSENDS", 8);
 
 	/* Supposedly the number of written bytes should be written as a
 	"recovery information" in the file trailer, but in reality qpress
 	always writes 8 zeros here. Let's do the same */
 
-	write_uint64_le(dest_ds, dest_file, 0);
+	write_uint64_le(dest_file, 0);
 
-	dest_ds->close(dest_file);
+	ds_close(dest_file);
 
 	MY_FREE(file);
 
@@ -322,39 +296,37 @@
 compress_deinit(ds_ctxt_t *ctxt)
 {
 	ds_compress_ctxt_t 	*comp_ctxt;
-	ds_ctxt_t		*dest_ctxt;
-	datasink_t		*dest_ds;
+
+	xb_a(ctxt->pipe_ctxt != NULL);
 
 	comp_ctxt = (ds_compress_ctxt_t *) ctxt->ptr;;
 
 	destroy_worker_threads(comp_ctxt->threads, comp_ctxt->nthreads);
 
-	dest_ctxt = comp_ctxt->dest_ctxt;
-	dest_ds = dest_ctxt->datasink;
-
-	dest_ds->deinit(dest_ctxt);
-
+	ds_destroy(ctxt->pipe_ctxt);
+
+	MY_FREE(ctxt->root);
 	MY_FREE(ctxt);
 }
 
 static inline
 int
-write_uint32_le(datasink_t *sink, ds_file_t *file, ulong n)
+write_uint32_le(ds_file_t *file, ulong n)
 {
 	char tmp[4];
 
 	int4store(tmp, n);
-	return sink->write(file, tmp, sizeof(tmp));
+	return ds_write(file, tmp, sizeof(tmp));
 }
 
 static inline
 int
-write_uint64_le(datasink_t *sink, ds_file_t *file, ulonglong n)
+write_uint64_le(ds_file_t *file, ulonglong n)
 {
 	char tmp[8];
 
 	int8store(tmp, n);
-	return sink->write(file, tmp, sizeof(tmp));
+	return ds_write(file, tmp, sizeof(tmp));
 }
 
 static
@@ -489,14 +461,3 @@
 
 	return NULL;
 }
-
-/* Return a target datasink for the specified compress datasink */
-ds_ctxt_t *
-compress_get_dest_ctxt(ds_ctxt_t *ctxt)
-{
-	ds_compress_ctxt_t *comp_ctxt;
-
-	comp_ctxt = (ds_compress_ctxt_t *) ctxt->ptr;
-
-	return comp_ctxt->dest_ctxt;
-}
 
=== renamed file 'src/compress.h' => 'src/ds_compress.h'
--- src/compress.h	2012-06-14 10:50:28 +0000
+++ src/ds_compress.h	2012-10-18 11:58:23 +0000
@@ -18,14 +18,11 @@
 
 *******************************************************/
 
-#ifndef XB_COMPRESS_H
-#define XB_COMPRESS_H
+#ifndef DS_COMPRESS_H
+#define DS_COMPRESS_H
 
 #include "datasink.h"
 
 extern datasink_t datasink_compress;
 
-/* Return a target datasink for the specified compress datasink */
-ds_ctxt_t *compress_get_dest_ctxt(ds_ctxt_t *);
-
 #endif
 
=== renamed file 'src/local.c' => 'src/ds_local.c'
--- src/local.c	2012-02-10 20:05:56 +0000
+++ src/ds_local.c	2012-10-18 11:58:23 +0000
@@ -57,7 +57,6 @@
 
 	ctxt = my_malloc(sizeof(ds_ctxt_t), MYF(MY_FAE));
 
-	ctxt->datasink = &datasink_local;
 	ctxt->root = my_strdup(root, MYF(MY_FAE));
 
 	return ctxt;
@@ -117,9 +116,7 @@
 	File fd = ((ds_local_file_t *) file->ptr)->fd;
 
 	if (!my_write(fd, buf, len, MYF(MY_WME | MY_NABP))) {
-#ifdef USE_POSIX_FADVISE
 		posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED);
-#endif
 		return 0;
 	}
 
 
=== renamed file 'src/local.h' => 'src/ds_local.h'
--- src/local.h	2012-02-10 20:05:56 +0000
+++ src/ds_local.h	2012-10-18 11:58:23 +0000
@@ -18,8 +18,8 @@
 
 *******************************************************/
 
-#ifndef XB_LOCAL_H
-#define XB_LOCAL_H
+#ifndef DS_LOCAL_H
+#define DS_LOCAL_H
 
 #include "datasink.h"
 
 
=== renamed file 'src/stream.c' => 'src/ds_stream.c'
--- src/stream.c	2012-04-22 13:04:24 +0000
+++ src/ds_stream.c	2012-10-18 11:58:23 +0000
@@ -24,6 +24,7 @@
 #include "common.h"
 #include "datasink.h"
 #include "xbstream.h"
+#include "xtrabackup.h"
 
 typedef struct {
 	struct archive	*archive;
@@ -88,7 +89,10 @@
 		}
 
 		if (archive_write_set_compression_none(a) != ARCHIVE_OK ||
-		    archive_write_set_format_pax_restricted(a) != ARCHIVE_OK) {
+		    archive_write_set_format_pax_restricted(a) != ARCHIVE_OK ||
+		    /* disable internal buffering so we don't have to flush the
+		    output in xtrabackup */
+		    archive_write_set_bytes_per_block(a, 0) != ARCHIVE_OK) {
 			msg("failed to set libarchive stream options: %s\n",
 			    archive_error_string(a));
 			archive_write_finish(a);
@@ -103,7 +107,6 @@
 	}
 
 
-	ctxt->datasink = &datasink_stream;
 	ctxt->ptr = stream_ctxt;
 
 	return ctxt;
 
=== renamed file 'src/stream.h' => 'src/ds_stream.h'
--- src/stream.h	2012-02-10 20:05:56 +0000
+++ src/ds_stream.h	2012-10-18 11:58:23 +0000
@@ -18,8 +18,8 @@
 
 *******************************************************/
 
-#ifndef XTRABACKUP_STREAM_H
-#define XTRABACKUP_STREAM_H
+#ifndef DS_STREAM_H
+#define DS_STREAM_H
 
 #include "datasink.h"
 
 
=== added file 'src/ds_tmpfile.c'
--- src/ds_tmpfile.c	1970-01-01 00:00:00 +0000
+++ src/ds_tmpfile.c	2012-10-18 11:58:23 +0000
@@ -0,0 +1,243 @@
+/******************************************************
+Copyright (c) 2012 Percona Inc.
+
+tmpfile datasink for XtraBackup.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+/* Do all writes to temporary files first, then pipe them to the specified
+datasink in a serialized way in deinit(). */
+
+#include <my_base.h>
+#include "common.h"
+#include "datasink.h"
+
+typedef struct {
+	pthread_mutex_t	 mutex;
+	LIST		*file_list;
+} ds_tmpfile_ctxt_t;
+
+typedef struct {
+	LIST		 list;
+	File		 fd;
+	char		*orig_path;
+	MY_STAT		 mystat;
+	ds_file_t	*file;
+} ds_tmp_file_t;
+
+static ds_ctxt_t *tmpfile_init(const char *root);
+static ds_file_t *tmpfile_open(ds_ctxt_t *ctxt, const char *path,
+			       MY_STAT *mystat);
+static int tmpfile_write(ds_file_t *file, const void *buf, size_t len);
+static int tmpfile_close(ds_file_t *file);
+static void tmpfile_deinit(ds_ctxt_t *ctxt);
+
+datasink_t datasink_tmpfile = {
+	&tmpfile_init,
+	&tmpfile_open,
+	&tmpfile_write,
+	&tmpfile_close,
+	&tmpfile_deinit
+};
+
+static ds_ctxt_t *
+tmpfile_init(const char *root)
+{
+	ds_ctxt_t		*ctxt;
+	ds_tmpfile_ctxt_t	*tmpfile_ctxt;
+
+	ctxt = my_malloc(sizeof(ds_ctxt_t) + sizeof(ds_tmpfile_ctxt_t),
+			 MYF(MY_FAE));
+	tmpfile_ctxt = (ds_tmpfile_ctxt_t *) (ctxt + 1);
+	tmpfile_ctxt->file_list = NULL;
+	if (pthread_mutex_init(&tmpfile_ctxt->mutex, NULL)) {
+
+		MY_FREE(ctxt);
+		return NULL;
+	}
+
+	ctxt->ptr = tmpfile_ctxt;
+	ctxt->root = my_strdup(root, MYF(MY_FAE));
+
+	return ctxt;
+}
+
+static ds_file_t *
+tmpfile_open(ds_ctxt_t *ctxt, const char *path,
+			       MY_STAT *mystat)
+{
+	ds_tmpfile_ctxt_t	*tmpfile_ctxt;
+	char			 tmp_path[FN_REFLEN];
+	ds_tmp_file_t		*tmp_file;
+	ds_file_t		*file;
+	size_t			 path_len;
+	File			 fd;
+
+	/* Create a temporary file in tmpdir. The file will be automatically
+	removed on close. Code copied from mysql_tmpfile(). */
+	fd = create_temp_file(tmp_path, my_tmpdir(&mysql_tmpdir_list),
+			      "xbtemp",
+#ifdef __WIN__
+			      O_BINARY | O_TRUNC | O_SEQUENTIAL |
+			      O_TEMPORARY | O_SHORT_LIVED |
+#endif /* __WIN__ */
+			      O_CREAT | O_EXCL | O_RDWR,
+			      MYF(MY_WME));
+
+#ifndef __WIN__
+	if (fd >= 0) {
+		/* On Windows, open files cannot be removed, but files can be
+		created with the O_TEMPORARY flag to the same effect
+		("delete on close"). */
+		unlink(path);
+	}
+#endif /* !__WIN__ */
+
+	if (fd < 0) {
+		return NULL;
+	}
+
+	path_len = strlen(path) + 1; /* terminating '\0' */
+
+	file = (ds_file_t *) my_malloc(sizeof(ds_file_t) +
+				       sizeof(ds_tmp_file_t) + path_len,
+				       MYF(MY_FAE));
+
+	tmp_file = (ds_tmp_file_t *) (file + 1);
+	tmp_file->file = file;
+	memcpy(&tmp_file->mystat, mystat, sizeof(MY_STAT));
+	/* Save a copy of 'path', since it may not be accessible later */
+	tmp_file->orig_path = (char *) tmp_file + sizeof(ds_tmp_file_t);
+
+	tmp_file->fd = fd;
+	memcpy(tmp_file->orig_path, path, path_len);
+
+	/* Store the real temporary file name in file->path */
+	file->path = my_strdup(tmp_path, MYF(MY_FAE));
+	file->ptr = tmp_file;
+
+	/* Store the file object in the list to be piped later */
+	tmpfile_ctxt = (ds_tmpfile_ctxt_t *) ctxt->ptr;
+	tmp_file->list.data = tmp_file;
+
+	pthread_mutex_lock(&tmpfile_ctxt->mutex);
+	tmpfile_ctxt->file_list = list_add(tmpfile_ctxt->file_list,
+					   &tmp_file->list);
+	pthread_mutex_unlock(&tmpfile_ctxt->mutex);
+
+	return file;
+}
+
+static int
+tmpfile_write(ds_file_t *file, const void *buf, size_t len)
+{
+	File fd = ((ds_tmp_file_t *) file->ptr)->fd;
+
+	if (!my_write(fd, buf, len, MYF(MY_WME | MY_NABP))) {
+		posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED);
+		return 0;
+	}
+
+	return 1;
+}
+
+static int
+tmpfile_close(ds_file_t *file)
+{
+	/* Do nothing -- we will close (and thus remove) the file after piping
+	it to the destination datasink in tmpfile_deinit(). */
+
+	MY_FREE(file->path);
+
+	return 0;
+}
+
+static void
+tmpfile_deinit(ds_ctxt_t *ctxt)
+{
+	LIST			*list;
+	ds_tmpfile_ctxt_t	*tmpfile_ctxt;
+	MY_STAT			 mystat;
+	ds_tmp_file_t 		*tmp_file;
+	ds_file_t		*dst_file;
+	ds_ctxt_t		*pipe_ctxt;
+	void			*buf = NULL;
+	const size_t		 buf_size = 1024 * 1024;
+	size_t			 bytes;
+
+	pipe_ctxt = ctxt->pipe_ctxt;
+	xb_a(pipe_ctxt != NULL);
+
+	buf = my_malloc(buf_size, MYF(MY_FAE));
+	
+	tmpfile_ctxt = (ds_tmpfile_ctxt_t *) ctxt->ptr;
+	list = tmpfile_ctxt->file_list;
+
+	/* Walk the files in the order they have been added */
+	list = list_reverse(list);
+	while (list != NULL) {
+		tmp_file = list->data;
+		/* Stat the file to replace size and mtime on the original
+		* mystat struct */
+		if (my_fstat(tmp_file->fd, &mystat, MYF(0))) {
+			msg("error: my_fstat() failed.\n");
+			exit(EXIT_FAILURE);
+		}
+		tmp_file->mystat.st_size = mystat.st_size;
+		tmp_file->mystat.st_mtime = mystat.st_mtime;
+
+		dst_file = ds_open(pipe_ctxt, tmp_file->orig_path,
+				   &tmp_file->mystat);
+		if (dst_file == NULL) {
+			msg("error: could not stream a temporary file to "
+			    "'%s'\n", tmp_file->orig_path);
+			exit(EXIT_FAILURE);
+		}
+
+		/* copy to the destination datasink */
+		posix_fadvise(tmp_file->fd, 0, 0, POSIX_FADV_SEQUENTIAL);
+		if (my_seek(tmp_file->fd, 0, SEEK_SET, MYF(0)) ==
+		    MY_FILEPOS_ERROR) {
+			msg("error: my_seek() failed for '%s', errno = %d.\n",
+			    tmp_file->file->path, my_errno);
+			exit(EXIT_FAILURE);
+		}
+		while ((bytes = my_read(tmp_file->fd, buf, buf_size,
+					MYF(MY_WME))) > 0) {
+			posix_fadvise(tmp_file->fd, 0, 0, POSIX_FADV_DONTNEED);
+			if (ds_write(dst_file, buf, bytes)) {
+				msg("error: cannot write to stream for '%s'.\n",
+				    tmp_file->orig_path);
+				exit(EXIT_FAILURE);
+			}
+		}
+		if (bytes == (size_t) -1) {
+			exit(EXIT_FAILURE);
+		}
+
+		my_close(tmp_file->fd, MYF(MY_WME));
+		ds_close(dst_file);
+
+		list = list_rest(list);
+		MY_FREE(tmp_file->file);
+	}
+
+	pthread_mutex_destroy(&tmpfile_ctxt->mutex);
+
+	MY_FREE(buf);
+	MY_FREE(ctxt->root);
+	MY_FREE(ctxt);
+}
 
=== added file 'src/ds_tmpfile.h'
--- src/ds_tmpfile.h	1970-01-01 00:00:00 +0000
+++ src/ds_tmpfile.h	2012-10-18 11:58:23 +0000
@@ -0,0 +1,28 @@
+/******************************************************
+Copyright (c) 2012 Percona Inc.
+
+tmpfile datasink for XtraBackup.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+#ifndef DS_TMPFILE_H
+#define DS_TMPFILE_H
+
+#include "datasink.h"
+
+extern datasink_t datasink_tmpfile;
+
+#endif
 
=== added file 'src/fil_cur.c'
--- src/fil_cur.c	1970-01-01 00:00:00 +0000
+++ src/fil_cur.c	2012-10-18 11:58:23 +0000
@@ -0,0 +1,270 @@
+/******************************************************
+XtraBackup: hot backup tool for InnoDB
+(c) 2009-2012 Percona Inc.
+Originally Created 3/3/2009 Yasufumi Kinoshita
+Written by Alexey Kopytov, Aleksandr Kuzminsky, Stewart Smith, Vadim Tkachenko,
+Yasufumi Kinoshita, Ignacio Nin and Baron Schwartz.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+/* Source file cursor implementation */
+
+#include <my_base.h>
+#include "innodb_int.h"
+#include <buf0buf.h>
+#include <fsp0types.h>
+#include "fil_cur.h"
+#include "common.h"
+#include "xtrabackup.h"
+
+/* Size of read buffer in pages */
+#define XB_FIL_CUR_PAGES 64
+
+/************************************************************************
+Open a source file cursor.
+
+@return XB_FIL_CUR_SUCCESS on success, XB_FIL_CUR_SKIP if the source file must
+be skipped and XB_FIL_CUR_ERROR on error. */
+xb_fil_cur_result_t
+xb_fil_cur_open(
+/*============*/
+	xb_fil_cur_t*	cursor,		/*!< out: source file cursor */
+	fil_node_t*	node,		/*!< in: source tablespace node */
+	uint		thread_n)	/*!< thread number for diagnostics */
+{
+	ulint	page_size;
+	ulint	page_size_shift;
+	ulint	zip_size;
+
+	/* Initialize these first so xb_fil_cur_close() handles them correctly
+	in case of error */
+	cursor->orig_buf = NULL;
+	cursor->file = XB_FILE_UNDEFINED;
+
+	cursor->space_id = node->space->id;
+	cursor->is_system = trx_sys_sys_space(node->space->id);
+
+	/* Make the file path relative to the backup root,
+	i.e. "ibdata1" for system tablespace or database/table.ibd for
+	per-table spaces. */
+	if (cursor->is_system) {
+		char *next, *p;
+
+		p = node->name;
+		while ((next = strstr(p, SRV_PATH_SEPARATOR_STR)) != NULL) {
+			p = next + 1;
+		}
+		strncpy(cursor->path, p, sizeof(cursor->path));
+	} else {
+		/* file per table style "./database/table.ibd" */
+		strncpy(cursor->path, node->name, sizeof(cursor->path));
+	}
+
+	/* Open the file */
+
+	if (my_stat(node->name, &cursor->statinfo, MYF(MY_WME)) == NULL) {
+		msg("[%02u] xtrabackup: Warning: cannot stat %s\n",
+		    thread_n, node->name);
+		return(XB_FIL_CUR_SKIP);
+	}
+
+	if (!node->open) {
+		ibool success;
+
+		cursor->file =
+			xb_file_create_no_error_handling(node->name,
+							 OS_FILE_OPEN,
+							 OS_FILE_READ_ONLY,
+							 &success);
+		if (!success) {
+			/* The following call prints an error message */
+			os_file_get_last_error(TRUE);
+
+			msg("[%02u] xtrabackup: Warning: cannot open %s\n"
+			    "[%02u] xtrabackup: Warning: We assume the "
+			    "table was dropped or renamed during "
+			    "xtrabackup execution and ignore the file.\n",
+			    thread_n, node->name, thread_n);
+			return(XB_FIL_CUR_SKIP);
+		}
+
+		xb_file_set_nocache(cursor->file, node->name, "OPEN");
+	} else {
+		cursor->file = node->handle;
+	}
+	posix_fadvise(cursor->file, 0, 0, POSIX_FADV_SEQUENTIAL);
+
+	/* Determine the page size */
+#ifndef INNODB_VERSION_SHORT
+	zip_size = UNIV_PAGE_SIZE;
+	page_size = UNIV_PAGE_SIZE;
+	page_size_shift = UNIV_PAGE_SIZE_SHIFT;
+#else
+	zip_size = xb_get_zip_size(cursor->file);
+	if (zip_size == ULINT_UNDEFINED) {
+		os_file_close(cursor->file);
+		return(XB_FIL_CUR_SKIP);
+	} else if (zip_size) {
+		page_size = zip_size;
+		page_size_shift = get_bit_shift(page_size);
+		msg("[%02u] %s is compressed with page size = "
+		    "%lu bytes\n", thread_n, node->name, page_size);
+		if (page_size_shift < 10 || page_size_shift > 14) {
+			msg("[%02u] xtrabackup: Error: Invalid "
+			    "page size: %lu.\n", thread_n, page_size);
+			ut_error;
+		}
+	} else {
+		page_size = UNIV_PAGE_SIZE;
+		page_size_shift = UNIV_PAGE_SIZE_SHIFT;
+	}
+#endif
+	cursor->page_size = page_size;
+	cursor->page_size_shift = page_size_shift;
+	cursor->zip_size = zip_size;
+
+	/* Allocate read buffer */
+	cursor->buf_size = XB_FIL_CUR_PAGES * page_size;
+	cursor->orig_buf = ut_malloc(cursor->buf_size + UNIV_PAGE_SIZE);
+	cursor->buf = ut_align(cursor->orig_buf, UNIV_PAGE_SIZE);
+
+	cursor->offset = 0;
+	cursor->buf_read = 0;
+	cursor->buf_npages = 0;
+	cursor->buf_offset = 0;
+	cursor->buf_page_no = 0;
+	cursor->thread_n = thread_n;
+
+	return(XB_FIL_CUR_SUCCESS);
+}
+
+/************************************************************************
+Reads and verifies the next block of pages from the source
+file. Positions the cursor after the last read non-corrupted page.
+
+@return XB_FIL_CUR_SUCCESS if some have been read successfully, XB_FIL_CUR_EOF
+if there are no more pages to read and XB_FIL_CUR_ERROR on error. */
+xb_fil_cur_result_t
+xb_fil_cur_read(
+/*============*/
+	xb_fil_cur_t*	cursor)	/*!< in/out: source file cursor */
+{
+	ibool			success;
+	ulint			page_size;
+	ulint			offset_high;
+	ulint			offset_low;
+	byte*			page;
+	ulint			i;
+	ulint			npages;
+	ulint			retry_count;
+	xb_fil_cur_result_t	ret;
+	IB_INT64		to_read;
+
+	page_size = cursor->page_size;
+
+	offset_high = (ulint) (cursor->offset >> 32);
+	offset_low = (ulint) (cursor->offset & 0xFFFFFFFFUL);
+
+	to_read = (IB_INT64) cursor->statinfo.st_size - cursor->offset;
+
+	if (to_read == 0LL) {
+		return(XB_FIL_CUR_EOF);
+	}
+
+	if (to_read > (IB_INT64) cursor->buf_size) {
+		to_read = (IB_INT64) cursor->buf_size;
+	}
+	ut_a(to_read > 0 && to_read <= 0xFFFFFFFFLL);
+	ut_a(to_read % page_size == 0);
+
+	npages = (ulint) (to_read >> cursor->page_size_shift);
+
+	retry_count = 10;
+	ret = XB_FIL_CUR_SUCCESS;
+
+read_retry:
+	xtrabackup_io_throttling();
+
+	cursor->buf_read = 0;
+	cursor->buf_npages = 0;
+	cursor->buf_offset = cursor->offset;
+	cursor->buf_page_no = (ulint) (cursor->offset >>
+				       cursor->page_size_shift);
+
+	success = os_file_read(cursor->file, cursor->buf,
+			       offset_low, offset_high, to_read);
+	if (!success) {
+		return(XB_FIL_CUR_ERROR);
+	}
+
+	/* check pages for corruption and re-read if necessary. i.e. in case of
+	partially written pages */
+	for (page = cursor->buf, i = 0; i < npages; page += page_size, i++) {
+		if (xb_buf_page_is_corrupted(page, cursor->zip_size))
+		{
+			ulint page_no = cursor->buf_page_no + i;
+
+			if (cursor->is_system &&
+			    page_no >= FSP_EXTENT_SIZE &&
+			    page_no < FSP_EXTENT_SIZE * 3) {
+				/* skip doublewrite buffer pages */
+				ut_a(page_size == UNIV_PAGE_SIZE);
+				msg("[%02lu] xtrabackup: "
+				    "Page %lu is a doublewrite buffer page, "
+				    "skipping.\n", cursor->thread_n, page_no);
+			} else {
+				retry_count--;
+				if (retry_count == 0) {
+					msg("[%02lu] xtrabackup: "
+					    "Error: failed to read page after "
+					    "10 retries. File %s seems to be "
+					    "corrupted.\n", cursor->thread_n,
+					    cursor->path);
+					ret = XB_FIL_CUR_ERROR;
+					break;
+				}
+				msg("[%02lu] xtrabackup: "
+				    "Database page corruption detected at page "
+				    "%lu, retrying...\n", cursor->thread_n,
+				    page_no);
+				goto read_retry;
+			}
+		}
+		cursor->buf_read += page_size;
+		cursor->buf_npages++;
+	}
+
+	posix_fadvise(cursor->file, 0, 0, POSIX_FADV_DONTNEED);
+
+	cursor->offset += page_size * i;
+
+	return(ret);
+}
+
+/************************************************************************
+Close the source file cursor opened with xb_fil_cur_open(). */
+void
+xb_fil_cur_close(
+/*=============*/
+	xb_fil_cur_t *cursor)	/*!< in/out: source file cursor */
+{
+	if (cursor->orig_buf != NULL) {
+		ut_free(cursor->orig_buf);
+	}
+	if (cursor->file != XB_FILE_UNDEFINED) {
+		os_file_close(cursor->file);
+	}
+}
 
=== added file 'src/fil_cur.h'
--- src/fil_cur.h	1970-01-01 00:00:00 +0000
+++ src/fil_cur.h	2012-10-18 11:58:23 +0000
@@ -0,0 +1,96 @@
+/******************************************************
+XtraBackup: hot backup tool for InnoDB
+(c) 2009-2012 Percona Inc.
+Originally Created 3/3/2009 Yasufumi Kinoshita
+Written by Alexey Kopytov, Aleksandr Kuzminsky, Stewart Smith, Vadim Tkachenko,
+Yasufumi Kinoshita, Ignacio Nin and Baron Schwartz.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+/* Source file cursor interface */
+
+#ifndef FIL_CUR_H
+#define FIL_CUR_H
+
+#include <my_dir.h>
+#include "innodb_int.h"
+
+typedef struct {
+	os_file_t	file;		/*!< source file handle */
+	char		path[FN_REFLEN];/*!< normalized file path */
+	MY_STAT		statinfo;	/*!< information about the file */
+	ulint		zip_size;	/*!< compressed page size in bytes or 0
+					for uncompressed pages */
+	ulint		page_size;	/*!< = zip_size for compressed pages or
+					UNIV_PAGE_SIZE for uncompressed ones */
+	ulint		page_size_shift;/*!< bit shift corresponding to
+					page_size */
+	IB_INT64	offset;		/*!< current file offset in bytes */
+	my_bool		is_system;	/*!< TRUE for system tablespace, FALSE
+					otherwise */
+	byte*		orig_buf;	/*!< read buffer */
+	byte*		buf;		/*!< aligned pointer for orig_buf */
+	ulint		buf_size;	/*!< buffer size in bytes */
+	ulint		buf_read;	/*!< number of read bytes in buffer
+					after the last cursor read */
+	ulint		buf_npages;	/*!< number of pages in buffer after the
+					last cursor read */
+	IB_INT64	buf_offset;	/*!< file offset of the first page in
+					buffer */
+	ulint		buf_page_no;	/*!< number of the first page in
+					buffer */
+	ulint		thread_n;	/*!< thread number for diagnostics */
+	ulint		space_id;	/*!< ID of tablespace */
+} xb_fil_cur_t;
+
+typedef enum {
+	XB_FIL_CUR_SUCCESS,
+	XB_FIL_CUR_SKIP,
+	XB_FIL_CUR_ERROR,
+	XB_FIL_CUR_EOF
+} xb_fil_cur_result_t;
+
+/************************************************************************
+Open a source file cursor.
+
+@return XB_FIL_CUR_SUCCESS on success, XB_FIL_CUR_SKIP if the source file must
+be skipped and XB_FIL_CUR_ERROR on error. */
+xb_fil_cur_result_t
+xb_fil_cur_open(
+/*============*/
+	xb_fil_cur_t*	cursor,		/*!< out: source file cursor */
+	fil_node_t*	node,		/*!< in: source tablespace node */
+	uint		thread_n);	/*!< thread number for diagnostics */
+
+/************************************************************************
+Reads and verifies the next block of pages from the source
+file. Positions the cursor after the last read non-corrupted page.
+
+@return XB_FIL_CUR_SUCCESS if some have been read successfully, XB_FIL_CUR_EOF
+if there are no more pages to read and XB_FIL_CUR_ERROR on error. */
+xb_fil_cur_result_t
+xb_fil_cur_read(
+/*============*/
+	xb_fil_cur_t*	cursor);	/*!< in/out: source file cursor */
+
+/************************************************************************
+Close the source file cursor opened with xb_fil_cur_open(). */
+void
+xb_fil_cur_close(
+/*=============*/
+	xb_fil_cur_t *cursor);	/*!< in/out: source file cursor */
+
+#endif
 
=== added file 'src/innodb_int.c'
--- src/innodb_int.c	1970-01-01 00:00:00 +0000
+++ src/innodb_int.c	2012-10-18 11:58:23 +0000
@@ -0,0 +1,983 @@
+/******************************************************
+XtraBackup: hot backup tool for InnoDB
+(c) 2009-2012 Percona Inc.
+Originally Created 3/3/2009 Yasufumi Kinoshita
+Written by Alexey Kopytov, Aleksandr Kuzminsky, Stewart Smith, Vadim Tkachenko,
+Yasufumi Kinoshita, Ignacio Nin and Baron Schwartz.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+/* InnoDB portability helpers and interface to internal functions */
+
+#include <my_base.h>
+#include <mysql_com.h>
+#include "innodb_int.h"
+#include <srv0srv.h>
+#include <ha_prototypes.h>
+#include <trx0trx.h>
+#include "common.h"
+
+extern long innobase_lock_wait_timeout;
+
+char *opt_mysql_tmpdir = NULL;
+MY_TMPDIR mysql_tmpdir_list;
+
+
+/****************************************************************//**
+A simple function to open or create a file.
+@return own: handle to the file, not defined if error, error number
+can be retrieved with os_file_get_last_error */
+os_file_t
+xb_file_create_no_error_handling(
+/*=============================*/
+	const char*	name,	/*!< in: name of the file or path as a
+				null-terminated string */
+	ulint		create_mode,/*!< in: OS_FILE_OPEN if an existing file
+				is opened (if does not exist, error), or
+				OS_FILE_CREATE if a new file is created
+				(if exists, error) */
+	ulint		access_type,/*!< in: OS_FILE_READ_ONLY,
+				OS_FILE_READ_WRITE, or
+				OS_FILE_READ_ALLOW_DELETE; the last option is
+				used by a backup program reading the file */
+	ibool*		success)/*!< out: TRUE if succeed, FALSE if error */
+{
+#if MYSQL_VERSION_ID > 50500
+	return os_file_create_simple_no_error_handling(
+		0, /* innodb_file_data_key */
+		name, create_mode, access_type, success);
+#else
+	return os_file_create_simple_no_error_handling(
+		name, create_mode, access_type, success);
+#endif
+}
+
+/****************************************************************//**
+Opens an existing file or creates a new.
+@return own: handle to the file, not defined if error, error number
+can be retrieved with os_file_get_last_error */
+os_file_t
+xb_file_create(
+/*===========*/
+	const char*	name,	/*!< in: name of the file or path as a
+				null-terminated string */
+	ulint		create_mode,/*!< in: OS_FILE_OPEN if an existing file
+				is opened (if does not exist, error), or
+				OS_FILE_CREATE if a new file is created
+				(if exists, error),
+				OS_FILE_OVERWRITE if a new file is created
+				or an old overwritten;
+				OS_FILE_OPEN_RAW, if a raw device or disk
+				partition should be opened */
+	ulint		purpose,/*!< in: OS_FILE_AIO, if asynchronous,
+				non-buffered i/o is desired,
+				OS_FILE_NORMAL, if any normal file;
+				NOTE that it also depends on type, os_aio_..
+				and srv_.. variables whether we really use
+				async i/o or unbuffered i/o: look in the
+				function source code for the exact rules */
+	ulint		type,	/*!< in: OS_DATA_FILE or OS_LOG_FILE */
+	ibool*		success)/*!< out: TRUE if succeed, FALSE if error */
+{
+#if MYSQL_VERSION_ID > 50500
+	return os_file_create(0 /* innodb_file_data_key */,
+			      name, create_mode, purpose, type, success);
+#else
+	return os_file_create(name, create_mode, purpose, type, success);
+#endif
+}
+
+/***********************************************************************//**
+Renames a file (can also move it to another directory). It is safest that the
+file is closed before calling this function.
+@return	TRUE if success */
+ibool
+xb_file_rename(
+/*===========*/
+	const char*	oldpath,/*!< in: old file path as a null-terminated
+				string */
+	const char*	newpath)/*!< in: new file path */
+{
+#if MYSQL_VERSION_ID > 50500
+	return os_file_rename(
+		0 /* innodb_file_data_key */, oldpath, newpath);
+#else
+	return os_file_rename(oldpath, newpath);
+#endif
+}
+
+void
+xb_file_set_nocache(
+/*================*/
+	os_file_t	fd,		/* in: file descriptor to alter */
+	const char*	file_name,	/* in: used in the diagnostic message */
+	const char*	operation_name) /* in: used in the diagnostic message,
+					we call os_file_set_nocache()
+					immediately after opening or creating
+					a file, so this is either "open" or
+					"create" */
+{
+#ifndef __WIN__
+	if (srv_unix_file_flush_method == SRV_UNIX_O_DIRECT) {
+		os_file_set_nocache(fd, file_name, operation_name);
+	}
+#endif
+}
+
+/***********************************************************************//**
+Compatibility wrapper around os_file_flush().
+@return	TRUE if success */
+ibool
+xb_file_flush(
+/*==========*/
+	os_file_t	file)	/*!< in, own: handle to a file */
+{
+#ifdef XTRADB_BASED
+	return os_file_flush(file, TRUE);
+#else
+	return os_file_flush(file);
+#endif
+}
+/*******************************************************************//**
+Returns the table space by a given id, NULL if not found. */
+fil_space_t*
+xb_space_get_by_id(
+/*================*/
+	ulint	id)	/*!< in: space id */
+{
+	fil_space_t*	space;
+
+	ut_ad(mutex_own(&fil_system->mutex));
+
+#ifdef INNODB_VERSION_SHORT
+	HASH_SEARCH(hash, fil_system->spaces, id,
+		    fil_space_t*, space,
+		    ut_ad(space->magic_n == FIL_SPACE_MAGIC_N),
+		    space->id == id);
+#else
+	HASH_SEARCH(hash, fil_system->spaces, id, space, space->id == id);
+#endif
+
+	return(space);
+}
+
+/*******************************************************************//**
+Returns the table space by a given name, NULL if not found. */
+fil_space_t*
+xb_space_get_by_name(
+/*==================*/
+	const char*	name)	/*!< in: space name */
+{
+	fil_space_t*	space;
+	ulint		fold;
+
+	ut_ad(mutex_own(&fil_system->mutex));
+
+#ifdef INNODB_VERSION_SHORT
+	fold = ut_fold_string(name);
+	HASH_SEARCH(name_hash, fil_system->name_hash, fold,
+		    fil_space_t*, space,
+		    ut_ad(space->magic_n == FIL_SPACE_MAGIC_N),
+		    !strcmp(name, space->name));
+#else
+	HASH_SEARCH(name_hash, fil_system->name_hash, ut_fold_string(name),
+		    space, 0 == strcmp(name, space->name));
+#endif
+
+	return(space);
+}
+
+/****************************************************************//**
+Create a new tablespace on disk and return the handle to its opened
+file. Code adopted from fil_create_new_single_table_tablespace with
+the main difference that only disk file is created without updating
+the InnoDB in-memory dictionary data structures.
+
+@return TRUE on success, FALSE on error.  */
+ibool
+xb_space_create_file(
+/*==================*/
+	const char*	path,		/*!<in: path to tablespace */
+	ulint		space_id,	/*!<in: space id */
+	ulint		flags __attribute__((unused)),/*!<in: tablespace
+					flags */
+	os_file_t*	file)		/*!<out: file handle */
+{
+	ibool		ret;
+	byte*		buf;
+	byte*		page;
+
+	*file = xb_file_create_no_error_handling(path, OS_FILE_CREATE,
+						 OS_FILE_READ_WRITE, &ret);
+	if (!ret) {
+		msg("xtrabackup: cannot create file %s\n", path);
+		return ret;
+	}
+
+	ret = os_file_set_size(path, *file,
+			       FIL_IBD_FILE_INITIAL_SIZE * UNIV_PAGE_SIZE, 0);
+	if (!ret) {
+		msg("xtrabackup: cannot set size for file %s\n", path);
+		os_file_close(*file);
+		os_file_delete(path);
+		return ret;
+	}
+
+	buf = ut_malloc(3 * UNIV_PAGE_SIZE);
+	/* Align the memory for file i/o if we might have O_DIRECT set */
+	page = ut_align(buf, UNIV_PAGE_SIZE);
+
+	memset(page, '\0', UNIV_PAGE_SIZE);
+
+#ifdef INNODB_VERSION_SHORT
+	fsp_header_init_fields(page, space_id, flags);
+	mach_write_to_4(page + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID, space_id);
+
+	if (!(flags & DICT_TF_ZSSIZE_MASK)) {
+		buf_flush_init_for_writing(page, NULL, 0);
+
+		ret = os_file_write(path, *file, page, 0, 0, UNIV_PAGE_SIZE);
+	}
+	else {
+		page_zip_des_t	page_zip;
+		ulint		zip_size;
+
+		zip_size = (PAGE_ZIP_MIN_SIZE >> 1)
+			<< ((flags & DICT_TF_ZSSIZE_MASK)
+			    >> DICT_TF_ZSSIZE_SHIFT);
+		page_zip_set_size(&page_zip, zip_size);
+		page_zip.data = page + UNIV_PAGE_SIZE;
+		fprintf(stderr, "zip_size = %lu\n", zip_size);
+
+#ifdef UNIV_DEBUG
+		page_zip.m_start =
+#endif /* UNIV_DEBUG */
+			page_zip.m_end = page_zip.m_nonempty =
+			page_zip.n_blobs = 0;
+
+		buf_flush_init_for_writing(page, &page_zip, 0);
+
+		ret = os_file_write(path, *file, page_zip.data, 0, 0,
+				    zip_size);
+	}
+#else
+	fsp_header_write_space_id(page, space_id);
+
+	buf_flush_init_for_writing(page, ut_dulint_zero, space_id, 0);
+
+	ret = os_file_write(path, *file, page, 0, 0, UNIV_PAGE_SIZE);
+#endif
+
+	ut_free(buf);
+
+	if (!ret) {
+		msg("xtrabackup: could not write the first page to %s\n",
+		    path);
+		os_file_close(*file);
+		os_file_delete(path);
+		return ret;
+	}
+
+	return TRUE;
+}
+
+
+#ifndef INNODB_VERSION_SHORT
+
+/********************************************************************//**
+Extract the compressed page size from table flags.
+@return	compressed page size, or 0 if not compressed */
+ulint
+dict_table_flags_to_zip_size(
+/*=========================*/
+	ulint	flags)	/*!< in: flags */
+{
+	ulint	zip_size = flags & DICT_TF_ZSSIZE_MASK;
+
+	if (UNIV_UNLIKELY(zip_size)) {
+		zip_size = ((PAGE_ZIP_MIN_SIZE >> 1)
+			    << (zip_size >> DICT_TF_ZSSIZE_SHIFT));
+
+		ut_ad(zip_size <= UNIV_PAGE_SIZE);
+	}
+
+	return(zip_size);
+}
+
+
+/*******************************************************************//**
+Free all spaces in space_list. */
+void
+fil_free_all_spaces(void)
+/*=====================*/
+{
+	fil_space_t*	space;
+
+	mutex_enter(&fil_system->mutex);
+
+	space = UT_LIST_GET_FIRST(fil_system->space_list);
+
+	while (space != NULL) {
+		fil_node_t*	node;
+		fil_space_t*	prev_space = space;
+
+		space = UT_LIST_GET_NEXT(space_list, space);
+
+		fil_space_free(prev_space->id, FALSE);
+	}
+
+	mutex_exit(&fil_system->mutex);
+}
+
+#endif
+
+void
+innobase_invalidate_query_cache(
+	trx_t*	trx,
+#ifndef INNODB_VERSION_SHORT
+	char*	full_name,
+#else
+	const char*	full_name,
+#endif
+	ulint	full_name_len)
+{
+	(void)trx;
+	(void)full_name;
+	(void)full_name_len;
+	/* do nothing */
+}
+
+int
+mysql_get_identifier_quote_char(
+	trx_t*		trx,
+	const char*	name,
+	ulint		namelen)
+{
+	(void)trx;
+	(void)name;
+	(void)namelen;
+	return '"';
+}
+
+void
+innobase_print_identifier(
+	FILE*	f,
+	trx_t*	trx __attribute__((unused)),
+	ibool	table_id __attribute__((unused)),
+	const char*	name,
+	ulint	namelen)
+{
+	const char*	s	= name;
+	const char*	e = s + namelen;
+	int		q;
+
+	q = '"';
+
+	putc(q, f);
+	while (s < e) {
+		int	c = *s++;
+		if (c == q) {
+			putc(c, f);
+		}
+		putc(c, f);
+	}
+	putc(q, f);
+}
+
+/**********************************************************************//**
+It should be safe to use lower_case_table_names=0 for xtrabackup. If it causes
+any problems, we can add the lower_case_table_names option to xtrabackup
+later.
+@return	0 */
+ulint
+innobase_get_lower_case_table_names(void)
+/*=====================================*/
+{
+	return(0);
+}
+
+/******************************************************************//**
+Strip dir name from a full path name and return only the file name
+@return file name or "null" if no file name */
+const char*
+innobase_basename(
+/*==============*/
+	const char*	path_name)	/*!< in: full path name */
+{
+	const char*	name = base_name(path_name);
+
+	return((name) ? name : "null");
+}
+
+/*****************************************************************//**
+Convert an SQL identifier to the MySQL system_charset_info (UTF-8)
+and quote it if needed.
+@return	pointer to the end of buf */
+static
+char*
+innobase_convert_identifier(
+/*========================*/
+	char*		buf,	/*!< out: buffer for converted identifier */
+	ulint		buflen,	/*!< in: length of buf, in bytes */
+	const char*	id,	/*!< in: identifier to convert */
+	ulint		idlen,	/*!< in: length of id, in bytes */
+	void*		thd __attribute__((unused)), 
+						/*!< in: MySQL connection thread, or NULL */
+	ibool		file_id __attribute__((unused)))
+						/*!< in: TRUE=id is a table or database name;
+						FALSE=id is an UTF-8 string */
+{
+	const char*	s	= id;
+	int		q;
+
+	/* See if the identifier needs to be quoted. */
+	q = '"';
+
+	if (q == EOF) {
+		if (UNIV_UNLIKELY(idlen > buflen)) {
+			idlen = buflen;
+		}
+		memcpy(buf, s, idlen);
+		return(buf + idlen);
+	}
+
+	/* Quote the identifier. */
+	if (buflen < 2) {
+		return(buf);
+	}
+
+	*buf++ = q;
+	buflen--;
+
+	for (; idlen; idlen--) {
+		int	c = *s++;
+		if (UNIV_UNLIKELY(c == q)) {
+			if (UNIV_UNLIKELY(buflen < 3)) {
+				break;
+			}
+
+			*buf++ = c;
+			*buf++ = c;
+			buflen -= 2;
+		} else {
+			if (UNIV_UNLIKELY(buflen < 2)) {
+				break;
+			}
+
+			*buf++ = c;
+			buflen--;
+		}
+	}
+
+	*buf++ = q;
+	return(buf);
+}
+
+/*****************************************************************//**
+Convert a table or index name to the MySQL system_charset_info (UTF-8)
+and quote it if needed.
+@return	pointer to the end of buf */
+char*
+innobase_convert_name(
+/*==================*/
+	char*		buf,	/*!< out: buffer for converted identifier */
+	ulint		buflen,	/*!< in: length of buf, in bytes */
+	const char*	id,	/*!< in: identifier to convert */
+	ulint		idlen,	/*!< in: length of id, in bytes */
+	void*		thd,	/*!< in: MySQL connection thread, or NULL */
+	ibool		table_id)/*!< in: TRUE=id is a table or database name;
+				FALSE=id is an index name */
+{
+	char*		s	= buf;
+	const char*	bufend	= buf + buflen;
+
+	if (table_id) {
+		const char*	slash = (const char*) memchr(id, '/', idlen);
+		if (!slash) {
+
+			goto no_db_name;
+		}
+
+		/* Print the database name and table name separately. */
+		s = innobase_convert_identifier(s, bufend - s, id, slash - id,
+						thd, TRUE);
+		if (UNIV_LIKELY(s < bufend)) {
+			*s++ = '.';
+			s = innobase_convert_identifier(s, bufend - s,
+							slash + 1, idlen
+							- (slash - id) - 1,
+							thd, TRUE);
+		}
+#ifdef INNODB_VERSION_SHORT
+	} else if (UNIV_UNLIKELY(*id == TEMP_INDEX_PREFIX)) {
+		/* Temporary index name (smart ALTER TABLE) */
+		const char temp_index_suffix[]= "--temporary--";
+
+		s = innobase_convert_identifier(buf, buflen, id + 1, idlen - 1,
+						thd, FALSE);
+		if (s - buf + (sizeof temp_index_suffix - 1) < buflen) {
+			memcpy(s, temp_index_suffix,
+			       sizeof temp_index_suffix - 1);
+			s += sizeof temp_index_suffix - 1;
+		}
+#endif
+	} else {
+no_db_name:
+		s = innobase_convert_identifier(buf, buflen, id, idlen,
+						thd, table_id);
+	}
+
+	return(s);
+
+}
+
+ibool
+trx_is_interrupted(
+	trx_t*	trx)
+{
+	(void)trx;
+	/* There are no mysql_thd */
+	return(FALSE);
+}
+
+int
+innobase_mysql_cmp(
+	int		mysql_type,
+	uint		charset_number,
+	unsigned char*	a,
+	unsigned int	a_length,
+	unsigned char*	b,
+	unsigned int	b_length)
+{
+	CHARSET_INFO*		charset;
+	enum enum_field_types	mysql_tp;
+	int                     ret;
+
+	DBUG_ASSERT(a_length != UNIV_SQL_NULL);
+	DBUG_ASSERT(b_length != UNIV_SQL_NULL);
+
+	mysql_tp = (enum enum_field_types) mysql_type;
+
+	switch (mysql_tp) {
+
+	case MYSQL_TYPE_BIT:
+	case MYSQL_TYPE_STRING:
+	case MYSQL_TYPE_VAR_STRING:
+	case FIELD_TYPE_TINY_BLOB:
+	case FIELD_TYPE_MEDIUM_BLOB:
+	case FIELD_TYPE_BLOB:
+	case FIELD_TYPE_LONG_BLOB:
+	case MYSQL_TYPE_VARCHAR:
+		/* Use the charset number to pick the right charset struct for
+		the comparison. Since the MySQL function get_charset may be
+		slow before Bar removes the mutex operation there, we first
+		look at 2 common charsets directly. */
+
+		if (charset_number == default_charset_info->number) {
+			charset = default_charset_info;
+		} else if (charset_number == my_charset_latin1.number) {
+			charset = &my_charset_latin1;
+		} else {
+			charset = get_charset(charset_number, MYF(MY_WME));
+
+			if (charset == NULL) {
+				msg("xtrabackup: InnoDB needs charset %lu for "
+				    "doing a comparison, but MySQL cannot "
+				    "find that charset.\n",
+				    (ulong) charset_number);
+				ut_a(0);
+			}
+		}
+
+		/* Starting from 4.1.3, we use strnncollsp() in comparisons of
+		non-latin1_swedish_ci strings. NOTE that the collation order
+		changes then: 'b\0\0...' is ordered BEFORE 'b  ...'. Users
+		having indexes on such data need to rebuild their tables! */
+
+		ret = charset->coll->strnncollsp(charset,
+				  a, a_length,
+						 b, b_length, 0);
+		if (ret < 0) {
+			return(-1);
+		} else if (ret > 0) {
+			return(1);
+		} else {
+			return(0);
+		}
+	default:
+		assert(0);
+	}
+
+	return(0);
+}
+
+ulint
+innobase_get_at_most_n_mbchars(
+	ulint charset_id,
+	ulint prefix_len,
+	ulint data_len,
+	const char* str)
+{
+	ulint char_length;	/* character length in bytes */
+	ulint n_chars;		/* number of characters in prefix */
+	CHARSET_INFO* charset;	/* charset used in the field */
+
+	charset = get_charset((uint) charset_id, MYF(MY_WME));
+
+	ut_ad(charset);
+	ut_ad(charset->mbmaxlen);
+
+	/* Calculate how many characters at most the prefix index contains */
+
+	n_chars = prefix_len / charset->mbmaxlen;
+
+	/* If the charset is multi-byte, then we must find the length of the
+	first at most n chars in the string. If the string contains less
+	characters than n, then we return the length to the end of the last
+	character. */
+
+	if (charset->mbmaxlen > 1) {
+		/* my_charpos() returns the byte length of the first n_chars
+		characters, or a value bigger than the length of str, if
+		there were not enough full characters in str.
+
+		Why does the code below work:
+		Suppose that we are looking for n UTF-8 characters.
+
+		1) If the string is long enough, then the prefix contains at
+		least n complete UTF-8 characters + maybe some extra
+		characters + an incomplete UTF-8 character. No problem in
+		this case. The function returns the pointer to the
+		end of the nth character.
+
+		2) If the string is not long enough, then the string contains
+		the complete value of a column, that is, only complete UTF-8
+		characters, and we can store in the column prefix index the
+		whole string. */
+
+		char_length = my_charpos(charset, str,
+						str + data_len, (int) n_chars);
+		if (char_length > data_len) {
+			char_length = data_len;
+		}
+	} else {
+		if (data_len < prefix_len) {
+			char_length = data_len;
+		} else {
+			char_length = prefix_len;
+		}
+	}
+
+	return(char_length);
+}
+
+ibool
+innobase_query_is_update(void)
+{
+	msg("xtrabackup: innobase_query_is_update() is called\n");
+	return(0);
+}
+
+#ifdef INNODB_VERSION_SHORT
+ulint
+innobase_raw_format(
+/*================*/
+	const char*	data,		/*!< in: raw data */
+	ulint		data_len,	/*!< in: raw data length
+					in bytes */
+	ulint		charset_coll,	/*!< in: charset collation */
+	char*		buf,		/*!< out: output buffer */
+	ulint		buf_size)	/*!< in: output buffer size
+					in bytes */
+{
+	(void)data;
+	(void)data_len;
+	(void)charset_coll;
+	(void)buf;
+	(void)buf_size;
+
+	msg("xtrabackup: innobase_raw_format() is called\n");
+	return(0);
+}
+
+ulong
+thd_lock_wait_timeout(
+/*==================*/
+	void*	thd)	/*!< in: thread handle (THD*), or NULL to query
+			the global innodb_lock_wait_timeout */
+{
+	(void)thd;
+	return(innobase_lock_wait_timeout);
+}
+
+ibool
+thd_supports_xa(
+/*============*/
+	void*	thd)	/*!< in: thread handle (THD*), or NULL to query
+			the global innodb_supports_xa */
+{
+	(void)thd;
+	return(FALSE);
+}
+
+ibool
+trx_is_strict(
+/*==========*/
+	trx_t*	trx)	/*!< in: transaction */
+{
+	(void)trx;
+	return(FALSE);
+}
+
+#ifdef XTRADB_BASED
+trx_t*
+innobase_get_trx()
+{
+	return(NULL);
+}
+
+ibool
+innobase_get_slow_log()
+{
+	return(FALSE);
+}
+#endif
+#endif
+
+ibool
+thd_is_replication_slave_thread(
+	void*	thd)
+{
+	(void)thd;
+	msg("xtrabackup: thd_is_replication_slave_thread() is called\n");
+	return(FALSE);
+}
+
+ibool
+thd_has_edited_nontrans_tables(
+	void*	thd)
+{
+	(void)thd;
+	msg("xtrabackup: thd_has_edited_nontrans_tables() is called\n");
+	return(FALSE);
+}
+
+ibool
+thd_is_select(
+	const void*	thd)
+{
+	(void)thd;
+	msg("xtrabackup: thd_is_select() is called\n");
+	return(FALSE);
+}
+
+void
+innobase_mysql_prepare_print_arbitrary_thd(void)
+{
+	/* do nothing */
+}
+
+void
+innobase_mysql_end_print_arbitrary_thd(void)
+{
+	/* do nothing */
+}
+
+void
+innobase_mysql_print_thd(
+	FILE*   f,
+	void*   input_thd,
+	uint	max_query_len)
+{
+	(void)f;
+	(void)input_thd;
+	(void)max_query_len;
+	msg("xtrabackup: innobase_mysql_print_thd() is called\n");
+}
+
+void
+innobase_get_cset_width(
+	ulint	cset,
+	ulint*	mbminlen,
+	ulint*	mbmaxlen)
+{
+	CHARSET_INFO*	cs;
+	ut_ad(cset < 256);
+	ut_ad(mbminlen);
+	ut_ad(mbmaxlen);
+
+	cs = all_charsets[cset];
+	if (cs) {
+		*mbminlen = cs->mbminlen;
+		*mbmaxlen = cs->mbmaxlen;
+	} else {
+		ut_a(cset == 0);
+		*mbminlen = *mbmaxlen = 0;
+	}
+}
+
+void
+innobase_convert_from_table_id(
+#ifdef INNODB_VERSION_SHORT
+	struct charset_info_st*	cs,
+#endif
+	char*	to,
+	const char*	from,
+	ulint	len)
+{
+#ifdef INNODB_VERSION_SHORT
+	(void)cs;
+#endif
+	(void)to;
+	(void)from;
+	(void)len;
+
+	msg("xtrabackup: innobase_convert_from_table_id() is called\n");
+}
+
+void
+innobase_convert_from_id(
+#ifdef INNODB_VERSION_SHORT
+	struct charset_info_st*	cs,
+#endif
+	char*	to,
+	const char*	from,
+	ulint	len)
+{
+#ifdef INNODB_VERSION_SHORT
+	(void)cs;
+#endif
+	(void)to;
+	(void)from;
+	(void)len;
+	msg("xtrabackup: innobase_convert_from_id() is called\n");
+}
+
+int
+innobase_strcasecmp(
+	const char*	a,
+	const char*	b)
+{
+	return(my_strcasecmp(&my_charset_utf8_general_ci, a, b));
+}
+
+void
+innobase_casedn_str(
+	char*	a)
+{
+	my_casedn_str(&my_charset_utf8_general_ci, a);
+}
+
+struct charset_info_st*
+innobase_get_charset(
+	void*   mysql_thd)
+{
+	(void)mysql_thd;
+	msg("xtrabackup: innobase_get_charset() is called\n");
+	return(NULL);
+}
+
+const char*
+innobase_get_stmt(
+	void*	mysql_thd,
+	size_t*	length)
+{
+	(void)mysql_thd;
+	(void)length;
+	msg("xtrabackup: innobase_get_stmt() is called\n");
+	return("nothing");
+}
+
+int
+innobase_mysql_tmpfile(void)
+{
+	char	filename[FN_REFLEN];
+	int	fd2 = -1;
+	File	fd = create_temp_file(filename, my_tmpdir(&mysql_tmpdir_list), "ib",
+#ifdef __WIN__
+				O_BINARY | O_TRUNC | O_SEQUENTIAL |
+				O_TEMPORARY | O_SHORT_LIVED |
+#endif /* __WIN__ */
+				O_CREAT | O_EXCL | O_RDWR,
+				MYF(MY_WME));
+	if (fd >= 0) {
+#ifndef __WIN__
+		/* On Windows, open files cannot be removed, but files can be
+		created with the O_TEMPORARY flag to the same effect
+		("delete on close"). */
+		unlink(filename);
+#endif /* !__WIN__ */
+		/* Copy the file descriptor, so that the additional resources
+		allocated by create_temp_file() can be freed by invoking
+		my_close().
+
+		Because the file descriptor returned by this function
+		will be passed to fdopen(), it will be closed by invoking
+		fclose(), which in turn will invoke close() instead of
+		my_close(). */
+#ifdef _WIN32
+		/* Note that on Windows, the integer returned by mysql_tmpfile
+		has no relation to C runtime file descriptor. Here, we need
+		to call my_get_osfhandle to get the HANDLE and then convert it
+		to C runtime filedescriptor. */
+		{
+			HANDLE hFile = my_get_osfhandle(fd);
+			HANDLE hDup;
+			BOOL bOK =
+				DuplicateHandle(GetCurrentProcess(), hFile,
+						GetCurrentProcess(), &hDup, 0,
+						FALSE, DUPLICATE_SAME_ACCESS);
+			if(bOK) {
+				fd2 = _open_osfhandle((intptr_t)hDup,0);
+			}
+			else {
+				my_osmaperr(GetLastError());
+				fd2 = -1;
+			}
+		}
+#else
+		fd2 = dup(fd);
+#endif
+		if (fd2 < 0) {
+			msg("xtrabackup: Got error %d on dup\n",fd2);
+		}
+		my_close(fd, MYF(MY_WME));
+	}
+	return(fd2);
+}
+
+#if MYSQL_VERSION_ID >= 50507
+/*
+   As of MySQL 5.5.7, InnoDB uses thd_wait plugin service.
+   We have to provide mock functions to avoid linker errors.
+*/
+#include <mysql/plugin.h>
+#include <mysql/service_thd_wait.h>
+
+void thd_wait_begin(MYSQL_THD thd, int wait_type)
+{
+	(void)thd;
+	(void)wait_type;
+	return;
+}
+
+void thd_wait_end(MYSQL_THD thd)
+{
+	(void)thd;
+	return;
+}
+
+#endif /* MYSQL_VERSION_ID >= 50507 */
 
=== added file 'src/innodb_int.h'
--- src/innodb_int.h	1970-01-01 00:00:00 +0000
+++ src/innodb_int.h	2012-10-18 11:58:23 +0000
@@ -0,0 +1,695 @@
+/******************************************************
+XtraBackup: hot backup tool for InnoDB
+(c) 2009-2012 Percona Inc.
+Originally Created 3/3/2009 Yasufumi Kinoshita
+Written by Alexey Kopytov, Aleksandr Kuzminsky, Stewart Smith, Vadim Tkachenko,
+Yasufumi Kinoshita, Ignacio Nin and Baron Schwartz.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+/* InnoDB portability helpers and interface to internal functions */
+
+#ifndef INNODB_INT_H
+#define INNODB_INT_H
+
+#include <my_base.h>
+#include <univ.i>
+#include <fil0fil.h>
+#include <os0file.h>
+#include <hash0hash.h>
+#include <btr0sea.h>
+#include <log0log.h>
+#include <log0recv.h>
+#include <trx0sys.h>
+
+#ifndef INNODB_VERSION_SHORT
+#  define IB_INT64 ib_longlong
+#  define LSN64 dulint
+#  define MACH_READ_64 mach_read_from_8
+#  define MACH_WRITE_64 mach_write_to_8
+#  define OS_MUTEX_CREATE() os_mutex_create(NULL)
+#  define xb_buf_page_is_corrupted(page, zip_size) buf_page_is_corrupted(page)
+#  define xb_fil_space_create(name, space_id, zip_size, purpose) \
+	fil_space_create(name, space_id, purpose)
+#  define PAGE_ZIP_MIN_SIZE_SHIFT	10
+#  define PAGE_ZIP_MIN_SIZE	(1 << PAGE_ZIP_MIN_SIZE_SHIFT)
+#  define DICT_TF_ZSSIZE_SHIFT	1
+#  define DICT_TF_ZSSIZE_MASK		(15 << DICT_TF_ZSSIZE_SHIFT)
+#  define DICT_TF_FORMAT_ZIP	1
+#  define DICT_TF_FORMAT_SHIFT		5
+#  define SRV_SHUTDOWN_NONE	0
+#else
+#  define IB_INT64 ib_int64_t
+#  define LSN64 ib_uint64_t
+#  if (MYSQL_VERSION_ID < 50500)
+#    define MACH_READ_64 mach_read_ull
+#    define MACH_WRITE_64 mach_write_ull
+#    define OS_MUTEX_CREATE() os_mutex_create(NULL)
+#  else
+#    define MACH_READ_64 mach_read_from_8
+#    define MACH_WRITE_64 mach_write_to_8
+#    define OS_MUTEX_CREATE() os_mutex_create()
+#  endif
+#  define xb_buf_page_is_corrupted(page, zip_size)	\
+	buf_page_is_corrupted(page, zip_size)
+#  define xb_fil_space_create(name, space_id, zip_size, purpose) \
+	fil_space_create(name, space_id, zip_size, purpose)
+#  define ut_dulint_zero 0
+#  define ut_dulint_cmp(A, B) (A > B ? 1 : (A == B ? 0 : -1))
+#  define ut_dulint_add(A, B) (A + B)
+#  define ut_dulint_minus(A, B) (A - B)
+#  define ut_dulint_align_down(A, B) (A & ~((ib_int64_t)B - 1))
+#  define ut_dulint_align_up(A, B) ((A + B - 1) & ~((ib_int64_t)B - 1))
+#endif
+
+#ifndef XTRADB_BASED
+#define trx_sys_sys_space(id) (id == 0)
+#endif
+
+#ifdef __WIN__
+#define SRV_PATH_SEPARATOR	'\\'
+#define SRV_PATH_SEPARATOR_STR	"\\"
+#else
+#define SRV_PATH_SEPARATOR	'/'
+#define SRV_PATH_SEPARATOR_STR	"/"
+#endif
+
+#ifndef UNIV_PAGE_SIZE_MAX
+#define UNIV_PAGE_SIZE_MAX UNIV_PAGE_SIZE
+#endif
+#ifndef UNIV_PAGE_SIZE_SHIFT_MAX
+#define UNIV_PAGE_SIZE_SHIFT_MAX UNIV_PAGE_SIZE_SHIFT
+#endif
+
+#ifdef __WIN__
+#define XB_FILE_UNDEFINED NULL
+#else
+#define XB_FILE_UNDEFINED (-1)
+#endif
+
+#ifdef INNODB_VERSION_SHORT
+#define XB_HASH_SEARCH(NAME, TABLE, FOLD, DATA, ASSERTION, TEST) \
+	HASH_SEARCH(NAME, TABLE, FOLD, xtrabackup_tables_t*, DATA, ASSERTION, \
+		    TEST)
+#else
+#define XB_HASH_SEARCH(NAME, TABLE, FOLD, DATA, ASSERTION, TEST) \
+	HASH_SEARCH(NAME, TABLE, FOLD, DATA, TEST)
+#endif
+
+/* ==start === definition at fil0fil.c === */
+// ##################################################################
+// NOTE: We should check the following definitions fit to the source.
+// ##################################################################
+
+#ifndef INNODB_VERSION_SHORT
+//5.0 5.1
+/* File node of a tablespace or the log data space */
+struct fil_node_struct {
+	fil_space_t*	space;	/* backpointer to the space where this node
+				belongs */
+	char*		name;	/* path to the file */
+	ibool		open;	/* TRUE if file open */
+	os_file_t	handle; /* OS handle to the file, if file open */
+	ibool		is_raw_disk;/* TRUE if the 'file' is actually a raw
+				device or a raw disk partition */
+	ulint		size;	/* size of the file in database pages, 0 if
+				not known yet; the possible last incomplete
+				megabyte may be ignored if space == 0 */
+	ulint		n_pending;
+				/* count of pending i/o's on this file;
+				closing of the file is not allowed if
+				this is > 0 */
+	ulint		n_pending_flushes;
+				/* count of pending flushes on this file;
+				closing of the file is not allowed if
+				this is > 0 */
+	ib_longlong	modification_counter;/* when we write to the file we
+				increment this by one */
+	ib_longlong	flush_counter;/* up to what modification_counter value
+				we have flushed the modifications to disk */
+	UT_LIST_NODE_T(fil_node_t) chain;
+				/* link field for the file chain */
+	UT_LIST_NODE_T(fil_node_t) LRU;
+				/* link field for the LRU list */
+	ulint		magic_n;
+};
+
+struct fil_space_struct {
+	char*		name;	/* space name = the path to the first file in
+				it */
+	ulint		id;	/* space id */
+	ib_longlong	tablespace_version;
+				/* in DISCARD/IMPORT this timestamp is used to
+				check if we should ignore an insert buffer
+				merge request for a page because it actually
+				was for the previous incarnation of the
+				space */
+	ibool		mark;	/* this is set to TRUE at database startup if
+				the space corresponds to a table in the InnoDB
+				data dictionary; so we can print a warning of
+				orphaned tablespaces */
+	ibool		stop_ios;/* TRUE if we want to rename the .ibd file of
+				tablespace and want to stop temporarily
+				posting of new i/o requests on the file */
+	ibool		stop_ibuf_merges;
+				/* we set this TRUE when we start deleting a
+				single-table tablespace */
+	ibool		is_being_deleted;
+				/* this is set to TRUE when we start
+				deleting a single-table tablespace and its
+				file; when this flag is set no further i/o
+				or flush requests can be placed on this space,
+				though there may be such requests still being
+				processed on this space */
+	ulint		purpose;/* FIL_TABLESPACE, FIL_LOG, or FIL_ARCH_LOG */
+	UT_LIST_BASE_NODE_T(fil_node_t) chain;
+				/* base node for the file chain */
+	ulint		size;	/* space size in pages; 0 if a single-table
+				tablespace whose size we do not know yet;
+				last incomplete megabytes in data files may be
+				ignored if space == 0 */
+	ulint		n_reserved_extents;
+				/* number of reserved free extents for
+				ongoing operations like B-tree page split */
+	ulint		n_pending_flushes; /* this is > 0 when flushing
+				the tablespace to disk; dropping of the
+				tablespace is forbidden if this is > 0 */
+	ulint		n_pending_ibuf_merges;/* this is > 0 when merging
+				insert buffer entries to a page so that we
+				may need to access the ibuf bitmap page in the
+				tablespade: dropping of the tablespace is
+				forbidden if this is > 0 */
+	hash_node_t	hash;	/* hash chain node */
+	hash_node_t	name_hash;/* hash chain the name_hash table */
+	rw_lock_t	latch;	/* latch protecting the file space storage
+				allocation */
+	UT_LIST_NODE_T(fil_space_t) unflushed_spaces;
+				/* list of spaces with at least one unflushed
+				file we have written to */
+	ibool		is_in_unflushed_spaces; /* TRUE if this space is
+				currently in the list above */
+	UT_LIST_NODE_T(fil_space_t) space_list;
+				/* list of all spaces */
+	ibuf_data_t*	ibuf_data;
+				/* insert buffer data */
+	ulint		magic_n;
+};
+typedef struct fil_system_struct	fil_system_t;
+struct fil_system_struct {
+	mutex_t		mutex;		/* The mutex protecting the cache */
+	hash_table_t*	spaces;		/* The hash table of spaces in the
+					system; they are hashed on the space
+					id */
+	hash_table_t*	name_hash;	/* hash table based on the space
+					name */
+	UT_LIST_BASE_NODE_T(fil_node_t) LRU;
+					/* base node for the LRU list of the
+					most recently used open files with no
+					pending i/o's; if we start an i/o on
+					the file, we first remove it from this
+					list, and return it to the start of
+					the list when the i/o ends;
+					log files and the system tablespace are
+					not put to this list: they are opened
+					after the startup, and kept open until
+					shutdown */
+	UT_LIST_BASE_NODE_T(fil_space_t) unflushed_spaces;
+					/* base node for the list of those
+					tablespaces whose files contain
+					unflushed writes; those spaces have
+					at least one file node where
+					modification_counter > flush_counter */
+	ulint		n_open;		/* number of files currently open */
+	ulint		max_n_open;	/* n_open is not allowed to exceed
+					this */
+	ib_longlong	modification_counter;/* when we write to a file we
+					increment this by one */
+	ulint		max_assigned_id;/* maximum space id in the existing
+					tables, or assigned during the time
+					mysqld has been up; at an InnoDB
+					startup we scan the data dictionary
+					and set here the maximum of the
+					space id's of the tables there */
+	ib_longlong	tablespace_version;
+					/* a counter which is incremented for
+					every space object memory creation;
+					every space mem object gets a
+					'timestamp' from this; in DISCARD/
+					IMPORT this is used to check if we
+					should ignore an insert buffer merge
+					request */
+	UT_LIST_BASE_NODE_T(fil_space_t) space_list;
+					/* list of all file spaces */
+};
+#else
+//Plugin ?
+/** File node of a tablespace or the log data space */
+struct fil_node_struct {
+	fil_space_t*	space;	/*!< backpointer to the space where this node
+				belongs */
+	char*		name;	/*!< path to the file */
+	ibool		open;	/*!< TRUE if file open */
+	os_file_t	handle;	/*!< OS handle to the file, if file open */
+	ibool		is_raw_disk;/*!< TRUE if the 'file' is actually a raw
+				device or a raw disk partition */
+	ulint		size;	/*!< size of the file in database pages, 0 if
+				not known yet; the possible last incomplete
+				megabyte may be ignored if space == 0 */
+	ulint		n_pending;
+				/*!< count of pending i/o's on this file;
+				closing of the file is not allowed if
+				this is > 0 */
+	ulint		n_pending_flushes;
+				/*!< count of pending flushes on this file;
+				closing of the file is not allowed if
+				this is > 0 */
+	ib_int64_t	modification_counter;/*!< when we write to the file we
+				increment this by one */
+	ib_int64_t	flush_counter;/*!< up to what
+				modification_counter value we have
+				flushed the modifications to disk */
+	UT_LIST_NODE_T(fil_node_t) chain;
+				/*!< link field for the file chain */
+	UT_LIST_NODE_T(fil_node_t) LRU;
+				/*!< link field for the LRU list */
+	ulint		magic_n;/*!< FIL_NODE_MAGIC_N */
+};
+
+struct fil_space_struct {
+	char*		name;	/*!< space name = the path to the first file in
+				it */
+	ulint		id;	/*!< space id */
+	ib_int64_t	tablespace_version;
+				/*!< in DISCARD/IMPORT this timestamp
+				is used to check if we should ignore
+				an insert buffer merge request for a
+				page because it actually was for the
+				previous incarnation of the space */
+	ibool		mark;	/*!< this is set to TRUE at database startup if
+				the space corresponds to a table in the InnoDB
+				data dictionary; so we can print a warning of
+				orphaned tablespaces */
+	ibool		stop_ios;/*!< TRUE if we want to rename the
+				.ibd file of tablespace and want to
+				stop temporarily posting of new i/o
+				requests on the file */
+	ibool		stop_ibuf_merges;
+				/*!< we set this TRUE when we start
+				deleting a single-table tablespace */
+	ibool		is_being_deleted;
+				/*!< this is set to TRUE when we start
+				deleting a single-table tablespace and its
+				file; when this flag is set no further i/o
+				or flush requests can be placed on this space,
+				though there may be such requests still being
+				processed on this space */
+	ulint		purpose;/*!< FIL_TABLESPACE, FIL_LOG, or
+				FIL_ARCH_LOG */
+	UT_LIST_BASE_NODE_T(fil_node_t) chain;
+				/*!< base node for the file chain */
+	ulint		size;	/*!< space size in pages; 0 if a single-table
+				tablespace whose size we do not know yet;
+				last incomplete megabytes in data files may be
+				ignored if space == 0 */
+	ulint		flags;	/*!< compressed page size and file format, or 0
+				*/
+	ulint		n_reserved_extents;
+				/*!< number of reserved free extents for
+				ongoing operations like B-tree page split */
+	ulint		n_pending_flushes; /*!< this is positive when flushing
+				the tablespace to disk; dropping of the
+				tablespace is forbidden if this is positive */
+	ulint		n_pending_ibuf_merges;/*!< this is positive
+				when merging insert buffer entries to
+				a page so that we may need to access
+				the ibuf bitmap page in the
+				tablespade: dropping of the tablespace
+				is forbidden if this is positive */
+	hash_node_t	hash;	/*!< hash chain node */
+	hash_node_t	name_hash;/*!< hash chain the name_hash table */
+#ifndef UNIV_HOTBACKUP
+	rw_lock_t	latch;	/*!< latch protecting the file space storage
+				allocation */
+#endif /* !UNIV_HOTBACKUP */
+	UT_LIST_NODE_T(fil_space_t) unflushed_spaces;
+				/*!< list of spaces with at least one unflushed
+				file we have written to */
+	ibool		is_in_unflushed_spaces; /*!< TRUE if this space is
+				currently in unflushed_spaces */
+#ifdef XTRADB_BASED
+	ibool		is_corrupt;
+#endif
+	UT_LIST_NODE_T(fil_space_t) space_list;
+				/*!< list of all spaces */
+	ulint		magic_n;/*!< FIL_SPACE_MAGIC_N */
+};
+
+typedef	struct fil_system_struct	fil_system_t;
+
+struct fil_system_struct {
+#ifndef UNIV_HOTBACKUP
+	mutex_t		mutex;		/*!< The mutex protecting the cache */
+#ifdef XTRADB_BASED
+	mutex_t		file_extend_mutex;
+#endif
+#endif /* !UNIV_HOTBACKUP */
+	hash_table_t*	spaces;		/*!< The hash table of spaces in the
+					system; they are hashed on the space
+					id */
+	hash_table_t*	name_hash;	/*!< hash table based on the space
+					name */
+	UT_LIST_BASE_NODE_T(fil_node_t) LRU;
+					/*!< base node for the LRU list of the
+					most recently used open files with no
+					pending i/o's; if we start an i/o on
+					the file, we first remove it from this
+					list, and return it to the start of
+					the list when the i/o ends;
+					log files and the system tablespace are
+					not put to this list: they are opened
+					after the startup, and kept open until
+					shutdown */
+	UT_LIST_BASE_NODE_T(fil_space_t) unflushed_spaces;
+					/*!< base node for the list of those
+					tablespaces whose files contain
+					unflushed writes; those spaces have
+					at least one file node where
+					modification_counter > flush_counter */
+	ulint		n_open;		/*!< number of files currently open */
+	ulint		max_n_open;	/*!< n_open is not allowed to exceed
+					this */
+	ib_int64_t	modification_counter;/*!< when we write to a file we
+					increment this by one */
+	ulint		max_assigned_id;/*!< maximum space id in the existing
+					tables, or assigned during the time
+					mysqld has been up; at an InnoDB
+					startup we scan the data dictionary
+					and set here the maximum of the
+					space id's of the tables there */
+	ib_int64_t	tablespace_version;
+					/*!< a counter which is incremented for
+					every space object memory creation;
+					every space mem object gets a
+					'timestamp' from this; in DISCARD/
+					IMPORT this is used to check if we
+					should ignore an insert buffer merge
+					request */
+	UT_LIST_BASE_NODE_T(fil_space_t) space_list;
+					/*!< list of all file spaces */
+	ibool		space_id_reuse_warned;
+					/* !< TRUE if fil_space_create()
+					has issued a warning about
+					potential space_id reuse */
+};
+
+#endif /* INNODB_VERSION_SHORT */
+
+extern fil_system_t*   fil_system;
+extern char *opt_mysql_tmpdir;
+extern MY_TMPDIR mysql_tmpdir_list;
+
+/** Value of fil_space_struct::magic_n */
+#define	FIL_SPACE_MAGIC_N	89472
+
+/* ==end=== definition  at fil0fil.c === */
+
+/* prototypes for static functions in original */
+#ifndef INNODB_VERSION_SHORT
+page_t*
+btr_node_ptr_get_child(
+/*===================*/
+				/* out: child page, x-latched */
+	rec_t*		node_ptr,/* in: node pointer */
+	const ulint*	offsets,/* in: array returned by rec_get_offsets() */
+	mtr_t*		mtr);	/* in: mtr */
+#else
+buf_block_t*
+btr_node_ptr_get_child(
+/*===================*/
+	const rec_t*	node_ptr,/*!< in: node pointer */
+	dict_index_t*	index,	/*!< in: index */
+	const ulint*	offsets,/*!< in: array returned by rec_get_offsets() */
+	mtr_t*		mtr);	/*!< in: mtr */
+
+buf_block_t*
+btr_root_block_get(
+/*===============*/
+	dict_index_t*	index,	/*!< in: index tree */
+	mtr_t*		mtr);	/*!< in: mtr */
+#endif
+
+int
+fil_file_readdir_next_file(
+/*=======================*/
+				/* out: 0 if ok, -1 if error even after the
+				retries, 1 if at the end of the directory */
+	ulint*		err,	/* out: this is set to DB_ERROR if an error
+				was encountered, otherwise not changed */
+	const char*	dirname,/* in: directory name or path */
+	os_file_dir_t	dir,	/* in: directory stream */
+	os_file_stat_t*	info);	/* in/out: buffer where the info is returned */
+
+ibool
+recv_check_cp_is_consistent(
+/*========================*/
+			/* out: TRUE if ok */
+	byte*	buf);	/* in: buffer containing checkpoint info */
+
+ulint
+recv_find_max_checkpoint(
+/*=====================*/
+					/* out: error code or DB_SUCCESS */
+	log_group_t**	max_group,	/* out: max group */
+	ulint*		max_field);	/* out: LOG_CHECKPOINT_1 or
+					LOG_CHECKPOINT_2 */
+
+ibool
+log_block_checksum_is_ok_or_old_format(
+/*===================================*/
+			/* out: TRUE if ok, or if the log block may be in the
+			format of InnoDB version < 3.23.52 */
+	byte*	block);	/* in: pointer to a log block */
+
+ulint
+open_or_create_log_file(
+/*====================*/
+					/* out: DB_SUCCESS or error code */
+	ibool	create_new_db,		/* in: TRUE if we should create a
+					new database */
+	ibool*	log_file_created,	/* out: TRUE if new log file
+					created */
+	ibool	log_file_has_been_opened,/* in: TRUE if a log file has been
+					opened before: then it is an error
+					to try to create another log file */
+	ulint	k,			/* in: log group number */
+	ulint	i);			/* in: log file number in group */
+
+ulint
+open_or_create_data_files(
+/*======================*/
+				/* out: DB_SUCCESS or error code */
+	ibool*	create_new_db,	/* out: TRUE if new database should be
+								created */
+#ifdef XTRADB_BASED
+	ibool*	create_new_doublewrite_file,
+#endif
+#ifdef UNIV_LOG_ARCHIVE
+	ulint*	min_arch_log_no,/* out: min of archived log numbers in data
+				files */
+	ulint*	max_arch_log_no,/* out: */
+#endif /* UNIV_LOG_ARCHIVE */
+	LSN64*	min_flushed_lsn,/* out: min of flushed lsn values in data
+				files */
+	LSN64*	max_flushed_lsn,/* out: */
+	ulint*	sum_of_new_sizes);/* out: sum of sizes of the new files added
+				  */
+
+/****************************************************************//**
+A simple function to open or create a file.
+@return own: handle to the file, not defined if error, error number
+can be retrieved with os_file_get_last_error */
+os_file_t
+xb_file_create_no_error_handling(
+/*=============================*/
+	const char*	name,	/*!< in: name of the file or path as a
+				null-terminated string */
+	ulint		create_mode,/*!< in: OS_FILE_OPEN if an existing file
+				is opened (if does not exist, error), or
+				OS_FILE_CREATE if a new file is created
+				(if exists, error) */
+	ulint		access_type,/*!< in: OS_FILE_READ_ONLY,
+				OS_FILE_READ_WRITE, or
+				OS_FILE_READ_ALLOW_DELETE; the last option is
+				used by a backup program reading the file */
+	ibool*		success);/*!< out: TRUE if succeed, FALSE if error */
+
+/****************************************************************//**
+Opens an existing file or creates a new.
+@return own: handle to the file, not defined if error, error number
+can be retrieved with os_file_get_last_error */
+os_file_t
+xb_file_create(
+/*===========*/
+	const char*	name,	/*!< in: name of the file or path as a
+				null-terminated string */
+	ulint		create_mode,/*!< in: OS_FILE_OPEN if an existing file
+				is opened (if does not exist, error), or
+				OS_FILE_CREATE if a new file is created
+				(if exists, error),
+				OS_FILE_OVERWRITE if a new file is created
+				or an old overwritten;
+				OS_FILE_OPEN_RAW, if a raw device or disk
+				partition should be opened */
+	ulint		purpose,/*!< in: OS_FILE_AIO, if asynchronous,
+				non-buffered i/o is desired,
+				OS_FILE_NORMAL, if any normal file;
+				NOTE that it also depends on type, os_aio_..
+				and srv_.. variables whether we really use
+				async i/o or unbuffered i/o: look in the
+				function source code for the exact rules */
+	ulint		type,	/*!< in: OS_DATA_FILE or OS_LOG_FILE */
+	ibool*		success);/*!< out: TRUE if succeed, FALSE if error */
+
+
+/***********************************************************************//**
+Renames a file (can also move it to another directory). It is safest that the
+file is closed before calling this function.
+@return	TRUE if success */
+ibool
+xb_file_rename(
+/*===========*/
+	const char*	oldpath,/*!< in: old file path as a null-terminated
+				string */
+	const char*	newpath);/*!< in: new file path */
+
+void
+xb_file_set_nocache(
+/*================*/
+	os_file_t	fd,		/* in: file descriptor to alter */
+	const char*	file_name,	/* in: used in the diagnostic message
+					*/
+	const char*	operation_name);/* in: used in the diagnostic message,
+					we call os_file_set_nocache()
+					immediately after opening or creating
+					a file, so this is either "open" or
+					"create" */
+
+/***********************************************************************//**
+Compatibility wrapper around os_file_flush().
+@return	TRUE if success */
+ibool
+xb_file_flush(
+/*==========*/
+	os_file_t	file);	/*!< in, own: handle to a file */
+
+/*******************************************************************//**
+Returns the table space by a given id, NULL if not found. */
+fil_space_t*
+xb_space_get_by_id(
+/*================*/
+	ulint	id);	/*!< in: space id */
+
+/*******************************************************************//**
+Returns the table space by a given name, NULL if not found. */
+fil_space_t*
+xb_space_get_by_name(
+/*==================*/
+	const char*	name);	/*!< in: space name */
+
+/****************************************************************//**
+Create a new tablespace on disk and return the handle to its opened
+file. Code adopted from fil_create_new_single_table_tablespace with
+the main difference that only disk file is created without updating
+the InnoDB in-memory dictionary data structures.
+
+@return TRUE on success, FALSE on error.  */
+ibool
+xb_space_create_file(
+/*==================*/
+	const char*	path,		/*!<in: path to tablespace */
+	ulint		space_id,	/*!<in: space id */
+	ulint		flags __attribute__((unused)),/*!<in: tablespace
+						      flags */
+	os_file_t*	file);		/*!<out: file handle */
+
+#ifndef INNODB_VERSION_SHORT
+
+/********************************************************************//**
+Extract the compressed page size from table flags.
+@return	compressed page size, or 0 if not compressed */
+ulint
+dict_table_flags_to_zip_size(
+/*=========================*/
+	ulint	flags);	/*!< in: flags */
+
+
+/*******************************************************************//**
+Free all spaces in space_list. */
+void
+fil_free_all_spaces(void);
+/*=====================*/
+
+#endif
+
+void
+innobase_mysql_prepare_print_arbitrary_thd(void);
+
+void
+innobase_mysql_end_print_arbitrary_thd(void);
+
+int
+mysql_get_identifier_quote_char(
+	trx_t*		trx,
+	const char*	name,
+	ulint		namelen);
+
+void
+innobase_print_identifier(
+	FILE*	f,
+	trx_t*	trx __attribute__((unused)),
+	ibool	table_id __attribute__((unused)),
+	const char*	name,
+	ulint	namelen);
+
+/**********************************************************************//**
+It should be safe to use lower_case_table_names=0 for xtrabackup. If it causes
+any problems, we can add the lower_case_table_names option to xtrabackup
+later.
+@return	0 */
+ulint
+innobase_get_lower_case_table_names(void);
+
+/******************************************************************//**
+Strip dir name from a full path name and return only the file name
+@return file name or "null" if no file name */
+const char*
+innobase_basename(
+/*==============*/
+	const char*	path_name);	/*!< in: full path name */
+
+int
+innobase_mysql_cmp(
+	int		mysql_type,
+	uint		charset_number,
+	unsigned char*	a,
+	unsigned int	a_length,
+	unsigned char*	b,
+	unsigned int	b_length);
+
+ibool
+innobase_query_is_update(void);
+
+#endif /* INNODB_INT_H */
 
=== modified file 'src/quicklz/quicklz.h'
--- src/quicklz/quicklz.h	2012-02-10 20:05:56 +0000
+++ src/quicklz/quicklz.h	2012-10-18 11:58:23 +0000
@@ -134,6 +134,7 @@
 size_t qlz_compress(const void *source, char *destination, size_t size, qlz_state_compress *state);
 size_t qlz_decompress(const char *source, void *destination, qlz_state_decompress *state);
 int qlz_get_setting(int setting);
+size_t qlz_size_header(const char *source);
 
 #if defined (__cplusplus)
 }
 
=== added file 'src/write_filt.c'
--- src/write_filt.c	1970-01-01 00:00:00 +0000
+++ src/write_filt.c	2012-10-18 11:58:23 +0000
@@ -0,0 +1,321 @@
+/******************************************************
+XtraBackup: hot backup tool for InnoDB
+(c) 2009-2012 Percona Inc.
+Originally Created 3/3/2009 Yasufumi Kinoshita
+Written by Alexey Kopytov, Aleksandr Kuzminsky, Stewart Smith, Vadim Tkachenko,
+Yasufumi Kinoshita, Ignacio Nin and Baron Schwartz.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+/* Page write filters implementation */
+
+#include <my_base.h>
+#include <fil0fil.h>
+#include <ut0mem.h>
+#include "common.h"
+#include "innodb_int.h"
+#include "write_filt.h"
+#include "fil_cur.h"
+#include "xtrabackup.h"
+
+/************************************************************************
+Write-through page write filter. */
+static my_bool wf_wt_init(xb_write_filt_ctxt_t *ctxt, char *dst_name,
+			  xb_fil_cur_t *cursor);
+static my_bool wf_wt_process(xb_write_filt_ctxt_t *ctxt, ds_file_t *dstfile);
+
+xb_write_filt_t wf_write_through = {
+	&wf_wt_init,
+	&wf_wt_process,
+	NULL,
+	NULL
+};
+
+/************************************************************************
+Incremental page write filter. */
+static my_bool wf_incremental_init(xb_write_filt_ctxt_t *ctxt, char *dst_name,
+				   xb_fil_cur_t *cursor);
+static my_bool wf_incremental_process(xb_write_filt_ctxt_t *ctxt,
+				      ds_file_t *dstfile);
+static my_bool wf_incremental_finalize(xb_write_filt_ctxt_t *ctxt,
+				       ds_file_t *dstfile);
+static void wf_incremental_deinit(xb_write_filt_ctxt_t *ctxt);
+
+xb_write_filt_t wf_incremental = {
+	&wf_incremental_init,
+	&wf_incremental_process,
+	&wf_incremental_finalize,
+	&wf_incremental_deinit
+};
+
+/************************************************************************
+Compact page write filter. */
+static my_bool wf_compact_init(xb_write_filt_ctxt_t *ctxt, char *dst_name,
+			       xb_fil_cur_t *cursor);
+static my_bool wf_compact_process(xb_write_filt_ctxt_t *ctxt,
+				  ds_file_t *dstfile);
+static my_bool wf_compact_finalize(xb_write_filt_ctxt_t *ctxt,
+				   ds_file_t *dstfile);
+
+xb_write_filt_t wf_compact = {
+	&wf_compact_init,
+	&wf_compact_process,
+	&wf_compact_finalize,
+	NULL
+};
+
+/************************************************************************
+Initialize incremental page write filter.
+
+@return TRUE on success, FALSE on error. */
+static my_bool
+wf_incremental_init(xb_write_filt_ctxt_t *ctxt, char *dst_name,
+		    xb_fil_cur_t *cursor)
+{
+	char				meta_name[FN_REFLEN];
+	xb_delta_info_t			info;
+	ulint				buf_size;
+	xb_wf_incremental_ctxt_t	*cp =
+		&(ctxt->u.wf_incremental_ctxt);
+
+	ctxt->cursor = cursor;
+
+	/* allocate buffer for incremental backup (4096 pages) */
+	buf_size = (UNIV_PAGE_SIZE_MAX / 4 + 1) * UNIV_PAGE_SIZE_MAX;
+	cp->delta_buf_base = ut_malloc(buf_size);
+	bzero(cp->delta_buf_base, buf_size);
+	cp->delta_buf = ut_align(cp->delta_buf_base, UNIV_PAGE_SIZE_MAX);
+
+	/* write delta meta info */
+	snprintf(meta_name, sizeof(meta_name), "%s%s", dst_name,
+		 XB_DELTA_INFO_SUFFIX);
+	info.page_size = cursor->page_size;
+	info.zip_size = cursor->zip_size;
+	info.space_id = cursor->space_id;
+	if (!xb_write_delta_metadata(meta_name, &info)) {
+		msg("[%02lu] xtrabackup: Error: "
+		    "failed to write meta info for %s\n",
+		    cursor->thread_n, cursor->path);
+		return(FALSE);
+	}
+
+	/* change the target file name, since we are only going to write
+	delta pages */
+	strcat(dst_name, ".delta");
+
+	mach_write_to_4(cp->delta_buf, 0x78747261UL); /*"xtra"*/
+	cp->npages = 1;
+
+	return(TRUE);
+}
+
+/************************************************************************
+Run the next batch of pages through incremental page write filter.
+
+@return TRUE on success, FALSE on error. */
+static my_bool
+wf_incremental_process(xb_write_filt_ctxt_t *ctxt, ds_file_t *dstfile)
+{
+	ulint				i;
+	xb_fil_cur_t			*cursor = ctxt->cursor;
+	ulint				page_size = cursor->page_size;
+	byte				*page;
+	xb_wf_incremental_ctxt_t	*cp = &(ctxt->u.wf_incremental_ctxt);
+
+	for (i = 0, page = cursor->buf; i < cursor->buf_npages;
+	     i++, page += page_size) {
+		if (ut_dulint_cmp(incremental_lsn,
+				  MACH_READ_64(page + FIL_PAGE_LSN)) >= 0) {
+			continue;
+		}
+
+		/* updated page */
+		if (cp->npages == page_size / 4) {
+			/* flush buffer */
+			if (ds_write(dstfile, cp->delta_buf,
+				     cp->npages * page_size)) {
+				return(FALSE);
+			}
+
+			/* clear buffer */
+			bzero(cp->delta_buf, page_size / 4 * page_size);
+			/*"xtra"*/
+			mach_write_to_4(cp->delta_buf, 0x78747261UL);
+			cp->npages = 1;
+		}
+
+		mach_write_to_4(cp->delta_buf + cp->npages * 4,
+				cursor->buf_page_no + i);
+		memcpy(cp->delta_buf + cp->npages * page_size, page,
+		       page_size);
+
+		cp->npages++;
+	}
+
+	return(TRUE);
+}
+
+/************************************************************************
+Flush the incremental page write filter's buffer.
+
+@return TRUE on success, FALSE on error. */
+static my_bool
+wf_incremental_finalize(xb_write_filt_ctxt_t *ctxt, ds_file_t *dstfile)
+{
+	xb_fil_cur_t			*cursor = ctxt->cursor;
+	ulint				page_size = cursor->page_size;
+	xb_wf_incremental_ctxt_t	*cp = &(ctxt->u.wf_incremental_ctxt);
+
+	if (cp->npages != page_size / 4) {
+		mach_write_to_4(cp->delta_buf + cp->npages * 4, 0xFFFFFFFFUL);
+	}
+
+	/* Mark the final block */
+	mach_write_to_4(cp->delta_buf, 0x58545241UL); /*"XTRA"*/
+
+	/* flush buffer */
+	if (ds_write(dstfile, cp->delta_buf, cp->npages * page_size)) {
+		return(FALSE);
+	}
+
+	return(TRUE);
+}
+
+/************************************************************************
+Free the incremental page write filter's buffer. */
+static void
+wf_incremental_deinit(xb_write_filt_ctxt_t *ctxt)
+{
+	xb_wf_incremental_ctxt_t	*cp = &(ctxt->u.wf_incremental_ctxt);
+
+	if (cp->delta_buf_base != NULL) {
+		ut_free(cp->delta_buf_base);
+	}
+}
+
+/************************************************************************
+Initialize the write-through page write filter.
+
+@return TRUE on success, FALSE on error. */
+static my_bool
+wf_wt_init(xb_write_filt_ctxt_t *ctxt, char *dst_name __attribute__((unused)),
+	   xb_fil_cur_t *cursor)
+{
+	ctxt->cursor = cursor;
+
+	return(TRUE);
+}
+
+/************************************************************************
+Write the next batch of pages to the destination datasink.
+
+@return TRUE on success, FALSE on error. */
+static my_bool
+wf_wt_process(xb_write_filt_ctxt_t *ctxt, ds_file_t *dstfile)
+{
+	xb_fil_cur_t			*cursor = ctxt->cursor;
+
+	if (ds_write(dstfile, cursor->buf, cursor->buf_read)) {
+		return(FALSE);
+	}
+
+	return(TRUE);
+}
+
+/************************************************************************
+Initialize the compact page write filter.
+
+@return TRUE on success, FALSE on error. */
+static my_bool
+wf_compact_init(xb_write_filt_ctxt_t *ctxt,
+		char *dst_name __attribute__((unused)), xb_fil_cur_t *cursor)
+{
+	xb_wf_compact_ctxt_t		*cp = &(ctxt->u.wf_compact_ctxt);
+
+	/* Don't compact the system table space */
+	cp->skip = cursor->is_system;
+
+	return(TRUE);
+}
+
+static my_bool
+check_if_skip_page(byte *page __attribute__((unused)))
+{
+	return(FALSE);
+}
+
+/************************************************************************
+Run the next batch of pages through the compact page write filter.
+
+@return TRUE on success, FALSE on error. */
+static my_bool
+wf_compact_process(xb_write_filt_ctxt_t *ctxt, ds_file_t *dstfile)
+{
+	xb_fil_cur_t		*cursor = ctxt->cursor;
+	ulint			 page_size = cursor->page_size;
+	byte			*page;
+	byte			*buf_end;
+	byte			*write_from;
+	xb_wf_compact_ctxt_t	*cp = &(ctxt->u.wf_compact_ctxt);
+
+	if (cp->skip) {
+		return(!ds_write(dstfile, cursor->buf, cursor->buf_read));
+	}
+
+	write_from = NULL;
+	buf_end = cursor->buf + cursor->buf_read;
+	for (page = cursor->buf; page < buf_end; page += page_size) {
+		if (!check_if_skip_page(page)) {
+			if (write_from == NULL) {
+				write_from = page;
+			}
+			continue;
+		}
+	}
+
+	/* Write the remaining pages in the buffer, if any */
+	if (write_from != NULL &&
+	    ds_write(dstfile, write_from, buf_end - write_from)) {
+		return(FALSE);
+	}
+
+	return(TRUE);
+}
+
+/************************************************************************
+Close the compact write filter's page map stream.
+
+@return TRUE on success, FALSE on error. */
+static my_bool
+wf_compact_finalize(xb_write_filt_ctxt_t *ctxt, ds_file_t *dstfile)
+{
+	xb_fil_cur_t			*cursor = ctxt->cursor;
+	ulint				page_size = cursor->page_size;
+	xb_wf_incremental_ctxt_t	*cp = &(ctxt->u.wf_incremental_ctxt);
+
+	if (cp->npages != page_size / 4) {
+		mach_write_to_4(cp->delta_buf + cp->npages * 4, 0xFFFFFFFFUL);
+	}
+
+	mach_write_to_4(cp->delta_buf, 0x58545241UL); /*"XTRA"*/
+
+	/* flush buffer */
+	if (ds_write(dstfile, cp->delta_buf, cp->npages * page_size)) {
+		return(FALSE);
+	}
+
+	return(TRUE);
+}
 
=== added file 'src/write_filt.h'
--- src/write_filt.h	1970-01-01 00:00:00 +0000
+++ src/write_filt.h	2012-10-18 11:58:23 +0000
@@ -0,0 +1,65 @@
+/******************************************************
+XtraBackup: hot backup tool for InnoDB
+(c) 2009-2012 Percona Inc.
+Originally Created 3/3/2009 Yasufumi Kinoshita
+Written by Alexey Kopytov, Aleksandr Kuzminsky, Stewart Smith, Vadim Tkachenko,
+Yasufumi Kinoshita, Ignacio Nin and Baron Schwartz.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+*******************************************************/
+
+/* Page write filter interface */
+
+#ifndef XB_WRITE_FILT_H
+#define XB_WRITE_FILT_H
+
+#include "fil_cur.h"
+#include "datasink.h"
+
+/* Incremental page filter context */
+typedef struct {
+	byte		*delta_buf_base;
+	byte		*delta_buf;
+	ulint		 npages;
+} xb_wf_incremental_ctxt_t;
+
+/* Compact page filter context */
+typedef struct {
+	my_bool	skip;
+} xb_wf_compact_ctxt_t;
+
+/* Page filter context used as an opaque structure by callers */
+typedef struct {
+	xb_fil_cur_t	*cursor;
+	union {
+		xb_wf_incremental_ctxt_t	wf_incremental_ctxt;
+		xb_wf_compact_ctxt_t		wf_compact_ctxt;
+	} u;
+} xb_write_filt_ctxt_t;
+
+
+typedef struct {
+	my_bool	(*init)(xb_write_filt_ctxt_t *ctxt, char *dst_name,
+			xb_fil_cur_t *cursor);
+	my_bool	(*process)(xb_write_filt_ctxt_t *ctxt, ds_file_t *dstfile);
+	my_bool	(*finalize)(xb_write_filt_ctxt_t *, ds_file_t *dstfile);
+	void (*deinit)(xb_write_filt_ctxt_t *);
+} xb_write_filt_t;
+
+extern xb_write_filt_t wf_write_through;
+extern xb_write_filt_t wf_incremental;
+extern xb_write_filt_t wf_compact;
+
+#endif /* XB_WRITE_FILT_H */
 
=== modified file 'src/xbstream.c'
--- src/xbstream.c	2012-02-10 20:05:56 +0000
+++ src/xbstream.c	2012-10-18 11:58:23 +0000
@@ -23,7 +23,7 @@
 #include <hash.h>
 #include "common.h"
 #include "xbstream.h"
-#include "local.h"
+#include "ds_local.h"
 
 #define XBSTREAM_VERSION "1.0"
 #define XBSTREAM_BUFFER_SIZE (1024 * 1024UL)
@@ -36,6 +36,12 @@
 	RUN_MODE_EXTRACT
 } run_mode_t;
 
+/* Need the following definitions to avoid linking with ds_stream.o,
+ds_compress.o, ds_tmpfile.o and their linking dependencies */
+datasink_t datasink_stream;
+datasink_t datasink_compress;
+datasink_t datasink_tmpfile;
+
 static run_mode_t 	opt_mode;
 static char *		opt_directory = NULL;
 static my_bool		opt_verbose = 0;
@@ -61,7 +67,7 @@
 typedef struct {
 	char 		*path;
 	uint		pathlen;
-	off_t		offset;
+	my_off_t	offset;
 	ds_ctxt_t	*ds_ctxt;
 	ds_file_t	*file;
 } file_entry_t;
@@ -199,9 +205,7 @@
 	uchar	buf[XBSTREAM_BUFFER_SIZE];
 	size_t	bytes;
 
-#ifdef USE_POSIX_FADVISE
 	posix_fadvise(file, 0, 0, POSIX_FADV_SEQUENTIAL);
-#endif
 
 	while ((bytes = my_read(file, buf, XBSTREAM_BUFFER_SIZE,
 				MYF(MY_WME))) > 0) {
@@ -210,9 +214,7 @@
 			    my_progname);
 			return 1;
 		}
-#ifdef USE_POSIX_FADVISE
 		posix_fadvise(file, 0, 0, POSIX_FADV_DONTNEED);
-#endif
 
 	}
 
@@ -291,7 +293,6 @@
 file_entry_new(ds_ctxt_t *ds_ctxt, const char *path, uint pathlen)
 {
 	file_entry_t	*entry;
-	datasink_t	*ds = ds_ctxt->datasink;
 	ds_file_t	*file;
 
 	entry = (file_entry_t *) my_malloc(sizeof(file_entry_t),
@@ -306,7 +307,7 @@
 	}
 	entry->pathlen = pathlen;
 
-	file = ds->open(ds_ctxt, path, NULL);
+	file = ds_open(ds_ctxt, path, NULL);
 	if (file == NULL) {
 		msg("%s: failed to create file.\n", my_progname);
 		goto err;
@@ -343,10 +344,7 @@
 void
 file_entry_free(file_entry_t *entry)
 {
-	ds_ctxt_t	*ds_ctxt = entry->ds_ctxt;
-	datasink_t	*ds = ds_ctxt->datasink;
-
-	ds->close(entry->file);
+	ds_close(entry->file);
 	MY_FREE(entry->path);
 	MY_FREE(entry);
 }
@@ -354,14 +352,13 @@
 static
 int
 mode_extract(int argc __attribute__((unused)),
-	     char **argv __attribute((unused)))
+	     char **argv __attribute__((unused)))
 {
 	xb_rstream_t		*stream;
 	xb_rstream_result_t	res;
 	xb_rstream_chunk_t	chunk;
 	HASH			filehash;
 	file_entry_t		*entry;
-	datasink_t		*ds;
 	ds_ctxt_t		*ds_ctxt;
 
 	stream = xb_stream_read_new();
@@ -371,8 +368,7 @@
 	}
 
 	/* If --directory is specified, it is already set as CWD by now. */
-	ds = &datasink_local;
-	ds_ctxt = ds->init(".");
+	ds_ctxt = ds_create(".", DS_TYPE_LOCAL);
 
 	if (my_hash_init(&filehash, &my_charset_bin, START_FILE_HASH_SIZE,
 			  0, 0, (my_hash_get_key) get_file_entry_key,
@@ -420,7 +416,7 @@
 			goto err;
 		}
 
-		if (ds->write(entry->file, chunk.data, chunk.length)) {
+		if (ds_write(entry->file, chunk.data, chunk.length)) {
 			msg("%s: my_write() failed.\n", my_progname);
 			goto err;
 		}
@@ -433,13 +429,13 @@
 	}
 
 	my_hash_free(&filehash);
-	ds->deinit(ds_ctxt);
+	ds_destroy(ds_ctxt);
 	xb_stream_read_done(stream);
 
 	return 0;
 err:
 	my_hash_free(&filehash);
-	ds->deinit(ds_ctxt);
+	ds_destroy(ds_ctxt);
 	xb_stream_read_done(stream);
 
 	return 1;
 
=== modified file 'src/xtrabackup.c'
--- src/xtrabackup.c	2012-09-13 10:28:54 +0000
+++ src/xtrabackup.c	2012-10-18 11:58:23 +0000
@@ -48,9 +48,9 @@
 
 //#define XTRABACKUP_TARGET_IS_PLUGIN
 
+#include <mysql_version.h>
 #include <my_base.h>
 #include <my_getopt.h>
-#include <mysql_version.h>
 #include <mysql_com.h>
 
 #if (MYSQL_VERSION_ID < 50100)
@@ -59,8 +59,6 @@
 #define G_PTR uchar*
 #endif
 
-#include <univ.i>
-#include <os0file.h>
 #include <os0thread.h>
 #include <srv0start.h>
 #include <srv0srv.h>
@@ -72,8 +70,6 @@
 #include <row0mysql.h>
 #include <row0sel.h>
 #include <row0upd.h>
-#include <log0log.h>
-#include <log0recv.h>
 #include <lock0lock.h>
 #include <dict0crea.h>
 #include <btr0cur.h>
@@ -81,9 +77,7 @@
 #include <btr0sea.h>
 #include <fsp0fsp.h>
 #include <sync0sync.h>
-#include <fil0fil.h>
 #include <trx0xa.h>
-#include <btr0sea.h>
 #include <log0recv.h>
 #include <fcntl.h>
 
@@ -93,676 +87,12 @@
 
 #include "common.h"
 #include "datasink.h"
-#include "local.h"
-#include "stream.h"
-#include "compress.h"
 
 #include "xb_regex.h"
-
-#ifndef INNODB_VERSION_SHORT
-#define IB_INT64 ib_longlong
-#define LSN64 dulint
-#define MACH_READ_64 mach_read_from_8
-#define MACH_WRITE_64 mach_write_to_8
-#define OS_MUTEX_CREATE() os_mutex_create(NULL)
-#define PAGE_ZIP_MIN_SIZE_SHIFT	10
-#define DICT_TF_ZSSIZE_SHIFT	1
-#define DICT_TF_FORMAT_ZIP	1
-#define DICT_TF_FORMAT_SHIFT		5
-#else
-#define IB_INT64 ib_int64_t
-#define LSN64 ib_uint64_t
-#if (MYSQL_VERSION_ID < 50500)
-#define MACH_READ_64 mach_read_ull
-#define MACH_WRITE_64 mach_write_ull
-#define OS_MUTEX_CREATE() os_mutex_create(NULL)
-#else
-#define MACH_READ_64 mach_read_from_8
-#define MACH_WRITE_64 mach_write_to_8
-#define OS_MUTEX_CREATE() os_mutex_create()
-#endif
-#define ut_dulint_zero 0
-#define ut_dulint_cmp(A, B) (A > B ? 1 : (A == B ? 0 : -1))
-#define ut_dulint_add(A, B) (A + B)
-#define ut_dulint_minus(A, B) (A - B)
-#define ut_dulint_align_down(A, B) (A & ~((ib_int64_t)B - 1))
-#define ut_dulint_align_up(A, B) ((A + B - 1) & ~((ib_int64_t)B - 1))
-#endif
-
-#ifdef __WIN__
-#define SRV_PATH_SEPARATOR	'\\'
-#define SRV_PATH_SEPARATOR_STR	"\\"	
-#else
-#define SRV_PATH_SEPARATOR	'/'
-#define SRV_PATH_SEPARATOR_STR	"/"
-#endif
-
-#ifndef UNIV_PAGE_SIZE_MAX
-#define UNIV_PAGE_SIZE_MAX UNIV_PAGE_SIZE
-#endif
-#ifndef UNIV_PAGE_SIZE_SHIFT_MAX
-#define UNIV_PAGE_SIZE_SHIFT_MAX UNIV_PAGE_SIZE_SHIFT
-#endif
-
-#if MYSQL_VERSION_ID >= 50507
-/*
-   As of MySQL 5.5.7, InnoDB uses thd_wait plugin service.
-   We have to provide mock functions to avoid linker errors.
-*/
-#include <mysql/plugin.h>
-#include <mysql/service_thd_wait.h>
-
-void thd_wait_begin(MYSQL_THD thd, int wait_type)
-{
-	(void)thd;
-	(void)wait_type;
-	return;
-}
-
-void thd_wait_end(MYSQL_THD thd)
-{
-	(void)thd;
-	return;
-}
-
-#endif /* MYSQL_VERSION_ID >= 50507 */
-
-/* prototypes for static functions in original */
-#ifndef INNODB_VERSION_SHORT
-page_t*
-btr_node_ptr_get_child(
-/*===================*/
-				/* out: child page, x-latched */
-	rec_t*		node_ptr,/* in: node pointer */
-	const ulint*	offsets,/* in: array returned by rec_get_offsets() */
-	mtr_t*		mtr);	/* in: mtr */
-#else
-buf_block_t*
-btr_node_ptr_get_child(
-/*===================*/
-	const rec_t*	node_ptr,/*!< in: node pointer */
-	dict_index_t*	index,	/*!< in: index */
-	const ulint*	offsets,/*!< in: array returned by rec_get_offsets() */
-	mtr_t*		mtr);	/*!< in: mtr */
-
-buf_block_t*
-btr_root_block_get(
-/*===============*/
-	dict_index_t*	index,	/*!< in: index tree */
-	mtr_t*		mtr);	/*!< in: mtr */
-#endif
-
-int
-fil_file_readdir_next_file(
-/*=======================*/
-				/* out: 0 if ok, -1 if error even after the
-				retries, 1 if at the end of the directory */
-	ulint*		err,	/* out: this is set to DB_ERROR if an error
-				was encountered, otherwise not changed */
-	const char*	dirname,/* in: directory name or path */
-	os_file_dir_t	dir,	/* in: directory stream */
-	os_file_stat_t*	info);	/* in/out: buffer where the info is returned */
-
-ibool
-recv_check_cp_is_consistent(
-/*========================*/
-			/* out: TRUE if ok */
-	byte*	buf);	/* in: buffer containing checkpoint info */
-
-ulint
-recv_find_max_checkpoint(
-/*=====================*/
-					/* out: error code or DB_SUCCESS */
-	log_group_t**	max_group,	/* out: max group */
-	ulint*		max_field);	/* out: LOG_CHECKPOINT_1 or
-					LOG_CHECKPOINT_2 */
-
-ibool
-log_block_checksum_is_ok_or_old_format(
-/*===================================*/
-			/* out: TRUE if ok, or if the log block may be in the
-			format of InnoDB version < 3.23.52 */
-	byte*	block);	/* in: pointer to a log block */
-
-ulint
-open_or_create_log_file(
-/*====================*/
-					/* out: DB_SUCCESS or error code */
-        ibool   create_new_db,          /* in: TRUE if we should create a
-                                        new database */
-	ibool*	log_file_created,	/* out: TRUE if new log file
-					created */
-	ibool	log_file_has_been_opened,/* in: TRUE if a log file has been
-					opened before: then it is an error
-					to try to create another log file */
-	ulint	k,			/* in: log group number */
-	ulint	i);			/* in: log file number in group */
-
-ulint
-open_or_create_data_files(
-/*======================*/
-				/* out: DB_SUCCESS or error code */
-	ibool*	create_new_db,	/* out: TRUE if new database should be
-								created */
-#ifdef XTRADB_BASED
-	ibool*	create_new_doublewrite_file,
-#endif 
-#ifdef UNIV_LOG_ARCHIVE
-	ulint*	min_arch_log_no,/* out: min of archived log numbers in data
-				files */
-	ulint*	max_arch_log_no,/* out: */
-#endif /* UNIV_LOG_ARCHIVE */
-	LSN64*	min_flushed_lsn,/* out: min of flushed lsn values in data
-				files */
-	LSN64*	max_flushed_lsn,/* out: */
-	ulint*	sum_of_new_sizes);/* out: sum of sizes of the new files added */
-
-void
-os_file_set_nocache(
-/*================*/
-	int		fd,		/* in: file descriptor to alter */
-	const char*	file_name,	/* in: used in the diagnostic message */
-	const char*	operation_name);	/* in: used in the diagnostic message,
-					we call os_file_set_nocache()
-					immediately after opening or creating
-					a file, so this is either "open" or
-					"create" */
-
-/****************************************************************//**
-A simple function to open or create a file.
-@return own: handle to the file, not defined if error, error number
-can be retrieved with os_file_get_last_error */
-UNIV_INLINE
-os_file_t
-xb_file_create_no_error_handling(
-/*=============================*/
-	const char*	name,	/*!< in: name of the file or path as a
-				null-terminated string */
-	ulint		create_mode,/*!< in: OS_FILE_OPEN if an existing file
-				is opened (if does not exist, error), or
-				OS_FILE_CREATE if a new file is created
-				(if exists, error) */
-	ulint		access_type,/*!< in: OS_FILE_READ_ONLY,
-				OS_FILE_READ_WRITE, or
-				OS_FILE_READ_ALLOW_DELETE; the last option is
-				used by a backup program reading the file */
-	ibool*		success);/*!< out: TRUE if succeed, FALSE if error */
-
-/****************************************************************//**
-Opens an existing file or creates a new.
-@return own: handle to the file, not defined if error, error number
-can be retrieved with os_file_get_last_error */
-UNIV_INLINE
-os_file_t
-xb_file_create(
-/*===========*/
-	const char*	name,	/*!< in: name of the file or path as a
-				null-terminated string */
-	ulint		create_mode,/*!< in: OS_FILE_OPEN if an existing file
-				is opened (if does not exist, error), or
-				OS_FILE_CREATE if a new file is created
-				(if exists, error),
-				OS_FILE_OVERWRITE if a new file is created
-				or an old overwritten;
-				OS_FILE_OPEN_RAW, if a raw device or disk
-				partition should be opened */
-	ulint		purpose,/*!< in: OS_FILE_AIO, if asynchronous,
-				non-buffered i/o is desired,
-				OS_FILE_NORMAL, if any normal file;
-				NOTE that it also depends on type, os_aio_..
-				and srv_.. variables whether we really use
-				async i/o or unbuffered i/o: look in the
-				function source code for the exact rules */
-	ulint		type,	/*!< in: OS_DATA_FILE or OS_LOG_FILE */
-	ibool*		success);/*!< out: TRUE if succeed, FALSE if error */
-
-
-/***********************************************************************//**
-Renames a file (can also move it to another directory). It is safest that the
-file is closed before calling this function.
-@return	TRUE if success */
-UNIV_INLINE
-ibool
-xb_file_rename(
-/*===========*/
-	const char*	oldpath,/*!< in: old file path as a null-terminated
-				string */
-	const char*	newpath);/*!< in: new file path */
-
-UNIV_INLINE
-void
-xb_file_set_nocache(
-/*================*/
-	os_file_t	fd,		/* in: file descriptor to alter */
-	const char*	file_name,	/* in: used in the diagnostic message */
-	const char*	operation_name);/* in: used in the diagnostic message,
-					we call os_file_set_nocache()
-					immediately after opening or creating
-					a file, so this is either "open" or
-					"create" */
-
-/***********************************************************************//**
-Compatibility wrapper around os_file_flush().
-@return	TRUE if success */
-static
-ibool
-xb_file_flush(
-/*==========*/
-	os_file_t	file);	/*!< in, own: handle to a file */
-
-/* ==start === definition at fil0fil.c === */
-// ##################################################################
-// NOTE: We should check the following definitions fit to the source.
-// ##################################################################
-
-#ifndef INNODB_VERSION_SHORT
-//5.0 5.1
-/* File node of a tablespace or the log data space */
-struct fil_node_struct {
-        fil_space_t*    space;  /* backpointer to the space where this node
-                                belongs */
-        char*           name;   /* path to the file */
-        ibool           open;   /* TRUE if file open */
-        os_file_t       handle; /* OS handle to the file, if file open */
-        ibool           is_raw_disk;/* TRUE if the 'file' is actually a raw
-                                device or a raw disk partition */
-        ulint           size;   /* size of the file in database pages, 0 if
-                                not known yet; the possible last incomplete
-                                megabyte may be ignored if space == 0 */
-        ulint           n_pending;
-                                /* count of pending i/o's on this file;
-                                closing of the file is not allowed if
-                                this is > 0 */
-        ulint           n_pending_flushes;
-                                /* count of pending flushes on this file;
-                                closing of the file is not allowed if
-                                this is > 0 */
-        ib_longlong     modification_counter;/* when we write to the file we
-                                increment this by one */
-        ib_longlong     flush_counter;/* up to what modification_counter value
-                                we have flushed the modifications to disk */
-        UT_LIST_NODE_T(fil_node_t) chain;
-                                /* link field for the file chain */
-        UT_LIST_NODE_T(fil_node_t) LRU;
-                                /* link field for the LRU list */
-        ulint           magic_n;
-};
-
-struct fil_space_struct {
-        char*           name;   /* space name = the path to the first file in
-                                it */
-        ulint           id;     /* space id */
-        ib_longlong     tablespace_version;
-                                /* in DISCARD/IMPORT this timestamp is used to
-                                check if we should ignore an insert buffer
-                                merge request for a page because it actually
-                                was for the previous incarnation of the
-                                space */
-        ibool           mark;   /* this is set to TRUE at database startup if
-                                the space corresponds to a table in the InnoDB
-                                data dictionary; so we can print a warning of
-                                orphaned tablespaces */
-        ibool           stop_ios;/* TRUE if we want to rename the .ibd file of
-                                tablespace and want to stop temporarily
-                                posting of new i/o requests on the file */
-        ibool           stop_ibuf_merges;
-                                /* we set this TRUE when we start deleting a
-                                single-table tablespace */
-        ibool           is_being_deleted;
-                                /* this is set to TRUE when we start
-                                deleting a single-table tablespace and its
-                                file; when this flag is set no further i/o
-                                or flush requests can be placed on this space,
-                                though there may be such requests still being
-                                processed on this space */
-        ulint           purpose;/* FIL_TABLESPACE, FIL_LOG, or FIL_ARCH_LOG */
-        UT_LIST_BASE_NODE_T(fil_node_t) chain;
-                                /* base node for the file chain */
-        ulint           size;   /* space size in pages; 0 if a single-table
-                                tablespace whose size we do not know yet;
-                                last incomplete megabytes in data files may be
-                                ignored if space == 0 */
-        ulint           n_reserved_extents;
-                                /* number of reserved free extents for
-                                ongoing operations like B-tree page split */
-        ulint           n_pending_flushes; /* this is > 0 when flushing
-                                the tablespace to disk; dropping of the
-                                tablespace is forbidden if this is > 0 */
-        ulint           n_pending_ibuf_merges;/* this is > 0 when merging
-                                insert buffer entries to a page so that we
-                                may need to access the ibuf bitmap page in the
-                                tablespade: dropping of the tablespace is
-                                forbidden if this is > 0 */
-        hash_node_t     hash;   /* hash chain node */
-        hash_node_t     name_hash;/* hash chain the name_hash table */
-        rw_lock_t       latch;  /* latch protecting the file space storage
-                                allocation */
-        UT_LIST_NODE_T(fil_space_t) unflushed_spaces;
-                                /* list of spaces with at least one unflushed
-                                file we have written to */
-        ibool           is_in_unflushed_spaces; /* TRUE if this space is
-                                currently in the list above */
-        UT_LIST_NODE_T(fil_space_t) space_list;
-                                /* list of all spaces */
-        ibuf_data_t*    ibuf_data;
-                                /* insert buffer data */
-        ulint           magic_n;
-};
-typedef struct fil_system_struct        fil_system_t;
-struct fil_system_struct {
-        mutex_t         mutex;          /* The mutex protecting the cache */
-        hash_table_t*   spaces;         /* The hash table of spaces in the
-                                        system; they are hashed on the space
-                                        id */
-        hash_table_t*   name_hash;      /* hash table based on the space
-                                        name */
-        UT_LIST_BASE_NODE_T(fil_node_t) LRU;
-                                        /* base node for the LRU list of the
-                                        most recently used open files with no
-                                        pending i/o's; if we start an i/o on
-                                        the file, we first remove it from this
-                                        list, and return it to the start of
-                                        the list when the i/o ends;
-                                        log files and the system tablespace are
-                                        not put to this list: they are opened
-                                        after the startup, and kept open until
-                                        shutdown */
-        UT_LIST_BASE_NODE_T(fil_space_t) unflushed_spaces;
-                                        /* base node for the list of those
-                                        tablespaces whose files contain
-                                        unflushed writes; those spaces have
-                                        at least one file node where
-                                        modification_counter > flush_counter */
-        ulint           n_open;         /* number of files currently open */
-        ulint           max_n_open;     /* n_open is not allowed to exceed
-                                        this */
-        ib_longlong     modification_counter;/* when we write to a file we
-                                        increment this by one */
-        ulint           max_assigned_id;/* maximum space id in the existing
-                                        tables, or assigned during the time
-                                        mysqld has been up; at an InnoDB
-                                        startup we scan the data dictionary
-                                        and set here the maximum of the
-                                        space id's of the tables there */
-        ib_longlong     tablespace_version;
-                                        /* a counter which is incremented for
-                                        every space object memory creation;
-                                        every space mem object gets a
-                                        'timestamp' from this; in DISCARD/
-                                        IMPORT this is used to check if we
-                                        should ignore an insert buffer merge
-                                        request */
-        UT_LIST_BASE_NODE_T(fil_space_t) space_list;
-                                        /* list of all file spaces */
-};
-#else
-//Plugin ?
-/** File node of a tablespace or the log data space */
-struct fil_node_struct {
-	fil_space_t*	space;	/*!< backpointer to the space where this node
-				belongs */
-	char*		name;	/*!< path to the file */
-	ibool		open;	/*!< TRUE if file open */
-	os_file_t	handle;	/*!< OS handle to the file, if file open */
-	ibool		is_raw_disk;/*!< TRUE if the 'file' is actually a raw
-				device or a raw disk partition */
-	ulint		size;	/*!< size of the file in database pages, 0 if
-				not known yet; the possible last incomplete
-				megabyte may be ignored if space == 0 */
-	ulint		n_pending;
-				/*!< count of pending i/o's on this file;
-				closing of the file is not allowed if
-				this is > 0 */
-	ulint		n_pending_flushes;
-				/*!< count of pending flushes on this file;
-				closing of the file is not allowed if
-				this is > 0 */
-	ib_int64_t	modification_counter;/*!< when we write to the file we
-				increment this by one */
-	ib_int64_t	flush_counter;/*!< up to what
-				modification_counter value we have
-				flushed the modifications to disk */
-	UT_LIST_NODE_T(fil_node_t) chain;
-				/*!< link field for the file chain */
-	UT_LIST_NODE_T(fil_node_t) LRU;
-				/*!< link field for the LRU list */
-	ulint		magic_n;/*!< FIL_NODE_MAGIC_N */
-};
-
-struct fil_space_struct {
-	char*		name;	/*!< space name = the path to the first file in
-				it */
-	ulint		id;	/*!< space id */
-	ib_int64_t	tablespace_version;
-				/*!< in DISCARD/IMPORT this timestamp
-				is used to check if we should ignore
-				an insert buffer merge request for a
-				page because it actually was for the
-				previous incarnation of the space */
-	ibool		mark;	/*!< this is set to TRUE at database startup if
-				the space corresponds to a table in the InnoDB
-				data dictionary; so we can print a warning of
-				orphaned tablespaces */
-	ibool		stop_ios;/*!< TRUE if we want to rename the
-				.ibd file of tablespace and want to
-				stop temporarily posting of new i/o
-				requests on the file */
-	ibool		stop_ibuf_merges;
-				/*!< we set this TRUE when we start
-				deleting a single-table tablespace */
-	ibool		is_being_deleted;
-				/*!< this is set to TRUE when we start
-				deleting a single-table tablespace and its
-				file; when this flag is set no further i/o
-				or flush requests can be placed on this space,
-				though there may be such requests still being
-				processed on this space */
-	ulint		purpose;/*!< FIL_TABLESPACE, FIL_LOG, or
-				FIL_ARCH_LOG */
-	UT_LIST_BASE_NODE_T(fil_node_t) chain;
-				/*!< base node for the file chain */
-	ulint		size;	/*!< space size in pages; 0 if a single-table
-				tablespace whose size we do not know yet;
-				last incomplete megabytes in data files may be
-				ignored if space == 0 */
-	ulint		flags;	/*!< compressed page size and file format, or 0 */
-	ulint		n_reserved_extents;
-				/*!< number of reserved free extents for
-				ongoing operations like B-tree page split */
-	ulint		n_pending_flushes; /*!< this is positive when flushing
-				the tablespace to disk; dropping of the
-				tablespace is forbidden if this is positive */
-	ulint		n_pending_ibuf_merges;/*!< this is positive
-				when merging insert buffer entries to
-				a page so that we may need to access
-				the ibuf bitmap page in the
-				tablespade: dropping of the tablespace
-				is forbidden if this is positive */
-	hash_node_t	hash;	/*!< hash chain node */
-	hash_node_t	name_hash;/*!< hash chain the name_hash table */
-#ifndef UNIV_HOTBACKUP
-	rw_lock_t	latch;	/*!< latch protecting the file space storage
-				allocation */
-#endif /* !UNIV_HOTBACKUP */
-	UT_LIST_NODE_T(fil_space_t) unflushed_spaces;
-				/*!< list of spaces with at least one unflushed
-				file we have written to */
-	ibool		is_in_unflushed_spaces; /*!< TRUE if this space is
-				currently in unflushed_spaces */
-#ifdef XTRADB_BASED
-	ibool		is_corrupt;
-#endif
-	UT_LIST_NODE_T(fil_space_t) space_list;
-				/*!< list of all spaces */
-	ulint		magic_n;/*!< FIL_SPACE_MAGIC_N */
-};
-
-typedef	struct fil_system_struct	fil_system_t;
-
-struct fil_system_struct {
-#ifndef UNIV_HOTBACKUP
-	mutex_t		mutex;		/*!< The mutex protecting the cache */
-#ifdef XTRADB_BASED
-	mutex_t		file_extend_mutex;
-#endif
-#endif /* !UNIV_HOTBACKUP */
-	hash_table_t*	spaces;		/*!< The hash table of spaces in the
-					system; they are hashed on the space
-					id */
-	hash_table_t*	name_hash;	/*!< hash table based on the space
-					name */
-	UT_LIST_BASE_NODE_T(fil_node_t) LRU;
-					/*!< base node for the LRU list of the
-					most recently used open files with no
-					pending i/o's; if we start an i/o on
-					the file, we first remove it from this
-					list, and return it to the start of
-					the list when the i/o ends;
-					log files and the system tablespace are
-					not put to this list: they are opened
-					after the startup, and kept open until
-					shutdown */
-	UT_LIST_BASE_NODE_T(fil_space_t) unflushed_spaces;
-					/*!< base node for the list of those
-					tablespaces whose files contain
-					unflushed writes; those spaces have
-					at least one file node where
-					modification_counter > flush_counter */
-	ulint		n_open;		/*!< number of files currently open */
-	ulint		max_n_open;	/*!< n_open is not allowed to exceed
-					this */
-	ib_int64_t	modification_counter;/*!< when we write to a file we
-					increment this by one */
-	ulint		max_assigned_id;/*!< maximum space id in the existing
-					tables, or assigned during the time
-					mysqld has been up; at an InnoDB
-					startup we scan the data dictionary
-					and set here the maximum of the
-					space id's of the tables there */
-	ib_int64_t	tablespace_version;
-					/*!< a counter which is incremented for
-					every space object memory creation;
-					every space mem object gets a
-					'timestamp' from this; in DISCARD/
-					IMPORT this is used to check if we
-					should ignore an insert buffer merge
-					request */
-	UT_LIST_BASE_NODE_T(fil_space_t) space_list;
-					/*!< list of all file spaces */
-	ibool		space_id_reuse_warned;
-					/* !< TRUE if fil_space_create()
-					has issued a warning about
-					potential space_id reuse */
-};
-
-#endif /* INNODB_VERSION_SHORT */
-
-typedef struct {
-	ulint	page_size;
-	ulint	zip_size;
-	ulint	space_id;
-} xb_delta_info_t;
-
-extern fil_system_t*   fil_system;
-
-/** Value of fil_space_struct::magic_n */
-#define	FIL_SPACE_MAGIC_N	89472
-
-/*******************************************************************//**
-Returns the table space by a given id, NULL if not found. */
-fil_space_t*
-xb_space_get_by_id(
-/*================*/
-	ulint	id);	/*!< in: space id */
-
-/*******************************************************************//**
-Returns the table space by a given name, NULL if not found. */
-fil_space_t*
-xb_space_get_by_name(
-/*==================*/
-	const char*	name);	/*!< in: space name */
-
-/*******************************************************************//**
-Returns the table space by a given id, NULL if not found. */
-fil_space_t*
-xb_space_get_by_id(
-/*================*/
-	ulint	id)	/*!< in: space id */
-{
-	fil_space_t*	space;
-
-	ut_ad(mutex_own(&fil_system->mutex));
-
-#ifdef INNODB_VERSION_SHORT
-	HASH_SEARCH(hash, fil_system->spaces, id,
-		    fil_space_t*, space,
-		    ut_ad(space->magic_n == FIL_SPACE_MAGIC_N),
-		    space->id == id);
-#else
-	HASH_SEARCH(hash, fil_system->spaces, id, space, space->id == id);
-#endif
-
-	return(space);
-}
-
-/*******************************************************************//**
-Returns the table space by a given name, NULL if not found. */
-fil_space_t*
-xb_space_get_by_name(
-/*==================*/
-	const char*	name)	/*!< in: space name */
-{
-	fil_space_t*	space;
-	ulint		fold;
-
-	ut_ad(mutex_own(&fil_system->mutex));
-
-#ifdef INNODB_VERSION_SHORT
-	fold = ut_fold_string(name);
-	HASH_SEARCH(name_hash, fil_system->name_hash, fold,
-		    fil_space_t*, space,
-		    ut_ad(space->magic_n == FIL_SPACE_MAGIC_N),
-		    !strcmp(name, space->name));
-#else
-	HASH_SEARCH(name_hash, fil_system->name_hash, ut_fold_string(name),
-		    space, 0 == strcmp(name, space->name));
-#endif
-
-	return(space);
-}
-
-#ifndef INNODB_VERSION_SHORT
-
-/*******************************************************************//**
-Free all spaces in space_list. */
-void
-fil_free_all_spaces(void)
-/*=====================*/
-{
-	fil_space_t*	space;
-
-	mutex_enter(&fil_system->mutex);
-
-	space = UT_LIST_GET_FIRST(fil_system->space_list);
-
-	while (space != NULL) {
-		fil_node_t*	node;
-		fil_space_t*	prev_space = space;
-
-		space = UT_LIST_GET_NEXT(space_list, space);
-
-		fil_space_free(prev_space->id, FALSE);
-	}
-
-	mutex_exit(&fil_system->mutex);
-}
-
-#define SRV_SHUTDOWN_NONE 0
-
-#endif
-
-/* ==end=== definition  at fil0fil.c === */
-
+#include "innodb_int.h"
+#include "fil_cur.h"
+#include "write_filt.h"
+#include "xtrabackup.h"
 
 my_bool innodb_inited= 0;
 
@@ -786,7 +116,6 @@
 lint io_ticket;
 os_event_t wait_throttle = NULL;
 
-my_bool xtrabackup_log_only = FALSE;
 char *xtrabackup_incremental = NULL;
 LSN64 incremental_lsn;
 LSN64 incremental_to_lsn;
@@ -850,27 +179,15 @@
 ib_uint64_t metadata_last_lsn = 0;
 #endif
 
-#define XB_DELTA_INFO_SUFFIX ".meta"
-
 #define XB_LOG_FILENAME "xtrabackup_logfile"
 
-#ifdef __WIN__
-#define XB_FILE_UNDEFINED NULL
-#else
-#define XB_FILE_UNDEFINED (-1)
-#endif
-
-int	dst_log_fd = XB_FILE_UNDEFINED;
-char	dst_log_path[FN_REFLEN];
+ds_file_t	*dst_log_file = NULL;
 
 /* === some variables from mysqld === */
 char mysql_real_data_home[FN_REFLEN] = "./";
 char *mysql_data_home= mysql_real_data_home;
 static char mysql_data_home_buff[2];
 
-char *opt_mysql_tmpdir = NULL;
-MY_TMPDIR mysql_tmpdir_list;
-
 const char *defaults_group = "mysqld";
 
 /* === static parameters in ha_innodb.cc */
@@ -897,7 +214,9 @@
 long innobase_open_files = 300L;
 
 long innobase_page_size = (1 << 14); /* 16KB */
+#ifdef XTRADB_BASED
 static ulong innobase_log_block_size = 512;
+#endif
 my_bool innobase_fast_checksum = FALSE;
 my_bool	innobase_extra_undoslots = FALSE;
 char*	innobase_doublewrite_file = NULL;
@@ -944,6 +263,16 @@
 
 static char *xtrabackup_debug_sync = NULL;
 
+static my_bool xtrabackup_compact = FALSE;
+
+/* Datasinks */
+static  ds_ctxt_t       *ds_data     = NULL;
+static  ds_ctxt_t       *ds_meta     = NULL;
+static  ds_ctxt_t       *ds_local    = NULL;
+static  ds_ctxt_t       *ds_compress = NULL;
+static  ds_ctxt_t       *ds_tmpfile  = NULL;
+static  ds_ctxt_t       *ds_stream   = NULL;
+
 /* ======== Datafiles iterator ======== */
 typedef struct {
 	fil_system_t *system;
@@ -1024,7 +353,6 @@
 	uint			*count;
 	os_mutex_t		count_mutex;
 	os_thread_id_t		id;
-	ds_ctxt_t		*ds_ctxt;
 } data_thread_ctxt_t;
 
 /* ======== for option and variables ======== */
@@ -1041,7 +369,6 @@
   OPT_XTRA_SUSPEND_AT_END,
   OPT_XTRA_USE_MEMORY,
   OPT_XTRA_THROTTLE,
-  OPT_XTRA_LOG_ONLY,
   OPT_XTRA_INCREMENTAL,
   OPT_XTRA_INCREMENTAL_BASEDIR,
   OPT_XTRA_EXTRA_LSNDIR,
@@ -1099,6 +426,7 @@
   OPT_INNODB_THREAD_CONCURRENCY,
   OPT_INNODB_THREAD_SLEEP_DELAY,
   OPT_XTRA_DEBUG_SYNC,
+  OPT_XTRA_COMPACT,