Merge lp:~sergei.glushchenko/percona-server/ps56-univ-log-archive into lp:percona-server/5.6

Proposed by Sergei Glushchenko
Status: Merged
Approved by: Alexey Kopytov
Approved revision: 324
Merged at revision: 326
Proposed branch: lp:~sergei.glushchenko/percona-server/ps56-univ-log-archive
Merge into: lp:percona-server/5.6
Prerequisite: lp:~sergei.glushchenko/percona-server/56-bug1157078-bug1157075
Diff against target: 2357 lines (+1394/-155)
43 files modified
Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_check.inc (+15/-0)
Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_cleanup.inc (+15/-0)
Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_count.inc (+9/-0)
Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_last_file.inc (+24/-0)
Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_setup.inc (+49/-0)
Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_stat.inc (+72/-0)
Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_workload.inc (+5/-0)
Percona-Server/mysql-test/suite/innodb/r/pecona_bug1155475.result (+23/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_innodb_log_archive_func.result (+24/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_log_arch_expire_sec.result (+27/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_log_arch_func.result (+21/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_purge_archived_logs_before.result (+37/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_purge_archived_logs_to.result (+35/-0)
Percona-Server/mysql-test/suite/innodb/t/pecona_bug1155475.test (+30/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_innodb_log_archive_func.test (+31/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_log_arch_expire_sec.test (+29/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_log_arch_func.test (+19/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_purge_archived_logs_before.test (+43/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_purge_archived_logs_to.test (+39/-0)
Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_arch_dir_basic.result (+38/-0)
Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_arch_expire_sec_basic.result (+38/-0)
Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_archive_basic.result (+38/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_arch_dir_basic.test (+60/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_arch_expire_sec_basic.test (+60/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_archive_basic.test (+61/-0)
Percona-Server/sql/handler.cc (+42/-0)
Percona-Server/sql/handler.h (+8/-0)
Percona-Server/sql/lex.h (+1/-0)
Percona-Server/sql/mysqld.cc (+2/-0)
Percona-Server/sql/sql_cmd.h (+2/-0)
Percona-Server/sql/sql_parse.cc (+43/-0)
Percona-Server/sql/sql_yacc.yy (+19/-1)
Percona-Server/storage/innobase/fil/fil0fil.cc (+34/-33)
Percona-Server/storage/innobase/handler/ha_innodb.cc (+78/-8)
Percona-Server/storage/innobase/include/fil0fil.h (+8/-6)
Percona-Server/storage/innobase/include/log0log.h (+20/-7)
Percona-Server/storage/innobase/include/log0recv.h (+2/-2)
Percona-Server/storage/innobase/include/srv0srv.h (+14/-0)
Percona-Server/storage/innobase/include/univ.i (+3/-0)
Percona-Server/storage/innobase/log/log0log.cc (+101/-62)
Percona-Server/storage/innobase/log/log0recv.cc (+18/-16)
Percona-Server/storage/innobase/srv/srv0srv.cc (+140/-0)
Percona-Server/storage/innobase/srv/srv0start.cc (+17/-20)
To merge this branch: bzr merge lp:~sergei.glushchenko/percona-server/ps56-univ-log-archive
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Approve
Review via email: mp+154428@code.launchpad.net

This proposal supersedes a proposal from 2013-02-10.

Description of the change

Implementation of log archiving based on branch by KAlexey
which enables UNIV_LOG_ARCHIVE
* variables names and startup parameters have been changed
  from InnoDB original to be same as were used in previous
  implementation
* archive log files naming has been changed from original
  zero-based sequence (0, 1, 2, 3) to sequence of start lsn
  of archived file. Also prefix has been changed to be
  the same as in previous implementation
* purge commands have been implemented
  (PURGE ARCHIVED LOGS BEFORE/TO)
* innodb_xtra_expire_archive_logs_sec has been implemented
* innodb_xtra_log_archive_throttle has not been implemented
  as not applicable
* testcase has been added (modified from original, as purge
  commands do not delete current archive in progress)

To post a comment you must log in.
Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote : Posted in a previous version of this proposal
Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote : Posted in a previous version of this proposal

Will post couple of questions here
1. This implementation is different from described in https://blueprints.launchpad.net/percona-server/+spec/log-archiving, do we need to disapprove this blueprint and create new one, or just change existing one to describe current implementation?
2. This "I would like to have corresponding binary log position for log archive.
This will allow to have point in time recovery with binary logs." has not been implemented in original implementation and there is no comments about it in previous MP. Do we need this feature?
3. Do we need variable (like have_xtra_log_archiving) to indicate that server has archiving support?
4. Do we really need to be able enable/disable archiving at runtime as BP above describes? We should handle partially written archives somehow in this case.

Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

Hi Sergei,

On Mon, Feb 11 2013 14:17:20 +0400, Sergei Glushchenko wrote:

> Will post couple of questions here
> 1. This implementation is different from described in https://blueprints.launchpad.net/percona-server/+spec/log-archiving, do we need to disapprove this blueprint and create new one, or just change existing one to describe current implementation?

I think just changing the existing one is fine.

> 2. This "I would like to have corresponding binary log position for log archive.
> This will allow to have point in time recovery with binary logs." has not been implemented in original implementation and there is no comments about it in previous MP. Do we need this feature?

Yes, was probably left out from the original implementation. How hard
would it be to implement it?

> 3. Do we need variable (like have_xtra_log_archiving) to indicate that
> server has archiving support?

I don't think so.

> 4. Do we really need to be able enable/disable archiving at runtime as BP above describes? We should handle partially written archives somehow in this case.

I think ability to control it at runtime is good. Partially written
files and gaps in archived files are IMHO a user's problem, i.e. there's
not much _we_ can do to _handle_ them.

--
Best regards,
Alexey.

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote : Posted in a previous version of this proposal

Alexey,

It's not clear to me how to obtain corresponding binlog position for the LSN at the start of archive. InnoDB know binlog position at transaction commit. And it possible to obtain current binlog position by invoking thd_binlog_pos, but the problem here is that at the moment when we write archived file current LSN might be much further than LSN we write into archive.

Is there any other way to get this?

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote : Posted in a previous version of this proposal

http://jenkins.percona.com/view/PS%205.6/job/percona-server-5.6-param/41/

  Implementation of log archiving based on branch by KAlexey
  which enables UNIV_LOG_ARCHIVE
  * variables and startup parameters have following names:
    - innodb_log_archive = ON/OFF
    - innodb_log_arch_dir = DIR
    - innodb_log_arch_expire_sec = N
  * archive log files naming has been changed from original
    zero-based sequence (0, 1, 2, 3) to sequence of start lsn
    of archived file. Also prefix has been changed to be
    the same as in previous implementation
  * purge commands have been implemented
    (PURGE ARCHIVED LOGS BEFORE/TO)
  * innodb_log_arch_expire_sec has been implemented
  * innodb_xtra_log_archive_throttle has not been implemented
    as it is not applicable
  * testcases were implemented

Revision history for this message
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal

Sergey, it seems percona_log_archiving-master.opt is needless because there is no corresponding .test-.result files and at least one option in the .opt file is wrong.

Revision history for this message
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal

It's just partial review:

1. Unused local variable space_arch_log_no in fil_open_single_table_tablespace();
2. I would use "#define IB_ARCHIVED_LOGS_PREFIX_LEN (sizeof(IB_ARCHIVED_LOGS_PREFIX) - 1)" instead of using the number;
3. I don't think it's a good idea to abort the server if the file name does not fit into the buffer in log_archived_file_name_gen(). We could use strncpy() and snprintf() instead of strcpy() and sprintf() and return some value that shows if operation is completed successfully. If it does not some warning should be emitted.

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote : Posted in a previous version of this proposal

Vlad,

What behavior do you propose for case 3? Server obviously cannot continue operation with given settings. Should log_archiving be turned off (silently or with warning issued to error log)? How can it will be visible to DBA (logs aren't being archived)?

Revision history for this message
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal

> Vlad,
>
> What behavior do you propose for case 3? Server obviously cannot continue
> operation with given settings. Should log_archiving be turned off (silently or
> with warning issued to error log)? How can it will be visible to DBA (logs
> aren't being archived)?

Logs archiving - its a backup feature. I think server must not stop if backup does not work. There should be error in error log with stopped log archiving.

Revision history for this message
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal
Download full text (4.3 KiB)

I'm testing the latest xtrabackup(lp:percona-xtrabackup) with this server and I have the following issue when I try to do backup:

xtrabackup: uses posix_fadvise().
xtrabackup: cd to /tmp/1/src/data
xtrabackup: Target instance is assumed as followings.
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 2097152
130313 0:06:46 InnoDB: Assertion failure in thread 140183767389952 in file fil0fil.c line 5519
InnoDB: Failing assertion: byte_offset % OS_FILE_LOG_BLOCK_SIZE == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.

I use the following test script to repeat it:
#!/bin/bash

BASE_DIR=/tmp/1/percona-current
DATA_DIR=/tmp/1/src/data
ARCHIVED_LOGS_DIR=/tmp/1/src/archived_logs
BACKUP_DIR=/tmp/1/src/backup
DST_DIR=/tmp/1/dst
XTRABACKUP_DIR=.
XTRABACKUP=$XTRABACKUP_DIR/xtrabackup_55
USER=some_user
EXTRA_OPTIONS="--innodb-file-per-table --innodb_log_file_size=2M"
SERVER_OPTIONS="--innodb_log_archive=ON --innodb_log_arch_dir=$ARCHIVED_LOGS_DIR"
LD_LIBRARY_PATH=$BASE_DIR/lib/mysql:$LD_LIBRARY_PATH
INSTALL_DB=$BASE_DIR/scripts/mysql_install_db
MYSQLD=$BASE_DIR/bin/mysqld
MYSQL=$BASE_DIR/bin/mysql

mkdir -p $DATA_DIR $ARCHIVED_LOGS_DIR $BACKUP_DIR $DST_DIR
rm -rf $DATA_DIR/* $ARCHIVED_LOGS_DIR/* $BACKUP_DIR/* $DST_DIR/*

$INSTALL_DB --no-defaults --basedir=$BASE_DIR --datadir=$DATA_DIR --user=$USER $EXTRA_OPTIONS
PID1=$!
sleep 2
echo '--------------------------after start-----------------------------'
                               ...

Read more...

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote : Posted in a previous version of this proposal

Vlad,

So we shouldn't probably fail in case if we cannot write to create new archive file (os_file_create is failed), any problems with IO and so on, which could lead to some nontrivial changes of InnoDB code. Your thoughts?

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote : Posted in a previous version of this proposal

Xtradb based version are failed with same assertion, however xtrabackup_plugin seems to able to perform backup.

Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

   - multiple lines violate the 80 chars limit, try:

: bzr diff -c-1 | grep -0 '^+ ' | awk 'length > 82'

   - there's still no reply in http://bugs.mysql.com/bug.php?id=68635
     We obviously cannot afford similar changes for every new statement
     type, so we should do something about it. What about adding
     performance_schema_max_statement_classes = 167 to
     mysql-test/include/default_mysqld.cnf?

   - the changes related to digest values is another "gift" from
     performance schema. Not sure what to do about it. Probably just
     "--replace-column ... <digest>" in the corresponding tests?

  - please use OS_FILE_MAX_PATH instead of hard-coded (and
    inconsistent) constants in log_group_archive() and
    log_group_recover_from_archive_file(). I don't think it then makes
    sense to assert on sufficient buffer space, as a lot of things
    would break if paths are longer than that. But you can keep the
    assertion if you like. The rest of InnoDB code seems to assume that
    never happens, but I agree we should fail *if* it does.

review: Needs Fixing
Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :
Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

Failure of XtraDB based xtrabackup binaries is a result of change which comes from innodb_files_extend.patch. This patch uses LOG_CHECKPOINT_ARCHIVED_LSN to store LSN offset of checkpoint within the group, assuming that "UNIV_LOG_ARCHIVE could not be enabled". So this feature is incompatible with any XtraDB based XtraBackup binary. It also should be taken into account when feature will be ported to 5.6.

Revision history for this message
Alexey Kopytov (akopytov) wrote :

The part of innodb_files_extend.patch that conflicts with UNIV_LOG_ARCHIVE is support for log files > 4G and thus will not be ported to 5.6, because it is implemented in upstream 5.6, but in a different way and without breaking UNIV_LOG_ARCHIVE.

Revision history for this message
Alexey Kopytov (akopytov) wrote :

Sergei,

   Summary of our IRC discussion for the record:
   - test case for bug #1155475
   - unified return types in sql/handler.cc
   - error on bogus filename in innobase_purge_archive_logs()

   Other comments:

   - archived_file_no / next_archived_file_no are now LSNs rather that
     ordinal file numbers. Which means their type should be lsn_t, not
     ulint. The same for 'file_no' argument f
     log_archived_file_name_gen().

   - the assertion and sprintf() in log_archived_file_name_gen() seem to
     assume the maximum length for a decimal representation of file_no
     is 16 characters, but it is a 64-bit integer, so the maximum length
     is 20 characters.

   - the following line in log_archived_file_name_gen() still is still
     longer than 80 chars:

: sprintf(buf + dirnamelen, IB_ARCHIVED_LOGS_PREFIX "%016lu", (ulong) file_no);

   - the following line in log_group_archive() is still longer than 80
     chars:

: n_files * (group->file_size - LOG_FILE_HDR_SIZE),

review: Needs Fixing
Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

There is an issue with archived file number and archived file offset. These values are written to checkpoint as 4-byte values, however they are naturally 64-bit (type of archived_offset/next_archived_offset should be changed to lsn_t as well).
Possible solution is not to write archived file number to archived file header and to checkpoint. Archived file number is the same as first LSN for archived log file.
I'm going to implement this and look how it goes.

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

Unfortunately archived file number have to be written to checkpoint. It cannot be calculated using checkpoint LSN as archive may be start by user at any time starting with any LSN. So I introduced two additional 4-byte fields in checkpoint to hold high 4-byte values for archive file number and archive file offset

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

Alexey,

This is what have been done to address your comments and issues
above

* type of archived_file_no, archived_offset, next_archived_file_no,
  next_archived_offset members of log_group_t has been changed to
  lsn_t
* two additional fields in chechpoint record have been introduced
  to store high 4 bytes of archived file number and offset in
  archived file
* numeric part of archived log file name has been changed to be
  20 characters width, zero padded
* testcase for bug #1155475 has been added
* return types has been changed to my_bool for MySQL handlers,
  and to dberr_t for InnoDB function (PURGE ARCHIVED LOGS commands)
* error on bogus file name has been fixed, check for it has been
  added to appropriate test case

http://jenkins.percona.com/view/PS%205.6/job/percona-server-5.6-param/65/

Revision history for this message
Alexey Kopytov (akopytov) wrote :

Sergei,

Introducing new fields to the on-disk data format doesn't look good.

It looks like we can just avoid using LOG_CHECKPOINT_ARCHIVED_FILE_NO
and LOG_CHECKPOINT_ARCHIVED_OFFSET altogether. Upstream InnoDB code
stores archived_file_no and archived_offset on disk, because it uses
file naming based on ordinal file numbers. So on a restart it has to get
the file number from the checkpoint header to figure out the file _name_
that should be used for further archive writes. It normally doesn't need
the offset, because on a clean restart or when log archiving is
re-enabled, it starts a new file (i.e. with a zero file offset). The
only case when the offset is used is when InnoDB recovers from a
crash. In which case it will proceed with writing to the incompletely
written file starting from archived_offset, so it can eventually write
the file full and mark it as completed.

Now we use LSN-based file naming, so we don't have that problem. With
the new archived_file_no semantics we have archived_lsn =
archived_file_no + archived_offset. We can still maintain them
internally for simplicity, just don't write them to disk.

We have to do more work on a restart though to avoid incompletely
written files in case of a crash:

1. Read archived_lsn from the checkpoint header.
2. Scan innodb_log_arch_dir and find the log file with the maximum LSN
   in its name
3. If no files are found, set archived_file_no to archived_lsn,
   archived_offset to 0 and return (i.e. start a new file)
4. If LOG_FILE_ARCH_COMPLETED in the found file is TRUE (i.e. we are restarting from a
   clean shutdown) do the same as in 3.
5. Assert that archived_lsn lies within the incompletely written file we
   have found (i.e. file_start_lsn + group->file_size > archived_lsn)
6. set archived_file_no to file_start_lsn, archived_offset to
   (archived_lsn - file_start_lsn) and return

Another option would be to give up on the LSN-based file names idea, but
it wouldn't be a complete solution, because as you noted, the offset
should still be a 64-bit value now when log files can be > 4G.

Which means there are actually 2 options: either extend the on-disk
format, or do more work on restart.

Other minor things I noticed when looking at the code:

- too long line:
: IB_ARCHIVED_LOGS_PREFIX, IB_ARCHIVED_LOGS_PREFIX_LEN)) {

- spaces instead of tabs here:

: if (fil_space_contains_node(group->archive_space_id,
: archived_log_filename) ||

review: Needs Fixing
Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

I understand the reasoning for keeping ondisk format unchanged, but I think proposed algorithm can be made simpler.

Lets suppose we store only file number, but not the offset (we use both current 4-byte file_no and offset for it). After restart, we can read archived file number and archived lsn from checkpoint. So no need to iterate files in directory!. From archived file header we can read start lsn for this file and get offset by using subtraction.

Does it look good?

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

And, yes, if file doesn't exists, just make assume offset=0

Revision history for this message
Alexey Kopytov (akopytov) wrote :

Sergei,

Agree, a combination of a 8-byte archived_file_no and archived_lsn provides us all the information we need and allows to avoid innodb_log_arch_dir scan. Good find!

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :
Revision history for this message
Alexey Kopytov (akopytov) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_check.inc'
2--- Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_check.inc 1970-01-01 00:00:00 +0000
3+++ Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_check.inc 2013-03-31 06:25:28 +0000
4@@ -0,0 +1,15 @@
5+# Check that there are no gaps in archived files
6+
7+--source percona_log_archiving_stat.inc
8+
9+let $lsn = `SELECT (($archived_logs_count - 1) * $LOG_FILE_LSN_STEP + $first_log_file_lsn)`;
10+
11+if ($last_log_file_lsn != $lsn)
12+{
13+ echo Expected last file lsn is $lsn but got $last_log_file_lsn;
14+ die Test failed;
15+}
16+
17+echo Check OK;
18+
19+let $LAST_ARCHIVED_LSN = $last_log_file_lsn;
20
21=== added file 'Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_cleanup.inc'
22--- Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_cleanup.inc 1970-01-01 00:00:00 +0000
23+++ Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_cleanup.inc 2013-03-31 06:25:28 +0000
24@@ -0,0 +1,15 @@
25+#
26+# Cleanup after log archiving testing
27+#
28+
29+DROP TABLE t;
30+
31+--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
32+--shutdown_server
33+--source include/wait_until_disconnected.inc
34+--enable_reconnect
35+--exec echo "restart: --innodb-log-file-size=1M" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
36+--source include/wait_until_connected_again.inc
37+
38+--remove_files_wildcard $MYSQLD_ARCHIVEDIR ib_log_archive_*
39+--rmdir $MYSQLD_ARCHIVEDIR
40
41=== added file 'Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_count.inc'
42--- Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_count.inc 1970-01-01 00:00:00 +0000
43+++ Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_count.inc 2013-03-31 06:25:28 +0000
44@@ -0,0 +1,9 @@
45+
46+--source percona_log_archiving_stat.inc
47+
48+let $lsn = $n_files * $LOG_FILE_LSN_STEP;
49+
50+--echo $archived_logs_count
51+
52+let $LAST_ARCHIVED_LSN = $last_log_file_lsn;
53+
54
55=== added file 'Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_last_file.inc'
56--- Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_last_file.inc 1970-01-01 00:00:00 +0000
57+++ Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_last_file.inc 2013-03-31 06:25:28 +0000
58@@ -0,0 +1,24 @@
59+
60+perl;
61+@files = glob($ENV{'MYSQLD_ARCHIVEDIR'}.'/ib_log_archive_*');
62+my $filepath, $filename, $dirname;
63+foreach my $file (@files) {
64+ ($dirname, $filename) = $file =~ m|^(.*[/\\])([^/\\]+?)$|;
65+ $filepath = $file;
66+}
67+
68+my @array = stat($filepath);
69+my $filemodifytime = $array[9];
70+my @t = localtime $filemodifytime;
71+my $modifytime = sprintf "%04u-%02u-%02u %02u:%02u:%02u",$t[5]+1900,$t[4]+1,$t[3],$t[2],$t[1],$t[0];
72+my $lsn = $filename =~ m/ib_log_archive_0*([1-9][0-9]+)$/;
73+open F, ">".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/percona_archived_logs.tmp' or die "Cannot open file percona_archived_logs.tmp for write";
74+printf F ("let \$last_log_file = %s;\n", $filename);
75+printf F ("let \$last_log_file_modify = %s;\n", $modifytime);
76+printf F ("let \$last_log_file_lsn = %s;\n", $lsn);
77+close F;
78+EOF
79+
80+--source $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp
81+# remove_file $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp;
82+
83
84=== added file 'Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_setup.inc'
85--- Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_setup.inc 1970-01-01 00:00:00 +0000
86+++ Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_setup.inc 2013-03-31 06:25:28 +0000
87@@ -0,0 +1,49 @@
88+#
89+# setup for log archive testing
90+#
91+
92+
93+call mtr.add_suppression("InnoDB: Resizing redo log");
94+call mtr.add_suppression("InnoDB: Starting to delete and rewrite log files");
95+call mtr.add_suppression("InnoDB: New log files created");
96+call mtr.add_suppression("InnoDB: The log sequence numbers [0-9]+ and [0-9]+ in ibdata files do not match the log sequence number [0-9]+ in the ib_logfiles");
97+
98+--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
99+--shutdown_server
100+--source include/wait_until_disconnected.inc
101+# Do something while server is down
102+--enable_reconnect
103+--exec echo "restart: --innodb_log_archive=ON --innodb_log_arch_dir=logs_archive --innodb-log-file-size=1M" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
104+--source include/wait_until_connected_again.inc
105+
106+--disable_warnings
107+drop table if exists t;
108+--enable_warnings
109+
110+let $MYSQLD_DATADIR= `SELECT @@datadir`;
111+let $MYSQLD_ARCHIVEDIR= `SELECT IF( locate('/', @@innodb_log_arch_dir) = 1, @@innodb_log_arch_dir, CONCAT(@@datadir, @@innodb_log_arch_dir))`;
112+let MYSQLD_ARCHIVEDIR = $MYSQLD_ARCHIVEDIR;
113+--mkdir $MYSQLD_ARCHIVEDIR
114+
115+let $LOG_FILE_HDR_SIZE = 2048;
116+let $LOG_FILE_SIZE = `SELECT @@innodb_log_file_size`;
117+let $LOG_FILE_LSN_STEP = `SELECT $LOG_FILE_SIZE - $LOG_FILE_HDR_SIZE`;
118+
119+create table t (a int not null) ENGINE=InnoDB;
120+
121+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
122+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
123+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
124+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
125+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
126+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
127+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
128+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
129+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
130+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
131+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
132+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
133+
134+--source percona_log_archiving_stat.inc
135+
136+let $LAST_ARCHIVED_LSN = $last_log_file_lsn;
137
138=== added file 'Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_stat.inc'
139--- Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_stat.inc 1970-01-01 00:00:00 +0000
140+++ Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_stat.inc 2013-03-31 06:25:28 +0000
141@@ -0,0 +1,72 @@
142+#
143+# Collect information about archived log files
144+#
145+
146+perl;
147+use Data::Dumper;
148+sub fn {
149+ my $filepath = shift;
150+ my ($dirname, $filename) = $filepath =~ m|^(.*[/\\])([^/\\]+?)$|;
151+
152+ return $filename;
153+}
154+
155+sub get_file_stat {
156+ my $filepath = shift;
157+ my $prefix = shift;
158+ my $filename = fn($filepath);
159+
160+ my @array = stat($filepath);
161+ my $filemodifytime = $array[9];
162+ my @t = localtime $filemodifytime;
163+ my $modifytime = sprintf "%04u-%02u-%02u %02u:%02u:%02u",$t[5]+1900,$t[4]+1,$t[3],$t[2],$t[1],$t[0];
164+ my ($lsn) = $filename =~ m/ib_log_archive_[0]*([1-9][0-9]+)$/;
165+
166+ my $f = {
167+ 'path' => $filepath,
168+ 'name' => $filename,
169+ 't' => $filemodifytime,
170+ 'modifytime' => $modifytime,
171+ 'lsn' => $lsn
172+ };
173+
174+ return $f;
175+}
176+
177+sub print_file_stat {
178+ my %stat = %{$_[0]};
179+ my $prefix = $_[1];
180+
181+ open F, ">>".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/percona_archived_logs.tmp' or die "Cannot open file percona_archived_logs.tmp for write";
182+ printf F ("let \$${prefix}_log_file = %s;\n", $stat{'name'});
183+ printf F ("let \$${prefix}_log_file_modify = %s;\n", $stat{'modifytime'});
184+ printf F ("let \$${prefix}_log_file_lsn = %s;\n", $stat{'lsn'});
185+ close F;
186+}
187+
188+@files = glob($ENV{'MYSQLD_ARCHIVEDIR'}.'/ib_log_archive_*');
189+
190+my @files_stat = ();
191+
192+foreach my $file (@files) {
193+ push(@files_stat, get_file_stat($file));
194+}
195+
196+$count = scalar(@files_stat);
197+$mid_n = 1;
198+$t = $files_stat[$mid_n]{'t'};
199+while (($files_stat[$mid_n]{'t'} == $t) && ($mid_n < $count - 1)) {$mid_n++};
200+print_file_stat(@files_stat[0], 'first');
201+print_file_stat(@files_stat[-1], 'last');
202+print_file_stat(@files_stat[$mid_n - 1], 'mid');
203+
204+open F, ">>".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/percona_archived_logs.tmp' or die "Cannot open file percona_archived_logs.tmp for write";
205+printf F ("let \$archived_logs_count = %s;\n", $count);
206+printf F ("let \$archived_logs_mid = %s;\n", $mid_n);
207+close F;
208+
209+EOF
210+
211+--source $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp
212+remove_file $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp;
213+
214
215=== added file 'Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_workload.inc'
216--- Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_workload.inc 1970-01-01 00:00:00 +0000
217+++ Percona-Server/mysql-test/suite/innodb/include/percona_log_archiving_workload.inc 2013-03-31 06:25:28 +0000
218@@ -0,0 +1,5 @@
219+#
220+# Perform insert into table to produce some log records
221+#
222+
223+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
224
225=== added file 'Percona-Server/mysql-test/suite/innodb/r/pecona_bug1155475.result'
226--- Percona-Server/mysql-test/suite/innodb/r/pecona_bug1155475.result 1970-01-01 00:00:00 +0000
227+++ Percona-Server/mysql-test/suite/innodb/r/pecona_bug1155475.result 2013-03-31 06:25:28 +0000
228@@ -0,0 +1,23 @@
229+call mtr.add_suppression("InnoDB: Resizing redo log");
230+call mtr.add_suppression("InnoDB: Starting to delete and rewrite log files");
231+call mtr.add_suppression("InnoDB: New log files created");
232+call mtr.add_suppression("InnoDB: The log sequence numbers [0-9]+ and [0-9]+ in ibdata files do not match the log sequence number [0-9]+ in the ib_logfiles");
233+drop table if exists t;
234+create table t (a int not null) ENGINE=InnoDB;
235+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
236+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
237+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
238+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
239+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
240+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
241+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
242+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
243+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
244+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
245+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
246+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
247+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
248+Check OK
249+PURGE ARCHIVED LOGS BEFORE NOW();
250+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
251+DROP TABLE t;
252
253=== added file 'Percona-Server/mysql-test/suite/innodb/r/percona_innodb_log_archive_func.result'
254--- Percona-Server/mysql-test/suite/innodb/r/percona_innodb_log_archive_func.result 1970-01-01 00:00:00 +0000
255+++ Percona-Server/mysql-test/suite/innodb/r/percona_innodb_log_archive_func.result 2013-03-31 06:25:28 +0000
256@@ -0,0 +1,24 @@
257+call mtr.add_suppression("InnoDB: Resizing redo log");
258+call mtr.add_suppression("InnoDB: Starting to delete and rewrite log files");
259+call mtr.add_suppression("InnoDB: New log files created");
260+call mtr.add_suppression("InnoDB: The log sequence numbers [0-9]+ and [0-9]+ in ibdata files do not match the log sequence number [0-9]+ in the ib_logfiles");
261+drop table if exists t;
262+create table t (a int not null) ENGINE=InnoDB;
263+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
264+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
265+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
266+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
267+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
268+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
269+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
270+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
271+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
272+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
273+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
274+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
275+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
276+SET GLOBAL innodb_log_archive = OFF;
277+Check OK
278+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
279+SET GLOBAL innodb_log_archive = ON;
280+DROP TABLE t;
281
282=== added file 'Percona-Server/mysql-test/suite/innodb/r/percona_log_arch_expire_sec.result'
283--- Percona-Server/mysql-test/suite/innodb/r/percona_log_arch_expire_sec.result 1970-01-01 00:00:00 +0000
284+++ Percona-Server/mysql-test/suite/innodb/r/percona_log_arch_expire_sec.result 2013-03-31 06:25:28 +0000
285@@ -0,0 +1,27 @@
286+call mtr.add_suppression("InnoDB: Resizing redo log");
287+call mtr.add_suppression("InnoDB: Starting to delete and rewrite log files");
288+call mtr.add_suppression("InnoDB: New log files created");
289+call mtr.add_suppression("InnoDB: The log sequence numbers [0-9]+ and [0-9]+ in ibdata files do not match the log sequence number [0-9]+ in the ib_logfiles");
290+drop table if exists t;
291+create table t (a int not null) ENGINE=InnoDB;
292+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
293+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
294+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
295+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
296+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
297+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
298+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
299+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
300+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
301+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
302+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
303+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
304+SET @save_log_arch_expire_sec = @@innodb_log_arch_expire_sec;
305+SET GLOBAL innodb_log_arch_expire_sec = 1;
306+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
307+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
308+SELECT sleep(4);
309+sleep(4)
310+0
311+SET GLOBAL innodb_log_arch_expire_sec = @save_log_arch_expire_sec;
312+DROP TABLE t;
313
314=== added file 'Percona-Server/mysql-test/suite/innodb/r/percona_log_arch_func.result'
315--- Percona-Server/mysql-test/suite/innodb/r/percona_log_arch_func.result 1970-01-01 00:00:00 +0000
316+++ Percona-Server/mysql-test/suite/innodb/r/percona_log_arch_func.result 2013-03-31 06:25:28 +0000
317@@ -0,0 +1,21 @@
318+call mtr.add_suppression("InnoDB: Resizing redo log");
319+call mtr.add_suppression("InnoDB: Starting to delete and rewrite log files");
320+call mtr.add_suppression("InnoDB: New log files created");
321+call mtr.add_suppression("InnoDB: The log sequence numbers [0-9]+ and [0-9]+ in ibdata files do not match the log sequence number [0-9]+ in the ib_logfiles");
322+drop table if exists t;
323+create table t (a int not null) ENGINE=InnoDB;
324+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
325+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
326+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
327+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
328+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
329+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
330+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
331+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
332+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
333+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
334+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
335+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
336+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
337+Check OK
338+DROP TABLE t;
339
340=== added file 'Percona-Server/mysql-test/suite/innodb/r/percona_purge_archived_logs_before.result'
341--- Percona-Server/mysql-test/suite/innodb/r/percona_purge_archived_logs_before.result 1970-01-01 00:00:00 +0000
342+++ Percona-Server/mysql-test/suite/innodb/r/percona_purge_archived_logs_before.result 2013-03-31 06:25:28 +0000
343@@ -0,0 +1,37 @@
344+call mtr.add_suppression("InnoDB: Resizing redo log");
345+call mtr.add_suppression("InnoDB: Starting to delete and rewrite log files");
346+call mtr.add_suppression("InnoDB: New log files created");
347+call mtr.add_suppression("InnoDB: The log sequence numbers [0-9]+ and [0-9]+ in ibdata files do not match the log sequence number [0-9]+ in the ib_logfiles");
348+drop table if exists t;
349+create table t (a int not null) ENGINE=InnoDB;
350+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
351+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
352+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
353+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
354+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
355+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
356+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
357+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
358+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
359+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
360+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
361+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
362+SELECT sleep(1);
363+sleep(1)
364+0
365+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
366+SELECT sleep(1);
367+sleep(1)
368+0
369+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
370+SELECT sleep(1);
371+sleep(1)
372+0
373+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
374+Check OK
375+PURGE ARCHIVED LOGS BEFORE 'val';
376+PURGE ARCHIVED LOGS BEFORE NOW();
377+PURGE ARCHIVED LOGS BEFORE DATE_ADD(NOW(), INTERVAL 7 DAY);
378+PURGE ARCHIVED LOGS BEFORE 'q';
379+ERROR HY000: Unknown error during log purge
380+DROP TABLE t;
381
382=== added file 'Percona-Server/mysql-test/suite/innodb/r/percona_purge_archived_logs_to.result'
383--- Percona-Server/mysql-test/suite/innodb/r/percona_purge_archived_logs_to.result 1970-01-01 00:00:00 +0000
384+++ Percona-Server/mysql-test/suite/innodb/r/percona_purge_archived_logs_to.result 2013-03-31 06:25:28 +0000
385@@ -0,0 +1,35 @@
386+call mtr.add_suppression("InnoDB: Resizing redo log");
387+call mtr.add_suppression("InnoDB: Starting to delete and rewrite log files");
388+call mtr.add_suppression("InnoDB: New log files created");
389+call mtr.add_suppression("InnoDB: The log sequence numbers [0-9]+ and [0-9]+ in ibdata files do not match the log sequence number [0-9]+ in the ib_logfiles");
390+drop table if exists t;
391+create table t (a int not null) ENGINE=InnoDB;
392+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
393+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
394+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
395+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
396+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
397+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
398+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
399+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
400+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
401+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
402+insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13);
403+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
404+SELECT sleep(1);
405+sleep(1)
406+0
407+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
408+SELECT sleep(1);
409+sleep(1)
410+0
411+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
412+SELECT sleep(1);
413+sleep(1)
414+0
415+insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 40000;
416+Check OK
417+PURGE ARCHIVED LOGS TO 'val';
418+PURGE ARCHIVED LOGS TO '#invalid_file';
419+ERROR HY000: Unknown error during log purge
420+DROP TABLE t;
421
422=== added file 'Percona-Server/mysql-test/suite/innodb/t/pecona_bug1155475.test'
423--- Percona-Server/mysql-test/suite/innodb/t/pecona_bug1155475.test 1970-01-01 00:00:00 +0000
424+++ Percona-Server/mysql-test/suite/innodb/t/pecona_bug1155475.test 2013-03-31 06:25:28 +0000
425@@ -0,0 +1,30 @@
426+#
427+# Test bug 1155475
428+#
429+
430+--source include/have_innodb.inc
431+--source include/not_embedded.inc
432+
433+--source ../include/percona_log_archiving_setup.inc
434+
435+--source ../include/percona_log_archiving_workload.inc
436+--source ../include/percona_log_archiving_check.inc
437+
438+if ($archived_logs_count == 0)
439+{
440+ die "0 archived logs have been produced";
441+}
442+
443+--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
444+--shutdown_server
445+--source include/wait_until_disconnected.inc
446+--enable_reconnect
447+--exec echo "restart: --innodb_log_archive=ON --innodb_log_arch_dir=logs_archive --innodb-log-file-size=1M" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
448+--source include/wait_until_connected_again.inc
449+
450+PURGE ARCHIVED LOGS BEFORE NOW();
451+
452+# server shouldn't crash
453+--source ../include/percona_log_archiving_workload.inc
454+
455+--source ../include/percona_log_archiving_cleanup.inc
456
457=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_innodb_log_archive_func.test'
458--- Percona-Server/mysql-test/suite/innodb/t/percona_innodb_log_archive_func.test 1970-01-01 00:00:00 +0000
459+++ Percona-Server/mysql-test/suite/innodb/t/percona_innodb_log_archive_func.test 2013-03-31 06:25:28 +0000
460@@ -0,0 +1,31 @@
461+#
462+# Test innodb_log_archive turning ON/OFF at runtime
463+#
464+
465+--source include/have_innodb.inc
466+--source include/not_embedded.inc
467+
468+--source ../include/percona_log_archiving_setup.inc
469+
470+
471+--source ../include/percona_log_archiving_workload.inc
472+
473+SET GLOBAL innodb_log_archive = OFF;
474+
475+--source ../include/percona_log_archiving_check.inc
476+
477+let $count = $archived_logs_count;
478+
479+--source ../include/percona_log_archiving_workload.inc
480+
481+SET GLOBAL innodb_log_archive = ON;
482+
483+--source ../include/percona_log_archiving_stat.inc
484+
485+if ($count != $archived_logs_count)
486+{
487+ die Test failed. Archived logs being written while innodb_log_archive being turned off.;
488+}
489+
490+--source ../include/percona_log_archiving_cleanup.inc
491+
492
493=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_log_arch_expire_sec.test'
494--- Percona-Server/mysql-test/suite/innodb/t/percona_log_arch_expire_sec.test 1970-01-01 00:00:00 +0000
495+++ Percona-Server/mysql-test/suite/innodb/t/percona_log_arch_expire_sec.test 2013-03-31 06:25:28 +0000
496@@ -0,0 +1,29 @@
497+#
498+# Test innodb_log_archive_expire_sec
499+#
500+
501+--source include/have_innodb.inc
502+--source include/not_embedded.inc
503+
504+--source ../include/percona_log_archiving_setup.inc
505+
506+SET @save_log_arch_expire_sec = @@innodb_log_arch_expire_sec;
507+SET GLOBAL innodb_log_arch_expire_sec = 1;
508+
509+--source ../include/percona_log_archiving_workload.inc
510+
511+--source ../include/percona_log_archiving_workload.inc
512+
513+SELECT sleep(4);
514+
515+--source ../include/percona_log_archiving_stat.inc
516+
517+if ($archived_logs_count > 1)
518+{
519+ die Test failed. Archived logs have nod been purged.;
520+}
521+
522+SET GLOBAL innodb_log_arch_expire_sec = @save_log_arch_expire_sec;
523+
524+--source ../include/percona_log_archiving_cleanup.inc
525+
526
527=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_log_arch_func.test'
528--- Percona-Server/mysql-test/suite/innodb/t/percona_log_arch_func.test 1970-01-01 00:00:00 +0000
529+++ Percona-Server/mysql-test/suite/innodb/t/percona_log_arch_func.test 2013-03-31 06:25:28 +0000
530@@ -0,0 +1,19 @@
531+#
532+# Test that log archiving is working (produce some files at least)
533+#
534+
535+--source include/have_innodb.inc
536+--source include/not_embedded.inc
537+
538+--source ../include/percona_log_archiving_setup.inc
539+
540+--source ../include/percona_log_archiving_workload.inc
541+--source ../include/percona_log_archiving_check.inc
542+
543+if ($archived_logs_count == 0)
544+{
545+ die "0 archived logs have been produced";
546+}
547+
548+--source ../include/percona_log_archiving_cleanup.inc
549+
550
551=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_purge_archived_logs_before.test'
552--- Percona-Server/mysql-test/suite/innodb/t/percona_purge_archived_logs_before.test 1970-01-01 00:00:00 +0000
553+++ Percona-Server/mysql-test/suite/innodb/t/percona_purge_archived_logs_before.test 2013-03-31 06:25:28 +0000
554@@ -0,0 +1,43 @@
555+#
556+# Test PURGE ARCHIVED LOGS BEFORE
557+#
558+
559+--source include/have_innodb.inc
560+--source include/not_embedded.inc
561+
562+--source ../include/percona_log_archiving_setup.inc
563+
564+SELECT sleep(1);
565+--source ../include/percona_log_archiving_workload.inc
566+SELECT sleep(1);
567+--source ../include/percona_log_archiving_workload.inc
568+SELECT sleep(1);
569+--source ../include/percona_log_archiving_workload.inc
570+--source ../include/percona_log_archiving_check.inc
571+
572+if ($archived_logs_count == 0)
573+{
574+ die "0 archived logs have been produced";
575+}
576+
577+--replace_result $mid_log_file_modify val
578+--eval PURGE ARCHIVED LOGS BEFORE '$mid_log_file_modify'
579+
580+let $count = `SELECT $archived_logs_count - $archived_logs_mid`;
581+
582+--source ../include/percona_log_archiving_stat.inc
583+
584+if ($archived_logs_count != $count)
585+{
586+ die "PURGE ARCHIVED LOGS BEFORE failed";
587+}
588+
589+# check that it works with NOW() and other functions
590+PURGE ARCHIVED LOGS BEFORE NOW();
591+PURGE ARCHIVED LOGS BEFORE DATE_ADD(NOW(), INTERVAL 7 DAY);
592+
593+# should fail
594+--error 1379
595+PURGE ARCHIVED LOGS BEFORE 'q';
596+
597+--source ../include/percona_log_archiving_cleanup.inc
598
599=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_purge_archived_logs_to.test'
600--- Percona-Server/mysql-test/suite/innodb/t/percona_purge_archived_logs_to.test 1970-01-01 00:00:00 +0000
601+++ Percona-Server/mysql-test/suite/innodb/t/percona_purge_archived_logs_to.test 2013-03-31 06:25:28 +0000
602@@ -0,0 +1,39 @@
603+#
604+# Test PURGE ARCHIVED LOGS TO
605+#
606+
607+--source include/have_innodb.inc
608+--source include/not_embedded.inc
609+
610+--source ../include/percona_log_archiving_setup.inc
611+
612+SELECT sleep(1);
613+--source ../include/percona_log_archiving_workload.inc
614+SELECT sleep(1);
615+--source ../include/percona_log_archiving_workload.inc
616+SELECT sleep(1);
617+--source ../include/percona_log_archiving_workload.inc
618+--source ../include/percona_log_archiving_check.inc
619+
620+if ($archived_logs_count == 0)
621+{
622+ die "0 archived logs have been produced";
623+}
624+
625+--replace_result $mid_log_file val
626+--eval PURGE ARCHIVED LOGS TO '$mid_log_file'
627+
628+let $count = `SELECT $archived_logs_count - $archived_logs_mid + 1`;
629+
630+--source ../include/percona_log_archiving_stat.inc
631+
632+if ($archived_logs_count != $count)
633+{
634+ die "PURGE ARCHIVED LOGS TO failed";
635+}
636+
637+#should fail
638+--error 1379
639+PURGE ARCHIVED LOGS TO '#invalid_file';
640+
641+--source ../include/percona_log_archiving_cleanup.inc
642
643=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_arch_dir_basic.result'
644--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_arch_dir_basic.result 1970-01-01 00:00:00 +0000
645+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_arch_dir_basic.result 2013-03-31 06:25:28 +0000
646@@ -0,0 +1,38 @@
647+SELECT @@GLOBAL.innodb_log_arch_dir;
648+@@GLOBAL.innodb_log_arch_dir
649+./
650+NULL Expected
651+SET @@GLOBAL.innodb_log_arch_dir=1;
652+ERROR HY000: Variable 'innodb_log_arch_dir' is a read only variable
653+Expected error 'Read only variable'
654+SELECT @@GLOBAL.innodb_log_arch_dir;
655+@@GLOBAL.innodb_log_arch_dir
656+./
657+NULL Expected
658+SELECT VARIABLE_VALUE
659+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES
660+WHERE VARIABLE_NAME='innodb_log_arch_dir';
661+VARIABLE_VALUE
662+./
663+empty string Expected
664+SELECT @@innodb_log_arch_dir;
665+@@innodb_log_arch_dir
666+./
667+NULL Expected
668+SELECT @@innodb_log_arch_dir;
669+@@innodb_log_arch_dir
670+./
671+NULL Expected
672+SELECT @@local.innodb_log_arch_dir;
673+ERROR HY000: Variable 'innodb_log_arch_dir' is a GLOBAL variable
674+Expected error 'Variable is a GLOBAL variable'
675+SELECT @@SESSION.innodb_log_arch_dir;
676+ERROR HY000: Variable 'innodb_log_arch_dir' is a GLOBAL variable
677+Expected error 'Variable is a GLOBAL variable'
678+SELECT @@GLOBAL.innodb_log_arch_dir;
679+@@GLOBAL.innodb_log_arch_dir
680+./
681+NULL Expected
682+SELECT innodb_log_arch_dir = @@SESSION.innodb_log_arch_dir;
683+ERROR 42S22: Unknown column 'innodb_log_arch_dir' in 'field list'
684+Expected error Unknown column 'innodb_log_arch_dir' in 'field list'
685
686=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_arch_expire_sec_basic.result'
687--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_arch_expire_sec_basic.result 1970-01-01 00:00:00 +0000
688+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_arch_expire_sec_basic.result 2013-03-31 06:25:28 +0000
689@@ -0,0 +1,38 @@
690+SELECT @@GLOBAL.innodb_log_arch_expire_sec INTO @save;
691+SELECT @@GLOBAL.innodb_log_arch_expire_sec;
692+@@GLOBAL.innodb_log_arch_expire_sec
693+0
694+0 Expected
695+SET @@GLOBAL.innodb_log_arch_expire_sec=1;
696+SELECT @@GLOBAL.innodb_log_arch_expire_sec;
697+@@GLOBAL.innodb_log_arch_expire_sec
698+1
699+1 Expected
700+SELECT VARIABLE_VALUE
701+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES
702+WHERE VARIABLE_NAME='innodb_log_arch_expire_sec';
703+VARIABLE_VALUE
704+1
705+1 Expected
706+SELECT @@innodb_log_arch_expire_sec;
707+@@innodb_log_arch_expire_sec
708+1
709+1 Expected
710+SELECT @@innodb_log_arch_expire_sec;
711+@@innodb_log_arch_expire_sec
712+1
713+1 Expected
714+SELECT @@local.innodb_log_arch_expire_sec;
715+ERROR HY000: Variable 'innodb_log_arch_expire_sec' is a GLOBAL variable
716+Expected error 'Variable is a GLOBAL variable'
717+SELECT @@SESSION.innodb_log_arch_expire_sec;
718+ERROR HY000: Variable 'innodb_log_arch_expire_sec' is a GLOBAL variable
719+Expected error 'Variable is a GLOBAL variable'
720+SELECT @@GLOBAL.innodb_log_arch_expire_sec;
721+@@GLOBAL.innodb_log_arch_expire_sec
722+1
723+1 Expected
724+SELECT innodb_log_arch_expire_sec = @@SESSION.innodb_log_arch_expire_sec;
725+ERROR 42S22: Unknown column 'innodb_log_arch_expire_sec' in 'field list'
726+Expected error Unknown column 'innodb_log_arch_expire_sec' in 'field list'
727+SET @@GLOBAL.innodb_log_arch_expire_sec = @save;
728
729=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_archive_basic.result'
730--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_archive_basic.result 1970-01-01 00:00:00 +0000
731+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_log_archive_basic.result 2013-03-31 06:25:28 +0000
732@@ -0,0 +1,38 @@
733+SELECT @@GLOBAL.innodb_log_archive;
734+@@GLOBAL.innodb_log_archive
735+0
736+0 Expected
737+SET @save_innodb_log_archive = @@GLOBAL.innodb_log_archive;
738+SET @@GLOBAL.innodb_log_archive=1;
739+SELECT @@GLOBAL.innodb_log_archive;
740+@@GLOBAL.innodb_log_archive
741+1
742+1 Expected
743+SELECT VARIABLE_VALUE
744+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES
745+WHERE VARIABLE_NAME='innodb_log_archive';
746+VARIABLE_VALUE
747+ON
748+ON Expected
749+SET @@GLOBAL.innodb_log_archive = @save_innodb_log_archive;
750+SELECT @@innodb_log_archive;
751+@@innodb_log_archive
752+0
753+0 Expected
754+SELECT @@innodb_log_archive;
755+@@innodb_log_archive
756+0
757+0 Expected
758+SELECT @@local.innodb_log_archive;
759+ERROR HY000: Variable 'innodb_log_archive' is a GLOBAL variable
760+Expected error 'Variable is a GLOBAL variable'
761+SELECT @@SESSION.innodb_log_archive;
762+ERROR HY000: Variable 'innodb_log_archive' is a GLOBAL variable
763+Expected error 'Variable is a GLOBAL variable'
764+SELECT @@GLOBAL.innodb_log_archive;
765+@@GLOBAL.innodb_log_archive
766+0
767+0 Expected
768+SELECT innodb_log_archive = @@SESSION.innodb_log_archive;
769+ERROR 42S22: Unknown column 'innodb_log_archive' in 'field list'
770+Expected error Unknown column 'innodb_log_archive' in 'field list'
771
772=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_arch_dir_basic.test'
773--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_arch_dir_basic.test 1970-01-01 00:00:00 +0000
774+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_arch_dir_basic.test 2013-03-31 06:25:28 +0000
775@@ -0,0 +1,60 @@
776+#######################################################
777+# Basic test for innodb_log_arch_dir variable #
778+#######################################################
779+
780+--source include/have_innodb.inc
781+
782+####################################################################
783+# Displaying default value #
784+####################################################################
785+SELECT @@GLOBAL.innodb_log_arch_dir;
786+--echo NULL Expected
787+
788+
789+####################################################################
790+# Check if Value can set #
791+####################################################################
792+
793+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
794+SET @@GLOBAL.innodb_log_arch_dir=1;
795+--echo Expected error 'Read only variable'
796+
797+SELECT @@GLOBAL.innodb_log_arch_dir;
798+--echo NULL Expected
799+
800+SELECT VARIABLE_VALUE
801+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES
802+WHERE VARIABLE_NAME='innodb_log_arch_dir';
803+--echo empty string Expected
804+
805+############################################
806+# Check accessing variable without GLOBAL #
807+############################################
808+SELECT @@innodb_log_arch_dir;
809+--echo NULL Expected
810+
811+
812+
813+##########################################################################
814+# Check if innodb_log_arch_dir can be accessed without @@ sign #
815+##########################################################################
816+
817+SELECT @@innodb_log_arch_dir;
818+--echo NULL Expected
819+
820+--Error ER_INCORRECT_GLOBAL_LOCAL_VAR
821+SELECT @@local.innodb_log_arch_dir;
822+--echo Expected error 'Variable is a GLOBAL variable'
823+
824+--Error ER_INCORRECT_GLOBAL_LOCAL_VAR
825+SELECT @@SESSION.innodb_log_arch_dir;
826+--echo Expected error 'Variable is a GLOBAL variable'
827+
828+SELECT @@GLOBAL.innodb_log_arch_dir;
829+--echo NULL Expected
830+
831+--Error ER_BAD_FIELD_ERROR
832+SELECT innodb_log_arch_dir = @@SESSION.innodb_log_arch_dir;
833+--echo Expected error Unknown column 'innodb_log_arch_dir' in 'field list'
834+
835+
836
837=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_arch_expire_sec_basic.test'
838--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_arch_expire_sec_basic.test 1970-01-01 00:00:00 +0000
839+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_arch_expire_sec_basic.test 2013-03-31 06:25:28 +0000
840@@ -0,0 +1,60 @@
841+###############################################################
842+# Basic test for innodb_log_arch_expire_sec variable #
843+###############################################################
844+
845+--source include/have_innodb.inc
846+
847+SELECT @@GLOBAL.innodb_log_arch_expire_sec INTO @save;
848+
849+####################################################################
850+# Displaying default value #
851+####################################################################
852+SELECT @@GLOBAL.innodb_log_arch_expire_sec;
853+--echo 0 Expected
854+
855+
856+####################################################################
857+# Check if Value can set #
858+####################################################################
859+
860+SET @@GLOBAL.innodb_log_arch_expire_sec=1;
861+
862+SELECT @@GLOBAL.innodb_log_arch_expire_sec;
863+--echo 1 Expected
864+
865+SELECT VARIABLE_VALUE
866+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES
867+WHERE VARIABLE_NAME='innodb_log_arch_expire_sec';
868+--echo 1 Expected
869+
870+############################################
871+# Check accessing variable without GLOBAL #
872+############################################
873+SELECT @@innodb_log_arch_expire_sec;
874+--echo 1 Expected
875+
876+
877+
878+##################################################################################
879+# Check if innodb_log_arch_expire_sec can be accessed without @@ sign #
880+##################################################################################
881+
882+SELECT @@innodb_log_arch_expire_sec;
883+--echo 1 Expected
884+
885+--Error ER_INCORRECT_GLOBAL_LOCAL_VAR
886+SELECT @@local.innodb_log_arch_expire_sec;
887+--echo Expected error 'Variable is a GLOBAL variable'
888+
889+--Error ER_INCORRECT_GLOBAL_LOCAL_VAR
890+SELECT @@SESSION.innodb_log_arch_expire_sec;
891+--echo Expected error 'Variable is a GLOBAL variable'
892+
893+SELECT @@GLOBAL.innodb_log_arch_expire_sec;
894+--echo 1 Expected
895+
896+--Error ER_BAD_FIELD_ERROR
897+SELECT innodb_log_arch_expire_sec = @@SESSION.innodb_log_arch_expire_sec;
898+--echo Expected error Unknown column 'innodb_log_arch_expire_sec' in 'field list'
899+
900+SET @@GLOBAL.innodb_log_arch_expire_sec = @save;
901
902=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_archive_basic.test'
903--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_archive_basic.test 1970-01-01 00:00:00 +0000
904+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_log_archive_basic.test 2013-03-31 06:25:28 +0000
905@@ -0,0 +1,61 @@
906+###################################################
907+# Basic test for innodb_log_archive variable #
908+###################################################
909+
910+--source include/have_innodb.inc
911+
912+####################################################################
913+# Displaying default value #
914+####################################################################
915+SELECT @@GLOBAL.innodb_log_archive;
916+--echo 0 Expected
917+
918+
919+####################################################################
920+# Check if Value can set #
921+####################################################################
922+
923+SET @save_innodb_log_archive = @@GLOBAL.innodb_log_archive;
924+SET @@GLOBAL.innodb_log_archive=1;
925+
926+SELECT @@GLOBAL.innodb_log_archive;
927+--echo 1 Expected
928+
929+SELECT VARIABLE_VALUE
930+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES
931+WHERE VARIABLE_NAME='innodb_log_archive';
932+--echo ON Expected
933+
934+SET @@GLOBAL.innodb_log_archive = @save_innodb_log_archive;
935+
936+############################################
937+# Check accessing variable without GLOBAL #
938+############################################
939+SELECT @@innodb_log_archive;
940+--echo 0 Expected
941+
942+
943+
944+##########################################################################
945+# Check if innodb_log_archive can be accessed without @@ sign #
946+##########################################################################
947+
948+SELECT @@innodb_log_archive;
949+--echo 0 Expected
950+
951+--Error ER_INCORRECT_GLOBAL_LOCAL_VAR
952+SELECT @@local.innodb_log_archive;
953+--echo Expected error 'Variable is a GLOBAL variable'
954+
955+--Error ER_INCORRECT_GLOBAL_LOCAL_VAR
956+SELECT @@SESSION.innodb_log_archive;
957+--echo Expected error 'Variable is a GLOBAL variable'
958+
959+SELECT @@GLOBAL.innodb_log_archive;
960+--echo 0 Expected
961+
962+--Error ER_BAD_FIELD_ERROR
963+SELECT innodb_log_archive = @@SESSION.innodb_log_archive;
964+--echo Expected error Unknown column 'innodb_log_archive' in 'field list'
965+
966+
967
968=== modified file 'Percona-Server/sql/handler.cc'
969--- Percona-Server/sql/handler.cc 2013-03-05 12:46:43 +0000
970+++ Percona-Server/sql/handler.cc 2013-03-31 06:25:28 +0000
971@@ -6867,6 +6867,48 @@
972 return result;
973 }
974
975+static my_bool purge_archive_logs_handlerton(THD *thd, plugin_ref plugin,
976+ void *arg)
977+{
978+ ulong before_timestamp= *(ulong*) arg;
979+ handlerton *hton= plugin_data(plugin, handlerton *);
980+
981+ if (hton->purge_archive_logs == NULL)
982+ return FALSE;
983+
984+ return hton->purge_archive_logs(hton, before_timestamp, NULL);
985+}
986+
987+bool ha_purge_archive_logs(THD *thd, handlerton *db_type, void* args)
988+{
989+ if (db_type == NULL)
990+ return plugin_foreach(thd, purge_archive_logs_handlerton,
991+ MYSQL_STORAGE_ENGINE_PLUGIN, args);
992+
993+ return false;
994+}
995+
996+static my_bool purge_archive_logs_to_handlerton(THD *thd, plugin_ref plugin,
997+ void *arg)
998+{
999+ const char* to_filename= (const char*) arg;
1000+ handlerton *hton= plugin_data(plugin, handlerton *);
1001+
1002+ if (hton->purge_archive_logs == NULL)
1003+ return FALSE;
1004+
1005+ return hton->purge_archive_logs(hton, 0, to_filename);
1006+}
1007+
1008+bool ha_purge_archive_logs_to(THD *thd, handlerton *db_type, void* args)
1009+{
1010+ if (db_type == NULL)
1011+ return plugin_foreach(thd, purge_archive_logs_to_handlerton,
1012+ MYSQL_STORAGE_ENGINE_PLUGIN, args);
1013+
1014+ return false;
1015+}
1016+
1017 /*
1018 Function to check if the conditions for row-based binlogging is
1019 correct for the table.
1020
1021=== modified file 'Percona-Server/sql/handler.h'
1022--- Percona-Server/sql/handler.h 2013-03-05 12:46:43 +0000
1023+++ Percona-Server/sql/handler.h 2013-03-31 06:25:28 +0000
1024@@ -881,6 +881,9 @@
1025 int (*fill_is_table)(handlerton *hton, THD *thd, TABLE_LIST *tables,
1026 class Item *cond,
1027 enum enum_schema_tables);
1028+ bool (*purge_archive_logs)(handlerton *hton, time_t before_date,
1029+ const char* to_filename);
1030+
1031 uint32 flags; /* global handler flags */
1032 /*
1033 Those handlerton functions below are properly initialized at handler
1034@@ -3375,6 +3378,11 @@
1035 int ha_prepare(THD *thd);
1036 int ha_recover(HASH *commit_list);
1037
1038+/* remove old archived transaction logs files */
1039+bool ha_purge_archive_logs(THD *thd, handlerton *db_type, void* args);
1040+bool ha_purge_archive_logs_to(THD *thd, handlerton *db_type, void* args);
1041+
1042+
1043 /*
1044 transactions: interface to low-level handlerton functions. These are
1045 intended to be used by the transaction coordinators to
1046
1047=== modified file 'Percona-Server/sql/lex.h'
1048--- Percona-Server/sql/lex.h 2013-03-05 12:46:43 +0000
1049+++ Percona-Server/sql/lex.h 2013-03-31 06:25:28 +0000
1050@@ -69,6 +69,7 @@
1051 { "ANALYZE", SYM(ANALYZE_SYM)}, // this one is for ANALYZE TABLE etc
1052 { "AND", SYM(AND_SYM)},
1053 { "ANY", SYM(ANY_SYM)},
1054+ { "ARCHIVED", SYM(ARCHIVED_SYM)},
1055 { "AS", SYM(AS)},
1056 { "ASC", SYM(ASC)},
1057 { "ASCII", SYM(ASCII_SYM)},
1058
1059=== modified file 'Percona-Server/sql/mysqld.cc'
1060--- Percona-Server/sql/mysqld.cc 2013-03-05 12:46:43 +0000
1061+++ Percona-Server/sql/mysqld.cc 2013-03-31 06:25:28 +0000
1062@@ -3418,6 +3418,8 @@
1063 {"prepare_sql", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PREPARE]), SHOW_LONG_STATUS},
1064 {"purge", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PURGE]), SHOW_LONG_STATUS},
1065 {"purge_before_date", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PURGE_BEFORE]), SHOW_LONG_STATUS},
1066+ {"purge_archived", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PURGE_ARCHIVE]), SHOW_LONG_STATUS},
1067+ {"purge_archived_before_date", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PURGE_ARCHIVE_BEFORE]), SHOW_LONG_STATUS},
1068 {"release_savepoint", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_RELEASE_SAVEPOINT]), SHOW_LONG_STATUS},
1069 {"rename_table", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_RENAME_TABLE]), SHOW_LONG_STATUS},
1070 {"rename_user", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_RENAME_USER]), SHOW_LONG_STATUS},
1071
1072=== modified file 'Percona-Server/sql/sql_cmd.h'
1073--- Percona-Server/sql/sql_cmd.h 2012-12-04 08:24:59 +0000
1074+++ Percona-Server/sql/sql_cmd.h 2013-03-31 06:25:28 +0000
1075@@ -89,6 +89,8 @@
1076 SQLCOM_SHOW_RELAYLOG_EVENTS,
1077 SQLCOM_GET_DIAGNOSTICS,
1078 SQLCOM_ALTER_USER,
1079+ SQLCOM_PURGE_ARCHIVE,
1080+ SQLCOM_PURGE_ARCHIVE_BEFORE,
1081 /*
1082 When a command is added here, be sure it's also added in mysqld.cc
1083 in "struct show_var_st status_vars[]= {" ...
1084
1085=== modified file 'Percona-Server/sql/sql_parse.cc'
1086--- Percona-Server/sql/sql_parse.cc 2013-03-05 12:46:43 +0000
1087+++ Percona-Server/sql/sql_parse.cc 2013-03-31 06:25:28 +0000
1088@@ -2655,6 +2655,49 @@
1089 res = purge_master_logs_before_date(thd, (ulong)it->val_int());
1090 break;
1091 }
1092+ case SQLCOM_PURGE_ARCHIVE:
1093+ {
1094+ if (check_global_access(thd, SUPER_ACL))
1095+ goto error;
1096+ /* PURGE ARCHIVED LOGS TO 'file' */
1097+ if (!ha_purge_archive_logs_to(NULL, NULL, lex->to_log)) {
1098+ my_ok(thd);
1099+ } else {
1100+ my_error(ER_LOG_PURGE_UNKNOWN_ERR, MYF(0), "PURGE ARCHIVE LOGS TO");
1101+ goto error;
1102+ }
1103+
1104+ break;
1105+ }
1106+ case SQLCOM_PURGE_ARCHIVE_BEFORE:
1107+ {
1108+ Item *it;
1109+
1110+ if (check_global_access(thd, SUPER_ACL))
1111+ goto error;
1112+ /* PURGE ARCHIVE LOGS BEFORE 'data' */
1113+ it= (Item *)lex->value_list.head();
1114+ if ((!it->fixed && it->fix_fields(lex->thd, &it)) ||
1115+ it->check_cols(1))
1116+ {
1117+ my_error(ER_WRONG_ARGUMENTS, MYF(0), "PURGE ARCHIVE LOGS BEFORE");
1118+ goto error;
1119+ }
1120+ it= new Item_func_unix_timestamp(it);
1121+ /*
1122+ it is OK only emulate fix_fieds, because we need only
1123+ value of constant
1124+ */
1125+ it->quick_fix_field();
1126+ ulong before_timestamp = (ulong)it->val_int();
1127+ if (!ha_purge_archive_logs(NULL, NULL, &before_timestamp )) {
1128+ my_ok(thd);
1129+ } else {
1130+ my_error(ER_LOG_PURGE_UNKNOWN_ERR, MYF(0), "PURGE ARCHIVE LOGS BEFORE");
1131+ goto error;
1132+ }
1133+ break;
1134+ }
1135 #endif
1136 case SQLCOM_SHOW_WARNS:
1137 {
1138
1139=== modified file 'Percona-Server/sql/sql_yacc.yy'
1140--- Percona-Server/sql/sql_yacc.yy 2013-03-05 12:46:43 +0000
1141+++ Percona-Server/sql/sql_yacc.yy 2013-03-31 06:25:28 +0000
1142@@ -1064,6 +1064,7 @@
1143 %token AND_AND_SYM /* OPERATOR */
1144 %token AND_SYM /* SQL-2003-R */
1145 %token ANY_SYM /* SQL-2003-R */
1146+%token ARCHIVED_SYM /* MYSQL */
1147 %token AS /* SQL-2003-R */
1148 %token ASC /* SQL-2003-N */
1149 %token ASCII_SYM /* MYSQL-FUNC */
1150@@ -12942,7 +12943,8 @@
1151 ;
1152
1153 purge_options:
1154- master_or_binary LOGS_SYM purge_option
1155+ ARCHIVED_SYM LOGS_SYM purge_archive_option
1156+ | master_or_binary LOGS_SYM purge_option
1157 ;
1158
1159 purge_option:
1160@@ -12959,6 +12961,21 @@
1161 }
1162 ;
1163
1164+purge_archive_option:
1165+ TO_SYM TEXT_STRING_sys
1166+ {
1167+ Lex->to_log = $2.str;
1168+ Lex->sql_command= SQLCOM_PURGE_ARCHIVE;
1169+ }
1170+ | BEFORE_SYM expr
1171+ {
1172+ LEX *lex= Lex;
1173+ lex->value_list.empty();
1174+ lex->value_list.push_front($2);
1175+ lex->sql_command= SQLCOM_PURGE_ARCHIVE_BEFORE;
1176+ }
1177+ ;
1178+
1179 /* kill threads */
1180
1181 kill:
1182@@ -14080,6 +14097,7 @@
1183 | AGGREGATE_SYM {}
1184 | ALGORITHM_SYM {}
1185 | ANALYSE_SYM {}
1186+ | ARCHIVED_SYM {}
1187 | ANY_SYM {}
1188 | AT_SYM {}
1189 | AUTO_INC {}
1190
1191=== modified file 'Percona-Server/storage/innobase/fil/fil0fil.cc'
1192--- Percona-Server/storage/innobase/fil/fil0fil.cc 2013-03-05 12:46:43 +0000
1193+++ Percona-Server/storage/innobase/fil/fil0fil.cc 2013-03-31 06:25:28 +0000
1194@@ -1211,6 +1211,39 @@
1195
1196 mutex_exit(&fil_system->mutex);
1197 }
1198+
1199+/****************************************************************//**
1200+Check is there node in file space with given name. */
1201+UNIV_INTERN
1202+ibool
1203+fil_space_contains_node(
1204+/*====================*/
1205+ ulint id, /*!< in: space id */
1206+ char* node_name) /*!< in: node name */
1207+{
1208+ fil_node_t* node;
1209+ fil_space_t* space;
1210+
1211+ mutex_enter(&fil_system->mutex);
1212+
1213+ space = fil_space_get_by_id(id);
1214+
1215+ ut_a(space);
1216+
1217+ for (node = UT_LIST_GET_FIRST(space->chain); node != NULL;
1218+ node = UT_LIST_GET_NEXT(chain, node)) {
1219+
1220+ if (ut_strcmp(node->name, node_name) == 0) {
1221+ mutex_exit(&fil_system->mutex);
1222+ return(TRUE);
1223+ }
1224+
1225+ }
1226+
1227+ mutex_exit(&fil_system->mutex);
1228+ return(FALSE);
1229+}
1230+
1231 #endif /* UNIV_LOG_ARCHIVE */
1232
1233 /*******************************************************************//**
1234@@ -1942,12 +1975,6 @@
1235 contain sensible data */
1236 ulint* flags, /*!< out: tablespace flags */
1237 ulint* space_id, /*!< out: tablespace ID */
1238-#ifdef UNIV_LOG_ARCHIVE
1239- ulint* min_arch_log_no, /*!< out: min of archived
1240- log numbers in data files */
1241- ulint* max_arch_log_no, /*!< out: max of archived
1242- log numbers in data files */
1243-#endif /* UNIV_LOG_ARCHIVE */
1244 lsn_t* min_flushed_lsn, /*!< out: min of flushed
1245 lsn values in data files */
1246 lsn_t* max_flushed_lsn) /*!< out: max of flushed
1247@@ -1976,10 +2003,7 @@
1248 if (!one_read_already) {
1249 *min_flushed_lsn = flushed_lsn;
1250 *max_flushed_lsn = flushed_lsn;
1251-#ifdef UNIV_LOG_ARCHIVE
1252- *min_arch_log_no = arch_log_no;
1253- *max_arch_log_no = arch_log_no;
1254-#endif /* UNIV_LOG_ARCHIVE */
1255+
1256 return;
1257 }
1258
1259@@ -1989,14 +2013,6 @@
1260 if (*max_flushed_lsn < flushed_lsn) {
1261 *max_flushed_lsn = flushed_lsn;
1262 }
1263-#ifdef UNIV_LOG_ARCHIVE
1264- if (*min_arch_log_no > arch_log_no) {
1265- *min_arch_log_no = arch_log_no;
1266- }
1267- if (*max_arch_log_no < arch_log_no) {
1268- *max_arch_log_no = arch_log_no;
1269- }
1270-#endif /* UNIV_LOG_ARCHIVE */
1271 }
1272
1273 /*================ SINGLE-TABLE TABLESPACES ==========================*/
1274@@ -3499,9 +3515,6 @@
1275 lsn_t lsn; /*!< Flushed LSN from header page */
1276 ulint id; /*!< Space ID */
1277 ulint flags; /*!< Tablespace flags */
1278-#ifdef UNIV_LOG_ARCHIVE
1279- ulint arch_log_no; /*!< latest archived log file number */
1280-#endif /* UNIV_LOG_ARCHIVE */
1281 };
1282
1283 /********************************************************************//**
1284@@ -3637,9 +3650,6 @@
1285 if (def.success) {
1286 fil_read_first_page(
1287 def.file, FALSE, &def.flags, &def.id,
1288-#ifdef UNIV_LOG_ARCHIVE
1289- &space_arch_log_no, &space_arch_log_no,
1290-#endif /* UNIV_LOG_ARCHIVE */
1291 &def.lsn, &def.lsn);
1292
1293 /* Validate this single-table-tablespace with SYS_TABLES,
1294@@ -3661,9 +3671,6 @@
1295 if (remote.success) {
1296 fil_read_first_page(
1297 remote.file, FALSE, &remote.flags, &remote.id,
1298-#ifdef UNIV_LOG_ARCHIVE
1299- &remote.arch_log_no, &remote.arch_log_no,
1300-#endif /* UNIV_LOG_ARCHIVE */
1301 &remote.lsn, &remote.lsn);
1302
1303 /* Validate this single-table-tablespace with SYS_TABLES,
1304@@ -3686,9 +3693,6 @@
1305 if (dict.success) {
1306 fil_read_first_page(
1307 dict.file, FALSE, &dict.flags, &dict.id,
1308-#ifdef UNIV_LOG_ARCHIVE
1309- &dict.arch_log_no, &dict.arch_log_no,
1310-#endif /* UNIV_LOG_ARCHIVE */
1311 &dict.lsn, &dict.lsn);
1312
1313 /* Validate this single-table-tablespace with SYS_TABLES,
1314@@ -3916,9 +3920,6 @@
1315 {
1316 fil_read_first_page(
1317 fsp->file, FALSE, &fsp->flags, &fsp->id,
1318-#ifdef UNIV_LOG_ARCHIVE
1319- &fsp->arch_log_no, &fsp->arch_log_no,
1320-#endif /* UNIV_LOG_ARCHIVE */
1321 &fsp->lsn, &fsp->lsn);
1322
1323 if (fsp->id == ULINT_UNDEFINED || fsp->id == 0) {
1324
1325=== modified file 'Percona-Server/storage/innobase/handler/ha_innodb.cc'
1326--- Percona-Server/storage/innobase/handler/ha_innodb.cc 2013-03-05 12:46:43 +0000
1327+++ Percona-Server/storage/innobase/handler/ha_innodb.cc 2013-03-31 06:25:28 +0000
1328@@ -989,6 +989,32 @@
1329 name to lower case */
1330
1331 /*************************************************************//**
1332+Removes old archived transaction log files.
1333+@return true on error */
1334+static bool innobase_purge_archive_logs(
1335+ handlerton *hton, /*!< in: InnoDB handlerton */
1336+ time_t before_date, /*!< in: all files modified
1337+ before timestamp should be removed */
1338+ const char* to_filename) /*!< in: this and earler files
1339+ should be removed */
1340+{
1341+ ulint err= DB_ERROR;
1342+ if (before_date > 0) {
1343+ err= purge_archived_logs(before_date, 0);
1344+ } else if (to_filename) {
1345+ if (is_prefix(to_filename, IB_ARCHIVED_LOGS_PREFIX)) {
1346+ unsigned long long log_file_lsn = strtoll(to_filename
1347+ + IB_ARCHIVED_LOGS_PREFIX_LEN,
1348+ NULL, 10);
1349+ if (log_file_lsn > 0 && log_file_lsn < ULLONG_MAX) {
1350+ err= purge_archived_logs(0, log_file_lsn);
1351+ }
1352+ }
1353+ }
1354+ return (err != DB_SUCCESS);
1355+}
1356+
1357+/*************************************************************//**
1358 Check for a valid value of innobase_commit_concurrency.
1359 @return 0 for valid innodb_commit_concurrency */
1360 static
1361@@ -2760,6 +2786,7 @@
1362
1363 innobase_hton->release_temporary_latches =
1364 innobase_release_temporary_latches;
1365+ innobase_hton->purge_archive_logs = innobase_purge_archive_logs;
1366
1367 innobase_hton->data = &innodb_api_cb;
1368
1369@@ -2862,12 +2889,9 @@
1370 }
1371
1372 #ifdef UNIV_LOG_ARCHIVE
1373- /* Since innodb_log_arch_dir has no relevance under MySQL,
1374- starting from 4.0.6 we always set it the same as
1375- innodb_log_group_home_dir: */
1376-
1377- innobase_log_arch_dir = innobase_log_group_home_dir;
1378-
1379+ if (!innobase_log_arch_dir) {
1380+ innobase_log_arch_dir = srv_log_group_home_dir;
1381+ }
1382 srv_arch_dir = innobase_log_arch_dir;
1383 #endif /* UNIG_LOG_ARCHIVE */
1384
1385@@ -13684,6 +13708,45 @@
1386 }
1387
1388 /****************************************************************//**
1389+Update the system variable innodb_log_arch_expire_sec using
1390+the "saved" value. This function is registered as a callback with MySQL. */
1391+static
1392+void
1393+innodb_log_archive_expire_update(
1394+/*==============================*/
1395+ THD* thd, /*!< in: thread handle */
1396+ struct st_mysql_sys_var* var, /*!< in: pointer to
1397+ system variable */
1398+ void* var_ptr, /*!< out: unused */
1399+ const void* save) /*!< in: immediate result
1400+ from check function */
1401+{
1402+ srv_log_arch_expire_sec = *(ulint*) save;
1403+}
1404+
1405+static
1406+void
1407+innodb_log_archive_update(
1408+/*======================*/
1409+ THD* thd,
1410+ struct st_mysql_sys_var* var,
1411+ void* var_ptr,
1412+ const void* save)
1413+{
1414+ my_bool in_val = *static_cast<const my_bool*>(save);
1415+
1416+ if (in_val) {
1417+ /* turn archiving on */
1418+ srv_log_archive_on = innobase_log_archive = 1;
1419+ log_archive_archivelog();
1420+ } else {
1421+ /* turn archivng off */
1422+ srv_log_archive_on = innobase_log_archive = 0;
1423+ log_archive_noarchivelog();
1424+ }
1425+}
1426+
1427+/****************************************************************//**
1428 Update the system variable innodb_max_dirty_pages_pct using the "saved"
1429 value. This function is registered as a callback with MySQL. */
1430 static
1431@@ -15486,14 +15549,20 @@
1432 "Where full logs should be archived.", NULL, NULL, NULL);
1433
1434 static MYSQL_SYSVAR_BOOL(log_archive, innobase_log_archive,
1435- PLUGIN_VAR_OPCMDARG | PLUGIN_VAR_READONLY,
1436- "Set to 1 if you want to have logs archived.", NULL, NULL, FALSE);
1437+ PLUGIN_VAR_OPCMDARG,
1438+ "Set to 1 if you want to have logs archived.",
1439+ NULL, innodb_log_archive_update, FALSE);
1440 #endif /* UNIV_LOG_ARCHIVE */
1441
1442 static MYSQL_SYSVAR_STR(log_group_home_dir, srv_log_group_home_dir,
1443 PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY,
1444 "Path to InnoDB log files.", NULL, NULL, NULL);
1445
1446+static MYSQL_SYSVAR_ULONG(log_arch_expire_sec,
1447+ srv_log_arch_expire_sec, PLUGIN_VAR_OPCMDARG,
1448+ "Expiration time for archived innodb transaction logs.",
1449+ NULL, innodb_log_archive_expire_update, 0, 0, ~0L, 0);
1450+
1451 static MYSQL_SYSVAR_ULONG(max_dirty_pages_pct, srv_max_buf_pool_modified_pct,
1452 PLUGIN_VAR_RQCMDARG,
1453 "Percentage of dirty pages allowed in bufferpool.",
1454@@ -16140,6 +16209,7 @@
1455 #ifdef UNIV_LOG_ARCHIVE
1456 MYSQL_SYSVAR(log_arch_dir),
1457 MYSQL_SYSVAR(log_archive),
1458+ MYSQL_SYSVAR(log_arch_expire_sec),
1459 #endif /* UNIV_LOG_ARCHIVE */
1460 MYSQL_SYSVAR(page_size),
1461 MYSQL_SYSVAR(log_buffer_size),
1462
1463=== modified file 'Percona-Server/storage/innobase/include/fil0fil.h'
1464--- Percona-Server/storage/innobase/include/fil0fil.h 2013-03-05 12:46:43 +0000
1465+++ Percona-Server/storage/innobase/include/fil0fil.h 2013-03-31 06:25:28 +0000
1466@@ -229,6 +229,14 @@
1467 ulint trunc_len); /*!< in: truncate by this much; it is an error
1468 if this does not equal to the combined size of
1469 some initial files in the space */
1470+/****************************************************************//**
1471+Check is there node in file space with given name. */
1472+UNIV_INTERN
1473+ibool
1474+fil_space_contains_node(
1475+/*====================*/
1476+ ulint id, /*!< in: space id */
1477+ char* node_name); /*!< in: node name */
1478 #endif /* UNIV_LOG_ARCHIVE */
1479 /*******************************************************************//**
1480 Creates a space memory object and puts it to the 'fil system' hash table.
1481@@ -371,12 +379,6 @@
1482 contain sensible data */
1483 ulint* flags, /*!< out: tablespace flags */
1484 ulint* space_id, /*!< out: tablespace ID */
1485-#ifdef UNIV_LOG_ARCHIVE
1486- ulint* min_arch_log_no, /*!< out: min of archived
1487- log numbers in data files */
1488- ulint* max_arch_log_no, /*!< out: max of archived
1489- log numbers in data files */
1490-#endif /* UNIV_LOG_ARCHIVE */
1491 lsn_t* min_flushed_lsn, /*!< out: min of flushed
1492 lsn values in data files */
1493 lsn_t* max_flushed_lsn); /*!< out: max of flushed
1494
1495=== modified file 'Percona-Server/storage/innobase/include/log0log.h'
1496--- Percona-Server/storage/innobase/include/log0log.h 2013-03-05 12:46:43 +0000
1497+++ Percona-Server/storage/innobase/include/log0log.h 2013-03-31 06:25:28 +0000
1498@@ -70,6 +70,10 @@
1499 /** Maximum number of log groups in log_group_t::checkpoint_buf */
1500 #define LOG_MAX_N_GROUPS 32
1501
1502+#define IB_ARCHIVED_LOGS_PREFIX "ib_log_archive_"
1503+#define IB_ARCHIVED_LOGS_PREFIX_LEN (sizeof(IB_ARCHIVED_LOGS_PREFIX) - 1)
1504+#define IB_ARCHIVED_LOGS_SERIAL_LEN 20
1505+
1506 /*******************************************************************//**
1507 Calculates where in log files we find a specified lsn.
1508 @return log file number */
1509@@ -283,8 +287,7 @@
1510 /*==============================*/
1511 const byte* buf, /*!< in: buffer containing checkpoint info */
1512 ulint n, /*!< in: nth slot */
1513- ulint* file_no,/*!< out: archived file number */
1514- ulint* offset);/*!< out: archived file offset */
1515+ lsn_t* file_no);/*!< out: archived file number */
1516 /******************************************************//**
1517 Writes checkpoint info to groups. */
1518 UNIV_INTERN
1519@@ -340,8 +343,18 @@
1520 log_archived_file_name_gen(
1521 /*=======================*/
1522 char* buf, /*!< in: buffer where to write */
1523+ ulint buf_len,/*!< in: buffer length */
1524 ulint id, /*!< in: group id */
1525- ulint file_no);/*!< in: file number */
1526+ lsn_t file_no);/*!< in: file number */
1527+
1528+UNIV_INTERN
1529+void
1530+log_archived_get_offset(
1531+/*====================*/
1532+ log_group_t* group, /*!< in: log group */
1533+ lsn_t file_no, /*!< in: archive log file number */
1534+ lsn_t archived_lsn, /*!< in: last archived LSN */
1535+ lsn_t* offset); /*!< out: offset within archived file */
1536 #else /* !UNIV_HOTBACKUP */
1537 /******************************************************//**
1538 Writes info to a buffer of a log group when log files are created in
1539@@ -736,19 +749,19 @@
1540 ulint archive_space_id;/*!< file space which
1541 implements the log group
1542 archive */
1543- ulint archived_file_no;/*!< file number corresponding to
1544+ lsn_t archived_file_no;/*!< file number corresponding to
1545 log_sys->archived_lsn */
1546- ulint archived_offset;/*!< file offset corresponding to
1547+ lsn_t archived_offset;/*!< file offset corresponding to
1548 log_sys->archived_lsn, 0 if we have
1549 not yet written to the archive file
1550 number archived_file_no */
1551- ulint next_archived_file_no;/*!< during an archive write,
1552+ lsn_t next_archived_file_no;/*!< during an archive write,
1553 until the write is completed, we
1554 store the next value for
1555 archived_file_no here: the write
1556 completion function then sets the new
1557 value to ..._file_no */
1558- ulint next_archived_offset; /*!< like the preceding field */
1559+ lsn_t next_archived_offset; /*!< like the preceding field */
1560 #endif /* UNIV_LOG_ARCHIVE */
1561 /*-----------------------------*/
1562 lsn_t scanned_lsn; /*!< used only in recovery: recovery scan
1563
1564=== modified file 'Percona-Server/storage/innobase/include/log0recv.h'
1565--- Percona-Server/storage/innobase/include/log0recv.h 2013-03-05 12:46:43 +0000
1566+++ Percona-Server/storage/innobase/include/log0recv.h 2013-03-31 06:25:28 +0000
1567@@ -297,14 +297,14 @@
1568 Recovers from archived log files, and also from log files, if they exist.
1569 @return error code or DB_SUCCESS */
1570 UNIV_INTERN
1571-ulint
1572+dberr_t
1573 recv_recovery_from_archive_start(
1574 /*=============================*/
1575 lsn_t min_flushed_lsn,/*!< in: min flushed lsn field from the
1576 data files */
1577 lsn_t limit_lsn, /*!< in: recover up to this lsn if
1578 possible */
1579- ulint first_log_no); /*!< in: number of the first archived
1580+ lsn_t first_log_no); /*!< in: number of the first archived
1581 log file to use in the recovery; the
1582 file will be searched from
1583 INNOBASE_LOG_ARCH_DIR specified in
1584
1585=== modified file 'Percona-Server/storage/innobase/include/srv0srv.h'
1586--- Percona-Server/storage/innobase/include/srv0srv.h 2013-03-05 12:46:43 +0000
1587+++ Percona-Server/storage/innobase/include/srv0srv.h 2013-03-31 06:25:28 +0000
1588@@ -348,6 +348,8 @@
1589 extern ulong srv_doublewrite_batch_size;
1590 extern ulong srv_checksum_algorithm;
1591
1592+extern ulong srv_log_arch_expire_sec;
1593+
1594 extern ulong srv_max_buf_pool_modified_pct;
1595 extern ulong srv_max_purge_lag;
1596 extern ulong srv_max_purge_lag_delay;
1597@@ -651,6 +653,18 @@
1598 void
1599 srv_export_innodb_status(void);
1600 /*==========================*/
1601+/*************************************************************//**
1602+Removes old archived transaction log files.
1603+Both parameters couldn't be provided at the same time.
1604+@return DB_SUCCESS on success, otherwise DB_ERROR */
1605+UNIV_INTERN
1606+dberr_t
1607+purge_archived_logs(
1608+ time_t before_date, /*!< in: all files modified
1609+ before timestamp should be removed */
1610+ lsn_t before_lsn); /*!< in: files with this lsn in name
1611+ and earler should be removed */
1612+/*==========================*/
1613 /*******************************************************************//**
1614 Get current server activity count. We don't hold srv_sys::mutex while
1615 reading this value as it is only used in heuristics.
1616
1617=== modified file 'Percona-Server/storage/innobase/include/univ.i'
1618--- Percona-Server/storage/innobase/include/univ.i 2013-03-05 12:46:43 +0000
1619+++ Percona-Server/storage/innobase/include/univ.i 2013-03-31 06:25:28 +0000
1620@@ -46,6 +46,9 @@
1621 #define INNODB_VERSION_MINOR 2
1622 #define INNODB_VERSION_BUGFIX MYSQL_VERSION_PATCH
1623
1624+/* Enable UNIV_LOG_ARCHIVE in XtraDB */
1625+#define UNIV_LOG_ARCHIVE 1
1626+
1627 /* The following is the InnoDB version as shown in
1628 SELECT plugin_version FROM information_schema.plugins;
1629 calculated in make_version_string() in sql/sql_show.cc like this:
1630
1631=== modified file 'Percona-Server/storage/innobase/log/log0log.cc'
1632--- Percona-Server/storage/innobase/log/log0log.cc 2013-03-05 12:46:43 +0000
1633+++ Percona-Server/storage/innobase/log/log0log.cc 2013-03-31 06:25:28 +0000
1634@@ -817,15 +817,11 @@
1635 rw_lock_create(archive_lock_key, &log_sys->archive_lock,
1636 SYNC_NO_ORDER_CHECK);
1637
1638- log_sys->archive_buf = NULL;
1639-
1640- /* ut_align(
1641- ut_malloc(LOG_ARCHIVE_BUF_SIZE
1642- + OS_FILE_LOG_BLOCK_SIZE),
1643- OS_FILE_LOG_BLOCK_SIZE); */
1644- log_sys->archive_buf_size = 0;
1645-
1646- /* memset(log_sys->archive_buf, '\0', LOG_ARCHIVE_BUF_SIZE); */
1647+ log_sys->archive_buf = static_cast<byte*>(
1648+ ut_align(mem_zalloc(LOG_ARCHIVE_BUF_SIZE
1649+ + OS_FILE_LOG_BLOCK_SIZE),
1650+ OS_FILE_LOG_BLOCK_SIZE));
1651+ log_sys->archive_buf_size = LOG_ARCHIVE_BUF_SIZE;
1652
1653 log_sys->archiving_on = os_event_create();
1654 #endif /* UNIV_LOG_ARCHIVE */
1655@@ -896,10 +892,10 @@
1656 mem_zalloc(sizeof(byte**) * n_files));
1657
1658 #ifdef UNIV_LOG_ARCHIVE
1659- group->archive_file_header_bufs_ptr = static_cast<byte*>(
1660+ group->archive_file_header_bufs_ptr = static_cast<byte**>(
1661 mem_zalloc( sizeof(byte*) * n_files));
1662
1663- group->archive_file_header_bufs = static_cast<byte*>(
1664+ group->archive_file_header_bufs = static_cast<byte**>(
1665 mem_zalloc(sizeof(byte*) * n_files));
1666 #endif /* UNIV_LOG_ARCHIVE */
1667
1668@@ -924,7 +920,7 @@
1669 #ifdef UNIV_LOG_ARCHIVE
1670 group->archive_space_id = archive_space_id;
1671
1672- group->archived_file_no = 0;
1673+ group->archived_file_no = LOG_START_LSN;
1674 group->archived_offset = 0;
1675 #endif /* UNIV_LOG_ARCHIVE */
1676
1677@@ -1708,15 +1704,13 @@
1678 /*==============================*/
1679 byte* buf, /*!< in: buffer for checkpoint info */
1680 ulint n, /*!< in: nth slot */
1681- ulint file_no,/*!< in: archived file number */
1682- ulint offset) /*!< in: archived file offset */
1683+ lsn_t file_no)/*!< in: archived file number */
1684 {
1685 ut_ad(n < LOG_MAX_N_GROUPS);
1686
1687- mach_write_to_4(buf + LOG_CHECKPOINT_GROUP_ARRAY
1688- + 8 * n + LOG_CHECKPOINT_ARCHIVED_FILE_NO, file_no);
1689- mach_write_to_4(buf + LOG_CHECKPOINT_GROUP_ARRAY
1690- + 8 * n + LOG_CHECKPOINT_ARCHIVED_OFFSET, offset);
1691+ mach_write_to_8(buf + LOG_CHECKPOINT_GROUP_ARRAY +
1692+ 8 * n + LOG_CHECKPOINT_ARCHIVED_FILE_NO,
1693+ file_no);
1694 }
1695
1696 /*******************************************************************//**
1697@@ -1727,15 +1721,12 @@
1698 /*==============================*/
1699 const byte* buf, /*!< in: buffer containing checkpoint info */
1700 ulint n, /*!< in: nth slot */
1701- ulint* file_no,/*!< out: archived file number */
1702- ulint* offset) /*!< out: archived file offset */
1703+ lsn_t* file_no)/*!< out: archived file number */
1704 {
1705 ut_ad(n < LOG_MAX_N_GROUPS);
1706
1707- *file_no = mach_read_from_4(buf + LOG_CHECKPOINT_GROUP_ARRAY
1708- + 8 * n + LOG_CHECKPOINT_ARCHIVED_FILE_NO);
1709- *offset = mach_read_from_4(buf + LOG_CHECKPOINT_GROUP_ARRAY
1710- + 8 * n + LOG_CHECKPOINT_ARCHIVED_OFFSET);
1711+ *file_no = mach_read_from_8(buf + LOG_CHECKPOINT_GROUP_ARRAY +
1712+ 8 * n + LOG_CHECKPOINT_ARCHIVED_FILE_NO);
1713 }
1714
1715 /******************************************************//**
1716@@ -1795,7 +1786,7 @@
1717 #endif /* UNIV_LOG_ARCHIVE */
1718
1719 for (i = 0; i < LOG_MAX_N_GROUPS; i++) {
1720- log_checkpoint_set_nth_group_info(buf, i, 0, 0);
1721+ log_checkpoint_set_nth_group_info(buf, i, 0);
1722 }
1723
1724 group2 = UT_LIST_GET_FIRST(log_sys->log_groups);
1725@@ -1803,10 +1794,9 @@
1726 while (group2) {
1727 log_checkpoint_set_nth_group_info(buf, group2->id,
1728 #ifdef UNIV_LOG_ARCHIVE
1729- group2->archived_file_no,
1730- group2->archived_offset
1731+ group2->archived_file_no
1732 #else /* UNIV_LOG_ARCHIVE */
1733- 0, 0
1734+ 0
1735 #endif /* UNIV_LOG_ARCHIVE */
1736 );
1737
1738@@ -2225,7 +2215,7 @@
1739 fil_io(OS_FILE_READ | OS_FILE_LOG, sync, group->space_id, 0,
1740 (ulint) (source_offset / UNIV_PAGE_SIZE),
1741 (ulint) (source_offset % UNIV_PAGE_SIZE),
1742- len, buf, NULL);
1743+ len, buf, (type == LOG_ARCHIVE) ? &log_archive_io : NULL);
1744
1745 start_lsn += len;
1746 buf += len;
1747@@ -2244,12 +2234,68 @@
1748 log_archived_file_name_gen(
1749 /*=======================*/
1750 char* buf, /*!< in: buffer where to write */
1751+ ulint buf_len,/*!< in: buffer length */
1752 ulint id __attribute__((unused)),
1753 /*!< in: group id;
1754 currently we only archive the first group */
1755- ulint file_no)/*!< in: file number */
1756-{
1757- sprintf(buf, "%sib_arch_log_%010lu", srv_arch_dir, (ulong) file_no);
1758+ lsn_t file_no)/*!< in: file number */
1759+{
1760+ ulint dirnamelen;
1761+
1762+ dirnamelen = strlen(srv_arch_dir);
1763+
1764+ ut_a(buf_len > dirnamelen +
1765+ IB_ARCHIVED_LOGS_SERIAL_LEN +
1766+ IB_ARCHIVED_LOGS_PREFIX_LEN + 2);
1767+
1768+ strcpy(buf, srv_arch_dir);
1769+
1770+ if (buf[dirnamelen-1] != SRV_PATH_SEPARATOR) {
1771+ buf[dirnamelen++] = SRV_PATH_SEPARATOR;
1772+ }
1773+ sprintf(buf + dirnamelen, IB_ARCHIVED_LOGS_PREFIX
1774+ "%0" IB_TO_STR(IB_ARCHIVED_LOGS_SERIAL_LEN) "llu",
1775+ file_no);
1776+}
1777+
1778+/******************************************************//**
1779+Get offset within archived log file to continue to write
1780+with. */
1781+UNIV_INTERN
1782+void
1783+log_archived_get_offset(
1784+/*=====================*/
1785+ log_group_t* group, /*!< in: log group */
1786+ lsn_t file_no, /*!< in: archive log file number */
1787+ lsn_t archived_lsn, /*!< in: last archived LSN */
1788+ lsn_t* offset) /*!< out: offset within archived file */
1789+{
1790+ char file_name[OS_FILE_MAX_PATH];
1791+ ibool exists;
1792+ os_file_type_t type;
1793+
1794+ log_archived_file_name_gen(file_name,
1795+ sizeof(file_name), group->id, file_no);
1796+
1797+ ut_a(os_file_status(file_name, &exists, &type));
1798+
1799+ if (!exists) {
1800+ *offset = 0;
1801+ return;
1802+ }
1803+
1804+ *offset = archived_lsn - file_no + LOG_FILE_HDR_SIZE;
1805+
1806+ if (archived_lsn != IB_ULONGLONG_MAX) {
1807+ *offset = archived_lsn - file_no + LOG_FILE_HDR_SIZE;
1808+ } else {
1809+ /* Archiving was OFF prior startup */
1810+ *offset = 0;
1811+ }
1812+
1813+ ut_a(group->file_size >= *offset + LOG_FILE_HDR_SIZE);
1814+
1815+ return;
1816 }
1817
1818 /******************************************************//**
1819@@ -2261,7 +2307,7 @@
1820 log_group_t* group, /*!< in: log group */
1821 ulint nth_file, /*!< in: header to the nth file in the
1822 archive log file space */
1823- ulint file_no, /*!< in: archived file number */
1824+ lsn_t file_no, /*!< in: archived file number */
1825 ib_uint64_t start_lsn) /*!< in: log file data starts at this
1826 lsn */
1827 {
1828@@ -2287,6 +2333,7 @@
1829 MONITOR_INC(MONITOR_LOG_IO);
1830
1831 fil_io(OS_FILE_WRITE | OS_FILE_LOG, TRUE, group->archive_space_id,
1832+ 0,
1833 dest_offset / UNIV_PAGE_SIZE,
1834 dest_offset % UNIV_PAGE_SIZE,
1835 2 * OS_FILE_LOG_BLOCK_SIZE,
1836@@ -2322,6 +2369,7 @@
1837 MONITOR_INC(MONITOR_LOG_IO);
1838
1839 fil_io(OS_FILE_WRITE | OS_FILE_LOG, TRUE, group->archive_space_id,
1840+ 0,
1841 dest_offset / UNIV_PAGE_SIZE,
1842 dest_offset % UNIV_PAGE_SIZE,
1843 OS_FILE_LOG_BLOCK_SIZE,
1844@@ -2340,7 +2388,7 @@
1845 os_file_t file_handle;
1846 lsn_t start_lsn;
1847 lsn_t end_lsn;
1848- char name[1024];
1849+ char name[OS_FILE_MAX_PATH];
1850 byte* buf;
1851 ulint len;
1852 ibool ret;
1853@@ -2372,12 +2420,19 @@
1854
1855 if (next_offset % group->file_size == 0) {
1856 open_mode = OS_FILE_CREATE;
1857+ if (n_files == 0) {
1858+ /* Adjust archived_file_no to match start_lsn
1859+ which is written in file header as well */
1860+ group->archived_file_no = start_lsn;
1861+ }
1862 } else {
1863 open_mode = OS_FILE_OPEN;
1864 }
1865
1866- log_archived_file_name_gen(name, group->id,
1867- group->archived_file_no + n_files);
1868+ log_archived_file_name_gen(name, sizeof(name), group->id,
1869+ group->archived_file_no +
1870+ n_files * (group->file_size -
1871+ LOG_FILE_HDR_SIZE));
1872
1873 file_handle = os_file_create(innodb_file_log_key,
1874 name, open_mode,
1875@@ -2414,13 +2469,14 @@
1876
1877 /* Add the archive file as a node to the space */
1878
1879- fil_node_create(name, group->file_size / UNIV_PAGE_SIZE,
1880- group->archive_space_id, FALSE);
1881+ ut_a(fil_node_create(name, group->file_size / UNIV_PAGE_SIZE,
1882+ group->archive_space_id, FALSE));
1883
1884 if (next_offset % group->file_size == 0) {
1885 log_group_archive_file_header_write(
1886 group, n_files,
1887- group->archived_file_no + n_files,
1888+ group->archived_file_no +
1889+ n_files * (group->file_size - LOG_FILE_HDR_SIZE),
1890 start_lsn);
1891
1892 next_offset += LOG_FILE_HDR_SIZE;
1893@@ -2451,6 +2507,7 @@
1894 MONITOR_INC(MONITOR_LOG_IO);
1895
1896 fil_io(OS_FILE_WRITE | OS_FILE_LOG, FALSE, group->archive_space_id,
1897+ 0,
1898 (ulint) (next_offset / UNIV_PAGE_SIZE),
1899 (ulint) (next_offset % UNIV_PAGE_SIZE),
1900 ut_calc_align(len, OS_FILE_LOG_BLOCK_SIZE), buf,
1901@@ -2469,7 +2526,8 @@
1902 goto loop;
1903 }
1904
1905- group->next_archived_file_no = group->archived_file_no + n_files;
1906+ group->next_archived_file_no = group->archived_file_no +
1907+ n_files * (group->file_size - LOG_FILE_HDR_SIZE);
1908 group->next_archived_offset = next_offset % group->file_size;
1909
1910 ut_a(group->next_archived_offset % OS_FILE_LOG_BLOCK_SIZE == 0);
1911@@ -2501,7 +2559,7 @@
1912 /*===================================*/
1913 {
1914 log_group_t* group;
1915- ulint end_offset;
1916+ lsn_t end_offset;
1917 ulint trunc_files;
1918 ulint n_files;
1919 ib_uint64_t start_lsn;
1920@@ -2822,18 +2880,8 @@
1921 trunc_len);
1922 if (increment_file_count) {
1923 group->archived_offset = 0;
1924- group->archived_file_no += 2;
1925 }
1926
1927-#ifdef UNIV_DEBUG
1928- if (log_debug_writes) {
1929- fprintf(stderr,
1930- "Incrementing arch file no to %lu"
1931- " in log group %lu\n",
1932- (ulong) group->archived_file_no + 2,
1933- (ulong) group->id);
1934- }
1935-#endif /* UNIV_DEBUG */
1936 }
1937 }
1938
1939@@ -3087,7 +3135,6 @@
1940 /*=======================================*/
1941 {
1942 lsn_t lsn;
1943- ulint arch_log_no;
1944 ulint count = 0;
1945 ulint total_trx;
1946 ulint pending_io;
1947@@ -3304,15 +3351,7 @@
1948 goto loop;
1949 }
1950
1951- arch_log_no = 0;
1952-
1953 #ifdef UNIV_LOG_ARCHIVE
1954- UT_LIST_GET_FIRST(log_sys->log_groups)->archived_file_no;
1955-
1956- if (0 == UT_LIST_GET_FIRST(log_sys->log_groups)->archived_offset) {
1957-
1958- arch_log_no--;
1959- }
1960
1961 log_archive_close_groups(TRUE);
1962 #endif /* UNIV_LOG_ARCHIVE */
1963@@ -3371,7 +3410,7 @@
1964 srv_shutdown_lsn = lsn;
1965
1966 if (!srv_read_only_mode) {
1967- fil_write_flushed_lsn_to_data_files(lsn, arch_log_no);
1968+ fil_write_flushed_lsn_to_data_files(lsn, 0);
1969
1970 fil_flush_file_spaces(FIL_TABLESPACE);
1971 }
1972@@ -3594,7 +3633,7 @@
1973
1974 #ifdef UNIV_LOG_ARCHIVE
1975 rw_lock_free(&log_sys->archive_lock);
1976- os_event_create();
1977+ os_event_free(log_sys->archiving_on);
1978 #endif /* UNIV_LOG_ARCHIVE */
1979
1980 #ifdef UNIV_LOG_DEBUG
1981
1982=== modified file 'Percona-Server/storage/innobase/log/log0recv.cc'
1983--- Percona-Server/storage/innobase/log/log0recv.cc 2013-03-05 12:46:43 +0000
1984+++ Percona-Server/storage/innobase/log/log0recv.cc 2013-03-31 06:25:28 +0000
1985@@ -3115,8 +3115,10 @@
1986
1987 while (group) {
1988 log_checkpoint_get_nth_group_info(buf, group->id,
1989- &(group->archived_file_no),
1990- &(group->archived_offset));
1991+ &(group->archived_file_no));
1992+
1993+ log_archived_get_offset(group, group->archived_file_no,
1994+ archived_lsn, &(group->archived_offset));
1995
1996 group = UT_LIST_GET_NEXT(log_groups, group);
1997 }
1998@@ -3690,7 +3692,7 @@
1999 os_offset_t read_offset;
2000 os_offset_t file_size;
2001 int input_char;
2002- char name[10000];
2003+ char name[OS_FILE_MAX_PATH];
2004
2005 ut_a(0);
2006
2007@@ -3699,7 +3701,8 @@
2008
2009 /* Add the file to the archive file space; open the file */
2010
2011- log_archived_file_name_gen(name, group->id, group->archived_file_no);
2012+ log_archived_file_name_gen(name, sizeof(name),
2013+ group->id, group->archived_file_no);
2014
2015 file_handle = os_file_create(innodb_file_log_key,
2016 name, OS_FILE_OPEN,
2017@@ -3748,20 +3751,19 @@
2018
2019 /* Add the archive file as a node to the space */
2020
2021- fil_node_create(name, 1 + file_size / UNIV_PAGE_SIZE,
2022- group->archive_space_id, FALSE);
2023-#if RECV_SCAN_SIZE < LOG_FILE_HDR_SIZE
2024-# error "RECV_SCAN_SIZE < LOG_FILE_HDR_SIZE"
2025-#endif
2026+ ut_a(fil_node_create(name, 1 + file_size / UNIV_PAGE_SIZE,
2027+ group->archive_space_id, FALSE));
2028+ ut_a(RECV_SCAN_SIZE >= LOG_FILE_HDR_SIZE);
2029
2030 /* Read the archive file header */
2031- fil_io(OS_FILE_READ | OS_FILE_LOG, TRUE, group->archive_space_id, 0, 0,
2032+ fil_io(OS_FILE_READ | OS_FILE_LOG, TRUE, group->archive_space_id, 0,
2033+ 0, 0,
2034 LOG_FILE_HDR_SIZE, buf, NULL);
2035
2036 /* Check if the archive file header is consistent */
2037
2038 if (mach_read_from_4(buf + LOG_GROUP_ID) != group->id
2039- || mach_read_from_4(buf + LOG_FILE_NO)
2040+ || mach_read_from_8(buf + LOG_FILE_START_LSN)
2041 != group->archived_file_no) {
2042 fprintf(stderr,
2043 "InnoDB: Archive file header inconsistent %s\n", name);
2044@@ -3828,7 +3830,7 @@
2045 #endif /* UNIV_DEBUG */
2046
2047 fil_io(OS_FILE_READ | OS_FILE_LOG, TRUE,
2048- group->archive_space_id, read_offset / UNIV_PAGE_SIZE,
2049+ group->archive_space_id, 0, read_offset / UNIV_PAGE_SIZE,
2050 read_offset % UNIV_PAGE_SIZE, len, buf, NULL);
2051
2052 ret = recv_scan_log_recs(
2053@@ -3863,14 +3865,14 @@
2054 Recovers from archived log files, and also from log files, if they exist.
2055 @return error code or DB_SUCCESS */
2056 UNIV_INTERN
2057-ulint
2058+dberr_t
2059 recv_recovery_from_archive_start(
2060 /*=============================*/
2061 ib_uint64_t min_flushed_lsn,/*!< in: min flushed lsn field from the
2062 data files */
2063 ib_uint64_t limit_lsn, /*!< in: recover up to this lsn if
2064 possible */
2065- ulint first_log_no) /*!< in: number of the first archived
2066+ lsn_t first_log_no) /*!< in: number of the first archived
2067 log file to use in the recovery; the
2068 file will be searched from
2069 INNOBASE_LOG_ARCH_DIR specified in
2070@@ -3880,7 +3882,7 @@
2071 ulint group_id;
2072 ulint trunc_len;
2073 ibool ret;
2074- ulint err;
2075+ dberr_t err;
2076
2077 ut_a(0);
2078
2079@@ -3939,7 +3941,7 @@
2080 trunc_len);
2081 }
2082
2083- group->archived_file_no++;
2084+ group->archived_file_no += group->file_size - LOG_FILE_HDR_SIZE;
2085 }
2086
2087 if (recv_sys->recovered_lsn < limit_lsn) {
2088
2089=== modified file 'Percona-Server/storage/innobase/srv/srv0srv.cc'
2090--- Percona-Server/storage/innobase/srv/srv0srv.cc 2013-03-05 12:46:43 +0000
2091+++ Percona-Server/storage/innobase/srv/srv0srv.cc 2013-03-31 06:25:28 +0000
2092@@ -67,6 +67,7 @@
2093 #include "os0sync.h" /* for HAVE_ATOMIC_BUILTINS */
2094 #include "srv0mon.h"
2095 #include "ut0crc32.h"
2096+#include "os0file.h"
2097
2098 #include "mysql/plugin.h"
2099 #include "mysql/service_thd_wait.h"
2100@@ -111,6 +112,7 @@
2101
2102 #ifdef UNIV_LOG_ARCHIVE
2103 UNIV_INTERN char* srv_arch_dir = NULL;
2104+UNIV_INTERN ulong srv_log_arch_expire_sec = 0;
2105 #endif /* UNIV_LOG_ARCHIVE */
2106
2107 /** Set if InnoDB must operate in read-only mode. We don't do any
2108@@ -1852,6 +1854,134 @@
2109 return(thread_active);
2110 }
2111
2112+/*************************************************************//**
2113+Removes old archived transaction log files.
2114+Both parameters couldn't be provided at the same time */
2115+dberr_t
2116+purge_archived_logs(
2117+ time_t before_date, /*!< in: all files modified
2118+ before timestamp should be removed */
2119+ lsn_t before_no) /*!< in: files with this number in name
2120+ and earler should be removed */
2121+{
2122+ log_group_t* group = UT_LIST_GET_FIRST(log_sys->log_groups);
2123+
2124+ os_file_dir_t dir;
2125+ os_file_stat_t fileinfo;
2126+ char archived_log_filename[OS_FILE_MAX_PATH];
2127+ char namegen[OS_FILE_MAX_PATH];
2128+ ulint dirnamelen;
2129+
2130+ if (srv_arch_dir) {
2131+ dir = os_file_opendir(srv_arch_dir, FALSE);
2132+ if (!dir) {
2133+ fprintf(stderr,
2134+ "InnoDB: Note: opening archived log directory "
2135+ "%s failed. "
2136+ "Purge archived logs are not available\n",
2137+ srv_arch_dir);
2138+ /* failed to open directory */
2139+ return(DB_ERROR);
2140+ }
2141+ } else {
2142+ /* log archive directory is not specified */
2143+ return(DB_ERROR);
2144+ }
2145+
2146+ dirnamelen = strlen(srv_arch_dir);
2147+
2148+ memcpy(archived_log_filename, srv_arch_dir, dirnamelen);
2149+ if (dirnamelen &&
2150+ archived_log_filename[dirnamelen - 1] != SRV_PATH_SEPARATOR) {
2151+ archived_log_filename[dirnamelen++] = SRV_PATH_SEPARATOR;
2152+ }
2153+
2154+ memset(&fileinfo, 0, sizeof(fileinfo));
2155+ while(!os_file_readdir_next_file(srv_arch_dir, dir,
2156+ &fileinfo) ) {
2157+ if (strncmp(fileinfo.name,
2158+ IB_ARCHIVED_LOGS_PREFIX, IB_ARCHIVED_LOGS_PREFIX_LEN)) {
2159+ continue;
2160+ }
2161+ if (dirnamelen + strlen(fileinfo.name) + 2 > OS_FILE_MAX_PATH)
2162+ continue;
2163+
2164+ snprintf(archived_log_filename + dirnamelen, OS_FILE_MAX_PATH,
2165+ "%s", fileinfo.name);
2166+
2167+ if (before_no) {
2168+ ib_uint64_t log_file_no = strtoull(fileinfo.name +
2169+ IB_ARCHIVED_LOGS_PREFIX_LEN,
2170+ NULL, 10);
2171+ if (log_file_no == 0 || before_no <= log_file_no) {
2172+ continue;
2173+ }
2174+ } else {
2175+ fileinfo.mtime = 0;
2176+ if (os_file_get_status(archived_log_filename,
2177+ &fileinfo, false) != DB_SUCCESS ||
2178+ fileinfo.mtime == 0) {
2179+ continue;
2180+ }
2181+
2182+ if (before_date == 0 || fileinfo.mtime > before_date) {
2183+ continue;
2184+ }
2185+ }
2186+
2187+ /* We are going to delete archived file. Acquire log_sys->mutex
2188+ to make sure that we are the only who try to delete file. This
2189+ also prevents log system from using this file. Do not delete
2190+ file if it is currently in progress of writting or have
2191+ pending IO. This is enforced by checking:
2192+ 1. fil_space_contains_node.
2193+ 2. group->archived_offset % group->file_size != 0, i.e.
2194+ there is archive in progress and we are going to delete it.
2195+ This covers 3 cases:
2196+ a. Usual case when we have one archive in progress,
2197+ both 1 and 2 are TRUE
2198+ b. When we have more then 1 archive in fil_space,
2199+ this can happen when flushed LSN range crosses file
2200+ boundary
2201+ c. When we have empty fil_space, but existing file will be
2202+ opened once archiving operation is requested. This usually
2203+ happens on startup.
2204+ */
2205+
2206+ mutex_enter(&log_sys->mutex);
2207+
2208+ log_archived_file_name_gen(namegen, sizeof(namegen),
2209+ group->id, group->archived_file_no);
2210+
2211+ if (fil_space_contains_node(group->archive_space_id,
2212+ archived_log_filename) ||
2213+ (group->archived_offset % group->file_size != 0 &&
2214+ strcmp(namegen, archived_log_filename) == 0)) {
2215+
2216+ mutex_exit(&log_sys->mutex);
2217+ continue;
2218+ }
2219+
2220+ if (os_file_delete_if_exists(archived_log_filename) == FALSE) {
2221+ fprintf(stderr,
2222+ "InnoDB: Note: can't delete archived log file "
2223+ "%s.\n", archived_log_filename);
2224+
2225+ mutex_exit(&log_sys->mutex);
2226+ os_file_closedir(dir);
2227+
2228+ return(DB_ERROR);
2229+ }
2230+
2231+ mutex_exit(&log_sys->mutex);
2232+ }
2233+
2234+ os_file_closedir(dir);
2235+
2236+ return(DB_SUCCESS);
2237+}
2238+
2239+
2240 /*******************************************************************//**
2241 Tells the InnoDB server that there has been activity in the database
2242 and wakes up the master thread if it is suspended (not sleeping). Used
2243@@ -2205,6 +2335,16 @@
2244 log_checkpoint(TRUE, FALSE);
2245 MONITOR_INC_TIME_IN_MICRO_SECS(MONITOR_SRV_CHECKPOINT_MICROSECOND,
2246 counter_time);
2247+
2248+ if (srv_shutdown_state > 0) {
2249+ return;
2250+ }
2251+
2252+ if (srv_log_arch_expire_sec) {
2253+ srv_main_thread_op_info = "purging archived logs";
2254+ purge_archived_logs(ut_time() - srv_log_arch_expire_sec,
2255+ 0);
2256+ }
2257 }
2258
2259 /*********************************************************************//**
2260
2261=== modified file 'Percona-Server/storage/innobase/srv/srv0start.cc'
2262--- Percona-Server/storage/innobase/srv/srv0start.cc 2013-03-05 12:46:43 +0000
2263+++ Percona-Server/storage/innobase/srv/srv0start.cc 2013-03-31 06:25:28 +0000
2264@@ -640,6 +640,11 @@
2265 }
2266 }
2267
2268+#ifdef UNIV_LOG_ARCHIVE
2269+ /* Create the file space object for archived logs. */
2270+ fil_space_create("arch_log_space", SRV_LOG_SPACE_FIRST_ID + 1,
2271+ 0, FIL_LOG);
2272+#endif
2273 log_group_init(0, srv_n_log_files,
2274 srv_log_file_size * UNIV_PAGE_SIZE,
2275 SRV_LOG_SPACE_FIRST_ID,
2276@@ -650,7 +655,12 @@
2277 /* Create a log checkpoint. */
2278 mutex_enter(&log_sys->mutex);
2279 ut_d(recv_no_log_write = FALSE);
2280- recv_reset_logs(lsn);
2281+ recv_reset_logs(
2282+#ifdef UNIV_LOG_ARCHIVE
2283+ UT_LIST_GET_FIRST(log_sys->log_groups)->archived_file_no,
2284+ TRUE,
2285+#endif
2286+ lsn);
2287 mutex_exit(&log_sys->mutex);
2288
2289 return(DB_SUCCESS);
2290@@ -738,9 +748,9 @@
2291 ibool* create_new_db, /*!< out: TRUE if new database should be
2292 created */
2293 #ifdef UNIV_LOG_ARCHIVE
2294- ulint* min_arch_log_no,/*!< out: min of archived log
2295+ lsn_t* min_arch_log_no,/*!< out: min of archived log
2296 numbers in data files */
2297- ulint* max_arch_log_no,/*!< out: max of archived log
2298+ lsn_t* max_arch_log_no,/*!< out: max of archived log
2299 numbers in data files */
2300 #endif /* UNIV_LOG_ARCHIVE */
2301 lsn_t* min_flushed_lsn,/*!< out: min of flushed lsn
2302@@ -963,9 +973,6 @@
2303 skip_size_check:
2304 fil_read_first_page(
2305 files[i], one_opened, &flags, &space,
2306-#ifdef UNIV_LOG_ARCHIVE
2307- min_arch_log_no, max_arch_log_no,
2308-#endif /* UNIV_LOG_ARCHIVE */
2309 min_flushed_lsn, max_flushed_lsn);
2310
2311 /* The first file of the system tablespace must
2312@@ -1443,8 +1450,8 @@
2313 lsn_t min_flushed_lsn;
2314 lsn_t max_flushed_lsn;
2315 #ifdef UNIV_LOG_ARCHIVE
2316- ulint min_arch_log_no;
2317- ulint max_arch_log_no;
2318+ lsn_t min_arch_log_no;
2319+ lsn_t max_arch_log_no;
2320 #endif /* UNIV_LOG_ARCHIVE */
2321 ulint sum_of_new_sizes;
2322 ulint sum_of_data_file_sizes;
2323@@ -1883,17 +1890,6 @@
2324 os_thread_create(io_handler_thread, n + i, thread_ids + i);
2325 }
2326
2327-#ifdef UNIV_LOG_ARCHIVE
2328- if (0 != ut_strcmp(srv_log_group_home_dir, srv_arch_dir)) {
2329- ut_print_timestamp(stderr);
2330- fprintf(stderr, " InnoDB: Error: you must set the log group home dir in my.cnf\n");
2331- ut_print_timestamp(stderr);
2332- fprintf(stderr, " InnoDB: the same as log arch dir.\n");
2333-
2334- return(DB_ERROR);
2335- }
2336-#endif /* UNIV_LOG_ARCHIVE */
2337-
2338 if (srv_n_log_files * srv_log_file_size * UNIV_PAGE_SIZE
2339 >= 512ULL * 1024ULL * 1024ULL * 1024ULL) {
2340 /* log_block_convert_lsn_to_no() limits the returned block
2341@@ -1982,7 +1978,6 @@
2342
2343 #ifdef UNIV_LOG_ARCHIVE
2344 srv_normalize_path_for_win(srv_arch_dir);
2345- srv_arch_dir = srv_add_path_separator_if_needed(srv_arch_dir);
2346 #endif /* UNIV_LOG_ARCHIVE */
2347
2348 dirnamelen = strlen(srv_log_group_home_dir);
2349@@ -2462,6 +2457,8 @@
2350 if (!srv_log_archive_on) {
2351 ut_a(DB_SUCCESS == log_archive_noarchivelog());
2352 } else {
2353+ bool start_archive;
2354+
2355 mutex_enter(&(log_sys->mutex));
2356
2357 start_archive = FALSE;

Subscribers

People subscribed via source and target branches