Merge lp:~ihanick/percona-server/5.1-innodb-log_archiving into lp:percona-server/5.1
- 5.1-innodb-log_archiving
- Merge into 5.1
Status: | Superseded |
---|---|
Proposed branch: | lp:~ihanick/percona-server/5.1-innodb-log_archiving |
Merge into: | lp:percona-server/5.1 |
Diff against target: |
1816 lines (+1290/-15) 26 files modified
Percona-Server/mysql-test/r/percona_server_variables_debug.result (+3/-0) Percona-Server/mysql-test/r/percona_server_variables_release.result (+3/-0) Percona-Server/mysql-test/suite/innodb_plugin/r/percona_log_archiving.result (+37/-0) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_log_archiving-master.opt (+1/-0) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_log_archiving.test (+128/-0) Percona-Server/sql/handler.cc (+54/-0) Percona-Server/sql/handler.h (+7/-0) Percona-Server/sql/lex.h (+1/-0) Percona-Server/sql/mysqld.cc (+2/-0) Percona-Server/sql/sql_lex.h (+2/-0) Percona-Server/sql/sql_parse.cc (+44/-0) Percona-Server/sql/sql_yacc.yy (+20/-0) Percona-Server/storage/innodb_plugin/fil/fil0fil.c (+222/-0) Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc (+99/-0) Percona-Server/storage/innodb_plugin/include/fil0fil.h (+14/-0) Percona-Server/storage/innodb_plugin/include/log0log.h (+18/-0) Percona-Server/storage/innodb_plugin/include/log0log.ic (+4/-0) Percona-Server/storage/innodb_plugin/include/os0file.h (+13/-0) Percona-Server/storage/innodb_plugin/include/srv0srv.h (+21/-0) Percona-Server/storage/innodb_plugin/include/srv0start.h (+22/-0) Percona-Server/storage/innodb_plugin/log/log0log.c (+83/-1) Percona-Server/storage/innodb_plugin/os/os0file.c (+92/-0) Percona-Server/storage/innodb_plugin/srv/srv0srv.c (+328/-0) Percona-Server/storage/innodb_plugin/srv/srv0start.c (+12/-14) doc/source/index.rst (+1/-0) doc/source/management/innodb_log_archiving.rst (+59/-0) |
To merge this branch: | bzr merge lp:~ihanick/percona-server/5.1-innodb-log_archiving |
Related bugs: | |
Related blueprints: |
Log Archiving for XtraDB
(High)
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Laurynas Biveinis | Pending | ||
Alexey Kopytov | Pending | ||
Review via email: mp+96733@code.launchpad.net |
This proposal supersedes a proposal from 2012-02-21.
This proposal has been superseded by a proposal from 2012-03-21.
Commit message
Description of the change
https:/
Based on original Yasufumi work: ~percona-
How it works:
1) Additional log archiving thread created inside innodb if log archiving is enabled
2) new transaction log file is created by log archiving script at the start or right after finishing archiving process
3) If active innodb transaction log file is full empty transaction log swapped with the earliest used transaction log file
4) If we have innodb_
5) if innodb_
6) test case now not using temporary file names and not affected by slow testing environment
7) fixed problems from last MP
8) cross-filesystem archived log renaming
Additionally the patch contains PURGE ARCHIVED LOGS {BEFORE|TO} command and a new handlerton call: purge_archive_logs
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal | # |
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal | # |
Nickolay,
I didn't review everything. Just thought it would be more efficient to do it iteratively.
Generally, I first suggest trying to make a debug build with your patch (I had a few issues see below), and then trying to run the percona_
So the comments I have so far:
- percona_
- compiler warning in srv0srv.c. I suggest replacing
if(expire_sec <= 0 || (ut_time() - fileinfo.mtime) <= expire_sec) {
with
if (ut_time() <= (ib_time_t)
- there is a compile-time assertion in debug builds, because the
patch introduces 2 new SQLCOM_ commands, but does not add the
corresponding SHOW STATUS variables to com_status_vars.
- there are unused 'protocol' variables in ha_purge_
and ha_purge_
- after fixing the above issues with debug builds, I got a runtime
assertion failure when trying to run the percona_log_archive
test. The reason is that PURGE ARCHIVED LOGS sends neither OK nor
error packet to the client. You should either call my_ok() and set
res to FALSE in mysql_execute_
res to TRUE, see purge_master_logs() for an example
- there are main.percona_
http://
- I think it's better to add the test to the innodb_plugin suite
rather than main one
- a more common way to pass test-specific options to mysqld is an
.opt file, not a .cnf file. there are lots of examples in existing
tests
- the test case does "drop table if exists t1;" but then creates and
uses table with the name 't'
- some unnecessary changes:
> @@ -645,5 +646,5 @@
> { "TRIM", SYM(TRIM)},
> { "VARIANCE", SYM(VARIANCE_SYM)},
> { "VAR_POP", SYM(VARIANCE_SYM)},
> - { "VAR_SAMP", SYM(VAR_SAMP_SYM)},
> + { "VAR_SAMP", SYM(VAR_SAMP_SYM)}
> };
>
and
> @@ -4560,6 +4560,8 @@
> return FALSE;
> }
>
> +
> +
> bool ha_show_status(THD *thd, handlerton *db_type, enum ha_stat_type stat)
> {
> List<Item> field_list;
>
and
> @@ -10709,9 +10710,12 @@ select_var_ident:
> ;
>
> purge_options:
> + ARCHIVED_SYM LOGS_SYM purge_archive_
> + |
> master_or_binary LOGS_SYM purge_option
> ;
>
> +
> purge_option:
> TO_SYM TEXT_STRING_sys
> {
>
- please remove commented code, e.g.:
> + //res = purge_master_
> + //fprintf(stderr, "purge archive logs command called %s\n", lex->to_log);
>
- SLEEPs in test cases almost never work. They definitely don't work
for synchronization, but unfortunately there's no way to
synchronize between mtr and InnoDB, especially when a background
InnoDB thread has to be executed synchronously. Which means we have
to modify tests to remove SLEEP()s possibly at the cost of test
covera...
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal | # |
Nickolay,
The patch looks generally good to me, but I have a bunch of relatively minor comments:
- the patch makes ARCHIVED a reserved keyword (so, for example,
"CREATE TABLE archived(...)" succeeds in upstream server, but fails
in PS). Please add ARCHIVED_SYM to the keyword_sp rule in sql_yacc.yy
- innobase_
something odd. So innobase_
time up to which logs must be purged as its 'before_date'
argument. Basically, all we need to do is to traverse all files in
the archived logs directory and delete those with mtime <
before_date, right? Here's what actually happens:
- innobase_
before_date and passes that as the expire_sec argument to
- expire_sec argument is unsigned. So if you specify a point in
time in the future, i.e. "PURGE ARCHIVED LOGS BEFORE NOW() +
INTERVAL 1 MONTH" an overflow occurs and unexpected things may
happen depending on sizeof(time_t) in the OS
- purge_archived_
adds expire_sec and compares it with _current_ time again. So the
exact cut-off time is drifting, which may become a problem when
file deletion takes a long time (which is quite possible)
- on a side note, purge_archive_
expired_sec is 0. Why?
So why not just pass before_date to purge_archived_
compare mtime with it directly?
- I think fil_swap_file() does too much unnecessary work on POSIX
systems. It flushes the file being swapped, then waits for flush to
complete, then closes the file, just because allegedly "we do not
trust the operating systems can rename an open file". Well, they
actually can. It is in the POSIX standard. Windows can do that too,
but that requires a special flag when opening the file. Since
Windows is not a priority for us, I suggest enclosing the
unnecessary for POSIX code in #ifdef __WIN__. We can fix Windows
later and remove that code in fil_swap_file() completely.
- fil_swap_file() could use some descriptive comments for the
function itself and its arguments. Also, fil_rotate_file() would
probably be a bit more descriptive name.
- innobase_
function body must be protected by log_sys-
otherwise multiple concurrent threads calling that function can
result in multiple log archive threads being created.
- in innobase_
innobase_
rather than innodb_* variables directly. The comment "where the
formal string goes" in the var_ptr comment looks to be copy-pasted
and not applicable to those 2 functions?
- please don't use the Yoda notation, as in "if (FALSE ==
log_
- please use OS_FILE_MAX_PATH instead of 10000 in
purge_
- there are 2 open_or_...
Nickolay Ihalainen (ihanick) wrote : Posted in a previous version of this proposal | # |
jenkins build for revision 435
http://
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal | # |
Nickolay,
If I'm not mistaken, the only change since my previous review comments is revision 435 with some changes to tests. It doesn't look like any of my comments have been addressed yet. Is that correct, or I am missing something? Just trying to understand if the patch is ready for review.
Nickolay Ihalainen (ihanick) wrote : Posted in a previous version of this proposal | # |
Alexey,
Sorry, just have found your comments at 2012-02-19.
I'm going to fix these problems today and tomorrow.
In lastest MP was fixed only 2012-02-13 problems.
Nickolay Ihalainen (ihanick) wrote : | # |
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal | # |
Nickolay,
I didn't have time to review everything, but the following code in os_file_
> + success = os_file_
> + ut_a(success);
> +
> + for (offset_high = 0; offset_
> + for (offset = 0; offset<
I see the following problems with it:
1. when size_high == 0 (i.e. the file size is < 4 GB), we will have 0 loop iterations, i.e. we won't copy anything.
2. when size_high = 2 and size_low = 1024*1024 (i.e. the file size is 8 GB + 1 MB), we will have 2 iterations, each copying 1 MB, so we'll copy 2 MB in total.
- 439. By Nickolay Ihalainen
-
Update branch to trunk
- 440. By Nickolay Ihalainen
-
Added missing comments
- 441. By Nickolay Ihalainen
-
code standards, comments, test for invalid purge archived logs to filename
- 442. By Nickolay Ihalainen
-
fix file copy for archived log located on separate disks
- 443. By Nickolay Ihalainen
-
pull changes from 5.1 trunk
- 444. By Nickolay Ihalainen
-
log archiving, copy log to separate disk: copy now uses 64bit offset, implemented throttling
- 445. By Nickolay Ihalainen
-
log archiving, copy log to separate disk, increase chunk to 64 innodb pages.
- 446. By Nickolay Ihalainen
-
fix file copy for archived log located on separate disks: prevent caching cleanup
- 447. By Nickolay Ihalainen
-
Add log archiving copy throttle variable to percona server variables test results
- 448. By Nickolay Ihalainen
-
Fixed deadlock if innodb_
xtra_log_ archive enabled dynamically. Correct posix_fadvise usage, indentation - 449. By Nickolay Ihalainen
-
merge with 5.1 trunk
Unmerged revisions
- 449. By Nickolay Ihalainen
-
merge with 5.1 trunk
- 448. By Nickolay Ihalainen
-
Fixed deadlock if innodb_
xtra_log_ archive enabled dynamically. Correct posix_fadvise usage, indentation - 447. By Nickolay Ihalainen
-
Add log archiving copy throttle variable to percona server variables test results
- 446. By Nickolay Ihalainen
-
fix file copy for archived log located on separate disks: prevent caching cleanup
- 445. By Nickolay Ihalainen
-
log archiving, copy log to separate disk, increase chunk to 64 innodb pages.
- 444. By Nickolay Ihalainen
-
log archiving, copy log to separate disk: copy now uses 64bit offset, implemented throttling
- 443. By Nickolay Ihalainen
-
pull changes from 5.1 trunk
- 442. By Nickolay Ihalainen
-
fix file copy for archived log located on separate disks
- 441. By Nickolay Ihalainen
-
code standards, comments, test for invalid purge archived logs to filename
- 440. By Nickolay Ihalainen
-
Added missing comments
Preview Diff
1 | === modified file 'Percona-Server/mysql-test/r/percona_server_variables_debug.result' |
2 | --- Percona-Server/mysql-test/r/percona_server_variables_debug.result 2012-01-30 04:30:33 +0000 |
3 | +++ Percona-Server/mysql-test/r/percona_server_variables_debug.result 2012-03-21 18:29:24 +0000 |
4 | @@ -161,6 +161,9 @@ |
5 | INNODB_USE_SYS_STATS_TABLE |
6 | INNODB_VERSION |
7 | INNODB_WRITE_IO_THREADS |
8 | +INNODB_XTRA_EXPIRE_ARCHIVE_LOGS_SEC |
9 | +INNODB_XTRA_LOG_ARCHIVE |
10 | +INNODB_XTRA_LOG_ARCHIVE_DIR |
11 | INSERT_ID |
12 | INTERACTIVE_TIMEOUT |
13 | JOIN_BUFFER_SIZE |
14 | |
15 | === modified file 'Percona-Server/mysql-test/r/percona_server_variables_release.result' |
16 | --- Percona-Server/mysql-test/r/percona_server_variables_release.result 2011-11-24 02:01:33 +0000 |
17 | +++ Percona-Server/mysql-test/r/percona_server_variables_release.result 2012-03-21 18:29:24 +0000 |
18 | @@ -159,6 +159,9 @@ |
19 | INNODB_USE_SYS_STATS_TABLE |
20 | INNODB_VERSION |
21 | INNODB_WRITE_IO_THREADS |
22 | +INNODB_XTRA_EXPIRE_ARCHIVE_LOGS_SEC |
23 | +INNODB_XTRA_LOG_ARCHIVE |
24 | +INNODB_XTRA_LOG_ARCHIVE_DIR |
25 | INSERT_ID |
26 | INTERACTIVE_TIMEOUT |
27 | JOIN_BUFFER_SIZE |
28 | |
29 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_log_archiving.result' |
30 | --- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_log_archiving.result 1970-01-01 00:00:00 +0000 |
31 | +++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_log_archiving.result 2012-03-21 18:29:24 +0000 |
32 | @@ -0,0 +1,37 @@ |
33 | +drop table if exists t; |
34 | +create table t (a int not null) ENGINE=InnoDB; |
35 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
36 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
37 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
38 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
39 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
40 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
41 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
42 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
43 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
44 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
45 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
46 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
47 | +SELECT sleep(1); |
48 | +sleep(1) |
49 | +0 |
50 | +PURGE ARCHIVED LOGS BEFORE ("tmpval" + INTERVAL 1 SECOND); |
51 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
52 | +PURGE ARCHIVED LOGS TO "last_log_file_next"; |
53 | +SET GLOBAL innodb_xtra_log_archive = OFF; |
54 | +SELECT @@innodb_xtra_log_archive; |
55 | +@@innodb_xtra_log_archive |
56 | +0 |
57 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
58 | +SET GLOBAL innodb_xtra_log_archive = ON; |
59 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
60 | +SELECT sleep(1); |
61 | +sleep(1) |
62 | +0 |
63 | +SET @saved_innodb_xtra_expire_archive_logs_sec = @@innodb_xtra_expire_archive_logs_sec; |
64 | +SET GLOBAL innodb_xtra_expire_archive_logs_sec = 1; |
65 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
66 | +SET GLOBAL innodb_xtra_expire_archive_logs_sec = @saved_innodb_xtra_expire_archive_logs_sec; |
67 | +create table archived(id int); |
68 | +drop table archived; |
69 | +DROP TABLE t; |
70 | |
71 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_log_archiving-master.opt' |
72 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_log_archiving-master.opt 1970-01-01 00:00:00 +0000 |
73 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_log_archiving-master.opt 2012-03-21 18:29:24 +0000 |
74 | @@ -0,0 +1,1 @@ |
75 | +--innodb_xtra_log_archive=1 --innodb_xtra_log_archive_dir=logs_archive --innodb_xtra_expire_archive_logs_sec=3600 --innodb_log_file_size=2M |
76 | |
77 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_log_archiving.test' |
78 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_log_archiving.test 1970-01-01 00:00:00 +0000 |
79 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_log_archiving.test 2012-03-21 18:29:24 +0000 |
80 | @@ -0,0 +1,128 @@ |
81 | +--source include/have_innodb.inc |
82 | +--disable_warnings |
83 | +drop table if exists t; |
84 | +--enable_warnings |
85 | + |
86 | +let $MYSQLD_DATADIR= `SELECT @@datadir`; |
87 | +let $MYSQLD_ARCHIVEDIR= `SELECT IF( locate('/', @@innodb_xtra_log_archive_dir) = 1, @@innodb_xtra_log_archive_dir, CONCAT(@@datadir, @@innodb_xtra_log_archive_dir))`; |
88 | +let MYSQLD_ARCHIVEDIR = $MYSQLD_ARCHIVEDIR; |
89 | +--mkdir $MYSQLD_ARCHIVEDIR |
90 | + |
91 | +create table t (a int not null) ENGINE=InnoDB; |
92 | + |
93 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
94 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
95 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
96 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
97 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
98 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
99 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
100 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
101 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
102 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
103 | +insert into t values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13); |
104 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
105 | + |
106 | +--list_files_append_file $MYSQLTEST_VARDIR/tmp/archived_transaction_logs.list $MYSQLD_ARCHIVEDIR ib_log_archive_* |
107 | +# Set the purge time 1 second after the last modify time of last ib_log_archive_* |
108 | +perl; |
109 | +open FSRC, "<".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/archived_transaction_logs.list' or die "Tmp file archived_transaction_logs.list not found"; |
110 | +open F, ">>".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/percona_archived_logs.tmp' or die "Tmp file percona_archived_logs.tmp not found"; |
111 | +my $line; |
112 | +while(<FSRC>) { |
113 | + chomp; |
114 | + if( index( $_,'ib_log_archive_') >= 0 ) { |
115 | + $line = $_; |
116 | + } |
117 | +} |
118 | + |
119 | +my $binlogpath = $ENV{'MYSQLD_ARCHIVEDIR'}.'/'.$line; |
120 | +my @array = stat($binlogpath); |
121 | +my $filemodifytime = $array[9]; |
122 | +my @t = localtime $filemodifytime; |
123 | +my $modifytime = sprintf "%04u-%02u-%02u %02u:%02u:%02u",$t[5]+1900,$t[4]+1,$t[3],$t[2],$t[1],$t[0]; |
124 | +printf F ("let \$last_log_file = %s;\n", $line); |
125 | +printf F ("let \$tmpval = %s;\n", $modifytime); |
126 | +close F; |
127 | +EOF |
128 | + |
129 | +--source $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp |
130 | +remove_file $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp; |
131 | +remove_file $MYSQLTEST_VARDIR/tmp/archived_transaction_logs.list; |
132 | + |
133 | +SELECT sleep(1); |
134 | + |
135 | +--replace_result $tmpval tmpval |
136 | +--eval PURGE ARCHIVED LOGS BEFORE ("$tmpval" + INTERVAL 1 SECOND) |
137 | +--list_files $MYSQLD_ARCHIVEDIR $last_log_file |
138 | + |
139 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
140 | + |
141 | +--list_files_append_file $MYSQLTEST_VARDIR/tmp/archived_transaction_logs.list $MYSQLD_ARCHIVEDIR ib_log_archive_* |
142 | +# get last archived transaction log file |
143 | +perl; |
144 | +open FSRC, "<".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/archived_transaction_logs.list' or die "Tmp file archived_transaction_logs.list not found"; |
145 | +open F, ">>".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/percona_archived_logs.tmp' or die "Tmp file percona_archived_logs.tmp not found"; |
146 | +my $line; |
147 | +my $next_lsn_file; |
148 | +while(<FSRC>) { |
149 | + chomp; |
150 | + if( m/ib_log_archive_(\d+)$/ ) { $line = $_;$next_lsn_file = sprintf "ib_log_archive_%016d", ($1+1); } |
151 | +} |
152 | + |
153 | +printf F ("let \$last_log_file = %s;\n", $line); |
154 | +printf F ("let \$last_log_file_next = %s;\n", $next_lsn_file); |
155 | +close F; |
156 | +EOF |
157 | + |
158 | +--source $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp |
159 | +remove_file $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp; |
160 | +remove_file $MYSQLTEST_VARDIR/tmp/archived_transaction_logs.list; |
161 | + |
162 | +--replace_result $last_log_file_next last_log_file_next |
163 | +--eval PURGE ARCHIVED LOGS TO "$last_log_file_next" |
164 | +--list_files $MYSQLD_ARCHIVEDIR $last_log_file |
165 | + |
166 | +# Check enable-disable |
167 | +SET GLOBAL innodb_xtra_log_archive = OFF; |
168 | +SELECT @@innodb_xtra_log_archive; |
169 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
170 | +--list_files $MYSQLD_ARCHIVEDIR ib_log_archive_* |
171 | +SET GLOBAL innodb_xtra_log_archive = ON; |
172 | + |
173 | + |
174 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
175 | + |
176 | +--list_files_append_file $MYSQLTEST_VARDIR/tmp/archived_transaction_logs.list $MYSQLD_ARCHIVEDIR ib_log_archive_* |
177 | +perl; |
178 | +open FSRC, "<".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/archived_transaction_logs.list' or die "Tmp file archived_transaction_logs.list not found"; |
179 | +open F, ">>".$ENV{'MYSQLTEST_VARDIR'}.'/tmp/percona_archived_logs.tmp' or die "Tmp file percona_archived_logs.tmp not found"; |
180 | +my $line; |
181 | +while(<FSRC>) { |
182 | + chomp; |
183 | + if( index( $_,'ib_log_archive_') >= 0 ) { $line = $_; } |
184 | +} |
185 | + |
186 | +printf F ("let \$last_log_file = %s;", $line); |
187 | +close F; |
188 | +EOF |
189 | + |
190 | +--source $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp |
191 | +remove_file $MYSQLTEST_VARDIR/tmp/percona_archived_logs.tmp; |
192 | +remove_file $MYSQLTEST_VARDIR/tmp/archived_transaction_logs.list; |
193 | + |
194 | +SELECT sleep(1); |
195 | + |
196 | +SET @saved_innodb_xtra_expire_archive_logs_sec = @@innodb_xtra_expire_archive_logs_sec; |
197 | +SET GLOBAL innodb_xtra_expire_archive_logs_sec = 1; |
198 | +insert into t (a) select t1.a from t t1, t t2, t t3 LIMIT 100000; |
199 | +--list_files $MYSQLD_ARCHIVEDIR $last_log_file |
200 | +SET GLOBAL innodb_xtra_expire_archive_logs_sec = @saved_innodb_xtra_expire_archive_logs_sec; |
201 | + |
202 | +create table archived(id int); |
203 | +drop table archived; |
204 | + |
205 | +DROP TABLE t; |
206 | + |
207 | +--remove_files_wildcard $MYSQLD_ARCHIVEDIR ib_log_archive_* |
208 | +--rmdir $MYSQLD_ARCHIVEDIR |
209 | |
210 | === modified file 'Percona-Server/sql/handler.cc' |
211 | --- Percona-Server/sql/handler.cc 2011-11-24 16:33:30 +0000 |
212 | +++ Percona-Server/sql/handler.cc 2012-03-21 18:29:24 +0000 |
213 | @@ -4597,6 +4597,60 @@ |
214 | return result; |
215 | } |
216 | |
217 | +static my_bool purge_archive_logs_handlerton(THD *thd, plugin_ref plugin, |
218 | + void *arg) |
219 | +{ |
220 | + ulong before_timestamp = *(ulong*) arg; |
221 | + const char* to_filename = NULL; |
222 | + handlerton *hton= plugin_data(plugin, handlerton *); |
223 | + |
224 | + if (NULL == hton->purge_archive_logs) { |
225 | + return TRUE; |
226 | + } |
227 | + |
228 | + if (hton->purge_archive_logs(hton, before_timestamp, to_filename)) |
229 | + return TRUE; |
230 | + return FALSE; |
231 | +} |
232 | + |
233 | +bool ha_purge_archive_logs(THD *thd, handlerton *db_type, void* args) |
234 | +{ |
235 | + if (db_type == NULL) |
236 | + { |
237 | + return plugin_foreach(thd, purge_archive_logs_handlerton, |
238 | + MYSQL_STORAGE_ENGINE_PLUGIN, args); |
239 | + } |
240 | + return false; |
241 | +} |
242 | + |
243 | +static my_bool purge_archive_logs_to_handlerton(THD *thd, plugin_ref plugin, |
244 | + void *arg) |
245 | +{ |
246 | + const char* to_filename = (const char*) arg; |
247 | + handlerton *hton= plugin_data(plugin, handlerton *); |
248 | + |
249 | + |
250 | + if (NULL == hton->purge_archive_logs) { |
251 | + return TRUE; |
252 | + } |
253 | + |
254 | + if (hton->purge_archive_logs(hton, 0, to_filename)) |
255 | + return TRUE; |
256 | + return FALSE; |
257 | +} |
258 | + |
259 | +bool ha_purge_archive_logs_to(THD *thd, handlerton *db_type, void* args) |
260 | +{ |
261 | + bool result; |
262 | + result = false; |
263 | + if (db_type == NULL) |
264 | + { |
265 | + result = plugin_foreach(thd, purge_archive_logs_to_handlerton, |
266 | + MYSQL_STORAGE_ENGINE_PLUGIN, args); |
267 | + } |
268 | + return result; |
269 | +} |
270 | + |
271 | /* |
272 | Function to check if the conditions for row-based binlogging is |
273 | correct for the table. |
274 | |
275 | === modified file 'Percona-Server/sql/handler.h' |
276 | --- Percona-Server/sql/handler.h 2012-01-17 14:05:16 +0000 |
277 | +++ Percona-Server/sql/handler.h 2012-03-21 18:29:24 +0000 |
278 | @@ -693,6 +693,8 @@ |
279 | int (*fill_files_table)(handlerton *hton, THD *thd, |
280 | TABLE_LIST *tables, |
281 | class Item *cond); |
282 | + int (*purge_archive_logs)(handlerton *hton, time_t before_date, const char* to_filename); |
283 | + |
284 | uint32 flags; /* global handler flags */ |
285 | /* |
286 | Those handlerton functions below are properly initialized at handler |
287 | @@ -2074,6 +2076,11 @@ |
288 | int ha_prepare(THD *thd); |
289 | int ha_recover(HASH *commit_list); |
290 | |
291 | +/* remove old archived transaction logs files */ |
292 | +bool ha_purge_archive_logs(THD *thd, handlerton *db_type, void* args); |
293 | +bool ha_purge_archive_logs_to(THD *thd, handlerton *db_type, void* args); |
294 | + |
295 | + |
296 | /* transactions: these functions never call handlerton functions directly */ |
297 | int ha_commit_trans(THD *thd, bool all); |
298 | int ha_autocommit_or_rollback(THD *thd, int error); |
299 | |
300 | === modified file 'Percona-Server/sql/lex.h' |
301 | --- Percona-Server/sql/lex.h 2011-11-24 02:01:23 +0000 |
302 | +++ Percona-Server/sql/lex.h 2012-03-21 18:29:24 +0000 |
303 | @@ -68,6 +68,7 @@ |
304 | { "ANALYZE", SYM(ANALYZE_SYM)}, |
305 | { "AND", SYM(AND_SYM)}, |
306 | { "ANY", SYM(ANY_SYM)}, |
307 | + { "ARCHIVED", SYM(ARCHIVED_SYM)}, |
308 | { "AS", SYM(AS)}, |
309 | { "ASC", SYM(ASC)}, |
310 | { "ASCII", SYM(ASCII_SYM)}, |
311 | |
312 | === modified file 'Percona-Server/sql/mysqld.cc' |
313 | --- Percona-Server/sql/mysqld.cc 2012-03-16 09:07:44 +0000 |
314 | +++ Percona-Server/sql/mysqld.cc 2012-03-21 18:29:24 +0000 |
315 | @@ -3070,6 +3070,8 @@ |
316 | {"prepare_sql", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PREPARE]), SHOW_LONG_STATUS}, |
317 | {"purge", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PURGE]), SHOW_LONG_STATUS}, |
318 | {"purge_before_date", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PURGE_BEFORE]), SHOW_LONG_STATUS}, |
319 | + {"purge_archived", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PURGE_ARCHIVE]), SHOW_LONG_STATUS}, |
320 | + {"purge_archived_before_date", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_PURGE_ARCHIVE_BEFORE]), SHOW_LONG_STATUS}, |
321 | {"release_savepoint", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_RELEASE_SAVEPOINT]), SHOW_LONG_STATUS}, |
322 | {"rename_table", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_RENAME_TABLE]), SHOW_LONG_STATUS}, |
323 | {"rename_user", (char*) offsetof(STATUS_VAR, com_stat[(uint) SQLCOM_RENAME_USER]), SHOW_LONG_STATUS}, |
324 | |
325 | === modified file 'Percona-Server/sql/sql_lex.h' |
326 | --- Percona-Server/sql/sql_lex.h 2011-11-24 02:01:33 +0000 |
327 | +++ Percona-Server/sql/sql_lex.h 2012-03-21 18:29:24 +0000 |
328 | @@ -130,6 +130,8 @@ |
329 | // TODO(mcallaghan): update status_vars in mysqld to export these |
330 | SQLCOM_SHOW_USER_STATS, SQLCOM_SHOW_TABLE_STATS, SQLCOM_SHOW_INDEX_STATS, |
331 | SQLCOM_SHOW_CLIENT_STATS, SQLCOM_SHOW_THREAD_STATS, |
332 | + SQLCOM_PURGE_ARCHIVE, |
333 | + SQLCOM_PURGE_ARCHIVE_BEFORE, |
334 | /* This should be the last !!! */ |
335 | SQLCOM_END |
336 | }; |
337 | |
338 | === modified file 'Percona-Server/sql/sql_parse.cc' |
339 | --- Percona-Server/sql/sql_parse.cc 2011-11-24 16:33:30 +0000 |
340 | +++ Percona-Server/sql/sql_parse.cc 2012-03-21 18:29:24 +0000 |
341 | @@ -2508,6 +2508,50 @@ |
342 | res = purge_master_logs_before_date(thd, (ulong)it->val_int()); |
343 | break; |
344 | } |
345 | + case SQLCOM_PURGE_ARCHIVE: |
346 | + { |
347 | + if (check_global_access(thd, SUPER_ACL)) |
348 | + goto error; |
349 | + /* PURGE ARCHIVED LOGS TO 'file' */ |
350 | + if (ha_purge_archive_logs_to(NULL, NULL, lex->to_log)) { |
351 | + my_ok(thd); |
352 | + } else { |
353 | + my_error(ER_LOG_PURGE_UNKNOWN_ERR, MYF(0), "PURGE ARCHIVE LOGS TO"); |
354 | + goto error; |
355 | + } |
356 | + |
357 | + break; |
358 | + } |
359 | + case SQLCOM_PURGE_ARCHIVE_BEFORE: |
360 | + { |
361 | + Item *it; |
362 | + |
363 | + if (check_global_access(thd, SUPER_ACL)) |
364 | + goto error; |
365 | + /* PURGE ARCHIVE LOGS BEFORE 'data' */ |
366 | + it= (Item *)lex->value_list.head(); |
367 | + if ((!it->fixed && it->fix_fields(lex->thd, &it)) || |
368 | + it->check_cols(1)) |
369 | + { |
370 | + my_error(ER_WRONG_ARGUMENTS, MYF(0), "PURGE ARCHIVE LOGS BEFORE"); |
371 | + goto error; |
372 | + } |
373 | + it= new Item_func_unix_timestamp(it); |
374 | + /* |
375 | + it is OK only emulate fix_fieds, because we need only |
376 | + value of constant |
377 | + */ |
378 | + it->quick_fix_field(); |
379 | + //res = purge_master_logs_before_date(thd, (ulong)it->val_int()); |
380 | + ulong before_timestamp = (ulong)it->val_int(); |
381 | + if (ha_purge_archive_logs(NULL, NULL, &before_timestamp )) { |
382 | + my_ok(thd); |
383 | + } else { |
384 | + my_error(ER_LOG_PURGE_UNKNOWN_ERR, MYF(0), "PURGE ARCHIVE LOGS BEFORE"); |
385 | + goto error; |
386 | + } |
387 | + break; |
388 | + } |
389 | #endif |
390 | case SQLCOM_SHOW_WARNS: |
391 | { |
392 | |
393 | === modified file 'Percona-Server/sql/sql_yacc.yy' |
394 | --- Percona-Server/sql/sql_yacc.yy 2011-11-24 16:33:30 +0000 |
395 | +++ Percona-Server/sql/sql_yacc.yy 2012-03-21 18:29:24 +0000 |
396 | @@ -714,6 +714,7 @@ |
397 | %token AND_AND_SYM /* OPERATOR */ |
398 | %token AND_SYM /* SQL-2003-R */ |
399 | %token ANY_SYM /* SQL-2003-R */ |
400 | +%token ARCHIVED_SYM /* MYSQL */ |
401 | %token AS /* SQL-2003-R */ |
402 | %token ASC /* SQL-2003-N */ |
403 | %token ASCII_SYM /* MYSQL-FUNC */ |
404 | @@ -10709,6 +10710,8 @@ |
405 | ; |
406 | |
407 | purge_options: |
408 | + ARCHIVED_SYM LOGS_SYM purge_archive_option |
409 | + | |
410 | master_or_binary LOGS_SYM purge_option |
411 | ; |
412 | |
413 | @@ -10726,6 +10729,22 @@ |
414 | } |
415 | ; |
416 | |
417 | +purge_archive_option: |
418 | + TO_SYM TEXT_STRING_sys |
419 | + { |
420 | + Lex->to_log = $2.str; |
421 | + Lex->sql_command= SQLCOM_PURGE_ARCHIVE; |
422 | + } |
423 | + | BEFORE_SYM expr |
424 | + { |
425 | + LEX *lex= Lex; |
426 | + lex->value_list.empty(); |
427 | + lex->value_list.push_front($2); |
428 | + lex->sql_command= SQLCOM_PURGE_ARCHIVE_BEFORE; |
429 | + } |
430 | + ; |
431 | + |
432 | + |
433 | /* kill threads */ |
434 | |
435 | kill: |
436 | @@ -11770,6 +11789,7 @@ |
437 | | AGGREGATE_SYM {} |
438 | | ALGORITHM_SYM {} |
439 | | ANY_SYM {} |
440 | + | ARCHIVED_SYM {} |
441 | | AT_SYM {} |
442 | | AUTHORS_SYM {} |
443 | | AUTO_INC {} |
444 | |
445 | === modified file 'Percona-Server/storage/innodb_plugin/fil/fil0fil.c' |
446 | --- Percona-Server/storage/innodb_plugin/fil/fil0fil.c 2011-12-09 16:53:35 +0000 |
447 | +++ Percona-Server/storage/innodb_plugin/fil/fil0fil.c 2012-03-21 18:29:24 +0000 |
448 | @@ -2639,6 +2639,228 @@ |
449 | return(success); |
450 | } |
451 | |
452 | +/********************************************************************//** |
453 | +Rotates tablespace like logrotate does: renames tablespace file, |
454 | +new tablespace file is created with the same name and tablespace header |
455 | +as original tablespace |
456 | +@return TRUE if success */ |
457 | +UNIV_INTERN |
458 | +ibool |
459 | +fil_rotate_file( |
460 | +/*==========*/ |
461 | + ulint id, /*!< in: space id */ |
462 | + ulint block_offset_in, /*!< in: header offset in a tablespace */ |
463 | + const char* old_file_rename_to, /*!< in: src tablespace file name */ |
464 | + const char* new_file_rename_from, /*!< in: dst tablespace file name */ |
465 | + ulint hdr_size) /*!< in: tablespace header size */ |
466 | + |
467 | +{ |
468 | + ibool success; |
469 | + fil_space_t* space; |
470 | + fil_node_t* node; |
471 | + ulint block_offset; |
472 | + os_file_t file; |
473 | + ulint size; |
474 | + ulint size_high; |
475 | + ulint count = 0; |
476 | + |
477 | + byte* buf_ptr = NULL; |
478 | + byte* buf = NULL; |
479 | + |
480 | + ut_a(id != 0); |
481 | + |
482 | +retry: |
483 | + block_offset = block_offset_in; |
484 | + count++; |
485 | + |
486 | + if (count > 1000) { |
487 | + ut_print_timestamp(stderr); |
488 | + fputs(" InnoDB: Warning: problems renaming ", stderr); |
489 | + ut_print_filename(stderr, old_file_rename_to); |
490 | + fputs(" or ", stderr); |
491 | + ut_print_filename(stderr, new_file_rename_from); |
492 | + fprintf(stderr, ", %lu iterations\n", (ulong) count); |
493 | + } |
494 | + |
495 | + mutex_enter(&fil_system->mutex); |
496 | + |
497 | + space = fil_space_get_by_id(id); |
498 | + |
499 | + if (space == NULL) { |
500 | + fprintf(stderr, |
501 | + "InnoDB: Error: cannot find space id %lu" |
502 | + " in the tablespace memory cache\n", |
503 | + (ulong) id); |
504 | + mutex_exit(&fil_system->mutex); |
505 | + |
506 | + return(FALSE); |
507 | + } |
508 | + |
509 | + if (count > 25000) { |
510 | +error_exit: |
511 | + space->stop_ios = FALSE; |
512 | + mutex_exit(&fil_system->mutex); |
513 | + |
514 | + if (buf_ptr) { |
515 | + ut_free(buf_ptr); |
516 | + } |
517 | + |
518 | + return(FALSE); |
519 | + } |
520 | + |
521 | + /* We temporarily close the .ibd file because we do not trust that |
522 | + operating systems can rename an open file. For the closing we have to |
523 | + wait until there are no pending i/o's or flushes on the file. */ |
524 | + |
525 | + space->stop_ios = TRUE; |
526 | + |
527 | + node = UT_LIST_GET_FIRST(space->chain); |
528 | + |
529 | + for (;;) { |
530 | + if (UNIV_UNLIKELY(node == NULL)) { |
531 | + fprintf(stderr, |
532 | + "InnoDB: Error: The specified block_offset %lu" |
533 | + " in space %lu,\n" |
534 | + "InnoDB: space name %s, which is outside the tablespace bounds.\n", |
535 | + (ulong) block_offset, (ulong) id, |
536 | + space->name); |
537 | + |
538 | + goto error_exit; |
539 | + } |
540 | + |
541 | + ut_a(node->size != 0); |
542 | + |
543 | + if (node->size > block_offset) { |
544 | + /* Found! */ |
545 | + break; |
546 | + } else { |
547 | + block_offset -= node->size; |
548 | + node = UT_LIST_GET_NEXT(chain, node); |
549 | + } |
550 | + } |
551 | + |
552 | + if (node->n_pending > 0 || node->n_pending_flushes > 0) { |
553 | + /* There are pending i/o's or flushes, sleep for a while and |
554 | + retry */ |
555 | + |
556 | + mutex_exit(&fil_system->mutex); |
557 | + |
558 | + os_thread_sleep(20000); |
559 | + |
560 | + goto retry; |
561 | + |
562 | + } else if (node->modification_counter > node->flush_counter) { |
563 | + /* Flush the space */ |
564 | + |
565 | + mutex_exit(&fil_system->mutex); |
566 | + |
567 | + os_thread_sleep(20000); |
568 | + |
569 | + fil_flush(id, TRUE); |
570 | + |
571 | + goto retry; |
572 | + |
573 | + } else if (node->open) { |
574 | + /* Close the file */ |
575 | + |
576 | + fil_node_close_file(node, fil_system); |
577 | + } |
578 | + |
579 | + /* store old header */ |
580 | + if (hdr_size > 0) { |
581 | + buf_ptr = ut_malloc(2 * hdr_size); |
582 | + buf = ut_align(buf_ptr, hdr_size); |
583 | + |
584 | + file = os_file_create(node->name, OS_FILE_OPEN, OS_FILE_NORMAL, |
585 | + OS_LOG_FILE, &success); |
586 | + ut_a(success); |
587 | + |
588 | + success = os_file_read(file, buf, 0, 0, hdr_size); |
589 | + ut_a(success); |
590 | + |
591 | + os_file_close(file); |
592 | + } |
593 | + |
594 | + /* check size of the new file */ |
595 | + file = os_file_create(new_file_rename_from, OS_FILE_OPEN, OS_FILE_NORMAL, |
596 | + OS_LOG_FILE, &success); |
597 | + if (!success) { |
598 | + fprintf(stderr, |
599 | + "InnoDB: Error in opening %s\n", new_file_rename_from); |
600 | + goto error_exit; |
601 | + } |
602 | + |
603 | + success = os_file_get_size(file, &size, &size_high); |
604 | + ut_a(success); |
605 | + |
606 | + if (size != (0xFFFFFFFFUL & (node->size << UNIV_PAGE_SIZE_SHIFT)) |
607 | + || size_high != (node->size >> (32 - UNIV_PAGE_SIZE_SHIFT))) { |
608 | + fprintf(stderr, |
609 | + "InnoDB: Error: file %s is" |
610 | + " of different size %lu %lu bytes\n" |
611 | + "InnoDB: than intended %lu %lu bytes!\n", |
612 | + new_file_rename_from, (ulong) size_high, (ulong) size, |
613 | + (ulong) (node->size >> (32 - UNIV_PAGE_SIZE_SHIFT)), |
614 | + (ulong) (0xFFFFFFFFUL & (node->size << UNIV_PAGE_SIZE_SHIFT))); |
615 | + os_file_close(file); |
616 | + goto error_exit; |
617 | + } |
618 | + |
619 | + /* copy stored header */ |
620 | + if (hdr_size > 0) { |
621 | + success = os_file_write(new_file_rename_from, file, buf, 0, 0, hdr_size); |
622 | + if (!success) { |
623 | + fprintf(stderr, |
624 | + "InnoDB: Error: failed to initialize" |
625 | + " header of %s\n", |
626 | + new_file_rename_from); |
627 | + os_file_close(file); |
628 | + goto error_exit; |
629 | + } |
630 | + success = os_file_flush(file, TRUE); |
631 | + if (!success) { |
632 | + fprintf(stderr, |
633 | + "InnoDB: Error: failed to initialize" |
634 | + " header of %s\n", |
635 | + new_file_rename_from); |
636 | + os_file_close(file); |
637 | + goto error_exit; |
638 | + } |
639 | + } |
640 | + |
641 | + success = os_file_close(file); |
642 | + ut_a(success); |
643 | + |
644 | + /* Rename the tablespace and the node in the memory cache */ |
645 | + success = os_file_rename(node->name, old_file_rename_to); |
646 | + |
647 | + if (success) { |
648 | + success = os_file_rename(new_file_rename_from, node->name); |
649 | + |
650 | + if (!success) { |
651 | + ut_a(os_file_rename(old_file_rename_to, node->name)); |
652 | + } |
653 | + } |
654 | + |
655 | + if (success) { |
656 | + fprintf(stderr, |
657 | + "InnoDB: file %s was renamed to %s\n" |
658 | + "InnoDB: file %s was renamed to %s\n", |
659 | + node->name, old_file_rename_to, |
660 | + new_file_rename_from, node->name); |
661 | + } |
662 | + |
663 | + space->stop_ios = FALSE; |
664 | + |
665 | + mutex_exit(&fil_system->mutex); |
666 | + |
667 | + if (buf_ptr) { |
668 | + ut_free(buf_ptr); |
669 | + } |
670 | + |
671 | + return(success); |
672 | +} |
673 | + |
674 | /*******************************************************************//** |
675 | Creates a new single-table tablespace to a database directory of MySQL. |
676 | Database directories are under the 'datadir' of MySQL. The datadir is the |
677 | |
678 | === modified file 'Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc' |
679 | --- Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2012-02-07 03:25:46 +0000 |
680 | +++ Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2012-03-21 18:29:24 +0000 |
681 | @@ -186,6 +186,9 @@ |
682 | static my_bool innobase_log_archive = FALSE; |
683 | static char* innobase_log_arch_dir = NULL; |
684 | #endif /* UNIV_LOG_ARCHIVE */ |
685 | +static my_bool innobase_xtra_log_archive = FALSE; |
686 | +static char* innobase_xtra_log_archive_dir = NULL; |
687 | +static ulong innodb_xtra_expire_archive_logs_sec = 0; |
688 | static my_bool innobase_use_doublewrite = TRUE; |
689 | static my_bool innobase_use_checksums = TRUE; |
690 | static my_bool innobase_fast_checksum = FALSE; |
691 | @@ -316,6 +319,27 @@ |
692 | static const char innobase_hton_name[]= "InnoDB"; |
693 | |
694 | /*************************************************************//** |
695 | +Removes old archived transaction log files. |
696 | +@return non-zero if error */ |
697 | +static int innobase_purge_archive_logs( |
698 | + handlerton *hton, /*!< in: InnoDB handlerton */ |
699 | + time_t before_date, /*!< in: all files modified |
700 | + before timestamp should be removed */ |
701 | + const char* to_filename) /*!< in: this and earler files |
702 | + should be removed */ |
703 | +{ |
704 | + if (before_date > 0) { |
705 | + purge_archived_logs(before_date, 0); |
706 | + } else if (to_filename) { |
707 | + if (to_filename == strstr(to_filename, IB_ARCHIVED_LOGS_PREFIX)) { |
708 | + unsigned long long log_file_lsn = strtoll(to_filename + strlen(IB_ARCHIVED_LOGS_PREFIX), NULL, 10); |
709 | + purge_archived_logs(0, log_file_lsn); |
710 | + } |
711 | + } |
712 | + return 0; |
713 | +} |
714 | + |
715 | +/*************************************************************//** |
716 | Check for a valid value of innobase_commit_concurrency. |
717 | @return 0 for valid innodb_commit_concurrency */ |
718 | static |
719 | @@ -2120,6 +2144,7 @@ |
720 | innobase_hton->flags=HTON_NO_FLAGS; |
721 | innobase_hton->release_temporary_latches=innobase_release_temporary_latches; |
722 | innobase_hton->alter_table_flags = innobase_alter_table_flags; |
723 | + innobase_hton->purge_archive_logs = innobase_purge_archive_logs; |
724 | |
725 | ut_a(DATA_MYSQL_TRUE_VARCHAR == (ulint)MYSQL_TYPE_VARCHAR); |
726 | |
727 | @@ -2489,6 +2514,14 @@ |
728 | #ifdef UNIV_LOG_ARCHIVE |
729 | srv_log_archive_on = (ulint) innobase_log_archive; |
730 | #endif /* UNIV_LOG_ARCHIVE */ |
731 | + log_xtra_log_archive_on = (ulint) innobase_xtra_log_archive; |
732 | + log_xtra_log_expire_sec = (ulint) innodb_xtra_expire_archive_logs_sec; |
733 | + |
734 | + if (innobase_xtra_log_archive_dir && strlen(innobase_xtra_log_archive_dir) > 0) { |
735 | + log_xtra_log_archive_dir = innobase_xtra_log_archive_dir; |
736 | + } else { |
737 | + log_xtra_log_archive_dir = innobase_log_group_home_dir; |
738 | + } |
739 | srv_log_buffer_size = (ulint) innobase_log_buffer_size; |
740 | |
741 | srv_buf_pool_size = (ulint) innobase_buffer_pool_size; |
742 | @@ -11142,6 +11175,56 @@ |
743 | } |
744 | |
745 | /****************************************************************//** |
746 | +Update the system variable innobase_log_archive using the "saved" |
747 | +value. This function is registered as a callback with MySQL. */ |
748 | +static |
749 | +void |
750 | +innobase_log_archive_update( |
751 | +/*==============================*/ |
752 | + THD* thd, /*!< in: thread handle */ |
753 | + struct st_mysql_sys_var* var, /*!< in: pointer to |
754 | + system variable */ |
755 | + void* var_ptr, /*!< out: unused */ |
756 | + const void* save) /*!< in: immediate result |
757 | + from check function */ |
758 | +{ |
759 | + mutex_enter(&(log_sys->archived_mutex)); |
760 | + if (*(my_bool*) save) { |
761 | + if (FALSE == log_xtra_log_archive_on) { |
762 | + innobase_xtra_log_archive=TRUE; |
763 | + log_xtra_log_archive_on=TRUE; |
764 | + os_thread_create(&srv_log_archive_thread, NULL, NULL); |
765 | + } |
766 | + } else { |
767 | + if (TRUE == log_xtra_log_archive_on) { |
768 | + innobase_xtra_log_archive=FALSE; |
769 | + log_xtra_log_archive_on=FALSE; |
770 | + os_event_set(log_sys->arch_event_rotate_start); |
771 | + } |
772 | + } |
773 | + mutex_exit(&(log_sys->archived_mutex)); |
774 | +} |
775 | + |
776 | +/****************************************************************//** |
777 | +Update the system variable innodb_xtra_expire_archive_logs_sec using |
778 | +the "saved" value. This function is registered as a callback with MySQL. */ |
779 | +static |
780 | +void |
781 | +innobase_log_archive_expire_update( |
782 | +/*==============================*/ |
783 | + THD* thd, /*!< in: thread handle */ |
784 | + struct st_mysql_sys_var* var, /*!< in: pointer to |
785 | + system variable */ |
786 | + void* var_ptr, /*!< out: unused */ |
787 | + const void* save) /*!< in: immediate result |
788 | + from check function */ |
789 | +{ |
790 | + mutex_enter(&(log_sys->archived_mutex)); |
791 | + log_xtra_log_expire_sec = innodb_xtra_expire_archive_logs_sec = *(ulint*) save; |
792 | + mutex_exit(&(log_sys->archived_mutex)); |
793 | +} |
794 | + |
795 | +/****************************************************************//** |
796 | Update the system variable innodb_old_blocks_pct using the "saved" |
797 | value. This function is registered as a callback with MySQL. */ |
798 | static |
799 | @@ -11502,6 +11585,19 @@ |
800 | "Set to 1 if you want to have logs archived.", NULL, NULL, FALSE); |
801 | #endif /* UNIV_LOG_ARCHIVE */ |
802 | |
803 | +static MYSQL_SYSVAR_BOOL(xtra_log_archive, innobase_xtra_log_archive, |
804 | + PLUGIN_VAR_OPCMDARG, |
805 | + "Set to 1 if you want to have logs archived.", NULL, innobase_log_archive_update, FALSE); |
806 | + |
807 | +static MYSQL_SYSVAR_STR(xtra_log_archive_dir, innobase_xtra_log_archive_dir, |
808 | + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, |
809 | + "Where full logs should be archived.", NULL, NULL, NULL); |
810 | + |
811 | +static MYSQL_SYSVAR_ULONG(xtra_expire_archive_logs_sec, innodb_xtra_expire_archive_logs_sec, |
812 | + PLUGIN_VAR_OPCMDARG, |
813 | + "Expire time for archived innodb transaction logs.", |
814 | + NULL, innobase_log_archive_expire_update, 0, 0, ~0L, 0); |
815 | + |
816 | static MYSQL_SYSVAR_STR(log_group_home_dir, innobase_log_group_home_dir, |
817 | PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, |
818 | "Path to InnoDB log files.", NULL, NULL, NULL); |
819 | @@ -11936,6 +12032,9 @@ |
820 | MYSQL_SYSVAR(log_arch_dir), |
821 | MYSQL_SYSVAR(log_archive), |
822 | #endif /* UNIV_LOG_ARCHIVE */ |
823 | + MYSQL_SYSVAR(xtra_log_archive), |
824 | + MYSQL_SYSVAR(xtra_log_archive_dir), |
825 | + MYSQL_SYSVAR(xtra_expire_archive_logs_sec), |
826 | MYSQL_SYSVAR(log_buffer_size), |
827 | MYSQL_SYSVAR(log_file_size), |
828 | MYSQL_SYSVAR(log_files_in_group), |
829 | |
830 | === modified file 'Percona-Server/storage/innodb_plugin/include/fil0fil.h' |
831 | --- Percona-Server/storage/innodb_plugin/include/fil0fil.h 2011-12-09 16:53:35 +0000 |
832 | +++ Percona-Server/storage/innodb_plugin/include/fil0fil.h 2012-03-21 18:29:24 +0000 |
833 | @@ -426,6 +426,20 @@ |
834 | const char* new_name); /*!< in: new table name in the standard |
835 | databasename/tablename format |
836 | of InnoDB */ |
837 | +/********************************************************************//** |
838 | +Rotates tablespace like logrotate does: renames tablespace file, |
839 | +new tablespace file is created with the same name and tablespace header |
840 | +as original tablespace |
841 | +@return TRUE if success */ |
842 | +UNIV_INTERN |
843 | +ibool |
844 | +fil_rotate_file( |
845 | +/*==========*/ |
846 | + ulint id, |
847 | + ulint block_offset_in, |
848 | + const char* old_file_rename_to, |
849 | + const char* new_file_rename_from, |
850 | + ulint hdr_size); |
851 | |
852 | /*******************************************************************//** |
853 | Creates a new single-table tablespace to a database directory of MySQL. |
854 | |
855 | === modified file 'Percona-Server/storage/innodb_plugin/include/log0log.h' |
856 | --- Percona-Server/storage/innodb_plugin/include/log0log.h 2011-11-24 02:00:43 +0000 |
857 | +++ Percona-Server/storage/innodb_plugin/include/log0log.h 2012-03-21 18:29:24 +0000 |
858 | @@ -56,6 +56,9 @@ |
859 | # define log_do_write TRUE |
860 | #endif /* UNIV_DEBUG */ |
861 | |
862 | +extern ibool log_xtra_log_archive_on; |
863 | +extern const char* log_xtra_log_archive_dir; |
864 | +extern time_t log_xtra_log_expire_sec; |
865 | /** Wait modes for log_write_up_to @{ */ |
866 | #define LOG_NO_WAIT 91 |
867 | #define LOG_WAIT_ONE_GROUP 92 |
868 | @@ -75,6 +78,17 @@ |
869 | /*==========================================*/ |
870 | ulint limit); /*!< in: limit to set */ |
871 | #endif /* !UNIV_HOTBACKUP */ |
872 | +/******************************************************//** |
873 | +Calculates the offset of an lsn within a log group. |
874 | +@return offset within the log group */ |
875 | +UNIV_INTERN |
876 | +ulint |
877 | +log_group_calc_lsn_offset( |
878 | +/*======================*/ |
879 | + ib_uint64_t lsn, /*!< in: lsn, must be within 4 GB of |
880 | + group->lsn */ |
881 | + const log_group_t* group); /*!< in: log group */ |
882 | + |
883 | /*******************************************************************//** |
884 | Calculates where in log files we find a specified lsn. |
885 | @return log file number */ |
886 | @@ -953,6 +967,10 @@ |
887 | become signaled */ |
888 | /* @} */ |
889 | #endif /* UNIV_LOG_ARCHIVE */ |
890 | + os_event_t arch_event_rotate_start; |
891 | + os_event_t arch_event_rotate_done; |
892 | + mutex_t archived_mutex; |
893 | + ib_uint64_t archived_lsn; |
894 | }; |
895 | |
896 | #ifdef UNIV_LOG_ARCHIVE |
897 | |
898 | === modified file 'Percona-Server/storage/innodb_plugin/include/log0log.ic' |
899 | --- Percona-Server/storage/innodb_plugin/include/log0log.ic 2010-06-10 14:31:28 +0000 |
900 | +++ Percona-Server/storage/innodb_plugin/include/log0log.ic 2012-03-21 18:29:24 +0000 |
901 | @@ -372,7 +372,11 @@ |
902 | |
903 | ut_ad(log_sys->buf_free <= log_sys->buf_size); |
904 | |
905 | + if (log_xtra_log_archive_on) |
906 | + mutex_enter(&(log_sys->archived_mutex)); |
907 | log_sys->lsn += len; |
908 | + if (log_xtra_log_archive_on) |
909 | + mutex_exit(&(log_sys->archived_mutex)); |
910 | |
911 | #ifdef UNIV_LOG_DEBUG |
912 | log_check_log_recs(log_sys->buf + log_sys->old_buf_free, |
913 | |
914 | === modified file 'Percona-Server/storage/innodb_plugin/include/os0file.h' |
915 | --- Percona-Server/storage/innodb_plugin/include/os0file.h 2011-11-24 02:00:47 +0000 |
916 | +++ Percona-Server/storage/innodb_plugin/include/os0file.h 2012-03-21 18:29:24 +0000 |
917 | @@ -403,6 +403,19 @@ |
918 | const char* oldpath, /*!< in: old file path as a |
919 | null-terminated string */ |
920 | const char* newpath); /*!< in: new file path */ |
921 | +/*************************************************************************** |
922 | +Renames a file (can also move it to another directory and another drive). |
923 | +If file should be placed to a different disks throttle will applied in IOPS. |
924 | +*/ |
925 | +ibool |
926 | +os_file_rename_throttled( |
927 | +/*===========*/ |
928 | + const char* oldpath, /* in: old file path as a |
929 | + null-terminated |
930 | + string */ |
931 | + const char* newpath, /* in: new file path */ |
932 | + int max_iops); /* in: maximum number of |
933 | + IO operations per sec*/ |
934 | /***********************************************************************//** |
935 | Closes a file handle. In case of error, error number can be retrieved with |
936 | os_file_get_last_error. |
937 | |
938 | === modified file 'Percona-Server/storage/innodb_plugin/include/srv0srv.h' |
939 | --- Percona-Server/storage/innodb_plugin/include/srv0srv.h 2011-11-24 02:01:25 +0000 |
940 | +++ Percona-Server/storage/innodb_plugin/include/srv0srv.h 2012-03-21 18:29:24 +0000 |
941 | @@ -185,6 +185,8 @@ |
942 | extern dulint srv_archive_recovery_limit_lsn; |
943 | #endif /* UNIV_LOG_ARCHIVE */ |
944 | |
945 | +#define IB_ARCHIVED_LOGS_PREFIX "ib_log_archive_" |
946 | + |
947 | extern char* srv_file_flush_method_str; |
948 | extern ulint srv_unix_file_flush_method; |
949 | extern ulint srv_win_file_flush_method; |
950 | @@ -445,6 +447,7 @@ |
951 | #endif |
952 | SRV_PURGE, /* thread purging undo records */ |
953 | SRV_PURGE_WORKER, /* thread purging undo records */ |
954 | + SRV_ARCHIVE, |
955 | SRV_MASTER /**< the master thread, (whose type number must |
956 | be biggest) */ |
957 | }; |
958 | @@ -643,6 +646,13 @@ |
959 | /*====================*/ |
960 | void* arg); /*!< in: a dummy parameter required by |
961 | os_thread_create */ |
962 | +/********************************************************************//** |
963 | +A thread which controls archiving transaction log files */ |
964 | +UNIV_INTERN |
965 | +os_thread_ret_t |
966 | +srv_log_archive_thread( |
967 | +/*===================*/ |
968 | + void* arg); |
969 | /******************************************************************//** |
970 | Outputs to a file the output of the InnoDB Monitor. |
971 | @return FALSE if not all information printed |
972 | @@ -665,6 +675,17 @@ |
973 | srv_export_innodb_status(void); |
974 | /*==========================*/ |
975 | |
976 | +/*************************************************************//** |
977 | +Removes old archived transaction log files. */ |
978 | +UNIV_INTERN |
979 | +void |
980 | +purge_archived_logs( |
981 | + time_t before_date, /*!< in: all files modified |
982 | + before timestamp should be removed */ |
983 | + unsigned long long before_lsn); /*!< in: files with this lsn in name |
984 | + and earler should be removed */ |
985 | +/*==========================*/ |
986 | + |
987 | /** Thread slot in the thread table */ |
988 | typedef struct srv_slot_struct srv_slot_t; |
989 | |
990 | |
991 | === modified file 'Percona-Server/storage/innodb_plugin/include/srv0start.h' |
992 | --- Percona-Server/storage/innodb_plugin/include/srv0start.h 2011-11-24 02:00:36 +0000 |
993 | +++ Percona-Server/storage/innodb_plugin/include/srv0start.h 2012-03-21 18:29:24 +0000 |
994 | @@ -37,6 +37,22 @@ |
995 | /*=======================*/ |
996 | char* str); /*!< in/out: null-terminated character string */ |
997 | /*********************************************************************//** |
998 | +Calculates the low 32 bits when a file size which is given as a number |
999 | +database pages is converted to the number of bytes. |
1000 | +@return low 32 bytes of file size when expressed in bytes */ |
1001 | +ulint |
1002 | +srv_calc_low32( |
1003 | +/*===========*/ |
1004 | + ulint file_size); /*!< in: file size in database pages */ |
1005 | +/*********************************************************************//** |
1006 | +Calculates the high 32 bits when a file size which is given as a number |
1007 | +database pages is converted to the number of bytes. |
1008 | +@return high 32 bytes of file size when expressed in bytes */ |
1009 | +ulint |
1010 | +srv_calc_high32( |
1011 | +/*============*/ |
1012 | + ulint file_size); /*!< in: file size in database pages */ |
1013 | +/*********************************************************************//** |
1014 | Reads the data files and their sizes from a character string given in |
1015 | the .cnf file. |
1016 | @return TRUE if ok, FALSE on parse error */ |
1017 | @@ -134,4 +150,10 @@ |
1018 | /** reserved for extra system tables */ |
1019 | #define SRV_EXTRA_SYS_SPACE_FIRST_ID 0xFFFFFFE0UL |
1020 | |
1021 | +#ifdef __WIN__ |
1022 | +#define SRV_PATH_SEPARATOR '\\' |
1023 | +#else |
1024 | +#define SRV_PATH_SEPARATOR '/' |
1025 | +#endif |
1026 | + |
1027 | #endif |
1028 | |
1029 | === modified file 'Percona-Server/storage/innodb_plugin/log/log0log.c' |
1030 | --- Percona-Server/storage/innodb_plugin/log/log0log.c 2011-11-24 02:01:25 +0000 |
1031 | +++ Percona-Server/storage/innodb_plugin/log/log0log.c 2012-03-21 18:29:24 +0000 |
1032 | @@ -97,6 +97,10 @@ |
1033 | UNIV_INTERN byte log_archive_io; |
1034 | #endif /* UNIV_LOG_ARCHIVE */ |
1035 | |
1036 | +UNIV_INTERN ibool log_xtra_log_archive_on = FALSE; |
1037 | + |
1038 | +UNIV_INTERN const char* log_xtra_log_archive_dir = NULL; |
1039 | +UNIV_INTERN time_t log_xtra_log_expire_sec; |
1040 | /* A margin for free space in the log buffer before a log entry is catenated */ |
1041 | #define LOG_BUF_WRITE_MARGIN (4 * OS_FILE_LOG_BLOCK_SIZE) |
1042 | |
1043 | @@ -287,10 +291,13 @@ |
1044 | ulint str_len) /*!< in: string length */ |
1045 | { |
1046 | log_t* log = log_sys; |
1047 | + log_group_t* group; |
1048 | ulint len; |
1049 | ulint data_len; |
1050 | byte* log_block; |
1051 | |
1052 | + group = UT_LIST_GET_FIRST(log->log_groups); |
1053 | + |
1054 | ut_ad(mutex_own(&(log->mutex))); |
1055 | part_loop: |
1056 | ut_ad(!recv_no_log_write); |
1057 | @@ -311,6 +318,25 @@ |
1058 | - LOG_BLOCK_TRL_SIZE; |
1059 | } |
1060 | |
1061 | + /* If the log write is across file border, should check the archive was done. */ |
1062 | + if (log_xtra_log_archive_on) { |
1063 | + if ((log_group_calc_lsn_offset(log->lsn, group) / group->file_size) |
1064 | + != (log_group_calc_lsn_offset(log->lsn + len, group) / group->file_size)) { |
1065 | + os_event_set(log_sys->arch_event_rotate_start); |
1066 | + } |
1067 | +wait_for_archived: |
1068 | + mutex_enter(&(log_sys->archived_mutex)); |
1069 | + if (log->lsn + len <= log_sys->archived_lsn + log_group_get_capacity(group)) { |
1070 | + mutex_exit(&(log_sys->archived_mutex)); |
1071 | + } else { |
1072 | + os_event_reset(log_sys->arch_event_rotate_done); |
1073 | + mutex_exit(&(log_sys->archived_mutex)); |
1074 | + os_event_set(log_sys->arch_event_rotate_start); /* for safety */ |
1075 | + os_event_wait(log_sys->arch_event_rotate_done); |
1076 | + goto wait_for_archived; |
1077 | + } |
1078 | + } |
1079 | + |
1080 | ut_memcpy(log->buf + log->buf_free, str, len); |
1081 | |
1082 | str_len -= len; |
1083 | @@ -327,12 +353,20 @@ |
1084 | log_sys->next_checkpoint_no); |
1085 | len += LOG_BLOCK_HDR_SIZE + LOG_BLOCK_TRL_SIZE; |
1086 | |
1087 | + if (log_xtra_log_archive_on) |
1088 | + mutex_enter(&(log_sys->archived_mutex)); |
1089 | log->lsn += len; |
1090 | + if (log_xtra_log_archive_on) |
1091 | + mutex_exit(&(log_sys->archived_mutex)); |
1092 | |
1093 | /* Initialize the next block header */ |
1094 | log_block_init(log_block + OS_FILE_LOG_BLOCK_SIZE, log->lsn); |
1095 | } else { |
1096 | + if (log_xtra_log_archive_on) |
1097 | + mutex_enter(&(log_sys->archived_mutex)); |
1098 | log->lsn += len; |
1099 | + if (log_xtra_log_archive_on) |
1100 | + mutex_exit(&(log_sys->archived_mutex)); |
1101 | } |
1102 | |
1103 | log->buf_free += len; |
1104 | @@ -552,7 +586,7 @@ |
1105 | /******************************************************//** |
1106 | Calculates the offset of an lsn within a log group. |
1107 | @return offset within the log group */ |
1108 | -static |
1109 | +UNIV_INTERN |
1110 | ulint |
1111 | log_group_calc_lsn_offset( |
1112 | /*======================*/ |
1113 | @@ -710,6 +744,11 @@ |
1114 | group = UT_LIST_GET_NEXT(log_groups, group); |
1115 | } |
1116 | |
1117 | + /* log_archive needs over 1 file margin */ |
1118 | + if (log_xtra_log_archive_on) { |
1119 | + smallest_capacity = smallest_archive_margin; |
1120 | + } |
1121 | + |
1122 | /* Add extra safety */ |
1123 | smallest_capacity = smallest_capacity - smallest_capacity / 10; |
1124 | |
1125 | @@ -786,6 +825,7 @@ |
1126 | log_sys = mem_alloc(sizeof(log_t)); |
1127 | |
1128 | mutex_create(&log_sys->mutex, SYNC_LOG); |
1129 | + mutex_create(&log_sys->archived_mutex, SYNC_NO_ORDER_CHECK); |
1130 | |
1131 | mutex_enter(&(log_sys->mutex)); |
1132 | |
1133 | @@ -872,6 +912,11 @@ |
1134 | log_sys->archiving_on = os_event_create(NULL); |
1135 | #endif /* UNIV_LOG_ARCHIVE */ |
1136 | |
1137 | + log_sys->arch_event_rotate_start = os_event_create(NULL); |
1138 | + os_event_set(log_sys->arch_event_rotate_start); |
1139 | + log_sys->arch_event_rotate_done = os_event_create(NULL); |
1140 | + os_event_set(log_sys->arch_event_rotate_done); |
1141 | + |
1142 | /*----------------------------*/ |
1143 | |
1144 | log_block_init(log_sys->buf, log_sys->lsn); |
1145 | @@ -1693,13 +1738,29 @@ |
1146 | log_complete_checkpoint(void) |
1147 | /*=========================*/ |
1148 | { |
1149 | + log_group_t* group; |
1150 | + |
1151 | ut_ad(mutex_own(&(log_sys->mutex))); |
1152 | ut_ad(log_sys->n_pending_checkpoint_writes == 0); |
1153 | |
1154 | + group = UT_LIST_GET_FIRST(log_sys->log_groups); |
1155 | + |
1156 | + if (log_xtra_log_archive_on) { |
1157 | + mutex_enter(&(log_sys->archived_mutex)); |
1158 | + |
1159 | + if ((log_group_calc_lsn_offset(log_sys->next_checkpoint_lsn, group) / group->file_size) |
1160 | + != (log_group_calc_lsn_offset(log_sys->last_checkpoint_lsn, group) / group->file_size)) { |
1161 | + os_event_set(log_sys->arch_event_rotate_start); |
1162 | + } |
1163 | + } |
1164 | + |
1165 | log_sys->next_checkpoint_no++; |
1166 | |
1167 | log_sys->last_checkpoint_lsn = log_sys->next_checkpoint_lsn; |
1168 | |
1169 | + if (log_xtra_log_archive_on) |
1170 | + mutex_exit(&(log_sys->archived_mutex)); |
1171 | + |
1172 | rw_lock_x_unlock_gen(&(log_sys->checkpoint_lock), LOG_CHECKPOINT); |
1173 | } |
1174 | |
1175 | @@ -3158,6 +3219,15 @@ |
1176 | goto loop; |
1177 | } |
1178 | |
1179 | + if (log_xtra_log_archive_on |
1180 | + && srv_n_threads_active[SRV_ARCHIVE] != 0) { |
1181 | + |
1182 | + mutex_exit(&kernel_mutex); |
1183 | + os_event_set(log_sys->arch_event_rotate_start); |
1184 | + |
1185 | + goto loop; |
1186 | + } |
1187 | + |
1188 | /* Check that the purge threads ended */ |
1189 | if (srv_use_purge_thread |
1190 | && (srv_n_threads_active[SRV_PURGE] != 0 |
1191 | @@ -3375,6 +3445,14 @@ |
1192 | log_sys->flushed_to_disk_lsn, |
1193 | log_sys->last_checkpoint_lsn); |
1194 | |
1195 | + if (log_xtra_log_archive_on) { |
1196 | + mutex_enter(&(log_sys->archived_mutex)); |
1197 | + fprintf(file, |
1198 | + "Log archived until %llu\n", |
1199 | + log_sys->archived_lsn); |
1200 | + mutex_exit(&(log_sys->archived_mutex)); |
1201 | + } |
1202 | + |
1203 | fprintf(file, |
1204 | "Max checkpoint age %lu\n" |
1205 | "Checkpoint age target %lu\n" |
1206 | @@ -3479,12 +3557,16 @@ |
1207 | rw_lock_free(&log_sys->checkpoint_lock); |
1208 | |
1209 | mutex_free(&log_sys->mutex); |
1210 | + mutex_free(&log_sys->archived_mutex); |
1211 | |
1212 | #ifdef UNIV_LOG_ARCHIVE |
1213 | rw_lock_free(&log_sys->archive_lock); |
1214 | os_event_create(log_sys->archiving_on); |
1215 | #endif /* UNIV_LOG_ARCHIVE */ |
1216 | |
1217 | + os_event_free(log_sys->arch_event_rotate_start); |
1218 | + os_event_free(log_sys->arch_event_rotate_done); |
1219 | + |
1220 | #ifdef UNIV_LOG_DEBUG |
1221 | recv_sys_debug_free(); |
1222 | #endif |
1223 | |
1224 | === modified file 'Percona-Server/storage/innodb_plugin/os/os0file.c' |
1225 | --- Percona-Server/storage/innodb_plugin/os/os0file.c 2011-12-15 15:27:38 +0000 |
1226 | +++ Percona-Server/storage/innodb_plugin/os/os0file.c 2012-03-21 18:29:24 +0000 |
1227 | @@ -1686,6 +1686,98 @@ |
1228 | #endif |
1229 | } |
1230 | |
1231 | +/*************************************************************************** |
1232 | +Renames a file (can also move it to another directory and another drive). |
1233 | +If file should be placed to a different disks throttle will applied in IOPS. |
1234 | +*/ |
1235 | +ibool |
1236 | +os_file_rename_throttled( |
1237 | +/*===========*/ |
1238 | + /* out: TRUE if success */ |
1239 | + const char* oldpath,/* in: old file path as a null-terminated |
1240 | + string */ |
1241 | + const char* newpath, /* in: new file path */ |
1242 | + int max_iops) /* in: maximum number of IO operations per sec*/ |
1243 | +{ |
1244 | +#ifdef __WIN__ |
1245 | + return os_file_rename(oldpath, newpath); |
1246 | +#else |
1247 | + int ret; |
1248 | + |
1249 | + ret = rename((const char*)oldpath, (const char*)newpath); |
1250 | + |
1251 | + if (ret != 0) { |
1252 | + ulint err; |
1253 | + err = (ulint) errno; |
1254 | + |
1255 | + if (err == EXDEV) { |
1256 | + ulint offset = 0; |
1257 | + ulint offset_high = 0; |
1258 | + ulint block_size = 64*UNIV_PAGE_SIZE; |
1259 | + os_file_t file; |
1260 | + os_file_t newfile; |
1261 | + byte* buf_ptr = NULL; |
1262 | + byte* buf = NULL; |
1263 | + ibool success; |
1264 | + ulint size_low; |
1265 | + ulint size_high; |
1266 | + |
1267 | + buf_ptr = ut_malloc(block_size); |
1268 | + buf = ut_align(buf_ptr, block_size); |
1269 | + |
1270 | + file = os_file_create(oldpath, OS_FILE_OPEN, OS_FILE_NORMAL, |
1271 | + OS_LOG_FILE, &success); |
1272 | + ut_a(success); |
1273 | + newfile = os_file_create(newpath, OS_FILE_CREATE, OS_FILE_NORMAL, |
1274 | + OS_LOG_FILE, &success); |
1275 | + ut_a(success); |
1276 | + |
1277 | + success = os_file_get_size(file, &size_low, &size_high); |
1278 | + ut_a(success); |
1279 | + |
1280 | + for (offset_high = 0; offset_high<size_high; ++offset_high) { |
1281 | + for (offset = 0; offset<size_low/block_size; ++offset) { |
1282 | + success = os_file_read(file, buf, offset, offset_high, block_size); |
1283 | + ut_a(success); |
1284 | + success = os_file_write(newpath, newfile, buf, offset, offset_high, block_size); |
1285 | + ut_a(success); |
1286 | + if(max_iops > 100 && (offset % (max_iops/100)) == 0 ) { |
1287 | + os_thread_sleep(10000 /* 0.01 sec */); |
1288 | + } |
1289 | + } |
1290 | + } |
1291 | + |
1292 | + if ((size_low % block_size) != 0) { |
1293 | + success = os_file_read(file, buf, offset, offset_high, (size_low % block_size)); |
1294 | + ut_a(success); |
1295 | + success = os_file_write(newpath, newfile, buf, offset, offset_high, (size_low % block_size)); |
1296 | + ut_a(success); |
1297 | + } |
1298 | + |
1299 | + success = os_file_flush(newfile, TRUE); |
1300 | + ut_a(success); |
1301 | + os_file_close(file); |
1302 | + os_file_close(newfile); |
1303 | + success = os_file_delete_if_exists(oldpath); |
1304 | + ut_a(success); |
1305 | + |
1306 | + if (buf_ptr) { |
1307 | + ut_free(buf_ptr); |
1308 | + } |
1309 | + } |
1310 | + else { |
1311 | + os_file_handle_error_no_exit(oldpath, "rename"); |
1312 | + |
1313 | + return(FALSE); |
1314 | + } |
1315 | + } |
1316 | + |
1317 | + return(TRUE); |
1318 | +#endif |
1319 | +} |
1320 | + |
1321 | + |
1322 | + |
1323 | /***********************************************************************//** |
1324 | Closes a file handle. In case of error, error number can be retrieved with |
1325 | os_file_get_last_error. |
1326 | |
1327 | === modified file 'Percona-Server/storage/innodb_plugin/srv/srv0srv.c' |
1328 | --- Percona-Server/storage/innodb_plugin/srv/srv0srv.c 2011-11-24 02:01:25 +0000 |
1329 | +++ Percona-Server/storage/innodb_plugin/srv/srv0srv.c 2012-03-21 18:29:24 +0000 |
1330 | @@ -2674,6 +2674,334 @@ |
1331 | OS_THREAD_DUMMY_RETURN; |
1332 | } |
1333 | |
1334 | +/*********************************************************************//** |
1335 | +Creates or opens the next log file and closes that */ |
1336 | +static |
1337 | +ulint |
1338 | +open_or_create_new_log_file( |
1339 | +/*========================*/ |
1340 | + const char* name) |
1341 | +{ |
1342 | + ibool ret; |
1343 | + ulint size; |
1344 | + ulint size_high; |
1345 | + os_file_t new_file; |
1346 | + |
1347 | + new_file = os_file_create(name, OS_FILE_CREATE, OS_FILE_NORMAL, |
1348 | + OS_LOG_FILE, &ret); |
1349 | + if (ret == FALSE) { |
1350 | + if (os_file_get_last_error(FALSE) != OS_FILE_ALREADY_EXISTS) { |
1351 | + fprintf(stderr, |
1352 | + "InnoDB: Error in creating" |
1353 | + " or opening %s\n", name); |
1354 | + |
1355 | + return(DB_ERROR); |
1356 | + } |
1357 | + |
1358 | + new_file = os_file_create(name, OS_FILE_OPEN, OS_FILE_AIO, |
1359 | + OS_LOG_FILE, &ret); |
1360 | + if (!ret) { |
1361 | + fprintf(stderr, |
1362 | + "InnoDB: Error in opening %s\n", name); |
1363 | + return(DB_ERROR); |
1364 | + } |
1365 | + |
1366 | + ret = os_file_get_size(new_file, &size, &size_high); |
1367 | + ut_a(ret); |
1368 | + |
1369 | + if (size != srv_calc_low32(srv_log_file_size) |
1370 | + || size_high != srv_calc_high32(srv_log_file_size)) { |
1371 | + |
1372 | + fprintf(stderr, |
1373 | + "InnoDB: Error: log file %s is" |
1374 | + " of different size %lu %lu bytes\n" |
1375 | + "InnoDB: than specified in the .cnf" |
1376 | + " file %lu %lu bytes! Rename or remove the file.\n", |
1377 | + name, (ulong) size_high, (ulong) size, |
1378 | + (ulong) srv_calc_high32(srv_log_file_size), |
1379 | + (ulong) srv_calc_low32(srv_log_file_size)); |
1380 | + |
1381 | + return(DB_ERROR); |
1382 | + } |
1383 | + } else { |
1384 | + ut_print_timestamp(stderr); |
1385 | + |
1386 | + fprintf(stderr, |
1387 | + " InnoDB: Log file %s did not exist:" |
1388 | + " new to be created\n", |
1389 | + name); |
1390 | + |
1391 | + fprintf(stderr, "InnoDB: Setting log file %s size to %lu MB\n", |
1392 | + name, (ulong) srv_log_file_size |
1393 | + >> (20 - UNIV_PAGE_SIZE_SHIFT)); |
1394 | + |
1395 | + fprintf(stderr, |
1396 | + "InnoDB: Database physically writes the file" |
1397 | + " full: wait...\n"); |
1398 | + |
1399 | + ret = os_file_set_size(name, new_file, |
1400 | + srv_calc_low32(srv_log_file_size), |
1401 | + srv_calc_high32(srv_log_file_size)); |
1402 | + if (!ret) { |
1403 | + fprintf(stderr, |
1404 | + "InnoDB: Error in creating %s:" |
1405 | + " probably out of disk space\n", |
1406 | + name); |
1407 | + |
1408 | + return(DB_ERROR); |
1409 | + } |
1410 | + |
1411 | + fprintf(stderr, |
1412 | + "InnoDB: Setting log file was done\n"); |
1413 | + } |
1414 | + |
1415 | + ret = os_file_close(new_file); |
1416 | + ut_a(ret); |
1417 | + |
1418 | + return(DB_SUCCESS); |
1419 | +} |
1420 | + |
1421 | +/*************************************************************//** |
1422 | +Removes old archived transaction log files. */ |
1423 | +void |
1424 | +purge_archived_logs( |
1425 | + time_t before_date, /*!< in: all files modified |
1426 | + before timestamp should be removed */ |
1427 | + unsigned long long before_lsn) /*!< in: files with this lsn in name |
1428 | + and earler should be removed */ |
1429 | +{ |
1430 | + os_file_dir_t dir; |
1431 | + os_file_stat_t fileinfo; |
1432 | + char archived_log_filename[OS_FILE_MAX_PATH]; |
1433 | + ulint dirnamelen; |
1434 | + |
1435 | + if (log_xtra_log_archive_dir) { |
1436 | + dir = os_file_opendir(log_xtra_log_archive_dir, FALSE); |
1437 | + if (!dir) { |
1438 | + fprintf(stderr, |
1439 | + "InnoDB: Note: opening archived log directory %s was failed. " |
1440 | + "Purge archived logs are not available\n", |
1441 | + log_xtra_log_archive_dir); |
1442 | + return; |
1443 | + } |
1444 | + } else { |
1445 | + return; |
1446 | + } |
1447 | + |
1448 | + dirnamelen = strlen(log_xtra_log_archive_dir); |
1449 | + |
1450 | + memcpy(archived_log_filename, log_xtra_log_archive_dir, dirnamelen); |
1451 | + if (dirnamelen && archived_log_filename[dirnamelen - 1] != SRV_PATH_SEPARATOR) { |
1452 | + archived_log_filename[dirnamelen++] = SRV_PATH_SEPARATOR; |
1453 | + } |
1454 | + |
1455 | + memset(&fileinfo, 0, sizeof(fileinfo)); |
1456 | + while(!os_file_readdir_next_file(log_xtra_log_archive_dir, dir, &fileinfo) ) { |
1457 | + if (fileinfo.name == strstr(fileinfo.name, IB_ARCHIVED_LOGS_PREFIX)) { |
1458 | + if (dirnamelen + strlen(fileinfo.name)+2 > OS_FILE_MAX_PATH) |
1459 | + continue; |
1460 | + |
1461 | + sprintf(archived_log_filename + dirnamelen, "%s", fileinfo.name); |
1462 | + |
1463 | + if (before_lsn) { |
1464 | + unsigned long long log_file_lsn = strtoll(fileinfo.name+ strlen(IB_ARCHIVED_LOGS_PREFIX), NULL, 10); |
1465 | + if (before_lsn <= log_file_lsn) { |
1466 | + continue; |
1467 | + } |
1468 | + } else { |
1469 | + fileinfo.mtime = 0; |
1470 | + if (os_file_get_status(archived_log_filename, &fileinfo) == FALSE || fileinfo.mtime == 0) { |
1471 | + fprintf(stderr, |
1472 | + "InnoDB: Note: read archived log last modification date for: %s.\n", |
1473 | + archived_log_filename); |
1474 | + continue; |
1475 | + } |
1476 | + |
1477 | + if (before_date == 0 || fileinfo.mtime > before_date) { |
1478 | + continue; |
1479 | + } |
1480 | + } |
1481 | + |
1482 | + if (os_file_delete(archived_log_filename) == FALSE) { |
1483 | + fprintf(stderr, |
1484 | + "InnoDB: Note: can't delete archived log file %s.\n", |
1485 | + archived_log_filename); |
1486 | + } |
1487 | + } |
1488 | + } |
1489 | + |
1490 | + os_file_closedir(dir); |
1491 | +} |
1492 | + |
1493 | +/*********************************************************************//** |
1494 | +A thread which controls archiving transaction log files */ |
1495 | +UNIV_INTERN |
1496 | +os_thread_ret_t |
1497 | +srv_log_archive_thread( |
1498 | +/*===================*/ |
1499 | + void* arg __attribute__((unused))) |
1500 | +{ |
1501 | + log_group_t* group = UT_LIST_GET_FIRST(log_sys->log_groups); |
1502 | + ibool next_is_prepared; |
1503 | + ulint err; |
1504 | + char new_file_name[OS_FILE_MAX_PATH]; |
1505 | + char archive_file_name[OS_FILE_MAX_PATH]; |
1506 | + char final_file_name[OS_FILE_MAX_PATH]; |
1507 | + ulint dirnamelen; |
1508 | + ulint final_dirnamelen; |
1509 | + |
1510 | + ut_a(log_xtra_log_archive_on); |
1511 | + |
1512 | + mutex_enter(&kernel_mutex); |
1513 | + srv_table_reserve_slot(SRV_ARCHIVE); |
1514 | + srv_n_threads_active[SRV_ARCHIVE]++; |
1515 | + mutex_exit(&kernel_mutex); |
1516 | + |
1517 | + /* initialization */ |
1518 | + /* assume before the current file was archived already */ |
1519 | + mutex_enter(&(log_sys->archived_mutex)); |
1520 | + log_sys->archived_lsn = log_sys->lsn |
1521 | + - ((log_group_calc_lsn_offset(log_sys->lsn - 1, group) |
1522 | + % group->file_size) + 1 - LOG_FILE_HDR_SIZE); |
1523 | + os_event_set(log_sys->arch_event_rotate_done); |
1524 | + mutex_exit(&(log_sys->archived_mutex)); |
1525 | + |
1526 | + fprintf(stderr, |
1527 | + "InnoDB: Note: transaction log archivation thread started.\n" |
1528 | + ); |
1529 | + |
1530 | + |
1531 | + /* confirm new file and create if needed */ |
1532 | + dirnamelen = strlen(srv_log_group_home_dirs[0]); |
1533 | + ut_a(dirnamelen < (sizeof new_file_name) - 31 - sizeof "ib_logfile"); |
1534 | + |
1535 | + if (log_xtra_log_archive_dir) { |
1536 | + final_dirnamelen = strlen(log_xtra_log_archive_dir); |
1537 | + memcpy(final_file_name, log_xtra_log_archive_dir, final_dirnamelen); |
1538 | + if (final_dirnamelen && final_file_name[final_dirnamelen - 1] != SRV_PATH_SEPARATOR) { |
1539 | + final_file_name[final_dirnamelen++] = SRV_PATH_SEPARATOR; |
1540 | + } |
1541 | + } else { |
1542 | + final_dirnamelen = 0; |
1543 | + } |
1544 | + |
1545 | + memcpy(new_file_name, srv_log_group_home_dirs[0], dirnamelen); |
1546 | + if (dirnamelen && new_file_name[dirnamelen - 1] != SRV_PATH_SEPARATOR) { |
1547 | + new_file_name[dirnamelen++] = SRV_PATH_SEPARATOR; |
1548 | + } |
1549 | + memcpy(archive_file_name, new_file_name, dirnamelen); |
1550 | + |
1551 | + sprintf(new_file_name + dirnamelen, "%s", "ib_next_logfile"); |
1552 | + |
1553 | + next_is_prepared = FALSE; |
1554 | + |
1555 | +loop: |
1556 | + os_event_reset(log_sys->arch_event_rotate_start); |
1557 | + |
1558 | + /* NOTICE: waiting the event log_sys->arch_event_rotate_done is done with |
1559 | + log_sys->mutex holding. |
1560 | + So, we cannot obtain log_sys->mutex in this thread. |
1561 | + The values shared with the other threads should be protected by log_sys->archived_mutex */ |
1562 | + |
1563 | +retry_create_file: |
1564 | + if (!next_is_prepared) { |
1565 | + err = open_or_create_new_log_file(new_file_name); |
1566 | + if (err == DB_SUCCESS) { |
1567 | + next_is_prepared = TRUE; |
1568 | + } else { |
1569 | + next_is_prepared = FALSE; |
1570 | + fprintf(stderr, |
1571 | + "InnoDB: Warning: The new log file cannot be created. " |
1572 | + "The above errors should be fixed\n" |
1573 | + " will retry 10 seconds later again.\n"); |
1574 | + os_thread_sleep(10000000); |
1575 | + goto retry_create_file; |
1576 | + } |
1577 | + } |
1578 | + |
1579 | + ut_a(next_is_prepared); |
1580 | + |
1581 | + mutex_enter(&(log_sys->archived_mutex)); |
1582 | + if (log_sys->last_checkpoint_lsn |
1583 | + >= log_sys->archived_lsn + (group->file_size - LOG_FILE_HDR_SIZE)) { |
1584 | + ulint hdr_offset; |
1585 | + ibool success; |
1586 | + |
1587 | + mutex_exit(&(log_sys->archived_mutex)); |
1588 | + trx_sys_print_mysql_binlog_offset(); |
1589 | + |
1590 | + sprintf(archive_file_name + dirnamelen, "%s%016lld", |
1591 | + IB_ARCHIVED_LOGS_PREFIX, log_sys->archived_lsn); |
1592 | + |
1593 | + if (final_dirnamelen) { |
1594 | + sprintf(final_file_name + final_dirnamelen, "%s%016lld", |
1595 | + IB_ARCHIVED_LOGS_PREFIX, log_sys->archived_lsn); |
1596 | + } |
1597 | + |
1598 | + hdr_offset = (log_group_calc_lsn_offset(log_sys->archived_lsn, group) |
1599 | + / group->file_size) * group->file_size / UNIV_PAGE_SIZE; |
1600 | + |
1601 | + success = fil_rotate_file(group->space_id, hdr_offset, |
1602 | + archive_file_name, new_file_name, |
1603 | + LOG_FILE_HDR_SIZE); |
1604 | + |
1605 | + if (success) { |
1606 | + mutex_enter(&(log_sys->archived_mutex)); |
1607 | + log_sys->archived_lsn += (group->file_size - LOG_FILE_HDR_SIZE); |
1608 | + os_event_set(log_sys->arch_event_rotate_done); |
1609 | + mutex_exit(&(log_sys->archived_mutex)); |
1610 | + next_is_prepared = FALSE; |
1611 | + |
1612 | + if (log_xtra_log_archive_dir && !(final_dirnamelen == dirnamelen && |
1613 | + 0 == strcmp(log_xtra_log_archive_dir, srv_log_group_home_dirs[0])) |
1614 | + ) { |
1615 | + if (FALSE == os_file_rename_throttled(archive_file_name, final_file_name, 0)) { |
1616 | + fprintf(stderr, |
1617 | + "InnoDB: Note: moving archived log to %s was failed\n", |
1618 | + final_file_name); |
1619 | + } |
1620 | + } |
1621 | + |
1622 | + if (log_xtra_log_expire_sec) { |
1623 | + purge_archived_logs(ut_time() - log_xtra_log_expire_sec, 0); |
1624 | + } |
1625 | + |
1626 | + |
1627 | + goto retry_create_file; |
1628 | + } else { |
1629 | + fprintf(stderr, |
1630 | + "InnoDB: Error: archiving %s was failed\n" |
1631 | + " will retry 10 seconds later again.\n", |
1632 | + archive_file_name); |
1633 | + os_thread_sleep(10000000); |
1634 | + goto retry_create_file; |
1635 | + } |
1636 | + } else { |
1637 | + mutex_exit(&(log_sys->archived_mutex)); |
1638 | + } |
1639 | + |
1640 | + if (srv_shutdown_state > 0 || log_xtra_log_archive_on == FALSE) { |
1641 | + fprintf(stderr, |
1642 | + "InnoDB: stopping transaction log archivation\n" |
1643 | + ); |
1644 | + goto exit_func; |
1645 | + } |
1646 | + |
1647 | + os_event_wait(log_sys->arch_event_rotate_start); |
1648 | + |
1649 | + goto loop; |
1650 | + |
1651 | +exit_func: |
1652 | + os_event_set(log_sys->arch_event_rotate_done); |
1653 | + |
1654 | + mutex_enter(&kernel_mutex); |
1655 | + srv_n_threads_active[SRV_ARCHIVE]--; |
1656 | + mutex_exit(&kernel_mutex); |
1657 | + os_thread_exit(NULL); |
1658 | + |
1659 | + OS_THREAD_DUMMY_RETURN; /* srv_log_archive_thread() */ |
1660 | +} |
1661 | + |
1662 | /*******************************************************************//** |
1663 | Tells the InnoDB server that there has been activity in the database |
1664 | and wakes up the master thread if it is suspended (not sleeping). Used |
1665 | |
1666 | === modified file 'Percona-Server/storage/innodb_plugin/srv/srv0start.c' |
1667 | --- Percona-Server/storage/innodb_plugin/srv/srv0start.c 2011-11-24 02:00:47 +0000 |
1668 | +++ Percona-Server/storage/innodb_plugin/srv/srv0start.c 2012-03-21 18:29:24 +0000 |
1669 | @@ -127,9 +127,9 @@ |
1670 | static ulint ios; |
1671 | |
1672 | /** io_handler_thread parameters for thread identification */ |
1673 | -static ulint n[SRV_MAX_N_IO_THREADS + 7 + UNIV_MAX_PARALLELISM]; |
1674 | +static ulint n[SRV_MAX_N_IO_THREADS + 8 + UNIV_MAX_PARALLELISM]; |
1675 | /** io_handler_thread identifiers */ |
1676 | -static os_thread_id_t thread_ids[SRV_MAX_N_IO_THREADS + 7 + UNIV_MAX_PARALLELISM]; |
1677 | +static os_thread_id_t thread_ids[SRV_MAX_N_IO_THREADS + 8 + UNIV_MAX_PARALLELISM]; |
1678 | |
1679 | /** We use this mutex to test the return value of pthread_mutex_trylock |
1680 | on successful locking. HP-UX does NOT return 0, though Linux et al do. */ |
1681 | @@ -490,12 +490,6 @@ |
1682 | } |
1683 | #endif /* !UNIV_HOTBACKUP */ |
1684 | |
1685 | -#ifdef __WIN__ |
1686 | -#define SRV_PATH_SEPARATOR '\\' |
1687 | -#else |
1688 | -#define SRV_PATH_SEPARATOR '/' |
1689 | -#endif |
1690 | - |
1691 | /*********************************************************************//** |
1692 | Normalizes a directory path for Windows: converts slashes to backslashes. */ |
1693 | UNIV_INTERN |
1694 | @@ -520,7 +514,6 @@ |
1695 | Calculates the low 32 bits when a file size which is given as a number |
1696 | database pages is converted to the number of bytes. |
1697 | @return low 32 bytes of file size when expressed in bytes */ |
1698 | -static |
1699 | ulint |
1700 | srv_calc_low32( |
1701 | /*===========*/ |
1702 | @@ -533,7 +526,6 @@ |
1703 | Calculates the high 32 bits when a file size which is given as a number |
1704 | database pages is converted to the number of bytes. |
1705 | @return high 32 bytes of file size when expressed in bytes */ |
1706 | -static |
1707 | ulint |
1708 | srv_calc_high32( |
1709 | /*============*/ |
1710 | @@ -1880,6 +1872,12 @@ |
1711 | os_thread_create(&srv_LRU_dump_restore_thread, NULL, |
1712 | thread_ids + 5 + SRV_MAX_N_IO_THREADS); |
1713 | |
1714 | + /* Create the thread which archives transaction log files */ |
1715 | + if (log_xtra_log_archive_on) { |
1716 | + os_thread_create(&srv_log_archive_thread, NULL, |
1717 | + thread_ids + 6 + SRV_MAX_N_IO_THREADS); |
1718 | + } |
1719 | + |
1720 | /* If srv_blocking_lru_restore is TRUE, load buffer pool contents |
1721 | synchronously */ |
1722 | if (srv_auto_lru_dump && srv_blocking_lru_restore) |
1723 | @@ -1909,13 +1907,13 @@ |
1724 | ulint i; |
1725 | |
1726 | os_thread_create(&srv_purge_thread, NULL, thread_ids |
1727 | - + (6 + SRV_MAX_N_IO_THREADS)); |
1728 | + + (7 + SRV_MAX_N_IO_THREADS)); |
1729 | |
1730 | for (i = 0; i < srv_use_purge_thread - 1; i++) { |
1731 | - n[7 + i + SRV_MAX_N_IO_THREADS] = i; /* using as index for arrays in purge_sys */ |
1732 | + n[8 + i + SRV_MAX_N_IO_THREADS] = i; /* using as index for arrays in purge_sys */ |
1733 | os_thread_create(&srv_purge_worker_thread, |
1734 | - n + (7 + i + SRV_MAX_N_IO_THREADS), |
1735 | - thread_ids + (7 + i + SRV_MAX_N_IO_THREADS)); |
1736 | + n + (8 + i + SRV_MAX_N_IO_THREADS), |
1737 | + thread_ids + (8 + i + SRV_MAX_N_IO_THREADS)); |
1738 | } |
1739 | } |
1740 | #ifdef UNIV_DEBUG |
1741 | |
1742 | === modified file 'doc/source/index.rst' |
1743 | --- doc/source/index.rst 2012-03-16 09:07:44 +0000 |
1744 | +++ doc/source/index.rst 2012-03-21 18:29:24 +0000 |
1745 | @@ -125,6 +125,7 @@ |
1746 | management/udf_maatkit |
1747 | management/innodb_fake_changes |
1748 | management/innodb_kill_idle_trx |
1749 | + management/innodb_log_archiving |
1750 | |
1751 | Diagnostics Improvements |
1752 | ======================== |
1753 | |
1754 | === added file 'doc/source/management/innodb_log_archiving.rst' |
1755 | --- doc/source/management/innodb_log_archiving.rst 1970-01-01 00:00:00 +0000 |
1756 | +++ doc/source/management/innodb_log_archiving.rst 2012-03-21 18:29:24 +0000 |
1757 | @@ -0,0 +1,59 @@ |
1758 | +.. _innodb_fast_index_creation: |
1759 | + |
1760 | +===================== |
1761 | + Innodb Log archiving |
1762 | +===================== |
1763 | + |
1764 | +The purpose of archiving logs is to be able to use them as a form of incremental backup. |
1765 | + |
1766 | +By replaying archived logs since the last backup taken with xtrabackup, you can get an online incremental backup. |
1767 | + |
1768 | +When log archiving is enabled, instead of rotating through log files, XtraDB will create a new one. There should always be one spare log file so that transactions are never stalled at log file rotation. |
1769 | + |
1770 | +``PURGE ARCHIVE LOGS`` |
1771 | +================ |
1772 | + |
1773 | +Syntax: |
1774 | + |
1775 | +PURGE ARCHIVED LOGS { TO 'filename' | BEFORE `datetime`} |
1776 | + |
1777 | +Examples: |
1778 | + |
1779 | +PURGE ARCHIVED LOGS TO 'ib_log_archive_000262325167341'; |
1780 | + |
1781 | +PURGE ARCHIVED LOGS BEFORE '2008-04-02 22:46:26'; |
1782 | + |
1783 | + |
1784 | + |
1785 | +System Variables |
1786 | +================ |
1787 | + |
1788 | +.. variable:: innodb_xtra_log_archive |
1789 | + |
1790 | + :cli: Yes |
1791 | + :conf: Yes |
1792 | + :scope: Global |
1793 | + :dyn: Yes |
1794 | + :vartype: Boolean |
1795 | + :default: OFF |
1796 | + :range: ON/OFF |
1797 | + |
1798 | +.. variable:: innodb_xtra_log_archive_dir |
1799 | + |
1800 | + :cli: Yes |
1801 | + :conf: Yes |
1802 | + :scope: Global |
1803 | + :dyn: No |
1804 | + :vartype: String |
1805 | + :default: '' |
1806 | + |
1807 | +.. variable:: innodb_xtra_expire_archive_logs_sec |
1808 | + |
1809 | + :cli: Yes |
1810 | + :conf: Yes |
1811 | + :scope: Global |
1812 | + :dyn: Yes |
1813 | + :vartype: Integer |
1814 | + :default: 0 |
1815 | + :range: 0- |
1816 | + |
Nickolay -
Overall impression looks good. I have some comments/questions:
What we are going to do when/if UNIV_LOG_ARCHIVE InnoDB log archiving
is enabled upstream? The implementations are not drop-in replacements
for each other (see below), so maybe it's good to prevent potential
user confusion by inventing another name for the XtraDB one?
What is the difference between this implementation and
UNIV_LOG_ARCHIVE anyway? As I see it:
1) this one is simpler and more parallel.
2) UNIV_LOG_ARCHIVE is able to archive up to the last checkpoint,
while this one is able to archive up to the last log file change.
Are there any other differences?
I am not sure about the archive file naming convention, i.e. including
the LSN in the file name instead of sequence number. With sequence
number it is easier to detect continuity errors when some file gets
lost. Perhaps include both?
SLEEP 4 in the test may result in non-deterministic behaviour on
loaded test machines. It's better to use DEBUG_SYNC facilities.
Why does log_sys->lsn need to be protected by log_sys- >archived_ mutex
too? I see no need to stop it advancing while the archiving is going
on as far as the log_write_low check is in place.
fil_swap_file does not have a header comment.
Please document the new fields in log_sys struct. I think a better
name for archived_mutex would be arch_sys_mutex or similar.
purge_archived_logs should have an InnoDB style header comment.
Please move SRV_PATH_SEPARATOR, srv_calc_low32, srv_calc_high32
definitions from srv0start.c to srv0start.h instead of copying them to
srv0srv.c
s/archivation/ archiving
s/"Log archived until"/"Log archived up to" in log_print()
Diff line 315: whitespace mismatch
Inconsistent code formatting: occurences of "if(foo)" or even
"if( foo)". Should be "if (foo)" everywhere.
I'd try to pretend to support multiple log groups with >log_groups, but I guess that's
UT_LIST_GET_NEXT loop over the log_sys-
strictly optional.