Percona Server moved to https://jira.percona.com/projects/PS

Merge lp:~akopytov/percona-server/bug1170103-5.5 into lp:percona-server/5.5

  1. bug1170103-5.5
  2. Merge into 5.5
Proposed by Alexey Kopytov
Status: Merged
Approved by: Alexey Kopytov
Approved revision: 503
Merged at revision: 513
Proposed branch: lp:~akopytov/percona-server/bug1170103-5.5
Merge into: lp:percona-server/5.5
Diff against target: 222 lines (+68/-13)
9 files modified
Percona-Server/mysql-test/t/percona_bug1170103.test (+23/-0)
Percona-Server/storage/innobase/handler/ha_innodb.cc (+4/-0)
Percona-Server/storage/innobase/include/srv0srv.h (+5/-0)
Percona-Server/storage/innobase/read/read0read.c (+6/-0)
Percona-Server/storage/innobase/row/row0sel.c (+2/-2)
Percona-Server/storage/innobase/srv/srv0srv.c (+14/-0)
Percona-Server/storage/innobase/trx/trx0sys.c (+2/-0)
Percona-Server/storage/innobase/trx/trx0trx.c (+1/-0)
build/percona-server.spec (+11/-11)
To merge this branch: bzr merge lp:~akopytov/percona-server/bug1170103-5.5
Reviewer Review Type Date Requested Status
Laurynas Biveinis (community) Approve
Review via email: mp+161066@code.launchpad.net

Description of the change

Since no GCA merge is possible, the branch is based on release-5.5.30-30.2.

http://jenkins.percona.com/view/PS%205.5/job/percona-server-5.5-param/723/

To post a comment you must log in.
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

The code is OK.

I am wondering about testing. I suspect that there should be existing MTR tests that would report this error under Valgrind. But as our awesome Jenkins setup Valgrind builds are broken since February 13th, I am waiting for some local tests to complete. Perhaps you have looked into this already? If there are none, perhaps we should add one?

review: Needs Information
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

I tried to grep the testsuite for any tests that have InnoDB and query cache enabled. I found three, out of which innodb.innodb, innodb.innodb_misc1 are Valgrind-clean with the bug present. innodb.innodb_mysql crashes.

lp:~akopytov/percona-server/bug1170103-5.5 updated
503. By Alexey Kopytov

Bug #1170103: Memory leak @ read_view_open_now...

row_search_check_if_query_cache_permitted() did not use the prebuilt
read view and allocated a new one on each call instead.

Fixed by reusing the prebuilt view appropriately.

This revision also adds additional status variables, should we have to
diagnose similar problems in future.

Revision history for this message
Alexey Kopytov (akopytov) wrote :

Added a minimal test case. http://jenkins.percona.com/view/PS%205.5/job/percona-server-5.5-param/724/

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

OK, thank you.

review: Approve
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Just noticed that it's "work in progress." Please set to Approved yourself is that was the wrong status, or ping me if a re-review is needed at a later time.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'Percona-Server/mysql-test/t/percona_bug1170103.test'
2--- Percona-Server/mysql-test/t/percona_bug1170103.test 1970-01-01 00:00:00 +0000
3+++ Percona-Server/mysql-test/t/percona_bug1170103.test 2013-04-27 10:07:32 +0000
4@@ -0,0 +1,23 @@
5+########################################################################
6+# Byg #1170103: Memory leak @ read_view_open_now...
7+########################################################################
8+
9+--source include/have_query_cache.inc
10+--source include/have_innodb.inc
11+
12+CREATE TABLE t(a INT) ENGINE=InnoDB;
13+INSERT INTO t VALUES (1), (2), (3);
14+
15+SET @old_query_cache_size=@@query_cache_size;
16+SET @old_query_cache_type=@@query_cache_type;
17+
18+SET GLOBAL query_cache_size=1024*1024;
19+SET GLOBAL query_cache_type=1;
20+
21+SELECT * FROM t;
22+SELECT * FROM t;
23+
24+SET GLOBAL query_cache_size=@old_query_cache_size;
25+SET GLOBAL query_cache_type=@old_query_cache_type;
26+
27+DROP TABLE t;
28
29=== modified file 'Percona-Server/storage/innobase/handler/ha_innodb.cc'
30--- Percona-Server/storage/innobase/handler/ha_innodb.cc 2013-04-11 21:03:27 +0000
31+++ Percona-Server/storage/innobase/handler/ha_innodb.cc 2013-04-27 10:07:32 +0000
32@@ -865,6 +865,10 @@
33 (char*) &export_vars.innodb_rows_read, SHOW_LONG},
34 {"rows_updated",
35 (char*) &export_vars.innodb_rows_updated, SHOW_LONG},
36+ {"read_views_memory",
37+ (char*) &export_vars.innodb_read_views_memory, SHOW_LONG},
38+ {"descriptors_memory",
39+ (char*) &export_vars.innodb_descriptors_memory, SHOW_LONG},
40 {"s_lock_os_waits",
41 (char*) &export_vars.innodb_s_lock_os_waits, SHOW_LONGLONG},
42 {"s_lock_spin_rounds",
43
44=== modified file 'Percona-Server/storage/innobase/include/srv0srv.h'
45--- Percona-Server/storage/innobase/include/srv0srv.h 2013-04-04 10:50:07 +0000
46+++ Percona-Server/storage/innobase/include/srv0srv.h 2013-04-27 10:07:32 +0000
47@@ -289,6 +289,9 @@
48 extern ulint srv_n_rows_deleted;
49 extern ulint srv_n_rows_read;
50
51+extern ulint srv_read_views_memory;
52+extern ulint srv_descriptors_memory;
53+
54 extern ibool srv_print_innodb_monitor;
55 extern ibool srv_print_innodb_lock_monitor;
56 extern ibool srv_print_innodb_tablespace_monitor;
57@@ -893,6 +896,8 @@
58 ulint innodb_rows_updated; /*!< srv_n_rows_updated */
59 ulint innodb_rows_deleted; /*!< srv_n_rows_deleted */
60 ulint innodb_truncated_status_writes; /*!< srv_truncated_status_writes */
61+ ulint innodb_read_views_memory; /*!< srv_read_views_memory */
62+ ulint innodb_descriptors_memory; /*!< srv_descriptors_memory */
63 ib_int64_t innodb_s_lock_os_waits;
64 ib_int64_t innodb_s_lock_spin_rounds;
65 ib_int64_t innodb_s_lock_spin_waits;
66
67=== modified file 'Percona-Server/storage/innobase/read/read0read.c'
68--- Percona-Server/storage/innobase/read/read0read.c 2013-03-25 09:59:39 +0000
69+++ Percona-Server/storage/innobase/read/read0read.c 2013-04-27 10:07:32 +0000
70@@ -150,6 +150,7 @@
71 {
72 if (view == NULL) {
73 view = ut_malloc(sizeof(read_view_t));
74+ srv_read_views_memory += sizeof(read_view_t);
75 view->max_descr = 0;
76 view->descriptors = NULL;
77 }
78@@ -159,6 +160,8 @@
79 /* avoid frequent re-allocations by extending the array to the
80 desired size + 10% */
81
82+ srv_read_views_memory += (n + n / 10 - view->max_descr) *
83+ sizeof(trx_id_t);
84 view->max_descr = n + n / 10;
85 view->descriptors = ut_realloc(view->descriptors,
86 view->max_descr *
87@@ -370,6 +373,9 @@
88 {
89 ut_ad(mutex_own(&kernel_mutex));
90
91+ srv_read_views_memory -= sizeof(read_view_t) +
92+ view->max_descr * sizeof(trx_id_t);
93+
94 if (view->descriptors != NULL) {
95 ut_free(view->descriptors);
96 }
97
98=== modified file 'Percona-Server/storage/innobase/row/row0sel.c'
99--- Percona-Server/storage/innobase/row/row0sel.c 2013-03-26 01:40:02 +0000
100+++ Percona-Server/storage/innobase/row/row0sel.c 2013-04-27 10:07:32 +0000
101@@ -4899,8 +4899,8 @@
102
103 trx->read_view =
104 read_view_open_now(trx->id,
105- NULL, TRUE);
106-
107+ trx->prebuilt_view, TRUE);
108+ trx->prebuilt_view = trx->read_view;
109 trx->global_read_view = trx->read_view;
110 }
111 }
112
113=== modified file 'Percona-Server/storage/innobase/srv/srv0srv.c'
114--- Percona-Server/storage/innobase/srv/srv0srv.c 2013-04-04 10:50:07 +0000
115+++ Percona-Server/storage/innobase/srv/srv0srv.c 2013-04-27 10:07:32 +0000
116@@ -463,6 +463,9 @@
117 UNIV_INTERN ulint srv_n_rows_deleted CACHE_ALIGNED = 0;
118 UNIV_INTERN ulint srv_n_rows_read CACHE_ALIGNED = 0;
119
120+UNIV_INTERN ulint srv_read_views_memory CACHE_ALIGNED = 0;
121+UNIV_INTERN ulint srv_descriptors_memory CACHE_ALIGNED = 0;
122+
123 UNIV_INTERN ulint srv_n_lock_deadlock_count CACHE_ALIGNED = 0;
124 UNIV_INTERN ulint srv_n_lock_wait_count CACHE_ALIGNED = 0;
125 UNIV_INTERN ulint srv_n_lock_wait_current_count CACHE_ALIGNED = 0;
126@@ -2129,6 +2132,9 @@
127 "; in additional pool allocated " ULINTPF "\n",
128 ut_total_allocated_memory,
129 mem_pool_get_reserved(mem_comm_pool));
130+ fprintf(file,
131+ "Total memory allocated by read views " ULINTPF "\n",
132+ srv_read_views_memory);
133 /* Calcurate reserved memories */
134 if (btr_search_sys && btr_search_sys->hash_index[0]->heap) {
135 btr_search_sys_subtotal = mem_heap_get_size(btr_search_sys->hash_index[0]->heap);
136@@ -2215,6 +2221,12 @@
137 fprintf(file, "%lu read views open inside InnoDB\n",
138 UT_LIST_GET_LEN(trx_sys->view_list));
139
140+ fprintf(file, "%lu transactions active inside InnoDB\n",
141+ UT_LIST_GET_LEN(trx_sys->trx_list));
142+
143+ fprintf(file, "%lu out of %lu descriptors used\n",
144+ trx_sys->descr_n_used, trx_sys->descr_n_max);
145+
146 if (UT_LIST_GET_LEN(trx_sys->view_list)) {
147 read_view_t* view = UT_LIST_GET_LAST(trx_sys->view_list);
148
149@@ -2524,6 +2536,8 @@
150 export_vars.innodb_rows_updated = srv_n_rows_updated;
151 export_vars.innodb_rows_deleted = srv_n_rows_deleted;
152 export_vars.innodb_truncated_status_writes = srv_truncated_status_writes;
153+ export_vars.innodb_read_views_memory = srv_read_views_memory;
154+ export_vars.innodb_descriptors_memory = srv_descriptors_memory;
155
156 #ifdef UNIV_DEBUG
157 if (trx_sys->max_trx_id < purge_sys->done_trx_no) {
158
159=== modified file 'Percona-Server/storage/innobase/trx/trx0sys.c'
160--- Percona-Server/storage/innobase/trx/trx0sys.c 2013-03-25 09:59:39 +0000
161+++ Percona-Server/storage/innobase/trx/trx0sys.c 2013-04-27 10:07:32 +0000
162@@ -1324,6 +1324,8 @@
163 TRX_DESCR_ARRAY_INITIAL_SIZE);
164 trx_sys->descr_n_max = TRX_DESCR_ARRAY_INITIAL_SIZE;
165 trx_sys->descr_n_used = 0;
166+ srv_descriptors_memory = TRX_DESCR_ARRAY_INITIAL_SIZE *
167+ sizeof(trx_id_t);
168
169 sys_header = trx_sysf_get(&mtr);
170
171
172=== modified file 'Percona-Server/storage/innobase/trx/trx0trx.c'
173--- Percona-Server/storage/innobase/trx/trx0trx.c 2013-03-26 01:40:02 +0000
174+++ Percona-Server/storage/innobase/trx/trx0trx.c 2013-04-27 10:07:32 +0000
175@@ -135,6 +135,7 @@
176 n_max * sizeof(trx_id_t));
177
178 trx_sys->descr_n_max = n_max;
179+ srv_descriptors_memory = n_max * sizeof(trx_id_t);
180 }
181
182 descr = trx_sys->descriptors + n_used - 1;
183
184=== modified file 'build/percona-server.spec'
185--- build/percona-server.spec 2013-04-23 06:02:56 +0000
186+++ build/percona-server.spec 2013-04-27 10:07:32 +0000
187@@ -725,6 +725,17 @@
188
189 if [ $1 -eq 1 ]; then
190 # ----------------------------------------------------------------------
191+# Create a MySQL user and group. Do not report any problems if it already
192+# exists.
193+# ----------------------------------------------------------------------
194+groupadd -r %{mysqld_group} 2> /dev/null || true
195+useradd -M -r -d $mysql_datadir -s /bin/bash -c "MySQL server" \
196+ -g %{mysqld_group} %{mysqld_user} 2> /dev/null || true
197+# The user may already exist, make sure it has the proper group nevertheless
198+# (BUG#12823)
199+usermod -g %{mysqld_group} %{mysqld_user} 2> /dev/null || true
200+
201+# ----------------------------------------------------------------------
202 # Create data directory if needed, check whether upgrade or install
203 # ----------------------------------------------------------------------
204 if [ ! -d $mysql_datadir ] ; then mkdir -m 755 $mysql_datadir; fi
205@@ -760,17 +771,6 @@
206 fi
207
208 # ----------------------------------------------------------------------
209-# Create a MySQL user and group. Do not report any problems if it already
210-# exists.
211-# ----------------------------------------------------------------------
212-groupadd -r %{mysqld_group} 2> /dev/null || true
213-useradd -M -r -d $mysql_datadir -s /bin/bash -c "MySQL server" \
214- -g %{mysqld_group} %{mysqld_user} 2> /dev/null || true
215-# The user may already exist, make sure it has the proper group nevertheless
216-# (BUG#12823)
217-usermod -g %{mysqld_group} %{mysqld_user} 2> /dev/null || true
218-
219-# ----------------------------------------------------------------------
220 # Initiate databases if needed
221 # ----------------------------------------------------------------------
222 # ----------------------------------------------------------------------

Subscribers

People subscribed via source and target branches