Multiple hash index partitions causes overly large hash index

Bug #1018264 reported by Raghavendra D Prabhu
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Fix Released
High
George Ormond Lorch III
5.1
Invalid
High
Unassigned
5.5
Fix Released
High
George Ormond Lorch III

Bug Description

Normally, adaptive hash index is initialized to 1/64 th of the buffer pool size.

in buf_pool_init:

 btr_search_sys_create(buf_pool_get_curr_size() / sizeof(void*) / 64);

However, innodb_adaptive_hash_index_partitions allows multiple adaptive hash index partitions to be created.

This leads to each partition being created with size of 1/64 of the buffer pool instead of 1/(64*number_of_partitions) and can cause overly high memory usage.

For instance, for a buffer pool of size 192G, without partitions it will be 3 GB but if the number of partitions is say, 8, it will be 24 G and will scale with that.

What is expected:
        The size of each partition should be decided after taking into account the number of buffer pool instances, ie.

in btr_search_sys_create:
                                    hash_size /= srv_buf_pool_instances

..otherwise each partition will be created with size of an otherwise single partition.

Related branches

Revision history for this message
Raghavendra D Prabhu (raghavendra-prabhu) wrote :

On a second thought, instead of

                                     hash_size /= srv_buf_pool_instances

 hash_size /= btr_search_index_num may be better since that is more directly related to the adaptive hash index.

Revision history for this message
George Ormond Lorch III (gl-az) wrote :

5.1 does not support multiple adaptive hash search partitions so bug as described does not apply to 5.1

Revision history for this message
SingerWang (wang) wrote :

I think each partition being of 1/(64*number_of_partitions) make sense..

Revision history for this message
George Ormond Lorch III (gl-az) wrote : Re: [Bug 1018264] Re: Multiple hash index partitions causes overly large hash index

Yes, that seems to be the appropriate way to allocate starting partition
sizes. A fix for PS 5.5 has been made and is currently being tested.

On 9/26/2012 11:28 AM, SingerWang wrote:
> I think each partition being of 1/(64*number_of_partitions) make sense..
>

--
George O. Lorch III
Software Engineer, Percona Inc.
+1-888-401-3401 x542 US/Arizona (GMT -7)
skype: george.ormond.lorch.iii

Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-568

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.