Merge lp:~michael.nelson/charms/trusty/elasticsearch/heap-config into lp:~onlineservices-charmers/charms/trusty/elasticsearch/elasticsearch2

Proposed by Michael Nelson
Status: Merged
Merged at revision: 42
Proposed branch: lp:~michael.nelson/charms/trusty/elasticsearch/heap-config
Merge into: lp:~onlineservices-charmers/charms/trusty/elasticsearch/elasticsearch2
Diff against target: 79 lines (+43/-0)
5 files modified
config.yaml (+7/-0)
playbook.yaml (+16/-0)
templates/elasticsearch.yml (+3/-0)
templates/etc_default.j2 (+14/-0)
templates/limits.conf (+3/-0)
To merge this branch: bzr merge lp:~michael.nelson/charms/trusty/elasticsearch/heap-config
Reviewer Review Type Date Requested Status
Wes Mason (community) Approve
Bret Barker Approve
Review via email: mp+277788@code.launchpad.net

Commit message

Add heap_size config option, defaulting to min(0.5*available_mem, 31g)

Description of the change

I've used http://stackoverflow.com/a/18152957/523729 as a basis. Testing it I can confirm locally that the heap size is updated in ES, using:

curl -s http://localhost:9200/_nodes/stats | python -m json.tool | grep heap_max_in_bytes

What I can't confirm is the memory lock - and that's either simply because I'm testing in an lxc container :P, or additionally because /etc/security/limits.d/ aren't applied to non-interactive users (ie. system jobs - I see this reported too).

I'll test on staging before landing. EDIT: Just tested on staging and confirm that setting the heap-size works as expected:

https://pastebin.canonical.com/144362/

Also, top shows a similar resident memory value too.

To post a comment you must log in.
Revision history for this message
Bret Barker (noise) wrote :

minor comment - with the conditional below, you can't set a heap of e.g. 1.4g and can't specify it as 1400m either. I know the recommended heap is 1/2 total ram, but for a 2g system it fails pretty miserably with any kind of load unless you do 1300-1500m heap.

Revision history for this message
Bret Barker (noise) wrote :

+1

review: Approve
Revision history for this message
Wes Mason (wesmason) wrote :

lgtm

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'config.yaml'
2--- config.yaml 2015-10-01 19:00:31 +0000
3+++ config.yaml 2015-11-18 20:40:41 +0000
4@@ -27,3 +27,10 @@
5 By default, the admin and peer ports (9200 and 9300) are only accessible
6 to clients and peers respectively. Switch this to false to enable access
7 from any machine.
8+ heap_size:
9+ type: string
10+ default: ""
11+ description: |
12+ By default, the heap size will be set to the maximum of half of the
13+ available memory and 30.5G. A number in gigabytes for the Elasticsearch
14+ heap size.
15
16=== modified file 'playbook.yaml'
17--- playbook.yaml 2015-10-27 22:23:15 +0000
18+++ playbook.yaml 2015-11-18 20:40:41 +0000
19@@ -37,6 +37,22 @@
20 notify:
21 - Restart ElasticSearch
22
23+ - name: Update defaults
24+ tags:
25+ - config-changed
26+ template: src={{ charm_dir }}/templates/etc_default.j2
27+ dest=/etc/default/elasticsearch
28+ notify:
29+ - Restart ElasticSearch
30+
31+ - name: Set Elasticsearch user limits
32+ tags:
33+ - upgrade-charm
34+ template: src={{ charm_dir }}/templates/limits.conf
35+ dest=/etc/security/limits.d/elasticsearch.conf
36+ notify:
37+ - Restart ElasticSearch
38+
39 - name: Open ES Port when exposed
40 command: open-port 9200
41 tags:
42
43=== modified file 'templates/elasticsearch.yml'
44--- templates/elasticsearch.yml 2015-10-30 03:32:29 +0000
45+++ templates/elasticsearch.yml 2015-11-18 20:40:41 +0000
46@@ -13,3 +13,6 @@
47 {% for reln in relations.peer %}
48 - {{ reln['private-address'] }}
49 {% endfor %}
50+
51+# https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html
52+bootstrap.mlockall: true
53
54=== added file 'templates/etc_default.j2'
55--- templates/etc_default.j2 1970-01-01 00:00:00 +0000
56+++ templates/etc_default.j2 2015-11-18 20:40:41 +0000
57@@ -0,0 +1,14 @@
58+# See https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html
59+{% set calculated_heap_size = ansible_memtotal_mb / 1024.0 / 2.0 %}
60+{# min/max filters only available in ansible 1.8 #}
61+{% if calculated_heap_size >= 31.5 %}{% set calculated_heap_size = 31 %}{% endif %}
62+{% set heap_size = heap_size | default(calculated_heap_size, true) | float %}
63+# Note: ES doesn't start if the heap size is not an int of either gigabytes (g)
64+# or megabytes (m)
65+{% if heap_size > 4 %}
66+ES_HEAP_SIZE={{ heap_size | int }}g
67+{% else %}
68+ES_HEAP_SIZE={{ (heap_size * 1024) | int }}m
69+{% endif %}
70+MAX_OPEN_FILES=65535
71+MAX_LOCKED_MEMORY=unlimited
72
73=== added file 'templates/limits.conf'
74--- templates/limits.conf 1970-01-01 00:00:00 +0000
75+++ templates/limits.conf 2015-11-18 20:40:41 +0000
76@@ -0,0 +1,3 @@
77+# Ensure ES user can run with max open files and unlimited memlock.
78+elasticsearch - nofile 65535
79+elasticsearch - memlock unlimited

Subscribers

People subscribed via source and target branches