Merge lp:~chris.macnaughton/charms/trusty/kibana/version_bump into lp:charms/trusty/kibana

Proposed by Chris MacNaughton on 2015-11-04
Status: Merged
Merged at revision: 19
Proposed branch: lp:~chris.macnaughton/charms/trusty/kibana/version_bump
Merge into: lp:charms/trusty/kibana
Diff against target: 484 lines (+163/-125)
17 files modified
README.md (+2/-2)
config.yaml (+3/-3)
files/charm/config.js (+0/-75)
files/charm/config.yml (+78/-0)
files/charm/kibana.init (+32/-0)
files/charm/kibana_lb.conf (+12/-0)
files/charm/nginx.conf (+3/-33)
files/charm/nginx_lb.conf (+1/-1)
hooks/config-changed (+2/-1)
hooks/install (+13/-5)
hooks/rest-relation-departed (+1/-0)
hooks/rest-relation-joined (+1/-0)
hooks/start (+1/-0)
hooks/stop (+1/-0)
metadata.yaml (+3/-1)
tests/11-scale-elastic.py (+3/-3)
tests/12-port-change.py (+7/-1)
To merge this branch: bzr merge lp:~chris.macnaughton/charms/trusty/kibana/version_bump
Reviewer Review Type Date Requested Status
charmers 2015-11-04 Pending
Andrew McLeod 2015-11-04 Pending
Review via email: mp+276709@code.launchpad.net

This proposal supersedes a proposal from 2015-10-09.

Commit message

Update Kibana to Kibana 4

Description of the change

Updates Kibana to Kibana 4 and actually gets running

To post a comment you must log in.
Andrew McLeod (admcleod) wrote : Posted in a previous version of this proposal

Hi Chris,

Nice work on the merge there, but I've run into some problems with bundletester

bundletester was run from the charm root as follows:

bundletester -e local -l DEBUG -vF

bundletester error log:

http://pastebin.ubuntu.com/12716770/

Kibana log entry for first bundletester error:

http://pastebin.ubuntu.com/12716784/

review: Needs Fixing
Andrew McLeod (admcleod) wrote : Posted in a previous version of this proposal

Hi Chris,

I've re-run bundletester on this in AWS and it all looks good until test 11 - the deployment is OK, I can access kibana GUI, however test 11 then tries to access kibana on port 9200 and times out:

Test 11:

url = 'http://%s:9200/_all/_search' % kibana_unit.info['public-address']
(diff comment line 412)

Test 12:

Also I set -F for bundletester so it continued with the tests - the next kibana service is deployed as kibana/2, but test 12 is looking for kibana/0:

kibana_unit = d.sentry.unit['kibana/0']

Perhaps you could use d.sentry['kibana'][0] to get the first kibana unit deployed?

bundletester output: http://pastebin.ubuntu.com/12894780/

review: Needs Fixing
Andrew McLeod (admcleod) wrote : Posted in a previous version of this proposal

Looks good, tests are passing now, Approve.

review: Approve
Charles Butler (lazypower) wrote : Posted in a previous version of this proposal

Nice work on this one Icey! +1

Thanks for the contribution. Approved and merged.

review: Approve
Michael Nelson (michael.nelson) wrote :

I think you'll find that the kibana_lb.conf below actually proxies *any* request for 9200 from the kibana unit through to the ES unit. That is, anyone who can access the kibana unit on port 80, is able to POST/DELETE whatever they like in ES.

I've fixed this, with a bunch of other smaller changes here:

https://code.launchpad.net/~canonical-is-sa/charms/trusty/kibana/kibana4/+merge/276944

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'README.md'
2--- README.md 2015-03-31 16:03:49 +0000
3+++ README.md 2015-11-04 21:51:00 +0000
4@@ -19,7 +19,7 @@
5 juju deploy logstash-indexer
6 juju add-relation logstash-indexer elasticsearch:cluster
7 juju deploy kibana
8- juju add-relation kibana elasticsearch:rest
9+ juju add-relation kibana elasticsearch
10 juju expose kibana
11
12 browse to http://ip-address to begin searching.
13@@ -31,7 +31,7 @@
14 configured values in 'kibana_source' and 'kibana_source_checksum', which
15 default to::
16
17- https://download.elasticsearch.org/kibana/kibana/kibana-3.1.1.tar.gz
18+ https://download.elastic.co/kibana/kibana/kibana-4.1.2-linux-x64.tar.gz
19
20 It will, obviously, require internet access on deployed nodes.
21
22
23=== modified file 'config.yaml'
24--- config.yaml 2015-04-01 20:07:40 +0000
25+++ config.yaml 2015-11-04 21:51:00 +0000
26@@ -1,15 +1,15 @@
27 options:
28 kibana_source:
29 type: string
30- default: 'https://download.elasticsearch.org/kibana/kibana/kibana-3.1.1.tar.gz'
31+ default: 'https://download.elastic.co/kibana/kibana/kibana-4.1.2-linux-x64.tar.gz'
32 description: URL to download the source for kibana
33 kibana_source_unpack_directory:
34 type: string
35- default: 'kibana-3.1.1'
36+ default: 'kibana-4.1.2-linux-x64'
37 description: the name of the directory the tar.gz will unpack to
38 kibana_source_checksum:
39 type: string
40- default: '74c6b8b79d1bfc7e4d5376e3c9ecb0fa1e79c986'
41+ default: '45e67114f7dac4ccac8118bf98ee8f6362c7a6a1'
42 description: the sha1sum of the kibana_source
43 port:
44 type: int
45
46=== removed file 'files/charm/config.js'
47--- files/charm/config.js 2014-04-14 15:59:48 +0000
48+++ files/charm/config.js 1970-01-01 00:00:00 +0000
49@@ -1,75 +0,0 @@
50-/** @scratch /configuration/config.js/1
51- *
52- * == Configuration
53- * config.js is where you will find the core Kibana configuration. This file contains parameter that
54- * must be set before kibana is run for the first time.
55- */
56-define(['settings'],
57-function (Settings) {
58-
59-
60- /** @scratch /configuration/config.js/2
61- *
62- * === Parameters
63- */
64- return new Settings({
65-
66- /** @scratch /configuration/config.js/5
67- *
68- * ==== elasticsearch
69- *
70- * The URL to your elasticsearch server. You almost certainly don't
71- * want +http://localhost:9200+ here. Even if Kibana and Elasticsearch are on
72- * the same host. By default this will attempt to reach ES at the same host you have
73- * kibana installed on. You probably want to set it to the FQDN of your
74- * elasticsearch host
75- */
76- elasticsearch: "http://"+window.location.hostname,
77-
78- /** @scratch /configuration/config.js/5
79- *
80- * ==== default_route
81- *
82- * This is the default landing page when you don't specify a dashboard to load. You can specify
83- * files, scripts or saved dashboards here. For example, if you had saved a dashboard called
84- * `WebLogs' to elasticsearch you might use:
85- *
86- * default_route: '/dashboard/elasticsearch/WebLogs',
87- */
88- default_route : '/dashboard/file/default.json',
89-
90- /** @scratch /configuration/config.js/5
91- *
92- * ==== kibana-int
93- *
94- * The default ES index to use for storing Kibana specific object
95- * such as stored dashboards
96- */
97- kibana_index: "kibana-int",
98-
99- /** @scratch /configuration/config.js/5
100- *
101- * ==== panel_name
102- *
103- * An array of panel modules available. Panels will only be loaded when they are defined in the
104- * dashboard, but this list is used in the "add panel" interface.
105- */
106- panel_names: [
107- 'histogram',
108- 'map',
109- 'goal',
110- 'table',
111- 'filtering',
112- 'timepicker',
113- 'text',
114- 'hits',
115- 'column',
116- 'trends',
117- 'bettermap',
118- 'query',
119- 'terms',
120- 'stats',
121- 'sparklines'
122- ]
123- });
124-});
125\ No newline at end of file
126
127=== added file 'files/charm/config.yml'
128--- files/charm/config.yml 1970-01-01 00:00:00 +0000
129+++ files/charm/config.yml 2015-11-04 21:51:00 +0000
130@@ -0,0 +1,78 @@
131+# Kibana is served by a back end server. This controls which port to use.
132+port: 5601
133+
134+# The host to bind the server to.
135+host: "0.0.0.0"
136+
137+# The Elasticsearch instance to use for all your queries.
138+elasticsearch_url: "http://localhost:9200"
139+
140+# preserve_elasticsearch_host true will send the hostname specified in `elasticsearch`. If you set it to false,
141+# then the host you use to connect to *this* Kibana instance will be sent.
142+elasticsearch_preserve_host: true
143+
144+# Kibana uses an index in Elasticsearch to store saved searches, visualizations
145+# and dashboards. It will create a new index if it doesn't already exist.
146+kibana_index: ".kibana"
147+
148+# If your Elasticsearch is protected with basic auth, this is the user credentials
149+# used by the Kibana server to perform maintence on the kibana_index at statup. Your Kibana
150+# users will still need to authenticate with Elasticsearch (which is proxied thorugh
151+# the Kibana server)
152+# kibana_elasticsearch_username: user
153+# kibana_elasticsearch_password: pass
154+
155+# If your Elasticsearch requires client certificate and key
156+# kibana_elasticsearch_client_crt: /path/to/your/client.crt
157+# kibana_elasticsearch_client_key: /path/to/your/client.key
158+
159+# If you need to provide a CA certificate for your Elasticsarech instance, put
160+# the path of the pem file here.
161+# ca: /path/to/your/CA.pem
162+
163+# The default application to load.
164+default_app_id: "discover"
165+
166+# Time in milliseconds to wait for elasticsearch to respond to pings, defaults to
167+# request_timeout setting
168+# ping_timeout: 1500
169+
170+# Time in milliseconds to wait for responses from the back end or elasticsearch.
171+# This must be > 0
172+request_timeout: 300000
173+
174+# Time in milliseconds for Elasticsearch to wait for responses from shards.
175+# Set to 0 to disable.
176+shard_timeout: 0
177+
178+# Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying
179+# startup_timeout: 5000
180+
181+# Set to false to have a complete disregard for the validity of the SSL
182+# certificate.
183+verify_ssl: true
184+
185+# SSL for outgoing requests from the Kibana Server (PEM formatted)
186+# ssl_key_file: /path/to/your/server.key
187+# ssl_cert_file: /path/to/your/server.crt
188+
189+# Set the path to where you would like the process id file to be created.
190+# pid_file: /var/run/kibana.pid
191+
192+# If you would like to send the log output to a file you can set the path below.
193+# This will also turn off the STDOUT log output.
194+# log_file: ./kibana.log
195+
196+# Plugins that are included in the build, and no longer found in the plugins/ folder
197+bundled_plugin_ids:
198+ - plugins/dashboard/index
199+ - plugins/discover/index
200+ - plugins/doc/index
201+ - plugins/kibana/index
202+ - plugins/markdown_vis/index
203+ - plugins/metric_vis/index
204+ - plugins/settings/index
205+ - plugins/table_vis/index
206+ - plugins/vis_types/index
207+ - plugins/visualize/index
208+
209
210=== added file 'files/charm/kibana.init'
211--- files/charm/kibana.init 1970-01-01 00:00:00 +0000
212+++ files/charm/kibana.init 2015-11-04 21:51:00 +0000
213@@ -0,0 +1,32 @@
214+# Kibana 4
215+#
216+description "Kibana 4"
217+
218+start on virtual-filesystems
219+stop on runlevel [06]
220+
221+# Respawn it if the process exits
222+respawn
223+
224+# We're setting high here, we'll re-limit below.
225+limit nofile 65550 65550
226+
227+setuid kibana
228+setgid kibana
229+
230+console log
231+script
232+ # Defaults - Please set KB_HOME
233+ PATH=/bin:/usr/bin:$PATH
234+ KB_HOME=/srv/kibana4
235+ KB_LOG_FILE=/var/log/kibana/kibana.log
236+ KB_NICE=19
237+
238+ HOME="${HOME:-$KB_HOME}"
239+
240+ # Export variables
241+ export PATH HOME
242+ test -n "${JAVACMD}" && export JAVACMD
243+
244+ exec nice -n ${KB_NICE} ${KB_HOME}/bin/kibana 2>&1 ${KB_LOG_FILE}
245+end script
246
247=== added file 'files/charm/kibana_lb.conf'
248--- files/charm/kibana_lb.conf 1970-01-01 00:00:00 +0000
249+++ files/charm/kibana_lb.conf 2015-11-04 21:51:00 +0000
250@@ -0,0 +1,12 @@
251+upstream kibana {
252+ server localhost:5601;
253+}
254+
255+server {
256+ listen *:9200;
257+
258+ location / {
259+ proxy_pass http://es_cluster;
260+ proxy_read_timeout 90;
261+ }
262+}
263\ No newline at end of file
264
265=== modified file 'files/charm/nginx.conf'
266--- files/charm/nginx.conf 2014-04-14 15:59:48 +0000
267+++ files/charm/nginx.conf 2015-11-04 21:51:00 +0000
268@@ -5,39 +5,9 @@
269 access_log /var/log/nginx/kibana.access.log;
270
271 location / {
272- root /srv/kibana3;
273- index index.html index.htm;
274- }
275-
276- location ~ ^/_aliases$ {
277- proxy_pass http://es_cluster;
278- proxy_read_timeout 90;
279- }
280- location ~ ^/.*/_aliases$ {
281- proxy_pass http://es_cluster;
282- proxy_read_timeout 90;
283- }
284- location ~ ^/_nodes$ {
285- proxy_pass http://es_cluster;
286- proxy_read_timeout 90;
287- }
288- location ~ ^/.*/_search$ {
289- proxy_pass http://es_cluster;
290- proxy_read_timeout 90;
291- }
292- location ~ ^/.*/_mapping$ {
293- proxy_pass http://es_cluster;
294- proxy_read_timeout 90;
295- }
296-
297- location ~ ^/kibana-int/dashboard/.*$ {
298- proxy_pass http://es_cluster;
299- proxy_read_timeout 90;
300- }
301- location ~ ^/kibana-int/temp.*$ {
302- proxy_pass http://es_cluster;
303- proxy_read_timeout 90;
304- }
305+ proxy_pass http://kibana;
306+ proxy_read_timeout 90;
307+ }
308 }
309
310
311
312=== modified file 'files/charm/nginx_lb.conf'
313--- files/charm/nginx_lb.conf 2013-09-29 21:58:50 +0000
314+++ files/charm/nginx_lb.conf 2015-11-04 21:51:00 +0000
315@@ -1,3 +1,3 @@
316 upstream es_cluster {
317 server localhost:9200;
318-}
319\ No newline at end of file
320+}
321
322=== modified file 'hooks/config-changed' (properties changed: -x to +x)
323--- hooks/config-changed 2015-06-18 20:10:31 +0000
324+++ hooks/config-changed 2015-11-04 21:51:00 +0000
325@@ -6,9 +6,10 @@
326
327 sed -i "s/listen.*/listen *:$PORT;/" /etc/nginx/sites-available/kibana
328
329-sed -i "s@elasticsearch: .*@elasticsearch: '//'+window.location.hostname+'$KIBANA_PATH',@" /srv/kibana3/config.js
330+# sed -i "s@elasticsearch: .*@elasticsearch: '//'+window.location.hostname+'$KIBANA_PATH',@" /srv/kibana4/config.js
331
332 service nginx reload
333+service kibana restart
334
335 [[ -r .PREVOPENPORT ]] && PREVOPENPORT=$(cat .PREVOPENPORT) || PREVOPENPORT=''
336 [[ -z "$PREVOPENPORT" ]] || close-port $PREVOPENPORT/tcp
337
338=== modified file 'hooks/install'
339--- hooks/install 2015-06-18 18:49:22 +0000
340+++ hooks/install 2015-11-04 21:51:00 +0000
341@@ -21,26 +21,34 @@
342 cp $KIBANA_TARBALL /tmp/kibana.tgz
343 fi
344
345+groupadd kibana
346+useradd -s /bin/false -g kibana kibana
347+mkdir -p /var/log/kibana
348+chown kibana:kibana /var/log/kibana
349+
350 install -o root -g root -m 0644 files/charm/nginx.conf /etc/nginx/sites-available/kibana
351 install -o root -g root -m 0644 files/charm/nginx_lb.conf /etc/nginx/sites-available/es_cluster
352-
353+install -o root -g root -m 0644 files/charm/kibana_lb.conf /etc/nginx/sites-available/kibana_load_balancer
354+install -o root -g root -m 0644 files/charm/kibana.init /etc/init/kibana.conf
355 pushd /srv
356
357-if [[ ! -d kibana3 ]]; then
358+if [[ ! -d kibana4 ]]; then
359 tar xzvf /tmp/kibana.tgz
360- mv ${KIBANA_SOURCE_UNPACK_DIRECTORY} kibana3
361+ mv ${KIBANA_SOURCE_UNPACK_DIRECTORY} kibana4
362 fi
363
364 rm -f /etc/nginx/sites-enabled/default
365
366 popd
367
368-rm -f /srv/kibana3/config.js
369-install -o root -g root -m 0644 files/charm/config.js /srv/kibana3/config.js
370+# rm -f /srv/kibana4/config.js
371+# install -o root -g root -m 0644 files/charm/config.js /srv/kibana4/config.js
372
373 hooks/config-changed
374
375 ln -s /etc/nginx/sites-available/kibana /etc/nginx/sites-enabled/kibana
376 ln -s /etc/nginx/sites-available/es_cluster /etc/nginx/sites-enabled/es_cluster
377+ln -s /etc/nginx/sites-available/kibana_load_balancer /etc/nginx/sites-enabled/kibana_load_balancer
378
379 service nginx restart
380+service kibana restart
381\ No newline at end of file
382
383=== modified file 'hooks/rest-relation-departed'
384--- hooks/rest-relation-departed 2013-11-26 08:51:58 +0000
385+++ hooks/rest-relation-departed 2015-11-04 21:51:00 +0000
386@@ -1,2 +1,3 @@
387 #!/bin/sh
388 juju-log $JUJU_REMOTE_UNIT departed
389+close-port 9200/tcp
390
391=== modified file 'hooks/rest-relation-joined'
392--- hooks/rest-relation-joined 2013-11-26 08:51:58 +0000
393+++ hooks/rest-relation-joined 2015-11-04 21:51:00 +0000
394@@ -1,2 +1,3 @@
395 #!/bin/bash
396 juju-log $JUJU_REMOTE_UNIT joined
397+open-port 9200/tcp
398
399=== modified file 'hooks/start' (properties changed: -x to +x)
400--- hooks/start 2015-06-18 18:49:22 +0000
401+++ hooks/start 2015-11-04 21:51:00 +0000
402@@ -4,3 +4,4 @@
403 # i.e. 'service apache2 start'
404
405 service nginx start
406+service kibana restart
407\ No newline at end of file
408
409=== modified file 'hooks/stop'
410--- hooks/stop 2015-06-18 20:10:31 +0000
411+++ hooks/stop 2015-11-04 21:51:00 +0000
412@@ -7,5 +7,6 @@
413 # rm /srv/webroot/server-live.txt && sleep 30
414
415 service nginx stop
416+service kibana stop
417 [[ -r .PREVOPENPORT ]] && PREVOPENPORT=$(cat .PREVOPENPORT) || PREVOPENPORT=''
418 [[ -z "$PREVOPENPORT" ]] || close-port $PREVOPENPORT/tcp
419
420=== modified file 'metadata.yaml'
421--- metadata.yaml 2015-04-08 20:21:47 +0000
422+++ metadata.yaml 2015-11-04 21:51:00 +0000
423@@ -1,6 +1,8 @@
424 name: kibana
425 summary: alternative (better) logstash frontend
426-maintainer: Paul Czarkowski <paul@paulcz.net>
427+maintainers:
428+ - Paul Czarkowski <paul@paulcz.net>
429+ - Chris MacNaughton <chris.macnaughton@canonical.com>
430 description: |
431 alternative (better) logstash frontend
432 tags:
433
434=== modified file 'tests/11-scale-elastic.py'
435--- tests/11-scale-elastic.py 2015-03-12 18:48:08 +0000
436+++ tests/11-scale-elastic.py 2015-11-04 21:51:00 +0000
437@@ -33,8 +33,8 @@
438 ##
439 # Set relationship aliases
440 ##
441-kibana_unit = d.sentry.unit['kibana/0']
442-elasticsearch_unit = d.sentry.unit['elasticsearch/1']
443+kibana_unit = d.sentry['kibana'][0]
444+elasticsearch_unit = d.sentry['elasticsearch'][1]
445
446
447 def test_web_interface():
448@@ -50,7 +50,7 @@
449
450 time.sleep(3)
451
452- url = 'http://%s/_all/_search' % kibana_unit.info['public-address']
453+ url = 'http://%s:9200/_all/_search' % kibana_unit.info['public-address']
454 test_dir = os.path.dirname(__file__)
455 test_data_file = os.path.join(test_dir, '11-scale-elastic-query.json')
456 with open(test_data_file) as fp:
457
458=== modified file 'tests/12-port-change.py'
459--- tests/12-port-change.py 2015-06-18 17:11:44 +0000
460+++ tests/12-port-change.py 2015-11-04 21:51:00 +0000
461@@ -8,8 +8,14 @@
462 d = amulet.Deployment(series='trusty')
463
464 d.add('kibana')
465+d.add('elasticsearch')
466+
467 d.configure('kibana', {'port': 81})
468
469+d.add_unit('elasticsearch', 2)
470+
471+d.relate('kibana:rest', 'elasticsearch:client')
472+
473 d.expose('kibana')
474
475 try:
476@@ -21,7 +27,7 @@
477
478
479 def test_web_interface():
480- kibana_unit = d.sentry.unit['kibana/0']
481+ kibana_unit = d.sentry['kibana'][0]
482 url = 'http://%s:81/' % kibana_unit.info['public-address']
483 r = requests.get(url)
484 if not r.ok:

Subscribers

People subscribed via source and target branches