Merge lp:~nicopace/charms/trusty/kibana/all-tests into lp:charms/trusty/kibana

[nicopace]

Merged kibana tests in one merge:
* basic test
* scale elastic

[amir sanjar (asanjar)]

2015-03-03 12:30:51 Starting deployment of local
2015-03-03 12:30:54 Deploying services...
2015-03-03 12:30:54 Deploying service elasticsearch using cs:trusty/elasticsearch-8
2015-03-03 12:31:03 Deploying service logstash-indexer using cs:precise/logstash-indexer-4
Traceback (most recent call last):
  File "/usr/bin/juju-deployer", line 9, in <module>
    load_entry_point('juju-deployer==0.4.3', 'console_scripts', 'juju-deployer')()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 130, in main
    run()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 228, in run
    importer.Importer(env, deployment, options).run()
  File "/usr/lib/python2.7/dist-packages/deployer/action/importer.py", line 202, in run
    self.add_units()
  File "/usr/lib/python2.7/dist-packages/deployer/action/importer.py", line 22, in add_units
    env_status = self.env.status()
  File "/usr/lib/python2.7/dist-packages/deployer/env/go.py", line 203, in status
    return self.client.get_stat()
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 599, in get_stat
    return StatusTranslator().run(watch)
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 999, in run
    self._unit(d)
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 1031, in _unit
    for p in ports:
TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/home/sanjar/development/charms/trusty/kibana/tests/11-scale-elastic.py", line 25, in <module>
    d.setup(timeout=seconds)
  File "/usr/lib/python3/dist-packages/amulet/deployer.py", line 315, in setup
    ], cwd=self.deployer_dir)
  File "/usr/lib/python3.4/subprocess.py", line 557, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['juju-deployer', '-W', '-L', '-c', '/tmp/amulet-juju-deployer-6ikpb2xh.json', '-e', 'local', '-t', '20100', 'local']' returned non-zero exit status 1

Shouldn't cs:precise/logstash-indexer be replaced by cs:trusty/logstash-indexer

review: Needs Fixing

[nicopace (nicopace)]

> Shouldn't cs:precise/logstash-indexer be replaced by cs:trusty/logstash-
> indexer

I didn't used a trusty version because there is no such version.

[nicopace (nicopace)]

@asanjar bump

[Kevin W Monroe (kwmonroe)]

Amir's failure was caused by juju:

https://bugs.launchpad.net/juju-core/+bug/1425435

And nicopace is correct, there's not a trusty version of logstash-indexer yet.. Using the precise charm for the test should be okay.

[Cory Johns (johnsca)]

With the work-around of `pip install -U juju-deployer`, I still had failures on the 11-scale-elastic.py test due to:

  * Bad path to 11-scale-elastic-query.json file
  * Invalid JSON in said query file (True instead of true)
  * Param error in requests.get() call (data instead of params)

I created a MP with fixes for these that allow all of the tests to pass for me: https://code.launchpad.net/~johnsca/charms/trusty/kibana/all-tests/+merge/252805

With my changes, I would give this a +1.

[nicopace (nicopace)]

Merge done :)

[Tim Van Steenburgh (tvansteenburgh)]

Nico, are you sure you merged Cory's changes into your branch? I don't see new commits, and the test still fails as of this morning (http://reports.vapour.ws/charm-test-details/charm-bundle-test-parent-158).

[nicopace (nicopace)]

@tvansteenburgh i have merged corys changes... i thought launchpad was going to do it by itself, but had to do it manually.

=== added file 'tests/11-scale-elastic-query.json'
--- tests/11-scale-elastic-query.json	1970-01-01 00:00:00 +0000
+++ tests/11-scale-elastic-query.json	2015-02-27 18:42:58 +0000
@@ -0,0 +1,49 @@
+{
+  "query": {
+    "filtered": {
+      "query": {
+        "bool": {
+          "should": [
+            {
+            "query_string": {
+              "query": "*"
+            }
+          }
+          ]
+        }
+      },
+      "filter": {
+        "bool": {
+          "must": [
+            {
+            "terms": {
+              "_type": [
+                "1"
+              ]
+            }
+          }
+          ]
+        }
+      }
+    }
+  },
+  "highlight": {
+    "fields": {},
+    "fragment_size": 2147483647,
+    "pre_tags": [
+      "@start-highlight@"
+    ],
+    "post_tags": [
+      "@end-highlight@"
+    ]
+  },
+  "size": 500,
+  "sort": [
+    {
+    "_score": {
+      "order": "desc",
+      "ignore_unmapped": True
+    }
+  }
+  ]
+}

=== added file 'tests/11-scale-elastic.py'
--- tests/11-scale-elastic.py	1970-01-01 00:00:00 +0000
+++ tests/11-scale-elastic.py	2015-02-27 18:42:58 +0000
@@ -0,0 +1,71 @@
+#!/usr/bin/python3
+import sys
+import time
+import json
+
+import amulet
+import requests
+
+seconds = 20000
+
+d = amulet.Deployment(series='trusty')
+
+d.add('kibana')
+d.add('elasticsearch')
+d.add('logstash-indexer', charm='cs:precise/logstash-indexer')
+
+d.add_unit('elasticsearch', 2)
+
+d.relate('logstash-indexer:cluster', 'elasticsearch:client')
+d.relate('kibana:rest', 'elasticsearch:client')
+
+d.expose('kibana')
+
+try:
+    d.setup(timeout=seconds)
+except amulet.helpers.TimeoutError:
+    amulet.raise_status(amulet.SKIP, msg="Environment wasn't stood up in time")
+except:
+    raise
+
+
+##
+# Set relationship aliases
+##
+kibana_unit = d.sentry.unit['kibana/0']
+elasticsearch_unit = d.sentry.unit['elasticsearch/1']
+
+
+def test_web_interface():
+
+    elasticsearch_unit.run("""
+        curl -X POST 'http://127.0.0.1:9200/person/1' -d '{
+        "info" : {
+                "height" : 2,
+                "width" : 20
+        }
+        }'
+    """)
+
+    time.sleep(3)
+
+    url = 'http://%s/_all/_search' % kibana_unit.info['public-address']
+    r = requests.get(url,
+                     data=json.load(open('11-scale-elastic-query.json')))
+    if not r.ok:
+        amulet.raise_status(amulet.FAIL,
+                            msg="Error connecting.")
+    try:
+        result = r.json()
+        if result['hits']['total'] != 1:
+            amulet.raise_status(amulet.FAIL,
+                                msg="Error inserting value in elasticsearch.")
+    except ValueError:
+        exc_type, value, traceback = sys.exc_info()
+        exc_text = "Exception: (%s) %s. %s" % (value, exc_type, traceback)
+        amulet.raise_status(amulet.FAIL,
+                            msg="Value returned not a valid json." + r.text +
+                            exc_text)
+
+
+test_web_interface()