Merge lp:~nicopace/charms/trusty/nagios/all-tests into lp:charms/trusty/nagios

[nicopace]

Integration of all nagios tests implemented:
* initial test
* nrpe test
* ssl test
* monitors interface test
* extraconfig test

[amir sanjar (asanjar)]

+1 all tests passed
(.venv)sanjar@sanjar-acer:~/development/charms$ bundletester -t nagios
nagios
    charm-proof PASS
    00-setup PASS
    10-initial-test PASS
    100-deploy PASS
    15-nrpe-test PASS
    20-ssl-test PASS
    21-monitors-interface-test PASS
    22-extraconfig-test PASS

PASS: 8 Total: 8 (0 sec)

review: Approve

[amir sanjar (asanjar)]

obviously bundletester itself is buggy, rerunning the test as follow
(.venv)sanjar@sanjar-acer:~/development/charms/trusty/nagios$ bundletester -v

[amir sanjar (asanjar)]

Test failed..
(.venv)sanjar@sanjar-acer:~/development/charms/trusty/nagios$ bundletester -v
nagios
    charm-proof PASS
[sudo] password for sanjar:
    00-setup PASS
    10-initial-test PASS
    100-deploy PASS
    15-nrpe-test ERROR

------------------------------------------------------------------------------
ERROR: nagios::15-nrpe-test
[/home/sanjar/development/charms/trusty/nagios/tests/15-nrpe-test exit 1]
2015-02-27 14:46:10 Starting deployment of local
2015-02-27 14:46:12 Deploying services...
2015-02-27 14:46:18 Adding relations...
2015-02-27 14:46:19 Adding relation nrpe:general-info <-> mediawiki:juju-info
Traceback (most recent call last):
  File "/usr/bin/juju-deployer", line 9, in <module>
    load_entry_point('juju-deployer==0.4.3', 'console_scripts', 'juju-deployer')()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 130, in main
    run()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 228, in run
    importer.Importer(env, deployment, options).run()
  File "/usr/lib/python2.7/dist-packages/deployer/action/importer.py", line 215, in run
    rels_created = self.add_relations()
  File "/usr/lib/python2.7/dist-packages/deployer/action/importer.py", line 131, in add_relations
    self.env.add_relation(end_a, end_b)
  File "/usr/lib/python2.7/dist-packages/deployer/env/go.py", line 34, in add_relation
    return self.client.add_relation(endpoint_a, endpoint_b)
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 666, in add_relation
    'Endpoints': [endpoint_a, endpoint_b]
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 148, in _rpc
    raise EnvError(result)
jujuclient.EnvError: <Env Error - Details:
 { u'Error': u'cannot add relation "nrpe:general-info mediawiki:juju-info": principal and subordinate services\' series must match',
    u'RequestId': 1,
    u'Response': { }}
 >
Traceback (most recent call last):
  File "/home/sanjar/development/charms/trusty/nagios/tests/15-nrpe-test", line 25, in <module>
    d.setup(timeout=seconds)
  File "/usr/lib/python3/dist-packages/amulet/deployer.py", line 315, in setup
    ], cwd=self.deployer_dir)
  File "/usr/lib/python3.4/subprocess.py", line 557, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['juju-deployer', '-W', '-L', '-c', '/tmp/amulet-juju-deployer-0_jxjpv4.json', '-e', 'local', '-t', '20100', 'local']' returned non-zero exit status 1

review: Disapprove

[Cory Johns (johnsca)]

Nicolas,

I think Amir's error was due to checking out the branch directly instead of merging it against the upstream. It seems that the missing default series was fixed in the upstream already. With that fix, however, there are still the following issues with the tests:

  * The first call to requests in test_web_interface_with_ssl needs to be https:// instead of http://

  * Both calls to requests in test_web_interface_with_ssl need to include verify=False to skip certificate validation

  * The tests need to call d.sentry.wait() immediately after d.setup() to ensure that the environment stabilizes before running the tests

review: Needs Fixing

[nicopace (nicopace)]

> obviously bundletester itself is buggy, rerunning the test as follow
> (.venv)sanjar@sanjar-acer:~/development/charms/trusty/nagios$ bundletester -v

Why do you say that bundletester is buggy?
It is the tool i've been using all these 3 months for executing the tests.

[lp:~nicopace/charms/trusty/nagios/all-tests]

32. By Nicolas Pace <email address hidden> on 2015-02-27

Get branch updates.

33. By Nicolas Pace <email address hidden> on 2015-02-27

Adds d.sentry.wait before setup, to stabilize the environment.

[nicopace (nicopace)]

> Nicolas,
>
> I think Amir's error was due to checking out the branch directly instead of
> merging it against the upstream. It seems that the missing default series was
> fixed in the upstream already. With that fix, however, there are still the
> following issues with the tests:
>
> * The first call to requests in test_web_interface_with_ssl needs to be
> https:// instead of http://

The first option tests the 'on' option. That means that nagios should provide http and https (that's why the first request is without ssl, and the second with.

>
> * Both calls to requests in test_web_interface_with_ssl need to include
> verify=False to skip certificate validation

The first one doesn't use ssl, so it is not required.

>
> * The tests need to call d.sentry.wait() immediately after d.setup() to
> ensure that the environment stabilizes before running the tests

I've updated the test to reflect this.

[Cory Johns (johnsca)]

> The first option tests the 'on' option. That means that nagios should provide
> http and https (that's why the first request is without ssl, and the second
> with.

Ah, fair enough. I think we saw a connection error on the plain http: request at one point, so assume that SSL on meant only SSL, but it makes much more sense that it would enable both and that the test should cover that. It's likely that the connection error we saw was actually due to the missing sentry.wait() calls, so I'll go ahead and re-run those tests to see if that resolved the issue.

[nicopace (nicopace)]

Bump

[nicopace (nicopace)]

So... @johnsca, is there any other conflict that is preventing this branch to be merged?

[Cory Johns (johnsca)]

Sorry for the delay. The 21-monitors-interface-test and 22-extraconfig-test tests are still missing d.sentry.wait(), but it turns out that the failure I was seeing was from the 21-monitors-interface-test test attempting to relate mysql and mediawiki when it doesn't deploy mediawiki.

review: Needs Fixing

[Cory Johns (johnsca)]

I created a MP against this branch with the fixes from my previous comment. With that applied, I would give this a +1.

https://code.launchpad.net/~johnsca/charms/trusty/nagios/all-tests/+merge/252786

review: Approve

[lp:~nicopace/charms/trusty/nagios/all-tests]

34. By Nicolas Pace <email address hidden> on 2015-03-18

Cory changes

[nicopace (nicopace)]

Merged cory's code

=== modified file 'tests/20-ssl-test'
--- tests/20-ssl-test	2015-02-03 19:48:51 +0000
+++ tests/20-ssl-test	2015-03-18 00:31:39 +0000
@@ -21,6 +21,7 @@
 
 try:
     d.setup(timeout=seconds)
+    d.sentry.wait()
 except amulet.helpers.TimeoutError:
     amulet.raise_status(amulet.SKIP, msg="Environment wasn't stood up in time")
 except:

=== added file 'tests/21-monitors-interface-test'
--- tests/21-monitors-interface-test	1970-01-01 00:00:00 +0000
+++ tests/21-monitors-interface-test	2015-03-18 00:31:39 +0000
@@ -0,0 +1,62 @@
+#!/usr/bin/python3
+
+import amulet
+import requests
+
+seconds = 20000
+
+d = amulet.Deployment(series='trusty')
+
+d.add('nagios')
+d.add('mysql')
+d.add('nrpe')
+d.add('mediawiki')
+
+d.relate('mysql:db', 'mediawiki:db')
+d.relate('nagios:monitors', 'mysql:monitors')
+d.relate('nrpe:general-info', 'mysql:juju-info')
+d.relate('nrpe:monitors', 'nagios:monitors')
+d.relate('nrpe:local-monitors', 'mysql:local-monitors')
+
+d.expose('nagios')
+
+try:
+    d.setup(timeout=seconds)
+    d.sentry.wait()
+except amulet.helpers.TimeoutError:
+    amulet.raise_status(amulet.SKIP, msg="Environment wasn't stood up in time")
+except:
+    raise
+
+
+##
+# Set relationship aliases
+##
+mysql_unit = d.sentry.unit['mysql/0']
+nagios_unit = d.sentry.unit['nagios/0']
+
+
+def test_nrpe_monitors_config():
+    # look for procrunning in nrpe config
+    try:
+        mysql_unit.file_contents('/etc/nagios/nrpe.d/procrunning_mysqld.cfg')
+    except IOError as e:
+        amulet.raise_status(amulet.ERROR,
+                            msg="procrunning config not found. Error:" +
+                            e.args[1])
+
+
+def test_nagios_monitors_response():
+    # look for mysql_database requests
+    nagpwd = nagios_unit.file_contents('/var/lib/juju/nagios.passwd').strip()
+    host_url = ("http://%s/cgi-bin/nagios3/status.cgi?"
+                "host=mysql-0")
+    r = requests.get(host_url % nagios_unit.info['public-address'],
+                     auth=('nagiosadmin', nagpwd))
+    if not r.text.find('mysql-0-basic'):
+        amulet.raise_status(amulet.ERROR,
+                            msg='Nagios is not monitoring the' +
+                            ' hosts it supposed to.')
+
+test_nrpe_monitors_config()
+test_nagios_monitors_response()

=== added file 'tests/22-extraconfig-test'
--- tests/22-extraconfig-test	1970-01-01 00:00:00 +0000
+++ tests/22-extraconfig-test	2015-03-18 00:31:39 +0000
@@ -0,0 +1,56 @@
+#!/usr/bin/python3
+
+import amulet
+import requests
+
+seconds = 20000
+
+d = amulet.Deployment(series='trusty')
+
+d.add('nagios')
+d.add('mysql')
+
+d.expose('nagios')
+d.expose('mysql')
+
+try:
+    d.setup(timeout=seconds)
+    d.sentry.wait()
+except amulet.helpers.TimeoutError:
+    amulet.raise_status(amulet.SKIP, msg="Environment wasn't stood up in time")
+except:
+    raise
+
+
+##
+# Set relationship aliases
+##
+mysql_unit = d.sentry.unit['mysql/0']
+nagios_unit = d.sentry.unit['nagios/0']
+
+d.configure('nagios', {
+    'extraconfig': """
+    define host{
+    use                     generic-host  ; Name of host template to use
+    host_name               server02
+    alias                   Server 02
+    address                 %s
+}""" % mysql_unit.info['public-address']
+})
+
+d.sentry.wait()
+
+
+def test_hosts_being_monitored():
+    nagpwd = nagios_unit.file_contents('/var/lib/juju/nagios.passwd').strip()
+    host_url = ("http://%s/cgi-bin/nagios3/status.cgi?"
+                "hostgroup=all&style=hostdetail")
+    r = requests.get(host_url % nagios_unit.info['public-address'],
+                     auth=('nagiosadmin', nagpwd))
+    if not r.text.find('server02'):
+        amulet.raise_status(amulet.ERROR,
+                            msg='Nagios is not monitoring the' +
+                            ' hosts it supposed to.')
+
+
+test_hosts_being_monitored()