Merge lp:~mbruzek/charms/precise/newrelic-php/tests into lp:~charmers/charms/precise/newrelic-php/trunk

[Matt Bruzek]

Adding tests for this charm

[Review Queue (review-queue)]

This items has failed automated testing! Results available here http://reports.vapour.ws/charm-tests/charm-bundle-test-10628-results

review: Needs Fixing (automated testing)

[lp:~mbruzek/charms/precise/newrelic-php/tests]

15. By Matt Bruzek on 2014-12-19

Adding newrelic key to test.

[Matt Bruzek (mbruzek)]

After adding the key the tests passed bundletester on my local machine.

[Review Queue (review-queue)]

This items has failed automated testing! Results available here http://reports.vapour.ws/charm-tests/charm-bundle-test-10829-results

review: Needs Fixing (automated testing)

[Antonio Rosales (arosales)]

As this merge proposal only add charm tests and does not touch charm code I would like to suggest to accept this merge proposal even though the charm is failing the tests for the following reasons:

1. A test is still valid even if the results aren't positive.
2. Having charm tests provides the first step to helping the charm author and community fix the charm. Specifically by providing a reproducible failure.
3. The tests are seed (basic) tests that an interested person can build off of.
4. As fixes are made to this charm automating charm testing will be able to run additional tests in addition to charm proof.
5. Having basic tests, as demonstrated here, helps ensure the charm is in a working (deployable) state given this charm is a recommended charm. A test failure can prompt a bug the charm author can take action on. The charm author may not have been aware of the bug had the charm not had even basic deployment tests.

In this case the test does deploy a valid thinkup environment, and @mbruzek also added a key to ensure the setup was not an issue in the tests. However, automated testing is showing that relation between newrelic and wordpress is failing. The test correctly deploys the environment, but the relation needs fixing which is the intent of the tests.

Thus, for the above reason I would suggest to accept this merge to add in tests.

+1 LGTM

review: Approve (community)

[Antonio Rosales (arosales)]

Bug opened for charm failures:
https://bugs.launchpad.net/charms/+source/newrelic-php/+bug/1232922

[Charles Butler (lazypower)]

Confirmed that the test failure is due to the charm code and not the test. I'm going to approve this as is with the hope that a maintainer will step into fixing the

unit-newrelic-php-0[19254]: 2015-01-09 21:03:28 INFO unit.newrelic-php/0.start logger.go:40 newrelic-php5: unrecognized service

Thanks for your contributions mbruzek and jose!

review: Approve

=== added directory 'tests'
=== added file 'tests/00-setup'
--- tests/00-setup	1970-01-01 00:00:00 +0000
+++ tests/00-setup	2014-12-19 22:08:38 +0000
@@ -0,0 +1,5 @@
+#!/bin/bash
+
+sudo add-apt-repository ppa:juju/stable -y
+sudo apt-get update
+sudo apt-get install amulet python3-requests -y

=== added file 'tests/99-autogen'
--- tests/99-autogen	1970-01-01 00:00:00 +0000
+++ tests/99-autogen	2014-12-19 22:08:38 +0000
@@ -0,0 +1,52 @@
+#!/usr/bin/env python3
+
+import amulet
+import requests
+import unittest
+
+
+class TestDeployment(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.deployment = amulet.Deployment(series='precise')
+
+        cls.deployment.add('newrelic-php')
+        cls.deployment.add('wordpress')
+        cls.deployment.configure('newrelic-php', {'key': '2e2b4406be1e34fe64b228ca7dc16d4ce555add5'})
+        cls.deployment.relate('newrelic-php:juju-info', 'wordpress:juju-info')
+
+        try:
+            cls.deployment.setup(timeout=900)
+            cls.deployment.sentry.wait()
+        except amulet.helpers.TimeoutError:
+            amulet.raise_status(amulet.SKIP, msg="Environment wasn't stood up in time")
+        except:
+            raise
+
+    def test_case(self):
+        # Now you can use self.deployment.sentry.unit[UNIT] to address each of
+        # the units and perform more in-depth steps.  You can also reference
+        # the first unit as self.unit.
+        # There are three test statuses that can be triggered with
+        # amulet.raise_status():
+        #   - amulet.PASS
+        #   - amulet.FAIL
+        #   - amulet.SKIP
+        # Each unit has the following methods:
+        #   - .info - An array of the information of that unit from Juju
+        #   - .file(PATH) - Get the details of a file on that unit
+        #   - .file_contents(PATH) - Get plain text output of PATH file from that unit
+        #   - .directory(PATH) - Get details of directory
+        #   - .directory_contents(PATH) - List files and folders in PATH on that unit
+        #   - .relation(relation, service:rel) - Get relation data from return service
+        #          add tests here to confirm service is up and working properly
+        # For example, to confirm that it has a functioning HTTP server:
+        #     page = requests.get('http://{}'.format(self.unit.info['public-address']))
+        #     page.raise_for_status()
+        # More information on writing Amulet tests can be found at:
+        #     https://juju.ubuntu.com/docs/tools-amulet.html
+        pass
+
+
+if __name__ == '__main__':
+    unittest.main()