Merge lp:~benji/juju-gui/sandboxify-tests into lp:juju-gui/experimental

[Benji York]

Add the ability to run local browser tests.

Some existing tests were moved from charm-requiring tests to local browser
tests.

https://codereview.appspot.com/40190043/

[Benji York (benji)]

Reviewers: mp+198410_code.launchpad.net,

Message:
Pre-review comments.

https://codereview.appspot.com/40190043/diff/1/Makefile
File Makefile (right):

https://codereview.appspot.com/40190043/diff/1/Makefile#newcode408
Makefile:408: # of this target when called by lbox. Please do not
change.
Typo fix, I think.

https://codereview.appspot.com/40190043/diff/1/Makefile#newcode445
Makefile:445: PYTHONPATH=lib virtualenv/bin/python
test/test_websocketreplay.py
We were using the system Python here.

https://codereview.appspot.com/40190043/diff/1/Makefile#newcode458
Makefile:458: # Wait for the display to be accessible.
Avoid racing with Xvfb to start up.

https://codereview.appspot.com/40190043/diff/1/test/browser.py
File test/browser.py (left):

https://codereview.appspot.com/40190043/diff/1/test/browser.py#oldcode214
test/browser.py:214: printerr('* Platform: local
{}'.format(get_platform(driver)))
This was annoying and non-useful.

https://codereview.appspot.com/40190043/diff/1/test/test_browser.py
File test/test_browser.py (right):

https://codereview.appspot.com/40190043/diff/1/test/test_browser.py#newcode1
test/test_browser.py:1: # This file is part of the Juju GUI, which lets
users view and manage Juju
These tests were moved here, unchanged.

https://codereview.appspot.com/40190043/diff/1/test/test_charm_running.py
File test/test_charm_running.py (left):

https://codereview.appspot.com/40190043/diff/1/test/test_charm_running.py#oldcode25
test/test_charm_running.py:25: class TestBasics(browser.TestCase):
These were moved elsewhere.

https://codereview.appspot.com/40190043/diff/1/test/test_charm_running.py#oldcode277
test/test_charm_running.py:277: class
TestAuthentication(browser.TestCase):
Moved.

https://codereview.appspot.com/40190043/diff/1/test/test_charm_running.py
File test/test_charm_running.py (right):

https://codereview.appspot.com/40190043/diff/1/test/test_charm_running.py#newcode192
test/test_charm_running.py:192: self.deploy('mysql')
This change made the tests start working when the GUI is not run from a
charm (as in the sandbox).

Description:
Add the ability to run local browser tests.

Some existing tests were moved from charm-requiring tests to local
browser
tests.

https://code.launchpad.net/~benji/juju-gui/sandboxify-tests/+merge/198410

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/40190043/

Affected files (+131, -74 lines):
   M HACKING
   M Makefile
   A [revision details]
   M test/browser.py
   A test/test_browser.py
   M test/test_browser_errors.py
   M test/test_charm_running.py
   M test/test_deploy_charm_for_testing.py

Unmerged revisions

1224. By Benji York on 2013-12-10

words

1223. By Benji York on 2013-12-10

reinstate tests

1222. By Benji York on 2013-12-10

abandon hope of starting the server automatically

1221. By Benji York on 2013-12-09

checkpoint

1220. By Benji York on 2013-12-06

merge from trunk

1219. By Benji York on 2013-12-06

checkpoint

=== modified file 'HACKING'
--- HACKING	2013-09-24 15:38:01 +0000
+++ HACKING	2013-12-10 14:54:36 +0000
@@ -60,7 +60,7 @@
   sudo apt-get update
   sudo apt-get install nodejs imagemagick python-sphinx python-yaml \
     python-tz python-virtualenv python-shelltoolbox python-selenium \
-    python-tornado python-gflags g++
+    python-tornado python-gflags g++ xvfb
 
 See :ref:`Browser Testing <browser-testing>` if you are curious about the
 reason for ``python-shelltoolbox`` and ``python-selenium``.

=== modified file 'Makefile'
--- Makefile	2013-10-31 19:12:42 +0000
+++ Makefile	2013-12-10 14:54:36 +0000
@@ -405,7 +405,7 @@
 prep: beautify lint
 
 # XXX bac: the order of test-debug and test-prod seems to affect the execution
-# of this target when called by lbox.  Please do note change.
+# of this target when called by lbox.  Please do not change.
 check: lint test-debug test-prod test-misc docs
 
 test/extracted_startup_code: app/index.html
@@ -426,6 +426,7 @@
 test-prep: test/test_startup.js
 
 test-filtering:
+	# Ensure no tests are disabled.
 	./bin/test-filtering
 
 test-debug: build-debug test-prep
@@ -441,10 +442,34 @@
 	./test-server.sh prod true
 
 test-misc:
-	PYTHONPATH=lib python test/test_websocketreplay.py
-	PYTHONPATH=lib python test/test_browser.py
-	PYTHONPATH=lib python test/test_deploy_charm_for_testing.py
-	PYTHONPATH=bin python test/test_http_server.py
+	PYTHONPATH=lib virtualenv/bin/python test/test_websocketreplay.py
+	PYTHONPATH=lib virtualenv/bin/python test/test_browser.py
+	PYTHONPATH=lib virtualenv/bin/python \
+	    test/test_deploy_charm_for_testing.py
+	PYTHONPATH=bin virtualenv/bin/python test/test_http_server.py
+
+test-browser: build-devel
+	# Tests that run in the browser.  A server has to be running on
+	# localhost:8888 for this to work.
+	# Make sure no display :34 exists before we start one.
+	DISPLAY=:34 xdpyinfo > /dev/null || exit 1
+	# Start Xvfb as a background process, capturing its PID.
+	$(eval xvfb_pid := $(shell Xvfb :34 2> /dev/null & echo $$!))
+	# Wait for the display to be accessible.
+	until (DISPLAY=:34 xdpyinfo > /dev/null); \
+	    do \
+	        echo "Waiting for Xvfb"; \
+	        sleep 1; \
+	    done
+	# Run the tests inside the virtual frame buffer, capturing the result
+	# of the test run.
+	$(eval result := $(shell \
+	    DISPLAY=:34 virtualenv/bin/python test/test_browser.py; \
+	    echo $$?))
+	# Stop the background processes.
+	kill $(xvfb_pid)
+	# Report the result of the test run.
+	exit $(result)
 
 test:
 	@echo "Deprecated. Please run either 'make test-prod' or 'make"

=== modified file 'test/browser.py'
--- test/browser.py	2013-09-19 14:31:53 +0000
+++ test/browser.py	2013-12-10 14:54:36 +0000
@@ -211,7 +211,6 @@
             capabilities = get_capabilities(name)
             driver = make_local_driver(name, capabilities)
             cls.remote_driver = False
-            printerr('* Platform: local {}'.format(get_platform(driver)))
         else:
             # Otherwise, set up a Saucelabs remote driver.
             capabilities = get_capabilities(browser_name)
@@ -232,7 +231,7 @@
         driver.implicitly_wait(20)
         driver.set_script_timeout(30)
         # We want to tell saucelabs when all the tests are done.
-        cls.app_url = os.environ['APP_URL']
+        cls.app_url = os.environ.get('APP_URL', 'http://localhost:8888')
         cls.driver = driver
 
     @classmethod

=== added file 'test/test_browser.py'
--- test/test_browser.py	1970-01-01 00:00:00 +0000
+++ test/test_browser.py	2013-12-10 14:54:36 +0000
@@ -0,0 +1,95 @@
+# This file is part of the Juju GUI, which lets users view and manage Juju
+# environments within a graphical interface (https://launchpad.net/juju-gui).
+# Copyright (C) 2012-2013 Canonical Ltd.
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License version 3, as published by
+# the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranties of MERCHANTABILITY,
+# SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from __future__ import print_function
+import unittest
+
+from selenium.common import exceptions
+
+import browser
+
+
+class TestBasics(browser.TestCase):
+
+    def test_title(self):
+        self.load()
+        self.assertTrue('Juju Admin' in self.driver.title)
+
+    def test_environment_name(self):
+        self.load()
+        self.handle_browser_warning()
+        # The next line attempts to work around an IE 10 fragility.
+        # The symptom we are trying to avoid is an error as follows:
+        # "JavaScript error (WARNING: The server did not provide any stacktrace
+        # information)""
+        self.wait_for_css_selector('svg')
+        body = self.driver.find_element_by_xpath('//body')
+        self.assertTrue('Environment on ' in body.text)
+
+    def test_environment_connection(self):
+        # The GUI connects to the API backend.
+        self.load()
+        self.handle_browser_warning()
+        # The next line attempts to work around an IE 10 fragility.
+        # The symptom we are trying to avoid is an error as follows:
+        # "JavaScript error (WARNING: The server did not provide any stacktrace
+        # information)""
+        self.wait_for_css_selector('svg')
+        script = 'return app && app.env && app.env.get("connected");'
+        self.wait_for_script(script, 'Environment not connected.')
+
+
+class TestAuthentication(browser.TestCase):
+
+    def is_authenticated(self):
+        """Return True if the user is authenticated, False otherwise."""
+        script = 'return app.env.userIsAuthenticated;'
+        return self.driver.execute_script(script)
+
+    def process_path(self, path):
+        """Load the given path, log out, log in again."""
+        self.load(path)
+        self.handle_browser_warning()
+        self.handle_login()
+        # Check the initial URL.
+        self.wait_for_path(path, error='Not in the initial path.')
+        self.assertTrue(self.is_authenticated(), 'initial state')
+        # Logout.
+        self.logout()
+        # Check redirection to /login/.
+        self.wait_for_path('/login/', error='Redirection to /login/ failed.')
+        self.assertFalse(self.is_authenticated(), 'after logging out')
+        # Login.
+        self.login()
+        # Ensure we are in the initial URL again.
+        self.wait_for_path(path, error='Post login redirection failed.')
+        self.assertTrue(self.is_authenticated(), 'after logging in again')
+
+    def test_root_page(self):
+        # It is possible to login to and logout from the root page.
+        self.process_path('/')
+
+    def test_service_page(self):
+        # It is possible to login to and logout from the service detail view.
+        self.process_path('/:gui:/service/haproxy/')
+
+    def test_unit_page(self):
+        # It is possible to login to and logout from the unit detail view.
+        self.process_path('/:gui:/unit/haproxy-0/')
+
+if __name__ == '__main__':
+    browser.browser_name = 'local-firefox'
+    unittest.main()

=== renamed file 'test/test_browser.py' => 'test/test_browser_errors.py'
=== modified file 'test/test_charm_running.py'
--- test/test_charm_running.py	2013-10-29 22:30:57 +0000
+++ test/test_charm_running.py	2013-12-10 14:54:36 +0000
@@ -22,34 +22,7 @@
 import browser
 
 
-class TestBasics(browser.TestCase):
-
-    def test_title(self):
-        self.load()
-        self.assertTrue('Juju Admin' in self.driver.title)
-
-    def test_environment_name(self):
-        self.load()
-        self.handle_browser_warning()
-        # The next line attempts to work around an IE 10 fragility.
-        # The symptom we are trying to avoid is an error as follows:
-        # "JavaScript error (WARNING: The server did not provide any stacktrace
-        # information)""
-        self.wait_for_css_selector('svg')
-        body = self.driver.find_element_by_xpath('//body')
-        self.assertTrue('Environment on ' in body.text)
-
-    def test_environment_connection(self):
-        # The GUI connects to the API backend.
-        self.load()
-        self.handle_browser_warning()
-        # The next line attempts to work around an IE 10 fragility.
-        # The symptom we are trying to avoid is an error as follows:
-        # "JavaScript error (WARNING: The server did not provide any stacktrace
-        # information)""
-        self.wait_for_css_selector('svg')
-        script = 'return app && app.env && app.env.get("connected");'
-        self.wait_for_script(script, 'Environment not connected.')
+class TestTests(browser.TestCase):
 
     def test_gui_unit_tests(self):
         # Ensure Juju GUI unit tests pass.
@@ -216,6 +189,7 @@
         # The service name is arbitrary, and connected to the charm name only
         # by default/convention. Since charms are deployed using the default
         # name, it is safe to reuse one of the service names here.
+        self.deploy('mysql')
         service = self.get_service_names().pop()
         self.deploy(service)
         notifications = self.get_notifications()
@@ -274,45 +248,6 @@
         self.assertEqual('haproxy/0', unit_name)
 
 
-class TestAuthentication(browser.TestCase):
-
-    def is_authenticated(self):
-        """Return True if the user is authenticated, False otherwise."""
-        script = 'return app.env.userIsAuthenticated;'
-        return self.driver.execute_script(script)
-
-    def process_path(self, path):
-        """Load the given path, log out, log in again."""
-        self.load(path)
-        self.handle_browser_warning()
-        self.handle_login()
-        # Check the initial URL.
-        self.wait_for_path(path, error='Not in the initial path.')
-        self.assertTrue(self.is_authenticated(), 'initial state')
-        # Logout.
-        self.logout()
-        # Check redirection to /login/.
-        self.wait_for_path('/login/', error='Redirection to /login/ failed.')
-        self.assertFalse(self.is_authenticated(), 'after logging out')
-        # Login.
-        self.login()
-        # Ensure we are in the initial URL again.
-        self.wait_for_path(path, error='Post login redirection failed.')
-        self.assertTrue(self.is_authenticated(), 'after logging in again')
-
-    def test_root_page(self):
-        # It is possible to login to and logout from the root page.
-        self.process_path('/')
-
-    def test_service_page(self):
-        # It is possible to login to and logout from the service detail view.
-        self.process_path('/:gui:/service/haproxy/')
-
-    def test_unit_page(self):
-        # It is possible to login to and logout from the unit detail view.
-        self.process_path('/:gui:/unit/haproxy-0/')
-
-
 class TestSandbox(browser.TestCase, DeployTestMixin):
 
     @classmethod

=== modified file 'test/test_deploy_charm_for_testing.py'
--- test/test_deploy_charm_for_testing.py	2013-08-06 13:21:03 +0000
+++ test/test_deploy_charm_for_testing.py	2013-12-10 14:54:36 +0000
@@ -54,6 +54,7 @@
 
         def sleep(seconds):
             raise ZZZZ
+
         with self.assertRaises(ZZZZ):
             wait_for_service(get_service_state, sleep)