Merge lp:~sinzui/launchpad/auto-yui-tests into lp:launchpad

[Curtis Hovey]

This is my branch to automatically test YUI unit tests in windmill.
The scope of this branch is only for the registry. This solution is not
comprehensive for launchpad, nor does it solve hard problems like
testing in iframes (a requirement for testing timelines and help)

    lp:~sinzui/launchpad/auto-yui-tests
    Diff size: 186
    Launchpad bug: None, this is to unblock the product branch.js feature.
    Test command: ./bin/test -vv -t test_yuitests
    Pre-implementation: gary, sidnei
    Target release: 10.04

Automatically add YUI unit tests to the test runner
---------------------------------------------------

The automated js test runner needs a harness to run the YUI unit test in
canonical/launchpad/javascript/*/test.

The following conditions must be met for a test to be automatically played
by windmill.
    * It must be lib/canonical/launchpad/javascript/registry/tests
    * The test js inject a node with id="complete" when complete.
    * The html file must match test_*.html
    * The html cannot test iframes
      * To accommodate iframes, the YUI test would need to inject the results
        in the parent page. This is work that does not solve the immediate
        issue.

Rules
-----

    The harness needs to:
    * Locate each html file like */javascript/test/test_*.html
    * Open it in the browser, wait for the log to populate
    * Verify there are no fail nodes in the log
    * Report failing notes as tests.

QA
--

None for the apps. This is successful if it correctly reports failures.

Lint
----

Linting changed files:
  configs/testrunner-appserver/yui-unittest.zcml
  lib/canonical/launchpad/javascript/registry/tests/test_milestone_table.js
  lib/lp/registry/windmill/tests/test_yuitests.py
  lib/lp/testing/__init__.py
  lib/lp/testing/views.py

Test
----

    * lib/canonical/launchpad/javascript/registry/tests/test_milestone_table.js
      * Added a method that provides a clear indication to windmill that the
        test is complete.
    * lib/lp/registry/windmill/tests/test_yuitests.py
      * Added a test harness to locate test_*.html in the registry YUI
        directory. Note that I renamed test_milestone_table.html too.
        The test waits for the signal that the YUI test runner is complete,
        then gets the page content, extracts the messages, and calls fail
        if there was a YUI failure.

Implementation
--------------

    * configs/testrunner-appserver/yui-unittest.zcml
      * Register a view that can serve the html pages through the webapp.
        This is a requirement because windmill cannot load a file from the
        filesystem--it is a js security prohibition. This view is only
        available to the test runner.
    * lib/lp/testing/__init__.py
      * Removed a duplicate class. This looks like a paste error.
    * lib/lp/testing/views.py
      * Create a view to retrieve the yui files that need to play in the
        windmill test.

[Brad Crittenden (bac)]

Curtis this branch looks good. As we discussed on IRC it will need to be modified to allow at least lp.code in addition to lp.registry to satisfy the immediate use case.

I look forward to trying it out.

review: Approve (code)

=== added file 'configs/testrunner-appserver/yui-unittest.zcml'
--- configs/testrunner-appserver/yui-unittest.zcml	1970-01-01 00:00:00 +0000
+++ configs/testrunner-appserver/yui-unittest.zcml	2010-04-02 14:27:39 +0000
@@ -0,0 +1,16 @@
+<!-- Copyright 2010 Canonical Ltd.  This software is licensed under the
+     GNU Affero General Public License version 3 (see the file LICENSE).
+-->
+
+<configure
+    xmlns="http://namespaces.zope.org/zope"
+    xmlns:browser="http://namespaces.zope.org/browser">
+
+    <browser:page
+        name="+yui-unittest"
+        for="canonical.launchpad.webapp.interfaces.ILaunchpadRoot"
+        class="lp.testing.views.YUITestFileView"
+        attribute="__call__"
+        permission="zope.Public"/>
+
+</configure>

=== renamed file 'lib/canonical/launchpad/javascript/registry/tests/milestone_table.html' => 'lib/canonical/launchpad/javascript/registry/tests/test_milestone_table.html'
=== modified file 'lib/canonical/launchpad/javascript/registry/tests/test_milestone_table.js'
--- lib/canonical/launchpad/javascript/registry/tests/test_milestone_table.js	2010-02-11 20:10:01 +0000
+++ lib/canonical/launchpad/javascript/registry/tests/test_milestone_table.js	2010-04-02 14:27:39 +0000
@@ -244,6 +244,12 @@
         }));
 
     // Lock, stock, and two smoking barrels.
+    var handle_complete = function(data) {
+        status_node = Y.Node.create(
+            '<p id="complete">Test status: complete</p>');
+        Y.get('body').appendChild(status_node);
+        };
+    Y.Test.Runner.on('complete', handle_complete);
     Y.Test.Runner.add(suite);
 
     var console = new Y.Console({newestOnTop: false});

=== renamed file 'lib/canonical/launchpad/javascript/registry/tests/timeline.html' => 'lib/canonical/launchpad/javascript/registry/tests/test_timeline.html'
=== modified file 'lib/canonical/launchpad/javascript/registry/tests/timeline.js'
--- lib/canonical/launchpad/javascript/registry/tests/timeline.js	2010-02-11 20:10:01 +0000
+++ lib/canonical/launchpad/javascript/registry/tests/timeline.js	2010-04-02 14:27:39 +0000
@@ -263,6 +263,18 @@
 
 
 Y.Test.Runner.add(suite);
+var handle_complete = function(data) {
+    var parent_log = Y.Node.create(
+        '<div id="log" style="display: none">' +
+        Y.get('#log').get('innerHTML') +  '</div>');
+    var parent_body = Y.one(parent.document).get('body');
+    parent_body.appendChild(parent_log);
+    var status_node = Y.Node.create(
+        '<p id="complete">Test status: complete</p>');
+    parent_body.appendChild(status_node);
+    };
+Y.Test.Runner.on('complete', handle_complete);
+
 
 var yconsole = new Y.Console({
     newestOnTop: false

=== added file 'lib/lp/registry/windmill/tests/test_yuitests.py'
--- lib/lp/registry/windmill/tests/test_yuitests.py	1970-01-01 00:00:00 +0000
+++ lib/lp/registry/windmill/tests/test_yuitests.py	2010-04-02 14:27:39 +0000
@@ -0,0 +1,21 @@
+# Copyright 2010 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Run YUI.test tests."""
+
+__metaclass__ = type
+__all__ = []
+
+from lp.testing import build_yui_unittest_suite, YUIUnitTestCase
+from lp.registry.windmill.testing import RegistryWindmillLayer
+
+
+class RegistryYUIUnitTestCase(YUIUnitTestCase):
+
+    layer = RegistryWindmillLayer
+    suite_name = 'RegistryYUIUnitTests'
+
+
+def test_suite():
+    app_testing_path = 'canonical/launchpad/javascript/registry/tests'
+    return build_yui_unittest_suite(app_testing_path, RegistryYUIUnitTestCase)

=== modified file 'lib/lp/testing/__init__.py'
--- lib/lp/testing/__init__.py	2010-03-18 10:05:01 +0000
+++ lib/lp/testing/__init__.py	2010-04-02 14:27:39 +0000
@@ -6,6 +6,7 @@
 __metaclass__ = type
 __all__ = [
     'ANONYMOUS',
+    'build_yui_unittest_suite',
     'capture_events',
     'FakeTime',
     'get_lsb_information',
@@ -29,6 +30,7 @@
     'validate_mock_class',
     'WindmillTestCase',
     'with_anonymous_login',
+    'YUIUnitTestCase',
     'ZopeTestInSubProcess',
     ]
 
@@ -43,6 +45,7 @@
 import sys
 import tempfile
 import time
+import unittest
 
 from bzrlib.branch import Branch as BzrBranch
 from bzrlib.bzrdir import BzrDir, format_registry
@@ -70,6 +73,7 @@
 from canonical.launchpad.webapp import errorlog
 from canonical.config import config
 from canonical.launchpad.webapp.interfaces import ILaunchBag
+from canonical.launchpad.windmill.testing import constants
 from lp.codehosting.vfs import branch_id_to_path, get_multi_server
 # Import the login and logout functions here as it is a much better
 # place to import them from in tests.
@@ -568,25 +572,65 @@
         self.client.open(url=u'http://launchpad.dev:8085')
 
 
-class WindmillTestCase(TestCaseWithFactory):
-    """A TestCase class for Windmill tests.
-
-    It provides a WindmillTestClient (self.client) with Launchpad's front
-    page loaded.
-    """
-
+class YUIUnitTestCase(WindmillTestCase):
+
+    layer = None
     suite_name = ''
 
+    _yui_results = None
+    _view_name = u'http://launchpad.dev:8085/+yui-unittest/'
+
+    def initialize(self, test_path):
+        self.test_path = test_path
+        self.yui_runner_url = self._view_name + test_path
+
     def setUp(self):
-        TestCaseWithFactory.setUp(self)
-        self.client = WindmillTestClient(self.suite_name)
-        # Load the front page to make sure we don't get fooled by stale pages
-        # left by the previous test. (For some reason, when you create a new
-        # WindmillTestClient you get a new session and everything, but if you
-        # do anything before you open() something you'd be operating on the
-        # page that was last accessed by the previous test, which is the cause
-        # of things like https://launchpad.net/bugs/515494)
-        self.client.open(url=u'http://launchpad.dev:8085')
+        super(YUIUnitTestCase, self).setUp()
+        client = self.client
+        client.open(url=self.yui_runner_url)
+        client.waits.forPageLoad(timeout=constants.PAGE_LOAD)
+        client.waits.forElement(id='complete')
+        response = client.commands.getPageText()
+        self._yui_results = {}
+        # Maybe testing.pages should move to lp to avoid circular imports.
+        from canonical.launchpad.testing.pages import find_tags_by_class
+        entries = find_tags_by_class(
+            response['result'], 'yui-console-entry-TestRunner')
+        for entry in entries:
+            category = entry.find(
+                attrs={'class': 'yui-console-entry-cat'})
+            if category is None:
+                continue
+            result = category.string
+            if result not in ('pass', 'fail'):
+                continue
+            message = entry.pre.string
+            test_name, ignore = message.split(':', 1)
+            self._yui_results[test_name] = dict(
+                result=result, message=message)
+
+    def runTest(self):
+        if self._yui_results is None or len(self._yui_results) == 0:
+            self.fail("Test harness or js failed.")
+        for test_name in self._yui_results:
+            result = self._yui_results[test_name]
+            self.assertTrue('pass' == result['result'],
+                    'Failure in %s.%s: %s' % (
+                        self.test_path, test_name, result['message']))
+
+
+def build_yui_unittest_suite(app_testing_path, yui_test_class):
+    suite = unittest.TestSuite()
+    testing_path = os.path.join(config.root, 'lib', app_testing_path)
+    unit_test_names = [
+        file_name for file_name in os.listdir(testing_path)
+        if file_name.startswith('test_') and file_name.endswith('.html')]
+    for unit_test_name in unit_test_names:
+        test_path = os.path.join(app_testing_path, unit_test_name)
+        test_case = yui_test_class()
+        test_case.initialize(test_path)
+        suite.addTest(test_case)
+    return suite
 
 
 class ZopeTestInSubProcess:

=== modified file 'lib/lp/testing/views.py'
--- lib/lp/testing/views.py	2009-12-14 22:00:41 +0000
+++ lib/lp/testing/views.py	2010-04-02 14:27:39 +0000
@@ -7,12 +7,16 @@
 __all__ = [
     'create_view',
     'create_initialized_view',
+    'YUITestFileView',
     ]
 
+import os
 
 from zope.component import getUtility, getMultiAdapter
 from zope.security.management import endInteraction, newInteraction
 
+from canonical.config import config
+from canonical.lazr import ExportedFolder
 from canonical.launchpad.layers import setFirstLayer
 from canonical.launchpad.webapp.interfaces import IPlacelessAuthUtility
 from canonical.launchpad.webapp.servers import LaunchpadTestRequest
@@ -72,3 +76,10 @@
         query_string, cookie, request, path_info)
     view.initialize()
     return view
+
+
+class YUITestFileView(ExportedFolder):
+    """Export the lib directory where the test assets reside."""
+
+    folder = os.path.join(config.root, 'lib/')
+    export_subdirectories = True