Merge lp:~lifeless/subunit/progress-gtk into lp:~subunit/subunit/trunk

[Robert Collins]

[Robert Collins (lifeless)]

This builds on progress-core to do a GTK progress bar. Woo. Shiny.

[lp:~lifeless/subunit/progress-gtk]

76. By Robert Collins on 2009-07-29

Handle python 2.5 a bit better in subunit2gtk.

77. By Robert Collins on 2009-07-29

Fix gtk support more.

78. By Robert Collins on 2009-07-30

Fix returning None from gobject IO callbacks.

Unmerged revisions

78. By Robert Collins on 2009-07-30

Fix returning None from gobject IO callbacks.

=== modified file 'NEWS'
--- NEWS	2009-07-22 09:39:00 +0000
+++ NEWS	2009-07-29 09:59:36 +0000
@@ -10,10 +10,26 @@
 
   IMPROVEMENTS:
 
+    * Subunit streams can now include optional, incremental lookahead
+      information about progress. This allows reporters to make estimates
+      about completion, when such information is available. See the README
+      under ``progress`` for more details.
+
+    * ``subunit2gtk`` has been added, a filter that shows a GTK summary of a
+      test stream.
+
+    * ``subunit2pyunit`` has a --progress flag which will cause the bzrlib
+      test reporter to be used, which has a textual progress bar. This requires
+      a recent bzrlib as a minor bugfix was required in bzrlib to support this.
+
   BUG FIXES:
 
   API CHANGES:
 
+    * When a progress: directive is encountered in a subunit stream, the
+      python bindings now call the ``progress(offset, whence)`` methd on
+      ``TestResult``.
+
     * When a time: directive is encountered in a subunit stream, the python
       bindings now call the ``time(seconds)`` method on ``TestResult``.
 

=== modified file 'README'
--- README	2009-07-22 09:39:00 +0000
+++ README	2009-07-29 09:59:36 +0000
@@ -47,6 +47,7 @@
 Subunit supplies the following filters:
  * tap2subunit - convert perl's TestAnythingProtocol to subunit.
  * subunit2pyunit - convert a subunit stream to pyunit test results.
+ * subunit2gtk - show a subunit stream in GTK.
  * subunit-filter - filter out tests from a subunit stream.
  * subunit-ls - list info about tests present in a subunit stream.
  * subunit-stats - generate a summary of a subunit stream.
@@ -130,12 +131,17 @@
  # needed and report to your result object.
  suite.run(result)
 
-subunit includes extensions to the python ``TestResult`` protocol. The
-``time(a_datetime)`` method is called (if present) when a ``time:``
+subunit includes extensions to the python ``TestResult`` protocol.
+
+The ``time(a_datetime)`` method is called (if present) when a ``time:``
 directive is encountered in a subunit stream. This is used to tell a TestResult
 about the time that events in the stream occured at, to allow reconstructing
 test timing from a stream.
 
+The ``progress(offset, whence)`` method controls progress data for a stream.
+The offset parameter is an int, and whence is one of subunit.SEEK_CUR,
+subunit.SEEK_SET.
+
 Finally, subunit.run is a convenience wrapper to run a python test suite via
 the command line, reporting via subunit::
 
@@ -216,10 +222,12 @@
 xfail[:] test label
 xfail[:] test label [
 ]
+progress: [+|-]X
 tags: [-]TAG ...
 time: YYYY-MM-DD HH:MM:SSZ
 unexpected output on stdout -> stdout.
-exit w/0 or last test -> error
+exit w/0 or last test completing -> error
+
 Tags given outside a test are applied to all following tests
 Tags given after a test: line and before the result line for the same test
 apply only to that test, and inheric the current global tags.
@@ -228,7 +236,19 @@
 In Python, tags are assigned to the .tags attribute on the RemoteTest objects
 created by the TestProtocolServer.
 
-The time element acts as a clock event - it sets the time for all future
+The progress directive is used to provide progress information about a stream
+so that stream consumer can provide completion estimates, progress bars and so
+on. Stream generators that know how many tests will be present in the stream
+should output "progress: COUNT". Stream filters that add tests should output
+"progress: +COUNT", and those that remove tests should output
+"progress: -COUNT". An absolute count should reset the progress indicators in
+use - it indicates that two separate streams from different generators have
+been trivially concatenated together, and there is no knowledge of how many
+more complete streams are incoming. Smart concatenation could scan each stream
+for their count and sum them, or alternatively translate absolute counts into
+relative counts inline.
+
+The time directive acts as a clock event - it sets the time for all future
 events. The value should be a valid ISO8601 time.
 
 The skip result is used to indicate a test that was found by the runner but not

=== added file 'filters/subunit2gtk'
--- filters/subunit2gtk	1970-01-01 00:00:00 +0000
+++ filters/subunit2gtk	2009-07-29 09:59:36 +0000
@@ -0,0 +1,229 @@
+#!/usr/bin/env python
+#  subunit: extensions to python unittest to get test results from subprocesses.
+#  Copyright (C) 2009  Robert Collins <[e-mail]>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+#
+
+### The GTK progress bar __init__ function is derived from the pygtk tutorial:
+# The PyGTK Tutorial is Copyright (C) 2001-2005 John Finlay.
+# 
+# The GTK Tutorial is Copyright (C) 1997 Ian Main.
+# 
+# Copyright (C) 1998-1999 Tony Gale.
+# 
+# Permission is granted to make and distribute verbatim copies of this manual
+# provided the copyright notice and this permission notice are preserved on all
+# copies.
+# 
+# Permission is granted to copy and distribute modified versions of this
+# document under the conditions for verbatim copying, provided that this
+# copyright notice is included exactly as in the original, and that the entire
+# resulting derived work is distributed under the terms of a permission notice
+# identical to this one.
+# 
+# Permission is granted to copy and distribute translations of this document
+# into another language, under the above conditions for modified versions.
+# 
+# If you are intending to incorporate this document into a published work,
+# please contact the maintainer, and we will make an effort to ensure that you
+# have the most up to date information available.
+# 
+# There is no guarantee that this document lives up to its intended purpose.
+# This is simply provided as a free resource. As such, the authors and
+# maintainers of the information provided within can not make any guarantee
+# that the information is even accurate.
+
+"""Display a subunit stream in a gtk progress window."""
+
+import os
+import sys
+import unittest
+
+import pygtk
+pygtk.require('2.0')
+import gtk, gtk.gdk, gobject
+
+from subunit import ProtocolTestCase, TestProtocolServer
+
+class GTKTestResult(unittest.TestResult):
+
+    def __init__(self):
+        super(GTKTestResult, self).__init__()
+        # Instance variables (in addition to TestResult)
+        self.window = None
+        self.run_label = None
+        self.ok_label = None
+        self.not_ok_label = None
+        self.total_tests = None
+
+        self.window = gtk.Window(gtk.WINDOW_TOPLEVEL)
+        self.window.set_resizable(True)
+
+        self.window.connect("destroy", gtk.main_quit)
+        self.window.set_title("Tests...")
+        self.window.set_border_width(0)
+
+        vbox = gtk.VBox(False, 5)
+        vbox.set_border_width(10)
+        self.window.add(vbox)
+        vbox.show()
+  
+        # Create a centering alignment object
+        align = gtk.Alignment(0.5, 0.5, 0, 0)
+        vbox.pack_start(align, False, False, 5)
+        align.show()
+
+        # Create the ProgressBar
+        self.pbar = gtk.ProgressBar()
+        align.add(self.pbar)
+        self.pbar.set_text("Running")
+        self.pbar.show()
+
+        separator = gtk.HSeparator()
+        vbox.pack_start(separator, False, False, 0)
+        separator.show()
+
+        # rows, columns, homogeneous
+        table = gtk.Table(2, 3, False)
+        vbox.pack_start(table, False, True, 0)
+        table.show()
+        # Show summary details about the run. Could use an expander.
+        label = gtk.Label("Run:")
+        table.attach(label, 0, 1, 1, 2, gtk.EXPAND | gtk.FILL,
+            gtk.EXPAND | gtk.FILL, 5, 5)
+        label.show()
+        self.run_label = gtk.Label("N/A")
+        table.attach(self.run_label, 1, 2, 1, 2, gtk.EXPAND | gtk.FILL,
+            gtk.EXPAND | gtk.FILL, 5, 5)
+        self.run_label.show()
+
+        label = gtk.Label("OK:")
+        table.attach(label, 0, 1, 2, 3, gtk.EXPAND | gtk.FILL,
+            gtk.EXPAND | gtk.FILL, 5, 5)
+        label.show()
+        self.ok_label = gtk.Label("N/A")
+        table.attach(self.ok_label, 1, 2, 2, 3, gtk.EXPAND | gtk.FILL,
+            gtk.EXPAND | gtk.FILL, 5, 5)
+        self.ok_label.show()
+
+        label = gtk.Label("Not OK:")
+        table.attach(label, 0, 1, 3, 4, gtk.EXPAND | gtk.FILL,
+            gtk.EXPAND | gtk.FILL, 5, 5)
+        label.show()
+        self.not_ok_label = gtk.Label("N/A")
+        table.attach(self.not_ok_label, 1, 2, 3, 4, gtk.EXPAND | gtk.FILL,
+            gtk.EXPAND | gtk.FILL, 5, 5)
+        self.not_ok_label.show()
+
+        self.window.show()
+        # For the demo.
+        self.window.set_keep_above(True)
+        self.window.present()
+
+    def stopTest(self, test):
+        super(GTKTestResult, self).stopTest(test)
+        if not self.total_tests:
+            self.pbar.pulse()
+        else:
+            self.pbar.set_fraction(self.testsRun/float(self.total_tests))
+
+    def stopTestRun(self):
+        try:
+            super(GTKTestResult, self).stopTestRun()
+        except AttributeError:
+            pass
+        self.pbar.set_text('Finished')
+
+    def addError(self, test, err):
+        super(GTKTestResult, self).addError(test, err)
+        self.update_counts()
+
+    def addFailure(self, test, err):
+        super(GTKTestResult, self).addFailure(test, err)
+        self.update_counts()
+
+    def addSuccess(self, test):
+        super(GTKTestResult, self).addSuccess(test)
+        self.update_counts()
+
+    def addSkip(self, test, reason):
+        super(GTKTestResult, self).addSkipSuccess(test, reason)
+        self.update_counts()
+
+    def addExpectedFailure(self, test, err):
+        super(GTKTestResult, self).addExpectedFailure(test, err)
+        self.update_counts()
+
+    def addUnexpectedSuccess(self, test):
+        super(GTKTestResult, self).addUnexpectedSuccess(test)
+        self.update_counts()
+
+    def progress(self, offset, whence):
+        if whence == os.SEEK_SET:
+            self.total_tests = offset
+        else:
+            self.total_tests += offset
+
+    def time(self, a_datetime):
+        # We don't try to estimate completion yet.
+        pass
+
+    def update_counts(self):
+        self.run_label.set_text(str(self.testsRun))
+        bad = len(self.failures + self.errors)
+        self.ok_label.set_text(str(self.testsRun - bad))
+        self.not_ok_label.set_text(str(bad))
+
+
+class GIOProtocolTestCase(object):
+
+    def __init__(self, stream, result, on_finish):
+        self.stream = stream
+        self.schedule_read()
+        self.hup_id = gobject.io_add_watch(stream, gobject.IO_HUP, self.hup)
+        self.protocol = TestProtocolServer(result)
+        self.on_finish = on_finish
+
+    def read(self, source, condition):
+        #NB: \o/ actually blocks
+        line = source.readline()
+        if not line:
+            self.protocol.lostConnection()
+            self.on_finish()
+            return False
+        self.protocol.lineReceived(line)
+        # schedule more IO shortly - if we say we're willing to do it
+        # immediately we starve things.
+        source_id = gobject.timeout_add(1, self.schedule_read)
+        return False
+
+    def schedule_read(self):
+        self.read_id = gobject.io_add_watch(self.stream, gobject.IO_IN, self.read)
+
+    def hup(self, source, condition):
+        self.protocol.lostConnection()
+        gobject.remove(self.read_id)
+        self.on_finish()
+
+
+result = GTKTestResult()
+test = GIOProtocolTestCase(sys.stdin, result, result.stopTestRun)
+gtk.main()
+if result.wasSuccessful():
+    exit_code = 0
+else:
+    exit_code = 1
+sys.exit(exit_code)

=== modified file 'filters/subunit2pyunit'
--- filters/subunit2pyunit	2009-02-15 11:55:00 +0000
+++ filters/subunit2pyunit	2009-07-28 21:44:28 +0000
@@ -17,15 +17,27 @@
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 #
 
-"""Filter a subunit stream through python's default unittest test runner."""
+"""Display a subunit stream through python's unittest test runner."""
 
+from optparse import OptionParser
 import sys
 import unittest
 
 from subunit import ProtocolTestCase, TestProtocolServer
 
-runner = unittest.TextTestRunner(verbosity=2)
+parser = OptionParser(description=__doc__)
+parser.add_option("--progress", action="store_true",
+    help="Use bzrlib's test reporter (requires bzrlib)",
+        default=False)
+(options, args) = parser.parse_args()
 test = ProtocolTestCase(sys.stdin)
+if options.progress:
+    from bzrlib.tests import TextTestRunner
+    from bzrlib import ui
+    ui.ui_factory = ui.make_ui_for_terminal(None, sys.stdout, sys.stderr)
+    runner = TextTestRunner()
+else:
+    runner = unittest.TextTestRunner(verbosity=2)
 if runner.run(test).wasSuccessful():
     exit_code = 0
 else:

=== modified file 'python/subunit/__init__.py'
--- python/subunit/__init__.py	2009-07-22 08:46:04 +0000
+++ python/subunit/__init__.py	2009-07-28 13:32:10 +0000
@@ -28,6 +28,10 @@
 import iso8601
 
 
+SEEK_CUR = os.SEEK_CUR
+SEEK_SET = os.SEEK_SET
+
+
 def test_suite():
     import subunit.tests
     return subunit.tests.test_suite()
@@ -200,6 +204,18 @@
         else:
             self.stdOutLineReceived(line)
 
+    def _handleProgress(self, offset, line):
+        """Process a progress directive."""
+        line = line[offset:].strip()
+        if line[0] in '+-':
+            whence = SEEK_CUR
+        else:
+            whence = SEEK_SET
+        delta = int(line)
+        progress_method = getattr(self.client, 'progress', None)
+        if callable(progress_method):
+            progress_method(delta, whence)
+
     def _handleTags(self, offset, line):
         """Process a tags command."""
         tags = line[offset:].split()
@@ -243,6 +259,8 @@
                     self._addError(offset, line)
                 elif cmd == 'failure':
                     self._addFailure(offset, line)
+                elif cmd == 'progress':
+                    self._handleProgress(offset, line)
                 elif cmd == 'skip':
                     self._addSkip(offset, line)
                 elif cmd in ('success', 'successful'):
@@ -355,6 +373,21 @@
         """Mark a test as starting its test run."""
         self._stream.write("test: %s\n" % test.id())
 
+    def progress(self, offset, whence):
+        """Provide indication about the progress/length of the test run.
+
+        :param offset: Information about the number of tests remaining. If
+            whence is SEEK_CUR, then offset increases/decreases the remaining
+            test count. If whence is SEEK_SET, then offset specifies exactly
+            the remaining test count.
+        :param whence: One of SEEK_CUR or SEEK_SET.
+        """
+        if whence == SEEK_CUR and offset > -1:
+            prefix = "+"
+        else:
+            prefix = ""
+        self._stream.write("progress: %s%s\n" % (prefix, offset))
+
     def time(self, a_datetime):
         """Inform the client of the time.
 

=== modified file 'python/subunit/test_results.py'
--- python/subunit/test_results.py	2009-07-22 23:17:18 +0000
+++ python/subunit/test_results.py	2009-07-28 13:32:10 +0000
@@ -87,6 +87,10 @@
         self._before_event()
         return self._call_maybe("addUnexpectedSuccess", test)
 
+    def progress(self, offset, whence):
+        self._before_event()
+        return self._call_maybe("progress", offset, whence)
+
     def wasSuccessful(self):
         self._before_event()
         return self.decorated.wasSuccessful()
@@ -124,6 +128,9 @@
         time = datetime.datetime.utcnow().replace(tzinfo=iso8601.Utc())
         self._call_maybe("time", time)
 
+    def progress(self, offset, whence):
+        return self._call_maybe("progress", offset, whence)
+
     @property
     def shouldStop(self):
         return self.decorated.shouldStop

=== modified file 'python/subunit/tests/test_test_protocol.py'
--- python/subunit/tests/test_test_protocol.py	2009-07-22 08:46:04 +0000
+++ python/subunit/tests/test_test_protocol.py	2009-07-28 13:32:10 +0000
@@ -37,6 +37,7 @@
         self.skip_calls = []
         self.start_calls = []
         self.success_calls = []
+        self.progress_calls = []
         self._time = None
         super(MockTestProtocolServerClient, self).__init__()
 
@@ -58,6 +59,9 @@
     def startTest(self, test):
         self.start_calls.append(test)
 
+    def progress(self, offset, whence):
+        self.progress_calls.append((offset, whence))
+
     def time(self, time):
         self._time = time
 
@@ -118,6 +122,11 @@
         self.assertEqual(protocol.success_calls, [])
         self.assertEqual(protocol.start_calls, [])
 
+    def test_progress(self):
+        protocol = MockTestProtocolServerClient()
+        protocol.progress(-1, subunit.SEEK_CUR)
+        self.assertEqual(protocol.progress_calls, [(-1, subunit.SEEK_CUR)])
+
 
 class TestTestImports(unittest.TestCase):
 
@@ -710,6 +719,36 @@
         self.success_quoted_bracket("success:")
 
 
+class TestTestProtocolServerProgress(unittest.TestCase):
+    """Test receipt of progress: directives."""
+
+    def test_progress_accepted_stdlib(self):
+        # With a stdlib TestResult, progress events are swallowed.
+        self.result = unittest.TestResult()
+        self.stream = StringIO()
+        self.protocol = subunit.TestProtocolServer(self.result,
+            stream=self.stream)
+        self.protocol.lineReceived("progress: 23")
+        self.protocol.lineReceived("progress: -2")
+        self.protocol.lineReceived("progress: +4")
+        self.assertEqual("", self.stream.getvalue())
+
+    def test_progress_accepted_extended(self):
+        # With a progress capable TestResult, progress events are emitted.
+        self.result = MockTestProtocolServerClient()
+        self.stream = StringIO()
+        self.protocol = subunit.TestProtocolServer(self.result,
+            stream=self.stream)
+        self.protocol.lineReceived("progress: 23")
+        self.protocol.lineReceived("progress: -2")
+        self.protocol.lineReceived("progress: +4")
+        self.assertEqual("", self.stream.getvalue())
+        self.assertEqual(
+            [(23, subunit.SEEK_SET), (-2, subunit.SEEK_CUR),
+            (4, subunit.SEEK_CUR)],
+            self.result.progress_calls)
+
+
 class TestTestProtocolServerStreamTags(unittest.TestCase):
     """Test managing tags on the protocol level."""
 
@@ -1005,6 +1044,18 @@
             self.io.getvalue(),
             'skip: %s [\nHas it really?\n]\n' % self.test.id())
 
+    def test_progress_set(self):
+        self.protocol.progress(23, subunit.SEEK_SET)
+        self.assertEqual(self.io.getvalue(), 'progress: 23\n')
+
+    def test_progress_neg_cur(self):
+        self.protocol.progress(-23, subunit.SEEK_CUR)
+        self.assertEqual(self.io.getvalue(), 'progress: -23\n')
+
+    def test_progress_pos_cur(self):
+        self.protocol.progress(23, subunit.SEEK_CUR)
+        self.assertEqual(self.io.getvalue(), 'progress: +23\n')
+
     def test_time(self):
         # Calling time() outputs a time signal immediately.
         self.protocol.time(

=== modified file 'python/subunit/tests/test_test_results.py'
--- python/subunit/tests/test_test_results.py	2009-07-22 23:17:18 +0000
+++ python/subunit/tests/test_test_results.py	2009-07-28 13:32:10 +0000
@@ -108,6 +108,9 @@
     def test_addUnexpectedSuccess(self):
         self.result.addUnexpectedSuccess(self)
 
+    def test_progress(self):
+        self.result.progress(1, os.SEEK_SET)
+
     def test_wasSuccessful(self):
         self.result.wasSuccessful()
 
@@ -135,6 +138,10 @@
         self.assertEqual(1, len(self.result.decorated._calls))
         self.assertNotEqual(None, self.result.decorated._calls[0])
 
+    def test_no_time_from_progress(self):
+        self.result.progress(1, os.SEEK_CUR)
+        self.assertEqual(0, len(self.result.decorated._calls))
+
     def test_no_time_from_shouldStop(self):
         self.result.decorated.stop()
         self.result.shouldStop