Merge lp:~mars/launchpad/fix-ec2test-utf-in-devel into lp:launchpad

[Māris Fogels]

Fix a number of errors related to the utf-8 charset in the ec2 test suite.

Hi,

This branch fixes a problem on production-devel and Maverick local developer stations where a test would fail with a UTF-8 decoding error. This patch has already landed in production (see https://code.edge.launchpad.net/~mars/launchpad/fix-ec2test-on-production-devel/+merge/37526).

In this new proposal I have applied the technique more widely to catch a number of similar errors identified in the Launchpad stable branch. I reproduced these errors using devel on Maverick, and have patched them accordingly. All tests in 'bin/test -cv devscripts.ec2test.tests.test_remote' now pass.

Maris

[Jonathan Lange (jml)]

Hey Mars,

I think the right approach is to change get_body_text() to use decode=True itself. The point of the helper is to get the text.

Also, strangely, these tests are failing for me in *stable*.

jml

review: Needs Fixing

[Leonard Richardson (leonardr)]

review: Approve

[Māris Fogels (mars)]

> Hey Mars,
>
> I think the right approach is to change get_body_text() to use decode=True
> itself. The point of the helper is to get the text.
>
> Also, strangely, these tests are failing for me in *stable*.
>
> jml

I saw this test failing for me in devel/Maverick, and in production-devel/Lucid. I don't know how it spread so widely.

I can fix the helper, but I also found that the reported TestResult changed if I did not do the calculations before the assertEquals() call. If the assertEquals() failed, then *its* TestResult would be used instead, to the great confusion of the developer.

I can change the helper.

[Māris Fogels (mars)]

Jono, does this fix look better?

[Michael Hudson-Doyle (mwhudson)]

Looks plausible. I'm taking your word that it fixes the tests :-)

review: Approve

=== modified file 'lib/devscripts/ec2test/tests/test_remote.py'
--- lib/devscripts/ec2test/tests/test_remote.py	2010-09-20 22:44:49 +0000
+++ lib/devscripts/ec2test/tests/test_remote.py	2010-10-06 22:07:04 +0000
@@ -122,6 +122,16 @@
             'full.log', 'summary.log', 'index.html', request, echo_to_stdout)
 
 
+def get_body_text(email):
+    """A helper to retrieve the text body of a test run mail.
+
+    :param email: An email.mime.MultiPartMime object.
+    """
+    # Stringify the utf8-encoded MIME text message part containing the
+    # test run summary.
+    return email.get_payload(0).get_payload(decode=True)
+
+
 class TestFlagFallStream(TestCase):
     """Tests for `FlagFallStream`."""
 
@@ -526,8 +536,8 @@
         [body, attachment] = email.get_payload()
         self.assertIsInstance(body, MIMEText)
         self.assertEqual('inline', body['Content-Disposition'])
-        self.assertEqual('text/plain; charset="utf-8"', body['Content-Type'])
-        self.assertEqual("foo", body.get_payload())
+        self.assertEqual('text/plain; charset="utf8"', body['Content-Type'])
+        self.assertEqual("foo", get_body_text(email))
 
     def test_report_email_attachment(self):
         req = self.make_request(emails=['[e-mail]'])
@@ -769,7 +779,7 @@
         [body, attachment] = email.get_payload()
         self.assertIsInstance(body, MIMEText)
         self.assertEqual('inline', body['Content-Disposition'])
-        self.assertEqual('text/plain; charset="utf-8"', body['Content-Type'])
+        self.assertEqual('text/plain; charset="utf8"', body['Content-Type'])
         self.assertEqual(
             logger.get_summary_contents(), body.get_payload(decode=True))
         self.assertIsInstance(attachment, MIMEApplication)
@@ -838,7 +848,7 @@
         self.assertNotEqual([], email_log)
         [tester_msg] = email_log
         self.assertEqual('[e-mail]', tester_msg['To'])
-        self.assertIn('ZeroDivisionError', str(tester_msg))
+        self.assertIn('ZeroDivisionError', get_body_text(tester_msg))
 
 
 class TestDaemonizationInteraction(TestCaseWithTransport, RequestHelpers):
@@ -887,9 +897,6 @@
 class TestResultHandling(TestCaseWithTransport, RequestHelpers):
     """Tests for how we handle the result at the end of the test suite."""
 
-    def get_body_text(self, email):
-        return email.get_payload()[0].get_payload()
-
     def make_empty_result(self):
         return TestResult()
 
@@ -943,7 +950,7 @@
         self.assertEqual(message, pqm_message)
         self.assertIn(
             'SUBMITTED TO PQM:\n%s' % (message['Subject'],),
-            self.get_body_text(user_message))
+            get_body_text(user_message))
 
     def test_doesnt_submit_to_pqm_on_failure(self):
         log = []
@@ -964,7 +971,7 @@
         [user_message] = log
         self.assertIn(
             '**NOT** submitted to PQM:\n%s' % (message['Subject'],),
-            self.get_body_text(user_message))
+            get_body_text(user_message))
 
     def test_email_refers_to_attached_log(self):
         log = []
@@ -975,7 +982,7 @@
         [user_message] = log
         self.assertIn(
             '(See the attached file for the complete log)\n',
-            self.get_body_text(user_message))
+            get_body_text(user_message))
 
     def test_email_body_is_format_result(self):
         # The body of the email sent to the user is the summary file.
@@ -986,10 +993,11 @@
         result = self.make_failing_result()
         logger.got_result(result)
         [user_message] = log
+        error_result_string = request.format_result(
+                result, logger._start_time, logger._end_time)
         self.assertEqual(
-            request.format_result(
-                result, logger._start_time, logger._end_time),
-            self.get_body_text(user_message).decode('quoted-printable'))
+            error_result_string,
+            get_body_text(user_message))
 
     def test_gzip_of_full_log_attached(self):
         # The full log is attached to the email.