Merge ~cjwatson/launchpad:py3-bug-text into launchpad:master

[Colin Watson]

Fix Bug:+text for Python 3

After converting BugTextView to message_as_bytes, we still find that Python 3 encodes email headers slightly differently, causing xx-bug-text-pages.txt to fail; so also rearrange the test to parse the comments as an email message and test the bits of them that we care about.

[Cristian Gonzalez (cristiangsp)]

Looks good!

review: Approve

diff --git a/lib/lp/bugs/browser/bug.py b/lib/lp/bugs/browser/bug.py
index eedf9b4..b1909e7 100644
--- a/lib/lp/bugs/browser/bug.py
+++ b/lib/lp/bugs/browser/bug.py
@@ -105,6 +105,7 @@ from lp.bugs.model.structuralsubscription import (
     get_structural_subscriptions_for_bug,
     )
 from lp.registry.interfaces.person import IPersonSet
+from lp.services.compat import message_as_bytes
 from lp.services.features import getFeatureFlag
 from lp.services.fields import DuplicateBug
 from lp.services.librarian.browser import ProxiedLibraryFileAlias
@@ -1163,13 +1164,12 @@ class BugTextView(LaunchpadView):
 
         for comment in comments:
             message = build_message(comment.text_for_display)
-            message['Author'] = comment.owner.unique_displayname.encode(
-                'utf-8')
+            message['Author'] = comment.owner.unique_displayname
             message['Date'] = format_rfc2822_date(comment.datecreated)
             message['Message-Id'] = comment.rfc822msgid
             comment_mime.attach(message)
 
-        return comment_mime.as_string().decode('utf-8')
+        return message_as_bytes(comment_mime).decode('utf-8')
 
     def render(self):
         """Return a text representation of the bug."""
diff --git a/lib/lp/bugs/stories/bugs/xx-bug-text-pages.txt b/lib/lp/bugs/stories/bugs/xx-bug-text-pages.txt
index 6d16ca2..aedbffd 100644
--- a/lib/lp/bugs/stories/bugs/xx-bug-text-pages.txt
+++ b/lib/lp/bugs/stories/bugs/xx-bug-text-pages.txt
@@ -95,7 +95,7 @@ all tasks related to that bug, presented in an easy-to-digest format:
     date-left-closed: ...
     reporter: Sample Person (name12)
     importance: Low
-    assignee: M...rk Sh...ttlew...rth (mark)
+    assignee: Márk Shúttlewòrth (mark)
     milestone: 
     <BLANKLINE>
     task: mozilla-firefox (Ubuntu)
@@ -119,31 +119,6 @@ all tasks related to that bug, presented in an easy-to-digest format:
     milestone: 
     <BLANKLINE>
     Content-Type: multipart/mixed; boundary="...
-    MIME-Version: 1.0
-    <BLANKLINE>
-    --...
-    Content-Type: text/plain; charset="utf-8"
-    Content-Transfer-Encoding: quoted-printable
-    <BLANKLINE>
-    Firefox needs to support embedded SVG images, now that the standard has
-    been finalised.
-    <BLANKLINE>
-    The SVG standard 1.0 is complete, and draft implementations for Firefox
-    exist. One of these implementations needs to be integrated with the base
-    install of Firefox. Ideally, the implementation needs to include support
-    for the manipulation of SVG objects from JavaScript to enable interactive
-    and dynamic SVG drawings.
-    --...
-    ...
-    --...
-    Content-Type: text/plain; charset="utf-8"
-    Content-Transfer-Encoding: quoted-printable
-    Author: ... (mark)
-    Date: ...
-    Message-Id: ...
-    <BLANKLINE>
-    comment for file with space
-    --...
 
 The multiple white spaces in the mime type of the second attachment
 are replaced by a single space.
@@ -154,6 +129,43 @@ are replaced by a single space.
     ' http://bugs.launchpad.test/.../file%20with%20space.txt text/plain;
     name="file with space.txt"'
 
+The comments are represented as a MIME message.
+
+    >>> import email
+    >>> from email.header import decode_header
+    >>> comments = email.message_from_string(
+    ...     text_bug[text_bug.find('Content-Type:'):]).get_payload()
+
+    >>> print(comments[0]['Content-Type'])
+    text/plain; charset="utf-8"
+    >>> 'Author' in comments[0]
+    False
+    >>> 'Date' in comments[0]
+    False
+    >>> 'Message-Id' in comments[0]
+    False
+    >>> print(comments[0].get_payload())
+    Firefox needs to support embedded SVG images, now that the standard has
+    been finalised.
+    <BLANKLINE>
+    The SVG standard 1.0 is complete, and draft implementations for Firefox
+    exist. One of these implementations needs to be integrated with the base
+    install of Firefox. Ideally, the implementation needs to include support
+    for the manipulation of SVG objects from JavaScript to enable interactive
+    and dynamic SVG drawings.
+
+    >>> print(comments[3]['Content-Type'])
+    text/plain; charset="utf-8"
+    >>> [(author_bytes, author_charset)] = decode_header(comments[3]['Author'])
+    >>> print(author_bytes.decode(author_charset))
+    Márk Shúttlewòrth (mark)
+    >>> 'Date' in comments[3]
+    True
+    >>> 'Message-Id' in comments[3]
+    True
+    >>> print(comments[3].get_payload())
+    comment for file with space
+
 
 == Text Pages from a Bug Task Context ==