Merge ~cjwatson/launchpad:loggerhead-fix-robots into launchpad:master

[Colin Watson]

launchpad_loggerhead: Fix robots.txt

On Python 3, the response to a request for `robots.txt` needs to be bytes or Gunicorn rejects it.

We didn't notice this on production or staging because this file is served via an Apache `Alias` directive rather than going via the appserver, but fixing it seems to make sense anyway.

[Ines Almeida (ines-almeida)]

LGTM

review: Approve

diff --git a/lib/launchpad_loggerhead/app.py b/lib/launchpad_loggerhead/app.py
index 20549b4..b7496ae 100644
--- a/lib/launchpad_loggerhead/app.py
+++ b/lib/launchpad_loggerhead/app.py
@@ -34,7 +34,7 @@ from lp.services.webapp.errorlog import ErrorReportingUtility
 from lp.services.webapp.vhosts import allvhosts
 from lp.xmlrpc import faults
 
-robots_txt = """\
+robots_txt = b"""\
 User-agent: *
 Disallow: /
 """
diff --git a/lib/launchpad_loggerhead/tests.py b/lib/launchpad_loggerhead/tests.py
index 1bc5fae..ba788ff 100644
--- a/lib/launchpad_loggerhead/tests.py
+++ b/lib/launchpad_loggerhead/tests.py
@@ -191,6 +191,13 @@ class TestWSGI(TestCaseWithFactory):
             "loggerhead-debug", Content(UTF8_TEXT, get_debug_log_bytes)
         )
 
+    def test_robots(self):
+        response = requests.get(
+            "http://127.0.0.1:%d/robots.txt" % config.codebrowse.port
+        )
+        self.assertEqual(200, response.status_code)
+        self.assertEqual(b"User-agent: *\nDisallow: /\n", response.content)
+
     def test_public_port_public_branch(self):
         # Requests for public branches on the public port are allowed.
         db_branch, _ = self.create_branch_and_tree()