Merge lp:~wgrant/launchpad/bug-694001-apache-username-spaces into lp:launchpad

[William Grant]

[r=jcsackett,sinzui][ui=none][bug=694001] Prevent the Apache log parser from choking on usernames with spaces.

Private PPA Apache logs contain unquoted usernames, which can be whatever the user wants -- even containing spaces or other strange characters. Since the default Apache combined log format uses spaces to delimit fields, the parser grabs fields with \S+ unless they are quoted. This makes it choke on lines with usernames containing spaces.

This branch fixes contrib.apachelog to match usernames with spaces. It's the only field in the default log format that can contain spaces, so it's still deterministically parsable.

[j.c.sackett (jcsackett)]

William--

This looks fine by me.

review: Approve (code*)

[Curtis Hovey (sinzui)]

This is good to land

review: Approve (code)

=== modified file 'lib/contrib/apachelog.py'
--- lib/contrib/apachelog.py	2009-04-29 19:10:17 +0000
+++ lib/contrib/apachelog.py	2011-01-19 00:11:54 +0000
@@ -159,7 +159,7 @@
             elif findpercent.search(element):
                 subpattern = r'(\[[^\]]+\])'
                 
-            elif element == '%U':
+            elif element in ('%U', '%u'):
                 subpattern = '(.+?)'
             
             subpatterns.append(subpattern)
 
=== modified file 'lib/lp/services/apachelogparser/tests/test_apachelogparser.py'
--- lib/lp/services/apachelogparser/tests/test_apachelogparser.py	2011-01-05 04:56:11 +0000
+++ lib/lp/services/apachelogparser/tests/test_apachelogparser.py	2011-01-19 00:11:54 +0000
@@ -68,6 +68,22 @@
         self.assertEqual(
             request, 'GET /10133748/cramfsswap_1.4.1.tar.gz HTTP/1.0')
 
+    def test_parsing_line_with_spaces_in_username(self):
+        # Some lines have spaces in the username, left unquoted by
+        # Apache. They can still be parsed OK, since no other fields
+        # have similar issues.
+        line = (r'1.1.1.1 - Some User [25/Jan/2009:15:48:07 +0000] "GET '
+                r'/10133748/cramfsswap_1.4.1.tar.gz HTTP/1.0" 200 12341 '
+                r'"http://foo.bar/?baz=\"bang\"" '
+                r'"\"Nokia2630/2.0 (05.20) Profile/MIDP-2.1 '
+                r'Configuration/CLDC-1.1\""')
+        host, date, status, request = get_host_date_status_and_request(line)
+        self.assertEqual(host, '1.1.1.1')
+        self.assertEqual(date, '[25/Jan/2009:15:48:07 +0000]')
+        self.assertEqual(status, '200')
+        self.assertEqual(
+            request, 'GET /10133748/cramfsswap_1.4.1.tar.gz HTTP/1.0')
+
     def test_day_extraction(self):
         date = '[13/Jun/2008:18:38:57 +0100]'
         self.assertEqual(get_day(date), datetime(2008, 6, 13))