Merge ~tcuthbert/container-log-archive-charm:master into container-log-archive-charm:master

[Thomas Cuthbert]

Work around nginx general access/error logs that emit the date as the first column

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[Stuart Bishop (stub)]

This looks like a library bug to me. Surely this case should be an netaddr.AddrFormatError rather than a generic ValueError? Anyway, given fixing upstream right now and Python stdlib doesn't offer usable IP address validation...

Sniffing the error message is rather fragile, and a last resort. How about:

ip = splits[0].decode('utf-8')
try:
    ip = netaddr.IPAddress(ip)
except (netaddr.AddrFormatError, ValueError) as e:
    [...]

Alternatively, explicitly check if splits[0] is a date before attempting to convert things. This would be easy using a regexp or strptime if the date format is stable.

review: Needs Fixing

[Joel Sing (jsing)]

LGTM, see nit inline.

review: Approve (+1)

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision c7cf260897b0e3b3441e812797420e4b2523111a

diff --git a/files/anonymize_web_log_ips.py b/files/anonymize_web_log_ips.py
index fe26128..fa37efc 100755
--- a/files/anonymize_web_log_ips.py
+++ b/files/anonymize_web_log_ips.py
@@ -4,6 +4,7 @@
 """
 import argparse
 import netaddr
+import re
 import sys
 
 
@@ -21,8 +22,13 @@ def anonymize(new_file, log_file, skip_ips=None, skip_private=False):
     for line in log_file:
         total += 1
         splits = line.split()
+        ip_raw = splits[0].decode('utf-8')
+        # Work around nginx general access/error logs that emit the date instead of IP as the first column.
+        if re.match('^\d+/\d+/\d+', ip_raw):
+            skip += 1
+            continue
         try:
-            ip = netaddr.IPAddress(splits[0].decode('utf-8'))
+            ip = netaddr.IPAddress(ip_raw)
         except netaddr.AddrFormatError:
             skip += 1
             continue