Merge lp:~hloeung/ubuntu-repository-cache/add-accept-encoding-headers into lp:ubuntu-repository-cache

[Haw Loeung]

Add 'Accept-Encoding' in nrpe and health checks

Upstream archives use mod_deflate which sets 'Vary: Accept-Encoding' on responses, so check_http should set that header.

This also reduces the logging spam from squid:

| 2021/03/10 00:15:00 kid1| varyEvaluateMatch: Oops. Not a Vary match on second attempt, 'http://archive.ubuntu.com/ubuntu/pool/' 'accept-encoding="*"'
| 2021/03/10 00:15:00 kid1| clientProcessHit: Vary object loop!

We also hope to reduce squid memory usage (mem_node) as discovered by
tcuthbert.

[Thomas Cuthbert (tcuthbert)]

LGTM +1

review: Approve

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision 330

=== modified file 'files/health_check.py'
--- files/health_check.py	2021-03-08 08:50:42 +0000
+++ files/health_check.py	2021-03-10 00:57:05 +0000
@@ -16,6 +16,7 @@
 CHECK_HEADERS = {
     'Host': 'archive.ubuntu.com',
     'User-agent': 'ubuntu-repository-cache/healthcheck',
+    'Accept-Encoding': '*',
 }
 CHECK_HOST = 'http://localhost/'
 CHECK_TIMEOUT = 5

=== modified file 'lib/ubuntu_repository_cache/apache.py'
--- lib/ubuntu_repository_cache/apache.py	2021-03-07 22:00:26 +0000
+++ lib/ubuntu_repository_cache/apache.py	2021-03-10 00:57:05 +0000
@@ -355,15 +355,22 @@
                 check_cmd=(
                     'check_http '
                     '--hostname ubuntu-archive-asm.trafficmanager.net '
-                    '--url "/{}/" --string "dists/" '
-                    '--expect \"200\\ OK\"'
+                    '--url "/{}/" '
+                    '--string "dists/" '
+                    '--expect \"200\\ OK\" '
+                    '--header \"Accept-Encoding:\\ \\*\"'
                 ).format(path_base),
             )
 
     nrpe_config.add_check(
         shortname='apache',
         description='Apache2 serving archive metadata',
-        check_cmd=('check_http --hostname localhost --url "/{}/" ' '--string "dists/" --expect \"200\\ OK\"').format(
-            path_base
-        ),
+        check_cmd=(
+            'check_http '
+            '--hostname localhost '
+            '--url "/{}/" '
+            '--string "dists/" '
+            '--expect \"200\\ OK\" '
+            '--header \"Accept-Encoding:\\ \\*\"'
+        ).format(path_base),
     )

=== modified file 'lib/ubuntu_repository_cache/squid.py'
--- lib/ubuntu_repository_cache/squid.py	2020-12-07 23:36:53 +0000
+++ lib/ubuntu_repository_cache/squid.py	2021-03-10 00:57:05 +0000
@@ -253,6 +253,11 @@
         shortname='squid',
         description='squid serving archive pool',
         check_cmd=(
-            'check_http --hostname localhost --url "/{}/pool/" ' '--string "main/" --expect \"200\\ OK\"'
+            'check_http '
+            '--hostname localhost '
+            '--url "/{}/pool/" '
+            '--string "main/" '
+            '--expect \"200\\ OK\" '
+            '--header \"Accept-Encoding:\\ \\*\"'
         ).format(path_base),
     )