Merge lp:~hloeung/ubuntu-repository-cache/extra-server-process into lp:ubuntu-repository-cache

[Haw Loeung]

Allow an additional process to reduce short outages on graceful restart - LP:1918211

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Unable to determine commit message from repository - please click "Set commit message" and enter the commit message manually.

[Paul Collins (pjdc)]

There's a comment in the code already saying "VMs perform badly with over 2500 workers". I definitely want the extra process, but this this seems like it'll double the number of workers... will that be a problem?

I'm wondering what the performance impact of more processes is. Ideally we want to have one process's worth of threads unused at all times (i'm pretty sure graceful is serialized). The upstream default seems to be StartServers 3....

[Paul Collins (pjdc)]

For what it's worth (I checked just now out of curiosity) the 100Gbps machines seem to have close to 200 apache2 processes each at the moment.

[Paul Collins (pjdc)]

review: Approve (lgtm)

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change has no commit message, setting status to needs review.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision 332

=== modified file 'lib/ubuntu_repository_cache/apache.py'
--- lib/ubuntu_repository_cache/apache.py	2021-03-10 00:46:28 +0000
+++ lib/ubuntu_repository_cache/apache.py	2021-03-11 23:09:41 +0000
@@ -92,7 +92,9 @@
     if not config['serverlimit']:
         # Benchmarking shows single process multiple threads performs better
         # than multiple processes multiple threads.
-        config['serverlimit'] = 1
+        # Unfortunately, we need more than one server process to allow for
+        # graceful restarts. See LP#1918211.
+        config['serverlimit'] = 2
 
     if not config['threadsperchild']:
         # We could take into consideration how much memory the system
@@ -104,8 +106,8 @@
         # (likely squid-deb-proxy bottleneck). But let's cap it here,
         # someone can override it by setting the
         # apache2_mpm_threadlimit charm config.
-        if config['threadsperchild'] > 2560:
-            config['threadsperchild'] = 2560
+        if (config['threadsperchild'] * config['serverlimit']) > 2560:
+            config['threadsperchild'] = int(2560 / config['serverlimit'])
 
     if not config['threadlimit']:
         # Hard limit should match per child limit.
 
=== modified file 'lib/ubuntu_repository_cache/tests/test_apache.py'
--- lib/ubuntu_repository_cache/tests/test_apache.py	2021-01-31 22:33:24 +0000
+++ lib/ubuntu_repository_cache/tests/test_apache.py	2021-03-11 23:09:41 +0000
@@ -38,12 +38,12 @@
         }
         expected = {
             'startservers': 1,
-            'minsparethreads': 512,
-            'maxsparethreads': 1024,
+            'minsparethreads': 1024,
+            'maxsparethreads': 2048,
             'threadlimit': 1024,
             'threadsperchild': 1024,
-            'serverlimit': 1,
-            'maxrequestworkers': 1024,
+            'serverlimit': 2,
+            'maxrequestworkers': 2048,
             'maxconnectionsperchild': 10000,
         }
         self.assertEqual(apache.tune_mpm_configs(config), expected)
@@ -63,9 +63,9 @@
             'startservers': 1,
             'minsparethreads': 1280,
             'maxsparethreads': 2560,
-            'threadlimit': 2560,
-            'threadsperchild': 2560,
-            'serverlimit': 1,
+            'threadlimit': 1280,
+            'threadsperchild': 1280,
+            'serverlimit': 2,
             'maxrequestworkers': 2560,
             'maxconnectionsperchild': 10000,
         }
@@ -84,12 +84,12 @@
         }
         expected = {
             'startservers': 1,
-            'minsparethreads': 128,
-            'maxsparethreads': 256,
+            'minsparethreads': 256,
+            'maxsparethreads': 512,
             'threadlimit': 256,
             'threadsperchild': 256,
-            'serverlimit': 1,
-            'maxrequestworkers': 256,
+            'serverlimit': 2,
+            'maxrequestworkers': 512,
             'maxconnectionsperchild': 10000,
         }
         self.assertEqual(apache.tune_mpm_configs(config), expected)
@@ -107,12 +107,12 @@
         }
         expected = {
             'startservers': 1,
-            'minsparethreads': 256,
-            'maxsparethreads': 512,
+            'minsparethreads': 512,
+            'maxsparethreads': 1024,
             'threadlimit': 512,
             'threadsperchild': 512,
-            'serverlimit': 1,
-            'maxrequestworkers': 512,
+            'serverlimit': 2,
+            'maxrequestworkers': 1024,
             'maxconnectionsperchild': 10000,
         }
         self.assertEqual(apache.tune_mpm_configs(config), expected)
 
=== modified file 'tests/unit/test_apache.py'
--- tests/unit/test_apache.py	2021-03-08 21:00:00 +0000
+++ tests/unit/test_apache.py	2021-03-11 23:09:41 +0000
@@ -238,10 +238,10 @@
             '/etc/apache2/conf-available/000mpm-worker.conf',
             {
                 'maxconnectionsperchild': 10000,
-                'maxrequestworkers': 512,
-                'maxsparethreads': 512,
-                'minsparethreads': 256,
-                'serverlimit': 1,
+                'maxrequestworkers': 1024,
+                'maxsparethreads': 1024,
+                'minsparethreads': 512,
+                'serverlimit': 2,
                 'startservers': 1,
                 'threadlimit': 512,
                 'threadsperchild': 512,