Add rate limiting on source IP to prevent DDoS
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Content Cache Charm |
Fix Released
|
Medium
|
Haw Loeung |
Bug Description
We had a situation with a small subset of IP was generating a lot of request, causing service disruption
```
Number of request | IP
60645 | 150.136.170.161
1577168 | 150.136.216.209
1425381 | 150.136.228.9
866199 | 150.136.33.22
```
We should add a rate limit per IP (or at least an option allowing to) in either haproxy or iptables to prevent this kind of thing from happening.
For haproxy:
* https:/
* https:/
For iptables, using the hashlimit extension seems relevant for the purpose here.
* http://
The threshold are yet to be determined but at least not allowing 100 connections from a single IP in a short period of time seems a good start.
Related branches
- James Simpson: Approve
- Canonical IS Reviewers: Pending requested
-
Diff: 460 lines (+293/-2)14 files modifiedlib/haproxy.py (+21/-1)
reactive/content_cache.py (+2/-0)
tests/unit/files/config_test_config.txt (+6/-1)
tests/unit/files/content_cache_rendered_haproxy_test_output.txt (+3/-0)
tests/unit/files/content_cache_rendered_haproxy_test_output_auto_maxconns.txt (+3/-0)
tests/unit/files/content_cache_rendered_haproxy_test_output_load_balancing_algorithm.txt (+3/-0)
tests/unit/files/content_cache_rendered_haproxy_test_output_processes_and_threads.txt (+3/-0)
tests/unit/files/content_cache_rendered_haproxy_test_output_processes_and_threads_haproxy2.txt (+3/-0)
tests/unit/files/content_cache_rendered_haproxy_test_output_rate_limiting.txt (+95/-0)
tests/unit/files/content_cache_rendered_haproxy_test_output_rate_limiting_missing_condition.txt (+92/-0)
tests/unit/files/haproxy_config_rendered_backends_stanzas_test_output.txt (+3/-0)
tests/unit/files/haproxy_config_rendered_test_output.txt (+3/-0)
tests/unit/files/haproxy_config_rendered_test_output2.txt (+3/-0)
tests/unit/test_content_cache.py (+53/-0)
Changed in content-cache-charm: | |
status: | New → Confirmed |
importance: | Undecided → Medium |
description: | updated |
Changed in content-cache-charm: | |
status: | Confirmed → In Progress |
assignee: | nobody → Haw Loeung (hloeung) |
Changed in content-cache-charm: | |
status: | In Progress → Fix Committed |
Changed in content-cache-charm: | |
status: | Fix Committed → Fix Released |
Let's make sure the threshold is high enough so we don't block things like VPN or university http proxies