Merge lp:~tribaal/charms/trusty/nova-cloud-controller/enable-api-rate-limiting into lp:~openstack-charmers-archive/charms/trusty/nova-cloud-controller/next

[Chris Glass]

This branch enables API rate-limiting in the nova-cloud-controller charm instead of the nova-compute one.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #11157 nova-cloud-controller-next for tribaal mp273217
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/12638435/
Build: http://10.245.162.77:8080/job/charm_lint_check/11157/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #10362 nova-cloud-controller-next for tribaal mp273217
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/10362/

[lp:~tribaal/charms/trusty/nova-cloud-controller/enable-api-rate-limiting]

198. By Chris Glass on 2015-10-02

Added extra blank line.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #11161 nova-cloud-controller-next for tribaal mp273217
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/11161/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #10365 nova-cloud-controller-next for tribaal mp273217
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/10365/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6972 nova-cloud-controller-next for tribaal mp273217
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12638894/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6972/

[lp:~tribaal/charms/trusty/nova-cloud-controller/enable-api-rate-limiting]

199. By Chris Glass on 2015-10-02

Setting expectations right for the failing test.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #11162 nova-cloud-controller-next for tribaal mp273217
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/11162/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6975 nova-cloud-controller-next for tribaal mp273217
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12639181/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6975/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6981 nova-cloud-controller-next for tribaal mp273217
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12639295/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6981/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6985 nova-cloud-controller-next for tribaal mp273217
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12639381/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6985/

[lp:~tribaal/charms/trusty/nova-cloud-controller/enable-api-rate-limiting]

200. By Chris Glass on 2015-10-02

An actual missing semicolon.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #11168 nova-cloud-controller-next for tribaal mp273217
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/11168/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #10371 nova-cloud-controller-next for tribaal mp273217
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/10371/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #6991 nova-cloud-controller-next for tribaal mp273217
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12640547/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6991/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #7001 nova-cloud-controller-next for tribaal mp273217
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12642142/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7001/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #7027 nova-cloud-controller-next for tribaal mp273217
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12644613/
Build: http://10.245.162.77:8080/job/charm_amulet_test/7027/

[Ryan Beisner (1chb1n)]

Apologize for test noise while we are sorting through infra issues.

...

However, this is the real failure observed in the amulet tests. It looks like the test will need some adjusting.

00:07:49.623 2015-10-03 03:04:58 Invalid config charm nova-compute api-rate-limit-rules=( POST, '*', .*, 9999, MINUTE );

review: Needs Fixing

[lp:~tribaal/charms/trusty/nova-cloud-controller/enable-api-rate-limiting]

201. By Chris Glass on 2015-10-05

Don't assign the same dict to both nova-compute and nova-cloud-controller.

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #11324 nova-cloud-controller-next for tribaal mp273217
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/11324/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #10518 nova-cloud-controller-next for tribaal mp273217
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/10518/

[Liam Young (gnuoy)]

Ran amulet manually: http://paste.ubuntu.com/12688874/ looks good!

juju-test INFO : Results: 9 passed, 0 failed, 0 errored

[Liam Young (gnuoy)]

Approved

review: Approve

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #7109 nova-cloud-controller-next for tribaal mp273217
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/7109/

=== modified file 'config.yaml'
--- config.yaml	2015-09-28 10:02:33 +0000
+++ config.yaml	2015-10-05 06:51:47 +0000
@@ -411,3 +411,12 @@
     description: |
       List of filter class names to use for filtering hosts when not specified in
       the request.
+  api-rate-limit-rules:
+    type: string
+    default:
+    description: |
+      The API rate-limit rules to use for the deployed nova API, if any.
+      Contents of this config options will be inserted in the api-paste.ini file
+      under the "filter:ratelimit" section as "limits". The syntax for these
+      rules is documented at
+      http://docs.openstack.org/kilo/config-reference/content/configuring-compute-API.html

=== modified file 'hooks/nova_cc_context.py'
--- hooks/nova_cc_context.py	2015-09-29 06:46:46 +0000
+++ hooks/nova_cc_context.py	2015-10-05 06:51:47 +0000
@@ -409,3 +409,12 @@
                 ctxt['html5proxy_base_url'] = url
 
         return ctxt
+
+
+class APIRateLimitingContext(context.OSContextGenerator):
+    def __call__(self):
+        ctxt = {}
+        rate_rules = config('api-rate-limit-rules')
+        if rate_rules:
+            ctxt['api_rate_limit_rules'] = rate_rules
+        return ctxt

=== modified file 'hooks/nova_cc_utils.py'
--- hooks/nova_cc_utils.py	2015-09-25 17:20:51 +0000
+++ hooks/nova_cc_utils.py	2015-10-05 06:51:47 +0000
@@ -199,7 +199,8 @@
     }),
     (NOVA_API_PASTE, {
         'services': [s for s in BASE_SERVICES if 'api' in s],
-        'contexts': [nova_cc_context.IdentityServiceContext()],
+        'contexts': [nova_cc_context.IdentityServiceContext(),
+                     nova_cc_context.APIRateLimitingContext()],
     }),
     (QUANTUM_CONF, {
         'services': ['quantum-server'],

=== modified file 'templates/icehouse/etc_nova_api-paste.ini'
--- templates/icehouse/etc_nova_api-paste.ini	2014-04-12 21:47:13 +0000
+++ templates/icehouse/etc_nova_api-paste.ini	2015-10-05 06:51:47 +0000
@@ -85,6 +85,9 @@
 
 [filter:ratelimit]
 paste.filter_factory = nova.api.openstack.compute.limits:RateLimitingMiddleware.factory
+{% if api_rate_limit_rules -%}
+limits = {{ api_rate_limit_rules }}
+{% endif -%}
 
 [filter:sizelimit]
 paste.filter_factory = nova.api.sizelimit:RequestBodySizeLimiter.factory

=== modified file 'templates/kilo/etc_nova_api-paste.ini'
--- templates/kilo/etc_nova_api-paste.ini	2015-03-24 17:42:42 +0000
+++ templates/kilo/etc_nova_api-paste.ini	2015-10-05 06:51:47 +0000
@@ -104,6 +104,9 @@
 
 [filter:ratelimit]
 paste.filter_factory = nova.api.openstack.compute.limits:RateLimitingMiddleware.factory
+{% if api_rate_limit_rules -%}
+limits = {{ api_rate_limit_rules }}
+{% endif -%}
 
 [filter:sizelimit]
 paste.filter_factory = oslo.middleware:RequestBodySizeLimiter.factory

=== modified file 'tests/basic_deployment.py'
--- tests/basic_deployment.py	2015-07-23 03:02:42 +0000
+++ tests/basic_deployment.py	2015-10-05 06:51:47 +0000
@@ -66,7 +66,9 @@
 
     def _configure_services(self):
         """Configure all of the services."""
-        nova_cc_config = nova_config = {}
+        nova_cc_config = {}
+        nova_config = {}
+
         if self.git:
             amulet_http_proxy = os.environ.get('AMULET_HTTP_PROXY')
 
@@ -98,6 +100,10 @@
             }
             nova_cc_config['openstack-origin-git'] = yaml.dump(openstack_origin_git)
             nova_config['openstack-origin-git'] = yaml.dump(openstack_origin_git)
+
+        # Add some rate-limiting options to the charm. These will noop before
+        # icehouse.
+        nova_cc_config['api-rate-limit-rules'] = "( POST, '*', .*, 9999, MINUTE );"
         keystone_config = {'admin-password': 'openstack',
                            'admin-token': 'ubuntutesting'}
         configs = {'nova-cloud-controller': nova_cc_config,
@@ -647,3 +653,25 @@
 
         u.delete_image(self.glance, image)
         u.delete_instance(self.nova_demo, instance)
+
+    def test_api_rate_limiting_is_enabled_for_icehouse_or_more(self):
+        """
+        The API rate limiting is enabled for icehouse or more. Otherwise the
+        api-paste.ini file is left untouched.
+        """
+        unit = self.nova_cc_sentry
+        conf = '/etc/nova/api-paste.ini'
+        section = "filter:ratelimit"
+        factory = ("nova.api.openstack.compute.limits:RateLimitingMiddleware"
+                   ".factory")
+
+        if self._get_openstack_release() >= self.precise_icehouse:
+            expected = {"paste.filter_factory": factory,
+                        "limits": "( POST, '*', .*, 9999, MINUTE );"}
+        else:
+            expected = {"paste.filter_factory": factory}
+
+        ret = u.validate_config_data(unit, conf, section, expected)
+        if ret:
+            message = "api paste config error: {}".format(ret)
+            amulet.raise_status(amulet.FAIL, msg=message)