Merge lp:~tribaal/charms/trusty/nova-compute/enable-api-rate-limiting into lp:~openstack-charmers-archive/charms/trusty/nova-compute/next

Proposed by Chris Glass on 2015-09-30
Status: Merged
Merged at revision: 164
Proposed branch: lp:~tribaal/charms/trusty/nova-compute/enable-api-rate-limiting
Merge into: lp:~openstack-charmers-archive/charms/trusty/nova-compute/next
Diff against target: 246 lines (+182/-1)
5 files modified
config.yaml (+10/-0)
hooks/nova_compute_context.py (+9/-0)
hooks/nova_compute_utils.py (+5/-0)
templates/api-paste.ini (+141/-0)
tests/basic_deployment.py (+17/-1)
To merge this branch: bzr merge lp:~tribaal/charms/trusty/nova-compute/enable-api-rate-limiting
Reviewer Review Type Date Requested Status
Liam Young 2015-09-30 Approve on 2015-10-01
Review via email: mp+272884@code.launchpad.net

Description of the change

This branch introduces a new api-rate-limit-rules charm config option controlling the rate-limiting
of the nova API by writing the passed value to the /etc/nova/api-paste.ini file.

To post a comment you must log in.
165. By Chris Glass on 2015-09-30

Forgot to check the output.

charm_lint_check #11089 nova-compute-next for tribaal mp272884
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/11089/

charm_unit_test #10295 nova-compute-next for tribaal mp272884
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/10295/

charm_lint_check #11090 nova-compute-next for tribaal mp272884
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/11090/

charm_unit_test #10297 nova-compute-next for tribaal mp272884
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/10297/

charm_amulet_test #6896 nova-compute-next for tribaal mp272884
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12625830/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6896/

charm_lint_check #11151 nova-compute-next for tribaal mp272884
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/11151/

Liam Young (gnuoy) wrote :

nova-compute install hook fails on ./tests/014-basic-precise-icehouse amulet test.

0 upgraded, 0 newly installed, 0 to remove and 19 not upgraded.
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/install.real", line 434, in <module>
    main()
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/install.real", line 428, in main
    hooks.execute(sys.argv)
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/core/hookenv.py", line 672, in execute
    self._hooks[hook_name]()
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 777, in wrapped_f
    set_os_workload_status(configs, required_interfaces, charm_func)
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 790, in set_os_workload_status
    incomplete_rel_data = incomplete_relation_data(configs, required_interfaces)
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 920, in incomplete_relation_data
    complete_ctxts = configs.complete_contexts()
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 295, in complete_contexts
    for i in six.itervalues(self.templates)]
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 120, in complete_contexts
    self.context()
  File "/var/lib/juju/agents/unit-nova-compute-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 107, in context
    ctxt.update(_ctxt)
TypeError: 'APIRateLimitingContext' object is not iterable

review: Needs Fixing
166. By Chris Glass on 2015-10-01

Fixing iterable idiocy.

Liam Young (gnuoy) wrote :

Manually ran amulet for trusty and precise. All good! Approved

review: Approve

charm_amulet_test #6926 nova-compute-next for tribaal mp272884
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/12633828/
Build: http://10.245.162.77:8080/job/charm_amulet_test/6926/

Ryan Beisner (1chb1n) wrote :

fwiw ^ The re-run (with test enviro looking sound) says precise-icehouse didn't have nova-api service started on the nova-compute/0 unit.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'config.yaml'
2--- config.yaml 2015-09-28 07:56:16 +0000
3+++ config.yaml 2015-10-01 15:08:20 +0000
4@@ -265,3 +265,13 @@
5 description: |
6 The pecentage of system memory to use for hugepages eg '10%' or the total
7 number of 2M hugepages - eg "1024".
8+ api-rate-limit-rules:
9+ type: string
10+ default:
11+ description: |
12+ The API rate-limit rules to use for the deployed nova API, if any.
13+ Contents of this config options will be inserted in the api-paste.ini file
14+ under the "filter:ratelimit" section as "limits". The syntax for these
15+ rules is documented at
16+ http://docs.openstack.org/kilo/config-reference/content/configuring-compute-API.html
17+
18
19=== modified file 'hooks/nova_compute_context.py'
20--- hooks/nova_compute_context.py 2015-09-28 16:28:13 +0000
21+++ hooks/nova_compute_context.py 2015-10-01 15:08:20 +0000
22@@ -496,3 +496,12 @@
23 ctxt['host_ip'] = host_ip
24
25 return ctxt
26+
27+
28+class APIRateLimitingContext(context.OSContextGenerator):
29+ def __call__(self):
30+ ctxt = {}
31+ rate_rules = config('api-rate-limit-rules')
32+ if rate_rules:
33+ ctxt['api_rate_limit_rules'] = rate_rules
34+ return ctxt
35
36=== modified file 'hooks/nova_compute_utils.py'
37--- hooks/nova_compute_utils.py 2015-09-29 15:14:35 +0000
38+++ hooks/nova_compute_utils.py 2015-10-01 15:08:20 +0000
39@@ -74,6 +74,7 @@
40 CEPH_CONF,
41 ceph_config_file,
42 HostIPContext,
43+ APIRateLimitingContext
44 )
45
46 CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'
47@@ -150,6 +151,7 @@
48 LIBVIRT_BIN = '/etc/default/libvirt-bin'
49 LIBVIRT_BIN_OVERRIDES = '/etc/init/libvirt-bin.override'
50 NOVA_CONF = '%s/nova.conf' % NOVA_CONF_DIR
51+API_PASTE_INI = '%s/api-paste.ini' % NOVA_CONF_DIR
52
53 BASE_RESOURCE_MAP = {
54 NOVA_CONF: {
55@@ -175,6 +177,9 @@
56 HostIPContext(),
57 context.LogLevelContext()],
58 },
59+ API_PASTE_INI: {
60+ 'services': ['nova-compute'],
61+ 'contexts': [APIRateLimitingContext()]}
62 }
63
64 LIBVIRT_RESOURCE_MAP = {
65
66=== added file 'templates/api-paste.ini'
67--- templates/api-paste.ini 1970-01-01 00:00:00 +0000
68+++ templates/api-paste.ini 2015-10-01 15:08:20 +0000
69@@ -0,0 +1,141 @@
70+###############################################################################
71+# [ WARNING ]
72+# Configuration file maintained by Juju. Local changes may be overwritten.
73+###############################################################################
74+############
75+# Metadata #
76+############
77+[composite:metadata]
78+use = egg:Paste#urlmap
79+/: meta
80+
81+[pipeline:meta]
82+pipeline = ec2faultwrap logrequest metaapp
83+
84+[app:metaapp]
85+paste.app_factory = nova.api.metadata.handler:MetadataRequestHandler.factory
86+
87+#######
88+# EC2 #
89+#######
90+
91+[composite:ec2]
92+use = egg:Paste#urlmap
93+/: ec2cloud
94+
95+[composite:ec2cloud]
96+use = call:nova.api.auth:pipeline_factory
97+noauth = ec2faultwrap logrequest ec2noauth cloudrequest validator ec2executor
98+noauth2 = ec2faultwrap logrequest ec2noauth cloudrequest validator ec2executor
99+keystone = ec2faultwrap logrequest ec2keystoneauth cloudrequest validator ec2executor
100+
101+[filter:ec2faultwrap]
102+paste.filter_factory = nova.api.ec2:FaultWrapper.factory
103+
104+[filter:logrequest]
105+paste.filter_factory = nova.api.ec2:RequestLogging.factory
106+
107+[filter:ec2lockout]
108+paste.filter_factory = nova.api.ec2:Lockout.factory
109+
110+[filter:ec2keystoneauth]
111+paste.filter_factory = nova.api.ec2:EC2KeystoneAuth.factory
112+
113+[filter:ec2noauth]
114+paste.filter_factory = nova.api.ec2:NoAuth.factory
115+
116+[filter:cloudrequest]
117+controller = nova.api.ec2.cloud.CloudController
118+paste.filter_factory = nova.api.ec2:Requestify.factory
119+
120+[filter:authorizer]
121+paste.filter_factory = nova.api.ec2:Authorizer.factory
122+
123+[filter:validator]
124+paste.filter_factory = nova.api.ec2:Validator.factory
125+
126+[app:ec2executor]
127+paste.app_factory = nova.api.ec2:Executor.factory
128+
129+#############
130+# OpenStack #
131+#############
132+
133+[composite:osapi_compute]
134+use = call:nova.api.openstack.urlmap:urlmap_factory
135+/: oscomputeversions
136+/v1.1: openstack_compute_api_v2
137+/v2: openstack_compute_api_v2
138+/v2.1: openstack_compute_api_v21
139+/v3: openstack_compute_api_v3
140+
141+[composite:openstack_compute_api_v2]
142+use = call:nova.api.auth:pipeline_factory
143+noauth = compute_req_id faultwrap sizelimit noauth ratelimit osapi_compute_app_v2
144+noauth2 = compute_req_id faultwrap sizelimit noauth2 ratelimit osapi_compute_app_v2
145+keystone = compute_req_id faultwrap sizelimit authtoken keystonecontext ratelimit osapi_compute_app_v2
146+keystone_nolimit = compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v2
147+
148+[composite:openstack_compute_api_v21]
149+use = call:nova.api.auth:pipeline_factory_v21
150+noauth = compute_req_id faultwrap sizelimit noauth osapi_compute_app_v21
151+noauth2 = compute_req_id faultwrap sizelimit noauth2 osapi_compute_app_v21
152+keystone = compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v21
153+
154+[composite:openstack_compute_api_v3]
155+use = call:nova.api.auth:pipeline_factory_v21
156+noauth = request_id faultwrap sizelimit noauth_v3 osapi_compute_app_v3
157+noauth2 = request_id faultwrap sizelimit noauth_v3 osapi_compute_app_v3
158+keystone = request_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v3
159+
160+[filter:request_id]
161+paste.filter_factory = oslo.middleware:RequestId.factory
162+
163+[filter:compute_req_id]
164+paste.filter_factory = nova.api.compute_req_id:ComputeReqIdMiddleware.factory
165+
166+[filter:faultwrap]
167+paste.filter_factory = nova.api.openstack:FaultWrapper.factory
168+
169+[filter:noauth]
170+paste.filter_factory = nova.api.openstack.auth:NoAuthMiddlewareOld.factory
171+
172+[filter:noauth2]
173+paste.filter_factory = nova.api.openstack.auth:NoAuthMiddleware.factory
174+
175+[filter:noauth_v3]
176+paste.filter_factory = nova.api.openstack.auth:NoAuthMiddlewareV3.factory
177+
178+[filter:ratelimit]
179+paste.filter_factory = nova.api.openstack.compute.limits:RateLimitingMiddleware.factory
180+{% if api_rate_limit_rules -%}
181+limits = {{ api_rate_limit_rules}}
182+{% endif -%}
183+
184+[filter:sizelimit]
185+paste.filter_factory = oslo.middleware:RequestBodySizeLimiter.factory
186+
187+[app:osapi_compute_app_v2]
188+paste.app_factory = nova.api.openstack.compute:APIRouter.factory
189+
190+[app:osapi_compute_app_v21]
191+paste.app_factory = nova.api.openstack.compute:APIRouterV21.factory
192+
193+[app:osapi_compute_app_v3]
194+paste.app_factory = nova.api.openstack.compute:APIRouterV3.factory
195+
196+[pipeline:oscomputeversions]
197+pipeline = faultwrap oscomputeversionapp
198+
199+[app:oscomputeversionapp]
200+paste.app_factory = nova.api.openstack.compute.versions:Versions.factory
201+
202+##########
203+# Shared #
204+##########
205+
206+[filter:keystonecontext]
207+paste.filter_factory = nova.api.auth:NovaKeystoneContext.factory
208+
209+[filter:authtoken]
210+paste.filter_factory = keystonemiddleware.auth_token:filter_factory
211
212=== modified file 'tests/basic_deployment.py'
213--- tests/basic_deployment.py 2015-07-21 18:02:54 +0000
214+++ tests/basic_deployment.py 2015-10-01 15:08:20 +0000
215@@ -68,7 +68,9 @@
216 def _configure_services(self):
217 """Configure all of the services."""
218 nova_config = {'config-flags': 'auto_assign_floating_ip=False',
219- 'enable-live-migration': 'False'}
220+ 'enable-live-migration': 'False',
221+ 'api-rate-limit-rules':
222+ '( POST, *, .*, 9999, MINUTE );'}
223 nova_cc_config = {}
224 if self.git:
225 amulet_http_proxy = os.environ.get('AMULET_HTTP_PROXY')
226@@ -466,6 +468,20 @@
227 message = "nova config error: {}".format(ret)
228 amulet.raise_status(amulet.FAIL, msg=message)
229
230+ def test_api_paste_config(self):
231+ """Check that the rate limiting is set on the nova api (for POSTs)."""
232+ unit = self.nova_compute_sentry
233+ conf = '/etc/nova/api-paste.ini'
234+ section = "filter:ratelimit"
235+ factory = ("nova.api.openstack.compute.limits:RateLimitingMiddleware"
236+ ".factory")
237+ expected = {"paste.filter_factory": factory,
238+ "limits": "( POST, *, .*, 9999, MINUTE );"}
239+ ret = u.validate_config_data(unit, conf, section, expected)
240+ if ret:
241+ message = "api paste config error: {}".format(ret)
242+ amulet.raise_status(amulet.FAIL, msg=message)
243+
244 def test_image_instance_create(self):
245 """Create an image/instance, verify they exist, and delete them."""
246 # NOTE(coreycb): Skipping failing test on essex until resolved. essex

Subscribers

People subscribed via source and target branches