Merge ~lloydwaltersj/maas:fix_oapi_get_config_description into maas:master
- Git
- lp:~lloydwaltersj/maas
- fix_oapi_get_config_description
- Merge into master
Status: | Merged |
---|---|
Approved by: | Jack Lloyd-Walters |
Approved revision: | 2e78d34bd2eee96abe4ceff798b675b7448f29bd |
Merge reported by: | MAAS Lander |
Merged at revision: | not available |
Proposed branch: | ~lloydwaltersj/maas:fix_oapi_get_config_description |
Merge into: | maas:master |
Diff against target: |
323 lines (+117/-36) 4 files modified
Makefile (+4/-1) src/maasserver/api/doc_oapi.py (+105/-27) src/maasserver/api/tests/test_oapi.py (+3/-7) src/maasserver/templates/openapi.html (+5/-1) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
MAAS Lander | Approve | ||
MAAS Maintainers | Pending | ||
Review via email: mp+451215@code.launchpad.net |
Commit message
Add formatting fix for https:/
Description of the change
Alexsander de Souza (alexsander-souza) : | # |
- dd40ddd... by Jack Lloyd-Walters
-
cleanup and respond to feedback
MAAS Lander (maas-lander) wrote : | # |
UNIT TESTS
-b fix_oapi_
STATUS: FAILED
LOG: http://
COMMIT: dd40ddd71042cb5
- eb20fea... by Jack Lloyd-Walters
-
update openapi template
Nick De Villiers (nickdv99) wrote : | # |
Once this has landed, I have a branch ready for maas.io that should fix the formatting on the frontend as well. Let me know once the YAML file we use for production has been updated
MAAS Lander (maas-lander) wrote : | # |
UNIT TESTS
-b fix_oapi_
STATUS: FAILED
LOG: http://
COMMIT: eb20fea54c621b9
- ada201d... by Jack Lloyd-Walters
-
linting
- af06abf... by Jack Lloyd-Walters
-
add offline docs to api page
- 485e6b2... by Jack Lloyd-Walters
-
add todo
- bb74991... by Jack Lloyd-Walters
-
add makefile targets
MAAS Lander (maas-lander) wrote : | # |
UNIT TESTS
-b fix_oapi_
STATUS: SUCCESS
COMMIT: 485e6b28a8a7843
MAAS Lander (maas-lander) wrote : | # |
UNIT TESTS
-b fix_oapi_
STATUS: SUCCESS
COMMIT: bb749919ff5dec2
Adam Collard (adam-collard) : | # |
- 2e78d34... by Jack Lloyd-Walters
-
fix makefile name
MAAS Lander (maas-lander) wrote : | # |
UNIT TESTS
-b fix_oapi_
STATUS: SUCCESS
COMMIT: 2e78d34bd2eee96
Preview Diff
1 | diff --git a/Makefile b/Makefile |
2 | index a22a714..4635d90 100644 |
3 | --- a/Makefile |
4 | +++ b/Makefile |
5 | @@ -259,7 +259,10 @@ api-docs.rst: bin/maas-region src/maasserver/api/doc_handler.py syncdb |
6 | openapi.yaml: bin/maas-region src/maasserver/api/doc_handler.py syncdb |
7 | bin/maas-region generate_oapi_spec > $@ |
8 | |
9 | -doc: api-docs.rst openapi.yaml swagger-css swagger-js |
10 | +openapi-doc: openapi.yaml swagger-css swagger-js |
11 | +.PHONY: openapi-doc |
12 | + |
13 | +doc: api-docs.rst openapi-doc |
14 | .PHONY: doc |
15 | |
16 | clean-ui: |
17 | diff --git a/src/maasserver/api/doc_oapi.py b/src/maasserver/api/doc_oapi.py |
18 | index 5ee4d55..e8e3f1b 100644 |
19 | --- a/src/maasserver/api/doc_oapi.py |
20 | +++ b/src/maasserver/api/doc_oapi.py |
21 | @@ -11,8 +11,10 @@ from inspect import getdoc, signature |
22 | import json |
23 | import re |
24 | from textwrap import dedent |
25 | +from typing import Any |
26 | |
27 | from django.http import HttpResponse |
28 | +from piston3.resource import Resource |
29 | import yaml |
30 | |
31 | from maasserver.api import support |
32 | @@ -29,8 +31,19 @@ PARAM_RE = re.compile( |
33 | r"^{(?P<param>\S+)}$", |
34 | ) |
35 | |
36 | +# https://github.com/canonical/maas.io/issues/806 |
37 | +# Match variables enclosed by :, ', ` within a docstring, variables cannot contain spaces or the |
38 | +# enclosing character within their definition |
39 | +MARKERS = [":", "'", "`"] |
40 | +MATCHES = [re.compile(rf" {k}[^ ^{k}]*{k} ") for k in MARKERS] |
41 | |
42 | -def landing_page(request): |
43 | +NEWLINES = re.compile(r"(?<![:.])[\n]+") |
44 | +WHITESPACE = re.compile(r"\n\s+") |
45 | +PUNCTUATION = re.compile(r"([^\w\s])\1+") |
46 | +POINTS = ["*", "+", "-"] |
47 | + |
48 | + |
49 | +def landing_page(request: str) -> HttpResponse: |
50 | """Render a landing page with pointers for the MAAS API. |
51 | |
52 | :return: An `HttpResponse` containing a JSON page with pointers to both |
53 | @@ -46,7 +59,7 @@ def landing_page(request): |
54 | ) |
55 | |
56 | |
57 | -def endpoint(request): |
58 | +def endpoint(request: str) -> HttpResponse: |
59 | """Render the OpenApi endpoint. |
60 | |
61 | :return: An `HttpResponse` containing a YAML document that complies |
62 | @@ -61,7 +74,7 @@ def endpoint(request): |
63 | ) |
64 | |
65 | |
66 | -def get_api_landing_page(): |
67 | +def get_api_landing_page() -> dict[str, str | Any]: |
68 | """Return the API landing page""" |
69 | description = { |
70 | "title": "MAAS API", |
71 | @@ -74,23 +87,29 @@ def get_api_landing_page(): |
72 | "title": "this document", |
73 | }, |
74 | { |
75 | + "path": "/MAAS/docs", |
76 | + "rel": "service-doc", |
77 | + "type": "text/html", |
78 | + "title": "offline MAAS documentation", |
79 | + }, |
80 | + { |
81 | "path": f"{settings.API_URL_PREFIX}openapi.yaml", |
82 | "rel": "service-desc", |
83 | "type": "application/openapi+yaml", |
84 | - "title": "the API definition", |
85 | + "title": "the OpenAPI definition", |
86 | }, |
87 | { |
88 | "path": "/MAAS/api/docs/", |
89 | "rel": "service-doc", |
90 | "type": "text/html", |
91 | - "title": "the API documentation", |
92 | + "title": "OpenAPI documentation", |
93 | }, |
94 | ], |
95 | } |
96 | return description |
97 | |
98 | |
99 | -def get_api_endpoint(): |
100 | +def get_api_endpoint() -> dict[str, str | Any]: |
101 | """Return the API endpoint""" |
102 | description = { |
103 | "openapi": "3.0.0", |
104 | @@ -112,7 +131,7 @@ def get_api_endpoint(): |
105 | return description |
106 | |
107 | |
108 | -def _get_maas_servers(): |
109 | +def _get_maas_servers() -> list[dict[str, str]]: |
110 | """Return a servers defintion of the public-facing MAAS address. |
111 | |
112 | :return: An object describing the MAAS public-facing server. |
113 | @@ -129,7 +148,7 @@ def _get_maas_servers(): |
114 | ] |
115 | |
116 | |
117 | -def _new_path_item(params): |
118 | +def _new_path_item(params: list[Any]) -> dict[str, dict[str, Any]]: |
119 | path_item = {} |
120 | for p in params: |
121 | path_item.setdefault("parameters", []).append( |
122 | @@ -143,17 +162,65 @@ def _new_path_item(params): |
123 | return path_item |
124 | |
125 | |
126 | -def _prettify(doc): |
127 | - """Cleans up text by replacing newlines with spaces, so that sentences are |
128 | - not broken apart prematurely. |
129 | - Respects paragraphing by not replacing newlines that occur after periods. |
130 | +def _prettify(doc: str) -> str: |
131 | + """Cleans up text by: |
132 | + - Dedenting text to the same depth. |
133 | + - Respecting paragraphing by not replacing newlines that occur after periods or colons |
134 | + - Removeing duplicate punctuation groups and replaces with singular |
135 | """ |
136 | - return re.sub("(?<![.\n])[\n]+", " ", dedent(doc)).strip() |
137 | + doc = dedent(doc) |
138 | + doc = NEWLINES.sub(" ", doc) |
139 | + doc = WHITESPACE.sub("\n", doc) |
140 | + doc = PUNCTUATION.sub(r"\1", doc) |
141 | + for idx, point in enumerate(POINTS): |
142 | + doc = re.sub(rf"\n\s*\{point}", f"\n{' '*2*idx}{point}", doc) |
143 | + return doc.strip() |
144 | + |
145 | + |
146 | +def _contains_variables(doc: str) -> list[str] | None: |
147 | + """Search for any instances of :variable:, ''variable'', or `variable`.""" |
148 | + for m in MATCHES: |
149 | + if (v := m.findall(doc[doc.find(":") :])) and len(v) > 1: |
150 | + return v |
151 | + |
152 | + |
153 | +def _parse_enumerable(doc: str) -> tuple[str, list[str]]: |
154 | + """Parse docstring for multiple variables. Clean up and represent as the correct |
155 | + form. (https://github.com/canonical/maas.io/issues/806)""" |
156 | + enumerable = {} |
157 | + if variables := _contains_variables(doc): |
158 | + for idx, var in enumerate(variables): |
159 | + start_pos = doc.find(var) + len(var) |
160 | + end_pos = ( |
161 | + len(doc) |
162 | + if idx >= len(variables) - 1 |
163 | + else doc.find(variables[idx + 1]) |
164 | + ) |
165 | + var_string = doc[start_pos:end_pos] |
166 | + doc = doc.replace(var + var_string, "") |
167 | + depth = MARKERS.index(var[1]) |
168 | + enumerable[var.strip(var[:2]) or " "] = { |
169 | + "description": _parse_enumerable(var_string)[0], |
170 | + "point": POINTS[depth], |
171 | + } |
172 | + doc = "\n".join( |
173 | + [f"{doc}"] |
174 | + + [ |
175 | + f"{v['point']} `{k}` {v['description'].strip('.,')}." |
176 | + for k, v in enumerable.items() |
177 | + ] |
178 | + ) |
179 | + return doc, list(enumerable.keys()) |
180 | |
181 | |
182 | def _render_oapi_oper_item( |
183 | - http_method, op, doc, uri_params, function, resources |
184 | -): |
185 | + http_method: str, |
186 | + op: str, |
187 | + doc: str, |
188 | + uri_params: Any, |
189 | + function: object, |
190 | + resources: set[Resource], |
191 | +) -> dict[str, str | Any]: |
192 | oper_id = op or support.OperationsResource.crudmap.get(http_method) |
193 | oper_obj = { |
194 | "operationId": f"{doc.name}_{oper_id}", |
195 | @@ -171,9 +238,13 @@ def _render_oapi_oper_item( |
196 | |
197 | |
198 | def _oapi_item_from_docstring( |
199 | - function, http_method, uri_params, doc, resources |
200 | -): |
201 | - def _type_to_string(schema): |
202 | + function: object, |
203 | + http_method: str, |
204 | + uri_params: Any, |
205 | + doc: str, |
206 | + resources: set[Resource], |
207 | +) -> dict[str, str | Any]: |
208 | + def _type_to_string(schema: str) -> str: |
209 | match schema: |
210 | case "Boolean": |
211 | return "boolean" |
212 | @@ -186,7 +257,7 @@ def _oapi_item_from_docstring( |
213 | case _: |
214 | return "object" |
215 | |
216 | - def _response_pair(ap_dict): |
217 | + def _response_pair(ap_dict: dict[str, str | Any]) -> list[str]: |
218 | status_code = "HTTP Status Code" |
219 | status = content = {} |
220 | paired = [] |
221 | @@ -215,11 +286,15 @@ def _oapi_item_from_docstring( |
222 | ap.parse(docstring) |
223 | ap_dict = ap.get_dict() |
224 | oper_obj["summary"] = ap_dict["description_title"].strip() |
225 | - oper_obj["description"] = _prettify(ap_dict["description"]) |
226 | + |
227 | + oper_obj["description"] = _prettify( |
228 | + _parse_enumerable(ap_dict["description"])[0] |
229 | + ) |
230 | + |
231 | if "deprecated" in oper_obj["description"].lower(): |
232 | oper_obj["deprecated"] = True |
233 | for param in ap_dict["params"]: |
234 | - description = _prettify(param["description_stripped"]) |
235 | + description = _parse_enumerable(param["description_stripped"])[0] |
236 | # LP 2009140 |
237 | stripped_name = PARAM_RE.match(param["name"]) |
238 | name = ( |
239 | @@ -236,7 +311,7 @@ def _oapi_item_from_docstring( |
240 | param_dict = { |
241 | "name": name, |
242 | "in": "path" if name in uri_params else "query", |
243 | - "description": description, |
244 | + "description": _prettify(description), |
245 | "schema": { |
246 | "type": _type_to_string(param["type"]), |
247 | }, |
248 | @@ -244,10 +319,11 @@ def _oapi_item_from_docstring( |
249 | } |
250 | oper_obj.setdefault("parameters", []).append(param_dict) |
251 | else: |
252 | - body.setdefault("properties", {})[name] = { |
253 | - "description": description, |
254 | + params_dict = { |
255 | + "description": _prettify(description), |
256 | "type": _type_to_string(param["type"]), |
257 | } |
258 | + body.setdefault("properties", {})[name] = params_dict |
259 | if required: |
260 | body.setdefault("required", []).append(name) |
261 | |
262 | @@ -336,13 +412,15 @@ def _oapi_item_from_docstring( |
263 | return oper_obj |
264 | |
265 | |
266 | -def _render_oapi_paths(): |
267 | +def _render_oapi_paths() -> dict[str, str | Any]: |
268 | from maasserver import urls_api as urlconf |
269 | |
270 | - def _resource_key(resource): |
271 | + def _resource_key(resource: Resource) -> str: |
272 | return resource.handler.__class__.__name__ |
273 | |
274 | - def _export_key(export): |
275 | + def _export_key( |
276 | + export: tuple[tuple[str, str], object] |
277 | + ) -> tuple[str, object]: |
278 | (http_method, op), function = export |
279 | return http_method, op or "", function |
280 | |
281 | diff --git a/src/maasserver/api/tests/test_oapi.py b/src/maasserver/api/tests/test_oapi.py |
282 | index d22c2cc..c28e18d 100644 |
283 | --- a/src/maasserver/api/tests/test_oapi.py |
284 | +++ b/src/maasserver/api/tests/test_oapi.py |
285 | @@ -138,12 +138,8 @@ represented in ASCII using ``bsondump example.bson`` and is for |
286 | demonstrative purposes.""" |
287 | |
288 | after = """\ |
289 | -Returns system details -- for example, LLDP and ``lshw`` XML dumps. |
290 | - |
291 | - |
292 | -Returns a ``{detail_type: xml, ...}`` map, where ``detail_type`` is something like "lldp" or "lshw". |
293 | - |
294 | - |
295 | -Note that this is returned as BSON and not JSON. This is for efficiency, but mainly because JSON can''t do binary content without applying additional encoding like base-64. The example output below is represented in ASCII using ``bsondump example.bson`` and is for demonstrative purposes.""" |
296 | +Returns system details - for example, LLDP and `lshw` XML dumps. |
297 | +Returns a `{detail_type: xml, .}` map, where `detail_type` is something like "lldp" or "lshw". |
298 | +Note that this is returned as BSON and not JSON. This is for efficiency, but mainly because JSON can't do binary content without applying additional encoding like base-64. The example output below is represented in ASCII using `bsondump example.bson` and is for demonstrative purposes.""" |
299 | |
300 | self.assertEqual(_prettify(before), after) |
301 | diff --git a/src/maasserver/templates/openapi.html b/src/maasserver/templates/openapi.html |
302 | index 4b75523..23186c8 100644 |
303 | --- a/src/maasserver/templates/openapi.html |
304 | +++ b/src/maasserver/templates/openapi.html |
305 | @@ -4,7 +4,8 @@ |
306 | <head> |
307 | <meta charset="UTF-8"> |
308 | <title>MAAS API</title> |
309 | - <link href="/MAAS/r/static/css/main.a4c6517c.css" rel="stylesheet"> |
310 | + <!-- TODO: Figure out how to update this programatically --> |
311 | + <link href="/MAAS/r/static/css/main.d1c59af9.css" rel="stylesheet"> |
312 | <style> |
313 | {% include "dist/swagger-ui.css" %} |
314 | </style> |
315 | @@ -21,6 +22,9 @@ |
316 | margin: 0 !important; |
317 | max-width: 100% !important; |
318 | } |
319 | + .swagger-ui .parameters-col_description { |
320 | + width: 85% !important |
321 | + } |
322 | </style> |
323 | </head> |
324 |
UNIT TESTS get_config_ description lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas
-b fix_oapi_
STATUS: SUCCESS 06227a22973de28 048c919be1
COMMIT: b69a16ef1a76364