Merge ~lloydwaltersj/maas:oapi-linter into maas:master

Proposed by Jack Lloyd-Walters
Status: Merged
Approved by: Jack Lloyd-Walters
Approved revision: 1cbbc4976c25603e92dad0e557d623ebe889a17f
Merge reported by: MAAS Lander
Merged at revision: not available
Proposed branch: ~lloydwaltersj/maas:oapi-linter
Merge into: maas:master
Diff against target: 135 lines (+79/-2)
3 files modified
Makefile (+6/-1)
setup.cfg (+10/-1)
utilities/check-oapi (+63/-0)
Reviewer Review Type Date Requested Status
MAAS Lander Approve
Alexsander de Souza Approve
Adam Collard (community) Abstain
Review via email: mp+428646@code.launchpad.net

Commit message

Add a linter for the OpenAPI spec

Description of the change

add the ability to lint the MAAS Http api against OAS3

To post a comment you must log in.
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/355/consoleText
COMMIT: 94339310e45dd8f4e22296b5c0e2436ab1c66120

review: Needs Fixing
Revision history for this message
Alexsander de Souza (alexsander-souza) wrote :

use tox to handle the test environment

review: Needs Fixing
Revision history for this message
Adam Collard (adam-collard) wrote :

Don't think we should consider this a linter in the same way that we do the other linters.

It should be implemented as a test in the api package, alongside the API definition, documentation etc.

If this fails, it's not because we've used a deprecated way of writing Python, or sorted imports differently, it's a failure close to the OpenAPI feature.

review: Needs Fixing
Revision history for this message
Alexsander de Souza (alexsander-souza) wrote :

> If this fails, it's not because we've used a deprecated way of writing Python,
> or sorted imports differently, it's a failure close to the OpenAPI feature.

I think this is real linter, because if this fails it means that we failed to write valid documentation for whatever we are doing, not that the OpenAPI feature has a bug.

~lloydwaltersj/maas:oapi-linter updated
cd623b8... by Jack Lloyd-Walters

transition to tox

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/362/consoleText
COMMIT: cd623b889ddf406d2aef707052f647e22488f931

review: Needs Fixing
~lloydwaltersj/maas:oapi-linter updated
a90fe6e... by Jack Lloyd-Walters

change how oapi doccs are generated for linting

Revision history for this message
Jack Lloyd-Walters (lloydwaltersj) wrote (last edit ):

> STATUS: FAILED
> LOG: http://maas-ci.internal:8080/job/maas-tester/362/consoleText
> COMMIT: cd623b889ddf406d2aef707052f647e22488f931

This is working exactly as expected, a successfull failure if you will!

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/363/consoleText
COMMIT: a90fe6e49bee699aa470cd374526654e94bcab95

review: Needs Fixing
Revision history for this message
Jack Lloyd-Walters (lloydwaltersj) wrote :

jenkins: !test

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/392/consoleText
COMMIT: a90fe6e49bee699aa470cd374526654e94bcab95

review: Needs Fixing
~lloydwaltersj/maas:oapi-linter updated
39c02c3... by Jack Lloyd-Walters

bring branch up to parity

c0695f9... by Jack Lloyd-Walters

revert yaml.dump change

Revision history for this message
Adam Collard (adam-collard) :
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/405/consoleText
COMMIT: c0695f98b1e9ced2ccb37447682b5c59fd50bb1d

review: Needs Fixing
~lloydwaltersj/maas:oapi-linter updated
4eaea74... by Jack Lloyd-Walters

remove make docs from lint

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/407/consoleText
COMMIT: 4eaea745bfebe77675dc6e03ae1047f4b182122e

review: Needs Fixing
Revision history for this message
Adam Collard (adam-collard) :
~lloydwaltersj/maas:oapi-linter updated
2c01771... by Jack Lloyd-Walters

remove hardcoded openapi file location

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/438/consoleText
COMMIT: 2c01771792cbdf7cb63bed44851791f6f354df7b

review: Needs Fixing
Revision history for this message
Adam Collard (adam-collard) :
~lloydwaltersj/maas:oapi-linter updated
6247138... by Jack Lloyd-Walters

bring branch up to parity

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/455/consoleText
COMMIT: 6247138baf4652bcd8ae945d9e173e0da87ccc99

review: Needs Fixing
~lloydwaltersj/maas:oapi-linter updated
b55d865... by Jack Lloyd-Walters

bring branch up to parity

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/487/consoleText
COMMIT: b55d865e978a03383164554460790a26ffe24b55

review: Needs Fixing
~lloydwaltersj/maas:oapi-linter updated
8421bf8... by Jack Lloyd-Walters

generate spec if missing

bb8d96d... by Jack Lloyd-Walters

make linter depend on spec

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: 8421bf8f4b32f39fa8592cf821d836f6ba501b29

review: Approve
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: bb8d96d2bf145e94f54b57cb39b01216d3871108

review: Approve
Revision history for this message
Adam Collard (adam-collard) :
review: Approve
Revision history for this message
Adam Collard (adam-collard) :
review: Abstain
~lloydwaltersj/maas:oapi-linter updated
921393f... by Jack Lloyd-Walters

remove help supression

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/506/consoleText
COMMIT: 921393ffd0cc2f1eabb8a6444afc8b3a49869182

review: Needs Fixing
Revision history for this message
Jack Lloyd-Walters (lloydwaltersj) wrote :

jenkins: !test

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: 921393ffd0cc2f1eabb8a6444afc8b3a49869182

review: Approve
Revision history for this message
Alexsander de Souza (alexsander-souza) wrote :

+1

review: Approve
~lloydwaltersj/maas:oapi-linter updated
46f3269... by Jack Lloyd-Walters

remove lint-oapi from lint target

1cbbc49... by Jack Lloyd-Walters

move oapi linter to test phase

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b oapi-linter lp:~lloydwaltersj/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: 1cbbc4976c25603e92dad0e557d623ebe889a17f

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/Makefile b/Makefile
index 5a5c2af..903c5af 100644
--- a/Makefile
+++ b/Makefile
@@ -121,7 +121,7 @@ go-bins:
121 $(MAKE) -j -C src/host-info build121 $(MAKE) -j -C src/host-info build
122.PHONY: go-bins122.PHONY: go-bins
123123
124test: test-missing-migrations test-py124test: test-missing-migrations test-py lint-oapi
125.PHONY: test125.PHONY: test
126126
127test-missing-migrations: bin/database bin/maas-region127test-missing-migrations: bin/database bin/maas-region
@@ -166,6 +166,11 @@ lint-py-linefeeds:
166 (echo "Lint check failed; run make format to fix DOS linefeeds."; false)166 (echo "Lint check failed; run make format to fix DOS linefeeds."; false)
167.PHONY: lint-py-linefeeds167.PHONY: lint-py-linefeeds
168168
169# Open API Spec
170lint-oapi: openapi.yaml
171 @tox -e oapi
172.PHONY: lint-oapi
173
169# Go fmt174# Go fmt
170lint-go:175lint-go:
171 @find src/ \( -name pkg -o -name vendor \) -prune -o -name '*.go' -exec gofmt -l {} + | \176 @find src/ \( -name pkg -o -name vendor \) -prune -o -name '*.go' -exec gofmt -l {} + | \
diff --git a/setup.cfg b/setup.cfg
index 5d02873..c4c7faa 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -94,6 +94,7 @@ lint_files =
94 src/provisioningserver/refresh/50-maas-01-commissioning94 src/provisioningserver/refresh/50-maas-01-commissioning
95 utilities/bin2python.py95 utilities/bin2python.py
96 utilities/check-imports96 utilities/check-imports
97 utilities/check-oapi
97 utilities/import-db-dump98 utilities/import-db-dump
98 utilities/python_standard_libs.py99 utilities/python_standard_libs.py
99 utilities/snap-bind-mounts100 utilities/snap-bind-mounts
@@ -109,6 +110,9 @@ deps_lint =
109 cogapp == 3.3.0110 cogapp == 3.3.0
110 click == 8.1.3111 click == 8.1.3
111112
113deps_oapi =
114 openapi-spec-validator == 0.4.0
115
112[flake8]116[flake8]
113ignore = E203, E266, E501, W503, W504117ignore = E203, E266, E501, W503, W504
114exclude =118exclude =
@@ -116,7 +120,7 @@ exclude =
116120
117[tox:tox]121[tox:tox]
118skipsdist = True122skipsdist = True
119envlist = format,lint123envlist = format,lint,oapi
120124
121[testenv:format]125[testenv:format]
122deps = {[globals]deps_lint}126deps = {[globals]deps_lint}
@@ -132,3 +136,8 @@ commands =
132 black --check {[globals]lint_files}136 black --check {[globals]lint_files}
133 flake8 {[globals]lint_files}137 flake8 {[globals]lint_files}
134 cog --check --verbosity=1 {[globals]cog_files}138 cog --check --verbosity=1 {[globals]cog_files}
139
140[testenv:oapi]
141deps = {[globals]deps_oapi}
142commands =
143 python3 utilities/check-oapi openapi.yaml
135\ No newline at end of file144\ No newline at end of file
diff --git a/utilities/check-oapi b/utilities/check-oapi
136new file mode 100755145new file mode 100755
index 0000000..d77ae23
--- /dev/null
+++ b/utilities/check-oapi
@@ -0,0 +1,63 @@
1#!/usr/bin/env python3
2
3"""Script to ensure that an OpenAPI spec matches OAS3"""
4
5import argparse
6import sys
7
8from openapi_spec_validator import openapi_v3_spec_validator as validator
9import yaml
10
11failure = "\033[0;31m"
12success = "\033[0;32m"
13end = "\033[0m"
14
15
16def fetch_errors(errors):
17 """Return all the errors in the oapi spec with
18 some nice formatting for the terminal
19 """
20 num = len(errors)
21 pad = len(str(num))
22 err_ = [f"{num} problems with the spec:"]
23 for i, err in enumerate(errors):
24 msg, reason, loc = str(err).split("\n\n")
25 i = str(i).rjust(pad)
26 err_ += [
27 end
28 + f"{i}: {msg}\n{failure}{reason}\n{loc}".replace(
29 "\n", "\n" + " " * (2 + pad)
30 )
31 ]
32 return failure + "\n\n".join(err for err in err_) + end
33
34
35def validate(spec):
36 """a wrapper around the spec validator for our purposes"""
37 errors = list(validator.iter_errors(spec))
38 if errors:
39 error = fetch_errors(errors)
40 raise Exception(error)
41 else:
42 print(success + "No errors found in spec!" + end)
43
44
45def main():
46 parser = argparse.ArgumentParser(
47 description="Check an OpenAPI yaml schema against OAS3"
48 )
49 parser.add_argument(
50 "api_spec",
51 help="path for the file to validate against.",
52 type=argparse.FileType("r"),
53 )
54 args = parser.parse_args()
55 with args.api_spec as f:
56 spec = yaml.load(f, Loader=yaml.FullLoader)
57 info = spec["info"]
58 print(f"Checking {info['title']} version {info['version']}")
59 validate(spec)
60
61
62if __name__ == "__main__":
63 sys.exit(main())

Subscribers

People subscribed via source and target branches