feat: add type and descriptions to prometheus output (#4475)
The `/_node/_local/_prometheus` is a missing `TYPE` annotation for
`couchdb_httpd_status_codes`.
In addition, it contains no `HELP` annotations, which
are useful when exploring the metrics, particularly where
metrics do not strictly match those returned by the `_stats` or
`_system` endpoints.
This PR adds the missing `TYPE` annotation and adds `HELP` annotations
to all metrics.
It also adds additional spacing between the metrics series, making it
easier for humans to parse.
## couch_prometheus_util:to_prom/3
`couch_prometheus_util:to_prom/3` is replaced by `couch_prometheus_util:to_prom/4`.
which now expects a description alongside the metric name and type.
## couch_prometheus_util:couch_to_prom/3
`couch_prometheus_util:couch_to_prom/3` now extracts the metrics
description from the metric metadata returned by `couch_stats`.
In some cases, where the metrics are transformed e.g. from multiple
metrics to a single metric with a tag, the description is explicitly
specified to match the new metric semantics.
- Add more tests to check sandboxing resets, and that docs are "frozen".
- Remove the extra `\n` and `"` around function body lines. Erlang can do
multi-line binaries just fine. Mark the sections with %erlfmt-ignore to the
formatter doesn't complain.
- Generalize `should_create_sandbox` test to check for the `not defined` string
only. Experimenting with QuickJS noticed that it uses single quotes
around`'Object.foo' is not defined` and SM doesn't. So check for `not
defined` part only as it's obvious enough what the check is about.
- Make sure to return test procs back to the pool. Previously, none of the
tests returned the processes back into the pool, and when the tests ended,
they were forcibly killed which resulted in log noise that looked like:
```
erl_child_setup: failed with error 32 on line 265
erl_child_setup: failed with error 32 on line 265
...
```
`couch_prometheus_util:counter_metric/1` is intended to add a
`_total` suffix to the name of metrics with type `counter` if
the name does not already end with `_total`.
The implementation was previously incorrect, resulting in metrics
with names like `document_purges_total_total`.
This adds basic eunit tests for the failure / success scenarios and
fixes the implementation.
It is a breaking change for consumers of the `_node/_local/_prometheus`
endpoint, however, since the following metric is renamed:
