Currently, when POSTing to `/_session` with a Content-Type header
other than either `application/x-www-form-urlencoded` or
`application/json`, the error response can be surprising.
This changes the response to 415 `bad_content_type` when it's not one
of the above.
By now most of the CouchDB implementations support `_bulk_get`, so
let's update the replicator to take advantage of that.
To be backwards compatible assume some endpoints will not support
`_bulk_get` and may return either a 500 or 400 error. In that case the
replicator will fall back to fetching individual document revisions
like it did previously. For additional backward compatibility, and to
keep things simple, support only the `application/json` `_bulk_get`
response format. (Ideally, we'd send multiple Accept headers with
various `q` preference parameters for `json` and `multipart/related`
content, then do the right thing based on the response, however, none
of the recent Apache CouchDB implementations support that scheme
properly).
Since fetching attachments with application/json response is not
optimal, attachments are fetched individually. This means there are
two main reasons for the replicator to fall back to fetching
individual revisions: 1) when _bulk_get endpoint is not supported and
2) when the document revisions contain attachments.
To avoid wasting resource repeatedly attempting to use `_bulk_get `and
then falling back to individual doc fetches, maintain some historical
stats about the rate of failure, and if it crosses a threshold, skip
calling `_bulk_get` altogether. This is implemented with a moving
exponential average, along with periodic probing to see if `_bulk_get`
usage becomes viable again.
To give the users some indication about how successful `_bulk_get`
usage is, introduce two replication statistics parameters:
* `bulk_get_attempts`: _bulk_get document revisions attempts made.
* `bulk_get_docs` : `_bulk_get` document revisions successfully retrieved.
These are persisted in the replication checkpoints along with the rest
of the job statistics and visible in `_scheduler/jobs` and
`_active_tasks` output.
Since we updated the replication job statistics, perform some minor
cleanups in that area:
- Stop using the process dictionary for the reporting timestamp. Use
a regular record state field instead.
- Use casts instead of a calls when possible. We still rely on
report_seq_done calls as a synchronization point to make sure we
don't overrun the message queues for the replication worker and
scheduler job process.
- Add stats update API functions instead of relying on naked
`gen_server` calls and casts. The functions make it clear which
process is being updated: the replication worker or the main
replication scheduler job process.
For testing, rely on the variety of existing replication tests running
and passing. The recently merged replication test overhaul from [pull
the tests form using the node-local (back-end API) to chttpd (the
cluster API), which actually implements `_bulk_get`. In this way, the
majority of replication tests should test the `_bulk_get` API usage
alongside whatever else they are testing. There there is new test
checking that `_bulk_get` fallback works and testing the
characteristics of the new statistics parameters.
d4c7273...
by
Robert Newson <email address hidden>
Merge pull request #4041 from apache/draft_allow_nested_json_claim_roles
src/chttpd/test/eunit/chttpd_util_test.erl:33:48: Warning: variable '_Persist' is already bound. If you mean to ignore this value, use '_' or a different underscore-prefixed name
Create an explicit `Persist` variable set `false` to suppress those warnings.
The test doesn't check if the hash algorithm is supported by the
erlang vm. The test for supported hash algorithms was only missing
in the test itself and not in CouchDB.
Refactor test and verify hash names during test runs.
The values of the new config parameter is a list of comma-separated values of Erlang hash algorithms.
An example:
hash_algorithms = sha256, sha, md5
This line will use and generate new cookies with the sha256 hash algorithm and accept/verify cookies with the given hash algorithms sha256, sha and md5.
