Optimise data transfer between principle and worker procs
SearchResultMinimal is used to transder results between
the worker processes and the main collector. As a result
it must be a small as possible in order to keep transfer
fast and memory footprint low. There were several
unnecessary variables being stored and duplicated in this
object that have now been removed maing the transfer
faster and using less memory. Transfers are also batched
so as to reduce interruption on searches.
Also removes unnecssary use of multiprocessing.Queue for
single thread usecase.
When applying constraints to a single line, if we
are unable to we used to assume this meant a valid
line but the effect was that the searcher would
treat all subsequent lines as valid for that
searchdef which would lead to false positives. We
now treat this scenario as a pass *only* if it is
part of a set of constraints being applied to a
searchdef where *all* others have passed.
* reduces sixe of log messages to essential information
* cleanup code style consistency and docstring
* ensure first and last line checked first before
full bisect
searchkit/constraints: rewrite of binary search algorithm
Implemented a new binary search algorithm that no longer needs
filemarkers or knowing the lines beforehand, which reduces the
time spent applying a SearchConstraintSearchSince to a file,
especially if the file is large in size.
Removed the following classes which are no longer necessary:
Instead of only supporting "strict" mode and silently skipping
files that raise a UnicodeDecodeError we now raise the error
and add a new "decode_errors" kwarg to FileSearcher that supports
setting alternate handlers such as backslashescape, ignore etc.
