Author: Edward Hope-Morley
Author Date: 2023-05-27 11:18:43 UTC

Fixes case where search paths overlap

If the paths used to register searches overlap once
expanded, they will cause the same file to be searched
concurrently which breaks the MPCache and is also
superfluous. This patch fixes that problem and applies
some minor optimisations to the way we extract
datetime from start of line to apply constraints.

Also fixes support for applying constraints to files
containing unicode characters by ensuring that we escape
rather than decode those charaters.

Author: Edward Hope-Morley
Author Date: 2023-05-27 11:18:43 UTC

Fixes case where search paths overlap

If the paths used to register searches overlap once
expanded, they will cause the same file to be searched
concurrently which breaks the MPCache and is also
superfluous. This patch fixes that problem and applies
some minor optimisations to the way we extract
datetime from start of line to apply constraints.

Also fixes support for applying constraints to files
containing unicode characters by ensuring that we escape
rather than decode those charaters.

Author: Edward Hope-Morley
Author Date: 2023-05-25 12:59:55 UTC

Display more stats in logs

Author: Edward Hope-Morley
Author Date: 2023-05-21 16:51:51 UTC

Add constraints timestamp matcher class

Author: Edward Hope-Morley
Author Date: 2023-05-11 14:07:04 UTC

Implement more efficient file indexing

The constraints engine indexes files
to get start of line pointers used in the binary
search. This is an expensive task for large files so
we ensure that we hold as little as possible in
memory and try to obtain line positions as fast as
possible. This is achieved with the addition of a
new MPCacheSharded class and faster line ending
lookup with reduced reliance on tell/seek.

Author: Edward Hope-Morley
Author Date: 2023-05-03 18:59:08 UTC

Reduce memory footprint of searches

Result search group values are now stored in a central index
so that identical values are only stored once and retreived
by reference. Also reduces the size of the result object send
back to the main process via results queue which greatly
reduces the time it takes the send/receive it.

We now use the shelve library to manage the MPCache underlying
key value store which reduces complexity and improves
performance (although there are more improvements that can be
done but will addresses in seperate patches).

Author: Edward Hope-Morley
Author Date: 2023-03-30 12:42:23 UTC

Support result field names

Result groups are by default retrieved using their numeric group
index. This patch adds support for retrieving them using string
names and optionally casting their value to a specific type.

Author: Edward Hope-Morley
Author Date: 2023-04-10 08:44:16 UTC

Update readme

Author: Edward Hope-Morley
Author Date: 2023-04-01 15:22:20 UTC

Use concurrent.futures.ProcessPoolExecutor

This is the second attempt at using this model. Also
makes some improvements such as:

* no more global queue index
* managed queue used for results
* better logging
* ensures pool worker processes are terminated

Author: Edward Hope-Morley
Author Date: 2023-03-30 21:12:04 UTC

Bump version

Author: Edward Hope-Morley
Author Date: 2023-03-28 13:49:12 UTC

Sequence search code cleanup

Cleans up the code used to perform sequence searches and
improves docstrings. Also extends SearchResultsCollection
with a new call to allow sequence results lookup using
tags rather than requiring the sequence object id.

Author: Edward Hope-Morley
Author Date: 2023-03-11 15:26:09 UTC

Use separate file for each key stored in MPCache

The MPCache will now store each key as a separate file
by default and can optionally be switched to store all
keys in a single file.

Resolves: #1

Author: Edward Hope-Morley
Author Date: 2023-01-16 20:56:28 UTC

Improve MP implementation

Improves the code around multiprocessing to be more
efficient and safer. Also generally refactors and
cleans up the code.