Integrate threat intel feeds (#669) (#690)
* add mapping for indices storing threat intel feed data
* fix feed indices mapping
* add threat intel feed data dao
* add threatIntelEnabled field in detector.
* add threat intel feed service and searching feeds
* ti feed data to doc level query convertor logic added
* plug threat intel feed into detector creation
* Preliminary framework for jobscheduler and datasource (#626)
* create doc level query from threat intel feed data index docs"
* handle threat intel enabled check during detector updation
* add tests for testing threat intel feed integration with detectors
* Threat intel feeds job runner and unit tests (#654)
* fix doc level query constructor (#651)
* add mapping for indices storing threat intel feed data
* fix feed indices mapping
* add threat intel feed data dao
* add threatIntelEnabled field in detector.
* add threat intel feed service and searching feeds
* ti feed data to doc level query convertor logic added
* plug threat intel feed into detector creation
* Preliminary framework for jobscheduler and datasource (#626)
* with listener and processor
* removed actions
* clean up
* added parser
* add unit tests
* refactored class names
* before moving db
* after moving db
* added actions to plugin and removed user schedule
* unit tests
* fix build error
* changed transport naming
---------
* converge job scheduler code with threat intel feed integration in detectors
* converge job scheduler and detector threat intel code
* add feed metadata config files in src and test
* adds ioc fields list in log type config files and ioc fields object in LogType POJO
* fix compilation issues in tests
* test udpate detector disabling threat intel
* add tests for detector creation and updation with threat intel
* Threat intel test (#673)
* add mapping for indices storing threat intel feed data
* fix feed indices mapping
* add threat intel feed data dao
* add threatIntelEnabled field in detector.
* add threat intel feed service and searching feeds
* ti feed data to doc level query convertor logic added
* plug threat intel feed into detector creation
* Preliminary framework for jobscheduler and datasource (#626)
* create doc level query from threat intel feed data index docs"
* handle threat intel enabled check during detector updation
* add tests for testing threat intel feed integration with detectors
* Threat intel feeds job runner and unit tests (#654)
* fix doc level query constructor (#651)
* add mapping for indices storing threat intel feed data
* fix feed indices mapping
* add threat intel feed data dao
* add threatIntelEnabled field in detector.
* add threat intel feed service and searching feeds
* ti feed data to doc level query convertor logic added
* plug threat intel feed into detector creation
* Preliminary framework for jobscheduler and datasource (#626)
* with listener and processor
* removed actions
* clean up
* added parser
* add unit tests
* refactored class names
* before moving db
* after moving db
* added actions to plugin and removed user schedule
* unit tests
* fix build error
* changed transport naming
---------
* converge job scheduler code with threat intel feed integration in detectors
* refactored out unecessary
* added headers and cleaned up
* converge job scheduler and detector threat intel code
* working on testing
* fixed the parser and build.gradle
* add mapping for indices storing threat intel feed data
* fix feed indices mapping
* add threat intel feed data dao
* add threatIntelEnabled field in detector.
* add threat intel feed service and searching feeds
* ti feed data to doc level query convertor logic added
* plug threat intel feed into detector creation
* Preliminary framework for jobscheduler and datasource (#626)
* create doc level query from threat intel feed data index docs"
* handle threat intel enabled check during detector updation
* add tests for testing threat intel feed integration with detectors
* Threat intel feeds job runner and unit tests (#654)
* fix doc level query constructor (#651)
* add mapping for indices storing threat intel feed data
* fix feed indices mapping
* add threat intel feed data dao
* add threatIntelEnabled field in detector.
* add threat intel feed service and searching feeds
* ti feed data to doc level query convertor logic added
* plug threat intel feed into detector creation
* Preliminary framework for jobscheduler and datasource (#626)
* with listener and processor
* removed actions
* clean up
* added parser
* add unit tests
* refactored class names
* before moving db
* after moving db
* added actions to plugin and removed user schedule
* unit tests
* fix build error
* changed transport naming
---------
* converge job scheduler code with threat intel feed integration in detectors
* converge job scheduler and detector threat intel code
* add feed metadata config files in src and test
* clean up some tests
* fixed merge conflicts
* adds ioc fields list in log type config files and ioc fields object in LogType POJO
* update csv parser and new metadata field
* fixed job scheduler interval settings
* add tests for ioc to fields for each log type
* removed wildcards
---------
* fix threat intel integ tests and add update detector logic
* JS for Threat intel feeds - changed extension (#675)
* merge conflicts
* fixed java wildcards and changed update key name
* integ test failing
* fix job scheduler params
* changed extension and has debug messages
* clean up
* fixed job scheduler plugin spi jar resolution
* cleaned up TODOs and changed job scheduler name
---------
* TIF Job Runner Cleanup (#676)
* merge conflicts
* fixed java wildcards and changed update key name
* integ test failing
* fix job scheduler params
* changed extension and has debug messages
* clean up
* fixed job scheduler plugin spi jar resolution
* cleaned up TODOs and changed job scheduler name
* removed google commons unused import, updated interval setting, removed rest action
* removed policy file and updated name for job scheduler
* responded to comments about parameter validator and TIFMetadata
* refactored ThreatIntelFeedDataService and changed variables to public static final where possible
* changed opensearch-sap-threatintel to opensearch-sap-threat-intel
---------
* fix TIFJobParameter class
* test detector updation when feed updation job runs
* removed delete job scheduler code and cleaned up (#678)
* working integ test (#680)
* fix timeout of tif job creation
* remove unncessary thread forking in put tif job action
* refactoring code to address review comments
* detector trigger detection types
* pull out threat intel rest tests into separate test class
* add detection types testing in detector trigger for rules and threat intel detection scenarios
* add license header
* add threat intel field aliases in mapping view response
* fix threat intel feed parser
* fix workflow failing test
* spotless check failures fixed
* remove dockerfile (#689)
---------
Signed-off-by: Surya Sashank Nistala <email address hidden>
Signed-off-by: Joanne Wang <email address hidden>
Signed-off-by: Joanne Wang <email address hidden>
Co-authored-by: Surya Sashank Nistala <email address hidden>
