~data-platform/opensearch-project-components/+git/opensearch-index-management:lp-2.8.0

Last commit made on 2023-08-14
Get this branch:
git clone -b lp-2.8.0 https://git.launchpad.net/~data-platform/opensearch-project-components/+git/opensearch-index-management
Members of Canonical Data Platform can upload to this branch. Log in for directions.

Branch merges

Recent commits

823222d... by Mehdi B.

changed gradle distro url

93c5af1... by Mehdi B.

changed gradle distro url

c611ac0... by "opensearch-trigger-bot[bot]" <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>

Bugfix/202 transform date add date conversion (#622) (#804)

* 202: Added format property when specifying the date histogram

Signed-off-by: Stevan Buzejic <email address hidden>

* 202: Added component responsible for building the target index mapping once the transform is being triggered.

Signed-off-by: Stevan Buzejic <email address hidden>

* 202: date_histogram considered in the case of the creating the target index for the date fields when transform is executed

Signed-off-by: Stevan Buzejic <email address hidden>

* 202: Enabled target index date field mappings if those fields are used in aggregations or as a term aggregation for defining the buckets

Signed-off-by: Stevan Buzejic <email address hidden>

* Updated code according to comments. Added targetIndexMapping when transform preview action is triggered

Signed-off-by: Stevan Buzejic <email address hidden>

* Updated schema versions

Signed-off-by: Stevan Buzejic <email address hidden>

* Addressed the comments

Signed-off-by: Stevan Buzejic <email address hidden>

* Refactored transform tests related with aggregation based on a date field.
Updated transform preview action to consider target index mapping when using a date field. Kept formatting of the date field in target index.

Signed-off-by: Stevan Buzejic <email address hidden>

* detekt fix

Signed-off-by: Stevan Buzejic <email address hidden>

* Added zone in IT

Signed-off-by: Stevan Buzejic <email address hidden>

* Added function for creating target index mapping that considers transform mapping json. Target date field mappings are generated after transform validation when running transform. Removed target index date field values formatting. emoved default format for date_histogram because of the rollup. Updated schema version in test.

Signed-off-by: Stevan Buzejic <email address hidden>

---------

Signed-off-by: Stevan Buzejic <email address hidden>
(cherry picked from commit 42833b12b292a291f56bc4f94264d32f33317bf2)

Co-authored-by: Stevan Buzejic <email address hidden>

5c42012... by "opensearch-trigger-bot[bot]" <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>

Use original PR numbers in 2.8 release notes. (#796) (#798)

* Use original PR numbers in 2.8 release notes.

Signed-off-by: Xuesong Luo <email address hidden>

---------

Signed-off-by: Xuesong Luo <email address hidden>
(cherry picked from commit 104c4b8ef4de78ac4cba3be58dcfb2d23aaedc1c)

Co-authored-by: Xuesong Luo <email address hidden>

168073e... by "opensearch-trigger-bot[bot]" <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>

Add 2.8 release notes (#794) (#795)

* Add 2.8 release notes

Signed-off-by: Xuesong Luo <email address hidden>
(cherry picked from commit d921f0be3eb20dd92cea5af2c2eeebd73017b508)

Co-authored-by: Xuesong Luo <email address hidden>

2c8866b... by "opensearch-trigger-bot[bot]" <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>

[Backport 2.x] Feature/index operation notification (#793)

* Feature/index operation notification (#712)

* Feature/index operation notification (#707)

* task completion listener for long running task

Signed-off-by: Hailong Cui <email address hidden>

* task completion listener for long running task

Signed-off-by: Hailong Cui <email address hidden>

* remove unused import

Signed-off-by: Hailong Cui <email address hidden>

---------

Signed-off-by: Hailong Cui <email address hidden>

* index action, no validation&test

Signed-off-by: zhichao-aws <email address hidden>

get action. no validation&test

Signed-off-by: zhichao-aws <email address hidden>

index and get action (draft)

changes for rebase main

Signed-off-by: zhichao-aws <email address hidden>

index and get, no validation&test

Signed-off-by: zhichao-aws <email address hidden>

* refactor on package name

Signed-off-by: Hailong Cui <email address hidden>

* +delete, change index

Signed-off-by: zhichao-aws <email address hidden>

* Implement notification with ActionFilter

Signed-off-by: Hailong Cui <email address hidden>

* Query notification channel

Signed-off-by: Hailong Cui <email address hidden>

* Notification message update

Signed-off-by: Hailong Cui <email address hidden>

* new version of crud

Signed-off-by: zhichao-aws <email address hidden>

* validate task id and action name; unitfy taskId

Signed-off-by: zhichao-aws <email address hidden>

* Add unit test case

Signed-off-by: Hailong Cui <email address hidden>

* refactor on response parser

Signed-off-by: Hailong Cui <email address hidden>

* fix unit test case

Signed-off-by: Hailong Cui <email address hidden>

* modify code structure, change ID to Id

Signed-off-by: zhichao-aws <email address hidden>

* add some test, delete message template

Signed-off-by: zhichao-aws <email address hidden>

* Add integration test

Signed-off-by: Hailong Cui <email address hidden>

* Add integration test for action listener

Signed-off-by: Hailong Cui <email address hidden>

* remove refresh policy in request

Signed-off-by: zhichao-aws <email address hidden>

* add some tests, fix actionfilter refresh parameter

Signed-off-by: zhichao-aws <email address hidden>

* Remove duplicate errors for reindex

Signed-off-by: Hailong Cui <email address hidden>

* add UT and IT

Signed-off-by: zhichao-aws <email address hidden>

* fix some code style

Signed-off-by: zhichao-aws <email address hidden>

* enable com.sun.net.httpserver.* for test

Signed-off-by: Hailong Cui <email address hidden>

* change index name, fix bug

Signed-off-by: zhichao-aws <email address hidden>

* modifications after meeting

Signed-off-by: zhichao-aws <email address hidden>

* don't continue wait for shard to be started for async mode

Signed-off-by: Hailong Cui <email address hidden>

* handle IndexNotFoundException for sending notification

Signed-off-by: Hailong Cui <email address hidden>

* add security test case

Signed-off-by: zhichao-aws <email address hidden>

* add security test case

Signed-off-by: zhichao-aws <email address hidden>

* refactor on file names

Signed-off-by: Hailong Cui <email address hidden>

* add success/failure condition

Signed-off-by: zhichao-aws <email address hidden>

* filter channels by conditions

Signed-off-by: Hailong Cui <email address hidden>

* Remove unused field enable for index .opensearch-admin-panel

Signed-off-by: Hailong Cui <email address hidden>

* add more integration testcase

Signed-off-by: Hailong Cui <email address hidden>

* change admin panel to control center

Signed-off-by: zhichao-aws <email address hidden>

* Update mock server address to localhost for docker env

Signed-off-by: Hailong Cui <email address hidden>

* remove unused field in response

Signed-off-by: zhichao-aws <email address hidden>

* update integ test

Signed-off-by: Hailong Cui <email address hidden>

* remove default for all

Signed-off-by: zhichao-aws <email address hidden>

* Remove default for all

Signed-off-by: Hailong Cui <email address hidden>

* remove filter by backend roles

Signed-off-by: zhichao-aws <email address hidden>

* fix build

Signed-off-by: zhichao-aws <email address hidden>

* optimize constructor code

Signed-off-by: zhichao-aws <email address hidden>

* add request doc id validation

Signed-off-by: zhichao-aws <email address hidden>

* merge get and search

Signed-off-by: zhichao-aws <email address hidden>

* remove get before delete

Signed-off-by: zhichao-aws <email address hidden>

* add node id validation

Signed-off-by: zhichao-aws <email address hidden>

* auto create doc for update

Signed-off-by: zhichao-aws <email address hidden>

* fix test case

Signed-off-by: zhichao-aws <email address hidden>

* delete restUpdateAction, fix test

Signed-off-by: zhichao-aws <email address hidden>

* add dry run option

Signed-off-by: zhichao-aws <email address hidden>

* fix bug, add test case

Signed-off-by: zhichao-aws <email address hidden>

* add xcontent tests

Signed-off-by: zhichao-aws <email address hidden>

* notification crud api

Signed-off-by: zhichao-aws <email address hidden>

* changes based on comments; add tests

Signed-off-by: zhichao-aws <email address hidden>

* changes for comment

Signed-off-by: zhichao-aws <email address hidden>

* change behavior for get action index not exists, fix test cases

Signed-off-by: zhichao-aws <email address hidden>

* fix merge

Signed-off-by: zhichao-aws <email address hidden>

* delay 5s for sending runtime notification

Signed-off-by: Hailong Cui <email address hidden>

* update log message

Signed-off-by: Hailong Cui <email address hidden>

* delay 5s for runtime notification policy

Signed-off-by: Hailong Cui <email address hidden>

* Update notification message

Signed-off-by: Hailong Cui <email address hidden>

* filter duplicate channel

Signed-off-by: Hailong Cui <email address hidden>

* exception handling

Signed-off-by: Hailong Cui <email address hidden>

* Add cluster name into title

Signed-off-by: Hailong Cui <email address hidden>

* fix UT for failure messagee

Signed-off-by: Hailong Cui <email address hidden>

* include cluster name into notification messag

Signed-off-by: Hailong Cui <email address hidden>

* fix UT failures

Signed-off-by: Hailong Cui <email address hidden>

* fix detekt issue

Signed-off-by: Hailong Cui <email address hidden>

* fix NPE

Signed-off-by: Hailong Cui <email address hidden>

* force merge message

Signed-off-by: Hailong Cui <email address hidden>

* remove duplicate error of reindex

Signed-off-by: Hailong Cui <email address hidden>

* notification message rewording

Signed-off-by: Hailong Cui <email address hidden>

* notification message wording change and unit test case

Signed-off-by: Hailong Cui <email address hidden>

* update block write wording

Signed-off-by: Hailong Cui <email address hidden>

* Add more unit test case

Signed-off-by: Hailong Cui <email address hidden>

* Add more logs

Signed-off-by: Hailong Cui <email address hidden>

---------

Signed-off-by: Hailong Cui <email address hidden>
Signed-off-by: zhichao-aws <email address hidden>
Co-authored-by: zhichao-aws <email address hidden>
(cherry picked from commit 89bd6de24e8ae57712d348c52686774d1c53f8ac)

* Fix backport failure

Signed-off-by: Hailong Cui <email address hidden>

---------

Signed-off-by: Hailong Cui <email address hidden>
Co-authored-by: Hailong Cui <email address hidden>

431fdac... by zhichao-aws <email address hidden>

[Backport 2.x] CRUD api for long running operation notification config (#791)

* CRUD api for long running operation notification config (#722)

* notification crud api

Signed-off-by: zhichao-aws <email address hidden>

* changes based on comments; add tests

Signed-off-by: zhichao-aws <email address hidden>

* changes for comment

Signed-off-by: zhichao-aws <email address hidden>

* change behavior for get action index not exists, fix test cases

Signed-off-by: zhichao-aws <email address hidden>

* fix weak password for changes in security plugin

Signed-off-by: zhichao-aws <email address hidden>

---------

Signed-off-by: zhichao-aws <email address hidden>
(cherry picked from commit 96f7380341af6e978081a7568e33dbf9cd58f7e3)

* fix test code apache http package version

Signed-off-by: zhichao-aws <email address hidden>

---------

Signed-off-by: zhichao-aws <email address hidden>

2a79766... by Petar Dzepina <email address hidden>

AddPolicyAction - Recursion removal (#779) (#785)

* recursion removal from AddPolicy Action

Signed-off-by: Petar Dzepina <email address hidden>

* added test

Signed-off-by: Petar Dzepina <email address hidden>

* debug logging

Signed-off-by: Petar Dzepina <email address hidden>

* removed single node testcase

Signed-off-by: Petar Dzepina <email address hidden>

* added security test; fixed index permission check

Signed-off-by: Petar Dzepina <email address hidden>

* test fix

Signed-off-by: Petar Dzepina <email address hidden>

* addressing comments

Signed-off-by: Petar Dzepina <email address hidden>

* test cleanup

Signed-off-by: Petar Dzepina <email address hidden>

* reverted security inject changes

Signed-off-by: Petar Dzepina <email address hidden>

* fixed weak password error when creating test user

Signed-off-by: Petar Dzepina <email address hidden>

* test tweak

Signed-off-by: Petar Dzepina <email address hidden>

---------

Signed-off-by: Petar Dzepina <email address hidden>
(cherry picked from commit 45b14765c2139095da9ce12c835a8bac35fa5fbb)

fa00e92... by Petar Dzepina <email address hidden>

[backport 2.x] Rollup Interceptor sourceIndex NPE fix (#773) (#781)

* Rollup Interceptor sourceIndex NPE fix (#773)

* initial commit

Signed-off-by: Petar Dzepina <email address hidden>

* detekt fix

Signed-off-by: Petar Dzepina <email address hidden>

* few fixes

Signed-off-by: Petar Dzepina <email address hidden>

* detekt override

Signed-off-by: Petar Dzepina <email address hidden>

* Added test

Signed-off-by: Petar Dzepina <email address hidden>

---------

Signed-off-by: Petar Dzepina <email address hidden>
(cherry picked from commit 4c60abd2d31f2738a08306944b61fcb53e6eb9a2)

* compile fix

Signed-off-by: Petar Dzepina <email address hidden>

---------

Signed-off-by: Petar Dzepina <email address hidden>

e2c0a0e... by "opensearch-trigger-bot[bot]" <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>

Date nanos rollup fix (#772) (#782)

* initial commit

Signed-off-by: Petar Dzepina <email address hidden>

* initial commit

Signed-off-by: Petar Dzepina <email address hidden>

* reverted pkgbuild.gradle

Signed-off-by: Petar Dzepina <email address hidden>

* ec

Signed-off-by: Petar Dzepina <email address hidden>

* rework

Signed-off-by: Petar Dzepina <email address hidden>

* ec

Signed-off-by: Petar Dzepina <email address hidden>

* ec

Signed-off-by: Petar Dzepina <email address hidden>

* ec

Signed-off-by: Petar Dzepina <email address hidden>

* fixed tests

Signed-off-by: Petar Dzepina <email address hidden>

* changed rollup id in test

Signed-off-by: Petar Dzepina <email address hidden>

---------

Signed-off-by: Petar Dzepina <email address hidden>
(cherry picked from commit 7ed62993ce6b7e53cd0ac5513b9d51110ce692a9)

Co-authored-by: Petar Dzepina <email address hidden>