Merge ~peppepetra/charm-graylog:lp-1954764 into charm-graylog:master
- Git
- lp:~peppepetra/charm-graylog
- lp-1954764
- Merge into master
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | James Troup | ||||
Approved revision: | ff2b261bc050cb011c2a344da1a0632c909cbe6b | ||||
Merged at revision: | 0d9b0f278badbf421d0b2566dda7659d5879fff2 | ||||
Proposed branch: | ~peppepetra/charm-graylog:lp-1954764 | ||||
Merge into: | charm-graylog:master | ||||
Diff against target: |
124 lines (+35/-7) 4 files modified
src/lib/charms/layer/graylog/snap_change.py (+3/-0) src/reactive/graylog.py (+24/-1) src/tests/functional/tests/tests.yaml (+1/-1) src/tests/unit/test_graylog.py (+7/-5) |
||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
🤖 prod-jenkaas-bootstack (community) | continuous-integration | Needs Fixing | |
Drew Freiberger (community) | Approve | ||
BootStack Reviewers | Pending | ||
Review via email: mp+413151@code.launchpad.net |
Commit message
Mitigate log4j CVE-2021-44228
Description of the change
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
A CI job is currently in progress. A follow up comment will be added when it completes.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:63fe5dff0c8
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
A CI job is currently in progress. A follow up comment will be added when it completes.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:e3d94613fdd
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
A CI job is currently in progress. A follow up comment will be added when it completes.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:b33915cf13b
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
A CI job is currently in progress. A follow up comment will be added when it completes.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:033234c8cbb
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:033234c8cbb
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:033234c8cbb
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Drew Freiberger (afreiberger) wrote : | # |
+1 on this approach
There's a bug with the new 3.x version that the conf file now matches the regex twice, which needs to be bypassed.
I would suggest adding in a forced writing of the new_conf file if it doesn't exist, rather than checking against the settings of the SHIPPED_
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:033234c8cbb
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
A CI job is currently in progress. A follow up comment will be added when it completes.
Drew Freiberger (afreiberger) wrote : | # |
CI appears to be failing at timeout for settling for graylog2-focal, but passed for graylog2-bionic. No errors, just not enough time to settle. Can we get a manual graylog2-focal test run outside CI and possibly move gl2-focal to a non-gating test?
I'm +1 on the change.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:87da3dec3d6
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Giuseppe Petralia (peppepetra) wrote : | # |
Proof of gl2 passing on serverstack https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
A CI job is currently in progress. A follow up comment will be added when it completes.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:7cdd02589d5
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
A CI job is currently in progress. A follow up comment will be added when it completes.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:00909086e19
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:00909086e19
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:00909086e19
https:/
Executed test runs:
None: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
A CI job is currently in progress. A follow up comment will be added when it completes.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:ff2b261bc05
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
Change successfully merged at revision 0d9b0f278badbf4
Preview Diff
1 | diff --git a/src/lib/charms/layer/graylog/snap_change.py b/src/lib/charms/layer/graylog/snap_change.py | |||
2 | index 9de8d45..59dd792 100644 | |||
3 | --- a/src/lib/charms/layer/graylog/snap_change.py | |||
4 | +++ b/src/lib/charms/layer/graylog/snap_change.py | |||
5 | @@ -106,3 +106,6 @@ def perform_snap_channel_change(new_channel): | |||
6 | 106 | data_changed("elasticsearch.relation", None) | 106 | data_changed("elasticsearch.relation", None) |
7 | 107 | data_changed("mongodb.uri", None) | 107 | data_changed("mongodb.uri", None) |
8 | 108 | remove_state("graylog.configured") | 108 | remove_state("graylog.configured") |
9 | 109 | else: | ||
10 | 110 | # If reconfiguration is not needed, restart the snap | ||
11 | 111 | host.service_start(SERVICE_NAME) | ||
12 | diff --git a/src/reactive/graylog.py b/src/reactive/graylog.py | |||
13 | index 2bb524d..4bfcac7 100644 | |||
14 | --- a/src/reactive/graylog.py | |||
15 | +++ b/src/reactive/graylog.py | |||
16 | @@ -1028,8 +1028,11 @@ def set_jvm_heap_size(heap_size="1G", conf_path=SERVER_DEFAULT_CONF_FILE): # no | |||
17 | 1028 | # file shipped out by the snap | 1028 | # file shipped out by the snap |
18 | 1029 | if not os.path.exists(conf_path): | 1029 | if not os.path.exists(conf_path): |
19 | 1030 | updated_conf_path = SHIPPED_SNAP_SERVER_DEFAULT_CONF_FILE | 1030 | updated_conf_path = SHIPPED_SNAP_SERVER_DEFAULT_CONF_FILE |
20 | 1031 | # if current config does not exist, force writing it even with default opts | ||
21 | 1032 | changed = True | ||
22 | 1031 | else: | 1033 | else: |
23 | 1032 | updated_conf_path = conf_path | 1034 | updated_conf_path = conf_path |
24 | 1035 | |||
25 | 1033 | with open(updated_conf_path, "rb") as conf_file: | 1036 | with open(updated_conf_path, "rb") as conf_file: |
26 | 1034 | for line in conf_file: | 1037 | for line in conf_file: |
27 | 1035 | line = line.decode() | 1038 | line = line.decode() |
28 | @@ -1037,8 +1040,20 @@ def set_jvm_heap_size(heap_size="1G", conf_path=SERVER_DEFAULT_CONF_FILE): # no | |||
29 | 1037 | if not m: | 1040 | if not m: |
30 | 1038 | conf.append(line) | 1041 | conf.append(line) |
31 | 1039 | continue | 1042 | continue |
33 | 1040 | new_opts = parse_java_opts(m.group(1), size) | 1043 | all_opts = parse_java_opts(m.group(1), size) |
34 | 1044 | |||
35 | 1045 | # Mitigate log4j CVE-2021-44228 | ||
36 | 1046 | mitigation_option = "-Dlog4j2.formatMsgNoLookups=true" | ||
37 | 1047 | if mitigation_option not in all_opts: | ||
38 | 1048 | all_opts.append(mitigation_option) | ||
39 | 1049 | |||
40 | 1050 | # Avoid repeating opts | ||
41 | 1051 | new_opts = [] | ||
42 | 1052 | for opt in all_opts: | ||
43 | 1053 | if not key_in_list(opt, conf): | ||
44 | 1054 | new_opts.append(opt) | ||
45 | 1041 | new_line = 'GRAYLOG_SERVER_JAVA_OPTS="{}"\n'.format(" ".join(new_opts)) | 1055 | new_line = 'GRAYLOG_SERVER_JAVA_OPTS="{}"\n'.format(" ".join(new_opts)) |
46 | 1056 | |||
47 | 1042 | if new_line != line: | 1057 | if new_line != line: |
48 | 1043 | hookenv.log( | 1058 | hookenv.log( |
49 | 1044 | 'Updating GRAYLOG_SERVER_JAVA_OPTS="{}".'.format("".join(new_opts)) | 1059 | 'Updating GRAYLOG_SERVER_JAVA_OPTS="{}".'.format("".join(new_opts)) |
50 | @@ -1294,3 +1309,11 @@ def _maybe_configure_graylog_jvm_keystore(): | |||
51 | 1294 | context={}, | 1309 | context={}, |
52 | 1295 | perms=0o644, | 1310 | perms=0o644, |
53 | 1296 | ) | 1311 | ) |
54 | 1312 | |||
55 | 1313 | |||
56 | 1314 | def key_in_list(key, lines): | ||
57 | 1315 | """Check if key is a substring of any lines.""" | ||
58 | 1316 | for line in lines: | ||
59 | 1317 | if key in line: | ||
60 | 1318 | return True | ||
61 | 1319 | return False | ||
62 | diff --git a/src/tests/functional/tests/tests.yaml b/src/tests/functional/tests/tests.yaml | |||
63 | index 9b53b3b..95eb05a 100644 | |||
64 | --- a/src/tests/functional/tests/tests.yaml | |||
65 | +++ b/src/tests/functional/tests/tests.yaml | |||
66 | @@ -1,7 +1,6 @@ | |||
67 | 1 | charm_name: graylog | 1 | charm_name: graylog |
68 | 2 | gate_bundles: | 2 | gate_bundles: |
69 | 3 | - gl2: bionic-graylog2 | 3 | - gl2: bionic-graylog2 |
70 | 4 | - gl2: focal-graylog2 | ||
71 | 5 | - gl3: bionic-graylog3 | 4 | - gl3: bionic-graylog3 |
72 | 6 | - gl3: bionic-graylog3-tls | 5 | - gl3: bionic-graylog3-tls |
73 | 7 | - gl3: focal-graylog3 | 6 | - gl3: focal-graylog3 |
74 | @@ -12,6 +11,7 @@ smoke_bundles: | |||
75 | 12 | - gl3: bionic-graylog3 | 11 | - gl3: bionic-graylog3 |
76 | 13 | dev_bundles: | 12 | dev_bundles: |
77 | 14 | - gl3: bionic-graylog3 | 13 | - gl3: bionic-graylog3 |
78 | 14 | - gl2: focal-graylog2 | ||
79 | 15 | tests: | 15 | tests: |
80 | 16 | - gl2: | 16 | - gl2: |
81 | 17 | - tests.test_legacy.LegacyTests | 17 | - tests.test_legacy.LegacyTests |
82 | diff --git a/src/tests/unit/test_graylog.py b/src/tests/unit/test_graylog.py | |||
83 | index 1f9a57a..1819b85 100644 | |||
84 | --- a/src/tests/unit/test_graylog.py | |||
85 | +++ b/src/tests/unit/test_graylog.py | |||
86 | @@ -472,13 +472,13 @@ GRAYLOG_COMMAND_WRAPPER="" | |||
87 | 472 | heap_default = u""" | 472 | heap_default = u""" |
88 | 473 | GRAYLOG_SERVER_JAVA_OPTS="-Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB \ | 473 | GRAYLOG_SERVER_JAVA_OPTS="-Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB \ |
89 | 474 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ | 474 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ |
91 | 475 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow" | 475 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j2.formatMsgNoLookups=true" |
92 | 476 | """ | 476 | """ |
93 | 477 | 477 | ||
94 | 478 | heap_2g = u""" | 478 | heap_2g = u""" |
95 | 479 | GRAYLOG_SERVER_JAVA_OPTS="-Xms2g -Xmx2g -XX:NewRatio=1 -server -XX:+ResizeTLAB \ | 479 | GRAYLOG_SERVER_JAVA_OPTS="-Xms2g -Xmx2g -XX:NewRatio=1 -server -XX:+ResizeTLAB \ |
96 | 480 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ | 480 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ |
98 | 481 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow" | 481 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j2.formatMsgNoLookups=true" |
99 | 482 | """ | 482 | """ |
100 | 483 | 483 | ||
101 | 484 | heap_not_present = u""" | 484 | heap_not_present = u""" |
102 | @@ -490,19 +490,21 @@ GRAYLOG_SERVER_JAVA_OPTS="-XX:NewRatio=1 -server -XX:+ResizeTLAB \ | |||
103 | 490 | heap_unset = u""" | 490 | heap_unset = u""" |
104 | 491 | GRAYLOG_SERVER_JAVA_OPTS="-XX:NewRatio=1 -server -XX:+ResizeTLAB \ | 491 | GRAYLOG_SERVER_JAVA_OPTS="-XX:NewRatio=1 -server -XX:+ResizeTLAB \ |
105 | 492 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ | 492 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ |
107 | 493 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow" | 493 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j2.formatMsgNoLookups=true" |
108 | 494 | """ | 494 | """ |
109 | 495 | 495 | ||
110 | 496 | heap_not_present = u""" | 496 | heap_not_present = u""" |
111 | 497 | GRAYLOG_SERVER_JAVA_OPTS="-XX:NewRatio=1 -server -XX:+ResizeTLAB \ | 497 | GRAYLOG_SERVER_JAVA_OPTS="-XX:NewRatio=1 -server -XX:+ResizeTLAB \ |
112 | 498 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ | 498 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ |
114 | 499 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Xms4g -Xmx4g" | 499 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j2.formatMsgNoLookups=true \ |
115 | 500 | -Xms4g -Xmx4g" | ||
116 | 500 | """ | 501 | """ |
117 | 501 | 502 | ||
118 | 502 | heap_twice = u""" | 503 | heap_twice = u""" |
119 | 503 | GRAYLOG_SERVER_JAVA_OPTS="-XX:NewRatio=1 -server -XX:+ResizeTLAB \ | 504 | GRAYLOG_SERVER_JAVA_OPTS="-XX:NewRatio=1 -server -XX:+ResizeTLAB \ |
120 | 504 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ | 505 | -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled \ |
122 | 505 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Xms8g -Xmx8g" | 506 | -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j2.formatMsgNoLookups=true \ |
123 | 507 | -Xms8g -Xmx8g" | ||
124 | 506 | """ | 508 | """ |
125 | 507 | 509 | ||
126 | 508 | 510 |
This merge proposal is being monitored by mergebot. Change the status to Approved to merge.