Merge ~peter-sabaini/charm-graylog:lp1758175 into charm-graylog:master
- Git
- lp:~peter-sabaini/charm-graylog
- lp1758175
- Merge into master
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Xav Paice | ||||
Approved revision: | adacec26185a959fc0660eca258f127469e38fc7 | ||||
Merged at revision: | 47d61ac0ad7465c6bae5b3198b3e95dd32c26bdf | ||||
Proposed branch: | ~peter-sabaini/charm-graylog:lp1758175 | ||||
Merge into: | charm-graylog:master | ||||
Diff against target: |
841 lines (+441/-17) 20 files modified
src/layer.yaml (+1/-0) src/lib/charms/layer/graylog/api.py (+11/-1) src/lib/charms/layer/graylog/constants.py (+6/-0) src/lib/charms/layer/graylog/utils.py (+9/-2) src/reactive/graylog.py (+72/-3) src/templates/default-graylog-server (+4/-0) src/templates/sync-graylog-snap (+7/-0) src/tests/functional/requirements.txt (+2/-0) src/tests/functional/tests/base.py (+48/-0) src/tests/functional/tests/bundles/bionic-graylog2.yaml (+6/-0) src/tests/functional/tests/bundles/bionic-graylog3-tls.yaml (+51/-0) src/tests/functional/tests/bundles/bionic-graylog3.yaml (+6/-0) src/tests/functional/tests/bundles/focal-graylog2.yaml (+6/-0) src/tests/functional/tests/bundles/focal-graylog3-tls.yaml (+51/-0) src/tests/functional/tests/bundles/focal-graylog3.yaml (+6/-0) src/tests/functional/tests/test_graylog_charm.py (+12/-5) src/tests/functional/tests/test_legacy.py (+29/-4) src/tests/functional/tests/tests.yaml (+15/-1) src/tests/unit/test_graylog.py (+86/-0) src/tests/unit/test_lib.py (+13/-1) |
||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Xav Paice (community) | Approve | ||
Drew Freiberger (community) | Approve | ||
Paul Goins | Approve | ||
Review via email: mp+396671@code.launchpad.net |
This proposal supersedes a proposal from 2021-01-21.
Commit message
Description of the change
Paul Goins (vultaire) wrote : Posted in a previous version of this proposal | # |
Paul Goins (vultaire) wrote : Posted in a previous version of this proposal | # |
Unit tests are failing; seems more mocking is needed. "make unittests" partial output showing the failure: https:/
(It's trying to do an apt-get install ca-certificates
Running functional tests now.
Paul Goins (vultaire) wrote : Posted in a previous version of this proposal | # |
Functional tests pass.
If you can run "make black" and correct the unit test failure, I think this is +1 from me.
Peter Sabaini (peter-sabaini) wrote : Posted in a previous version of this proposal | # |
I've mocked out the java CA install and blackened, please take a look?
Drew Freiberger (afreiberger) wrote : | # |
Expected status blocked on bionic-
Unit Workload Agent Machine Public address Ports Message
easyrsa/0* active idle 0 10.0.8.81 Certificate Authority connected.
elastic/0* active idle 1 10.0.8.91 9200/tcp Unit is ready
graylog/0* waiting idle 2 10.0.8.8 9000/tcp Waiting for: filebeat, REST API
nrpe/0* active idle 10.0.8.8 icmp,5666/tcp ready
mongo/0* active idle 3 10.0.8.92 27017/tcp,
nagios/0* active idle 4 10.0.8.126 80/tcp ready
ubuntu/0* active idle 5 10.0.8.135 ready
filebeat/0* active idle 10.0.8.135 Filebeat ready.
Drew Freiberger (afreiberger) wrote : | # |
Graylog service not starting snap version is 3/stable with tls enabled:
2021-01-
2021-01-
2021-01-
graylog traceback has this as the cause:
Caused by: org.glassfish.
Snap is running graylog with the following cli args: (most specifically: -Djavax.
root 7680 1 0 21:24 ? 00:00:03 /snap/graylog/
Drew Freiberger (afreiberger) wrote : | # |
After adding the sync-graylog-snap file into /etc/ca-
Drew Freiberger (afreiberger) wrote : | # |
Getting random snap core install errors (this was on bionic-graylog3-tls model.
https:/
I feel like there weren't random functest errors in 20.10, I'm running functest against that release to see results for stable/20.10 with current upstream bits and bobs to see if it's in the upstream or in our charm/functests that need repairing.
Drew Freiberger (afreiberger) wrote : | # |
We have uncovered a number of issues with our testing infrastructure (juju not updating daily images, zaza not upgrading packages, systemd bug affecting ES patched a couple weeks ago) that are causing random failures of the functional tests, but 30% pass w/out issue. I suggest merging this and opening a bug report for racy tests.
Preview Diff
1 | diff --git a/src/layer.yaml b/src/layer.yaml | |||
2 | index 04e6d2a..5173d7e 100644 | |||
3 | --- a/src/layer.yaml | |||
4 | +++ b/src/layer.yaml | |||
5 | @@ -3,6 +3,7 @@ includes: | |||
6 | 3 | - 'layer:basic' | 3 | - 'layer:basic' |
7 | 4 | - 'layer:snap' | 4 | - 'layer:snap' |
8 | 5 | - 'layer:leadership' | 5 | - 'layer:leadership' |
9 | 6 | - 'layer:tls-client' | ||
10 | 6 | - 'interface:elasticsearch' | 7 | - 'interface:elasticsearch' |
11 | 7 | - 'interface:elastic-beats' | 8 | - 'interface:elastic-beats' |
12 | 8 | - 'interface:http' | 9 | - 'interface:http' |
13 | diff --git a/src/lib/charms/layer/graylog/api.py b/src/lib/charms/layer/graylog/api.py | |||
14 | index 98e52cf..3d9bf93 100644 | |||
15 | --- a/src/lib/charms/layer/graylog/api.py | |||
16 | +++ b/src/lib/charms/layer/graylog/api.py | |||
17 | @@ -4,6 +4,12 @@ import os | |||
18 | 4 | 4 | ||
19 | 5 | import requests | 5 | import requests |
20 | 6 | 6 | ||
21 | 7 | # When using 'certifi' from the virtualenv, the system-wide certificates store | ||
22 | 8 | # is not used, so installed certificates won't be used to validate hosts. | ||
23 | 9 | # Adding the system CA bundle | ||
24 | 10 | # https://git.launchpad.net/ubuntu/+source/python-certifi/tree/debian/patches/0001-Use-Debian-provided-etc-ssl-certs-ca-certificates.cr.patch | ||
25 | 11 | SYSTEM_CA_BUNDLE = "/etc/ssl/certs/ca-certificates.crt" | ||
26 | 12 | |||
27 | 7 | # We are in a charm environment | 13 | # We are in a charm environment |
28 | 8 | charm = False | 14 | charm = False |
29 | 9 | if os.environ.get("JUJU_UNIT_NAME"): | 15 | if os.environ.get("JUJU_UNIT_NAME"): |
30 | @@ -20,7 +26,9 @@ def get_ignore_indexer_failures_file(): # noqa: D103 | |||
31 | 20 | class GraylogApi: | 26 | class GraylogApi: |
32 | 21 | """Manage Graylog via its API.""" | 27 | """Manage Graylog via its API.""" |
33 | 22 | 28 | ||
35 | 23 | def __init__(self, base_url, username, password, token_name="graylog-api"): | 29 | def __init__( |
36 | 30 | self, base_url, username, password, token_name="graylog-api", verify=None | ||
37 | 31 | ): | ||
38 | 24 | """Initialize HTTP values.""" | 32 | """Initialize HTTP values.""" |
39 | 25 | self.base_url = base_url | 33 | self.base_url = base_url |
40 | 26 | if not base_url.endswith("/"): | 34 | if not base_url.endswith("/"): |
41 | @@ -33,6 +41,7 @@ class GraylogApi: | |||
42 | 33 | self.input_types = None | 41 | self.input_types = None |
43 | 34 | self.req_timeout = 3 | 42 | self.req_timeout = 3 |
44 | 35 | self.req_retries = 4 | 43 | self.req_retries = 4 |
45 | 44 | self.verify = verify | ||
46 | 36 | 45 | ||
47 | 37 | def request(self, path, method="GET", data={}, params=None): # noqa: C901 | 46 | def request(self, path, method="GET", data={}, params=None): # noqa: C901 |
48 | 38 | """Retrieve data by URL.""" | 47 | """Retrieve data by URL.""" |
49 | @@ -59,6 +68,7 @@ class GraylogApi: | |||
50 | 59 | params=params, | 68 | params=params, |
51 | 60 | headers=headers, | 69 | headers=headers, |
52 | 61 | timeout=self.req_timeout, | 70 | timeout=self.req_timeout, |
53 | 71 | verify=SYSTEM_CA_BUNDLE if self.verify is None else self.verify, | ||
54 | 62 | ) | 72 | ) |
55 | 63 | if resp.ok: | 73 | if resp.ok: |
56 | 64 | if method == "DELETE": | 74 | if method == "DELETE": |
57 | diff --git a/src/lib/charms/layer/graylog/constants.py b/src/lib/charms/layer/graylog/constants.py | |||
58 | index db38aa0..ce2bc59 100644 | |||
59 | --- a/src/lib/charms/layer/graylog/constants.py | |||
60 | +++ b/src/lib/charms/layer/graylog/constants.py | |||
61 | @@ -1,5 +1,9 @@ | |||
62 | 1 | """Options used by the Graylog charm.""" | 1 | """Options used by the Graylog charm.""" |
63 | 2 | import os | ||
64 | 3 | |||
65 | 4 | |||
66 | 2 | SNAP_NAME = "graylog" | 5 | SNAP_NAME = "graylog" |
67 | 6 | SNAP_COMMON_DIR = "/var/snap/graylog/common/" | ||
68 | 3 | CONF_FILE = "/var/snap/graylog/common/server.conf" | 7 | CONF_FILE = "/var/snap/graylog/common/server.conf" |
69 | 4 | SNAP_CONF_FILE = "/snap/graylog/current/etc/graylog/server/server.conf" | 8 | SNAP_CONF_FILE = "/snap/graylog/current/etc/graylog/server/server.conf" |
70 | 5 | # /snap/graylog is a read-only squashfs so we use the initial settings | 9 | # /snap/graylog is a read-only squashfs so we use the initial settings |
71 | @@ -12,3 +16,5 @@ ELASTICSEARCH_DISCOVERY_PORT = "9300" | |||
72 | 12 | SERVICE_NAME = "snap.graylog.graylog" | 16 | SERVICE_NAME = "snap.graylog.graylog" |
73 | 13 | DEFAULT_REST_API_TIMEOUT = 120 | 17 | DEFAULT_REST_API_TIMEOUT = 120 |
74 | 14 | NAGIOS_USERNAME = "nagios" | 18 | NAGIOS_USERNAME = "nagios" |
75 | 19 | CERT_PATH = os.path.join(SNAP_COMMON_DIR, "server.crt") | ||
76 | 20 | CERT_KEY_PATH = os.path.join(SNAP_COMMON_DIR, "server.key") | ||
77 | diff --git a/src/lib/charms/layer/graylog/utils.py b/src/lib/charms/layer/graylog/utils.py | |||
78 | index 115e672..6b020c5 100644 | |||
79 | --- a/src/lib/charms/layer/graylog/utils.py | |||
80 | +++ b/src/lib/charms/layer/graylog/utils.py | |||
81 | @@ -4,10 +4,10 @@ from urllib.parse import urlparse | |||
82 | 4 | from charmhelpers.core import hookenv | 4 | from charmhelpers.core import hookenv |
83 | 5 | 5 | ||
84 | 6 | from charms.layer.graylog.constants import SNAP_NAME | 6 | from charms.layer.graylog.constants import SNAP_NAME |
85 | 7 | from charms.reactive import get_flags | ||
86 | 7 | 8 | ||
87 | 8 | import netifaces | 9 | import netifaces |
88 | 9 | 10 | ||
89 | 10 | |||
90 | 11 | # Allow mocking of other layers during unit tests | 11 | # Allow mocking of other layers during unit tests |
91 | 12 | try: | 12 | try: |
92 | 13 | from charms.layer import snap | 13 | from charms.layer import snap |
93 | @@ -20,7 +20,14 @@ def get_api_port(): # noqa: D103 | |||
94 | 20 | 20 | ||
95 | 21 | 21 | ||
96 | 22 | def get_api_url(): # noqa: D103 | 22 | def get_api_url(): # noqa: D103 |
98 | 23 | return "http://127.0.0.1:{}/api/".format(get_api_port()) | 23 | return "{}://127.0.0.1:{}/api/".format(get_api_protocol(), get_api_port()) |
99 | 24 | |||
100 | 25 | |||
101 | 26 | def get_api_protocol(): # noqa: D103 | ||
102 | 27 | if "graylog.certificates.configured" in get_flags(): | ||
103 | 28 | return "https" | ||
104 | 29 | else: | ||
105 | 30 | return "http" | ||
106 | 24 | 31 | ||
107 | 25 | 32 | ||
108 | 26 | def is_v2(): # noqa: D103 | 33 | def is_v2(): # noqa: D103 |
109 | diff --git a/src/reactive/graylog.py b/src/reactive/graylog.py | |||
110 | index 9cc3a93..4f4a9ee 100644 | |||
111 | --- a/src/reactive/graylog.py | |||
112 | +++ b/src/reactive/graylog.py | |||
113 | @@ -2,23 +2,29 @@ | |||
114 | 2 | import hashlib | 2 | import hashlib |
115 | 3 | import os | 3 | import os |
116 | 4 | import re | 4 | import re |
117 | 5 | import socket | ||
118 | 6 | import subprocess | ||
119 | 5 | import time | 7 | import time |
120 | 6 | from urllib.parse import urlparse | 8 | from urllib.parse import urlparse |
121 | 7 | 9 | ||
122 | 8 | from charmhelpers.contrib.charmsupport import nrpe | 10 | from charmhelpers.contrib.charmsupport import nrpe |
124 | 9 | from charmhelpers.core import hookenv, host, unitdata | 11 | from charmhelpers.core import hookenv, host, templating, unitdata |
125 | 12 | from charmhelpers.fetch import filter_installed_packages, install | ||
126 | 10 | 13 | ||
127 | 11 | import charms.leadership | 14 | import charms.leadership |
129 | 12 | from charms.layer import snap | 15 | from charms.layer import snap, tls_client |
130 | 13 | from charms.layer.graylog import ( | 16 | from charms.layer.graylog import ( |
131 | 14 | GraylogApi, | 17 | GraylogApi, |
132 | 15 | LogExtractPipeline, | 18 | LogExtractPipeline, |
133 | 16 | get_api_port, | 19 | get_api_port, |
134 | 20 | get_api_protocol, | ||
135 | 17 | get_api_url, | 21 | get_api_url, |
136 | 18 | is_v2, | 22 | is_v2, |
137 | 19 | validate_api_uri, | 23 | validate_api_uri, |
138 | 20 | ) | 24 | ) |
139 | 21 | from charms.layer.graylog.constants import ( | 25 | from charms.layer.graylog.constants import ( |
140 | 26 | CERT_KEY_PATH, | ||
141 | 27 | CERT_PATH, | ||
142 | 22 | CONF_FILE, | 28 | CONF_FILE, |
143 | 23 | DEFAULT_REST_API_TIMEOUT, | 29 | DEFAULT_REST_API_TIMEOUT, |
144 | 24 | ELASTICSEARCH_DISCOVERY_PORT, | 30 | ELASTICSEARCH_DISCOVERY_PORT, |
145 | @@ -200,7 +206,8 @@ def configure_graylog(): # noqa: C901 | |||
146 | 200 | # intead of guessing the best IP. | 206 | # intead of guessing the best IP. |
147 | 201 | set_conf("http_bind_address", "0.0.0.0:9000") | 207 | set_conf("http_bind_address", "0.0.0.0:9000") |
148 | 202 | set_conf( | 208 | set_conf( |
150 | 203 | "http_publish_uri", validate_api_uri("http://0.0.0.0:{}/".format(api_port)) | 209 | "http_publish_uri", |
151 | 210 | validate_api_uri("{}://0.0.0.0:{}/".format(get_api_protocol(), api_port)), | ||
152 | 204 | ) | 211 | ) |
153 | 205 | 212 | ||
154 | 206 | if conf["web_endpoint_uri"]: | 213 | if conf["web_endpoint_uri"]: |
155 | @@ -1045,6 +1052,49 @@ def configure_nagios(nagios): | |||
156 | 1045 | set_state("graylog_nagios.configured") | 1052 | set_state("graylog_nagios.configured") |
157 | 1046 | 1053 | ||
158 | 1047 | 1054 | ||
159 | 1055 | @when("certificates.available") | ||
160 | 1056 | @when_not("graylog.certificates.configured") | ||
161 | 1057 | def tls_request_certificate(tls): | ||
162 | 1058 | """Create a server certificate for this server.""" | ||
163 | 1059 | # ca-certificates-java generates the ca-certificates in a jvm compatible | ||
164 | 1060 | # keystore | ||
165 | 1061 | if filter_installed_packages(["ca-certificates-java"]): | ||
166 | 1062 | install(["ca-certificates-java"], fatal=True) | ||
167 | 1063 | |||
168 | 1064 | _maybe_install_ca_certificates_hook() | ||
169 | 1065 | _maybe_configure_graylog_jvm_keystore() | ||
170 | 1066 | |||
171 | 1067 | # Use the public ip of this unit as the Common Name for the certificate. | ||
172 | 1068 | common_name = hookenv.unit_public_ip() | ||
173 | 1069 | # Get a list of Subject Alt Names for the certificate. | ||
174 | 1070 | sans = [] | ||
175 | 1071 | sans.append(hookenv.unit_public_ip()) | ||
176 | 1072 | sans.append(hookenv.unit_private_ip()) | ||
177 | 1073 | sans.append(socket.gethostname()) | ||
178 | 1074 | sans.append("127.0.0.1") | ||
179 | 1075 | sans.append("localhost") | ||
180 | 1076 | tls_client.request_server_cert( | ||
181 | 1077 | common_name, sans, crt_path=CERT_PATH, key_path=CERT_KEY_PATH | ||
182 | 1078 | ) | ||
183 | 1079 | |||
184 | 1080 | set_conf("http_enable_tls", "true") | ||
185 | 1081 | set_conf("http_tls_cert_file", CERT_PATH) | ||
186 | 1082 | set_conf("http_tls_key_file", CERT_KEY_PATH) | ||
187 | 1083 | |||
188 | 1084 | set_conf("rest_enable_tls", "true") | ||
189 | 1085 | set_conf("rest_tls_cert_file", CERT_PATH) | ||
190 | 1086 | set_conf("rest_tls_key_file", CERT_KEY_PATH) | ||
191 | 1087 | |||
192 | 1088 | set_conf("web_enable_tls", "true") | ||
193 | 1089 | set_conf("web_tls_cert_file", CERT_PATH) | ||
194 | 1090 | set_conf("web_tls_key_file", CERT_KEY_PATH) | ||
195 | 1091 | |||
196 | 1092 | set_conf("http_publish_uri", "https://0.0.0.0:{}/".format(get_api_port())) | ||
197 | 1093 | |||
198 | 1094 | set_state("graylog.certificates.configured") | ||
199 | 1095 | set_state("graylog.needs_restart") | ||
200 | 1096 | |||
201 | 1097 | |||
202 | 1048 | def get_default_graylog_client(): # noqa: D103 | 1098 | def get_default_graylog_client(): # noqa: D103 |
203 | 1049 | db = unitdata.kv() | 1099 | db = unitdata.kv() |
204 | 1050 | return GraylogApi( | 1100 | return GraylogApi( |
205 | @@ -1068,3 +1118,22 @@ def _verify_rest_api_is_alive(timeout=DEFAULT_REST_API_TIMEOUT): | |||
206 | 1068 | if time.time() - start_ts > timeout: | 1118 | if time.time() - start_ts > timeout: |
207 | 1069 | raise ApiTimeout() | 1119 | raise ApiTimeout() |
208 | 1070 | hookenv.log("REST API is up") | 1120 | hookenv.log("REST API is up") |
209 | 1121 | |||
210 | 1122 | |||
211 | 1123 | def _maybe_install_ca_certificates_hook(): | ||
212 | 1124 | templating.render( | ||
213 | 1125 | "sync-graylog-snap", | ||
214 | 1126 | "/etc/ca-certificates/update.d/sync-graylog-snap", | ||
215 | 1127 | context={}, | ||
216 | 1128 | perms=0o755, | ||
217 | 1129 | ) | ||
218 | 1130 | subprocess.check_call(["update-ca-certificates"]) | ||
219 | 1131 | |||
220 | 1132 | |||
221 | 1133 | def _maybe_configure_graylog_jvm_keystore(): | ||
222 | 1134 | templating.render( | ||
223 | 1135 | "default-graylog-server", | ||
224 | 1136 | "/var/snap/graylog/current/default-graylog-server", | ||
225 | 1137 | context={}, | ||
226 | 1138 | perms=0o644, | ||
227 | 1139 | ) | ||
228 | diff --git a/src/templates/default-graylog-server b/src/templates/default-graylog-server | |||
229 | 1071 | new file mode 100644 | 1140 | new file mode 100644 |
230 | index 0000000..ace24cb | |||
231 | --- /dev/null | |||
232 | +++ b/src/templates/default-graylog-server | |||
233 | @@ -0,0 +1,4 @@ | |||
234 | 1 | # | ||
235 | 2 | # This file is managed by juju. | ||
236 | 3 | # | ||
237 | 4 | GRAYLOG_SERVER_JAVA_OPTS="-Djavax.net.ssl.trustStore=/var/snap/graylog/common/cacerts" | ||
238 | diff --git a/src/templates/sync-graylog-snap b/src/templates/sync-graylog-snap | |||
239 | 0 | new file mode 100755 | 5 | new file mode 100755 |
240 | index 0000000..6d08b7d | |||
241 | --- /dev/null | |||
242 | +++ b/src/templates/sync-graylog-snap | |||
243 | @@ -0,0 +1,7 @@ | |||
244 | 1 | #!/bin/sh | ||
245 | 2 | # | ||
246 | 3 | # This file is managed by juju. | ||
247 | 4 | # | ||
248 | 5 | |||
249 | 6 | set -e | ||
250 | 7 | rsync /etc/ssl/certs/java/cacerts /var/snap/graylog/common/cacerts | ||
251 | diff --git a/src/tests/functional/requirements.txt b/src/tests/functional/requirements.txt | |||
252 | index 7345526..ecf87be 100644 | |||
253 | --- a/src/tests/functional/requirements.txt | |||
254 | +++ b/src/tests/functional/requirements.txt | |||
255 | @@ -1,2 +1,4 @@ | |||
256 | 1 | tenacity | 1 | tenacity |
257 | 2 | git+https://github.com/openstack-charmers/zaza.git#egg=zaza | 2 | git+https://github.com/openstack-charmers/zaza.git#egg=zaza |
258 | 3 | netifaces | ||
259 | 4 | charmhelpers | ||
260 | diff --git a/src/tests/functional/tests/base.py b/src/tests/functional/tests/base.py | |||
261 | 3 | new file mode 100644 | 5 | new file mode 100644 |
262 | index 0000000..46eb75f | |||
263 | --- /dev/null | |||
264 | +++ b/src/tests/functional/tests/base.py | |||
265 | @@ -0,0 +1,48 @@ | |||
266 | 1 | """Base class for Testing.""" | ||
267 | 2 | import logging | ||
268 | 3 | import os | ||
269 | 4 | import tempfile | ||
270 | 5 | import unittest | ||
271 | 6 | |||
272 | 7 | from zaza import model | ||
273 | 8 | |||
274 | 9 | |||
275 | 10 | class BaseTestCase(unittest.TestCase): | ||
276 | 11 | """Base class for graylog functional testing.""" | ||
277 | 12 | |||
278 | 13 | @classmethod | ||
279 | 14 | def setUpClass(cls): | ||
280 | 15 | """Configure the test's environment. | ||
281 | 16 | |||
282 | 17 | - Get the CA from easyrsa (when available) and write it on disk. | ||
283 | 18 | """ | ||
284 | 19 | if cls.get_api_protocol() == "https": | ||
285 | 20 | rel_id = model.get_relation_id( | ||
286 | 21 | "graylog", "easyrsa", remote_interface_name="client" | ||
287 | 22 | ) | ||
288 | 23 | easyrsa_unit = model.get_units("easyrsa")[0] | ||
289 | 24 | cmd = "relation-get -r client:{} ca {}".format( | ||
290 | 25 | rel_id, easyrsa_unit.entity_id | ||
291 | 26 | ) | ||
292 | 27 | logging.info(cmd) | ||
293 | 28 | result = model.run_on_unit(easyrsa_unit.entity_id, cmd) | ||
294 | 29 | |||
295 | 30 | with tempfile.NamedTemporaryFile(mode="w", delete=False) as f: | ||
296 | 31 | f.write(result["Stdout"]) | ||
297 | 32 | f.flush() | ||
298 | 33 | cls.ca_path = f.name | ||
299 | 34 | else: | ||
300 | 35 | cls.ca_path = None | ||
301 | 36 | |||
302 | 37 | @classmethod | ||
303 | 38 | def tearDownClass(cls): | ||
304 | 39 | """Remove the CA file that was written to disk during the setUp.""" | ||
305 | 40 | if cls.ca_path: | ||
306 | 41 | os.remove(cls.ca_path) | ||
307 | 42 | |||
308 | 43 | @classmethod | ||
309 | 44 | def get_api_protocol(cls): # noqa: D102 | ||
310 | 45 | if model.get_relation_id("graylog", "easyrsa"): | ||
311 | 46 | return "https" | ||
312 | 47 | else: | ||
313 | 48 | return "http" | ||
314 | diff --git a/src/tests/functional/tests/bundles/bionic-graylog2.yaml b/src/tests/functional/tests/bundles/bionic-graylog2.yaml | |||
315 | index bd256db..27703fd 100644 | |||
316 | --- a/src/tests/functional/tests/bundles/bionic-graylog2.yaml | |||
317 | +++ b/src/tests/functional/tests/bundles/bionic-graylog2.yaml | |||
318 | @@ -30,6 +30,10 @@ applications: | |||
319 | 30 | nrpe: | 30 | nrpe: |
320 | 31 | charm: cs:nrpe | 31 | charm: cs:nrpe |
321 | 32 | 32 | ||
322 | 33 | nagios: | ||
323 | 34 | charm: cs:nagios | ||
324 | 35 | num_units: 1 | ||
325 | 36 | |||
326 | 33 | relations: | 37 | relations: |
327 | 34 | - - ubuntu | 38 | - - ubuntu |
328 | 35 | - filebeat | 39 | - filebeat |
329 | @@ -43,3 +47,5 @@ relations: | |||
330 | 43 | - haproxy | 47 | - haproxy |
331 | 44 | - - graylog | 48 | - - graylog |
332 | 45 | - nrpe | 49 | - nrpe |
333 | 50 | - - nagios | ||
334 | 51 | - nrpe | ||
335 | diff --git a/src/tests/functional/tests/bundles/bionic-graylog3-tls.yaml b/src/tests/functional/tests/bundles/bionic-graylog3-tls.yaml | |||
336 | 46 | new file mode 100644 | 52 | new file mode 100644 |
337 | index 0000000..954f9c5 | |||
338 | --- /dev/null | |||
339 | +++ b/src/tests/functional/tests/bundles/bionic-graylog3-tls.yaml | |||
340 | @@ -0,0 +1,51 @@ | |||
341 | 1 | series: bionic | ||
342 | 2 | |||
343 | 3 | applications: | ||
344 | 4 | ubuntu: | ||
345 | 5 | charm: cs:ubuntu | ||
346 | 6 | num_units: 1 | ||
347 | 7 | |||
348 | 8 | filebeat: | ||
349 | 9 | charm: cs:filebeat | ||
350 | 10 | num_units: 0 | ||
351 | 11 | |||
352 | 12 | graylog: | ||
353 | 13 | num_units: 1 | ||
354 | 14 | series: bionic | ||
355 | 15 | options: | ||
356 | 16 | channel: 3/stable | ||
357 | 17 | |||
358 | 18 | elastic: | ||
359 | 19 | charm: cs:elasticsearch | ||
360 | 20 | num_units: 1 | ||
361 | 21 | |||
362 | 22 | mongo: | ||
363 | 23 | charm: cs:mongodb | ||
364 | 24 | num_units: 1 | ||
365 | 25 | |||
366 | 26 | nrpe: | ||
367 | 27 | charm: cs:nrpe | ||
368 | 28 | |||
369 | 29 | nagios: | ||
370 | 30 | charm: cs:nagios | ||
371 | 31 | num_units: 1 | ||
372 | 32 | |||
373 | 33 | easyrsa: | ||
374 | 34 | charm: cs:~containers/easyrsa | ||
375 | 35 | num_units: 1 | ||
376 | 36 | |||
377 | 37 | relations: | ||
378 | 38 | - - ubuntu | ||
379 | 39 | - filebeat | ||
380 | 40 | - - graylog:beats | ||
381 | 41 | - filebeat:logstash | ||
382 | 42 | - - graylog | ||
383 | 43 | - mongo | ||
384 | 44 | - - graylog | ||
385 | 45 | - elastic | ||
386 | 46 | - - graylog | ||
387 | 47 | - nrpe | ||
388 | 48 | - - nagios | ||
389 | 49 | - nrpe | ||
390 | 50 | - - easyrsa:client | ||
391 | 51 | - graylog:certificates | ||
392 | diff --git a/src/tests/functional/tests/bundles/bionic-graylog3.yaml b/src/tests/functional/tests/bundles/bionic-graylog3.yaml | |||
393 | index 596cf7d..33ac7e5 100644 | |||
394 | --- a/src/tests/functional/tests/bundles/bionic-graylog3.yaml | |||
395 | +++ b/src/tests/functional/tests/bundles/bionic-graylog3.yaml | |||
396 | @@ -30,6 +30,10 @@ applications: | |||
397 | 30 | nrpe: | 30 | nrpe: |
398 | 31 | charm: cs:nrpe | 31 | charm: cs:nrpe |
399 | 32 | 32 | ||
400 | 33 | nagios: | ||
401 | 34 | charm: cs:nagios | ||
402 | 35 | num_units: 1 | ||
403 | 36 | |||
404 | 33 | relations: | 37 | relations: |
405 | 34 | - - ubuntu | 38 | - - ubuntu |
406 | 35 | - filebeat | 39 | - filebeat |
407 | @@ -43,3 +47,5 @@ relations: | |||
408 | 43 | - haproxy | 47 | - haproxy |
409 | 44 | - - graylog | 48 | - - graylog |
410 | 45 | - nrpe | 49 | - nrpe |
411 | 50 | - - nagios | ||
412 | 51 | - nrpe | ||
413 | diff --git a/src/tests/functional/tests/bundles/focal-graylog2.yaml b/src/tests/functional/tests/bundles/focal-graylog2.yaml | |||
414 | index 368432e..9cc6016 100644 | |||
415 | --- a/src/tests/functional/tests/bundles/focal-graylog2.yaml | |||
416 | +++ b/src/tests/functional/tests/bundles/focal-graylog2.yaml | |||
417 | @@ -30,6 +30,10 @@ applications: | |||
418 | 30 | nrpe: | 30 | nrpe: |
419 | 31 | charm: cs:nrpe | 31 | charm: cs:nrpe |
420 | 32 | 32 | ||
421 | 33 | nagios: | ||
422 | 34 | charm: cs:nagios | ||
423 | 35 | num_units: 1 | ||
424 | 36 | |||
425 | 33 | relations: | 37 | relations: |
426 | 34 | - - ubuntu | 38 | - - ubuntu |
427 | 35 | - filebeat | 39 | - filebeat |
428 | @@ -43,3 +47,5 @@ relations: | |||
429 | 43 | - haproxy | 47 | - haproxy |
430 | 44 | - - graylog | 48 | - - graylog |
431 | 45 | - nrpe | 49 | - nrpe |
432 | 50 | - - nagios | ||
433 | 51 | - nrpe | ||
434 | diff --git a/src/tests/functional/tests/bundles/focal-graylog3-tls.yaml b/src/tests/functional/tests/bundles/focal-graylog3-tls.yaml | |||
435 | 46 | new file mode 100644 | 52 | new file mode 100644 |
436 | index 0000000..100ccb4 | |||
437 | --- /dev/null | |||
438 | +++ b/src/tests/functional/tests/bundles/focal-graylog3-tls.yaml | |||
439 | @@ -0,0 +1,51 @@ | |||
440 | 1 | series: bionic | ||
441 | 2 | |||
442 | 3 | applications: | ||
443 | 4 | ubuntu: | ||
444 | 5 | charm: cs:ubuntu | ||
445 | 6 | num_units: 1 | ||
446 | 7 | |||
447 | 8 | filebeat: | ||
448 | 9 | charm: cs:filebeat | ||
449 | 10 | num_units: 0 | ||
450 | 11 | |||
451 | 12 | graylog: | ||
452 | 13 | num_units: 1 | ||
453 | 14 | series: focal | ||
454 | 15 | options: | ||
455 | 16 | channel: 3/stable | ||
456 | 17 | |||
457 | 18 | elastic: | ||
458 | 19 | charm: cs:elasticsearch | ||
459 | 20 | num_units: 1 | ||
460 | 21 | |||
461 | 22 | mongo: | ||
462 | 23 | charm: cs:mongodb | ||
463 | 24 | num_units: 1 | ||
464 | 25 | |||
465 | 26 | nrpe: | ||
466 | 27 | charm: cs:nrpe | ||
467 | 28 | |||
468 | 29 | nagios: | ||
469 | 30 | charm: cs:nagios | ||
470 | 31 | num_units: 1 | ||
471 | 32 | |||
472 | 33 | easyrsa: | ||
473 | 34 | charm: cs:~containers/easyrsa | ||
474 | 35 | num_units: 1 | ||
475 | 36 | |||
476 | 37 | relations: | ||
477 | 38 | - - ubuntu | ||
478 | 39 | - filebeat | ||
479 | 40 | - - graylog:beats | ||
480 | 41 | - filebeat:logstash | ||
481 | 42 | - - graylog | ||
482 | 43 | - mongo | ||
483 | 44 | - - graylog | ||
484 | 45 | - elastic | ||
485 | 46 | - - graylog | ||
486 | 47 | - nrpe | ||
487 | 48 | - - nagios | ||
488 | 49 | - nrpe | ||
489 | 50 | - - easyrsa:client | ||
490 | 51 | - graylog:certificates | ||
491 | diff --git a/src/tests/functional/tests/bundles/focal-graylog3.yaml b/src/tests/functional/tests/bundles/focal-graylog3.yaml | |||
492 | index cb26f52..e2e2a41 100644 | |||
493 | --- a/src/tests/functional/tests/bundles/focal-graylog3.yaml | |||
494 | +++ b/src/tests/functional/tests/bundles/focal-graylog3.yaml | |||
495 | @@ -30,6 +30,10 @@ applications: | |||
496 | 30 | nrpe: | 30 | nrpe: |
497 | 31 | charm: cs:nrpe | 31 | charm: cs:nrpe |
498 | 32 | 32 | ||
499 | 33 | nagios: | ||
500 | 34 | charm: cs:nagios | ||
501 | 35 | num_units: 1 | ||
502 | 36 | |||
503 | 33 | relations: | 37 | relations: |
504 | 34 | - - ubuntu | 38 | - - ubuntu |
505 | 35 | - filebeat | 39 | - filebeat |
506 | @@ -43,3 +47,5 @@ relations: | |||
507 | 43 | - haproxy | 47 | - haproxy |
508 | 44 | - - graylog | 48 | - - graylog |
509 | 45 | - nrpe | 49 | - nrpe |
510 | 50 | - - nagios | ||
511 | 51 | - nrpe | ||
512 | diff --git a/src/tests/functional/tests/test_graylog_charm.py b/src/tests/functional/tests/test_graylog_charm.py | |||
513 | index df763b6..83d58e7 100644 | |||
514 | --- a/src/tests/functional/tests/test_graylog_charm.py | |||
515 | +++ b/src/tests/functional/tests/test_graylog_charm.py | |||
516 | @@ -2,12 +2,13 @@ | |||
517 | 2 | 2 | ||
518 | 3 | import logging | 3 | import logging |
519 | 4 | import time | 4 | import time |
520 | 5 | import unittest | ||
521 | 6 | 5 | ||
522 | 7 | from api import GraylogApi | 6 | from api import GraylogApi |
523 | 8 | 7 | ||
524 | 9 | import tenacity | 8 | import tenacity |
525 | 10 | 9 | ||
526 | 10 | from tests.base import BaseTestCase | ||
527 | 11 | |||
528 | 11 | import zaza.model as model | 12 | import zaza.model as model |
529 | 12 | 13 | ||
530 | 13 | 14 | ||
531 | @@ -17,7 +18,7 @@ DEFAULT_API_PORT = "9001" | |||
532 | 17 | DEFAULT_WEB_PORT = "9000" | 18 | DEFAULT_WEB_PORT = "9000" |
533 | 18 | 19 | ||
534 | 19 | 20 | ||
536 | 20 | class BaseGraylogTest(unittest.TestCase): | 21 | class BaseGraylogTest(BaseTestCase): |
537 | 21 | """Base for Graylog charm tests.""" | 22 | """Base for Graylog charm tests.""" |
538 | 22 | 23 | ||
539 | 23 | @classmethod | 24 | @classmethod |
540 | @@ -56,10 +57,14 @@ class BaseGraylogTest(unittest.TestCase): | |||
541 | 56 | "org.graylog.plugins.threatintel.ThreatIntelPlugin", | 57 | "org.graylog.plugins.threatintel.ThreatIntelPlugin", |
542 | 57 | } | 58 | } |
543 | 58 | 59 | ||
545 | 59 | api_url = "http://{}:{}/api".format(cls.graylog_ip, cls.api_port) | 60 | api_url = "{}://{}:{}/api".format( |
546 | 61 | cls.get_api_protocol(), cls.graylog_ip, cls.api_port | ||
547 | 62 | ) | ||
548 | 60 | action = model.run_action_on_leader(cls.application_name, "show-admin-password") | 63 | action = model.run_action_on_leader(cls.application_name, "show-admin-password") |
549 | 61 | res = action.data["results"] | 64 | res = action.data["results"] |
551 | 62 | cls.api = GraylogApi(api_url, "admin", res["admin-password"]) | 65 | cls.api = GraylogApi( |
552 | 66 | api_url, "admin", res["admin-password"], verify=cls.ca_path | ||
553 | 67 | ) | ||
554 | 63 | # try harder to get an api connection to cater for overloaded testsystems | 68 | # try harder to get an api connection to cater for overloaded testsystems |
555 | 64 | cls.api.req_timeout = REQ_TIMEOUT | 69 | cls.api.req_timeout = REQ_TIMEOUT |
556 | 65 | logging.debug("API at {}".format(api_url)) | 70 | logging.debug("API at {}".format(api_url)) |
557 | @@ -75,7 +80,9 @@ class CharmOperationTest(BaseGraylogTest): | |||
558 | 75 | until the CURL_TIMEOUT because it may take a few seconds for the | 80 | until the CURL_TIMEOUT because it may take a few seconds for the |
559 | 76 | graylog systemd service to start. | 81 | graylog systemd service to start. |
560 | 77 | """ | 82 | """ |
562 | 78 | curl_command = "curl http://localhost:{}".format(self.api_port) | 83 | curl_command = "curl {}://localhost:{}".format( |
563 | 84 | self.get_api_protocol(), self.api_port | ||
564 | 85 | ) | ||
565 | 79 | timeout = time.time() + CURL_TIMEOUT | 86 | timeout = time.time() + CURL_TIMEOUT |
566 | 80 | while time.time() < timeout: | 87 | while time.time() < timeout: |
567 | 81 | response = model.run_on_unit(self.lead_unit_name, curl_command) | 88 | response = model.run_on_unit(self.lead_unit_name, curl_command) |
568 | diff --git a/src/tests/functional/tests/test_legacy.py b/src/tests/functional/tests/test_legacy.py | |||
569 | index 5ee2eea..fcea119 100644 | |||
570 | --- a/src/tests/functional/tests/test_legacy.py | |||
571 | +++ b/src/tests/functional/tests/test_legacy.py | |||
572 | @@ -1,9 +1,14 @@ | |||
573 | 1 | """Graylog v2 legacy tests.""" | 1 | """Graylog v2 legacy tests.""" |
574 | 2 | import logging | ||
575 | 2 | import re | 3 | import re |
576 | 3 | import unittest | 4 | import unittest |
577 | 4 | 5 | ||
578 | 5 | from api import GraylogApi | 6 | from api import GraylogApi |
579 | 6 | 7 | ||
580 | 8 | from charmhelpers.core.decorators import retry_on_exception | ||
581 | 9 | |||
582 | 10 | from tests.base import BaseTestCase | ||
583 | 11 | |||
584 | 7 | import yaml | 12 | import yaml |
585 | 8 | 13 | ||
586 | 9 | from zaza import model | 14 | from zaza import model |
587 | @@ -14,7 +19,7 @@ DEFAULT_API_PORT = "9001" # Graylog 2 only | |||
588 | 14 | DEFAULT_WEB_PORT = "9000" | 19 | DEFAULT_WEB_PORT = "9000" |
589 | 15 | 20 | ||
590 | 16 | 21 | ||
592 | 17 | class LegacyTests(unittest.TestCase): | 22 | class LegacyTests(BaseTestCase): |
593 | 18 | """These tests were ported from tests/test_10_basic.py in changeset a3b54c2. | 23 | """These tests were ported from tests/test_10_basic.py in changeset a3b54c2. |
594 | 19 | 24 | ||
595 | 20 | They were temporarily deleted during the conversion from Amulet to Zaza. | 25 | They were temporarily deleted during the conversion from Amulet to Zaza. |
596 | @@ -24,8 +29,12 @@ class LegacyTests(unittest.TestCase): | |||
597 | 24 | 29 | ||
598 | 25 | def test_api_ready(self): | 30 | def test_api_ready(self): |
599 | 26 | """Curl the api endpoint on the graylog unit.""" | 31 | """Curl the api endpoint on the graylog unit.""" |
600 | 32 | protocol = self.get_api_protocol() | ||
601 | 27 | port = self.get_api_port() | 33 | port = self.get_api_port() |
603 | 28 | curl_command = "curl http://localhost:{} --connect-timeout 30".format(port) | 34 | curl_command = ("curl {}://localhost:{} " "--connect-timeout 30").format( |
604 | 35 | protocol, port | ||
605 | 36 | ) | ||
606 | 37 | logging.info(curl_command) | ||
607 | 29 | self.run_command("graylog/0", curl_command) | 38 | self.run_command("graylog/0", curl_command) |
608 | 30 | 39 | ||
609 | 31 | def test_elasticsearch_active(self): | 40 | def test_elasticsearch_active(self): |
610 | @@ -60,10 +69,16 @@ class LegacyTests(unittest.TestCase): | |||
611 | 60 | """Return the list of API clients to Graylog units.""" | 69 | """Return the list of API clients to Graylog units.""" |
612 | 61 | graylog_units = juju.get_full_juju_status().applications["graylog"]["units"] | 70 | graylog_units = juju.get_full_juju_status().applications["graylog"]["units"] |
613 | 62 | graylog_addrs = [unit["public-address"] for unit in graylog_units.values()] | 71 | graylog_addrs = [unit["public-address"] for unit in graylog_units.values()] |
614 | 72 | protocol = self.get_api_protocol() | ||
615 | 63 | port = self.get_api_port() | 73 | port = self.get_api_port() |
616 | 64 | password = self.get_graylog_admin_password() | 74 | password = self.get_graylog_admin_password() |
617 | 65 | return [ | 75 | return [ |
619 | 66 | GraylogApi("http://{}:{}/api".format(graylog_addr, port), "admin", password) | 76 | GraylogApi( |
620 | 77 | "{}://{}:{}/api".format(protocol, graylog_addr, port), | ||
621 | 78 | "admin", | ||
622 | 79 | password, | ||
623 | 80 | verify=self.ca_path, | ||
624 | 81 | ) | ||
625 | 67 | for graylog_addr in graylog_addrs | 82 | for graylog_addr in graylog_addrs |
626 | 68 | ] | 83 | ] |
627 | 69 | 84 | ||
628 | @@ -76,8 +91,17 @@ class LegacyTests(unittest.TestCase): | |||
629 | 76 | password = result.data["results"]["admin-password"] | 91 | password = result.data["results"]["admin-password"] |
630 | 77 | return password | 92 | return password |
631 | 78 | 93 | ||
633 | 79 | def test_website_active(self): | 94 | def test_website_active_with_haproxy(self): |
634 | 80 | """Verify if the Graylog endpoints are correctly configured.""" | 95 | """Verify if the Graylog endpoints are correctly configured.""" |
635 | 96 | try: | ||
636 | 97 | model.get_application("haproxy") | ||
637 | 98 | except KeyError: | ||
638 | 99 | reason = ( | ||
639 | 100 | "haproxy is not in the model, assuming there is " | ||
640 | 101 | "no front loadbalancer" | ||
641 | 102 | ) | ||
642 | 103 | raise unittest.SkipTest(reason) | ||
643 | 104 | |||
644 | 81 | data = juju.get_relation_from_unit("haproxy/0", "graylog/0", "website") | 105 | data = juju.get_relation_from_unit("haproxy/0", "graylog/0", "website") |
645 | 82 | self.assertEqual(data["port"], DEFAULT_WEB_PORT) | 106 | self.assertEqual(data["port"], DEFAULT_WEB_PORT) |
646 | 83 | 107 | ||
647 | @@ -117,6 +141,7 @@ class LegacyTests(unittest.TestCase): | |||
648 | 117 | ) | 141 | ) |
649 | 118 | self.assertTrue(re.search(r"CRITICAL", cfg)) | 142 | self.assertTrue(re.search(r"CRITICAL", cfg)) |
650 | 119 | 143 | ||
651 | 144 | @retry_on_exception(num_retries=5, base_delay=2, exc_type=AssertionError) | ||
652 | 120 | def get_file_contents(self, unit, filename): | 145 | def get_file_contents(self, unit, filename): |
653 | 121 | """Retrieve content of a file in a remote unit.""" | 146 | """Retrieve content of a file in a remote unit.""" |
654 | 122 | result = self.run_command(unit, "cat '{}'".format(filename)) | 147 | result = self.run_command(unit, "cat '{}'".format(filename)) |
655 | diff --git a/src/tests/functional/tests/tests.yaml b/src/tests/functional/tests/tests.yaml | |||
656 | index c52c200..47db931 100644 | |||
657 | --- a/src/tests/functional/tests/tests.yaml | |||
658 | +++ b/src/tests/functional/tests/tests.yaml | |||
659 | @@ -1,9 +1,11 @@ | |||
660 | 1 | charm_name: graylog | 1 | charm_name: graylog |
661 | 2 | gate_bundles: | 2 | gate_bundles: |
662 | 3 | - gl2: bionic-graylog2 | 3 | - gl2: bionic-graylog2 |
663 | 4 | - gl3: bionic-graylog3 | ||
664 | 5 | - gl2: focal-graylog2 | 4 | - gl2: focal-graylog2 |
665 | 5 | - gl3: bionic-graylog3 | ||
666 | 6 | - gl3: bionic-graylog3-tls | ||
667 | 6 | - gl3: focal-graylog3 | 7 | - gl3: focal-graylog3 |
668 | 8 | - gl3: focal-graylog3-tls | ||
669 | 7 | # - gl2: bionic-graylog2-ha | 9 | # - gl2: bionic-graylog2-ha |
670 | 8 | # - gl3: bionic-graylog3-ha | 10 | # - gl3: bionic-graylog3-ha |
671 | 9 | smoke_bundles: | 11 | smoke_bundles: |
672 | @@ -28,3 +30,15 @@ target_deploy_status: | |||
673 | 28 | nrpe: | 30 | nrpe: |
674 | 29 | workload-status: blocked | 31 | workload-status: blocked |
675 | 30 | workload-status-message: Nagios server not configured or related | 32 | workload-status-message: Nagios server not configured or related |
676 | 33 | mongo: | ||
677 | 34 | workload-status: active | ||
678 | 35 | workload-status-message: 'Unit is ready' | ||
679 | 36 | graylog: | ||
680 | 37 | workload-status: active | ||
681 | 38 | workload-status-message: 'Ready with: filebeat, elasticsearch, mongodb' | ||
682 | 39 | elastic: | ||
683 | 40 | workload-status: active | ||
684 | 41 | workload-status-message: 'Unit is ready' | ||
685 | 42 | easyrsa: | ||
686 | 43 | workload-status-message: 'Certificate Authority connected.' | ||
687 | 44 | |||
688 | diff --git a/src/tests/unit/test_graylog.py b/src/tests/unit/test_graylog.py | |||
689 | index c723ce1..a9b0ae8 100644 | |||
690 | --- a/src/tests/unit/test_graylog.py | |||
691 | +++ b/src/tests/unit/test_graylog.py | |||
692 | @@ -1,10 +1,15 @@ | |||
693 | 1 | """Tests around the graylog reactive script.""" | 1 | """Tests around the graylog reactive script.""" |
694 | 2 | import os | 2 | import os |
695 | 3 | import socket | ||
696 | 3 | import sys | 4 | import sys |
697 | 4 | import tempfile | 5 | import tempfile |
698 | 5 | import unittest | 6 | import unittest |
699 | 6 | from unittest import mock | 7 | from unittest import mock |
700 | 7 | 8 | ||
701 | 9 | from charms.layer.graylog.constants import ( | ||
702 | 10 | CERT_KEY_PATH, | ||
703 | 11 | CERT_PATH, | ||
704 | 12 | ) | ||
705 | 8 | from charms.layer.graylog.snap_change import ( | 13 | from charms.layer.graylog.snap_change import ( |
706 | 9 | ChannelChangeStatus, | 14 | ChannelChangeStatus, |
707 | 10 | _is_channel_valid, | 15 | _is_channel_valid, |
708 | @@ -17,6 +22,8 @@ leader_mock = mock.Mock() | |||
709 | 17 | sys.modules["charms.leadership"] = leader_mock | 22 | sys.modules["charms.leadership"] = leader_mock |
710 | 18 | snap_mock = mock.Mock() | 23 | snap_mock = mock.Mock() |
711 | 19 | sys.modules["charms.layer.snap"] = snap_mock | 24 | sys.modules["charms.layer.snap"] = snap_mock |
712 | 25 | tls_client_mock = mock.Mock() | ||
713 | 26 | sys.modules["charms.layer.tls_client"] = tls_client_mock | ||
714 | 20 | 27 | ||
715 | 21 | from files import check_graylog_health # noqa: E402 | 28 | from files import check_graylog_health # noqa: E402 |
716 | 22 | 29 | ||
717 | @@ -28,6 +35,7 @@ from reactive.graylog import ( # noqa: E402 | |||
718 | 28 | refresh_graylog, | 35 | refresh_graylog, |
719 | 29 | set_conf, | 36 | set_conf, |
720 | 30 | set_jvm_heap_size, | 37 | set_jvm_heap_size, |
721 | 38 | tls_request_certificate, | ||
722 | 31 | update_config, | 39 | update_config, |
723 | 32 | upgrade_charm, | 40 | upgrade_charm, |
724 | 33 | ) | 41 | ) |
725 | @@ -555,3 +563,81 @@ class TestIsChannelValid(unittest.TestCase): | |||
726 | 555 | "2", # track w/o risk | 563 | "2", # track w/o risk |
727 | 556 | ): | 564 | ): |
728 | 557 | self.assertFalse(_is_channel_valid(channel)) | 565 | self.assertFalse(_is_channel_valid(channel)) |
729 | 566 | |||
730 | 567 | |||
731 | 568 | class TestTlsClient(unittest.TestCase): | ||
732 | 569 | """Test TLS Client integration.""" | ||
733 | 570 | |||
734 | 571 | @mock.patch("charmhelpers.core.host.mkdir") | ||
735 | 572 | @mock.patch("charmhelpers.core.host.write_file") | ||
736 | 573 | @mock.patch("charmhelpers.core.hookenv.charm_dir") | ||
737 | 574 | @mock.patch("charms.layer.graylog.utils.is_v2") | ||
738 | 575 | @mock.patch("reactive.graylog.set_conf") | ||
739 | 576 | @mock.patch("reactive.graylog.hookenv.unit_public_ip") | ||
740 | 577 | @mock.patch("reactive.graylog.hookenv.unit_private_ip") | ||
741 | 578 | @mock.patch("reactive.graylog.install") | ||
742 | 579 | @mock.patch("subprocess.check_call") | ||
743 | 580 | def test_request_certificate( | ||
744 | 581 | self, | ||
745 | 582 | mock_check_call, | ||
746 | 583 | mock_install, | ||
747 | 584 | mock_private_ip, | ||
748 | 585 | mock_public_ip, | ||
749 | 586 | mock_set_conf, | ||
750 | 587 | mock_is_v2, | ||
751 | 588 | mock_charm_dir, | ||
752 | 589 | mock_write_file, | ||
753 | 590 | mock_mkdir, | ||
754 | 591 | ): # noqa: D102 | ||
755 | 592 | charm_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../") | ||
756 | 593 | mock_charm_dir.return_value = charm_dir | ||
757 | 594 | mock_is_v2.return_value = False | ||
758 | 595 | mock_public_ip.return_value = "1.2.3.4" | ||
759 | 596 | mock_private_ip.return_value = "5.6.7.8" | ||
760 | 597 | mock_tls = mock.Mock() | ||
761 | 598 | tls_request_certificate(mock_tls) | ||
762 | 599 | |||
763 | 600 | tls_client_mock.request_server_cert.assert_called_with( | ||
764 | 601 | "1.2.3.4", | ||
765 | 602 | ["1.2.3.4", "5.6.7.8", socket.gethostname(), "127.0.0.1", "localhost"], | ||
766 | 603 | crt_path=CERT_PATH, | ||
767 | 604 | key_path=CERT_KEY_PATH, | ||
768 | 605 | ) | ||
769 | 606 | mock_set_conf.assert_has_calls( | ||
770 | 607 | [ | ||
771 | 608 | mock.call("rest_enable_tls", "true"), | ||
772 | 609 | mock.call("rest_tls_cert_file", CERT_PATH), | ||
773 | 610 | mock.call("rest_tls_key_file", CERT_KEY_PATH), | ||
774 | 611 | mock.call("web_enable_tls", "true"), | ||
775 | 612 | mock.call("web_tls_cert_file", CERT_PATH), | ||
776 | 613 | mock.call("web_tls_key_file", CERT_KEY_PATH), | ||
777 | 614 | ] | ||
778 | 615 | ) | ||
779 | 616 | |||
780 | 617 | with open(os.path.join(charm_dir, "templates/sync-graylog-snap"), "rb") as f: | ||
781 | 618 | sync_script = f.read().strip() | ||
782 | 619 | |||
783 | 620 | with open( | ||
784 | 621 | os.path.join(charm_dir, "templates/default-graylog-server"), "rb" | ||
785 | 622 | ) as f: | ||
786 | 623 | default_graylog_server = f.read().strip() | ||
787 | 624 | |||
788 | 625 | mock_write_file.assert_has_calls( | ||
789 | 626 | [ | ||
790 | 627 | mock.call( | ||
791 | 628 | "/etc/ca-certificates/update.d/sync-graylog-snap", | ||
792 | 629 | sync_script, | ||
793 | 630 | "root", | ||
794 | 631 | "root", | ||
795 | 632 | 0o755, | ||
796 | 633 | ), | ||
797 | 634 | mock.call( | ||
798 | 635 | "/var/snap/graylog/current/default-graylog-server", | ||
799 | 636 | default_graylog_server, | ||
800 | 637 | "root", | ||
801 | 638 | "root", | ||
802 | 639 | 0o644, | ||
803 | 640 | ), | ||
804 | 641 | ] | ||
805 | 642 | ) | ||
806 | 643 | mock_check_call.assert_called_with(["update-ca-certificates"]) | ||
807 | diff --git a/src/tests/unit/test_lib.py b/src/tests/unit/test_lib.py | |||
808 | index 2f7a7c3..5849ac5 100644 | |||
809 | --- a/src/tests/unit/test_lib.py | |||
810 | +++ b/src/tests/unit/test_lib.py | |||
811 | @@ -12,17 +12,29 @@ from charms.layer.graylog import ( | |||
812 | 12 | class TestLibraryUtils(unittest.TestCase): | 12 | class TestLibraryUtils(unittest.TestCase): |
813 | 13 | """Test lib.charms.layer.graylog utils.""" | 13 | """Test lib.charms.layer.graylog utils.""" |
814 | 14 | 14 | ||
815 | 15 | @mock.patch("charms.layer.graylog.utils.get_flags") | ||
816 | 15 | @mock.patch("charms.layer.graylog.utils.is_v2") | 16 | @mock.patch("charms.layer.graylog.utils.is_v2") |
818 | 16 | def test_api(self, mock_v2): | 17 | def test_api(self, mock_v2, mock_get_flags): |
819 | 17 | """Validate the api port/url match what we expect for v2 and v3.""" | 18 | """Validate the api port/url match what we expect for v2 and v3.""" |
820 | 18 | mock_v2.return_value = True | 19 | mock_v2.return_value = True |
821 | 19 | self.assertEqual(get_api_port(), "9001") | 20 | self.assertEqual(get_api_port(), "9001") |
822 | 21 | mock_get_flags.return_value = [] | ||
823 | 20 | self.assertEqual(get_api_url(), "http://127.0.0.1:9001/api/") | 22 | self.assertEqual(get_api_url(), "http://127.0.0.1:9001/api/") |
824 | 21 | 23 | ||
825 | 24 | mock_get_flags.reset_mock() | ||
826 | 25 | mock_get_flags.return_value = ["graylog.certificates.configured"] | ||
827 | 26 | self.assertEqual(get_api_url(), "https://127.0.0.1:9001/api/") | ||
828 | 27 | |||
829 | 22 | mock_v2.return_value = False | 28 | mock_v2.return_value = False |
830 | 23 | self.assertEqual(get_api_port(), "9000") | 29 | self.assertEqual(get_api_port(), "9000") |
831 | 30 | mock_get_flags.reset_mock() | ||
832 | 31 | mock_get_flags.return_value = [] | ||
833 | 24 | self.assertEqual(get_api_url(), "http://127.0.0.1:9000/api/") | 32 | self.assertEqual(get_api_url(), "http://127.0.0.1:9000/api/") |
834 | 25 | 33 | ||
835 | 34 | mock_get_flags.reset_mock() | ||
836 | 35 | mock_get_flags.return_value = ["graylog.certificates.configured"] | ||
837 | 36 | self.assertEqual(get_api_url(), "https://127.0.0.1:9000/api/") | ||
838 | 37 | |||
839 | 26 | @mock.patch("charms.layer.graylog.utils.is_v2") | 38 | @mock.patch("charms.layer.graylog.utils.is_v2") |
840 | 27 | def test_validate_api_url(self, mock_v2): | 39 | def test_validate_api_url(self, mock_v2): |
841 | 28 | """Validate the URI suffix is valid for v2 and v3.""" | 40 | """Validate the URI suffix is valid for v2 and v3.""" |
This review is large, but I actually found the overall review to be good. I have no real critical feedback.
Re: the zaza bundles, something we've been adopting in other charms to reduce duplication is to have a single base bundle with all the common apps/relations ("base.yaml", not listed in tests.yaml), replace the existing bundles with symlinks to that base bundle, and then have all the per-bundle customizations moved to overlays. I don't think that's critical for this MR, but I'm mentioning it as it seems a good practice to reduce duplication and the chance of accidental divergence between bundles.
Re: code changes: the only thing I would suggest is that "make black" is run; src/tests/ functional/ tests/test_ graylog_ upgrade. py is not passing "make lint" but will simply by running "make black" and committing the changes.
I'll get this running a full test suite on my charmlab box... I'm +1 once the "make black" changes are in, assuming no failures arise during tests.