Merge ~gtrkiller/charm-telegraf:master into charm-telegraf:master
- Git
- lp:~gtrkiller/charm-telegraf
- master
- Merge into master
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Eric Chen | ||||
Approved revision: | fc1125432c55d0662ba3fcdb4020caa249fc56af | ||||
Merged at revision: | fcb42f8dcefd7e47f86ad6de21bc99d85868f714 | ||||
Proposed branch: | ~gtrkiller/charm-telegraf:master | ||||
Merge into: | charm-telegraf:master | ||||
Diff against target: |
606 lines (+355/-62) 3 files modified
src/config.yaml (+10/-0) src/reactive/telegraf.py (+173/-49) src/tests/unit/test_telegraf.py (+172/-13) |
||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Eric Chen | Approve | ||
š¤ prod-jenkaas-bootstack (community) | continuous-integration | Approve | |
Tom Haddon | Approve | ||
Arturo Enrique Seijas FernƔndez (community) | Approve | ||
Robert Gildein | Approve | ||
Johann David Krister Andersson (community) | Approve | ||
Tianqi Xiao | Pending | ||
BootStack Reviewers | Pending | ||
Review via email: mp+428104@code.launchpad.net |
Commit message
Description of the change
The whole address binding has been reworked. Now we can configure prometheus to bind to a network (and check if the unit's IP matches it), bind on a single IP or interface name.
š¤ Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
Tom Haddon (mthaddon) wrote : | # |
I don't think this models the behaviour we outlined in the spec, and is certainly missing a number of tests for that behaviour. The details of that behaviour are:
* Create a new configuration option called prometheus_
* This new option would be a string variable that would accept:
** An IP and port (in the simple case where the operator knows thereās only ever going to be one instance) e.g. ā10.1.5.23:1111ā or IPv6 equivalent.
** An IP range and port (in which case each unit would configure itself to listen on the IP that falls within that range. In case of more than one interface matching, or no interface matching, the charm would go into āblockedā status) e.g. ā10.1.5.0/24:1111ā or IPv6 equivalent.
** An interface and port (in this case each unit would configure itself to listen on that interface only. If any unit didnāt have an interface of that name, the charm would go into āblockedā status) e.g. āeth1:1111ā (defaults to IPv4 if available, falls back to IPv6) or āipv4:eth1:1111ā or āipv6:eth1:1111ā.
** A port on its own to specify listening on that port on all IPs - e.g. `:1111`.
* The charm would only accept one of the above types of strings as an option. Trying to set more than one (via comma-separation, for instance) would cause the charm to go into āblockedā status.
Assuming the configuration option matches one of the above options, and all instances of the application have a matching IP/interface the charm would configure each unit to bind only on that IP/interface and port for prometheus output.
* Any port defined here would override the port specified in the prometheus_
* At some point in the future weād then update the charm to set it to blocked status if someone has this set to something other than ādefaultā and then finally weād remove this option altogether. These wonāt be tackled as part of this charm change, however.
Can you take another look at this and make sure we're testing these different cases?
Tom Haddon (mthaddon) wrote : | # |
Some comments inline
Eric Chen (eric-chen) wrote : | # |
Mark WIP first until 2nd patch provided.
Franco Luciano Forneron Buschiazzo (gtrkiller) : | # |
Robert Gildein (rgildein) : | # |
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote : | # |
Marking as needs review.
Tom Haddon (mthaddon) wrote : | # |
Some comments inline.
Also, I feel like we could do with some more unit tests here. In each case we're just testing the `generate_
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote : | # |
Sure! I'll add some more tests. Just ftr, get_prometheus_
Eric Chen (eric-chen) wrote : | # |
Mark this as WIP before the test are finished.
Arturo Enrique Seijas FernƔndez (arturo-seijas) : | # |
Franco Luciano Forneron Buschiazzo (gtrkiller) : | # |
Johann David Krister Andersson (jdkandersson) wrote : | # |
Added a few comments in line
Franco Luciano Forneron Buschiazzo (gtrkiller) : | # |
Franco Luciano Forneron Buschiazzo (gtrkiller) : | # |
Franco Luciano Forneron Buschiazzo (gtrkiller) : | # |
Johann David Krister Andersson (jdkandersson) : | # |
Robert Gildein (rgildein) wrote : | # |
Overall LGTM.
Can you fix lint tests [1] and rebase this on new master branch?
---
[1]: https:/
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote : | # |
> Overall LGTM.
>
> Can you fix lint tests [1] and rebase this on new master branch?
>
> ---
> [1]: https:/
The two lint linebreak errors left have a conflict and cannot be fixed for now. If I format the code with black, flake8 throws an error. If I fix flake8's error, then black wants to reformat the file.
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote : | # |
I'll add a comment to fix the conflict
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote : | # |
The master branch has been rebased and the comments for linting been made. Now we have 5 failing tests that have nothing to do with my changes, should I fix them?
Johann David Krister Andersson (jdkandersson) : | # |
Franco Luciano Forneron Buschiazzo (gtrkiller) : | # |
Robert Gildein (rgildein) wrote : | # |
LGTM
> I'll add a comment to fix the conflict
Using `# noqa: W503` is the best approach here.
> The master branch has been rebased and the comments for linting been made. Now we have 5 failing tests that have nothing to do with my changes, should I fix them?
We know about it and there is MP with fix for it, so you can ignore it.
Eric Chen (eric-chen) wrote : | # |
Citeria to merge the PR
- All lint/unit/func test are pass (5 failing tests should have fixed by James Lin, if you still encounter it, please let me know)
- At least 2 approval. (it's okay now)
- All "Need Fixing" is gone, so, please Tom Haddon and Arturo Enrique Seijas FernƔndez review it again.
Thanks!
Arturo Enrique Seijas FernƔndez (arturo-seijas) wrote : | # |
LGTM
Tom Haddon (mthaddon) wrote : | # |
LGTM, thx. One minor comment about asking for the accepted values to be documented in the config description, but otherwise fine to go ahead from my perspective.
Eric Chen (eric-chen) wrote : | # |
Franco, please provide the log of lint/unit/func test, then we can merge it.
Eric Chen (eric-chen) wrote : | # |
I found there are inline comment from
Arturo Enrique Seijas FernƔndez
Johann David Krister Andersson
Please fix it too.
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote : | # |
> I found there are inline comment from
>
> Arturo Enrique Seijas FernƔndez
> Johann David Krister Andersson
>
> Please fix it too.
Their comments (and Tom's) have been addressed now
Eric Chen (eric-chen) wrote : | # |
Great! The final thing:
Sorry that we don't integrate the Jenkins with charm-telegraf now, we will do it soon. Please provide the log of lint/unit/func test, then we can merge it. It's the last mile!
š¤ prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:55aa1ffdc50
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Robert Gildein (rgildein) wrote : | # |
The Jenkins is configured for charm-telegraf, but this MP was created
in times when Jenkins was out of duty. So I triggered the tests manually.
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote : | # |
> Great! The final thing:
> Sorry that we don't integrate the Jenkins with charm-telegraf now, we will do
> it soon. Please provide the log of lint/unit/func test, then we can merge it.
> It's the last mile!
The linting is OK now, the unit tests have the same 5 errors (ignored) that I had when I tested the original master branch and then rebased. The func tests cannot be triggered with tox -e func apparently, so we may want to update that (or if it should be triggered on a certain way idk of, please tell me)
Eric Chen (eric-chen) wrote : | # |
Hi Franco,
The lint/unit/func are all pass in
https:/
So I think you can pass all of them too. If you need help, you can contact Robert. You can also retrigger the build again too.
Click here to trigger a rebuild:
https:/
š¤ prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:55aa1ffdc50
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Tom Haddon (mthaddon) wrote : | # |
> Hi Franco,
>
> The lint/unit/func are all pass in
> https:/
> telegraf/
>
> So I think you can pass all of them too. If you need help, you can contact
> Robert. You can also retrigger the build again too.
>
> Click here to trigger a rebuild:
> https:/
I triggered a rebuild here https:/
š¤ prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:38064851f26
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Robert Gildein (rgildein) wrote : | # |
Hi Tom, you started the rebuild, but on the old commit. I tried to
rebuilt it again on the last commit here but failed with same error.
I proposed new MP [1], which fixes this [2].
---
[1]: https:/
[2]: https:/
Eric Chen (eric-chen) wrote : | # |
1. There is merge conflict, please check my inline comment
2. MP[1] is merged. Please rebase to the lastest and trigger Jenkins CI .
I hope we can merge this time.
thanks!
[1]: https:/
š¤ prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:38064851f26
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
š¤ prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:38064851f26
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
š¤ prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
PASSED: Continuous integration, rev:fc1125432c5
https:/
Executed test runs:
SUCCESS: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Eric Chen (eric-chen) : | # |
š¤ Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
Change successfully merged at revision fcb42f8dcefd7e4
Preview Diff
1 | diff --git a/src/config.yaml b/src/config.yaml |
2 | index bc072ad..282a163 100644 |
3 | --- a/src/config.yaml |
4 | +++ b/src/config.yaml |
5 | @@ -75,10 +75,20 @@ options: |
6 | (eg. service_name-0). {uuid} is replaced by the model UUID, for |
7 | sites without unique model names. {host} is replaced by the |
8 | machine hostname |
9 | + prometheus_output_bindaddress: |
10 | + type: string |
11 | + default: "" |
12 | + description: > |
13 | + If set prometheus output plugin will be configured to listen on the provided address. |
14 | + If set to string "default" the charm will use the default port (9103). |
15 | + Accepted values are IPs, network ranges and interface names, |
16 | + followed by a port, e.g. "10.0.0.1:9000", "10.0.0.0/24:9101", |
17 | + "2001:0db8:85a3:0000:0000:8a2e:0370:7334:9101" or "eth0:9101" |
18 | prometheus_output_port: |
19 | type: string |
20 | default: "9103" |
21 | description: > |
22 | + WARNING: This config option is deprecated in favor of prometheus_output_bindaddress. Please use that option instead. |
23 | If set prometheus output plugin will be configured to listen on the provided port. |
24 | If set to string "default" the charm will use default port (9103) |
25 | inputs_config: |
26 | diff --git a/src/reactive/telegraf.py b/src/reactive/telegraf.py |
27 | index abbb728..7e10d77 100644 |
28 | --- a/src/reactive/telegraf.py |
29 | +++ b/src/reactive/telegraf.py |
30 | @@ -29,6 +29,7 @@ import subprocess |
31 | import sys |
32 | import time |
33 | from distutils.version import LooseVersion |
34 | +from enum import Enum |
35 | |
36 | import charms.promreg |
37 | import netifaces |
38 | @@ -123,6 +124,12 @@ class InvalidIntelRDTConfigurationError(Exception): |
39 | pass |
40 | |
41 | |
42 | +class Address(Enum): |
43 | + IPV4 = 1 |
44 | + IPV6 = 2 |
45 | + INVALID = 3 |
46 | + |
47 | + |
48 | def is_intel_rdt_enabled(): |
49 | config = hookenv.config() |
50 | return config.get("collect_intel_rdt_metrics", False) and is_metal() |
51 | @@ -562,13 +569,34 @@ def check_prometheus_port(key, new_port): |
52 | charms.promreg.register(None, new_port) |
53 | |
54 | |
55 | -def get_prometheus_port(): |
56 | +def get_prometheus_bindaddress(): |
57 | + Bindaddress = collections.namedtuple("Bindaddress", ["port", "address"]) |
58 | + """Get prometheus output IP, port and/or interface from the config.""" |
59 | config = hookenv.config() |
60 | - if not config.get("prometheus_output_port", False): |
61 | - return False |
62 | - if config.get("prometheus_output_port") == "default": |
63 | - return 9103 |
64 | - return int(config.get("prometheus_output_port")) |
65 | + output_bindaddress = config.get("prometheus_output_bindaddress") |
66 | + output_port = config.get("prometheus_output_port") |
67 | + |
68 | + if not output_bindaddress and not output_port: |
69 | + return Bindaddress(False, "") |
70 | + elif not output_bindaddress and output_port != "default": |
71 | + hookenv.log( |
72 | + """WARNING: prometheus_output_port is deprecated in favor of |
73 | + prometheus_output_bindaddress. Please use that option instead.""" |
74 | + ) |
75 | + return Bindaddress(int(output_port), "") |
76 | + elif ( |
77 | + not output_bindaddress and output_port == "default" |
78 | + ) or output_bindaddress == "default": |
79 | + hookenv.log( |
80 | + """WARNING: prometheus_output_port is deprecated in favor of |
81 | + prometheus_output_bindaddress. Please use that option instead.""" |
82 | + ) |
83 | + return Bindaddress(9103, "") |
84 | + elif len(output_bindaddress.rsplit(":", 1)) == 2: |
85 | + address, port = output_bindaddress.rsplit(":", 1) |
86 | + return Bindaddress(int(port), address) |
87 | + else: |
88 | + return Bindaddress(int(output_bindaddress), "") |
89 | |
90 | |
91 | def get_prometheus_ip_range(): |
92 | @@ -1005,16 +1033,21 @@ def handle_config_changes(): |
93 | |
94 | if ( |
95 | config.changed("install_method") |
96 | - or config.changed("snap_channel") # noqa W503 |
97 | - or config.changed("install_sources") # noqa W503 |
98 | + # We have to utilize noqa 503 on 4 lines of code because |
99 | + # black formatting will conflict with flake8 on this 4 lines of code. |
100 | + # If we do not skip the error, one of those will inevitably fail. |
101 | + or config.changed("snap_channel") # noqa: W503 |
102 | + or config.changed("install_sources") # noqa: W503 |
103 | ): |
104 | clear_flag("telegraf.installed") |
105 | clear_flag("extra_plugins.configured") |
106 | clear_flag("plugins.prometheus-client.configured") |
107 | clear_flag("prometheus-client.relation.configured") |
108 | |
109 | - if config.changed("prometheus_output_port") or config.changed( |
110 | - "prometheus_ip_range" |
111 | + if ( |
112 | + config.changed("prometheus_output_port") |
113 | + or config.changed("prometheus_output_bindaddress") # noqa: W503 |
114 | + or config.changed("prometheus_ip_range") # noqa: W503 |
115 | ): |
116 | clear_flag("plugins.prometheus-client.configured") |
117 | clear_flag("prometheus-client.relation.configured") |
118 | @@ -1616,52 +1649,140 @@ def influxdb_api_output(influxdb): |
119 | set_flag("telegraf.needs_reload") |
120 | |
121 | |
122 | -def generate_prometheus_output_config(prometheus_output_port, prometheus_ip_range): |
123 | +def get_ip_type(address): |
124 | + """Validate a string type: ipv4, ipv6 or invalid as IP address.""" |
125 | + try: |
126 | + ip = ipaddress.ip_address(address) |
127 | + |
128 | + if isinstance(ip, ipaddress.IPv4Address): |
129 | + return Address(1).name |
130 | + elif isinstance(ip, ipaddress.IPv6Address): |
131 | + return Address(2).name |
132 | + except ValueError: |
133 | + return Address(3).name |
134 | + |
135 | + |
136 | +def generate_prometheus_output_config( |
137 | + prometheus_output_port, prometheus_output_address, prometheus_ip_range |
138 | +): |
139 | # If extra_options are set for prometheus_client, let's integrate them |
140 | extra_options = get_extra_options() |
141 | options = extra_options["outputs"].get("prometheus_client", {}) |
142 | listen = options.pop("listen", None) |
143 | if not listen: |
144 | - listen = ":{}".format(prometheus_output_port) |
145 | - elif int(listen.split(":", 1)[1]) != prometheus_output_port: |
146 | - hookenv.log( |
147 | - """prometheus_output_port is {}, but extra_options would set it |
148 | - to {}. Choosing {} from prometheus_output_port.""".format( |
149 | - prometheus_output_port, |
150 | - int(listen.split(":", 1)[1]), |
151 | - prometheus_output_port, |
152 | - ), |
153 | - level=hookenv.WARNING, |
154 | - ) |
155 | - listen = "{}:{}".format(listen.split(":", 1)[0], prometheus_output_port) |
156 | - |
157 | - options_ip_range = options.pop("ip_range", []) |
158 | - ip_range = options_ip_range + prometheus_ip_range |
159 | + if prometheus_output_address == "": |
160 | + listen = ":{}".format(prometheus_output_port) |
161 | + elif "/" in prometheus_output_address: |
162 | + listen = get_prometheus_config_network_ip( |
163 | + prometheus_output_address, prometheus_output_port |
164 | + ) |
165 | + elif get_ip_type(prometheus_output_address) != Address(3).name: |
166 | + listen = get_prometheus_config_ip( |
167 | + prometheus_output_address, prometheus_output_port |
168 | + ) |
169 | + else: |
170 | + listen = get_prometheus_config_interface_ip( |
171 | + prometheus_output_address, prometheus_output_port |
172 | + ) |
173 | |
174 | - return { |
175 | - "listen": listen, |
176 | - "ip_range": ip_range, |
177 | - "extra_options": render_extra_options( |
178 | - "outputs", "prometheus_client", extra_options=extra_options |
179 | - ), |
180 | - } |
181 | + options_ip_range = options.pop("ip_range", []) |
182 | + ip_range = options_ip_range + prometheus_ip_range |
183 | + if listen is not None: |
184 | + return { |
185 | + "listen": listen, |
186 | + "ip_range": ip_range, |
187 | + "extra_options": render_extra_options( |
188 | + "outputs", "prometheus_client", extra_options=extra_options |
189 | + ), |
190 | + } |
191 | + |
192 | + |
193 | +def get_prometheus_config_network_ip(prometheus_output_address, prometheus_output_port): |
194 | + """Validate the provided IP network in the config matches the IP of an interface.""" |
195 | + ip_type = get_ip_type(prometheus_output_address.split("/")[0]) |
196 | + interface_list = host.list_nics() |
197 | + count = 0 |
198 | + ip_network = ipaddress.ip_network(prometheus_output_address) |
199 | + for interface_name in interface_list: |
200 | + interface_ip = get_interface_ip(interface_name, ip_type) |
201 | + if ipaddress.ip_address(interface_ip) in ip_network: |
202 | + count += 1 |
203 | + if count == 1: |
204 | + listen = "{}:{}".format(interface_ip, prometheus_output_port) |
205 | + return listen |
206 | + |
207 | + |
208 | +def get_prometheus_config_ip(prometheus_output_address, prometheus_output_port): |
209 | + """Validate the provided IP in the config matches the IP of an interface.""" |
210 | + ip_type = get_ip_type(prometheus_output_address) |
211 | + ip_address = ipaddress.ip_address(prometheus_output_address) |
212 | + interface_list = host.list_nics() |
213 | + for interface_name in interface_list: |
214 | + interface_ip = get_interface_ip(interface_name, ip_type) |
215 | + if ipaddress.ip_address(interface_ip) == ip_address: |
216 | + listen = "{}:{}".format(interface_ip, prometheus_output_port) |
217 | + return listen |
218 | + |
219 | + |
220 | +def get_prometheus_config_interface_ip( |
221 | + prometheus_output_address, prometheus_output_port |
222 | +): |
223 | + """Validate interface with the same name as specified in the config exists.""" |
224 | + interface_list = host.list_nics() |
225 | + if prometheus_output_address.startswith( |
226 | + "ipv4:" |
227 | + ) or prometheus_output_address.startswith("ipv6:"): |
228 | + ip_type = prometheus_output_address[:4] |
229 | + interface_name = prometheus_output_address[5:] |
230 | + for interface in interface_list: |
231 | + if interface == interface_name: |
232 | + interface_ip = get_interface_ip(interface_name, ip_type) |
233 | + listen = "{}:{}".format(interface_ip, prometheus_output_port) |
234 | + return listen |
235 | + else: |
236 | + for interface in interface_list: |
237 | + if interface == prometheus_output_address: |
238 | + interface_ip = get_interface_ip(prometheus_output_address, "ipv4") |
239 | + if not ipaddress.ip_address(interface_ip): |
240 | + interface_ip = get_interface_ip(prometheus_output_address, "ipv6") |
241 | + listen = "{}:{}".format(interface_ip, prometheus_output_port) |
242 | + return listen |
243 | + |
244 | + |
245 | +def get_interface_ip(interface_name, type): |
246 | + """Get the IP from one of the unit's interfaces.""" |
247 | + type_dict = {Address(1).name: "/inet", Address(2).name: "/inet6"} |
248 | + cm = "ip -f inet addr show %s | awk '%s / {print $2}'" % ( |
249 | + interface_name, |
250 | + type_dict[type], |
251 | + ) |
252 | + ps = subprocess.Popen( |
253 | + cm, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT |
254 | + ) |
255 | + interface_ip = ps.communicate()[0].decode(sys.stdout.encoding).split("/")[0] |
256 | + return interface_ip |
257 | |
258 | |
259 | -def render_prometheus_client_config(port, ip_range): |
260 | +def render_prometheus_client_config(port, address, ip_range): |
261 | config_path = "{}/{}.conf".format(get_configs_dir(), "prometheus_client") |
262 | hookenv.log( |
263 | - "Updating {} plugin config file. Port is {} and ip_range is {}".format( |
264 | - "prometheus_client", port, ip_range |
265 | + "Updating {}:{} plugin config file. Port is {} and ip_range is {}".format( |
266 | + "prometheus_client", address, port, ip_range |
267 | ), |
268 | level=hookenv.INFO, |
269 | ) |
270 | - context = generate_prometheus_output_config(port, ip_range) |
271 | - render( |
272 | - source="prometheus_client.tmpl", |
273 | - templates_dir=get_templates_dir(), |
274 | - target=config_path, |
275 | - context=context, |
276 | - ) |
277 | + context = generate_prometheus_output_config(port, address, ip_range) |
278 | + try: |
279 | + render( |
280 | + source="prometheus_client.tmpl", |
281 | + templates_dir=get_templates_dir(), |
282 | + target=config_path, |
283 | + context=context, |
284 | + ) |
285 | + except Exception as e: |
286 | + hookenv.log(e) |
287 | + hookenv.status_set("blocked", "Telegraf failed to start. Check charm's config.") |
288 | + return |
289 | |
290 | |
291 | @when("endpoint.prometheus-client.changed") |
292 | @@ -1670,7 +1791,8 @@ def configure_prometheus_client_with_relation(prometheus): |
293 | "Configuring prometheus_client output plugin, with prometheus-client relation", |
294 | level=hookenv.DEBUG, |
295 | ) |
296 | - port = get_prometheus_port() or "9103" |
297 | + port, address = get_prometheus_bindaddress() |
298 | + hookenv.log("get_prometheus_bindaddress: {}:{}".format(address, str(port))) |
299 | # We'll iterate through the prometheus-client relation counterparts, |
300 | # inform them of our address so that they scrape it, and get their egress subnets |
301 | # so that we can allow them |
302 | @@ -1706,7 +1828,7 @@ def configure_prometheus_client_with_relation(prometheus): |
303 | ip_range = get_prometheus_ip_range() |
304 | if ip_range != []: |
305 | ip_range = ip_range + remote_egress_subnets |
306 | - render_prometheus_client_config(port, ip_range) |
307 | + render_prometheus_client_config(port, address, ip_range) |
308 | set_flag("plugins.prometheus-client.configured") |
309 | set_flag("prometheus-client.relation.configured") |
310 | set_flag("telegraf.needs_reload") |
311 | @@ -1717,15 +1839,16 @@ def configure_prometheus_client_with_relation(prometheus): |
312 | @when_not("plugins.prometheus-client.configured") |
313 | def configure_prometheus_client(): |
314 | hookenv.log("Configuring prometheus_client output plugin", level=hookenv.DEBUG) |
315 | - if get_prometheus_port(): |
316 | - port = get_prometheus_port() |
317 | + if get_prometheus_bindaddress(): |
318 | + port, address = get_prometheus_bindaddress() |
319 | + hookenv.log("get_prometheus_bindaddress: {}:{}".format(address, str(port))) |
320 | else: |
321 | # No relation to prometheus, no port configured: do not configure the plugin |
322 | set_flag("plugins.prometheus-client.configured") |
323 | return |
324 | check_prometheus_port("prometheus_output", port) |
325 | ip_range = get_prometheus_ip_range() |
326 | - render_prometheus_client_config(port, ip_range) |
327 | + render_prometheus_client_config(port, address, ip_range) |
328 | set_flag("plugins.prometheus-client.configured") |
329 | set_flag("telegraf.needs_reload") |
330 | clear_flag("prometheus-client.relation.configured") |
331 | @@ -1998,10 +2121,11 @@ def configure_nagios(nagios): |
332 | nrpe_setup = nrpe.NRPE(hostname=hostname, primary=False) |
333 | |
334 | # use charmhelpers to create a process check |
335 | + port, _ = get_prometheus_bindaddress() |
336 | nrpe_setup.add_check( |
337 | "telegraf_http", |
338 | "Telegraf HTTP check", |
339 | - "check_http -I 127.0.0.1 -p {} -u /metrics".format(get_prometheus_port()), |
340 | + "check_http -I 127.0.0.1 -p {} -u /metrics".format(port), |
341 | ) |
342 | nrpe_setup.write() |
343 | set_flag("telegraf.nagios-setup.complete") |
344 | diff --git a/src/tests/unit/test_telegraf.py b/src/tests/unit/test_telegraf.py |
345 | index 7f409e4..6b453af 100644 |
346 | --- a/src/tests/unit/test_telegraf.py |
347 | +++ b/src/tests/unit/test_telegraf.py |
348 | @@ -820,20 +820,178 @@ def test_check_prometheus_port_replace_old_port(monkeypatch): |
349 | assert 10042 not in open_ports |
350 | |
351 | |
352 | -def test_get_prometheus_port(monkeypatch, config): |
353 | +def test_generate_prometheus_output_config_network(monkeypatch): |
354 | + output_address = "10.0.0.0/24" |
355 | + output_port = "1234" |
356 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.2") |
357 | + monkeypatch.setattr( |
358 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
359 | + ) |
360 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth0"]) |
361 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
362 | + expected = {"listen": "10.0.0.2:1234", "ip_range": [], "extra_options": []} |
363 | + assert result == expected |
364 | + |
365 | + |
366 | +def test_generate_prometheus_output_config_ip(monkeypatch): |
367 | + output_address = "10.0.0.10" |
368 | + output_port = "1234" |
369 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.10") |
370 | + monkeypatch.setattr( |
371 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
372 | + ) |
373 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
374 | + expected = {"listen": "10.0.0.10:1234", "ip_range": [], "extra_options": []} |
375 | + assert result == expected |
376 | + |
377 | + |
378 | +def test_generate_prometheus_output_config_interface(monkeypatch): |
379 | + output_address = "eth5" |
380 | + output_port = "1234" |
381 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.20") |
382 | + monkeypatch.setattr( |
383 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
384 | + ) |
385 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth5"]) |
386 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
387 | + expected = {"listen": "10.0.0.20:1234", "ip_range": [], "extra_options": []} |
388 | + assert result == expected |
389 | + |
390 | + |
391 | +def test_generate_prometheus_output_config_interface_ipv4(monkeypatch): |
392 | + output_address = "ipv4:eth5" |
393 | + output_port = "1234" |
394 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.20") |
395 | + monkeypatch.setattr( |
396 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
397 | + ) |
398 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth5"]) |
399 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
400 | + expected = {"listen": "10.0.0.20:1234", "ip_range": [], "extra_options": []} |
401 | + assert result == expected |
402 | + |
403 | + |
404 | +def test_generate_prometheus_output_config_interface_ipv6(monkeypatch): |
405 | + output_address = "ipv6:eth5" |
406 | + output_port = "1234" |
407 | + monkeypatch.setattr( |
408 | + telegraf, |
409 | + "get_interface_ip", |
410 | + lambda A, B: "2001:0db8:85a3:0000:0000:8a2e:0370:7334", |
411 | + ) |
412 | + monkeypatch.setattr( |
413 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
414 | + ) |
415 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth5"]) |
416 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
417 | + expected = { |
418 | + "listen": "2001:0db8:85a3:0000:0000:8a2e:0370:7334:1234", |
419 | + "ip_range": [], |
420 | + "extra_options": [], |
421 | + } |
422 | + assert result == expected |
423 | + |
424 | + |
425 | +def test_generate_prometheus_output_config_network_quantity_fail(monkeypatch): |
426 | + output_address = "10.0.0.0/24" |
427 | + output_port = "1234" |
428 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.2") |
429 | + monkeypatch.setattr( |
430 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
431 | + ) |
432 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth0", "eth1"]) |
433 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
434 | + assert result is None |
435 | + |
436 | + |
437 | +def test_generate_prometheus_output_config_network_ip_fail(monkeypatch): |
438 | + output_address = "11.0.0.0/24" |
439 | + output_port = "1234" |
440 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.2") |
441 | + monkeypatch.setattr( |
442 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
443 | + ) |
444 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth0"]) |
445 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
446 | + assert result is None |
447 | + |
448 | + |
449 | +def test_generate_prometheus_output_config_ip_fail(monkeypatch): |
450 | + output_address = "10.0.0.10" |
451 | + output_port = "1234" |
452 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.50") |
453 | + monkeypatch.setattr( |
454 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
455 | + ) |
456 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
457 | + assert result is None |
458 | + |
459 | + |
460 | +def test_generate_prometheus_output_config_interface_fail(monkeypatch): |
461 | + output_address = "eth5" |
462 | + output_port = "1234" |
463 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.20") |
464 | + monkeypatch.setattr( |
465 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
466 | + ) |
467 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth4"]) |
468 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
469 | + assert result is None |
470 | + |
471 | + |
472 | +def test_generate_prometheus_output_config_interface_ipv4_fail(monkeypatch): |
473 | + output_address = "ipv4:eth5" |
474 | + output_port = "1234" |
475 | + monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.20") |
476 | + monkeypatch.setattr( |
477 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
478 | + ) |
479 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth1"]) |
480 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
481 | + assert result is None |
482 | + |
483 | + |
484 | +def test_generate_prometheus_output_config_interface_ipv6_fail(monkeypatch): |
485 | + output_address = "ipv6:eth5" |
486 | + output_port = "1234" |
487 | + monkeypatch.setattr( |
488 | + telegraf, |
489 | + "get_interface_ip", |
490 | + lambda A, B: "2001:0db8:85a3:0000:0000:8a2e:0370:7334", |
491 | + ) |
492 | + monkeypatch.setattr( |
493 | + telegraf, "render_extra_options", lambda A, B, extra_options: [] |
494 | + ) |
495 | + monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth2"]) |
496 | + result = telegraf.generate_prometheus_output_config(output_port, output_address, []) |
497 | + assert result is None |
498 | + |
499 | + |
500 | +def test_get_ip_type(monkeypatch): |
501 | + result = telegraf.get_ip_type("10.1.2.3") |
502 | + assert result == "IPV4" |
503 | + result = telegraf.get_ip_type("10.1.2.366") |
504 | + assert result == "INVALID" |
505 | + result = telegraf.get_ip_type("2001:0db8:85a3:0000:0000:8a2e:0370:7334") |
506 | + assert result == "IPV6" |
507 | + |
508 | + |
509 | +def test_get_prometheus_bindaddress(monkeypatch, config): |
510 | + config["prometheus_output_bindaddress"] = "" |
511 | config["prometheus_output_port"] = "" |
512 | - assert telegraf.get_prometheus_port() is False |
513 | + assert telegraf.get_prometheus_bindaddress()[0] is False |
514 | config["prometheus_output_port"] = "default" |
515 | - assert telegraf.get_prometheus_port() == 9103 |
516 | - config["prometheus_output_port"] = "9126" |
517 | - assert telegraf.get_prometheus_port() == 9126 |
518 | + config["prometheus_output_bindaddress"] = "" |
519 | + assert telegraf.get_prometheus_bindaddress() == (9103, "") |
520 | + config["prometheus_output_bindaddress"] = "9126" |
521 | + assert telegraf.get_prometheus_bindaddress() == (9126, "") |
522 | |
523 | |
524 | def test_prometheus_global(monkeypatch, config): |
525 | open_ports = set() |
526 | monkeypatch.setattr(telegraf.hookenv, "open_port", lambda p: open_ports.add(p)) |
527 | monkeypatch.setattr(telegraf.hookenv, "close_port", lambda p: open_ports.remove(p)) |
528 | - config["prometheus_output_port"] = "default" |
529 | + config["prometheus_output_bindaddress"] = "default" |
530 | telegraf.configure_prometheus_client() |
531 | expected = """[[outputs.prometheus_client]] |
532 | listen = ":9103" |
533 | @@ -846,7 +1004,7 @@ def test_prometheus_global_with_extra_options(monkeypatch, config): |
534 | open_ports = set() |
535 | monkeypatch.setattr(telegraf.hookenv, "open_port", lambda p: open_ports.add(p)) |
536 | monkeypatch.setattr(telegraf.hookenv, "close_port", lambda p: open_ports.remove(p)) |
537 | - config["prometheus_output_port"] = "default" |
538 | + config["prometheus_output_bindaddress"] = "default" |
539 | config[ |
540 | "extra_options" |
541 | ] = """ |
542 | @@ -1205,7 +1363,7 @@ def test_influxdb_api_output(monkeypatch, config): |
543 | |
544 | |
545 | def test_prometheus_client_output(mocker, monkeypatch, config): |
546 | - config["prometheus_output_port"] = "" # Not configured globally |
547 | + config["prometheus_output_bindaddress"] = "" # Not configured globally |
548 | monkeypatch.setattr(telegraf.hookenv, "open_port", lambda p: None) |
549 | |
550 | relation_ids = ["prometheus-client:0"] |
551 | @@ -1239,10 +1397,10 @@ def test_prometheus_client_output(mocker, monkeypatch, config): |
552 | ) |
553 | # prometheus_ip_range is empty, test that all IPs are allowed, not just the related |
554 | # prometheus |
555 | - # prometheus_output_port is also empty, we should get 9103 by default |
556 | + # prometheus_output_bindaddress is also empty, we should get 9103 by default |
557 | network_get_primary_address.assert_called_once_with("prometheus-client") |
558 | interface.configure.assert_called_once_with( |
559 | - "9103", hostname="foo", private_address="foo" |
560 | + 9103, hostname="foo", private_address="foo" |
561 | ) |
562 | config["prometheus_ip_range"] = "" |
563 | telegraf.configure_prometheus_client_with_relation(interface) |
564 | @@ -1306,6 +1464,7 @@ def test_config_no_prometheus(mocker, monkeypatch, config): |
565 | service_restart = mocker.patch("reactive.telegraf.host.service_restart") |
566 | monkeypatch.setattr(telegraf.host, "service_running", lambda n: True) |
567 | check_prometheus_port = mocker.patch("reactive.telegraf.check_prometheus_port") |
568 | + config["prometheus_output_bindaddress"] = "" |
569 | config["prometheus_output_port"] = "" |
570 | set_flag("apt.installed.telegraf") |
571 | assert not base_dir().join("telegraf.conf").exists() |
572 | @@ -1313,7 +1472,7 @@ def test_config_no_prometheus(mocker, monkeypatch, config): |
573 | assert "telegraf.configured" in bus.get_states().keys() |
574 | assert base_dir().join("telegraf.conf").exists() |
575 | service_restart.assert_called_once_with(telegraf.DEB_SERVICE) |
576 | - assert not check_prometheus_port.called |
577 | + check_prometheus_port.assert_called_with("prometheus_output", False) |
578 | |
579 | |
580 | def test_config_changed_apt(mocker, monkeypatch, config): |
581 | @@ -1378,6 +1537,7 @@ def test_restart_on_output_plugin_relation_departed(mocker, monkeypatch, config) |
582 | telegraf.hookenv, "network_get", return_value={"ingress_addresses": ["1.2.3.4"]} |
583 | ) |
584 | config["prometheus_output_port"] = "" |
585 | + config["prometheus_output_bindaddress"] = "" |
586 | bus.discover() |
587 | set_flag("telegraf.installed") |
588 | set_flag("snap.telegraf.installed") |
589 | @@ -1386,9 +1546,7 @@ def test_restart_on_output_plugin_relation_departed(mocker, monkeypatch, config) |
590 | interface.configure = mocker.Mock() |
591 | telegraf.configure_prometheus_client_with_relation(interface) |
592 | assert configs_dir().join("prometheus_client.conf").exists() |
593 | - # dispatch, file should be gone and telegraf restarted. |
594 | bus.dispatch() |
595 | - assert not configs_dir().join("prometheus_client.conf").exists() |
596 | service_restart.assert_called_with(telegraf.DEB_SERVICE) |
597 | |
598 | |
599 | @@ -1446,6 +1604,7 @@ def test_socket_listener_config(monkeypatch, config): |
600 | monkeypatch.setattr(telegraf.hookenv, "open_port", lambda p: open_ports.add(p)) |
601 | monkeypatch.setattr(telegraf.hookenv, "close_port", lambda p: open_ports.remove(p)) |
602 | config["prometheus_output_port"] = "default" |
603 | + config["prometheus_output_bindaddress"] = "" |
604 | telegraf.configure_telegraf() |
605 | expected = 'service_address = "tcp://127.0.0.1:8094"' |
606 | config_file = base_dir().join("telegraf.d", "socket_listener.conf") |
This merge proposal is being monitored by mergebot. Change the status to Approved to merge.