Merge ~gtrkiller/charm-telegraf:master into charm-telegraf:master

Proposed by Franco Luciano Forneron Buschiazzo
Status: Merged
Approved by: Eric Chen
Approved revision: fc1125432c55d0662ba3fcdb4020caa249fc56af
Merged at revision: fcb42f8dcefd7e47f86ad6de21bc99d85868f714
Proposed branch: ~gtrkiller/charm-telegraf:master
Merge into: charm-telegraf:master
Diff against target: 606 lines (+355/-62)
3 files modified
src/config.yaml (+10/-0)
src/reactive/telegraf.py (+173/-49)
src/tests/unit/test_telegraf.py (+172/-13)
Reviewer Review Type Date Requested Status
Eric Chen Approve
šŸ¤– prod-jenkaas-bootstack (community) continuous-integration Approve
Tom Haddon Approve
Arturo Enrique Seijas FernƔndez (community) Approve
Robert Gildein Approve
Johann David Krister Andersson (community) Approve
Tianqi Xiao Pending
BootStack Reviewers Pending
Review via email: mp+428104@code.launchpad.net

Description of the change

The whole address binding has been reworked. Now we can configure prometheus to bind to a network (and check if the unit's IP matches it), bind on a single IP or interface name.

To post a comment you must log in.
Revision history for this message
šŸ¤– Canonical IS Merge Bot (canonical-is-mergebot) wrote :

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
Tom Haddon (mthaddon) wrote :

I don't think this models the behaviour we outlined in the spec, and is certainly missing a number of tests for that behaviour. The details of that behaviour are:

* Create a new configuration option called prometheus_output_bindaddresses.
* This new option would be a string variable that would accept:
** An IP and port (in the simple case where the operator knows thereā€™s only ever going to be one instance) e.g. ā€˜10.1.5.23:1111ā€™ or IPv6 equivalent.
** An IP range and port (in which case each unit would configure itself to listen on the IP that falls within that range. In case of more than one interface matching, or no interface matching, the charm would go into ā€œblockedā€ status) e.g. ā€˜10.1.5.0/24:1111ā€™ or IPv6 equivalent.
** An interface and port (in this case each unit would configure itself to listen on that interface only. If any unit didnā€™t have an interface of that name, the charm would go into ā€œblockedā€ status) e.g. ā€˜eth1:1111ā€™ (defaults to IPv4 if available, falls back to IPv6) or ā€˜ipv4:eth1:1111ā€™ or ā€˜ipv6:eth1:1111ā€™.
** A port on its own to specify listening on that port on all IPs - e.g. `:1111`.
* The charm would only accept one of the above types of strings as an option. Trying to set more than one (via comma-separation, for instance) would cause the charm to go into ā€œblockedā€ status.
Assuming the configuration option matches one of the above options, and all instances of the application have a matching IP/interface the charm would configure each unit to bind only on that IP/interface and port for prometheus output.
* Any port defined here would override the port specified in the prometheus_output_port configuration option. We would update the description of that configuration option to mention that this option is deprecated, and we should log a warning in juju if anyone has this option set to anything other than ā€œdefaultā€.
* At some point in the future weā€™d then update the charm to set it to blocked status if someone has this set to something other than ā€œdefaultā€ and then finally weā€™d remove this option altogether. These wonā€™t be tackled as part of this charm change, however.

Can you take another look at this and make sure we're testing these different cases?

review: Needs Fixing
Revision history for this message
Tom Haddon (mthaddon) wrote :

Some comments inline

Revision history for this message
Eric Chen (eric-chen) wrote :

Mark WIP first until 2nd patch provided.

Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) :
Revision history for this message
Robert Gildein (rgildein) :
review: Needs Fixing
Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote :

Marking as needs review.

Revision history for this message
Tom Haddon (mthaddon) wrote :

Some comments inline.

Also, I feel like we could do with some more unit tests here. In each case we're just testing the `generate_prometheus_output_config` method with one interface that matches each use case. Can we have some tests where things don't match, or there are multiple interfaces so it's clear what happens in those cases? I think it's also worth testing some of the other methods you've added here, particularly `get_prometheus_bindaddress`.

Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote :

Sure! I'll add some more tests. Just ftr, get_prometheus_bindaddress already has a test for it :)

Revision history for this message
Eric Chen (eric-chen) wrote :

Mark this as WIP before the test are finished.

Revision history for this message
Arturo Enrique Seijas FernƔndez (arturo-seijas) :
review: Needs Fixing
Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) :
Revision history for this message
Johann David Krister Andersson (jdkandersson) wrote :

Added a few comments in line

review: Needs Fixing
Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) :
Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) :
Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) :
Revision history for this message
Johann David Krister Andersson (jdkandersson) :
Revision history for this message
Robert Gildein (rgildein) wrote :

Overall LGTM.

Can you fix lint tests [1] and rebase this on new master branch?

---
[1]: https://pastebin.ubuntu.com/

Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote :

> Overall LGTM.
>
> Can you fix lint tests [1] and rebase this on new master branch?
>
> ---
> [1]: https://pastebin.ubuntu.com/
The two lint linebreak errors left have a conflict and cannot be fixed for now. If I format the code with black, flake8 throws an error. If I fix flake8's error, then black wants to reformat the file.

Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote :

I'll add a comment to fix the conflict

Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote :

The master branch has been rebased and the comments for linting been made. Now we have 5 failing tests that have nothing to do with my changes, should I fix them?

Revision history for this message
Johann David Krister Andersson (jdkandersson) :
review: Approve
Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) :
Revision history for this message
Robert Gildein (rgildein) wrote :

LGTM

> I'll add a comment to fix the conflict

Using `# noqa: W503` is the best approach here.

> The master branch has been rebased and the comments for linting been made. Now we have 5 failing tests that have nothing to do with my changes, should I fix them?

We know about it and there is MP with fix for it, so you can ignore it.

review: Approve
Revision history for this message
Eric Chen (eric-chen) wrote :

Citeria to merge the PR

- All lint/unit/func test are pass (5 failing tests should have fixed by James Lin, if you still encounter it, please let me know)
- At least 2 approval. (it's okay now)
- All "Need Fixing" is gone, so, please Tom Haddon and Arturo Enrique Seijas FernƔndez review it again.

Thanks!

Revision history for this message
Arturo Enrique Seijas FernƔndez (arturo-seijas) wrote :

LGTM

review: Approve
Revision history for this message
Tom Haddon (mthaddon) wrote :

LGTM, thx. One minor comment about asking for the accepted values to be documented in the config description, but otherwise fine to go ahead from my perspective.

review: Approve
Revision history for this message
Eric Chen (eric-chen) wrote :

Franco, please provide the log of lint/unit/func test, then we can merge it.

review: Needs Information
Revision history for this message
Eric Chen (eric-chen) wrote :

I found there are inline comment from

Arturo Enrique Seijas FernƔndez
Johann David Krister Andersson

Please fix it too.

review: Needs Fixing
Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote :

> I found there are inline comment from
>
> Arturo Enrique Seijas FernƔndez
> Johann David Krister Andersson
>
> Please fix it too.

Their comments (and Tom's) have been addressed now

Revision history for this message
Eric Chen (eric-chen) wrote :

Great! The final thing:
Sorry that we don't integrate the Jenkins with charm-telegraf now, we will do it soon. Please provide the log of lint/unit/func test, then we can merge it. It's the last mile!

Revision history for this message
šŸ¤– prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Robert Gildein (rgildein) wrote :

The Jenkins is configured for charm-telegraf, but this MP was created
in times when Jenkins was out of duty. So I triggered the tests manually.

Revision history for this message
Franco Luciano Forneron Buschiazzo (gtrkiller) wrote :

> Great! The final thing:
> Sorry that we don't integrate the Jenkins with charm-telegraf now, we will do
> it soon. Please provide the log of lint/unit/func test, then we can merge it.
> It's the last mile!

The linting is OK now, the unit tests have the same 5 errors (ignored) that I had when I tested the original master branch and then rebased. The func tests cannot be triggered with tox -e func apparently, so we may want to update that (or if it should be triggered on a certain way idk of, please tell me)

https://pastebin.canonical.com/p/685jHyPyvD/

Revision history for this message
Eric Chen (eric-chen) wrote :

Hi Franco,

The lint/unit/func are all pass in
https://code.launchpad.net/~rgildein/charm-telegraf/+git/charm-telegraf/+merge/429255

So I think you can pass all of them too. If you need help, you can contact Robert. You can also retrigger the build again too.

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-telegraf-ci/147//rebuild

review: Needs Fixing
Revision history for this message
šŸ¤– prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Tom Haddon (mthaddon) wrote :

> Hi Franco,
>
> The lint/unit/func are all pass in
> https://code.launchpad.net/~rgildein/charm-telegraf/+git/charm-
> telegraf/+merge/429255
>
> So I think you can pass all of them too. If you need help, you can contact
> Robert. You can also retrigger the build again too.
>
> Click here to trigger a rebuild:
> https://jenkins.canonical.com/bootstack/job/lp-charm-telegraf-ci/147//rebuild

I triggered a rebuild here https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/881/console but that now seems to have pulled in pyproject-flake8==5.0.4 rather than pyproject-flake8==0.0.1a5 which was used for previous runs, and the test output is now reporting hundreds of docstring and line too long errors. Can you advise what approach you'd like Franco to take here? Should he pin the version of pyproject-flake8 or take some other approach?

Revision history for this message
šŸ¤– prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Robert Gildein (rgildein) wrote :

Hi Tom, you started the rebuild, but on the old commit. I tried to
rebuilt it again on the last commit here but failed with same error.
I proposed new MP [1], which fixes this [2].

---
[1]: https://code.launchpad.net/~rgildein/charm-telegraf/+git/charm-telegraf/+merge/429753
[2]: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/884/console

Revision history for this message
Eric Chen (eric-chen) wrote :

1. There is merge conflict, please check my inline comment
2. MP[1] is merged. Please rebase to the lastest and trigger Jenkins CI .
I hope we can merge this time.
thanks!

[1]: https://code.launchpad.net/~rgildein/charm-telegraf/+git/charm-telegraf/+merge/429753

review: Needs Fixing
Revision history for this message
šŸ¤– prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
šŸ¤– prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
šŸ¤– prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Approve (continuous-integration)
Revision history for this message
Eric Chen (eric-chen) :
review: Approve
Revision history for this message
šŸ¤– Canonical IS Merge Bot (canonical-is-mergebot) wrote :

Change successfully merged at revision fcb42f8dcefd7e47f86ad6de21bc99d85868f714

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/src/config.yaml b/src/config.yaml
2index bc072ad..282a163 100644
3--- a/src/config.yaml
4+++ b/src/config.yaml
5@@ -75,10 +75,20 @@ options:
6 (eg. service_name-0). {uuid} is replaced by the model UUID, for
7 sites without unique model names. {host} is replaced by the
8 machine hostname
9+ prometheus_output_bindaddress:
10+ type: string
11+ default: ""
12+ description: >
13+ If set prometheus output plugin will be configured to listen on the provided address.
14+ If set to string "default" the charm will use the default port (9103).
15+ Accepted values are IPs, network ranges and interface names,
16+ followed by a port, e.g. "10.0.0.1:9000", "10.0.0.0/24:9101",
17+ "2001:0db8:85a3:0000:0000:8a2e:0370:7334:9101" or "eth0:9101"
18 prometheus_output_port:
19 type: string
20 default: "9103"
21 description: >
22+ WARNING: This config option is deprecated in favor of prometheus_output_bindaddress. Please use that option instead.
23 If set prometheus output plugin will be configured to listen on the provided port.
24 If set to string "default" the charm will use default port (9103)
25 inputs_config:
26diff --git a/src/reactive/telegraf.py b/src/reactive/telegraf.py
27index abbb728..7e10d77 100644
28--- a/src/reactive/telegraf.py
29+++ b/src/reactive/telegraf.py
30@@ -29,6 +29,7 @@ import subprocess
31 import sys
32 import time
33 from distutils.version import LooseVersion
34+from enum import Enum
35
36 import charms.promreg
37 import netifaces
38@@ -123,6 +124,12 @@ class InvalidIntelRDTConfigurationError(Exception):
39 pass
40
41
42+class Address(Enum):
43+ IPV4 = 1
44+ IPV6 = 2
45+ INVALID = 3
46+
47+
48 def is_intel_rdt_enabled():
49 config = hookenv.config()
50 return config.get("collect_intel_rdt_metrics", False) and is_metal()
51@@ -562,13 +569,34 @@ def check_prometheus_port(key, new_port):
52 charms.promreg.register(None, new_port)
53
54
55-def get_prometheus_port():
56+def get_prometheus_bindaddress():
57+ Bindaddress = collections.namedtuple("Bindaddress", ["port", "address"])
58+ """Get prometheus output IP, port and/or interface from the config."""
59 config = hookenv.config()
60- if not config.get("prometheus_output_port", False):
61- return False
62- if config.get("prometheus_output_port") == "default":
63- return 9103
64- return int(config.get("prometheus_output_port"))
65+ output_bindaddress = config.get("prometheus_output_bindaddress")
66+ output_port = config.get("prometheus_output_port")
67+
68+ if not output_bindaddress and not output_port:
69+ return Bindaddress(False, "")
70+ elif not output_bindaddress and output_port != "default":
71+ hookenv.log(
72+ """WARNING: prometheus_output_port is deprecated in favor of
73+ prometheus_output_bindaddress. Please use that option instead."""
74+ )
75+ return Bindaddress(int(output_port), "")
76+ elif (
77+ not output_bindaddress and output_port == "default"
78+ ) or output_bindaddress == "default":
79+ hookenv.log(
80+ """WARNING: prometheus_output_port is deprecated in favor of
81+ prometheus_output_bindaddress. Please use that option instead."""
82+ )
83+ return Bindaddress(9103, "")
84+ elif len(output_bindaddress.rsplit(":", 1)) == 2:
85+ address, port = output_bindaddress.rsplit(":", 1)
86+ return Bindaddress(int(port), address)
87+ else:
88+ return Bindaddress(int(output_bindaddress), "")
89
90
91 def get_prometheus_ip_range():
92@@ -1005,16 +1033,21 @@ def handle_config_changes():
93
94 if (
95 config.changed("install_method")
96- or config.changed("snap_channel") # noqa W503
97- or config.changed("install_sources") # noqa W503
98+ # We have to utilize noqa 503 on 4 lines of code because
99+ # black formatting will conflict with flake8 on this 4 lines of code.
100+ # If we do not skip the error, one of those will inevitably fail.
101+ or config.changed("snap_channel") # noqa: W503
102+ or config.changed("install_sources") # noqa: W503
103 ):
104 clear_flag("telegraf.installed")
105 clear_flag("extra_plugins.configured")
106 clear_flag("plugins.prometheus-client.configured")
107 clear_flag("prometheus-client.relation.configured")
108
109- if config.changed("prometheus_output_port") or config.changed(
110- "prometheus_ip_range"
111+ if (
112+ config.changed("prometheus_output_port")
113+ or config.changed("prometheus_output_bindaddress") # noqa: W503
114+ or config.changed("prometheus_ip_range") # noqa: W503
115 ):
116 clear_flag("plugins.prometheus-client.configured")
117 clear_flag("prometheus-client.relation.configured")
118@@ -1616,52 +1649,140 @@ def influxdb_api_output(influxdb):
119 set_flag("telegraf.needs_reload")
120
121
122-def generate_prometheus_output_config(prometheus_output_port, prometheus_ip_range):
123+def get_ip_type(address):
124+ """Validate a string type: ipv4, ipv6 or invalid as IP address."""
125+ try:
126+ ip = ipaddress.ip_address(address)
127+
128+ if isinstance(ip, ipaddress.IPv4Address):
129+ return Address(1).name
130+ elif isinstance(ip, ipaddress.IPv6Address):
131+ return Address(2).name
132+ except ValueError:
133+ return Address(3).name
134+
135+
136+def generate_prometheus_output_config(
137+ prometheus_output_port, prometheus_output_address, prometheus_ip_range
138+):
139 # If extra_options are set for prometheus_client, let's integrate them
140 extra_options = get_extra_options()
141 options = extra_options["outputs"].get("prometheus_client", {})
142 listen = options.pop("listen", None)
143 if not listen:
144- listen = ":{}".format(prometheus_output_port)
145- elif int(listen.split(":", 1)[1]) != prometheus_output_port:
146- hookenv.log(
147- """prometheus_output_port is {}, but extra_options would set it
148- to {}. Choosing {} from prometheus_output_port.""".format(
149- prometheus_output_port,
150- int(listen.split(":", 1)[1]),
151- prometheus_output_port,
152- ),
153- level=hookenv.WARNING,
154- )
155- listen = "{}:{}".format(listen.split(":", 1)[0], prometheus_output_port)
156-
157- options_ip_range = options.pop("ip_range", [])
158- ip_range = options_ip_range + prometheus_ip_range
159+ if prometheus_output_address == "":
160+ listen = ":{}".format(prometheus_output_port)
161+ elif "/" in prometheus_output_address:
162+ listen = get_prometheus_config_network_ip(
163+ prometheus_output_address, prometheus_output_port
164+ )
165+ elif get_ip_type(prometheus_output_address) != Address(3).name:
166+ listen = get_prometheus_config_ip(
167+ prometheus_output_address, prometheus_output_port
168+ )
169+ else:
170+ listen = get_prometheus_config_interface_ip(
171+ prometheus_output_address, prometheus_output_port
172+ )
173
174- return {
175- "listen": listen,
176- "ip_range": ip_range,
177- "extra_options": render_extra_options(
178- "outputs", "prometheus_client", extra_options=extra_options
179- ),
180- }
181+ options_ip_range = options.pop("ip_range", [])
182+ ip_range = options_ip_range + prometheus_ip_range
183+ if listen is not None:
184+ return {
185+ "listen": listen,
186+ "ip_range": ip_range,
187+ "extra_options": render_extra_options(
188+ "outputs", "prometheus_client", extra_options=extra_options
189+ ),
190+ }
191+
192+
193+def get_prometheus_config_network_ip(prometheus_output_address, prometheus_output_port):
194+ """Validate the provided IP network in the config matches the IP of an interface."""
195+ ip_type = get_ip_type(prometheus_output_address.split("/")[0])
196+ interface_list = host.list_nics()
197+ count = 0
198+ ip_network = ipaddress.ip_network(prometheus_output_address)
199+ for interface_name in interface_list:
200+ interface_ip = get_interface_ip(interface_name, ip_type)
201+ if ipaddress.ip_address(interface_ip) in ip_network:
202+ count += 1
203+ if count == 1:
204+ listen = "{}:{}".format(interface_ip, prometheus_output_port)
205+ return listen
206+
207+
208+def get_prometheus_config_ip(prometheus_output_address, prometheus_output_port):
209+ """Validate the provided IP in the config matches the IP of an interface."""
210+ ip_type = get_ip_type(prometheus_output_address)
211+ ip_address = ipaddress.ip_address(prometheus_output_address)
212+ interface_list = host.list_nics()
213+ for interface_name in interface_list:
214+ interface_ip = get_interface_ip(interface_name, ip_type)
215+ if ipaddress.ip_address(interface_ip) == ip_address:
216+ listen = "{}:{}".format(interface_ip, prometheus_output_port)
217+ return listen
218+
219+
220+def get_prometheus_config_interface_ip(
221+ prometheus_output_address, prometheus_output_port
222+):
223+ """Validate interface with the same name as specified in the config exists."""
224+ interface_list = host.list_nics()
225+ if prometheus_output_address.startswith(
226+ "ipv4:"
227+ ) or prometheus_output_address.startswith("ipv6:"):
228+ ip_type = prometheus_output_address[:4]
229+ interface_name = prometheus_output_address[5:]
230+ for interface in interface_list:
231+ if interface == interface_name:
232+ interface_ip = get_interface_ip(interface_name, ip_type)
233+ listen = "{}:{}".format(interface_ip, prometheus_output_port)
234+ return listen
235+ else:
236+ for interface in interface_list:
237+ if interface == prometheus_output_address:
238+ interface_ip = get_interface_ip(prometheus_output_address, "ipv4")
239+ if not ipaddress.ip_address(interface_ip):
240+ interface_ip = get_interface_ip(prometheus_output_address, "ipv6")
241+ listen = "{}:{}".format(interface_ip, prometheus_output_port)
242+ return listen
243+
244+
245+def get_interface_ip(interface_name, type):
246+ """Get the IP from one of the unit's interfaces."""
247+ type_dict = {Address(1).name: "/inet", Address(2).name: "/inet6"}
248+ cm = "ip -f inet addr show %s | awk '%s / {print $2}'" % (
249+ interface_name,
250+ type_dict[type],
251+ )
252+ ps = subprocess.Popen(
253+ cm, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT
254+ )
255+ interface_ip = ps.communicate()[0].decode(sys.stdout.encoding).split("/")[0]
256+ return interface_ip
257
258
259-def render_prometheus_client_config(port, ip_range):
260+def render_prometheus_client_config(port, address, ip_range):
261 config_path = "{}/{}.conf".format(get_configs_dir(), "prometheus_client")
262 hookenv.log(
263- "Updating {} plugin config file. Port is {} and ip_range is {}".format(
264- "prometheus_client", port, ip_range
265+ "Updating {}:{} plugin config file. Port is {} and ip_range is {}".format(
266+ "prometheus_client", address, port, ip_range
267 ),
268 level=hookenv.INFO,
269 )
270- context = generate_prometheus_output_config(port, ip_range)
271- render(
272- source="prometheus_client.tmpl",
273- templates_dir=get_templates_dir(),
274- target=config_path,
275- context=context,
276- )
277+ context = generate_prometheus_output_config(port, address, ip_range)
278+ try:
279+ render(
280+ source="prometheus_client.tmpl",
281+ templates_dir=get_templates_dir(),
282+ target=config_path,
283+ context=context,
284+ )
285+ except Exception as e:
286+ hookenv.log(e)
287+ hookenv.status_set("blocked", "Telegraf failed to start. Check charm's config.")
288+ return
289
290
291 @when("endpoint.prometheus-client.changed")
292@@ -1670,7 +1791,8 @@ def configure_prometheus_client_with_relation(prometheus):
293 "Configuring prometheus_client output plugin, with prometheus-client relation",
294 level=hookenv.DEBUG,
295 )
296- port = get_prometheus_port() or "9103"
297+ port, address = get_prometheus_bindaddress()
298+ hookenv.log("get_prometheus_bindaddress: {}:{}".format(address, str(port)))
299 # We'll iterate through the prometheus-client relation counterparts,
300 # inform them of our address so that they scrape it, and get their egress subnets
301 # so that we can allow them
302@@ -1706,7 +1828,7 @@ def configure_prometheus_client_with_relation(prometheus):
303 ip_range = get_prometheus_ip_range()
304 if ip_range != []:
305 ip_range = ip_range + remote_egress_subnets
306- render_prometheus_client_config(port, ip_range)
307+ render_prometheus_client_config(port, address, ip_range)
308 set_flag("plugins.prometheus-client.configured")
309 set_flag("prometheus-client.relation.configured")
310 set_flag("telegraf.needs_reload")
311@@ -1717,15 +1839,16 @@ def configure_prometheus_client_with_relation(prometheus):
312 @when_not("plugins.prometheus-client.configured")
313 def configure_prometheus_client():
314 hookenv.log("Configuring prometheus_client output plugin", level=hookenv.DEBUG)
315- if get_prometheus_port():
316- port = get_prometheus_port()
317+ if get_prometheus_bindaddress():
318+ port, address = get_prometheus_bindaddress()
319+ hookenv.log("get_prometheus_bindaddress: {}:{}".format(address, str(port)))
320 else:
321 # No relation to prometheus, no port configured: do not configure the plugin
322 set_flag("plugins.prometheus-client.configured")
323 return
324 check_prometheus_port("prometheus_output", port)
325 ip_range = get_prometheus_ip_range()
326- render_prometheus_client_config(port, ip_range)
327+ render_prometheus_client_config(port, address, ip_range)
328 set_flag("plugins.prometheus-client.configured")
329 set_flag("telegraf.needs_reload")
330 clear_flag("prometheus-client.relation.configured")
331@@ -1998,10 +2121,11 @@ def configure_nagios(nagios):
332 nrpe_setup = nrpe.NRPE(hostname=hostname, primary=False)
333
334 # use charmhelpers to create a process check
335+ port, _ = get_prometheus_bindaddress()
336 nrpe_setup.add_check(
337 "telegraf_http",
338 "Telegraf HTTP check",
339- "check_http -I 127.0.0.1 -p {} -u /metrics".format(get_prometheus_port()),
340+ "check_http -I 127.0.0.1 -p {} -u /metrics".format(port),
341 )
342 nrpe_setup.write()
343 set_flag("telegraf.nagios-setup.complete")
344diff --git a/src/tests/unit/test_telegraf.py b/src/tests/unit/test_telegraf.py
345index 7f409e4..6b453af 100644
346--- a/src/tests/unit/test_telegraf.py
347+++ b/src/tests/unit/test_telegraf.py
348@@ -820,20 +820,178 @@ def test_check_prometheus_port_replace_old_port(monkeypatch):
349 assert 10042 not in open_ports
350
351
352-def test_get_prometheus_port(monkeypatch, config):
353+def test_generate_prometheus_output_config_network(monkeypatch):
354+ output_address = "10.0.0.0/24"
355+ output_port = "1234"
356+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.2")
357+ monkeypatch.setattr(
358+ telegraf, "render_extra_options", lambda A, B, extra_options: []
359+ )
360+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth0"])
361+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
362+ expected = {"listen": "10.0.0.2:1234", "ip_range": [], "extra_options": []}
363+ assert result == expected
364+
365+
366+def test_generate_prometheus_output_config_ip(monkeypatch):
367+ output_address = "10.0.0.10"
368+ output_port = "1234"
369+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.10")
370+ monkeypatch.setattr(
371+ telegraf, "render_extra_options", lambda A, B, extra_options: []
372+ )
373+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
374+ expected = {"listen": "10.0.0.10:1234", "ip_range": [], "extra_options": []}
375+ assert result == expected
376+
377+
378+def test_generate_prometheus_output_config_interface(monkeypatch):
379+ output_address = "eth5"
380+ output_port = "1234"
381+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.20")
382+ monkeypatch.setattr(
383+ telegraf, "render_extra_options", lambda A, B, extra_options: []
384+ )
385+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth5"])
386+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
387+ expected = {"listen": "10.0.0.20:1234", "ip_range": [], "extra_options": []}
388+ assert result == expected
389+
390+
391+def test_generate_prometheus_output_config_interface_ipv4(monkeypatch):
392+ output_address = "ipv4:eth5"
393+ output_port = "1234"
394+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.20")
395+ monkeypatch.setattr(
396+ telegraf, "render_extra_options", lambda A, B, extra_options: []
397+ )
398+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth5"])
399+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
400+ expected = {"listen": "10.0.0.20:1234", "ip_range": [], "extra_options": []}
401+ assert result == expected
402+
403+
404+def test_generate_prometheus_output_config_interface_ipv6(monkeypatch):
405+ output_address = "ipv6:eth5"
406+ output_port = "1234"
407+ monkeypatch.setattr(
408+ telegraf,
409+ "get_interface_ip",
410+ lambda A, B: "2001:0db8:85a3:0000:0000:8a2e:0370:7334",
411+ )
412+ monkeypatch.setattr(
413+ telegraf, "render_extra_options", lambda A, B, extra_options: []
414+ )
415+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth5"])
416+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
417+ expected = {
418+ "listen": "2001:0db8:85a3:0000:0000:8a2e:0370:7334:1234",
419+ "ip_range": [],
420+ "extra_options": [],
421+ }
422+ assert result == expected
423+
424+
425+def test_generate_prometheus_output_config_network_quantity_fail(monkeypatch):
426+ output_address = "10.0.0.0/24"
427+ output_port = "1234"
428+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.2")
429+ monkeypatch.setattr(
430+ telegraf, "render_extra_options", lambda A, B, extra_options: []
431+ )
432+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth0", "eth1"])
433+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
434+ assert result is None
435+
436+
437+def test_generate_prometheus_output_config_network_ip_fail(monkeypatch):
438+ output_address = "11.0.0.0/24"
439+ output_port = "1234"
440+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.2")
441+ monkeypatch.setattr(
442+ telegraf, "render_extra_options", lambda A, B, extra_options: []
443+ )
444+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth0"])
445+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
446+ assert result is None
447+
448+
449+def test_generate_prometheus_output_config_ip_fail(monkeypatch):
450+ output_address = "10.0.0.10"
451+ output_port = "1234"
452+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.50")
453+ monkeypatch.setattr(
454+ telegraf, "render_extra_options", lambda A, B, extra_options: []
455+ )
456+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
457+ assert result is None
458+
459+
460+def test_generate_prometheus_output_config_interface_fail(monkeypatch):
461+ output_address = "eth5"
462+ output_port = "1234"
463+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.20")
464+ monkeypatch.setattr(
465+ telegraf, "render_extra_options", lambda A, B, extra_options: []
466+ )
467+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth4"])
468+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
469+ assert result is None
470+
471+
472+def test_generate_prometheus_output_config_interface_ipv4_fail(monkeypatch):
473+ output_address = "ipv4:eth5"
474+ output_port = "1234"
475+ monkeypatch.setattr(telegraf, "get_interface_ip", lambda A, B: "10.0.0.20")
476+ monkeypatch.setattr(
477+ telegraf, "render_extra_options", lambda A, B, extra_options: []
478+ )
479+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth1"])
480+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
481+ assert result is None
482+
483+
484+def test_generate_prometheus_output_config_interface_ipv6_fail(monkeypatch):
485+ output_address = "ipv6:eth5"
486+ output_port = "1234"
487+ monkeypatch.setattr(
488+ telegraf,
489+ "get_interface_ip",
490+ lambda A, B: "2001:0db8:85a3:0000:0000:8a2e:0370:7334",
491+ )
492+ monkeypatch.setattr(
493+ telegraf, "render_extra_options", lambda A, B, extra_options: []
494+ )
495+ monkeypatch.setattr(telegraf.host, "list_nics", lambda: ["eth2"])
496+ result = telegraf.generate_prometheus_output_config(output_port, output_address, [])
497+ assert result is None
498+
499+
500+def test_get_ip_type(monkeypatch):
501+ result = telegraf.get_ip_type("10.1.2.3")
502+ assert result == "IPV4"
503+ result = telegraf.get_ip_type("10.1.2.366")
504+ assert result == "INVALID"
505+ result = telegraf.get_ip_type("2001:0db8:85a3:0000:0000:8a2e:0370:7334")
506+ assert result == "IPV6"
507+
508+
509+def test_get_prometheus_bindaddress(monkeypatch, config):
510+ config["prometheus_output_bindaddress"] = ""
511 config["prometheus_output_port"] = ""
512- assert telegraf.get_prometheus_port() is False
513+ assert telegraf.get_prometheus_bindaddress()[0] is False
514 config["prometheus_output_port"] = "default"
515- assert telegraf.get_prometheus_port() == 9103
516- config["prometheus_output_port"] = "9126"
517- assert telegraf.get_prometheus_port() == 9126
518+ config["prometheus_output_bindaddress"] = ""
519+ assert telegraf.get_prometheus_bindaddress() == (9103, "")
520+ config["prometheus_output_bindaddress"] = "9126"
521+ assert telegraf.get_prometheus_bindaddress() == (9126, "")
522
523
524 def test_prometheus_global(monkeypatch, config):
525 open_ports = set()
526 monkeypatch.setattr(telegraf.hookenv, "open_port", lambda p: open_ports.add(p))
527 monkeypatch.setattr(telegraf.hookenv, "close_port", lambda p: open_ports.remove(p))
528- config["prometheus_output_port"] = "default"
529+ config["prometheus_output_bindaddress"] = "default"
530 telegraf.configure_prometheus_client()
531 expected = """[[outputs.prometheus_client]]
532 listen = ":9103"
533@@ -846,7 +1004,7 @@ def test_prometheus_global_with_extra_options(monkeypatch, config):
534 open_ports = set()
535 monkeypatch.setattr(telegraf.hookenv, "open_port", lambda p: open_ports.add(p))
536 monkeypatch.setattr(telegraf.hookenv, "close_port", lambda p: open_ports.remove(p))
537- config["prometheus_output_port"] = "default"
538+ config["prometheus_output_bindaddress"] = "default"
539 config[
540 "extra_options"
541 ] = """
542@@ -1205,7 +1363,7 @@ def test_influxdb_api_output(monkeypatch, config):
543
544
545 def test_prometheus_client_output(mocker, monkeypatch, config):
546- config["prometheus_output_port"] = "" # Not configured globally
547+ config["prometheus_output_bindaddress"] = "" # Not configured globally
548 monkeypatch.setattr(telegraf.hookenv, "open_port", lambda p: None)
549
550 relation_ids = ["prometheus-client:0"]
551@@ -1239,10 +1397,10 @@ def test_prometheus_client_output(mocker, monkeypatch, config):
552 )
553 # prometheus_ip_range is empty, test that all IPs are allowed, not just the related
554 # prometheus
555- # prometheus_output_port is also empty, we should get 9103 by default
556+ # prometheus_output_bindaddress is also empty, we should get 9103 by default
557 network_get_primary_address.assert_called_once_with("prometheus-client")
558 interface.configure.assert_called_once_with(
559- "9103", hostname="foo", private_address="foo"
560+ 9103, hostname="foo", private_address="foo"
561 )
562 config["prometheus_ip_range"] = ""
563 telegraf.configure_prometheus_client_with_relation(interface)
564@@ -1306,6 +1464,7 @@ def test_config_no_prometheus(mocker, monkeypatch, config):
565 service_restart = mocker.patch("reactive.telegraf.host.service_restart")
566 monkeypatch.setattr(telegraf.host, "service_running", lambda n: True)
567 check_prometheus_port = mocker.patch("reactive.telegraf.check_prometheus_port")
568+ config["prometheus_output_bindaddress"] = ""
569 config["prometheus_output_port"] = ""
570 set_flag("apt.installed.telegraf")
571 assert not base_dir().join("telegraf.conf").exists()
572@@ -1313,7 +1472,7 @@ def test_config_no_prometheus(mocker, monkeypatch, config):
573 assert "telegraf.configured" in bus.get_states().keys()
574 assert base_dir().join("telegraf.conf").exists()
575 service_restart.assert_called_once_with(telegraf.DEB_SERVICE)
576- assert not check_prometheus_port.called
577+ check_prometheus_port.assert_called_with("prometheus_output", False)
578
579
580 def test_config_changed_apt(mocker, monkeypatch, config):
581@@ -1378,6 +1537,7 @@ def test_restart_on_output_plugin_relation_departed(mocker, monkeypatch, config)
582 telegraf.hookenv, "network_get", return_value={"ingress_addresses": ["1.2.3.4"]}
583 )
584 config["prometheus_output_port"] = ""
585+ config["prometheus_output_bindaddress"] = ""
586 bus.discover()
587 set_flag("telegraf.installed")
588 set_flag("snap.telegraf.installed")
589@@ -1386,9 +1546,7 @@ def test_restart_on_output_plugin_relation_departed(mocker, monkeypatch, config)
590 interface.configure = mocker.Mock()
591 telegraf.configure_prometheus_client_with_relation(interface)
592 assert configs_dir().join("prometheus_client.conf").exists()
593- # dispatch, file should be gone and telegraf restarted.
594 bus.dispatch()
595- assert not configs_dir().join("prometheus_client.conf").exists()
596 service_restart.assert_called_with(telegraf.DEB_SERVICE)
597
598
599@@ -1446,6 +1604,7 @@ def test_socket_listener_config(monkeypatch, config):
600 monkeypatch.setattr(telegraf.hookenv, "open_port", lambda p: open_ports.add(p))
601 monkeypatch.setattr(telegraf.hookenv, "close_port", lambda p: open_ports.remove(p))
602 config["prometheus_output_port"] = "default"
603+ config["prometheus_output_bindaddress"] = ""
604 telegraf.configure_telegraf()
605 expected = 'service_address = "tcp://127.0.0.1:8094"'
606 config_file = base_dir().join("telegraf.d", "socket_listener.conf")

Subscribers

People subscribed via source and target branches