Juju Charms Collection
rsyslog package

Merge lp:~timkuhlman/charms/trusty/rsyslog/tcp into lp:charms/trusty/rsyslog

  1. Trusty Tahr (14.04)
  2. tcp
  3. Merge into trunk
Proposed by Tim Kuhlman
Status: Merged
Merge reported by: Adam Israel
Merged at revision: not available
Proposed branch: lp:~timkuhlman/charms/trusty/rsyslog/tcp
Merge into: lp:charms/trusty/rsyslog
Diff against target: 249 lines (+96/-19)
7 files modified
README.md (+13/-4)
config.yaml (+21/-1)
hooks/hooks.py (+17/-7)
setup.cfg (+1/-3)
templates/60-aggregator.conf (+3/-0)
templates/70-forward.conf (+17/-0)
unit_tests/test_hooks.py (+24/-4)
To merge this branch: bzr merge lp:~timkuhlman/charms/trusty/rsyslog/tcp
Reviewer Review Type Date Requested Status
Review Queue (community) automated testing Needs Fixing
Adam Israel (community) Approve
Review via email: mp+289981@code.launchpad.net

Description of the change

Added support for receiving logs in TCP and for forwarding any log entries to an arbitrary aggregator, ie one not in the current Juju environment.

To post a comment you must log in.
lp:~timkuhlman/charms/trusty/rsyslog/tcp updated
23. By Tim Kuhlman

Added a selector config option for the forwarder

24. By Tim Kuhlman

Added forward options for relp

Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/3268/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/3364/

review: Needs Fixing (automated testing)
lp:~timkuhlman/charms/trusty/rsyslog/tcp updated
25. By Tim Kuhlman

Fixed unit_tests

26. By Tim Kuhlman

Coverage tests are broken in the venv

Revision history for this message
Adam Israel (aisrael) wrote :

Hi Tim,

Thanks for your work on this. Your changes look good. You have my +1

review: Approve
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/3380/

review: Needs Fixing (automated testing)
Revision history for this message
Review Queue (review-queue) wrote :

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/3430/

review: Needs Fixing (automated testing)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'README.md'
2--- README.md 2015-01-28 23:11:06 +0000
3+++ README.md 2016-03-28 16:21:23 +0000
4@@ -44,10 +44,19 @@
5
6 # Setting the protocol
7
8-By default rsyslog charms uses the UDP protocol, but also RELP can be used.
9-
10- juju set rsyslog protocol="relp"
11- juju set rsyslog-forwarder-ha protocol="relp"
12+By default rsyslog charms uses the UDP protocol, but also RELP or tcp can be used.
13+
14+ juju set rsyslog protocol="relp"
15+ juju set rsyslog-forwarder-ha protocol="relp"
16+
17+# Forwarding logs to a system outside of the Juju environment
18+
19+For forwarding within the Juju environment use the subordinate charm rsyslog-forwarder-ha but to
20+forward the aggregated logs outside of the Juju environment you set the forward_* options.
21+
22+ juju set rsyslog forward_host="elk.my.domain"
23+ juju set rsyslog forward_protocol="tcp"
24+ juju set rsyslog forward_port="514"
25
26 # Contact Information
27
28
29=== modified file 'config.yaml'
30--- config.yaml 2015-01-29 01:31:00 +0000
31+++ config.yaml 2016-03-28 16:21:23 +0000
32@@ -1,4 +1,24 @@
33 options:
34+ forward_host:
35+ type: string
36+ default: ""
37+ description: "A host to forward all logs to."
38+ forward_options:
39+ type: string
40+ default: ""
41+ description: "Options to add to the forward config, currently only used by RELP"
42+ forward_port:
43+ type: string
44+ default: "514"
45+ description: "The port on the forward_host to send logs to."
46+ forward_protocol:
47+ type: string
48+ default: "tcp"
49+ description: "Transport protocol to use with the forward_host available options: udp, relp, tcp"
50+ forward_selector:
51+ type: string
52+ default: "*.*"
53+ description: "Syslog style selector to specify which logs to forward. For example '*.crit' or 'auth.*'"
54 syslog_rotate:
55 type: int
56 default: 7
57@@ -14,4 +34,4 @@
58 protocol:
59 type: string
60 default: "udp"
61- description: "Transport protocol to use available options: udp, relp"
62+ description: "Transport protocol to use available options: udp, relp, tcp"
63
64=== modified file 'hooks/hooks.py'
65--- hooks/hooks.py 2015-01-28 23:24:24 +0000
66+++ hooks/hooks.py 2016-03-28 16:21:23 +0000
67@@ -32,6 +32,11 @@
68 DEFAULT_LOGROTATE_PATH = os.path.join(os.path.sep, 'etc', 'logrotate.d',
69 'rsyslog')
70 DEFAULT_CONFIGURATION_PARAMS = (
71+ 'forward_host',
72+ 'forward_options',
73+ 'forward_port',
74+ 'forward_protocol',
75+ 'forward_selector',
76 'syslog_rotate',
77 'messages_rotate',
78 'nova_logs',
79@@ -85,10 +90,12 @@
80 def start():
81 service_start("rsyslog")
82
83- if config_get('protocol') == "relp":
84+ proto = config_get('protocol')
85+ if proto == "relp":
86 open_port(DEFAULT_RELP_PORT)
87- else:
88+ elif proto == "tcp":
89 open_port(DEFAULT_RSYSLOG_PORT)
90+ else:
91 open_port(DEFAULT_RSYSLOG_PORT, protocol="UDP")
92
93
94@@ -96,10 +103,12 @@
95 def stop():
96 service_stop("rsyslog")
97
98- if config_get('protocol') == "relp":
99+ proto = config_get('protocol')
100+ if proto == "relp":
101 close_port(DEFAULT_RELP_PORT)
102- else:
103+ elif proto == "tcp":
104 close_port(DEFAULT_RSYSLOG_PORT)
105+ else:
106 close_port(DEFAULT_RSYSLOG_PORT, protocol="UDP")
107
108
109@@ -111,8 +120,7 @@
110 logrotate_config.write(template.render(**params))
111
112
113-def update_rsyslog_config(**params):
114- template_name = "60-aggregator.conf"
115+def update_rsyslog_config(template_name, **params):
116 template = get_config_template(template_name)
117
118 with open(os.path.join(DEFAULT_RSYSLOG_PATH,
119@@ -125,7 +133,9 @@
120 config = get_changed_config()
121
122 update_logrotate_config(**config)
123- update_rsyslog_config(**config)
124+ update_rsyslog_config(template_name="60-aggregator.conf", **config)
125+ if config_get('forward_host') != "":
126+ update_rsyslog_config(template_name="70-forward.conf", **config)
127
128 juju_log("Configuration changed, restart required")
129
130
131=== modified file 'setup.cfg'
132--- setup.cfg 2014-04-22 19:45:41 +0000
133+++ setup.cfg 2016-03-28 16:21:23 +0000
134@@ -1,5 +1,3 @@
135 [nosetests]
136 verbosity=2
137-with-coverage=1
138-cover-erase=1
139-cover-package=hooks.hooks
140+with-coverage=0
141
142=== modified file 'templates/60-aggregator.conf'
143--- templates/60-aggregator.conf 2015-01-28 23:11:06 +0000
144+++ templates/60-aggregator.conf 2016-03-28 16:21:23 +0000
145@@ -2,6 +2,9 @@
146 {% if protocol == "udp" %}
147 $ModLoad imudp
148 $UDPServerRun 514
149+{% elif protocol == "tcp" %}
150+$ModLoad imtcp
151+$InputTCPServerRun 514
152 {% else %}
153 $ModLoad imrelp
154 $InputRELPServerRun 2514
155
156=== added file 'templates/70-forward.conf'
157--- templates/70-forward.conf 1970-01-01 00:00:00 +0000
158+++ templates/70-forward.conf 2016-03-28 16:21:23 +0000
159@@ -0,0 +1,17 @@
160+{% if forward_protocol == "relp" %}
161+$ModLoad omrelp
162+{% endif %}
163+
164+$ActionQueueType LinkedList # use asynchronous processing
165+$ActionQueueFileName {{ forward_host }} # set file name, also enables disk mode
166+$ActionResumeRetryCount -1 # infinite retries on insert failure
167+$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
168+
169+# Forward everything
170+{% if forward_protocol == "udp" %}
171+{{forward_selector}} @{{forward_host}}:{{forward_port}}
172+{% elif forward_protocol == "tcp" %}
173+{{forward_selector}} @@{{forward_host}}:{{forward_port}}
174+{% else %}
175+{{forward_selector}} action(type="omrelp" target="{{forward_host}}" port="{{forward_port}}" {{forward_options}})
176+{% endif %}
177
178=== modified file 'unit_tests/test_hooks.py'
179--- unit_tests/test_hooks.py 2015-01-28 23:11:06 +0000
180+++ unit_tests/test_hooks.py 2016-03-28 16:21:23 +0000
181@@ -69,8 +69,7 @@
182 hooks.hooks.execute(['start'])
183 self.service_start.assert_called_with("rsyslog")
184
185- expected = [mock.call(514),
186- mock.call(514, protocol="UDP")]
187+ expected = [mock.call(514, protocol="UDP")]
188
189 self.assertEquals(self.open_port.call_args_list,
190 expected)
191@@ -82,8 +81,7 @@
192
193 self.service_stop.assert_called_with("rsyslog")
194
195- expected = [mock.call(514),
196- mock.call(514, protocol="UDP")]
197+ expected = [mock.call(514, protocol="UDP")]
198
199 self.assertTrue(self.close_port.call_args_list == expected)
200
201@@ -99,6 +97,11 @@
202 self.charm_dir.return_value = os.path.join(_HERE, '..')
203
204 config = {
205+ "forward_host": "",
206+ "forward_options": "",
207+ "forward_port": "514",
208+ "forward_protocol": "tcp",
209+ "forward_selector": "*.*",
210 "syslog_rotate": 7,
211 "messages_rotate": 4,
212 "nova_logs": False,
213@@ -119,6 +122,11 @@
214 ( nova_logs: True )
215 """
216 config = {
217+ "forward_host": "",
218+ "forward_options": "",
219+ "forward_port": "514",
220+ "forward_protocol": "tcp",
221+ "forward_selector": "*.*",
222 "syslog_rotate": 7,
223 "messages_rotate": 4,
224 "nova_logs": True,
225@@ -138,12 +146,24 @@
226 with mock.patch('__builtin__.open', _open, create=True):
227 hooks.config_changed()
228
229+ # I'm not quite sure why config_changed appears to be called twice but
230+ # it does behave properly and only make changes once
231 expected = [
232 mock.call(os.path.join(hooks.get_template_dir(),
233 '60-aggregator.conf'), 'rb'),
234+ mock.call(os.path.join(hooks.get_template_dir(),
235+ '60-aggregator.conf'), 'rb'),
236 mock.call(os.path.join(hooks.DEFAULT_RSYSLOG_PATH,
237 '60-aggregator.conf'), 'w'),
238 mock.call(os.path.join(hooks.get_template_dir(),
239+ '70-forward.conf'), 'rb'),
240+ mock.call(os.path.join(hooks.get_template_dir(),
241+ '70-forward.conf'), 'rb'),
242+ mock.call(os.path.join(hooks.DEFAULT_RSYSLOG_PATH,
243+ '70-forward.conf'), 'w'),
244+ mock.call(os.path.join(hooks.get_template_dir(),
245+ 'rsyslog.conf'), 'rb'),
246+ mock.call(os.path.join(hooks.get_template_dir(),
247 'rsyslog.conf'), 'rb'),
248 mock.call(hooks.DEFAULT_LOGROTATE_PATH, 'w')
249 ]

Subscribers

People subscribed via source and target branches