Merge lp:~tmm1/graphite/udp-receiver into lp:~graphite-dev/graphite/main

Proposed by Aman Gupta on 2011-07-21
Status: Merged
Merge reported by: chrismd
Merged at revision: not available
Proposed branch: lp:~tmm1/graphite/udp-receiver
Merge into: lp:~graphite-dev/graphite/main
Diff against target: 103 lines (+29/-3) 5 files modified
To merge this branch: bzr merge lp:~tmm1/graphite/udp-receiver
Reviewer Review Type Date Requested Status
chrismd 2011-07-21 Needs Fixing on 2011-07-21
Review via email: mp+68622@code.launchpad.net
To post a comment you must log in.
lp:~tmm1/graphite/udp-receiver updated on 2011-07-21
434. By Aman Gupta on 2011-07-21

Only show remote hostname on invalid datagrams

chrismd (chrismd) wrote :

tmm1 this change looks good except I think the datagram receiver should allow for multiple lines. This will make the data format the same as the current tcp port 2003 receiver, and also the udp listener should use udp port 2003 by default as well since it will use the data format.

review: Needs Fixing
Aman Gupta (tmm1) wrote :

Good call on both points.

Is it worth having UDP_RECEIVER_* configuration, or should I re-use LINE_RECEIVER_* for both TCP/UDP. I suppose it could be useful to listen for datagrams on a separate ip/port.

lp:~tmm1/graphite/udp-receiver updated on 2011-07-21
435. By Aman Gupta on 2011-07-21

Listen for datagrams on the same port as the line receiver by default

436. By Aman Gupta on 2011-07-21

UDP Reciever can accept multiple lines per datagram

Preview Diff

1=== modified file 'carbon/conf/carbon.amqp.conf.example'
2--- carbon/conf/carbon.amqp.conf.example 2011-04-01 06:01:35 +0000
3+++ carbon/conf/carbon.amqp.conf.example 2011-07-21 03:08:32 +0000
4@@ -32,6 +32,9 @@
5 LINE_RECEIVER_INTERFACE = 0.0.0.0
6 LINE_RECEIVER_PORT = 2003
7
8+UDP_RECEIVER_INTERFACE = 0.0.0.0
9+UDP_RECEIVER_PORT = 2003
10+
11 PICKLE_RECEIVER_INTERFACE = 0.0.0.0
12 PICKLE_RECEIVER_PORT = 2004
13
14
15=== modified file 'carbon/conf/carbon.conf.example'
16--- carbon/conf/carbon.conf.example 2011-04-05 21:42:51 +0000
17+++ carbon/conf/carbon.conf.example 2011-07-21 03:08:32 +0000
18@@ -30,6 +30,9 @@
19 LINE_RECEIVER_INTERFACE = 0.0.0.0
20 LINE_RECEIVER_PORT = 2003
21
22+UDP_RECEIVER_INTERFACE = 0.0.0.0
23+UDP_RECEIVER_PORT = 2003
24+
25 PICKLE_RECEIVER_INTERFACE = 0.0.0.0
26 PICKLE_RECEIVER_PORT = 2004
27
28
29=== modified file 'carbon/lib/carbon/conf.py'
30--- carbon/lib/carbon/conf.py 2011-07-20 23:15:06 +0000
31+++ carbon/lib/carbon/conf.py 2011-07-21 03:08:32 +0000
32@@ -35,6 +35,8 @@
33 MAX_CREATES_PER_MINUTE='inf',
34 LINE_RECEIVER_INTERFACE='0.0.0.0',
35 LINE_RECEIVER_PORT=2003,
36+ UDP_RECEIVER_INTERFACE='0.0.0.0',
37+ UDP_RECEIVER_PORT=2003,
38 PICKLE_RECEIVER_INTERFACE='0.0.0.0',
39 PICKLE_RECEIVER_PORT=2004,
40 CACHE_QUERY_INTERFACE='0.0.0.0',
41
42=== modified file 'carbon/lib/carbon/listeners.py'
43--- carbon/lib/carbon/listeners.py 2010-06-06 16:24:30 +0000
44+++ carbon/lib/carbon/listeners.py 2011-07-21 03:08:32 +0000
45@@ -1,5 +1,5 @@
46 from twisted.internet import reactor
47-from twisted.internet.protocol import Factory
48+from twisted.internet.protocol import Factory, DatagramProtocol
49 from twisted.internet.error import ConnectionDone
50 from twisted.protocols.basic import LineOnlyReceiver, Int32StringReceiver
51 from carbon.cache import MetricCache
52@@ -42,6 +42,19 @@
53 metricReceived(metric, datapoint)
54
55
56+class MetricDatagramReceiver(LoggingMixin, DatagramProtocol):
57+ def datagramReceived(self, data, (host, port)):
58+ for line in data.splitlines():
59+ try:
60+ metric, value, timestamp = line.strip().split()
61+ datapoint = ( float(timestamp), float(value) )
62+
63+ increment('metricsReceived')
64+ metricReceived(metric, datapoint)
65+ except:
66+ log.listener('invalid line received from client %s, ignoring' % host)
67+
68+
69 class MetricPickleReceiver(LoggingMixin, Int32StringReceiver):
70 MAX_LENGTH = 2 ** 20
71
72
73=== modified file 'carbon/lib/carbon/service.py'
74--- carbon/lib/carbon/service.py 2011-07-08 20:25:00 +0000
75+++ carbon/lib/carbon/service.py 2011-07-21 03:08:32 +0000
76@@ -16,13 +16,13 @@
77 from os.path import exists
78
79 from twisted.application.service import MultiService
80-from twisted.application.internet import TCPServer, TCPClient
81+from twisted.application.internet import TCPServer, TCPClient, UDPServer
82 from twisted.internet.protocol import ServerFactory
83
84
85 def createBaseService(config):
86 from carbon.conf import settings
87- from carbon.listeners import MetricLineReceiver, MetricPickleReceiver
88+ from carbon.listeners import MetricLineReceiver, MetricPickleReceiver, MetricDatagramReceiver
89
90 root_service = MultiService()
91 root_service.setName(settings.program)
92@@ -52,6 +52,11 @@
93 service = TCPServer(int(port), factory, interface=interface)
94 service.setServiceParent(root_service)
95
96+ service = UDPServer(int(settings.UDP_RECEIVER_PORT),
97+ MetricDatagramReceiver(),
98+ interface=settings.UDP_RECEIVER_INTERFACE)
99+ service.setServiceParent(root_service)
100+
101 if use_amqp:
102 factory = amqp_listener.createAMQPListener(
103 amqp_user, amqp_password,

Subscribers

People subscribed via source and target branches