Merge lp:~lifeless/python-oops-amqp/misc into lp:python-oops-amqp

Proposed by Robert Collins on 2012-08-10
Status: Merged
Approved by: William Grant on 2012-08-10
Approved revision: 16
Merged at revision: 18
Proposed branch: lp:~lifeless/python-oops-amqp/misc
Merge into: lp:python-oops-amqp
Diff against target: 367 lines (+132/-28)
10 files modified
Makefile (+13/-0)
NEWS (+12/-0)
README (+6/-6)
oops_amqp/__init__.py (+1/-1)
oops_amqp/publisher.py (+3/-3)
oops_amqp/tests/test_publisher.py (+9/-9)
oops_amqp/tests/test_receiver.py (+6/-6)
oops_amqp/trace.py (+74/-0)
setup.py (+6/-2)
versions.cfg (+2/-1)
To merge this branch: bzr merge lp:~lifeless/python-oops-amqp/misc
Reviewer Review Type Date Requested Status
William Grant code Approve on 2012-08-10
James Westby (community) 2012-08-10 Approve on 2012-08-10
Review via email: mp+119076@code.launchpad.net

Description of the change

This updates the API to match current oops releases, and adds a useful trace helper.

To post a comment you must log in.
lp:~lifeless/python-oops-amqp/misc updated on 2012-08-10
16. By Robert Collins on 2012-08-10

Release 0.0.7.

James Westby (james-w) wrote :

281 + description = "Load OOPS reports into oops-tools from AMQP."

Also, I think setup.py should change the requirement to oops>=0.0.11.

Otherwise I think this is ok.

Thanks,

James

James Westby (james-w) :
review: Approve
lp:~lifeless/python-oops-amqp/misc updated on 2012-08-10
17. By Robert Collins on 2012-08-10

Fix bad trace description.

18. By Robert Collins on 2012-08-10

Encode the version constraint for oops in setup.py.

William Grant (wgrant) :
review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'Makefile'
2--- Makefile 1970-01-01 00:00:00 +0000
3+++ Makefile 2012-08-10 03:02:19 +0000
4@@ -0,0 +1,13 @@
5+all:
6+
7+bin/buildout: buildout.cfg versions.cfg setup.py download-cache eggs
8+ ./bootstrap.py \
9+ --setup-source=download-cache/ez_setup.py \
10+ --download-base=download-cache/dist --eggs=eggs
11+
12+
13+download-cache:
14+ bzr checkout --lightweight lp:lp-source-dependencies download-cache
15+
16+eggs:
17+ mkdir eggs
18
19=== modified file 'NEWS'
20--- NEWS 2012-02-10 14:47:11 +0000
21+++ NEWS 2012-08-10 03:02:19 +0000
22@@ -3,6 +3,18 @@
23
24 Changes and improvements to oops-amqp, grouped by release.
25
26+NEXT
27+----
28+
29+0.0.7
30+-----
31+
32+* New script 'oops-amqp-trace' which will trace oops reports received on an
33+ AMQP exchange. (Robert Collins)
34+
35+* Updated to support the new publisher API in oops 0.0.11 and above. This is
36+ incompatible with older versions of oops. (Robert Collins)
37+
38 0.0.6
39 -----
40
41
42=== modified file 'README'
43--- README 2011-12-08 10:52:36 +0000
44+++ README 2012-08-10 03:02:19 +0000
45@@ -29,7 +29,7 @@
46
47 * bson
48
49-* oops (http://pypi.python.org/pypi/oops)
50+* oops (http://pypi.python.org/pypi/oops) 0.0.11 or newer.
51
52 * amqplib
53
54@@ -63,7 +63,7 @@
55 Provide the publisher to your OOPS config::
56
57 >>> config = oops.Config()
58- >>> config.publishers.append(publisher)
59+ >>> config.publisher = publisher
60
61 Any oops published via that config will now be sent via amqp.
62
63@@ -82,14 +82,14 @@
64 the publication failed. To prevent losing the OOPS its a good idea to have a
65 fallback publisher - either another AMQP publisher (to a different server) or
66 one that spools locally (where you can pick up the OOPSes via rsync or some
67-other mechanism. Using the oops standard helper publish_new_only will let you
68-wrap the fallback publisher so that it only gets invoked if the primary
69+other mechanism. Using the oops standard helper publish_with_fallback will let
70+you wrap the fallback publisher so that it only gets invoked if the primary
71 method failed::
72
73 >>> fallback_factory = partial(amqp.Connection, host="otherserver:5672",
74 ... userid="guest", password="guest", virtual_host="/", insist=False)
75 >>> fallback_publisher = oops_amqp.Publisher(fallback_factory, "oopses", "")
76- >>> config.publishers.append(publish_new_only(fallback_publisher))
77+ >>> config.publisher = publish_with_fallback(publisher, fallback_publisher)
78
79 Receiving from AMQP
80 +++++++++++++++++++
81@@ -106,7 +106,7 @@
82
83 >>> publisher = oops_datedir_repo.DateDirRepo('.', inherit_id=True)
84 >>> config = oops.Config()
85- >>> config.publishers.append(publisher.publish)
86+ >>> config.publisher = publisher.publish
87 >>> receiver = oops_amqp.Receiver(config, factory, "my queue")
88 >>> receiver.run_forever()
89
90
91=== modified file 'oops_amqp/__init__.py'
92--- oops_amqp/__init__.py 2012-02-13 00:03:12 +0000
93+++ oops_amqp/__init__.py 2012-08-10 03:02:19 +0000
94@@ -97,7 +97,7 @@
95 # established at this point, and setup.py will use a version of next-$(revno).
96 # If the releaselevel is 'final', then the tarball will be major.minor.micro.
97 # Otherwise it is major.minor.micro~$(revno).
98-__version__ = (0, 0, 6, 'beta', 0)
99+__version__ = (0, 0, 7, 'final', 0)
100
101 __all__ = [
102 'Publisher',
103
104=== modified file 'oops_amqp/publisher.py'
105--- oops_amqp/publisher.py 2012-02-09 23:13:45 +0000
106+++ oops_amqp/publisher.py 2012-08-10 03:02:19 +0000
107@@ -88,7 +88,7 @@
108 message.properties["delivery_mode"] = 2
109 channel = self.get_channel()
110 if channel is None:
111- return None
112+ return []
113 try:
114 channel.basic_publish(
115 message, self.exchange_name, routing_key=self.routing_key)
116@@ -96,7 +96,7 @@
117 self.channels.channel = None
118 if is_amqplib_connection_error(e):
119 # Could not connect / interrupted connection
120- return None
121+ return []
122 # Unknown error mode : don't hide it.
123 raise
124- return report['id']
125+ return [report['id']]
126
127=== modified file 'oops_amqp/tests/test_publisher.py'
128--- oops_amqp/tests/test_publisher.py 2012-02-09 23:13:45 +0000
129+++ oops_amqp/tests/test_publisher.py 2012-08-10 03:02:19 +0000
130@@ -40,9 +40,9 @@
131 reference_oops = {'id': 'kept', 'akey': 'avalue'}
132 oops = dict(reference_oops)
133 expected_id = 'kept'
134- oops_id = publisher(oops)
135+ oops_ids = publisher(oops)
136 # Publication returns the oops ID allocated.
137- self.assertEqual(expected_id, oops_id)
138+ self.assertEqual([expected_id], oops_ids)
139 # The oops should not be altered by publication.
140 self.assertEqual(reference_oops, oops)
141 # The received OOPS should have the ID embedded and be a bson dict.
142@@ -66,14 +66,14 @@
143 oops = dict(reference_oops)
144 id_bson = md5(bson.dumps(oops)).hexdigest()
145 expected_id = "OOPS-%s" % id_bson
146- oops_id = publisher(oops)
147+ oops_ids = publisher(oops)
148 # Publication returns the oops ID allocated.
149- self.assertEqual(expected_id, oops_id)
150+ self.assertEqual([expected_id], oops_ids)
151 # The oops should not be altered by publication.
152 self.assertEqual(reference_oops, oops)
153 # The received OOPS should have the ID embedded and be a bson dict.
154 expected_oops = dict(reference_oops)
155- expected_oops['id'] = oops_id
156+ expected_oops['id'] = oops_ids[0]
157 def check_oops(msg):
158 self.assertEqual(expected_oops, bson.loads(msg.body))
159 channel.basic_ack(msg.delivery_tag)
160@@ -98,11 +98,11 @@
161 publisher = Publisher(
162 self.connection_factory, queue.exchange_name, "")
163 oops = {'akey': 42}
164- self.assertEqual(None, publisher(oops))
165+ self.assertEqual([], publisher(oops))
166 finally:
167 self.rabbit.runner._start()
168 queue.channel = self.connection_factory().channel()
169- self.assertNotEqual(None, publisher(oops))
170+ self.assertNotEqual([], publisher(oops))
171
172 def test_publish_amqp_down_after_use(self):
173 # If amqp goes down after its been successfully used, None is returned
174@@ -119,9 +119,9 @@
175 # release.
176 self.rabbit.runner._stop()
177 try:
178- self.assertEqual(None, publisher(oops))
179+ self.assertEqual([], publisher(oops))
180 finally:
181 self.rabbit.runner._start()
182 queue.channel = self.connection_factory().channel()
183- self.assertNotEqual(None, publisher(oops))
184+ self.assertNotEqual([], publisher(oops))
185
186
187=== modified file 'oops_amqp/tests/test_receiver.py'
188--- oops_amqp/tests/test_receiver.py 2012-02-09 23:13:45 +0000
189+++ oops_amqp/tests/test_receiver.py 2012-08-10 03:02:19 +0000
190@@ -39,7 +39,7 @@
191 reports = []
192 def capture(report):
193 reports.append(report)
194- return report['id']
195+ return [report['id']]
196 expected_report = {'id': 'foo', 'otherkey': 42}
197 message = amqp.Message(bson.dumps(expected_report))
198 channel = self.useFixture(
199@@ -51,7 +51,7 @@
200 channel.basic_publish(
201 amqp.Message(sentinel), queue.exchange_name, routing_key="")
202 config = Config()
203- config.publishers.append(capture)
204+ config.publisher = capture
205 receiver = Receiver(config, self.connection_factory, queue.queue_name)
206 receiver.sentinel = sentinel
207 receiver.run_forever()
208@@ -62,7 +62,7 @@
209 reports = []
210 def capture(report):
211 reports.append(report)
212- return report['id']
213+ return [report['id']]
214 expected_report = {'id': 'foo', 'otherkey': 42}
215 message = amqp.Message(bson.dumps(expected_report))
216 channel = self.useFixture(
217@@ -71,7 +71,7 @@
218 channel.basic_publish(
219 message, queue.exchange_name, routing_key="")
220 config = Config()
221- config.publishers.append(capture)
222+ config.publisher = capture
223 # We don't want to loop forever: patch the channel so that after one
224 # call to wait (which will get our injected message) the loop will shut
225 # down.
226@@ -134,7 +134,7 @@
227 reports = []
228 def capture(report):
229 reports.append(report)
230- return report['id']
231+ return [report['id']]
232 expected_report = {'id': 'foo', 'otherkey': 42}
233 message = amqp.Message(bson.dumps(expected_report))
234 channel = self.useFixture(
235@@ -142,7 +142,7 @@
236 queue = self.useFixture(QueueFixture(channel, self.getUniqueString))
237 channel.basic_publish(message, queue.exchange_name, routing_key="")
238 config = Config()
239- config.publishers.append(capture)
240+ config.publisher = capture
241 state = {}
242 def error_once(func):
243 def wrapped(*args, **kwargs):
244
245=== added file 'oops_amqp/trace.py'
246--- oops_amqp/trace.py 1970-01-01 00:00:00 +0000
247+++ oops_amqp/trace.py 2012-08-10 03:02:19 +0000
248@@ -0,0 +1,74 @@
249+# Copyright (c) 2012, Canonical Ltd
250+#
251+# This program is free software: you can redistribute it and/or modify
252+# it under the terms of the GNU Lesser General Public License as published by
253+# the Free Software Foundation, version 3 only.
254+#
255+# This program is distributed in the hope that it will be useful,
256+# but WITHOUT ANY WARRANTY; without even the implied warranty of
257+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
258+# GNU Lesser General Public License for more details.
259+#
260+# You should have received a copy of the GNU Lesser General Public License
261+# along with this program. If not, see <http://www.gnu.org/licenses/>.
262+# GNU Lesser General Public License version 3 (see the file LICENSE).
263+
264+"""Trace OOPS reports coming from an AMQP queue."""
265+
266+from functools import partial
267+import sys
268+import optparse
269+from textwrap import dedent
270+
271+import amqplib.client_0_8 as amqp
272+import oops
273+import oops_amqp
274+
275+import anybson as bson
276+
277+
278+def main(argv=None):
279+ if argv is None:
280+ argv=sys.argv
281+ usage = dedent("""\
282+ %prog [options]
283+
284+ The following options must be supplied:
285+ --host
286+
287+ e.g.
288+ oops-amqp-trace --host "localhost:3472"
289+
290+ If you do not have a persistent queue, you should run this script
291+ before generating oopses, as AMQP will discard messages with no
292+ consumers.
293+ """)
294+ description = "Trace OOPS reports coming from an AMQP queue."
295+ parser = optparse.OptionParser(
296+ description=description, usage=usage)
297+ parser.add_option('--host', help="AQMP host / host:port.")
298+ parser.add_option('--username', help="AQMP username.", default="guest")
299+ parser.add_option('--password', help="AQMP password.", default="guest")
300+ parser.add_option('--vhost', help="AMQP vhost.", default="/")
301+ parser.add_option('--exchange', help="AMQP exchange name.", default="oopses")
302+ options, args = parser.parse_args(argv[1:])
303+ def needed(optname):
304+ if getattr(options, optname, None) is None:
305+ raise ValueError('option "%s" must be supplied' % optname)
306+ needed('host')
307+ factory = partial(
308+ amqp.Connection, host=options.host, userid=options.username,
309+ password=options.password, virtual_host=options.vhost)
310+ connection = factory()
311+ channel = connection.channel()
312+ channel.exchange_declare(options.exchange, type="fanout", durable=False,
313+ auto_delete=True)
314+ queue = channel.queue_declare(durable=False, auto_delete=True)[0]
315+ channel.queue_bind(queue, options.exchange)
316+ config = oops.Config()
317+ config.publisher = oops.pprint_to_stream(sys.stdout)
318+ receiver = oops_amqp.Receiver(config, factory, queue)
319+ try:
320+ receiver.run_forever()
321+ except KeyboardInterrupt:
322+ pass
323
324=== modified file 'setup.py'
325--- setup.py 2012-02-10 14:47:11 +0000
326+++ setup.py 2012-08-10 03:02:19 +0000
327@@ -23,7 +23,7 @@
328 os.path.join(os.path.dirname(__file__), 'README'), 'rb').read()
329
330 setup(name="oops_amqp",
331- version="0.0.6",
332+ version="0.0.7",
333 description=\
334 "OOPS AMQP transport.",
335 long_description=description,
336@@ -41,7 +41,7 @@
337 ],
338 install_requires = [
339 'pymongo',
340- 'oops',
341+ 'oops>=0.0.11',
342 'amqplib',
343 ],
344 extras_require = dict(
345@@ -51,4 +51,8 @@
346 'testtools',
347 ]
348 ),
349+ entry_points=dict(
350+ console_scripts=[ # `console_scripts` is a magic name to setuptools
351+ 'oops-amqp-trace = oops_amqp.trace:main',
352+ ]),
353 )
354
355=== modified file 'versions.cfg'
356--- versions.cfg 2011-10-10 21:49:50 +0000
357+++ versions.cfg 2012-08-10 03:02:19 +0000
358@@ -6,7 +6,8 @@
359 bson = 0.3.2
360 fixtures = 0.3.6
361 iso8601 = 0.1.4
362-oops = 0.0.6
363+oops = 0.0.13
364+pymongo = 2.1.1
365 pytz = 2010o
366 rabbitfixture = 0.3.2
367 setuptools = 0.6c11

Subscribers

People subscribed via source and target branches

to all changes: