Merge lp:~james-w/python-oops-celery/oops-reporter into lp:python-oops-celery

Proposed by James Westby
Status: Merged
Approved by: James Westby
Approved revision: 5
Merged at revision: 3
Proposed branch: lp:~james-w/python-oops-celery/oops-reporter
Merge into: lp:python-oops-celery
Diff against target: 443 lines (+397/-0)
9 files modified
.bzrignore (+2/-0)
.testr.conf (+3/-0)
LICENSE (+165/-0)
README (+37/-0)
oops_celery/__init__.py (+1/-0)
oops_celery/oops_reporter.py (+26/-0)
oops_celery/tests/__init__.py (+9/-0)
oops_celery/tests/test_oops_reporter.py (+130/-0)
setup.py (+24/-0)
To merge this branch: bzr merge lp:~james-w/python-oops-celery/oops-reporter
Reviewer Review Type Date Requested Status
Sidnei da Silva (community) Approve
Review via email: mp+90307@code.launchpad.net

Commit message

Initial oops integration for celery.

Description of the change

Hi,

Here's my initial code for celery integration, it's small, but it's about as
big as I can make it while staying independent of project details.

Thanks,

James

To post a comment you must log in.
Revision history for this message
Sidnei da Silva (sidnei) wrote :

Looks good to me. +1!

review: Approve
5. By James Westby

Fix the call to dirname.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file '.bzrignore'
2--- .bzrignore 1970-01-01 00:00:00 +0000
3+++ .bzrignore 2012-01-26 19:14:26 +0000
4@@ -0,0 +1,2 @@
5+.testrepository
6+oops_celery.egg-info
7
8=== added file '.testr.conf'
9--- .testr.conf 1970-01-01 00:00:00 +0000
10+++ .testr.conf 2012-01-26 19:14:26 +0000
11@@ -0,0 +1,3 @@
12+[DEFAULT]
13+test_command=python -m subunit.run $IDLIST
14+test_id_list_default=oops_celery.tests.test_suite
15
16=== added file 'LICENSE'
17--- LICENSE 1970-01-01 00:00:00 +0000
18+++ LICENSE 2012-01-26 19:14:26 +0000
19@@ -0,0 +1,165 @@
20+ GNU LESSER GENERAL PUBLIC LICENSE
21+ Version 3, 29 June 2007
22+
23+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
24+ Everyone is permitted to copy and distribute verbatim copies
25+ of this license document, but changing it is not allowed.
26+
27+
28+ This version of the GNU Lesser General Public License incorporates
29+the terms and conditions of version 3 of the GNU General Public
30+License, supplemented by the additional permissions listed below.
31+
32+ 0. Additional Definitions.
33+
34+ As used herein, "this License" refers to version 3 of the GNU Lesser
35+General Public License, and the "GNU GPL" refers to version 3 of the GNU
36+General Public License.
37+
38+ "The Library" refers to a covered work governed by this License,
39+other than an Application or a Combined Work as defined below.
40+
41+ An "Application" is any work that makes use of an interface provided
42+by the Library, but which is not otherwise based on the Library.
43+Defining a subclass of a class defined by the Library is deemed a mode
44+of using an interface provided by the Library.
45+
46+ A "Combined Work" is a work produced by combining or linking an
47+Application with the Library. The particular version of the Library
48+with which the Combined Work was made is also called the "Linked
49+Version".
50+
51+ The "Minimal Corresponding Source" for a Combined Work means the
52+Corresponding Source for the Combined Work, excluding any source code
53+for portions of the Combined Work that, considered in isolation, are
54+based on the Application, and not on the Linked Version.
55+
56+ The "Corresponding Application Code" for a Combined Work means the
57+object code and/or source code for the Application, including any data
58+and utility programs needed for reproducing the Combined Work from the
59+Application, but excluding the System Libraries of the Combined Work.
60+
61+ 1. Exception to Section 3 of the GNU GPL.
62+
63+ You may convey a covered work under sections 3 and 4 of this License
64+without being bound by section 3 of the GNU GPL.
65+
66+ 2. Conveying Modified Versions.
67+
68+ If you modify a copy of the Library, and, in your modifications, a
69+facility refers to a function or data to be supplied by an Application
70+that uses the facility (other than as an argument passed when the
71+facility is invoked), then you may convey a copy of the modified
72+version:
73+
74+ a) under this License, provided that you make a good faith effort to
75+ ensure that, in the event an Application does not supply the
76+ function or data, the facility still operates, and performs
77+ whatever part of its purpose remains meaningful, or
78+
79+ b) under the GNU GPL, with none of the additional permissions of
80+ this License applicable to that copy.
81+
82+ 3. Object Code Incorporating Material from Library Header Files.
83+
84+ The object code form of an Application may incorporate material from
85+a header file that is part of the Library. You may convey such object
86+code under terms of your choice, provided that, if the incorporated
87+material is not limited to numerical parameters, data structure
88+layouts and accessors, or small macros, inline functions and templates
89+(ten or fewer lines in length), you do both of the following:
90+
91+ a) Give prominent notice with each copy of the object code that the
92+ Library is used in it and that the Library and its use are
93+ covered by this License.
94+
95+ b) Accompany the object code with a copy of the GNU GPL and this license
96+ document.
97+
98+ 4. Combined Works.
99+
100+ You may convey a Combined Work under terms of your choice that,
101+taken together, effectively do not restrict modification of the
102+portions of the Library contained in the Combined Work and reverse
103+engineering for debugging such modifications, if you also do each of
104+the following:
105+
106+ a) Give prominent notice with each copy of the Combined Work that
107+ the Library is used in it and that the Library and its use are
108+ covered by this License.
109+
110+ b) Accompany the Combined Work with a copy of the GNU GPL and this license
111+ document.
112+
113+ c) For a Combined Work that displays copyright notices during
114+ execution, include the copyright notice for the Library among
115+ these notices, as well as a reference directing the user to the
116+ copies of the GNU GPL and this license document.
117+
118+ d) Do one of the following:
119+
120+ 0) Convey the Minimal Corresponding Source under the terms of this
121+ License, and the Corresponding Application Code in a form
122+ suitable for, and under terms that permit, the user to
123+ recombine or relink the Application with a modified version of
124+ the Linked Version to produce a modified Combined Work, in the
125+ manner specified by section 6 of the GNU GPL for conveying
126+ Corresponding Source.
127+
128+ 1) Use a suitable shared library mechanism for linking with the
129+ Library. A suitable mechanism is one that (a) uses at run time
130+ a copy of the Library already present on the user's computer
131+ system, and (b) will operate properly with a modified version
132+ of the Library that is interface-compatible with the Linked
133+ Version.
134+
135+ e) Provide Installation Information, but only if you would otherwise
136+ be required to provide such information under section 6 of the
137+ GNU GPL, and only to the extent that such information is
138+ necessary to install and execute a modified version of the
139+ Combined Work produced by recombining or relinking the
140+ Application with a modified version of the Linked Version. (If
141+ you use option 4d0, the Installation Information must accompany
142+ the Minimal Corresponding Source and Corresponding Application
143+ Code. If you use option 4d1, you must provide the Installation
144+ Information in the manner specified by section 6 of the GNU GPL
145+ for conveying Corresponding Source.)
146+
147+ 5. Combined Libraries.
148+
149+ You may place library facilities that are a work based on the
150+Library side by side in a single library together with other library
151+facilities that are not Applications and are not covered by this
152+License, and convey such a combined library under terms of your
153+choice, if you do both of the following:
154+
155+ a) Accompany the combined library with a copy of the same work based
156+ on the Library, uncombined with any other library facilities,
157+ conveyed under the terms of this License.
158+
159+ b) Give prominent notice with the combined library that part of it
160+ is a work based on the Library, and explaining where to find the
161+ accompanying uncombined form of the same work.
162+
163+ 6. Revised Versions of the GNU Lesser General Public License.
164+
165+ The Free Software Foundation may publish revised and/or new versions
166+of the GNU Lesser General Public License from time to time. Such new
167+versions will be similar in spirit to the present version, but may
168+differ in detail to address new problems or concerns.
169+
170+ Each version is given a distinguishing version number. If the
171+Library as you received it specifies that a certain numbered version
172+of the GNU Lesser General Public License "or any later version"
173+applies to it, you have the option of following the terms and
174+conditions either of that published version or of any later version
175+published by the Free Software Foundation. If the Library as you
176+received it does not specify a version number of the GNU Lesser
177+General Public License, you may choose any version of the GNU Lesser
178+General Public License ever published by the Free Software Foundation.
179+
180+ If the Library as you received it specifies that a proxy can decide
181+whether future versions of the GNU Lesser General Public License shall
182+apply, that proxy's public statement of acceptance of any version is
183+permanent authorization for you to choose that version for the
184+Library.
185
186=== added file 'README'
187--- README 1970-01-01 00:00:00 +0000
188+++ README 2012-01-26 19:14:26 +0000
189@@ -0,0 +1,37 @@
190+python-oops-celery
191+==================
192+
193+Integration so that failed celery tasks trigger Oopses.
194+
195+Depedencies
196+-----------
197+
198+python-oops
199+celery
200+
201+To run the tests you also need:
202+
203+python-testtools
204+
205+Use
206+---
207+
208+In your celery worker process run:
209+
210+ from oops_celery import setup_oops_reporter
211+
212+ setup_oops_reporter(config)
213+
214+where `config` is the `oops.Config` object you wish to use.
215+
216+Running Tests
217+-------------
218+
219+Install `testrepository` and `python-subunit` and run
220+
221+ testr init
222+ testr run
223+
224+or just run
225+
226+ python -m testtools.run oops_celery.tests.test_suite
227
228=== added directory 'oops_celery'
229=== added file 'oops_celery/__init__.py'
230--- oops_celery/__init__.py 1970-01-01 00:00:00 +0000
231+++ oops_celery/__init__.py 2012-01-26 19:14:26 +0000
232@@ -0,0 +1,1 @@
233+from .oops_reporter import TaskOopsReporter, setup_oops_reporter
234
235=== added file 'oops_celery/oops_reporter.py'
236--- oops_celery/oops_reporter.py 1970-01-01 00:00:00 +0000
237+++ oops_celery/oops_reporter.py 2012-01-26 19:14:26 +0000
238@@ -0,0 +1,26 @@
239+from celery import signals
240+
241+
242+class TaskOopsReporter(object):
243+
244+ def __init__(self, config):
245+ self.config = config
246+
247+ def on_failure(self, signal=None, sender=None, task_id=None, exception=None,
248+ args=None, kwargs=None, einfo=None, traceback=None):
249+ context = dict()
250+ if einfo is not None:
251+ context['exc_info'] = einfo.exc_info
252+ report = self.config.create(context)
253+ report.update({
254+ 'celery.task_id': task_id,
255+ 'celery.task_args': args,
256+ 'celery.task_kwargs': kwargs,
257+ 'celery.sender': sender,
258+ })
259+ self.config.publish(report)
260+
261+
262+def setup_oops_reporter(config):
263+ reporter = TaskOopsReporter(config)
264+ signals.task_failure.connect(reporter.on_failure, weak=False)
265
266=== added directory 'oops_celery/tests'
267=== added file 'oops_celery/tests/__init__.py'
268--- oops_celery/tests/__init__.py 1970-01-01 00:00:00 +0000
269+++ oops_celery/tests/__init__.py 2012-01-26 19:14:26 +0000
270@@ -0,0 +1,9 @@
271+import unittest
272+
273+def test_suite():
274+ module_names = [
275+ 'oops_celery.tests.test_oops_reporter',
276+ ]
277+ loader = unittest.TestLoader()
278+ suite = loader.loadTestsFromNames(module_names)
279+ return suite
280
281=== added file 'oops_celery/tests/test_oops_reporter.py'
282--- oops_celery/tests/test_oops_reporter.py 1970-01-01 00:00:00 +0000
283+++ oops_celery/tests/test_oops_reporter.py 2012-01-26 19:14:26 +0000
284@@ -0,0 +1,130 @@
285+import sys
286+import traceback
287+
288+from celery import signals
289+from celery.datastructures import ExceptionInfo
290+import oops
291+from testtools import TestCase
292+
293+from oops_celery.oops_reporter import setup_oops_reporter, TaskOopsReporter
294+
295+
296+class TaskOopsReporterTests(TestCase):
297+
298+ def get_einfo(self):
299+ try:
300+ raise ValueError('a message')
301+ except ValueError:
302+ exc_info = sys.exc_info()
303+ return ExceptionInfo(exc_info)
304+
305+ def setUp(self):
306+ super(TaskOopsReporterTests, self).setUp()
307+ self.oopses = []
308+ config = self.get_oops_config()
309+ self.recorder = TaskOopsReporter(config)
310+
311+ def get_oops_config(self):
312+ config = oops.Config()
313+ config.publishers.append(self.store_oops)
314+ return config
315+
316+ def store_oops(self, oops):
317+ self.oopses.append(oops)
318+
319+ def test_on_failure_publishes_oops(self):
320+ einfo = self.get_einfo()
321+ self.recorder.on_failure()
322+ self.assertEqual(1, len(self.oopses))
323+
324+ def test_on_failure_includes_task_id(self):
325+ einfo = self.get_einfo()
326+ task_id = 'atask'
327+ self.recorder.on_failure(task_id=task_id)
328+ self.assertEqual(task_id, self.oopses[0]['celery.task_id'])
329+
330+ def test_on_failure_includes_args(self):
331+ einfo = self.get_einfo()
332+ args = [8, 9]
333+ self.recorder.on_failure(args=args)
334+ self.assertEqual(args, self.oopses[0]['celery.task_args'])
335+
336+ def test_on_failure_includes_kwargs(self):
337+ einfo = self.get_einfo()
338+ kwargs = dict(a=1, b=2)
339+ self.recorder.on_failure(kwargs=kwargs)
340+ self.assertEqual(kwargs, self.oopses[0]['celery.task_kwargs'])
341+
342+ def test_on_failure_includes_tb_text(self):
343+ einfo = self.get_einfo()
344+ self.recorder.on_failure(einfo=einfo)
345+ self.assertEqual(u''.join(traceback.format_tb(einfo.tb)), self.oopses[0]['tb_text'])
346+
347+ def test_on_failure_includes_exception_type(self):
348+ einfo = self.get_einfo()
349+ self.recorder.on_failure(einfo=einfo)
350+ self.assertEqual(einfo.type.__name__, self.oopses[0]['type'])
351+
352+ def test_on_failure_includes_exception_value(self):
353+ einfo = self.get_einfo()
354+ self.recorder.on_failure(einfo=einfo)
355+ self.assertEqual(unicode(einfo.exception), self.oopses[0]['value'])
356+
357+ def test_on_failure_includes_sender(self):
358+ sender = object()
359+ self.recorder.on_failure(sender=sender)
360+ self.assertEqual(sender, self.oopses[0]['celery.sender'])
361+
362+ def test_on_failure_accepts_signal_kwarg(self):
363+ # Test there's no error if celery passes the signal kwarg
364+ self.recorder.on_failure(signal=object())
365+
366+ def test_on_failure_accepts_exception_kwarg(self):
367+ # Test there's no error if celery passes the exception kwarg
368+ self.recorder.on_failure(exception=object())
369+
370+ def test_on_failure_accepts_traceback_kwarg(self):
371+ # Test there's no error if celery passes the traceback kwarg
372+ self.recorder.on_failure(traceback=object())
373+
374+
375+class SetupOopsReporterTests(TestCase):
376+
377+ def setUp(self):
378+ super(SetupOopsReporterTests, self).setUp()
379+ original_receivers = signals.task_failure.receivers
380+ def restore_receivers():
381+ signals.task_failure.receivers = original_receivers
382+ self.addCleanup(restore_receivers)
383+ signals.task_failure.receivers = []
384+
385+ def setup_oops_reporter(self, config=None):
386+ self.assertEqual(0, len(signals.task_failure.receivers))
387+ if config is None:
388+ config = oops.Config()
389+ setup_oops_reporter(config)
390+
391+ def test_registers_callback(self):
392+ self.setup_oops_reporter()
393+ self.assertEqual(1, len(signals.task_failure.receivers))
394+
395+ def test_callback_is_a_reporter(self):
396+ self.setup_oops_reporter()
397+ receiver = signals.task_failure.receivers[0][1]
398+ self.assertIsInstance(receiver.im_self, TaskOopsReporter)
399+
400+ def test_uses_supplied_config(self):
401+ config = oops.Config()
402+ self.setup_oops_reporter(config=config)
403+ receiver = signals.task_failure.receivers[0][1]
404+ self.assertEqual(config, receiver.im_self.config)
405+
406+ def test_generates_oops(self):
407+ config = oops.Config()
408+ oopses = []
409+ def save_oops(oops):
410+ oopses.append(oops)
411+ config.publishers.append(save_oops)
412+ self.setup_oops_reporter(config=config)
413+ signals.task_failure.send(object())
414+ self.assertEqual(1, len(oopses))
415
416=== added file 'setup.py'
417--- setup.py 1970-01-01 00:00:00 +0000
418+++ setup.py 2012-01-26 19:14:26 +0000
419@@ -0,0 +1,24 @@
420+import os
421+
422+from setuptools import setup, find_packages
423+
424+setup(
425+ name='oops_celery',
426+ version='0.0.1',
427+ packages=find_packages(),
428+ include_package_data=True,
429+ maintainer='Launchpad developers',
430+ maintainer_email='launchpad-devs@lists.launchpad.net',
431+ description='Oops integration for celery',
432+ long_description=open(os.path.join(os.path.dirname(__file__), 'README')).read(),
433+ license='LGPLv3',
434+ url='http://launchpad.net/python-oops-celery',
435+ download_url='https://launchpad.net/python-oops-celery/+download',
436+ test_suite='oops_celery.tests',
437+ install_requires = [
438+ 'oops',
439+ 'celery',
440+ ],
441+ # Auto-conversion to Python 3.
442+ use_2to3=True,
443+ )

Subscribers

People subscribed via source and target branches