Merge lp:~james-w/python-oops-celery/oops-reporter into lp:python-oops-celery
- oops-reporter
- Merge into trunk
Proposed by
James Westby
Status: | Merged |
---|---|
Approved by: | James Westby |
Approved revision: | 5 |
Merged at revision: | 3 |
Proposed branch: | lp:~james-w/python-oops-celery/oops-reporter |
Merge into: | lp:python-oops-celery |
Diff against target: |
443 lines (+397/-0) 9 files modified
.bzrignore (+2/-0) .testr.conf (+3/-0) LICENSE (+165/-0) README (+37/-0) oops_celery/__init__.py (+1/-0) oops_celery/oops_reporter.py (+26/-0) oops_celery/tests/__init__.py (+9/-0) oops_celery/tests/test_oops_reporter.py (+130/-0) setup.py (+24/-0) |
To merge this branch: | bzr merge lp:~james-w/python-oops-celery/oops-reporter |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Sidnei da Silva (community) | Approve | ||
Review via email: mp+90307@code.launchpad.net |
Commit message
Initial oops integration for celery.
Description of the change
Hi,
Here's my initial code for celery integration, it's small, but it's about as
big as I can make it while staying independent of project details.
Thanks,
James
To post a comment you must log in.
- 5. By James Westby
-
Fix the call to dirname.
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === added file '.bzrignore' |
2 | --- .bzrignore 1970-01-01 00:00:00 +0000 |
3 | +++ .bzrignore 2012-01-26 19:14:26 +0000 |
4 | @@ -0,0 +1,2 @@ |
5 | +.testrepository |
6 | +oops_celery.egg-info |
7 | |
8 | === added file '.testr.conf' |
9 | --- .testr.conf 1970-01-01 00:00:00 +0000 |
10 | +++ .testr.conf 2012-01-26 19:14:26 +0000 |
11 | @@ -0,0 +1,3 @@ |
12 | +[DEFAULT] |
13 | +test_command=python -m subunit.run $IDLIST |
14 | +test_id_list_default=oops_celery.tests.test_suite |
15 | |
16 | === added file 'LICENSE' |
17 | --- LICENSE 1970-01-01 00:00:00 +0000 |
18 | +++ LICENSE 2012-01-26 19:14:26 +0000 |
19 | @@ -0,0 +1,165 @@ |
20 | + GNU LESSER GENERAL PUBLIC LICENSE |
21 | + Version 3, 29 June 2007 |
22 | + |
23 | + Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> |
24 | + Everyone is permitted to copy and distribute verbatim copies |
25 | + of this license document, but changing it is not allowed. |
26 | + |
27 | + |
28 | + This version of the GNU Lesser General Public License incorporates |
29 | +the terms and conditions of version 3 of the GNU General Public |
30 | +License, supplemented by the additional permissions listed below. |
31 | + |
32 | + 0. Additional Definitions. |
33 | + |
34 | + As used herein, "this License" refers to version 3 of the GNU Lesser |
35 | +General Public License, and the "GNU GPL" refers to version 3 of the GNU |
36 | +General Public License. |
37 | + |
38 | + "The Library" refers to a covered work governed by this License, |
39 | +other than an Application or a Combined Work as defined below. |
40 | + |
41 | + An "Application" is any work that makes use of an interface provided |
42 | +by the Library, but which is not otherwise based on the Library. |
43 | +Defining a subclass of a class defined by the Library is deemed a mode |
44 | +of using an interface provided by the Library. |
45 | + |
46 | + A "Combined Work" is a work produced by combining or linking an |
47 | +Application with the Library. The particular version of the Library |
48 | +with which the Combined Work was made is also called the "Linked |
49 | +Version". |
50 | + |
51 | + The "Minimal Corresponding Source" for a Combined Work means the |
52 | +Corresponding Source for the Combined Work, excluding any source code |
53 | +for portions of the Combined Work that, considered in isolation, are |
54 | +based on the Application, and not on the Linked Version. |
55 | + |
56 | + The "Corresponding Application Code" for a Combined Work means the |
57 | +object code and/or source code for the Application, including any data |
58 | +and utility programs needed for reproducing the Combined Work from the |
59 | +Application, but excluding the System Libraries of the Combined Work. |
60 | + |
61 | + 1. Exception to Section 3 of the GNU GPL. |
62 | + |
63 | + You may convey a covered work under sections 3 and 4 of this License |
64 | +without being bound by section 3 of the GNU GPL. |
65 | + |
66 | + 2. Conveying Modified Versions. |
67 | + |
68 | + If you modify a copy of the Library, and, in your modifications, a |
69 | +facility refers to a function or data to be supplied by an Application |
70 | +that uses the facility (other than as an argument passed when the |
71 | +facility is invoked), then you may convey a copy of the modified |
72 | +version: |
73 | + |
74 | + a) under this License, provided that you make a good faith effort to |
75 | + ensure that, in the event an Application does not supply the |
76 | + function or data, the facility still operates, and performs |
77 | + whatever part of its purpose remains meaningful, or |
78 | + |
79 | + b) under the GNU GPL, with none of the additional permissions of |
80 | + this License applicable to that copy. |
81 | + |
82 | + 3. Object Code Incorporating Material from Library Header Files. |
83 | + |
84 | + The object code form of an Application may incorporate material from |
85 | +a header file that is part of the Library. You may convey such object |
86 | +code under terms of your choice, provided that, if the incorporated |
87 | +material is not limited to numerical parameters, data structure |
88 | +layouts and accessors, or small macros, inline functions and templates |
89 | +(ten or fewer lines in length), you do both of the following: |
90 | + |
91 | + a) Give prominent notice with each copy of the object code that the |
92 | + Library is used in it and that the Library and its use are |
93 | + covered by this License. |
94 | + |
95 | + b) Accompany the object code with a copy of the GNU GPL and this license |
96 | + document. |
97 | + |
98 | + 4. Combined Works. |
99 | + |
100 | + You may convey a Combined Work under terms of your choice that, |
101 | +taken together, effectively do not restrict modification of the |
102 | +portions of the Library contained in the Combined Work and reverse |
103 | +engineering for debugging such modifications, if you also do each of |
104 | +the following: |
105 | + |
106 | + a) Give prominent notice with each copy of the Combined Work that |
107 | + the Library is used in it and that the Library and its use are |
108 | + covered by this License. |
109 | + |
110 | + b) Accompany the Combined Work with a copy of the GNU GPL and this license |
111 | + document. |
112 | + |
113 | + c) For a Combined Work that displays copyright notices during |
114 | + execution, include the copyright notice for the Library among |
115 | + these notices, as well as a reference directing the user to the |
116 | + copies of the GNU GPL and this license document. |
117 | + |
118 | + d) Do one of the following: |
119 | + |
120 | + 0) Convey the Minimal Corresponding Source under the terms of this |
121 | + License, and the Corresponding Application Code in a form |
122 | + suitable for, and under terms that permit, the user to |
123 | + recombine or relink the Application with a modified version of |
124 | + the Linked Version to produce a modified Combined Work, in the |
125 | + manner specified by section 6 of the GNU GPL for conveying |
126 | + Corresponding Source. |
127 | + |
128 | + 1) Use a suitable shared library mechanism for linking with the |
129 | + Library. A suitable mechanism is one that (a) uses at run time |
130 | + a copy of the Library already present on the user's computer |
131 | + system, and (b) will operate properly with a modified version |
132 | + of the Library that is interface-compatible with the Linked |
133 | + Version. |
134 | + |
135 | + e) Provide Installation Information, but only if you would otherwise |
136 | + be required to provide such information under section 6 of the |
137 | + GNU GPL, and only to the extent that such information is |
138 | + necessary to install and execute a modified version of the |
139 | + Combined Work produced by recombining or relinking the |
140 | + Application with a modified version of the Linked Version. (If |
141 | + you use option 4d0, the Installation Information must accompany |
142 | + the Minimal Corresponding Source and Corresponding Application |
143 | + Code. If you use option 4d1, you must provide the Installation |
144 | + Information in the manner specified by section 6 of the GNU GPL |
145 | + for conveying Corresponding Source.) |
146 | + |
147 | + 5. Combined Libraries. |
148 | + |
149 | + You may place library facilities that are a work based on the |
150 | +Library side by side in a single library together with other library |
151 | +facilities that are not Applications and are not covered by this |
152 | +License, and convey such a combined library under terms of your |
153 | +choice, if you do both of the following: |
154 | + |
155 | + a) Accompany the combined library with a copy of the same work based |
156 | + on the Library, uncombined with any other library facilities, |
157 | + conveyed under the terms of this License. |
158 | + |
159 | + b) Give prominent notice with the combined library that part of it |
160 | + is a work based on the Library, and explaining where to find the |
161 | + accompanying uncombined form of the same work. |
162 | + |
163 | + 6. Revised Versions of the GNU Lesser General Public License. |
164 | + |
165 | + The Free Software Foundation may publish revised and/or new versions |
166 | +of the GNU Lesser General Public License from time to time. Such new |
167 | +versions will be similar in spirit to the present version, but may |
168 | +differ in detail to address new problems or concerns. |
169 | + |
170 | + Each version is given a distinguishing version number. If the |
171 | +Library as you received it specifies that a certain numbered version |
172 | +of the GNU Lesser General Public License "or any later version" |
173 | +applies to it, you have the option of following the terms and |
174 | +conditions either of that published version or of any later version |
175 | +published by the Free Software Foundation. If the Library as you |
176 | +received it does not specify a version number of the GNU Lesser |
177 | +General Public License, you may choose any version of the GNU Lesser |
178 | +General Public License ever published by the Free Software Foundation. |
179 | + |
180 | + If the Library as you received it specifies that a proxy can decide |
181 | +whether future versions of the GNU Lesser General Public License shall |
182 | +apply, that proxy's public statement of acceptance of any version is |
183 | +permanent authorization for you to choose that version for the |
184 | +Library. |
185 | |
186 | === added file 'README' |
187 | --- README 1970-01-01 00:00:00 +0000 |
188 | +++ README 2012-01-26 19:14:26 +0000 |
189 | @@ -0,0 +1,37 @@ |
190 | +python-oops-celery |
191 | +================== |
192 | + |
193 | +Integration so that failed celery tasks trigger Oopses. |
194 | + |
195 | +Depedencies |
196 | +----------- |
197 | + |
198 | +python-oops |
199 | +celery |
200 | + |
201 | +To run the tests you also need: |
202 | + |
203 | +python-testtools |
204 | + |
205 | +Use |
206 | +--- |
207 | + |
208 | +In your celery worker process run: |
209 | + |
210 | + from oops_celery import setup_oops_reporter |
211 | + |
212 | + setup_oops_reporter(config) |
213 | + |
214 | +where `config` is the `oops.Config` object you wish to use. |
215 | + |
216 | +Running Tests |
217 | +------------- |
218 | + |
219 | +Install `testrepository` and `python-subunit` and run |
220 | + |
221 | + testr init |
222 | + testr run |
223 | + |
224 | +or just run |
225 | + |
226 | + python -m testtools.run oops_celery.tests.test_suite |
227 | |
228 | === added directory 'oops_celery' |
229 | === added file 'oops_celery/__init__.py' |
230 | --- oops_celery/__init__.py 1970-01-01 00:00:00 +0000 |
231 | +++ oops_celery/__init__.py 2012-01-26 19:14:26 +0000 |
232 | @@ -0,0 +1,1 @@ |
233 | +from .oops_reporter import TaskOopsReporter, setup_oops_reporter |
234 | |
235 | === added file 'oops_celery/oops_reporter.py' |
236 | --- oops_celery/oops_reporter.py 1970-01-01 00:00:00 +0000 |
237 | +++ oops_celery/oops_reporter.py 2012-01-26 19:14:26 +0000 |
238 | @@ -0,0 +1,26 @@ |
239 | +from celery import signals |
240 | + |
241 | + |
242 | +class TaskOopsReporter(object): |
243 | + |
244 | + def __init__(self, config): |
245 | + self.config = config |
246 | + |
247 | + def on_failure(self, signal=None, sender=None, task_id=None, exception=None, |
248 | + args=None, kwargs=None, einfo=None, traceback=None): |
249 | + context = dict() |
250 | + if einfo is not None: |
251 | + context['exc_info'] = einfo.exc_info |
252 | + report = self.config.create(context) |
253 | + report.update({ |
254 | + 'celery.task_id': task_id, |
255 | + 'celery.task_args': args, |
256 | + 'celery.task_kwargs': kwargs, |
257 | + 'celery.sender': sender, |
258 | + }) |
259 | + self.config.publish(report) |
260 | + |
261 | + |
262 | +def setup_oops_reporter(config): |
263 | + reporter = TaskOopsReporter(config) |
264 | + signals.task_failure.connect(reporter.on_failure, weak=False) |
265 | |
266 | === added directory 'oops_celery/tests' |
267 | === added file 'oops_celery/tests/__init__.py' |
268 | --- oops_celery/tests/__init__.py 1970-01-01 00:00:00 +0000 |
269 | +++ oops_celery/tests/__init__.py 2012-01-26 19:14:26 +0000 |
270 | @@ -0,0 +1,9 @@ |
271 | +import unittest |
272 | + |
273 | +def test_suite(): |
274 | + module_names = [ |
275 | + 'oops_celery.tests.test_oops_reporter', |
276 | + ] |
277 | + loader = unittest.TestLoader() |
278 | + suite = loader.loadTestsFromNames(module_names) |
279 | + return suite |
280 | |
281 | === added file 'oops_celery/tests/test_oops_reporter.py' |
282 | --- oops_celery/tests/test_oops_reporter.py 1970-01-01 00:00:00 +0000 |
283 | +++ oops_celery/tests/test_oops_reporter.py 2012-01-26 19:14:26 +0000 |
284 | @@ -0,0 +1,130 @@ |
285 | +import sys |
286 | +import traceback |
287 | + |
288 | +from celery import signals |
289 | +from celery.datastructures import ExceptionInfo |
290 | +import oops |
291 | +from testtools import TestCase |
292 | + |
293 | +from oops_celery.oops_reporter import setup_oops_reporter, TaskOopsReporter |
294 | + |
295 | + |
296 | +class TaskOopsReporterTests(TestCase): |
297 | + |
298 | + def get_einfo(self): |
299 | + try: |
300 | + raise ValueError('a message') |
301 | + except ValueError: |
302 | + exc_info = sys.exc_info() |
303 | + return ExceptionInfo(exc_info) |
304 | + |
305 | + def setUp(self): |
306 | + super(TaskOopsReporterTests, self).setUp() |
307 | + self.oopses = [] |
308 | + config = self.get_oops_config() |
309 | + self.recorder = TaskOopsReporter(config) |
310 | + |
311 | + def get_oops_config(self): |
312 | + config = oops.Config() |
313 | + config.publishers.append(self.store_oops) |
314 | + return config |
315 | + |
316 | + def store_oops(self, oops): |
317 | + self.oopses.append(oops) |
318 | + |
319 | + def test_on_failure_publishes_oops(self): |
320 | + einfo = self.get_einfo() |
321 | + self.recorder.on_failure() |
322 | + self.assertEqual(1, len(self.oopses)) |
323 | + |
324 | + def test_on_failure_includes_task_id(self): |
325 | + einfo = self.get_einfo() |
326 | + task_id = 'atask' |
327 | + self.recorder.on_failure(task_id=task_id) |
328 | + self.assertEqual(task_id, self.oopses[0]['celery.task_id']) |
329 | + |
330 | + def test_on_failure_includes_args(self): |
331 | + einfo = self.get_einfo() |
332 | + args = [8, 9] |
333 | + self.recorder.on_failure(args=args) |
334 | + self.assertEqual(args, self.oopses[0]['celery.task_args']) |
335 | + |
336 | + def test_on_failure_includes_kwargs(self): |
337 | + einfo = self.get_einfo() |
338 | + kwargs = dict(a=1, b=2) |
339 | + self.recorder.on_failure(kwargs=kwargs) |
340 | + self.assertEqual(kwargs, self.oopses[0]['celery.task_kwargs']) |
341 | + |
342 | + def test_on_failure_includes_tb_text(self): |
343 | + einfo = self.get_einfo() |
344 | + self.recorder.on_failure(einfo=einfo) |
345 | + self.assertEqual(u''.join(traceback.format_tb(einfo.tb)), self.oopses[0]['tb_text']) |
346 | + |
347 | + def test_on_failure_includes_exception_type(self): |
348 | + einfo = self.get_einfo() |
349 | + self.recorder.on_failure(einfo=einfo) |
350 | + self.assertEqual(einfo.type.__name__, self.oopses[0]['type']) |
351 | + |
352 | + def test_on_failure_includes_exception_value(self): |
353 | + einfo = self.get_einfo() |
354 | + self.recorder.on_failure(einfo=einfo) |
355 | + self.assertEqual(unicode(einfo.exception), self.oopses[0]['value']) |
356 | + |
357 | + def test_on_failure_includes_sender(self): |
358 | + sender = object() |
359 | + self.recorder.on_failure(sender=sender) |
360 | + self.assertEqual(sender, self.oopses[0]['celery.sender']) |
361 | + |
362 | + def test_on_failure_accepts_signal_kwarg(self): |
363 | + # Test there's no error if celery passes the signal kwarg |
364 | + self.recorder.on_failure(signal=object()) |
365 | + |
366 | + def test_on_failure_accepts_exception_kwarg(self): |
367 | + # Test there's no error if celery passes the exception kwarg |
368 | + self.recorder.on_failure(exception=object()) |
369 | + |
370 | + def test_on_failure_accepts_traceback_kwarg(self): |
371 | + # Test there's no error if celery passes the traceback kwarg |
372 | + self.recorder.on_failure(traceback=object()) |
373 | + |
374 | + |
375 | +class SetupOopsReporterTests(TestCase): |
376 | + |
377 | + def setUp(self): |
378 | + super(SetupOopsReporterTests, self).setUp() |
379 | + original_receivers = signals.task_failure.receivers |
380 | + def restore_receivers(): |
381 | + signals.task_failure.receivers = original_receivers |
382 | + self.addCleanup(restore_receivers) |
383 | + signals.task_failure.receivers = [] |
384 | + |
385 | + def setup_oops_reporter(self, config=None): |
386 | + self.assertEqual(0, len(signals.task_failure.receivers)) |
387 | + if config is None: |
388 | + config = oops.Config() |
389 | + setup_oops_reporter(config) |
390 | + |
391 | + def test_registers_callback(self): |
392 | + self.setup_oops_reporter() |
393 | + self.assertEqual(1, len(signals.task_failure.receivers)) |
394 | + |
395 | + def test_callback_is_a_reporter(self): |
396 | + self.setup_oops_reporter() |
397 | + receiver = signals.task_failure.receivers[0][1] |
398 | + self.assertIsInstance(receiver.im_self, TaskOopsReporter) |
399 | + |
400 | + def test_uses_supplied_config(self): |
401 | + config = oops.Config() |
402 | + self.setup_oops_reporter(config=config) |
403 | + receiver = signals.task_failure.receivers[0][1] |
404 | + self.assertEqual(config, receiver.im_self.config) |
405 | + |
406 | + def test_generates_oops(self): |
407 | + config = oops.Config() |
408 | + oopses = [] |
409 | + def save_oops(oops): |
410 | + oopses.append(oops) |
411 | + config.publishers.append(save_oops) |
412 | + self.setup_oops_reporter(config=config) |
413 | + signals.task_failure.send(object()) |
414 | + self.assertEqual(1, len(oopses)) |
415 | |
416 | === added file 'setup.py' |
417 | --- setup.py 1970-01-01 00:00:00 +0000 |
418 | +++ setup.py 2012-01-26 19:14:26 +0000 |
419 | @@ -0,0 +1,24 @@ |
420 | +import os |
421 | + |
422 | +from setuptools import setup, find_packages |
423 | + |
424 | +setup( |
425 | + name='oops_celery', |
426 | + version='0.0.1', |
427 | + packages=find_packages(), |
428 | + include_package_data=True, |
429 | + maintainer='Launchpad developers', |
430 | + maintainer_email='launchpad-devs@lists.launchpad.net', |
431 | + description='Oops integration for celery', |
432 | + long_description=open(os.path.join(os.path.dirname(__file__), 'README')).read(), |
433 | + license='LGPLv3', |
434 | + url='http://launchpad.net/python-oops-celery', |
435 | + download_url='https://launchpad.net/python-oops-celery/+download', |
436 | + test_suite='oops_celery.tests', |
437 | + install_requires = [ |
438 | + 'oops', |
439 | + 'celery', |
440 | + ], |
441 | + # Auto-conversion to Python 3. |
442 | + use_2to3=True, |
443 | + ) |
Looks good to me. +1!