Merge lp:~rackspace-titan/nova/log_all_stack_traces into lp:~hudson-openstack/nova/trunk

Proposed by Dan Prince
Status: Merged
Approved by: Paul Voccio
Approved revision: 1579
Merged at revision: 1614
Proposed branch: lp:~rackspace-titan/nova/log_all_stack_traces
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 47 lines (+3/-11)
2 files modified
nova/exception.py (+0/-7)
nova/tests/test_exception.py (+3/-4)
To merge this branch: bzr merge lp:~rackspace-titan/nova/log_all_stack_traces
Reviewer Review Type Date Requested Status
Matt Dietz (community) Approve
Todd Willey (community) Needs Information
Brian Lamar (community) Approve
Review via email: mp+75663@code.launchpad.net

Description of the change

Update exception.wrap_exception so that all exceptions (not just Error and NovaException types) get logged correctly.

To post a comment you must log in.
Revision history for this message
Brian Lamar (blamar) wrote :

I, too, support the removal of this block unless someone has a reason to keep. The attached bug has hit me a couple times and makes debugging difficult.

review: Approve
Revision history for this message
Todd Willey (xtoddx) wrote :

What is hard about debugging? You get the traceback from logger.exception, don't you? I'm not necessarily a fan of keeping it, but knowing where expected error cases are vs. brand new unanticipated errors has some value to developers.

review: Needs Information
Revision history for this message
Dan Prince (dan-prince) wrote :

> What is hard about debugging? You get the traceback from logger.exception,
> don't you? I'm not necessarily a fan of keeping it, but knowing where
> expected error cases are vs. brand new unanticipated errors has some value to
> developers.

Hey Todd,

Yes. I got a traceback that looked like this:

2011-09-15 12:37:02,599 ERROR nova.rpc [-] Exception during message handling
(nova.rpc): TRACE: Traceback (most recent call last):
(nova.rpc): TRACE: File "/usr/lib/pymodules/python2.6/nova/rpc/impl_kombu.py", line 620, in _process_data
(nova.rpc): TRACE: rval = node_func(context=ctxt, **node_args)
(nova.rpc): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 129, in wrapped
(nova.rpc): TRACE: raise Error(str(e))
(nova.rpc): TRACE: Error: list index out of range
(nova.rpc): TRACE:

---

Which pretty much tells me nothing! The issue I have with the code I removed is it effectively erases the full stack traces from runtime exceptions. Essentially anything that doesn't extend Error or NovaException won't get logged with a full stack trace.

The root cause of the exceptions getting wiped out is utils.to_primative which seems to have been added intentially but IMHO is somewhat of an evil method. Probably best not to get into that here however.

Revision history for this message
Matt Dietz (cerberus) wrote :

Rationale makes sense to me

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'nova/exception.py'
2--- nova/exception.py 2011-09-15 21:58:22 +0000
3+++ nova/exception.py 2011-09-16 03:04:17 +0000
4@@ -121,13 +121,6 @@
5 notifier.notify(publisher_id, temp_type, temp_level,
6 payload)
7
8- if (not isinstance(e, Error) and
9- not isinstance(e, NovaException)):
10- #exc_type, exc_value, exc_traceback = sys.exc_info()
11- LOG.exception(_('Uncaught exception'))
12- #logging.error(traceback.extract_stack(exc_traceback))
13- raise Error(str(e))
14-
15 # re-raise original exception since it may have been clobbered
16 raise exc_info[0], exc_info[1], exc_info[2]
17
18
19=== modified file 'nova/tests/test_exception.py'
20--- nova/tests/test_exception.py 2011-07-01 14:38:17 +0000
21+++ nova/tests/test_exception.py 2011-09-16 03:04:17 +0000
22@@ -74,14 +74,13 @@
23
24 def test_wrap_exception_throws_exception(self):
25 wrapped = exception.wrap_exception()
26- # Note that Exception is converted to Error ...
27- self.assertRaises(exception.Error, wrapped(bad_function_exception))
28+ self.assertRaises(Exception, wrapped(bad_function_exception))
29
30 def test_wrap_exception_with_notifier(self):
31 notifier = FakeNotifier()
32 wrapped = exception.wrap_exception(notifier, "publisher", "event",
33 "level")
34- self.assertRaises(exception.Error, wrapped(bad_function_exception))
35+ self.assertRaises(Exception, wrapped(bad_function_exception))
36 self.assertEquals(notifier.provided_publisher, "publisher")
37 self.assertEquals(notifier.provided_event, "event")
38 self.assertEquals(notifier.provided_priority, "level")
39@@ -91,7 +90,7 @@
40 def test_wrap_exception_with_notifier_defaults(self):
41 notifier = FakeNotifier()
42 wrapped = exception.wrap_exception(notifier)
43- self.assertRaises(exception.Error, wrapped(bad_function_exception))
44+ self.assertRaises(Exception, wrapped(bad_function_exception))
45 self.assertEquals(notifier.provided_publisher, None)
46 self.assertEquals(notifier.provided_event, "bad_function_exception")
47 self.assertEquals(notifier.provided_priority, notifier.ERROR)