Access violation in epicsTime::epicsTime (segmentation fault)

Bug #1377113 reported by Uros Mitrovic
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
EPICS Base
Won't Fix
Medium
Jeff Hill

Bug Description

When calling function epicsTime::epicsTime in "epicsTime.cpp" sometimes (cca. once in five times), I receive unhandled exception and consequently my Ioc crashes. I'm using Visual Studio 2010 Express on Windows 8.1 Pro and EPICS base 3.14.12 and win32-x86 architecture. I've also tried on Windows 7 Pro, but the same issue persists.

Revision history for this message
Uros Mitrovic (mitar173) wrote :
Revision history for this message
Andrew Johnson (anj) wrote :

The C++ API for the epicsTime class can and does throw C++ exceptions to report errors. Your code must not allow those exceptions to escape from your C++ code up to the aSub record's process routine as may be happening here or your IOC will crash. Code running within the IOC that uses epicsTime objects must catch and handle any exceptions itself. If you don't want to do that, use the C API for epicsTime instead which catches them for you (but don't forget to check the error status return values that it returns instead).

The class definition in epicsTime.h defines two private exception classes that it might throw which are not derived from std::exception, but it can also throw std::exception classes as well. Once you've discovered what kind of exception is being thrown we can look into what could be causing it.

HTH,

- Andrew

Revision history for this message
Jeff Hill (johill-lanl) wrote :

It would appear, after looking at the stack trace, that new is in fact throwing the exception. This can happen, in situations where you haven't actually run low on memory, if a code has clobbered the linked list implementing pool, by for example, writing outside of the bounds of almost any block of allocated memory.

FWIW, when writing a try catch block in C++ before returning into C code its ok to catch all exceptions with ... because we shouldn't allow _any_ C++ exception to propagate into C code resulting in undefined behavior.

Revision history for this message
Uros Mitrovic (mitar173) wrote :

Thanks for the quick response. I'm calling actually, epicsTimeFromTM C function as it is seen in Call Stack. I'm getting unhandled exception (Unhandled exception at 0x00337437 in btsControlIoc.exe: 0xC0000005: Access violation reading location 0x00000014) in "epicsTime.cpp" in method epicsTime::epicsTime. So it seems the lti is not initialized for some reason.

Best regards,
Uros

Revision history for this message
Jeff Hill (johill-lanl) wrote :

Correct, thanks for your comments, I should had a close look.

After closer inspection I suspect that this "Access Violation" probably is happening due to some issue with the proper orchestration of the initialization of static C++ objects. This code "static epicsTimeLoadTimeInit & lti = * new epicsTimeLoadTimeInit ();" is actually following conventional wisdom for how this type of static C++ objects should be instantiated (i.e. the use of new for a static pointer versus the simpler approach of "static epicsTimeLoadTimeInit lti;" which is generally considered to be more likely to cause issues).

However, the issue becomes more complex, and also more unclear in the older c++ standards, when multiple threads are running. Its probably a good idea at this point to obtain an inspection of the current state of the other threads in this process when this happens, and in particular the main thread. Is there some code running before main executes?

As I recall the Visual Studio 2010 has had some known problems in this area, and also possibly that this version of visual studio doesn't use any mutual exclusion to protect the private once-only initialization boolean flag protecting this type of static variables from simultaneous entry into this function by two threads.

I actually wrote some c++ template code implementing compiler specific thread safe protection for initialization of such static variables which exists in some epics work that I completed based on the CVS main trunk, but unfortunately it isn't present in either of the bazaar R3.14 or R3.15 branches.

Also, the C++ 11 standard (and possibly earlier) is quite specific about how mutual exclusion should be provided for such initialization so newer compilers tend to fix this issue. As I recall almost all version of gcc use mutual exclusion to protect this type of intialization, and also perhaps that this issue is fixed in some more recent version of visual c++.

All the best,

Jeff

Revision history for this message
Uros Mitrovic (mitar173) wrote :

Hi Jeff,

no code is running before main execution. In the attachment is the screenshot of the main thread only, as I have vast number of threads. One thing I've noticed, and namely when I change line:
static epicsTimeLoadTimeInit & lti = * new epicsTimeLoadTimeInit ();
to
static epicsTimeLoadTimeInit lti

then I don't get any exception.

All the best,
Uros

Revision history for this message
Uros Mitrovic (mitar173) wrote :
Revision history for this message
Jeff Hill (johill-lanl) wrote :

> I have vast number of threads.

If many threads are already running then perhaps its implausible for this to be the first call to epicsTime::epicsTime and therefore my scenario in comment 5 is perhaps also appearing now to be less likely.

Changing to "static epicsTimeLoadTimeInit lti" might rearrange memory so that the already instantiated object doesn't become an unlucky recipient for corruption?

A tool like valgrind might be helpful. On windows there are various commercial products like purify and the intel tools. The commercial products tend to be quite compiler version specific.

This web site suggests that valgrind might also be run from a mingw installation? The win32-x86-mingw build is using gcc but is otherwise quite similar to the native windows port of EPICS that is used with win32-x86 (and the MS compiler).

http://sourceforge.net/p/valgrind4win/wiki/DevelopmentEnvironment/

Revision history for this message
mdavidsaver (mdavidsaver) wrote :

Jeff/Uros, any news on this issue?

Changed in epics-base:
assignee: nobody → Jeff Hill (johill-lanl)
importance: Undecided → Medium
status: New → Triaged
Andrew Johnson (anj)
Changed in epics-base:
status: Triaged → Incomplete
Revision history for this message
mdavidsaver (mdavidsaver) wrote :

For archival purposes. A possible fix done some time ago. I've never been able to reproduce this. The present code behaves correctly on even not so recent gcc as a implementation defined behavior. With c++11 this becomes the standard behavior.

Andrew Johnson (anj)
Changed in epics-base:
status: Incomplete → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.