Conference clt::cma

1495.0. "bugcheck debug help?" by CHEFS::DAVIDSONS () Tue Feb 25 1997 14:57

    The following o/p shows:
    
    - application core dumping when not run from debugger
    - receiving SEGV when run from debuuger
    
    Problem is in the 2nd scenario 'show thread' or anything else
    causes a core dump.
    
    Any idea how to debug this?
    (I know it won't be a DECthread problem)
    
    Does 'where' always dump stack of current thread?
    
    Thanks,
    	Stuart.
     
environment:

Ladebug Debugger Version 4.0-14A
Digital UNIX V3.2D-1
    
%DECthreads bugcheck (version V3.12-311), terminating execution.
% Running on DEC OSF/1 AXP [OSF1 alpha V3.2(41); cpu type 30, configured for 1
%  cpu, 1 cpu in box, 96Mb]
% Reason: test and set: high order bits corrupt at 0x4e288

# file core
core:   core dump, generated from 'nss.exe'
# decladebug /usr/sbin/nss.exe core
Welcome to the Ladebug Debugger Version 4.0-14A
------------------
object file name: /usr/sbin/nss.exe
core file name: core
Reading symbolic information ...done
Core file produced from executable nss.exe
Thread 0xfffffc000463a0a0 terminated at PC 0x3ff8051b298 by signal SEGV
(ladebug) where
>0  0x3ff8051b298 in /usr/shlib/libc_r.so
#1  0x3ff8057003c in exc_raise(0x3ffc01f71d0, 0x0, 0x3ffc01f72e8, 0x35,
0x3ffbff
1b72c, 0x0) DebugInformationStrippedFromFile87:???
#2  0x3ff80570198 in exc_raise_status(0x45586732, 0x2, 0x0, 0x0, 0x0, 0x0)
Debug
InformationStrippedFromFile87:???
#3  0x3ff8056fe54 in cma__error(0x0, 0x0, 0x0, 0x0, 0x3ff8057b818, 0x2de08)
Debu
gInformationStrippedFromFile86:???
#4  0x3ff8057b814 in /usr/shlib/libpthreads.so
(ladebug) dump
The "dump" command has failed because the debuggee process is not active.
(ladebug) call cma_debug_cmd("show -c")
Thread state is inaccessible for executing call command.
(ladebug) show thread
  Thread #  Id                  State
> 1         0xfffffc000463a0a0  dead
  2         0xfffffc000339fae0  dead
  3         0xfffffc000463a000  dead
  4         0xfffffc000463b4a0  dead
  5         0xfffffc000463aaa0  dead
(ladebug)

running the program from decladebug:

Thread received signal SEGV
stopped at [???: ??? 0x3ff8308b834]
(ladebug) where
>0  0x3ff8308b834 in /usr/shlib/libc_r.so
#1  0x3ffbff18220 in DnaCreateTowerRaw(0x3d608, 0xffffffffffffffff, 0x3e808,
0x0, 0x3ffc1cb72e8, 0x6d) DebugInformationStrippedFromFile0:???
#2  0x3ffbff1adac in UnknownProcedure38FromFile0(0x2de88, 0x43fa8, 0x0, 0x27,
0x3d008, 0x10000000b) DebugInformationStrippedFromFile0:???
#3  0x3ffbff1b728 in dnaNodenameToTowerSet(0x0, 0x43fa8, 0x2de88, 0x45f58, 0x0,
0x45f58) DebugInformationStrippedFromFile0:???
#4  0x3ffbff1d880 in UnknownProcedure61FromFile0(0x3e9b8, 0x1, 0x3e9, 0x0, 0x0,
0x43fa8) DebugInformationStrippedFromFile0:???
#5  0x3ffbff1e158 in dnaConnectNodename(0x3e808, 0x12002f6b0, 0x0, 0x0, 0x0,
0x11ffffcd8) DebugInformationStrippedFromFile0:???
#6  0x120082e3c in ((TP4_Connector*)0x3ae08)->activate() ss_tp4_connector.C:59
#7  0x120090270 in ((Socket*)0x3ae08)->Activate() ss_socket.C:352
#8  0x1200be484 in ((Application_Thread*)0x3d498)->Init()
ss_application_thread.C:203
#9  0x1200e9308 in Thread_Base::Main_loop(arg=0x3d498) Thread_Base.C:365
#10 0x3ff82041ccc in /usr/shlib/libpthreads.so
(ladebug) call cma_debug_cmd("show -c")
Current thread is 4 (0x3e418)
(ladebug) thread 4
Thread received signal SEGV
/usr/sbin/nss: 3585 Abort - core dumped

.0> Any idea how to debug this?

You are the victim of a promiscuous random memory corruptor.  I, myself,
would recommend a code review looking for use of unitialized (local) pointer
variables and writing beyond array boundaries.  Or, you could try using a
tool such as Atom/Third Degree.

.0> Does 'where' always dump stack of current thread?

Yes.

.0> % Reason: test and set: high order bits corrupt at 0x4e288

This indicates a memory corruptor.

Regarding your first debug senario, core files left by SEGVs (at least prior
to V4) don't contain anything useful on the stack.  (Sorry 'bout that, but
it's an old story.)  And, of course you cannot call cma_debug_cmd() when
analyzing a core dump because there's no process there to run it in.  But,
you -can- run the process under the debugger and catch it right at the point
of the SEGV, so you're alright (sort of).

In your second debugging session, you can see that the victim of the
corruption is a libc(_r) routine called from DNA.  (I don't know that that
helps much, given that DNA has been stripped, but...)

.0> (ladebug) thread 4
.0> Thread received signal SEGV

Given that something has stomped all over your process's memory, it's not all
that surprising that the "thread" command dies.  (It's too bad that it takes
out Ladebug when it does, but c'est la vie...)


		Sorry I can't be of more help,


				Webb

    Thanks. I tried using Third Degree before posting the base note
    but as the man page says it's no use on threaded code, also the
    instrumented exe was smaller than the original and nothing much
    appeared in the log.
    
    Looks like V4.0 is utopian but this project won't move from V3.2*
    
    I'm considering using the public domain debug_malloc library, any
    thoughts?
    
    Stuart.

.2> I'm considering using the public domain debug_malloc library, any
.2> thoughts?

Yeah:  I'll be impressed if it's thread-safe (and works).


					Webb