Solved

understanding pstack output

Posted on 2016-11-02
2
11 Views
Last Modified: 2016-11-23
Hi All,
Am new to pstack. Can you please let me know what is happening with the below process. Process stays in busy state while in this condition. Not sure if it is a deadlock or what wrong has gone with the process.
[root@Site-B-NE5 /]# /root/pstack 2566
Thread 19 (Thread 0x484ba290 (LWP 3102)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x15755f08, mutex=0x15755f38)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x0f963650 in util::ConditionMutex::Lock() () from ./usr/st/lib/libcore.so
#4  0x1219b8a4 in lps::CmlSession::Lock() ()
#5  0x1219d128 in lps::CmlSession::CmlSession(lps::CmlSession::SessionType, std::string const&) ()
#6  0x127c2ea8 in lps::PhyCard::PhyCard(std::string const&, lpsdevice::Card::Slot, mgmt::Type, type::PhyCardId, type::ShelfId) ()
#7  0x0fcd92c0 in lps::Sr9624CardFactory::CreateCard(std::string const&, lpsdevice::Card::Type, lpsdevice::Card::Slot, mgmt::Type, type::PhyCardId, type::ShelfId) () from ./usr/st/lib/libdevice-cfgd.so
#8  0x127b3cdc in lps::DeviceManager::HandleCardEvent(std::string const&, mgmt::Type, mgmt::Action, bool, bool, bool) ()
#9  0x127b6b1c in lps::DeviceManager::_QueuedHandleMessage(pubsub::Message const&) ()
#10 0x0f890038 in pubsub::QueuedSubscriberImpl::_QueueHandler(_CORBA_ConstrType_Variable_Var<pubsub::Message>) () from ./usr/st/lib/libcore.so
#11 0x0f8917f4 in util::_MemFunWorkerVal<pubsub::QueuedSubscriberImpl, _CORBA_ConstrType_Variable_Var<pubsub::Message> >::operator()(_CORBA_ConstrType_Variable_Var<pubsub::Message>&) const () from ./usr/st/lib/libcore.so
#12 0x0f89360c in util::WorkQueue<_CORBA_ConstrType_Variable_Var<pubsub::Message>, std::deque<_CORBA_ConstrType_Variable_Var<pubsub::Message>, std::allocator<_CORBA_ConstrType_Variable_Var<pubsub::Message> > >, util::NotifierStub<_CORBA_ConstrType_Variable_Var<pubsub::Message> >, util::Extractor<_CORBA_ConstrType_Variable_Var<pubsub::Message>, std::deque<_CORBA_ConstrType_Variable_Var<pubsub::Message>, std::allocator<_CORBA_ConstrType_Variable_Var<pubsub::Message> > >, util::NotifierStub<_CORBA_ConstrType_Variable_Var<pubsub::Message> > > >::_Dispatch(util::RefPtr<util::ThreadPoolRunnable>) () from ./usr/st/lib/libcore.so
#13 0x0f891468 in util::WorkQueue<_CORBA_ConstrType_Variable_Var<pubsub::Message>, std::deque<_CORBA_ConstrType_Variable_Var<pubsub::Message>, std::allocator<_CORBA_ConstrType_Variable_Var<pubsub::Message> > >, util::NotifierStub<_CORBA_ConstrType_Variable_Var<pubsub::Message> >, util::Extractor<_CORBA_ConstrType_Variable_Var<pubsub::Message>, std::deque<_CORBA_ConstrType_Variable_Var<pubsub::Message>, std::allocator<_CORBA_ConstrType_Variable_Var<pubsub::Message> > >, util::NotifierStub<_CORBA_ConstrType_Variable_Var<pubsub::Message> > > >::_Dispatcher::operator()() () from ./usr/st/lib/libcore.so
#14 0x0f98cfa4 in util::ThreadPool::_ThreadMain(void*) ()
#15 0x0f988e10 in ?? () from ./usr/st/lib/libcore.so
#16 0x0f989090 in util::Thread::_ThreadMain(void*) ()
#17 0x0fdf7cf0 in start_thread (arg=0x484ba290) at pthread_create.c:310
#18 0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 18 (Thread 0x48cba290 (LWP 3103)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x15164690, mutex=0x15164658)
#1  0x0f9c8130 in thread::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f9a5910 in trace::event::Queue::_getNext(trace::event::Queue::Hint&, trace::event::Buffer&, int&, bool, bool) () from ./usr/st/lib/libcore.so
#3  0x0f9b3c2c in trace::sink::Manager::Internal::_readThread() ()
#4  0x0f9b3ce0 in trace::sink::Manager::Internal::_readMain(void*) ()
#5  0x0f9c7aa0 in thread::Group::_thread() () from ./usr/st/lib/libcore.so
#6  0x0f9c7c04 in thread::Group::_main(void*) () from ./usr/st/lib/libcore.so
#7  0x0f9c81e4 in thread::Thread::_main(void*) () from ./usr/st/lib/libcore.so
#8  0x0fdf7cf0 in start_thread (arg=0x48cba290) at pthread_create.c:310
#9  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 17 (Thread 0x48cf9290 (LWP 3104)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x15725b98, mutex=0x15725b40)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x0f98cc90 in util::ThreadPool::_GetWork() () from ./usr/st/lib/libcore.so
#4  0x0f98d01c in util::ThreadPool::_ThreadMain(void*) ()
#5  0x0f988e10 in ?? () from ./usr/st/lib/libcore.so
#6  0x0f989090 in util::Thread::_ThreadMain(void*) ()
#7  0x0fdf7cf0 in start_thread (arg=0x48cf9290) at pthread_create.c:310
#8  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 16 (Thread 0x48f3e290 (LWP 3105)):
#0  0x0fdfcdbc in __pthread_cond_timedwait (cond=0xfa9ba98, mutex=0xfa9ba38,
#1  0x0f974cc4 in util::SimpleMutex::TimedWait(pthread_cond_t*, timespec const&) () from ./usr/st/lib/libcore.so
#2  0x0f963a30 in util::ConditionVariable::WaitWithTimeout(util::SimpleMutex*, timespec const&) () from ./usr/st/lib/libcore.so
#3  0x0f9913d0 in util::TimerScheduler::_WaitForTimeout() ()
#4  0x0f991558 in util::TimerScheduler::_Thread() ()
#5  0x11c08a18 in util::RunnableAdapter<void, void>::operator()() ()
#6  0x0f989090 in util::Thread::_ThreadMain(void*) ()
#7  0x0fdf7cf0 in start_thread (arg=0x48f3e290) at pthread_create.c:310
#8  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 15 (Thread 0x498ff290 (LWP 3106)):
#0  0x0ff1f87c in select () from ./usr/st/lib/libc.so.6
#1  0x0f4ad5e0 in do_select (t=0x498fe7e8, e=0x0, w=0x0, r=0x498fe768,
#2  omni::SocketCollection::Select (this=this@entry=0x15737568)
#3  0x0f1af380 in omni::sslEndpoint::AcceptAndMonitor (this=0x15737560,
#4  0x0f4910bc in omni::giopRendezvouser::execute (this=0x15726410)
#5  0x0f43ed64 in real_run (this=0x15737210)
#6  omniAsyncWorkerInfo::run (this=this@entry=0x498fe9a8)
#7  0x0f43fb10 in omniAsyncWorker::run (this=<optimized out>)
#8  0x0f37ab1c in omni_thread_wrapper (ptr=0x15737210)
#9  0x0fdf7cf0 in start_thread (arg=0x498ff290) at pthread_create.c:310
#10 0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 14 (Thread 0x4a0ff290 (LWP 3107)):
#0  0x0ff1f87c in select () from ./usr/st/lib/libc.so.6
#1  0x0f4ad5e0 in do_select (t=0x4a0fe8a8, e=0x0, w=0x0, r=0x4a0fe828,
#2  omni::SocketCollection::Select (this=this@entry=0x15738750)
#3  0x0f4cff8c in omni::tcpEndpoint::AcceptAndMonitor (this=0x15738748,
#4  0x0f4910bc in omni::giopRendezvouser::execute (this=0x15737360)
#5  0x0f43ed64 in real_run (this=0x15737388)
#6  omniAsyncWorkerInfo::run (this=this@entry=0x4a0fe9a8)
#7  0x0f43fb10 in omniAsyncWorker::run (this=<optimized out>)
#8  0x0f37ab1c in omni_thread_wrapper (ptr=0x15737388)
#9  0x0fdf7cf0 in start_thread (arg=0x4a0ff290) at pthread_create.c:310
#10 0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 13 (Thread 0x4a8ff290 (LWP 3108)):
#0  0x0fdfcdbc in __pthread_cond_timedwait (cond=0x15727570, mutex=0x15727548,
#1  0x0f37a540 in omni_condition::timedwait (this=0x15727568,
#2  0x0f4881a0 in omni::Scavenger::execute (this=0x15738c20)
#3  0x0f43ed64 in real_run (this=0x15739240)
#4  omniAsyncWorkerInfo::run (this=this@entry=0x4a8fe9a8)
#5  0x0f43fb10 in omniAsyncWorker::run (this=<optimized out>)
#6  0x0f37ab1c in omni_thread_wrapper (ptr=0x15739240)
#7  0x0fdf7cf0 in start_thread (arg=0x4a8ff290) at pthread_create.c:310
#8  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 12 (Thread 0x48f7d290 (LWP 3110)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x1574d570, mutex=0x1574d514)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x0f89a048 in MessageQueueOperator::operator()() ()
#4  0x0f989090 in util::Thread::_ThreadMain(void*) ()
#5  0x0fdf7cf0 in start_thread (arg=0x48f7d290) at pthread_create.c:310
#6  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 11 (Thread 0x48fbc290 (LWP 3111)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x1574d6d0, mutex=0x1574d674)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x0f897724 in ?? () from ./usr/st/lib/libcore.so
#4  0x0f989090 in util::Thread::_ThreadMain(void*) ()
#5  0x0fdf7cf0 in start_thread (arg=0x48fbc290) at pthread_create.c:310
#6  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 10 (Thread 0x4c5ff290 (LWP 3262)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x15102b50, mutex=0x15102af0)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x11bc86dc in cml::RpcImpl::LockFromServer(char const*) ()
#4  0x129cdfec in ?? ()
#5  0x0f475fec in doLocalCall (servant=0xbffff330, this=0x4c5fe728)
#6  omniCallHandle::upcall (this=0x4c5fe828, servant=0xbffff330, desc=...)
#7  0x129dd7a8 in cml::_impl_Rpc::_dispatch(omniCallHandle&) ()
#8  0x0f466ca0 in omni::omniOrbPOA::dispatch (this=<optimized out>,
#9  0x0f449758 in omniLocalIdentity::dispatch (this=0x16274e90, handle=...)
#10 0x0f492ed0 in omni::GIOP_S::handleRequest (this=this@entry=0x4ad00d38)
#11 0x0f492c0c in omni::GIOP_S::dispatcher (this=0x4ad00d38)
#12 0x0f4908d0 in omni::giopWorker::real_execute (this=0x4ad00bd0)
#13 0x0f49081c in omni::giopWorker::execute (this=<optimized out>)
#14 0x0f43ed64 in real_run (this=0x4ad00bf8)
#15 omniAsyncWorkerInfo::run (this=this@entry=0x4c5fe9a8)
#16 0x0f43fb10 in omniAsyncWorker::run (this=<optimized out>)
#17 0x0f37ab1c in omni_thread_wrapper (ptr=0x4ad00bf8)
#18 0x0fdf7cf0 in start_thread (arg=0x4c5ff290) at pthread_create.c:310
#19 0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 9 (Thread 0x4cdff290 (LWP 3354)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x4a9197e0, mutex=0x4a919788)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x11b93ce8 in cml::ExternalEventBase::Data::Wait() ()
#4  0x11b945c4 in cml::ExternalEventBase::Data::GetTableEntryList() ()
#5  0x11c66ab4 in chassis::SwapImpl::SwapNpbSliceGroupIn(type::NpbSliceGroupId)
#6  0x135e4ce4 in ?? ()
#7  0x0f475fec in doLocalCall (servant=0x15759984, this=0x4cdfe738)
#8  omniCallHandle::upcall (this=0x4cdfe828, servant=0x15759984, desc=...)
#9  0x135e77b8 in chassis::_impl_Swap::_dispatch(omniCallHandle&) ()
#10 0x0f466ca0 in omni::omniOrbPOA::dispatch (this=<optimized out>,
#11 0x0f449758 in omniLocalIdentity::dispatch (this=0x157557b8, handle=...)
#12 0x0f492ed0 in omni::GIOP_S::handleRequest (this=this@entry=0x4a902580)
#13 0x0f492c0c in omni::GIOP_S::dispatcher (this=0x4a902580)
#14 0x0f4908d0 in omni::giopWorker::real_execute (this=0x4ad05070)
#15 0x0f49081c in omni::giopWorker::execute (this=<optimized out>)
#16 0x0f43ed64 in real_run (this=0x4ad05098)
#17 omniAsyncWorkerInfo::run (this=this@entry=0x4cdfe9a8)
#18 0x0f43fb10 in omniAsyncWorker::run (this=<optimized out>)
#19 0x0f37ab1c in omni_thread_wrapper (ptr=0x4ad05098)
#20 0x0fdf7cf0 in start_thread (arg=0x4cdff290) at pthread_create.c:310
#21 0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 8 (Thread 0x4aa3e290 (LWP 3414)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x15755f08, mutex=0x15755f38)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x0f963650 in util::ConditionMutex::Lock() () from ./usr/st/lib/libcore.so
#4  0x11b76050 in cml::Configuration::SwapinHandler() ()
#5  0x11bcc0a0 in cml::Swapin::operator()() ()
#6  0x0f989090 in util::Thread::_ThreadMain(void*) ()
#7  0x0fdf7cf0 in start_thread (arg=0x4aa3e290) at pthread_create.c:310
#8  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 7 (Thread 0x48ffb290 (LWP 3468)):
#0  0x0fdfcdbc in __pthread_cond_timedwait (cond=0x48e076c8, mutex=0x48e07798,
#1  0x0f974cc4 in util::SimpleMutex::TimedWait(pthread_cond_t*, timespec const&) () from ./usr/st/lib/libcore.so
#2  0x0f963a30 in util::ConditionVariable::WaitWithTimeout(util::SimpleMutex*, timespec const&) () from ./usr/st/lib/libcore.so
#3  0x0f963a84 in util::ConditionVariable::WaitWithTimeout(util::SimpleMutex*, long) () from ./usr/st/lib/libcore.so
#4  0x1154e030 in lps::LPSCluster::operator()() ()
#5  0x0f989090 in util::Thread::_ThreadMain(void*) ()
#6  0x0fdf7cf0 in start_thread (arg=0x48ffb290) at pthread_create.c:310
#7  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 6 (Thread 0x4e0ff290 (LWP 3536)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x4d87cf58, mutex=0x4d87cf00)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x11b93ce8 in cml::ExternalEventBase::Data::Wait() ()
#4  0x11b945c4 in cml::ExternalEventBase::Data::GetTableEntryList() ()
#5  0x11c625f0 in chassis::SwapImpl::SwapIn(type::PhySlotId, type::PhyCardId, chassis::SliceId, chassis::PortList const&) ()
#6  0x135e4ae8 in ?? ()
#7  0x0f475fec in doLocalCall (servant=0x15759984, this=0x4e0fe738)
#8  omniCallHandle::upcall (this=0x4e0fe828, servant=0x15759984, desc=...)
#9  0x135e75e8 in chassis::_impl_Swap::_dispatch(omniCallHandle&) ()
#10 0x0f466ca0 in omni::omniOrbPOA::dispatch (this=<optimized out>,
#11 0x0f449758 in omniLocalIdentity::dispatch (this=0x157557b8, handle=...)
#12 0x0f492ed0 in omni::GIOP_S::handleRequest (this=this@entry=0x4d84da40)
#13 0x0f492c0c in omni::GIOP_S::dispatcher (this=0x4d84da40)
#14 0x0f4908d0 in omni::giopWorker::real_execute (this=0x4ce8c070)
#15 0x0f49081c in omni::giopWorker::execute (this=<optimized out>)
#16 0x0f43ed64 in real_run (this=0x4ce51c00)
#17 omniAsyncWorkerInfo::run (this=this@entry=0x4e0fe9a8)
#18 0x0f43fb10 in omniAsyncWorker::run (this=<optimized out>)
#19 0x0f37ab1c in omni_thread_wrapper (ptr=0x4ce51c00)
#20 0x0fdf7cf0 in start_thread (arg=0x4e0ff290) at pthread_create.c:310
#21 0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 5 (Thread 0x4e8ff290 (LWP 3604)):
#0  0x0fe00b98 in recv () from ./usr/st/lib/libpthread.so.0
#1  0x0f4cebd0 in omni::tcpConnection::Recv (this=0x4ce53fd8, buf=0x4ce556a8,
#2  0x0f48b05c in omni::giopStream::inputMessage (this=this@entry=0x4ce52484)
#3  0x0f4a3b58 in omni::giopImpl12::inputNewServerMessage (
#4  0x0f4a40b4 in inputMessageBegin (
#5  omni::giopImpl12::inputMessageBegin (g=0x4ce52484,
#6  0x0f492ad8 in omni::GIOP_S::dispatcher (this=0x4ce52480)
#7  0x0f4908d0 in omni::giopWorker::real_execute (this=0x4ce859f0)
#8  0x0f49081c in omni::giopWorker::execute (this=<optimized out>)
#9  0x0f43ed64 in real_run (this=0x4ce51fd0)
#10 omniAsyncWorkerInfo::run (this=this@entry=0x4e8fe9a8)
#11 0x0f43fb10 in omniAsyncWorker::run (this=<optimized out>)
#12 0x0f37ab1c in omni_thread_wrapper (ptr=0x4ce51fd0)
#13 0x0fdf7cf0 in start_thread (arg=0x4e8ff290) at pthread_create.c:310
#14 0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 4 (Thread 0x522ff290 (LWP 3721)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x15755f08, mutex=0x15755f38)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x0f963650 in util::ConditionMutex::Lock() () from ./usr/st/lib/libcore.so
#4  0x1219b8a4 in lps::CmlSession::Lock() ()
#5  0x1219d128 in lps::CmlSession::CmlSession(lps::CmlSession::SessionType, std::string const&) ()
#6  0x127c2ea8 in lps::PhyCard::PhyCard(std::string const&, lpsdevice::Card::Slot, mgmt::Type, type::PhyCardId, type::ShelfId) ()
#7  0x0fcd92c0 in lps::Sr9624CardFactory::CreateCard(std::string const&, lpsdevice::Card::Type, lpsdevice::Card::Slot, mgmt::Type, type::PhyCardId, type::ShelfId) () from ./usr/st/lib/libdevice-cfgd.so
#8  0x127b3cdc in lps::DeviceManager::HandleCardEvent(std::string const&, mgmt::Type, mgmt::Action, bool, bool, bool) ()
#9  0x127b546c in lps::DeviceManager::AssignSlotInCache(_CORBA_ObjRef_Var<laurel::_objref_SlotIdl, laurel::SlotIdl_Helper>, bool) ()
#10 0x127b73fc in lps::DeviceManager::UpdateCardPortCache() ()
#11 0x127b7d40 in lps::DeviceManager::CheckCacheDirty() ()
#12 0x127b9620 in lps::DeviceManager::FindCard(lpsdevice::Card::Slot, type::ShelfId) ()
#13 0x1284157c in lpsutil::Util::GetCardId(type::PhySlotId, type::PhyCardId&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&)
#14 0x12843560 in lpsutil::Util::GetCardIdSlotIdPortNum(std::string const&, type::PhySlotId*, type::PhyCardId*, unsigned int*) ()
#15 0x125fe884 in cml::PortVerifier* lps::PortUtil::GetTableEntry<cml::PortVerifier>(std::string const&, unsigned int, cml::Error&) ()
#16 0x123cff74 in lps::OmsSubInterfaceImpl::GetConfigState(std::string const&, lpsdevice::OmsSubInterface::ConfigState&, bool) ()
#17 0x12315aa4 in lps::PortManagerImpl::discoverCommonSubIfs(std::string&, unsigned long, unsigned long&, std::string&) ()
#18 0x12318664 in lps::PortManagerImpl::discoverFirstNSubIfs(unsigned long) ()
#19 0x12c86314 in ?? ()
#20 0x0f475fec in doLocalCall (servant=0x4d7e3c34, this=0x522fe658)
#21 omniCallHandle::upcall (this=0x522fe828, servant=0x4d7e3c34, desc=...)
#22 0x12c87220 in lpsdevice::_impl_PortManager::_dispatch(omniCallHandle&) ()
#23 0x0f466ca0 in omni::omniOrbPOA::dispatch (this=<optimized out>,
#24 0x0f449758 in omniLocalIdentity::dispatch (this=this@entry=0x522fe758,
#25 0x0f46c868 in omni::omniOrbPOA::dispatch_to_sl (
#26 0x0f466fc8 in omni::omniOrbPOA::dispatch (this=0x490086b8, handle=...,
#27 0x0f492f58 in omni::GIOP_S::handleRequest (this=this@entry=0x16022470)
#28 0x0f492c0c in omni::GIOP_S::dispatcher (this=0x16022470)
#29 0x0f4908d0 in omni::giopWorker::real_execute (this=0x4a9192d8)
#30 0x0f49081c in omni::giopWorker::execute (this=<optimized out>)
#31 0x0f43ed64 in real_run (this=0x4a919300)
#32 omniAsyncWorkerInfo::run (this=this@entry=0x522fe9a8)
#33 0x0f43fb10 in omniAsyncWorker::run (this=<optimized out>)
#34 0x0f37ab1c in omni_thread_wrapper (ptr=0x4a919300)
#35 0x0fdf7cf0 in start_thread (arg=0x522ff290) at pthread_create.c:310
#36 0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 3 (Thread 0x4aa7d290 (LWP 3730)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x15755f08, mutex=0x15755f38)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x0f963650 in util::ConditionMutex::Lock() () from ./usr/st/lib/libcore.so
#4  0x11bc78e8 in ?? ()
#5  0x0f98cfa4 in util::ThreadPool::_ThreadMain(void*) ()
#6  0x0f988e10 in ?? () from ./usr/st/lib/libcore.so
#7  0x0f989090 in util::Thread::_ThreadMain(void*) ()
#8  0x0fdf7cf0 in start_thread (arg=0x4aa7d290) at pthread_create.c:310
#9  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 2 (Thread 0x4aafb290 (LWP 3793)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0x15f71f10, mutex=0x15f71eb8)
#1  0x0f975280 in util::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f963be0 in util::ConditionVariable::Wait(util::SimpleMutex*) ()
#3  0x119ab6b4 in lpseventmgr::Observer<lpseventmgr::SubscriberKey, lpsdevice::EventSubscriber, lpsdevice::EventSubscriber_Helper, util::RefPtr<lpseventmgr::Event<lpsdevice::LpsMessageAtom, _CORBA_ConstrType_Variable_Var<lpsdevice::LpsMessageAtom> > >, lpseventmgr::Event<lpsdevice::LpsMessageAtom, _CORBA_ConstrType_Variable_Var<lpsdevice::LpsMessageAtom> > >::operator()() ()
#4  0x0f989090 in util::Thread::_ThreadMain(void*) ()
#5  0x0fdf7cf0 in start_thread (arg=0x4aafb290) at pthread_create.c:310
#6  0x0ff275fc in clone () from ./usr/st/lib/libc.so.6
Thread 1 (Thread 0x4809fc60 (LWP 2566)):
#0  0x0fdfc978 in __pthread_cond_wait (cond=0xbffff2a8, mutex=0xbffff2d8)
#1  0x0f9c8130 in thread::SimpleMutex::Wait(pthread_cond_t*) ()
#2  0x0f9c84b4 in thread::Thread::Block() () from ./usr/st/lib/libcore.so
#3  0x0f9893a4 in util::Thread::Block() () from ./usr/st/lib/libcore.so
#4  0x110d54b8 in main ()

Your inputs will be of great help.
Thanks in advance.
0
Comment
Question by:Shirya J
  • 2
2 Comments
 
LVL 34

Accepted Solution

by:
Duncan Roe earned 500 total points (awarded by participants)
ID: 41873264
All threads are waiting for something to happen. Some threads will additionally wake on a timeout. __pthread_cond_wait waits for a condition. __pthread_cond_timedwait waits for a condition or a timeout, whichever happens first. recv (thread 5) is waiting for data. select (threads 14 & 15) waits for an i/o operation to complete or optionally for a timeout (I can't tell from this trace if a timeout is specified).
IN SUMMARY: the trace looks to be entirely normal. There may be a logic error which causes a deadlock, but a stack trace will not show you that.
What software is process 2566 by the way?
That's a nice-looking version of pstack you have - where did you get it please? (URL if possible)
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 41898869
The author has not seen fit to respond so we must assume he is happy with the response. If not, he has 4 days to object.
I wish he would have told me where he got that pstack from, or even just the Linux distribution it came with.
I combed the net and couldn't find one that worked: pstack-gdb comes close but mysteriously fails for me if I install the ruby extension.
As a result, I wrote my own pstack. Look out for it soon at github/duncan-roe. It's an expect script that runs gdb, and works at least on Linux and Cygwin, 32 or 64 bit.
0

Featured Post

Complete Microsoft Windows PC® & Mac Backup

Backup and recovery solutions to protect all your PCs & Mac– on-premises or in remote locations. Acronis backs up entire PC or Mac with patented reliable disk imaging technology and you will be able to restore workstations to a new, dissimilar hardware in minutes.

Join & Write a Comment

Daily system administration tasks often require administrators to connect remote systems. But allowing these remote systems to accept passwords makes these systems vulnerable to the risk of brute-force password guessing attacks. Furthermore there ar…
Over the last ten+ years I have seen Linux configuration tools come and go. In the early days there was the tried-and-true, all-powerful linuxconf that many thought would remain the one and only Linux configuration tool until the end of times. Well,…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now