Solved

Stuck thread in application running in Welogic 10.3.5

Posted on 2014-09-18
13
428 Views
Last Modified: 2014-10-19
We have encountered Stuck thread in application running in Welogic 10.3.5, below is the cause found in the logs:
Caused By: java.net.SocketException: Broken pipe
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
        at oracle.net.ns.DataPacket.send(DataPacket.java:200)
        at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:230)
        at oracle.net.ano.AnoComm.b(Unknown Source)
        at oracle.net.ano.Ano.negotiation(Unknown Source)
        at oracle.net.ns.NSProtocol.connect(NSProtocol.java:438)
        at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1056)
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:308)
        at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:538)
        at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:228)
        at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
        at weblogic.jdbc.common.internal.ConnectionEnvFactory.makeConnection(ConnectionEnvFactory.java:350)
        at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:236)
        at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1249)
        at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1166)
        at weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:450)
        at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:342)
        at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:419)
        at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:324)
        at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:94)
        at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:63)
        at weblogic.jdbc.jts.Driver.newConnection(Driver.java:915)
        at weblogic.jdbc.jts.Driver.createLocalConnection(Driver.java:327)
        at weblogic.jdbc.jts.Driver.connect(Driver.java:173)
        at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:364)
        at com.fortressit.ejb.EJB11.EJB11Context.getConnection(EJB11Context.java:144)
        at com.sns.base.util.DBConnection.getConnection(DBConnection.java:139)
0
Comment
Question by:SANJAY HALDAR
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 6
13 Comments
 
LVL 62

Expert Comment

by:gheist
ID: 40331961
Database connection disconnected mid transaction, and exception was not handled...  
What makes you think there is a stuck thread? Was it normal before?
0
 

Author Comment

by:SANJAY HALDAR
ID: 40336621
Thanks for the response.
Stuck thread noticed in one of our instance and we have seen the above mentioned traces in the logs. This API functions normally, is surge in the incoming request can trigger the broken pipe. What could the reason for the "Database connection disconnected mid transaction"
0
 
LVL 62

Expert Comment

by:gheist
ID: 40336642
Any firewall around database? Like windows or iptables?
0
Get MongoDB database support online, now!

At Percona’s web store you can order your MongoDB database support needs in minutes. No hassles, no fuss, just pick and click. Pay online with a credit card. Handle your MongoDB database support now!

 

Author Comment

by:SANJAY HALDAR
ID: 40338339
There is generic firewall for the network as always.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40338378
So that kills the connections, enjoy
0
 

Author Comment

by:SANJAY HALDAR
ID: 40343427
you mean to say the firewall kills the connection, if yes, I have the following queries:

# If firewall is killing the  connections, then we should encounter the issue always.
# What is the background of proving that  firewall is killing the  connection, so that we can take up to the infra level.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40344384
# it is not killing connection. it drops connection state and no longer forwards packets
# experience.
0
 

Author Comment

by:SANJAY HALDAR
ID: 40345491
So, the deficiency which we have in the code is that the exception wasn't handled.
BTW, can it be ensured that firewall doesn't drop the connection state, any measure?
0
 
LVL 62

Expert Comment

by:gheist
ID: 40345608
Dont use long idle connections.
0
 

Author Comment

by:SANJAY HALDAR
ID: 40352225
Would it be possible for you to demonstrate a sample piece of code which ensures that the connections are not hold idle OR to avoid using the idle connection
0
 
LVL 62

Expert Comment

by:gheist
ID: 40352250
socket was broken... What else you need.
0
 

Author Comment

by:SANJAY HALDAR
ID: 40361485
Was your last comment relevant to my previous query
0
 
LVL 62

Accepted Solution

by:
gheist earned 500 total points
ID: 40361577
Stack trace comes from:
at com.fortressit.ejb.EJB11.EJB11Context.getConnection(EJB11Context.java:144)
        at com.sns.base.util.DBConnection.getConnection(DBConnection.java:139)


so check that line and handle the exception when socket times out.
or close them before firewall does
or make some query on schedule
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Verbose logging is used to diagnose garbage collector problems. By default, -verbose:gc output is written to either native_stderr.log or native_stdout.log.   It is also possible to redirect the logs to a user-specified file. This article will de…
Preface There are many applications where some computing systems need have their system clocks running synchronized within a small margin and eventually need to be in sync with the global time. There are different solutions for this, i.e. the W3…
Michael from AdRem Software outlines event notifications and Automatic Corrective Actions in network monitoring. Automatic Corrective Actions are scripts, which can automatically run upon discovery of a certain undesirable condition in your network.…
Visualize your data even better in Access queries. Given a date and a value, this lesson shows how to compare that value with the previous value, calculate the difference, and display a circle if the value is the same, an up triangle if it increased…
Suggested Courses

632 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question