• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 11453
  • Last Modified:

WebSphere Application Cannot re-establish connection after DB restart

Hi,

I have applications running on WAS 6.0.2.31cluster which has been configured with Oracle RAC 10G.

Currently there was a problem with the Oracle 10g nodes and both went down, Once the Db is up WAS applications and SIBus couldnt re-establish the connetion, to fix this we were forced to re-start the Applicaiton servers.

Is there any fix/workaround to re-establish connection once the DB is back online.

Any suggestions/help is really appreciated.

Many Thanks
Dipin
0
dipinci
Asked:
dipinci
  • 8
  • 8
1 Solution
 
HonorGodSoftware EngineerCommented:
Have you had this situation before, or since?

Did you try stopping and restarting the application?

Did you try "testing" the connection?
0
 
dipinciAuthor Commented:
Ys I had this situation couple of times, I have always tried to restart the applications. Test connection  from Admin console is succeful once the DB is online.
0
 
HonorGodSoftware EngineerCommented:
But did you try stopping and starting the application (not the AppServer) to clear up the issue?

Or, did you try testing the connection from the datasource configuration panel when the issue occurred?
0
Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

 
dipinciAuthor Commented:
Restarted the application Server because I have SIBus associated with the Application Server.  Tested the datasource connection from the admin console
0
 
HonorGodSoftware EngineerCommented:
What version of WebSphere is being used, and on what Operating System?
0
 
dipinciAuthor Commented:
WAS 6.0.2.31 and OS AIX 5.3. WAS version is mentioned in the question.
0
 
HonorGodSoftware EngineerCommented:
Sorry about that.  You are correct.  I'm just in the habit of asking for the version.  I apologize.

Anyway, do you have verbose logs associated with the attempts to re-establish connection to the DB after they all get recycled?
0
 
dipinciAuthor Commented:
Yes, we have. I have a doubt how this Verbose GC related with this problem.
0
 
HonorGodSoftware EngineerCommented:
Verbose logs are not verbose GC logs.  Verbose just means "lots".  The default log level for WebSphere is "informational messages only", it would appear that we need more "verbose" messages, so that we can tell What WebSphere is trying to do, and what happens through out the process.

Does this make sense?
0
 
dipinciAuthor Commented:
I know that default log level is info, but there is no verbose log level. It will be trace fatal,trace,error....
I have the default logging enabled
0
 
HonorGodSoftware EngineerCommented:
by verbose, I meant "more than the default".  That's all.

What messages do you get when you encounter the problem?
0
 
dipinciAuthor Commented:
Below message logged

Server-1
4/27/09 23:11:01:268 AST] 0000aa8f ExceptionInte E com.qatarairways.ibe.platform.interceptor.ExceptionInterceptor afterThrowing MESSEGE ERRORKEY=  module.connection.prob   ,DATABASE ERROR CODE :17002
                                 org.springframework.dao.DataAccessResourceFailureException: Hibernate operation: Cannot open connection; SQL [???]; Io exception: The Network Adapter could not establish the connectionDSRA0010E: SQL State = null, Error Code = 17,002DSRA0010E: SQL State = null, Error Code = 17,002; nested exception is java.sql.SQLException: Io exception: The Network Adapter could not establish the connectionDSRA0010E: SQL State = null, Error Code = 17,002DSRA0010E: SQL State = null, Error Code = 17,002
java.sql.SQLException: Io exception: The Network Adapter could not establish the connectionDSRA0010E: SQL State = null, Error Code = 17,002DSRA0010E: SQL State = null, Error Code = 17,002


Server-2
[4/27/09 23:11:14:794 AST] 00017f18 ConnectionEve A   J2CA0056I: The Connection Manager received a fatal connection error from the Resource Adaptor for resource com.qatarairways.ibe.datasource.  The exception which was received is com.ibm.websphere.ce.cm.StaleConnectionException: Io exception: Connection reset
[4/27/09 23:11:14:883 AST] 00017f18 AbstractBatch W org.hibernate.jdbc.AbstractBatcher closeQueryStatement exception clearing maxRows/queryTimeout
                                 com.ibm.websphere.ce.cm.ObjectClosedException: DSRA9110E: Statement is closed.
0
 
HonorGodSoftware EngineerCommented:
The Server-1 information points to the application exception, but the Server-2 is a little more interesting.

The following document from the IBM website has proven very helpful in identifying, and resolving "Stale Connection" issues.

http://www.IBM.com/support/docview.wss?rs=180&uid=swg21063645

It also points to a white paper that is good.
0
 
dipinciAuthor Commented:
The purge policy set for the Connection pool is for entire pool, not sure about the code level change.

I am trying to find out some configuration/workaround to retry the connection from the Application Server.
0
 
HonorGodSoftware EngineerCommented:
ok, please let me know what I can do to help.
0
 
dipinciAuthor Commented:
I could find some fix the problem by configuring preTest string and  custom property.
This question can be closed.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: MCSA MCSE Windows Server 2012

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

  • 8
  • 8
Tackle projects and never again get stuck behind a technical roadblock.
Join Now