Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

WebSphere Application Cannot re-establish connection after DB restart

Posted on 2009-04-29
16
Medium Priority
?
11,023 Views
Last Modified: 2013-12-14
Hi,

I have applications running on WAS 6.0.2.31cluster which has been configured with Oracle RAC 10G.

Currently there was a problem with the Oracle 10g nodes and both went down, Once the Db is up WAS applications and SIBus couldnt re-establish the connetion, to fix this we were forced to re-start the Applicaiton servers.

Is there any fix/workaround to re-establish connection once the DB is back online.

Any suggestions/help is really appreciated.

Many Thanks
Dipin
0
Comment
Question by:dipinci
  • 8
  • 8
16 Comments
 
LVL 41

Expert Comment

by:HonorGod
ID: 24274125
Have you had this situation before, or since?

Did you try stopping and restarting the application?

Did you try "testing" the connection?
0
 
LVL 4

Author Comment

by:dipinci
ID: 24274282
Ys I had this situation couple of times, I have always tried to restart the applications. Test connection  from Admin console is succeful once the DB is online.
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 24274331
But did you try stopping and starting the application (not the AppServer) to clear up the issue?

Or, did you try testing the connection from the datasource configuration panel when the issue occurred?
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 
LVL 4

Author Comment

by:dipinci
ID: 24274374
Restarted the application Server because I have SIBus associated with the Application Server.  Tested the datasource connection from the admin console
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 24274699
What version of WebSphere is being used, and on what Operating System?
0
 
LVL 4

Author Comment

by:dipinci
ID: 24283722
WAS 6.0.2.31 and OS AIX 5.3. WAS version is mentioned in the question.
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 24284400
Sorry about that.  You are correct.  I'm just in the habit of asking for the version.  I apologize.

Anyway, do you have verbose logs associated with the attempts to re-establish connection to the DB after they all get recycled?
0
 
LVL 4

Author Comment

by:dipinci
ID: 24288790
Yes, we have. I have a doubt how this Verbose GC related with this problem.
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 24289828
Verbose logs are not verbose GC logs.  Verbose just means "lots".  The default log level for WebSphere is "informational messages only", it would appear that we need more "verbose" messages, so that we can tell What WebSphere is trying to do, and what happens through out the process.

Does this make sense?
0
 
LVL 4

Author Comment

by:dipinci
ID: 24290480
I know that default log level is info, but there is no verbose log level. It will be trace fatal,trace,error....
I have the default logging enabled
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 24290587
by verbose, I meant "more than the default".  That's all.

What messages do you get when you encounter the problem?
0
 
LVL 4

Author Comment

by:dipinci
ID: 24292739
Below message logged

Server-1
4/27/09 23:11:01:268 AST] 0000aa8f ExceptionInte E com.qatarairways.ibe.platform.interceptor.ExceptionInterceptor afterThrowing MESSEGE ERRORKEY=  module.connection.prob   ,DATABASE ERROR CODE :17002
                                 org.springframework.dao.DataAccessResourceFailureException: Hibernate operation: Cannot open connection; SQL [???]; Io exception: The Network Adapter could not establish the connectionDSRA0010E: SQL State = null, Error Code = 17,002DSRA0010E: SQL State = null, Error Code = 17,002; nested exception is java.sql.SQLException: Io exception: The Network Adapter could not establish the connectionDSRA0010E: SQL State = null, Error Code = 17,002DSRA0010E: SQL State = null, Error Code = 17,002
java.sql.SQLException: Io exception: The Network Adapter could not establish the connectionDSRA0010E: SQL State = null, Error Code = 17,002DSRA0010E: SQL State = null, Error Code = 17,002


Server-2
[4/27/09 23:11:14:794 AST] 00017f18 ConnectionEve A   J2CA0056I: The Connection Manager received a fatal connection error from the Resource Adaptor for resource com.qatarairways.ibe.datasource.  The exception which was received is com.ibm.websphere.ce.cm.StaleConnectionException: Io exception: Connection reset
[4/27/09 23:11:14:883 AST] 00017f18 AbstractBatch W org.hibernate.jdbc.AbstractBatcher closeQueryStatement exception clearing maxRows/queryTimeout
                                 com.ibm.websphere.ce.cm.ObjectClosedException: DSRA9110E: Statement is closed.
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 24294088
The Server-1 information points to the application exception, but the Server-2 is a little more interesting.

The following document from the IBM website has proven very helpful in identifying, and resolving "Stale Connection" issues.

http://www.IBM.com/support/docview.wss?rs=180&uid=swg21063645

It also points to a white paper that is good.
0
 
LVL 4

Author Comment

by:dipinci
ID: 24294131
The purge policy set for the Connection pool is for entire pool, not sure about the code level change.

I am trying to find out some configuration/workaround to retry the connection from the Application Server.
0
 
LVL 41

Expert Comment

by:HonorGod
ID: 24294167
ok, please let me know what I can do to help.
0
 
LVL 4

Accepted Solution

by:
dipinci earned 0 total points
ID: 24405323
I could find some fix the problem by configuring preTest string and  custom property.
This question can be closed.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In our object-oriented world the class is a minimal unit, a brick for constructing our applications. It is an abstraction and we know well how to use it. In well-designed software we are not usually interested in knowing how objects look in memory. …
Verbose logging is used to diagnose garbage collector problems. By default, -verbose:gc output is written to either native_stderr.log or native_stdout.log.   It is also possible to redirect the logs to a user-specified file. This article will de…
THe viewer will learn how to use NetBeans IDE 8.0 for Windows to perform CRUD operations on a MySql database.
The viewer will learn how to synchronize PHP projects with a remote server in NetBeans IDE 8.0 for Windows.
Suggested Courses

578 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question