Solved

Tomcat Hangs twice or three times a day

Posted on 2013-05-27
4
1,094 Views
Last Modified: 2013-07-03
Hi,

I have a Tomcat Server with some applications who asks a Mysql Database, and twice or three times a day It hangs. I need to stop and start it to bring it up online again. I've seen these errors:

ADVERTENCIA: Connection has been abandoned PooledConnection[com.mysql.jdbc.JDBC4Connection@9168b4]:java.lang.Exception
      at org.apache.tomcat.jdbc.pool.ConnectionPool.getThreadDump

...

com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
...
java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space
...

Here is the configuration of the Two pools in our Application:

  <Resource name="jdbc/PconDB"
  auth="Container"
  type="javax.sql.DataSource"
  factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
  testWhileIdle="true"
            testOnBorrow="true"
            testOnReturn="false"
            validationQuery="SELECT 1"
            validationInterval="30000"
            timeBetweenEvictionRunsMillis="30000"
            maxActive="300"
            minIdle="30"
            maxWait="10000"
            initialSize="30"
            removeAbandonedTimeout="60"
            removeAbandoned="true"
            logAbandoned="true"
            minEvictableIdleTimeMillis="30000"
            jmxEnabled="true"
            username="system"
            password="----"
            driverClassName="com.mysql.jdbc.Driver"
            url="jdbc:mysql://172.16.2.179:3306/pcondb?autoReconnect=true"/>              


  <Resource name="jdbc/EBS"
  auth="Container"
  type="javax.sql.DataSource"
  factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
  testWhileIdle="true"
            testOnBorrow="true"
            testOnReturn="false"
            validationQuery="SELECT 1 FROM DUAL"
            validationInterval="30000"
            timeBetweenEvictionRunsMillis="30000"
            maxActive="100"
            minIdle="20"
            maxWait="10000"
            initialSize="20"
            removeAbandonedTimeout="60"
            removeAbandoned="true"
            logAbandoned="true"
            minEvictableIdleTimeMillis="30000"
            jmxEnabled="true"
            username="apps"
            password="---"
            driverClassName="oracle.jdbc.driver.OracleDriver"
            url="jdbc:oracle:thin:@//172.16.2.249:1521/prod"/>  



We are usint a VM CEntOS 5.8 x64 on Esxi, and Tomcat 7.0.35 with jre1_6_0_27.

Can you give me an advice about what I need to do to get rid of these errors, and avoid to restart the Tomcat Server every 4-5 hours  because it hangs?

Thanks in Advance.

Josep
catalina.out.txt.zip
0
Comment
Question by:josepvallejo
  • 3
4 Comments
 
LVL 35

Accepted Solution

by:
mccarl earned 500 total points
ID: 39200143
You would more likely need to look more at the actual applications rather than anything Tomcat specific. Did you develop these applications yourself? Or are they third party apps?
0
 

Author Comment

by:josepvallejo
ID: 39200565
Hi mccarl,

Yes, These applications are internal and developped by a programmer. He is trying to solve some problems and errors related to queries, but there still aren't any improvements, so we were thinking about checking some parameteres to stabilize the system while the Programmer is solving Application errors.

The mainly thing is that this Web Server ins in production and is getting worse.

I don't know if we could do something to make it run a little bit better and not dying twice o three times a day.

Thanks in Advance for your answer.
Josep.
0
 

Author Comment

by:josepvallejo
ID: 39221620
I have stabilized the problem putting the parameter "RemoveAbandonedTimeout="6000" in both pools and modifying the script to reboot Tomcat every day at 3:00AM (it wasn't working before).

I've put this parameter in Tomcat options: -XX:-UseGCOverheadLimit

Please, close the request .

Josep.
0
 

Author Closing Comment

by:josepvallejo
ID: 39296219
Finally as mccarl said, it was a leaking problem due to a bad application design. It was nearly impossible to stabilize the system adjusting parameters and restarting it periodically. Finally the programmer  saw some problems about not closing sessions with mysql.close and not  catching exceptions properly with try-catch.
0

Featured Post

Building an interactive eFuture classroom

Watch and learn how ATEN provided a total control system solution including seamless switching matrix switch, HDBaseT extenders, PDU, lighting control to build an interactive eFuture classroom.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
move expression web site to a new server 13 77
Formating field inside mysql query 2 33
show child records separated by commas 12 38
Not listening to where 1 23
One of the typical problems I have experienced is when you have to move a web server from one hosting site to another. You normally prepare all on the new host, transfer the site, change DNS and cross your fingers hoping all will be ok on new server…
Meet the world's only “Transparent Cloud™” from Superb Internet Corporation. Now, you can experience firsthand a cloud platform that consistently outperforms Amazon Web Services (AWS), IBM’s Softlayer, and Microsoft’s Azure when it comes to CPU and …
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…
Are you ready to implement Active Directory best practices without reading 300+ pages? You're in luck. In this webinar hosted by Skyport Systems, you gain insight into Microsoft's latest comprehensive guide, with tips on the best and easiest way…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question