?
Solved

Tomcat Hangs twice or three times a day

Posted on 2013-05-27
4
Medium Priority
?
1,127 Views
Last Modified: 2013-07-03
Hi,

I have a Tomcat Server with some applications who asks a Mysql Database, and twice or three times a day It hangs. I need to stop and start it to bring it up online again. I've seen these errors:

ADVERTENCIA: Connection has been abandoned PooledConnection[com.mysql.jdbc.JDBC4Connection@9168b4]:java.lang.Exception
      at org.apache.tomcat.jdbc.pool.ConnectionPool.getThreadDump

...

com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
...
java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space
...

Here is the configuration of the Two pools in our Application:

  <Resource name="jdbc/PconDB"
  auth="Container"
  type="javax.sql.DataSource"
  factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
  testWhileIdle="true"
            testOnBorrow="true"
            testOnReturn="false"
            validationQuery="SELECT 1"
            validationInterval="30000"
            timeBetweenEvictionRunsMillis="30000"
            maxActive="300"
            minIdle="30"
            maxWait="10000"
            initialSize="30"
            removeAbandonedTimeout="60"
            removeAbandoned="true"
            logAbandoned="true"
            minEvictableIdleTimeMillis="30000"
            jmxEnabled="true"
            username="system"
            password="----"
            driverClassName="com.mysql.jdbc.Driver"
            url="jdbc:mysql://172.16.2.179:3306/pcondb?autoReconnect=true"/>              


  <Resource name="jdbc/EBS"
  auth="Container"
  type="javax.sql.DataSource"
  factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
  testWhileIdle="true"
            testOnBorrow="true"
            testOnReturn="false"
            validationQuery="SELECT 1 FROM DUAL"
            validationInterval="30000"
            timeBetweenEvictionRunsMillis="30000"
            maxActive="100"
            minIdle="20"
            maxWait="10000"
            initialSize="20"
            removeAbandonedTimeout="60"
            removeAbandoned="true"
            logAbandoned="true"
            minEvictableIdleTimeMillis="30000"
            jmxEnabled="true"
            username="apps"
            password="---"
            driverClassName="oracle.jdbc.driver.OracleDriver"
            url="jdbc:oracle:thin:@//172.16.2.249:1521/prod"/>  



We are usint a VM CEntOS 5.8 x64 on Esxi, and Tomcat 7.0.35 with jre1_6_0_27.

Can you give me an advice about what I need to do to get rid of these errors, and avoid to restart the Tomcat Server every 4-5 hours  because it hangs?

Thanks in Advance.

Josep
catalina.out.txt.zip
0
Comment
Question by:josepvallejo
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
4 Comments
 
LVL 36

Accepted Solution

by:
mccarl earned 1500 total points
ID: 39200143
You would more likely need to look more at the actual applications rather than anything Tomcat specific. Did you develop these applications yourself? Or are they third party apps?
0
 

Author Comment

by:josepvallejo
ID: 39200565
Hi mccarl,

Yes, These applications are internal and developped by a programmer. He is trying to solve some problems and errors related to queries, but there still aren't any improvements, so we were thinking about checking some parameteres to stabilize the system while the Programmer is solving Application errors.

The mainly thing is that this Web Server ins in production and is getting worse.

I don't know if we could do something to make it run a little bit better and not dying twice o three times a day.

Thanks in Advance for your answer.
Josep.
0
 

Author Comment

by:josepvallejo
ID: 39221620
I have stabilized the problem putting the parameter "RemoveAbandonedTimeout="6000" in both pools and modifying the script to reboot Tomcat every day at 3:00AM (it wasn't working before).

I've put this parameter in Tomcat options: -XX:-UseGCOverheadLimit

Please, close the request .

Josep.
0
 

Author Closing Comment

by:josepvallejo
ID: 39296219
Finally as mccarl said, it was a leaking problem due to a bad application design. It was nearly impossible to stabilize the system adjusting parameters and restarting it periodically. Finally the programmer  saw some problems about not closing sessions with mysql.close and not  catching exceptions properly with try-catch.
0

Featured Post

Get real performance insights from real users

Key features:
- Total Pages Views and Load times
- Top Pages Viewed and Load Times
- Real Time Site Page Build Performance
- Users’ Browser and Platform Performance
- Geographic User Breakdown
- And more

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows the steps required to install WordPress on Azure. Web Apps, Mobile Apps, API Apps, or Functions, in Azure all these run in an App Service plan. WordPress is no exception and requires an App Service Plan and Database to install
In this blog post, we’ll look at how using thread_statistics can cause high memory usage.
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
Suggested Courses

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question