[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

Tomcat Hangs twice or three times a day

Posted on 2013-05-27
4
Medium Priority
?
1,140 Views
Last Modified: 2013-07-03
Hi,

I have a Tomcat Server with some applications who asks a Mysql Database, and twice or three times a day It hangs. I need to stop and start it to bring it up online again. I've seen these errors:

ADVERTENCIA: Connection has been abandoned PooledConnection[com.mysql.jdbc.JDBC4Connection@9168b4]:java.lang.Exception
      at org.apache.tomcat.jdbc.pool.ConnectionPool.getThreadDump

...

com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
...
java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space
...

Here is the configuration of the Two pools in our Application:

  <Resource name="jdbc/PconDB"
  auth="Container"
  type="javax.sql.DataSource"
  factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
  testWhileIdle="true"
            testOnBorrow="true"
            testOnReturn="false"
            validationQuery="SELECT 1"
            validationInterval="30000"
            timeBetweenEvictionRunsMillis="30000"
            maxActive="300"
            minIdle="30"
            maxWait="10000"
            initialSize="30"
            removeAbandonedTimeout="60"
            removeAbandoned="true"
            logAbandoned="true"
            minEvictableIdleTimeMillis="30000"
            jmxEnabled="true"
            username="system"
            password="----"
            driverClassName="com.mysql.jdbc.Driver"
            url="jdbc:mysql://172.16.2.179:3306/pcondb?autoReconnect=true"/>              


  <Resource name="jdbc/EBS"
  auth="Container"
  type="javax.sql.DataSource"
  factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
  testWhileIdle="true"
            testOnBorrow="true"
            testOnReturn="false"
            validationQuery="SELECT 1 FROM DUAL"
            validationInterval="30000"
            timeBetweenEvictionRunsMillis="30000"
            maxActive="100"
            minIdle="20"
            maxWait="10000"
            initialSize="20"
            removeAbandonedTimeout="60"
            removeAbandoned="true"
            logAbandoned="true"
            minEvictableIdleTimeMillis="30000"
            jmxEnabled="true"
            username="apps"
            password="---"
            driverClassName="oracle.jdbc.driver.OracleDriver"
            url="jdbc:oracle:thin:@//172.16.2.249:1521/prod"/>  



We are usint a VM CEntOS 5.8 x64 on Esxi, and Tomcat 7.0.35 with jre1_6_0_27.

Can you give me an advice about what I need to do to get rid of these errors, and avoid to restart the Tomcat Server every 4-5 hours  because it hangs?

Thanks in Advance.

Josep
catalina.out.txt.zip
0
Comment
Question by:josepvallejo
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
4 Comments
 
LVL 36

Accepted Solution

by:
mccarl earned 1500 total points
ID: 39200143
You would more likely need to look more at the actual applications rather than anything Tomcat specific. Did you develop these applications yourself? Or are they third party apps?
0
 

Author Comment

by:josepvallejo
ID: 39200565
Hi mccarl,

Yes, These applications are internal and developped by a programmer. He is trying to solve some problems and errors related to queries, but there still aren't any improvements, so we were thinking about checking some parameteres to stabilize the system while the Programmer is solving Application errors.

The mainly thing is that this Web Server ins in production and is getting worse.

I don't know if we could do something to make it run a little bit better and not dying twice o three times a day.

Thanks in Advance for your answer.
Josep.
0
 

Author Comment

by:josepvallejo
ID: 39221620
I have stabilized the problem putting the parameter "RemoveAbandonedTimeout="6000" in both pools and modifying the script to reboot Tomcat every day at 3:00AM (it wasn't working before).

I've put this parameter in Tomcat options: -XX:-UseGCOverheadLimit

Please, close the request .

Josep.
0
 

Author Closing Comment

by:josepvallejo
ID: 39296219
Finally as mccarl said, it was a leaking problem due to a bad application design. It was nearly impossible to stabilize the system adjusting parameters and restarting it periodically. Finally the programmer  saw some problems about not closing sessions with mysql.close and not  catching exceptions properly with try-catch.
0

Featured Post

Simplify Your Workload with One Tool

How do you combat today’s intelligent hacker while managing multiple domains and platforms? By simplifying your workload with one tool. With Lunarpages hosting through Plesk Onyx, you can:

Automate SSL generation and installation with two clicks
Experience total server control

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

By, Vadim Tkachenko. In this article we’ll look at ClickHouse on its one year anniversary.
In this article, I’ll talk about multi-threaded slave statistics printed in MySQL error log file.
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
Suggested Courses

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question