?
Solved

Tomcat JVM Inaccessible  - Logs Clean

Posted on 2009-05-07
14
Medium Priority
?
651 Views
Last Modified: 2013-12-02
I have been running into this situation every 1-2 days on my production site where my Tomcat 5.5.7 JVM will become unresponsive. My Tomcat logs are clean. I have included a dump of my memory usage below. I notice that the Committed & Max are the same for the Eden Space, Survivor Space, & Old Gen, yet the used is considerably lower. Can someone please explain and offer suggestions:

MEMORY TYPE: Code Cache  
 Used:        22937280  
 Committed:   23035904
 Max:         50331648  
   
 MEMORY TYPE: PS Eden Space  
 Used:        28793728  
 Committed:   51183616
 Max:         51183616  
   
 MEMORY TYPE: PS Survivor Space  
 Used:        1935936  
 Committed:   4325376
 Max:         4325376  
   
 MEMORY TYPE: PS Old Gen  
 Used:        179478224  
 Committed:   477233152
 Max:         477233152  
   
 MEMORY TYPE: PS Perm Gen  
 Used:        66484456  
 Committed:   71172096
 Max:         268435456    
 
 Used Memory:  210207888
 Free Memory:  322534256
 Total Memory: 532742144
0
Comment
Question by:scott_m_ruby
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 5
14 Comments
 
LVL 92

Expert Comment

by:objects
ID: 24400901
sounds like you are running of request threads.
Do a thread dump when it is unresponsive to check whats going on
may be that you just need to increase the thread pool size

0
 

Author Comment

by:scott_m_ruby
ID: 24400931
Can you give me the syntax for doing a thread dump?
0
 
LVL 92

Expert Comment

by:objects
ID: 24400938
send the tomcat process a QUIT signal

what OS?

0
Learn how to optimize MySQL for your business need

With the increasing importance of apps & networks in both business & personal interconnections, perfor. has become one of the key metrics of successful communication. This ebook is a hands-on business-case-driven guide to understanding MySQL query parameter tuning & database perf

 

Author Comment

by:scott_m_ruby
ID: 24400947
Linux
0
 
LVL 92

Expert Comment

by:objects
ID: 24400957
kill -quit

the thread dump will get dumped to stdout, which usually ends up going to catalina.out

0
 

Author Comment

by:scott_m_ruby
ID: 24416844
Tomcat became unresponsive again this afternoon and I executed the thread dump as you suggested. It looks like I am using my maxThreads of 150. I will plan to increase but was hoping that you could look at my dump file to help diagnose what the "problem" may be. I have attached the portion of my catalina.out containing the thread dump. Also, as a note when I execute the kill -quit <tomcat pid> my process did not get killed. Should it have, normally?

Thanks,
Scott
catalina.log
0
 
LVL 92

Accepted Solution

by:
objects earned 2000 total points
ID: 24417342
appears to be lots of waiting on your connection pool.
check the size of your pool and the amount of database connections available
and make sure you close connections when you no longer need them

>  Should it have, normally?

no it doesn't kill the process just sends it a signal

0
 

Author Comment

by:scott_m_ruby
ID: 24437439
I did double my maxThreads from 150 to 300 and increase my max connections for my database connection pool from 25 to 50. The problem did not occur on Tuesday or Wednesday. In my monitoring via the Tomcat Manager web app I saw the thread count at 25 all day both days. I did notice today that I had 37 database connections gobbled up in about 10 seconds and I don't even have that many active users. Any ideas on how to get a handle on? Is there any java profiling tool or Tomcat monitoring tool that can help?
0
 
LVL 92

Expert Comment

by:objects
ID: 24437457
sounds better.

> . I did notice today that I had 37 database connections gobbled up in about 10 seconds and I don't even have that many active users.

if you are checking at the db then that could be ok as the pool will keep them open for a period (in case they can be reused)
0
 

Author Comment

by:scott_m_ruby
ID: 24530763
I wanted to check back with you as I believe I need a Tomcat monitoring tool that will allow me to see what is going on with my database connection pool as I continue to have periods throughout the day where the app tries to gobble up large number of connections and that seems to be causing our problems.
0
 
LVL 92

Expert Comment

by:objects
ID: 24532300
you can check the db to see how many connections are currently open

0

Featured Post

Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
When the s#!t hits the fan, you don’t have time to look up who’s on call, draft emails, call collaborators, or send text messages. An instant chat window is definitely the way to go, especially one like HipChat. HipChat is a true business app. An…
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.
How to fix incompatible JVM issue while installing Eclipse While installing Eclipse in windows, got one error like above and unable to proceed with the installation. This video describes how to successfully install Eclipse. How to solve incompa…
Suggested Courses
Course of the Month12 days, 7 hours left to enroll

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question