[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

Tomcat near 100% use on Windows 2008

Posted on 2013-01-28
13
Medium Priority
?
1,864 Views
Last Modified: 2013-01-31
We have a Windows 2008 R2 virtual machine on an ESXi server.  It only runs one application that requires Apache, Tomcat, Java and PostgreSQL.  We have the latest versions of these programs. The machine has 1TB of disk space and 8GB of RAM.  When I start the Tomcat 7 service, it starts using around 95% of CPU in a span of a minute, after which the total utilization of CPU by the machine is 100%.  Process Explorer shows that the number of threads for this process goes up to 91 and one of them shows over 15,000,000 Cycles Delta and uses all the CPU.  While the machine is stalled by Tomcat, the commit charge for the machine is at 12% and physical memory usage at 25%, only the CPU is maxed out.

I added CATALINA_OPTS=-Xms4096m -Xmx512m to the Environment Variables, but it doesn't seem to help.  I believe there is a memory leak here, but not sure how to pinpoint it using free tools and patch this leak.
0
Comment
Question by:lolekbolek
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
13 Comments
 

Author Comment

by:lolekbolek
ID: 38827861
I am attaching everything from the Tomcat's logs directory, including the output of the Probe application that I added to Tomcat.  The last file shows what kind of information Process Explorer lists for the 91 threads that are part of the tomcat7.exe process, including the first one that is eating up all the CPU.
catalina.2013-01-28.log
commons-daemon.2013-01-28.log
localhost.2013-01-28.log
manager.2013-01-28.log
probe.log
tomcat7-stderr.2013-01-28.log
tomcat7-stdout.2013-01-28.log
Tomcat-7-threads-in-Process-Expl.txt
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38828020
I haven't time to pore through these logs now, so just one tip: you can forget about HD/RAM considerations almost certainly. What you should be doing is looking for code that's cpu-intensive for whatever reason.
0
 

Author Comment

by:lolekbolek
ID: 38830558
I would love to do that, but, as I said, I am not sure how to.
0
NFR key for Veeam Agent for Linux

Veeam is happy to provide a free NFR license for one year.  It allows for the non‑production use and valid for five workstations and two servers. Veeam Agent for Linux is a simple backup tool for your Linux installations, both on‑premises and in the public cloud.

 
LVL 82

Expert Comment

by:leakim971
ID: 38836984
0
 
LVL 37

Expert Comment

by:ArneLovius
ID: 38837160
There are also free java profiling tools available.

Considering the rest if the software stack, I'd suggest trying it on Linux rather than Windows...
0
 
LVL 27

Accepted Solution

by:
mrcoffee365 earned 2000 total points
ID: 38838463
This is too hard a problem for us to help with remotely.  Hire someone who knows java and tomcat and have them look  at the problem on your machine.

There's nothing wrong with running tomcat on a Windows box.

Hopefully you mistyped when you said what you set memory to in catalina opts.
CATALINA_OPTS=-Xms(min heap)m -Xmx(max heap)m
In your post you said you made the minimum 4 times your max.  If you actually did that, then maybe that's your problem?

You might also have to increase PermGen memory space, with an option like:
-XX:MaxPermSize=128m (or whatever would be more than you have now)

You are running a ton of software with tomcat and it will take quite a bit of time for someone to track down your runaway app (assuming there is one).

If the problem really is the memory allocation for tomcat, then setting catalina opts is ... it might work, but why aren't you using the Configure Tomcat app?  If tomcat is running as a service, which I'd expect is how you have it set up, I'm not sure that catalina opts will definitely get used.
0
 

Author Comment

by:lolekbolek
ID: 38841177
We fixed it by applying the suggestion from Apache Tomcat forum, adding the following to the command line options to the Tomcat Window service tomcat7/bin/tomcat7w.exe:

-Xmx4096m
-XX:MaxPermSize=512m
-XX:+UseTLAB
-XX:+UseConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled

Now, Tomcat uses only a few percents of CPU.
1
 

Author Comment

by:lolekbolek
ID: 38842140
I've requested that this question be closed as follows:

Accepted answer: 0 points for lolekbolek's comment #a38841177

for the following reason:

It was the option that worked, as opposed to general comments or links to tools that we should try to use on our own.
1
 
LVL 27

Expert Comment

by:mrcoffee365
ID: 38842091
Glad you solved it.  I was telling you some of that already, so it's too bad that you decided not to award any points for the expert help you got.
0
 

Author Comment

by:lolekbolek
ID: 38842138
mrcoffee365 provided a good answer and deserves the points, while I still give myself a pat on the back for finding a stanza that I could use as is (for dummy solution that I needed).  Could you please cancel the closing for now and allow me to award mrcoffee365 the points?
0
 

Author Closing Comment

by:lolekbolek
ID: 38842141
Mostly right on the money.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
Resolving an irritating Remote Desktop connection that stops your saved credentials from being used.
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…
This tutorial will walk an individual through the steps necessary to join and promote the first Windows Server 2012 domain controller into an Active Directory environment running on Windows Server 2008. Determine the location of the FSMO roles by lo…
Suggested Courses

656 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question