[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1910
  • Last Modified:

Tomcat near 100% use on Windows 2008

We have a Windows 2008 R2 virtual machine on an ESXi server.  It only runs one application that requires Apache, Tomcat, Java and PostgreSQL.  We have the latest versions of these programs. The machine has 1TB of disk space and 8GB of RAM.  When I start the Tomcat 7 service, it starts using around 95% of CPU in a span of a minute, after which the total utilization of CPU by the machine is 100%.  Process Explorer shows that the number of threads for this process goes up to 91 and one of them shows over 15,000,000 Cycles Delta and uses all the CPU.  While the machine is stalled by Tomcat, the commit charge for the machine is at 12% and physical memory usage at 25%, only the CPU is maxed out.

I added CATALINA_OPTS=-Xms4096m -Xmx512m to the Environment Variables, but it doesn't seem to help.  I believe there is a memory leak here, but not sure how to pinpoint it using free tools and patch this leak.
0
lolekbolek
Asked:
lolekbolek
1 Solution
 
lolekbolekAuthor Commented:
I am attaching everything from the Tomcat's logs directory, including the output of the Probe application that I added to Tomcat.  The last file shows what kind of information Process Explorer lists for the 91 threads that are part of the tomcat7.exe process, including the first one that is eating up all the CPU.
catalina.2013-01-28.log
commons-daemon.2013-01-28.log
localhost.2013-01-28.log
manager.2013-01-28.log
probe.log
tomcat7-stderr.2013-01-28.log
tomcat7-stdout.2013-01-28.log
Tomcat-7-threads-in-Process-Expl.txt
0
 
CEHJCommented:
I haven't time to pore through these logs now, so just one tip: you can forget about HD/RAM considerations almost certainly. What you should be doing is looking for code that's cpu-intensive for whatever reason.
0
 
lolekbolekAuthor Commented:
I would love to do that, but, as I said, I am not sure how to.
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
leakim971PluritechnicianCommented:
0
 
ArneLoviusCommented:
There are also free java profiling tools available.

Considering the rest if the software stack, I'd suggest trying it on Linux rather than Windows...
0
 
mrcoffee365Commented:
This is too hard a problem for us to help with remotely.  Hire someone who knows java and tomcat and have them look  at the problem on your machine.

There's nothing wrong with running tomcat on a Windows box.

Hopefully you mistyped when you said what you set memory to in catalina opts.
CATALINA_OPTS=-Xms(min heap)m -Xmx(max heap)m
In your post you said you made the minimum 4 times your max.  If you actually did that, then maybe that's your problem?

You might also have to increase PermGen memory space, with an option like:
-XX:MaxPermSize=128m (or whatever would be more than you have now)

You are running a ton of software with tomcat and it will take quite a bit of time for someone to track down your runaway app (assuming there is one).

If the problem really is the memory allocation for tomcat, then setting catalina opts is ... it might work, but why aren't you using the Configure Tomcat app?  If tomcat is running as a service, which I'd expect is how you have it set up, I'm not sure that catalina opts will definitely get used.
0
 
lolekbolekAuthor Commented:
We fixed it by applying the suggestion from Apache Tomcat forum, adding the following to the command line options to the Tomcat Window service tomcat7/bin/tomcat7w.exe:

-Xmx4096m
-XX:MaxPermSize=512m
-XX:+UseTLAB
-XX:+UseConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled

Now, Tomcat uses only a few percents of CPU.
1
 
lolekbolekAuthor Commented:
I've requested that this question be closed as follows:

Accepted answer: 0 points for lolekbolek's comment #a38841177

for the following reason:

It was the option that worked, as opposed to general comments or links to tools that we should try to use on our own.
1
 
mrcoffee365Commented:
Glad you solved it.  I was telling you some of that already, so it's too bad that you decided not to award any points for the expert help you got.
0
 
lolekbolekAuthor Commented:
mrcoffee365 provided a good answer and deserves the points, while I still give myself a pat on the back for finding a stanza that I could use as is (for dummy solution that I needed).  Could you please cancel the closing for now and allow me to award mrcoffee365 the points?
0
 
lolekbolekAuthor Commented:
Mostly right on the money.
0

Featured Post

Restore individual SQL databases with ease

Veeam Explorer for Microsoft SQL Server delivers an easy-to-use, wizard-driven interface for restoring your databases from a backup. No expert SQL background required. Web interface provides a complete view of all available SQL databases to simplify the recovery of lost database

Tackle projects and never again get stuck behind a technical roadblock.
Join Now