Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

Tomcat JVM : Too many open files

Posted on 2009-04-14
4
Medium Priority
?
2,478 Views
Last Modified: 2013-12-02
i have an application running on tomcat application server.  After sometime of running it goes down with "Too many open files" error.

Here is the error.

Apr 14, 2009 6:22:53 PM org.apache.catalina.core.StandardServer await
SEVERE: StandardServer.await: accept:
java.net.SocketException: Too many open files
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        at java.net.ServerSocket.implAccept(ServerSocket.java:450)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at org.apache.catalina.core.StandardServer.await(StandardServer.java:388)
        at org.apache.catalina.startup.Catalina.await(Catalina.java:615)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:575)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:275)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Apr 14, 2009 6:22:53 PM org.apache.coyote.http11.Http11BaseProtocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8082
Apr 14, 2009 6:22:53 PM org.apache.tomcat.util.net.PoolTcpEndpoint acceptSocket
SEVERE: Endpoint ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=8082] ignored exception: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        at java.net.ServerSocket.implAccept(ServerSocket.java:450)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60)
        at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:407)
        at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:70)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:595)
java.lang.NullPointerException
*****************************************************
 
When I checked the server, I found more than 400 connection made to other server. which are in CLOSE_WAIT state.


java      31550 tomcat2  157u     IPv6 1472334157                 TCP cihcispapp099.corporate.ge.com:34609->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)
java      31550 tomcat2  158u     IPv6 1472335993                 TCP cihcispapp099.corporate.ge.com:34665->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)
java      31550 tomcat2  159u     IPv6 1472338064                 TCP cihcispapp099.corporate.ge.com:34717->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)
java      31550 tomcat2  161u     IPv6 1472351704                 TCP cihcispapp099.corporate.ge.com:35036->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)
java      31550 tomcat2  168u     IPv6 1472365912                 TCP cihcispapp099.corporate.ge.com:35420->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)

****************************************************************

I want to drill down the issue as which specific class or method is making this connection ? How can I start this troubleshooting ? Can someone please guide how to proceed ?
0
Comment
Question by:Prajal_Sutaria
  • 2
4 Comments
 
LVL 23

Expert Comment

by:Ajay-Singh
ID: 24144591
Whats the OS here? In case of Unix, you can tighten the DOS attach
parameters. I have a feeling that the system is under DOS attack.
0
 
LVL 1

Author Comment

by:Prajal_Sutaria
ID: 24144598
Its a Linux server. I do not understand where you are trying to point to ?
0
 
LVL 23

Expert Comment

by:Ajay-Singh
ID: 24144814
DOS, (Denial of Service) more here,
http://en.wikipedia.org/wiki/Denial-of-service_attack


is a type of attack, where the user opens a lot of sockets in very short
time and flood the sockets. The legitimate users, will not be able to
connect to server.
0
 
LVL 3

Accepted Solution

by:
CptGiggles earned 1000 total points
ID: 24148231
"I found more than 400 connection made to other server"

If both servers that are being used belong to you (Directly or Indirectly) I would tend to think that you could have some typos in your application/configuration that opens files but doesn't fully close them....


We had a problem with a JBoss server at my workplace that was doing something like this (JBoss runs an embedded Tomcat, so I would think they could be the same issue)
If however you think this is not an issue with the application, you can always increase the maximum open files configuration :

LINK: http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/

I also realize that most people would disagree with me but, sometimes a good old fashioned reboot solves a lot of lingering processes problems (Defunt Javas and such) which can do some crazy stuff to your machines.


0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Are you developing a Java application and want to create Excel Spreadsheets? You have come to the right place, this article will describe how you can create Excel Spreadsheets from a Java Application. For the purposes of this article, I will be u…
Upgrading Tomcat – There are a couple of methods to upgrade Tomcat is to use The Apache Installer is to download and unzip and run the services.bat remove|install Tomcat6 Because of the App that we are working with, we can only use Tomcat 6.…
The viewer will learn how to implement Singleton Design Pattern in Java.
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.
Suggested Courses
Course of the Month15 days, 4 hours left to enroll

577 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question