Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Tomcat JVM : Too many open files

Posted on 2009-04-14
4
Medium Priority
?
2,452 Views
Last Modified: 2013-12-02
i have an application running on tomcat application server.  After sometime of running it goes down with "Too many open files" error.

Here is the error.

Apr 14, 2009 6:22:53 PM org.apache.catalina.core.StandardServer await
SEVERE: StandardServer.await: accept:
java.net.SocketException: Too many open files
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        at java.net.ServerSocket.implAccept(ServerSocket.java:450)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at org.apache.catalina.core.StandardServer.await(StandardServer.java:388)
        at org.apache.catalina.startup.Catalina.await(Catalina.java:615)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:575)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:275)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Apr 14, 2009 6:22:53 PM org.apache.coyote.http11.Http11BaseProtocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8082
Apr 14, 2009 6:22:53 PM org.apache.tomcat.util.net.PoolTcpEndpoint acceptSocket
SEVERE: Endpoint ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=8082] ignored exception: java.net.SocketException: Too many open files
java.net.SocketException: Too many open files
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        at java.net.ServerSocket.implAccept(ServerSocket.java:450)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60)
        at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:407)
        at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:70)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:595)
java.lang.NullPointerException
*****************************************************
 
When I checked the server, I found more than 400 connection made to other server. which are in CLOSE_WAIT state.


java      31550 tomcat2  157u     IPv6 1472334157                 TCP cihcispapp099.corporate.ge.com:34609->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)
java      31550 tomcat2  158u     IPv6 1472335993                 TCP cihcispapp099.corporate.ge.com:34665->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)
java      31550 tomcat2  159u     IPv6 1472338064                 TCP cihcispapp099.corporate.ge.com:34717->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)
java      31550 tomcat2  161u     IPv6 1472351704                 TCP cihcispapp099.corporate.ge.com:35036->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)
java      31550 tomcat2  168u     IPv6 1472365912                 TCP cihcispapp099.corporate.ge.com:35420->corpp602-V04.corporate.ge.com:http (CLOSE_WAIT)

****************************************************************

I want to drill down the issue as which specific class or method is making this connection ? How can I start this troubleshooting ? Can someone please guide how to proceed ?
0
Comment
Question by:Prajal_Sutaria
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 23

Expert Comment

by:Ajay-Singh
ID: 24144591
Whats the OS here? In case of Unix, you can tighten the DOS attach
parameters. I have a feeling that the system is under DOS attack.
0
 
LVL 1

Author Comment

by:Prajal_Sutaria
ID: 24144598
Its a Linux server. I do not understand where you are trying to point to ?
0
 
LVL 23

Expert Comment

by:Ajay-Singh
ID: 24144814
DOS, (Denial of Service) more here,
http://en.wikipedia.org/wiki/Denial-of-service_attack


is a type of attack, where the user opens a lot of sockets in very short
time and flood the sockets. The legitimate users, will not be able to
connect to server.
0
 
LVL 3

Accepted Solution

by:
CptGiggles earned 1000 total points
ID: 24148231
"I found more than 400 connection made to other server"

If both servers that are being used belong to you (Directly or Indirectly) I would tend to think that you could have some typos in your application/configuration that opens files but doesn't fully close them....


We had a problem with a JBoss server at my workplace that was doing something like this (JBoss runs an embedded Tomcat, so I would think they could be the same issue)
If however you think this is not an issue with the application, you can always increase the maximum open files configuration :

LINK: http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/

I also realize that most people would disagree with me but, sometimes a good old fashioned reboot solves a lot of lingering processes problems (Defunt Javas and such) which can do some crazy stuff to your machines.


0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Are you developing a Java application and want to create Excel Spreadsheets? You have come to the right place, this article will describe how you can create Excel Spreadsheets from a Java Application. For the purposes of this article, I will be u…
This article is about some of the basic and important steps to be used to improve the performance in web-sphere commerce application development. 1) Always leverage the Dyna-caching facility provided by the product 2) Remove the unwanted code …
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
How to fix incompatible JVM issue while installing Eclipse While installing Eclipse in windows, got one error like above and unable to proceed with the installation. This video describes how to successfully install Eclipse. How to solve incompa…
Suggested Courses

604 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question