Solved

Java Memory Management- IOEXception: cannot create native thread, IOException: too many open files

Posted on 2004-07-30
7
604 Views
Last Modified: 2013-12-03
Hi,
I am currently having some memory management problems and was wondering if anyone could help.  The problem is that with my program I am either getting a java IOException: cannot create  native thread or IOException: too many files open.

To begin with the first one would appaer to being caused by creating too many threads.  Using -Xss enables the program to run longer and potentially for long enough but this is not ideal as the program is obvicously creating an increasing amount of threads when this is supposed to be a fixed number.  The problem may lie with the structure.  

The program initially creates 50 threads.  Each of these threads calls a function which itself creates a thread which attempts to do something for 12 seconds and then is killed.  I was wondering if the method below might be enabling the no of threads to increase with time or perhaps the solution to the probem lies somewhere else in the program.

 private String function() {
            
      Thread t = new Thread() {
            public void run() {
                  try {
                        
                        }
                        
                  } catch (Exception e) {
                        
                  }
            }
      };
      t.start();

      try {
            t.join(12000);
            
      } catch (InterruptedException e) {
            
      }
      return value;
  }
 
Also an IOException: too many files open is created when there should not be nearly enough files open to create this exception.  Again each of the 50 threads calls a function.  This function opens up to seperate .txt files (each time), writes too them and then closes them.  Prehaps the design of the function is causing the problem.

public void writeList(int pageno,String url, ArrayList links, int tno, String pageContents ) {
      
      PrintWriter out = null;
      PrintWriter out2 = null;
      

      String file ="name.txt";
        String file2="name2.txt";
 

    try {
       
        out = new PrintWriter(
                  new FileWriter(file));
      out.println("various information");                            
           out.close();
   

    out2 = new PrintWriter(
        new FileWriter(file2));
      out2.println("more information");
                 out2.close();
                 
    }
    catch (IOException e) {
        System.err.println("Caught IOException: " + e.getMessage());
    }
   
            
    }
   


 If anyone has any ideas on how to solve these issues it would be greatly appreciated, as the number of hairs on my head is rapidly decreasing with time.
Cheers,
everton690
 




 
0
Comment
Question by:everton690
  • 2
  • 2
  • 2
  • +1
7 Comments
 
LVL 35

Expert Comment

by:TimYates
ID: 11675382
Can you put some debug into your thread constructor to verify that you are only creating 100 threads?

Why do you create the first 50 threads to create the second 50?

Cant you just create the 2nd 50?

I assume you are on Linux too, as each thread creates a file as well (for it's process info), so this might be adding to your problem...
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11675452
You should close those Writers in a finally block, since if an exception is thrown,  they won't get closed, leading to a too many open files situation
0
 
LVL 9

Expert Comment

by:Venci75
ID: 11675490
>>
 t.start();

     try {
          t.join(12000);
         
     } catch (InterruptedException e) {
         
     }
>>

the join() method won't kill the thread 't' - it just waits mostly 12 seconds for 't' to die. In case 't' is still alive - the current thread will continue its execution. May be your code should be like this:

t.join(12000);
if (t.isAlive()) {
  t.interrupt();
}
0
PeopleSoft Has Never Been Easier

PeopleSoft Adoption Made Smooth & Simple!

On-The-Job Training Is made Intuitive & Easy With WalkMe's On-Screen Guidance Tool.  Claim Your Free WalkMe Account Now

 

Author Comment

by:everton690
ID: 11675493
Hi Tim,
I will use the thread constructor to see if this will unearth anything.

The first 50 threads are necessary as they are used to make 1000's of calls to the function described above.  So in effect 1000s of threads are created and destroyed (although the IOException would suggest that not all are dsetroyed).

I am on Linux and was not aware that each thread creates a file.  When the thread is killed is this file also destroyed?.  
0
 
LVL 35

Assisted Solution

by:TimYates
TimYates earned 50 total points
ID: 11675518
> When the thread is killed is this file also destroyed?.  

yup :-)

I think they are created in /var/proc or /proc or something like that :-)

CEHJ's comment is very valid too :-)
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 200 total points
ID: 11675533
For this kind of thing you should probably use a thread pool:

http://jakarta.apache.org/commons/sandbox/threadpool/
0
 

Author Comment

by:everton690
ID: 11684016
Cheers CEHJ,
I think a threadpool is definately the way to go and should prevent the memory problems. :)
0

Featured Post

Instantly Create Instructional Tutorials

Contextual Guidance at the moment of need helps your employees adopt to new software or processes instantly. Boost knowledge retention and employee engagement step-by-step with one easy solution.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
SHA2 certs for IIS AND Java? 2 122
Print Rhino Java Array in Javascript 1 39
DTD and JAVA versions 1 33
activeMQ Queue Messages Enqueued and Messages Dequeued 1 39
By the end of 1980s, object oriented programming using languages like C++, Simula69 and ObjectPascal gained momentum. It looked like programmers finally found the perfect language. C++ successfully combined the object oriented principles of Simula w…
Java Flight Recorder and Java Mission Control together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Java Flight Recorder is a profiling and event collectio…
Viewers will learn about the regular for loop in Java and how to use it. Definition: Break the for loop down into 3 parts: Syntax when using for loops: Example using a for loop:
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question