• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 614
  • Last Modified:

Java Memory Management- IOEXception: cannot create native thread, IOException: too many open files

Hi,
I am currently having some memory management problems and was wondering if anyone could help.  The problem is that with my program I am either getting a java IOException: cannot create  native thread or IOException: too many files open.

To begin with the first one would appaer to being caused by creating too many threads.  Using -Xss enables the program to run longer and potentially for long enough but this is not ideal as the program is obvicously creating an increasing amount of threads when this is supposed to be a fixed number.  The problem may lie with the structure.  

The program initially creates 50 threads.  Each of these threads calls a function which itself creates a thread which attempts to do something for 12 seconds and then is killed.  I was wondering if the method below might be enabling the no of threads to increase with time or perhaps the solution to the probem lies somewhere else in the program.

 private String function() {
            
      Thread t = new Thread() {
            public void run() {
                  try {
                        
                        }
                        
                  } catch (Exception e) {
                        
                  }
            }
      };
      t.start();

      try {
            t.join(12000);
            
      } catch (InterruptedException e) {
            
      }
      return value;
  }
 
Also an IOException: too many files open is created when there should not be nearly enough files open to create this exception.  Again each of the 50 threads calls a function.  This function opens up to seperate .txt files (each time), writes too them and then closes them.  Prehaps the design of the function is causing the problem.

public void writeList(int pageno,String url, ArrayList links, int tno, String pageContents ) {
      
      PrintWriter out = null;
      PrintWriter out2 = null;
      

      String file ="name.txt";
        String file2="name2.txt";
 

    try {
       
        out = new PrintWriter(
                  new FileWriter(file));
      out.println("various information");                            
           out.close();
   

    out2 = new PrintWriter(
        new FileWriter(file2));
      out2.println("more information");
                 out2.close();
                 
    }
    catch (IOException e) {
        System.err.println("Caught IOException: " + e.getMessage());
    }
   
            
    }
   


 If anyone has any ideas on how to solve these issues it would be greatly appreciated, as the number of hairs on my head is rapidly decreasing with time.
Cheers,
everton690
 




 
0
everton690
Asked:
everton690
  • 2
  • 2
  • 2
  • +1
2 Solutions
 
TimYatesCommented:
Can you put some debug into your thread constructor to verify that you are only creating 100 threads?

Why do you create the first 50 threads to create the second 50?

Cant you just create the 2nd 50?

I assume you are on Linux too, as each thread creates a file as well (for it's process info), so this might be adding to your problem...
0
 
CEHJCommented:
You should close those Writers in a finally block, since if an exception is thrown,  they won't get closed, leading to a too many open files situation
0
 
Venci75Commented:
>>
 t.start();

     try {
          t.join(12000);
         
     } catch (InterruptedException e) {
         
     }
>>

the join() method won't kill the thread 't' - it just waits mostly 12 seconds for 't' to die. In case 't' is still alive - the current thread will continue its execution. May be your code should be like this:

t.join(12000);
if (t.isAlive()) {
  t.interrupt();
}
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
everton690Author Commented:
Hi Tim,
I will use the thread constructor to see if this will unearth anything.

The first 50 threads are necessary as they are used to make 1000's of calls to the function described above.  So in effect 1000s of threads are created and destroyed (although the IOException would suggest that not all are dsetroyed).

I am on Linux and was not aware that each thread creates a file.  When the thread is killed is this file also destroyed?.  
0
 
TimYatesCommented:
> When the thread is killed is this file also destroyed?.  

yup :-)

I think they are created in /var/proc or /proc or something like that :-)

CEHJ's comment is very valid too :-)
0
 
CEHJCommented:
For this kind of thing you should probably use a thread pool:

http://jakarta.apache.org/commons/sandbox/threadpool/
0
 
everton690Author Commented:
Cheers CEHJ,
I think a threadpool is definately the way to go and should prevent the memory problems. :)
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 2
  • 2
  • 2
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now