?
Solved

Java Memory Management- IOEXception: cannot create native thread, IOException: too many open files

Posted on 2004-07-30
7
Medium Priority
?
608 Views
Last Modified: 2013-12-03
Hi,
I am currently having some memory management problems and was wondering if anyone could help.  The problem is that with my program I am either getting a java IOException: cannot create  native thread or IOException: too many files open.

To begin with the first one would appaer to being caused by creating too many threads.  Using -Xss enables the program to run longer and potentially for long enough but this is not ideal as the program is obvicously creating an increasing amount of threads when this is supposed to be a fixed number.  The problem may lie with the structure.  

The program initially creates 50 threads.  Each of these threads calls a function which itself creates a thread which attempts to do something for 12 seconds and then is killed.  I was wondering if the method below might be enabling the no of threads to increase with time or perhaps the solution to the probem lies somewhere else in the program.

 private String function() {
            
      Thread t = new Thread() {
            public void run() {
                  try {
                        
                        }
                        
                  } catch (Exception e) {
                        
                  }
            }
      };
      t.start();

      try {
            t.join(12000);
            
      } catch (InterruptedException e) {
            
      }
      return value;
  }
 
Also an IOException: too many files open is created when there should not be nearly enough files open to create this exception.  Again each of the 50 threads calls a function.  This function opens up to seperate .txt files (each time), writes too them and then closes them.  Prehaps the design of the function is causing the problem.

public void writeList(int pageno,String url, ArrayList links, int tno, String pageContents ) {
      
      PrintWriter out = null;
      PrintWriter out2 = null;
      

      String file ="name.txt";
        String file2="name2.txt";
 

    try {
       
        out = new PrintWriter(
                  new FileWriter(file));
      out.println("various information");                            
           out.close();
   

    out2 = new PrintWriter(
        new FileWriter(file2));
      out2.println("more information");
                 out2.close();
                 
    }
    catch (IOException e) {
        System.err.println("Caught IOException: " + e.getMessage());
    }
   
            
    }
   


 If anyone has any ideas on how to solve these issues it would be greatly appreciated, as the number of hairs on my head is rapidly decreasing with time.
Cheers,
everton690
 




 
0
Comment
Question by:everton690
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
  • +1
7 Comments
 
LVL 35

Expert Comment

by:TimYates
ID: 11675382
Can you put some debug into your thread constructor to verify that you are only creating 100 threads?

Why do you create the first 50 threads to create the second 50?

Cant you just create the 2nd 50?

I assume you are on Linux too, as each thread creates a file as well (for it's process info), so this might be adding to your problem...
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11675452
You should close those Writers in a finally block, since if an exception is thrown,  they won't get closed, leading to a too many open files situation
0
 
LVL 9

Expert Comment

by:Venci75
ID: 11675490
>>
 t.start();

     try {
          t.join(12000);
         
     } catch (InterruptedException e) {
         
     }
>>

the join() method won't kill the thread 't' - it just waits mostly 12 seconds for 't' to die. In case 't' is still alive - the current thread will continue its execution. May be your code should be like this:

t.join(12000);
if (t.isAlive()) {
  t.interrupt();
}
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:everton690
ID: 11675493
Hi Tim,
I will use the thread constructor to see if this will unearth anything.

The first 50 threads are necessary as they are used to make 1000's of calls to the function described above.  So in effect 1000s of threads are created and destroyed (although the IOException would suggest that not all are dsetroyed).

I am on Linux and was not aware that each thread creates a file.  When the thread is killed is this file also destroyed?.  
0
 
LVL 35

Assisted Solution

by:TimYates
TimYates earned 200 total points
ID: 11675518
> When the thread is killed is this file also destroyed?.  

yup :-)

I think they are created in /var/proc or /proc or something like that :-)

CEHJ's comment is very valid too :-)
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 800 total points
ID: 11675533
For this kind of thing you should probably use a thread pool:

http://jakarta.apache.org/commons/sandbox/threadpool/
0
 

Author Comment

by:everton690
ID: 11684016
Cheers CEHJ,
I think a threadpool is definately the way to go and should prevent the memory problems. :)
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This was posted to the Netbeans forum a Feb, 2010 and I also sent it to Verisign. Who didn't help much in my struggles to get my application signed. ------------------------- Start The idea here is to target your cell phones with the correct…
Ever visit a website where you spotted a really cool looking Font, yet couldn't figure out which font family it belonged to, or how to get a copy of it for your own use? This article explains the process of doing exactly that, as well as showing how…
Video by: Michael
Viewers learn about how to reduce the potential repetitiveness of coding in main by developing methods to perform specific tasks for their program. Additionally, objects are introduced for the purpose of learning how to call methods in Java. Define …
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.
Suggested Courses
Course of the Month9 days, 11 hours left to enroll

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question