Solved

Java Memory Management- IOEXception: cannot create native thread, IOException: too many open files

Posted on 2004-07-30
7
602 Views
Last Modified: 2013-12-03
Hi,
I am currently having some memory management problems and was wondering if anyone could help.  The problem is that with my program I am either getting a java IOException: cannot create  native thread or IOException: too many files open.

To begin with the first one would appaer to being caused by creating too many threads.  Using -Xss enables the program to run longer and potentially for long enough but this is not ideal as the program is obvicously creating an increasing amount of threads when this is supposed to be a fixed number.  The problem may lie with the structure.  

The program initially creates 50 threads.  Each of these threads calls a function which itself creates a thread which attempts to do something for 12 seconds and then is killed.  I was wondering if the method below might be enabling the no of threads to increase with time or perhaps the solution to the probem lies somewhere else in the program.

 private String function() {
            
      Thread t = new Thread() {
            public void run() {
                  try {
                        
                        }
                        
                  } catch (Exception e) {
                        
                  }
            }
      };
      t.start();

      try {
            t.join(12000);
            
      } catch (InterruptedException e) {
            
      }
      return value;
  }
 
Also an IOException: too many files open is created when there should not be nearly enough files open to create this exception.  Again each of the 50 threads calls a function.  This function opens up to seperate .txt files (each time), writes too them and then closes them.  Prehaps the design of the function is causing the problem.

public void writeList(int pageno,String url, ArrayList links, int tno, String pageContents ) {
      
      PrintWriter out = null;
      PrintWriter out2 = null;
      

      String file ="name.txt";
        String file2="name2.txt";
 

    try {
       
        out = new PrintWriter(
                  new FileWriter(file));
      out.println("various information");                            
           out.close();
   

    out2 = new PrintWriter(
        new FileWriter(file2));
      out2.println("more information");
                 out2.close();
                 
    }
    catch (IOException e) {
        System.err.println("Caught IOException: " + e.getMessage());
    }
   
            
    }
   


 If anyone has any ideas on how to solve these issues it would be greatly appreciated, as the number of hairs on my head is rapidly decreasing with time.
Cheers,
everton690
 




 
0
Comment
Question by:everton690
  • 2
  • 2
  • 2
  • +1
7 Comments
 
LVL 35

Expert Comment

by:TimYates
ID: 11675382
Can you put some debug into your thread constructor to verify that you are only creating 100 threads?

Why do you create the first 50 threads to create the second 50?

Cant you just create the 2nd 50?

I assume you are on Linux too, as each thread creates a file as well (for it's process info), so this might be adding to your problem...
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 11675452
You should close those Writers in a finally block, since if an exception is thrown,  they won't get closed, leading to a too many open files situation
0
 
LVL 9

Expert Comment

by:Venci75
ID: 11675490
>>
 t.start();

     try {
          t.join(12000);
         
     } catch (InterruptedException e) {
         
     }
>>

the join() method won't kill the thread 't' - it just waits mostly 12 seconds for 't' to die. In case 't' is still alive - the current thread will continue its execution. May be your code should be like this:

t.join(12000);
if (t.isAlive()) {
  t.interrupt();
}
0
The Eight Noble Truths of Backup and Recovery

How can IT departments tackle the challenges of a Big Data world? This white paper provides a roadmap to success and helps companies ensure that all their data is safe and secure, no matter if it resides on-premise with physical or virtual machines or in the cloud.

 

Author Comment

by:everton690
ID: 11675493
Hi Tim,
I will use the thread constructor to see if this will unearth anything.

The first 50 threads are necessary as they are used to make 1000's of calls to the function described above.  So in effect 1000s of threads are created and destroyed (although the IOException would suggest that not all are dsetroyed).

I am on Linux and was not aware that each thread creates a file.  When the thread is killed is this file also destroyed?.  
0
 
LVL 35

Assisted Solution

by:TimYates
TimYates earned 50 total points
ID: 11675518
> When the thread is killed is this file also destroyed?.  

yup :-)

I think they are created in /var/proc or /proc or something like that :-)

CEHJ's comment is very valid too :-)
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 200 total points
ID: 11675533
For this kind of thing you should probably use a thread pool:

http://jakarta.apache.org/commons/sandbox/threadpool/
0
 

Author Comment

by:everton690
ID: 11684016
Cheers CEHJ,
I think a threadpool is definately the way to go and should prevent the memory problems. :)
0

Featured Post

VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
numbers ascending pyramid 101 195
jar file executable 12 52
Basic Java Case or If-Else statement... 3 50
expectj telnet failing 5 36
Java contains several comparison operators (e.g., <, <=, >, >=, ==, !=) that allow you to compare primitive values. However, these operators cannot be used to compare the contents of objects. Interface Comparable is used to allow objects of a cl…
By the end of 1980s, object oriented programming using languages like C++, Simula69 and ObjectPascal gained momentum. It looked like programmers finally found the perfect language. C++ successfully combined the object oriented principles of Simula w…
Viewers learn about the third conditional statement “else if” and use it in an example program. Then additional information about conditional statements is provided, covering the topic thoroughly. Viewers learn about the third conditional statement …
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…

773 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question