Solved

JSP (too may open files error)

Posted on 2008-06-13
9
657 Views
Last Modified: 2013-11-23
Hi,

I have a large JSP page and i am using tomcat 5.0 with JDK 1.5_15 release on a linux server.

The JSP code opens the files , writes into them and then closes them. For a stress test if i keep using this opage back and forth during the day, i get the dreaded error thrown to me saying (too many open files). and then nothing works after that unless i restart tomcat server.

I believe there is a way to increase the max open files on unix server but i really want to fix it in my code. I am pasting the sample code which is how the file opening and closing is done in almost all the methods of this page. Please see if you can spot something wrong which is not letting the files close properly etc ?
===========
    String chgT3Prio (HttpServletRequest request, String prio, int seq, int t1cnt)
    {

            
      int count = 0;
      String filePath = "/some/path/myfile1";
      FileLock lock1 = null;
      FileChannel channel1 = null;
      
      BufferedReader reportFile = null;
      FileOutputStream out = null;
    PrintStream p = null;

      try {
          String tmpFileName = "/some/path/myfile1.tmp";
            RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();

          try
          {
              //--FileOutputStream out;
              //FileOutputStream out = null;
              //PrintStream p = null;
              
              String line = null;
   
              File tmpFile  = new File (tmpFileName);
              if (tmpFile.exists()) {
                  tmpFile.delete();
              }

              out = new FileOutputStream(tmpFileName, true);
              p = new PrintStream(out);

              reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));

              //////////////////////////////////////////
              // 1/2. change priority of target
              //////////////////////////////////////////
              while((line = reportFile.readLine())!=null) {
                StringTokenizer line_data = new StringTokenizer(line, "|", false);
                String ftype = line_data.nextToken();
   
                if (ftype.equals("T3"))
                      count++;
   
                if (count == seq) {
                    String loc_a    = line_data.nextToken();
                    String loc_z    = line_data.nextToken();
                    String fac_type = line_data.nextToken();
                    String fac_num  = line_data.nextToken();
                    String mile     = line_data.nextToken();
                    String ucr      = line_data.nextToken();
                    String user     = line_data.nextToken();
                    String sub_dt   = line_data.nextToken();
                    String sub_type = line_data.nextToken();
                    line_data.nextToken();
                    String long_time = line_data.nextToken();
                    String trailer = line_data.nextToken();
   
                    String newT3 = "T3|" + loc_a  + "|" + loc_z  + "|" + fac_type  + "|" + fac_num  + "|" + mile  +
                                   "|" + ucr  + "|" + user  + "|" + sub_dt  + "|" + sub_type + "|";
   
                    if (prio.equals("Normal")) {
                        newT3 += "High";
                    }else if (prio.equals("High")) {
                        newT3 += "Normal";
                    }
                    newT3 += "|" + long_time;
                    newT3 += "|" + trailer;
   
                    try {
                      p.println (newT3);
                    } catch (Exception e) {
                      System.err.println ("Error writing to log");
                    }
                    count = -32767;
                } else {
                    try {
                      p.println (line);
                    } catch (Exception e) {
                      System.err.println ("Error writing to log");
                    }
                  }
              }
              
                p.close();
              reportFile.close();

              
          } catch (FileNotFoundException f) {
              System.out.println(f);
          } catch (IOException e) {
              System.out.println(e);
          }
          
      } catch (FileNotFoundException fe) {
          System.out.println(fe);
      } catch (IOException e1) {
          System.out.println(e1);
      } finally {
          try {
              lock1.release();                        
            
          } catch (IOException e3) {
          }
      }




String chgT3Prio (HttpServletRequest request, String prio, int seq, int t1cnt)
    {
 
	       
	int count = 0;
	String filePath = "/some/path/myfile1";
	FileLock lock1 = null;
	FileChannel channel1 = null;
	
	BufferedReader reportFile = null;
	FileOutputStream out = null;
    PrintStream p = null; 
 
	try {
	    String tmpFileName = "/some/path/myfile1.tmp";
            RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();
 
	    try 
	    {
	        //--FileOutputStream out;
	        //FileOutputStream out = null;
	        //PrintStream p = null; 
	        
	        String line = null;
    
	        File tmpFile  = new File (tmpFileName);
	        if (tmpFile.exists()) {
	            tmpFile.delete();
	        }
 
	        out = new FileOutputStream(tmpFileName, true);
	        p = new PrintStream(out);
 
	        reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));
 
	        //////////////////////////////////////////
	        // 1/2. change priority of target
	        //////////////////////////////////////////
	        while((line = reportFile.readLine())!=null) {
		    StringTokenizer line_data = new StringTokenizer(line, "|", false);
		    String ftype = line_data.nextToken();
    
		    if (ftype.equals("T3"))
	                count++;
    
		    if (count == seq) {
		        String loc_a    = line_data.nextToken();
		        String loc_z    = line_data.nextToken();
		        String fac_type = line_data.nextToken();
		        String fac_num  = line_data.nextToken();
		        String mile     = line_data.nextToken();
		        String ucr      = line_data.nextToken();
		        String user     = line_data.nextToken();
		        String sub_dt   = line_data.nextToken();
		        String sub_type = line_data.nextToken();
		        line_data.nextToken();
		        String long_time = line_data.nextToken();
		        String trailer = line_data.nextToken();
    
		        String newT3 = "T3|" + loc_a  + "|" + loc_z  + "|" + fac_type  + "|" + fac_num  + "|" + mile  + 
		    		       "|" + ucr  + "|" + user  + "|" + sub_dt  + "|" + sub_type + "|";
    
		        if (prio.equals("Normal")) {
		            newT3 += "High";
		        }else if (prio.equals("High")) {
		            newT3 += "Normal";
		        }
		        newT3 += "|" + long_time;
		        newT3 += "|" + trailer;
    
		        try {
			    p.println (newT3);
		        } catch (Exception e) {
			    System.err.println ("Error writing to log");
		        }
		        count = -32767;
		    } else {
		        try {
			    p.println (line);
		        } catch (Exception e) {
			    System.err.println ("Error writing to log");
		        }
	            }
	        }
	        
                p.close();
	        reportFile.close();
 
	         
	    } catch (FileNotFoundException f) {
	        System.out.println(f);
	    } catch (IOException e) {
	        System.out.println(e);
	    } 
	    
	} catch (FileNotFoundException fe) {
	    System.out.println(fe);
	} catch (IOException e1) {
	    System.out.println(e1);
	} finally {
	    try {
	        lock1.release();	     	        
		
	    } catch (IOException e3) {
	    }
	}

Open in new window

0
Comment
Question by:Ade101
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
9 Comments
 
LVL 2

Expert Comment

by:Airyck666
ID: 21783649
Try closing the FileOutputStream out.


p.close();
reportFile.close();
out.close();

Open in new window

0
 

Author Comment

by:Ade101
ID: 21789122
I'd tried this too but no help :/

For locking the file is this the right code ?

 RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();

if yes then still we dont use the raf1 anywhere other than just initializing it and locking it. For read and write we use FileOutPutStream and PrintStream instead. Should I be closing raf1 also?

I am new to this.. Thnaks.
0
 
LVL 2

Expert Comment

by:Airyck666
ID: 21789167
Are you seeing any exceptions when you run it?
0
Turn Insights Into Action

You’ve already invested in ITSM tools, chat applications, automation utilities, and more. Fortify these solutions with intelligent communications so you can drive business processes forward.

With xMatters, you'll never miss a beat.

 
LVL 2

Expert Comment

by:Airyck666
ID: 21789213
The file locking looks okay.
A couple of things you could try:

Try seperating this definition and explicitly closing the FileInputStream.
reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));
see Code snippet

Another thing you can try is to close all your connections in your finally block to ensure these resources are released if there is a problem.

And the catch block at line 92-96 can be removed (along with the corresponding try) as your're catching these exceptions below.
FileInputStream fis = new FileInputStream(filePath);
reportFile = new BufferedReader(new InputStreamReader(fis));
// And then explicitly close this connection in the finally block

Open in new window

0
 

Author Comment

by:Ade101
ID: 21789262
The error I am receiving is:

Java.io.FileNotFoundException: myfilesname.txt (Too many open files)

Once this error is thrown no further file can be opened through JSP unless i restart the web server.
0
 

Author Comment

by:Ade101
ID: 21795680
I tried your suggestions i.e. seperating the FileInputString defination and closing eveything in the finally block too but after regression testing the page, i get the error again and for a resolution i have to restarted the web server which is even more annoying.

Can you please point out what can be the reason?

Thanks a lot.
handleDate() called ...handleDate() called ...java.io.FileNotFoundException: /home/mytestdata/MyTestFile (Too many open files)
WebappClassLoader: Failed to open JAR
java.util.zip.ZipException: error in opening zip file
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:203)
        at java.util.jar.JarFile.<init>(JarFile.java:132)
        at java.util.jar.JarFile.<init>(JarFile.java:97)
        at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1549)
        at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1771)
        at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1575)
        at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:860)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1307)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1189)
        at java.beans.Introspector.instantiate(Introspector.java:1453)
        at java.beans.Introspector.findExplicitBeanInfo(Introspector.java:410)
        at java.beans.Introspector.<init>(Introspector.java:359)
        at java.beans.Introspector.getBeanInfo(Introspector.java:159)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:896)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:926)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:801)
        at org.apache.commons.beanutils.PropertyUtilsBean.getSimpleProperty(PropertyUtilsBean.java:1119)
        at org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:686)
        at org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:715)
        at org.apache.commons.beanutils.PropertyUtils.getProperty(PropertyUtils.java:290)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:255)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
        at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
        at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:595)
WebappClassLoader: Failed to open JAR
java.util.zip.ZipException: error in opening zip file
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:203)
        at java.util.jar.JarFile.<init>(JarFile.java:132)
        at java.util.jar.JarFile.<init>(JarFile.java:97)
        at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1549)
        at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1771)
        at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1575)
        at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:860)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1307)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1189)
        at java.beans.Introspector.instantiate(Introspector.java:1453)
        at java.beans.Introspector.findExplicitBeanInfo(Introspector.java:410)
        at java.beans.Introspector.<init>(Introspector.java:359)
        at java.beans.Introspector.getBeanInfo(Introspector.java:159)
        at java.beans.Introspector.getBeanInfo(Introspector.java:220)
        at java.beans.Introspector.<init>(Introspector.java:368)
        at java.beans.Introspector.getBeanInfo(Introspector.java:159)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:896)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:926)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:801)
        at org.apache.commons.beanutils.PropertyUtilsBean.getSimpleProperty(PropertyUtilsBean.java:1119)
        at org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:686)
        at org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:715)
        at org.apache.commons.beanutils.PropertyUtils.getProperty(PropertyUtils.java:290)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:255)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
        at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
        at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:595)

Open in new window

0
 
LVL 2

Accepted Solution

by:
Airyck666 earned 100 total points
ID: 21796267
here... I rewote it slightly to double check that all streams are being closed.  There were a couple not being closed.  Try this.
    String chgT3Prio(HttpServletRequest request, String prio, int seq, int t1cnt) {
 
        String filePath = "/some/path/myfile1";
        FileLock lock1 = null;
        FileChannel channel1 = null;
        RandomAccessFile raf1 = null;
 
        BufferedReader reportFile = null;
        FileOutputStream out = null;
        FileInputStream fileInputStream = null;
        InputStreamReader inputStreamReader = null;
        
        PrintStream p = null;
 
 
        try {
            String tmpFileName = "/some/path/myfile1.tmp";
            
            raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();
 
            out = new FileOutputStream(tmpFileName, true);
            p = new PrintStream(out);
            fileInputStream = new FileInputStream(filePath);
            inputStreamReader = new InputStreamReader(fileInputStream); 
 
            // create report file
            reportFile = new BufferedReader(inputStreamReader);
            
            
            File tmpFile = new File(tmpFileName);
            if (tmpFile.exists()) {
                tmpFile.delete();
            }
 
            
 
            String line = null;
            int count = 0;
            // ////////////////////////////////////////
            // 1/2. change priority of target
            // ////////////////////////////////////////
            while ((line = reportFile.readLine()) != null) {
                StringTokenizer line_data = new StringTokenizer(line, "|", false);
                String ftype = line_data.nextToken();
 
                if (ftype.equals("T3"))
                    count++;
 
                if (count == seq) {
                    String loc_a = line_data.nextToken();
                    String loc_z = line_data.nextToken();
                    String fac_type = line_data.nextToken();
                    String fac_num = line_data.nextToken();
                    String mile = line_data.nextToken();
                    String ucr = line_data.nextToken();
                    String user = line_data.nextToken();
                    String sub_dt = line_data.nextToken();
                    String sub_type = line_data.nextToken();
                    line_data.nextToken();
                    String long_time = line_data.nextToken();
                    String trailer = line_data.nextToken();
 
                    String newT3 = "T3|" + loc_a + "|" + loc_z + "|" + fac_type + "|" + fac_num + "|" + mile + "|" + ucr + "|"
                            + user + "|" + sub_dt + "|" + sub_type + "|";
 
                    if (prio.equals("Normal")) {
                        newT3 += "High";
                    } else if (prio.equals("High")) {
                        newT3 += "Normal";
                    }
                    newT3 += "|" + long_time;
                    newT3 += "|" + trailer;
 
                    try {
                        p.println(newT3);
                    } catch (Exception e) {
                        System.err.println("Error writing to log");
                    }
                    count = -32767;
 
                } else {
                    try {
                        p.println(line);
                    } catch (Exception e) {
                        System.err.println("Error writing to log");
                    }
                }
            }
 
        } catch (FileNotFoundException fe) {
            System.out.println(fe);
        } catch (IOException e1) {
            System.out.println(e1);
        } finally {
            p.close();
            
                try {
                    // close all connections, etc
                    lock1.release();
                    channel1.close();
                    raf1.close();
                    reportFile.close();
                    out.close();
                    fileInputStream.close();
                    inputStreamReader.close();
                    
                } catch (IOException e) {
                    e.printStackTrace();
                }
        }
    }

Open in new window

0
 
LVL 2

Expert Comment

by:Airyck666
ID: 21796436
Oh... and if that doesn't work, try using the lsof command to see what files are open.
http://linux.about.com/library/cmd/blcmdl8_lsof.htm
0
 

Author Comment

by:Ade101
ID: 21805561
Airyck666,

Thanks a lot. The problem seemed to be resolved now. i wasnt closing the RandomFileAccess and FileChannel objects (I thought their instance donest actually open the real file on OS just like the File object). Moving everything to the finally block solved the problem and now when i play around with the pages which opens and closes the files, the command 'cat /proc/sys/fs/file-nr' on linux server shows that the all files get closed.

Many thanks.
0

Featured Post

Transaction Monitoring Vs. Real User Monitoring

Synthetic Transaction Monitoring Vs. Real User Monitoring: When To Use Each Approach? In this article, we will discuss two major monitoring approaches: Synthetic Transaction and Real User Monitoring.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
Viewers learn about the “for” loop and how it works in Java. By comparing it to the while loop learned before, viewers can make the transition easily. You will learn about the formatting of the for loop as we write a program that prints even numbers…
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

690 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question