?
Solved

JSP (too may open files error)

Posted on 2008-06-13
9
Medium Priority
?
658 Views
Last Modified: 2013-11-23
Hi,

I have a large JSP page and i am using tomcat 5.0 with JDK 1.5_15 release on a linux server.

The JSP code opens the files , writes into them and then closes them. For a stress test if i keep using this opage back and forth during the day, i get the dreaded error thrown to me saying (too many open files). and then nothing works after that unless i restart tomcat server.

I believe there is a way to increase the max open files on unix server but i really want to fix it in my code. I am pasting the sample code which is how the file opening and closing is done in almost all the methods of this page. Please see if you can spot something wrong which is not letting the files close properly etc ?
===========
    String chgT3Prio (HttpServletRequest request, String prio, int seq, int t1cnt)
    {

            
      int count = 0;
      String filePath = "/some/path/myfile1";
      FileLock lock1 = null;
      FileChannel channel1 = null;
      
      BufferedReader reportFile = null;
      FileOutputStream out = null;
    PrintStream p = null;

      try {
          String tmpFileName = "/some/path/myfile1.tmp";
            RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();

          try
          {
              //--FileOutputStream out;
              //FileOutputStream out = null;
              //PrintStream p = null;
              
              String line = null;
   
              File tmpFile  = new File (tmpFileName);
              if (tmpFile.exists()) {
                  tmpFile.delete();
              }

              out = new FileOutputStream(tmpFileName, true);
              p = new PrintStream(out);

              reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));

              //////////////////////////////////////////
              // 1/2. change priority of target
              //////////////////////////////////////////
              while((line = reportFile.readLine())!=null) {
                StringTokenizer line_data = new StringTokenizer(line, "|", false);
                String ftype = line_data.nextToken();
   
                if (ftype.equals("T3"))
                      count++;
   
                if (count == seq) {
                    String loc_a    = line_data.nextToken();
                    String loc_z    = line_data.nextToken();
                    String fac_type = line_data.nextToken();
                    String fac_num  = line_data.nextToken();
                    String mile     = line_data.nextToken();
                    String ucr      = line_data.nextToken();
                    String user     = line_data.nextToken();
                    String sub_dt   = line_data.nextToken();
                    String sub_type = line_data.nextToken();
                    line_data.nextToken();
                    String long_time = line_data.nextToken();
                    String trailer = line_data.nextToken();
   
                    String newT3 = "T3|" + loc_a  + "|" + loc_z  + "|" + fac_type  + "|" + fac_num  + "|" + mile  +
                                   "|" + ucr  + "|" + user  + "|" + sub_dt  + "|" + sub_type + "|";
   
                    if (prio.equals("Normal")) {
                        newT3 += "High";
                    }else if (prio.equals("High")) {
                        newT3 += "Normal";
                    }
                    newT3 += "|" + long_time;
                    newT3 += "|" + trailer;
   
                    try {
                      p.println (newT3);
                    } catch (Exception e) {
                      System.err.println ("Error writing to log");
                    }
                    count = -32767;
                } else {
                    try {
                      p.println (line);
                    } catch (Exception e) {
                      System.err.println ("Error writing to log");
                    }
                  }
              }
              
                p.close();
              reportFile.close();

              
          } catch (FileNotFoundException f) {
              System.out.println(f);
          } catch (IOException e) {
              System.out.println(e);
          }
          
      } catch (FileNotFoundException fe) {
          System.out.println(fe);
      } catch (IOException e1) {
          System.out.println(e1);
      } finally {
          try {
              lock1.release();                        
            
          } catch (IOException e3) {
          }
      }




String chgT3Prio (HttpServletRequest request, String prio, int seq, int t1cnt)
    {
 
	       
	int count = 0;
	String filePath = "/some/path/myfile1";
	FileLock lock1 = null;
	FileChannel channel1 = null;
	
	BufferedReader reportFile = null;
	FileOutputStream out = null;
    PrintStream p = null; 
 
	try {
	    String tmpFileName = "/some/path/myfile1.tmp";
            RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();
 
	    try 
	    {
	        //--FileOutputStream out;
	        //FileOutputStream out = null;
	        //PrintStream p = null; 
	        
	        String line = null;
    
	        File tmpFile  = new File (tmpFileName);
	        if (tmpFile.exists()) {
	            tmpFile.delete();
	        }
 
	        out = new FileOutputStream(tmpFileName, true);
	        p = new PrintStream(out);
 
	        reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));
 
	        //////////////////////////////////////////
	        // 1/2. change priority of target
	        //////////////////////////////////////////
	        while((line = reportFile.readLine())!=null) {
		    StringTokenizer line_data = new StringTokenizer(line, "|", false);
		    String ftype = line_data.nextToken();
    
		    if (ftype.equals("T3"))
	                count++;
    
		    if (count == seq) {
		        String loc_a    = line_data.nextToken();
		        String loc_z    = line_data.nextToken();
		        String fac_type = line_data.nextToken();
		        String fac_num  = line_data.nextToken();
		        String mile     = line_data.nextToken();
		        String ucr      = line_data.nextToken();
		        String user     = line_data.nextToken();
		        String sub_dt   = line_data.nextToken();
		        String sub_type = line_data.nextToken();
		        line_data.nextToken();
		        String long_time = line_data.nextToken();
		        String trailer = line_data.nextToken();
    
		        String newT3 = "T3|" + loc_a  + "|" + loc_z  + "|" + fac_type  + "|" + fac_num  + "|" + mile  + 
		    		       "|" + ucr  + "|" + user  + "|" + sub_dt  + "|" + sub_type + "|";
    
		        if (prio.equals("Normal")) {
		            newT3 += "High";
		        }else if (prio.equals("High")) {
		            newT3 += "Normal";
		        }
		        newT3 += "|" + long_time;
		        newT3 += "|" + trailer;
    
		        try {
			    p.println (newT3);
		        } catch (Exception e) {
			    System.err.println ("Error writing to log");
		        }
		        count = -32767;
		    } else {
		        try {
			    p.println (line);
		        } catch (Exception e) {
			    System.err.println ("Error writing to log");
		        }
	            }
	        }
	        
                p.close();
	        reportFile.close();
 
	         
	    } catch (FileNotFoundException f) {
	        System.out.println(f);
	    } catch (IOException e) {
	        System.out.println(e);
	    } 
	    
	} catch (FileNotFoundException fe) {
	    System.out.println(fe);
	} catch (IOException e1) {
	    System.out.println(e1);
	} finally {
	    try {
	        lock1.release();	     	        
		
	    } catch (IOException e3) {
	    }
	}

Open in new window

0
Comment
Question by:Ade101
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
9 Comments
 
LVL 2

Expert Comment

by:Airyck666
ID: 21783649
Try closing the FileOutputStream out.


p.close();
reportFile.close();
out.close();

Open in new window

0
 

Author Comment

by:Ade101
ID: 21789122
I'd tried this too but no help :/

For locking the file is this the right code ?

 RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();

if yes then still we dont use the raf1 anywhere other than just initializing it and locking it. For read and write we use FileOutPutStream and PrintStream instead. Should I be closing raf1 also?

I am new to this.. Thnaks.
0
 
LVL 2

Expert Comment

by:Airyck666
ID: 21789167
Are you seeing any exceptions when you run it?
0
Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

 
LVL 2

Expert Comment

by:Airyck666
ID: 21789213
The file locking looks okay.
A couple of things you could try:

Try seperating this definition and explicitly closing the FileInputStream.
reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));
see Code snippet

Another thing you can try is to close all your connections in your finally block to ensure these resources are released if there is a problem.

And the catch block at line 92-96 can be removed (along with the corresponding try) as your're catching these exceptions below.
FileInputStream fis = new FileInputStream(filePath);
reportFile = new BufferedReader(new InputStreamReader(fis));
// And then explicitly close this connection in the finally block

Open in new window

0
 

Author Comment

by:Ade101
ID: 21789262
The error I am receiving is:

Java.io.FileNotFoundException: myfilesname.txt (Too many open files)

Once this error is thrown no further file can be opened through JSP unless i restart the web server.
0
 

Author Comment

by:Ade101
ID: 21795680
I tried your suggestions i.e. seperating the FileInputString defination and closing eveything in the finally block too but after regression testing the page, i get the error again and for a resolution i have to restarted the web server which is even more annoying.

Can you please point out what can be the reason?

Thanks a lot.
handleDate() called ...handleDate() called ...java.io.FileNotFoundException: /home/mytestdata/MyTestFile (Too many open files)
WebappClassLoader: Failed to open JAR
java.util.zip.ZipException: error in opening zip file
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:203)
        at java.util.jar.JarFile.<init>(JarFile.java:132)
        at java.util.jar.JarFile.<init>(JarFile.java:97)
        at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1549)
        at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1771)
        at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1575)
        at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:860)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1307)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1189)
        at java.beans.Introspector.instantiate(Introspector.java:1453)
        at java.beans.Introspector.findExplicitBeanInfo(Introspector.java:410)
        at java.beans.Introspector.<init>(Introspector.java:359)
        at java.beans.Introspector.getBeanInfo(Introspector.java:159)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:896)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:926)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:801)
        at org.apache.commons.beanutils.PropertyUtilsBean.getSimpleProperty(PropertyUtilsBean.java:1119)
        at org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:686)
        at org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:715)
        at org.apache.commons.beanutils.PropertyUtils.getProperty(PropertyUtils.java:290)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:255)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
        at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
        at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:595)
WebappClassLoader: Failed to open JAR
java.util.zip.ZipException: error in opening zip file
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:203)
        at java.util.jar.JarFile.<init>(JarFile.java:132)
        at java.util.jar.JarFile.<init>(JarFile.java:97)
        at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1549)
        at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1771)
        at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1575)
        at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:860)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1307)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1189)
        at java.beans.Introspector.instantiate(Introspector.java:1453)
        at java.beans.Introspector.findExplicitBeanInfo(Introspector.java:410)
        at java.beans.Introspector.<init>(Introspector.java:359)
        at java.beans.Introspector.getBeanInfo(Introspector.java:159)
        at java.beans.Introspector.getBeanInfo(Introspector.java:220)
        at java.beans.Introspector.<init>(Introspector.java:368)
        at java.beans.Introspector.getBeanInfo(Introspector.java:159)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:896)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:926)
        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:801)
        at org.apache.commons.beanutils.PropertyUtilsBean.getSimpleProperty(PropertyUtilsBean.java:1119)
        at org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:686)
        at org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:715)
        at org.apache.commons.beanutils.PropertyUtils.getProperty(PropertyUtils.java:290)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:255)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
        at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
        at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:595)

Open in new window

0
 
LVL 2

Accepted Solution

by:
Airyck666 earned 400 total points
ID: 21796267
here... I rewote it slightly to double check that all streams are being closed.  There were a couple not being closed.  Try this.
    String chgT3Prio(HttpServletRequest request, String prio, int seq, int t1cnt) {
 
        String filePath = "/some/path/myfile1";
        FileLock lock1 = null;
        FileChannel channel1 = null;
        RandomAccessFile raf1 = null;
 
        BufferedReader reportFile = null;
        FileOutputStream out = null;
        FileInputStream fileInputStream = null;
        InputStreamReader inputStreamReader = null;
        
        PrintStream p = null;
 
 
        try {
            String tmpFileName = "/some/path/myfile1.tmp";
            
            raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();
 
            out = new FileOutputStream(tmpFileName, true);
            p = new PrintStream(out);
            fileInputStream = new FileInputStream(filePath);
            inputStreamReader = new InputStreamReader(fileInputStream); 
 
            // create report file
            reportFile = new BufferedReader(inputStreamReader);
            
            
            File tmpFile = new File(tmpFileName);
            if (tmpFile.exists()) {
                tmpFile.delete();
            }
 
            
 
            String line = null;
            int count = 0;
            // ////////////////////////////////////////
            // 1/2. change priority of target
            // ////////////////////////////////////////
            while ((line = reportFile.readLine()) != null) {
                StringTokenizer line_data = new StringTokenizer(line, "|", false);
                String ftype = line_data.nextToken();
 
                if (ftype.equals("T3"))
                    count++;
 
                if (count == seq) {
                    String loc_a = line_data.nextToken();
                    String loc_z = line_data.nextToken();
                    String fac_type = line_data.nextToken();
                    String fac_num = line_data.nextToken();
                    String mile = line_data.nextToken();
                    String ucr = line_data.nextToken();
                    String user = line_data.nextToken();
                    String sub_dt = line_data.nextToken();
                    String sub_type = line_data.nextToken();
                    line_data.nextToken();
                    String long_time = line_data.nextToken();
                    String trailer = line_data.nextToken();
 
                    String newT3 = "T3|" + loc_a + "|" + loc_z + "|" + fac_type + "|" + fac_num + "|" + mile + "|" + ucr + "|"
                            + user + "|" + sub_dt + "|" + sub_type + "|";
 
                    if (prio.equals("Normal")) {
                        newT3 += "High";
                    } else if (prio.equals("High")) {
                        newT3 += "Normal";
                    }
                    newT3 += "|" + long_time;
                    newT3 += "|" + trailer;
 
                    try {
                        p.println(newT3);
                    } catch (Exception e) {
                        System.err.println("Error writing to log");
                    }
                    count = -32767;
 
                } else {
                    try {
                        p.println(line);
                    } catch (Exception e) {
                        System.err.println("Error writing to log");
                    }
                }
            }
 
        } catch (FileNotFoundException fe) {
            System.out.println(fe);
        } catch (IOException e1) {
            System.out.println(e1);
        } finally {
            p.close();
            
                try {
                    // close all connections, etc
                    lock1.release();
                    channel1.close();
                    raf1.close();
                    reportFile.close();
                    out.close();
                    fileInputStream.close();
                    inputStreamReader.close();
                    
                } catch (IOException e) {
                    e.printStackTrace();
                }
        }
    }

Open in new window

0
 
LVL 2

Expert Comment

by:Airyck666
ID: 21796436
Oh... and if that doesn't work, try using the lsof command to see what files are open.
http://linux.about.com/library/cmd/blcmdl8_lsof.htm
0
 

Author Comment

by:Ade101
ID: 21805561
Airyck666,

Thanks a lot. The problem seemed to be resolved now. i wasnt closing the RandomFileAccess and FileChannel objects (I thought their instance donest actually open the real file on OS just like the File object). Moving everything to the finally block solved the problem and now when i play around with the pages which opens and closes the files, the command 'cat /proc/sys/fs/file-nr' on linux server shows that the all files get closed.

Many thanks.
0

Featured Post

How to Create Failover DNS Record Sets in Route 53

Route 53 has the ability to easily configure DNS record sets specifically for failover scenarios. These failover record sets can be configured to failover to full-blown deployments in other regions or to a static HTML page that informs your customers of the issue.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This was posted to the Netbeans forum a Feb, 2010 and I also sent it to Verisign. Who didn't help much in my struggles to get my application signed. ------------------------- Start The idea here is to target your cell phones with the correct…
Introduction This article is the second of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers the basic installation and configuration of the test automation tools used by…
Viewers will learn about the different types of variables in Java and how to declare them. Decide the type of variable desired: Put the keyword corresponding to the type of variable in front of the variable name: Use the equal sign to assign a v…
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.
Suggested Courses
Course of the Month9 days, 23 hours left to enroll

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question