Solved

JSP (too may open files error)

Posted on 2008-06-13
9
639 Views
Last Modified: 2013-11-23
Hi,

I have a large JSP page and i am using tomcat 5.0 with JDK 1.5_15 release on a linux server.

The JSP code opens the files , writes into them and then closes them. For a stress test if i keep using this opage back and forth during the day, i get the dreaded error thrown to me saying (too many open files). and then nothing works after that unless i restart tomcat server.

I believe there is a way to increase the max open files on unix server but i really want to fix it in my code. I am pasting the sample code which is how the file opening and closing is done in almost all the methods of this page. Please see if you can spot something wrong which is not letting the files close properly etc ?
===========
    String chgT3Prio (HttpServletRequest request, String prio, int seq, int t1cnt)
    {

            
      int count = 0;
      String filePath = "/some/path/myfile1";
      FileLock lock1 = null;
      FileChannel channel1 = null;
      
      BufferedReader reportFile = null;
      FileOutputStream out = null;
    PrintStream p = null;

      try {
          String tmpFileName = "/some/path/myfile1.tmp";
            RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();

          try
          {
              //--FileOutputStream out;
              //FileOutputStream out = null;
              //PrintStream p = null;
              
              String line = null;
   
              File tmpFile  = new File (tmpFileName);
              if (tmpFile.exists()) {
                  tmpFile.delete();
              }

              out = new FileOutputStream(tmpFileName, true);
              p = new PrintStream(out);

              reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));

              //////////////////////////////////////////
              // 1/2. change priority of target
              //////////////////////////////////////////
              while((line = reportFile.readLine())!=null) {
                StringTokenizer line_data = new StringTokenizer(line, "|", false);
                String ftype = line_data.nextToken();
   
                if (ftype.equals("T3"))
                      count++;
   
                if (count == seq) {
                    String loc_a    = line_data.nextToken();
                    String loc_z    = line_data.nextToken();
                    String fac_type = line_data.nextToken();
                    String fac_num  = line_data.nextToken();
                    String mile     = line_data.nextToken();
                    String ucr      = line_data.nextToken();
                    String user     = line_data.nextToken();
                    String sub_dt   = line_data.nextToken();
                    String sub_type = line_data.nextToken();
                    line_data.nextToken();
                    String long_time = line_data.nextToken();
                    String trailer = line_data.nextToken();
   
                    String newT3 = "T3|" + loc_a  + "|" + loc_z  + "|" + fac_type  + "|" + fac_num  + "|" + mile  +
                                   "|" + ucr  + "|" + user  + "|" + sub_dt  + "|" + sub_type + "|";
   
                    if (prio.equals("Normal")) {
                        newT3 += "High";
                    }else if (prio.equals("High")) {
                        newT3 += "Normal";
                    }
                    newT3 += "|" + long_time;
                    newT3 += "|" + trailer;
   
                    try {
                      p.println (newT3);
                    } catch (Exception e) {
                      System.err.println ("Error writing to log");
                    }
                    count = -32767;
                } else {
                    try {
                      p.println (line);
                    } catch (Exception e) {
                      System.err.println ("Error writing to log");
                    }
                  }
              }
              
                p.close();
              reportFile.close();

              
          } catch (FileNotFoundException f) {
              System.out.println(f);
          } catch (IOException e) {
              System.out.println(e);
          }
          
      } catch (FileNotFoundException fe) {
          System.out.println(fe);
      } catch (IOException e1) {
          System.out.println(e1);
      } finally {
          try {
              lock1.release();                        
            
          } catch (IOException e3) {
          }
      }




String chgT3Prio (HttpServletRequest request, String prio, int seq, int t1cnt)

    {
 

	       

	int count = 0;

	String filePath = "/some/path/myfile1";

	FileLock lock1 = null;

	FileChannel channel1 = null;

	

	BufferedReader reportFile = null;

	FileOutputStream out = null;

    PrintStream p = null; 
 

	try {

	    String tmpFileName = "/some/path/myfile1.tmp";

            RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");

            channel1 = raf1.getChannel();

            lock1 = channel1.lock();
 

	    try 

	    {

	        //--FileOutputStream out;

	        //FileOutputStream out = null;

	        //PrintStream p = null; 

	        

	        String line = null;

    

	        File tmpFile  = new File (tmpFileName);

	        if (tmpFile.exists()) {

	            tmpFile.delete();

	        }
 

	        out = new FileOutputStream(tmpFileName, true);

	        p = new PrintStream(out);
 

	        reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));
 

	        //////////////////////////////////////////

	        // 1/2. change priority of target

	        //////////////////////////////////////////

	        while((line = reportFile.readLine())!=null) {

		    StringTokenizer line_data = new StringTokenizer(line, "|", false);

		    String ftype = line_data.nextToken();

    

		    if (ftype.equals("T3"))

	                count++;

    

		    if (count == seq) {

		        String loc_a    = line_data.nextToken();

		        String loc_z    = line_data.nextToken();

		        String fac_type = line_data.nextToken();

		        String fac_num  = line_data.nextToken();

		        String mile     = line_data.nextToken();

		        String ucr      = line_data.nextToken();

		        String user     = line_data.nextToken();

		        String sub_dt   = line_data.nextToken();

		        String sub_type = line_data.nextToken();

		        line_data.nextToken();

		        String long_time = line_data.nextToken();

		        String trailer = line_data.nextToken();

    

		        String newT3 = "T3|" + loc_a  + "|" + loc_z  + "|" + fac_type  + "|" + fac_num  + "|" + mile  + 

		    		       "|" + ucr  + "|" + user  + "|" + sub_dt  + "|" + sub_type + "|";

    

		        if (prio.equals("Normal")) {

		            newT3 += "High";

		        }else if (prio.equals("High")) {

		            newT3 += "Normal";

		        }

		        newT3 += "|" + long_time;

		        newT3 += "|" + trailer;

    

		        try {

			    p.println (newT3);

		        } catch (Exception e) {

			    System.err.println ("Error writing to log");

		        }

		        count = -32767;

		    } else {

		        try {

			    p.println (line);

		        } catch (Exception e) {

			    System.err.println ("Error writing to log");

		        }

	            }

	        }

	        

                p.close();

	        reportFile.close();
 

	         

	    } catch (FileNotFoundException f) {

	        System.out.println(f);

	    } catch (IOException e) {

	        System.out.println(e);

	    } 

	    

	} catch (FileNotFoundException fe) {

	    System.out.println(fe);

	} catch (IOException e1) {

	    System.out.println(e1);

	} finally {

	    try {

	        lock1.release();	     	        

		

	    } catch (IOException e3) {

	    }

	}

Open in new window

0
Comment
Question by:Ade101
  • 5
  • 4
9 Comments
 
LVL 2

Expert Comment

by:Airyck666
ID: 21783649
Try closing the FileOutputStream out.


p.close();

reportFile.close();

out.close();

Open in new window

0
 

Author Comment

by:Ade101
ID: 21789122
I'd tried this too but no help :/

For locking the file is this the right code ?

 RandomAccessFile raf1 = new RandomAccessFile(filePath, "rw");
            channel1 = raf1.getChannel();
            lock1 = channel1.lock();

if yes then still we dont use the raf1 anywhere other than just initializing it and locking it. For read and write we use FileOutPutStream and PrintStream instead. Should I be closing raf1 also?

I am new to this.. Thnaks.
0
 
LVL 2

Expert Comment

by:Airyck666
ID: 21789167
Are you seeing any exceptions when you run it?
0
 
LVL 2

Expert Comment

by:Airyck666
ID: 21789213
The file locking looks okay.
A couple of things you could try:

Try seperating this definition and explicitly closing the FileInputStream.
reportFile = new BufferedReader(new InputStreamReader(new FileInputStream(filePath)));
see Code snippet

Another thing you can try is to close all your connections in your finally block to ensure these resources are released if there is a problem.

And the catch block at line 92-96 can be removed (along with the corresponding try) as your're catching these exceptions below.
FileInputStream fis = new FileInputStream(filePath);

reportFile = new BufferedReader(new InputStreamReader(fis));

// And then explicitly close this connection in the finally block

Open in new window

0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 

Author Comment

by:Ade101
ID: 21789262
The error I am receiving is:

Java.io.FileNotFoundException: myfilesname.txt (Too many open files)

Once this error is thrown no further file can be opened through JSP unless i restart the web server.
0
 

Author Comment

by:Ade101
ID: 21795680
I tried your suggestions i.e. seperating the FileInputString defination and closing eveything in the finally block too but after regression testing the page, i get the error again and for a resolution i have to restarted the web server which is even more annoying.

Can you please point out what can be the reason?

Thanks a lot.
handleDate() called ...handleDate() called ...java.io.FileNotFoundException: /home/mytestdata/MyTestFile (Too many open files)

WebappClassLoader: Failed to open JAR

java.util.zip.ZipException: error in opening zip file

        at java.util.zip.ZipFile.open(Native Method)

        at java.util.zip.ZipFile.<init>(ZipFile.java:203)

        at java.util.jar.JarFile.<init>(JarFile.java:132)

        at java.util.jar.JarFile.<init>(JarFile.java:97)

        at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1549)

        at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1771)

        at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1575)

        at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:860)

        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1307)

        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1189)

        at java.beans.Introspector.instantiate(Introspector.java:1453)

        at java.beans.Introspector.findExplicitBeanInfo(Introspector.java:410)

        at java.beans.Introspector.<init>(Introspector.java:359)

        at java.beans.Introspector.getBeanInfo(Introspector.java:159)

        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:896)

        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:926)

        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:801)

        at org.apache.commons.beanutils.PropertyUtilsBean.getSimpleProperty(PropertyUtilsBean.java:1119)

        at org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:686)

        at org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:715)

        at org.apache.commons.beanutils.PropertyUtils.getProperty(PropertyUtils.java:290)

        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:255)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)

        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)

        at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)

        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)

        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)

        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)

        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)

        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)

        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)

        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)

        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)

        at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)

        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)

        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)

        at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)

        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)

        at java.lang.Thread.run(Thread.java:595)

WebappClassLoader: Failed to open JAR

java.util.zip.ZipException: error in opening zip file

        at java.util.zip.ZipFile.open(Native Method)

        at java.util.zip.ZipFile.<init>(ZipFile.java:203)

        at java.util.jar.JarFile.<init>(JarFile.java:132)

        at java.util.jar.JarFile.<init>(JarFile.java:97)

        at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1549)

        at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1771)

        at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1575)

        at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:860)

        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1307)

        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1189)

        at java.beans.Introspector.instantiate(Introspector.java:1453)

        at java.beans.Introspector.findExplicitBeanInfo(Introspector.java:410)

        at java.beans.Introspector.<init>(Introspector.java:359)

        at java.beans.Introspector.getBeanInfo(Introspector.java:159)

        at java.beans.Introspector.getBeanInfo(Introspector.java:220)

        at java.beans.Introspector.<init>(Introspector.java:368)

        at java.beans.Introspector.getBeanInfo(Introspector.java:159)

        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:896)

        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptors(PropertyUtilsBean.java:926)

        at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:801)

        at org.apache.commons.beanutils.PropertyUtilsBean.getSimpleProperty(PropertyUtilsBean.java:1119)

        at org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:686)

        at org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:715)

        at org.apache.commons.beanutils.PropertyUtils.getProperty(PropertyUtils.java:290)

        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:255)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)

        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)

        at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)

        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)

        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)

        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)

        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)

        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)

        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

        at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)

        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)

        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)

        at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)

        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)

        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)

        at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)

        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)

        at java.lang.Thread.run(Thread.java:595)

Open in new window

0
 
LVL 2

Accepted Solution

by:
Airyck666 earned 100 total points
ID: 21796267
here... I rewote it slightly to double check that all streams are being closed.  There were a couple not being closed.  Try this.
    String chgT3Prio(HttpServletRequest request, String prio, int seq, int t1cnt) {
 

        String filePath = "/some/path/myfile1";

        FileLock lock1 = null;

        FileChannel channel1 = null;

        RandomAccessFile raf1 = null;
 

        BufferedReader reportFile = null;

        FileOutputStream out = null;

        FileInputStream fileInputStream = null;

        InputStreamReader inputStreamReader = null;

        

        PrintStream p = null;
 
 

        try {

            String tmpFileName = "/some/path/myfile1.tmp";

            

            raf1 = new RandomAccessFile(filePath, "rw");

            channel1 = raf1.getChannel();

            lock1 = channel1.lock();
 

            out = new FileOutputStream(tmpFileName, true);

            p = new PrintStream(out);

            fileInputStream = new FileInputStream(filePath);

            inputStreamReader = new InputStreamReader(fileInputStream); 
 

            // create report file

            reportFile = new BufferedReader(inputStreamReader);

            

            

            File tmpFile = new File(tmpFileName);

            if (tmpFile.exists()) {

                tmpFile.delete();

            }
 

            
 

            String line = null;

            int count = 0;

            // ////////////////////////////////////////

            // 1/2. change priority of target

            // ////////////////////////////////////////

            while ((line = reportFile.readLine()) != null) {

                StringTokenizer line_data = new StringTokenizer(line, "|", false);

                String ftype = line_data.nextToken();
 

                if (ftype.equals("T3"))

                    count++;
 

                if (count == seq) {

                    String loc_a = line_data.nextToken();

                    String loc_z = line_data.nextToken();

                    String fac_type = line_data.nextToken();

                    String fac_num = line_data.nextToken();

                    String mile = line_data.nextToken();

                    String ucr = line_data.nextToken();

                    String user = line_data.nextToken();

                    String sub_dt = line_data.nextToken();

                    String sub_type = line_data.nextToken();

                    line_data.nextToken();

                    String long_time = line_data.nextToken();

                    String trailer = line_data.nextToken();
 

                    String newT3 = "T3|" + loc_a + "|" + loc_z + "|" + fac_type + "|" + fac_num + "|" + mile + "|" + ucr + "|"

                            + user + "|" + sub_dt + "|" + sub_type + "|";
 

                    if (prio.equals("Normal")) {

                        newT3 += "High";

                    } else if (prio.equals("High")) {

                        newT3 += "Normal";

                    }

                    newT3 += "|" + long_time;

                    newT3 += "|" + trailer;
 

                    try {

                        p.println(newT3);

                    } catch (Exception e) {

                        System.err.println("Error writing to log");

                    }

                    count = -32767;
 

                } else {

                    try {

                        p.println(line);

                    } catch (Exception e) {

                        System.err.println("Error writing to log");

                    }

                }

            }
 

        } catch (FileNotFoundException fe) {

            System.out.println(fe);

        } catch (IOException e1) {

            System.out.println(e1);

        } finally {

            p.close();

            

                try {

                    // close all connections, etc

                    lock1.release();

                    channel1.close();

                    raf1.close();

                    reportFile.close();

                    out.close();

                    fileInputStream.close();

                    inputStreamReader.close();

                    

                } catch (IOException e) {

                    e.printStackTrace();

                }

        }

    }

Open in new window

0
 
LVL 2

Expert Comment

by:Airyck666
ID: 21796436
Oh... and if that doesn't work, try using the lsof command to see what files are open.
http://linux.about.com/library/cmd/blcmdl8_lsof.htm
0
 

Author Comment

by:Ade101
ID: 21805561
Airyck666,

Thanks a lot. The problem seemed to be resolved now. i wasnt closing the RandomFileAccess and FileChannel objects (I thought their instance donest actually open the real file on OS just like the File object). Moving everything to the finally block solved the problem and now when i play around with the pages which opens and closes the files, the command 'cat /proc/sys/fs/file-nr' on linux server shows that the all files get closed.

Many thanks.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

I had a project requirement for a displaying a user workbench .This workbench would consist multiple data grids .In each grid the user will be able to see a large number of data. These data grids should allow the user to 1. Sort 2. Export the …
In this post we will learn how to connect and configure Android Device (Smartphone etc.) with Android Studio. After that we will run a simple Hello World Program.
Viewers learn about the “while” loop and how to utilize it correctly in Java. Additionally, viewers begin exploring how to include conditional statements within a while loop and avoid an endless loop. Define While Loop: Basic Example: Explanatio…
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now