Java - MultiThreading and exception

Hi Experts,

Would appreciate help with this question pls.
I have another Question as below:

http://www.experts-exchange.com/Programming/Programming_Languages/Java/Q_21409857.html

related to this, I am having couple of problems while trying to test suggestions given in above question by you experts and so I thought I will open a new question so keep it in separate thread.

While trying to test this out with many threads (30 threads) or 50 threads.. I am getting exception as below:


java.sql.SQLException: JZ006: Caught IOException: java.net.SocketException: Too
many open files
        at java.lang.Throwable.fillInStackTrace(Native Method)
        at java.lang.Throwable.fillInStackTrace(Compiled Code)
        at java.lang.Throwable.<init>(Compiled Code)
        at java.lang.Exception.<init>(Compiled Code)
        at java.sql.SQLException.<init>(SQLException.java:64)
        at com.sybase.jdbc2.jdbc.ErrorMessage.raiseError(ErrorMessage.java:423)
        at com.sybase.jdbc2.tds.Tds.handleIOE(Compiled Code)
        at com.sybase.jdbc2.tds.Tds.login(Compiled Code)
        at com.sybase.jdbc2.jdbc.SybConnection.tryLogin(Compiled Code)
        at com.sybase.jdbc2.jdbc.SybConnection.regularConnect(Compiled Code)
        at com.sybase.jdbc2.jdbc.SybConnection.<init>(Compiled Code)
        at com.sybase.jdbc2.jdbc.SybConnection.<init>(Compiled Code)
        at com.sybase.jdbc2.jdbc.SybDriver.createConnection(Compiled Code)
        at com.sybase.jdbc2.jdbc.SybDriver.connect(Compiled Code)
        at com.sybase.jdbc2.jdbc.SybDriver.connect(Compiled Code)
        at com.sybase.jdbc2.jdbc.SybDataSource.getConnection(Compiled Code)
        at com.citigroup.cpacs.util.DB$1.createObject(Compiled Code)
        at com.citi.cititech.eae.util.object.ObjectPoolImpl.addObject(Compiled C
ode)
        at com.citi.cititech.eae.util.object.ObjectPoolImpl.getPooledObject(Comp
iled Code)
        at com.citigroup.cpacs.util.DB.getConnection(Compiled Code)
        at com.citigroup.cpacs.mapper.sybase.AbstractMapper.abstractFind(Compile
d Code)
        at com.citigroup.cpacs.mapper.sybase.GeneralProfileMapperImpl.find(Compi
led Code)
        at com.citigroup.cpacs.ClientProfileService.getGeneralProfile(Compiled C
ode)
        at com.citigroup.cpacs.ccs.ChargeCalculationService.applyCharges(Compile
d Code)
        at com.citigroup.cpacs.ccs.ChargeCalculationService.doCalculation(Compil
ed Code)
        at com.citigroup.cpacs.test.TestDriver.testCalculation(Compiled Code)
        at com.citigroup.cpacs.test.ThreadCpacsCore.run(ThreadCpacsCore.java:35


any ideas??

This happens when I raise conn pool to 200 conn's..

does this mean DB cannot accept that many conns.?

If I lower the limit, I run out of conn pool and it reaches the limit.

any ideas how can I test this out..???

narravAsked:
Who is Participating?
 
TimYatesCommented:
Errr... you're not closing your preparedStatement either...

Id put all 3 (rs, findStatement, and conn) in a finally block like:

     protected Object abstractFind(Object key) throws SQLException
     {
          Object result = null;
          Connection conn = null;
          ResultSet rs = null;
          PreparedStatement findStatement = null ;
          try
          {    
               conn = DB.getConnection(MapperFactory.getDataSourceName());
               
               findStatement = conn.prepareStatement(findStatement());

               loadFinderStatement(key, findStatement);
               rs = findStatement.executeQuery();
               if (rs.next())
               {     result = load(rs);
               }
          }
          catch (SQLException e)
          {
                throw new SQLException("Exception initialising mapper");
               //throw new SQLException("Exception initialising mapper!", e);
          }
          catch (Exception ex) {
          }
          finally
          {
              try { if( rs != null ) rs.close() ; } catch( SQLException ex ) {}
              try { if( findStatement != null ) findStatement.close() ; } catch( SQLException ex ) {}
              DB.cleanUp(conn);
          }
          return result;

     }
0
 
TimYatesCommented:
See this:

http://www.webservertalk.com/archive135-2005-2-921383.html

Are you doing Class.forName on every thread?  You only need to do it once :-/

It sounds like you're not closing a connection properly...

Tim
0
 
TimYatesCommented:
Can you post your code?
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
Jim CakalicSenior Developer/ArchitectCommented:
Is it possible that you have a Connection leak somewhere? Are you always closing Connections in a finally block?
What platform are you on? You might be running up against a per-process limit on open file handles ...

Jim
0
 
narravAuthor Commented:

import java.util.Hashtable;
import java.util.ArrayList;
import com.citigroup.cpacs.test.utils.StringExt;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;




/**
 * @author ms97167
 *
 * TODO To change the template for this generated type comment go to
 * Window - Preferences - Java - Code Style - Code Templates
 */
public class MultiThreadCpacsCoreTest {

      public static void main(String[] args) throws Exception
      {
       fileName = args[0];
       System.out.println("FileName is: " + fileName);
       MultiThreadCpacsCoreTest m = new MultiThreadCpacsCoreTest();
       m.executeFile();
      }


 public void executeFile() throws IOException {

           if (this.fileName.trim() == "") {
              throw new IOException("File name not been set!.");
           }

           BufferedReader reader = new BufferedReader(new FileReader(this.fileName));
           if (reader ==null) {
              System.out.println("No file to read");
           }

           String sLine = reader.readLine();
           Hashtable  ht = new Hashtable();
           ArrayList threadCpacsCore=new ArrayList();
           while (sLine != null) {
                System.out.println("Processing line:" + sLine);
                String asFieldValues[] = StringExt.split(sLine,",");
                // break each pair into its components and build a map
                for (int inCont=0; inCont < asFieldValues.length; inCont++) {
                  String lsPairvalues[] = StringExt.split(asFieldValues[inCont],"=");
                  if (lsPairvalues.length ==2) {
                     ht.put(lsPairvalues[0],lsPairvalues[1]);

                  }

                }

            ThreadCpacsCore tpc = new ThreadCpacsCore(ht.get("ThreadName").toString(),ht.get("InputFile").toString(), ht.get("OutputFile").toString(),ht.get("RunOption").toString());
            threadCpacsCore.add(tpc);
            ht.clear();

                System.out.println("===========================================");

                sLine = reader.readLine();
           }


           reader.close();


         for(int index = 0;index < threadCpacsCore.size();index++) {
               ThreadCpacsCore tcp = (ThreadCpacsCore) threadCpacsCore.get(index);
               tcp.start();
          }
         }


       private static String fileName = "";

}

public class ThreadCpacsCore extends Thread {


      /**
       * @param arg0
       */
      public ThreadCpacsCore(String threadName,String inputFile, String outputFile,String runOption) {
            super(threadName);
        this.inputFile=inputFile;
        this.outputFile=outputFile;
        this.runOption=runOption;
      }
      /* (non-Javadoc)
       * @see java.lang.Runnable#run()
       */
      public void run() {
         try {
            System.out.println("Executing Thread : " + this.getName());

            TestDriver testDriver = new TestDriver(inputFile, outputFile, runOption,-1);
            testDriver.testCalculation();
} catch(Exception e) {
System.out.println("Got Exception  " + e);
      }
}

      private String inputFile=null;
      private String outputFile=null;
      private String runOption=null;
}


and then each thread invokes instance of TestDriver.. do you need TestDriver?? It is a huge program.. in case you need to see I can email it to you, or whatever you prefer..

I am closing the conn as below:



rotected Object abstractFind(Object key) throws SQLException
      {
            Object result = null;
            Connection conn = null;
            try
            {      conn = DB.getConnection(MapperFactory.getDataSourceName());
                  ResultSet rs = null;
                  PreparedStatement findStatement = conn.prepareStatement(findStatement());

                  loadFinderStatement(key, findStatement);
                  rs = findStatement.executeQuery();
                  if (rs.next())
                  {      result = load(rs);
                  }
                  rs.close();
                  DB.cleanUp(conn);
            }
            catch (SQLException e)
            {
                   throw new SQLException("Exception initialising mapper");
                  //throw new SQLException("Exception initialising mapper!", e);
            }
            catch (Exception ex) {
            }
            return result;

      }

so I am not doing it in finally block.. i will change it do it in finally block.. but looks like this is good catch Jim, I will fix it.. but it will not fix this problem.. right??

i am running on solaris 5.8

how to find open file handles per-process?
0
 
Jim CakalicSenior Developer/ArchitectCommented:
I concur, Tim. Maybe even encapsulate this in your in DB class by adding another static method that takes the object references:

    public static void cleanUp(Connection conn, Statement stmt, ResultSet rset) {
        try { if( rs != null ) rs.close() ; } catch( SQLException ex ) {}
        try { if( findStatement != null ) findStatement.close() ; } catch( SQLException ex ) {}
        cleanUp(conn);
    }

And then call this in the finally block. I've found it to be a very useful idiom.

Jim
0
 
TimYatesCommented:
Yeah, I always use that too :-)
0
 
Jim CakalicSenior Developer/ArchitectCommented:
Of course, it works best when I change the copy and paste code to actually compile before posting it!
:-o
0
 
narravAuthor Commented:
Oops,

I am sorry, I already have cleanUp in DB.java

I tried using it in finally block, but independant of this issue, I am facing problem while trying to run this in Multiple Threads.

Could you please take a look at my other question
http://www.experts-exchange.com/Programming/Programming_Languages/Java/Q_21409857.html
which is kind of related to this and suggest something please..

0
 
TimYatesCommented:
>> I am sorry, I already have cleanUp in DB.java

Event the preparedstatement?

How are you closing that?

Tim
0
 
TimYatesCommented:
I meant:

Even the PreparedStatement
0
 
narravAuthor Commented:
well, there are diffent methods in DB.java.. I was just calling DB.cleanup(conn) but I am doing it based on your first suggestion..

I am closing rs and ps per your way and I was closing using DB.cleanup(conn) method..

so basically this is what I am doing:

      protected Object abstractFind(Object key) throws SQLException
      {
            Object result = null;
            Connection conn = null;
            ResultSet rs = null;
            PreparedStatement findStatement = null;
            try
            {      conn = DB.getConnection(MapperFactory.getDataSourceName());
                  //ResultSet rs = null;
                   findStatement = conn.prepareStatement(findStatement());

                  loadFinderStatement(key, findStatement);
                  rs = findStatement.executeQuery();
                  if (rs.next())
                  {      result = load(rs);
                  }
                  //rs.close();
                  //DB.cleanUp(conn);
            }
            catch (SQLException e)
            {
                   throw new SQLException("Exception initialising mapper");
                  //throw new SQLException("Exception initialising mapper!", e);
            }
            catch (Exception ex) {
                   System.out.println("AbstractMapper$$abstractFind General Exception " + ex);
                   ex.printStackTrace();
            }

             finally
                      {
                          System.out.println("AbstractMapper$$finally closing conn");
                          try { if( rs != null ) rs.close() ; } catch( SQLException ex ) {}
                          try { if( findStatement != null ) findStatement.close() ; } catch( SQLException ex ) {}
                          DB.cleanUp(conn);
                      }


            return result;

      }

I guess this is correct also right?
0
 
TimYatesCommented:
yeah...does it still crash?
0
 
narravAuthor Commented:
no, its not crashing now.. may be this was the problem..

but i am facing some issues with multithreading with this cleanUp method as well as at some other parts of code..

i got some of the issues fixed with some of experts as well.. but with synchronizing and cloning approach.. i want to look for other approach.. to get everything at method variable level rather than cloning or synchronizing methods as this is not accpetable to company here..

can u please look into other question and reply as well.. though all experts are helping me.. but i really want to get max suggestions to figure this issue out..

thanks
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.