[x]
Posted via EE Mobile

Search, ask, and monitor your questions on the go with EE Mobile. Visit Experts Exchange from your mobile device and never be out of touch again.

Question
[x]
Attachment Details
[x]
The Solution Rating System

With so many solutions, how can you tell which solutions are most likely to help you and which ones are not? To provide you with a tool to use, we rate our solutions based on various elements that most accurately determine if a solution is a quality solution. To explain what factors affect the solution rating, here are the elements we take into consideration when formulating our solution rating.

  • The Grade of the Solution
  • The Zone Rank of the Expert Providing the Solution
  • The Number of Author and Expert Comments
  • The Number of Experts Contributing
  • The Feedback of the Community

Your Input Matters
Because of the way the system is set up, the most important variable in this equation is you. As a member of Experts Exchange, you are able to cast your vote on the quality of the solutions in regard to how complete, accurate, helpful and easy to understand each solution is. When you provide your feedback, each rating is adjusted accordingly. So, if you see a solution that has a poor rating that you think is a good solution, let us know by rating it. As you do, the rating will be adjusted and will become more accurate for other members of our site.

If you have any suggestions that you would like to make for our rating system, please ask a question in the Suggestions Zone of Community Support.

Thank you!

6.8

Best way to perform massive # of inserts from java...

Asked by jbaisden in Java Programming Language

Tags: java, inserts, million, prepared



Hi experts. I am working on a batch process that is quite lengthy. It tries to find an association between one thing and another based on about 51 conditions. Each condition, if matched, adds a point so to speak. For each match that has received a certain number of points or more, I add that object (it's a business entity object) to an array list. The last thing the batch does before completing is call the Data Access method of this object and pass the arraylist of matches to the method. The following code is how I attempt to perform the insert. Keep in mind that we are using a prepared statement here.

                  INSERT_NEW_MATCHES +=
                  "      INSERT INTO Mytable   " +
                  "      (  F1, F2, F3, F4, F5)   " ;
                  
                  System.out.println("in insert method - before for loop. SCMLS size: " + scmls.size() );
                  for(int i=0; i < scmls.size()  ; i++ )
                  {
                        //iterationIndex = i;
                        //System.out.println("in insert method - before for get in for loop. Iteration: " + i );
                        //scml = (myObj)scmls.get(i);
                        INSERT_NEW_MATCHES +=
                              " SELECT       " +
                                                 "
                                                                                                                       ? ," +
                                                 " ? ," +  
                                               " ? ," +  
                          "            SYSDATE, " +
                                                                           " ? " +
                            " FROM DUAL " +
                            " UNION " ;

                  }

Followed By:

                  if( INSERT_NEW_MATCHES.indexOf("UNION") > -1 )
                        INSERT_NEW_MATCHES = INSERT_NEW_MATCHES.substring(0, INSERT_NEW_MATCHES.lastIndexOf("UNION"));
                  
                  System.out.println("FINAL INSERT QUERY: " + INSERT_NEW_MATCHES );
                  
                  pstmt = con.prepareStatement(INSERT_NEW_MATCHES );
                  
                  for(int a=0;a<scmls.size() ;a++)
                  {
                        iterationIndex = a;
                        scml = (StudentCampusMatchList)scmls.get(a);
                        pstmt.setInt(++count, scml.F1() );
                        pstmt.setInt(++count, scml.F2());
                        pstmt.setDouble(++count, scml.F3());
                        pstmt.setString(++count, scml.F4() );
                  }

                  System.out.println("in insert method - before executeUpdate ");
                  int rc = pstmt.executeUpdate();      

As you can imagine this is pretty heavy on the memory. I did not anticipate that this process would have to insert 1 million plus records, but it does. I realize that I will likely run into a memory issue with the string. My question is, how can I implement this in a better way? What ever the suggestion is, it must be able to utilize an arraylist of business objects.

Thanks ahead of time guys!            
[+][-]10/23/06 06:42 AM, ID: 17788395Accepted Solution

View this solution now by starting your 30-day free trial. Setting up your free trial is quick, easy, and secure. We will return you to this solution, unlocked, when you're done.

About this solution

Zone: Java Programming Language
Tags: java, inserts, million, prepared
Sign Up Now!
Solution Provided By: karanw
Participating Experts: 2
Solution Grade: A
 
[+][-]10/23/06 07:44 AM, ID: 17788876Author Comment

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 30-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]10/23/06 01:21 PM, ID: 17791360Assisted Solution

Assisted solutions are selected by the member who asked the question as a comment that contributed to their question's solution.

Start your 30-day free trial to view this Assisted Solution or ask the Experts your question.

 
[+][-]10/24/06 11:45 AM, ID: 17798264Author Comment

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 30-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]10/24/06 11:54 AM, ID: 17798362Expert Comment

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 30-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]10/24/06 12:15 PM, ID: 17798554Author Comment

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 30-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]10/24/06 12:25 PM, ID: 17798638Expert Comment

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 30-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]10/24/06 01:10 PM, ID: 17799037Expert Comment

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 30-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]11/15/06 08:48 AM, ID: 17948500Author Comment

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 30-day free trial to view this Author Comment or ask the Experts your question.

 
 
Loading Advertisement...
20091111-EE-VQP-89