[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

How to make this sql processing faster?

Posted on 2006-04-08
9
Medium Priority
?
284 Views
Last Modified: 2010-05-19
Hi,

My code below does this,

-it reads rows from a db table row by row
-every row that is reads it gets the user info and send data to the user info via a method sendText()

Now, my current code takes very long to run especially when there's lots of rows. My requests are,

-Suggestion on how to load my rows of data into memory then send one by one from mem
-Suggestion on improving sql processing of this code.

----CODE--------
/*
 * Classname: FortuneTellerReadingsDelivery.java Description: Generic class to send
 * readings to fortune teller subscribers
 *
 * Author: melwong Date started: 11-Mar-2004
 *
 * Version: 1.0
 */

package com.jm.fortuneteller;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;

import org.apache.commons.dbutils.DbUtils;

import com.jm.config.DbConfig;
import com.jm.config.FileConfig;
import com.jm.config.PropertyFileReader;
import com.jm.gateway.OutboundResponse;
import com.jm.util.KeywordProcessor;

public class FortuneTellerReadingsDelivery {
      
      private Connection conn = null;
      
      //Melvin:16/03/06
      public FortuneTellerReadingsDelivery(Connection conn) {
            this.conn = conn;
            
      }


      public void processDelivery (String message) {
            
            int[] fortuneReadings = null;
            int groupId = 0;
            int serviceId = 0;
            int subkeyword1Id = 0;
            String lineFeed = "";
            String serviceKeyword = "";
            String fortuneCookie = "";
            String readings = "";
            String phoneNumber = "";
            String sqlStr = "";
            
            //Melvin:16/03/06
            //Connection conn = null;
            
            PreparedStatement statement = null;
            ResultSet rs = null;
            KeywordProcessor keyPro = null;
            OutboundResponse outRes = null;
            FortuneTellerUtils util = null;

            keyPro = new KeywordProcessor();
            outRes = new OutboundResponse();
            util = new FortuneTellerUtils();
            
            lineFeed = System.getProperty("line.separator");
            
            //To get the subscribers table
            serviceId = keyPro.getServiceId(message);
            subkeyword1Id = keyPro.getSubkeyword1Id(message, serviceId);
            groupId = keyPro.getGroupId(subkeyword1Id);
            serviceKeyword = keyPro.getServiceKeyword(groupId);
            
            //Melvin:16/03/06
            //Get one fortune cookie for all subscribers
            fortuneCookie = util.getFortuneCookie();

            try {
                  //Melvin:16/03/06
                  //DbUtils.loadDriver(DbConfig.dbDriver);
                  //conn = DriverManager.getConnection(DbConfig.dbUrl);

                  sqlStr = "SELECT DISTINCT phoneNumber FROM "
                              + serviceKeyword + "_fortuneteller_users"
                              + " WHERE inactive = 0";
                  
                  statement = conn
                              .prepareStatement(sqlStr);
                  rs = statement.executeQuery();

                  while (rs.next()) {
                        phoneNumber = rs.getString(1);
                        
                        fortuneReadings = util.getFortuneReadings();
                        
                        //Melvin:16/03/06
                        //fortuneCookie = util.getFortuneCookie();
                        
                        readings = "Love:" + fortuneReadings[0] + "%" + lineFeed
                                          + "Money:" + fortuneReadings[1] + "%" + lineFeed
                                          + "Attitude:" + fortuneReadings[2] + "%";
                        
                        //Melvin: 02/08/05
                        outRes.processText(groupId, "Your today's fortune cookie: " + fortuneCookie + lineFeed + readings, phoneNumber);
                        //outRes.processText(groupId, fortuneCookie + lineFeed + readings, phoneNumber);

                  }

            } catch (SQLException e) {
                  throw new FortuneTellerException(this.getClass().toString(), e);
            } finally{
                  DbUtils.closeQuietly(rs);
                  DbUtils.closeQuietly(statement);
                  
                  //Melvin:16/03/06
                  //DbUtils.closeQuietly(conn);
            }

      }

      
      public static void main(String args[]) {

            String message = "";
            String absPath = "";
            PropertyFileReader readProp = null;
            DbConfig initDbConfig = null;
            FortuneTellerReadingsDelivery fortuneTellerReadingsDelivery = null;
            
            //Melvin:16/03/06
            Connection conn = null;

            if (args.length != 0) {

                  absPath = args[0];

                  for (int i = 1; i < args.length; i++) {
                        message += args[i];
                        message += " ";
                  }
                  message = message.trim();

                  System.out.println("absPath=" + absPath);
                  System.out.println("message=" + message);

            } else {
                  System.err
                              .println("FortuneTellerReadingsDelivery: ERROR: Invalid command line arguments.");
                  System.err.println("\tFortuneTellerReadingsDelivery: Usage:");
                  System.err
                              .println("\tjava FortuneTellerReadingsDelivery <full webapp path> <full SMS message>");
                  System.err
                              .println("\tE.g: java FortuneTellerReadingsDelivery C:\\tomcat\\webapps\\jm HI FC");
                  System.exit(1);
            }

            //Read properties must process first, then initialize db
            readProp = new PropertyFileReader(absPath);
            
            //To initialize the db stuffs
            initDbConfig = new DbConfig();

            //To get URL for WAP Push and app full path
            FileConfig.setSysConfig();
            
            //Melvin:16/03/06
            try {
                  DbUtils.loadDriver(DbConfig.dbDriver);
                  conn = DriverManager.getConnection(DbConfig.dbUrl);
                  fortuneTellerReadingsDelivery = new FortuneTellerReadingsDelivery(conn);
                  fortuneTellerReadingsDelivery.processDelivery(message);
            } catch (SQLException e) {
                  throw new FortuneTellerException("FortuneTellerReadingsDelivery", e);
            } finally {
                  DbUtils.closeQuietly(conn);
            }
            //fortuneTellerReadingsDelivery = new FortuneTellerReadingsDelivery();

            //fortuneTellerReadingsDelivery.processDelivery(message);

      }

}

---EOF CODE-----
0
Comment
Question by:mwhc
  • 3
  • 3
  • 3
9 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 16408105
You can start with:

a. bind the parameter to the PS

statement.setString(1, serviceKeyword);

b. make sure there's an index on phone number column


Does the below need to be called in the loop?

>>fortuneReadings = util.getFortuneReadings();
0
 
LVL 92

Expert Comment

by:objects
ID: 16408977
Don't use a PreparedSTatement, just use a standard one using createStatement()

And if your result set size is large then try adjusting the fetch size of the resukt set

0
 

Author Comment

by:mwhc
ID: 16409941
Hi CEHJ,

a. What is PS

>>Does the below need to be called in the loop?
>>fortuneReadings = util.getFortuneReadings();
Yes is does.

To objects,
How do I adjust the fetch size of the result set?
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:mwhc
ID: 16409949
BTW,

What do you mean by "make sure there's an index on phone number column". This is my table structure,

CREATE TABLE `just_fortuneteller_users` (
  `userId` int(11) NOT NULL auto_increment,
  `phoneNumber` varchar(100) NOT NULL default '',
  `registrationTime` datetime NOT NULL default '0000-00-00 00:00:00',
  `totalMessage` int(11) NOT NULL default '0',
  `inactive` int(11) NOT NULL default '0',
  PRIMARY KEY  (`userId`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Is this ok?

Melv
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 600 total points
ID: 16410492
>>a. What is PS

PreparedStatement

>>What do you mean by "make sure there's an index on phone number column". This is my table structure,

try the following in your CREATE TABLE

phoneNumber varchar(100) NOT NULL, INDEX USING BTREE (phoneNumber)
0
 
LVL 92

Expert Comment

by:objects
ID: 16413234
> How do I adjust the fetch size of the result set?

rs.setFetchSize(size);

http://java.sun.com/j2se/1.5.0/docs/api/java/sql/ResultSet.html#setFetchSize(int)
0
 

Author Comment

by:mwhc
ID: 16415787
Hi,

I'm learning on indexing now. But I found out that indexing is useful especially in the WHERE statement. Will it be useful in my phoneNumber col since it's not in the WHERE statement?
0
 
LVL 92

Expert Comment

by:objects
ID: 16415803
doubt it, your better off indexing on inactive.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16416582
>>But I found out that indexing is useful especially in the WHERE statement.
>>your better off indexing on inactive.

Sorry mwhc, inactive is the correct column

0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

An old method to applying the Singleton pattern in your Java code is to check if a static instance, defined in the same class that needs to be instantiated once and only once, is null and then create a new instance; otherwise, the pre-existing insta…
Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.
This video teaches viewers about errors in exception handling.
Suggested Courses
Course of the Month20 days, 5 hours left to enroll

872 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question