Solved

With more than 100,000 Record, Locate Methode is Lame duck...How to Get higher speed in searching progress....

Posted on 2001-09-04
6
284 Views
Last Modified: 2010-04-06
          I?m writing a code with A Medium size table, ?Member? Table. Problems I have encounter is that I try to create Search-Form to provide user interface by letting user key in MemberId in Edit Box and write OnChange Event of EditBox. Form only have a DBGrid with 3 Columns of MemberId, MemberName, MemberSurname. But searching speed suffer user too much and make my Search-Form lame duck...By using a Locate Methode, I?ve created Index Name ?Member01? in which indexed on Field ?MemberId? but it?s still Run with too bad speed
        I wonder if there?s anyway to write a Search-Form to provide user interface by entering a MemberId and Dataset move to Such Record first found
        For example, Table contain 3 Record
          Record 1              MemberId = 0001  MemberName = Name01 ....
          Record 2              MemberId = 0101 MemberName = Name02
          Record 3              MemberId = 0201 MemberName = Name03
     If User Key in 01   Form Must move RecordPointer to Record 2, if User Key in 02 RecordPointer must move to Record 3 and So on...
           What I want is that Record 999       Form Must move Record Pointer to Record of MemebrId 99900 Within 2 Seconds At least on Pentium II-400  , say, must be faster depend on Spec of Computer running it. Any real expert out there Show me how to do so, I?ll give you 200 points
0
Comment
Question by:DOOM_DAEMON
6 Comments
 
LVL 10

Expert Comment

by:Jacco
Comment Utility
What component are you using?

TQuery.Locate never uses index

Regards Jacco
0
 
LVL 3

Expert Comment

by:karouri
Comment Utility
Listening.
I just hope that these 100,000 records are not in a Microsoft Access database..
0
 
LVL 27

Expert Comment

by:kretzschmar
Comment Utility
listening
(multiple ways possible, but some more info needed)
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 3

Accepted Solution

by:
oraelbis earned 200 total points
Comment Utility
I recomend to use TClientDataSet(from Midas tab) and create indexes on the fly(creating indexes very fast).
After creating indexes you can use them for fast searching, or filtering. This component support additional filter options.(Note: before close it you may free all created indexes).
There is some code from my TClientDataset descendant (you cand find this components sourcecode on http://bs-elbis.chat.ru) with supporting search function. Use Tedit control on your form such as:
procedure TfrmEmployee.edtSearchChange(Sender: TObject);
begin
  //cdsEmloyee TBS_Client dataset
  cdsEmloyee.SearchCdsRecord(edtSearch);
end;


function TBS_ClientDataSet.SearchCdsRecord(const AedtSearch:Tedit):boolean;
var
  sFieldsNames:String;
  i:integer;
  theSearchText:String;
  aSubStr:String;
  theSearchArray:Variant;

begin
  sFieldsNames:='';
  theSearchText:=AedtSearch.text;
  Result:=False;
  try
    theSearchArray := VarArrayCreate([0, Self.IndexFieldCount - 1], varVariant);
    for i := 0 to Self.IndexFieldCount - 1 do
      begin
        if sFieldsNames<>'' then
          sFieldsNames:=sFieldsNames+';';
        sFieldsNames:=sFieldsNames + Self.IndexFields[i].FieldName;
        if Pos(';',theSearchText)>0 then
          begin
            aSubStr:=Copy(theSearchText,1,Pos(';',theSearchText)-1);
            theSearchText:=Copy(theSearchText,Pos(';',theSearchText)+1,
                           Length(theSearchText)-(Pos(';',theSearchText)));
          end
        else
          begin
            aSubStr:=theSearchText;
            theSearchText:='';
          end;
        try
          case Self.IndexFields[i].DataType of
            ftSmallint,ftInteger,ftBytes,ftVarBytes,
            ftAutoInc,ftLargeint,ftTypedBinary:
              if aSubStr<>'' then
                theSearchArray[i]:=StrToInt(aSubStr)
              else
                theSearchArray[i]:=0;
            ftWord,ftFloat,ftBCD,ftCurrency:
              if aSubStr<>'' then
                theSearchArray[i]:=StrToFloat(aSubStr)
              else
                theSearchArray[i]:=0;
            ftDate, ftTime, ftDateTime:
              theSearchArray[i]:=StrToDate(aSubStr);
          else
            theSearchArray[i]:=aSubStr;
          end;
        except
          theSearchArray[i]:=aSubStr;
        end;
      end;

    if Self.Locate(sFieldsNames, theSearchArray,
       [loCaseInsensitive,loPartialKey]) then
      begin
        AedtSearch.Font.Color:=clWindowText;
      end
    else
      AedtSearch.Font.Color:=clRed; {Can't find}
    Result:=True;
  except      //Try restore last position
    Result:=False;
  end;
end;

procedure TBS_ClientDataSet.FreeCreatedIndexes;
var
  i:integer;
begin
  if Self.Active then
  try
    IndexDefs.Update; //Refresh
    for i := 0 to IndexDefs.Count - 1 do
      if (IndexDefs.Items[i].Name<>'DEFAULT_ORDER') and (IndexDefs.Items[i].Name<>'CHANGEINDEX')
         and (IndexDefs.Items[i].Name<>'PRIMARY_KEY') then
           begin
             if (Self.IndexName=IndexDefs.Items[i].Name) then
               Self.IndexName:='';
             DeleteIndex(IndexDefs.Items[i].Name);
           end;
  except
    raise;
  end;
end;

////////////////////////////////////////////////////
I use TDBGreedEh descendant, for automatic index creating.
There is part of code from it:
function TBS_DBGridEh.BS_MultiSort:boolean;
var
  i                   :integer;
  theIndexName        :String;
  theIndexFields      :String;
  theDescFields       :String;
  theOldCursor        :TCursor;
  theBS_ClientDataSet :TBS_ClientDataSet;
  fIndexPresent       :boolean;
  s                   :String;

 function DeleteStr(str:String; substr:String): String;
 var i:Integer;
 begin
   i := Pos(substr,str);
   if i <> 0 then
     Delete(str,i,Length(substr));
   Result := str;
 end;

begin
 Result:=False;
 theOldCursor:=Screen.Cursor;
 Screen.Cursor:=crHourGlass;
 try
  if Assigned(Self.DataSource.DataSet as TBS_ClientDataSet)
    and Self.DataSource.DataSet.Active then
    begin
      theBS_ClientDataSet:=TBS_ClientDataSet(Self.DataSource.DataSet);

      theIndexName := '';
      theDescFields:= '';
      theIndexFields:= '';
      for i := 0 to Self.SortMarkedColumns.Count-1 do
        if Self.SortMarkedColumns[i].Title.SortMarker = smUpEh then
          begin
            theIndexFields:=theIndexFields+ Self.SortMarkedColumns[i].FieldName+';';
            theIndexName := theIndexName + Self.SortMarkedColumns[i].FieldName + '_DESC_';
          end
        else
          begin
            theIndexFields:=theIndexFields+ Self.SortMarkedColumns[i].FieldName+';';
            theIndexName := theIndexName + Self.SortMarkedColumns[i].FieldName + '_';
            theDescFields:=theDescFields+ Self.SortMarkedColumns[i].FieldName+';';
          end;

        if theIndexName <> '' then theIndexName := 'idx' + Copy(theIndexName,1,Length(theIndexName)-1);
          theIndexName := DeleteStr(theIndexName,'1');
        if theIndexFields <> '' then theIndexFields := Copy(theIndexFields,1,Length(theIndexFields)-1);
          theIndexFields := DeleteStr(theIndexFields,'1');
        if theDescFields <> '' then theDescFields := Copy(theDescFields,1,Length(theDescFields)-1);
          theDescFields := DeleteStr(theDescFields,'1');

        fIndexPresent:=False;
        theBS_ClientDataSet.IndexDefs.Update;//Refresh
        //If index exist, then set it
        for i := 0 to theBS_ClientDataSet.IndexDefs.Count - 1 do
        if LowerCase(theBS_ClientDataSet.IndexDefs.Items[i].Name) = LowerCase(theIndexName) then
          begin
            fIndexPresent:=True; // Sign
            theBS_ClientDataSet.IndexName := theBS_ClientDataSet.IndexDefs.Items[i].Name;
          end;
        //If index not exist, then try create it
        if Not(fIndexPresent) then
          begin
            theBS_ClientDataSet.AddIndex(theIndexName, theIndexFields,[ixCaseInsensitive],theDescFields,'',0);
          end;

        theBS_ClientDataSet.IndexName := theIndexName;
        Result:=True;
    end;
  finally
    Screen.Cursor:=theOldCursor;
  end;
end;

////////////////////////////////////////////////////////////////////
And there is using filter for searching example(Wark very fast):
procedure TfrmSPORG.edtSearchChange(Sender: TObject);
var
  sFieldsNames  : String;
  i             : integer;
begin
  sFieldsNames:='';
  for i := 0 to FinDM.cdsOrganization.IndexFieldCount - 1 do
    if sFieldsNames='' then
      begin
        sFieldsNames:=sFieldsNames + FinDM.cdsOrganization.IndexFields[i].FieldName;
        break;
      end;
  {$D+L+}
  case FinDM.cdsOrganization.IndexFields[i].DataType of
    ftString, ftFixedChar, ftWideString, ftGuid:
    begin

      if (edtSearch.Text='') or (sFieldsNames='') then
        begin
          FinDM.cdsOrganization.Filtered:=False;
          FinDM.cdsOrganization.First;
        end
      else
        begin
          FinDM.cdsOrganization.Filter:='';
           FinDM.cdsOrganization.Filter:='('+sFieldsNames
             +' LIKE '+ #39+'%'+edtSearch.Text+'%'+#39+')'
             +' or ('+sFieldsNames+'='+#39+edtSearch.Text+#39+')';
        end;
        FinDM.cdsOrganization.Filtered:=True;
        FinDM.cdsOrganization.First;
        FinDM.cdsOrganization.SearchCdsRecord(edtSearch);
    end
  else
    FinDM.cdsOrganization.Filtered:=False;
    FinDM.cdsOrganization.SearchCdsRecord(edtSearch);
  end
end;
 
0
 
LVL 22

Expert Comment

by:mnasman
Comment Utility
as I tested before (but with oracle database)
ADOTable and ADOQuery are much faster than the Table and Query
that's mean use ADO it will be faster than BDE
but still more information needed

0
 
LVL 1

Expert Comment

by:Moondancer
Comment Utility
ADMINISTRATION WILL BE CONTACTING YOU SHORTLY.  Moderators Computer101 or Netminder will return to finalize these if they are still open in 14 days.  Experts, please post closing recommendations before that time.

Below are your open questions as of today.  Questions which have been inactive for 21 days or longer are considered to be abandoned and for those, your options are:
1. Accept a Comment As Answer (use the button next to the Expert's name).
2. Close the question if the information was not useful to you, but may help others. You must tell the participants why you wish to do this, and allow for Expert response.  This choice will include a refund to you, and will move this question to our PAQ (Previously Asked Question) database.  If you found information outside this question thread, please add it.
3. Ask Community Support to help split points between participating experts, or just comment here with details and we'll respond with the process.
4. Delete the question (if it has no potential value for others).
   --> Post comments for expert of your intention to delete and why
   --> YOU CANNOT DELETE A QUESTION with comments; special handling by a Moderator is required.

For special handling needs, please post a zero point question in the link below and include the URL (question QID/link) that it regards with details.
http://www.experts-exchange.com/jsp/qList.jsp?ta=commspt
 
Please click this link for Help Desk, Guidelines/Member Agreement and the Question/Answer process.  http://www.experts-exchange.com/jsp/cmtyHelpDesk.jsp

Click you Member Profile to view your question history and please keep them updated. If you are a KnowledgePro user, use the Power Search option to find them.  

Questions which are LOCKED with a Proposed Answer but do not help you, should be rejected with comments added.  When you grade the question less than an A, please comment as to why.  This helps all involved, as well as others who may access this item in the future.  PLEASE DO NOT AWARD POINTS TO ME.

To view your open questions, please click the following link(s) and keep them all current with updates.
http://www.experts-exchange.com/questions/Q.20110714.html
http://www.experts-exchange.com/questions/Q.20173874.html
http://www.experts-exchange.com/questions/Q.20170377.html
http://www.experts-exchange.com/questions/Q.20178176.html
http://www.experts-exchange.com/questions/Q.20178177.html
http://www.experts-exchange.com/questions/Q.20229473.html
http://www.experts-exchange.com/questions/Q.20279642.html

To view your locked questions, please click the following link(s) and evaluate the proposed answer.
http://www.experts-exchange.com/questions/Q.20220086.html

*****  E X P E R T S    P L E A S E  ******  Leave your closing recommendations.
If you are interested in the cleanup effort, please click this link
http://www.experts-exchange.com/jsp/qManageQuestion.jsp?ta=commspt&qid=20274643
POINTS FOR EXPERTS awaiting comments are listed in the link below
http://www.experts-exchange.com/commspt/Q.20277028.html
 
Moderators will finalize this question if in @14 days Asker has not responded.  This will be moved to the PAQ (Previously Asked Questions) at zero points, deleted or awarded.
 
Thanks everyone.
Moondancer
Moderator @ Experts Exchange
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

The uses clause is one of those things that just tends to grow and grow. Most of the time this is in the main form, as it's from this form that all others are called. If you have a big application (including many forms), the uses clause in the in…
Introduction The parallel port is a very commonly known port, it was widely used to connect a printer to the PC, if you look at the back of your computer, for those who don't have newer computers, there will be a port with 25 pins and a small print…
This video discusses moving either the default database or any database to a new volume.
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now