Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

With more than 100,000 Record, Locate Methode is Lame duck...How to Get higher speed in searching progress....

Posted on 2001-09-04
6
Medium Priority
?
295 Views
Last Modified: 2010-04-06
          I?m writing a code with A Medium size table, ?Member? Table. Problems I have encounter is that I try to create Search-Form to provide user interface by letting user key in MemberId in Edit Box and write OnChange Event of EditBox. Form only have a DBGrid with 3 Columns of MemberId, MemberName, MemberSurname. But searching speed suffer user too much and make my Search-Form lame duck...By using a Locate Methode, I?ve created Index Name ?Member01? in which indexed on Field ?MemberId? but it?s still Run with too bad speed
        I wonder if there?s anyway to write a Search-Form to provide user interface by entering a MemberId and Dataset move to Such Record first found
        For example, Table contain 3 Record
          Record 1              MemberId = 0001  MemberName = Name01 ....
          Record 2              MemberId = 0101 MemberName = Name02
          Record 3              MemberId = 0201 MemberName = Name03
     If User Key in 01   Form Must move RecordPointer to Record 2, if User Key in 02 RecordPointer must move to Record 3 and So on...
           What I want is that Record 999       Form Must move Record Pointer to Record of MemebrId 99900 Within 2 Seconds At least on Pentium II-400  , say, must be faster depend on Spec of Computer running it. Any real expert out there Show me how to do so, I?ll give you 200 points
0
Comment
Question by:DOOM_DAEMON
6 Comments
 
LVL 10

Expert Comment

by:Jacco
ID: 6453093
What component are you using?

TQuery.Locate never uses index

Regards Jacco
0
 
LVL 3

Expert Comment

by:karouri
ID: 6453794
Listening.
I just hope that these 100,000 records are not in a Microsoft Access database..
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6455848
listening
(multiple ways possible, but some more info needed)
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 3

Accepted Solution

by:
oraelbis earned 800 total points
ID: 6457587
I recomend to use TClientDataSet(from Midas tab) and create indexes on the fly(creating indexes very fast).
After creating indexes you can use them for fast searching, or filtering. This component support additional filter options.(Note: before close it you may free all created indexes).
There is some code from my TClientDataset descendant (you cand find this components sourcecode on http://bs-elbis.chat.ru) with supporting search function. Use Tedit control on your form such as:
procedure TfrmEmployee.edtSearchChange(Sender: TObject);
begin
  //cdsEmloyee TBS_Client dataset
  cdsEmloyee.SearchCdsRecord(edtSearch);
end;


function TBS_ClientDataSet.SearchCdsRecord(const AedtSearch:Tedit):boolean;
var
  sFieldsNames:String;
  i:integer;
  theSearchText:String;
  aSubStr:String;
  theSearchArray:Variant;

begin
  sFieldsNames:='';
  theSearchText:=AedtSearch.text;
  Result:=False;
  try
    theSearchArray := VarArrayCreate([0, Self.IndexFieldCount - 1], varVariant);
    for i := 0 to Self.IndexFieldCount - 1 do
      begin
        if sFieldsNames<>'' then
          sFieldsNames:=sFieldsNames+';';
        sFieldsNames:=sFieldsNames + Self.IndexFields[i].FieldName;
        if Pos(';',theSearchText)>0 then
          begin
            aSubStr:=Copy(theSearchText,1,Pos(';',theSearchText)-1);
            theSearchText:=Copy(theSearchText,Pos(';',theSearchText)+1,
                           Length(theSearchText)-(Pos(';',theSearchText)));
          end
        else
          begin
            aSubStr:=theSearchText;
            theSearchText:='';
          end;
        try
          case Self.IndexFields[i].DataType of
            ftSmallint,ftInteger,ftBytes,ftVarBytes,
            ftAutoInc,ftLargeint,ftTypedBinary:
              if aSubStr<>'' then
                theSearchArray[i]:=StrToInt(aSubStr)
              else
                theSearchArray[i]:=0;
            ftWord,ftFloat,ftBCD,ftCurrency:
              if aSubStr<>'' then
                theSearchArray[i]:=StrToFloat(aSubStr)
              else
                theSearchArray[i]:=0;
            ftDate, ftTime, ftDateTime:
              theSearchArray[i]:=StrToDate(aSubStr);
          else
            theSearchArray[i]:=aSubStr;
          end;
        except
          theSearchArray[i]:=aSubStr;
        end;
      end;

    if Self.Locate(sFieldsNames, theSearchArray,
       [loCaseInsensitive,loPartialKey]) then
      begin
        AedtSearch.Font.Color:=clWindowText;
      end
    else
      AedtSearch.Font.Color:=clRed; {Can't find}
    Result:=True;
  except      //Try restore last position
    Result:=False;
  end;
end;

procedure TBS_ClientDataSet.FreeCreatedIndexes;
var
  i:integer;
begin
  if Self.Active then
  try
    IndexDefs.Update; //Refresh
    for i := 0 to IndexDefs.Count - 1 do
      if (IndexDefs.Items[i].Name<>'DEFAULT_ORDER') and (IndexDefs.Items[i].Name<>'CHANGEINDEX')
         and (IndexDefs.Items[i].Name<>'PRIMARY_KEY') then
           begin
             if (Self.IndexName=IndexDefs.Items[i].Name) then
               Self.IndexName:='';
             DeleteIndex(IndexDefs.Items[i].Name);
           end;
  except
    raise;
  end;
end;

////////////////////////////////////////////////////
I use TDBGreedEh descendant, for automatic index creating.
There is part of code from it:
function TBS_DBGridEh.BS_MultiSort:boolean;
var
  i                   :integer;
  theIndexName        :String;
  theIndexFields      :String;
  theDescFields       :String;
  theOldCursor        :TCursor;
  theBS_ClientDataSet :TBS_ClientDataSet;
  fIndexPresent       :boolean;
  s                   :String;

 function DeleteStr(str:String; substr:String): String;
 var i:Integer;
 begin
   i := Pos(substr,str);
   if i <> 0 then
     Delete(str,i,Length(substr));
   Result := str;
 end;

begin
 Result:=False;
 theOldCursor:=Screen.Cursor;
 Screen.Cursor:=crHourGlass;
 try
  if Assigned(Self.DataSource.DataSet as TBS_ClientDataSet)
    and Self.DataSource.DataSet.Active then
    begin
      theBS_ClientDataSet:=TBS_ClientDataSet(Self.DataSource.DataSet);

      theIndexName := '';
      theDescFields:= '';
      theIndexFields:= '';
      for i := 0 to Self.SortMarkedColumns.Count-1 do
        if Self.SortMarkedColumns[i].Title.SortMarker = smUpEh then
          begin
            theIndexFields:=theIndexFields+ Self.SortMarkedColumns[i].FieldName+';';
            theIndexName := theIndexName + Self.SortMarkedColumns[i].FieldName + '_DESC_';
          end
        else
          begin
            theIndexFields:=theIndexFields+ Self.SortMarkedColumns[i].FieldName+';';
            theIndexName := theIndexName + Self.SortMarkedColumns[i].FieldName + '_';
            theDescFields:=theDescFields+ Self.SortMarkedColumns[i].FieldName+';';
          end;

        if theIndexName <> '' then theIndexName := 'idx' + Copy(theIndexName,1,Length(theIndexName)-1);
          theIndexName := DeleteStr(theIndexName,'1');
        if theIndexFields <> '' then theIndexFields := Copy(theIndexFields,1,Length(theIndexFields)-1);
          theIndexFields := DeleteStr(theIndexFields,'1');
        if theDescFields <> '' then theDescFields := Copy(theDescFields,1,Length(theDescFields)-1);
          theDescFields := DeleteStr(theDescFields,'1');

        fIndexPresent:=False;
        theBS_ClientDataSet.IndexDefs.Update;//Refresh
        //If index exist, then set it
        for i := 0 to theBS_ClientDataSet.IndexDefs.Count - 1 do
        if LowerCase(theBS_ClientDataSet.IndexDefs.Items[i].Name) = LowerCase(theIndexName) then
          begin
            fIndexPresent:=True; // Sign
            theBS_ClientDataSet.IndexName := theBS_ClientDataSet.IndexDefs.Items[i].Name;
          end;
        //If index not exist, then try create it
        if Not(fIndexPresent) then
          begin
            theBS_ClientDataSet.AddIndex(theIndexName, theIndexFields,[ixCaseInsensitive],theDescFields,'',0);
          end;

        theBS_ClientDataSet.IndexName := theIndexName;
        Result:=True;
    end;
  finally
    Screen.Cursor:=theOldCursor;
  end;
end;

////////////////////////////////////////////////////////////////////
And there is using filter for searching example(Wark very fast):
procedure TfrmSPORG.edtSearchChange(Sender: TObject);
var
  sFieldsNames  : String;
  i             : integer;
begin
  sFieldsNames:='';
  for i := 0 to FinDM.cdsOrganization.IndexFieldCount - 1 do
    if sFieldsNames='' then
      begin
        sFieldsNames:=sFieldsNames + FinDM.cdsOrganization.IndexFields[i].FieldName;
        break;
      end;
  {$D+L+}
  case FinDM.cdsOrganization.IndexFields[i].DataType of
    ftString, ftFixedChar, ftWideString, ftGuid:
    begin

      if (edtSearch.Text='') or (sFieldsNames='') then
        begin
          FinDM.cdsOrganization.Filtered:=False;
          FinDM.cdsOrganization.First;
        end
      else
        begin
          FinDM.cdsOrganization.Filter:='';
           FinDM.cdsOrganization.Filter:='('+sFieldsNames
             +' LIKE '+ #39+'%'+edtSearch.Text+'%'+#39+')'
             +' or ('+sFieldsNames+'='+#39+edtSearch.Text+#39+')';
        end;
        FinDM.cdsOrganization.Filtered:=True;
        FinDM.cdsOrganization.First;
        FinDM.cdsOrganization.SearchCdsRecord(edtSearch);
    end
  else
    FinDM.cdsOrganization.Filtered:=False;
    FinDM.cdsOrganization.SearchCdsRecord(edtSearch);
  end
end;
 
0
 
LVL 22

Expert Comment

by:Mohammed Nasman
ID: 6468790
as I tested before (but with oracle database)
ADOTable and ADOQuery are much faster than the Table and Query
that's mean use ADO it will be faster than BDE
but still more information needed

0
 
LVL 1

Expert Comment

by:Moondancer
ID: 6976068
ADMINISTRATION WILL BE CONTACTING YOU SHORTLY.  Moderators Computer101 or Netminder will return to finalize these if they are still open in 14 days.  Experts, please post closing recommendations before that time.

Below are your open questions as of today.  Questions which have been inactive for 21 days or longer are considered to be abandoned and for those, your options are:
1. Accept a Comment As Answer (use the button next to the Expert's name).
2. Close the question if the information was not useful to you, but may help others. You must tell the participants why you wish to do this, and allow for Expert response.  This choice will include a refund to you, and will move this question to our PAQ (Previously Asked Question) database.  If you found information outside this question thread, please add it.
3. Ask Community Support to help split points between participating experts, or just comment here with details and we'll respond with the process.
4. Delete the question (if it has no potential value for others).
   --> Post comments for expert of your intention to delete and why
   --> YOU CANNOT DELETE A QUESTION with comments; special handling by a Moderator is required.

For special handling needs, please post a zero point question in the link below and include the URL (question QID/link) that it regards with details.
http://www.experts-exchange.com/jsp/qList.jsp?ta=commspt
 
Please click this link for Help Desk, Guidelines/Member Agreement and the Question/Answer process.  http://www.experts-exchange.com/jsp/cmtyHelpDesk.jsp

Click you Member Profile to view your question history and please keep them updated. If you are a KnowledgePro user, use the Power Search option to find them.  

Questions which are LOCKED with a Proposed Answer but do not help you, should be rejected with comments added.  When you grade the question less than an A, please comment as to why.  This helps all involved, as well as others who may access this item in the future.  PLEASE DO NOT AWARD POINTS TO ME.

To view your open questions, please click the following link(s) and keep them all current with updates.
http://www.experts-exchange.com/questions/Q.20110714.html
http://www.experts-exchange.com/questions/Q.20173874.html
http://www.experts-exchange.com/questions/Q.20170377.html
http://www.experts-exchange.com/questions/Q.20178176.html
http://www.experts-exchange.com/questions/Q.20178177.html
http://www.experts-exchange.com/questions/Q.20229473.html
http://www.experts-exchange.com/questions/Q.20279642.html

To view your locked questions, please click the following link(s) and evaluate the proposed answer.
http://www.experts-exchange.com/questions/Q.20220086.html

*****  E X P E R T S    P L E A S E  ******  Leave your closing recommendations.
If you are interested in the cleanup effort, please click this link
http://www.experts-exchange.com/jsp/qManageQuestion.jsp?ta=commspt&qid=20274643 
POINTS FOR EXPERTS awaiting comments are listed in the link below
http://www.experts-exchange.com/commspt/Q.20277028.html
 
Moderators will finalize this question if in @14 days Asker has not responded.  This will be moved to the PAQ (Previously Asked Questions) at zero points, deleted or awarded.
 
Thanks everyone.
Moondancer
Moderator @ Experts Exchange
0

Featured Post

Vote for the Most Valuable Expert

It’s time to recognize experts that go above and beyond with helpful solutions and engagement on site. Choose from the top experts in the Hall of Fame or on the right rail of your favorite topic page. Look for the blue “Nominate” button on their profile to vote.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this tutorial I will show you how to use the Windows Speech API in Delphi. I will only cover basic functions such as text to speech and controlling the speed of the speech. SAPI Installation First you need to install the SAPI type library, th…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
Integration Management Part 2
Is your data getting by on basic protection measures? In today’s climate of debilitating malware and ransomware—like WannaCry—that may not be enough. You need to establish more than basics, like a recovery plan that protects both data and endpoints.…
Suggested Courses

877 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question