Solved

With more than 100,000 Record, Locate Methode is Lame duck...How to Get higher speed in searching progress....

Posted on 2001-09-04
6
285 Views
Last Modified: 2010-04-06
          I?m writing a code with A Medium size table, ?Member? Table. Problems I have encounter is that I try to create Search-Form to provide user interface by letting user key in MemberId in Edit Box and write OnChange Event of EditBox. Form only have a DBGrid with 3 Columns of MemberId, MemberName, MemberSurname. But searching speed suffer user too much and make my Search-Form lame duck...By using a Locate Methode, I?ve created Index Name ?Member01? in which indexed on Field ?MemberId? but it?s still Run with too bad speed
        I wonder if there?s anyway to write a Search-Form to provide user interface by entering a MemberId and Dataset move to Such Record first found
        For example, Table contain 3 Record
          Record 1              MemberId = 0001  MemberName = Name01 ....
          Record 2              MemberId = 0101 MemberName = Name02
          Record 3              MemberId = 0201 MemberName = Name03
     If User Key in 01   Form Must move RecordPointer to Record 2, if User Key in 02 RecordPointer must move to Record 3 and So on...
           What I want is that Record 999       Form Must move Record Pointer to Record of MemebrId 99900 Within 2 Seconds At least on Pentium II-400  , say, must be faster depend on Spec of Computer running it. Any real expert out there Show me how to do so, I?ll give you 200 points
0
Comment
Question by:DOOM_DAEMON
6 Comments
 
LVL 10

Expert Comment

by:Jacco
ID: 6453093
What component are you using?

TQuery.Locate never uses index

Regards Jacco
0
 
LVL 3

Expert Comment

by:karouri
ID: 6453794
Listening.
I just hope that these 100,000 records are not in a Microsoft Access database..
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6455848
listening
(multiple ways possible, but some more info needed)
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 3

Accepted Solution

by:
oraelbis earned 200 total points
ID: 6457587
I recomend to use TClientDataSet(from Midas tab) and create indexes on the fly(creating indexes very fast).
After creating indexes you can use them for fast searching, or filtering. This component support additional filter options.(Note: before close it you may free all created indexes).
There is some code from my TClientDataset descendant (you cand find this components sourcecode on http://bs-elbis.chat.ru) with supporting search function. Use Tedit control on your form such as:
procedure TfrmEmployee.edtSearchChange(Sender: TObject);
begin
  //cdsEmloyee TBS_Client dataset
  cdsEmloyee.SearchCdsRecord(edtSearch);
end;


function TBS_ClientDataSet.SearchCdsRecord(const AedtSearch:Tedit):boolean;
var
  sFieldsNames:String;
  i:integer;
  theSearchText:String;
  aSubStr:String;
  theSearchArray:Variant;

begin
  sFieldsNames:='';
  theSearchText:=AedtSearch.text;
  Result:=False;
  try
    theSearchArray := VarArrayCreate([0, Self.IndexFieldCount - 1], varVariant);
    for i := 0 to Self.IndexFieldCount - 1 do
      begin
        if sFieldsNames<>'' then
          sFieldsNames:=sFieldsNames+';';
        sFieldsNames:=sFieldsNames + Self.IndexFields[i].FieldName;
        if Pos(';',theSearchText)>0 then
          begin
            aSubStr:=Copy(theSearchText,1,Pos(';',theSearchText)-1);
            theSearchText:=Copy(theSearchText,Pos(';',theSearchText)+1,
                           Length(theSearchText)-(Pos(';',theSearchText)));
          end
        else
          begin
            aSubStr:=theSearchText;
            theSearchText:='';
          end;
        try
          case Self.IndexFields[i].DataType of
            ftSmallint,ftInteger,ftBytes,ftVarBytes,
            ftAutoInc,ftLargeint,ftTypedBinary:
              if aSubStr<>'' then
                theSearchArray[i]:=StrToInt(aSubStr)
              else
                theSearchArray[i]:=0;
            ftWord,ftFloat,ftBCD,ftCurrency:
              if aSubStr<>'' then
                theSearchArray[i]:=StrToFloat(aSubStr)
              else
                theSearchArray[i]:=0;
            ftDate, ftTime, ftDateTime:
              theSearchArray[i]:=StrToDate(aSubStr);
          else
            theSearchArray[i]:=aSubStr;
          end;
        except
          theSearchArray[i]:=aSubStr;
        end;
      end;

    if Self.Locate(sFieldsNames, theSearchArray,
       [loCaseInsensitive,loPartialKey]) then
      begin
        AedtSearch.Font.Color:=clWindowText;
      end
    else
      AedtSearch.Font.Color:=clRed; {Can't find}
    Result:=True;
  except      //Try restore last position
    Result:=False;
  end;
end;

procedure TBS_ClientDataSet.FreeCreatedIndexes;
var
  i:integer;
begin
  if Self.Active then
  try
    IndexDefs.Update; //Refresh
    for i := 0 to IndexDefs.Count - 1 do
      if (IndexDefs.Items[i].Name<>'DEFAULT_ORDER') and (IndexDefs.Items[i].Name<>'CHANGEINDEX')
         and (IndexDefs.Items[i].Name<>'PRIMARY_KEY') then
           begin
             if (Self.IndexName=IndexDefs.Items[i].Name) then
               Self.IndexName:='';
             DeleteIndex(IndexDefs.Items[i].Name);
           end;
  except
    raise;
  end;
end;

////////////////////////////////////////////////////
I use TDBGreedEh descendant, for automatic index creating.
There is part of code from it:
function TBS_DBGridEh.BS_MultiSort:boolean;
var
  i                   :integer;
  theIndexName        :String;
  theIndexFields      :String;
  theDescFields       :String;
  theOldCursor        :TCursor;
  theBS_ClientDataSet :TBS_ClientDataSet;
  fIndexPresent       :boolean;
  s                   :String;

 function DeleteStr(str:String; substr:String): String;
 var i:Integer;
 begin
   i := Pos(substr,str);
   if i <> 0 then
     Delete(str,i,Length(substr));
   Result := str;
 end;

begin
 Result:=False;
 theOldCursor:=Screen.Cursor;
 Screen.Cursor:=crHourGlass;
 try
  if Assigned(Self.DataSource.DataSet as TBS_ClientDataSet)
    and Self.DataSource.DataSet.Active then
    begin
      theBS_ClientDataSet:=TBS_ClientDataSet(Self.DataSource.DataSet);

      theIndexName := '';
      theDescFields:= '';
      theIndexFields:= '';
      for i := 0 to Self.SortMarkedColumns.Count-1 do
        if Self.SortMarkedColumns[i].Title.SortMarker = smUpEh then
          begin
            theIndexFields:=theIndexFields+ Self.SortMarkedColumns[i].FieldName+';';
            theIndexName := theIndexName + Self.SortMarkedColumns[i].FieldName + '_DESC_';
          end
        else
          begin
            theIndexFields:=theIndexFields+ Self.SortMarkedColumns[i].FieldName+';';
            theIndexName := theIndexName + Self.SortMarkedColumns[i].FieldName + '_';
            theDescFields:=theDescFields+ Self.SortMarkedColumns[i].FieldName+';';
          end;

        if theIndexName <> '' then theIndexName := 'idx' + Copy(theIndexName,1,Length(theIndexName)-1);
          theIndexName := DeleteStr(theIndexName,'1');
        if theIndexFields <> '' then theIndexFields := Copy(theIndexFields,1,Length(theIndexFields)-1);
          theIndexFields := DeleteStr(theIndexFields,'1');
        if theDescFields <> '' then theDescFields := Copy(theDescFields,1,Length(theDescFields)-1);
          theDescFields := DeleteStr(theDescFields,'1');

        fIndexPresent:=False;
        theBS_ClientDataSet.IndexDefs.Update;//Refresh
        //If index exist, then set it
        for i := 0 to theBS_ClientDataSet.IndexDefs.Count - 1 do
        if LowerCase(theBS_ClientDataSet.IndexDefs.Items[i].Name) = LowerCase(theIndexName) then
          begin
            fIndexPresent:=True; // Sign
            theBS_ClientDataSet.IndexName := theBS_ClientDataSet.IndexDefs.Items[i].Name;
          end;
        //If index not exist, then try create it
        if Not(fIndexPresent) then
          begin
            theBS_ClientDataSet.AddIndex(theIndexName, theIndexFields,[ixCaseInsensitive],theDescFields,'',0);
          end;

        theBS_ClientDataSet.IndexName := theIndexName;
        Result:=True;
    end;
  finally
    Screen.Cursor:=theOldCursor;
  end;
end;

////////////////////////////////////////////////////////////////////
And there is using filter for searching example(Wark very fast):
procedure TfrmSPORG.edtSearchChange(Sender: TObject);
var
  sFieldsNames  : String;
  i             : integer;
begin
  sFieldsNames:='';
  for i := 0 to FinDM.cdsOrganization.IndexFieldCount - 1 do
    if sFieldsNames='' then
      begin
        sFieldsNames:=sFieldsNames + FinDM.cdsOrganization.IndexFields[i].FieldName;
        break;
      end;
  {$D+L+}
  case FinDM.cdsOrganization.IndexFields[i].DataType of
    ftString, ftFixedChar, ftWideString, ftGuid:
    begin

      if (edtSearch.Text='') or (sFieldsNames='') then
        begin
          FinDM.cdsOrganization.Filtered:=False;
          FinDM.cdsOrganization.First;
        end
      else
        begin
          FinDM.cdsOrganization.Filter:='';
           FinDM.cdsOrganization.Filter:='('+sFieldsNames
             +' LIKE '+ #39+'%'+edtSearch.Text+'%'+#39+')'
             +' or ('+sFieldsNames+'='+#39+edtSearch.Text+#39+')';
        end;
        FinDM.cdsOrganization.Filtered:=True;
        FinDM.cdsOrganization.First;
        FinDM.cdsOrganization.SearchCdsRecord(edtSearch);
    end
  else
    FinDM.cdsOrganization.Filtered:=False;
    FinDM.cdsOrganization.SearchCdsRecord(edtSearch);
  end
end;
 
0
 
LVL 22

Expert Comment

by:Mohammed Nasman
ID: 6468790
as I tested before (but with oracle database)
ADOTable and ADOQuery are much faster than the Table and Query
that's mean use ADO it will be faster than BDE
but still more information needed

0
 
LVL 1

Expert Comment

by:Moondancer
ID: 6976068
ADMINISTRATION WILL BE CONTACTING YOU SHORTLY.  Moderators Computer101 or Netminder will return to finalize these if they are still open in 14 days.  Experts, please post closing recommendations before that time.

Below are your open questions as of today.  Questions which have been inactive for 21 days or longer are considered to be abandoned and for those, your options are:
1. Accept a Comment As Answer (use the button next to the Expert's name).
2. Close the question if the information was not useful to you, but may help others. You must tell the participants why you wish to do this, and allow for Expert response.  This choice will include a refund to you, and will move this question to our PAQ (Previously Asked Question) database.  If you found information outside this question thread, please add it.
3. Ask Community Support to help split points between participating experts, or just comment here with details and we'll respond with the process.
4. Delete the question (if it has no potential value for others).
   --> Post comments for expert of your intention to delete and why
   --> YOU CANNOT DELETE A QUESTION with comments; special handling by a Moderator is required.

For special handling needs, please post a zero point question in the link below and include the URL (question QID/link) that it regards with details.
http://www.experts-exchange.com/jsp/qList.jsp?ta=commspt
 
Please click this link for Help Desk, Guidelines/Member Agreement and the Question/Answer process.  http://www.experts-exchange.com/jsp/cmtyHelpDesk.jsp

Click you Member Profile to view your question history and please keep them updated. If you are a KnowledgePro user, use the Power Search option to find them.  

Questions which are LOCKED with a Proposed Answer but do not help you, should be rejected with comments added.  When you grade the question less than an A, please comment as to why.  This helps all involved, as well as others who may access this item in the future.  PLEASE DO NOT AWARD POINTS TO ME.

To view your open questions, please click the following link(s) and keep them all current with updates.
http://www.experts-exchange.com/questions/Q.20110714.html
http://www.experts-exchange.com/questions/Q.20173874.html
http://www.experts-exchange.com/questions/Q.20170377.html
http://www.experts-exchange.com/questions/Q.20178176.html
http://www.experts-exchange.com/questions/Q.20178177.html
http://www.experts-exchange.com/questions/Q.20229473.html
http://www.experts-exchange.com/questions/Q.20279642.html

To view your locked questions, please click the following link(s) and evaluate the proposed answer.
http://www.experts-exchange.com/questions/Q.20220086.html

*****  E X P E R T S    P L E A S E  ******  Leave your closing recommendations.
If you are interested in the cleanup effort, please click this link
http://www.experts-exchange.com/jsp/qManageQuestion.jsp?ta=commspt&qid=20274643 
POINTS FOR EXPERTS awaiting comments are listed in the link below
http://www.experts-exchange.com/commspt/Q.20277028.html
 
Moderators will finalize this question if in @14 days Asker has not responded.  This will be moved to the PAQ (Previously Asked Questions) at zero points, deleted or awarded.
 
Thanks everyone.
Moondancer
Moderator @ Experts Exchange
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Need Delphi function to get Youtube video title 5 228
PDF library for Delphi 2 107
Delphi application Soap connection 5 96
URL for downloading Google Chrome for Win XP 2 126
Introduction The parallel port is a very commonly known port, it was widely used to connect a printer to the PC, if you look at the back of your computer, for those who don't have newer computers, there will be a port with 25 pins and a small print…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…
As a trusted technology advisor to your customers you are likely getting the daily question of, ‘should I put this in the cloud?’ As customer demands for cloud services increases, companies will see a shift from traditional buying patterns to new…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now