Explain why this is so slowwwwwww. I need to learn this stuff

Respektable
Respektable used Ask the Experts™
on
Would someone please explain to me why method A takes 20 milliseconds.... and method B takes almost 3 seconds? I can't understand why adding to a stringlist would be faster than appending to a string.

procedure TForm1.LoadStringList;
var S:String; x:integer; ID:String;
begin
 for x:= 0 to 10000 do begin
  id:= Inttostr(x);
  SL.add(id);
 End;

//this is very fast
end;

procedure TForm1.LoadPlainDumbString;
  var S:String; x:integer; ID:String;
begin

 for x:= 0 to 10000 do begin
  id:= Inttostr(x);
 S := S + id;

//this is VERY slow. Why?
end;
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Software Engineer, Advisory
Top Expert 2005
Commented:

Appending to string requires reallocations (allocate memory, copy the current contents, append the new contents)

Your method A, using string lists, allocates PString pointers (for the internal dynamic array) in blocks, and appending of strings only requires an allocate and copy.

Russell

Author

Commented:
Ah,, well that explains it.Thanks  Maybe you can also explain the fastest way to build an index to a stringlist.

I find that using Stringlist.indexofName(NameX) to be rather slow.

I needed a faster method, so I re-indexed the entire list into a simplestring....

eg. IndexString:= Name1~0; Name2~1; Name3=2... etc

I then find the ID with  

  p:=  Pos(String,1,length(IndexString);

Then I copy and extract the index value from the string. This gives me StringList[x]

This is many, many times faster than a stringlist looking up a name with indexofnames.  1) Is there an even faster way to retrieve the index value? 2) is there a faster way to build then index than loading a stringlist first? (now that i know that the list is faster than appending)
Russell LibbySoftware Engineer, Advisory
Top Expert 2005

Commented:

Well....

The reason why IndexOfName is so slow is that it walks from first -> last item. So the search time is directly related to (1) the number of items (2) where the search item falls in the list.

This could be made faster, but it would require sorting the list. The problem with that is the items will be moved around, and it sounds like the index is important to you.

A solution to this would be to use a sorted TStringList, but when you perform the add, you use AddObject(), that way you can maintain the natural order, ex:

List.AddObject(Name, Pointer(List.Count))
List.AddObject(Name, Pointer(List.Count))
List.AddObject(Name, Pointer(List.Count))

The Find function could then be used (it performs a split search) to get a starting point in the list, which you could then perform your checks on. A side note on this; better to keep list unsorted while adding large numbers of items, due to the shifting that would take place (memory copies)

Or, you could use the Pos function like you have above, but perform it on the List.Text property. If you look at how that property works, it iterates the list of strings once to get the total size needed for the result string, allocates that much string space (using setlength), then copies (moves) the strings into the result string.

Hope this helps,
Russell





Author

Commented:
Thanks russell. I wasn't aware I could use Find Object like that.
My current pos works rather fast. Like 1 millisecond... compared to 50-60 for a standard stringlist (on 50000 records).  I can live with 1 millisecond per search ;)

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial