What is the fastest array in delphi

I have a File, a record type file or an untype file and during startup I want to load the 3 records into an array
Im not sure if:

--------------------------
var a1:Array of String //dynamic array
SetValue(a1, count+1);
a1[count]:= Myrecord1 //adds many records, about 900k
for count = ...
if a1[count] = 'TheOtherRecord' then ...
--------------------------

is faster than

------------------------
var a2:TstringList //hashed array
a2.Add(MyRecord1) //adds many records, about 900k
a2.Names(count)
for count = ...
if a2.Names(count) = 'TheOtherRecord' then ...
-----------------------


And I did not test it yet, because my records size is not enough to test, But if anyone of you has experience or a sure knowledged of fastest array in delphi, just answer what is it in your openion.

Thanks
LVL 14
systanAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

cjcsoftCommented:
code1 is faster than code2,

TStringList's source code will show you why it's slower than Array of String.

1.
 TStringList.add() finally called InserItem(),  it did a lot of things in InsertItem(().


procedure TStringList.InsertItem(Index: Integer; const S: string; AObject: TObject);
begin
  Changing;
  if FCount = FCapacity then Grow;
  if Index < FCount then
    System.Move(FList^[Index], FList^[Index + 1],
      (FCount - Index) * SizeOf(TStringItem));
  with FList^[Index] do
  begin
    Pointer(FString) := nil;
    FObject := AObject;
    FString := S;
  end;
  Inc(FCount);
  Changed;
end;


2.
 TStrings.Names() finally called ExtractName(),  it's even slower than InsertItem()


function TStrings.ExtractName(const S: string): string;
var
  P: Integer;
begin
  Result := S;
  P := AnsiPos(NameValueSeparator, Result);
  if P <> 0 then
    SetLength(Result, P-1) else
    SetLength(Result, 0);
end;


you can see every time you call the names(), it calls a AnsiPos(). it may cost some of time especially in a loop.




0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Emmanuel PASQUIERFreelance Project ManagerCommented:
first, this does not make much sense : "I have a File, a record type file or an untype file and during startup I want to load the 3 records into an array" .

second, you are comparing two different things, as Names property of TStringList will get you the 'name' of a name=value pair in your stringlist. So it has to extract the string before the '=' that **might** be there. I suppose that is not what you want. You should use the Items property instead, which is also the default array property of a TStringList.

for i:=0 to List.Count-1 do if List.Items[i]='something' Then ...
which can be also written like :
for i:=0 to List.Count-1 do if List[i]='something' Then ...

If you want to search a value in a StringList, easiest way is the use of IndexOf method :
i:=List.IndexOf('something');
if i>=0 Then ...

Note that TStringList is NOT a hashlist. But it can be a sorted list (if you don't care about the order of the values). If that is so (just set the Sorted property to true before adding values), IndexOf will work very fast. Otherwise, IndexOf will work pretty much the same as looking in all the values of an array.

In short, TStringList is ALWAYS a better option than a string array you manage yourself, because it has much more useful methods already implemented, and when used with the correct options for your needs, will be fast enough.
0
systanAuthor Commented:
Thank you

Actually, I don't need sorting or any properties in an array.
I just want to clarify that UNTYPED FILE (binary file?)is what I am trying to say.

This one:
:
Type TMyRec = Record
Name: String[15];
////Temporary Only
////MidInitial: Char;
////Age: Integer;
end;

////Global Variables
Const fn = 'UnTypedFile.Dat';
var mLength: Integer;
var PmyRec: TMyRec;
var MyStringArray: Array of String;
var ie: Integer;

procedure StartLoadRec;
var i: integer;
begin
i:=0;
mLength:=0;

 if FileExists(fn) then
    Stream := TFileStream.Create(fn, fmOpenReadWrite)
  else
    Stream := TFileStream.Create(fn, fmCreate);

    while stream.Position < stream.Size  do
    begin
    Stream.ReadBuffer(PmyRec, SizeOf(TmyRec));
    SetLength(MyStringArray, i);
    MyStringArray[i]:= PmyRec.Name;
    i:=i+1;
    end;

    Stream.Free;
    mLength := mLength + i;
end;


////Supposing PmyRec has 900 thousand records Loaded in MyStringArray
Procedure JustCompareEachOne(StringFromOther);
begin
for ie := 0 to (mLength -1) do
begin
////ThisString(StringFromOther) is Function Return Value of Some String
////Crypted is a Function incrypte/decrypte, because before saving the records, it is incrypted
if ThisString(StringFromOther) = Crypted(MyStringArray[ie]) then
begin
////AddToListBox is a procedure adding it to Listbox if are same
AddToListBox( Crypted(MyStringArray[ie]) );
end;
end;


procedure SaveRec(_JustSomeStringHere_);
begin
Stream := TFileStream.Create(fn, fmOpenWrite);
Stream.Position := Stream.Size;
PmyRec.Name := Crypted(_JustSomeStringHere_);
Stream.WriteBuffer(PmyRec,  SizeOf(TmyRec));
Stream.Free;
////Load to Memory - Again when Saving
SetLength(MyStringArray, mLength +1);
MyStringArray[mLength] := Crypted(_JustSomeStringHere_);
end;


//...
//Some codes here to complete the program ...
//...
//End.


NOW
Do you think <TStringList> is _Faster_ if I implement it on that same Code Structure ??
Or there is an <another> that is  _FasterThan_  both of them ??

Excute me for my connected question here:
If i Replace TFileStream with SqLite Database ( during Load/Read time of Recordss ),
Do you think UnTyped-File is  _FasterThan_  SqLite on Reading recordss like my sample above?
0
OWASP: Threats Fundamentals

Learn the top ten threats that are present in modern web-application development and how to protect your business from them.

Emmanuel PASQUIERFreelance Project ManagerCommented:
basically, even the best DataBase have to READ data in a file at some point. So NO, nothing can be quicker than direct file loading.

At this point of your code, I don't see why you bother with TFileStream instead of plain old typed files (see AssignFile, Reset , Read, CloseFile in Delphi Help).

Your JustCompareEachOne is so cryptic that we can't help you much. I'm just wondering why you are calling Crypted on your strings again and again. Why not just do it once and for all on all strings in StartLoadRec ? If you are so much after performance....

As for TStringList, it's very convenient if you have a need to store objects related with a unique string key, and find them quickly. But if you are trying to find those objects based on some calculated value, it's not going to help much. Still, it couldn't hurt more than a basic array.
0
systanAuthor Commented:
>Do you think <TStringList> is _Faster_ if I implement it on that same Code Structure ??
So your answer is NO!, meaning [ array of string is faster ] in my code structure.

>Or there is an <another> that is  _FasterThan_  both of them ??  
Nothing or no comment?  Ok

>If i Replace TFileStream with SqLite Database ( during Load/Read time of Recordss ),
>Do you think UnTyped-File is  _FasterThan_  SqLite on Reading recordss like my sample above?
So NO, nothing can be quicker than direct file loading.

Thank you,
I'll see your advice about cryted
0
Emmanuel PASQUIERFreelance Project ManagerCommented:
> So your answer is NO!, meaning [ array of string is faster ] in my code structure.

No, that's not what I said. if you want to search a list of strings or string+object (see AddObject method of TStringList) , using the string as key, then :
a) if you can afford having your list sorted, then IndexOf will be a lot quicker than anything else, even if you don't actually NEED it to be sorted that is what you SHOULD do.
b) if you can't afford sorting the list because you NEED it to keep its natural order, then IndexOf will be only so little slower that it won't be a problem, considering all the great features you can implement easily using TStringList instead of a plain array.

Your needs are not yet clear enough so that I can give you anything else than global advices. TStringList seems to be what you need, now, and with next requirements.
0
systanAuthor Commented:
Ok,   I got it,   but my code structure is not searching a list of strings,   and I don't want to sort it,   because if you look at my code structure,   you will find just <comparing strings>.    Thats what I ask in details, which is faster when comparing strings only,  is it the variable data type <array of strings> or the data type <TstringList>

But now your advice  is to use <TstringList>, but you mean if I'm going use it with sorting or searching.


And cJcSoft answer is to use <array of strings> and showed some disadvantages of <TstringList>


Thank you to both of you
I'll think what is best, really  (in comparing strings) or even adding thousands of  strings in memory sequencially).
0
Emmanuel PASQUIERFreelance Project ManagerCommented:
a string is a string whether it's in a record array or in a stringlist. Comparing them takes the same time once located.
cJcsoft is not accurate on some points.
1) InsertItem does a few things, yes, but they are mainly optimizations related to its capacity to insert items and not just add at the end, and to grow not at each added string as you are doing with arrays, but with some anticipation. It maybe a bit slower to use TStringList that can do much, compared to manually doing only what you need with arrays, but that's NOTHING compared to other tasks like comparing 2 strings, or loop through all values without optimizations like sort.
2) ExtractName is slow yes, but is not what you want/should use. Use the Items property directly. That's almost as quick as an array, and used the same way.

I advise you to use TStringList because it's what would leave you with more options to make your app grow in functionality. But if you are more confident with the usage of arrays, and you won't have that much changes, then go with arrays. a TStringList IS a dynamic array if you look deeper in its implementation. Which is why you won't see much difference between the 2. You can reinvent the wheel if that's what takes you the less time.
0
aikimarkCommented:
what is missing from this problem is the context.  Your question asks which of two configurations is faster, but doesn't ask what is the fastest possible method to manage X data with Y volume and Z processing operations.

I suggest you look at the speed and functionality of TClientDataset, using the CDS (binary) format for loading your data.

By the way...is your string data simple ASCII characters (<127) or more complex strings?
0
systanAuthor Commented:
@epasquier, Thank you for the clarification and advice

@aikimark, Yes it only has ascii simple characters, but what do you mean <127 ?
0
aikimarkCommented:
127 is the byte value of the simple ASCII character set.  The extended ASCII characters have byte values 128-255.

The simple ASCII character set might allow you to store your data as PCHAR, which might provide alternative speedy implementations.
0
systanAuthor Commented:
Ok, do you mean by setting < array of PChar >  instead of < array of String > would make a difference in speed?
0
aikimarkCommented:
You have only hinted at the data structure you need to store, so I'm not going to suggest anything until I know more.  It is possible that you have a very complex data structure and not just an iterative string/pchar set.

We don't know what kind of processing you require once you have read the text from the hard drive.

Context is king to understanding your problem.
0
systanAuthor Commented:
ID: 30849136
If you've read my code structure above, that's the only important of my code, loading and comparing strings, no sorting, no searching.
0
aikimarkCommented:
@systan

You are presenting us with a partial view of code you have written.  What are you trying to DO with the data?  What kind of data will you have in production?
0
systanAuthor Commented:
Actually that are my test, I'm just knowing delphi, thats all, Thanks.
0
aikimarkCommented:
you are learning to program with Delphi (Object Pascal)?
0
systanAuthor Commented:
ah, Ok, thanks.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Delphi

From novice to tech pro — start learning today.