ThievingSix
asked on
Fastest way to read text and parse
I want to find a fast way to read large(100mb+) text files and parse them into a ListBox(TStringList).
ASKER
LoadFromFile is much slower than even readln, and you can't use a progressbar or processmessages.
in case you want to parse fragments and put them into the ListBox tzaje a look to
https://www.experts-exchange.com/questions/20698441/Searching-with-Assembly-code-embedded.html
meikl ;-)
https://www.experts-exchange.com/questions/20698441/Searching-with-Assembly-code-embedded.html
meikl ;-)
tzaje -> take
odd fingers :-))
meikl ;-)
odd fingers :-))
meikl ;-)
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
> LoadFromFile is much slower than even readln, and you can't use a progressbar or processmessages.
the speed increase I was talking about was the beginupdate...endupdate
like I said... it depends on how you intend to parse...I wouldn't use LoadFromFile in many situations (for large files), I'd use TFileStream, then working on the bytes in memory or with a TStringStream/Pchar etc. But I really need to know what kind of parsing you are doing before I can tailor a faster solution for you.
the speed increase I was talking about was the beginupdate...endupdate
like I said... it depends on how you intend to parse...I wouldn't use LoadFromFile in many situations (for large files), I'd use TFileStream, then working on the bytes in memory or with a TStringStream/Pchar etc. But I really need to know what kind of parsing you are doing before I can tailor a faster solution for you.
here's an example of simply reading a big string that I use (thanks ciuly a year ago for providing it for me)
var
BigText: string;
procedure TBigFileParser.LoadFile(filename: string);
var
FS: TFileStream;
begin
FS := TFileStream.create(filename, fmOpenRead);
try
SetLength(BigText, FS.Size);
FS.Read(PAnsiChar(BigText)^, FS.Size);
finally
FS.Free;
end;
end;
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
listbox1.items.BeginUpdate
try
listbox1.Items.LoadFromFil
finally
listbox1.Items.EndUpdate;
end;
As for the parsing, it depends what you are doing, string manipulation is notoriously slow, but depending on your requirements, using a TStream may be beneficial