Link to home
Start Free TrialLog in
Avatar of ThievingSix
ThievingSixFlag for United States of America

asked on

Fastest way to read text and parse

I want to find a fast way to read large(100mb+) text files and parse them into a ListBox(TStringList).
Avatar of TheRealLoki
TheRealLoki
Flag of New Zealand image

reading them can be as fast as
listbox1.items.BeginUpdate;
try
  listbox1.Items.LoadFromFile('test.txt');
finally
  listbox1.Items.EndUpdate;
end;

As for the parsing, it depends what you are doing, string manipulation is notoriously slow, but depending on your requirements, using a TStream may be beneficial
Avatar of ThievingSix

ASKER

LoadFromFile is much slower than even readln, and you can't use a progressbar or processmessages.
in case you want to parse fragments and put them into the ListBox tzaje a look to

https://www.experts-exchange.com/questions/20698441/Searching-with-Assembly-code-embedded.html

meikl ;-)
tzaje -> take

odd fingers :-))

meikl ;-)
SOLUTION
Avatar of dinilud
dinilud
Flag of India image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
> LoadFromFile is much slower than even readln, and you can't use a progressbar or processmessages.
the speed increase I was talking about was the beginupdate...endupdate

like I said... it depends on how you intend to parse...I wouldn't use LoadFromFile in many situations (for large files), I'd use TFileStream, then working on the bytes in memory or with a TStringStream/Pchar etc. But I really need to know what kind of parsing you are doing before I can tailor a faster solution for you.
here's an example of simply reading a big string that I use (thanks ciuly a year ago for providing it for me)

var
  BigText: string;
 
procedure TBigFileParser.LoadFile(filename: string);
  var
    FS: TFileStream;
begin
  FS := TFileStream.create(filename, fmOpenRead);
  try
    SetLength(BigText, FS.Size);
    FS.Read(PAnsiChar(BigText)^, FS.Size);
  finally
    FS.Free;
  end;
end;

Open in new window

ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial