Objects in TList too slow

I want a tree with Folder names and cumulated size in it.

Goal: Scan through folders plus subfolders and create an tree with size and cumulated size of folders.

Dirty solution: No tree rather then a sum of files in that tree in a stringlist. Browsing not possible.

Better: An size object with folder name and size, parent folder and children in a tlist. The scan creates an object tree with all those data. Browsing through tree possible.

I have allready created both classes.


The problem is, that the stringlist approach is faster! .. (for the first scan)
In the object list I have to create an object a thousand times, whereby I fill the stringlist with thousand strings in a shorter time.

Is there a better way?
Is there a way to improve the speed of the object creation???
(A size object has names, size, parent folder and child folderobjects in a tlist.  Creating those objects takes the longest time).
No goal is to create +1000 objects when I start the program!

Points for a managable solution which helps me to make this stuff faster.
I want to be able to browse through the results. (I use a treeview to display like explorer)
LVL 1
hush021299Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

kretzschmarCommented:
i would advice to use mike lischke's VirtualStringTree

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Lee_NoverCommented:
or check VirtualShellTools based on mikes' VTV that do just what you want :)
kretzschmarCommented:
seems i'm a bit outdated :-))
Exploring ASP.NET Core: Fundamentals

Learn to build web apps and services, IoT apps, and mobile backends by covering the fundamentals of ASP.NET Core and  exploring the core foundations for app libraries.

hush021299Author Commented:
ups youre fast.
I am talking about a class only.
I currently use some different components to display this stuff(TTreeview, TChart) but this is just the visual part.
I consider to display this stuff in the VTV later on, but this will be another story. Btw. VTV needs to have an object which will be displayed, so the same thing is required.

I just want to get the non visual part in line.

I think maybe I should do this in threads. The first instance of folders on a root folder, each in a different thread. But first I dont know much about threads, second the number of folders can not be limited and third, how to bring it together?

kretzschmarCommented:
maybe you should show, how you do it now

maybe it is better to use records rather than objects
(no object-overhead), but this depends on what you do

usual the sort into a tree costs more time than the collection itself

meikl ;-)
Lee_NoverCommented:
creating 1000 objects shouldn't take more than 10ms really
it's the searching part that's slow
hush021299Author Commented:

This one builds the  object tree and count folders (also visualized in the statusbar)
after that it calc size, which is indicated in a gauge. Finally he builds the treeview.
The counting of the folders, equally building the object tree with this many tlists take long. Counting size afterwards is fast.

This is the code:

 
I call it like:

           newFolder := STree.RootFolder.AddFolder(EName.Text);
            STree.RootFolder.CreateObjecttree(newFolder);

unit SizeTreeU;


interface
uses Windows, Messages, SysUtils, Controls, Forms, Dialogs, Classes;
const
    WM_SCANPROGRESS = WM_USER;
    WM_SCANMAX = $0401; //WM_USER;
    WM_SCANCOUNT = $0402; //WM_USER;

type
    TSizeTree = class;
    TSizeFolder = class;

    TSizeTree = class
    private
        FRootFolder: TSizeFolder;
        FL: string; //list of all folders
        function CalcFolders(Fol: TSizeFolder): TSizefolder;
        function RecurseSize(Fol: TSizeFolder): TSizefolder;
   //function ClearNode(Node: TSizeFolder): TSizeFolder;
    public
        destructor destroy; override;
        constructor create;
        function CalcFolderList:boolean;
        function FindFolder(FolderPath: string): TSizeFolder;
        property RootFolder: TSizeFolder read FRootFolder;

    end;

    TSizeFolder = class
    private
        FSizeOfFiles, FTotalsize: extended;
        Folders: TList;
        FFName: string;
        FShortName: string;
        FParentFolder: TSizeFolder;

        constructor CreateRootFolder;
        function GetSizeOfFiles: extended;
        procedure SetSizeOfFiles(const Value: extended);
        function GetTotalSize: extended;
        procedure SetTotalSize(const Value: extended);
        //procedure CumulateTotalSize;
        function ListFolders(StartFolder: TSizeFolder): TSizeFolder;
        function CreateDirObjectTree(StartFolder: TSizeFolder): TSizeFolder;

    public

        constructor Create(Parent: TSizeFolder; const FName: string);
        destructor Destroy; override;
        function GetParentFolder(Folder: TSizeFolder): TSizeFolder;
        function AddFolder(const FolderName: string): TSizeFolder;
        function getFolderCount: Integer;
        function getFolder(index: Integer): TSizeFolder;

        property FName: string read FFname;
        property ShortName: string read FShortname write FShortname;
        //property NoOfFolder: integer read FNoOfFolder write fNoOfFolder;
        function CreateObjectTree(StartFolder: TSizeFolder): TSizeFolder;
        property SizeOfFiles: extended read GetSizeOfFiles write SetSizeOfFiles;
        property CumulatedSize: extended read GetTotalSize write SetTotalSize;
    end;


function GetSizeOfFilesInDir(Folder: string): extended;


var
    FNoOfFolder: integer;
    ProgressMax: integer;
    TotalProgress: integer;
    FBreak:boolean;

implementation

uses SizeTreeMainU;

function GetSizeOfFilesInDir(Folder: string): extended;
//get the cumulated file size per dir
var
    Search: TSearchrec; //frei:Double;
    aPath: string; Asize: extended;
begin

    apath := IncludeTrailingPathDelimiter(Folder);
    Asize := 0;
    if FindFirst(APath + '*.*', faAnyFile, search) = 0 then
        repeat
            if (Search.Name[1] <> '.') and (Search.Name[1] <> '..') then
            begin
                Asize := Asize + ((search.Size / 1024 / 1024));
            end
        until FindNext(Search) <> 0;
    result := aSize;
end;


constructor TSizeTree.create;
begin
    FRootFolder := TSizeFolder.CreateRootFolder;
    //FRootFolder.FshortName:='RooT';
    //sl := tstringlist.create;
end;

destructor TSizeTree.destroy;
begin

    FRootFolder.Free;
    FRootFolder := nil;
end;

function TSizeFolder.AddFolder(
    const FolderName: string): TSizeFolder;
begin
    assert(FolderName <> '');
    result := TSizeFolder.Create(Self, FolderName);
    Folders.Add(result);
end;

constructor TSizeFolder.Create(Parent: TSizeFolder;
    const FName: string);
begin
    FBreak:=false;
    Self.FParentFolder := Parent;
    Self.FFName := FName;
    //List for the folders
    Folders := TList.Create;
end;

constructor TSizeFolder.CreateRootFolder;
begin
    //1st TList, this is for rootfolder
    Folders := TList.Create;
end;

destructor TSizeFolder.Destroy;
var i: Integer;
begin
 // first destroy all Folders
    for i := 0 to getFolderCount - 1 do
        getFolder(i).Free;

    Folders.Free;
    Folders := nil;
    inherited;
end;

function TSizeFolder.getFolder(index: Integer): TSizeFolder;
begin
    assert((index >= 0) and (index < getFolderCount));
    result := TSizeFolder(Folders[index]);
end;

function TSizeFolder.getFolderCount: Integer;
begin
    result := Folders.Count;
end;

function TSizeFolder.GetSizeOfFiles: extended;
begin
    result := FSizeOfFiles;
end;


function TSizeFolder.ListFolders(StartFolder: TSizeFolder): TSizeFolder;
//builds an object tree with all dirs and subdirs in the rootdir
var
    Search: TSearchrec;
    aPath: string;
    AFolder: TSizeFolder;
begin
    assert(StartFolder.FName <> '');
    aPath := IncludeTrailingPathDelimiter(StartFolder.FName);
    if FindFirst(APath + '*.*', faAnyFile, search) = 0 then
        repeat
            if ((Search.Attr and faDirectory) = faDirectory) and
                (Search.Name[1] <> '.') and not fBreak then
            begin

                AFolder := startfolder.AddFolder(APath + Search.name);
                AFolder.ShortName := Search.name;
                inc(fNoOfFolder);

               
                Application.ProcessMessages;
                  if GetKeyState(VK_Escape) and 128 = 128 then
                  begin
                    FBreak:=True;
                    break;
                  end;


                SendMessage(Form1.Handle, WM_SCANCOUNT, fNoOfFolder, 0);
                application.ProcessMessages;
                ListFolders(AFolder);
            end;
        until FindNext(Search) <> 0;
    FindClose(Search);
    result := AFolder; //ich glaube das ist der letzte folder in der liste
end;

function TSizeFolder.CreateObjectTree(StartFolder: TSizeFolder): TSizeFolder;
var //create the list of stree objects (folders)
    AFolder: TSizeFolder;
begin
    //Rootfolder
    AFolder := startfolder;
    AFolder.ShortName := extractfilename(Afolder.FFName);
    //calc all files in root folder
    Afolder.SetSizeOfFiles(GetSizeOfFilesinDir(afolder.FFName));
    //make object tree with dir objects
    result := CreateDirObjectTree(Afolder);



end;

procedure TSizeFolder.SetSizeOfFiles(const Value: extended);
begin
    FSizeOfFiles := FSizeOfFiles + value;
end;

function TSizeFolder.GetTotalSize: extended;
begin
    result := FTotalsize;
end;

procedure TSizeFolder.SetTotalSize(const Value: extended);
begin
    FTotalSize := Value;
end;

function TSizeFolder.GetParentFolder(Folder: TSizeFolder): TSizeFolder;
begin
    Assert(Folder <> nil);
    result := folder.FParentFolder;
end;


function TSizeFolder.CreateDirObjectTree(StartFolder: TSizeFolder): TSizeFolder;
begin
    FNoOfFolder := 0;
    FBreak:=false;
    result := ListFolders(StartFolder);
end;

function TSizeTree.CalcFolderList:boolean;
begin
    result:=not FBreak;
    if not FBreak then
    begin
    //gauge
    TotalProgress := 0;
    ProgressMax := fNoOfFolder;
    SendMessage(Form1.Handle, WM_SCANMAX, ProgressMax, 0);

    if FRootfolder <> nil then
        CalcFolders(FRootFolder);
    end else FBreak:=false;
end;

function Round2Decimal(Value: currency): currency;
var
    aux: Extended;
begin
    aux := Frac(Value);
    aux := aux * 100;
    aux := Round(aux);
    aux := aux / 100;
    Result := Int(Value) + aux;

end;


function TSizeTree.CalcFolders(Fol: TSizeFolder): TSizefolder;
//recursive, daher kann ich den root hier nicht bearbeiten
var i, c: integer; //s:string;
    AFol: TSizefolder;
begin
    for i := 0 to fol.getFolderCount - 1 do // downto 0 do
    begin
        AFol := fol.getFolder(i);
        AFol.SetSizeOfFiles(GetSizeOfFilesinDir(AFol.FFName));
        RecurseSize(aFol);
        //gauge
        TotalProgress := TotalProgress + (1);
        SendMessage(Form1.Handle, WM_SCANPROGRESS, TotalProgress, 0);
        Application.ProcessMessages;
        //end gauge
        CalcFolders(afol);
    end;
end;

function TSizeTree.RecurseSize(Fol: TSizeFolder): TSizefolder;
var i: integer; //s:string;
    TFS, SOF: extended;
    PFol: TSizefolder;

    function Recurse(Fol: TSizefolder): TSizefolder;
    var PFol: TSizeFolder;
    begin
        pFol := Fol.GetParentFolder(Fol);
        if pFol <> FRootFolder then
        begin
            PFol.FTotalsize := PFol.FTotalsize + TFS + SOF;
            if PFol.GetParentFolder(PFol) <> nil then Recurse(pFol);
        end;
    end;

begin
    SOF := Fol.FSizeOfFiles;
    TFS := Fol.FTotalsize;
    if Fol <> FRootFolder then Recurse(FOL);
end;

{
procedure TSizeTree.ClearAll;
begin
   self := nil;
end;
}

function TSizeTree.FindFolder(FolderPath: string): TSizeFolder;
var node: TSizeFolder;

    function find(folderpath: string; fol: tsizefolder): tsizefolder;
    var afol: TSizefolder; i, c: integer;
    begin

        c := fol.getfoldercount;
        if c > 0 then
            for i := 0 to c - 1 do
            begin
                aFol := fol.getFolder(i);
                if aFol.FFName = excludetrailingbackslash(Folderpath) then
                begin
                    node := aFol;
                    result := aFol;

                end else
                    if result <> nil then find(folderpath, afol);
            end;
    end;

begin
    result := nil;
    find(folderpath, FRootfolder.getFolder(0));

    if node = nil
        then
        result := FRootfolder.getFolder(0) else result := node;
end;


end.
hush021299Author Commented:
Its a pitty, I will ask this again in a while. I hope kretschmar will find a solution

regards
hh
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Delphi

From novice to tech pro — start learning today.