[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Object Tree based on Folder Structure. How to improve speed?

Posted on 2005-04-14
17
Medium Priority
?
359 Views
Last Modified: 2010-04-05
Based on
http://www.experts-exchange.com/Programming/Programming_Languages/Delphi/Q_21243022.html

In above thread is a class which scans a directory structure to build a object tree based on the folder structure.
Unfortunately the search is somewhat slower then another approach which uses stringlists.

In both cases I want to display a folder with his children and the size of files in every child, as well as for the parent folder. It is possible from here to browse through the object tree, because every object contains a tlist with its children, who still refer to the parent. Browsing need no recalculation, since all values are stored in a object. Problem is that the object might be too big, but on the other hand I dont know how to reduce complexity of object without missing the advantages using it.

The object tree need to create an object for every folder. This seems to slow down the process. However, having all the size infos in every folder object makes it great to browse. Also the usage of the class is very easy since I heavily relay (e.g. in a Virtualtreeview to display the stuff) on the advantage using an object e.g.: sizetree.foldername or sizetree.totalsize or so.

Is there an approach to make the thing faster, at the same time still providing some convenience when using (not calculate again everytime I select a child at new parent)
0
Comment
Question by:hush021299
  • 9
  • 7
17 Comments
 
LVL 6

Expert Comment

by:pritaeas
ID: 13780594
Well, if you put the building in a thread, then you can remove the ProcessMessages and SendMessage in e.g. the ListFolders function. This would speed things up a bit.
0
 
LVL 1

Author Comment

by:hush021299
ID: 13784788
I havent done threads yet, but removing thos process.. and sendm.. does speed up a bit, but not essential.
0
 
LVL 34

Accepted Solution

by:
Slick812 earned 1000 total points
ID: 13786120
hello  hush, , since you say that object creation is slowing down your folder search, then it seems that you might should not do so much object creation. One thing I would do is to store the folder information, like file size, in the Tree View TTreeNode for each node, so you can read it from the selected tree Node and display it on your form. . . .
Here is some code for a DirSearch unit that uses the methods I think will do this job, this will create just one object TDirSearch and then use an Array of records to get all of the folder and sub-folder size information. For the TDirSearch, I use a Pointer to a Dynamic Array called  pSubDir  to access the TDirSearch folder's sub-folders, I have used it as a pointer because that is how I have the sub-folders as a pointer to an array. I was forced to use a pointer to an array because delphi will not allow using that record in that record's definition, but you can use a pointer to it. . .

Here is the code for the DirSearch unit - - - -





unit DirSearch;

interface

uses ComCtrls;

type
// PDirInfo is used to strore size data in the Tree View TTreeNode.Data
  PDirInfo = ^TDirInfo;
  TDirInfo = record // record used for the TreeView node Data pointer
    Size, AllSize: Double; // exact same numeric values in the TDirSize record below
    NumSub, AllSub, NumFiles, AllFiles: Cardinal;
    end;


  PAryDS = ^TAryDS;
  // you can declare pointer types before you define the type it is based on

// the TDirSize record will have all the info needed for a Folder and it's sub folders
  PDirSize = ^TDirSize;
  TDirSize = record
    Size, AllSize: Double; // I use a Double (Real) type because it has 8 bytes
    NumSub, AllSub, NumFiles, AllFiles: Cardinal;
  // the values not used in the TreeView TTreeInfo Record are below the
  // values that are used, so I can use the CopyMemory function to fill tree view
    Path, Name: String;
    SubDir: PAryDS;
  // you can NOT use a record type inside the same record type,
  // so I have to use a pointer type PAryDS
  // you will need to Call New( SubDir) function to initialize this SubDir
  end;

  TAryDS = Array of TDirSize;
  // I use a Pointer to this array type in the TDirSize record
 

  TDirSearch = class
    // I usually place all my fields in the protected section
    // so if I make a decendent, I can have access to all fields
    protected
    FSubDir: PAryDS; // maintain the pointer to the array
    FPath, FName: String;
    FNumFiles, FAllFiles, FNumSub, FAllSub, FmsgAmt: Cardinal;
    FSizeMeg, FAllSize: Double;
    FsysNT, FStop: Boolean;
    function GetDirSize(const Folder: String; var NumFiles: Cardinal): Double;
    procedure Search4Sub(var DirSz: TDirSize);
    procedure Init(const aPath: String);
    procedure SetPath(Value: String);
    procedure ClearMem;

    public
      pSubDir: PAryDS;
   { pSubDir is a pointer to an Array which contains a record for each sub-Folder
     and each record for a folder will contain a PAryDS array with all of it's sub-folders
     this starts out as NIL, you must test it before you try and use it }

      constructor Create(const FolderPath: String);
    {when you create the TDirSearch, the root folder path is set, but no disk
     folder or file search is done}
      destructor Destroy; override;

    { the GetSubFolders procedure will do a disk search for sub folders
      it does NOT do a file search. You must call this before you call ScanForFiles.
      only sets the numSubDir below, the rest are still zero}
      procedure GetSubFolders;

    { you call the ScanForFiles procedure to search the folders for files and
      file sizes, this sets 5 of the Folder Info values below -
      SizeMeg, TotalSize, FilesInDir, TotalFiles, TotalDir
      this does nothing if the GetSubFolders has not been called.}
    procedure ScanForFiles;

    // call Cancel to stop an ongoing folder or file search
      procedure Cancel;

      // you can change the path and then do a GetSubFolders
      property Path: String read FPath write SetPath;

    // all properties below are read only
      property Name: String read FName; // folder name, no path
      property SizeMeg: Double read FSizeMeg;
      property TotalSize: Double read FAllSize;
      property FilesInDir: Cardinal read FNumFiles;
      property TotalFiles: Cardinal read FAllFiles;
      property numSubDir: Cardinal read FNumSub; // only value set after GetSubFolders
      property TotalDir: Cardinal read FAllSub;
      property UserCanceled: Boolean read FStop;
    // Read UserCanceled to see if the GetSubFolders and ScanForFiles has completed

    end;


implementation

uses
  Windows, Forms, SysUtils, FileCtrl;




procedure TDirSearch.Init(const aPath: String);
var
i: Integer;
begin
// sets the root folder path and name, zeros field values
FPath := aPath;

if FPath[Length(FPath)] = '\' then // remove the \
  SetLength(FPath, Length(FPath) -1);
i := LastDelimiter('\', FPath);
FName := Copy(FPath, i + 1, MaxInt);

FSizeMeg := 0;
FAllSize := 0;
FNumFiles := 0;
FAllFiles := 0;
FNumSub := 0;
FAllSub := 0;
FStop := False;
FmsgAmt := 0;
end;



constructor TDirSearch.Create(const FolderPath: String);
begin
inherited Create;
if not DirectoryExists(FolderPath) then
  raise Exception.Create('ERROR - TDirSearch Create Failure, Folder does not exist');
FsysNT := (Win32Platform = VER_PLATFORM_WIN32_NT) and (Win32MajorVersion >= 5);
Init(FolderPath);

FSubDir := nil;
pSubDir := nil;
end;

destructor TDirSearch.Destroy;
begin
ClearMem;// you need to release all the array pointers
inherited Destroy;
end;

function TDirSearch.GetDirSize(const Folder: String; var NumFiles: Cardinal): Double;
var
hFind : THandle;
FndData: TWin32FindData;
ErrorMode: Word;

begin
// this will scan a folder and get the file number and file size it contains
Result := 0;
NumFiles := 0;
Application.ProcessMessages;
if Application.Terminated or
  (GetKeyState(VK_Escape) and 128 = 128) then
  FStop := True;
if FStop then Exit;
ErrorMode := SetErrorMode(SEM_FailCriticalErrors);
if FsysNt then
  hFind := Cardinal(FindFirstFileEx(PChar(Folder + '\*.*'), FindExInfoStandard,
                    @FndData, FindExSearchNameMatch, nil, 0))
  else
  hFind := FindFirstFile(PChar(Folder + '\*.*'), FndData);

SetErrorMode(ErrorMode);
if hFind <> INVALID_HANDLE_VALUE then
  with FndData do
  begin
  if (dwFileAttributes and FILE_ATTRIBUTE_DIRECTORY) = 0 then
    begin
    Inc(NumFiles);
    Inc(FMsgAmt);
    if nFileSizeHigh = 0 then
      Result := Result + (nFileSizeLow / 1048576)
      else
      Result := Result + (nFileSizeLow+(FndData.nFileSizeHigh shl 32) / 1048576);
    end;
  while FindNextFile(hFind, FndData) do
    begin
    if (dwFileAttributes and FILE_ATTRIBUTE_DIRECTORY) = 0 then
      begin
      Inc(NumFiles);
      Inc(FMsgAmt);
      if nFileSizeHigh = 0 then
        Result := Result + (nFileSizeLow / 1048576)
        else
        Result := Result + (nFileSizeLow+(FndData.nFileSizeHigh shl 32) / 1048576);
      if FMsgAmt > 350 then
        begin
        // I call ProcessMessages every 350 to keep thread from freezing the Form
        FMsgAmt := 0;
        Application.ProcessMessages;
        if Application.Terminated or
           (GetKeyState(VK_Escape) and 128 = 128) then
           FStop := True;
        if FStop Then
          begin
          windows.FindClose(hFind);
          Exit;
          end;
        end;
      end;
    end;
  windows.FindClose(hFind);
  end;

end;

procedure TDirSearch.ScanForFiles;
var
sd, i: Integer;

  procedure GetSize(var DSz :TDirSize);
  var
  c, n: Integer;
  begin
  // this procedure calls itself to get all sub folders file sizes
  for c := High(DSz.SubDir^) downto 0 do
    with DSz.SubDir^[c] do
    begin
    if NumSub = 0 then
      begin;
      if FStop Then Exit;
      Size := GetDirSize(Path, numFiles);
      AllSize := Size;
      AllSub := NumSub;
      AllFiles := numFiles;
      end  else
      begin
      if FStop Then Exit;
      GetSize(DSz.SubDir^[c]); // get sub folders first
      Size := GetDirSize(Path, numFiles);
      AllSize := Size;
      AllFiles := numFiles;
      AllSub:= NumSub;
      for n := 0 to High(SubDir^) do
        begin
        AllSub := AllSub + SubDir^[n].AllSub;
        AllSize := AllSize + SubDir^[n].AllSize;
        AllFiles := AllFiles + SubDir^[n].AllFiles;
        end;
      end;
    end;
  end;


begin
if Application.Terminated then
  begin
  ClearMem;
  FNumSub := 0;
  Exit;
  end;
// this procedure will call GetDirSize for every folder in the arrays
// this sets 5 of the numeric values for the folders
// 3 of the values (totals) are set after the file search of sub folders
if FSubDir = nil then
  raise Exception.Create('ERROR - ScanForFolders has Not been Done');
FStop:= False;
FSizeMeg := 0;
FSizeMeg := GetDirSize(FPath, FNumFiles);
FMsgAmt := 0;

for sd := High(FSubDir^) downto 0 do
  with FSubDir^[sd] do
  begin
  if NumSub > 0 then
    GetSize(FSubDir^[sd]); // get sub folders first so Total amounts will be set
  if FStop Then Exit;
  Size := GetDirSize(Path, numFiles);
  AllSize := Size;
  AllFiles := numFiles;
  AllSub := numSub;
  if SubDir = nil then Continue;
  for i := 0 to High(SubDir^) do
    begin
    AllSub := AllSub + SubDir^[i].AllSub;
    AllFiles := AllFiles + SubDir^[i].AllFiles;
    AllSize := AllSize + SubDir^[i].AllSize;
    end;
  end;

FAllSize := FSizeMeg;
FAllFiles := FNumFiles;
FAllSub := FNumSub;

if FSubDir = Nil then Exit;
for i := 0 to High(FSubDir^) do
  begin
  FAllSub := FAllSub + FSubDir^[i].AllSub;
  FAllFiles := FAllFiles + FSubDir^[i].AllFiles;
  FAllSize := FAllSize + FSubDir^[i].AllSize;
  end;
end;

procedure TDirSearch.Search4Sub(var DirSz: TDirSize);
var
hFind : THandle;
FndData: TWin32FindData;
ErrorMode: Word;
i: Integer;

  procedure addIt;
  begin
  // sets the length of array and puts the path and name
  with DirSz, FndData do
    begin
    if NumSub = 0 then
      New(SubDir);
//New(DirSz.SubDir); // you Must initialize every array pointer or access violations
// you Must also Dispose of the array pointer later, when resetting folders or close
    Inc(NumSub);
    Inc(FMsgAmt);
    SetLength(SubDir^, Length(SubDir^)+1);
    SubDir^[High(SubDir^)].Path := Path+'\'+ cFileName;
    SubDir^[High(SubDir^)].Name := cFileName;
    end;
  end;


begin
// this does a folder search for sub folders in each folder
DirSz.NumSub := 0;
DirSz.SubDir := nil;
Application.ProcessMessages;
if Application.Terminated or
  (GetKeyState(VK_Escape) and 128 = 128) then
  FStop := True;
if FStop then Exit;
Inc(FmsgAmt);
ErrorMode := SetErrorMode(SEM_FailCriticalErrors);
// I use the FindFirstFileEx with FindExSearchLimitToDirectories in NT systems
// I believe it is faster than the FindFirstFile for Dir
if FsysNT then
  hFind := Cardinal(FindFirstFileEx(PChar(DirSz.Path + '\*.*'),
              FindExInfoStandard, @FndData, FindExSearchLimitToDirectories, nil, 0))
  else
  hFind := FindFirstFile(PChar(DirSz.Path + '\*.*'), FndData);
SetErrorMode(ErrorMode);

if hFind <> INVALID_HANDLE_VALUE then
  with FndData do
  begin
  if (dwFileAttributes and FILE_ATTRIBUTE_DIRECTORY) <> 0 then
    if (cFileName[0] <> '.') and (PWORD(@cFileName)^ <> $2E2E) then
      addIt;

  while FindNextFile(hFind, FndData) do
    begin
    if FMsgAmt > 200 then
        begin
        FMsgAmt := 0;
        Application.ProcessMessages;
        if Application.Terminated or
           (GetKeyState(VK_Escape) and 128 = 128) then
           FStop := True;// stop if program close
        if FStop then
          begin windows.FindClose(hFind); Exit; end;
        end;
    if (dwFileAttributes and FILE_ATTRIBUTE_DIRECTORY) <> 0 then
    if (cFileName[0] <> '.') and (PWORD(@cFileName)^ <> $2E2E) then
      begin
      addIt;
      // alllow ProcessMessages every 200 folders, keep Form responding

      end;
    end;
  windows.FindClose(hFind);
  end;

if DirSz.SubDir <> nil then
for i := 0 to High(DirSz.SubDir^) do
  begin
  // loop through all sub folders and get their subs
  if FMsgAmt > 200 then
    begin
    FMsgAmt := 0;
    Application.ProcessMessages;
    if Application.Terminated or
       (GetKeyState(VK_Escape) and 128 = 128) then
       FStop := True;
    if FStop Then Exit;
    end;
  Search4Sub(DirSz.SubDir^[i]);
  end;
end;

procedure TDirSearch.ClearMem;
var
i: Integer;

  procedure DisMem(var DSz :TDirSize);
  var
  c: Integer;
  begin
  for c := {DSz.NumSub-1 }High(DSz.SubDir^) downto 0 do
    with DSz.SubDir^[c] do
    begin
    if NumSub = 0 then
      begin
      if SubDir <> nil then
        Dispose(SubDir);
      end else
      //if DSz.SubDir^[c] <> nil then
        DisMem(DSz.SubDir^[c]);
    end;
  Dispose(DSz.SubDir);
  end;

begin
// IMPORTANT, , you Must Dispose the array pointer memory to free it and
// the records in the array
if FSubDir <> nil then
  begin
  for i := High(FSubDir^) downto 0 do
    begin
    if FSubDir^[i].NumSub = 0 then
      begin
      if FSubDir^[i].SubDir <> nil then
      Dispose(FSubDir^[i].SubDir)
      end else
      DisMem(FSubDir^[i])
    end;
  Dispose(FSubDir);
  FSubDir := nil;
  pSubDir := nil;
  end;
end;


procedure TDirSearch.GetSubFolders;
var
DirSize: TDirSize;
begin
// this procedure will reset values and search for all sub-folders
ClearMem;
if Application.Terminated then
  begin
  FNumSub := 0;
  Exit;
  end;
DirSize.Name := FName;
DirSize.Path := FPath;
FmsgAmt := 0;
FStop:= False;
Search4Sub(DirSize); // Search4Sub should get all sub folders
FNumSub := DirSize.NumSub;
FSubDir := DirSize.SubDir;
pSubDir := FSubDir;
end;

procedure TDirSearch.SetPath(Value: String);
begin
if not DirectoryExists(Value) then
  raise Exception.Create('ERROR - TDirSearch Path Failure, Folder does not exist');
ClearMem; // need to release memoryof arrays
Init(Value);
end;

procedure TDirSearch.Cancel;
begin
FStop := True;
end;

end.

0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 34

Expert Comment

by:Slick812
ID: 13786158
here is some code in a TForm unit that I used to have the TDirSearch fill a TreeView - -



// button click that starts the folder search
procedure TForm1.but_DoFolderListClick(Sender: TObject);
var
DirSearch1: TDirSearch;

begin
DirSearch1 := TDirSearch.Create('D:\Stuff');

DirSearch1.GetSubFolders; // will get sub-folders, not file sizes
ShowMessage(DirSearch1.Path+' '+IntToStr(DirSearch1.numSubDir));
// only the numSubDir folder info is filled with GetSubFolders
DirSearch1.Path := 'H:'; // use for base Drive folder
DirSearch1.GetSubFolders;
if DirSearch1.UserCanceled then
  begin
  // if user cancels then UserCanceled is true and folder info is incomplete
  Label2.Caption := 'User has Canceled';
  FreeAndNil(DirSearch1);
  Exit;
  end;
ShowMessage(DirSearch1.Path+' '+IntToStr(DirSearch1.numSubDir));

DirSearch1.ScanForFiles;
// ScanForFiles will search all subfolders and get the file sizes
if DirSearch1.UserCanceled then
  begin
  Label2.Caption := 'User has Canceled';
  FreeAndNil(DirSearch1);
  Exit;
  end;
ShowMessage('Size '+FloatToStr(DirSearch1.TotalSize));
ShowMessage('Total files '+IntToStr(DirSearch1.TotalFiles)+' Total Dir '+IntToStr(DirSearch1.TotalDir));

FillTreeView(TreeView1, DirSearch1.pSubDir);
// the FillTreeView procedure will create all the tree nodes and transfer all sub-folder info

FreeAndNil(DirSearch1);
// all folder size info is now in the TreeView nodes, so you can Free DirSearch1
end;




procedure TForm1.FillTreeView(TV: TTreeView; pDAry: PAryDS);

  procedure ReTree(var DirSz: TDirSize; reNode: TTreeNode);
  var
  c: Integer;
  TreeRe: TTreeNode;
  pDInfo: PDirInfo;
  begin
  // this procedure is called for every sub-folder to add dir nodes
  for c := 0 to High(DirSz.SubDir^) do
    with DirSz do
    begin
    TreeRe := TV.Items.AddChild(reNode,SubDir^[c].Name+' * '+
                                   IntToStr(SubDir^[c].NumFiles));
    New(pDInfo);
    CopyMemory(pDInfo, @SubDir^[c], SizeOf(TDirInfo));
    TreeRe.Data := pDInfo;
    if SubDir^[c].NumSub > 0 then
      ReTree(SubDir^[c], TreeRe);
    end;
  end;


var
pDInfo: PDirInfo;
TreeR, TreeC: TTreeNode;
i: Integer;

begin
// this procedure will fill a TTreeView with all the folders in a TDirSearch
// pSubDir array of TDirSize records.
if (not Assigned(TV)) or (pDAry = nil) then Exit;
TV.Items.Clear;
if  (Length(pDAry^) = 0) then Exit;

with TV.Items do
  begin
  with pDAry^[0] do
    begin
    TreeR := Add(nil, Name+' * '+
               IntToStr(NumFiles)); // Add a root node
    New(pDInfo);
   // you will need to initilize the record pointer with the New( ) function
   // you will also need to free this pointer memory with the Dispose( )
    CopyMemory(pDInfo, @pDAry^[0], SizeOf(TDirInfo));
    // since all of the data in the TTreeInfo is numeric I use CopyMemory
    // to place these values into the pTInfo^ record
    TreeR.Data := pDInfo;
    if NumSub > 0 then
      ReTree(pDAry^[0], TreeR);
    end;

  for i := 1 to High(pDAry^) do
    with pDAry^[i] do
    begin
    TreeC := Add(TreeR, Name+' * '+
                 IntToStr(NumFiles));
    New(pDInfo);
    CopyMemory(pDInfo, @pDAry^[i], SizeOf(TDirInfo));
    TreeC.Data := pDInfo;
    if NumSub > 0 then
      ReTree(pDAry^[i], TreeC);
    end;
  end;


TV.OnDeletion := TVDeletion;
// IMPORTANT, you must get the TreeView's OnDeletion event to Dispose of the memory
// I set it here, but you can do it in the object inspector for tree view
end;




procedure TForm1.TreeView1Change(Sender: TObject; Node: TTreeNode);
var
pDInfo: PDirInfo;
begin
// this is the Tree View OnChange event which will show the selected folder size info in 6 Labels
if Node.Data <> nil then
  begin
  pDInfo := Node.Data;
  with pDInfo^ do
    begin
    Label2.Caption := 'Size of Folder '+FloatToStrF(Size,ffNumber,18,4)+' Megs';
    Label3.Caption := 'TotalSize of Folder '+FloatToStrF(AllSize,ffNumber,18,4)+' Megs';
    Label4.Caption := 'Number of files '+IntToStr(NumFiles);
    Label5.Caption := 'Total Number of files '+IntToStr(AllFiles);
    Label6.Caption := 'Number of Sub Dir '+IntToStr(NumSub);
    Label7.Caption := 'Total Number of Sub Dir '+IntToStr(AllSub);
    end;
  end;
end;



procedure TForm1.TVDeletion(Sender: TObject; Node: TTreeNode);
begin
// whenever a tree node is deleted form the Tree View, this is called
if Node.Data <> nil then
  Dispose(Node.Data); // free the memory
end;


 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

this seems to work at a fast rate on my XP machine, and only a little slower on my win 98 machine
I hope you are able to get something usefull from this
0
 
LVL 1

Author Comment

by:hush021299
ID: 13788919
TQ
I get it running and it seems to be another approach with the same results. I will award the points, but I still have a couple of questions before I try to understand more of the code. I will build something to compare the speed, which is btw. only possible during the first scan.

1 I dont want to save the sizes in the treeview, because then I heavily depend on a virtual component. Is it also possible to store it somewhere else?
2 You create the tDirsize with a directory and later I add another path. Where is that for? Shouldnt it be the same?
begin
DirSearch1 := TDirSearch.Create('D:\My Documents\My Programs');
DirSearch1.GetSubFolders; // will get sub-folders, not file sizes
ShowMessage(DirSearch1.Path+' '+IntToStr(DirSearch1.numSubDir));
// only the numSubDir folder info is filled with GetSubFolders
DirSearch1.Path := 'D:'; // use for base Drive folder
0
 
LVL 1

Author Comment

by:hush021299
ID: 13788936
..and why do I get twice the message D:7 (7 Folders in D)
0
 
LVL 1

Author Comment

by:hush021299
ID: 13790282
I ve got your code running and it produces almost the same like my stuff.

Now I did a comparison, and after a restart of Win2000 all 3 variants (your code, an old code of mine just with a tree view, and my app) show almost same results (!:05-1:08 for my D:\ Hard disk).

The second run is always faster, take about for seconds with the small apps and 6 with mine, whereby I lead this to the visual controls.

At the end I can not get it faster.
I assume there is no way to improve the speed further.

But again thanks for the code
0
 
LVL 34

Expert Comment

by:Slick812
ID: 13792014
I  am not really able to understand some of what you are asking. . .you ask -  "I assume there is no way to improve the speed further."
There seems to always be ways to increase speed for code operations, however the speed inprovements are so small and you may have to do very low level code work, that it is not worth the trouble. . . If you are going to be reading a hard disk for it's folder and file information, then the hard disk functional sector access speed will be the limiting factor, no matter what code or coding methods you use, , the speed of a disk operations is far far slower than code operations, so even if you get faster code methods, the disk reading of FAT from the disk platters will probally not speed up. As far as I know you will need to have the hard disk read it's FAT for the FindFirstFile operations. . . .

also I did some perormance test for the creation of a simple object, and it was very fast, so just because there is object creation (lots of objects) it should not slow it down very much, however if you do operations in these objects that take processor time during it's creation, then it get's slower

as to the -

DirSearch1 := TDirSearch.Create('D:\My Documents\My Programs');
DirSearch1.GetSubFolders; // will get sub-folders, not file sizes
ShowMessage(DirSearch1.Path+' '+IntToStr(DirSearch1.numSubDir));
// only the numSubDir folder info is filled with GetSubFolders
DirSearch1.Path := 'D:'; // use for base Drive folder



all of that is just demo code to show how to use the TDirSearch

if I were actually using it I would use code like this

DirSearch1 := TDirSearch.Create('D:');
DirSearch1.GetSubFolders;
if DirSearch1.UserCanceled then
  begin
  // if user cancels then UserCanceled is true and folder info is incomplete
  Label2.Caption := 'User has Canceled';
  FreeAndNil(DirSearch1);
  Exit;
  end;
DirSearch1.ScanForFiles;
if DirSearch1.UserCanceled then
  begin
  Label2.Caption := 'User has Canceled';
  FreeAndNil(DirSearch1);
  Exit;
  end;
FillTreeView(TreeView1, DirSearch1.pSubDir);
FreeAndNil(DirSearch1);


 = = = = = = = = = = = = = = = = = = = = = = = =
you ask - " I dont want to save the sizes in the treeview, because then I heavily depend on a virtual component. Is it also possible to store it somewhere else?"
the answer is yes
0
 
LVL 1

Author Comment

by:hush021299
ID: 13805385
You are right. There is an improvement, but I think the savings are less then I had expected. But I see, that the code is ok as it is, which saves me headache for major renewal actions.
I look how I can improve a little using a profiler then.

Yes one more maybe:
Does it make sense to use threads? (Have never used threads)
Or, maybe run a thread on every drive at once?
0
 
LVL 34

Expert Comment

by:Slick812
ID: 13808827
I Hope that you can get some ideas to learn coding from my code example above, I hate to feel like I spent the time to do that for nothing. . .

OK about Threads. . . As I have come to understand them. . . Threads are a way for the windows system to give different "Processes" a way to "Share" the CPU processing time without ending a code segment or function, , , ,, A Single CPU (processor) can only do ONE THING AT A TIME (except some of the newest CPU or machines with more than one CPU), so the windows system has a method to change which process code will be sent to and used by the CPU during several simutaneous (at the same time) calls to use the CPU, Threads have been usefull to allow one thread to do a "Long time" data process, while the another thread, the main thread, runs the GUI window and is not blocked. .  however many programmers (I thought this at first) beleive that threads can "Speed UP" code operations. .  This is only true if one code operation is blocking or delaying another code operation that is NOT slow because of CPU usage ( Like waiting for data from a slow internet connection), ,  It is NOT TRUE that placing code operations in different Threads will "Speed Up" to time it taks to do ALL of the code in the combined operations, because the CPU can only do one thing at a time, useng sevaral threads may acually slow it down, , , (if you have several CPU in a machine it can speed it up)

File access is the same story, the CPU and Hard disk can only do one file access at a time no matter from which process or thread file access is called, so using threads would not speed up the time needed to the disk read operations, I do not beleive,  (again if there is more than one hard disk and more then one CPU and more than one hardware disk controller you may get a speed increase), There are times where you will need threads, so you should learn how to use them (examples in the Demo files for Delphi and in Delphi help), but for many things, speed increase may not be a good reason to use threads
0
 
LVL 1

Author Comment

by:hush021299
ID: 13873963
PAryDS = ^TAryDS;
  // you can declare pointer types before you define the type it is based on
....
tree view
    Path, Name: String;
    SubDir: PAryDS;
  // you can NOT use a record type inside the same record type,
  // so I have to use a pointer type PAryDS
  // you will need to Call New( SubDir) function to initialize this SubDir
  end;
0
 
LVL 1

Author Comment

by:hush021299
ID: 13873997
Sorry, I clicked to early.
Thanks again for your support, and yes I try to understand your coding. However I am under strong pressure in my job, so programming moved a bit in the background in the last weeks.
But here is one question to above record.
Why do you( one) uses mostly pointers for records,
second:
if it is only because you need to call the record from inside then I dont understand why is that so.
I thought Delphi use Object referencies, doing anything to pointer anyway.
?

Threads:
OK, got it.
I ve got that idea because I had the code of a program doing comparable things on the hard disk as our code.
However, this prog claimed that it works faster, when scanning all drives together, rather then drive by drive. I did not understand why.
0
 
LVL 34

Expert Comment

by:Slick812
ID: 13880056
OK,
I did all of my code methods in my  TDirSearch  as a result of you saying that because you created and  used Objects, your "Speed" was much Slower than when you did not use objects. ( I do not beleive that creating the Objects was what caused your speed to be slower ).  So I tried to code it with arrays of records instead of TLists, and I wanted to only create just ONE TObject the  TDirSearch.. .  this is why I used the pointer to array inside of the record, You could have just used a TList instead of the pointer to an array, or a TObject. . .
For me I sometimes use the pointers and the New( ) so I can use a more flexible coding methods, but in my TDirSearch it was because I wanted NOT to use an Object of any kind (because of your Objects were slow statement) in my records or anywhere in the TDirSearch.

Somethig I wanted to tell you but desided not to was - That you can create your own decendent of a TList, that has a List of a certain Object insted of a List of pointers, and you can Include function and procedures in your TList that will do work on the Objects in your list.

- - - - -

I have also had Components and units that were made by someone else, that claimed to use threads and would "Speed UP" Disk read, disk write, processing time, ect. . . but when I tested some of them, they did not acually show any amount of speed increase that was worth the trouble.

If you don't have time for this, then just go on with your work. . .
0
 
LVL 1

Author Comment

by:hush021299
ID: 13901594
So thanks a lot, I further study your code.
With regards to my time this is perfect, because it saves time. However, just my response time is somewhat slower due to my actual work.

0
 
LVL 34

Expert Comment

by:Slick812
ID: 13901860
another thing,  you said something -

depend on a virtual component

I used the method in my code for storing information for a TreeView, by using the Data of a Tree Node, , like -

TreeR.Data := pDInfo;

this is a old and established way to store your data for a Tree Node in the Tree Node, you can add a string to the record stored with the Path for the node, or you can add or subtract anything to this record and store it with the tree node
0
 
LVL 1

Author Comment

by:hush021299
ID: 14151865
Hello Slick812,

after a "working break" I have one more question in the content of this topic.
How can I scan all drives at once?
Can this be done without sequential work for all drives in a way like c:, d: ...
I think I could use (like mentioned above) a thread for each drive. So I can create up to 24 threads (However, I have not much experience working on threads) for up to 24 drives.

THis could be a seperate topic, If this could be a solution
0
 
LVL 34

Expert Comment

by:Slick812
ID: 14156409
I am not sure how to comment on this?, to Scan a drive, with the current size of Hard Drives and storage capasities, will take awhile to do, I know on my machines. I have very many drives (12 on one machine), but there is never more than 2  physical Hard Drves, I have Logical partitions on a single hard drive which will show 8 Drives in the MyComputer window for a single hard drive. . .

but you still will have to use the computers hardware (the CPU, the separate drive controlers and the hard drive)  to do any hard drive scan,  - - -  SO, as I tried to tell you before, running the scans at the same time may slow it down, instead of speed it up, and The system (both the windows operating system and the hardware system) are what accually does the drive scans, in you code you do not do any of the real code work (reading the FAT tables and analizing the data there) you just call a system function to do it. . . if you really want to do a speed increase, you might have to do some very low level code work for the drive access of the FAT table data but I would guess that it may be only a very slight improvement (maybe 3 or 4 percent), since the system designes have many, many years of experience and there code for this must be optimized by now. . .

If you need any more info on this , you should open up a  NEW QUESTION as a seperate topic. . . .
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The uses clause is one of those things that just tends to grow and grow. Most of the time this is in the main form, as it's from this form that all others are called. If you have a big application (including many forms), the uses clause in the in…
In my programming career I have only very rarely run into situations where operator overloading would be of any use in my work.  Normally those situations involved math with either overly large numbers (hundreds of thousands of digits or accuracy re…
This lesson discusses how to use a Mainform + Subforms in Microsoft Access to find and enter data for payments on orders. The sample data comes from a custom shop that builds and sells movable storage structures that are delivered to your property. …
Look below the covers at a subform control , and the form that is inside it. Explore properties and see how easy it is to aggregate, get statistics, and synchronize results for your data. A Microsoft Access subform is used to show relevant calcul…
Suggested Courses

872 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question