Solved

Memory issues: custom object takes 4 times more memory than I calculated

Posted on 2004-10-24
258 Views
Last Modified: 2010-04-05
Consider the following definition of objects:

cell_status = (H, A1, A2, A3, D, E);

TCell = class(TObject)
  status : cell_status;
  to_update : cell_status;
  lifetime: word;
  resist: byte;
  activated: bool;
  constructor create();
end;

TLattice = class(TObject)
  cell : array of array of TCell;
  size_dim1, size_dim2, lifetime : word;
  cellCountH, cellCountA1, cellCountA2, cellCountA3, cellCountD : array of longword;
  active_drugs : byte;
  constructor create(Dimensie1, Dimensie2 : Integer);
  destructor Destroy;
    override;
end;

var
  bufferzone :integer = 10;


constructors:

constructor TCell.create();
begin
  self.lifetime := 0;
  self.status := H;
  self.to_update := H;
  self.resist := 0;
  self.activated := false;
end;

constructor TLattice.create(Dimensie1, Dimensie2 : Integer);
var
  Temp_Cell : TCell;
  Teller1, Teller2 : Integer;
begin

    setlength(cell,Dimensie1+(2*bufferzone),Dimensie2+(2*bufferzone));  // set dynamic array => dimensions + bufferzone(bufferzone = 10)
    self.size_dim1 := Dimensie1;
    self.size_dim2 := Dimensie2;

    setlength(cellCountH,1000);
    setlength(cellCountA1,1000);
    setlength(cellCountA2,1000);
    setlength(cellCountA3,1000);
    setlength(cellCountD,1000);

    for Teller1 := 0 to Dimensie1 + (2*bufferzone -1) do
    begin // vul lattice 1e dimensie
      for Teller2 := 0 to Dimensie2 + (2*bufferzone -1) do  // vul lattice 2e dimensie
      begin
        Temp_Cell := TCell.create();
        self.cell[Teller1,Teller2] := Temp_Cell;
      end; // einde vul lattice 2e dimensie
    end; // einde vul lattice 1e dimensie
end; // einde constructor lattice

Considering the storage needed for:

Lattice := TLattice.create(700,700);

To my calculations this would give:

2*cell_status(=2),1*word(=2),1*byte(=1),1*bool(=1) = 6 bytes per TCell, 720*720*6 = 3110400 per TLattice(with TLattice.create(700,700);)
size_dim1, size_dim2, lifetime : word => 3*2 = 6 per TLattice
active_drugs : byte; => 1*1 per TLattice
cellCountH, cellCountA1, cellCountA2, cellCountA3, cellCountD : array of longword; 5*1000*4 = 20000 bytes per TLattice
=================================
3130407 bytes per TLattice
~
3057 Kbytes per TLattice
~
3 MB per TLattice
(+ perhaps some extra memory to hold pointers to memory locations)

When I check using the windows taskmanager, each TLattice eats up 12200 KB ~ 12 MB
What am I missing here?

Furthermore, I'm creating two more objects for each TLattice, 2 TMargolusArray.
These should just contain pointers to the TCells in TLattice, but each one adds another 12 MB(more or less). Any suggestions on how to keep memory costs down??
This is quite important to me since I need to create about 80 sets of these objects(would be 36*80 = 2880 MB)
while this should be(well, only according to what I calculated :) 80 * 3 = 240 + 80 * 2 * (some amount of memory, not more than 3 MB anyway)

TMargolusBlock = class(TObject)
  cell : array[0..3] of TCell;
  cellCountH, cellCountA1, cellCountA2, cellCountA3, cellCountD : longword;
  LatticeReference1, LatticeReference2 : word;
  constructor create(cell0,cell1,cell2,cell3 : TCell);
  destructor Destroy;
    override;
end;

TMargolusArray = class(TObject)
  mblock : array of array of TMargolusBlock;
  constructor create(size : Integer);
  destructor Destroy;
    override;
end;

constructor TMargolusBlock.create(cell0,cell1,cell2,cell3 : TCell);
begin
  self.cell[0] := cell0;
  self.cell[1] := cell1;
  self.cell[2] := cell2;
  self.cell[3] := cell3;
  self.cellCountH := 0;
  self.cellCountA1 := 0;
  self.cellCountA2 := 0;
  self.cellCountA3 := 0;
  self.cellCountD := 0;
end;

constructor TMargolusArray.create(size : Integer);
begin
  setlength(mblock, size+1, size+1);
end;


Below the TCells of TLattice are assigned to TMargolusBlock in TMargolusArray(I left out some details about really fitting all TCells in MargolusArrayB):

  Dimensie1 := Lattice.size_dim1;
  MargolusArraySize := Dimensie1 div 2;
 
  MargolusArrayA := TMargolusArray.create(MargolusArraySize);
  MargolusArrayB := TMargolusArray.create(MargolusArraySize+1);

  for Teller := 0 to MargolusArraySize - 1 do
  begin
    for Teller2 := 0 to MargolusArraySize - 1 do
    begin
      index1 := Teller * 2+bufferzone;
      index2 := Teller2 * 2+bufferzone;

      cellA0 := Lattice.cell[index1,index2];
      cellA1 := Lattice.cell[index1+1,index2];
      cellA2 := Lattice.cell[index1,index2+1];
      cellA3 := Lattice.cell[index1+1,index2+1];
      cellB0 := Lattice.cell[index1+1,index2+1];
      cellB1 := Lattice.cell[index1+2,index2+1];
      cellB2 := Lattice.cell[index1+1,index2+2];
      cellB3 := Lattice.cell[index1+2,index2+2];

      MargolusArrayA.mblock[Teller, Teller2].Free();
      MargolusArrayB.mblock[Teller+1, Teller2+1].Free();

      MargolusArrayA.mblock[Teller, Teller2] := TMargolusBlock.create(cellA0,cellA1,cellA2,cellA3);
      MargolusArrayB.mblock[Teller+1, Teller2+1] := TMargolusBlock.create(cellB0,cellB1,cellB2,cellB3);
      MargolusArrayA.mblock[Teller, Teller2].LatticeReference1 := index1;
      MargolusArrayA.mblock[Teller, Teller2].LatticeReference2 := index2;
      MargolusArrayB.mblock[Teller+1, Teller2+1].LatticeReference1 := index1 + 1;
      MargolusArrayB.mblock[Teller+1, Teller2+1].LatticeReference2 := index2 + 1;


    end; // einde Teller2
  end; // einde Teller

Help very much appreciated!!




0
Question by:reynaerde
    11 Comments
     
    LVL 7

    Assisted Solution

    by:LRHGuy
    The confusion probably stems from the fact that your objects are classes, which uses more memory for overhead.

    With record and other types, you get the memory needed with

    sizeof(variable);

    You can get the exact amount of memory used for any class object with:

    classname.instancesize;

    for example:

    (2*sizeof(cell_status)) +
    (sizeof(word)) +
    (sizeof(byte)) +
    (sizeof (boolean)) +
    (720*720*tcell.instancesize)

    Hope that helps clear up the mystery...
    0
     
    LVL 7

    Assisted Solution

    by:LRHGuy
    Also, "bool" is 4 bytes, "boolean" is 1.

    If you let the compiler get the sizes for you, as I showed above, you'll get more accurate metrics.

    Also, "instancesize" of course doesn't include any memory you allocate in the "constructor" method, since it doesn't know about it.
    0
     
    LVL 7

    Assisted Solution

    by:LRHGuy
    Also, don't forget to include the size of the tcell array or array itself, another 720*720*sizeof(tcell). Since tcell is a pointer to a class, it's 4 bytes, i.e.  2,073,600 bytes for the array pointing to the tcell objects, which takes 6,739,200 bytes, bringing it to 8,812,800 bytes. That at least accounts for some of the mystery....
    0
     
    LVL 7

    Assisted Solution

    by:LRHGuy
    Giving it some more thought, I believe the memory manager will allocate on 16-byte boundries, which means the tcells which I think need 13 bytes each, actually eat up 16 each, i.e. 8,294,400 .... bringing the total to: 10,368,000.
    0
     

    Author Comment

    by:reynaerde
    Ok, thank you. Considering what you wrote I put TCell into a packed record and changed bool to boolean.
    Now TLattice takes up something like 3MB, which was kinda what I expected, very good :)
    Unfortunately, the TMargolusArray is still using loads of memory.
    Just before closing this question, can anyone tell me what would be the most efficient way of storing pointers?

    Basically what I want to do is this:
    I have 1 object(TLattice) which holds 700*700 cells(TCell)

    Now, I also have 2 objects(TMargolusArray) which hold pointers to these cells in a structured way.

    But, 1 such an object uses more memory than the TLattice object itself. It does make some sense though, since a pointer takes up 4 bytes, there are 4 pointers in each TMargolusBlock, 4 longwords and 2 words totalling 4*4 + 4*4 + 2*2 = 36 bytes for each block. There are (TLattice.size / 2 blocks)^2 =  350^2 blocks
    350^2 * 36 ~4 MB + some extra ~ 5 MB. I'm not sure if I can do anything about the longwords and words, but is there a way to minimize pointer storage space or is it always 4 bytes?
    0
     
    LVL 7

    Accepted Solution

    by:
    I'm reasonably sure a pointer to an instance of an object (class) is always 4 bytes with today's Delphi.

    Since you appear to save the index of the starting cell in the block, maybe you could reference a global cells array using the index, rather than the direct reference to the cells?

          cellA0 := Lattice.cell[latticeReference1,LatticeReference2];

    The MargolusBlock would look up the cell (as above) whenever it needed the reference, instead of storing it.

    Just a thought.
    0
     

    Author Comment

    by:reynaerde
    Yes, that could be a good idea, I will look into it. Thanks a lot for your help :)
    0
     

    Author Comment

    by:reynaerde
    Oh, by the way, I'm now using pointers to a record(TCell), does this change anything? Or is a pointer to a class just a large of a pointer to a record?
    I could imagine there would be some optimalization since the size of a record is always known..
    0
     
    LVL 7

    Expert Comment

    by:LRHGuy
    I believe they are the same size.
    0
     
    LVL 33

    Expert Comment

    by:Slick812
    seems like you accepted answer while I was doing this, but since I did it already, here it is -

    I am not real sure about what to comment about, the Delphi memory manager will handle all of the memory allocation for objects and dynamic arrays, so I would guess you would have to alter or write your own memory manager to get to do something different?
    What I have done, is to Create my own Memory Blocks to use in arrays, these arrays are of a Dynamic sort of array, but they are Not a Delphi dynamic array, I use a pointer to array type, so I can allocate and Free my own memory for arrays, I do not really see this as an advantage in the things you are doing here, but it does give you the control of the memory allocated for your arrays.
    I do not see any reason for your TCell to be an object, since you do not seem to use any of the TObjects properties, so I have changed it to a record.


    here are the types I use


    type
      cell_status = (H, A1, A2, A3, D, E);

      TCell = packed record
        // using packed record reduces mem use
        status, to_update: cell_status;
        lifetime: word;
        resist: byte;
        activated: boolean;
        end;

      ParyCell = ^TaryCell; // A Pointer to an array of cell
      TaryCell = Array[Word] of TCell;

    // I had to split the 2 dimention array in to 2 separate arrays for mem allocation

      ParyPCells = ^TaryPCells; // a pointer to an array of ParyCell
      TaryPCells = Array[Word] of ParyCell;

      ParyCard = ^TaryCard; // pointer to an array of Cardinal
      TaryCard = Array[Word] of Cardinal;

      TLattice = class
        pCells : ParyPCells;{this 2 dimentional array starts out with no allocated memory}
        size_dim1, size_dim2, lifetime : word;
        pCellCountH, pCellCountA1, pCellCountA2, pCellCountA3, pCellCountD : ParyCard;
        // all of the cardinal arrays start out with no memory
        active_drugs : byte;
        constructor Create(Dimensie1, Dimensie2 : Integer);
        destructor Destroy; override;
        end;


    // end of type


    constructor TLattice.Create(Dimensie1, Dimensie2 : Integer);
    const
    CountSize = 1000;
    bufferzone = 1;
    // this bufferzone at 10 Really ADDS up alot of extra memory use
    // do you even need this bufferzone ?

    var
    i: Integer;
    begin
    {you will need to allocate memory for All of the members of the arrays}
    pCells := AllocMem((Dimensie1+bufferzone)* SizeOf(ParyCell));
    for i := 0 to Dimensie1-1 do // get mem for each pCells
      pCells[i] := AllocMem((Dimensie2+bufferzone)* SizeOf(TCell));

    // I use AllocMem because it Zeros the memory
    // so it does the same as your  TCell.create

    size_dim1 := Dimensie1;
    size_dim2 := Dimensie2;
    lifeTime := 0;

    pCellCountH := Allocmem(CountSize*SizeOf(Cardinal));
    // need to get mem for all cardinal arrays
    pCellCountA1 := AllocMem(CountSize* SizeOf(Cardinal));
    pCellCountA2 := AllocMem(CountSize* SizeOf(Cardinal));
    pCellCountA3 := AllocMem(CountSize* SizeOf(Cardinal));
    pCellCountD := AllocMem(CountSize* SizeOf(Cardinal));

    active_drugs := 0;
    end;


    destructor TLattice.Destroy;
    var
    i: Integer;
    begin
    for i := 0 to size_dim1-1 do
      FreeMem(pCells[i]); // free mem of each pCells

    // free all memory you allocated
    FreeMem(pCells);
    FreeMem(pCellCountH);
    FreeMem(pCellCountA1);
    FreeMem(pCellCountA2);
    FreeMem(pCellCountA3);
    FreeMem(pCellCountD);
    end;

    // end of TLattice

    // this is a button click event that will get the HeapStatus to check memory usage


    procedure TForm1.sbut_TestObjectClick(Sender: TObject);

    var
    Latice1: TLattice;
    HeapStat: THeapStatus;
    StartMem, DuringMem, AfterMem, CommittedMem: Cardinal;
    begin
    HeapStat := GetHeapStatus;
    StartMem := HeapStat.TotalAllocated;
    // get heap mem for stages
    ShowMessage(IntToStr(SizeOf(TCell)));
    Latice1 := TLattice.create(700,700);
    try
    HeapStat := GetHeapStatus;
    DuringMem := HeapStat.TotalAllocated; // increases about 3 megs on my machine
    CommittedMem := HeapStat.TotalCommitted;
    if Latice1.pCells[2][1].status = H then
      ShowMessage('Is H , size_dim1 is '+IntToStr(Latice1.size_dim1));

    Latice1.pCells[2][2].status := A2;
    Latice1.pCells[699][699].lifetime := 7531;
    Latice1.pCellCountH[500] := MaxDWord;
    ShowMessage(IntToStr(Latice1.pCells[699][699].lifetime)+'   '+
                IntToStr(Latice1.pCellCountH[500]));

    finally
     FreeAndNil(Latice1);
     end;
    HeapStat := GetHeapStatus;
    AfterMem := HeapStat.TotalAllocated;
    ShowMessage('Start '+IntToStr(StartMem)+' During '+IntToStr(DuringMem)+
                ' Committed '+IntToStr(CommittedMem)+' After '+IntToStr(AfterMem));

    end;
    0
     
    LVL 33

    Expert Comment

    by:Slick812
    you could also use a packed array for the TLattice and just have an initiazation function, sort of


    function LatticeCreate(Dimensie1, Dimensie2 : Integer): TLattice; //TLattice as a record
    const
    CountSize = 1000;
    bufferzone = 1;
    // this bufferzone at 10 Really ADDS up alot of extra memory use
    // do you even need this bufferzone ?

    var
    i: Integer;
    begin
    with Result do
    begin
    {you will need to allocate memory for All of the members of the arrays}
    pCells := AllocMem((Dimensie1+bufferzone)* SizeOf(ParyCell));
    for i := 0 to Dimensie1-1 do // get mem for each pCells
      pCells[i] := AllocMem((Dimensie2+bufferzone)* SizeOf(TCell));

    // I use AllocMem because it Zeros the memory
    // so it does the same as your  TCell.create

    size_dim1 := Dimensie1;
    size_dim2 := Dimensie2;
    lifeTime := 0;

    pCellCountH := Allocmem(CountSize*SizeOf(Cardinal));
    // need to get mem for all cardinal arrays
    pCellCountA1 := AllocMem(CountSize* SizeOf(Cardinal));
    pCellCountA2 := AllocMem(CountSize* SizeOf(Cardinal));
    pCellCountA3 := AllocMem(CountSize* SizeOf(Cardinal));
    pCellCountD := AllocMem(CountSize* SizeOf(Cardinal));

    active_drugs := 0;
    end;
    end;
    0

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Why You Should Analyze Threat Actor TTPs

    After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

    A lot of questions regard threads in Delphi.   One of the more specific questions is how to show progress of the thread.   Updating a progressbar from inside a thread is a mistake. A solution to this would be to send a synchronized message to the…
    In this tutorial I will show you how to use the Windows Speech API in Delphi. I will only cover basic functions such as text to speech and controlling the speed of the speech. SAPI Installation First you need to install the SAPI type library, th…
    This Experts Exchange video Micro Tutorial shows how to tell Microsoft Office that a word is NOT spelled correctly. Microsoft Office has a built-in, main dictionary that is shared by Office apps, including Excel, Outlook, PowerPoint, and Word. When …
    Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…

    934 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    8 Experts available now in Live!

    Get 1:1 Help Now