Delphi Dynamic Array Contiguity

I have an application that dumps data from a complicated dynamic array to disk and then reads it back in again at a later point to minimize memory usage.

The array (of singles) has four dimensions and is created in two steps:

   
 SetLength(SDVuser,MaxRegionTypes+1);
    for i := 0 to MaxRegionTypes do
      SetLength(SDVuser[i],MaxSubRegions[i]+1,  MaxSimDailyVars+1,  MaxDays+1);

Open in new window


So in others words, the first dimension is created first and then the 2nd/3rd/4th dimensions are created next because the size of the second dimension varies.

The writing to and reading from file F (file of single) is done using BlockWrite and BlockRead commands:

       
 for rt := 0 to MaxRegionTypes do
          begin
            RecordSize := (MaxSubRegions[rt]+1)*
                          (MaxSimDailyVars+1) *
                          (MaxDays+1);
            BlockRead(F,SDVuser[rt,0,0,0],RecordSize);
          end;

Open in new window


This assumes that the 2nd/3rd/4th dimensions of the SDVuser array are CONTIGUOUS in memory.  Documentation and on-line comment that I can see on dynamic arrays suggests this is true.

But I sometimes get Range Check errors associated with the BlockRead which I am thinking might be caused by non-contiguity.  I am also getting odd exceptions occurring elsewhere which again I think may be caused by memory being trampled because the array is not in fact contiguous.

Question 1: Does the SetLength para create a contiguous block of memory for the 3 dimensions I create in one go?
Question 2: Do the secondary SetLength operations cause the entire 4 dimensional structure to be recreated each time to ensue that the WHOLE array is contiguous - my observation is that assuming this causes lots more problems elsewhere and so I have avoided making this assumption.

Hope someone can clarify the situation

Best wishes

Jon
jon_rsAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

MerijnBSr. Software EngineerCommented:
In Delphi dynamic arrays are put in one single large memory block.

I'm not 100% sure, but I think that when you do a SetLength() on the first dimension, all the dynamic arrays in the second dimension will be set to 0 length (or to unknown status).

So in short:

1) no, only for the first dimension
2) yes

One note: depending on which Delphi version you are using large dynamic arrays can cause problems for the standard memory manager (memory fragmentation = memory leak while you don't actually leak memory).
0
aikimarkCommented:
I hope I'm supporting what MerijnB stated.

It looks like your code is creating and shaping an array of variable-sized arrays.  As such, there can be no expectation of contiguous memory use.

Also, when storing data in arrays, the data might actually be stored in the (supplied) dimensions in reverse order of what you actually supplied.  For instance, if you created and array of size (X, Y, Z), the data values might be physically stored in (Z, Y, X) order.  I have take advantage of this to do block memory copy operations of 'row' data by allocating and populating the array with the row data as the right most.
0
jon_rsAuthor Commented:
Hi both

Thanks for your responses.
I have taken a step back and rewritten the code so that it makes NO assumptions about contiguity of  a multi dimensioned dynamic array and my memory problems have gone away.

So I think in summary:

A single dimensioned dynamic array IS contiguous
An array of variable sized arrays constructed in a dynamic array will almost certainly NOT be contiguous (although elements of the individual dimensions MIGHT be).
Relying on any sort of contiguity of a multi-dimensioned dynamic array is dangerous and to be avoided.

Jon
0
MerijnBSr. Software EngineerCommented:
Maybe to support your conclusion (and maybe I misunderstood your initial question):

A dynamic array with one dimension will always be contiguous. So the first dimension of your array is contiguous, and all the dynamic arrays in the second dimension are all contiguous (in itself).

Afaik a dynamic array works like a pointer, so your first dimension is basically an array of pointers. This means that the whole thing together won't be contiguous! (ofc it can be, that will be luck :)).
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jon_rsAuthor Commented:
MerijnB

Thanks.  I think my original question was not as clear as it could have been :-)

As you say I can see no reason why the second set of 3 dimensional arrays should not be contiguous, but my findings suggest that if you pass the starting address of that part of the array to the BlockWrite/BlockRead you don't always get want you think you might.

My solution has been to create a new single dimensioned array (which definitely is contiguous) and to copy (element by element) the stuff in the 3-dimensional array into that contiguous block and then to write that out, and then to reverse the process for a BlockRead.  This makes the whole thing stable again.

Jon
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Delphi

From novice to tech pro — start learning today.