maximum size of dynamic arrays in delphi

I don't know the way it is worked out (so can't be much help really), but it is to do with actual and virtual memory.  Your paging file comes into this (probably more than physical memory).

Make sure your paging file is at least twice the size of the array you want to create.

What is the paging file?

Anyway, to easily describe the issue I am attaching the code for a sample project. I has one form Form1 with a button Button1 and a textbox Edit1. If one sets the size of the array to 100,000,000 the program runs fine with an increase in the memory consumption of 600,000,000bytes=600MB
If one sets the size to 200,000,000 an Out-of-memory message is output and if one sets it to1,000,000,000 a range-check error is created. What can I do to increase such limits?
Note that increasing the maximum stack size did not affect the above described behaviour.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               C O D E
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls;

type
  TForm1 = class(TForm)
    Button1: TButton;
    Edit1: TEdit;
    procedure Button1Click(Sender: TObject);
    function StringToInt ( str : string) : integer;
  private
   //
  public
    TmpArray: array of real;
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.Button1Click(Sender: TObject);
var
  size: integer;
begin
    size:= StringToInt(Edit1.Text);
    SetLength(TmpArray,size);

end;

function TForm1.StringToInt ( str : string) : integer;
var
    Code,i : integer;
begin
    Val(str, i, Code);
    if Code <> 0 then i := 0;
    StringToInt := i;
end;
end.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               E N D   C O D E
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First off use the StrToInt() and StrToInt64() functions. Second, switch "Size: Integer" to "Size: Int64". Second do you have sufficient ram to handle 200,000,000(1.5gb of memory)?

It does not matter how much ram you have, as you are both missing one crucial point.

The address space of a 32bit process is limited to 4GB of addressable memory. 2GB for the application, 2GB for the system. That means that you are limited to (at most) 2GB, and trying to define anything larger than 2GB will throw a range check. For example, because real is 6 bytes, but the array is not packed, it will actually allocate 8 bytes per element, giving you a max definable size of 268,435,455. Anything larger will throw a range check as it >= 2GB limit.

As to the out of memory error, that is going to happen before you even get close to the 2GB limit; normally you will see it at or around 1.4-1.5 GB allocation. You will probably find that you can hit an array of 180,000,000, but not 190,000,000. Also keep in mind that if the array is already allocated, then setting a new length forces a reallocation, and in many cases the reallocation cannot be done in place, requiring CurrentSize + NewSize to be available in order for the allocation to succeed. To handle this, you should be using SetLength with a size of zero to release the memory before allocating a different size (assuming that you are discarding what is currently in the array).

But really, if you have to process this much data then you should probably consider working with temporary files and process the data in chuncks. An example of this technique (albeit in C) can be found here:

http://www.geocities.com/krishnapg/TerabyteArrays.html

Regards,
Russell

Many thanks for the responses.

Note that in my particular case, it is not possible to allocate memory to temporary files. The program is roughly speaking a simulation of structures in earthquake. With a typical  time step of 0.01sec and average duration of an earthquake of 20sec and 5-10iterations per step to achieve solution (it is an iterative process) the data are accessed 100*20*5=10,000 times per run. Saving and retrieving data from temporary files is simply too slow, considering that an average run of a case like that would take up to 10h.

Anyway, all is very clear now, I just have a couple of clarifying questions more::
1) What is the reason for the difference in the messages, i.e for 200,000,000 elements it is an out-of-memory and for 1,000,000,000 elements it is a out-of-range error?
2) Is there a way of knowing in advance that a SetLength(TmpArray,size); command will lead in an out-of-memory error? For instance, a procedure call that calculates the amount of memory that is available? I need this so that to output a user-friendly message to users (suggesting possible measures to reduce memory consumption),  instead of a simple out-of-memory message? Note that the limits of the application with typical modelling are approximately for a 100-storey building (way too much). The out-of-memory message that we got was due to ridiculous modelling requirements by the user (hence I am looking for a way of guiding such' stupid' users).