Read a text file and parse data into a unmanaged Double Array

It depends on your file
You have read a Line that contain data.
The simplest case the file only contains 30000*75 doubles each double on a separate line
is quite easy just convert yout string to a double there are several functions that can do that
have for example a look to strtod But you also can use sscanf and others.
infront of the while loop add i=0 and j=0
and within the while loop each time you have assigned a value to the array using
for example sscanf(Line,"%f",&DataArray[i][j]);
i++;
if(i>=30000)
{
    i=0;
    j++;
}
Of couse you have to take care on to large files where are more data in and you have to use other algorithms if your file structure is more complex.

I really need a C++.NET implementation
following in the .net code used.
Managed and Unmanaged arrays/code
is a issue, I think.
Thx.

The code you have is almost right, this works ok for me:

      StreamReader^ sr;
      String^ line;
      sr = File::OpenText("c:\\data.txt");

      double arr[100];
      size_t i = 0;
      while(!sr->EndOfStream)  // i and j loops omitted to be brief.
      {
            line = sr->ReadLine()->Trim()->ToString();
            arr[i++] = Double::Parse(line);
            //loop over i, j
      }

This compiles fine but at runtime:
"Unhandled Exception and
string is not correct format."

String is managed, c Array is not.
There is a conversion or casting I'm missing.

No - I don't think so - the code I have works fine for me - the managed/unmanaged conversion is fine in my code sample.  

The problem may be the format of the text file that you have.  Can you post a line from the text file that you are using?

Here are 2 lines of data I am trying to read into a unmanaged array. I see the below got
word wrapped so I split the 2 lines with a blank row, that does not exits in the real data.
The data is 68 numbers per row, and thousands of rows.

 17.4599   29.3400   35.3700   -1.0000   -1.0000    1.0000    1.0000  640.3441  -43.0918    0.2433    2.8766    1.0000    0.0000    0.0000    0.0000    6.6724    9.0899    9.8099    9.1269   14.1299   22.2799    1.2356    1.1229    5.4716    1.6973  635.6559  655.5140  641.4299  652.6500  641.4299  652.6500  647.1783  643.9916  653.6500  655.7900    0.5950  659.0800  659.0800  633.5100  633.5100    0.4936  633.5100  633.5100    2.4423  652.6500    0.8295  653.6500  656.9700  656.9700    0.4873  659.0800    0.6342  663.0000  635.1900  627.6300    0.5526  627.6300  627.6300    0.8713  627.6300    0.5035  627.6300  0.20  19960318   64143   65265   64143   65265

 17.8299   29.3400   35.3700   -1.0000   -1.0000    2.0000    0.0000  645.0565   11.4634    0.2374   13.8133    0.2962    0.4467    0.0000    0.0000    6.9350    9.0249   10.0937    8.9864   11.7999   20.1399    1.1305    1.1946    5.5292    0.9106  635.7143  657.7856  652.6500  656.1799  649.8000  651.6900  647.1207  646.3792  652.6500  653.6500    0.6077  659.0800  659.0800  633.5100  633.5100    0.5040  633.5100  633.5100    2.0625  656.1799    0.8295  656.1799  656.9700  656.9700    0.5027  656.9700    0.5858  663.0000  638.3500  627.6300    0.4614  627.6300  627.6300    0.7876  627.6300    0.0476  627.6300  0.20  19960319   65265   65618   64980   65169

That worked. This works great for a 1 d array. But when I move arr to 2-d
I get the unhandled exception. I should be able to

double arr[10000][75]={0};
int j=0;
array<Char>^sep = gcnew array<Char>{' '};
      array<String^>^ splitDoubles;
      while(!sr->EndOfStream)  // i and j loops omitted to be brief.
      {
            line = sr->ReadLine()->Trim()->ToString();
            splitDoubles = line->Split(sep,System::StringSplitOptions::RemoveEmptyEntries);
            for(int n = 0; n<splitDoubles->Length; n++)
            {
                  arr[j][i++] = Double::Parse(splitDoubles[n]);
            }
                j++;
      }

If the example you gave me is accurate then it seems like it may contain blank lines as well as lines with numbers.  You should possibly check for that after reading each line.

If that isn't the case - can you give details of the exception and where the problem occurs?

sry - you said blank line isn't in the real data... so can you let me know exactly what it is that is failing?

I used a 2 d array in place of the 1 d array
and NOT incremented over the rows just to test if a 2 d
unmanaged array can be written to.
(line of code commented as to when it crashes)
I get a "An unhandled exception of type 'System.StackOverflowException'
but it occurs at line 123. Again, the 1 d array works great!

private: System::Void button1_Click(System::Object^  sender, System::EventArgs^  e)
{
      double arr[30000]={0};
                    double arr2[30000][75]={0};
      int linecount=0;
       StreamReader^ sr;
      String ^line;
      String^ stringline;
      size_t i=0;
      size_t j=0;
      String^ myfile = "c:\\temp\\testdata.txt";   ///LINE 123<<<<<<
      array<Char>^sep = gcnew array<Char>{' '};
      array<String^>^ splitDoubles;
      sr = File::OpenText(myfile);
      j=0;
      while (!sr->EndOfStream)
      {
      line = sr->ReadLine()->Trim()->ToString();
      splitDoubles = line->Split(sep,System::StringSplitOptions::RemoveEmptyEntries);
      i=0;
      for(int n = 0; n<splitDoubles->Length; n++)
                {
arr2[i++][j] = Double::Parse(splitDoubles[n]); //>>>>2-D use this and it crashes. Just overwriting jth row!
//arr[i++] = Double::Parse(splitDoubles[n]);   //>>>>>1-D use this and it works fine
                }
      }
      sr->Close();
      this->label1->Text="Import Success";
      ////CODE TO DISPLAY, ETC////
 }

Not for points, but the size of your 2 dimensional array is too large to fit on the stack.  The default stack size on 32bit platforms is 1MB: http://support.microsoft.com/kb/50950.  You've got 30000 * 75 * sizeof(double) = 18000000 bytes allocated in just your array.

You'll need to use the heap to store that size of a variable.  Dynamically declare your storage for it (or use a vector).

> Not for points, but the size of your 2 dimensional array is too large to fit on the stack

I suspect you deserve some points for this - I suspect that this is the answer - especially given the 'stack overflow' message...

allocate the array like this, as suggested by clockwatcher...

double (*arr2)[75] = new double[30000][75];

then remember to free the memory as appropriate...

Even if I reduce the array down to
arr2[3000][68] it crashes so
stack overloading is not the only problem.
I worked around it for rows<=1000  by translating rows of 1 d array
into the unmanaged 2 d array, but there must be
a convention between
array<Form1^,2>  ^arr2 = gcnew array<Form1^,2>(SIZE_X,SIZE_Y);
and a unmanaged arr2unmanaged[SIZE_X, SIZE_Y]
Cant find much info on how to read into a unmanaged array from a
.NET streamreader.
Is this a Marshal.Copy Issue?
System::Runtime::Interop;
Marshal::Copy(...

The stack size is 1MB.

 68 * 3000 * 8 = 1,632,000 bytes =~ 1.6 MB --- you're still over the maximum stack size.  

I am certain this is nothing to do with the interop - and more to do with the array handling.

The array copy part is not important - you are extracting a single .NET double (from parse) and writing it directly into an unmanaged array... there should be no problems with that.

Did you try it with a dynamic array?  

Sorry about delay in response - I think our time zones are siginifcantly different...

No problem..I have a proto that's working declaring an object array
then converting that object to a double. I'm cleaning it up and will repost ASAP.
Almost done.
Thx.

///Problem Issue was declaring the object array.
//Need to read data to managed array and copy to unmanaged array.
//There may be a "Copy" in the namespace somewhere!
//Critical snippets are:

array<Object^,2> ^arr3 = gcnew array<Object^,2>(10000,68);  //This was the missing line

line = sr->ReadLine()->Trim()->ToString();                         
splitDoubles = line->Split(sep,System::StringSplitOptions::RemoveEmptyEntries);
for(int n = 0; n<splitDoubles->Length; n++)
                {
arr3[j,i++] = Double::Parse(splitDoubles[n]);   //use this and it works fine
                }
double v1[68]={0};
for (int m=0;m<68;m++)
{
v1[m]=System::Convert::ToDouble(arr3[n,m]);  //v1 is c array. This was the missing line
}
//loop over lines n