Link to home
Create AccountLog in
Avatar of NAB015
NAB015Flag for United States of America

asked on

Grabbing data from a file. end of line problem.

Hello experts.

So I have some wonderful code that doesn't work!

Below is both the data file and the program. I want to fill matrix X with the values. However the way it works now it thinks a period is the end of the line and fills the matrix there after with 0's. Any quick fixes or is there a better alternative?

Thanks!  

(The Jmax stuff is just in case the rows are of different length)
1 2 3 4 5 6.7 7 8 9 4 
1 1 1 1 1.3 1 1 1 1 1 
2 5 2 2.3 2 2 5 2 2 2 
3 3 3 3 3 10 3 3 3 3

Open in new window

#include <iostream>   // for input/output via screen
#include <fstream>    // for input/output via files
#include <string>     // to enable data-type string
#include <cmath>          // for mathematical functions
#include <cstdlib>        //  -"-
#include <sstream>
using namespace std;
using std::cerr;
using std::cout;
using std::endl;
using std::ifstream;


// This program reads values from the file 'example.txt'


int X[10000][100000];

int main()
{
   ifstream indata; // indata is like cin
   int num; // variable for input value

  indata.open("example.txt"); // opens the file
   if(!indata) { // file couldn't be opened
      cerr << "Error: file could not be opened" << endl;
      exit(1);
   }

size_t i = 0, j = 0;

int jmax=0;
std::string line;
while (getline(indata, line)) {     // read the next line of data from the file, into a string
    j = 0;
    std::stringstream ss(line);     // a stringstream used for parsing the line
    int num = 0;
    while (ss >> num) {             // get the next int value from the line
        X[i][j] = num;              // and store it in the array
        ++j;
        if(j>jmax) jmax=j;
    }
    ++i;
}


cout << "i= "<< i << "   jmax= "<< jmax<<endl;
cout << "The data read in is: " << endl;
for(int l = 0; l <= i;l++){
	for(int k = 0; k<=jmax;k++){
	cout << X[l][k] << " ";
	}
cout << endl;
}

  return 0;
}

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of Infinity08
Infinity08
Flag of Belgium image

Link to home
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
See answer
Are you sure that is a period? Have you looked at the data with a hex editor? Post hexdump of 'example.txt'. Might as well eliminate the obvious stuff
Avatar of NAB015

ASKER

Infinity,

using double to define the matrix X gives me this error:
/tmp/ccgw3plS.o: In function `__static_initialization_and_destruction_0(int, int)':
import.cc:(.text+0x1e): relocation truncated to fit: R_X86_64_32 against `.bss'
/tmp/ccgw3plS.o: In function `__tcf_0':
import.cc:(.text+0x61): relocation truncated to fit: R_X86_64_32 against `.bss'
collect2: ld returned 1 exit status
import.out: Command not found.

TomasP,
Sorry I don't know how to look at in a hex editor. I am pretty sure it is a period. The data was saved through excel as a .txt file and those values were decimals.

Thanks- NB
>> using double to define the matrix X gives me this error:

Could you show the exact and complete code that generated those errors ?
Avatar of NAB015

ASKER

Exact code from command line:

[nab015@linuxremote1 ~/Capstone_s2011/NN.dir]$ g++ -o import.out import.cc ; import.out
/tmp/ccgw3plS.o: In function `__static_initialization_and_destruction_0(int, int)':
import.cc:(.text+0x1e): relocation truncated to fit: R_X86_64_32 against `.bss'
/tmp/ccgw3plS.o: In function `__tcf_0':
import.cc:(.text+0x61): relocation truncated to fit: R_X86_64_32 against `.bss'
collect2: ld returned 1 exit status
import.out: Command not found.

Using the exact code posted above except replaced int with double @ double X[10000][100000];
That's the output, not the code ;) By code I mean the source C++ code.


Anyway : if you kept the same dimensions for the 2D array, then that is very likely to be your problem. Assuming that sizeof(double) = 8 :

        10000 * 100000 * 8 = 8000000000

That's almost 8 GB of memory ... That seems a bit excessive ;)
>> Using the exact code posted above except replaced int with double @ double X[10000][100000];

You'll also want to use double instead of int when reading the value from the file. ie. num has to be a double too.
Avatar of NAB015

ASKER

So making the matrix smaller did allow it to run but did not solve the problem.

I will eventually need the matrix to be pretty large, would 1,000 * 400 * 8 = 32000000 be to large?

Any other ideas?
>> So making the matrix smaller did allow it to run but did not solve the problem.

See my previous post - you'll also need to make num a double - otherwise you'll still try to read an int from the file.
Avatar of NAB015

ASKER

There we go. The I had changed the first num but not the second one.

Thanks Infinity.
Avatar of NAB015

ASKER

Thanks Infinity!

Defining variables is a pain. Oh well C++.
Ok, since you are sure that the values in the file are
1.3 and 2.3
then your parsing the information into an integer is not optimal. These are floating point numbers. Your parser would read the numbers up to the period on the line and then the numbers after the period.
Change the array or change the parser to pull all fields that are white space seperated
That way you will get the fields into a float/double and then decide it you want to "cast" away the value to the right of the decimal point or preserve it and change your matrix to a floating point type
Avatar of rwwff
rwwff

1,000 x 400 is fine, size wise; for exceptionally huge arrays like your original posting, you're going to have to come up with a problem specific way of handling the storage, because handling the storage can easily become the limiter on performance if its done in a way that poorly matches the way your application utilizes the data once it has been read.