NAB015
asked on
Grabbing data from a file. end of line problem.
Hello experts.
So I have some wonderful code that doesn't work!
Below is both the data file and the program. I want to fill matrix X with the values. However the way it works now it thinks a period is the end of the line and fills the matrix there after with 0's. Any quick fixes or is there a better alternative?
Thanks!
(The Jmax stuff is just in case the rows are of different length)
So I have some wonderful code that doesn't work!
Below is both the data file and the program. I want to fill matrix X with the values. However the way it works now it thinks a period is the end of the line and fills the matrix there after with 0's. Any quick fixes or is there a better alternative?
Thanks!
(The Jmax stuff is just in case the rows are of different length)
1 2 3 4 5 6.7 7 8 9 4
1 1 1 1 1.3 1 1 1 1 1
2 5 2 2.3 2 2 5 2 2 2
3 3 3 3 3 10 3 3 3 3
#include <iostream> // for input/output via screen
#include <fstream> // for input/output via files
#include <string> // to enable data-type string
#include <cmath> // for mathematical functions
#include <cstdlib> // -"-
#include <sstream>
using namespace std;
using std::cerr;
using std::cout;
using std::endl;
using std::ifstream;
// This program reads values from the file 'example.txt'
int X[10000][100000];
int main()
{
ifstream indata; // indata is like cin
int num; // variable for input value
indata.open("example.txt"); // opens the file
if(!indata) { // file couldn't be opened
cerr << "Error: file could not be opened" << endl;
exit(1);
}
size_t i = 0, j = 0;
int jmax=0;
std::string line;
while (getline(indata, line)) { // read the next line of data from the file, into a string
j = 0;
std::stringstream ss(line); // a stringstream used for parsing the line
int num = 0;
while (ss >> num) { // get the next int value from the line
X[i][j] = num; // and store it in the array
++j;
if(j>jmax) jmax=j;
}
++i;
}
cout << "i= "<< i << " jmax= "<< jmax<<endl;
cout << "The data read in is: " << endl;
for(int l = 0; l <= i;l++){
for(int k = 0; k<=jmax;k++){
cout << X[l][k] << " ";
}
cout << endl;
}
return 0;
}
ASKER CERTIFIED SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
Are you sure that is a period? Have you looked at the data with a hex editor? Post hexdump of 'example.txt'. Might as well eliminate the obvious stuff
ASKER
Infinity,
using double to define the matrix X gives me this error:
/tmp/ccgw3plS.o: In function `__static_initialization_a nd_destruc tion_0(int , int)':
import.cc:(.text+0x1e): relocation truncated to fit: R_X86_64_32 against `.bss'
/tmp/ccgw3plS.o: In function `__tcf_0':
import.cc:(.text+0x61): relocation truncated to fit: R_X86_64_32 against `.bss'
collect2: ld returned 1 exit status
import.out: Command not found.
TomasP,
Sorry I don't know how to look at in a hex editor. I am pretty sure it is a period. The data was saved through excel as a .txt file and those values were decimals.
Thanks- NB
using double to define the matrix X gives me this error:
/tmp/ccgw3plS.o: In function `__static_initialization_a
import.cc:(.text+0x1e): relocation truncated to fit: R_X86_64_32 against `.bss'
/tmp/ccgw3plS.o: In function `__tcf_0':
import.cc:(.text+0x61): relocation truncated to fit: R_X86_64_32 against `.bss'
collect2: ld returned 1 exit status
import.out: Command not found.
TomasP,
Sorry I don't know how to look at in a hex editor. I am pretty sure it is a period. The data was saved through excel as a .txt file and those values were decimals.
Thanks- NB
>> using double to define the matrix X gives me this error:
Could you show the exact and complete code that generated those errors ?
Could you show the exact and complete code that generated those errors ?
ASKER
Exact code from command line:
[nab015@linuxremote1 ~/Capstone_s2011/NN.dir]$ g++ -o import.out import.cc ; import.out
/tmp/ccgw3plS.o: In function `__static_initialization_a nd_destruc tion_0(int , int)':
import.cc:(.text+0x1e): relocation truncated to fit: R_X86_64_32 against `.bss'
/tmp/ccgw3plS.o: In function `__tcf_0':
import.cc:(.text+0x61): relocation truncated to fit: R_X86_64_32 against `.bss'
collect2: ld returned 1 exit status
import.out: Command not found.
Using the exact code posted above except replaced int with double @ double X[10000][100000];
[nab015@linuxremote1 ~/Capstone_s2011/NN.dir]$ g++ -o import.out import.cc ; import.out
/tmp/ccgw3plS.o: In function `__static_initialization_a
import.cc:(.text+0x1e): relocation truncated to fit: R_X86_64_32 against `.bss'
/tmp/ccgw3plS.o: In function `__tcf_0':
import.cc:(.text+0x61): relocation truncated to fit: R_X86_64_32 against `.bss'
collect2: ld returned 1 exit status
import.out: Command not found.
Using the exact code posted above except replaced int with double @ double X[10000][100000];
That's the output, not the code ;) By code I mean the source C++ code.
Anyway : if you kept the same dimensions for the 2D array, then that is very likely to be your problem. Assuming that sizeof(double) = 8 :
10000 * 100000 * 8 = 8000000000
That's almost 8 GB of memory ... That seems a bit excessive ;)
Anyway : if you kept the same dimensions for the 2D array, then that is very likely to be your problem. Assuming that sizeof(double) = 8 :
10000 * 100000 * 8 = 8000000000
That's almost 8 GB of memory ... That seems a bit excessive ;)
>> Using the exact code posted above except replaced int with double @ double X[10000][100000];
You'll also want to use double instead of int when reading the value from the file. ie. num has to be a double too.
You'll also want to use double instead of int when reading the value from the file. ie. num has to be a double too.
ASKER
So making the matrix smaller did allow it to run but did not solve the problem.
I will eventually need the matrix to be pretty large, would 1,000 * 400 * 8 = 32000000 be to large?
Any other ideas?
I will eventually need the matrix to be pretty large, would 1,000 * 400 * 8 = 32000000 be to large?
Any other ideas?
>> So making the matrix smaller did allow it to run but did not solve the problem.
See my previous post - you'll also need to make num a double - otherwise you'll still try to read an int from the file.
See my previous post - you'll also need to make num a double - otherwise you'll still try to read an int from the file.
ASKER
There we go. The I had changed the first num but not the second one.
Thanks Infinity.
Thanks Infinity.
:)
ASKER
Thanks Infinity!
Defining variables is a pain. Oh well C++.
Defining variables is a pain. Oh well C++.
Ok, since you are sure that the values in the file are
1.3 and 2.3
then your parsing the information into an integer is not optimal. These are floating point numbers. Your parser would read the numbers up to the period on the line and then the numbers after the period.
Change the array or change the parser to pull all fields that are white space seperated
That way you will get the fields into a float/double and then decide it you want to "cast" away the value to the right of the decimal point or preserve it and change your matrix to a floating point type
1.3 and 2.3
then your parsing the information into an integer is not optimal. These are floating point numbers. Your parser would read the numbers up to the period on the line and then the numbers after the period.
Change the array or change the parser to pull all fields that are white space seperated
That way you will get the fields into a float/double and then decide it you want to "cast" away the value to the right of the decimal point or preserve it and change your matrix to a floating point type
1,000 x 400 is fine, size wise; for exceptionally huge arrays like your original posting, you're going to have to come up with a problem specific way of handling the storage, because handling the storage can easily become the limiter on performance if its done in a way that poorly matches the way your application utilizes the data once it has been read.