Go Premium for a chance to win a PS4. Enter to Win

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 797
  • Last Modified:

Writing Space Delimited file

Hi,

I am trying to read a space delimited file using C++ but i realised that my program is slow. It take minutes to parse a 1mb txt file. below is my code for reading the space delimited file. Is there a better way to readline() and straight get all the space delimited words into an array?


UserReport analyseFileUser(string str_IPAddress)
	{
		UserReport _UserReport;
		vector<string> tabData;
		while(! fileIn.eof())
		{
			string currentLine = "";
 
			while(! fileIn.eof())
			{
				//Get an input line
				char c_LastChar;
				fileIn.get(c_LastChar);
				
 
				if(fileIn.fail())
					//if read file fail at this point, just exit and parse the line
					break;
				if(c_LastChar != '\r' && c_LastChar != '\n')
					//if c_LastChar is not at \n means it is not end of line yet, continue the loop
					currentLine = currentLine + c_LastChar;
				if(c_LastChar == '\n')
					//quit the inner while loop to read the nextline
					break;
			}
			
			if(currentLine.length() > 0)
			{
				
				int int_Cnt;
				string str_CurrentString = "";
				tabData.clear();
 
				for(int_Cnt=0;int_Cnt < (int)currentLine.length()-1;int_Cnt++)
				{
					if(currentLine[int_Cnt] != ' ')
					{
						str_CurrentString = str_CurrentString + currentLine[int_Cnt];
					}
					else
					{
						tabData.insert(tabData.end(), str_CurrentString);
						str_CurrentString = "";
					}
				}
				
				if(tabData.at(2) == str_IPAddress && tabData.at(3) == "778899C" && tabData.at(5) == "GXJFUH")
				{
					for(int_Cnt=0;int_Cnt<(int)tabData.size()-1;int_Cnt++)
					{
						int int_byte;
						string str_Host = getHostName(tabData.at(6));
						string byte = tabData.at(4);
						int_byte = stringToInt(byte);
						_UserReport.setUserIP(str_IPAddress);
						_UserReport.addGETCnt();
						_UserReport.addLineCnt();
						_UserReport.insertAddress(str_Host);
						_UserReport.addByteTotal(int_byte);
					}
				}
			}
			
			
		}
		return _UserReport;
	}

Open in new window

0
jaxrpc
Asked:
jaxrpc
2 Solutions
 
WodCommented:
You could try this (example from: http://www.experts-exchange.com/Programming/Programming_Languages/Cplusplus/Q_20651518.html):
#include <iostream>
#include <iomanip>
#include <fstream>
#include <vector>
#include <string>
 
using namespace std;
 
int main()
{
      ifstream in("test.csv");
      vector < vector <string> > data;
      string element, delimiters = ",\n\r";
      int row = 0;
      char ch;
 
      data.push_back( vector <string>() );
 
      while( in.read( (char*)&ch, 1 ) )
      {
 
            if( delimiters.find_first_of(ch) == delimiters.npos )
            {
                  element += ch;
            }
            else
            {
                  if( ch != '\r' )
                  {
                        data[row].push_back( element );
                        element = "";
 
                        if( ch == '\n' )
                        {
                              data.push_back( vector <string>() );
                              row++;
                        }
                  }
            }
      }
 
      if( element.size() > 0 )
            data[row].push_back( element );
 
      in.close();
 
      for( unsigned int x = 0; x < data.size(); x++ )
      {
            for( unsigned int y = 0; y < data[x].size(); y++ )
            {
                  cout << data[x][y] << ",";
            }
            cout << endl;
      }
 
      return 0;
}

Open in new window

0
 
Infinity08Commented:
Why read one character at a time. Why not use getline to read a line, and then split it up using a combination of find and substr :

        http://www.cplusplus.com/reference/string/getline.html
        http://www.cplusplus.com/reference/string/string/find.html
        http://www.cplusplus.com/reference/string/string/substr.html
0
 
grg99Commented:
You're not going to make any progress until you narrow down the area of inefficiency.

Many times I've thought the slowness was in one area, and I wasted time speeding up the wrong code.

If you have a code profiler, use it first to get some idea what is slow.

If not, just comment out a section of code at a time and see how the speed changes.

I'd first try commenting out the code that takes the final parsed value and stores it away.   All that GetHostName could be really slow, we don't know.



0
 
jaxrpcAuthor Commented:
Hi, i tried using getline, substr and find. the speed is 80% better now. i think it is the reading of file that makes my program slow.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Tackle projects and never again get stuck behind a technical roadblock.
Join Now