Solved

Writing Space Delimited file

Posted on 2007-12-03
4
793 Views
Last Modified: 2008-02-01
Hi,

I am trying to read a space delimited file using C++ but i realised that my program is slow. It take minutes to parse a 1mb txt file. below is my code for reading the space delimited file. Is there a better way to readline() and straight get all the space delimited words into an array?


UserReport analyseFileUser(string str_IPAddress)
	{
		UserReport _UserReport;
		vector<string> tabData;
		while(! fileIn.eof())
		{
			string currentLine = "";
 
			while(! fileIn.eof())
			{
				//Get an input line
				char c_LastChar;
				fileIn.get(c_LastChar);
				
 
				if(fileIn.fail())
					//if read file fail at this point, just exit and parse the line
					break;
				if(c_LastChar != '\r' && c_LastChar != '\n')
					//if c_LastChar is not at \n means it is not end of line yet, continue the loop
					currentLine = currentLine + c_LastChar;
				if(c_LastChar == '\n')
					//quit the inner while loop to read the nextline
					break;
			}
			
			if(currentLine.length() > 0)
			{
				
				int int_Cnt;
				string str_CurrentString = "";
				tabData.clear();
 
				for(int_Cnt=0;int_Cnt < (int)currentLine.length()-1;int_Cnt++)
				{
					if(currentLine[int_Cnt] != ' ')
					{
						str_CurrentString = str_CurrentString + currentLine[int_Cnt];
					}
					else
					{
						tabData.insert(tabData.end(), str_CurrentString);
						str_CurrentString = "";
					}
				}
				
				if(tabData.at(2) == str_IPAddress && tabData.at(3) == "778899C" && tabData.at(5) == "GXJFUH")
				{
					for(int_Cnt=0;int_Cnt<(int)tabData.size()-1;int_Cnt++)
					{
						int int_byte;
						string str_Host = getHostName(tabData.at(6));
						string byte = tabData.at(4);
						int_byte = stringToInt(byte);
						_UserReport.setUserIP(str_IPAddress);
						_UserReport.addGETCnt();
						_UserReport.addLineCnt();
						_UserReport.insertAddress(str_Host);
						_UserReport.addByteTotal(int_byte);
					}
				}
			}
			
			
		}
		return _UserReport;
	}

Open in new window

0
Comment
Question by:jaxrpc
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
4 Comments
 
LVL 7

Accepted Solution

by:
Wod earned 250 total points
ID: 20395428
You could try this (example from: http://www.experts-exchange.com/Programming/Programming_Languages/Cplusplus/Q_20651518.html):
#include <iostream>
#include <iomanip>
#include <fstream>
#include <vector>
#include <string>
 
using namespace std;
 
int main()
{
      ifstream in("test.csv");
      vector < vector <string> > data;
      string element, delimiters = ",\n\r";
      int row = 0;
      char ch;
 
      data.push_back( vector <string>() );
 
      while( in.read( (char*)&ch, 1 ) )
      {
 
            if( delimiters.find_first_of(ch) == delimiters.npos )
            {
                  element += ch;
            }
            else
            {
                  if( ch != '\r' )
                  {
                        data[row].push_back( element );
                        element = "";
 
                        if( ch == '\n' )
                        {
                              data.push_back( vector <string>() );
                              row++;
                        }
                  }
            }
      }
 
      if( element.size() > 0 )
            data[row].push_back( element );
 
      in.close();
 
      for( unsigned int x = 0; x < data.size(); x++ )
      {
            for( unsigned int y = 0; y < data[x].size(); y++ )
            {
                  cout << data[x][y] << ",";
            }
            cout << endl;
      }
 
      return 0;
}

Open in new window

0
 
LVL 53

Assisted Solution

by:Infinity08
Infinity08 earned 250 total points
ID: 20395472
Why read one character at a time. Why not use getline to read a line, and then split it up using a combination of find and substr :

        http://www.cplusplus.com/reference/string/getline.html
        http://www.cplusplus.com/reference/string/string/find.html
        http://www.cplusplus.com/reference/string/string/substr.html
0
 
LVL 22

Expert Comment

by:grg99
ID: 20395577
You're not going to make any progress until you narrow down the area of inefficiency.

Many times I've thought the slowness was in one area, and I wasted time speeding up the wrong code.

If you have a code profiler, use it first to get some idea what is slow.

If not, just comment out a section of code at a time and see how the speed changes.

I'd first try commenting out the code that takes the final parsed value and stores it away.   All that GetHostName could be really slow, we don't know.



0
 

Author Comment

by:jaxrpc
ID: 20395629
Hi, i tried using getline, substr and find. the speed is 80% better now. i think it is the reading of file that makes my program slow.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
I could not build boost code, 10 107
Are the two lines the same? If not what is the difference? 2 29
I could not set window to top 4 42
boost::uuid crashes 17 38
Often, when implementing a feature, you won't know how certain events should be handled at the point where they occur and you'd rather defer to the user of your function or class. For example, a XML parser will extract a tag from the source code, wh…
Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question