Reading 10,000 records from a txt file

Use new to allocate memory and then delete the memory by using delete after the record is read..
if u post the code I can help you to correct it.

//For example:

string log_line;
ifstream log_file("access.log");

while(!log_file.eof())
{
   getline(log_file,log_line); // read a line of any length.
   cout<<log_line<<endl;
}
log_file.close();

>> the program gets terminated

Is there an access violation? Did you check with the debugger what happened?

You could make a try - catch block around, so you might get the error cause in the catch block. The code snippet from above looks fine - beside of the fact that outputting 10000 lines will last some time...

Regards, Alex

> about 500kB

That sounds suspiciously like DOS (the dreaded 640K ceiling which leaves old timers hearabouts in cold sweats). You're not using Visual C 1.5 to generate a DOS executable, are you?

> hearabouts

hereabouts   (the spelling mistake was symptomatic of a cold sweat)

>> Visual C 1.5 to generate a DOS executable

Do you mean Visual C++ 1.5.2 to generate a Win 3.xx executable?

There wasn't a 640K limit  in VC++ 1.x, but i remember that MFC CString allocated a minimum of 256 bytes (or more..) for any string. So 16 MB RAM wasn't enough to read 10000 CString objects from database...

Regards, Alex

>> So 16 MB RAM

Maybe it was  4 MB ;-)

Cold sweats seem to have different effects on different people ...

MS DOS has (can we say had?) 640K of addressable conventional memory. A pointer could conveniently be used to address any 64K within that space but you would need to do clever things with far pointer arithmetic to address > 64K (recall huge pointers and pointer normalisation on far pointers... shudder). If you needed to address more than 640K, you needed to do clever things with extended or expanded memory. I'm not really up with version numbers of Microsoft C (I was a Turbo C fan back in those days), but I *think* that Microsoft's Visual C version 1 compilers were for DOS and Windows 3.x only. If you run a DOS executable on a Win32 platform, it runs in 8086 emulation, which means that you are back in the bad old days of 640K. I had no idea that they had things like CString back in those days. I expect the internal storage was in Cuniform :-)

//main.cpp
#include <stdio.h>
#include <string>
#include <fstream>
#include <iostream>

#include "record.h"

using namespace std;



string prefix;
string firstName;
string MiddleInit;
string lastName;

string title;
string company;
string department;
string address;
string city;
string state;
string zip1;
string zip2;

Record *record;

bool ReadFile(string Infilename, string Outfilename)
{
      ifstream filein;
      ofstream fileout;
      string temp;
      int pos;

      filein.open(Infilename.c_str(), ios::in);
      fileout.open (Outfilename.c_str (), ios::out);

      if ( filein.fail() )
      {
            cout << "Cannot open file to read." << endl;
            return false;
      }

      if ( fileout.fail() )
      {
            cout << "Cannot open file to save." << endl;
            return false;
      }

      int size = 0;


      while (!filein.eof() && size < 5000) //if I take out the size < 5000 or put in a number > 5000, the program got terminated with an error
      {
            getline(filein, temp);
            pos = temp.find (",", 0);
            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            prefix = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            firstName = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            MiddleInit = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            lastName = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            title = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            company = temp.substr (1, pos - 2);
            
            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            department = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            address = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            city = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            state = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            zip1 = temp.substr (1, pos - 2);

            temp.erase (0, pos + 1);
            pos = temp.find (",", 0);
            zip2 = temp.substr (1, pos - 2);

            record = new Record(prefix, firstName, MiddleInit, lastName, title, company, department, address, city, state, zip1, zip2);

            fileout << record->getPrefix() << " " << record->getfirstName() << " " << record->getMiddleInit() << " " << record->getlastName() << endl;
            if (record->getTitle() !="")
                  fileout << record->getTitle() << endl;
            fileout << record->getCompany() << endl;
            if (record->getDept() !="")
                  fileout << record->getDept() << endl;
            fileout << record->getAdd() << endl;
            fileout << record->getCity() << " " << record->getState() << " " << record->getZip1() << "-" << record->getZip2() << endl << endl;
            
            size++;
            delete record;

      }

}      


void main()
{
      string Infilename, Outfilename;
      cout << "Enter a file name to read: ";
      cin >> Infilename;
      cout << "Enter a file name to save: ";
      cin >> Outfilename;
      bool success = ReadFile (Infilename, Outfilename);


}


//record.h
#include <stdio.h>
#include <string>
#include <iostream>

using namespace std;

class Record
{
private:
      string prefix;
      string firstName;
      string MiddleInit;
      string lastName;
      string title;
      string company;
      string department;
      string address;
      string city;
      string state;
      string zip1;
      string zip2;
public:
      Record( const string str1 = "",  const string str2 ="" ,  const string str3 ="",  const string str4 ="",  const string str5 ="",  const string str6="",  const string str7="",  const string str8="",  const string str9="",  const string str10="",  const string str11="", const string str12="")
      {
            prefix = str1;
            firstName = str2;
            MiddleInit = str3;
            lastName = str4;
            title = str5;
            company = str6;
            department = str7;
            address = str8;
            city = str9;
            state = str10;
            zip1 = str11;
            zip2 = str12;
      }
      set( const string str1,  const string str2,  const string str3,  const string str4,  const string str5,  const string str6,  const string str7,  const string str8,  const string str9,  const string str10,  const string str11, const string str12)
      {
            prefix = str1;
            firstName = str2;
            MiddleInit = str3;
            lastName = str4;
            title = str5;
            company = str6;
            department = str7;
            address = str8;
            city = str9;
            state = str10;
            zip1 = str11;
            zip2 = str12;
      }
      string getPrefix()
      {
            return prefix;
      }

      string getfirstName()
      {
            return firstName;
      }

      string getMiddleInit()
      {
            return MiddleInit;
      }

      string getlastName()
      {
            return lastName;
      }

      string getTitle()
      {
            return title;
      }

      string getCompany()
      {
            return company;
      }

      string getDept()
      {
            return department;
      }

      string getAdd()
      {
            return address;
      }

      string getCity()
      {
            return city;
      }

      string getState()
      {
            return state;
      }

      string getZip1()
      {
            return zip1;
      }

      string getZip2()
      {
            return zip2;
      }


};



This is how the input file looks like. It has a few thousands lines like this. I don't know how the users get this file, but this will be the input to my program.

"00000001","MS","D","L","GOLDFARB","","DIRECTOR","INSTITUE OF MANAGEMENT & ADMIN","DEPT 82216","29 W 35TH ST","NEW YORK","NY","10001","2299",""
"01759019","MR","RICH","","MULHOLAND","","PURCHASING AGENT","J L STRUCTURAL INC","","111 STATION ST","ALIQUIPPA","PA","15001","3610",""

My error says "abnormal program termination". If I hit "Retry" then the outputs still come out correctly. But I would lile to have the users not see this error. Is there any way you know of, please?

Thank you very much for your help.

I got your code to work. I have to test against the actual file that has more than 5000 records. Can you mail me the file that has all the comma separated records information. Above is just 2 records.

mail at pb_india AT HOTMAIL DOT COM

Meanwhile try this:
Replace your record.cpp with this one. Let me know how it goes.
//main.cpp
#include <stdio.h>
#include <string>
#include <fstream>
#include <iostream>

#include "record.h"

using namespace std;



string prefix;
string firstName;
string MiddleInit;
string lastName;

string title;
string company;
string department;
string address;
string city;
string state;
string zip1;
string zip2;

Record *record=NULL;
void clearIT(void){
 prefix.erase();
 firstName.erase();
 MiddleInit.erase();
 lastName.erase();

 title.erase();
 company.erase();
 department.erase();
 address.erase();
 city.erase();
 state.erase();
 zip1.erase();
 zip2.erase();

}
bool ReadFile(string Infilename, string Outfilename)
{
     ifstream filein;
     ofstream fileout;
     string temp="";
     int pos=0;

     filein.open(Infilename.c_str(), ios::in);
     fileout.open (Outfilename.c_str (), ios::out);

     if ( filein.fail() )
     {
          cout << "Cannot open file to read." << endl;
          return false;
     }

     if ( fileout.fail() )
     {
          cout << "Cannot open file to save." << endl;
          return false;
     }

     int size = 0;


     while (!filein.eof() )
     {
                  clearIT();
          getline(filein, temp);
          pos = temp.find (",", 0);
          temp.erase (0, pos + 1);
          pos = temp.find (",", 0);
          if(pos>2){
                    prefix = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    firstName = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    MiddleInit = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    lastName = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    title = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    company = temp.substr (1, pos - 2);
         
                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    department = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    address = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    city = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    state = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    zip1 = temp.substr (1, pos - 2);

                    temp.erase (0, pos + 1);
                    pos = temp.find (",", 0);
                    zip2 = temp.substr (1, pos - 2);
              }
          record = new Record(prefix, firstName, MiddleInit, lastName, title, company, department, address, city, state, zip1, zip2);

          fileout << record->getPrefix() << " " << record->getfirstName() << " " << record->getMiddleInit() << " " << record->getlastName() << endl;
          if (record->getTitle() !="")
               fileout << record->getTitle() << endl;
          fileout << record->getCompany() << endl;
          if (record->getDept() !="")
               fileout << record->getDept() << endl;
          fileout << record->getAdd() << endl;
          fileout << record->getCity() << " " << record->getState() << " " << record->getZip1() << "-" << record->getZip2() << endl << endl;
         
          size++;
              if(record){
                  delete record;
                  record=NULL;
              }
              temp.erase();

     }

}    


void main()
{
     string Infilename, Outfilename;
     cout << "Enter a file name to read: ";
     cin >> Infilename;
     cout << "Enter a file name to save: ";
     cin >> Outfilename;
     bool success = ReadFile (Infilename, Outfilename);


}

 while (!filein.eof() )
     {
         getline(filein, temp);

That sequence looks wrong as you don't check getline error. At end-of-file the last call to getline() most probably fails, what sets the stream status to fail and not to eof. The condition size < 5000 simply prevents your prog from reaching file end.

I would suggest to try that:

    while (!getline(filein, temp))
    {
         ...

Regards, Alex


     

>> I had no idea that they had things like CString back in those days.

I started with 16 Bit VC++ 1.3 and MFC 1.0 in 1993. The platform was Windows 3.11. The difference to Borland C++ was that VC++ had an integrated  'visual' Resource Editor - called ApStudio and MFC with Scribble Tutorial.. ODBC Access came with VC 1.4 in spring 1994 (CRecordset class and jet engine driver for Access 1.1). DOS applications could be compiled using a DOS-Extender (i forgot the name as i had no DOS past but came from VMS).

There was a Bit VC 1.1 (same time as 16 Bit 1.3) for Windows NT 3.5 (that had a Win 3.11 GUI).

Regards, Alex

>> There was a Bit VC 1.1

Seems i have to replace my keyboard. Should be "There was a 32 Bit VC 1.1".

<Apologies for the continued trip down memory lane...>

DOS extenders like Phar Lap were only needed to extend the capabilities of DOS to break through the 640K barrier. I did pitifully little Windows 3.x stuff with Borland's OWL (Object Windows Library). Borland didn't seem to license much off Microsoft... possibly only the resource compiler. But most of my development at that time was text mode, compiled with a 1989 copy of Turbo C 1.0. which could fit onto a bootable 720K floppy with enough space for pkzip and a text editor but which wasn't to be trusted with floating point.

<Returning to the current decade... and the thread>

I think you're right about getline or perhaps an insufficient number of commas in an errant record making npos == string::npos, which isn't tested for. It certainly isn't a DOS thing :-)