Solved

ifstream and ofstream..  save/read from memory?

Posted on 2004-08-12
8
922 Views
Last Modified: 2012-05-05
i got a third party library and one of the function takes an ifstream or ofstream for saving and reading data to/from a file. Consider that i can't modify that libray.. is there something i can do so i can get it to write to (or read from) a memory buffer instead of a physical file?
I want to do this because calling those function calls is the only way for me to retrieve the whole data stored in that object.. but writing/reading from disk slows down the process a lot!

Thanks
0
Comment
Question by:rw8
  • 3
  • 2
  • 2
  • +1
8 Comments
 
LVL 17

Expert Comment

by:rstaveley
ID: 11791312
ifstream and ofstream are streams, which means that they buffer and therefore ought to be fast compared with unbuffered file reads/writes. You could increase the size of the buffer, I guess. Try the following, which you'll need to tweak for UN*X, because it uses a Windoze timer:
--------8<--------
#include <windows.h>
#include <iostream>
#include <fstream>
#include <vector>

int main(int argc,const char* argv[])
{
      if (argc != 2)
            return std::cerr << "Usage: " << *argv << " {repetitions}\n\n\tTry 1024,2048,4096",1;

      int repetitions = atoi(*++argv);

      // String to log
      std::vector<char> test(1025,'x');test[1024] = '\0';

// Normal logging
{
      DWORD start = GetTickCount();
      {
            std::ofstream fout("logfile_normal.txt");
            DWORD start = GetTickCount();
            for (int i = 0;i < 1024;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Normal run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Normal run took " << finish-start << " millisecs including ctor and dtor\n";
}

// Fast logging
{
      DWORD start = GetTickCount();
      const std::streamsize BufferSize = 1024*1024;      /* Buffer size of 1M */
      std::vector<char> buf(BufferSize);
      {
            std::ofstream fout("logfile_fast.txt");
            fout.rdbuf()->pubsetbuf(&buf[0],buf.size());
            DWORD start = GetTickCount();
            for (int i = 0;i < 1024;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Fast run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Fast run took " << finish-start << " millisecs including ctor and dtor\n";
}

}
--------8<--------
Notice that you get a performance hit whenever the buffer gets flushed to disk, which is why I added a timer for the dtor in each case.
0
 
LVL 17

Expert Comment

by:rstaveley
ID: 11791385
The illustration would have been better, if I'd put repetitions into the loop rather than leaving my hard-coded value. Try the following with command line parameters 1024, 2048 and 4096. 1024 repeatitions of a string 1024 characters long fits into the large 1M buffer, 2048 repetitions requires one flush of the buffer. You should get the idea.

It is informative - this time 8-)

--------8<--------
#include <windows.h>
#include <iostream>
#include <fstream>
#include <vector>

int main(int argc,const char* argv[])
{
      if (argc != 2)
            return std::cerr << "Usage: " << *argv << " {repetitions}\n\n\tTry 1024,2048,4096",1;

      int repetitions = atoi(*++argv);

      // String to log
      std::vector<char> test(1025,'x');test[1024] = '\0';

// Normal logging
{
      DWORD start = GetTickCount();
      {
            std::ofstream fout("logfile_normal.txt");
            DWORD start = GetTickCount();
            for (int i = 0;i < repetitions;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Normal run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Normal run took " << finish-start << " millisecs including ctor and dtor\n";
}

// Fast logging
{
      DWORD start = GetTickCount();
      const std::streamsize BufferSize = 1024*1024;      /* Buffer size of 1M */
      std::vector<char> buf(BufferSize);
      {
            std::ofstream fout("logfile_fast.txt");
            fout.rdbuf()->pubsetbuf(&buf[0],buf.size());
            DWORD start = GetTickCount();
            for (int i = 0;i < repetitions;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Fast run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Fast run took " << finish-start << " millisecs including ctor and dtor\n";
}

}
--------8<--------
0
 
LVL 11

Expert Comment

by:bcladd
ID: 11792242
The library functions take ofstream (ifstream), not ostream (istream) references? If they take an ostream reference you can pass in an ostringstream and that will write into memory rather than out to the disk. You can then do whatever you want with the resulting string value in the buffer.

-bcl
0
Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 4

Accepted Solution

by:
anthony_w earned 125 total points
ID: 11793178
If the functions really do take ofstream and ifstream references, and you really don't want to handle the I/O, then provided they don't call open/close/is_open, which are the only {o|i}fstream-specific functions, you can just get away with deriving new classes from std::ofstream and std::ifstream, which override all the protected methods, and forward them to a std::stringstream member.

If the functions call the rdbuf member function, then you will also need to derive a class from std::filebuf, so you can return a pointer to it. This class will similarly need to forward all its members to the buffer of the contained stringstream
0
 

Author Comment

by:rw8
ID: 11806930
it's taking ofstream and ifstream reference and I don't really want to handle the I/O myself.

Anthony_w: I'm using Visual C++ 6.0. I don't know much about the different stream class really and the MSDN help doesn't really talk too much about the std::stringstream.
Anyway, i've tried deriving a class from ofstream:


//copied from the other class that use ofstream
#if (_MSC_VER >= 1300) || defined(USE_NEW_IOSTREAMS)
#include <fstream>
using namespace std;
#else
#ifndef _INC_FSTREAM
#include <fstream.h>
#
#endif
#endif


#if !defined(AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_)
#define AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_

#if _MSC_VER > 1000
#pragma once
#endif // _MSC_VER > 1000

class memofstream : public ofstream  
{
public:
      memofstream();
      virtual ~memofstream();

private:      
      stringstream m_stringStream;
};

#endif // !defined(AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_)



and when i compile it, the compiler complains:
syntax error : missing ';' before identifier 'm_stringStream'
'stringstream' : missing storage-class or type specifiers
'm_stringStream' : missing storage-class or type specifiers


so what did i do wrong?
0
 

Author Comment

by:rw8
ID: 11806945
btw, i believe i'm using:
#include<fstream.h>
not
#include <fstream>
using namespace std;


the reason is.. if i replace:

#if (_MSC_VER >= 1300) || defined(USE_NEW_IOSTREAMS)
#include <fstream>
using namespace std;
#else
#ifndef _INC_FSTREAM
#include <fstream.h>
#
#endif
#endif


with

#include <fstream>
using namespace std;


i got heaps of:    "'ofstream' : ambiguous symbol" error on the other file that use ofstream
0
 
LVL 4

Expert Comment

by:anthony_w
ID: 11807465
std::stringstream is in the <sstream> header, so you need

#include <sstream>

at the top to make it compile.

0
 
LVL 17

Expert Comment

by:rstaveley
ID: 11808676
>  got heaps of:    "'ofstream' : ambiguous symbol" error on the other file that use ofstream

You should use the standard headers with VC 6.

Can you compile and execute the following?
--------8<--------
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

void some_func(ostream& os) // Your library function, taking ostream&
{
      os << "Hello, stream\n";
}

int main()
{
      ostringstream oss;
      some_func(oss);
      cout << oss.str();
      return 0;
}
--------8<--------

If so, you should be able to do likewise with your class, when you've got the includes right.
0

Featured Post

Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Handle Exceptions during instantiation 28 422
Least Squares Curve Fitting 4 96
Add values of each row in an array 3 63
designing in object programming 12 94
Many modern programming languages support the concept of a property -- a class member that combines characteristics of both a data member and a method.  These are sometimes called "smart fields" because you can add logic that is applied automaticall…
Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question