Solved

ifstream and ofstream..  save/read from memory?

Posted on 2004-08-12
8
910 Views
Last Modified: 2012-05-05
i got a third party library and one of the function takes an ifstream or ofstream for saving and reading data to/from a file. Consider that i can't modify that libray.. is there something i can do so i can get it to write to (or read from) a memory buffer instead of a physical file?
I want to do this because calling those function calls is the only way for me to retrieve the whole data stored in that object.. but writing/reading from disk slows down the process a lot!

Thanks
0
Comment
Question by:rw8
  • 3
  • 2
  • 2
  • +1
8 Comments
 
LVL 17

Expert Comment

by:rstaveley
ID: 11791312
ifstream and ofstream are streams, which means that they buffer and therefore ought to be fast compared with unbuffered file reads/writes. You could increase the size of the buffer, I guess. Try the following, which you'll need to tweak for UN*X, because it uses a Windoze timer:
--------8<--------
#include <windows.h>
#include <iostream>
#include <fstream>
#include <vector>

int main(int argc,const char* argv[])
{
      if (argc != 2)
            return std::cerr << "Usage: " << *argv << " {repetitions}\n\n\tTry 1024,2048,4096",1;

      int repetitions = atoi(*++argv);

      // String to log
      std::vector<char> test(1025,'x');test[1024] = '\0';

// Normal logging
{
      DWORD start = GetTickCount();
      {
            std::ofstream fout("logfile_normal.txt");
            DWORD start = GetTickCount();
            for (int i = 0;i < 1024;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Normal run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Normal run took " << finish-start << " millisecs including ctor and dtor\n";
}

// Fast logging
{
      DWORD start = GetTickCount();
      const std::streamsize BufferSize = 1024*1024;      /* Buffer size of 1M */
      std::vector<char> buf(BufferSize);
      {
            std::ofstream fout("logfile_fast.txt");
            fout.rdbuf()->pubsetbuf(&buf[0],buf.size());
            DWORD start = GetTickCount();
            for (int i = 0;i < 1024;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Fast run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Fast run took " << finish-start << " millisecs including ctor and dtor\n";
}

}
--------8<--------
Notice that you get a performance hit whenever the buffer gets flushed to disk, which is why I added a timer for the dtor in each case.
0
 
LVL 17

Expert Comment

by:rstaveley
ID: 11791385
The illustration would have been better, if I'd put repetitions into the loop rather than leaving my hard-coded value. Try the following with command line parameters 1024, 2048 and 4096. 1024 repeatitions of a string 1024 characters long fits into the large 1M buffer, 2048 repetitions requires one flush of the buffer. You should get the idea.

It is informative - this time 8-)

--------8<--------
#include <windows.h>
#include <iostream>
#include <fstream>
#include <vector>

int main(int argc,const char* argv[])
{
      if (argc != 2)
            return std::cerr << "Usage: " << *argv << " {repetitions}\n\n\tTry 1024,2048,4096",1;

      int repetitions = atoi(*++argv);

      // String to log
      std::vector<char> test(1025,'x');test[1024] = '\0';

// Normal logging
{
      DWORD start = GetTickCount();
      {
            std::ofstream fout("logfile_normal.txt");
            DWORD start = GetTickCount();
            for (int i = 0;i < repetitions;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Normal run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Normal run took " << finish-start << " millisecs including ctor and dtor\n";
}

// Fast logging
{
      DWORD start = GetTickCount();
      const std::streamsize BufferSize = 1024*1024;      /* Buffer size of 1M */
      std::vector<char> buf(BufferSize);
      {
            std::ofstream fout("logfile_fast.txt");
            fout.rdbuf()->pubsetbuf(&buf[0],buf.size());
            DWORD start = GetTickCount();
            for (int i = 0;i < repetitions;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Fast run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Fast run took " << finish-start << " millisecs including ctor and dtor\n";
}

}
--------8<--------
0
 
LVL 11

Expert Comment

by:bcladd
ID: 11792242
The library functions take ofstream (ifstream), not ostream (istream) references? If they take an ostream reference you can pass in an ostringstream and that will write into memory rather than out to the disk. You can then do whatever you want with the resulting string value in the buffer.

-bcl
0
 
LVL 4

Accepted Solution

by:
anthony_w earned 125 total points
ID: 11793178
If the functions really do take ofstream and ifstream references, and you really don't want to handle the I/O, then provided they don't call open/close/is_open, which are the only {o|i}fstream-specific functions, you can just get away with deriving new classes from std::ofstream and std::ifstream, which override all the protected methods, and forward them to a std::stringstream member.

If the functions call the rdbuf member function, then you will also need to derive a class from std::filebuf, so you can return a pointer to it. This class will similarly need to forward all its members to the buffer of the contained stringstream
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 

Author Comment

by:rw8
ID: 11806930
it's taking ofstream and ifstream reference and I don't really want to handle the I/O myself.

Anthony_w: I'm using Visual C++ 6.0. I don't know much about the different stream class really and the MSDN help doesn't really talk too much about the std::stringstream.
Anyway, i've tried deriving a class from ofstream:


//copied from the other class that use ofstream
#if (_MSC_VER >= 1300) || defined(USE_NEW_IOSTREAMS)
#include <fstream>
using namespace std;
#else
#ifndef _INC_FSTREAM
#include <fstream.h>
#
#endif
#endif


#if !defined(AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_)
#define AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_

#if _MSC_VER > 1000
#pragma once
#endif // _MSC_VER > 1000

class memofstream : public ofstream  
{
public:
      memofstream();
      virtual ~memofstream();

private:      
      stringstream m_stringStream;
};

#endif // !defined(AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_)



and when i compile it, the compiler complains:
syntax error : missing ';' before identifier 'm_stringStream'
'stringstream' : missing storage-class or type specifiers
'm_stringStream' : missing storage-class or type specifiers


so what did i do wrong?
0
 

Author Comment

by:rw8
ID: 11806945
btw, i believe i'm using:
#include<fstream.h>
not
#include <fstream>
using namespace std;


the reason is.. if i replace:

#if (_MSC_VER >= 1300) || defined(USE_NEW_IOSTREAMS)
#include <fstream>
using namespace std;
#else
#ifndef _INC_FSTREAM
#include <fstream.h>
#
#endif
#endif


with

#include <fstream>
using namespace std;


i got heaps of:    "'ofstream' : ambiguous symbol" error on the other file that use ofstream
0
 
LVL 4

Expert Comment

by:anthony_w
ID: 11807465
std::stringstream is in the <sstream> header, so you need

#include <sstream>

at the top to make it compile.

0
 
LVL 17

Expert Comment

by:rstaveley
ID: 11808676
>  got heaps of:    "'ofstream' : ambiguous symbol" error on the other file that use ofstream

You should use the standard headers with VC 6.

Can you compile and execute the following?
--------8<--------
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

void some_func(ostream& os) // Your library function, taking ostream&
{
      os << "Hello, stream\n";
}

int main()
{
      ostringstream oss;
      some_func(oss);
      cout << oss.str();
      return 0;
}
--------8<--------

If so, you should be able to do likewise with your class, when you've got the includes right.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Unlike C#, C++ doesn't have native support for sealing classes (so they cannot be sub-classed). At the cost of a virtual base class pointer it is possible to implement a pseudo sealing mechanism The trick is to virtually inherit from a base class…
Templates For Beginners Or How To Encourage The Compiler To Work For You Introduction This tutorial is targeted at the reader who is, perhaps, familiar with the basics of C++ but would prefer a little slower introduction to the more ad…
The goal of the tutorial is to teach the user how to use functions in C++. The video will cover how to define functions, how to call functions and how to create functions prototypes. Microsoft Visual C++ 2010 Express will be used as a text editor an…
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now