Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

ifstream and ofstream..  save/read from memory?

Posted on 2004-08-12
8
Medium Priority
?
994 Views
Last Modified: 2012-05-05
i got a third party library and one of the function takes an ifstream or ofstream for saving and reading data to/from a file. Consider that i can't modify that libray.. is there something i can do so i can get it to write to (or read from) a memory buffer instead of a physical file?
I want to do this because calling those function calls is the only way for me to retrieve the whole data stored in that object.. but writing/reading from disk slows down the process a lot!

Thanks
0
Comment
Question by:rw8
  • 3
  • 2
  • 2
  • +1
8 Comments
 
LVL 17

Expert Comment

by:rstaveley
ID: 11791312
ifstream and ofstream are streams, which means that they buffer and therefore ought to be fast compared with unbuffered file reads/writes. You could increase the size of the buffer, I guess. Try the following, which you'll need to tweak for UN*X, because it uses a Windoze timer:
--------8<--------
#include <windows.h>
#include <iostream>
#include <fstream>
#include <vector>

int main(int argc,const char* argv[])
{
      if (argc != 2)
            return std::cerr << "Usage: " << *argv << " {repetitions}\n\n\tTry 1024,2048,4096",1;

      int repetitions = atoi(*++argv);

      // String to log
      std::vector<char> test(1025,'x');test[1024] = '\0';

// Normal logging
{
      DWORD start = GetTickCount();
      {
            std::ofstream fout("logfile_normal.txt");
            DWORD start = GetTickCount();
            for (int i = 0;i < 1024;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Normal run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Normal run took " << finish-start << " millisecs including ctor and dtor\n";
}

// Fast logging
{
      DWORD start = GetTickCount();
      const std::streamsize BufferSize = 1024*1024;      /* Buffer size of 1M */
      std::vector<char> buf(BufferSize);
      {
            std::ofstream fout("logfile_fast.txt");
            fout.rdbuf()->pubsetbuf(&buf[0],buf.size());
            DWORD start = GetTickCount();
            for (int i = 0;i < 1024;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Fast run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Fast run took " << finish-start << " millisecs including ctor and dtor\n";
}

}
--------8<--------
Notice that you get a performance hit whenever the buffer gets flushed to disk, which is why I added a timer for the dtor in each case.
0
 
LVL 17

Expert Comment

by:rstaveley
ID: 11791385
The illustration would have been better, if I'd put repetitions into the loop rather than leaving my hard-coded value. Try the following with command line parameters 1024, 2048 and 4096. 1024 repeatitions of a string 1024 characters long fits into the large 1M buffer, 2048 repetitions requires one flush of the buffer. You should get the idea.

It is informative - this time 8-)

--------8<--------
#include <windows.h>
#include <iostream>
#include <fstream>
#include <vector>

int main(int argc,const char* argv[])
{
      if (argc != 2)
            return std::cerr << "Usage: " << *argv << " {repetitions}\n\n\tTry 1024,2048,4096",1;

      int repetitions = atoi(*++argv);

      // String to log
      std::vector<char> test(1025,'x');test[1024] = '\0';

// Normal logging
{
      DWORD start = GetTickCount();
      {
            std::ofstream fout("logfile_normal.txt");
            DWORD start = GetTickCount();
            for (int i = 0;i < repetitions;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Normal run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Normal run took " << finish-start << " millisecs including ctor and dtor\n";
}

// Fast logging
{
      DWORD start = GetTickCount();
      const std::streamsize BufferSize = 1024*1024;      /* Buffer size of 1M */
      std::vector<char> buf(BufferSize);
      {
            std::ofstream fout("logfile_fast.txt");
            fout.rdbuf()->pubsetbuf(&buf[0],buf.size());
            DWORD start = GetTickCount();
            for (int i = 0;i < repetitions;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Fast run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Fast run took " << finish-start << " millisecs including ctor and dtor\n";
}

}
--------8<--------
0
 
LVL 11

Expert Comment

by:bcladd
ID: 11792242
The library functions take ofstream (ifstream), not ostream (istream) references? If they take an ostream reference you can pass in an ostringstream and that will write into memory rather than out to the disk. You can then do whatever you want with the resulting string value in the buffer.

-bcl
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 4

Accepted Solution

by:
anthony_w earned 500 total points
ID: 11793178
If the functions really do take ofstream and ifstream references, and you really don't want to handle the I/O, then provided they don't call open/close/is_open, which are the only {o|i}fstream-specific functions, you can just get away with deriving new classes from std::ofstream and std::ifstream, which override all the protected methods, and forward them to a std::stringstream member.

If the functions call the rdbuf member function, then you will also need to derive a class from std::filebuf, so you can return a pointer to it. This class will similarly need to forward all its members to the buffer of the contained stringstream
0
 

Author Comment

by:rw8
ID: 11806930
it's taking ofstream and ifstream reference and I don't really want to handle the I/O myself.

Anthony_w: I'm using Visual C++ 6.0. I don't know much about the different stream class really and the MSDN help doesn't really talk too much about the std::stringstream.
Anyway, i've tried deriving a class from ofstream:


//copied from the other class that use ofstream
#if (_MSC_VER >= 1300) || defined(USE_NEW_IOSTREAMS)
#include <fstream>
using namespace std;
#else
#ifndef _INC_FSTREAM
#include <fstream.h>
#
#endif
#endif


#if !defined(AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_)
#define AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_

#if _MSC_VER > 1000
#pragma once
#endif // _MSC_VER > 1000

class memofstream : public ofstream  
{
public:
      memofstream();
      virtual ~memofstream();

private:      
      stringstream m_stringStream;
};

#endif // !defined(AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_)



and when i compile it, the compiler complains:
syntax error : missing ';' before identifier 'm_stringStream'
'stringstream' : missing storage-class or type specifiers
'm_stringStream' : missing storage-class or type specifiers


so what did i do wrong?
0
 

Author Comment

by:rw8
ID: 11806945
btw, i believe i'm using:
#include<fstream.h>
not
#include <fstream>
using namespace std;


the reason is.. if i replace:

#if (_MSC_VER >= 1300) || defined(USE_NEW_IOSTREAMS)
#include <fstream>
using namespace std;
#else
#ifndef _INC_FSTREAM
#include <fstream.h>
#
#endif
#endif


with

#include <fstream>
using namespace std;


i got heaps of:    "'ofstream' : ambiguous symbol" error on the other file that use ofstream
0
 
LVL 4

Expert Comment

by:anthony_w
ID: 11807465
std::stringstream is in the <sstream> header, so you need

#include <sstream>

at the top to make it compile.

0
 
LVL 17

Expert Comment

by:rstaveley
ID: 11808676
>  got heaps of:    "'ofstream' : ambiguous symbol" error on the other file that use ofstream

You should use the standard headers with VC 6.

Can you compile and execute the following?
--------8<--------
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

void some_func(ostream& os) // Your library function, taking ostream&
{
      os << "Hello, stream\n";
}

int main()
{
      ostringstream oss;
      some_func(oss);
      cout << oss.str();
      return 0;
}
--------8<--------

If so, you should be able to do likewise with your class, when you've got the includes right.
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Often, when implementing a feature, you won't know how certain events should be handled at the point where they occur and you'd rather defer to the user of your function or class. For example, a XML parser will extract a tag from the source code, wh…
What is C++ STL?: STL stands for Standard Template Library and is a part of standard C++ libraries. It contains many useful data structures (containers) and algorithms, which can spare you a lot of the time. Today we will look at the STL Vector. …
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…

578 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question