Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

ifstream and ofstream..  save/read from memory?

Posted on 2004-08-12
8
Medium Priority
?
965 Views
Last Modified: 2012-05-05
i got a third party library and one of the function takes an ifstream or ofstream for saving and reading data to/from a file. Consider that i can't modify that libray.. is there something i can do so i can get it to write to (or read from) a memory buffer instead of a physical file?
I want to do this because calling those function calls is the only way for me to retrieve the whole data stored in that object.. but writing/reading from disk slows down the process a lot!

Thanks
0
Comment
Question by:rw8
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
  • 2
  • +1
8 Comments
 
LVL 17

Expert Comment

by:rstaveley
ID: 11791312
ifstream and ofstream are streams, which means that they buffer and therefore ought to be fast compared with unbuffered file reads/writes. You could increase the size of the buffer, I guess. Try the following, which you'll need to tweak for UN*X, because it uses a Windoze timer:
--------8<--------
#include <windows.h>
#include <iostream>
#include <fstream>
#include <vector>

int main(int argc,const char* argv[])
{
      if (argc != 2)
            return std::cerr << "Usage: " << *argv << " {repetitions}\n\n\tTry 1024,2048,4096",1;

      int repetitions = atoi(*++argv);

      // String to log
      std::vector<char> test(1025,'x');test[1024] = '\0';

// Normal logging
{
      DWORD start = GetTickCount();
      {
            std::ofstream fout("logfile_normal.txt");
            DWORD start = GetTickCount();
            for (int i = 0;i < 1024;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Normal run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Normal run took " << finish-start << " millisecs including ctor and dtor\n";
}

// Fast logging
{
      DWORD start = GetTickCount();
      const std::streamsize BufferSize = 1024*1024;      /* Buffer size of 1M */
      std::vector<char> buf(BufferSize);
      {
            std::ofstream fout("logfile_fast.txt");
            fout.rdbuf()->pubsetbuf(&buf[0],buf.size());
            DWORD start = GetTickCount();
            for (int i = 0;i < 1024;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Fast run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Fast run took " << finish-start << " millisecs including ctor and dtor\n";
}

}
--------8<--------
Notice that you get a performance hit whenever the buffer gets flushed to disk, which is why I added a timer for the dtor in each case.
0
 
LVL 17

Expert Comment

by:rstaveley
ID: 11791385
The illustration would have been better, if I'd put repetitions into the loop rather than leaving my hard-coded value. Try the following with command line parameters 1024, 2048 and 4096. 1024 repeatitions of a string 1024 characters long fits into the large 1M buffer, 2048 repetitions requires one flush of the buffer. You should get the idea.

It is informative - this time 8-)

--------8<--------
#include <windows.h>
#include <iostream>
#include <fstream>
#include <vector>

int main(int argc,const char* argv[])
{
      if (argc != 2)
            return std::cerr << "Usage: " << *argv << " {repetitions}\n\n\tTry 1024,2048,4096",1;

      int repetitions = atoi(*++argv);

      // String to log
      std::vector<char> test(1025,'x');test[1024] = '\0';

// Normal logging
{
      DWORD start = GetTickCount();
      {
            std::ofstream fout("logfile_normal.txt");
            DWORD start = GetTickCount();
            for (int i = 0;i < repetitions;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Normal run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Normal run took " << finish-start << " millisecs including ctor and dtor\n";
}

// Fast logging
{
      DWORD start = GetTickCount();
      const std::streamsize BufferSize = 1024*1024;      /* Buffer size of 1M */
      std::vector<char> buf(BufferSize);
      {
            std::ofstream fout("logfile_fast.txt");
            fout.rdbuf()->pubsetbuf(&buf[0],buf.size());
            DWORD start = GetTickCount();
            for (int i = 0;i < repetitions;i++)
                  fout << &test[0];
            DWORD finish = GetTickCount();
            std::cout << "Fast run took " << finish-start << " millisecs\n";
      }
      DWORD finish = GetTickCount();
      std::cout << "Fast run took " << finish-start << " millisecs including ctor and dtor\n";
}

}
--------8<--------
0
 
LVL 11

Expert Comment

by:bcladd
ID: 11792242
The library functions take ofstream (ifstream), not ostream (istream) references? If they take an ostream reference you can pass in an ostringstream and that will write into memory rather than out to the disk. You can then do whatever you want with the resulting string value in the buffer.

-bcl
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 4

Accepted Solution

by:
anthony_w earned 500 total points
ID: 11793178
If the functions really do take ofstream and ifstream references, and you really don't want to handle the I/O, then provided they don't call open/close/is_open, which are the only {o|i}fstream-specific functions, you can just get away with deriving new classes from std::ofstream and std::ifstream, which override all the protected methods, and forward them to a std::stringstream member.

If the functions call the rdbuf member function, then you will also need to derive a class from std::filebuf, so you can return a pointer to it. This class will similarly need to forward all its members to the buffer of the contained stringstream
0
 

Author Comment

by:rw8
ID: 11806930
it's taking ofstream and ifstream reference and I don't really want to handle the I/O myself.

Anthony_w: I'm using Visual C++ 6.0. I don't know much about the different stream class really and the MSDN help doesn't really talk too much about the std::stringstream.
Anyway, i've tried deriving a class from ofstream:


//copied from the other class that use ofstream
#if (_MSC_VER >= 1300) || defined(USE_NEW_IOSTREAMS)
#include <fstream>
using namespace std;
#else
#ifndef _INC_FSTREAM
#include <fstream.h>
#
#endif
#endif


#if !defined(AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_)
#define AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_

#if _MSC_VER > 1000
#pragma once
#endif // _MSC_VER > 1000

class memofstream : public ofstream  
{
public:
      memofstream();
      virtual ~memofstream();

private:      
      stringstream m_stringStream;
};

#endif // !defined(AFX_MEMOFSTREAM_H__61B81A47_311F_4D1D_9F5C_B191931A7CF6__INCLUDED_)



and when i compile it, the compiler complains:
syntax error : missing ';' before identifier 'm_stringStream'
'stringstream' : missing storage-class or type specifiers
'm_stringStream' : missing storage-class or type specifiers


so what did i do wrong?
0
 

Author Comment

by:rw8
ID: 11806945
btw, i believe i'm using:
#include<fstream.h>
not
#include <fstream>
using namespace std;


the reason is.. if i replace:

#if (_MSC_VER >= 1300) || defined(USE_NEW_IOSTREAMS)
#include <fstream>
using namespace std;
#else
#ifndef _INC_FSTREAM
#include <fstream.h>
#
#endif
#endif


with

#include <fstream>
using namespace std;


i got heaps of:    "'ofstream' : ambiguous symbol" error on the other file that use ofstream
0
 
LVL 4

Expert Comment

by:anthony_w
ID: 11807465
std::stringstream is in the <sstream> header, so you need

#include <sstream>

at the top to make it compile.

0
 
LVL 17

Expert Comment

by:rstaveley
ID: 11808676
>  got heaps of:    "'ofstream' : ambiguous symbol" error on the other file that use ofstream

You should use the standard headers with VC 6.

Can you compile and execute the following?
--------8<--------
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;

void some_func(ostream& os) // Your library function, taking ostream&
{
      os << "Hello, stream\n";
}

int main()
{
      ostringstream oss;
      some_func(oss);
      cout << oss.str();
      return 0;
}
--------8<--------

If so, you should be able to do likewise with your class, when you've got the includes right.
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Unlike C#, C++ doesn't have native support for sealing classes (so they cannot be sub-classed). At the cost of a virtual base class pointer it is possible to implement a pseudo sealing mechanism The trick is to virtually inherit from a base class…
In days of old, returning something by value from a function in C++ was necessarily avoided because it would, invariably, involve one or even two copies of the object being created and potentially costly calls to a copy-constructor and destructor. A…
The viewer will be introduced to the technique of using vectors in C++. The video will cover how to define a vector, store values in the vector and retrieve data from the values stored in the vector.
The viewer will be introduced to the member functions push_back and pop_back of the vector class. The video will teach the difference between the two as well as how to use each one along with its functionality.

670 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question