Solved

dump one stream into another without multiple copies

Posted on 2013-01-08
8
329 Views
Last Modified: 2013-01-15
I'm working on a little utility application which needs to have a place to send error messages even if a console or file isn't available yet.

The idea is to cache messages into a stringstream until a file can be opened (fstream), then dump the contents of the cache stream into the "true" message stream.  Even if it has to wait until the destructor is called.

Stripped down to its essentials:

class ostream_sink 
{
 private:
 bool m_caching;
 std::string m_filename;
 std::ostringstream *m_cache;
 std::ofstream *m_dest;
 // imagine constructor and destructor details, and setting the file name
 bool open_file(); // implementation omitted
 public:
 void start_caching() { m_caching = true; }
 void stop_caching(); // question is about implmenting this routine

 std::ostream &stream();
 std::ostream &stream() const;
 template <class T> ostream_sink &operator<<(T const &v) { stream() << v; }
 template <class T> ostream_sink &operator<<(T const &v) const { stream() << v; }
};

 std::ostream &stream() const 
{
 if (m_caching) 
    return *m_cache;
   else return *m_dest; 
}

std::ostream &ostream_sink::stream ()
{ if (m_caching) if (open_file()) stop_caching(); 
   if (m_caching) 
      return *m_cache; 
      else return *m_dest; 
}

Open in new window


I suppose I could always use the stringstream's str() mehod:

ostream_sink::stop_caching ()
{
 if (m_caching) 
  {
   (*m_dest) << m_cache->str();
   m_cache->str(string()); 
   m_caching = false;
 }
}

Open in new window


but that string gets copied twice.   Is there a way do dump the cache's streambuf into the file stream's?  You certainly can't swap a stringbuf with a filebuf.....
0
Comment
Question by:Spencer Simpson
  • 4
  • 2
  • 2
8 Comments
 
LVL 86

Expert Comment

by:jkr
Comment Utility
The snippets above actually do look OK. Could you provide some more code? Maybe the problem is located elsewhere...
0
 
LVL 1

Author Comment

by:Spencer Simpson
Comment Utility
It's not a "somethign doesn't work" problem, it's a question of efficiency.    The snippets provide a simplified example of the concept.

m_cache->str() returns a copy of the string, which might have a large amount of text in it. Putting it into the m_dest copies the text again.  So not only is a large amount of memory allocated and deallocated for the temporary string, there are potentially two fairly large copy operations involved.
0
 
LVL 86

Expert Comment

by:jkr
Comment Utility
Ah, OK, now I see what you mean. In that case, you could use a 'basic_filebuf' directly and use the stringstream's buffer as the source using 'pubsetbuf()', see e.g. the example at http://msdn.microsoft.com/en-us/library/vstudio/tzf8k3z8%28v=vs.100%29.aspx ("basic_filebuf Class" - scroll down). However, I haven't tried that myself.
0
 
LVL 86

Expert Comment

by:jkr
Comment Utility
Oh, maybe also of interst in this context: http://stackoverflow.com/questions/2079912/simpler-way-to-create-a-c-memorystream-from-char-size-t-without-copying-t ("Simpler way to create a C++ memorystream from (char*, size_t), without copying the data?")
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 1

Author Comment

by:Spencer Simpson
Comment Utility
After some examination, it appears that basic_streambuf has protected member functions that would let you do precisely this if they weren't protected.

The solution appears to be abandoning this approach and taking the one the STL gods would tell you to take, if you asked you about anything past the basic funcitonality of these streams: roll your own streambuf class.
0
 
LVL 22

Accepted Solution

by:
ambience earned 450 total points
Comment Utility
Another solution is to use strstream rather than stringstream even though in general its not a good practice but since you are encapsulating everything it may be worth it. The benefit is that it wont copy buffers around and you can even initialize an strstream around a static buffer. The str() of strstream would give you a pointer to its internal buffer and the pcount() would give the size of data.

dest->write(...)

Even if you write your own streambuf implementation the best you can do is build some sort of buffer pooling, otherwise you'd still have to write to ofstream and then that becomes essentially the same as using an strstream instead.
0
 
LVL 86

Assisted Solution

by:jkr
jkr earned 50 total points
Comment Utility
>>After some examination, it appears that basic_streambuf has protected member functions
>>that would let you do precisely this if they weren't protected.

Derive from it and add a public member that provides access to the protected member ;o)
0
 
LVL 22

Assisted Solution

by:ambience
ambience earned 450 total points
Comment Utility
Unless the size of error messages is bounded by an upper limit writing a custom streambuf is complicated compared to using strstream.

But if you must go that route then theres no point in writing a custom stringstream and then copy to an ofstream. The best solution would be to rather implement a custom ofstream that has caching and retrying capabilities. This way you'd be able to avoid the inevitable copying from temp stream to the filestream. Further optimization would be to use memory mapped files internally but fallback to in memory buffers should the files can not be mapped. Once the file has opened switch to direct mem-mapped IO.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Article by: SunnyDark
This article's goal is to present you with an easy to use XML wrapper for C++ and also present some interesting techniques that you might use with MS C++. The reason I built this class is to ease the pain of using XML files with C++, since there is…
This article shows you how to optimize memory allocations in C++ using placement new. Applicable especially to usecases dealing with creation of large number of objects. A brief on problem: Lets take example problem for simplicity: - I have a G…
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now