Go Premium for a chance to win a PS4. Enter to Win

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 342
  • Last Modified:

dump one stream into another without multiple copies

I'm working on a little utility application which needs to have a place to send error messages even if a console or file isn't available yet.

The idea is to cache messages into a stringstream until a file can be opened (fstream), then dump the contents of the cache stream into the "true" message stream.  Even if it has to wait until the destructor is called.

Stripped down to its essentials:

class ostream_sink 
{
 private:
 bool m_caching;
 std::string m_filename;
 std::ostringstream *m_cache;
 std::ofstream *m_dest;
 // imagine constructor and destructor details, and setting the file name
 bool open_file(); // implementation omitted
 public:
 void start_caching() { m_caching = true; }
 void stop_caching(); // question is about implmenting this routine

 std::ostream &stream();
 std::ostream &stream() const;
 template <class T> ostream_sink &operator<<(T const &v) { stream() << v; }
 template <class T> ostream_sink &operator<<(T const &v) const { stream() << v; }
};

 std::ostream &stream() const 
{
 if (m_caching) 
    return *m_cache;
   else return *m_dest; 
}

std::ostream &ostream_sink::stream ()
{ if (m_caching) if (open_file()) stop_caching(); 
   if (m_caching) 
      return *m_cache; 
      else return *m_dest; 
}

Open in new window


I suppose I could always use the stringstream's str() mehod:

ostream_sink::stop_caching ()
{
 if (m_caching) 
  {
   (*m_dest) << m_cache->str();
   m_cache->str(string()); 
   m_caching = false;
 }
}

Open in new window


but that string gets copied twice.   Is there a way do dump the cache's streambuf into the file stream's?  You certainly can't swap a stringbuf with a filebuf.....
0
Spencer Simpson
Asked:
Spencer Simpson
  • 4
  • 2
  • 2
3 Solutions
 
jkrCommented:
The snippets above actually do look OK. Could you provide some more code? Maybe the problem is located elsewhere...
0
 
Spencer SimpsonAuthor Commented:
It's not a "somethign doesn't work" problem, it's a question of efficiency.    The snippets provide a simplified example of the concept.

m_cache->str() returns a copy of the string, which might have a large amount of text in it. Putting it into the m_dest copies the text again.  So not only is a large amount of memory allocated and deallocated for the temporary string, there are potentially two fairly large copy operations involved.
0
 
jkrCommented:
Ah, OK, now I see what you mean. In that case, you could use a 'basic_filebuf' directly and use the stringstream's buffer as the source using 'pubsetbuf()', see e.g. the example at http://msdn.microsoft.com/en-us/library/vstudio/tzf8k3z8%28v=vs.100%29.aspx ("basic_filebuf Class" - scroll down). However, I haven't tried that myself.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
jkrCommented:
Oh, maybe also of interst in this context: http://stackoverflow.com/questions/2079912/simpler-way-to-create-a-c-memorystream-from-char-size-t-without-copying-t ("Simpler way to create a C++ memorystream from (char*, size_t), without copying the data?")
0
 
Spencer SimpsonAuthor Commented:
After some examination, it appears that basic_streambuf has protected member functions that would let you do precisely this if they weren't protected.

The solution appears to be abandoning this approach and taking the one the STL gods would tell you to take, if you asked you about anything past the basic funcitonality of these streams: roll your own streambuf class.
0
 
ambienceCommented:
Another solution is to use strstream rather than stringstream even though in general its not a good practice but since you are encapsulating everything it may be worth it. The benefit is that it wont copy buffers around and you can even initialize an strstream around a static buffer. The str() of strstream would give you a pointer to its internal buffer and the pcount() would give the size of data.

dest->write(...)

Even if you write your own streambuf implementation the best you can do is build some sort of buffer pooling, otherwise you'd still have to write to ofstream and then that becomes essentially the same as using an strstream instead.
0
 
jkrCommented:
>>After some examination, it appears that basic_streambuf has protected member functions
>>that would let you do precisely this if they weren't protected.

Derive from it and add a public member that provides access to the protected member ;o)
0
 
ambienceCommented:
Unless the size of error messages is bounded by an upper limit writing a custom streambuf is complicated compared to using strstream.

But if you must go that route then theres no point in writing a custom stringstream and then copy to an ofstream. The best solution would be to rather implement a custom ofstream that has caching and retrying capabilities. This way you'd be able to avoid the inevitable copying from temp stream to the filestream. Further optimization would be to use memory mapped files internally but fallback to in memory buffers should the files can not be mapped. Once the file has opened switch to direct mem-mapped IO.
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

  • 4
  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now