Solved

dump one stream into another without multiple copies

Posted on 2013-01-08
8
333 Views
Last Modified: 2013-01-15
I'm working on a little utility application which needs to have a place to send error messages even if a console or file isn't available yet.

The idea is to cache messages into a stringstream until a file can be opened (fstream), then dump the contents of the cache stream into the "true" message stream.  Even if it has to wait until the destructor is called.

Stripped down to its essentials:

class ostream_sink 
{
 private:
 bool m_caching;
 std::string m_filename;
 std::ostringstream *m_cache;
 std::ofstream *m_dest;
 // imagine constructor and destructor details, and setting the file name
 bool open_file(); // implementation omitted
 public:
 void start_caching() { m_caching = true; }
 void stop_caching(); // question is about implmenting this routine

 std::ostream &stream();
 std::ostream &stream() const;
 template <class T> ostream_sink &operator<<(T const &v) { stream() << v; }
 template <class T> ostream_sink &operator<<(T const &v) const { stream() << v; }
};

 std::ostream &stream() const 
{
 if (m_caching) 
    return *m_cache;
   else return *m_dest; 
}

std::ostream &ostream_sink::stream ()
{ if (m_caching) if (open_file()) stop_caching(); 
   if (m_caching) 
      return *m_cache; 
      else return *m_dest; 
}

Open in new window


I suppose I could always use the stringstream's str() mehod:

ostream_sink::stop_caching ()
{
 if (m_caching) 
  {
   (*m_dest) << m_cache->str();
   m_cache->str(string()); 
   m_caching = false;
 }
}

Open in new window


but that string gets copied twice.   Is there a way do dump the cache's streambuf into the file stream's?  You certainly can't swap a stringbuf with a filebuf.....
0
Comment
Question by:Spencer Simpson
  • 4
  • 2
  • 2
8 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 38755929
The snippets above actually do look OK. Could you provide some more code? Maybe the problem is located elsewhere...
0
 
LVL 1

Author Comment

by:Spencer Simpson
ID: 38756002
It's not a "somethign doesn't work" problem, it's a question of efficiency.    The snippets provide a simplified example of the concept.

m_cache->str() returns a copy of the string, which might have a large amount of text in it. Putting it into the m_dest copies the text again.  So not only is a large amount of memory allocated and deallocated for the temporary string, there are potentially two fairly large copy operations involved.
0
 
LVL 86

Expert Comment

by:jkr
ID: 38756237
Ah, OK, now I see what you mean. In that case, you could use a 'basic_filebuf' directly and use the stringstream's buffer as the source using 'pubsetbuf()', see e.g. the example at http://msdn.microsoft.com/en-us/library/vstudio/tzf8k3z8%28v=vs.100%29.aspx ("basic_filebuf Class" - scroll down). However, I haven't tried that myself.
0
Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

 
LVL 86

Expert Comment

by:jkr
ID: 38756244
Oh, maybe also of interst in this context: http://stackoverflow.com/questions/2079912/simpler-way-to-create-a-c-memorystream-from-char-size-t-without-copying-t ("Simpler way to create a C++ memorystream from (char*, size_t), without copying the data?")
0
 
LVL 1

Author Comment

by:Spencer Simpson
ID: 38759761
After some examination, it appears that basic_streambuf has protected member functions that would let you do precisely this if they weren't protected.

The solution appears to be abandoning this approach and taking the one the STL gods would tell you to take, if you asked you about anything past the basic funcitonality of these streams: roll your own streambuf class.
0
 
LVL 22

Accepted Solution

by:
ambience earned 450 total points
ID: 38760840
Another solution is to use strstream rather than stringstream even though in general its not a good practice but since you are encapsulating everything it may be worth it. The benefit is that it wont copy buffers around and you can even initialize an strstream around a static buffer. The str() of strstream would give you a pointer to its internal buffer and the pcount() would give the size of data.

dest->write(...)

Even if you write your own streambuf implementation the best you can do is build some sort of buffer pooling, otherwise you'd still have to write to ofstream and then that becomes essentially the same as using an strstream instead.
0
 
LVL 86

Assisted Solution

by:jkr
jkr earned 50 total points
ID: 38761031
>>After some examination, it appears that basic_streambuf has protected member functions
>>that would let you do precisely this if they weren't protected.

Derive from it and add a public member that provides access to the protected member ;o)
0
 
LVL 22

Assisted Solution

by:ambience
ambience earned 450 total points
ID: 38762275
Unless the size of error messages is bounded by an upper limit writing a custom streambuf is complicated compared to using strstream.

But if you must go that route then theres no point in writing a custom stringstream and then copy to an ofstream. The best solution would be to rather implement a custom ofstream that has caching and retrying capabilities. This way you'd be able to avoid the inevitable copying from temp stream to the filestream. Further optimization would be to use memory mapped files internally but fallback to in memory buffers should the files can not be mapped. Once the file has opened switch to direct mem-mapped IO.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Often, when implementing a feature, you won't know how certain events should be handled at the point where they occur and you'd rather defer to the user of your function or class. For example, a XML parser will extract a tag from the source code, wh…
What is C++ STL?: STL stands for Standard Template Library and is a part of standard C++ libraries. It contains many useful data structures (containers) and algorithms, which can spare you a lot of the time. Today we will look at the STL Vector. …
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
The viewer will learn how to pass data into a function in C++. This is one step further in using functions. Instead of only printing text onto the console, the function will be able to perform calculations with argumentents given by the user.

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question