Solved

dump one stream into another without multiple copies

Posted on 2013-01-08
8
335 Views
Last Modified: 2013-01-15
I'm working on a little utility application which needs to have a place to send error messages even if a console or file isn't available yet.

The idea is to cache messages into a stringstream until a file can be opened (fstream), then dump the contents of the cache stream into the "true" message stream.  Even if it has to wait until the destructor is called.

Stripped down to its essentials:

class ostream_sink 
{
 private:
 bool m_caching;
 std::string m_filename;
 std::ostringstream *m_cache;
 std::ofstream *m_dest;
 // imagine constructor and destructor details, and setting the file name
 bool open_file(); // implementation omitted
 public:
 void start_caching() { m_caching = true; }
 void stop_caching(); // question is about implmenting this routine

 std::ostream &stream();
 std::ostream &stream() const;
 template <class T> ostream_sink &operator<<(T const &v) { stream() << v; }
 template <class T> ostream_sink &operator<<(T const &v) const { stream() << v; }
};

 std::ostream &stream() const 
{
 if (m_caching) 
    return *m_cache;
   else return *m_dest; 
}

std::ostream &ostream_sink::stream ()
{ if (m_caching) if (open_file()) stop_caching(); 
   if (m_caching) 
      return *m_cache; 
      else return *m_dest; 
}

Open in new window


I suppose I could always use the stringstream's str() mehod:

ostream_sink::stop_caching ()
{
 if (m_caching) 
  {
   (*m_dest) << m_cache->str();
   m_cache->str(string()); 
   m_caching = false;
 }
}

Open in new window


but that string gets copied twice.   Is there a way do dump the cache's streambuf into the file stream's?  You certainly can't swap a stringbuf with a filebuf.....
0
Comment
Question by:Spencer Simpson
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 2
  • 2
8 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 38755929
The snippets above actually do look OK. Could you provide some more code? Maybe the problem is located elsewhere...
0
 
LVL 1

Author Comment

by:Spencer Simpson
ID: 38756002
It's not a "somethign doesn't work" problem, it's a question of efficiency.    The snippets provide a simplified example of the concept.

m_cache->str() returns a copy of the string, which might have a large amount of text in it. Putting it into the m_dest copies the text again.  So not only is a large amount of memory allocated and deallocated for the temporary string, there are potentially two fairly large copy operations involved.
0
 
LVL 86

Expert Comment

by:jkr
ID: 38756237
Ah, OK, now I see what you mean. In that case, you could use a 'basic_filebuf' directly and use the stringstream's buffer as the source using 'pubsetbuf()', see e.g. the example at http://msdn.microsoft.com/en-us/library/vstudio/tzf8k3z8%28v=vs.100%29.aspx ("basic_filebuf Class" - scroll down). However, I haven't tried that myself.
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 86

Expert Comment

by:jkr
ID: 38756244
Oh, maybe also of interst in this context: http://stackoverflow.com/questions/2079912/simpler-way-to-create-a-c-memorystream-from-char-size-t-without-copying-t ("Simpler way to create a C++ memorystream from (char*, size_t), without copying the data?")
0
 
LVL 1

Author Comment

by:Spencer Simpson
ID: 38759761
After some examination, it appears that basic_streambuf has protected member functions that would let you do precisely this if they weren't protected.

The solution appears to be abandoning this approach and taking the one the STL gods would tell you to take, if you asked you about anything past the basic funcitonality of these streams: roll your own streambuf class.
0
 
LVL 22

Accepted Solution

by:
ambience earned 450 total points
ID: 38760840
Another solution is to use strstream rather than stringstream even though in general its not a good practice but since you are encapsulating everything it may be worth it. The benefit is that it wont copy buffers around and you can even initialize an strstream around a static buffer. The str() of strstream would give you a pointer to its internal buffer and the pcount() would give the size of data.

dest->write(...)

Even if you write your own streambuf implementation the best you can do is build some sort of buffer pooling, otherwise you'd still have to write to ofstream and then that becomes essentially the same as using an strstream instead.
0
 
LVL 86

Assisted Solution

by:jkr
jkr earned 50 total points
ID: 38761031
>>After some examination, it appears that basic_streambuf has protected member functions
>>that would let you do precisely this if they weren't protected.

Derive from it and add a public member that provides access to the protected member ;o)
0
 
LVL 22

Assisted Solution

by:ambience
ambience earned 450 total points
ID: 38762275
Unless the size of error messages is bounded by an upper limit writing a custom streambuf is complicated compared to using strstream.

But if you must go that route then theres no point in writing a custom stringstream and then copy to an ofstream. The best solution would be to rather implement a custom ofstream that has caching and retrying capabilities. This way you'd be able to avoid the inevitable copying from temp stream to the filestream. Further optimization would be to use memory mapped files internally but fallback to in memory buffers should the files can not be mapped. Once the file has opened switch to direct mem-mapped IO.
0

Featured Post

[Webinar] Code, Load, and Grow

Managing multiple websites, servers, applications, and security on a daily basis? Join us for a webinar on May 25th to learn how to simplify administration and management of virtual hosts for IT admins, create a secure environment, and deploy code more effectively and frequently.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Article by: SunnyDark
This article's goal is to present you with an easy to use XML wrapper for C++ and also present some interesting techniques that you might use with MS C++. The reason I built this class is to ease the pain of using XML files with C++, since there is…
Container Orchestration platforms empower organizations to scale their apps at an exceptional rate. This is the reason numerous innovation-driven companies are moving apps to an appropriated datacenter wide platform that empowers them to scale at a …
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question