?
Solved

writing to multiple filehandles - buffer

Posted on 2011-02-22
3
Medium Priority
?
453 Views
Last Modified: 2012-05-11
I have filehandles in an array that access named pipes. I need to write to them in a round robin fashion, and not block because the reader of its named pipe is being slow (with in reason).

I believe I need large buffers on each filehandle. Is this correct? How do I do it?
0
Comment
Question by:modsiw
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 13

Expert Comment

by:Carl Bohman
ID: 34963157
I'm not aware of any way to buffer the output in the way you are asking.  The operating system is likely where you need to look, since it is the operating system is in control of the buffering for named pipes (to the best of my knowledge).  One page I found on a quick search hinted that the buffer may be linked to the page size, but this may or may not be true for a given operating system.
0
 
LVL 3

Author Comment

by:modsiw
ID: 34964383
I want to buffer inside the perl application before it writes out to the OS. It already does this, but I need to increase that buffer.
0
 
LVL 13

Accepted Solution

by:
Carl Bohman earned 2000 total points
ID: 34964671
There are the setvbuf methods in Perl's FileHandle and IO::Handle modules.  Perhaps you could use this to do what you are looking for?  I'm not certain that Perl has a way of changing the output buffer size.

Otherwise, you might want to look into something like creating separate threads that do the buffering for you.  One way to do this would be to have a shared array (see threads::shared module) that the originating thread pushes to.  The writer thread then waits for the array to be non-empty and shifts the values off of it to write to the named pipe.  This way, only the thread gets blocked if the pipe is full while the shared array is your buffer (which is bounded only by available memory by default).

Of course this is only one approach (which I've never tried myself, but it should work).  One nice thing about this approach is that you can change your round-robin technique into one that looks for the buffer/queue that is the least full before writing (which could increase overall throughput if any of the threads start getting backed up).
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Many time we need to work with multiple files all together. If its windows system then we can use some GUI based editor to accomplish our task. But what if you are on putty or have only CLI(Command Line Interface) as an option to  edit your files. I…
A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question