Solved

writing to multiple filehandles - buffer

Posted on 2011-02-22
3
395 Views
Last Modified: 2012-05-11
I have filehandles in an array that access named pipes. I need to write to them in a round robin fashion, and not block because the reader of its named pipe is being slow (with in reason).

I believe I need large buffers on each filehandle. Is this correct? How do I do it?
0
Comment
Question by:modsiw
  • 2
3 Comments
 
LVL 13

Expert Comment

by:Carl Bohman
Comment Utility
I'm not aware of any way to buffer the output in the way you are asking.  The operating system is likely where you need to look, since it is the operating system is in control of the buffering for named pipes (to the best of my knowledge).  One page I found on a quick search hinted that the buffer may be linked to the page size, but this may or may not be true for a given operating system.
0
 
LVL 3

Author Comment

by:modsiw
Comment Utility
I want to buffer inside the perl application before it writes out to the OS. It already does this, but I need to increase that buffer.
0
 
LVL 13

Accepted Solution

by:
Carl Bohman earned 500 total points
Comment Utility
There are the setvbuf methods in Perl's FileHandle and IO::Handle modules.  Perhaps you could use this to do what you are looking for?  I'm not certain that Perl has a way of changing the output buffer size.

Otherwise, you might want to look into something like creating separate threads that do the buffering for you.  One way to do this would be to have a shared array (see threads::shared module) that the originating thread pushes to.  The writer thread then waits for the array to be non-empty and shifts the values off of it to write to the named pipe.  This way, only the thread gets blocked if the pipe is full while the shared array is your buffer (which is bounded only by available memory by default).

Of course this is only one approach (which I've never tried myself, but it should work).  One nice thing about this approach is that you can change your round-robin technique into one that looks for the buffer/queue that is the least full before writing (which could increase overall throughput if any of the threads start getting backed up).
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Email validation in proper way is  very important validation required in any web pages. This code is self explainable except that Regular Expression which I used for pattern matching. I originally published as a thread on my website : http://www…
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now