Solved

writing to multiple filehandles - buffer

Posted on 2011-02-22
3
414 Views
Last Modified: 2012-05-11
I have filehandles in an array that access named pipes. I need to write to them in a round robin fashion, and not block because the reader of its named pipe is being slow (with in reason).

I believe I need large buffers on each filehandle. Is this correct? How do I do it?
0
Comment
Question by:modsiw
  • 2
3 Comments
 
LVL 13

Expert Comment

by:Carl Bohman
ID: 34963157
I'm not aware of any way to buffer the output in the way you are asking.  The operating system is likely where you need to look, since it is the operating system is in control of the buffering for named pipes (to the best of my knowledge).  One page I found on a quick search hinted that the buffer may be linked to the page size, but this may or may not be true for a given operating system.
0
 
LVL 3

Author Comment

by:modsiw
ID: 34964383
I want to buffer inside the perl application before it writes out to the OS. It already does this, but I need to increase that buffer.
0
 
LVL 13

Accepted Solution

by:
Carl Bohman earned 500 total points
ID: 34964671
There are the setvbuf methods in Perl's FileHandle and IO::Handle modules.  Perhaps you could use this to do what you are looking for?  I'm not certain that Perl has a way of changing the output buffer size.

Otherwise, you might want to look into something like creating separate threads that do the buffering for you.  One way to do this would be to have a shared array (see threads::shared module) that the originating thread pushes to.  The writer thread then waits for the array to be non-empty and shifts the values off of it to write to the named pipe.  This way, only the thread gets blocked if the pipe is full while the shared array is your buffer (which is bounded only by available memory by default).

Of course this is only one approach (which I've never tried myself, but it should work).  One nice thing about this approach is that you can change your round-robin technique into one that looks for the buffer/queue that is the least full before writing (which could increase overall throughput if any of the threads start getting backed up).
0

Featured Post

Is Your AD Toolbox Looking More Like a Toybox?

Managing Active Directory can get complicated.  Often, the native tools for managing AD are just not up to the task.  The largest Active Directory installations in the world have relied on one tool to manage their day-to-day administration tasks: Hyena. Start your trial today.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

There are many situations when we need to display the data in sorted order. For example: Student details by name or by rank or by total marks etc. If you are working on data driven based projects then you will use sorting techniques very frequently.…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This tutorial gives a high-level tour of the interface of Marketo (a marketing automation tool to help businesses track and engage prospective customers and drive them to purchase). You will see the main areas including Marketing Activities, Design …

778 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question