We help IT Professionals succeed at work.

We've partnered with Certified Experts, Carl Webster and Richard Faulkner, to bring you a podcast all about Citrix Workspace, moving to the cloud, and analytics & intelligence. Episode 2 coming soon!Listen Now

x

Fushing Data or disabling buffering on a named pipe (CreateProcess)

peeldog
peeldog asked
on
Medium Priority
1,990 Views
Last Modified: 2013-12-03
I have a CreateProcess which has the output redirected to a named pipe.  The other end of the named is being read by a 2 threads, one for stdout and one for stderr.  The data comes through okay, but mostly in chunks of 1024.  The problem is that I need to get the data right away and don't want to wait for 1024 bytes before it is sent.  

I have tried:
 - changing the size of the buffer set in CreateNamedPipe to small values.
 - Setting the flags in createFile for the write end of the named pipe to FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING before they get passed to the subprocess
 - Using overlapped file handles
 - Using Anonymous Pipes
 - Banging my head against the wall.

But I can't seem to get any data over the pipe before 1024 or before the app ends.

Is there anyway to flush the named pipe on the read end to get anything that has been written to the write end but is being held in a buffer... or disable the buffers?

Thanks!


Comment
Watch Question

Author

Commented:
My thinking was this question is more about how to flush a pipe.  If there is no way to flush a pipe and the only way to get the data before 1024 bytes (or however many the system determines) is to read the low level console buffer, then yes, I guess it would be another view of the same question.
jkr
CERTIFIED EXPERT
Top Expert 2012

Commented:
What about using 'FlushFileBuffers()' for the pipe handle?

Author

Commented:
In the docs for that function it says:

"The file handle must have the GENERIC_WRITE access right"

The reader threads have the read end of the pipe and I pass the write end to the console.  I tried keeping a copy of the write end that I pass and flushing that, as well flushing the end that I am reading from, but didn't seem to have any effect.

I also tried changing which end of the named pipe is obtained by Create file, but no effect.  This is what I have right now:

hStdoutRead = CreateNamedPipe("\\\\.\\PIPE\\alcatraz_StdOutRead",
                   PIPE_ACCESS_INBOUND, PIPE_TYPE_BYTE | PIPE_WAIT,
                   PIPE_UNLIMITED_INSTANCES,  128,  128, 15000, &saAttr);

 hStdoutWrite = CreateFile("\\\\.\\PIPE\\alcatraz_StdOutRead", GENERIC_WRITE,  
                   FILE_SHARE_READ | FILE_SHARE_WRITE, &saAttr, CREATE_ALWAYS,  
                   FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING , 0);

CreateNamedPipe("\\\\.\\PIPE\\alcatraz_StdOutRead",
                   PIPE_ACCESS_INBOUND, PIPE_TYPE_BYTE | PIPE_WAIT,
                   PIPE_UNLIMITED_INSTANCES,  128,  128, 15000, &saAttr);

hStdoutWrite = CreateFile("\\\\.\\PIPE\\alcatraz_StdOutRead", GENERIC_WRITE,  
                   FILE_SHARE_READ | FILE_SHARE_WRITE, &saAttr, CREATE_ALWAYS,  
                   FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING , 0);

Should FlushFileBuffers work?  If so on what handle? (in the parent app - I can't modify the child process code).

jkr
CERTIFIED EXPERT
Top Expert 2012

Commented:
'FlushFileBuffers()' only makes sense on the writing side of the pipe.

Commented:

Are we sure that the data is getting buffered in pipe itself ? It could be the console re-direct buffering.
Since you're creating the pipe with NO_BUFFERING flag, AFAIK it should not do buffering.
May be its a good idea to write your own pipe server and client and see if you can re-produce the problem.
Taking your "other" question to consideration, I feel like you are dealing with a console application which
has its own console buffer handling. Before diving further into the named pipe buffering, its worth thinking in that way.

good luck.

Author

Commented:
Yup, I think you are right.  Some investigation leads me to believe that it's the stdlib buffers not clearing to the pipe.  Is there any way I can clear or read those buffers without modifying the application?  Is there also a "console re-direct" buffer?  Do you have any references?

Thanks for your help!
Commented:

hmm.. interesting.. I haven't tried this but I guess you migth have to take a different route.

Since the child application has his own console buffer handling, you cannot depend on standard ios.
One thing you can try is to attach to the child's console by calling AttachConsole().
Once you attach to the other console, you can read the console output the way discussed in
http://homepages.tesco.net/J.deBoynePollard/FGA/capture-console-win32.html

You might have to face some windows version issues here.
But the idea is to takeover/attach to the child process's console.

hope that helps.

Not the solution you were looking for? Getting a personalized solution is easy.

Ask the Experts
Access more of Experts Exchange with a free account
Thanks for using Experts Exchange.

Create a free account to continue.

Limited access with a free account allows you to:

  • View three pieces of content (articles, solutions, posts, and videos)
  • Ask the experts questions (counted toward content limit)
  • Customize your dashboard and profile

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.