?
Solved

Fushing Data or disabling buffering on a named pipe (CreateProcess)

Posted on 2006-03-19
8
Medium Priority
?
1,503 Views
Last Modified: 2013-12-03
I have a CreateProcess which has the output redirected to a named pipe.  The other end of the named is being read by a 2 threads, one for stdout and one for stderr.  The data comes through okay, but mostly in chunks of 1024.  The problem is that I need to get the data right away and don't want to wait for 1024 bytes before it is sent.  

I have tried:
 - changing the size of the buffer set in CreateNamedPipe to small values.
 - Setting the flags in createFile for the write end of the named pipe to FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING before they get passed to the subprocess
 - Using overlapped file handles
 - Using Anonymous Pipes
 - Banging my head against the wall.

But I can't seem to get any data over the pipe before 1024 or before the app ends.

Is there anyway to flush the named pipe on the read end to get anything that has been written to the write end but is being held in a buffer... or disable the buffers?

Thanks!


0
Comment
Question by:peeldog
  • 3
  • 3
  • 2
8 Comments
 
LVL 8

Expert Comment

by:mxjijo
ID: 16233963
0
 
LVL 2

Author Comment

by:peeldog
ID: 16237278
My thinking was this question is more about how to flush a pipe.  If there is no way to flush a pipe and the only way to get the data before 1024 bytes (or however many the system determines) is to read the low level console buffer, then yes, I guess it would be another view of the same question.
0
 
LVL 86

Expert Comment

by:jkr
ID: 16237547
What about using 'FlushFileBuffers()' for the pipe handle?
0
Get your Conversational Ransomware Defense e‑book

This e-book gives you an insight into the ransomware threat and reviews the fundamentals of top-notch ransomware preparedness and recovery. To help you protect yourself and your organization. The initial infection may be inevitable, so the best protection is to be fully prepared.

 
LVL 2

Author Comment

by:peeldog
ID: 16237745
In the docs for that function it says:

"The file handle must have the GENERIC_WRITE access right"

The reader threads have the read end of the pipe and I pass the write end to the console.  I tried keeping a copy of the write end that I pass and flushing that, as well flushing the end that I am reading from, but didn't seem to have any effect.

I also tried changing which end of the named pipe is obtained by Create file, but no effect.  This is what I have right now:

hStdoutRead = CreateNamedPipe("\\\\.\\PIPE\\alcatraz_StdOutRead",
                   PIPE_ACCESS_INBOUND, PIPE_TYPE_BYTE | PIPE_WAIT,
                   PIPE_UNLIMITED_INSTANCES,  128,  128, 15000, &saAttr);

 hStdoutWrite = CreateFile("\\\\.\\PIPE\\alcatraz_StdOutRead", GENERIC_WRITE,  
                   FILE_SHARE_READ | FILE_SHARE_WRITE, &saAttr, CREATE_ALWAYS,  
                   FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING , 0);

CreateNamedPipe("\\\\.\\PIPE\\alcatraz_StdOutRead",
                   PIPE_ACCESS_INBOUND, PIPE_TYPE_BYTE | PIPE_WAIT,
                   PIPE_UNLIMITED_INSTANCES,  128,  128, 15000, &saAttr);

hStdoutWrite = CreateFile("\\\\.\\PIPE\\alcatraz_StdOutRead", GENERIC_WRITE,  
                   FILE_SHARE_READ | FILE_SHARE_WRITE, &saAttr, CREATE_ALWAYS,  
                   FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING , 0);

Should FlushFileBuffers work?  If so on what handle? (in the parent app - I can't modify the child process code).

0
 
LVL 86

Expert Comment

by:jkr
ID: 16238086
'FlushFileBuffers()' only makes sense on the writing side of the pipe.
0
 
LVL 8

Expert Comment

by:mxjijo
ID: 16238221

Are we sure that the data is getting buffered in pipe itself ? It could be the console re-direct buffering.
Since you're creating the pipe with NO_BUFFERING flag, AFAIK it should not do buffering.
May be its a good idea to write your own pipe server and client and see if you can re-produce the problem.
Taking your "other" question to consideration, I feel like you are dealing with a console application which
has its own console buffer handling. Before diving further into the named pipe buffering, its worth thinking in that way.

good luck.

0
 
LVL 2

Author Comment

by:peeldog
ID: 16250055
Yup, I think you are right.  Some investigation leads me to believe that it's the stdlib buffers not clearing to the pipe.  Is there any way I can clear or read those buffers without modifying the application?  Is there also a "console re-direct" buffer?  Do you have any references?

Thanks for your help!
0
 
LVL 8

Accepted Solution

by:
mxjijo earned 2000 total points
ID: 16250332

hmm.. interesting.. I haven't tried this but I guess you migth have to take a different route.

Since the child application has his own console buffer handling, you cannot depend on standard ios.
One thing you can try is to attach to the child's console by calling AttachConsole().
Once you attach to the other console, you can read the console output the way discussed in
http://homepages.tesco.net/J.deBoynePollard/FGA/capture-console-win32.html

You might have to face some windows version issues here.
But the idea is to takeover/attach to the child process's console.

hope that helps.
0

Featured Post

NFR key for Veeam Agent for Linux

Veeam is happy to provide a free NFR license for one year.  It allows for the non‑production use and valid for five workstations and two servers. Veeam Agent for Linux is a simple backup tool for your Linux installations, both on‑premises and in the public cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows how to make a Windows 7 gadget that accepts files dropped from the Windows Explorer.  It also illustrates how to give your gadget a non-rectangular shape and how to add some nifty visual effects to text displayed in a your gadget.…
Entering time in Microsoft Access can be difficult. An input mask often bothers users more than helping them and won't catch all typing errors. This article shows how to create a textbox for 24-hour time input with full validation politely catching …
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA. http://www.experts-exchange.com/videos/1478/Excel-Error-Handlin…
Are you ready to place your question in front of subject-matter experts for more timely responses? With the release of Priority Question, Premium Members, Team Accounts and Qualified Experts can now identify the emergent level of their issue, signal…

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question