asked on

Fushing Data or disabling buffering on a named pipe (CreateProcess)

I have a CreateProcess which has the output redirected to a named pipe. The other end of the named is being read by a 2 threads, one for stdout and one for stderr. The data comes through okay, but mostly in chunks of 1024. The problem is that I need to get the data right away and don't want to wait for 1024 bytes before it is sent.

I have tried:
- changing the size of the buffer set in CreateNamedPipe to small values.
- Setting the flags in createFile for the write end of the named pipe to FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING before they get passed to the subprocess
- Using overlapped file handles
- Using Anonymous Pipes
- Banging my head against the wall.

But I can't seem to get any data over the pipe before 1024 or before the app ends.

Is there anyway to flush the named pipe on the read end to get anything that has been written to the write end but is being held in a buffer... or disable the buffers?

Thanks!

mxjijo

I guess https://www.experts-exchange.com/questions/21779650/Capture-standard-output-from-CreateProcess-problem-low-level-write.html
is another view of the same qn ?

peeldog

ASKER

My thinking was this question is more about how to flush a pipe. If there is no way to flush a pipe and the only way to get the data before 1024 bytes (or however many the system determines) is to read the low level console buffer, then yes, I guess it would be another view of the same question.

jkr

What about using 'FlushFileBuffers()' for the pipe handle?

peeldog

ASKER

In the docs for that function it says:

"The file handle must have the GENERIC_WRITE access right"

The reader threads have the read end of the pipe and I pass the write end to the console. I tried keeping a copy of the write end that I pass and flushing that, as well flushing the end that I am reading from, but didn't seem to have any effect.

I also tried changing which end of the named pipe is obtained by Create file, but no effect. This is what I have right now:

hStdoutRead = CreateNamedPipe("\\\\.\\PIPE\\alcatraz_StdOutRead",
                   PIPE_ACCESS_INBOUND, PIPE_TYPE_BYTE | PIPE_WAIT,
                   PIPE_UNLIMITED_INSTANCES, 128, 128, 15000, &saAttr);

hStdoutWrite = CreateFile("\\\\.\\PIPE\\alcatraz_StdOutRead", GENERIC_WRITE,
                   FILE_SHARE_READ | FILE_SHARE_WRITE, &saAttr, CREATE_ALWAYS,
                   FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING , 0);

CreateNamedPipe("\\\\.\\PIPE\\alcatraz_StdOutRead",
                   PIPE_ACCESS_INBOUND, PIPE_TYPE_BYTE | PIPE_WAIT,
                   PIPE_UNLIMITED_INSTANCES, 128, 128, 15000, &saAttr);

hStdoutWrite = CreateFile("\\\\.\\PIPE\\alcatraz_StdOutRead", GENERIC_WRITE,
                   FILE_SHARE_READ | FILE_SHARE_WRITE, &saAttr, CREATE_ALWAYS,
                   FILE_FLAG_WRITE_THROUGH |FILE_FLAG_NO_BUFFERING , 0);

Should FlushFileBuffers work? If so on what handle? (in the parent app - I can't modify the child process code).

jkr

'FlushFileBuffers()' only makes sense on the writing side of the pipe.

mxjijo

Are we sure that the data is getting buffered in pipe itself ? It could be the console re-direct buffering.
Since you're creating the pipe with NO_BUFFERING flag, AFAIK it should not do buffering.
May be its a good idea to write your own pipe server and client and see if you can re-produce the problem.
Taking your "other" question to consideration, I feel like you are dealing with a console application which
has its own console buffer handling. Before diving further into the named pipe buffering, its worth thinking in that way.

good luck.

peeldog

ASKER

Yup, I think you are right. Some investigation leads me to believe that it's the stdlib buffers not clearing to the pipe. Is there any way I can clear or read those buffers without modifying the application? Is there also a "console re-direct" buffer? Do you have any references?

Thanks for your help!

ASKER CERTIFIED SOLUTION

mxjijo

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial