steva
asked on
Bypassing cache for network reads
NT seems to always pass disk data destined for the network through the cache, even if the file is opened unbuffered. But passing something like a 3 GByte CAT scan through the cache just flushes data that might have been hit and slows throughput. Does anyone know of a way to read remote data without having it go through the server cache?
ASKER
Turning off WriteBehind in the redirector would seem to just keep the server from caching writes from the network and slow throughput by requiring the data to get to the disk before the network request could be ACK'd. I'm happy with write throughput now because I _can_ put write data into the cache at the server without waiting for it to get onto the disk. My problem is reading from the server. I don't want the server to run data it gets from the disk through the cache before putting it onto the network.
I notice that even though I set the FILE_FLAG_NO_BUFFERING flag when I open the file with CreateFile in the client, the SMB NT_CREATE_ANDX command going over the wire to open the file in the server does _not_ have the NO_BUFFERING flag set in ExtFileAtrributes, indicating to me that LanManager is failing to tell the server "don't cache this file." So how do you tell the server from Win32 in a client not to cache a file. Or is it that NT Server MUST cache disk data read for the network, for some reason, so LanManager ignores the FILE_FLAG_NO_BUFFERING flag in the CreateFile call. But then why bother to put a NO_BUFFERING flag in ExtFileAtrributes if NT never sets it. Perhaps the real question is, how do you get LanManager to set the NO_BUFFERING flag in the NT_CREATE_ANDX command?
I notice that even though I set the FILE_FLAG_NO_BUFFERING flag when I open the file with CreateFile in the client, the SMB NT_CREATE_ANDX command going over the wire to open the file in the server does _not_ have the NO_BUFFERING flag set in ExtFileAtrributes, indicating to me that LanManager is failing to tell the server "don't cache this file." So how do you tell the server from Win32 in a client not to cache a file. Or is it that NT Server MUST cache disk data read for the network, for some reason, so LanManager ignores the FILE_FLAG_NO_BUFFERING flag in the CreateFile call. But then why bother to put a NO_BUFFERING flag in ExtFileAtrributes if NT never sets it. Perhaps the real question is, how do you get LanManager to set the NO_BUFFERING flag in the NT_CREATE_ANDX command?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Check out the last two paragraphs of my above answer to see why it doesnot work over a network.
Srini.
Srini.
ASKER
Srini,
I'm afraid you've answered the question correctly, though I was hoping for a different answer.
The KB article says,
"The redirector and server cannot properly implement the full semantics of FILE_FLAG_NO_BUFFERING over the network. In particular, the requirement for sector-sized, sector-aligned I/O cannot be met."
I think it's more a question of Microsoft not wanting to do it badly enough yet. Surely SRV could give the file system on the server a buffer that was sector-sized and sector-aligned.
The performance hit from passing disk data through the NT cache on the way to an application is HUGE! We have a disk subsystem that transfers 60 MBytes/s to an application that opens a local file on the disk unbuffered. But when the application opens the same file buffered it only gets the data at 17 MBytes/s because of all the overhead setting up cache pages and faulting the data in. As the networks and disks that connect to NT get faster, NT has to get faster by bypassing the cache for large files. At least that's my 2 cents worth.
Thanks for the answer.
Steve
I'm afraid you've answered the question correctly, though I was hoping for a different answer.
The KB article says,
"The redirector and server cannot properly implement the full semantics of FILE_FLAG_NO_BUFFERING over the network. In particular, the requirement for sector-sized, sector-aligned I/O cannot be met."
I think it's more a question of Microsoft not wanting to do it badly enough yet. Surely SRV could give the file system on the server a buffer that was sector-sized and sector-aligned.
The performance hit from passing disk data through the NT cache on the way to an application is HUGE! We have a disk subsystem that transfers 60 MBytes/s to an application that opens a local file on the disk unbuffered. But when the application opens the same file buffered it only gets the data at 17 MBytes/s because of all the overhead setting up cache pages and faulting the data in. As the networks and disks that connect to NT get faster, NT has to get faster by bypassing the cache for large files. At least that's my 2 cents worth.
Thanks for the answer.
Steve
I agree with you, totally...
Srini.
Srini.
HKEY_LOCAL_MACHINE
SYSTEM
CurrentControlSet
Services
Rdr
Parameters
Value: UseWriteBehind
Typ: Reg_DWORD
Valuet: 0