Bypassing cache for network reads

NT seems to always pass disk data destined for the network through the cache, even if the file is opened unbuffered. But passing something like a 3 GByte CAT scan through the cache just flushes data that might have been hit and slows throughput. Does anyone know of a way to read remote data  without having it go through the server cache?

stevaAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

TSauerCommented:
This registry Key will help you:

HKEY_LOCAL_MACHINE
 SYSTEM
  CurrentControlSet
   Services
    Rdr
     Parameters

     Value:    UseWriteBehind
     Typ:      Reg_DWORD

    Valuet:     0

0
stevaAuthor Commented:
Turning off WriteBehind in the redirector would seem to just keep the server from caching writes from the network and slow throughput by requiring the data to get to the disk before the network request could be ACK'd. I'm happy with write throughput now because I _can_ put write data into the cache at the server without waiting for it to get onto the disk. My problem is reading from the server.  I don't want the server to run data it gets from the disk through the cache before putting it onto the network.

I notice that even though I set the FILE_FLAG_NO_BUFFERING flag when I open the file with CreateFile in the client, the SMB NT_CREATE_ANDX command going over the wire to open the file in the server does _not_ have the NO_BUFFERING flag set in ExtFileAtrributes, indicating to me that LanManager is failing to tell the server "don't cache this file." So how do you tell the server from Win32 in a client not to cache a file. Or is it that NT Server MUST cache disk data read for the network, for some reason,  so LanManager ignores the FILE_FLAG_NO_BUFFERING flag in the CreateFile call. But then why bother to put a NO_BUFFERING flag in ExtFileAtrributes if NT never sets it.  Perhaps the real question is, how do you get LanManager to set the NO_BUFFERING flag in the NT_CREATE_ANDX command?


0
snimmagaCommented:
When using FILE_FLAG_NO_BUFFERING, disk reads and writes must be done on sector boundaries, and buffer addresses must be aligned on disk sector boundaries in memory.
 
These restrictions are necessary because the buffer that you pass to the read or write API is used directly for I/O at the device level; at that level, your buffer addresses and sector sizes must satisfy any processor and media alignment estrictions of the hardware you are running on.
 
The Windows 95 CDFS (CD-ROM File System) does not support the
FILE_FLAG_NO_BUFFERING flag for CreateFile(). While a Windows 95 FSD, such as VFAT, may implement it, FILE_FLAG_NO_BUFFERING is not a required flag for file system drivers, and it is not supported by CDFS.
 
This code fragment demonstrates how to sector-align data in a buffer and pass it to CreateFile():
 
  char buf[2 * SECTOR_SIZE - 1], *p;
 
  p = (char *) ((DWORD) (buf + SECTOR_SIZE - 1) & ~(SECTOR_SIZE - 1));
  h = CreateFile(argv[1], GENERIC_READ | GENERIC_WRITE,
      FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, CREATE_ALWAYS,
      FILE_ATTRIBUTE_NORMAL | FILE_FLAG_NO_BUFFERING, NULL);
  WriteFile(h, p, SECTOR_SIZE, &dwWritten, NULL);
 
The pointer p is sector-aligned and points within the buffer.
 
When opening a remote file over the network, the server always caches and ignores the no buffering flag specified by the client. This is by design. The redirector and server cannot properly implement the full semantics of FILE_FLAG_NO_BUFFERING over the network. In particular, the requirement for sector-sized, sector-aligned I/O cannot be met. Therefore, when a Win32-
based application asks for FILE_FLAG_NO_BUFFERING, the redirector and server treat this as a request for  FILE_FLAG_WRITE_THROUGH. The file is not cached at the client, writes go directly to the server and to the disk on the server, and the read/write sizes on the network are exactly what the
application asks for. However, the file is cached on the server.
 
Not caching the client can have a different effect, depending on the type of I/O. You eliminate the cache hits or read ahead, but you also may reduce the size of transmits and receives. In general, for sequential I/O, it is a good idea to cache on the client. For small, random access I/O, it is often best not to cache.

Good luck..
Srini.
Ps: Further info in article Q.99794
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

snimmagaCommented:
Check out the last two paragraphs of my above answer to see why it doesnot work over a network.
Srini.
0
stevaAuthor Commented:
Srini,

I'm afraid you've answered the question correctly, though I was hoping for a different answer.  

The KB article says,

"The redirector and server cannot properly implement the full semantics of FILE_FLAG_NO_BUFFERING over the network. In particular, the requirement for sector-sized, sector-aligned I/O cannot be met."  

I think it's more a question of Microsoft not wanting to do it badly enough yet. Surely SRV could give the file system on the server a buffer that was sector-sized and sector-aligned.

The performance hit from passing disk data through the NT cache on the way to an application is HUGE! We have a disk subsystem that transfers 60 MBytes/s to an application that opens a local file on the disk unbuffered. But when the application opens the same file buffered it only gets the data at 17 MBytes/s because of all the overhead setting up cache pages and faulting the data in. As the networks and disks that connect to NT get faster, NT has to get faster by bypassing the cache for large files. At least that's my 2 cents worth.

Thanks for the answer.

Steve


0
snimmagaCommented:
I agree with you, totally...
Srini.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Networking

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.