Solved

Bypassing cache for network reads

Posted on 1998-04-27
6
540 Views
Last Modified: 2013-12-19
NT seems to always pass disk data destined for the network through the cache, even if the file is opened unbuffered. But passing something like a 3 GByte CAT scan through the cache just flushes data that might have been hit and slows throughput. Does anyone know of a way to read remote data  without having it go through the server cache?

0
Comment
Question by:steva
  • 3
  • 2
6 Comments
 
LVL 3

Expert Comment

by:TSauer
ID: 1559991
This registry Key will help you:

HKEY_LOCAL_MACHINE
 SYSTEM
  CurrentControlSet
   Services
    Rdr
     Parameters

     Value:    UseWriteBehind
     Typ:      Reg_DWORD

    Valuet:     0

0
 

Author Comment

by:steva
ID: 1559992
Turning off WriteBehind in the redirector would seem to just keep the server from caching writes from the network and slow throughput by requiring the data to get to the disk before the network request could be ACK'd. I'm happy with write throughput now because I _can_ put write data into the cache at the server without waiting for it to get onto the disk. My problem is reading from the server.  I don't want the server to run data it gets from the disk through the cache before putting it onto the network.

I notice that even though I set the FILE_FLAG_NO_BUFFERING flag when I open the file with CreateFile in the client, the SMB NT_CREATE_ANDX command going over the wire to open the file in the server does _not_ have the NO_BUFFERING flag set in ExtFileAtrributes, indicating to me that LanManager is failing to tell the server "don't cache this file." So how do you tell the server from Win32 in a client not to cache a file. Or is it that NT Server MUST cache disk data read for the network, for some reason,  so LanManager ignores the FILE_FLAG_NO_BUFFERING flag in the CreateFile call. But then why bother to put a NO_BUFFERING flag in ExtFileAtrributes if NT never sets it.  Perhaps the real question is, how do you get LanManager to set the NO_BUFFERING flag in the NT_CREATE_ANDX command?


0
 
LVL 5

Accepted Solution

by:
snimmaga earned 250 total points
ID: 1559993
When using FILE_FLAG_NO_BUFFERING, disk reads and writes must be done on sector boundaries, and buffer addresses must be aligned on disk sector boundaries in memory.
 
These restrictions are necessary because the buffer that you pass to the read or write API is used directly for I/O at the device level; at that level, your buffer addresses and sector sizes must satisfy any processor and media alignment estrictions of the hardware you are running on.
 
The Windows 95 CDFS (CD-ROM File System) does not support the
FILE_FLAG_NO_BUFFERING flag for CreateFile(). While a Windows 95 FSD, such as VFAT, may implement it, FILE_FLAG_NO_BUFFERING is not a required flag for file system drivers, and it is not supported by CDFS.
 
This code fragment demonstrates how to sector-align data in a buffer and pass it to CreateFile():
 
  char buf[2 * SECTOR_SIZE - 1], *p;
 
  p = (char *) ((DWORD) (buf + SECTOR_SIZE - 1) & ~(SECTOR_SIZE - 1));
  h = CreateFile(argv[1], GENERIC_READ | GENERIC_WRITE,
      FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, CREATE_ALWAYS,
      FILE_ATTRIBUTE_NORMAL | FILE_FLAG_NO_BUFFERING, NULL);
  WriteFile(h, p, SECTOR_SIZE, &dwWritten, NULL);
 
The pointer p is sector-aligned and points within the buffer.
 
When opening a remote file over the network, the server always caches and ignores the no buffering flag specified by the client. This is by design. The redirector and server cannot properly implement the full semantics of FILE_FLAG_NO_BUFFERING over the network. In particular, the requirement for sector-sized, sector-aligned I/O cannot be met. Therefore, when a Win32-
based application asks for FILE_FLAG_NO_BUFFERING, the redirector and server treat this as a request for  FILE_FLAG_WRITE_THROUGH. The file is not cached at the client, writes go directly to the server and to the disk on the server, and the read/write sizes on the network are exactly what the
application asks for. However, the file is cached on the server.
 
Not caching the client can have a different effect, depending on the type of I/O. You eliminate the cache hits or read ahead, but you also may reduce the size of transmits and receives. In general, for sequential I/O, it is a good idea to cache on the client. For small, random access I/O, it is often best not to cache.

Good luck..
Srini.
Ps: Further info in article Q.99794
0
Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

 
LVL 5

Expert Comment

by:snimmaga
ID: 1559994
Check out the last two paragraphs of my above answer to see why it doesnot work over a network.
Srini.
0
 

Author Comment

by:steva
ID: 1559995
Srini,

I'm afraid you've answered the question correctly, though I was hoping for a different answer.  

The KB article says,

"The redirector and server cannot properly implement the full semantics of FILE_FLAG_NO_BUFFERING over the network. In particular, the requirement for sector-sized, sector-aligned I/O cannot be met."  

I think it's more a question of Microsoft not wanting to do it badly enough yet. Surely SRV could give the file system on the server a buffer that was sector-sized and sector-aligned.

The performance hit from passing disk data through the NT cache on the way to an application is HUGE! We have a disk subsystem that transfers 60 MBytes/s to an application that opens a local file on the disk unbuffered. But when the application opens the same file buffered it only gets the data at 17 MBytes/s because of all the overhead setting up cache pages and faulting the data in. As the networks and disks that connect to NT get faster, NT has to get faster by bypassing the cache for large files. At least that's my 2 cents worth.

Thanks for the answer.

Steve


0
 
LVL 5

Expert Comment

by:snimmaga
ID: 1559996
I agree with you, totally...
Srini.
0

Featured Post

New My Cloud Pro Series - organize everything!

With space to keep virtually everything, the My Cloud Pro Series offers your team the network storage to edit, save and share production files from anywhere with an internet connection. Compatible with both Mac and PC, you're able to protect your content regardless of OS.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Can't access PCs by IP address. 20 61
Lost hyper V virtual network 1 43
FTP output from Wireshak 6 74
SQL Server Reporting Services Service Start Timeout 4 48
A brief overview to explain gateways, default gateways and static routes OR NO - you CANNOT have two default gateways on the same server, PC or other Windows-based network device. In simple terms a gateway is formed when a computer such as a serv…
Have you ever set up your wireless router at home or in the office to find that you little pop-up bubble in the bottom right-hand corner of Windows read "IP Conflict - One of more computers on the network have been assigned the following IP address"…
This is used to tweak the memory usage for your computer, it is used for servers more so than workstations but just be careful editing registry settings as it may cause irreversible results. I hold no responsibility for anything you do to the regist…
This is an introductory video for CloudBerry Managed Backup. You will learn how to sign up with the service and get started in a few minutes.

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now