Solved

HowTo: Flush o/s cache with win32 api call(s)?

Posted on 2006-06-21
9
1,482 Views
Last Modified: 2013-11-20
Sorry if this if OT because it deals with the WinApi rather than MFC specifically.

I'm doing some tests to speed up file handling using vc6 and WinXp-Sp2 on a development computer that has 1 gig of memory and an AMD-64-3800.

I'd like to clear/flush the o/s cache before doing the timings so as to get valid "1st time through" elapsed times. Otherwise, the 2nd and subsequent tests are much faster and don't tell me much.

Is there a Win32 api call to accomplish this? Or series of calls?

Now I'm doing the equivalent of xcopy of a large directory to another directory ... this seems to work, but is tedious and time-consuming.

0
Comment
Question by:newton-allan
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
9 Comments
 
LVL 22

Expert Comment

by:mahesh1402
ID: 16957395
You can flush a files buffers with FlushFileBuffers. To flush all open files on a volume, call FlushFileBuffers with a handle to the volume.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/flushfilebuffers.asp

But.. i'm not sure if this is what you need in your case.

-MAHESH
0
 
LVL 1

Author Comment

by:newton-allan
ID: 16958101
I don't think this is what I'm looking for. The documentation indicates this will flush an open file that has pending writes.

On an older computer, I'm trying to speed up the reading of a 4 meg file into a memory buffer. The first read might take 100 ms, and then the next read might take 10ms because of o/s caching. I would like to have an api-call before the read that would empty the o/s cache so I could see what actually causes the reading to speed up or slow down.
0
 
LVL 22

Expert Comment

by:mahesh1402
ID: 16958363
http://support.microsoft.com/kb/q110092/ <== As given here you can flush 32-bit file access cache (VCACHE)  by running MS-DOS Prompt and the pressing CTRL+C or CTRL+BREAK.

So what you need simulating Ctrl+C on DOS window .... So IF THERE IS NO API call for this you may do that by opening it programatically and simulating Ctrl+C on DOS window using CreateProcess or ShellExec and with following code to simulate Ctrl+C:

CWnd* myhwnd;
CString ClassName;
CString WindowText;
ClassName = "ConsoleWindowClass";
myhwnd =FindWindow(ClassName, WindowText);
     
myhwnd->SetForegroundWindow();
keybd_event(VK_CONTROL,0,0,0);
keybd_event (0x43, 0, 0, 0 ); //Send the C key (43 is "C")
keybd_event (0x43, 0, KEYEVENTF_KEYUP, 0);
keybd_event (VK_CONTROL, 0, KEYEVENTF_KEYUP, 0);// 'Left Control Up

-MAHESH

0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 86

Accepted Solution

by:
jkr earned 125 total points
ID: 16960442
Call 'CreateFile()' with 'FILE_FLAG_NO_BUFFERING' to achieve the desired effect.
0
 
LVL 1

Author Comment

by:newton-allan
ID: 16972183
Ooops ... I accepted the answer too quickly before doing further checking. I got the same timings, but then when I rebooted and re-ran the test.exe file to read the 4 meg file, the first run took much longer.

One thing I did find out ... having the FILE_FLAG_NO_BUFFERING makes the ReadFile much faster for doing a single all-at-once read.

Without the FILE_FLAG_NO_BUFFERING on a reasonably modern AMD-64 3700, 1 gb memory and 250gb drive, the first read was about 120 milliseconds, and the second run was about 8 ms.  With FILE_FLAG_NO_BUFFERING, the numbers were 42 ms and 1.5 ms. Big difference.

Here's the code:

#include <stdio.h>
#include <windows.h>
#include "HiResTimer.h"

#define VPL_SIZE      (5000000)
char    vplBuf[VPL_SIZE];

void main(void)
{
   int    numRead;
   HANDLE fileHandle;
   BOOL   bFlag;
   double elapsed;

   HrtInit();
   HrtSetPriority(ABOVE_NORMAL_PRIORITY_CLASS);
   HrtStart();

   fileHandle = CreateFile("BigFile.txt",
                           GENERIC_READ, 0,
                           0, OPEN_EXISTING,
                           FILE_ATTRIBUTE_NORMAL
                           | FILE_FLAG_NO_BUFFERING
                           | FILE_FLAG_SEQUENTIAL_SCAN
                           | FILE_ATTRIBUTE_READONLY,
                           0);

   if (fileHandle == INVALID_HANDLE_VALUE) {
      printf("Unable to open file\n");
      return;
   }
   bFlag = ReadFile(fileHandle, vplBuf, VPL_SIZE,
                   (DWORD *)&numRead, NULL);
   
   CloseHandle(fileHandle);
   elapsed = HrtElapsedMillis();
   printf("Elapsed: %.3f\n", elapsed);
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 16972681
Have you checked the docs on that also?

An application must meet certain requirements when working with files opened with FILE_FLAG_NO_BUFFERING:

File access must begin at byte offsets within the file that are integer multiples of the volume's sector size.

File access must be for numbers of bytes that are integer multiples of the volume's sector size. For example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1024, or 2048 bytes, but not of 335, 981, or 7171 bytes.

Buffer addresses for read and write operations must be sector aligned (aligned on addresses in memory that are integer multiples of the volume's sector size).

One way to align buffers on integer multiples of the volume sector size is to use VirtualAlloc to allocate the buffers. It allocates memory that is aligned on addresses that are integer multiples of the operating system's memory page size. Because both memory page and volume sector sizes are powers of 2, this memory is also aligned on addresses that are integer multiples of a volume's sector size.

0
 
LVL 1

Author Comment

by:newton-allan
ID: 16974301
I tried the VirtualAlloc call with appropriate alignment, and I'm seeing more consistent timings. Thanks ... and leaving the question as PAQ is fine.

Does it appear I setup and used the VirtualAlloc call correctly?

#define VIRTUAL_ALLOC_BUF_SZ (1024*1024)
void ReadOneBigFileWithUnbufferedCreateFile(void)
{
  int    numRead;
  HANDLE fileHandle;
  BOOL   bFlag;

  HiResTimerInit();
  HiResTimerSetPriority(HIGH_PRIORITY_CLASS);

  char*  pVirtualAlloc = VirtualAlloc(NULL,
                                      VIRTUAL_ALLOC_BUF_SZ,
                                      MEM_COMMIT,
                                      PAGE_READWRITE);

  fileHandle = CreateFile("..\\..\\..\\EsvOtNtNoTags.vpl",
                          GENERIC_READ, 0,
                          0, OPEN_EXISTING,
                          FILE_FLAG_NO_BUFFERING,
                          0);

  if (fileHandle == INVALID_HANDLE_VALUE) {
     printf("Unable to open file\n");
     return;
  }
  bFlag = ReadFile(fileHandle, pVirtualAlloc,
                  VIRTUAL_ALLOC_BUF_SZ,
                  (DWORD *)&numRead, NULL);
   
  CloseHandle(fileHandle);

  double elapsed = HiResTimerElapsedMillis();
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 16989166
Sorry, I almost missed your last comment. Yes, that looks good to me, at least right now I can't spot anything that would seem off.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
function returning CString in a regular win32 DLL 8 216
How to create frequencies of a variable from SAS dataset? 10 146
WinWaitActive parameters 12 31
sumHeights  challenge 17 83
Introduction: Hints for the grid button.  Nested classes, templated collections.  Squash that darned bug! Continuing from the sixth article about sudoku.   Open the project in visual studio. First we will finish with the SUD_SETVALUE messa‚Ķ
Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question