Solved

HowTo: Flush o/s cache with win32 api call(s)?

Posted on 2006-06-21
9
1,474 Views
Last Modified: 2013-11-20
Sorry if this if OT because it deals with the WinApi rather than MFC specifically.

I'm doing some tests to speed up file handling using vc6 and WinXp-Sp2 on a development computer that has 1 gig of memory and an AMD-64-3800.

I'd like to clear/flush the o/s cache before doing the timings so as to get valid "1st time through" elapsed times. Otherwise, the 2nd and subsequent tests are much faster and don't tell me much.

Is there a Win32 api call to accomplish this? Or series of calls?

Now I'm doing the equivalent of xcopy of a large directory to another directory ... this seems to work, but is tedious and time-consuming.

0
Comment
Question by:newton-allan
  • 3
  • 3
  • 2
9 Comments
 
LVL 22

Expert Comment

by:mahesh1402
ID: 16957395
You can flush a files buffers with FlushFileBuffers. To flush all open files on a volume, call FlushFileBuffers with a handle to the volume.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/flushfilebuffers.asp

But.. i'm not sure if this is what you need in your case.

-MAHESH
0
 
LVL 1

Author Comment

by:newton-allan
ID: 16958101
I don't think this is what I'm looking for. The documentation indicates this will flush an open file that has pending writes.

On an older computer, I'm trying to speed up the reading of a 4 meg file into a memory buffer. The first read might take 100 ms, and then the next read might take 10ms because of o/s caching. I would like to have an api-call before the read that would empty the o/s cache so I could see what actually causes the reading to speed up or slow down.
0
 
LVL 22

Expert Comment

by:mahesh1402
ID: 16958363
http://support.microsoft.com/kb/q110092/ <== As given here you can flush 32-bit file access cache (VCACHE)  by running MS-DOS Prompt and the pressing CTRL+C or CTRL+BREAK.

So what you need simulating Ctrl+C on DOS window .... So IF THERE IS NO API call for this you may do that by opening it programatically and simulating Ctrl+C on DOS window using CreateProcess or ShellExec and with following code to simulate Ctrl+C:

CWnd* myhwnd;
CString ClassName;
CString WindowText;
ClassName = "ConsoleWindowClass";
myhwnd =FindWindow(ClassName, WindowText);
     
myhwnd->SetForegroundWindow();
keybd_event(VK_CONTROL,0,0,0);
keybd_event (0x43, 0, 0, 0 ); //Send the C key (43 is "C")
keybd_event (0x43, 0, KEYEVENTF_KEYUP, 0);
keybd_event (VK_CONTROL, 0, KEYEVENTF_KEYUP, 0);// 'Left Control Up

-MAHESH

0
 
LVL 86

Accepted Solution

by:
jkr earned 125 total points
ID: 16960442
Call 'CreateFile()' with 'FILE_FLAG_NO_BUFFERING' to achieve the desired effect.
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 1

Author Comment

by:newton-allan
ID: 16972183
Ooops ... I accepted the answer too quickly before doing further checking. I got the same timings, but then when I rebooted and re-ran the test.exe file to read the 4 meg file, the first run took much longer.

One thing I did find out ... having the FILE_FLAG_NO_BUFFERING makes the ReadFile much faster for doing a single all-at-once read.

Without the FILE_FLAG_NO_BUFFERING on a reasonably modern AMD-64 3700, 1 gb memory and 250gb drive, the first read was about 120 milliseconds, and the second run was about 8 ms.  With FILE_FLAG_NO_BUFFERING, the numbers were 42 ms and 1.5 ms. Big difference.

Here's the code:

#include <stdio.h>
#include <windows.h>
#include "HiResTimer.h"

#define VPL_SIZE      (5000000)
char    vplBuf[VPL_SIZE];

void main(void)
{
   int    numRead;
   HANDLE fileHandle;
   BOOL   bFlag;
   double elapsed;

   HrtInit();
   HrtSetPriority(ABOVE_NORMAL_PRIORITY_CLASS);
   HrtStart();

   fileHandle = CreateFile("BigFile.txt",
                           GENERIC_READ, 0,
                           0, OPEN_EXISTING,
                           FILE_ATTRIBUTE_NORMAL
                           | FILE_FLAG_NO_BUFFERING
                           | FILE_FLAG_SEQUENTIAL_SCAN
                           | FILE_ATTRIBUTE_READONLY,
                           0);

   if (fileHandle == INVALID_HANDLE_VALUE) {
      printf("Unable to open file\n");
      return;
   }
   bFlag = ReadFile(fileHandle, vplBuf, VPL_SIZE,
                   (DWORD *)&numRead, NULL);
   
   CloseHandle(fileHandle);
   elapsed = HrtElapsedMillis();
   printf("Elapsed: %.3f\n", elapsed);
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 16972681
Have you checked the docs on that also?

An application must meet certain requirements when working with files opened with FILE_FLAG_NO_BUFFERING:

File access must begin at byte offsets within the file that are integer multiples of the volume's sector size.

File access must be for numbers of bytes that are integer multiples of the volume's sector size. For example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1024, or 2048 bytes, but not of 335, 981, or 7171 bytes.

Buffer addresses for read and write operations must be sector aligned (aligned on addresses in memory that are integer multiples of the volume's sector size).

One way to align buffers on integer multiples of the volume sector size is to use VirtualAlloc to allocate the buffers. It allocates memory that is aligned on addresses that are integer multiples of the operating system's memory page size. Because both memory page and volume sector sizes are powers of 2, this memory is also aligned on addresses that are integer multiples of a volume's sector size.

0
 
LVL 1

Author Comment

by:newton-allan
ID: 16974301
I tried the VirtualAlloc call with appropriate alignment, and I'm seeing more consistent timings. Thanks ... and leaving the question as PAQ is fine.

Does it appear I setup and used the VirtualAlloc call correctly?

#define VIRTUAL_ALLOC_BUF_SZ (1024*1024)
void ReadOneBigFileWithUnbufferedCreateFile(void)
{
  int    numRead;
  HANDLE fileHandle;
  BOOL   bFlag;

  HiResTimerInit();
  HiResTimerSetPriority(HIGH_PRIORITY_CLASS);

  char*  pVirtualAlloc = VirtualAlloc(NULL,
                                      VIRTUAL_ALLOC_BUF_SZ,
                                      MEM_COMMIT,
                                      PAGE_READWRITE);

  fileHandle = CreateFile("..\\..\\..\\EsvOtNtNoTags.vpl",
                          GENERIC_READ, 0,
                          0, OPEN_EXISTING,
                          FILE_FLAG_NO_BUFFERING,
                          0);

  if (fileHandle == INVALID_HANDLE_VALUE) {
     printf("Unable to open file\n");
     return;
  }
  bFlag = ReadFile(fileHandle, pVirtualAlloc,
                  VIRTUAL_ALLOC_BUF_SZ,
                  (DWORD *)&numRead, NULL);
   
  CloseHandle(fileHandle);

  double elapsed = HiResTimerElapsedMillis();
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 16989166
Sorry, I almost missed your last comment. Yes, that looks good to me, at least right now I can't spot anything that would seem off.
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Suggested Solutions

This is to be the first in a series of articles demonstrating the development of a complete windows based application using the MFC classes.  I’ll try to keep each article focused on one (or a couple) of the tasks that one may meet.   Introductio…
Have you tried to learn about Unicode, UTF-8, and multibyte text encoding and all the articles are just too "academic" or too technical? This article aims to make the whole topic easy for just about anyone to understand.
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
This video discusses moving either the default database or any database to a new volume.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now