Solved

HowTo: Flush o/s cache with win32 api call(s)?

Posted on 2006-06-21
9
1,475 Views
Last Modified: 2013-11-20
Sorry if this if OT because it deals with the WinApi rather than MFC specifically.

I'm doing some tests to speed up file handling using vc6 and WinXp-Sp2 on a development computer that has 1 gig of memory and an AMD-64-3800.

I'd like to clear/flush the o/s cache before doing the timings so as to get valid "1st time through" elapsed times. Otherwise, the 2nd and subsequent tests are much faster and don't tell me much.

Is there a Win32 api call to accomplish this? Or series of calls?

Now I'm doing the equivalent of xcopy of a large directory to another directory ... this seems to work, but is tedious and time-consuming.

0
Comment
Question by:newton-allan
  • 3
  • 3
  • 2
9 Comments
 
LVL 22

Expert Comment

by:mahesh1402
ID: 16957395
You can flush a files buffers with FlushFileBuffers. To flush all open files on a volume, call FlushFileBuffers with a handle to the volume.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/flushfilebuffers.asp

But.. i'm not sure if this is what you need in your case.

-MAHESH
0
 
LVL 1

Author Comment

by:newton-allan
ID: 16958101
I don't think this is what I'm looking for. The documentation indicates this will flush an open file that has pending writes.

On an older computer, I'm trying to speed up the reading of a 4 meg file into a memory buffer. The first read might take 100 ms, and then the next read might take 10ms because of o/s caching. I would like to have an api-call before the read that would empty the o/s cache so I could see what actually causes the reading to speed up or slow down.
0
 
LVL 22

Expert Comment

by:mahesh1402
ID: 16958363
http://support.microsoft.com/kb/q110092/ <== As given here you can flush 32-bit file access cache (VCACHE)  by running MS-DOS Prompt and the pressing CTRL+C or CTRL+BREAK.

So what you need simulating Ctrl+C on DOS window .... So IF THERE IS NO API call for this you may do that by opening it programatically and simulating Ctrl+C on DOS window using CreateProcess or ShellExec and with following code to simulate Ctrl+C:

CWnd* myhwnd;
CString ClassName;
CString WindowText;
ClassName = "ConsoleWindowClass";
myhwnd =FindWindow(ClassName, WindowText);
     
myhwnd->SetForegroundWindow();
keybd_event(VK_CONTROL,0,0,0);
keybd_event (0x43, 0, 0, 0 ); //Send the C key (43 is "C")
keybd_event (0x43, 0, KEYEVENTF_KEYUP, 0);
keybd_event (VK_CONTROL, 0, KEYEVENTF_KEYUP, 0);// 'Left Control Up

-MAHESH

0
 
LVL 86

Accepted Solution

by:
jkr earned 125 total points
ID: 16960442
Call 'CreateFile()' with 'FILE_FLAG_NO_BUFFERING' to achieve the desired effect.
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 1

Author Comment

by:newton-allan
ID: 16972183
Ooops ... I accepted the answer too quickly before doing further checking. I got the same timings, but then when I rebooted and re-ran the test.exe file to read the 4 meg file, the first run took much longer.

One thing I did find out ... having the FILE_FLAG_NO_BUFFERING makes the ReadFile much faster for doing a single all-at-once read.

Without the FILE_FLAG_NO_BUFFERING on a reasonably modern AMD-64 3700, 1 gb memory and 250gb drive, the first read was about 120 milliseconds, and the second run was about 8 ms.  With FILE_FLAG_NO_BUFFERING, the numbers were 42 ms and 1.5 ms. Big difference.

Here's the code:

#include <stdio.h>
#include <windows.h>
#include "HiResTimer.h"

#define VPL_SIZE      (5000000)
char    vplBuf[VPL_SIZE];

void main(void)
{
   int    numRead;
   HANDLE fileHandle;
   BOOL   bFlag;
   double elapsed;

   HrtInit();
   HrtSetPriority(ABOVE_NORMAL_PRIORITY_CLASS);
   HrtStart();

   fileHandle = CreateFile("BigFile.txt",
                           GENERIC_READ, 0,
                           0, OPEN_EXISTING,
                           FILE_ATTRIBUTE_NORMAL
                           | FILE_FLAG_NO_BUFFERING
                           | FILE_FLAG_SEQUENTIAL_SCAN
                           | FILE_ATTRIBUTE_READONLY,
                           0);

   if (fileHandle == INVALID_HANDLE_VALUE) {
      printf("Unable to open file\n");
      return;
   }
   bFlag = ReadFile(fileHandle, vplBuf, VPL_SIZE,
                   (DWORD *)&numRead, NULL);
   
   CloseHandle(fileHandle);
   elapsed = HrtElapsedMillis();
   printf("Elapsed: %.3f\n", elapsed);
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 16972681
Have you checked the docs on that also?

An application must meet certain requirements when working with files opened with FILE_FLAG_NO_BUFFERING:

File access must begin at byte offsets within the file that are integer multiples of the volume's sector size.

File access must be for numbers of bytes that are integer multiples of the volume's sector size. For example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1024, or 2048 bytes, but not of 335, 981, or 7171 bytes.

Buffer addresses for read and write operations must be sector aligned (aligned on addresses in memory that are integer multiples of the volume's sector size).

One way to align buffers on integer multiples of the volume sector size is to use VirtualAlloc to allocate the buffers. It allocates memory that is aligned on addresses that are integer multiples of the operating system's memory page size. Because both memory page and volume sector sizes are powers of 2, this memory is also aligned on addresses that are integer multiples of a volume's sector size.

0
 
LVL 1

Author Comment

by:newton-allan
ID: 16974301
I tried the VirtualAlloc call with appropriate alignment, and I'm seeing more consistent timings. Thanks ... and leaving the question as PAQ is fine.

Does it appear I setup and used the VirtualAlloc call correctly?

#define VIRTUAL_ALLOC_BUF_SZ (1024*1024)
void ReadOneBigFileWithUnbufferedCreateFile(void)
{
  int    numRead;
  HANDLE fileHandle;
  BOOL   bFlag;

  HiResTimerInit();
  HiResTimerSetPriority(HIGH_PRIORITY_CLASS);

  char*  pVirtualAlloc = VirtualAlloc(NULL,
                                      VIRTUAL_ALLOC_BUF_SZ,
                                      MEM_COMMIT,
                                      PAGE_READWRITE);

  fileHandle = CreateFile("..\\..\\..\\EsvOtNtNoTags.vpl",
                          GENERIC_READ, 0,
                          0, OPEN_EXISTING,
                          FILE_FLAG_NO_BUFFERING,
                          0);

  if (fileHandle == INVALID_HANDLE_VALUE) {
     printf("Unable to open file\n");
     return;
  }
  bFlag = ReadFile(fileHandle, pVirtualAlloc,
                  VIRTUAL_ALLOC_BUF_SZ,
                  (DWORD *)&numRead, NULL);
   
  CloseHandle(fileHandle);

  double elapsed = HiResTimerElapsedMillis();
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 16989166
Sorry, I almost missed your last comment. Yes, that looks good to me, at least right now I can't spot anything that would seem off.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Introduction: Dialogs (2) modeless dialog and a worker thread.  Handling data shared between threads.  Recursive functions. Continuing from the tenth article about sudoku.   Last article we worked with a modal dialog to help maintain informat…
If you use Adobe Reader X it is possible you can't open OLE PDF documents in the standard. The reason is the 'save box mode' in adobe reader X. Many people think the protected Mode of adobe reader x is only to stop the write access. But this fe…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
In this video I am going to show you how to back up and restore Office 365 mailboxes using CodeTwo Backup for Office 365. Learn more about the tool used in this video here: http://www.codetwo.com/backup-for-office-365/ (http://www.codetwo.com/ba…

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now