Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

HowTo: Flush o/s cache with win32 api call(s)?

Posted on 2006-06-21
9
Medium Priority
?
1,492 Views
Last Modified: 2013-11-20
Sorry if this if OT because it deals with the WinApi rather than MFC specifically.

I'm doing some tests to speed up file handling using vc6 and WinXp-Sp2 on a development computer that has 1 gig of memory and an AMD-64-3800.

I'd like to clear/flush the o/s cache before doing the timings so as to get valid "1st time through" elapsed times. Otherwise, the 2nd and subsequent tests are much faster and don't tell me much.

Is there a Win32 api call to accomplish this? Or series of calls?

Now I'm doing the equivalent of xcopy of a large directory to another directory ... this seems to work, but is tedious and time-consuming.

0
Comment
Question by:newton-allan
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
9 Comments
 
LVL 22

Expert Comment

by:mahesh1402
ID: 16957395
You can flush a files buffers with FlushFileBuffers. To flush all open files on a volume, call FlushFileBuffers with a handle to the volume.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/flushfilebuffers.asp

But.. i'm not sure if this is what you need in your case.

-MAHESH
0
 
LVL 1

Author Comment

by:newton-allan
ID: 16958101
I don't think this is what I'm looking for. The documentation indicates this will flush an open file that has pending writes.

On an older computer, I'm trying to speed up the reading of a 4 meg file into a memory buffer. The first read might take 100 ms, and then the next read might take 10ms because of o/s caching. I would like to have an api-call before the read that would empty the o/s cache so I could see what actually causes the reading to speed up or slow down.
0
 
LVL 22

Expert Comment

by:mahesh1402
ID: 16958363
http://support.microsoft.com/kb/q110092/ <== As given here you can flush 32-bit file access cache (VCACHE)  by running MS-DOS Prompt and the pressing CTRL+C or CTRL+BREAK.

So what you need simulating Ctrl+C on DOS window .... So IF THERE IS NO API call for this you may do that by opening it programatically and simulating Ctrl+C on DOS window using CreateProcess or ShellExec and with following code to simulate Ctrl+C:

CWnd* myhwnd;
CString ClassName;
CString WindowText;
ClassName = "ConsoleWindowClass";
myhwnd =FindWindow(ClassName, WindowText);
     
myhwnd->SetForegroundWindow();
keybd_event(VK_CONTROL,0,0,0);
keybd_event (0x43, 0, 0, 0 ); //Send the C key (43 is "C")
keybd_event (0x43, 0, KEYEVENTF_KEYUP, 0);
keybd_event (VK_CONTROL, 0, KEYEVENTF_KEYUP, 0);// 'Left Control Up

-MAHESH

0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 86

Accepted Solution

by:
jkr earned 500 total points
ID: 16960442
Call 'CreateFile()' with 'FILE_FLAG_NO_BUFFERING' to achieve the desired effect.
0
 
LVL 1

Author Comment

by:newton-allan
ID: 16972183
Ooops ... I accepted the answer too quickly before doing further checking. I got the same timings, but then when I rebooted and re-ran the test.exe file to read the 4 meg file, the first run took much longer.

One thing I did find out ... having the FILE_FLAG_NO_BUFFERING makes the ReadFile much faster for doing a single all-at-once read.

Without the FILE_FLAG_NO_BUFFERING on a reasonably modern AMD-64 3700, 1 gb memory and 250gb drive, the first read was about 120 milliseconds, and the second run was about 8 ms.  With FILE_FLAG_NO_BUFFERING, the numbers were 42 ms and 1.5 ms. Big difference.

Here's the code:

#include <stdio.h>
#include <windows.h>
#include "HiResTimer.h"

#define VPL_SIZE      (5000000)
char    vplBuf[VPL_SIZE];

void main(void)
{
   int    numRead;
   HANDLE fileHandle;
   BOOL   bFlag;
   double elapsed;

   HrtInit();
   HrtSetPriority(ABOVE_NORMAL_PRIORITY_CLASS);
   HrtStart();

   fileHandle = CreateFile("BigFile.txt",
                           GENERIC_READ, 0,
                           0, OPEN_EXISTING,
                           FILE_ATTRIBUTE_NORMAL
                           | FILE_FLAG_NO_BUFFERING
                           | FILE_FLAG_SEQUENTIAL_SCAN
                           | FILE_ATTRIBUTE_READONLY,
                           0);

   if (fileHandle == INVALID_HANDLE_VALUE) {
      printf("Unable to open file\n");
      return;
   }
   bFlag = ReadFile(fileHandle, vplBuf, VPL_SIZE,
                   (DWORD *)&numRead, NULL);
   
   CloseHandle(fileHandle);
   elapsed = HrtElapsedMillis();
   printf("Elapsed: %.3f\n", elapsed);
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 16972681
Have you checked the docs on that also?

An application must meet certain requirements when working with files opened with FILE_FLAG_NO_BUFFERING:

File access must begin at byte offsets within the file that are integer multiples of the volume's sector size.

File access must be for numbers of bytes that are integer multiples of the volume's sector size. For example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1024, or 2048 bytes, but not of 335, 981, or 7171 bytes.

Buffer addresses for read and write operations must be sector aligned (aligned on addresses in memory that are integer multiples of the volume's sector size).

One way to align buffers on integer multiples of the volume sector size is to use VirtualAlloc to allocate the buffers. It allocates memory that is aligned on addresses that are integer multiples of the operating system's memory page size. Because both memory page and volume sector sizes are powers of 2, this memory is also aligned on addresses that are integer multiples of a volume's sector size.

0
 
LVL 1

Author Comment

by:newton-allan
ID: 16974301
I tried the VirtualAlloc call with appropriate alignment, and I'm seeing more consistent timings. Thanks ... and leaving the question as PAQ is fine.

Does it appear I setup and used the VirtualAlloc call correctly?

#define VIRTUAL_ALLOC_BUF_SZ (1024*1024)
void ReadOneBigFileWithUnbufferedCreateFile(void)
{
  int    numRead;
  HANDLE fileHandle;
  BOOL   bFlag;

  HiResTimerInit();
  HiResTimerSetPriority(HIGH_PRIORITY_CLASS);

  char*  pVirtualAlloc = VirtualAlloc(NULL,
                                      VIRTUAL_ALLOC_BUF_SZ,
                                      MEM_COMMIT,
                                      PAGE_READWRITE);

  fileHandle = CreateFile("..\\..\\..\\EsvOtNtNoTags.vpl",
                          GENERIC_READ, 0,
                          0, OPEN_EXISTING,
                          FILE_FLAG_NO_BUFFERING,
                          0);

  if (fileHandle == INVALID_HANDLE_VALUE) {
     printf("Unable to open file\n");
     return;
  }
  bFlag = ReadFile(fileHandle, pVirtualAlloc,
                  VIRTUAL_ALLOC_BUF_SZ,
                  (DWORD *)&numRead, NULL);
   
  CloseHandle(fileHandle);

  double elapsed = HiResTimerElapsedMillis();
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 16989166
Sorry, I almost missed your last comment. Yes, that looks good to me, at least right now I can't spot anything that would seem off.
0

Featured Post

Moving data to the cloud? Find out if you’re ready

Before moving to the cloud, it is important to carefully define your db needs, plan for the migration & understand prod. environment. This wp explains how to define what you need from a cloud provider, plan for the migration & what putting a cloud solution into practice entails.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction: Load and Save to file, Document-View interaction inside the SDI. Continuing from the second article about sudoku.   Open the project in visual studio. From the class view select CSudokuDoc and double click to open the header …
Introduction: Dialogs (1) modal - maintaining the database. Continuing from the ninth article about sudoku.   You might have heard of modal and modeless dialogs.  Here with this Sudoku application will we use one of each type: a modal dialog …
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
In this brief tutorial Pawel from AdRem Software explains how you can quickly find out which services are running on your network, or what are the IP addresses of servers responsible for each service. Software used is freeware NetCrunch Tools (https…

664 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question