Dennis Miller
asked on
Data not being written to disk - carefully placed breakpoints fix the behavior
I have a small program written in C++ with Win32 that writes data to the slack space of a file; it does this by writing to the data to the file with WriteFile() and then shrinking the file with SetEndOfFile() to leave the original file in tact, but still have the written data remain on disk. By examining my hard drive with Winhex, I've confirmed that this approach works, but only in rather unusual circumstances.
Basically, whenever I run my program normally, either by itself or in the Visual Studio debugger, I can't see any changes in the data on the disk after it finishes; Windows confirms that the file itself was modified, but Winhex doesn't show the new slack space data I wrote. However, if I start the program in the debugger with a breakpoint right after my call to WriteFile(), when the breakpoint is hit, I can see the changes on the hard drive in Winhex immediately. I figured that the buffered data to write wasn't getting flushed out in time, so I put in a call to FlushFileBuffers() right after WriteFile(), but I was still getting the same behavior. However, by putting in other meaningless calls to take up time, like a call to system("pause") after WriteFile(), the program successfully wrote the data to disk no matter how I ran it.
So, does anyone know what's really going on here? How can I ensure (without using hacks like system("pause")) that my program always writes its data to disk? Any help would be appreciated.
The code in question (that requires a breakpoint on the call to SetFilePointer()) is attached.
Basically, whenever I run my program normally, either by itself or in the Visual Studio debugger, I can't see any changes in the data on the disk after it finishes; Windows confirms that the file itself was modified, but Winhex doesn't show the new slack space data I wrote. However, if I start the program in the debugger with a breakpoint right after my call to WriteFile(), when the breakpoint is hit, I can see the changes on the hard drive in Winhex immediately. I figured that the buffered data to write wasn't getting flushed out in time, so I put in a call to FlushFileBuffers() right after WriteFile(), but I was still getting the same behavior. However, by putting in other meaningless calls to take up time, like a call to system("pause") after WriteFile(), the program successfully wrote the data to disk no matter how I ran it.
So, does anyone know what's really going on here? How can I ensure (without using hacks like system("pause")) that my program always writes its data to disk? Any help would be appreciated.
The code in question (that requires a breakpoint on the call to SetFilePointer()) is attached.
SetFilePointer(current_file, 0, NULL, FILE_END);
for (int i = 0; i < cluster_size / 2; ++i)
WriteFile(current_file, "01", 2, &bytes_returned, NULL);
SetFilePointer(current_file, -cluster_size, NULL, FILE_END);
SetEndOfFile(current_file);
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
The required pause is the clue:
I think that your device driver is just optimized. Before its caching logic does the physical write-to-disk, it checks down the queue a bit and sees that the file will soon be truncated, making the disk write unnecessary.
That would be not unlike the way an optimizing complier will "optimize away" intermediate results in a calculation series, and in fact discard the entrie code sequence, when it sees that the next statement assigns a literal value to the variable, overwriting it anyway.
There might be an IOCTL function, or other mechanism to override the standard operations of the cache control logic, or there might not be.
You might be able to experiment on a thumb drive in which you have turned off write-behind option ("optimize for quick removal") to see if it affects the issue.
I think that your device driver is just optimized. Before its caching logic does the physical write-to-disk, it checks down the queue a bit and sees that the file will soon be truncated, making the disk write unnecessary.
That would be not unlike the way an optimizing complier will "optimize away" intermediate results in a calculation series, and in fact discard the entrie code sequence, when it sees that the next statement assigns a literal value to the variable, overwriting it anyway.
There might be an IOCTL function, or other mechanism to override the standard operations of the cache control logic, or there might not be.
You might be able to experiment on a thumb drive in which you have turned off write-behind option ("optimize for quick removal") to see if it affects the issue.
ASKER
Thanks for the advice Dan; I had disregarded optimizations as the source of the problem when I saw that optimizations were disabled in my VS project settings.
I tried running the program on a thumb drive with "optimize for quick removal" checked, but the same things happened. I'll take your advice and continue to search for any methods that might help me override this behavior. In the worst case scenario, I can just have my program pause for a moment before truncating the file, but this will be a pretty big performance hit, since it operates on a lot of files in one run.
I'll leave the question open until I come to a final solution.
I tried running the program on a thumb drive with "optimize for quick removal" checked, but the same things happened. I'll take your advice and continue to search for any methods that might help me override this behavior. In the worst case scenario, I can just have my program pause for a moment before truncating the file, but this will be a pretty big performance hit, since it operates on a lot of files in one run.
I'll leave the question open until I come to a final solution.
Of course disk caching optimization is totally unrelated to compiler code optimization...
For more diagnostic info, you could try:
IOCTL_DISK_GET_CACHE_INFOR
http://msdn.microsoft.com/en-us/library/aa365165(VS.85).aspx
For more diagnostic info, you could try:
IOCTL_DISK_GET_CACHE_INFOR
http://msdn.microsoft.com/en-us/library/aa365165(VS.85).aspx
ASKER
Calling FlushFileBuffers() worked. Before, I was calling it on the handle to the file, but when I called on a handle to the drive holding the file, it worked. Thanks for the help.
ASKER