danteleone
asked on
How do you write to end of files opened with FILE_FLAG_NO_BUFFERING?
Files opened with FiLE_FLAG_NO_BUFFERING need to be written in chunks that are multiples of the sector size. I have to write a huge file, but the data size is not a multiple of the sector size. How can I write the last handful of bytes? I've looked into write a full block and then truncating the file, but I don't see how to do that either since SetFilePointer requires that you set the file pointer to sector boundaries.
ASKER
Thanks, but the problem is that the files that I'm working with are huge (this one is 180 GB), and I'm hitting a hard limit in Windows on the total size of files opened with buffering (150 GB), because the kernel runs out of memory maintaining its cache data structure. So I can't open the file normally to truncate it.
You are writing a 150 GB file and worrying about the max extra 511 bytes at the end?
What if you access if for GENERIC_WRITE? Does the system choke in that case?
-- Dan
What if you access if for GENERIC_WRITE? Does the system choke in that case?
-- Dan
If you aren't reading from the file, then you can
use FILE_FLAG_WRITE_THROUGH, this disables write buffering
but I think lifts the sector write limitation.
use FILE_FLAG_WRITE_THROUGH, this disables write buffering
but I think lifts the sector write limitation.
ASKER
Thanks, but the problem is that the files that I'm working with are huge (this one is 180 GB), and I'm hitting a hard limit in Windows on the total size of files opened with buffering (150 GB), because the kernel runs out of memory maintaining its cache data structure. So I can't open the file normally to truncate it.
ASKER
Ignore my duplicate last comment...
I didn't get a chance to try the FILE_FLAG_WRITE_THROUGH flag, but the documentation isn't promising, since it does say that caching can still take place.
In any event, I believe I've found a solution: the ntdll call NtSetInformationFile allows you to explicitly set the file size. So I can write a final full sector, and then truncate using that.
I didn't get a chance to try the FILE_FLAG_WRITE_THROUGH flag, but the documentation isn't promising, since it does say that caching can still take place.
In any event, I believe I've found a solution: the ntdll call NtSetInformationFile allows you to explicitly set the file size. So I can write a final full sector, and then truncate using that.
It sound as if you have solved this yourself. Please look here for instructions on how to procede:
http://www.apollois.com/EE/Help/Closing_Questions.htm#Refund
-- Dan
http://www.apollois.com/EE/Help/Closing_Questions.htm#Refund
-- Dan
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
2) Close it.
3) Reopen it normally (without FiLE_FLAG_NO_BUFFERING).
4) Seek to the remembered 'actual' end of file.
5) Call SetEndOfFile
There ya do!
-- Dan