Solved

How do you write to end of files opened with FILE_FLAG_NO_BUFFERING?

Posted on 2003-03-12
8
1,341 Views
Last Modified: 2013-12-03
Files opened with FiLE_FLAG_NO_BUFFERING need to be written in chunks that are multiples of the sector size.  I have to write a huge file, but the data size is not a multiple of the sector size.  How can I write the last handful of bytes?  I've looked into write a full block and then truncating the file, but I don't see how to do that either since SetFilePointer requires that you set the file pointer to sector boundaries.
0
Comment
Question by:danteleone
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 49

Expert Comment

by:DanRollins
ID: 8126858
1) Write the file, including the slack at the end.  But make a note of the 'actual' end of file.
2) Close it.  
3) Reopen it normally (without FiLE_FLAG_NO_BUFFERING).
4) Seek to the remembered 'actual' end of file.
5) Call SetEndOfFile

There ya do!
-- Dan
0
 

Author Comment

by:danteleone
ID: 8128067
Thanks, but the problem is that the files that I'm working with are huge (this one is 180 GB), and I'm hitting a hard limit in Windows on the total size of files opened with buffering (150 GB), because the kernel runs out of memory maintaining its cache data structure.  So I can't open the file normally to truncate it.
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 8131468
You are writing a 150 GB file and worrying about the max extra 511 bytes at the end?

What if you access if for GENERIC_WRITE?  Does the system choke in that case?

-- Dan
0
Creating Instructional Tutorials  

For Any Use & On Any Platform

Contextual Guidance at the moment of need helps your employees/users adopt software o& achieve even the most complex tasks instantly. Boost knowledge retention, software adoption & employee engagement with easy solution.

 
LVL 1

Expert Comment

by:Hermetic
ID: 8141761
If you aren't reading from the file, then you can
use FILE_FLAG_WRITE_THROUGH, this disables write buffering
but I think lifts the sector write limitation.
0
 

Author Comment

by:danteleone
ID: 8155035
Thanks, but the problem is that the files that I'm working with are huge (this one is 180 GB), and I'm hitting a hard limit in Windows on the total size of files opened with buffering (150 GB), because the kernel runs out of memory maintaining its cache data structure.  So I can't open the file normally to truncate it.
0
 

Author Comment

by:danteleone
ID: 8155054
Ignore my duplicate last comment...

I didn't get a chance to try the FILE_FLAG_WRITE_THROUGH flag, but the documentation isn't promising, since it does say that caching can still take place.

In any event, I believe I've found a solution: the ntdll call NtSetInformationFile allows you to explicitly set the file size.  So I can write a final full sector, and then truncate using that.
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 8155300
It sound as if you have solved this yourself.  Please look here for instructions on how to procede:
   http://www.apollois.com/EE/Help/Closing_Questions.htm#Refund
-- Dan
0
 

Accepted Solution

by:
SpideyMod earned 0 total points
ID: 8157103
PAQ'd and all 75 points refunded.

SpideyMod
Community Support Moderator @Experts Exchange
0

Featured Post

Creating Instructional Tutorials  

For Any Use & On Any Platform

Contextual Guidance at the moment of need helps your employees/users adopt software o& achieve even the most complex tasks instantly. Boost knowledge retention, software adoption & employee engagement with easy solution.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows a few slightly more advanced techniques for Windows 7 gadget programming, including how to save and restore user settings for your gadget and how to populate the "details" panel that is displayed in the Windows 7 gadget gallery.  …
This article describes how to add a user-defined command button to the Windows 7 Explorer toolbar.  In the previous article (http://www.experts-exchange.com/A_2172.html), we saw how to put the Delete button back there where it belongs.  "Delete" is …
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA. http://www.experts-exchange.com/videos/1478/Excel-Error-Handlin…
NetCrunch network monitor is a highly extensive platform for network monitoring and alert generation. In this video you'll see a live demo of NetCrunch with most notable features explained in a walk-through manner. You'll also get to know the philos…

628 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question