Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

How do you write to end of files opened with FILE_FLAG_NO_BUFFERING?

Posted on 2003-03-12
8
Medium Priority
?
1,370 Views
Last Modified: 2013-12-03
Files opened with FiLE_FLAG_NO_BUFFERING need to be written in chunks that are multiples of the sector size.  I have to write a huge file, but the data size is not a multiple of the sector size.  How can I write the last handful of bytes?  I've looked into write a full block and then truncating the file, but I don't see how to do that either since SetFilePointer requires that you set the file pointer to sector boundaries.
0
Comment
Question by:danteleone
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 49

Expert Comment

by:DanRollins
ID: 8126858
1) Write the file, including the slack at the end.  But make a note of the 'actual' end of file.
2) Close it.  
3) Reopen it normally (without FiLE_FLAG_NO_BUFFERING).
4) Seek to the remembered 'actual' end of file.
5) Call SetEndOfFile

There ya do!
-- Dan
0
 

Author Comment

by:danteleone
ID: 8128067
Thanks, but the problem is that the files that I'm working with are huge (this one is 180 GB), and I'm hitting a hard limit in Windows on the total size of files opened with buffering (150 GB), because the kernel runs out of memory maintaining its cache data structure.  So I can't open the file normally to truncate it.
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 8131468
You are writing a 150 GB file and worrying about the max extra 511 bytes at the end?

What if you access if for GENERIC_WRITE?  Does the system choke in that case?

-- Dan
0
Get your Disaster Recovery as a Service basics

Disaster Recovery as a Service is one go-to solution that revolutionizes DR planning. Implementing DRaaS could be an efficient process, easily accessible to non-DR experts. Learn about monitoring, testing, executing failovers and failbacks to ensure a "healthy" DR environment.

 
LVL 1

Expert Comment

by:Hermetic
ID: 8141761
If you aren't reading from the file, then you can
use FILE_FLAG_WRITE_THROUGH, this disables write buffering
but I think lifts the sector write limitation.
0
 

Author Comment

by:danteleone
ID: 8155035
Thanks, but the problem is that the files that I'm working with are huge (this one is 180 GB), and I'm hitting a hard limit in Windows on the total size of files opened with buffering (150 GB), because the kernel runs out of memory maintaining its cache data structure.  So I can't open the file normally to truncate it.
0
 

Author Comment

by:danteleone
ID: 8155054
Ignore my duplicate last comment...

I didn't get a chance to try the FILE_FLAG_WRITE_THROUGH flag, but the documentation isn't promising, since it does say that caching can still take place.

In any event, I believe I've found a solution: the ntdll call NtSetInformationFile allows you to explicitly set the file size.  So I can write a final full sector, and then truncate using that.
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 8155300
It sound as if you have solved this yourself.  Please look here for instructions on how to procede:
   http://www.apollois.com/EE/Help/Closing_Questions.htm#Refund
-- Dan
0
 

Accepted Solution

by:
SpideyMod earned 0 total points
ID: 8157103
PAQ'd and all 75 points refunded.

SpideyMod
Community Support Moderator @Experts Exchange
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article describes a technique for converting RTF (Rich Text Format) data to HTML and provides C++ source that does it all in just a few lines of code. Although RTF is coming to be considered a "legacy" format, it is still in common use... po…
For most people, the WrapPanel seems like a magic when they switch from WinForms to WPF. Most of us will think that the code that is used to write a control like that would be difficult. However, most of the work is done by the WPF engine, and the W…
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA. http://www.experts-exchange.com/videos/1478/Excel-Error-Handlin…
This is my first video review of Microsoft Bookings, I will be doing a part two with a bit more information, but wanted to get this out to you folks.

721 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question