Link to home
Start Free TrialLog in
Avatar of gaggio
gaggio

asked on

Is WriteFile() limited to 64MB buffer writes?

WriteFile() returns 0 when I ask it to write down a 67108864 byte buffer...

GetLastError() returns error code #1450 (Not enough system resources)

Anybody has any idea of the reason of this behavior, and how to solve it?

An obvious solution is of course to reduce the size of my buffer, but I am willing to do so only if there is no other solution.

Thanks for your help
Avatar of SteveGTR
SteveGTR
Flag of United States of America image

I ran the code below on my XP Pro box 512MB on memory and it ran fine:

  HANDLE hFile =
    ::CreateFile("c:\\temp\\temp.dat", GENERIC_WRITE, 0, NULL,
      CREATE_ALWAYS,  FILE_ATTRIBUTE_NORMAL, NULL);

  if (hFile != INVALID_HANDLE_VALUE)
    {
    char* pBuff = (char*) calloc(sizeof(char), 67108864);
    unsigned long bytesOut = 0L;

    BOOL bCode = ::WriteFile(hFile, pBuff, 67108864, &bytesOut, FALSE);

    if (bCode != 0)
      TRACE("It worked, %ld bytes written\n", bytesOut);
    else
      TRACE("It didn't work, GetLastError(): %u\n", GetLastError());

    ::CloseHandle(hFile);
    free(pBuff);
    }

I couldn't find any information on WriteFile and your error.
Avatar of gaggio
gaggio

ASKER

Well, sorry but I forgot to say something quite important: I am working in the following mode:

hFile = CreateFile(filename, GENERIC_READ | GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_FLAG_WRITE_THROUGH|FILE_FLAG_NO_BUFFERING, NULL);

Could you try to run the test with this instead?

Thanks
ASKER CERTIFIED SOLUTION
Avatar of SteveGTR
SteveGTR
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
That is 64MB so your assumption is correct.
Avatar of gaggio

ASKER

Thanks, you will get some points when I close the question, but before doing so I would like to know THE REASON of this 64MB limit.

And is it possible to increase it?
I'd imagine that the reason is that there isn't limitless amounts of memory and 64MB is a pretty generous buffer size in my view. As for increasing that buffer, I'd have to sniff around. Those kinds of settings are generally configurable.
Avatar of gaggio

ASKER

Thanks, well, if you could find this somewhere that would be awesome!
I couldn't find any information on this setting. Sorry :(
Avatar of gaggio

ASKER

I am going to wait a little more before closing the question, in case somebody finds anything about it...
If you remove the  FILE_FLAG_NO_BUFFERING  flag WriteFile function will return 1. Having this flag turns off system cache and hence WriteFile fails. If you have to have this flag you can use VirtualAlloc function to specify the memory caching settings.

If your application performs file input or output without using the intermediate buffering or caching provided by the system, then only this flag need to be used.

An application must meet certain requirements when working with files that are opened with FILE_FLAG_NO_BUFFERING:

1. File access must begin at byte offsets within a file that are integer multiples of the volume sector size.
2. File access must be for numbers of bytes that are integer multiples of the volume sector size. For example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1024, or 2048 bytes, but not of 335, 981, or 7171 bytes.
3. Buffer addresses for read and write operations should be sector aligned, which means aligned on addresses in memory that are integer multiples of the volume sector size. Depending on the disk, this requirement may not be enforced.

One way to align buffers on integer multiples of the volume sector size is to use VirtualAlloc to allocate the buffers. It allocates memory that is aligned on addresses that are integer multiples of the operating system's memory page size. Because both memory page and volume sector sizes are powers of 2, this memory is also aligned on addresses that are integer multiples of a volume sector size.

An application can determine a volume sector size by calling the GetDiskFreeSpace function.

You can additionally refer to
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/reserving_and_committing_memory.asp
Avatar of gaggio

ASKER

Thank you but I already had this information: I know how to use search engines as well as you!
You are not responding to my question at all, you are just pasting from MSDN...
Still waiting...
Gaggio,

I tried to bring some more details about the  FILE_FLAG_NO_BUFFERING  flag and the reason for WriteFile method failure. Off course MSDN is the bible that everyone refers to. But comments are not just blindly posted by copy & paste. I think you can understand that.

If someone tries to help or share knowledge with you, it is not a good practice to turn them out with comments like this.

Avatar of gaggio

ASKER

Well, thanks then if you were sincerely trying to help.
I was simply trying to show that there are many "experts" who just do not want to spend any time on a question and may get cheap points because of people who did not even try google to get information before wasting their points on this board.
But this question is difficult and I won't be able to close it with an answer like the one you wrote.
My answer identifies the problem.
Avatar of DanRollins
SteveGTR answered the original question and though the specific reason for the 64MB limit was not found, I agree that he deserves the points.