?
Solved

Bytes read during an Asychronous File Read?

Posted on 2004-04-09
3
Medium Priority
?
562 Views
Last Modified: 2008-02-01
I want to start processing on a data from a file while it is being loaded. I figure the only way for me to do this is with an asynchronous file read, but I can't seem to find a way to determine how much of the file has been loaded. If I know how much is loaded, I can limit my processing to the data that is loaded and not worry about processing data locations before they are filled. MSDN says that GetOverlappedResult() will return the number of bytes transferred, but in practice the call fails during the read and returns a byte value only when the read is completed.

unsigned int array[40000000];
HANDLE inputFile;

inputFile = CreateFile("test.file", GENERIC_READ, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL | FILE_FLAG_SEQUENTIAL_SCAN | FILE_FLAG_OVERLAPPED, NULL);
ReadFileEx(inputFile, &array[0], 40000000, &asyncSettings, NULL);

if(GetOverlappedResult(inputFile, &asyncSettings, &bytesRead, FALSE)) {
     AfxMessageBox("It worked!");   //it never works...
}
//Process Data in a Loop based on the number of bytes read      

Anyone have any ideas how to do this?
0
Comment
Question by:noble24
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 23

Expert Comment

by:brettmjohnson
ID: 10794667
ReadFileEx() will not call your callback routine until the entire 40MB has been read.

The easier way to do this is to use multiple threads.   One thread reading the file
in smaller chunks, while the other thread processes the data that has been read.

I have also done this in the past using file mapping, with one thread skipping along
the file's address space reading one word of each VM page, and the other thread
processing the data that has been mapped in.

0
 
LVL 22

Assisted Solution

by:grg99
grg99 earned 600 total points
ID: 10794737
Simple, read the file as you're doing, in async mode, but read smaller chunks.
Say read in one megabyte per read request, but do 40 read requests, all in parallel.
You'll get 40 callbacks, one for each one megabyte chunk.

Or if you want to minimize the number of async requests in the air, jut have two requests going, then in the callback schedule a new one.
That way you'll always have a read request going.

0
 
LVL 4

Accepted Solution

by:
caner_elci earned 800 total points
ID: 10795141
The last parameter of ReadFileEx() is the callback function which will be called when async read operation is completed. So you should not just give NULL as a callback function. Define one something like this:

void WINAPI ReadCompleteCallBack( DWORD dwError, DWORD dwBytesRead, LPOVERLAPPED lpOverlapped )

and as you can see, this callback function has all information you need. Number of bytes read (which is the same with the one you give in ReadFileEx unless it's end of file), any error code and your OVERLAPPED structure.

You should create a buffer with proper size because you'll have to wait until it's filled even this is a async operation. Then you can re-ReadFileEx in your callback function for the rest of data. But be careful here, you should advance Offset field of your OVERLAPPED structure before calling further ReadFileEx() because it won't be incremented automatically. Also you should check if it's end of file, and close your handle and inform your main loop that it's over.

When you call ReadFileEx() first time (not in the callback function) you should use a alertable function such as SleepEx or MsgWaitForMultipleObjectsEx (there are more of them)...

One more tip, there is a hEvent field in OVERLAPPED structure which is free to use. You can create a simple structure which has some fields like hWnd of your parent window, EOF state etc... And you can pass it by OVERLAPPED structure. That will help you to send messages to your main window to inform the status of your async read. For example, WM_USER + 1 may be a read operation having data in LPARAM of the message and data length in WPARAM; or you can send WM_USER + 3 to inform that it's EOF... etc

Anyway, I also recommend you to use multiple threads if possible.. it would be easier and looks better...

Caner ELCI
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When writing generic code, using template meta-programming techniques, it is sometimes useful to know if a type is convertible to another type. A good example of when this might be is if you are writing diagnostic instrumentation for code to generat…
C++ Properties One feature missing from standard C++ that you will find in many other Object Oriented Programming languages is something called a Property (http://www.experts-exchange.com/Programming/Languages/CPP/A_3912-Object-Properties-in-C.ht…
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.
The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.
Suggested Courses

765 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question