ifstream + Binary file + Input -- Problems

Posted on 2006-06-20
Last Modified: 2013-11-15
Basically I'm reading a file which is 1201*1201*2 bytes (2.8 megs) in size. I have several thousands of these files.
Upon inspection, these files are exactly this size.

These files contain short-values in big-endian format. My program converts them. But that matters little to this question.

When I attempt to read the entire file into memory, Many times will return false.
This isually happens around byte # 69632  and this isn't file specific.

I've tried reading the entire file in 1 shot  (, 1201*1201*2))  
I've tried reading it in via a normal buffer  (, 1201))   ... process buffer ...
I've tried reading it in one byte at a time (, 1)) ... process AChar...

Here's the code using a normal sized buffer[1201]

                        char C; int Counter = 0; char buffer[1201]; int H;
                        while(Counter < 1201 * 1201 * 2){//, 1)){
                                    MessageBox(NULL, "It's not good", "Nope", MB_OK);
                    , 1201);
                              for(H = 0; H < 1201; H++){

Note: FileData is an array of Chars.    Char* FileData = new Char[1201*1201*2]

Note: I had an old system which would open a file, and seek around in that file for specific bytes that were needed... but now I pretty much need every byte in every file, so I'm hoping to just read it all in at once and access it in an array (the old seek seek seek seek strategy was pretty slow for this).

Confused and Frustrated
Question by:oxygen_728
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 8
  • 2
  • 2
  • +1
LVL 86

Accepted Solution

jkr earned 450 total points
ID: 16945946
Stupid Q - how are you opening the file? Are you using 'ios_base::binary' as the open mode?

Also, if you are on windows, you might want to consider reading the file using Win32 APIs, e.g.


DWORD dwSize = 1201 * 1201 *2;
DWORD dwRead;

ReadFile(hFile,(LPVOID FileData,dwSize,&dwRead,NULL);


if (dwSize != dwRead) {

  // error

Author Comment

ID: 16946239
I'm sorry, I thought i included that.,ios::in | ios::binary);

I got pretty frustrated looking through the archived questions on EE - almost EVERYBODY that had a problem similar to mine wasn't using the ios::binary flag.

I'm trying to use the ReadFile technique now.... but I'm getting some strange error codes.

It may be a hardware issue... I'm using an external USB hard drive... and It currently won't let me copy files from it to an internal hard drive via Windows Explorer.

It keeps giving me a message popup in the bottom right of my taskbar:

"Delayed Write Error"
"Windows was unable to savea ll the data for the file G:.  ..... "

I've got to be social for a few hours, I'll start hacking away at it when I return.

Thanks for the tip about the windows ReadFile() thing... the error codes it provides are very informative.

LVL 14

Assisted Solution

wayside earned 50 total points
ID: 16946320
If you don't open using binary mode, when you read a ^Z (control-Z, decimal byte 26) it will be interpreted as the end of file marker.  This might be why your read fails at a certain point - look at your data file in a hex editor and see if there is a ^Z near the spot where the read fails.

In addition, <CR><LF> pairs are translated to a single <LF>, which is probably not what you want either.

> I'm hoping to just read it all in at once and access it in an array

If what you want is to access your file as if it were an array, you might consider opening it as a memory-mapped file. This frees you from doing any input yourself, and you can treat the file exactly as an array.
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!


Author Comment

ID: 16946537
wayside: Sorry I wasn't clear above... I'm opening as binary
wayside: Thanks for the tip about a memory mapped file - I wonder if it's faster to use a memory mapped file or to simply read an entire file into an array and index it via an array?

LVL 86

Expert Comment

ID: 16946561
There is no difference in reading the file with one call compared to opening it as a memory mapped file, the result is the same.

Author Comment

ID: 16946693
ok thanks, as soon as I get back home I'll hack away at it
LVL 17

Expert Comment

ID: 16947018
> It may be a hardware issue

You might want to check out the Microsoft patch recommended in the following thread:

Author Comment

ID: 16947074
Hey i'll try that out when I get home. It'd be nice if that fixed the problem... I also suspect the directory the files are stored in is corrupt after reading that thread. I will have to "scan-disk" it.

LVL 14

Expert Comment

ID: 16947719
>  There is no difference in reading the file with one call compared to opening it as a memory mapped file,

I don't think this is necessarily true, it can be faster, it depends on what else is going on in the program, how the memory is accessed, and how much can go on in the background. At worst, it will be no different than reading the entire file in one read, but it may be faster. You'd have to benchmark the app to see.


Author Comment

ID: 16947972
I kinda figured they'd be about equal

Author Comment

ID: 16948232
Ok I solved the problem by enabling write caching on my hard drive... which seems counterintuitive... but when you're doing with millions of lines of code and two separate things like USB and HDD transfer are expected to work properly on the first try, who knows what works?

Thanks for the tips

Author Comment

ID: 16948235
Again, to restate for anybody stumbling on this thread:

My problem was an incorrectly working hard drive.

The most common solution to this problem is to verify that you are opening the file with ios::binary

LVL 17

Expert Comment

ID: 16948948
Enabling write caching sounds counterintuitive as you say. If I was you (and could spare the time to do it), I'd follow this up in the hardware TAs here at EE, and see what the search facility comes up with on "delayed write error" - i.e.

Here's my guess: Write-caching is now causing your HDD to need to do fewer seeks for writing and whatever the fundamental problem is that you have, write caching is making it occur less often. Defragmentation may help too. My guess is that during the delayed write error event, I/O is blocked and reads fail. It would be better if you could address the fundamental problem. The hardware experts are the ones to ask, but certainly you should take a look at your event log to see if you are still getting delayed write errors albeit less often.

Author Comment

ID: 16949598
Ok will do thanks rstaveley, much appreciated

Featured Post

Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Abstract Express Replacement Software 12 61
moto 4g plus phone how to delete videos and check space used 13 87
Best way to encrypt a xls file 8 64
Esxi host upgrade 16 99
Workplace bullying has increased with the use of email and social media. Retain evidence of this with email archiving to protect your employees.
Invest in your employees with these five simple steps to improve employee engagement and retention.
Using Adobe Premiere Pro, the viewer will learn how to set up a sequence with proper settings, importing pictures, rendering, and exporting the finished product.
The viewer will learn how to successfully download and install the SARDU utility on Windows 7, without downloading adware.

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question