We help IT Professionals succeed at work.

file size limit for ifstream? getting a core dump with 68MB input file.

briank
briank asked
on
Medium Priority
829 Views
Last Modified: 2008-02-01
I'm working on a program that reads in a file to get data.  I have no problems with the program until the user-specified file starts to get really large.  When testing with a 68MB text file, ifstream fails on the first attempt (if(!fin) passes, where "fin" is the ifstream variable), but if you give it the same file name a second time around (without quitting out of the program), it core dumps.  To add to the mystery, it doesn't necessarily core dump in the same place each time, and it never cores on anything dealing with the ifstream... it usually cores on something completely unrelated like a simple assignment a few lines down in the code.

Again, it works fine with smaller files (I've tested up to 6MB with no problem), even if you read them over and over, or if you switch files while the program is still running...  

I have the program echoing back a messsage to the user if the ifstream fails with the name of the file it failed on... this message comes up with a '!' appended to the end of the file name (I'm baffled by this) when ifstream fails on this very large file.

The code has been compiled an run on Linux 7.1 and Windows NT4.0, yielding the same results.  The Linux machine has 1GB of RAM, the Windows machine has 96MB.
Comment
Watch Question

Software Craftsman
Commented:
Generally when you get unexpected and inconsistent behaviour as you describe, it is indicative of a memory error such as writing past the end of a buffer, using already freed memory, etc.

How large is the program?  Would it be possible to post some source code?  What compilers are you using?

Commented:
The problem is almost certainly in your code, not fstream.

If you are attempting to store all the data in memory, you might simply be running out of heap space and will need to redesign your app to be less memory intensive.

Commented:
I think you forgot this question. I will ask Community Support to close it unless you finalize it within 7 days. Unless there is objection or further activity,  I will suggest to PAQ at zero points.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!
======
Werner
Force-accepted by
Netminder
CS Moderator