Output filestream corrputing global pointer

Posted on 1998-08-15
Last Modified: 2010-04-01
I'm using Watcom C/C++ 10.6, programming under a flat memory model with DOS/4GW.

The program originally allocated some 5000 or so objects each approximately 1K in size.  Each successive run of the code reads in the old saved files creates typcially an additional 10-200 objects of approximately the same size (progressive runs typically increases the number of objects allocated, so the first run added another 10 or so, the next one another 15-20, and so forth, this peaks off at around run 10, then tapers of slowly), which are then written back to disk for the next run.  Everything was working fine until the 20th run of code (where I'm now at somewhere around 12000-13000 objects).  It makes no difference if I run the code from a windows 95 dos box, from a dos boot, from the debugger.

Within my code I have the following lines

items * item = item_list;  // item_list is a global pointer to a double linked list of items
ofstream report; ("report.txt", ios::append);
report << item -> number () << endl;    // this is the line that is causing the problems
report.close ();

The compiled code (w/ debugging info) is just under 1 meg (if this makes any difference) - not very practical for me to list the entire source code - but I believe I have given all the pertinent code anyway.
number () is an inline function for class items - which reads short number (void) {return item_number;}
item_number is of course a member of items and is of type short

Although my initial impression for the cause of this is to think memory link I am having a hard time believing that this is the cause for the following reasons:

1) The memory allocation for all of the objects is fairly straight forward - allocate 1K (approx) of memory, load the info from disk, and then link the item to the previously loaded object.  Repeat this process until all objects are loaded.
2) I found one memory leak very early on.  But it wasn't a memory leak of writing to an unitialized pointer, but rather a failure to deallocate memory at program finish.
3) When stepping through the debugger I switched over to the assembly mneumonic (sp?) view, and stepped through the report << item -> number () << endl; line.  I stepped into the compilers filestream write functions and find the following - the information is written, then as the code begins to ret out of all the called routines I come across the following
sub esp,eax
eax at the time is set to 0x00000200
esp - I forget the exact value, but is something like 0x0045D80C
As soon as I process over that line esp is decremented as expected, but suddenly my globabl pointer item_list (which initially has a value of something like 0x0045D5AC) gets mangled to 0x18B8845A (or some such - I forget exactly what the LSB comes out to).  
A few lines later esp gets changed again, and my pointer is further mangled.

The following items have changed in the code since the run where it worked and the run where it didn't.
Added some ,'s to some of the output - but all of this extra ,'s in the output come later in the code so shouldn't have any direct effect on the problem from my softwares standpoint (bug in the compiler maybe?)

I also added the following to one of the member functions of class item
if (!strcmp (name, "externally generated string"))
code to single out the operations performed in a particular case.  This section of the code is not iterative - that is, it only gets run once for a given program run.

When this crashed the code I alterd the if statements above to be
#ifdef OLD

This solved the problem for a little bit, until some 200-300 lines later in the code I have
ofstream report; ("report.txt", ios::append);
report << formatted_string << endl;

formatted_string is a global string of 10000 bytes in length, that gets build dependent on various parameters and so forth.  in the particular case where it crashed it is null terminated at around 1500 characters.

The only difference with this crash and the previous one - is it corrupts a 2nd global pointer to a second double linked list.  It does so again by the occurance of sub esp,eax after writing the string out to the file.

Other notes that may help - I have run the code on a 2nd set of data that starts at just over 10,000 objects, and it runs fine (I haven't run it on this set of data more than twice now - the program run is quite lengthy).

At the point the original set of data crashes I've created perhaps 30 of the 100+ objects that will get created this run.

I've also double checked my open files.  report.txt is the only file open at the time - all others have been closed.

I have to get this resolved as soon as possible.

Am I overlooking something in my code that I should double check?  Is it a compiler bug? (I couldn't find anything on PowerSofts web site - but part of their support pages were down).  Do I have a bad memory chip?  processor?
Question by:bbangerter
  • 4
  • 3

Author Comment

ID: 1170389
One more thing.  If I remove the offending line (aka, comment it out) the code seems to run fine (at least up to this point - it is about 40 minutes into is 1 and 1/2 hour run).  The problem is my report file is going to be missing some data (a very small amount of data - but I'd still like for it to be there).

Other notes - the processor I have is an AMD K5 PR100 (if it matters)
LVL 22

Expert Comment

ID: 1170390
the code

sub esp,eax
eax at the time is set to 0x00000200

is making room on the stack.  (Stack grows down, so you subtract to put things on the stack) Most likely it is adding a large object to the stack.  This should not cause a problem.  This cannot corrupt global memory unless the stack runs into the global memory and that is not possible in win32 (it is possible in other environments).  Are you in win32?  In win16 that could be what is happining.

If you wanted to send me the code and files, I could look at them (probably tommorrow.)
LVL 22

Expert Comment

ID: 1170391
Okay I read the whole thing twice and on the 3rd time see that you are in a flat model dos program.  I don't know how the stack is handled there,but It is possible that your stack is growing down into your global memory.  Can you compile this as a win32 console application?  That will detect if your stack collides with another segment.

Note you can test to see if that is what is happening.  Take a look at the address of the stack pointer  Is that at about the same address as the global memory pointer that is being corrupted?  Note you aren't comparing to the stack pointer to the location pointed to by the pointer, but the location that the pointer is stored in.  If the two are even close (within a couple hundred bytes, say) then you have a problem.
Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.


Author Comment

ID: 1170392
Uh yes, sorry about that.  32-bit DOS application.

Ok, significant registers just before ptr corruption
EAX 0x0200
ESP 0x0045D7E8
Where item_list is stored - 0x0045D5AC

So if I've done my math correctly they are 482 bytes between the top (bottom?) of my stack and the location of my item_list ptr.  After the sub esp,eax
ESP is of course 0x0045D5E8, a mere 150 bytes between the two.
No pushes or pops have happened when the ptr is corrupted though, so I don't understand why I lose my pointer (admittadly though I understand stacks conceptually - I don't understand the internal workings of one within the registers of the CPU).

With some very minimal tweaking I compiled it as a console app and it seems to run fine.
The only problem with this is I lost my DOS graphics.  I only use these the first time the program is run on a data set anyway, so I can work around this if need be, since the problem doesn't appear to crop up until several runs later, but would prefer not to have to do it this way of course.  So indications seem to indicate I'm mangling my globabls through an overgrown stack.  The real question now is how to solve this problem.  I don't think I have any way of specifying a larger stack space, but I'm looking into my compilers docs to verify this.

Would shrinking the size of my executable code put my globals at a lower memory location?  There are a number of things I could do to shrink the code size.  If this is the solution though, then how come my code doesn't crash with the first run on a data set?

Most of my allocated objects don't require the full size, and I could trim a lot of them down if need be, but these shouldn't affect my stack or code space allotment anyway as these are being stored on the heap.
LVL 22

Accepted Solution

nietod earned 500 total points
ID: 1170393
>> Uh yes, sorry about that.  32-bit DOS application.
You said it.  I just missed it--twice.

>> So if I've done my math correctly they are 482 bytes between the top (bottom?) of my stack and the location of my
>>     item_list ptr.  After the sub esp,eax
>>     ESP is of course 0x0045D5E8, a mere 150 bytes between the two

That is too close.  The stack is constantly growing above what the debugger shows as the stack top.  That is because additional information is pushed onto the stack as the debugger is entered and exited and as interrrupts occur.   That 150 bytes is problaby getting used and even if it is not, there are probably other globals that are getting wiped out.

>> Would shrinking the size of my executable code put my globals at a lower memory location?
Probably and the stack would probably just start lower.  So there would be no benefit.

>>The real question now is how to solve this problem.  I don't think I
>> have any way of specifying a larger stack space,
You probably do.  The other option is to limit your stack usage.  I can discuss that if you aren't familiar with what sort of things could be using the stack space.

>>Most of my allocated objects don't require the full size, and I could trim a
>> lot of them down if need be, but these  shouldn't affect my stack or code
>> space allotment anyway as these are being stored on the heap.
No that doesn't sound like it would help.  Look for highly recursive procedures that can be made non-recursive (Or whose stach usage could be improved.)  Look for local arrays or other large data structures that can be handled differently.
With more info about the code and the nature of the code, I might be able to provide more concrete suggestions.

Author Comment

ID: 1170394
I figured out how to expand my stack, it was defaulted at 4k, so I simply doubled it and that will give me plenty.  There are some recursive functions but those are limited in their iterations.  As to why it crashed on this particular run as opposed to not crashing with my second data set, is because there are several dozen functions that may be called dependent on external data.  This particular run called a function that hadn't been used in any previous runs, and it required a bit more stack space for local variables.

Although I don't think the question itself is worth the 500 points, the quick response you have given certainly has been for me.  I was able to run the code saturday night as a window console app to get by for the time, but now have it fully functioning under DOS.  Incidentally running it as a window console pointed out a few places where I check memory on invalid pointers as well, so probably eliminated potential furture bugs as well.  Thanks for the help.

LVL 22

Expert Comment

ID: 1170395
I would think 8 K is small for a stack in a 32 bit application.  So be prepared to raise it again.

I would agree that 500 points is very excessive, but hey you were the one who assigned the points!  : - )  I ussually ask a low point value and then raise the points if I feel experts are doing work that warrants it..

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Unlike C#, C++ doesn't have native support for sealing classes (so they cannot be sub-classed). At the cost of a virtual base class pointer it is possible to implement a pseudo sealing mechanism The trick is to virtually inherit from a base class…
Introduction This article is a continuation of the C/C++ Visual Studio Express debugger series. Part 1 provided a quick start guide in using the debugger. Part 2 focused on additional topics in breakpoints. As your assignments become a little more …
The goal of the tutorial is to teach the user how to use functions in C++. The video will cover how to define functions, how to call functions and how to create functions prototypes. Microsoft Visual C++ 2010 Express will be used as a text editor an…
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.

765 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question