Solved

new operator causes user breakpoint

Posted on 2001-06-12
30
2,187 Views
Last Modified: 2008-02-20
Hi!

I'm writing an application where I use a lot of objects (classes). Somtimes I get a user breakpoint (int 3) when I use 'new' to create a new object.
First I thourght that I was out of heap space, but if that was the case I would just get a NULL-pointer.
The breakpoint occours in ntdll.dll at address 0x77F9EEA9.
My debugger writes:

HEAP[app.exe]: HEAP: Free Heap block 1815380 modified at 1815b44 after it was freed

My guess is that 'new' tries to free some memory, but why?

I don't expect any of you to give an exact answer - just some help that can lead me to a solution.
0
Comment
Question by:toe_dst
  • 9
  • 6
  • 4
  • +8
30 Comments
 
LVL 4

Expert Comment

by:prashant_n_mhatre
ID: 6182052
I also observed that...
0
 
LVL 86

Expert Comment

by:jkr
ID: 6182112
This seems to be related to a memory leak - you might want to check out http://support.microsoft.com/support/kb/articles/q268/3/43.ASP ('Umdhtools.exe: How to Use Umdh.exe to Find Memory Leaks')
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6182143
> This seems to be related to a memory leak

maybe the opposite, maybe whatever you are using to *hold* the pointer has been freed.  That error implies something like that.

It would imply you are doing something like:

int** pointers;
pointers = new int*[10];
...
delete [] pointers;
...
pointers[2] = new int;

although, obviously not so trivially.
0
 
LVL 2

Expert Comment

by:kmv
ID: 6182204
>First I thourght that I was out of heap space, but if that was the case I would just get a NULL-pointer.
No. If compiler wrote according C++ standart (as MSVC) you get exception ('Can`t allocate memory' or smth like this).
What compiler do you use?
Could you post constructor code (and possible constructors of your base classes)?

Best regards.
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6182238
> No. If compiler wrote according C++ standart (as MSVC) you get exception

MSVC does not handle out of memory exceptions according to the C++ standard.  MSVC returns a NULL pointer by default but this can be changed using the set_new_handler function (so this is more or less standard old C++ behaviour).

The C++ standard says that a std::bad_alloc exception should be thrown on failure of new.

The error message appears to be fairly specific - overwriting freed memory.  An language exception would behave differently.
0
 
LVL 2

Expert Comment

by:kmv
ID: 6182301
About exception. I use MSVC 98 sp 5. I tried to allocate big amount of memory and get exception. I didn't changed standart handler function. MSDN says that NULL pointer is returned but I trust to my hands and my eyes.

>>The error message appears to be fairly specific - overwriting freed memory.  An language exception would
behave differently.
I agree with you. Sure I could say 'it's a memory leak' or smth like this. But it is done by previous peoples. So I asked you to write code which cause problem. Your eyes may not find error, may be other programmers can help you find bug.

Best regards.
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6182384
> About exception. I use MSVC 98 sp 5. I tried to allocate big amount of memory and get exception

that is probably because you are passing MSVC and oversized integral constant - that does generate an exception.  In general that will not be the case, however, try the following program, it certainly works for me with MSVC:

#include <limits.h>

void main()
{
    size_t chunk = 10000000;
    int* i = new int[chunk];
    while (i != 0)
    {
        i = new int[chunk];
    }
}
0
 
LVL 3

Accepted Solution

by:
JackThornton earned 150 total points
ID: 6183268
Memory managers do not constantly run in the background, looping to see if their memory structures have been corrupted. The only opportunity that debug-enabled memory managers have to test the integrity of their structures is when you make a call into the memory manager.

That being said, when you called "new", that was the first opportunity that the memory manager had to check for over/underwrites etc. since the bug occurred. The bug does not happen with the "new", the debug version of "new" merely calls on other code to check the heap. Your overwrite of freed memory occured an indeterminate amount of time prior to your call of new.

Take the message to heart and search for places where you may be writing to an already freed block of memory (otherwise known as a dangling pointer). It may help to "fencepost" the problem by using memcheck (see CRTDBG.H) to explicitly invoke heap validation without actually allocating or freeing memory. This will work well if you're not multi-threaded; if you are multi-threaded, finding the problem can be a little harder.

I'm going to look for some code of mine that may help you pinpoint your problem. If I find it, I'll repost.

- jack
0
 
LVL 3

Expert Comment

by:JackThornton
ID: 6183372
I don't want to knock the debug libraries, they have great facilities for checking to see if you've done bad things with allocated memory. The problem with them is that they tell you after the fact. It's like if you have a problem with a burglar, so you put a piece of string across your door. When you come home, you can see the string is missing which tells you that the burglar has come, but it doesn't help you catch the burglar. What you really want is a burglar alarm to catch the bad guy in the act.

I had a nasty problem like this once in a heavily multi-threaded application, where I had a publish-subscribe mechanism and once forgot to unsubscribe before deleting an object. Another thread would come along every 200ms and write through the pointer, which gave interrmittant symptoms seemingly unrelated to the actual bug. So I wrote a "burglar alarm" for catching writes to already freed memory:

http://www.cetasoft.com/JackThornton/code/TestAllocator.h
http://www.cetasoft.com/JackThornton/code/TestAllocator.cpp
http://www.cetasoft.com/JackThornton/code/CriticalSection.h

It is a replacement memory manager that you use just long enough to catch the culprit in the act. It maintains a very large range of memory and allocates pages at a time regardless of how small the request is. When the block is freed, the memory underneath it gets decommitted, so any furthur writes through the freed pointer cause an immediate page fault (exception 0xC0000005). It can also be set up to leave either the page before and/or after the allocated block uncommitted so you immediately catch over/underwrites. Catch this exception in the debugger, and you immediately see who is using the bad pointer.

hope this helps

- jack
0
 
LVL 3

Expert Comment

by:JackThornton
ID: 6183678
BTW - to use this you should include TestAllocator.h in your stdafx.h file so that all mallocs/frees/news/deletes get replaced. You can then turn the allocator on and off by declaring the USE_TESTALLOCATOR macro in your project settings (C/C++, General, Preprocessor definitions) or explicitly in the stdafx.h file.

- jack
0
 
LVL 2

Expert Comment

by:MadYugoslav
ID: 6184981
You need to check all of Your constructors and destructors. This is only possible if You post some code.
0
 

Author Comment

by:toe_dst
ID: 6186862
Hmm...it seems that I have solved the problem by making my own new and delete operators:

inline void *operator new(unsigned int size)
{
    return HeapAlloc(GetProcessHeap(),0,size);
}
inline void operator delete(void* p)
{
    if(p)
        HeapFree(GetProcessHeap(),0,p);
}

I dont't know if this is the best way of solving it, but it seems to work.
I am just a little bit affraid that there is still a bug in my code.
Anybody knows what is going on?
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6186887
> Anybody knows what is going on?

Almost certainly there is still a bug in your code.

By defining your own new/delete you have bypassed the error checking that new/delete do when they are executed in a debug build.

It is most likely that building the program in release mode would have had the same effect.

As JackThornton said, this error almost certainly due to the fact that you are prematurely freeing memory somewhere (or else something with a dangling reference is accessing it).  

This kind of exception is there to help you...all you have done is disabled it.
0
 

Author Comment

by:toe_dst
ID: 6186901
It seems to be the fact that I'm using HeapAlloc() and HeapFree() instaed of malloc() and free().
Maybe HeapFree checks if the heap block is valid before freeing it.
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6186940
> It seems to be the fact that I'm using HeapAlloc() and HeapFree()

right, because by doing this you are avoiding the checks that the debug version of the code makes, look at this link:

http://msdn.microsoft.com/library/devprods/vs6/visualc/vccore/_crt__malloc_dbg.htm

It talks a little bit about the use of the debug version of malloc in your code.  If you are compiling your code with the _DEBUG flag (normal for a debug build) then malloc will be replaced by malloc_dbg in your code.

You are just avoiding the issue by doing this...
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:toe_dst
ID: 6187010
>Almost certainly there is still a bug in your code.

I suspected that.
I've been trying to locate the bug (or memory leak or whatever it is) for days, but the exception only shows up sometimes (in what seems to be random places).
Well, back to debugging...
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6187050
> but the exception only shows up sometimes (in what seems to be random places).

this is the classic memory bug.

The approach you take may depend on the type of program you are debugging.

If you have money to spend on this, you might consider a tool like Rational's Purify, Numega BoundsChecker or ParaSoft's Insure++.  They can all help with this kind of thing.

The only other option is careful analysis.  Start checking memory allocation, deallocation and access (especially via loops to arrays).

You could consider making use of the debug heap that malloc_dbg provides and put checks in your code that check memory that is about to be accessed before it happens.


0
 
LVL 3

Expert Comment

by:JackThornton
ID: 6187079
Try using my replacement allocator - it will almost certainly throw an exception at the exact point where your error really is the first time you run it, so you can fix it. After weeks of beating my brains out with my strange random memory problem, and a few hours of writing this malloc/free replacement, it found the error the first time I ran it.  It then took me about 5 minutes to fix my bug.

- jack
0
 

Author Comment

by:toe_dst
ID: 6191134
I think I have located a bug, but I'm not sure.
This is a very simplyfied version of some place in my code:

String GetNewString()
{
    return *new String();
}

void do_something()
{
    String s;

    s = GetNewString();
}

As far as I can see, constructers and destructors are called nicely, but the object created whith 'new' is not destroyed.
Is there another way to do this?
0
 

Author Comment

by:toe_dst
ID: 6191161
I'm not sure I explained that very clearly.
The GetNewString() function is not just a constructor function. In my app it gets a text string.
0
 
LVL 2

Expert Comment

by:MadYugoslav
ID: 6191175
Change in this:

String* GetNewString()
{
   return new String();
}
void do_something()
{
   String* s;
   s = GetNewString();
   del s;
}

or make function to destroy content of alocated String (for example):
void DestroyString(String* a)
{
    del a;
}
0
 
LVL 2

Expert Comment

by:MadYugoslav
ID: 6191181
Change in this:

String* GetNewString()
{
   return new String();
}
void do_something()
{
   String* s;
   s = GetNewString();
   del s;
}

or make function to destroy content of alocated String (for example):
void DestroyString(String* a)
{
    del a;
}
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6191229
Why do you want code like this?

Why don't you use the much simpler?

String GetNewString()
{
   return String();
}

the effect *should* be exactly the same, because in the return statement the String class copy constructor is called (in both cases).  The advantage with this version is obviously that there is no memory leak.

Is this some home grown String class?

0
 

Author Comment

by:toe_dst
ID: 6191232
Yes, I know I can do that, but what I don't get is why my own code seemed to work. The destructor is called, but the object is not destroyed. Isn't that strange?
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6191283
> The destructor is called, but the object is not destroyed. Isn't that strange?

A temporary object is being created somewhere in the code.  That temporary object will be destroyed (hence you may see a destructor call).  However, the object allocated with new will not be destroyed (how can it, there is no corresponding call to delete).
0
 
LVL 9

Expert Comment

by:jasonclarke
ID: 6191334
Try this example, it will show you how a temporary is created with code like yours:

class String
{
public:
    static size_t count;

    String()
    { id = ++count; cout << "String::Constructor Call, id = " << id << endl; }
    String(const String& rhs)
    { id = ++count; cout << "String::Copy Constructor Call, id = " << id << endl; }

    String& operator=(const String& rhs) { return *this; }
   
    ~String() { cout << "String::Destructor Call, id = " << id << endl; }

    int id;
};

size_t String::count = 0;

String GetNewString()
{
   return *new String();
}

void do_something()
{
   String s;

   s = GetNewString();
}

void main()
{
    do_something();
}
0
 
LVL 1

Expert Comment

by:ua1zcl
ID: 6203641
As i can remember, operator new is using malloc
in C++ support library, when memory is getting from
application heap of reserved pages. I had
similar error and spent lot of time. Finaly i found
that only 1 byte wrote out of block overwrites
system (or runtime support) memory service information.
And there no error at a moment when curruption uccured,  
but later, when i called "new" or malloc again.
 
0
 
LVL 3

Expert Comment

by:romanm
ID: 6270040
I've encountered this problem when the application uses an incompatible run-time dll.
that is if your application is in debug and is linked to a release run-time dll, etc...
0
 
LVL 11

Expert Comment

by:griessh
ID: 6817253
I think you forgot this question. I will ask Community Support to close it unless you finalize it within 7 days. Unless there is objection or further activity,  I will suggest to accept "jasonclarke, JackThornton" comment(s) as an answer.

If you think your question was not answered at all, you can post a request in Community support (please include this link) to refund your points.
The link to the Community Support area is: http://www.experts-exchange.com/jsp/qList.jsp?ta=commspt

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!
======
Werner
0
 
LVL 5

Expert Comment

by:Netminder
ID: 6833342
Per recommendation comment force/accepted by

Netminder
Community Support Moderator
Experts Exchange

jasonclarke: points for you at http://www.experts-exchange.com/jsp/qShow.jsp?ta=cplusprog&qid=20272071
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Many modern programming languages support the concept of a property -- a class member that combines characteristics of both a data member and a method.  These are sometimes called "smart fields" because you can add logic that is applied automaticall…
Container Orchestration platforms empower organizations to scale their apps at an exceptional rate. This is the reason numerous innovation-driven companies are moving apps to an appropriated datacenter wide platform that empowers them to scale at a …
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now