set_new_handler not working?

Posted on 2003-03-24
Medium Priority
Last Modified: 2012-05-04
I'm trying to use a custom new_handler, in case the new operator cannot get enough memory for my object.
When I run the following code, instead of getting the error message I specify when I run out of memory, the program terminates with the output "Failed".

The code is:

#include <iostream>
#include <cstdlib>
#include <new>
using namespace std;

int counter = 0;

void out_of_memory() {
  cerr << "memory exhausted after " << counter
    << " allocations!" << endl;

int main() {
  while(1) {
    int* ia = new int[1000]; // Exhausts memory

This code is from "Thinking in C++ 2nd Ed, Vol 1" by Bruce Eckel, available from www.BruceEckel.com (with slight modifications)

I'm running linux mandrake 9.0, and g++ version 3.2  (3.2-1mdk).

Can anyone explain why my program terminates without my new_handler being called?


Question by:sacka
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 12

Expert Comment

ID: 8195484
I wouldn't use iostream in a memory handler the way you do.

It will typically try to allocate memory and since it is already inside memory allocation (out_of_memory is called before the memory allocation is done) it will just give up and write 'Failed' instead of doing the output to cerr.

Try to use very basic I/O, if you're on unix stick to write(2, msg,strlen(msg)); to do your output inside out_of_memory.

In Win32 you can also use the same write(2,msg,strlen(msg)) since it won't do any new.

You can also try a simple printf(stderr, ..); it might allocate memory but as it is a C function it will use malloc and not new.

Since you are out of memory it indicates that the run time is in trouble. You then have 2 options, either use non-C++ functions to do your I/O OR you can in the early stage of your program allocate a huge block of memory:

char * safety = new char[8192];

Then when you're out of memory you first do a delete [] safety and then you do your cerr etc....


char * safety = 0;

int main()
   safety = new char[8192];

void out_of_memory()
   delete [] safety;
   cerr .....;


Author Comment

ID: 8199324

Thanks for your comment.

I tried using the safety method, with no success.
I also tried using printf, same thing.

Which header files do I need to include to call write(,,)?  I couldn't find documentation that mentioned that function.

Is there something weird about my system?  What results do you get running the code I posted initially?  Or the code with your suggestions?


LVL 12

Accepted Solution

Salte earned 1000 total points
ID: 8202003

I haven't run the code on my system.

To get read() and write() functions you can include <unistd.h> under linux. Not sure what you need to include on your system. If you do a grep or some such on your include dir you will probably find the file that declares the functions.

set_new_handler() is called by new when it request more space from the system and it doesn't get that space. In this case it calls set_new_handler which is supposed to ensure that more space is available or to exit the process but if the new handler returns it should ensure that more space is available so new will try to request again. If it fails this second time, new will either return 0 or throw an exception.

It is odd that the safey method didn't work because in this case you were supposed to have lots of free space that could be used and although cerr << ... ; might allocate some space it shouldn't allocate that much space. However, there might be a possibility of the functions failing because the stack has reached the end.

The point is that the system allocates the stack at some place in your process memory and it typically places the heap just next to it and the heap and the stack grow in opposite directions so they grow towards each other. This means that if you keep allocating more space you might sooner or later reach the place of where the stack is and then it will fail even if you release more memory. It won't fail then due to lack of heap space but due to lack of stack space.

One solution here would be to switch stack (VERY tricky) or to release that memory space that was allocated close to the stack, i.e. the block that is closest to the stack.

Unfortunatley it isn't really very easy to find that block and you typically have to write very machine specific code to handle this. On a pentium processor the stack grows downwards while the heap grows upward and so if you search and find the heap block that has an address very close to your stack and find the heap block that has address closest to the stack then that is the block you should free up before attempting to call any functions to do IO.

On a PC running windows you should also use WriteFile() to stdout from the handler, this will not use much space from your own stack since that will mostly go directly to the system which uses its own stack.

On a Unix system you should use write(1,...) to write the message since that is directly a system call. On my Linux machine the write function is available by including unistd.h.

I am not sure but I suspect that the reason why it fails is not due to lack of heap space (which it has after you deleted the preallocated space) but lack of stack space since the heap has eaten all the memory below the current stackframe. This is a very bad situation since you can't call any function.

However, the C++ run time system is supposed to catch this situation and throw an exception in this situation - or maybe you have to include a specific file <stdexcept> or some such to activate that.


Expert Comment

ID: 8209565
works perfectly under mingw/gcc 3.2 and under redhat 7.2 and gcc 3.2.2.  what you have might be a compiler and/or a lib bug.  try to upgrade to 3.2.2

LVL 11

Expert Comment

ID: 9558706
No comment has been added lately, so it's time to clean up this TA.
I will leave a recommendation in the Cleanup topic area that this question is:

Answered: Points to Salte

Please leave any comments here within the next seven days. Experts: Silence
means you don't care.


-bcl (bcladd)
EE Cleanup Volunteer


Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Unlike C#, C++ doesn't have native support for sealing classes (so they cannot be sub-classed). At the cost of a virtual base class pointer it is possible to implement a pseudo sealing mechanism The trick is to virtually inherit from a base class…
  Included as part of the C++ Standard Template Library (STL) is a collection of generic containers. Each of these containers serves a different purpose and has different pros and cons. It is often difficult to decide which container to use and …
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
The viewer will learn how to pass data into a function in C++. This is one step further in using functions. Instead of only printing text onto the console, the function will be able to perform calculations with argumentents given by the user.

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question