Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win


Handling stack overflow in Linux

Posted on 2004-10-15
Medium Priority
Last Modified: 2012-08-13

I need to handle stack overflow in Linux from programs written in C/C++. I've surprisingly found that it's not not easy... In Windows 2000 I'm using structured exception handling for this purpose, but what mechanism should be used in Linux? Writing handlers for all signals that can be handled didn't help me. The result of program execution is always Segmentation fault.

Thanks in advance,
Question by:Foma
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 12

Expert Comment

ID: 12318014
Hi Foma,
I'm not aware of any tools which prevent stack overflows. You can set the stack limit with "limit" or "ulimit", but you cannot really detect them as such.

What are you putting on the stack which causes an overflow? Normally it's better to revise the initial design than to chase after error symptoms later.



Author Comment

ID: 12318177
Hi Stefan!
Actually, lots of program need to handle stack overflow. For example i'm writing interpreter for a programming language. And of course, a user can write a program in this language that will cause infinite number of function calls, which leads to stack overflow.
Hm, I'm sure Java virtual machine, for instance, handle this problem correctly on any platform, including Linux.

Best regards,
LVL 12

Expert Comment

ID: 12321389
True, but I don't think the Java virtual machine uses the system's stack for calling java methods.

Your interpreter should probably not recursive itself. I'm quite sure it is possible to calculate the maximum stack size.

A simple method is to call
pmap $$

...and then you see the current process's stack size + location. You can then easily add a call to check the stack to each function invocation. Make sure you have a bit of stack to spare when you trigger the overflow exception.

That's a classic for a simple assembly subroutine, something like (GCC's AT&T syntax):
    cmp $_stack_limit,%esp
    ble _overflow

...whereas "overflow" is a C routine which handles the stack overflow itself. You could even have the ASM in a little macro which you call in all the critical functions, as the overhead is minimal.

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!


Accepted Solution

mtmike earned 375 total points
ID: 12374921
Catching SIGSEGV should work, except that signal handlers are executed on the current stack. So, when the stack overflows the signal handler will cause a segmentation fault too. To avoid this, you will have to establish an alternate signal stack using sigaltstack() and pass the SA_ONSTACK flag to sigaction().

Assisted Solution

Sadrul earned 150 total points
ID: 12377291
you can also give libsigsegv a try. here's the URL: http://libsigsegv.sourceforge.net

-- Adil
LVL 12

Expert Comment

ID: 12409108
I strongly discourage catching segmentation faults. Chances are high that you get hundreds of them in a very short time. And you have no guarantee that they are related to your stack problem.

The worst case would be that you're missing a core dump which was created by a NULL reference or something similar, and pass on to the user a totally wrong "stack overflow message".

Author Comment

ID: 12433426
Thank you very much for your responces!
To mtmike:
Thank you. Simple solution and it works. Unfortunately I cannot distinguish stack overflow from other segmentation violations…
To Sadrul:
Nice library. It allows handling exactly stack overflows. It distinguishes arbitrary segmentation fault and stack overflow using some heuristics (I saw it in its source code, but I didn’t manage to look into it yet). Unfortunately, this library is not suitable to me because I need to return from signal handler to program automatically, what is achieved by using sigaction with SA_SIGINFO. But the library uses sigaction w/o this flag for Linux…
To stefan73:
I understand your position, but I can’t agree with you. Yes, in general it’s better not to get segmentation faults at all and write programs in such a manner that you will never get access violation. But sometimes it’s very hard or impossible… Concerning my program, I do my own memory management and SIGSEGV signal (when the address in selected diapason) is normal behavior of the program. When SIGSEGV signal is got I perform mmap call to correct the problem. In other case it can be stack overflow or error in program. I can suspect that it is stack overflow and make an effort to release resources. If during this step SIGSEGV arises, that means error in program and I just terminate the program… I think it not an ideal solution, but it is satisfactory for me

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Have you ever been frustrated by having to click seven times in order to retrieve a small bit of information from the web, always the same seven clicks, scrolling down and down until you reach your target? When you know the benefits of the command l…
The purpose of this article is to demonstrate how we can upgrade Python from version 2.7.6 to Python 2.7.10 on the Linux Mint operating system. I am using an Oracle Virtual Box where I have installed Linux Mint operating system version 17.2. Once yo…
How to fix incompatible JVM issue while installing Eclipse While installing Eclipse in windows, got one error like above and unable to proceed with the installation. This video describes how to successfully install Eclipse. How to solve incompa…
Please read the paragraph below before following the instructions in the video — there are important caveats in the paragraph that I did not mention in the video. If your PaperPort 12 or PaperPort 14 is failing to start, or crashing, or hanging, …
Suggested Courses

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question