Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Exception / stack problem (VC++ 6.0)

Posted on 2004-03-21
20
Medium Priority
?
1,475 Views
Last Modified: 2007-12-19
I need to test what happens when I cause a stack overflow, so I wrote this tiny test:

  __try
  {
    void * pMem0 = _alloca(0xFFFFFFFF);
    memset(pMem0, 0, 0xFFFFFFFF);
  }
  __except(EXCEPTION_EXECUTE_HANDLER)
  {
    MessageBox(NULL, "Exception!", "Exception", MB_OK);
  }

From the theory, I cannot allocate 0xFFFFFFFF on the stack on a 32 Bit machine. But I get no exception in this line ! Without the "memset" operation, the program runs through without any problem. Strange.
So I took the "memset" operation after the alloca, and now I am told "Unhandled Exception in Test.exe". Even more strange! I have an exception handler, I tried alternatively with "normal try/catch", but to the same result. "Enable exception handling" is switched on. I even set the /F0x64 switch just to make sure that I have a small stack, but to the same result.

So I have two questions:
1) Why doesn't my _alloca call cause any exception ?
2) Why is the memset exception not caught by my handler ?
0
Comment
Question by:PC-Alex
  • 9
  • 7
  • 3
  • +1
20 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 10645055
>>But I get no exception in this line !

That is because you are most likely to get a NULL pointer back, that is why 'memset()' leads to a crash.
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10645075
This is an excellent idea, I did not check this yet, because it is not documented in MSDN.

MSDN just says clearly: "A stack overflow exception is generated if the space cannot be allocated."

I'll check it at once!
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10645077
Nope.

pMem0 is 0x0012ff14
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 86

Expert Comment

by:jkr
ID: 10645100
Checked it, too. It seems that there is a lot going wrong here that should not happen. With or without the SEH handler, the program just terminates after 'memset()' - and the stack is completely blown away...
0
 
LVL 86

Expert Comment

by:jkr
ID: 10645106
>>and the stack is completely blown away...

BTW, this is the reason the exception handler does not kick in, since SEH is stack-frame based.
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10645113
Do you have any idea how I can make sure at runtime that my stack is big enough to handle my _alloca() - request ?
0
 
LVL 86

Expert Comment

by:jkr
ID: 10645134
Well, it is difficult to determine the stack size left at runtime, but http://support.microsoft.com/default.aspx?scid=kb;en-us;315937#5 ("HOW TO: Trap Stack Overflow in a Visual C++ Application - Trapping the Exception with __try and __except (Full Solution)") might help.
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10645160
jkr,

thanks for the link. This article helps me in enlarging the stack at runtime, but it does not help protect myself from a failed _alloca call; I need to get into the catch handler to handle the situation in any way, but I cannot reproduce such a stack-overflow situation caused by _alloca.
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10645174
At the moment I think that the guard page is not tested effectively; if I look at the assembler code in chkstk.asm, I cannot see some write operation into the newly allocated area, so I wonder how _alloca will detect that at all ?

Even more strange: The ESP register is not changed at all by the _alloca call. The _alloca result is the ESP value.
0
 
LVL 86

Expert Comment

by:jkr
ID: 10645181
>>The ESP register is not changed at all by the _alloca call. The _alloca result is the ESP value.

That is the way it should be - it will point at the base of the free stack.
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10645201
>>That is the way it should be

Only the second part of the sentence.
Calling _alloca with reasonable values always changes the ESP register (it MUST).
0
 
LVL 86

Expert Comment

by:jkr
ID: 10645207
>>Calling _alloca with reasonable values always changes the ESP register (it MUST).

Yup, that's right.
0
 
LVL 2

Expert Comment

by:guntherothk
ID: 10645427
There is a difference between C++ exceptions and Windows Structured Exceptions. You have to convert windows structured exceptions into C++ structured exceptions. There is a call called _set_se_translator() that does the job. You also have potentially to change a compiler flag to allow asynchronous exceptions. Converting structured exceptions arising from the hardware into C++ exceptions doesn't always work properly unless you've set asynchronous exception mode using the compiler flag /EHa, which is not the default.

How about calling a function recursively that allocates a local array of 10,000 characters. It won't take long for such a function to exhaust the stack, and you'll probably return more elegantly.

void crashme()
{
    char crasharray[10000];
    crashme();
}

This probably simulates more accurately what you are testing for anyway.
0
 
LVL 86

Expert Comment

by:jkr
ID: 10645439
But, how would a SE translator help here?
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10646919
guntherotk,

thanks for your comment. The crashme() example is exactly the same test as is proposed by jkr's link above; but this test does not help me further. I repeat my problem: I need to allocate memory on the stack at some places in my app, and don't use new/delete, for several reasons (performance, potential mem leaks, etc). Sometimes the application crashes randomly at the customer, and I need to find out if this is caused by "out-of-stackspace" situations, where the usage of _alloca is to blame.

So my task is simply: Generate an exception that occurs when _alloca fetches more memory, than is available on stack, and handle the situation.

But I cannot reproduce such a case. As I wrote above, I tried __try/__except as well as try/catch, and nothing was caught, so I do not think that any translation can help (if you still insist that it might help, could you please tell me how, in more detail ?).

I tried /EHa together with /GX, but with no change of behaviour. The combination /GX with /EHs is the same.

To be continued ...
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10646968
... I continued experimentation, thought: I will watch the ESP register, and if it changed, I have just allocated successully.

The new prog looks like this:

void * pMem0 = _alloca(0x000000FF);
void * pMem1 = _alloca(0x0000FFFF);
void * pMem2 = _alloca(0x00FFFFFF);
void * pMem3 = _alloca(0xFFFFFFFF);

I watch it under the debugger, still with a 100-Byte-Stack ( /F0x64 ) . So what happens ?

Line "pMem0" changes ESP from 0012FEFC to 0012FDFC (this is one Byte too much, but roughly behaves as expected)
Next line: ESP changes from 0012FDFC to 0011FDFC (once again: roughly as expected, but one byte too much)
Next line, big surprise: ESP is unchanged, and ... I catch an exception ! First-chance exception in Test.exe: 0xC00000FD: Stack Overflow.

So I have some change in questions:
1) why doesn't _alloca(0xFFFFFFFF) do (or throw) anything ?
2) why can I successfully allocate more than my predefined stacksize (100 Bytes) ?
3) why is always one Byte more allocated than desired ?

Again I analyzed CHTSTK.ASM and think, question 2 might be answered with the fact that _PAGESIZE_ is hardcoded 0x1000, so the /F switch makes only sense with multiples of 0x1000.

Question 3, the one byte, might be the way ECX is handled, saved with a "push" and restored with a "MOV". But I do not exactly understand why this is done so.
0
 
LVL 2

Accepted Solution

by:
Bronek-K earned 1000 total points
ID: 10647380
You cannot allocate memory of size 0xFFFFFFFF . That's because anytime you call _alloca, number you pass as argument gets rounded up to nearest multiple of 4. If you pass anything more than 0xFFFFFFFC, integer overflow happens and effectively you will alloca(0) from stack, which obviously succeeds. About your second question: there is hard-coded minimum size of stack 4096 bytes (this is size of single memory page). If you specify smaller size in compiler options, it will get rounded up to 4096. You can see that with dumpbin tool (dumpbin /headers your.exe). And your third question: this is actually result of round-up to multiple of 4 I described in first point. If you do _alloca(0x000000FE) or _alloca(0x000000FD) you will see that 0x100 bytes got allocated from stack.
Two advices: if your program runs on recent version of Windows, you need to call _resetstkoflw inside __catch block in order to enable handling next stack overflow exception. If you do not do that, your program will fail with next stack overflow exception (instead of passing control to __catch block). You also should not create C++ objects on stack inside __try block, because SEH exception handling mechanism does not guarantee calling destructors for these objects.

0
 
LVL 2

Expert Comment

by:Bronek-K
ID: 10647398
In case you wonder "why this roundup to multiple of 4" let me remind you that x86 is 32 bit CPU, which means that its most natural (and fastest) way of handling data is in 32-bit words, ie. 4 bytes.
0
 
LVL 1

Author Comment

by:PC-Alex
ID: 10655506
Thank you Bronek-K for the answer. Would you please tell me just one thing more ?

I examined the _alloca assembler code in chktsk.asm and want to understand on what command exactly the 4-byte-alignment is done. Is it done implicitly on a call to push or sub or something like that ? Or where else is the 0xFF rounded to 0x100 ?
0
 
LVL 2

Expert Comment

by:Bronek-K
ID: 10655569
If 0xFF is submitted as constant literal (ie. is known during compilation) it will happen during compilation, just before calling _chkstk. You will see in disassembly of your code that 0xFF got replaced by 0x100. If 0xFF is submitted as variable (not known during compilation), then simple calculation is performed before calling _chkstk
  mov  eax,dword ptr [bytes] ; bytes is your variable
  add  eax,3
  and  eax,0FFFFFFFCh
This will rund up number to nearest multiple of 4. BTW: there is no __catch block in SEH, it's called __except instead. Sorry for mistake.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

IntroductionThis article is the second in a three part article series on the Visual Studio 2008 Debugger.  It provides tips in setting and using breakpoints. If not familiar with this debugger, you can find a basic introduction in the EE article loc…
Article by: evilrix
Looking for a way to avoid searching through large data sets for data that doesn't exist? A Bloom Filter might be what you need. This data structure is a probabilistic filter that allows you to avoid unnecessary searches when you know the data defin…
The goal of the tutorial is to teach the user how to use functions in C++. The video will cover how to define functions, how to call functions and how to create functions prototypes. Microsoft Visual C++ 2010 Express will be used as a text editor an…
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.
Suggested Courses

886 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question