release build: getting an assert on CDialog::DoModal within a dialog-based app


I have a dialog-based app that causes assert whenever I try to do DoModal on any dialog. This happens in the release build only - debug works fine.

A bit of poking around pointed to the source of assert in AfxGetResourceHandle():

    ASSERT(afxCurrentResourceHandle != NULL)

The confusing part is that I call AfxGetResourceHandle() just before doing dlg.DoModal() that is failing, and returned HINSTANCE is fine.

So, something changes from the start of CDialog::DoModal to the point where AfxGetResourceHandle() is called, that I just can't see:

int CDialog::DoModal()
      // can be constructed with a resource template or InitModalIndirect
      ASSERT(m_lpszTemplateName != NULL || m_hDialogTemplate != NULL ||
            m_lpDialogTemplate != NULL);

      // load resource as necessary
      LPCDLGTEMPLATE lpDialogTemplate = m_lpDialogTemplate;
      HGLOBAL hDialogTemplate = m_hDialogTemplate;
      HINSTANCE hInst = AfxGetResourceHandle();

Any help appreciated!!

Who is Participating?
It would have to relate to whatever the UDM handler was doing and and what point in the normal sequence of events that it was being sent.  

In general (and to avoid hard-to-debug message-related errors) I prefer to use PostMessage rather than SendMessage.  If there is any quirky "unfinished business" then posting the message makes it much more likely that things will be handled correctly.  For instance, when setting the current selection for a CListCtrl there are messages flying all over the place for a while and interrupting the sequence can end up in recursion or other problems.  Same thing if messages happen early in the CDialog::Create() cycle.  

Given the nature of this error, I'd say that creation of some control was not complete at the time you sent a message to a handler that expected the dialog and all of its controls to be fully created.

If you defined your UDM using
then you should have no problem of conflicts with standard Windows messages.  If you are sending private messages to another application program of a suite of interacting programs, then you should use values returned by RegisterWindowMessage()

-- Dan
Is the dialog resource included in your release build?
AndyAinscowFreelance programmer / ConsultantCommented:
<I have a dialog-based app that causes assert whenever I try to do DoModal on any dialog. This happens in the release build only - debug works fine.>

?? - ASSERT in release build is a null statement - how can a non-existant line of code fail?
Cloud Class® Course: Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

aquarius003Author Commented:
Yes, dialog resource is included. There are no resource conflicts or anything like that.

Andy: I got the location of the assert when I changed the project settings to produce a PDB with debug information. Maybe ASSERT is not empty for that case??
aquarius003Author Commented:
Actually, afxCurrentResourceHandle is defined as:

 #define afxCurrentResourceHandle    AfxGetModuleState()->m_hCurrentResourceHandle

So, my assert is an access violation within AfxGetModuleState(). Unfortunately, it won't let me step in to see, but it could be that pState (_afxThreadState) is NULL, and being de-referenced, rather than afxBaseModuleState.

      _AFX_THREAD_STATE* pState = _afxThreadState;
      AFX_MODULE_STATE* pResult;
      if (pState->m_pModuleState != NULL)
            // thread state's module state serves as override
            pResult = pState->m_pModuleState;
            // otherwise, use global app state
            pResult = _afxBaseModuleState.GetData();
      ASSERT(pResult != NULL);
      return pResult;
Is this taking place inside of code in a DLL?  If so, make sure you are using:

       AFX_MANAGE_STATE( AfxGetStaticModuleState() );

at the start of any code that may be called by the main EXE.  Some entry points are kinda sneaky...
aquarius003Author Commented:
Thanks for the suggestion, Dan. I already had a similar issue with a DLL, so I know about doing AFX_MANAGE_STATE for every exported function.

However, my current problem has nothing to do with a DLL. There are some functions called by the application itself, from another DLL, but everything related to the dialogs that I'm seeing issues with, is witin the app.

This is really, really annoying, since everything appears to be kosher.
I would recommend calling AfxSetResourceHandle with the Instance handle of your extension dll before calling your Dialog class.  like this

HINSTANCE hOldInst = AfxGetResourceHandle();

CMyDialog dlg;



AndyAinscowFreelance programmer / ConsultantCommented:
PDB files - I'm no expert with them.

DoModal is a virtual function.  That means you could, temporarily, make a copy of the standard CDialog DoModal and create your own CMyDialog::DoModal.  In the copy put some AfxMessageBox calls to track how far it gets before the failure and also what values are present in a second iteration.


then when you know between which lines it fails at
CString s;
s.Format("something is %ld", lSomething);
aquarius003Author Commented:
Mahesh, I tried that already: doing AfxGetResourceHandle() and using the HINSTANCE to do AfxSetResourceHandle() just before calling DoModal. Doesn't make any diff.

Andy: good idea, will try and get back to you.

Thanks !!!
aquarius003Author Commented:
I resolved the issue and it had to do with a message handler added for a new message between two app components.


When I removed sending of this message, WM_MY_BLAH,  to the target window, and replaced it with a direct method invocation, no problems were happening any more.

I initially tried changing the base for user-defined message IDs, but that didn't do much good. I'm still not sure what exactly is happening (the problem seemed very 'slick' - as soon as I'd trace it to one location, it would skip to another, so to speak), since there are other user-defined message IDs that cause no problem.

I've read in a posting that user-defined message IDs can cause issues in release mode, if they overlap with IDs already used by the system, and this led me to poke around them. Not that I understand what's really going on. But hey, at least it's not crashing any longer.

Thanks for your help and suggestions!
aquarius003Author Commented:
Btw, if anyone can point to the connection between the user-defined messages and the crash I was getting, I'd be more than happy to assign points to them. :)
AndyAinscowFreelance programmer / ConsultantCommented:
For custom messages I usually use
const UINT AA_WM_XYZ = ::RegisterWindowMessage(_T("AA_WM_XYZ"));
which guarantees a unique ID for AA_WM_XYZ BUT, very important, any call to RegisterWindowMessage with the same string returns the same unique value - very good for passing between apps.

message map entry

and function
aquarius003Author Commented:
Thanks for the answers! Dan was right on the spot - changing SendMessage to PostMessage solved the issue.

It's much better to understand what's going on, not just having it working. :)

AndyAinscowFreelance programmer / ConsultantCommented:
<It's much better to understand what's going on, not just having it working.>

You do understand the VERY LARGE difference between SendMessage and PostMessage I hope.
aquarius003Author Commented:
AndyAinscowFreelance programmer / ConsultantCommented:
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.