Solved

Strange error before even WinMain gets called

Posted on 2003-11-03
23
548 Views
Last Modified: 2010-05-18
Hi,

I noticed this a while back already when stepping through the code starting in my WinMain entry point. I have a watch set at @ERR and when I enter the debugging mode and the program starts in WinMain it already displays error 6 (The handle is invalid). Well I never really cared much about it as it didn't seem to bother how my program ran. However today it did caused some trouble. I couldn't get some bytes written to a file and I had no idea what I was doing wrong. After endless searching I decided to use SetLastError(0) just before I write to the file to see if the flag gets changed, and to my surprise it changed to error 6 again. The handle of the file was not INVALID_HANDLE_VALUE however, which I double checked! It was created by CreateFile() which did not return an error, the handle of the created file always is 0x00000fd4 somehow though. Well somehow I think these two oddities are linked and I was hoping if anyone has noticed something like this before and knows a solution, tip, anything :-)

FYI:

- I'm using Visual Studio .NET 2002 (still waiting for the new one)
- It happens in any windows application project (with the default settings)
- I'm not using any library or anything like that which changes the entry point, nor am I overriding the WinMainCRTStartup

Hope you can help :-)
0
Comment
Question by:LuCkY
  • 8
  • 7
  • 4
  • +2
23 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 9673284
Since 'CreateFile()' does not report an error, the 'INVALID_HANDLE_VALUE' is not related to that API call. Get FileMon from http://www.sysinternals.com/ntw2k/source/filemon.shtml and check what file system operations your API performs. The error code you see might be generated from an undrlying system call. Also, '0x00000fd4' looks pretty healthy for a handle, which is nothing but an abstract descriptor - see http://msdn.microsoft.com/library/en-us/dngenlib/html/msdn_handles1.asp ("Give Me a Handle, and I'll Show You an Object")
0
 

Expert Comment

by:jbuote1
ID: 9673631
// Since 'CreateFile()' does not report an error, the 'INVALID_HANDLE_VALUE'
// is not related to that API call.

Hmm,
Yes I'm not an expert, but I beg to differ jkr..
Maybe I don't understand your comment, but
CreateFile() returns a valid handle to the file(object), or if there is an error,
It returns INVALID_HANDLE_VALUE.
If there is an error, then you need to call GetLastError() to see what the error was..
Sounds like that's what LuCky did here...

Am I missing something?

My first thought is maybe a handle out of scope or not using the variable they thought they were?
Just a thought....
0
 
LVL 86

Expert Comment

by:jkr
ID: 9673660
>>CreateFile() returns a valid handle to the file(object), or if there is an error,
>>It returns INVALID_HANDLE_VALUE.

Yes, but INVALID_HANDLE_VALUE != 0x00000fd4
0
 

Expert Comment

by:jbuote1
ID: 9673675
Absolutely agreed....
Just thought maybe there might be confusion to the Author is all...
That's when I thought of Out of scope, or a different variable...
Not trying to step on toes here....
Or CAUSE confusion...

No harm meant....
Thanks.
0
 

Expert Comment

by:jbuote1
ID: 9673703
LuCkY,

Just to be clear,
jkr IS classified an expert...
I am NOT.
So listen to him, Not me....

Just had a thought is all.
0
 
LVL 1

Author Comment

by:LuCkY
ID: 9674335
jkr might be an expert, but you understood what I ment and not jkr ;-) I did indeed mean that it doesn't return INVALID_HANDLE_VALUE and that it doesn't report an error either, which I check with GetLastError() to be sure. I understand the principles of a file handle, though it seemed funny to me that it gives the same handle every time, independant on the parameters I pass the CreateFile.

To clarify a bit more, I created a new project and I have overriden WinMainCRTStartup. The error is gone. This however is of course not desired behaviour as it calls the constructors for the static class etc. and I really don't feel like coding that myself and risking more errors that way. The problem lies in the CRT it seems, so I went searching the source files from the CRT for the WinMainCRTStartup. It provides two functions however from which I'm not even sure if that's the one I'm dealing with, so it hasn't got much use to do anything with that either. If it really is the CRT that has a bug, then I wonder if someone else might have noticed it before. I'll try compiling the application in Visual Studio 6.0 later and inform you if it works then.
0
 
LVL 86

Expert Comment

by:jkr
ID: 9674371
>>but you understood what I ment and not jkr ;-)

Hmm, I still am pretty sure that if 'CreateFile()' does not return 'INVALID_HANDLE_VALUE', it is not complaining about a problem :o)
0
 
LVL 1

Author Comment

by:LuCkY
ID: 9674452
I just compiled it with Visual Studio 6.0 and it gives the same error. Well I though that could make sense as both Visual Studio 6.0 as .NET load the source files from the Platform SDK before the provided source files with Visual Studio. So I deleted all references to the Platform SDK and I still have the error. I guess it must have something to do with the file operations then.

I tried writing the file two different ways too, with WriteFile/CreateFile and fopen/fwrite. Both didn't work however.
0
 
LVL 1

Author Comment

by:LuCkY
ID: 9674499
>>Hmm, I still am pretty sure that if 'CreateFile()' does not return 'INVALID_HANDLE_VALUE', it is not complaining about a problem :o)

I know and I didn't say you don't, it's just that I tried that because I want to know 100% sure that isn't happening. I mean what is happening now isn't suppose to happen already, so I don't just take it what MSDN says in this case :-)

I downloaded filemon and ... well first of all I was amazed by the amount of traffic :-) Within a minute 10's of thousands queries, read's and write's were being made while I don't even have that much programs running. When I finally figured out how to just reduce it to logging the stuff my program does I have no idea what to do with this information however. I see that it makes a couple of reads and writes and that's all just nice and all, but what can I learn from it, which could possibly lead me to a solution of this problem?
0
 
LVL 1

Author Comment

by:LuCkY
ID: 9674525
>>Since 'CreateFile()' does not report an error, the 'INVALID_HANDLE_VALUE' is not related to that API call

I never said that by the way, you must have misunderstood me. It is WriteFile that gives the error and not CreateFile

>>After endless searching I decided to use SetLastError(0) just before I write to the file to see if the flag gets changed, and to my surprise it changed to error 6 again

That's what I said

>>The handle of the file was not INVALID_HANDLE_VALUE however

That's what I said about the response on the error as error 6 implies that I pass an invalid handle to the WriteFile function
0
 
LVL 1

Author Comment

by:LuCkY
ID: 9674536
Oh and before you go say that there are more invalid handles than INVALID_HANDLE_VALUE, yeah I know... :-) That's why I remarked that I found it odd to get 0x00000fd4 as file handle everytime I call CreateFile
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 16

Expert Comment

by:_nn_
ID: 9674653
Let's go a step forward : since you proved it's probably a problem with the CRT, can you tell us what version you're using. If you're linking dynamically, I'd suggest to test linking the CRT statically and see what is happening. Rationale : suspecting that some installed software may have goofed the CRT DLLs on your box (or trying to eliminate that remote possibility)
0
 

Expert Comment

by:jbuote1
ID: 9674931
LuCkY,

First off, jkr DID understand what you meant....
After further reading he said that "Since CreateFile doesn't return an error, then INVALID_HANDLE_VALUE doesn't apply to that API"  (to paraphrase)
He meant that since CreateFile didn't give you the error(which it can) then your problem is NOT with CreateFile()... I Totally Agree with jkr on that.....
Your problem IS elswhere.

But now your dealing with the CRT stuff....
My background is in VERY Basic stuff...
I don't write gui's or anything Like that.
I intercept API's, and do SCADA systems...

My Personal advice is to listen to folks like jkr, and take it wisely...
CRT Stuff, well phrased another way, I might know, but I don't...
I'm NOT an expert in what your problem is... Sounds like jkr is...
My only thought was if a class, or other function opened the handle, then another class or function may try to write to that handle, but the proper handle was never passed correctly...
I can't see your code to know...

Sounds to me like the variable that holds the handle isn't global, or even a class member, but rather local to each function....

Now that's a BIG stretch not knowing your code, so AGAIN, I defer to those such as jkr to help you the best...

0
 

Expert Comment

by:jbuote1
ID: 9674971
Oh, and CreateFile WILL give you the same handle over and over...
It uses the latest avaiable handle...
Has to do with memory, allocations and all that stuff that jkr could probably describe better if your interested.
If that happens to be 0x00000fd4 then that' s the available handle

I can't count how many times in debug I get 0x00000120 as a handle EVERY time....
0
 

Expert Comment

by:jbuote1
ID: 9675052
You know,
My intention was to prevent confusion, however I seem to have created it..
My sincere apologies to ALL for that.

Also, Sounds like _nn_ has a real good thought too....
Please everyone,
I'm sorry to have caused an issue here...

Joe
0
 
LVL 86

Expert Comment

by:jkr
ID: 9675787
Hum, let's take an unusual approach - could you post the code? :o)
0
 

Expert Comment

by:jbuote1
ID: 9676331
I have to agree... Can you post the code?

jkr,
Am I making ANY sense at all to you?
I just joined, and would like to help not hinder....
jbuote1@cox.net
since it seems off topic.. Not good here....
No offense meant.
0
 
LVL 48

Expert Comment

by:AlexFM
ID: 9676347
Hmm,
I am not expert but I think this is a good idea.
0
 
LVL 1

Author Comment

by:LuCkY
ID: 9676766
#include <windows.h>

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, PSTR szCmdLine, int iShowCmd)
{
      return 0;
}

Thats all :-D

however if I add this:

void WinMainCRTStartup() {
      STARTUPINFO startInfo={sizeof(STARTUPINFO),0};

      GetStartupInfo(&startInfo);

      ExitProcess(WinMain(GetModuleHandle(NULL), NULL, GetCommandLine(), ((startInfo.dwFlags & STARTF_USESHOWWINDOW) ? startInfo.wShowWindow : SW_SHOWDEFAULT)));
}

the error is gone (but that's only the error that is already there before I even enter WinMain I told you about). If I step through the code in the debug version and am at the bracket (the '{' of WinMain) it already displays @ERR == 6. This doesn't mean the CRT code is goofed, which I think it isn't (as I used different versions of it (ie the platform SDK one, the visual studio 6.0 one and the visual studio .NET one). I just suspect in the CRT code they write some stuff as well which is not working, which causes the error register to display 6.

The code which is actually failing is this:

#ifdef _DEBUG

HANDLE hLogFile;

int __cdecl ReportHook(int nReportType, char* szMsg, int* pnRet)
{
      DWORD dwBytesWritten = 0;
      DWORD dwBytesToWrite = strlen(szMsg);

      SetLastError(0); // Reset

      // Write it to the log file
      if (hLogFile != NULL)
            WriteFile(hLogFile, szMsg, dwBytesToWrite, &dwBytesWritten, NULL);

      // Error register now is 6 again

      // Just do the default stuff
      return FALSE;
}

#endif // _DEBUG

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, PSTR szCmdLine, int iShowCmd)
{
#ifdef _DEBUG
      // Hook the _CrtDbgReport function
      _CrtSetReportHook(ReportHook);

      // Create a error log to write debugging information to
      hLogFile = CreateFile(_T("error.log"), GENERIC_WRITE, FILE_SHARE_READ, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
      ASSERT(hLogFile != INVALID_HANDLE_VALUE);

      // Tell the debugger to check for memory leaks
      _CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF);
#endif // _DEBUG

      // Create memory leak
      int * nLen = new int[1000];

#ifdef _DEBUG
      // Close the created handle to the file
      CloseHandle(hLogFile);
#endif // _DEBUG

      // Exit application
      return 0;
}

You probably ask why on earth do I use the debug hook to simply write the stuff to a file, well I'll tell you, because I tried. I used _CrtSetReportMode(_CRT_WARN, _CRTDBG_MODE_FILE) with _CrtSetReportFile(_CRT_WARN, hLogFile), but it didn't work either... You know what is fun too (which probably explains where it is going wrong), if I use

_RPT0(_CRT_WARN,"Why don't you work?");

It just writes the message with no problem! In fact if I ask the CRT to log ASSERTS to a file it works like a charm. It just seems that the memory leak warnings can't get written. Well I took a look at those strings and they look pretty normal. The first one is exactly "Detected memory leaks!\n". So I tried doing this:

_RPT0(_CRT_WARN,"Detected memory leaks!\n");

And guess what, it works... I am sure that it actually reaches the hook as I place a breakpoint inside it. I'm totally goofed by this one and I will let you experts break your brains on this one :-) If you need more information/code whatever just ask.
0
 
LVL 16

Expert Comment

by:_nn_
ID: 9676971
Everything is plain normal : you close the hLogFile handle before returning from WinMain(). The ReportHook() is called _after_ this, at the closing brace, and naturally, hLogFile is now invalid.
0
 
LVL 1

Author Comment

by:LuCkY
ID: 9677138
Doh! You are totally right, that's just plain stupid :-) What remains though is that @ERR is at 6 before WinMain gets called. What's up with that? This is a problem inside the CRT I suppose...
0
 
LVL 16

Accepted Solution

by:
_nn_ earned 500 total points
ID: 9677190
Not really. It may just happen that a non-fatal error occured in the startup code. On my system I sometimes noticed a 126 (module not found) once, and next time I ran the same program (without changing anything), it was a 0. I wouldn't bother much about it.

Well, I investigated a bit further : it seems these errors 6 (there are actually 3 occuring before entering WinMain) come from the _ioinit() function. When the CRT tries to initilize its stdin, stdout and stderr file handles. It happens only for binaries compiled for the windows subsystem (because there is no console).

Hope it explains.
0
 
LVL 1

Author Comment

by:LuCkY
ID: 9677820
I can't believe that I made such a stupid mistake :-) I never had problems with the error at 6 indeed, so I guess I'll simply live with it. I'll take a look at the CRT code when I'm home, so I understand what is going wrong. Thanks for your helpfull insights :-)
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

When writing generic code, using template meta-programming techniques, it is sometimes useful to know if a type is convertible to another type. A good example of when this might be is if you are writing diagnostic instrumentation for code to generat…
Errors will happen. It is a fact of life for the programmer. How and when errors are detected have a great impact on quality and cost of a product. It is better to detect errors at compile time, when possible and practical. Errors that make their wa…
The viewer will learn how to pass data into a function in C++. This is one step further in using functions. Instead of only printing text onto the console, the function will be able to perform calculations with argumentents given by the user.
The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now