Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

DbgHelp fails with large structures

Posted on 2008-11-07
15
Medium Priority
?
723 Views
Last Modified: 2013-12-14
Hi,

I'm trying to use dbghelp to know the exact contents of a binary file. Everything works fine expect in programs with very large structures. For example, using the following test program

DbgHelp behaves very strangly. It gets the address of the the member variables 0 to 3213 but then gets the address of var_0 and then the calls to SymGetTypeInfo fail.

Am I doing something wrong or this is a bug?

Any help would be welcomed.


typedef struct {
     float var_0;
     float var_1;
     ....
     float var_5000;
} stTest_t;
 
stTest_t stTest;
 
void main()
{
     stTest.var_1 = 2;
}

Open in new window

0
Comment
Question by:goloap
  • 8
  • 7
15 Comments
 
LVL 49

Expert Comment

by:DanRollins
ID: 22914765
What code are you using in the GetType parameter?  What are you passing in the pInfo parameter?
What is error code is returned by GetLastError ?
Have you tried using the SymGetTypeInfoEx function?
0
 

Author Comment

by:goloap
ID: 22923780
I'm using the same code as similar examples on the web (see bellow)

GetLastError returns a random value (really large negative value)

Not yet, looks too much of a pain the setup the structure correctly. But I may have to do it.

// If the symbol has children
if (FALSE == SymGetTypeInfo(SYMBOL_HANDLE, ModuleBase, pSymInfo->TypeIndex, TI_GET_CHILDRENCOUNT, &ChildCount))
{
	CString ErrString;
	ErrString.Format("ERROR: Unable to get type information for symbol: %s", BaseName );
	ThrowException(ErrString);
}
	
if (ChildCount != 0 )
{
	DWORD dwKind = 0;
	// If this is a User Defined Type (UDT) which is a structure
	SymGetTypeInfo(SYMBOL_HANDLE, ModuleBase, pSymInfo->TypeIndex, TI_GET_UDTKIND, &dwKind))
	if (dwKind == UdtStruct)
	{
		int FindChildrenSize = sizeof(TI_FINDCHILDREN_PARAMS) + ChildCount*sizeof(ULONG); 
		TI_FINDCHILDREN_PARAMS* pFC = (TI_FINDCHILDREN_PARAMS*)malloc( FindChildrenSize ); 
		memset( pFC, 0, FindChildrenSize ); 
		pFC->Count = ChildCount;
							
		// Get the info for the type
		SymGetTypeInfo( SYMBOL_HANDLE, ModuleBase, pSymInfo->TypeIndex, TI_FINDCHILDREN, pFC ) ) 
		DWORD i =0 ;
 
		for (i = 0; i < ChildCount; i++)
		{
			// Get Name 
			WCHAR *pName = 0;
			SymGetTypeInfo( SYMBOL_HANDLE, ModuleBase, pFC->ChildId[i], TI_GET_SYMNAME, &pName ); 
						
			// Offset relative to parent
			DWORD dwOffset = 0;
			SymGetTypeInfo( SYMBOL_HANDLE, ModuleBase, pFC->ChildId[i], TI_GET_OFFSET, &dwOffset );
			
...

Open in new window

0
 
LVL 49

Expert Comment

by:DanRollins
ID: 22925698
That "random number" from GetLastError might look less random when displayed in hexadecimal format.  Post it here, or use the "Error Lookup" tool to see if it provides additional information.
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:goloap
ID: 22941193
Made some changes, and now I end up with an error code of 1on the 3214th element of a structure. The weird bit is that element 3213 is fine, the next element returned however is element 0 and then I get the error. It seems that the pFC array gets corrupted.

Sorry for the slownest of my replies, I'm very busy on other projects also, but thanks for your help.
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 22943804
What is the value of ChildCount in the above example?
0
 

Author Comment

by:goloap
ID: 22974087
5001. So everything should be alright.
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 23009669
Just for fun, create an extra-large buffer to hold the response data.  Right now you are calculating the size in your FindChildrenSize variable.  If you increase that to, say 1MB (ridiculously large) do you still have the same error?
0
 

Author Comment

by:goloap
ID: 23014169
Valid idea. Change my malloc call to
TI_FINDCHILDREN_PARAMS* pFC = (TI_FINDCHILDREN_PARAMS*)malloc( 1024*1024 );

but still same error.

This is ridiculous, any good microsoft forum I could ask?
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 23017038
I don't know where else to ask.
If you suspect that the data is getting corrupted, then check your suspicion. Use the memory viewer to look in the buffer before you start to process the data. It should be all zeros (except Count) right before the call and after the call, it should hav the Count and Start value followed by 5000 4-byte values. Look at the area starting at about 12,856 bytes from the start. I'd expect the data to be "eyeball uniform" right up to offset 20,000.
Another angle:
How does the VC++ debugger behave with such a large structure? Does it have any problem displaying all 5000 of the fields?
0
 
LVL 49

Expert Comment

by:DanRollins
ID: 23017326
Another note: In the only examples I could find, they all assumed that there would be a smallish number of fields. One expected at most 256, and the Mat Pietrek article source code:
    http://filezilla.cvs.sourceforge.net/viewvc/*checkout*/filezilla/FileZilla%20Server/source/misc/WheatyExceptionReport.cpp?revision=1.1
assumes a max of 1024. It doesn't even appear to attempt to handle the situation where then value could be larger than that. The same is true with this code that appear to assume there would never be more than 200 (and takes no precautions against that possibility):
    http://www.reactos.org/pipermail/ros-diffs/2008-August/025027.html
Perhaps the guys who wrote these know something that we don't. Or perhaps they think that a structure with 5000 named fields is so unlikely that it's not worth the hassle of supporting...
0
 

Author Comment

by:goloap
ID: 23027508
I've checked the memory corruption bit, and it is not the problem. Since right the memory the data stored in the pFC is wrong after the call to TI_FINDCHILDREN as shows the memory viewer pFC = 0x010A3280 (pFC->Count = 5002 = 0x138a) &pFC->ChildId[3214] = 0x010A3280. You can see that the childId resets to the childId of item 0 for no reasons.

Visual Studio does not have a problem with displaying the large structure.
0x010A0040  138a 0000 0000 0000 0174 0000 0175 0000  `.......t...u...
0x010A0050  0176 0000 0177 0000 0178 0000 0179 0000  v...w...x...y...
0x010A0060  017a 0000 017b 0000 017c 0000 017d 0000  z...{...|...}...
 
...
 
0x010A3270  0dfe 0000 0dff 0000 0e00 0000 0e01 0000  þ...ÿ...........
0x010A3280  0174 0000 0e02 0000 0e03 0000 0e04 0000  t...............
0x010A3290  0179 0000 0e05 0000 0e06 0000 0e07 0000  y...............
0x010A32A0  017e 0000 017f 0000 0180 0000 0181 0000  ~.......¬.......

Open in new window

0
 
LVL 49

Expert Comment

by:DanRollins
ID: 23033156
Good feedback. The more I look into this, the more it appears to be bug in the library code. I'm sure you've read this MSDN article, but I want to post it here as the definitive info on DbgHelp:
    Improved Error Reporting with DBGHELP 5.1 APIs / Matt Pietrek
    http://msdn.microsoft.com/en-us/magazine/cc301692.aspx
Another good article with source code here:
    How to use dbghelp to access type information
    http://www.debuginfo.com/articles/dbghelptypeinfo.html
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
I note that you are not checking the return code from the relevant SymGetTypeInfo call. If it is 0, then you could call GetLastError().
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Just more thrashing about (trying to narrow down the issue):
1) What if you decrease the size of your structure to 3200? Does it work fine then? Increase it and continue to examine the memory dump above. Does it malfunction when there are 3213 fields or does it only malf when you get higher? Does it malf at a different place?
2) What if you have 2000 fields, but each is a two-field UDT? That is, I wonder if the problem relates to the total of all fields (including grandchildren), or just the total children of the structure.
Again, I'm sorry that I don't have an answer, and frankly doing the above tests will not solve anything. But it might generate a lead or at least some info you can pass to Microsoft when you make an official bug report.
-- Dan
0
 
LVL 49

Assisted Solution

by:DanRollins
DanRollins earned 1600 total points
ID: 23033274
As to solving this problem, about the only thing I can think of is to (again) suggest that you give a try to:
   SymGetTypeInfoEx Function
   http://msdn.microsoft.com/en-us/library/ms681350(VS.85).aspx
since it appears to have the same functionality, but goes about it in a different way -- a way which may not impose internal limitations...
Another alternative would be to delve into the complex DIASDK:
   Debug Interface Access SDK
   http://msdn.microsoft.com/en-us/library/x93ctkx8.aspx
which provides a COM interface way to access the debug symbols. For instance:
   http://msdn.microsoft.com/en-us/library/sdes2ybc.aspx
shows c++ source code to enumerate through a collection of fields in a UDT after calling IDiaSession::findChildren
0
 

Author Comment

by:goloap
ID: 23035280
Good eye for the missing GetLastError(). But I had removed it when I pasted my code in the window because it made for too many indentation. The last error code is still 1.

==============================
3200 items: ok
3213 items: ok
3214 items: ok
3215 items: executes ok but last item of the structure is shown to be the first one. I.e. element 3214 (0 based numbering) should be var_3214 but is displayed ad var_0 address 0x0
3216 item: fail on element 3215 with error code 1.

All the children count values are correct

===================================

if I declare the following

typedef struct{
      uint16_T a;
      uint16_T b;
}stMini_t;

typedef struct{
        stMini_t var_0;
        stMini_t var_1;
        ...
        stMini_t var_1999;
} stTest_t;

This works fine, but increasing the number of elements in stTest_t will result in the same behaviour.

==================================================

For the DIA SDK, i would really like not to have to redo my work. And in addition the original program is written in Visual C++ 6 so I doubt the DIA SDK would work with it. (I have a separate test program under Visual Studio 2005 EE with which I'm doing this debugging).

=================================================

I had quickly tried the SymGetTypeInfoEx function, but it is a monster function and hadn't gotten it to work. Maybe i'll give it another go since I am less busy.

Thanks for sticking with this Dan. Even if it is not fixed yet, your ideas help.
0
 

Accepted Solution

by:
goloap earned 0 total points
ID: 23035997
FINALLY I got it to work. The key was SymGetTypeInfoEx. I had to thinker to get it to work, but in the end it works. I post the magic code below in case somebody else is in the same situation.

Thanks for your help Dan, I'll give you the points because you helped me.
IMAGEHLP_GET_TYPE_INFO_PARAMS stTypeInfoEx;
ULONG TypeIds[] = {pSymInfo->TypeIndex};
IMAGEHLP_SYMBOL_TYPE_INFO ReqKinds[] = {TI_GET_OFFSET, TI_GET_SYMNAME};
ULONG_PTR ReqOffsets[] = {0, sizeof(DWORD)};
ULONG ReqSizes[] = {sizeof(DWORD), sizeof(WCHAR*)};
struct Record
{
	DWORD offset;
	WCHAR* pName;
};
struct Record *buffer;
buffer = (struct Record *)malloc(dwCount * sizeof(struct Record));
memset(buffer, 0, dwCount*sizeof(struct Record));
 
memset(&stTypeInfoEx, 0, sizeof(stTypeInfoEx));
stTypeInfoEx.SizeOfStruct = sizeof(stTypeInfoEx);
stTypeInfoEx.Flags = IMAGEHLP_GET_TYPE_INFO_CHILDREN;
stTypeInfoEx.NumIds = 1;
stTypeInfoEx.TypeIds = &TypeIds;
stTypeInfoEx.TagFilter = 0xFFFF; //0xFFFFFFFFFFFFFFFFul;
stTypeInfoEx.NumReqs = 2;
stTypeInfoEx.ReqKinds = ReqKinds;
stTypeInfoEx.ReqOffsets = ReqOffsets;
stTypeInfoEx.ReqSizes = ReqSizes;
stTypeInfoEx.ReqStride = sizeof(struct Record);
stTypeInfoEx.BufferSize = dwCount * sizeof(struct Record);
stTypeInfoEx.Buffer = (PVOID)buffer;
 
if (!SymGetTypeInfoEx(hBinFile, qwModuleBase, &stTypeInfoEx))
{
	printf("Error[%i]: SymGetTypeInfoEx failed\n", GetLastError());
}

Open in new window

0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Article by: SunnyDark
This article's goal is to present you with an easy to use XML wrapper for C++ and also present some interesting techniques that you might use with MS C++. The reason I built this class is to ease the pain of using XML files with C++, since there is…
Container Orchestration platforms empower organizations to scale their apps at an exceptional rate. This is the reason numerous innovation-driven companies are moving apps to an appropriated datacenter wide platform that empowers them to scale at a …
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.
Suggested Courses

564 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question