Need a Windows API to retrieve memory usage (private bytes usage) for the current process

Hi expert,

Anyone know a windows API to retrieve memory usage (something equivalence to the perfmon private bytes)...

I am tracking down a memory issue in our application,so far purify and bunceChecker did not find that problem for me.  I tried ProcessExplorer but the private bytes should in KB...I need an API that I constantly dump out the memory Private Bytes allocated for the current process.

Thanks.
4eyesgirlAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

evilrixSenior Software Engineer (Avast)Commented:
This may help you.

Diagnosing heap related memory leaks.

You can use _CrtSetDbgFlag to enable CRT heap allocation debugging. This should be at the very start of your program.
E.g. _CrtSetDbgFlag ( _CrtSetDbgFlag( _CRTDBG_REPORT_FLAG ) | _CRTDBG_LEAK_CHECK_DF );
http://msdn2.microsoft.com/en-us/library/974tc9t1(VS.80).aspx

You can use _CrtDumpMemoryLeaks to generate an error report if the application failed to free all the memory it allocated. This should be at the very end of your program.
http://msdn2.microsoft.com/en-us/library/d41t22sb(VS.80).aspx

Use can use _CrtSetBreakAlloc or _crtBreakAlloc to set break points where specific heap is allocated (as reported by CrtDumpMemoryLeaks) so that you can see where the problem starts
http://support.microsoft.com/kb/151585

Memory leak detection and isolation: http://msdn2.microsoft.com/en-us/library/x98tx3cf(VS.80).aspx
0
jkrCommented:
Add the following to a central header file:

#ifdef _DEBUG
#ifndef _DBG_NEW

#include <crtdbg.h>

inline void* __operator_new(size_t __n) {
     return ::operator new(__n,_NORMAL_BLOCK,__FILE__,__LINE__);
}
inline void* _cdecl operator new(size_t __n,const char* __fname,int __line) {
     return ::operator new(__n,_NORMAL_BLOCK,__fname,__line);
}
inline void _cdecl operator delete(void* __p,const char*,int) {
     ::operator delete(__p);
}

#define _DBG_NEW new(__FILE__,__LINE__)
#define new _DBG_NEW


#endif // _DBG_NEW
#else

#define __operator_new(__n) operator new(__n)

#endif

Then, add

               int tmpFlag;

               // Get the current state of the flag
               // and store it in a temporary variable
               tmpFlag = _CrtSetDbgFlag( _CRTDBG_REPORT_FLAG );

               // Turn On (OR) - Keep freed memory blocks in the
               // heap’s linked list and mark them as freed
               tmpFlag |= _CRTDBG_LEAK_CHECK_DF;

               // Set the new state for the flag
               _CrtSetDbgFlag( tmpFlag );


to your app's start code and call

               _CrtDumpMemoryLeaks ();


to before the program ends and you'll get the line number where the allocation occured.

(Ref.: http:Q_21009673.html)
0
4eyesgirlAuthor Commented:
Isn't Purify and BounceChecker is kind of using this type of debugging for memory issue?  Note that the memory issue that I am investigate (could be related to logical rather than static memory allocation) and that's why I don't think this will help.

What I need is an API that will dump out exact private byte usage for the current process.  IF you are familiar with Process Explorer, it shows the private byte usage but it is in KB...I want an API to dump out the exact bytes.

0
Exploring ASP.NET Core: Fundamentals

Learn to build web apps and services, IoT apps, and mobile backends by covering the fundamentals of ASP.NET Core and  exploring the core foundations for app libraries.

jkrCommented:
BTW, to really read the "private bytes" as you were asking, see http://support.microsoft.com/kb/296794 ("Sample Code to Detect Performance Services That Leak Memory") and the accompanying code at http://download.microsoft.com/download/whistler/sample/1.0/nt45xp/en-us/memleaktest.exe - it does exactly that.
0
jkrCommented:
>>Isn't Purify and BounceChecker is kind of using this type of debugging for
>>memory issue?

Usually, yes.

>>What I need is an API that will dump out exact private byte usage for the
>>current process.

See the MSDN article above, it does that. Yet the "private bytes" won't help to track down where the leak occurred.
0
4eyesgirlAuthor Commented:
Do you know what this win API function GlobalMemoryStatusEx does then?
0
evilrixSenior Software Engineer (Avast)Commented:
The point of my first post, in case you didn't realize, is that the debug CRT allows you to set break points at the point of heap memory allocation based upon leaked allocation blocks. This should allow you to identify exactly what leaks.

Also, you might find the following a useful read: http://shsc.info/WindowsMemoryManagement
0
jkrCommented:
>>Do you know what this win API function GlobalMemoryStatusEx does then?

It fills in a struct:  http://msdn2.microsoft.com/en-us/library/aa366770(VS.85).aspx

typedef struct _MEMORYSTATUSEX {
  DWORD dwLength;
  DWORD dwMemoryLoad;
  DWORDLONG ullTotalPhys;
  DWORDLONG ullAvailPhys;
  DWORDLONG ullTotalPageFile;
  DWORDLONG ullAvailPageFile;
  DWORDLONG ullTotalVirtual;
  DWORDLONG ullAvailVirtual;
  DWORDLONG ullAvailExtendedVirtual;
} MEMORYSTATUSEX,  *LPMEMORYSTATUSEX;

But that's not the "private bytes" you're looking for.


0
evilrixSenior Software Engineer (Avast)Commented:
>> But that's not the "private bytes" you're looking for.
That's very Jedi of you jkr :)
0
4eyesgirlAuthor Commented:
I know about the structure, but what are those values mean?  None of the value is the private byte for the process?
0
jkrCommented:
No, none of them. From MSDN: "Private bytes include memory that is committed and marked MEM_PRIVATE, data that is not mapped, and executable pages that have been written to."
0
4eyesgirlAuthor Commented:
Need to verify with the expert:

Step 1.  Find the central header file
     Add #define _CRTDBG_MAP_ALLOC
     include <crtdbg.h>  ???

   Do I need this #ifdef _DEBUG
                          #ifndef _DBG_NEW  ?  I think this is default included if I built debug version, correct?
           
What are the following code does?
inline void* __operator_new(size_t __n) {
     return ::operator new(__n,_NORMAL_BLOCK,__FILE__,__LINE__);
}
inline void* _cdecl operator new(size_t __n,const char* __fname,int __line) {
     return ::operator new(__n,_NORMAL_BLOCK,__fname,__line);
}
inline void _cdecl operator delete(void* __p,const char*,int) {
     ::operator delete(__p);
}

#define _DBG_NEW new(__FILE__,__LINE__)
#define new _DBG_NEW


#endif // _DBG_NEW
#else

#define __operator_new(__n) operator new(__n)

#endif
0
jkrCommented:
>>   Do I need this #ifdef _DEBUG
>>                          #ifndef _DBG_NEW  ?  I think this is default included if I built >>debug version, correct?

Yes, but the #define is still needed - '_DEBUG' will be set.

>>What are the following code does?

They re-route allocation calls to the debug versions that take the file name and line number of the caller to detect the occurrance of the leaks later.
0
4eyesgirlAuthor Commented:

>>What are the following code does?

They re-route allocation calls to the debug versions that take the file name and line number of the caller to detect the occurrance of the leaks later.

Isn't that equivalence to #define _CRTDBG_MAP_ALLOC does?

Also, My app has two part (dll and the test driver).

So I should add the #include <crtdbg.h> in the the central header in the dll?
What about setting the _CrtSetDbgFlag?  In the main() int the test driver?  Or the entry point of the dll? Call this _CrtDumpMemoryLeaks (); in the test driver main() before return 0?



0
jkrCommented:
You can do both. Yet if you add that to the DLL, it will dump the pending allocations in the driver also, thus it might be better to only put that into the driver code to ensure you'll only see real memory leaks.
0
4eyesgirlAuthor Commented:
You can do both. Yet if you add that to the DLL, it will dump the pending allocations in the driver also, thus it
might be better to only put that into the driver code to ensure you'll only see real memory leaks.

I don't understand your response.

Are you saying that I should put the _CrtSetDbgFlag in the main() and _CrtDumpMemoryLeaks () before exiting the test driver?

Would this detect leaks inside the dll?
0
jkrCommented:
>>Are you saying that I should put the _CrtSetDbgFlag in the main() and
>>_CrtDumpMemoryLeaks () before exiting the test driver?
>>Would this detect leaks inside the dll?

Yes to both. That's exactly how it works. BTW, you need to make sure that both the DLL and the driver are set to "Use Runtime Library: (Multithreaded) (Debug) DLL" to rule out some other issues also (if that is not set, both the app and the DLL have their own heap, which is not good). Also, you will only see that output in the output window of the debugger - or use 'DebugView' (http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx) to see that info.
0
4eyesgirlAuthor Commented:
okay.  I will give it a try.  I might need some help in the next couple hours.  Hopefully you will still be online.
Thanks.
0
4eyesgirlAuthor Commented:
Looks like this utility does not work with Stlport?

\release\stlport\inc\cstdlib(51) : error C2873: 'calloc' : symbol cannot be used in a using-declaration
\release\stlport\inc\cstdlib(52) : error C2039: 'free' : is not a member of 'operator``global namespace'''
\release\stlport\inc\cstdlib(52) : error C2873: 'free' : symbol cannot be used in a using-declaration
0
jkrCommented:
strange, neither 'calloc()' not 'free()' are used in the above code, so it shouldn't be caused by that.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Development

From novice to tech pro — start learning today.