ADO Memory Leak

VC++ 6.0 SP5, MDAC 2.8, W2K Pro SP3
I am developing an NT service that uploads data to a database. It will run 24/7. Because of this, memory leaks are of particular concern to me. I wrote a simple application that executes a select from a database continuously until you press a key. After starting the exe, I took a look at the task manager and saw the "mem usage" grow at a linear rate. There doesn't appear to be an upper bound to this growth.

01 #include <stdio.h>
02 #include <conio.h>
03 #import "c:\program files\common files\system\ado\msado15.dll" rename("EOF","adoEOF")
05 struct InitOle {
06 InitOle()  { ::CoInitialize(NULL); }
07 ~InitOle() { ::CoUninitialize();   }
08 } _init_InitOle_;
10 void hitdb();
12 int main(int argc, char* argv[])
13 {
14      bool done = false;
17      printf("\npress any key to cancel the test...");
18      do
19      {
20            if(kbhit()) done = true;
21            hitdb();
22      } while (!done);
23      getch();
24      return 0;
25 }
27 void hitdb()
28 {
29      try
30      {
31            ADODB::_RecordsetPtr   Rs1 = NULL;
32            Rs1.CreateInstance( __uuidof( ADODB::Recordset ) );
33            Rs1->Open( L"SELECT count(*) FROM counts",
34                  L"DSN=linestatus",
35                  ADODB::adOpenForwardOnly,
36                  ADODB::adLockReadOnly, -1 );
37            Rs1->Close();
38            Rs1 = NULL;
39      }
40      catch (_com_error &e) { printf("%s\n",e.ErrorMessage()); }
41      catch (...) { printf("Unspecified Error\n"); }
42 }

If I comment out lines 33 - 37 then there is no memory leak, leading me to believe the problem lies with ADO. Adding Rs1.Release(); to line 37.5 doesn't help. Explicitly creating a connection object doesn't help either, in fact, the leak occurs if I only open a connection to the db and nothing else. Additionally, I have tried to use a DSNless connection to avoid ODBC, but without luck. It seems that the memory used *should* return to a constant level after each call to hitdb(), but this doesn't appear to be the case.
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.


> After starting the exe, I took a look at the task manager and saw the "mem
> usage" grow at a linear rate. There doesn't appear to be an upper bound to this growth.

I've run into something similiar with ADO, but the memory usage always would drop back down after I closed the recordset.  But you're clearly closing the recordset, so that should cover it.

But the task manager isn't really sufficient for diagnosing a memory leak.  Instead, use the Performance Monitor, and add the following items:
     Memory Object -> Available Bytes
     Process -> YourProcess -> Handle Count
     Process -> YourProcess -> Pool Nonpaged Bytes
     Process -> YourProcess -> Pool Pages Bytes
     Process -> YourProcess -> Private Bytes

If the "Private Bytes" of your application increases consistantly, without decreasing, then you have a memory leak.

Hope that helps,

> If I comment out lines 33 - 37 then there is no memory leak, leading me to
> believe the problem lies with ADO.

You aren't the only one with this issue.  Read this:

That's for an Access database, but one of the replies says that there is similiar behavior with other providers.

Have you tried dumping the connection when you're done.

Rs1->putref_ActiveConnection(NULL); // Line 36.5
Cloud Class® Course: Certified Penetration Testing

This CPTE Certified Penetration Testing Engineer course covers everything you need to know about becoming a Certified Penetration Testing Engineer. Career Path: Professional roles include Ethical Hackers, Security Consultants, System Administrators, and Chief Security Officers.

celestianAuthor Commented:
  I had considered perfmon a more fine grained tool, and if taskman showed an increasing usage of memory, so would perfmon.

  I added the line you specified, but it did not aleviate the problem.


My point is that you might not have a memory leak at all.  The Windows memory manager may be doing some things that look odd to you, because it thinks thats the most efficient way to do it.  Maybe you have TONS of RAM on the system, maybe there is nothing else competing for the resources... Who knows?  My point is that "Mem Usage" doesn't indicate a memory leak.

The "Mem Usage" field of the task manager corresponds to the "Working Set" property of the Performance counter, which isn't even one of the ones they tell you to check for a memory leak.  The important measurement is "Private Bytes".  If you want to use the Task Manager, then do this:

1) View -> Select Columns
2) Check "Virtual Memory Size"

If that value keeps going up and up, then you might have a memory leak... But until you know for sure, you could be chasing a ghost.

celestianAuthor Commented:
I ran an extended test to see how the memory would be affected:

time                peak mem usage     vm size
12:19              10,948                   8,516
1:19                12,536                   19,408
3:05                13,260                   40,120
4:01                13,260                   51,264
~48 hrs later    17,344                   83,096

Since the program is doing the same thing over and over, I guess I would expect these numbers to remain constant. Since they are not, I am exploring alternatives to ADO.

Okay, that is definitely, a memory leak.

I was going to run the code that you posted, but I noticed something that may be your issue:
     38          Rs1 = NULL;

Rs1 is a smart pointer that will clean up after itself.  When you set it to NULL, you're clearing out the pointer that it is supposed to clean up, so when the variable falls out of scope, there is no way to clean it up.  Try removing line 38 and see if you get the same results.

Yes, I know lots of sample VB code does things like:
     Set Rs1 = Nothing

But, when you do that, you are releasing the smart pointer.

If you still are looking for an alternative, then have you looked into OLEDB?  (ADO is basically a higher level wrapper around OLEDB)  Now, OLEDB is much more detailed (and therefore, more to learn before using it), but it gives you access to things on a really low level.  This page gives you some idea of the differences:

ATL even provides you with some really nice templates for creating OLEDB classes, and VS will give you a wizard to generate classes based on a specific table, view, or stored procedure.

celestianAuthor Commented:
I have run some more detailed tests in an effort to try and understand what is going on here. I used the following code:

#include <stdio.h>
#include <conio.h>
#import <msado15.dll> rename("EOF","adoEOF")

struct InitOle {
  InitOle()  { ::CoInitialize(NULL); }
  ~InitOle() { ::CoUninitialize();   }
} _init_InitOle_;

void hitdb();

int main(int argc, char* argv[])
  bool done = false;
  char c;
  long l = 0;

  printf("\npress any key to start the test...");
  do { Sleep(300);} while (!kbhit()); getch();
  printf("\n'p' to pause 'x' to exit");
    if(kbhit()) {
      c = getch();
      if (c == 'p') {
        printf("\n%d iterations completed",l);
        printf("\npress any key to continue the test...");
        do { Sleep(300); } while (!kbhit()); getch();
      } else if (c == 'x') {
        done = true;
      if (!done) printf("\n'p' to pause 'x' to exit");
  } while (!done);

  return 0;
void hitdb() {
    ADODB::_ConnectionPtr  Cn1;
    Cn1->CursorLocation = ADODB::adUseClient;
  catch (_com_error &e) { printf("%s\n",e.ErrorMessage()); }
  catch (...) { printf("Unspecified Error\n"); }

I stepped through the code and noticed that as calls were made to hitdb(), threads were created and destroyed, no surprise there. Also, each time a thread was created, 3 handles were also created, but when the thread was destroyed, only 2 handles were returned. This happened each time as I allowed 1 call to hitdb(), and the handle counter grew by one after each call. When I let the loop continue without interruption, I noticed that this didn't always happen. After 671 calls to hitdb(), I saw 184 handles, when after 1 loop there had only been 137. I ran the test continuously 2 more times until I saw 184 handles on 4 threads. This happend after around 480 and 356 iterations. It seems that whatever is going on in this multi-threaded mess, there are handles to something being created, and they are not always destroyed. This might very well be the cause of the memory leak.

Looking for information on the internet related to C++ and ADO is difficult at best, perhaps because others have noticed problems as well and just don't use the two together. I also noticed that when I created my first ADO Connection instance, 50 dlls were loaded. And I am not doing any kind of data manipulation, I am just openning a connection and then closing it right away. I will look into OLEDB and <sqlext.h> and see if I can get something that works a little better.

I will let this question stay open for a little while in case there is someone who knows of a database c++ solution that doesn't leak memory.
What is the backend data source provider that you are using?  SQL Server?  Access?  Oracle?

celestianAuthor Commented:
I have tried Oracle on a remote server and a local copy of MySQL server. For Oracle I used a DSNless connection with OraOLEDB.Oracle.1 as the provider, and for MySQL I set up a system dsn with the MySQL ODBC 3.51 Driver. As it so happens both the above code samples are using the DSN with MySQL, but for the memory tests I posted that showed the leak, I was using Oracle. Either way I notice the leak.
I ran the code that you did, but with the SQL Server provider, and I wasn't able to reproduce the leaks.  Maybe the leaks are in the provider layer, not in ADO itself...  Which doesn't help you out, but it would explain why others haven't noticed it yet.


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
By writing code that doesn't have any. Clearly, if your code has new operations, delete operations, and pointer arithmetic all over the place, you are going to mess up somewhere and get leaks, stray pointers, etc. This is true independently of how conscientious you are with your allocations: eventually the complexity of the code will overcome the time and effort you can afford. It follows that successful techniques rely on hiding allocation and deallocation inside more manageable types. Good examples are the standard containers. They manage memory for their elements better than you could without disproportionate effort. Consider writing this without the help of string and vector:
      using namespace std;

      int main()      // small program messing around with strings
            cout << "enter some whitespace-separated words:\n";
            vector<string> v;
            string s;
            while (cin>>s) v.push_back(s);


            string cat;
            typedef vector<string>::const_iterator Iter;
            for (Iter p = v.begin(); p!=v.end(); ++p) cat += *p+"+";
            cout << cat << '\n';

What would be your chance of getting it right the first time? And how would you know you didn't have a leak?
Note the absence of explicit memory management, macros, casts, overflow checks, explicit size limits, and pointers. By using a function object and a standard algorithm, I could have eliminated the pointer-like use of the iterator, but that seemed overkill for such a tiny program.

These techniques are not perfect and it is not always easy to use them systematically. However, they apply surprisingly widely and by reducing the number of explicit allocations and deallocations you make the remaining examples much easier to keep track of. As early as 1981, I pointed out that by reducing the number of objects that I had to keep track of explicitly from many tens of thousands to a few dozens, I had reduced the intellectual effort needed to get the program right from a Herculean task to something manageable, or even easy.

If your application area doesn't have libraries that make programming that minimizes explicit memory management easy, then the fastest way of getting your program complete and correct might be to first build such a library.

Templates and the standard libraries make this use of containers, resource handles, etc., much easier than it was even a few years ago. The use of exceptions makes it close to essential.

If you cannot handle allocation/deallocation implicitly as part of an object you need in your application anyway, you can use a resource handle to minimize the chance of a leak. Here is an example where I need to return an object allocated on the free store from a function. This is an opportunity to forget to delete that object. After all, we cannot tell just looking at pointer whether it needs to be deallocated and if so who is responsible for that. Using a resource handle, here the standard library auto_ptr, makes it clear where the responsibility lies:

      using namespace std;

      struct S {
            S() { cout << "make an S\n"; }
            ~S() { cout << "destroy an S\n"; }
            S(const S&) { cout << "copy initialize an S\n"; }
            S& operator=(const S&) { cout << "copy assign an S\n"; }

      S* f()
            return new S;      // who is responsible for deleting this S?

      auto_ptr<S> g()
            return auto_ptr<S>(new S);      // explicitly transfer responsibility for deleting this S

      int main()
            cout << "start main\n";
            S* p = f();
            cout << "after f() before g()\n";
      //      S* q = g();      // caught by compiler
            auto_ptr<S> q = g();
            cout << "exit main\n";
            // leaks *p
            // implicitly deletes *q

Think about resources in general, rather than simply about memory.

If systematic application of these techniques is not possible in your environment (you have to use code from elsewhere, part of your program was written by Neanderthals, etc.), be sure to use a memory leak detector as part of your standard development procedure, or plug in a garbage collector.

Did that answer apply to this question?
No comment has been added lately, so it's time to clean up this TA.
I will leave the following recommendation for this question in the Cleanup topic area:

Split: Dexstar {http:#9766879} & Paladin_VB {http:#9804532}

Please leave any comments here within the next seven days.

EE Cleanup Volunteer
Good comment Paladin_VB

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Editors IDEs

From novice to tech pro — start learning today.