Solved

Memory Leak in Server using the Delphi SOAP Framework

Posted on 2004-04-21
7
1,086 Views
Last Modified: 2010-04-05
Hi there

I have implemented a SOAP Server using the Delphi 7 SOAP components (together with an INDY HTTP Server).

I left all the implementations to the methods of my class (descendant of TInvokableClass) empty. After issuing a few thousand calls to several of the methods (with a VB 6 client / MSSOAP) I realized that my little server had a huge memory leak.

A little research lead me to the article

http://community.borland.com/article/0,1410,28344,00.html

and using the KMM tool described there let me find, that every method call that arrives at my Server leads to a creation of an Object of my TInvokableClass descendants Class. The Creator of TInvokableClass contains a TSOAPHeaders.Create which, according to KMM has never been freed again. This leads to a loss of 32 bytes for every call to my server.

Can anybody tell me why the the Object of class TSOAPHeaders is created, but never freed ? What do I have to do to get rid of it ?

By the way: I tried to free it manually in the destructor of my class, which lead KMM to tell me, that everything was alright now. But in the Task managers process/memory usage monitor it still looks exactly the same as without freeing the TSOAPHeaders, which lets me think I am definitely doing something wrong. Can anyone explain this discrepancy ?

cheers
TomBig
0
Comment
Question by:TomBig
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
7 Comments
 
LVL 2

Expert Comment

by:RHenningsgard
ID: 10890094
Hmm... this is only a shot in the dark, but are any hugestrings allocated in the TSOAPHeaders?  I have written zillions of lines of Delphi, and have occasionally found string space that wasn't being recovered on the destructor method of objects.  I found this by using a phenomenal tool called "MemProof" by Atanas Stoyanov.

I just checked the web link from Atanas' MEMPROOF "About" box, and his site is moved or dead.  As great as the tool was, I'll bet he's still around.  Try a Google search for MEMPROOF and DELPHI, maybe Stoyanov, too.

As for my solution to the occasional hugestring memory leak, I simply went to the destructors of all of my objects (and a few in the VCL) and explicitly released the strings, i.e. LeakyString := '';  That always took care of it.

FWIW:  I encountered the leaks in a "clean-room" web server I wrote from the sockets level back in 1997.  The server uses tons of hugestrings, and it runs for three or four months at a crack between reboots of the server machine.  Since I applied my technique of setting all strings to equal nothing in the destructors, my server code hasn't leaked a single byte.

I hope this helps!

RHenningsgard
0
 
LVL 1

Author Comment

by:TomBig
ID: 10899200
RHenningsgard

thanks a lot for your comment. I found a link to MemProof in the arcticle mentioned in my Question. In the meanwhile I did some extensive testing (using KMM and MemProof) and by now I am definitely sure that the problem is the creation of a TSOAPHeaders object in the creator of TInvokableClass that is never being freed again. After I have overwritten the destructor in my derived class and freed the TSOAPHeaders instance manually, MemProof as well as KMM tell me that I dont have a memory leak (With MemProof I get the number of live pointers, which doesn't increase.)

What still bothers me a lot is, that the Task Manager still shows a steady increase in memory usage in my process, although occasionally it just suddenly drops a lot. Do you have any idea why that could be ? Could there be a kind of memory leak that wouldn't be monitored by MemProof ?

Besides I wonder if its just my poor Delphi version that doesn't release the TSOAPHeaders object. No one ever had this problem ?

Cheers
TomBig
0
 
LVL 2

Expert Comment

by:RHenningsgard
ID: 10899500
I have trouble reconciling the following two observations from your previous post:

<<I am definitely sure that the problem is the creation of a TSOAPHeaders object in the creator of TInvokableClass that is never being freed again>>

<<still bothers me a lot is, that the Task Manager still shows a steady increase in memory usage in my process, although occasionally it just suddenly drops a lot>>

So I have to ask the obvious questions:  

1) Precisely how is it that you can prove your application is leaking memory?

2) How long must you run the application before it hangs?

One suggestion:  I suggest you go to your system settings and force the virtual memory paging file size to zero.  Although I've not had to test this, friends of mine have suggested that temporarily defeating the virtual memory yields far more predictable behavior from the OS's memory management code.

Another observation:  Most O.S. memory managers with which I've had experience do not parcel out memory to applications in bite-sized chunks, as that would be very inefficient.  They give the application big chunks, and then the applications take responsibility for managing the little bits.  The effect of this would be consistent with the allocation/deallocation behavior you've described, as seen from Task Manager.

RHenningsgard
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 1

Author Comment

by:TomBig
ID: 10900527
Rob

this is really a big mess.

I followed your advice and set the virtual memory paging size to 0. I started 3 clients and submitted some thounsand calls to the server. It ran out of memory within a minute or so. I guess this also answers your first question.

At the moment I have not a clue about wht is happening. The facts (as known at the moment) are:
- MemProof tells me the server is NOT leaking (number of live pointers remains constant, all other numbers remain constant, I get a number of errors caused by some Indy classes I am using)
- Task Manager tells me that memory consumption is continoually increasing in time. Finally the server hangs with "Out of memory"

I guess I have to think it all over until monday. Maybe you have another clue ?

Best wishes
TomBig
0
 
LVL 2

Accepted Solution

by:
RHenningsgard earned 500 total points
ID: 10900962
<<this is really a big mess>>

Actually, from where I sit, you're making great progress!  If you can't get it to fail, you can't fix it.  So you've very effectively created a situation in which you can successfully make it fail.  (Now the problem is rigidly bounded).

<<I followed your advice and set the virtual memory paging size to 0. I started 3 clients and submitted some thounsand calls to the server. It ran out of memory within a minute or so. I guess this also answers your first question.>>

Perfect!  Perhaps if you can remove a memory stick, you can get it to fail without keeping you waiting so long.

<<At the moment I have not a clue about what is happening.>>

No, you know exactly what is happening.  Now you have to determine _why_ it is happening.

<<Maybe you have another clue ?>>

<<I get a number of errors caused by some Indy classes I am using>>

Here's your next step:  You must start slicing the program in half.  Comment out a bunch of its functionality and try to make it fail again.  After you comment out a bunch and it still fails, make a backup copy of the whole thing.  Then comment out a bunch more, and see if it fails.  If so, back up again, and repeat the process.  Eventually, by definition, sooner or later it will not fail (hopefully _before_ your mainline is reduced to "begin end.").  At that point, whatever you just eliminated is the place to look more closely.  Obviously, as soon as you comment out (or otherwise eliminate) something which then allows the program to run without leaking its available memory down to zero, you have taken a huge step towards identifying the culprit.

This process is admittedly crude, but it is inescapably effective.  If you just keep bisecting the program, eventually you will have nothing left but the problem itself.  I've used this process hundreds of times in all kinds of systems of hardware and software, and I have never failed to find a problem.  Be patient, keep chipping away at it, and you will absolutely find the source of your trouble.

RHenningsgard
0
 
LVL 1

Author Comment

by:TomBig
ID: 10919469
Rob.

thanks for all your comments. Somehow, I really dont know how, the memory usage monitor in the taskmanager now coincides with the info I get from MemProof. I did some more testing during the day and EVERYTHING looks stable now. No more crashes and memory usage at a very reasonable level.

As you took most time helping me and gave me some reasonable live time advice I suppose you should earn the points.

thanks
TomBig
0
 
LVL 2

Expert Comment

by:RHenningsgard
ID: 10919661
<<I did some more testing during the day and EVERYTHING looks stable now>>

Er... what did you change?  How did you fix the leak?  Can you make it come back and go away at will?  (If so, how?)

<<No more crashes and memory usage at a very reasonable level.>>

For now... unless you've actually found the cause of the problem and corrected it.  I don't mean to sound pessimistic, but I've been at this game for a long time, and there's nothing worse than a problem that shows up and then disappears intermittently.

Thank you for the points and all, but I really have the feeling we didn't get to the bottom of this one (yet).  Let me know if I can help further.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

A lot of questions regard threads in Delphi.   One of the more specific questions is how to show progress of the thread.   Updating a progressbar from inside a thread is a mistake. A solution to this would be to send a synchronized message to the…
In my programming career I have only very rarely run into situations where operator overloading would be of any use in my work.  Normally those situations involved math with either overly large numbers (hundreds of thousands of digits or accuracy re…
Are you ready to implement Active Directory best practices without reading 300+ pages? You're in luck. In this webinar hosted by Skyport Systems, you gain insight into Microsoft's latest comprehensive guide, with tips on the best and easiest way…

738 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question