Solved

Memory Leak in Server using the Delphi SOAP Framework

Posted on 2004-04-21
7
1,035 Views
Last Modified: 2010-04-05
Hi there

I have implemented a SOAP Server using the Delphi 7 SOAP components (together with an INDY HTTP Server).

I left all the implementations to the methods of my class (descendant of TInvokableClass) empty. After issuing a few thousand calls to several of the methods (with a VB 6 client / MSSOAP) I realized that my little server had a huge memory leak.

A little research lead me to the article

http://community.borland.com/article/0,1410,28344,00.html

and using the KMM tool described there let me find, that every method call that arrives at my Server leads to a creation of an Object of my TInvokableClass descendants Class. The Creator of TInvokableClass contains a TSOAPHeaders.Create which, according to KMM has never been freed again. This leads to a loss of 32 bytes for every call to my server.

Can anybody tell me why the the Object of class TSOAPHeaders is created, but never freed ? What do I have to do to get rid of it ?

By the way: I tried to free it manually in the destructor of my class, which lead KMM to tell me, that everything was alright now. But in the Task managers process/memory usage monitor it still looks exactly the same as without freeing the TSOAPHeaders, which lets me think I am definitely doing something wrong. Can anyone explain this discrepancy ?

cheers
TomBig
0
Comment
Question by:TomBig
  • 4
  • 3
7 Comments
 
LVL 2

Expert Comment

by:RHenningsgard
Comment Utility
Hmm... this is only a shot in the dark, but are any hugestrings allocated in the TSOAPHeaders?  I have written zillions of lines of Delphi, and have occasionally found string space that wasn't being recovered on the destructor method of objects.  I found this by using a phenomenal tool called "MemProof" by Atanas Stoyanov.

I just checked the web link from Atanas' MEMPROOF "About" box, and his site is moved or dead.  As great as the tool was, I'll bet he's still around.  Try a Google search for MEMPROOF and DELPHI, maybe Stoyanov, too.

As for my solution to the occasional hugestring memory leak, I simply went to the destructors of all of my objects (and a few in the VCL) and explicitly released the strings, i.e. LeakyString := '';  That always took care of it.

FWIW:  I encountered the leaks in a "clean-room" web server I wrote from the sockets level back in 1997.  The server uses tons of hugestrings, and it runs for three or four months at a crack between reboots of the server machine.  Since I applied my technique of setting all strings to equal nothing in the destructors, my server code hasn't leaked a single byte.

I hope this helps!

RHenningsgard
0
 
LVL 1

Author Comment

by:TomBig
Comment Utility
RHenningsgard

thanks a lot for your comment. I found a link to MemProof in the arcticle mentioned in my Question. In the meanwhile I did some extensive testing (using KMM and MemProof) and by now I am definitely sure that the problem is the creation of a TSOAPHeaders object in the creator of TInvokableClass that is never being freed again. After I have overwritten the destructor in my derived class and freed the TSOAPHeaders instance manually, MemProof as well as KMM tell me that I dont have a memory leak (With MemProof I get the number of live pointers, which doesn't increase.)

What still bothers me a lot is, that the Task Manager still shows a steady increase in memory usage in my process, although occasionally it just suddenly drops a lot. Do you have any idea why that could be ? Could there be a kind of memory leak that wouldn't be monitored by MemProof ?

Besides I wonder if its just my poor Delphi version that doesn't release the TSOAPHeaders object. No one ever had this problem ?

Cheers
TomBig
0
 
LVL 2

Expert Comment

by:RHenningsgard
Comment Utility
I have trouble reconciling the following two observations from your previous post:

<<I am definitely sure that the problem is the creation of a TSOAPHeaders object in the creator of TInvokableClass that is never being freed again>>

<<still bothers me a lot is, that the Task Manager still shows a steady increase in memory usage in my process, although occasionally it just suddenly drops a lot>>

So I have to ask the obvious questions:  

1) Precisely how is it that you can prove your application is leaking memory?

2) How long must you run the application before it hangs?

One suggestion:  I suggest you go to your system settings and force the virtual memory paging file size to zero.  Although I've not had to test this, friends of mine have suggested that temporarily defeating the virtual memory yields far more predictable behavior from the OS's memory management code.

Another observation:  Most O.S. memory managers with which I've had experience do not parcel out memory to applications in bite-sized chunks, as that would be very inefficient.  They give the application big chunks, and then the applications take responsibility for managing the little bits.  The effect of this would be consistent with the allocation/deallocation behavior you've described, as seen from Task Manager.

RHenningsgard
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 1

Author Comment

by:TomBig
Comment Utility
Rob

this is really a big mess.

I followed your advice and set the virtual memory paging size to 0. I started 3 clients and submitted some thounsand calls to the server. It ran out of memory within a minute or so. I guess this also answers your first question.

At the moment I have not a clue about wht is happening. The facts (as known at the moment) are:
- MemProof tells me the server is NOT leaking (number of live pointers remains constant, all other numbers remain constant, I get a number of errors caused by some Indy classes I am using)
- Task Manager tells me that memory consumption is continoually increasing in time. Finally the server hangs with "Out of memory"

I guess I have to think it all over until monday. Maybe you have another clue ?

Best wishes
TomBig
0
 
LVL 2

Accepted Solution

by:
RHenningsgard earned 500 total points
Comment Utility
<<this is really a big mess>>

Actually, from where I sit, you're making great progress!  If you can't get it to fail, you can't fix it.  So you've very effectively created a situation in which you can successfully make it fail.  (Now the problem is rigidly bounded).

<<I followed your advice and set the virtual memory paging size to 0. I started 3 clients and submitted some thounsand calls to the server. It ran out of memory within a minute or so. I guess this also answers your first question.>>

Perfect!  Perhaps if you can remove a memory stick, you can get it to fail without keeping you waiting so long.

<<At the moment I have not a clue about what is happening.>>

No, you know exactly what is happening.  Now you have to determine _why_ it is happening.

<<Maybe you have another clue ?>>

<<I get a number of errors caused by some Indy classes I am using>>

Here's your next step:  You must start slicing the program in half.  Comment out a bunch of its functionality and try to make it fail again.  After you comment out a bunch and it still fails, make a backup copy of the whole thing.  Then comment out a bunch more, and see if it fails.  If so, back up again, and repeat the process.  Eventually, by definition, sooner or later it will not fail (hopefully _before_ your mainline is reduced to "begin end.").  At that point, whatever you just eliminated is the place to look more closely.  Obviously, as soon as you comment out (or otherwise eliminate) something which then allows the program to run without leaking its available memory down to zero, you have taken a huge step towards identifying the culprit.

This process is admittedly crude, but it is inescapably effective.  If you just keep bisecting the program, eventually you will have nothing left but the problem itself.  I've used this process hundreds of times in all kinds of systems of hardware and software, and I have never failed to find a problem.  Be patient, keep chipping away at it, and you will absolutely find the source of your trouble.

RHenningsgard
0
 
LVL 1

Author Comment

by:TomBig
Comment Utility
Rob.

thanks for all your comments. Somehow, I really dont know how, the memory usage monitor in the taskmanager now coincides with the info I get from MemProof. I did some more testing during the day and EVERYTHING looks stable now. No more crashes and memory usage at a very reasonable level.

As you took most time helping me and gave me some reasonable live time advice I suppose you should earn the points.

thanks
TomBig
0
 
LVL 2

Expert Comment

by:RHenningsgard
Comment Utility
<<I did some more testing during the day and EVERYTHING looks stable now>>

Er... what did you change?  How did you fix the leak?  Can you make it come back and go away at will?  (If so, how?)

<<No more crashes and memory usage at a very reasonable level.>>

For now... unless you've actually found the cause of the problem and corrected it.  I don't mean to sound pessimistic, but I've been at this game for a long time, and there's nothing worse than a problem that shows up and then disappears intermittently.

Thank you for the points and all, but I really have the feeling we didn't get to the bottom of this one (yet).  Let me know if I can help further.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

In this tutorial I will show you how to use the Windows Speech API in Delphi. I will only cover basic functions such as text to speech and controlling the speed of the speech. SAPI Installation First you need to install the SAPI type library, th…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

6 Experts available now in Live!

Get 1:1 Help Now