NOT4IRS
asked on
Memory leaking in a clustered environment
I have a web service that was created with VB .Net under Visual Studio 2003. Although the web service main code is VB, it calls an unmanaged C++ object that invokes a Microfocus COBOL dll. The web service generally runs on a Windows 2003 Server under IIS 6.0 using Version 2.0 of the .NET framework, although the Web Service is compiled with Version 1.1 of the .NET framework.
Running in this environment the Web Service was tested exhaustively by running a multi-threaded client process that ran 8 simultaneous threads invoking the web server over 26,000 times within a 3 hour period and the w3wp.exe process stabilized at 167 meg memory utilization. Individual unit tests of the unmanaged components verified that there was no memory leaks detected, not even small ones.
Recently this Web Service was installed on a couple of Windows 2003 Servers that have been clustered to provide high availability. When running in this clustered environment the Web Service began to continuously consume memory until all 3 GB of memory on the server was exhausted and a System.OutOfMemory exception occurred. This was done relatively quickly with as little as 2 or 3 thousand requests being serviced by the web service. Obviously very little of the memory consumed by the Web Service is being released.
Basically, it looks as though we have no memory leaks at all in a non-clustered environment, yet in a clustered environment we are not only seeing a memory leak but it's leaking like a sieve.
Is there something different about memory management when services are running in a clustered environment?
Monday morning the customer reported a failure of the MIM in their production environment with a SYSTEM.OUTOFMEMORY exception.
Analysis of the w3wp.exe process (via the Task Manager) on the server indicated that the process was consuming memory at a high rate and this error occurred when the entire 3 GB on the server was exhausted.
The Maximum User Memory parameter in the Default Application Pool Properties was set to 192 megabytes causing the process to be restarted when this threshold was reached.
This setting resolved the SYSTEM.OUTOFMEMORY issue and allowed the customer to continue processing. However, they reported that the process was being restarted every minute.
Running in this environment the Web Service was tested exhaustively by running a multi-threaded client process that ran 8 simultaneous threads invoking the web server over 26,000 times within a 3 hour period and the w3wp.exe process stabilized at 167 meg memory utilization. Individual unit tests of the unmanaged components verified that there was no memory leaks detected, not even small ones.
Recently this Web Service was installed on a couple of Windows 2003 Servers that have been clustered to provide high availability. When running in this clustered environment the Web Service began to continuously consume memory until all 3 GB of memory on the server was exhausted and a System.OutOfMemory exception occurred. This was done relatively quickly with as little as 2 or 3 thousand requests being serviced by the web service. Obviously very little of the memory consumed by the Web Service is being released.
Basically, it looks as though we have no memory leaks at all in a non-clustered environment, yet in a clustered environment we are not only seeing a memory leak but it's leaking like a sieve.
Is there something different about memory management when services are running in a clustered environment?
Monday morning the customer reported a failure of the MIM in their production environment with a SYSTEM.OUTOFMEMORY exception.
Analysis of the w3wp.exe process (via the Task Manager) on the server indicated that the process was consuming memory at a high rate and this error occurred when the entire 3 GB on the server was exhausted.
The Maximum User Memory parameter in the Default Application Pool Properties was set to 192 megabytes causing the process to be restarted when this threshold was reached.
This setting resolved the SYSTEM.OUTOFMEMORY issue and allowed the customer to continue processing. However, they reported that the process was being restarted every minute.
ASKER CERTIFIED SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.