problem with ASp.net / ISS / w3wp.exe using 100% CPU

There are memory leaks from the application code.
That means when you create objects such as Product after you use the instance of the Product, you have to release memory by disposing them or set to null value after use the instance. To call the Dispose() method, you have to implement IDispose interface.
   product.Dispose()   or product = null

Are you using in memory session management, ie. SessionState is "InProc" in web.config.
Are you adding so many collections into ASP.NET Session or ViewState.
There are tools to monitor session objects such as CLR Profiler for .NET.
http://odetocode.com/Blogs/scott/archive/2005/12/11/2611.aspx

Antivirus program might hung since CPU usage increasing upto 100%.

Could you able to explain the ASP.NET application that you are referring, if you need more details.

Hi Thanks for this.

I've checked through the code and it all seems fine. any objects are bing set to nothing and disposed is called where available.

the server just got to 100% CPU usage again. unfortunalty someone did an iisreset before i could look into things properly (it's a live server, so it had to be brought back up straight away).

i do know that w3wp.exe was using about 167,200k memory.

ive watched the memory usage and it does give memory back, although over time the usage does increase - about 5mb an hour. Usage of the application and the amount of data being pulled up does increase during the day, though.


one other thing to note is that it uses AJAX heavily to do automatic refreshes of data - like a monitor screen. as soon as this page is initially called the memory usage does jump up a bit. i cant see anything wrong in the code. AJAX does a partial refresh of the records every 60 seconds (time up to 30 users a time). But like i said, the w3wp.exe was only using 167mb when the CPU got to 100%.

The Application pool is set to recycle if it hits 90%. this was set yesterday by someone after the CPU usage for w3wp.exe hit 100%, so i dont think that is a problem.

if you stop and then resetart IIS the applicaiton is rebuilt in under 3 seconds and does not use anywhere near 100% CPU.

the server has 8gig physical memory by the way.

thanks in advance.

Use of AJAX does not have such effect?

How do you manage the session? Is it inproc or sql server session?
If it;s a very data intensive and high-end application, you should maintain the session in the sql server, not inproc.
When it's InProc, it will use the same IIS worker process to maintain session objects.

I have seen that where IIS worker process (w3wp.exe) was consuming more than 2GB, so usage of more than that also possible.

Use of AJAX does not have such effect...

How do you manage the session? Is it inproc or sql server session?
If it;s a very data intensive and high-end application, you should maintain the session in the sql server, not inproc.
When it's InProc, it will use the same IIS worker process to maintain session objects.

I have seen that where IIS worker process (w3wp.exe) was consuming more than 2GB, so usage of more than that also possible.

Hi,

do you have php 5 installed on that server? We had a similar problem after upgrading from php 4 to php 5 on an iis box running dot.net. We rollback to php 4.47 and the problem dissapeared.


Thanks

Jose

Hi.

we don't have PHP installed on the server.

The state is inProc. the process has recycled itself OK the last couple of days (at set time periods before working time) and has not exceeded 165 mb for any length of time. it is giving memory back as well - at one point it was 200mb then fell back to 150mb.

We haven't had the problem since Thursday, so i'll keep monitoring. One of our guys is pretty convinced it might by Norton anti Virus causing problems as someone wrongly installed the client version on the server.

Will keep you posted.

Hllo Patrick,

I found a thread on this forum that talk about a buffer overflow protection in an antivurus. May be it's the same in the case of Norton

http://dotnetfreak.co.uk/blog/archive/2005/02/15/asp-net-w3wp-exe-100-cpu-usage-and-mcafee-virusscan-enterprise.aspx

Thanks
Jose

Jose, sending something related to Mcafee virus scan.

Well, it's still been fine all day today! if it hasn't given us any more problems by tomorrow i think we'll have to assume it was one of those things we aren't going to find easily!

Just had the same problem. It fell over with w3wp.exe using 99% cpu. it was using 207mb memory at the time, which does not seem excessive.

it might have given us some more information this time - in the system event viewer there is an entry (it's a warning): Application pool 'HarmoniAppPool' exceeded its job limit settings. Source W3SVC. Event ID 1025.

unfortunatly the pool recycled itself pretty quickly as it is set to do when it hits 90% CPU usage and so i'm not sure if this event actually triggered the failure or was a result of the failure. i'll Google it now.

Since we have decided to check the virus scanner utilization, can you please stop/kill the virus scanner and check whether CPU usage is reducing. Go to task manager to check whether which process has taken so much of time.

The Virus scanner has been disabled. if it has the problem today, that will rule NAV out!

Hi again.

it doesnt look like the antivirus was causing the problem. yesterday the memory usage grew to 1.2 gig (then the CPU usage started to rise steadily to 100%).

sachintana - could you provide a bit more details on how to monitor the memory the asp.net application is consuming and releasing rather than the w3wp.exe process itself.

 i've had a bit of a play with a brand new single file asp.net 2 application on a different server. all this page does is bind a gridview using a sqldatarouce. i set this up to make sure ajax wasn't causing a problem and to see what happens with the memory usage without any custom code.

just by refreshing that simple page i saw the memory usage of w2wp.exe go from 25mb to 37mb (there are about 40 records being returned in the datasource.

then as a further test, i also did a test page creating an sqlconnection, sql command and a datareader and just loaded the same data into the data reader, then closing everything. again, by just refreshing this page (there was no UI or NON user code) the memory usage kept increasing. the code from page_load is below.



 this is the only code in the application and it is the only application running on that machine (there is nothing else running under w3wp.exe)

i can understand the memory growth to a point (as the garbage disposal might not have done its thing yet), but i dont know how to monitor the actual amount of memory the application is really using. my last test showed that w3wp.exe was consuming memory and not releasing it after execution, but the code is simple and everything is closed. therefore the memory used for the actual application must be being released,,,?

            Dim SqlConnection1 As New System.Data.SqlClient.SqlConnection
            Dim cmdSelectDetails As New System.Data.SqlClient.SqlCommand
            SqlConnection1.ConnectionString = ConfigurationManager.ConnectionStrings("MWLProjectsConnectionString").ConnectionString
            cmdSelectDetails.Connection = SqlConnection1
            cmdSelectDetails.CommandType = System.Data.CommandType.StoredProcedure
 
 
 
            SqlConnection1.Open()
 
            cmdSelectDetails.CommandText = "select_customer_from_partial"
            cmdSelectDetails.Parameters.Add(New System.Data.SqlClient.SqlParameter("@prefix", System.Data.SqlDbType.NVarChar)).Value = "a"
            Dim dtrreader As System.Data.SqlClient.SqlDataReader
 
            dtrreader = cmdSelectDetails.ExecuteReader()
 
            dtrreader.Close()
            dtrreader = Nothing
            SqlConnection1.Close()
            SqlConnection1.Dispose()
            SqlConnection1 = Nothing
            cmdSelectDetails.Dispose()
            cmdSelectDetails = Nothing

Select allOpen in new window

Hi you could try to increase the max number of worker processes under application pools.

 

Hi.

as we dont use sessions in the app (apart from to remember the last filter that a person had applied) we've setup the app pool to be a lot less sympathetic - it now recycles at 400mb memory usage. it'll be interesting to see if it hits that today.

the problem we have is it takes a long time to actually start having problems.

if we have another problem with it today i'll increase the worker processes and see what happens then.

Thanks for your help. i now have a much more through idea of what asp.net and IIS are doing in the background with regard to the memroy.

as it turns out the application was not leaking memory. the problem seemed to be caused by a  section of code which was reading from and writing to cache frequently. Having turned off this bit of code we have not had a problem.

Notes for other people (we've been helped by an MS Escalation engineer called Tess Ferrandez and Samiwel Thomas). Tess's blog details lots of memory debugging ideas and examples:

http://blogs.msdn.com/tess/

Also they looked through the dump for the application and saw no real issues, even though the w3wp.exe process was around 330mb. So don't worry too much if the w3wp.exe process does consume a fair amount of memory, as long as the application and server are stable!

Hi Patrick,

glad you solved your problem. What was causing the CPU pb?

Jose

can't say for definite exactly what the mechanics of the problem where, but as soon as we took out the code which was writing to and reading from cache repeatedly (ever 10 seconds per user) we have not had the same issue.

I assume the overhead of repeatedly working with the cache programatically was causing the issue in some way.

Thanks for sharing the Debugging Blog http://blogs.msdn.com/tess/, that;s really useful in debugging production systems.

It should be noted that the CLR profiler is pretty much useless on a production server running more than one worker process.