Link to home
Create AccountLog in
Avatar of samabdelhamid
samabdelhamid

asked on

App Pool dropping requests after a restart

Hi All,
We have a few application pools in our application.
After stopping, and then starting an application pool, we make it available on our load balancer.
Onced it is available, users begin to use it, and it is serving requests, however, after sometime, it stops serving, and users get some errors. We can see this if we monitor it with perfmon.

We have seen it do this up to 3 times after a restart, and then it behaves itself.

Can anyone suggest where to start looking for such issues, we're all scratching our heads..
Avatar of becraig
becraig
Flag of United States of America image

Does the server give a 503 at any point ?

This sounds like your server might be under excessive load.
Do you have keep alives switched on ?

You should also check your web server logs to see what http responses are sent to the client before this unavailability.

If you see any 503s you may want to get a dump of the worker process as well as look into throttling requests to your server.  You might be running out of cpu or memory (possible memory leak in your application ?)

The server web logs would be the best place to start.
Avatar of samabdelhamid
samabdelhamid

ASKER

]It is under load, as this is what we are testing. However, not excessive, as the other app pools are handling the load with no issues.
WE dont know why it would begin to handle requests and then drop them.
Here is a screenshot of our perfmon.


Black line is Requests Total
Red line is Processor Time.

You can see how it starts serving and then drops, then starts serving again..

[embed=file 614597
img0000.jpg
Crack open the IIS logs and see what the return codes are at the point when the app is failing, the cpu load does look a little high ( however I don't know the specs on the server)


Scan the logs for the time around the failures and see what the return error codes are when requests begin to get dropped.
Did some research, found this, at the exact times we recycled the app pools in event vwr.

 A process serving application pool 'AppPool12' suffered a fatal communication error with the Windows Process Activation Service. The process id was '9960'. The data field contains the error number.
ASKER CERTIFIED SOLUTION
Avatar of becraig
becraig
Flag of United States of America image

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
The issue was a file called ABCPDF.DLL
This is a PDF generator (off the shelf). What was happening, is that multiple users were trying to access this DLL, normally this is ok, as its multi threaded, however, the first time the DLL runs and is loaded into memory, it can only be one thread. What was happening is the first user was hitting it, loading it into memory, the second user would come and also load it into memory, because user1 had not finished loading into memory while user 2 began loading it into memory. It would then fault, and the app pool would restart.

A very difficult issue to find. We tried sync locks, but to no avail. So we compiled it at app startup, and it then this issue went away.

Cheers