Link to home
Start Free TrialLog in
Avatar of StarDusterII
StarDusterII

asked on

IIS Stopped serving pages for 5 min but other users unaffected, why?

I've had this happen a couple of times now... I'm running an IIS 7.0 web server on a couple of different computers (on two different hosting environments) that's serving .asp's with SQL Server connections.  At one point, IIS stops allowing connections for about 5 minutes.  

On the last such event, I was remoted into the server with RDP and could logon to the web site, access pages, and do everything as normal.  However, I tried to log on to the system from another laptop and it timed out.   I had the Task Manager open and the system was hardly working (99% idle, 80% memory, networking < 1%, etc.)

There was nothing of note in any of the Windows Event logs and nothing in any of my application logs.  No application pools or web sites were stopped.

I'm wondering if any of the IIS settings would have an effect on this... positive or negative.  Does anyone know what could be going on or have anything to try?  I'm out of ideas!
Avatar of arnold
arnold
Flag of United States of America image

What components were needed? Database check by the initial access? Is database backups run at that time? Check the IIS log to see whether it saw the connection.
Your site might be under DDoS or plain dos attack when this shows up.
Do you have external firewall through which you can see stats on number of incoming connections?
I interpret *.asp that data are fetched from the sql in the background. Dependend on the way, how the application works, it can be the IIS or the client, whcih fetches the data. If IIS connects to the database, you should also inspec the event logs of IIS and SQL to have an idea, what is going on. Maybe ther are to many connections to the SLQ server at the same time (i.e if it is SQL express).
Also inspect the client libraries on the clients, so you may try to update the native SQL client on the IIS or client, depended from the question, which machine fetches the data.
Usually the native cleint software should have the same version / patch level than the sql.
Avatar of StarDusterII
StarDusterII

ASKER

Will gather information and let you know.
One entry in the System Log:

A process serving application pool 'DefaultAppPool' exceeded time limits during shut down. The process id was '5684'.

The SQL logs are free of anything unusual.  Bembi, you are correct.  The .asp's are built by IIS accessing the database and creating the html that's sent to the client.  All updates have been applied.   However, I'm not sure how to inspect the "client libraries"... I don't understand how that would come into play nor if native SQL Client is even running.
If you have multiple applications/sites customizing each site/application o use their own.
To start with you should test with your busiest site separating to see whether you see app pool recycling errors
The recycling settings might be what triggers the recycling right when the site is seeing large amount of traffic.
My answer was possibly a bit gerneric

With client libraries I mean every component, which belongs to SQL server, the simplest one is SQL server native clinet. Just keep the libraries on the update level of the SQL server.
(Client is here usually the IIS machine, but even can be the end client, dependend how the SQL access is managed).

At the end, for your own asp solution your connection setting determine, which components are use, so i.e. OLEDB or SNAC, for foreign applications it may some times not so quite clear, what is used.

For your App-Pool error I would also follow arnold, possibly to many apps on the same pool what may produce some deadlocks or interferences. So one pool for one application is a good idea, consumes more ram of course, but allows more individual settings for each app.
There is only one web site and two apps within that one.  I'll create a separate pool for each and see what that does.

As far as the libraries go, we don't use SQL server native client as far as I know.  IIS is the only thing connected to the SQL server.
Right, your applications either use DSN or DSnless connection with sql native client or ODBC/ADO connection pooling,etc. many options available.
OK. let me shoot into another direction....
As I understand you right, only one user at a time is thrown out for a limitied time.
After this everything is fine..., right??

Have you thought about a kind of flood mitigation? Means if the number of requests from one single user (or source IP) goes over a defined limit, the connection is refused for some time. Such a mechanism exists in the operationg system but can even be a mechanism from a virus sanner or installed firewall product.

In IIS you can enable the normal logs as well as a failed connection log for each web site. If something is denied directly from IIS, you should see it in the iIS Logs / failed connection logs. If it is a OS mechanism, the source is the event log. If it is any other application, it can be logged into the event log or in a application specific log.
When this happens, no one can log on.  It denies access to everyone, except I was able to access the web site while RDP'd into the server.   When I tried to access the web from my laptop browser, it wouldn't respond... to me or anyone else for a period of time.

I'll change the logs and see if there is anything in there.
Since I have no idea how your site works, it is impossible to guesstimate what might be going on.  If your site is not even loading, that could mean if you have a firewall that the number of sessions it can handle are being exceeded.  Since you do not have two separate systems one serves one site, and another serves another side, there is no way to remove the incoming connection handling as a possible cause without looking to see whether there is anything on the router/firewall that could explain it (network Saturation during the period in question) It is possible to allow RDP (QoS prioritization) while HTTP being saturated.

Internet <=> router/firewall <=> [saturation level reached ]            Web server
internet <=> router/firewall <=> RDP       (connect to web server works) /

Using the logs specifically whether connections were being seen on the web server during the five minute interval.

Using monitoring tools such as Nagios, OpenNMS, zabbix to both monitor the web server internally and one that monitors the web server from an external source could provide a context under which you can determine what if anything is causing this issue as well as have more data that will help identify the issue.

With those types of monitoring tools, you can also identify if this issue occurs at the same time like clock work, at which point looking through what tasks you have setup on the web server, database server such a backups, etc. that could explain this issue.
if you suspect that scheduled tasks have no impact on this, changing the schedule from one of the tasks that deal with the database to another time, then if the issue shifts with this task, you have found your culprit.
ASKER CERTIFIED SOLUTION
Avatar of Bembi
Bembi
Flag of Germany image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Didn't actually fix the problem of course, but the tools were an excellent idea.