IIS6.0
Win2003 Std w/ SP2
SQL Server 2005 w/ SP2
The web sitehas been extremely unstable since going into production (~1wk ago), crashing several times per day. Were seeing this error about the same time as the website becomes unavailable and Im wondering if you guys have seen this before:
-----------------------------------------------------------------------------------------------
Event Type: Error
Event Source: Report Server Windows Service (MSSQLSERVER)
Event Category: Management
Event ID: 107
Date: 2/21/2008
Time: 8:12:55 AM
User: N/A
Computer: WWW2
Description:
Report Server Windows Service (MSSQLSERVER) cannot connect to the report server database.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
And in the SQL logs at C:\Program Files\Microsoft SQL Server\MSSQL.2\Reporting Services\LogFiles, I see the last two lines of the message below just before the site goes down (Ive included some of the lines above that section in case it is helpful to you):
------------------------------------------------------------------------
ReportingServicesService!library!a!2/21/2008-07:07:38:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.ReportServerDatabaseUnavailableException: The report server cannot open a connection to the report server database. A connection to the database is required for all requests and processing., ;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.ReportServerDatabaseUnavailableException: The report server cannot open a connection to the report server database. A connection to the database is required for all requests and processing. ---> System.Data.SqlClient.SqlException: An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)
at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at Microsoft.ReportingServices.Library.ConnectionManager.OpenConnection()
--- End of inner exception stack trace ---
ReportingServicesService!servicecontroller!9!2/21/2008-08:12:39:: i INFO: Recycling the service from default domain
ReportingServicesService!servicecontroller!9!2/21/2008-08:12:39:: i INFO: New app domain started
---------------------------------------------------------------------
Youll see that the last line of this error seems to be starting a new connection with the SQL Reporting Service. Then the first error up at the top of this email happens within milliseconds. And after that, the site goes down.
In the SQL Server Config ManageràSQL Native Client ConfigàClient Protocols and SQL Server 2005 Network ConfigàProtocols for MSSQLSERVER, I see that the TCP keep-alive is 30,000. A search online indicates this value is in milliseconds, but if I ignore that and assume it is seconds, that's about 8 hours, which is the regular interval at which the site seems to go down. This could be entirely coincidental, but may be helpful.
We're not actually using the SQl Reporting database, so I stopped that and the Windows service of the same name.
Rebooting the server always (temporarily) solves the problem, but since this is a product server, that's obviously not acceptable. This site briefly went through a "demo" stage on this same hardware and we didn't have these problems. Before going into production, I created a new local account for the website to run as and set file/folder permissions appropriately (I think). I also set the Windows firewall to only allow ports TCP 80, 443, 1433. The last one is the SQL default port and if I don't include it in the firewall exceptions then the site doesn't run at all. I also set TCP/IP filtering in the properties of the NIC also specifying these same ports/protocols. Obviously in troubleshooting one would think, "well, what has changed since it last worked?". I'm not inclined to think these changes are the issue since it runs under this config for at least 8hrs at a time.
And yet more info:
The server has two NIC's, one with the public IP info, the other internal (private IP, etc.), both with statically assigned info. They have two different gateways (public INT has a public IP, private INT has an internal LAN gateway). This didn't happen when I initially set up the server, but since the problems began whenever I open and close the NIC properties pages I get a little pop-up saying "there are two different gateways, are you sure you want to do this...?".
Whenever the site is unavailable, if I do an "ipconfig /all", the public interface doesn't show a value for 'default gateway'. If I look at the NIC properties it still clearly shows the correct gateway. I am not able to ping the public gateway IP during these times.
I've contacted the datacenter hosting this server and they assure me there are no network issues going on, which I'm inclined to believe since none of my other servers have any problem, including other web serves using the same gateway.
Although a reboot does always "fix" this, sometimes just RDP'ing to the box seems to "wake up" the connection and the site comes back up.
I know this is a lot of info, but i'm hoping something here will help you help me.
by: JulianvaPosted on 2008-02-23 at 11:39:47ID: 20966290
Disable one of the Nic see if that makes a difference.