Martin81
asked on
SQL Server Browser Service Continually stopping
I have an Server 2008 R2 server running SQL 2008 SP2, the SQL Server Browser Service stops every 5 mins, I can start it again and it stops exactly 5 mins after the last time it stopped, to the second.
e.g it stops at 19:05:18, I start at 19:06, stops at 19:10:18, start at 19:13, stops at 19:15:18.
So far I've tried
enabling named pipes
changing the service log on account from local service to NT AUTHORITY\SYSTEM
changing the service log on account to my own domain admin account
Rebooting the server
netstat -a -b to check nothing else is using port 1434
After a bit of googling I tried running sqlbrowser.exe -c to troubleshoot and I get no errors, and curiously that keeps running as long as I let it, it doesn't crash after 5 mins.
c:\Program Files (x86)\Microsoft SQL Server\90\Shared>sqlbrowse r.exe -c
SQLBrowser: starting up in console mode
SQLBrowser: starting up SSRP redirection service
SQLBrowser is successfully listening on ::[1434]
SQLBrowser is successfully listening on 0.0.0.0[1434]
[5172]: Waiting for next request...
[5004]: Waiting for next request...
[1352]: Waiting for next request...
[2912]: Waiting for next request...
[5172]: Received request...
[5004]: Received request...
[1352]: Received request...
[2912]: Received request...
[5172]: Waiting for next request...
[5004]: Waiting for next request...
[1352]: Waiting for next request...
[2912]: Waiting for next request...
[5004]: Received request...
CLNT_UCAST_INST w/refresh from 10.100.32.136[60843]
CLNT_UCAST_INST from 10.100.32.136[60843]
[5004]: Waiting for next request...
[2912]: Received request...
[2912]: Waiting for next request...
[2912]: Received request...
CLNT_UCAST_INST w/refresh from 10.100.32.136[60844]
CLNT_UCAST_INST from 10.100.32.136[60844]
[2912]: Waiting for next request...
[5004]: Received request...
[5004]: Waiting for next request...
[2912]: Received request...
CLNT_UCAST_INST w/refresh from 10.100.32.136[60845]
CLNT_UCAST_INST from 10.100.32.136[60845]
[2912]: Waiting for next request...
[5004]: Received request...
[5004]: Waiting for next request...
[5004]: Received request...
CLNT_UCAST_INST w/refresh from 10.100.32.136[60846]
CLNT_UCAST_INST from 10.100.32.136[60846]
[5004]: Waiting for next request...
[2912]: Received request...
[2912]: Waiting for next request...
e.g it stops at 19:05:18, I start at 19:06, stops at 19:10:18, start at 19:13, stops at 19:15:18.
So far I've tried
enabling named pipes
changing the service log on account from local service to NT AUTHORITY\SYSTEM
changing the service log on account to my own domain admin account
Rebooting the server
netstat -a -b to check nothing else is using port 1434
After a bit of googling I tried running sqlbrowser.exe -c to troubleshoot and I get no errors, and curiously that keeps running as long as I let it, it doesn't crash after 5 mins.
c:\Program Files (x86)\Microsoft SQL Server\90\Shared>sqlbrowse
SQLBrowser: starting up in console mode
SQLBrowser: starting up SSRP redirection service
SQLBrowser is successfully listening on ::[1434]
SQLBrowser is successfully listening on 0.0.0.0[1434]
[5172]: Waiting for next request...
[5004]: Waiting for next request...
[1352]: Waiting for next request...
[2912]: Waiting for next request...
[5172]: Received request...
[5004]: Received request...
[1352]: Received request...
[2912]: Received request...
[5172]: Waiting for next request...
[5004]: Waiting for next request...
[1352]: Waiting for next request...
[2912]: Waiting for next request...
[5004]: Received request...
CLNT_UCAST_INST w/refresh from 10.100.32.136[60843]
CLNT_UCAST_INST from 10.100.32.136[60843]
[5004]: Waiting for next request...
[2912]: Received request...
[2912]: Waiting for next request...
[2912]: Received request...
CLNT_UCAST_INST w/refresh from 10.100.32.136[60844]
CLNT_UCAST_INST from 10.100.32.136[60844]
[2912]: Waiting for next request...
[5004]: Received request...
[5004]: Waiting for next request...
[2912]: Received request...
CLNT_UCAST_INST w/refresh from 10.100.32.136[60845]
CLNT_UCAST_INST from 10.100.32.136[60845]
[2912]: Waiting for next request...
[5004]: Received request...
[5004]: Waiting for next request...
[5004]: Received request...
CLNT_UCAST_INST w/refresh from 10.100.32.136[60846]
CLNT_UCAST_INST from 10.100.32.136[60846]
[5004]: Waiting for next request...
[2912]: Received request...
[2912]: Waiting for next request...
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Tried with AV disabled and got the same issue.
I noticed that SQL server browser shows as a separate program in programs and features so I tried a repair installation, that failed as the msi it was installed from was missing. I found another copy of the msi from another server and ran the repair installaion - same problem. Then I ran the msi to re-install, it's actually a newer version. 10.5 so it ran an upgrade, rebooted, the service starts but still stops after 5 mins.
I noticed that SQL server browser shows as a separate program in programs and features so I tried a repair installation, that failed as the msi it was installed from was missing. I found another copy of the msi from another server and ran the repair installaion - same problem. Then I ran the msi to re-install, it's actually a newer version. 10.5 so it ran an upgrade, rebooted, the service starts but still stops after 5 mins.
So you upgraded SQL Browser to 2008 R2 (that's 10.5) and it still crashes after 5 minutes?
Just to clarify, are you seeing the service actually stop every 5 minutes (and is it 5 minutes exactly, or in the neighborhood of 5 minutes?) or is the service just becoming unrepsonsive after around 5 minutes? I've never seen either before, but I'd troubleshoot them differently.
If the service is actually crashing, you can at least mitigate it by enabling the Windows Service to restart automatically if it stops unexpectedly, which will keep it generally available for users, though not resolve the actual problem.
Just to clarify, are you seeing the service actually stop every 5 minutes (and is it 5 minutes exactly, or in the neighborhood of 5 minutes?) or is the service just becoming unrepsonsive after around 5 minutes? I've never seen either before, but I'd troubleshoot them differently.
If the service is actually crashing, you can at least mitigate it by enabling the Windows Service to restart automatically if it stops unexpectedly, which will keep it generally available for users, though not resolve the actual problem.
ASKER
It does stop, it shows as stopped in services mmc, I have the recovery options set to restart on failure but it doesn't actually start the service back up.
It is exactly every 5 mins it stops, not 5 after I start it either, 5 mins since it last crashed. So if I leave it stopped for 4 mins then start it, it will crash after 1 min.
It is exactly every 5 mins it stops, not 5 after I start it either, 5 mins since it last crashed. So if I leave it stopped for 4 mins then start it, it will crash after 1 min.
ASKER
"So you upgraded SQL Browser to 2008 R2 (that's 10.5) and it still crashes after 5 minutes?"
Yep
Yep
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
See screenshots of the 'service has shutdown' events I'm getting in event veiwer, exactly 5 mins apart.
Capture.JPG
Capture2.JPG
Capture.JPG
Capture2.JPG
The service in your list is just a link to the EXE with some parameters - I wouldn't expect recreating it to alter the behavior at all, but it was worth a shot.
What happens if you wait longer than 5 minutes before starting the service again - does it crash immediately? Or does it wait until the next 5 minute interval and then crash at that time? Since it's happening every 5 minutes, I'm inclined to suggest that it's some kind of malformed request coming from a client - not something I've seen the Browser choke on before, but possible. I found an MSDN post here talking about it:
http://blogs.msdn.com/b/mangeshd/archive/2009/08/10/sqlbrowser-fails-with-the-sqlbrowser-service-was-unable-to-process-a-client-request.aspx
Do you have any related events in your Windows or SQL Server Event log around the time of the service crashes? If you do, please post them here.
What happens if you wait longer than 5 minutes before starting the service again - does it crash immediately? Or does it wait until the next 5 minute interval and then crash at that time? Since it's happening every 5 minutes, I'm inclined to suggest that it's some kind of malformed request coming from a client - not something I've seen the Browser choke on before, but possible. I found an MSDN post here talking about it:
http://blogs.msdn.com/b/mangeshd/archive/2009/08/10/sqlbrowser-fails-with-the-sqlbrowser-service-was-unable-to-process-a-client-request.aspx
Do you have any related events in your Windows or SQL Server Event log around the time of the service crashes? If you do, please post them here.
ASKER
If I wait longer that 5 mins, say 7 mins before I restart the service, it will crash exactly 10 mins since it last crashed.
Could it be a stored procedure causing the crash? I assume stored procedures would run on a regular schedule? We have an in house application that queries one of the databases that stores user information, then provisions AD user accounts based on the data it pulls. There's a service associated with that which I stopped to see if that made any difference, which it didn't, but I guess a stored procedure could still be running even with that service stopped?
I'm off home now, it's 23:15 here. I'll have a read of the article you posted and post back tomorrow. Thanks for your help, much appreciated.
Could it be a stored procedure causing the crash? I assume stored procedures would run on a regular schedule? We have an in house application that queries one of the databases that stores user information, then provisions AD user accounts based on the data it pulls. There's a service associated with that which I stopped to see if that made any difference, which it didn't, but I guess a stored procedure could still be running even with that service stopped?
I'm off home now, it's 23:15 here. I'll have a read of the article you posted and post back tomorrow. Thanks for your help, much appreciated.
I definitely suspect it's event-driven by something that's either happening locally or on a remote client, but is unrelated to the service itself - since it happens on five minute intervals, regardless of whether the service is running the whole time, that seems the most likely option.
I'm not sure what flexibility you have, but can you disconnect this server from the network during some kind of maintenance window? Then you could see if it still happens without a network connection - if it doesn't, then it's something happening as a result of a malformed client request of some kind. It would be less telling if it still happened when the network was disconnected, but at least then you could eliminate a remote client as the source and just look locally.
I'm not sure what flexibility you have, but can you disconnect this server from the network during some kind of maintenance window? Then you could see if it still happens without a network connection - if it doesn't, then it's something happening as a result of a malformed client request of some kind. It would be less telling if it still happened when the network was disconnected, but at least then you could eliminate a remote client as the source and just look locally.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
.
ASKER
I'd not thought of AV, I'll give it a try with the AV disabled. Windows Firewall is off on the server so I don't think it's a firewall issue.