Solved

SQL server stopped without any reason

Posted on 2016-08-05
27
129 Views
Last Modified: 2016-09-11
hi,

one of our SQL server running SQL server 2008 with SP4 stopped without any reason and the error from the log is:


1)            A user request from the session with SPID 64 generated a fatal exception. SQL Server is terminating this session. Contact Product Support Services with the dump produced in the log directory.
2)            SQL Server Assertion: File: <qxcntxt.cpp>, line=1052 Failed Assertion = 'cref == 0'. This error may be timing-related. If the error persists after rerunning the statement, use DBCC CHECKDB to check the database for structural integrity, or restart the server to ensure in-memory data structures are not corrupted
3)            SQL Server is terminating because of fatal exception c0000005. This error may be caused by an unhandled Win32 or C++ exception, or by an access violation encountered during exception handling. Check the SQL error log for any related stack dumps or messages. This exception forces SQL Server to shutdown. To recover from this error, restart the server (unless SQLAgent is configured to auto restart).
4) Database Engine Instance=SQL01-CDS;Mail PID=8544;Error Message:

1) Exception Information
===================
Exception Type: System.Data.SqlClient.SqlException
Errors: System.Data.SqlClient.SqlErrorCollection
Class: 11
LineNumber: 0
Number: -2
Procedure:
Server: 192.168.108.6,1435
State: 0
Source: .Net SqlClient Data Provider
ErrorCode: -2146232060
Message: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
Data: System.Collections.ListDictionaryInternal
TargetSite: Void OnError(System.Data.SqlClient.SqlException, Boolean)
HelpLink: NULL

StackTrace Information
===================
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject stateObj, UInt32 error)
   at System.Data.SqlClient.TdsParserStateObject.ReadSni(DbAsyncResult asyncResult, TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParserStateObject.ReadNetworkPacket()
   at System.Data.SqlClient.TdsParserStateObject.ReadBuffer()
   at System.Data.SqlClient.TdsParserStateObject.ReadByte()
  at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at Microsoft.SqlServer.Management.SqlIMail.Server.DataAccess.DataAccessAdapter.PostResponse(Response r, Guid convHandle)
   at Microsoft.SqlServer.Management.SqlIMail.IMailProcess.ThreadCallBack.PostResponseIfNeeded(MailInfo mi, Response r)
   at Microsoft.SqlServer.Management.SqlIMail.IMailProcess.ThreadCallBack.MailOperation(Object o)
0
Comment
Question by:marrowyung
  • 16
  • 8
  • 2
  • +1
27 Comments
 
LVL 45

Expert Comment

by:Vitor Montalvão
ID: 41744102
You may need to collect all the logs and dump files and send it to Microsoft support.  We can't do much with the information you provided.
0
 
LVL 11

Assisted Solution

by:Nakul Vachhrajani
Nakul Vachhrajani earned 125 total points
ID: 41744158
Once you have collected the error log dumps, please try to do the following:
1. Ensure that the system time is synchronized to a central time server and is correct
2. Try to restart the SQL Server Service. If a corruption occurred in the tempdb, restarting the service will reset the tempdb and the service should successfully restart
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41746717
our checkdb result returns good :

CHECKDB found 0 allocation errors and 0 consistency errors in database
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41751486
any comment ?
0
 
LVL 45

Expert Comment

by:Vitor Montalvão
ID: 41751597
Looks like you didn't follow our recommendation to collect the logs and you went to a different path (nobody asked about CHECKDB).

If you want to follow our recommendation then collect the log information. You can use SQLdiag utility for that.

Cheers
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41751831
"Looks like you didn't follow our recommendation to collect the logs and you went to a different path (nobody asked about CHECKDB)."

you mean this :

  "You may need to collect all the logs and dump files and send it to Microsoft support."

restart SQL server has been followed and it doesn't help.
0
 
LVL 45

Assisted Solution

by:Vitor Montalvão
Vitor Montalvão earned 125 total points
ID: 41751839
restart SQL server has been followed and it doesn't help.
That was only a guess from Nakul in case of tempdb corruption.
I, myself, can't say nothing without getting the result from SQLdiag.
0
 
LVL 57

Accepted Solution

by:
Jim Dettman (Microsoft MVP/ EE MVE) earned 250 total points
ID: 41751909
Is this running on a virtual box?   If so, you might want to wait and see if it happens again.  I'd also review the servers event logs to see if anything unusual is going on with the server.

As pointed out in the error message:

"this error may be caused by an unhandled Win32 or C++ exception, or by an access violation encountered during exception handling. "

 The error c0000005 is a memory access violation.  The root cause of that was:

"Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding."

 or in other words, the server simply took too long while trying to get data.  This was part of a mail process, which might have a bug (in that it had an unhandled exception).

As far as the "fix":

"This exception forces SQL Server to shutdown. To recover from this error, restart the server (unless SQLAgent is configured to auto restart)."

  Already suggested and done.

"This error may be timing-related. If the error persists after rerunning the statement, use DBCC CHECKDB to check the database for structural integrity, or restart the server to ensure in-memory data "

  You've also done the DBCC CHECKDB with no errors, so chances are it was timing related.   I'd wait to see if it happens again, and if so:

"Contact Product Support Services with the dump produced in the log directory."

or as Vitor said, post the result of SQLDiag.

Jim.
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41753563
"Is this running on a virtual box?  "

all SQL serve here run under hyper-V .

" I'd also review the servers event logs to see if anything unusual is going on with the server.
"
the log is the one I check from the log.
0
 
LVL 57
ID: 41753594
<<the log is the one I check from the log.>>

I meant outside of the posted error; network issues?  storage?

Something that might explain why a timeout occurred.

Jim.
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41757337
"I meant outside of the posted error; network issues?  storage? "

I checked all that and we also have network problem too but the network problem is one day later and it cause replication lag behind.

the SQL server restart one ahead.
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41757343
vitor,

sorry the SQLdiag page is a bit complex and I wan to use the simpiest way to run that command, what syntax you usually use to run that ?
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41757368
0
Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

 
LVL 1

Author Comment

by:marrowyung
ID: 41757373
I don't even find example .xml file from http://sqlnexus.codeplex.com/
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41762036
any update for me?
0
 
LVL 45

Expert Comment

by:Vitor Montalvão
ID: 41781128
I am sorry, marrowyung. I missed somehow your comments for this question.

actually what XML file you use ? the default one in the same path ?
Yes, the default one should be fine.
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41781351
hi,

yeah, I know there are default one, but usually what is the full command you use for that?

that command page is a bit complex to read.
0
 
LVL 45

Expert Comment

by:Vitor Montalvão
ID: 41781483
Check if with this tool it will be more easy for you to configure the XML file.
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41784195
hi all,

these errors just come again:

DESCRIPTION:	A user request from the session with SPID 164 generated a fatal exception. SQL Server is terminating this session. Contact Product Support Services with the dump produced in the log directory.

DESCRIPTION:	Internal error. Buffer provided to read column value is too small. Run DBCC CHECKDB to check for any corruption.

Open in new window


before that I can only see our idera compliance manger is running and collect trace file.
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41784522
send all information to idera and see what they said.

SQL server with SP3 don't have that problem but at this moment, it seems the one with SP4 has problem.

we will upgrade it to SQL server 2014 with SP2+ CU1 very soon but experience this, this error already strong enough for us to restart SQL server as error keep coming. we have no choice.
0
 
LVL 45

Expert Comment

by:Vitor Montalvão
ID: 41784552
It will also be good to send the dump file to Microsoft.
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41785661
yes, but I think I will upload to here first.

which link I can send to MS? they cost ?
0
 
LVL 45

Expert Comment

by:Vitor Montalvão
ID: 41785669
which link I can send to MS? they cost ?
Is the .dmp file. I'm guessing that your company has a support contract with MS. That's usual when you have their products.
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41785851
I think I will try sQLdiag tomrorrow or some day this week and update you all.
0
 
LVL 45

Expert Comment

by:Vitor Montalvão
ID: 41785853
SQLDiag is also able to read dump files so it's a good idea too.
Cheers
0
 
LVL 1

Author Comment

by:marrowyung
ID: 41793676
I guess now we know why that box stop without any reason as we are setting up Idera compliance manager (CM) to audit a SQL 2008 server with SP4, it doesn't even happen to SP3 at all.

DESCRIPTION:    A user request from the session with SPID 207 generated a fatal exception. SQL Server is terminating this session. Contact Product Support Services with the dump produced in the log directory.

Is caused by CM is the audited SQL server is running SQL server 2008 with SP4 when DML audit is turned on

This one:


DESCRIPTION:    Internal error. Buffer provided to read column value is too small. Run DBCC CHECKDB to check for any corruption.

Is caused by CM is the audited SQL server is running SQL server 2008 with SP4 when Before and AFTER audit is on.
0
 
LVL 1

Author Closing Comment

by:marrowyung
ID: 41793679
tks all for that.
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

In this article we will get to know that how can we recover deleted data if it happens accidently. We really can recover deleted rows if we know the time when data is deleted by using the transaction log.
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
Familiarize people with the process of retrieving data from SQL Server using an Access pass-thru query. Microsoft Access is a very powerful client/server development tool. One of the ways that you can retrieve data from a SQL Server is by using a pa…
Viewers will learn how the fundamental information of how to create a table.

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now