Link to home
Start Free TrialLog in
Avatar of Sbsystems
Sbsystems

asked on

Windows Server 2008 R2 Random Blue Screen Hyper-V

Hi EE,

I have a Windows Server 2008 R2 RDS Server that is randomly blue screening. I have attached a dump below but can't seem to see why its occurring. This is a virtualized server running on Windows Server Standard 2012 R2 and this issue was occurring until I completely rebuild it over a month ago, but it seems to have reappeared.

Can anyone please help?

Thanks
Capture.PNG
Avatar of Mal Osborne
Mal Osborne
Flag of Australia image

Looks like an errant kernel mode driver issue. Possibly something like antivirus, or monitoring.

Try throwing the minidump at the below online dump analyser and see if it points to anything.

http://www.osronline.com/page.cfm?name=analyze
Avatar of Sbsystems
Sbsystems

ASKER

Instant Online Crash Analysis, brought to you by OSR Open Systems Resources, Inc.

Show DivPrimary Analysis

Crash Dump Analysis provided by OSR Open Systems Resources, Inc. (http://www.osr.com)
Online Crash Dump Analysis Service
See http://www.osronline.com for more information
Windows 7 Kernel Version 7601 (Service Pack 1) MP (6 procs) Free x64
Product: Server, suite: TerminalServer
Built by: 7601.18933.amd64fre.win7sp1_gdr.150715-0600
Machine Name:
Kernel base = 0xfffff800`0161f000 PsLoadedModuleList = 0xfffff800`01866730
Debug session time: Thu Oct  1 01:53:02.426 2015 (UTC - 4:00)
System Uptime: 0 days 12:50:49.541
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CRITICAL_OBJECT_TERMINATION (f4)
A process or thread crucial to system operation has unexpectedly exited or been
terminated.
Several processes and threads are necessary for the operation of the
system; when they are terminated (for any reason), the system can no
longer function.
Arguments:
Arg1: 0000000000000003, Process
Arg2: fffffa8008b99060, Terminating object
Arg3: fffffa8008b99340, Process image file name
Arg4: fffff80001996e70, Explanatory message (ascii)

Debugging Details:
------------------

----- ETW minidump data unavailable-----*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!KPRCB                                      ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!KPRCB                                      ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
TRIAGER: Could not open triage file : e:\dump_analysis\program\triage\modclass.ini, error 2
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that   ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: nt!_KPRCB                                     ***
***                                                                   ***
*************************************************************************

PROCESS_OBJECT: fffffa8008b99060

IMAGE_NAME:  csrss.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  0

MODULE_NAME: csrss

FAULTING_MODULE: 0000000000000000

EXCEPTION_CODE: (Win32) 0 (0) - The operation completed successfully.

BUGCHECK_STR:  0xF4_fffff88000000000

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  WIN7_DRIVER_FAULT_SERVER

CURRENT_IRQL:  0

STACK_TEXT:  
fffff880`05ce69c8 fffff800`01a28cb2 : 00000000`000000f4 00000000`00000003 fffffa80`08b99060 fffffa80`08b99340 : nt!KeBugCheckEx
fffff880`05ce69d0 fffff800`019dfd8b : ffffffff`ffffffff fffffa80`08112b50 fffffa80`08b99060 fffffa80`07abab10 : nt!PspCatchCriticalBreak+0x92
fffff880`05ce6a10 fffff800`0194a484 : ffffffff`ffffffff 00000000`00000001 fffffa80`08b99060 00000000`00000008 : nt! ?? ::NNGAKEGL::`string'+0x29ce6
fffff880`05ce6a60 fffff800`01691853 : fffffa80`08b99060 fffff880`00000000 fffffa80`08112b50 fffffa80`00000004 : nt!NtTerminateProcess+0xf4
fffff880`05ce6ae0 00000000`7739dc7a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
00000000`02a9e808 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x7739dc7a


STACK_COMMAND:  kb

FOLLOWUP_NAME:  MachineOwner

FAILURE_BUCKET_ID:  X64_0xF4_fffff88000000000_IMAGE_csrss.exe

BUCKET_ID:  X64_0xF4_fffff88000000000_IMAGE_csrss.exe

Followup: MachineOwner
---------



This free analysis is provided by OSR Open Systems Resources, Inc.
 Want a deeper understanding of crash dump analysis? Check out our Windows Kernel Debugging and Crash Dump Analysis Seminar (opens in new tab/window)
Show DivCrash Code Links



Show DivLoaded Module List



Show DivRaw Stack Contents



Show DivDump Header Information



Show DivStrings
OK, nothing much of use there this time.

You may need to start uninstalling anything with a kernel mode driver, and see when the problem goes away. Backups software, antivirus, 3rd party disk encryption and inventory agents often include a kernel mode driver.
CSRSS.exe is part of the interpretation system between 32-bit (Win32) and 64-bit subsystems.

Is there A/V in the VM? Uninstall it.

What apps are running in the VM?

Have the Integration Services been updated at all since the VM was deployed? If the host has been updated sometimes IS also needs to be updated within the VM. The host's Event Logs will flag such discrepancies.

Run the IS installer to see if it accepts the need to update.
We originally build the VM on another host then copied the vhd to a new host and rebuild the virtual.

I ran the IS installer overnight and it says its up to date.  

Trend Micro Worry Free Business Security Advanced Agent is running on it also.

Applications:

MYOB AO
MYOB AO Classic
MYOB Accountright
Office 2010
Adobe Reader
Fiscal Finance
Firefox
Internet Explorer
Backup Exec 2015 Remote Agent
Trend Micro Worry Free Business Security Advanced Agent
Cute PDF
Cisco VPN Client
Cashflow Manager 9
ASKER CERTIFIED SOLUTION
Avatar of Mal Osborne
Mal Osborne
Flag of Australia image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Without the Backup Agent we cant backup the server,

Without the Trend Micro Agent we wont be protected from virus infections,

Without the Cisco VPN Client we cant lodge tax returns,

Are there any other alternatives?
Replace BuE with ShadowProtect.

Trend is a troublemaker in our books. Most infections today get past traditional A/V. Nothing beats a good GPO security structure in a RDS setting.

Cisco VPN Client has been on the radar for bad conflicts with updates lately.
Unfortunately, that's not an option and I haven't experienced any issues with BE2015.

What security settings would you suggest applying via GPO?

I understand however the Cisco VPN client is required to lodge tax returns within MYOB AO.
I think I have found the issue to be with the Cisco VPN Client. A user was lodging a tax return just before the crash occurred. Going to reinstall over the long weekend and see how we go. Will keep this post updated.

Thanks everyone for your help :)
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.