Link to home
Start Free TrialLog in
Avatar of SECC_IT
SECC_ITFlag for United States of America

asked on

Analyze WinDbg Crash

Our Remote Desktop Servers crash on a regular basis. It happens when they are overloaded - high CPU use. There are three of them; the latter two were cloned from the first one. The same behavoir happens in all three.

These machines are Server 2008 64 bit Virtual Machines hosted on a Windows Server 2008 64 bit machine using Hyper-V. They all have 4 processors assigned and more RAM than they need.

The host server has a total of 8 cores available through two quad core 2.8Ghz processors. There are three host servers, all with the same configuration. The crashing happens on the VMs only, never the hosts. The most a host has on it are three VMs, all similarly configured. The only VMs that reboot are the ones with FMP/SQL on them.

There is a blue screen error, which you will see below. The analyzed contents of the crash dump file are below that. I ran it through WinDbg, and posted that info.

The key thing here to understand is this:

RDS1 was stable until FileMaker Pro 11 and the SQL client were installed as part of a maximEyes 11 installation (optimetry software by First Insight).

This crashing has been occurring since the upgrade.

The crashing is obviously related to the FMP and SQL software. Whether it is the software itself, or simply the high CPU use caused by the software I have not been able to figure out. The server can have anywhere from 15 to 20 users logged on, all using the FMP software, some using the SQL.

The Blue Screen message is from the details on the box that appears in Windows after a system reboots.

The Crash Dump is the analysis done in WinDbg.


-----------BLUE SCREEN------------
Problem signature:
  Problem Event Name:      BlueScreen
  OS Version:      6.1.7601.2.1.0.18.10
  Locale ID:      1033

Additional information about the problem:
  BCCode:      ab
  BCP1:      0000000000000020
  BCP2:      0000000000000350
  BCP3:      0000000000000000
  BCP4:      0000000000000001
  OS Version:      6_1_7601
  Service Pack:      1_0
  Product:      18_3

----------CRASH DUMP FILE------------

     Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck AB, {1d, 3a0, 0, 1}

Probably pool corruption caused by Tag:  Pool

Followup: MachineOwner
---------

3: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

SESSION_HAS_VALID_POOL_ON_EXIT (ab)
Caused by a session driver not freeing its pool allocations prior to a
session unload.  This indicates a bug in win32k.sys, atmfd.dll,
rdpdd.dll or a video driver.
Arguments:
Arg1: 000000000000001d, session ID
Arg2: 00000000000003a0, number of paged pool bytes that are leaking
Arg3: 0000000000000000, number of nonpaged pool bytes that are leaking
Arg4: 0000000000000001, total number of paged and nonpaged allocations that are leaking.
      nonpaged allocations are in the upper half of this word,
      paged allocations are in the lower half of this word.

Debugging Details:
------------------


CORRUPTING_POOL_TAG:  Pool

DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT

BUGCHECK_STR:  0xAB

PROCESS_NAME:  csrss.exe

CURRENT_IRQL:  0

LAST_CONTROL_TRANSFER:  from fffff80001a6e61f to fffff800016cf1c0

STACK_TEXT:  
fffff880`0ddc0ac8 fffff800`01a6e61f : 00000000`000000ab 00000000`0000001d 00000000`000003a0 00000000`00000000 : nt!KeBugCheckEx
fffff880`0ddc0ad0 fffff800`0190b917 : fffff880`04f0bb40 fffff880`04f0b000 fffff880`04f0b000 fffffa80`27f8cb00 : nt!MiCheckSessionPoolAllocations+0x13f
fffff880`0ddc0b10 fffff800`01a08fa5 : fffff880`0ddc0ba8 fffffa80`27f8cb00 ffffffff`ffffffd5 fffff880`04f0b000 : nt!MiDereferenceSessionFinal+0x137
fffff880`0ddc0bb0 fffff800`016a039c : fffff800`01854940 00000000`00000001 00000000`00000000 fffffa80`28066060 : nt! ?? ::NNGAKEGL::`string'+0x25025
fffff880`0ddc0be0 fffff800`019a06da : fffff8a0`1f6da060 00000000`00000000 00000000`00000000 fffffa80`27f8cb00 : nt!MmCleanProcessAddressSpace+0x610
fffff880`0ddc0c30 fffff800`019a0abd : 00000000`00000000 fffff800`01965e01 00000000`00000000 fffffa80`28021b00 : nt!PspExitThread+0x56a
fffff880`0ddc0d30 fffff800`016bfec6 : fffff880`020e1180 00000000`00000080 fffffa80`27f8cb00 00000000`00000200 : nt!PspTerminateThreadByPointer+0x4d
fffff880`0ddc0d80 00000000`00000000 : fffff880`0ddc1000 fffff880`0ddbb000 fffff880`0ddc0710 00000000`00000000 : nt!KxStartSystemThread+0x16


STACK_COMMAND:  kb

FOLLOWUP_IP:
nt!MiCheckSessionPoolAllocations+13f
fffff800`01a6e61f cc              int     3

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  nt!MiCheckSessionPoolAllocations+13f

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

DEBUG_FLR_IMAGE_TIMESTAMP:  4fa390f3

IMAGE_NAME:  memory_corruption

FAILURE_BUCKET_ID:  X64_CORRUPTING_POOLTAG_Pool

BUCKET_ID:  X64_CORRUPTING_POOLTAG_Pool

Followup: MachineOwner
---------
Avatar of Leon Fester
Leon Fester
Flag of South Africa image

General troubleshooting for BSOD would include making sure that drivers and service packs are up to date.

Considering you're running Virtual servers, I'd suggest that you check both the host and the guest for latest updates.

The possibility exists that corruption could have been caused by Virus or storage issues so it would be good to run hdd scan and av scans.

I've seen this discussion on other forums:
http://forums.citrix.com/message.jspa?messageID=1584697
http://www.brianmadden.com/forums/t/2/20935.aspx
Avatar of SECC_IT

ASKER

I am using Server 2008 R2, so the first link does not apply. The citrix link had a possible fix, but when I tried to apply it to my server I received a "this does not apply" message.

Any other thoughts out there?
ASKER CERTIFIED SOLUTION
Avatar of SECC_IT
SECC_IT
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of SECC_IT

ASKER

Provided own answer.