Link to home
Start Free TrialLog in
Avatar of djMundy
djMundyFlag for Australia

asked on

Server BSOD, seeking advice on reading debug information

Hi Experts,

I have a terminal server that is blue screening at seemingly random times throughout the day. The server is running Windows 2003 x64bit version.

I've done some reading and installed windbg, but this is the first time I've attempted this and am after some help to read the output. I'm confused as to whether it's a driver issue, faulty RAM, or faulting application (I see reference to outlook.exe)

Many thanks,
Daniel
Microsoft (R) Windows Debugger Version 6.8.0004.0 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
 
 
Loading Dump File [C:\WINDOWS\MEMORY.DMP]
Kernel Summary Dump File: Only kernel address space is available
 
Symbol search path is: SRV*c:\temp*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows Server 2003 Kernel Version 3790 (Service Pack 2) MP (8 procs) Free x64
Product: Server, suite: TerminalServer
Built by: 3790.srv03_sp2_gdr.070321-2337
Kernel base = 0xfffff800`01000000 PsLoadedModuleList = 0xfffff800`011d4140
Debug session time: Thu Dec  6 16:32:41.194 2007 (GMT+10)
System Uptime: 0 days 2:22:20.031
Loading Kernel Symbols
......................................................................................................................
Loading User Symbols
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
Loading unloaded module list
.....
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
 
Use !analyze -v to get detailed debugging information.
 
BugCheck 4E, {99, 0, 0, 0}
 
Page c3f2a not present in the dump file. Type ".hh dbgerr004" for details
Page cec78 not present in the dump file. Type ".hh dbgerr004" for details
Page c402b not present in the dump file. Type ".hh dbgerr004" for details
Page cedc0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page cecbb not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fee not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page cecbe not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f19 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page cecc7 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page cecfb not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
 
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
 
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
Probably caused by : memory_corruption ( nt!MiFreeWsleList+505 )
 
Followup: MachineOwner
---------
 
7: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
 
PFN_LIST_CORRUPT (4e)
Typically caused by drivers passing bad memory descriptor lists (ie: calling
MmUnlockPages twice with the same list, etc).  If a kernel debugger is
available get the stack trace.
Arguments:
Arg1: 0000000000000099, A PTE or PFN is corrupt
Arg2: 0000000000000000, page frame number
Arg3: 0000000000000000, current page state
Arg4: 0000000000000000, 0
 
Debugging Details:
------------------
 
Page c3f2a not present in the dump file. Type ".hh dbgerr004" for details
Page cec78 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3f53 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd5 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3fd0 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3f5d not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page c3edf not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
Page ced49 not present in the dump file. Type ".hh dbgerr004" for details
 
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
 
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
 
OVERLAPPED_MODULE: Address regions for 'eeCtrl64' and 'imapi.sys' overlap
 
BUGCHECK_STR:  0x4E_99
 
DEFAULT_BUCKET_ID:  DRIVER_FAULT
 
PROCESS_NAME:  OUTLOOK.EXE
 
CURRENT_IRQL:  2
 
LAST_CONTROL_TRANSFER:  from fffff800010e7a90 to fffff8000102e890
 
STACK_TEXT:  
fffffadf`1cd6fa08 fffff800`010e7a90 : 00000000`0000004e 00000000`00000099 00000000`00000000 00000000`00000000 : nt!KeBugCheckEx
fffffadf`1cd6fa10 fffff800`010e87d1 : fffffadf`32386f40 fffffadf`1cd6fb70 fffffadf`34cdf660 fffff700`000844a0 : nt!MiFreeWsleList+0x505
fffffadf`1cd6fb50 fffff800`013cdf55 : fffff97f`f7db6e80 ffffffff`ffffffff fffffadf`34cdf660 fffffadf`32386f40 : nt!MiEmptyWorkingSet+0x24e
fffffadf`1cd6fc00 fffff800`013ce3db : fffffadf`32386c20 fffff97f`f3d31010 00000000`00000000 00000000`00000001 : nt!MmAdjustWorkingSetSizeEx+0x85
fffffadf`1cd6fc70 fffff97f`ff1be527 : fffff97f`f7db6e80 fffff97f`f7990000 00000000`00000db6 00000000`00000006 : nt!MmAdjustWorkingSetSize+0x1b
fffffadf`1cd6fcc0 fffff97f`ff10ffae : fffff680`00000001 fffffadf`00000006 fffff97f`f7d6a2c0 fffff97f`f7db6e80 : win32k!xxxMinMaximize+0x1218
fffffadf`1cd6fe20 fffff97f`ff1aa3ca : fffff97f`f7db6e80 fffffa80`00000008 00000000`00000006 00000000`00000000 : win32k!xxxShowWindow+0x189
fffffadf`1cd6feb0 fffff97f`ff11e480 : fffff97f`f7db6e80 00000000`0000f020 00000000`00000112 fffff97f`f7db6e80 : win32k!xxxSysCommand+0x3fe
fffffadf`1cd6ff40 fffff97f`ff0a02d7 : 00000000`00000001 fffff97f`ff0d5f0a 00000000`00020210 00000000`000403cf : win32k!xxxRealDefWindowProc+0xbe6
fffffadf`1cd70070 fffff97f`ff0a5a31 : fffffa80`0558ef48 00000000`00020210 00000000`00000112 00000000`0000029e : win32k!NtUserfnDWORD+0x27
fffffadf`1cd700b0 fffff800`0102e33d : fffff6fb`7dbed000 00000000`00000000 fffffadf`32386f90 fffffadf`1cd70260 : win32k!NtUserMessageCall+0x142
fffffadf`1cd70150 00000000`6b2b5e8a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x3
00000000`0013d398 fffff800`01026640 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x6b2b5e8a
fffffadf`1cd70550 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 fffffadf`1cd70c70 : nt!KiCallUserMode
 
 
STACK_COMMAND:  kb
 
FOLLOWUP_IP: 
nt!MiFreeWsleList+505
fffff800`010e7a90 cc              int     3
 
SYMBOL_STACK_INDEX:  1
 
SYMBOL_NAME:  nt!MiFreeWsleList+505
 
FOLLOWUP_NAME:  MachineOwner
 
MODULE_NAME: nt
 
DEBUG_FLR_IMAGE_TIMESTAMP:  460280f7
 
IMAGE_NAME:  memory_corruption
 
FAILURE_BUCKET_ID:  X64_0x4E_99_nt!MiFreeWsleList+505
 
BUCKET_ID:  X64_0x4E_99_nt!MiFreeWsleList+505
 
Followup: MachineOwner
---------

Open in new window

Avatar of djMundy
djMundy
Flag of Australia image

ASKER

I'm not sure if I'm looking at the right parts, but after running "!analyze -v" I see the following that may be useful:

249: PFN_LIST_CORRUPT (4e)
460: OVERLAPPED_MODULE: Address regions for 'eeCtrl64' and 'imapi.sys' overlap
464: DEFAULT_BUCKET_ID:  DRIVER_FAULT
466: PROCESS_NAME:  OUTLOOK.EXE
505: IMAGE_NAME:  memory_corruption

I'm also not sure about all of the "Page xx not present in the dump file" messages - are they anything to worry about? Reading the help found by typing ".hh dbgerr004" I see that I can suppress the messages, but I'm not sure if I should. I think I could also fix this by telling Windows to do a full dump (currently it's only dumping the kernel) but again not sure if this is required.

Thanks,
Daniel
ASKER CERTIFIED SOLUTION
Avatar of sirbounty
sirbounty
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of djMundy

ASKER

Hi sirbounty,

Thanks for your response. I've just checked another dump file and I don't think it's Outlook causing the crash,

PROCESS_NAME: Idle

I'm heading on site at the end of the day to start a memtest, I'll post back the results tomorrow. We can't have any downtime on this server, do you think 15 hours will be enough time to test the 8GB RAM this server has?

Page file is set to 12288MB and is on the system drive (C:) which is a RAID1 mirror on 2x 70GB 15k RPM SAS drives. I've been monitoring the server using perfmon and the page file usage never goes above about 15%.

When I try to change the debugging option to save a full dump, the only options are none, small memory dump (128k), and kernel memory dump. Do you know why this could be? The C: drive has plenty of free space (32GB).

Depending on whether this resolves the issue I'll try disabling some programs (the users are restricted to running only a select few applications by group policy) but this will have a fair impact on their work.

Many thanks,
Daniel
I'm glad you weren't able to change it to a full...no, you don't want a full dump with 8GB RAM and 12GB pagefile - why's the page file so large?  I'd suggest dropping that down to 2048 max.  
Also, can you post your boot.ini file here?
Avatar of djMundy

ASKER

We've also been having performance issues with the server, someone suggested increasing the page file, so I set it to 1.5x 8GB. I'll drop it down to 4096MB - I feel that 2048 may be too low as the server is currently using 15% of 12288MB, which is about 1800MB.

I ran a memtest last night, it only completed 7 passes but showed no errors. I'll be able to run a longer test on the weekend.
I'm not on our senior engineering team, but I know we never go above 2048 on our page file, for any server with above 2GB of RAM installed.

http://support.microsoft.com/kb/889654 details how to find the actual setting that should be used, but technically that large page file could lead to excessive paging, something that shouldn't be necessary with 8GB of RAM. (unless you're running more than term server?).

Nothing squirrlely in the boot.ini, I take it?
Avatar of djMundy

ASKER

I haven't changed anything in boot.ini so it's pretty much default (see code snippet).

All that's running on the server is Server 2003 x64 with terminal server, applications are Office 2000, EA3000 (a government supplied application for employment agencies) and a database client (FileMaker Pro 8.5).

I'm reading through that document now to determine the best page file size.

I'm also about to search HP's website for any new BIOS updates.

Thanks,
Daniel
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003 Standard x64 Edition" /noexecute=optout /fastdetect

Open in new window

Avatar of djMundy

ASKER

I've attached another memory dump in case it's useful.

Thanks,
Daniel

Microsoft (R) Windows Debugger Version 6.8.0004.0 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
 
 
Loading Dump File [C:\WINDOWS\MEMORY.DMP]
Kernel Summary Dump File: Only kernel address space is available
 
Symbol search path is: SRV*c:\temp*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows Server 2003 Kernel Version 3790 (Service Pack 2) MP (8 procs) Free x64
Product: Server, suite: TerminalServer
Built by: 3790.srv03_sp2_gdr.070321-2337
Kernel base = 0xfffff800`01000000 PsLoadedModuleList = 0xfffff800`011d4140
Debug session time: Wed Dec 12 13:30:59.232 2007 (GMT+10)
System Uptime: 0 days 5:07:30.687
Loading Kernel Symbols
......................................................................................................................
Loading User Symbols
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
Loading unloaded module list
.....
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
 
Use !analyze -v to get detailed debugging information.
 
BugCheck 3B, {c0000005, fffff80001033153, fffffadf1c1580b0, 0}
 
Page c4aa7 not present in the dump file. Type ".hh dbgerr004" for details
Page cebb5 not present in the dump file. Type ".hh dbgerr004" for details
 
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
 
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
Probably caused by : ntkrnlmp.exe ( nt!ExAcquireResourceSharedLite+23 )
 
Followup: MachineOwner
---------
 
3: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
 
SYSTEM_SERVICE_EXCEPTION (3b)
An exception happened while executing a system service routine.
Arguments:
Arg1: 00000000c0000005, Exception code that caused the bugcheck
Arg2: fffff80001033153, Address of the exception record for the exception that caused the bugcheck
Arg3: fffffadf1c1580b0, Address of the context record for the exception that caused the bugcheck
Arg4: 0000000000000000, zero.
 
Debugging Details:
------------------
 
Page c4aa7 not present in the dump file. Type ".hh dbgerr004" for details
Page cebb5 not present in the dump file. Type ".hh dbgerr004" for details
 
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
 
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
 
OVERLAPPED_MODULE: Address regions for 'eeCtrl64' and 'imapi.sys' overlap
 
EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".
 
FAULTING_IP: 
nt!ExAcquireResourceSharedLite+23
fffff800`01033153 f0480fba696000  lock bts qword ptr [rcx+60h],0
 
CONTEXT:  fffffadf1c1580b0 -- (.cxr 0xfffffadf1c1580b0)
rax=fffffadf323afbf0 rbx=0000000000000000 rcx=0000000000000000
rdx=0000000000000001 rsi=0000000000000001 rdi=fffffadf323afbf0
rip=fffff80001033153 rsp=fffffadf1c1588c0 rbp=fffffa800d770280
 r8=0000000000000000  r9=0000000000000000 r10=000000000000000c
r11=fffffadf323a0000 r12=fffffa80101e0540 r13=fffffa8008122530
r14=fffff982fd76d024 r15=0000000000000000
iopl=0         nv up di ng nz na po nc
cs=0010  ss=0018  ds=002b  es=002b  fs=0053  gs=002b             efl=00010086
nt!ExAcquireResourceSharedLite+0x23:
fffff800`01033153 f0480fba696000  lock bts qword ptr [rcx+60h],0 ds:002b:00000000`00000060=????????????????
Resetting default scope
 
DEFAULT_BUCKET_ID:  DRIVER_FAULT
 
BUGCHECK_STR:  0x3B
 
PROCESS_NAME:  WINWORD.EXE
 
CURRENT_IRQL:  1
 
LAST_CONTROL_TRANSFER:  from fffff8000125a1b8 to fffff80001033153
 
STACK_TEXT:  
fffffadf`1c1588c0 fffff800`0125a1b8 : 00000000`00000000 fffffa80`08183290 fffffa80`101e0540 fffffa80`0bdcb780 : nt!ExAcquireResourceSharedLite+0x23
fffffadf`1c158910 fffff800`0125e1a8 : fffffa80`0d770280 00000000`0006c020 fffffa80`101e0540 fffffa80`000866e0 : nt!CmpCheckNotifyAccess+0x1d9
fffffadf`1c1589c0 fffff800`01278804 : 00000000`00000000 fffffadf`1c158cf0 00000000`ffffffff 00000000`00000000 : nt!CmpReportNotifyHelper+0x10a
fffffadf`1c158a50 fffff800`0127830e : fffffa80`090b22c8 fffffadf`1c158b40 fffffa80`00000003 00000000`002de128 : nt!CmSetValueKey+0x885
fffffadf`1c158b10 fffff800`0102e33d : fffffadf`3196cc20 00000000`0013e410 00000000`00000000 00000000`0013e5b0 : nt!NtSetValueKey+0x31a
fffffadf`1c158c00 00000000`77ef0fea : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x3
00000000`0013dfb8 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x77ef0fea
 
 
FOLLOWUP_IP: 
nt!ExAcquireResourceSharedLite+23
fffff800`01033153 f0480fba696000  lock bts qword ptr [rcx+60h],0
 
SYMBOL_STACK_INDEX:  0
 
SYMBOL_NAME:  nt!ExAcquireResourceSharedLite+23
 
FOLLOWUP_NAME:  MachineOwner
 
MODULE_NAME: nt
 
IMAGE_NAME:  ntkrnlmp.exe
 
DEBUG_FLR_IMAGE_TIMESTAMP:  460280f7
 
STACK_COMMAND:  .cxr 0xfffffadf1c1580b0 ; kb
 
FAILURE_BUCKET_ID:  X64_0x3B_nt!ExAcquireResourceSharedLite+23
 
BUCKET_ID:  X64_0x3B_nt!ExAcquireResourceSharedLite+23
 
Followup: MachineOwner
---------

Open in new window

Is this after making any changes?
Looks like it stayed up a bit longer, but still not looking good...
Avatar of djMundy

ASKER

I haven't changed anything yet. I'll be reducing the page file tonight, and BIOS update tomorrow morning.

It's crashing on average once or twice a day, except not every day..

- Today: once
- Yesterday: twice
- Mon: twice

Last week:
- Mon: once
- Tue: didn't crash
- Wed: twice
- Thu: twice
- Fri: didn't crash
Rather than looking at how often it crashes, try to analyze what's going on when it does...
I don't know how much logging you're doing, but I'd be turning on auditing (possibly), and perfmon (definitely) to see if that gives you some further indications as well...

I'll look at this latest log in a bit more detail tomorrow morning.
Avatar of djMundy

ASKER

Hi sirbounty,

I've been running perfmon for the last week and a half, but only just had a chance to look at them. What I find interesting is that it crashed on 15/12 at 11:36AM, a Saturday, when there were only 4 active TS sessions. CPU, memory, network and disk usage were very very low leading up to the crash.

How would you suggest I proceed? I've not explored auditing before and aren't quite sure where to start.

Thanks for all your help.

Daniel
Avatar of djMundy

ASKER

I didn't end up getting it fixed but the server is being rebuilt as a file / Exchange server. I'll close the question and give points anyway as the suggestions were good and it probably was an application issue, just hard to test.

Thanks for your time and ideas sirbounty.

Cheers,
Daniel