Solved

BSOD -- analylzing dmp file

Posted on 2013-12-23
34
398 Views
Last Modified: 2013-12-31
Running Windows 7 Professional SP-1, I've had 4-5 instances of BSOD over the past month. Some of them are PFN_LIST_CORRUPT and some are IRQL_NOT_LESS_OR_EQUAL.  I have WER-sysdata and minidump files associated with the crashes.  

I've tested my hard disks, and one of them needed to be replaced, but the BSOD have continued.  I ran memtest86, which did not report any problems.  Driver verifier found a driver issue, but I've used that driver for years without incident.  Updating to the most recent version of the driver didn't prevent BSOD from happening.

I'm trying to use Windows Debugging Tools to analyze the .dmp files associated with these events, but I have no experience with this.  The console window says "Loading User Symbols", and nothing further appears there.  The "Symbols" folder has some contents, but I don't know its significance.  Guidance would be appreciated.
0
Comment
Question by:ddantes
  • 16
  • 11
  • 4
  • +2
34 Comments
 
LVL 1

Assisted Solution

by:Dragon911
Dragon911 earned 250 total points
ID: 39737062
Try this one from Nirsoft: http://www.nirsoft.net/utils/blue_screen_view.html

Here's a good link for the PFN Corrupt_list: http://www.osronline.com/article.cfm?article=334 .

Original content of that site removed, as their ToU explicitly prohibit republishing. And besides that it is copyright infriction.

Qlemo, Topic Advisor
0
 

Author Comment

by:ddantes
ID: 39737134
Thank you for your helpful input.  Using BlueScreenViewer, the files identified with respect to these incidents are:  hal.dll, ntoskrnl.exe, and tcpip.sys in one instance.
0
 
LVL 1

Expert Comment

by:Dragon911
ID: 39737144
Any major changes to the system lately?

Any changes/updates to the drivers? In particular- the NIC?

Have you run the system file checker? SFC /Scannow. Save your data and try running that.
0
 

Author Comment

by:ddantes
ID: 39737160
There are frequent changes to the system.  The most major change was replacement of the system hard disc, after the first BSOD, when the previous disc was found to fail some tests.  I use RamDisk by SuperSpeed, and I updated the driver, because it caused BSOD during driver verifier.  The updated driver causes a BSOD with driver verifier as well.   I ran SFC with no apparent errors.  There haven't been recent changes to NIC drivers or network hardware.
0
 
LVL 18

Expert Comment

by:awawada
ID: 39737387
Can you upload us the Minidumps from %SystemRoot%\windows\minidump\ ?
0
 

Author Comment

by:ddantes
ID: 39737407
112313-25272-01.dmp
120713-30154-01.dmp
121113-28267-01.dmp

Here are three .dmp files associated with PFN_LIST_CORRUPT BSOD.  The BSOD which was IRQ_NOT_LESS_OR_EQUAL didn't produce a .dmp file because, somehow, the configuration setting to generate a report was changed to "no report."  I've set it back to generate a minidump.
0
 
LVL 91

Expert Comment

by:nobus
ID: 39737470
121113 and 120713 says :  IMAGE_NAME:  memory_corruption
try replacing the ram
0
 

Author Comment

by:ddantes
ID: 39737579
Thank you for your recommendation.  I will try replacing one module at a time, and I'll post back afterwards.  May I ask, wouldn't memtest86 have discovered a memory issue?   One other question:  I'm looking at those dmp files with BlueScreenView, and I don't see any mention of "IMAGE_NAME:  memory_corruption".  Where do you find that reference?
0
 
LVL 91

Expert Comment

by:nobus
ID: 39737795
>>  May I ask, wouldn't memtest86 have discovered a memory issue?   <<  Normally YES
but there is NO diagnostic or software that covers all possible failures, so i always tell the people that a diag is ok for 99%.
But it keeps being possible it is something else; eg : ram controller, internal cache in the cpu, or even motherboard
0
 

Author Comment

by:ddantes
ID: 39738345
Thank you.  I would still like to know where in the .dmp file report you were able to see "IMAGE_NAME:  memory_corruption"?
0
 
LVL 91

Accepted Solution

by:
nobus earned 250 total points
ID: 39738448
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

PFN_LIST_CORRUPT (4e)
Typically caused by drivers passing bad memory descriptor lists (ie: calling
MmUnlockPages twice with the same list, etc).  If a kernel debugger is
available get the stack trace.
Arguments:
Arg1: 0000009a,
Arg2: 0005ae7e
Arg3: 00000006
Arg4: 00000005

Debugging Details:
------------------


BUGCHECK_STR:  0x4E_9a

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  WIN7_DRIVER_FAULT

PROCESS_NAME:  System

CURRENT_IRQL:  0

LAST_CONTROL_TRANSFER:  from 83305408 to 832f9bfc

STACK_TEXT:  
c7d489f0 83305408 0000004e 0000009a 0005ae7e nt!KeBugCheckEx+0x1e
c7d48a08 8333a12a 00000000 00001000 00000000 nt!MiBadRefCount+0x26
c7d48ae8 8333bef1 a1af6000 00001000 17c6537b nt!MiFreePoolPages+0x124
c7d48b50 832b7e75 a1af6000 00000000 00000002 nt!ExFreePoolWithTag+0x436
c7d48b64 834486ce 00000000 c000009a 83350580 nt!MmFreeAccessPfnBuffer+0x2f
c7d48bc0 8345ca87 00000000 8a905030 00000000 nt!PfpFlushBuffers+0x2ba
c7d48c50 8342413d 83350580 ed376c7a 00000000 nt!PfTLoggingWorker+0xaa
c7d48c90 832cb559 8345c9d7 83350580 00000000 nt!PspSystemThreadStartup+0x9e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x19


STACK_COMMAND:  kb

FOLLOWUP_IP:
nt!MiBadRefCount+26
83305408 cc              int     3

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  nt!MiBadRefCount+26

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

DEBUG_FLR_IMAGE_TIMESTAMP:  521e9cb6

IMAGE_NAME:  memory_corruption                 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

FAILURE_BUCKET_ID:  0x4E_9a_nt!MiBadRefCount+26

BUCKET_ID:  0x4E_9a_nt!MiBadRefCount+26

Followup: MachineOwner
---------
0
 

Author Comment

by:ddantes
ID: 39738453
What application are you using to examine the .dmp file?
0
 
LVL 91

Expert Comment

by:nobus
ID: 39738895
0
 
LVL 68

Expert Comment

by:Qlemo
ID: 39739874
Windbg is "Debugging Tools for Windows", or what you called "Windows Debugging Tools". In the File menu you will find "Open Crash Dump", which is exactly what you'll need.
Setting up the correct symbol path requires a symbol server for downloading the corresponding PDB files (symbol files). Usually you set that up as
 srv*c:\Symbols*http://msdl.microsoft.com/download/symbols
where C:\Symbols is where the files should be stored. You should always use the same path, as the Windows PDB files are large and many, and downloading them requires a lot of time. Hence it is a good idea to set the environment variable _NT_SYMBOL_PATH to that value shown above (unless you have another debugger/IDE like Visual Studio in use on that machine).
0
 

Author Comment

by:ddantes
ID: 39740281
I'm trying to reproduce Nobus' findings.  Using  Windbg 6.5.000.3.7.  Here is a screen shot of the console.  I don't know how to find the report which describes the memory issue. debug console
0
 
LVL 91

Expert Comment

by:nobus
ID: 39740340
there's alot after the user symbols, see here the first part :
Microsoft (R) Windows Debugger Version 6.2.9200.20512 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\Users\Nobus64\Downloads\121113-28267-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: .sympath SRV*c:\localsymbols*http://msdl.microsoft.com/download/symbols;symsrv*symsrv.dll*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows 7 Kernel Version 7601 (Service Pack 1) MP (4 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 7601.18247.x86fre.win7sp1_gdr.130828-1532
Machine Name:
Kernel base = 0x8321b000 PsLoadedModuleList = 0x833644d0
Debug session time: Wed Dec 11 10:20:30.262 2013 (UTC + 1:00)
System Uptime: 2 days 9:32:12.089
Loading Kernel Symbols
...............................................................
................................................................
................................................................
..
Loading User Symbols
Loading unloaded module list
..........................
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

UseMicrosoft (R) Windows Debugger Version 6.2.9200.20512 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\Users\Nobus64\Downloads\121113-28267-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: .sympath SRV*c:\localsymbols*http://msdl.microsoft.com/download/symbols;symsrv*symsrv.dll*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows 7 Kernel Version 7601 (Service Pack 1) MP (4 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 7601.18247.x86fre.win7sp1_gdr.130828-1532
Machine Name:
Kernel base = 0x8321b000 PsLoadedModuleList = 0x833644d0
Debug session time: Wed Dec 11 10:20:30.262 2013 (UTC + 1:00)
System Uptime: 2 days 9:32:12.089
Loading Kernel Symbols
...............................................................
................................................................
................................................................
..
Loading User Symbols
Loading unloaded module list
..........................
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 4E, {9a, 5ae7e, 6, 5}

Probably caused by : memory_corruption ( nt!MiBadRefCount+26 )

Followup: MachineOwner
---------
-v to get detailed debugging information.

BugCheck 4E, {9a, 5ae7e, 6, 5}

Probably caused by : memory_corruption ( nt!MiBadRefCount+26 )

Followup: MachineOwner
---------


Microsoft (R) Windows Debugger Version 6.2.9200.20512 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\Users\Nobus64\Downloads\121113-28267-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: .sympath SRV*c:\localsymbols*http://msdl.microsoft.com/download/symbols;symsrv*symsrv.dll*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows 7 Kernel Version 7601 (Service Pack 1) MP (4 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 7601.18247.x86fre.win7sp1_gdr.130828-1532
Machine Name:
Kernel base = 0x8321b000 PsLoadedModuleList = 0x833644d0
Debug session time: Wed Dec 11 10:20:30.262 2013 (UTC + 1:00)
System Uptime: 2 days 9:32:12.089
Loading Kernel Symbols
...............................................................
................................................................
................................................................
..
Loading User Symbols
Loading unloaded module list
..........................
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 4E, {9a, 5ae7e, 6, 5}

Probably caused by : memory_corruption ( nt!MiBadRefCount+26 )

Followup: MachineOwner

then i used the  !analyze -v command

---------
0
 

Author Comment

by:ddantes
ID: 39741241
Thank you for supplying that output.  Maybe I didn't let the bugcheck run long enough.  I left it for around a half hour, and nothing appeared after what shows in the screen shot.
0
Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 
LVL 91

Expert Comment

by:nobus
ID: 39741317
it does not take 2 minutes here, so probably you have aother problems :
a corrupt OS, or bad hardware
you can test by running the debug on a working pc
0
 
LVL 68

Expert Comment

by:Qlemo
ID: 39741380
It might last the first time you do that, because the necessary PDB files need to be downloaded. Check your symbol folder for changes. But is should not last 30 minutes or more...
0
 

Author Comment

by:ddantes
ID: 39742321
To test debug on a working pc, would I use the crash dump from the problem pc, but run the utility on the working one?
0
 
LVL 68

Expert Comment

by:Qlemo
ID: 39742445
Yes, you can use any PC for crash dump analysis. It is even a good idea to use always the same one if you have to do that often.
0
 

Author Comment

by:ddantes
ID: 39743026
I got the same result using a computer which isn't having BSOD issues.  "Loading unloaded module list" is the last entry.  No bugcheck analysis after that.
0
 
LVL 91

Expert Comment

by:nobus
ID: 39743206
and which dmp file are you using?
0
 

Author Comment

by:ddantes
ID: 39743762
I've tried with 120713-30154-01.dmp  and  121113-28267-01.dmp
0
 
LVL 68

Expert Comment

by:Qlemo
ID: 39744099
If set the symbol path, and then opened 120713-30154-01.dmp as crash dump in WinDbg, and that is the result:
Loading Dump File [C:\temp\ee\120713-30154-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: srv*c:\WebSymbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 7 Kernel Version 7601 (Service Pack 1) MP (4 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 7601.18247.x86fre.win7sp1_gdr.130828-1532
Machine Name:
Kernel base = 0x83201000 PsLoadedModuleList = 0x8334a4d0
Debug session time: Sat Dec  7 18:01:53.724 2013 (UTC + 1:00)
System Uptime: 2 days 14:09:43.926
Loading Kernel Symbols
...............................................................
................................................................
................................................................
..
Loading User Symbols
Loading unloaded module list
.........................
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 4E, {9a, cbdbb, 6, b}

Probably caused by : memory_corruption ( nt!MiBadRefCount+26 )

Followup: MachineOwner
---------

Open in new window

Clicking on the !analyze -v link reveals little more info in that case - probably too much memory corrupted already. But what we can see is ExFreePoolWithTag, which is associated with Paged Memory Pool (data which can be put into virtual memory, swapped out of physical memory). That kind of memory is used by drivers to store data not necessary to be available all the time, like string buffers for logging. That might be a hint or not. Sadly, there is no trace of the culprit driver ...
0
 

Author Comment

by:ddantes
ID: 39744153
Thank you.   For some reason, I can't get the debugging tool to display a bugcheck report.

I am using Superspeed's RamDisk Plus, so my paging file is on a RamDisk.  But that has worked fine for a couple of years without any BSOD, so I didn't suspect it was causing this.
0
 
LVL 91

Expert Comment

by:nobus
ID: 39744404
test it on another PC, to get familiar with the setup
0
 

Author Comment

by:ddantes
ID: 39744879
Nobus, are you suggesting testing the bug check tool or RamDisk on another computer?  I have had RamDisk on another computer, as well as this one, for a couple of years, without difficulty.  I tested the bug check tool on the other computer, and it didn't produce the report which both experts posted here.
0
 
LVL 91

Expert Comment

by:nobus
ID: 39745632
i never used the word "ramdisk"
i suggest only to test windbg on another PC
0
 

Author Comment

by:ddantes
ID: 39746536
Thank you.  I tested windbg on another PC, with the same result as on this PC.  No bugcheck, only the preliminary lines of information, appears in the console.
0
 
LVL 91

Expert Comment

by:nobus
ID: 39747660
then you did not install it properly imo.
did you install the standalone version, or as part of WDK?
0
 

Author Comment

by:ddantes
ID: 39747699
The version which I installed gave a choice of which components to install.  It was called SDK for Windows 7 and .NET Framework 4.  I selected "Common Utilities>Debugging Tools for Windows."   That produced an installer, dbg x86.msi, which I executed.
0
 
LVL 91

Expert Comment

by:nobus
ID: 39748050
strange. i never had such problem; unless you have other provblems with the hard- or software?
0
 

Author Closing Comment

by:ddantes
ID: 39748726
Input from all participating Experts is appreciated.  If the issue persists, I'll try replacing the RAM.  I've held off on that because a second Mem86 test was, again, negative for errors, and at the moment I am not seeing BSOD.
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

After playing around with my ASUS 1215n (http://www.asus.de/product.aspx?P_ID=HrglRhH8D60Rmlv3) Netbook, I finally managed to get smooth HD 1080p (http://en.wikipedia.org/wiki/1080p) playback of videos on it. Second Generation Intel Atom (http://en.…
I found out last night windows update has a problem regarding 4 latest updates that fail.  The way to get all 4 them installed is install sp1 first and restart then one by one with a resart in between as they fail every time if all the four updates …
This Micro Tutorial will teach you how to the overview of Microsoft Security Essentials. This is a free anti-virus software that guards your PC against viruses, spyware, worms, and other malicious software. This will be demonstrated using Windows…
The viewer will learn how to successfully create a multiboot device using the SARDU utility on Windows 7. Start the SARDU utility: Change the image directory to wherever you store your ISOs, this will prevent you from having 2 copies of an ISO wit…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now