birdistheword
asked on
Server 2008 Random BSOD rdbss.sys
Hi,
I have an IBM X3500 7977 Server with a Dell MD1000 Disk array & Perc6/E Raid card and RSAII Slimline attached. This is running Windows Server 2008 with SP2 with all latest windows updates. Roles installed are file server, print server & terminal server. LAN is connected via Teamed broadcom onboard nics.
I'm finding randomly between a few days and a few weeks the server will BSOD with the error rdbss.sys - see the minidump analysis below.
Things I have tried:
1. Updating all drivers & firmware from IBM website where possible;
2. Updating the Broadcom NIC teaming software & drivers from broadcom website past the version that IBM supplies
3. Disabling all forms of offloading on the NIC's
4. Applying the file serving hotfixes listed here http://support.microsoft.com/kb/2473205 that includes rdbss.sys
None of the above attempts have made any difference to the random crashes that are always with the same rdbss.sys error.
3rd party software installed on the server includes: Sophos (latest version) - however I've tried disabling everythings sophos and the problem still occurs; Adobe Acrobat, Office 2007. The crash can occur regardless of if anyone is using the terminal server or not (via RDP).
This one has got me stumped - any suggestions would be appreciated
I have an IBM X3500 7977 Server with a Dell MD1000 Disk array & Perc6/E Raid card and RSAII Slimline attached. This is running Windows Server 2008 with SP2 with all latest windows updates. Roles installed are file server, print server & terminal server. LAN is connected via Teamed broadcom onboard nics.
I'm finding randomly between a few days and a few weeks the server will BSOD with the error rdbss.sys - see the minidump analysis below.
Things I have tried:
1. Updating all drivers & firmware from IBM website where possible;
2. Updating the Broadcom NIC teaming software & drivers from broadcom website past the version that IBM supplies
3. Disabling all forms of offloading on the NIC's
4. Applying the file serving hotfixes listed here http://support.microsoft.com/kb/2473205 that includes rdbss.sys
None of the above attempts have made any difference to the random crashes that are always with the same rdbss.sys error.
3rd party software installed on the server includes: Sophos (latest version) - however I've tried disabling everythings sophos and the problem still occurs; Adobe Acrobat, Office 2007. The crash can occur regardless of if anyone is using the terminal server or not (via RDP).
This one has got me stumped - any suggestions would be appreciated
Crash Dump Analysis provided by OSR Open Systems Resources, Inc. (http://www.osr.com)
Online Crash Dump Analysis Service
See http://www.osronline.com for more information
Windows Server 2008/Windows Vista Kernel Version 6002 (Service Pack 2) MP (8 procs) Free x64
Product: Server, suite: TerminalServer
Built by: 6002.22662.amd64fre.vistasp2_ldr.110617-0336
Machine Name:
Kernel base = 0xfffff800`02213000 PsLoadedModuleList = 0xfffff800`023d6dd0
Debug session time: Wed Oct 26 20:01:00.705 2011 (UTC - 4:00)
System Uptime: 2 days 0:26:38.824
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
RDR_FILE_SYSTEM (27)
If you see RxExceptionFilter on the stack then the 2nd and 3rd parameters are the
exception record and context record. Do a .cxr on the 3rd parameter and then kb to
obtain a more informative stack trace.
The high 16 bits of the first parameter is the RDBSS bugcheck code, which is defined
as follows:
RDBSS_BUG_CHECK_CACHESUP = 0xca550000,
RDBSS_BUG_CHECK_CLEANUP = 0xc1ee0000,
RDBSS_BUG_CHECK_CLOSE = 0xc10e0000,
RDBSS_BUG_CHECK_NTEXCEPT = 0xbaad0000,
Arguments:
Arg1: 00000000baad0075
Arg2: fffffa600a687848
Arg3: fffffa600a687220
Arg4: fffffa60063ab53a
Debugging Details:
------------------
OVERLAPPED_MODULE: Address regions for 'Dxapi' and 'crashdmp.sys' overlap
EXCEPTION_RECORD: fffffa600a687848 -- (.exr 0xfffffa600a687848)
ExceptionAddress: fffffa60063ab53a (mrxsmb10!MRxSmbDeferredCreate+0x000000000000018a)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: 0000000000000058
Attempt to read from address 0000000000000058
CONTEXT: fffffa600a687220 -- (.cxr 0xfffffa600a687220)
rax=fffffa8012055010 rbx=fffff88022fa9610 rcx=fffffa8018bc1ee8
rdx=0000000000000000 rsi=fffff8802b821850 rdi=0000000000000000
rip=fffffa60063ab53a rsp=fffffa600a687a80 rbp=fffffa80138227a0
r8=0000000000000058 r9=fffffa8019631060 r10=0000000000000024
r11=fffffa800d048ff8 r12=fffff88032123010 r13=fffff88022fa9698
r14=fffff8802b821958 r15=fffffa8018bc19a0
iopl=0 nv up ei ng nz na po nc
cs=0010 ss=0018 ds=002b es=002b fs=0053 gs=002b efl=00010286
mrxsmb10!MRxSmbDeferredCreate+0x18a:
fffffa60`063ab53a 8b4758 mov eax,dword ptr [rdi+58h] ds:002b:00000000`00000058=????????
Resetting default scope
CUSTOMER_CRASH_COUNT: 1
DEFAULT_BUCKET_ID: NULL_CLASS_PTR_DEREFERENCE
PROCESS_NAME: explorer.exe
CURRENT_IRQL: 0
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".
EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".
EXCEPTION_PARAMETER1: 0000000000000000
EXCEPTION_PARAMETER2: 0000000000000058
READ_ADDRESS: GetPointerFromAddress: unable to read from fffff80002439080
0000000000000058
FOLLOWUP_IP:
mrxsmb10!MRxSmbDeferredCreate+18a
fffffa60`063ab53a 8b4758 mov eax,dword ptr [rdi+58h]
FAULTING_IP:
mrxsmb10!MRxSmbDeferredCreate+18a
fffffa60`063ab53a 8b4758 mov eax,dword ptr [rdi+58h]
BUGCHECK_STR: 0x27
LAST_CONTROL_TRANSFER: from fffffa60063a5fb4 to fffffa60063ab53a
STACK_TEXT:
fffffa60`0a687a80 fffffa60`063a5fb4 : 00000000`00000000 fffff880`2b821958 00000000`00000000 fffff800`0225b45e : mrxsmb10!MRxSmbDeferredCreate+0x18a
fffffa60`0a687b10 fffffa60`063a4b65 : fffffa60`06398110 00000000`00000002 00000000`00000001 fffff880`32123010 : mrxsmb10!MRxSmbFsControl+0x14c
fffffa60`0a687c60 fffffa60`06359928 : fffffa80`138227a0 fffffa80`16f4a290 fffffa80`150099a0 fffffa80`150099a0 : mrxsmb10!MRxSmbFsCtl+0x275
fffffa60`0a687ca0 fffffa60`0638224a : fffffa80`150099a0 fffffa80`16f4a290 fffffa80`150099a0 fffffa80`139e1de0 : mrxsmb!SmbShellFsCtl+0x48
fffffa60`0a687cd0 fffffa60`06382553 : fffffa80`150099a0 fffff880`32123010 fffff880`2b821850 00000000`000007ff : mrxsmb10!MRxSmbQueryResumeKey+0xfa
fffffa60`0a687d10 fffffa60`063aceb0 : 00000000`00000000 fffff880`22fa9610 fffff880`2b821850 fffffa80`150099a0 : mrxsmb10!MRxSmbCreateShadowSrvOpen+0x9b
fffffa60`0a687d60 fffffa60`063b1789 : fffffa80`00100001 019db1de`d53e8000 fffff880`00000001 fffffa60`00000000 : mrxsmb10!MRxSmbCreateFileSuccessTail+0x4f0
fffffa60`0a687e40 fffffa60`063ac509 : 00000000`00000000 fffff880`32123010 fffffa80`1726c2d0 fffffa80`1726c2d0 : mrxsmb10!MRxSmbPseudoOpenTailFromGFAResponse+0x109
fffffa60`0a687ea0 fffffa60`063b575b : fffffa80`1726c200 fffffa80`697fbafb fffff880`0f30f110 fffff880`22fa9610 : mrxsmb10!SmbPseExchangeStart_Create+0x261
fffffa60`0a687fa0 fffffa60`063ab2bd : fffffa80`1205aba0 fffffa80`1726c2d0 fffffa80`150099a0 00000000`00000000 : mrxsmb10!SmbCeInitiateExchange+0x47f
fffffa60`0a688010 fffffa60`05135d03 : 00000000`00000000 00000000`00000000 fffff880`2b821850 fffff880`32123140 : mrxsmb10!MRxSmbCreate+0x789
fffffa60`0a6880e0 fffffa60`05134b8e : fffffa80`150099a0 00000000`00000000 fffffa80`150099a0 fffffa60`0a6881a8 : rdbss!RxCollapseOrCreateSrvOpen+0x437
fffffa60`0a688160 fffffa60`05132958 : fffffa80`150099a0 fffffa80`16f4a290 fffffa60`0a688290 fffffa80`15009901 : rdbss!RxCreateFromNetRoot+0x75e
fffffa60`0a688230 fffffa60`0510e29c : fffffa80`150099a0 fffffa80`16f4a290 fffffa80`0f7069b0 fffffa80`16f4a3a8 : rdbss!RxCommonCreate+0x374
fffffa60`0a6882f0 fffffa60`0512f78e : fffffa80`16f4a290 fffffa80`11326000 00000000`00000000 fffffa80`11326040 : rdbss!RxFsdCommonDispatch+0x7fc
fffffa60`0a6883f0 fffffa60`0636e2cf : fffffa80`16f4a290 00000000`6349754d fffffa80`16f4a3a8 fffffa80`11326040 : rdbss!RxFsdDispatch+0x21a
fffffa60`0a688460 fffffa60`013df5f9 : fffffa80`1583e400 fffffa80`16f4a290 fffffa80`0ecbad80 fffff880`008283b0 : mrxsmb!MRxSmbFsdDispatch+0xbf
fffffa60`0a6884a0 fffffa60`013df130 : fffff880`008283b0 fffffa80`1583e400 00000000`00000001 fffffa80`16f4a290 : mup!MupiCallUncProvider+0x159
fffffa60`0a688510 fffffa60`013e00ef : fffffa80`16f4a438 fffffa80`16f4a290 fffffa80`0ecbad80 00000000`00000000 : mup!MupStateMachine+0x120
fffffa60`0a688560 fffffa60`00d42e91 : fffffa80`0e071010 fffffa80`16f4a290 fffffa80`173e5d00 fffffa80`0ecbad80 : mup!MupCreate+0x2c3
fffffa60`0a6885e0 fffffa60`00d5f26c : fffffa80`0e070840 fffffa80`0e071010 fffffa80`16f4a200 fffffa60`0a6886a0 : fltmgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x211
fffffa60`0a688650 fffff800`024d6da9 : 00000000`00000005 fffffa80`16f4a290 00000000`00000040 00000000`00000000 : fltmgr!FltpCreate+0x25d
fffffa60`0a688700 fffff800`024cf562 : fffffa80`0e070e40 00000000`00000000 fffffa80`16a5b630 00000000`00000001 : nt!IopParseDevice+0x5f9
fffffa60`0a6888a0 fffff800`024d00f5 : 00000000`00000000 fffffa80`16a5b738 00000000`00000900 00000000`00000000 : nt!ObpLookupObjectName+0x593
fffffa60`0a6889b0 fffff800`024d5417 : 00000000`00100001 00000000`00100001 00000000`00000001 fffffa60`0a688ca0 : nt!ObOpenObjectByName+0x2f5
fffffa60`0a688a80 fffff800`024c48e0 : 00000000`02bdeff8 fffffa60`00100001 00000000`0000003d 00000000`02bdef70 : nt!IopCreateFile+0x287
fffffa60`0a688b20 fffff800`02264933 : fffffa80`19631060 00000000`02bdf518 fffffa60`0a688bc8 fffff900`c1006e20 : nt!NtOpenFile+0x58
fffffa60`0a688bb0 00000000`76ea6e4a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
00000000`02bdef08 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x76ea6e4a
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: mrxsmb10!MRxSmbDeferredCreate+18a
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: mrxsmb10
IMAGE_NAME: mrxsmb10.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 4e147dee
STACK_COMMAND: .cxr 0xfffffa600a687220 ; kb
FAILURE_BUCKET_ID: X64_0x27_mrxsmb10!MRxSmbDeferredCreate+18a
BUCKET_ID: X64_0x27_mrxsmb10!MRxSmbDeferredCreate+18a
Followup: MachineOwner
---------
noxcho
Is there NVidia chipset on your mainboard?
ASKER
@noxcho - No this is an IBM x3500 7977 mainboard... no Nvidia in sight.
Do you have another network card to install it through PCI port and disable Broadcom to see if this fixes it?
ASKER
@noxcho - yes this is what i'm trying next since rdbss.sys seemed releated to the network stack so I thought might as well try changing the NIC
It could be even driver problem. I mean network card driver. So if you put there another NIC with new driver then it should shed some light on this issue.
ASKER
Well after a few days of having the network card changed it finally crashed today with the same rdbss.sys error.. so its definitely not network related ...
This time it crashed exactly when i tried to preview a large jpeg through terminal server using windows built in photo viewer ... however after reboot previewing the same jpeg worked fine...
I'm thinking I might have to rebuild the server with Server 2008 R2 instead... not fun
This time it crashed exactly when i tried to preview a large jpeg through terminal server using windows built in photo viewer ... however after reboot previewing the same jpeg worked fine...
I'm thinking I might have to rebuild the server with Server 2008 R2 instead... not fun
When viewing picture. Could it be video adapter driver? Which video card is used by you?
ASKER
When viewing picture. Could it be video adapter driver? Which video card is used by you?
There is no video card - I was viewing the picture via remote desktop/terminal services/RDP .... this crash has occured even when the machine in question (and terminal server user) wasn't in use.
There is no video card - I was viewing the picture via remote desktop/terminal services/RDP .... this crash has occured even when the machine in question (and terminal server user) wasn't in use.
Ops, looks like I misunderstood it. Ok. Then rebuild it. That Vista crap is always instable.
ASKER
SUCCESS!! I have a completely reproducable situation that is causing this BSOD - While in a Terminal Server /RDP session.. Right click on ANY Image file, be it jpeg, png etc... and press Preview which theortically opens it with the windows photo viewer... and instantly connection is lost / BSOD with the same rdbss.sys error! ...
Now how to fix this? What is the preview function in Server 2008 for images normally linked to? Windows Photo Viewer?
Now how to fix this? What is the preview function in Server 2008 for images normally linked to? Windows Photo Viewer?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
SOLUTION: Remove "Desktop Experience" from Features.... it's easier to not use this preview function then to figure out why its causing BSOD. Cheers.
ASKER
I hope this helps someone as it certainly stumped me - I highly recommend NOT using the Desktop Experience feature on Server 2008 in terminal services mode.... If you need the useful features like Disk Cleanup Tool (cleanmgr.exe) and Charmap - just google for how to install those WITHOUT the Desktop Experience feature and you'll avoid a world of possible pain :)
glad you found the root cause of this problem.
ASKER
Further update to this issue: The problem occured again today - this time while pressing 'extract' with the program 7-Zip to extract a file on the local network drive (referencing the disks on the same server) while on the terminal server.... So it's not limited to windows photo previewer but is actually and underlying program and looks like both 7-Zip and windows photo viewer call files in the same way.... Since its completely reproducible I'm going to try several fixes (changing permissions etc) to see if I can stop the server BSODing.
I think it is time to rebuild machine with Windows 2008 R2.
ASKER
Update: I found the underlying issue! ... It's caused by having SMB2 disabled
Someone else actually has this exact same issue ... as documented here http://social.technet.microsoft.com/Forums/en-US/itprovistasecurity/thread/97657c43-de53-43b2-9cc0-3d9f6fe7dd6e
The reason I had to disable SMB2 is because of files not showing up in mapped network drives... as documented here http://social.technet.microsoft.com/Forums/en/w7itprogeneral/thread/947489ae-dc86-45f0-ad5e-463a62e1d59f so I can't exactly re-enable SMB2
So I don't think a rebuild is going to help here as the problem occurs on Server 2008R2 as well... Just have to live with the occasional program not working in terminal services ... e.g 7-Zip... photo previewer.... I wonder how many more...
So basically I don't think its possible to fix this problem.... its either Default/SMB2 and have random issues with files not showing up in network shares... or SMB1 and crash when files opened with specific programs from network shares referencing the local computer...
Someone else actually has this exact same issue ... as documented here http://social.technet.microsoft.com/Forums/en-US/itprovistasecurity/thread/97657c43-de53-43b2-9cc0-3d9f6fe7dd6e
The reason I had to disable SMB2 is because of files not showing up in mapped network drives... as documented here http://social.technet.microsoft.com/Forums/en/w7itprogeneral/thread/947489ae-dc86-45f0-ad5e-463a62e1d59f so I can't exactly re-enable SMB2
So I don't think a rebuild is going to help here as the problem occurs on Server 2008R2 as well... Just have to live with the occasional program not working in terminal services ... e.g 7-Zip... photo previewer.... I wonder how many more...
So basically I don't think its possible to fix this problem.... its either Default/SMB2 and have random issues with files not showing up in network shares... or SMB1 and crash when files opened with specific programs from network shares referencing the local computer...