MD3000i RAID controller module Memory Parity Error!

We have a DELL MD3000i array with dual-controllers (A/A). Last week, one of the controllers went bad due to memory parity error (or aka. Memory Consistency Error) and we took a huge performance hit. We ordered a replacement and swapped it out, the error was still there.  We thought we may have received a defective part, so we got another controller, but the error doesn't go away. We took out the battery and memory from the controller, waited 2 mins, then popped it back in, but still no luck. Does any one know how to correct this issue? Does something else needs to be done beside putting the controller in offline mode before swapping?

Summary : RAID Controller Module Memory Consistency Error
Storage array:  MDPLSHELP01
Component          reporting problem:  RAID Controller Module in slot 0
 Status:           Online
 Location:  RAID Controller          Module/Expansion enclosure, RAID Controller Module in slot 0
Component          requiring service:  RAID Controller Module in slot 0
 Service          action (removal) allowed:  No 
 Service action LED on component:  No
 Replacement          part number:  
 Board ID:  1532
 Submodel          ID:  63
 Serial          number:  63V10XX
Michael McGovernVP of Information TechnologyAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Coolie SheppardSystems EngineerCommented:
May have to come down to flashing the firmware for the error to go away:

https://www.dell.com/community/PowerVault/MD3000-Raid-Controller-Module-Memory-Parity-Error/td-p/4549647

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
andyalderCommented:
AFAIK the cache modules have ECC memory so can;t have parity errors. Memory consistency us more likely to be the cache on one controller not matching that on the other after mirroring one to the other. I would shutdown and power off, then clear the cache on the other controller. i.e. do the same battery removal on both.

You can set mirrorcacheenabled=false with SMCLI to get rid of the performance hit when only using one controller but obviously that is a risk as write cache is only stored on one controller in that mode.
Michael McGovernVP of Information TechnologyAuthor Commented:
We're going to try the method suggested by andyalder during our maintenance window tonight.
Michael McGovernVP of Information TechnologyAuthor Commented:
Powering down the array and pulling out both controller with battery reset didn't work. Updating the firmware cleared the error! Thanks everyone.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Dell

From novice to tech pro — start learning today.