Solved

Failure in controller of HP StorageWorks MSA2012fc

Posted on 2014-10-15
8
914 Views
Last Modified: 2014-10-17
Hi experts, I have the next critical error on MSA logs (attached):
TUE OCT 07 11:40:55 2014 [314] #A16253: MSA2012fc Array SN#00C0FFD839E1 Controller A CRITICAL: FRU type: RAID IOM B, problem: encl 0. Product ID: AJ744A, S/N: 3CL922S426 rev: H.  Related event ID: 10016252, type: 313

Open in new window

The array connects to an Oracle RAC server with two instances
The AlertLog of Oracle Instance #2 says:
Errors in file /cots/oracle/app/oracle/admin/xa21/bdump/xa212_j001_30215.trc:
ORA-27091: unable to queue I/O
ORA-27072: File I/O error
Linux-x86_64 Error: 5: Input/output error
Additional information: 4
Additional information: 1529712
Additional information: -1

Open in new window

The Oracle instance #1 does not report errors.
The version of S.O is:
uname -a
Linux 2.6.18-53.1.21.el5 #1 SMP Wed May 7 08:42:34 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux

Open in new window

and the versions in MSA:
# versions
Controller A Versions
---------------------
Storage Controller CPU Type   : Celeron 566MHz
Storage Controller Firmware   : J200P30
Storage Controller Memory     : F300R22
Storage Controller Loader     : 15.010
Management Controller Firmware: W420R52
Management Controller Loader  : 12.013
Expander Controller Firmware  : 3022
CPLD Revision                 : 27
Hardware Revision             : LCA 56
Host Interface Module         : 26
Host Interface Module Model   : 1

Controller B Versions
---------------------
Storage Controller CPU Type   : Celeron 566MHz
Storage Controller Firmware   : J200P30
Storage Controller Memory     : F300R22
Storage Controller Loader     : 15.010
Management Controller Firmware: W420R52
Management Controller Loader  : 12.013
Expander Controller Firmware  : 3022
CPLD Revision                 : 27
Hardware Revision             : LCA 56
Host Interface Module         : 26
Host Interface Module Model   : 1

Open in new window

Could you please suggest me the steps to find the problem and its resolution?
Thankyou
Regards
logs-from-20140804.log
0
Comment
Question by:carlino70
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
8 Comments
 
LVL 42

Assisted Solution

by:paulsolov
paulsolov earned 250 total points
ID: 40383604
I would give HP a call and send over the logs.  Take a look at your fibre channel paths on the servers to see if all paths are showing, most likely the controller has failed over but you should look at getting a controller replaced if it's a hardware issue
0
 
LVL 7

Expert Comment

by:Stampel
ID: 40383856
When you look at your MSA, do you have one or two controllers ?
Don't you see any orange light / LED message on it ?
0
 

Author Comment

by:carlino70
ID: 40384207
paulsolov, the warranty expired. I searched for similar cases in the HP forums with various attempted solutions, with varying results.
0
Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

 

Author Comment

by:carlino70
ID: 40384224
Stampel, the MSA has 2 controllers.
No colors orange light / display in the Led's
Currently the leds 'LINK' of each input FO are off, on controller #1. But I see information from A y B, with up/down link, permanently:
TUE OCT 14 14:28:09 2014 [111] #A16361: MSA2012fc Array SN#00C0FFD839E1 Controller A INFORMATIONAL: Host link up Chan1: 4 Loop IDs, External Device(s)
TUE OCT 14 14:28:08 2014 [112] #A16360: MSA2012fc Array SN#00C0FFD839E1 Controller A WARNING: Host link down Chan1
TUE OCT 14 14:28:08 2014 [111] #A16359: MSA2012fc Array SN#00C0FFD839E1 Controller A INFORMATIONAL: Host link up Chan0: 4 Loop IDs, External Device(s)
TUE OCT 14 14:28:08 2014 [112] #A16358: MSA2012fc Array SN#00C0FFD839E1 Controller A WARNING: Host link down Chan0
TUE OCT 14 14:28:08 2014 [111] #B10072: MSA2012fc Array SN#00C0FFD839E1 Controller B INFORMATIONAL: Host link up Chan0: 4 Loop IDs, External Device(s)
TUE OCT 14 14:28:08 2014 [112] #B10071: MSA2012fc Array SN#00C0FFD839E1 Controller B WARNING: Host link down Chan0
TUE OCT 14 14:28:08 2014 [111] #B10070: MSA2012fc Array SN#00C0FFD839E1 Controller B INFORMATIONAL: Host link up Chan1: 4 Loop IDs, External Device(s)
TUE OCT 14 14:28:08 2014 [112] #B10069: MSA2012fc Array SN#00C0FFD839E1 Controller B WARNING: Host link down Chan1

Open in new window

0
 
LVL 7

Accepted Solution

by:
Stampel earned 250 total points
ID: 40384464
Did/could you upgrade your controllers to latest firmware first ?
0
 

Author Comment

by:carlino70
ID: 40384495
not yet. I wanted to get an accurate diagnosis before
0
 
LVL 7

Expert Comment

by:Stampel
ID: 40386924
The real important message i can see is from your logs is :
"CRITICAL: RAID controller B failed, reason PCIE link recovery failed" + Failover ...
MSA2012fc and MSA2000 have had many problems, i would upgrade firmware first just in case.

Do you still have File I/O error on Oracle RAC ?
0
 

Author Comment

by:carlino70
ID: 40387239
Stampel, I/O Errors no longer appear.
But the controller is working like simple configuration.
I will be doing a firmware upgrade, and evenual replacement controller.
Thanks for your comments.
Regards.
0

Featured Post

Edgartown IT Case Study

Learn about Edgartown's quest to ensure the safety and security of the entire town's employee and citizen data. Read the case study!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Lets start to have a small explanation what is VAAI(vStorage API for Array Integration ) and what are the benefits using it. VAAI is an API framework in VMware that enable some Storage tasks. It first presented in ESXi 4.1, but only after 5.x sup…
The business world is becoming increasingly integrated with tech. It’s not just for a select few anymore — but what about if you have a small business? It may be easier than you think to integrate technology into your small business, and it’s likely…
This video Micro Tutorial explains how to clone a hard drive using a commercial software product for Windows systems called Casper from Future Systems Solutions (FSS). Cloning makes an exact, complete copy of one hard disk drive (HDD) onto another d…
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question