Solved

How do i fix cissesrv errors incorrectly reporting disk drive removal

Posted on 2010-09-09
10
9,625 Views
Last Modified: 2012-05-10
We are using an HP ProLiant ML350G6 server with an embedded P410i smart array controller with 6x146GB disks. Every so often it reports that a disk has been removed and subsequently reinserted, and that it has to rebuild the physical and logical volumes, however the disk has never been removed. Has anyone come across this before and found a resolution?

The full sequence of events is:

Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:22:47
Event ID:      24582
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
A SAS physical drive located in bay 5 was removed. The drive can be found in box 1 which is attached  to port 2I of array controller P410i [Embedded].
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24582</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:22:47.000Z" />
    <EventRecordID>15408</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data> SAS </Data>
    <Data>5</Data>
    <Data>removed</Data>
    <Data>1</Data>
    <Data>2I</Data>
  </EventData>
</Event>

Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:22:47
Event ID:      24595
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
A drive failure notification has been received for the SAS physical drive located in bay 5.  This drive can be found in box 1 which is connected to port 2I of the array controller P410i [Embedded].  The failure reason received from the HP Smart Array firmware is: REMOVED_IN_HOT_PLUG.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="50229">24595</EventID>
    <Level>2</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:22:47.000Z" />
    <EventRecordID>15409</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data> SAS </Data>
    <Data>5</Data>
    <Data>1</Data>
    <Data>2I</Data>
    <Data>REMOVED_IN_HOT_PLUG</Data>
  </EventData>
</Event>

Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:22:47
Event ID:      24598
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
Logical drive 1 of array controller P410i [Embedded] has encountered a status change from:  

Status: OK  
to  
Status: INTERIM RECOVERY MODE
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24598</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:22:47.000Z" />
    <EventRecordID>15410</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>1</Data>
    <Data>OK</Data>
    <Data>INTERIM RECOVERY MODE</Data>
  </EventData>
</Event>

Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:22:47
Event ID:      24598
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
Logical drive 2 of array controller P410i [Embedded] has encountered a status change from:  

Status: OK  
to  
Status: INTERIM RECOVERY MODE
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24598</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:22:47.000Z" />
    <EventRecordID>15411</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>2</Data>
    <Data>OK</Data>
    <Data>INTERIM RECOVERY MODE</Data>
  </EventData>
</Event>

Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:22:47
Event ID:      24582
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
A SAS physical drive located in bay 5 was inserted. The drive can be found in box 1 which is attached  to port 2I of array controller P410i [Embedded].
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24582</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:22:47.000Z" />
    <EventRecordID>15412</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data> SAS </Data>
    <Data>5</Data>
    <Data>inserted</Data>
    <Data>1</Data>
    <Data>2I</Data>
  </EventData>
</Event>

Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:22:47
Event ID:      24598
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
Logical drive 1 of array controller P410i [Embedded] has encountered a status change from:  

Status: INTERIM RECOVERY MODE  
to  
Status: READY FOR RECOVERY
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24598</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:22:47.000Z" />
    <EventRecordID>15413</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>1</Data>
    <Data>INTERIM RECOVERY MODE</Data>
    <Data>READY FOR RECOVERY</Data>
  </EventData>
</Event>

Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:22:47
Event ID:      24598
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
Logical drive 2 of array controller P410i [Embedded] has encountered a status change from:  

Status: INTERIM RECOVERY MODE  
to  
Status: READY FOR RECOVERY
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24598</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:22:47.000Z" />
    <EventRecordID>15414</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>2</Data>
    <Data>INTERIM RECOVERY MODE</Data>
    <Data>READY FOR RECOVERY</Data>
  </EventData>
</Event>

Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:22:47
Event ID:      24598
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
Logical drive 1 of array controller P410i [Embedded] has encountered a status change from:  

Status: READY FOR RECOVERY  
to  
Status: RECOVERING
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24598</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:22:47.000Z" />
    <EventRecordID>15415</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>1</Data>
    <Data>READY FOR RECOVERY</Data>
    <Data>RECOVERING</Data>
  </EventData>
</Event>




Log Name:      System
Source:        HP Smart Array
Date:          08/09/2010 18:23:46
Event ID:      203
Task Category: Primary Storage Device
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
A physical drive status has changed to Rebuilding. Data is now being rebuilt onto this drive from the RAID set.
 
User Action
No action is recommended.
 
WBEM Indication Properties
AlertingElementFormat: 2 0x2 (CIMObjectPath)
AlertingManagedElement: "\\HWEDHPV\ROOT\HPQ:HPSA_DiskDrive.CreationClassName="HPSA_DiskDrive",DeviceID="5001438007CBDEF0:4",SystemCreationClassName="HPSA_ArraySystem",SystemName="5001438007CBDEF0""
AlertType: 5 0x5 (Device Alert)
Description: "A physical drive status has changed to Rebuilding. Data is now being rebuilt onto this drive from the RAID set."
DeviceControllerLogicalLocation[0]: "Smart Array P410i in Slot 0"
DeviceModel: "EG0146FAWHU"
EventCategory: 6 0x6 (Primary Storage Device)
EventID: "203"
EventTime: "20100908182338.565000-000"
HWFirmwareVersion[0]: "HPDD"
HWLogicalLocation[0]: "Port:2I Box:1 Bay:5"
HWManufacturer: "HP"
HWPhysicalLocation: "Box 1 Bay 5"
HWSerialNumber[0]: "3SD35GK300009037KLQ5"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{ADC690B6-052B-45FE-A73F-C2A352242973}"
IndicationTime: "20100908182343.573000+060"
NetworkAddresses[0]: "fe80::c17c:eb2d:89dc:a1bd%1034"
NetworkAddresses[1]: "192.168.5.16"
NetworkAddresses[2]: "fe80::44ef:3fa7:5928:daed%1033"
NetworkAddresses[3]: "169.254.48.111"
NetworkAddresses[4]: "fe80::34b7:b748:56c4:953c%12"
NetworkAddresses[5]: "192.168.5.4"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 2 0x2 (Information)
ProbableCause: 1 0x1 (Other)
ProbableCauseDescription: "Physical Drive Status Changed to Rebuilding"
ProviderName: "HP Smart Array"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "No action is recommended."
Summary: "Physical drive is now rebuilding."
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2010.03.01"
SystemFirmwareVersion[1]: "2010.03.01"
SystemGUID: "30303734-3536-5A43-4A30-313230364531"
SystemModel: "ProLiant ML350 G6"
SystemName: "HWEDHPV"
SystemProductID: "470065-164"
SystemSerialNumber: "CZJ01206E1"
TIME_CREATED: 129284402266776594 0x1cb4f7a981c2412
VariableNames[0]: "FailureDescription"
VariableNames[1]: "LastFailureDescription"
VariableNames[2]: "DriveInterface"
VariableTypes[0]: 1 0x1 (string)
VariableTypes[1]: 1 0x1 (string)
VariableTypes[2]: 4 0x4 (uint16)
VariableValues[0]: "Rebuilding"
VariableValues[1]: "Hot removed"
VariableValues[2]: "3"
 
For more information, please contact HP Support.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="HP Smart Array" />
    <EventID Qualifiers="16384">203</EventID>
    <Level>4</Level>
    <Task>6</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:23:46.000Z" />
    <EventRecordID>15416</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>A physical drive status has changed to Rebuilding. Data is now being rebuilt onto this drive from the RAID set.</Data>
    <Data>No action is recommended.</Data>
    <Data>AlertingElementFormat: 2 0x2 (CIMObjectPath)
AlertingManagedElement: "\\HWEDHPV\ROOT\HPQ:HPSA_DiskDrive.CreationClassName="HPSA_DiskDrive",DeviceID="5001438007CBDEF0:4",SystemCreationClassName="HPSA_ArraySystem",SystemName="5001438007CBDEF0""
AlertType: 5 0x5 (Device Alert)
Description: "A physical drive status has changed to Rebuilding. Data is now being rebuilt onto this drive from the RAID set."
DeviceControllerLogicalLocation[0]: "Smart Array P410i in Slot 0"
DeviceModel: "EG0146FAWHU"
EventCategory: 6 0x6 (Primary Storage Device)
EventID: "203"
EventTime: "20100908182338.565000-000"
HWFirmwareVersion[0]: "HPDD"
HWLogicalLocation[0]: "Port:2I Box:1 Bay:5"
HWManufacturer: "HP"
HWPhysicalLocation: "Box 1 Bay 5"
HWSerialNumber[0]: "3SD35GK300009037KLQ5"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{ADC690B6-052B-45FE-A73F-C2A352242973}"
IndicationTime: "20100908182343.573000+060"
NetworkAddresses[0]: "fe80::c17c:eb2d:89dc:a1bd%1034"
NetworkAddresses[1]: "192.168.5.16"
NetworkAddresses[2]: "fe80::44ef:3fa7:5928:daed%1033"
NetworkAddresses[3]: "169.254.48.111"
NetworkAddresses[4]: "fe80::34b7:b748:56c4:953c%12"
NetworkAddresses[5]: "192.168.5.4"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 2 0x2 (Information)
ProbableCause: 1 0x1 (Other)
ProbableCauseDescription: "Physical Drive Status Changed to Rebuilding"
ProviderName: "HP Smart Array"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "No action is recommended."
Summary: "Physical drive is now rebuilding."
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2010.03.01"
SystemFirmwareVersion[1]: "2010.03.01"
SystemGUID: "30303734-3536-5A43-4A30-313230364531"
SystemModel: "ProLiant ML350 G6"
SystemName: "HWEDHPV"
SystemProductID: "470065-164"
SystemSerialNumber: "CZJ01206E1"
TIME_CREATED: 129284402266776594 0x1cb4f7a981c2412
VariableNames[0]: "FailureDescription"
VariableNames[1]: "LastFailureDescription"
VariableNames[2]: "DriveInterface"
VariableTypes[0]: 1 0x1 (string)
VariableTypes[1]: 1 0x1 (string)
VariableTypes[2]: 4 0x4 (uint16)
VariableValues[0]: "Rebuilding"
VariableValues[1]: "Hot removed"
VariableValues[2]: "3"
</Data>
    <Data>For more information, please contact HP Support.</Data>
  </EventData>
</Event>


Log Name:      System
Source:        HP Smart Array
Date:          08/09/2010 18:23:46
Event ID:      103
Task Category: Primary Storage Device
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
A logical drive is in a degraded state. Device is still servicing I/O but redundancy may be degraded or lost.
 
User Action
Check status of physical drives. Degraded status may be caused by failed or missing physical drives, or a data recovery operation on one or more physical drives.
 
WBEM Indication Properties
AlertingElementFormat: 2 0x2 (CIMObjectPath)
AlertingManagedElement: "\\HWEDHPV\ROOT\HPQ:HPSA_StorageVolume.CreationClassName="HPSA_StorageVolume",DeviceID="5001438007CBDEF0-LD1",SystemCreationClassName="HPSA_ArraySystem",SystemName="5001438007CBDEF0""
AlertType: 5 0x5 (Device Alert)
Description: "A logical drive is in a degraded state. Device is still servicing I/O but redundancy may be degraded or lost."
DeviceOperationalStatus[0]: 3 0x3 (Degraded)
DeviceOperationalStatus[1]: 32777 0x8009 (Recovering)
DeviceOperationalStatusPrevious[0]: 2 0x2 (OK)
EventCategory: 6 0x6 (Primary Storage Device)
EventID: "103"
EventTime: "20100908182338.565000-000"
HWLogicalLocation[0]: "Logical Drive 2 (RAID 50)"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{006734A8-977B-4802-B051-1AEF7F34C20A}"
IndicationTime: "20100908182346.708000+060"
NetworkAddresses[0]: "fe80::c17c:eb2d:89dc:a1bd%1034"
NetworkAddresses[1]: "192.168.5.16"
NetworkAddresses[2]: "fe80::44ef:3fa7:5928:daed%1033"
NetworkAddresses[3]: "169.254.48.111"
NetworkAddresses[4]: "fe80::34b7:b748:56c4:953c%12"
NetworkAddresses[5]: "192.168.5.4"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 3 0x3 (Degraded/Warning)
ProbableCause: 86 0x56 (Disk Failure)
ProbableCauseDescription: "Logical State Changed to Degraded"
ProviderName: "HP Smart Array"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "Check status of physical drives. Degraded status may be caused by failed or missing physical drives, or a data recovery operation on one or more physical drives."
Summary: "Logical drive in degraded state"
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2010.03.01"
SystemFirmwareVersion[1]: "2010.03.01"
SystemGUID: "30303734-3536-5A43-4A30-313230364531"
SystemModel: "ProLiant ML350 G6"
SystemName: "HWEDHPV"
SystemProductID: "470065-164"
SystemSerialNumber: "CZJ01206E1"
TIME_CREATED: 129284402268024596 0x1cb4f7a982f2f14
VariableNames[0]: "FailureDescription"
VariableTypes[0]: 1 0x1 (string)
VariableValues[0]: "Recovering"
 
For more information, please contact HP Support.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="HP Smart Array" />
    <EventID Qualifiers="32768">103</EventID>
    <Level>3</Level>
    <Task>6</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:23:46.000Z" />
    <EventRecordID>15417</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>A logical drive is in a degraded state. Device is still servicing I/O but redundancy may be degraded or lost.</Data>
    <Data>Check status of physical drives. Degraded status may be caused by failed or missing physical drives, or a data recovery operation on one or more physical drives.</Data>
    <Data>AlertingElementFormat: 2 0x2 (CIMObjectPath)
AlertingManagedElement: "\\HWEDHPV\ROOT\HPQ:HPSA_StorageVolume.CreationClassName="HPSA_StorageVolume",DeviceID="5001438007CBDEF0-LD1",SystemCreationClassName="HPSA_ArraySystem",SystemName="5001438007CBDEF0""
AlertType: 5 0x5 (Device Alert)
Description: "A logical drive is in a degraded state. Device is still servicing I/O but redundancy may be degraded or lost."
DeviceOperationalStatus[0]: 3 0x3 (Degraded)
DeviceOperationalStatus[1]: 32777 0x8009 (Recovering)
DeviceOperationalStatusPrevious[0]: 2 0x2 (OK)
EventCategory: 6 0x6 (Primary Storage Device)
EventID: "103"
EventTime: "20100908182338.565000-000"
HWLogicalLocation[0]: "Logical Drive 2 (RAID 50)"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{006734A8-977B-4802-B051-1AEF7F34C20A}"
IndicationTime: "20100908182346.708000+060"
NetworkAddresses[0]: "fe80::c17c:eb2d:89dc:a1bd%1034"
NetworkAddresses[1]: "192.168.5.16"
NetworkAddresses[2]: "fe80::44ef:3fa7:5928:daed%1033"
NetworkAddresses[3]: "169.254.48.111"
NetworkAddresses[4]: "fe80::34b7:b748:56c4:953c%12"
NetworkAddresses[5]: "192.168.5.4"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 3 0x3 (Degraded/Warning)
ProbableCause: 86 0x56 (Disk Failure)
ProbableCauseDescription: "Logical State Changed to Degraded"
ProviderName: "HP Smart Array"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "Check status of physical drives. Degraded status may be caused by failed or missing physical drives, or a data recovery operation on one or more physical drives."
Summary: "Logical drive in degraded state"
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2010.03.01"
SystemFirmwareVersion[1]: "2010.03.01"
SystemGUID: "30303734-3536-5A43-4A30-313230364531"
SystemModel: "ProLiant ML350 G6"
SystemName: "HWEDHPV"
SystemProductID: "470065-164"
SystemSerialNumber: "CZJ01206E1"
TIME_CREATED: 129284402268024596 0x1cb4f7a982f2f14
VariableNames[0]: "FailureDescription"
VariableTypes[0]: 1 0x1 (string)
VariableValues[0]: "Recovering"
</Data>
    <Data>For more information, please contact HP Support.</Data>
  </EventData>
</Event>


Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:23:48
Event ID:      24598
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
Logical drive 1 of array controller P410i [Embedded] has encountered a status change from:  

Status: RECOVERING  
to  
Status: OK
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24598</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:23:48.000Z" />
    <EventRecordID>15418</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>1</Data>
    <Data>RECOVERING</Data>
    <Data>OK</Data>
  </EventData>
</Event>


Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:23:48
Event ID:      24598
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
Logical drive 2 of array controller P410i [Embedded] has encountered a status change from:  

Status: READY FOR RECOVERY  
to  
Status: RECOVERING
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24598</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:23:48.000Z" />
    <EventRecordID>15419</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>2</Data>
    <Data>READY FOR RECOVERY</Data>
    <Data>RECOVERING</Data>
  </EventData>
</Event>


Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 18:53:19
Event ID:      24598
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
Logical drive 2 of array controller P410i [Embedded] has encountered a status change from:  

Status: RECOVERING  
to  
Status: OK
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24598</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:53:19.000Z" />
    <EventRecordID>15420</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>2</Data>
    <Data>RECOVERING</Data>
    <Data>OK</Data>
  </EventData>
</Event>


Log Name:      System
Source:        HP Smart Array
Date:          08/09/2010 18:53:46
Event ID:      201
Task Category: Primary Storage Device
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
A physical drive status has changed from a non-OK state to OK.
 
User Action
No action is recommended.
 
WBEM Indication Properties
AlertingElementFormat: 2 0x2 (CIMObjectPath)
AlertingManagedElement: "\\HWEDHPV\ROOT\HPQ:HPSA_DiskDrive.CreationClassName="HPSA_DiskDrive",DeviceID="5001438007CBDEF0:4",SystemCreationClassName="HPSA_ArraySystem",SystemName="5001438007CBDEF0""
AlertType: 5 0x5 (Device Alert)
Description: "A physical drive status has changed from a non-OK state to OK."
DeviceControllerLogicalLocation[0]: "Smart Array P410i in Slot 0"
DeviceModel: "EG0146FAWHU"
EventCategory: 6 0x6 (Primary Storage Device)
EventID: "201"
EventTime: "20100908185338.637000-000"
HWFirmwareVersion[0]: "HPDD"
HWLogicalLocation[0]: "Port:2I Box:1 Bay:5"
HWManufacturer: "HP"
HWPhysicalLocation: "Box 1 Bay 5"
HWSerialNumber[0]: "3SD35GK300009037KLQ5"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{24FDD0C6-9A89-42EC-8354-4EE07F5EF205}"
IndicationTime: "20100908185343.644000+060"
NetworkAddresses[0]: "fe80::c17c:eb2d:89dc:a1bd%1034"
NetworkAddresses[1]: "192.168.5.16"
NetworkAddresses[2]: "fe80::44ef:3fa7:5928:daed%1033"
NetworkAddresses[3]: "169.254.48.111"
NetworkAddresses[4]: "fe80::34b7:b748:56c4:953c%12"
NetworkAddresses[5]: "192.168.5.4"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 2 0x2 (Information)
ProbableCause: 1 0x1 (Other)
ProbableCauseDescription: "Physical Drive Status Changed to OK"
ProviderName: "HP Smart Array"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "No action is recommended."
Summary: "Physical Drive status OK"
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2010.03.01"
SystemFirmwareVersion[1]: "2010.03.01"
SystemGUID: "30303734-3536-5A43-4A30-313230364531"
SystemModel: "ProLiant ML350 G6"
SystemName: "HWEDHPV"
SystemProductID: "470065-164"
SystemSerialNumber: "CZJ01206E1"
TIME_CREATED: 129284420266712209 0x1cb4f7ec8fd5c91
VariableNames[0]: "FailureDescription"
VariableNames[1]: "LastFailureDescription"
VariableNames[2]: "DriveInterface"
VariableTypes[0]: 1 0x1 (string)
VariableTypes[1]: 1 0x1 (string)
VariableTypes[2]: 4 0x4 (uint16)
VariableValues[0]: "OK"
VariableValues[1]: "Hot removed"
VariableValues[2]: "3"
 
For more information, please contact HP Support.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="HP Smart Array" />
    <EventID Qualifiers="16384">201</EventID>
    <Level>4</Level>
    <Task>6</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:53:46.000Z" />
    <EventRecordID>15421</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>A physical drive status has changed from a non-OK state to OK.</Data>
    <Data>No action is recommended.</Data>
    <Data>AlertingElementFormat: 2 0x2 (CIMObjectPath)
AlertingManagedElement: "\\HWEDHPV\ROOT\HPQ:HPSA_DiskDrive.CreationClassName="HPSA_DiskDrive",DeviceID="5001438007CBDEF0:4",SystemCreationClassName="HPSA_ArraySystem",SystemName="5001438007CBDEF0""
AlertType: 5 0x5 (Device Alert)
Description: "A physical drive status has changed from a non-OK state to OK."
DeviceControllerLogicalLocation[0]: "Smart Array P410i in Slot 0"
DeviceModel: "EG0146FAWHU"
EventCategory: 6 0x6 (Primary Storage Device)
EventID: "201"
EventTime: "20100908185338.637000-000"
HWFirmwareVersion[0]: "HPDD"
HWLogicalLocation[0]: "Port:2I Box:1 Bay:5"
HWManufacturer: "HP"
HWPhysicalLocation: "Box 1 Bay 5"
HWSerialNumber[0]: "3SD35GK300009037KLQ5"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{24FDD0C6-9A89-42EC-8354-4EE07F5EF205}"
IndicationTime: "20100908185343.644000+060"
NetworkAddresses[0]: "fe80::c17c:eb2d:89dc:a1bd%1034"
NetworkAddresses[1]: "192.168.5.16"
NetworkAddresses[2]: "fe80::44ef:3fa7:5928:daed%1033"
NetworkAddresses[3]: "169.254.48.111"
NetworkAddresses[4]: "fe80::34b7:b748:56c4:953c%12"
NetworkAddresses[5]: "192.168.5.4"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 2 0x2 (Information)
ProbableCause: 1 0x1 (Other)
ProbableCauseDescription: "Physical Drive Status Changed to OK"
ProviderName: "HP Smart Array"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "No action is recommended."
Summary: "Physical Drive status OK"
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2010.03.01"
SystemFirmwareVersion[1]: "2010.03.01"
SystemGUID: "30303734-3536-5A43-4A30-313230364531"
SystemModel: "ProLiant ML350 G6"
SystemName: "HWEDHPV"
SystemProductID: "470065-164"
SystemSerialNumber: "CZJ01206E1"
TIME_CREATED: 129284420266712209 0x1cb4f7ec8fd5c91
VariableNames[0]: "FailureDescription"
VariableNames[1]: "LastFailureDescription"
VariableNames[2]: "DriveInterface"
VariableTypes[0]: 1 0x1 (string)
VariableTypes[1]: 1 0x1 (string)
VariableTypes[2]: 4 0x4 (uint16)
VariableValues[0]: "OK"
VariableValues[1]: "Hot removed"
VariableValues[2]: "3"
</Data>
    <Data>For more information, please contact HP Support.</Data>
  </EventData>
</Event>


Log Name:      System
Source:        HP Smart Array
Date:          08/09/2010 18:53:46
Event ID:      101
Task Category: Primary Storage Device
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
A logical drive status has changed from a non-OK state to OK.
 
User Action
No action is recommended.
 
WBEM Indication Properties
AlertingElementFormat: 2 0x2 (CIMObjectPath)
AlertingManagedElement: "\\HWEDHPV\ROOT\HPQ:HPSA_StorageVolume.CreationClassName="HPSA_StorageVolume",DeviceID="5001438007CBDEF0-LD1",SystemCreationClassName="HPSA_ArraySystem",SystemName="5001438007CBDEF0""
AlertType: 5 0x5 (Device Alert)
Description: "A logical drive status has changed from a non-OK state to OK."
DeviceOperationalStatus[0]: 2 0x2 (OK)
DeviceOperationalStatusPrevious[0]: 3 0x3 (Degraded)
DeviceOperationalStatusPrevious[1]: 32777 0x8009 (Recovering)
EventCategory: 6 0x6 (Primary Storage Device)
EventID: "101"
EventTime: "20100908185338.652000-000"
HWLogicalLocation[0]: "Logical Drive 2 (RAID 50)"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{BB0D2DDE-07F4-4F97-B190-437B4843AF2E}"
IndicationTime: "20100908185346.686000+060"
NetworkAddresses[0]: "fe80::c17c:eb2d:89dc:a1bd%1034"
NetworkAddresses[1]: "192.168.5.16"
NetworkAddresses[2]: "fe80::44ef:3fa7:5928:daed%1033"
NetworkAddresses[3]: "169.254.48.111"
NetworkAddresses[4]: "fe80::34b7:b748:56c4:953c%12"
NetworkAddresses[5]: "192.168.5.4"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 2 0x2 (Information)
ProbableCause: 1 0x1 (Other)
ProbableCauseDescription: "Logical Drive Status Changed to OK"
ProviderName: "HP Smart Array"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "No action is recommended."
Summary: "Logical drive status OK"
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2010.03.01"
SystemFirmwareVersion[1]: "2010.03.01"
SystemGUID: "30303734-3536-5A43-4A30-313230364531"
SystemModel: "ProLiant ML350 G6"
SystemName: "HWEDHPV"
SystemProductID: "470065-164"
SystemSerialNumber: "CZJ01206E1"
TIME_CREATED: 129284420267804211 0x1cb4f7ec90e0633
VariableNames[0]: "FailureDescription"
VariableTypes[0]: 1 0x1 (string)
VariableValues[0]: "N/A"
 
For more information, please contact HP Support.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="HP Smart Array" />
    <EventID Qualifiers="16384">101</EventID>
    <Level>4</Level>
    <Task>6</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T17:53:46.000Z" />
    <EventRecordID>15422</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>A logical drive status has changed from a non-OK state to OK.</Data>
    <Data>No action is recommended.</Data>
    <Data>AlertingElementFormat: 2 0x2 (CIMObjectPath)
AlertingManagedElement: "\\HWEDHPV\ROOT\HPQ:HPSA_StorageVolume.CreationClassName="HPSA_StorageVolume",DeviceID="5001438007CBDEF0-LD1",SystemCreationClassName="HPSA_ArraySystem",SystemName="5001438007CBDEF0""
AlertType: 5 0x5 (Device Alert)
Description: "A logical drive status has changed from a non-OK state to OK."
DeviceOperationalStatus[0]: 2 0x2 (OK)
DeviceOperationalStatusPrevious[0]: 3 0x3 (Degraded)
DeviceOperationalStatusPrevious[1]: 32777 0x8009 (Recovering)
EventCategory: 6 0x6 (Primary Storage Device)
EventID: "101"
EventTime: "20100908185338.652000-000"
HWLogicalLocation[0]: "Logical Drive 2 (RAID 50)"
ImpactedDomain: 4 0x4 (System)
IndicationIdentifier: "{BB0D2DDE-07F4-4F97-B190-437B4843AF2E}"
IndicationTime: "20100908185346.686000+060"
NetworkAddresses[0]: "fe80::c17c:eb2d:89dc:a1bd%1034"
NetworkAddresses[1]: "192.168.5.16"
NetworkAddresses[2]: "fe80::44ef:3fa7:5928:daed%1033"
NetworkAddresses[3]: "169.254.48.111"
NetworkAddresses[4]: "fe80::34b7:b748:56c4:953c%12"
NetworkAddresses[5]: "192.168.5.4"
OSType: 103 0x67 (Microsoft Windows Server 2008 R2)
OSVersion: "6.1.7600"
PerceivedSeverity: 2 0x2 (Information)
ProbableCause: 1 0x1 (Other)
ProbableCauseDescription: "Logical Drive Status Changed to OK"
ProviderName: "HP Smart Array"
ProviderVersion: "2.5.0.0"
RecommendedActions[0]: "No action is recommended."
Summary: "Logical drive status OK"
SystemCreationClassName: "HP_WinComputerSystem"
SystemFirmwareVersion[0]: "2010.03.01"
SystemFirmwareVersion[1]: "2010.03.01"
SystemGUID: "30303734-3536-5A43-4A30-313230364531"
SystemModel: "ProLiant ML350 G6"
SystemName: "HWEDHPV"
SystemProductID: "470065-164"
SystemSerialNumber: "CZJ01206E1"
TIME_CREATED: 129284420267804211 0x1cb4f7ec90e0633
VariableNames[0]: "FailureDescription"
VariableTypes[0]: 1 0x1 (string)
VariableValues[0]: "N/A"
</Data>
    <Data>For more information, please contact HP Support.</Data>
  </EventData>
</Event>


Log Name:      System
Source:        Cissesrv
Date:          08/09/2010 21:42:54
Event ID:      24684
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      HWEDHPV
Description:
The consistency initialization pass has completed for logical drive 1 of array controller P410i [Embedded].
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Cissesrv" />
    <EventID Qualifiers="17461">24684</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2010-09-08T20:42:54.000Z" />
    <EventRecordID>15423</EventRecordID>
    <Channel>System</Channel>
    <Computer>HWEDHPV</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Cissesrv</Data>
    <Data>P410i [Embedded]</Data>
    <Data>1</Data>
  </EventData>
</Event>
0
Comment
Question by:solplus
  • 6
  • 3
10 Comments
 
LVL 55

Expert Comment

by:andyalder
ID: 33635724
Not reading through all that, is it always bay 5? It'll be a lot easier to see using the Systems Management Homepage integrated log viewer as that lists all hardware related problems.

First thing to do is to upgrade the firmware on both the controller and the disk drives to the latest version.
0
 

Author Comment

by:solplus
ID: 33636707
Thanks andyalder.

Will look into the bay query and get back to you.
The server is running Hyper V Server 2008 and doesnt have a gui. Can the Systems Management homepage be installed from a command line prompt? If so any guidance on how to do it would be much appreciated.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 33636897
SMH should be installed if you did a smartstart install, try opening a webpage to http://<server IP>:2301 or HTTPS://<serverIP>:2381

Haven't tried installing from command line but if you download the Proliant Support Pack you can do a push install from a remote PC.
0
 

Author Comment

by:solplus
ID: 33637058
Thanks andyalder.
We havent done a push install before. How do we do it?
0
Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

 
LVL 55

Expert Comment

by:andyalder
ID: 33637436
It's pretty self evident, run the prog on a remote PC and there's option to put remote name/address in and click green arrow. It won't install stuff that's not relevant but it may cause a reboot or two if it loads newer drivers so don't do in business hours.
0
 

Author Comment

by:solplus
ID: 33657702
Logged a call with HP. They want me to run the Array diagnostics. That means shutting down the server and booting from the CD. The problem is that we only notived the errors in the event viewer when we were investigating why backups were going slow and failing, so we do not have a good backup of the server and are a bit reticent to shutdown in this situation in case it does not come back up. At least at the moment it is working. That is aslo why we have not followed up your suggestion of installing the Systems Management Homepage, as you thought it might cause a reboot. Currenty backing up the virtual servers and virtual discs to an external disc and will look for a slot to down the server after that has completed. I really cannot see why  the Array Diagnostics have to be run at boot time from the CD. I would have though that there should be an online tool that can be used to interogate the array?
0
 

Author Comment

by:solplus
ID: 33657714
Sorry, to answer your earlier question. Yes it is alway bay 5 and it would appear to have been happening on and of for a few weeks. The problem is that removing/replacing a disc is in theory legitamate, so the removal and reinstallation and not reported as errors, or even warnings, but are reported as informational and get lost in the 1000's of other informational events.
0
 

Accepted Solution

by:
solplus earned 0 total points
ID: 34390060
Apologies for not closing this question. After much (and I mean a lot of) persuation, we managed to convince HP that this must be a disc problem. They finally agreed to send out a replacement disc. When we installed the disc, the "information" events stopped. I would prefer that this questiion be closed with this as the solution to encourage anyone else who encounters the same issue to log it as a hardware fault!
0
 

Author Closing Comment

by:solplus
ID: 34421711
If the event log says that a disc is being removed and you know that it is not, then it has to be a problem with the software, the bay or the disc. If it always reports the disc being removed from the same bay, then the problem has to be the bay or the disc. As the disc is easier to replace than the bay, I suggest trying a replacement disc to eliminate this. If you do not have a replacement disc then log a hardware fault and grit your teeth until they are fed up running diagnostices.
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Suggested Solutions

Every server (virtual or physical) needs a console: and the console can be provided through hardware directly connected, software for remote connections, local connections, through a KVM, etc. This document explains the different types of consol…
Data center, now-a-days, is referred as the home of all the advanced technologies. In-fact, most of the businesses are now establishing their entire organizational structure around the IT capabilities.
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…
This tutorial will walk an individual through the process of transferring the five major, necessary Active Directory Roles, commonly referred to as the FSMO roles to another domain controller. Log onto the new domain controller with a user account t…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now