Solaris 10 - HBA port offline

Just to be sure,
You have two DUAL Port HBAs Emulex
or do you have a single multi-channel quad port HBA?
 
both ports on each card or all ports on the quad port card are connected to a switch or directly to a SAN?
Check the switch/SAN to make sure they are properly zoned.  Note that the disconnected/offline indicators relfect an unknown for the type.

Is the port on the switch/SAN reflect that it is connected?
Do you use something similar to a multipathing PowerPATh from EMC? that is configured in a failover mode versus load-balance mode?

From the information provided, you have two dual port HBAs, one port on each HBA is connected, the anther port is not in use?
If this was the case, everything looks normal to me.

So I have to offline the c9 for example and online the c6?

What would be the comand to do it?

Tks for the reply and sorry for the newbie question...

But I need to have c5 and c6 Online.. I dont care for c9 and c10... Is this something doable?

Or it involves cable changing?

PS: how do you know that c5 and 6 are on one HBA and c9 and 10 are on another HBA?

Maybe what I need is to have something like this in the luxadm then,right?
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    CONNECTED

Instead on what I have now:
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    NOT CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    NOT CONNECTED

From the Serial Number of HBAs above, that c5 and c6 on the same HBA. Now you have better configuration. Why you want both ports on the same card online instead one port on each HBA online?

Im having a problem that some LUNs are not visible in this node....

So comparing to another node in the same cluser where these LUNs are visible I noticed that on this node the c5 and c6 are configured, so I figured if I copy the conf. from the other "good" node I would be ok...

So in the good node where the LUns are there I have this:
 # cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             scsi-sas     connected    configured   unknown
c1::dsk/c1t6d0                 CD-ROM       connected    configured   unknown
c3                             scsi-sas     connected    configured   unknown
c3::w5000cca015393dad,0        disk-path    connected    configured   unknown
c4                             scsi-sas     connected    unconfigured unknown
c5                             fc-fabric    connected    configured   unknown
c5::10000000c995ef9d           unknown      connected    unconfigured unknown
c5::10000000c9aee644           unknown      connected    unconfigured unknown
c5::10000090fa345b14           unknown      connected    unconfigured unknown
c5::10000090fa346bbe           unknown      connected    unconfigured unknown
c5::203200a0b8742a9c           disk         connected    unconfigured unknown
c5::203300a0b8742a9c           disk         connected    unconfigured unknown
c5::203400a0b84930ed           disk         connected    unconfigured unknown
c5::203400a0b849350f           disk         connected    configured   unknown
c5::203400a0b8742b24           disk         connected    unconfigured unknown
c5::203500a0b8742b24           disk         connected    unconfigured unknown
c5::20360080e53ed738           disk         connected    configured   unknown
c5::20370080e53ed738           disk         connected    configured   unknown
c6                             fc-fabric    connected    configured   unknown
c6::10000000c995edd0           unknown      connected    unconfigured unknown
c6::10000000c9aee645           unknown      connected    unconfigured unknown
c6::202200a0b8742a9c           disk         connected    unconfigured unknown
c6::202300a0b8742a9c           disk         connected    unconfigured unknown
c6::202400a0b84930ed           disk         connected    configured   unknown
c6::202400a0b849350f           disk         connected    configured   unknown
c6::202400a0b8742b24           disk         connected    unconfigured unknown
c6::202500a0b8742b24           disk         connected    unconfigured unknown
c7                             scsi-sas     connected    unconfigured unknown
c8                             fc           connected    unconfigured unknown
c9                             fc           connected    unconfigured unknown
c10                            scsi-sas     connected    configured   unknown
c10::w5000cca0153684f5,0       disk-path    connected    configured   unknown
On the faulty node I have the c6 as unconfigured:
# cfgadm -al
# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             scsi-sas     connected    configured   unknown
c1::dsk/c1t6d0                 CD-ROM       connected    configured   unknown
c3                             scsi-sas     connected    configured   unknown
c3::w5000cca02277ebf5,0        disk-path    connected    configured   unknown
c4                             scsi-sas     connected    unconfigured unknown
c5                             fc-fabric    connected    configured   unknown
c5::10000000c995ef9d           unknown      connected    unconfigured unknown
c5::10000000c9aee644           unknown      connected    unconfigured unknown
c5::10000000c9aef3ba           unknown      connected    unconfigured unknown
c5::10000090fa346bbe           unknown      connected    unconfigured unknown
c5::203200a0b8742a9c           disk         connected    unconfigured unknown
c5::203300a0b8742a9c           disk         connected    unconfigured unknown
c5::203400a0b84930ed           disk         connected    unconfigured unknown
c5::203400a0b849350f           disk         connected    configured   unknown
c5::203400a0b8742b24           disk         connected    unconfigured unknown
c5::203500a0b8742b24           disk         connected    unconfigured unknown
c5::20360080e53ed738           disk         connected    configured   unknown
c5::20370080e53ed738           disk         connected    configured   unknown
c6                             fc           connected    unconfigured unknown
c7                             scsi-sas     connected    configured   unknown
c7::w5000cca0227acb01,0        disk-path    connected    configured   unknown
c8                             scsi-sas     connected    unconfigured unknown
c9                             fc-fabric    connected    configured   unknown
c9::10000000c995ef9d           unknown      connected    unconfigured unknown
c9::10000000c9aee644           unknown      connected    unconfigured unknown
c9::10000000c9aef3ba           unknown      connected    unconfigured unknown
c9::10000090fa345b14           unknown      connected    unconfigured unknown
c9::203200a0b8742a9c           disk         connected    unconfigured unknown
c9::203300a0b8742a9c           disk         connected    unconfigured unknown
c9::203400a0b84930ed           disk         connected    unconfigured unknown
c9::203400a0b849350f           disk         connected    configured   unknown
c9::203400a0b8742b24           disk         connected    unconfigured unknown
c9::203500a0b8742b24           disk         connected    unconfigured unknown
c9::20360080e53ed738           disk         connected    configured   unknown
c9::20370080e53ed738           disk         connected    configured   unknown
c10                            fc           connected    unconfigured unknown


====

So thats why I was asking to have c5 and c6 configured and Online.

Thsi is the good node:
 # fcinfo hba-port
HBA Port WWN: 10000000c9aef3ba
        OS Device Name: /dev/cfg/c5
        Manufacturer: Emulex
        Model: LPe12002-S
        Firmware Version: 1.11a5 (U3D1.11A5)
        FCode/BIOS Version: Boot:5.03a0 Fcode:3.01a1
        Serial Number: 0999VM0-1103001MQ7
        Driver Name: emlxs
        Driver Version: 2.60h (2010.10.22.16.55)
        Type: N-port
        State: online
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: 8Gb
        Node WWN: 20000000c9aef3ba
HBA Port WWN: 10000000c9aef3bb
        OS Device Name: /dev/cfg/c6
        Manufacturer: Emulex
        Model: LPe12002-S
        Firmware Version: 1.11a5 (U3D1.11A5)
        FCode/BIOS Version: Boot:5.03a0 Fcode:3.01a1
        Serial Number: 0999VM0-1103001MQ7
        Driver Name: emlxs
        Driver Version: 2.60h (2010.10.22.16.55)
        Type: N-port
        State: online
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: 8Gb
        Node WWN: 20000000c9aef3bb

Note that as ganjos pointed out, this might not solve the issue you have.  You have to make sure that the switch is zoned correctly for the missing LUNS/PATHS to be serviced by the port to which you are connecting.

Did the move of the ports fixed the issue?
Did someone move the port from C6 to c9 thinking it will move the path/lun and work without any changes?

I assumed the setting was right, only cable pluged into wrong port.

Havent moved it yet.. Only tomorrow I can change the cable manually.

I think the person who set the cables put it by mistake.

The thing that I think that the machine (M4000) has only two HBA ports.... if so, what should I do?

Sry I mean T3 server.

You have showed four ports here:
# luxadm -e port
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    NOT CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    NOT CONNECTED

Do you have a fibre switch to which the T3 is connected and to which the solaris boxes are connected?

You could pull the wrong cable making all luns inaccessible.

the display of
offline
online
online
offline

are not a be all and end all for functionality such that if you had
online
online
offline
offline
is better than
offline
offline
online
online

There must be a reason for moving that will resolve the underlying issue.

As ganjos pointed out and I agree that using Two HBAs provides you with a tolerance for HBA failure.

I have four Gigabit Ethernet ports NET-0, 1, 2, 3... but I dont think these ports refers to it...

The thing is how do I know which port will be c6 or c9..... Anyway I will check tomorrow.

Tks for the help so far.

Tks for the reply.

Yes I have a fiber switch connected to the T3 and where the disk arrays are connected as well.

I will confirm but as far as I know there are two ports available for fiber in the T3 and today I have fiber cables connected to both of them.

# luxadm -e port
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    NOT CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    NOT CONNECTED

Sorry if its a beginner question.. Im trying to learn/understand.. but if I have connected to both of ports in the T3.. shoudlnt all ports shows be connected?

Is the port: /devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl
only somehow the port pci@1 "used twice" or its a completely different port?

If its only one port "used twice", if I already have the cable there shouldnt I have both "interfaces" of it CONNECT?

Tks for the help so far.

Tks for the explanation.... I had never touched the "disks" part of the setup.. thats what Im trying to understand.. tks for the help.

I was told that there is no conf. in the brocade done... so its basically just plug and play.. then You have access to the disk array....

One stupid question is: this HBA ports... (the 4 of then, 2 for each board) they are at the server not the disk array, right?

Im asking this because I will confirm today but I pretty sure that the server has only 2 HBA ports, not 4....

And since the other server connects to the brocade and to the diskarray, I dont think between the brocade and the disk array there is any misplaced cable....

One thing I read search around was that I havent put the new host to the array storage pools masking list... do you think this can be the problem here and the cables are actually ok?

If the cables are incorrect placed, after I place then correctly do I have to do any command or its just wait?

Tks,
Joao

Yes for the last question.

I have moved the fibers and now it looks ok:

Faulty node now:
# luxadm -e port
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    CONNECTED
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    NOT CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      NOT CONNECTED

And,
cfgadm -al -o show_FCP_dev
.
.
c6                             fc-fabric    connected    configured   unknown
c6::10000000c995ef9d           unknown      connected    unconfigured unknown
c6::10000000c9aee644           unknown      connected    unconfigured unknown
c6::10000000c9aef3ba           unknown      connected    unconfigured unknown
c6::10000090fa345b14           unknown      connected    unconfigured unknown
c6::203200a0b8742a9c           disk         connected    unconfigured unknown
c6::203300a0b8742a9c           disk         connected    unconfigured unknown
c6::203400a0b84930ed           disk         connected    unconfigured unknown
c6::203400a0b849350f,31        disk         connected    configured   unknown
c6::203400a0b8742b24           disk         connected    unconfigured unknown
c6::203500a0b8742b24           disk         connected    unconfigured unknown
c6::20360080e53ed738,0         disk         connected    configured   unknown
c6::20360080e53ed738,1         disk         connected    configured   unknown
c6::20360080e53ed738,2         disk         connected    configured   unknown
c6::20360080e53ed738,3         disk         connected    configured   unknown
c6::20360080e53ed738,4         disk         connected    configured   unknown
c6::20360080e53ed738,5         disk         connected    configured   unknown
c6::20360080e53ed738,6         disk         connected    configured   unknown
c6::20360080e53ed738,31        disk         connected    configured   unknown
c6::20370080e53ed738,0         disk         connected    configured   unknown
c6::20370080e53ed738,1         disk         connected    configured   unknown
c6::20370080e53ed738,2         disk         connected    configured   unknown
c6::20370080e53ed738,3         disk         connected    configured   unknown
c6::20370080e53ed738,4         disk         connected    configured   unknown
c6::20370080e53ed738,5         disk         connected    configured   unknown
c6::20370080e53ed738,6         disk         connected    configured   unknown
c6::20370080e53ed738,31        disk         connected    configured   unknown

And,
HBA Port WWN: 10000090fa345b15
        OS Device Name: /dev/cfg/c6
        Manufacturer: Emulex
        Model: LPe12002-S
        Firmware Version: 2.01a4 (U3D2.01A4)
        FCode/BIOS Version: Boot:5.03a0 Fcode:3.01a1
        Serial Number: 4925382+13180000OQ
        Driver Name: emlxs
        Driver Version: 2.80.8.0 (2012.09.17.15.45)
        Type: N-port
        State: online
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: 8Gb
        Node WWN: 20000090fa345b15

====

The thing is taht I sill cant see the c6 LUNs in the format command... is there any command to be issues so I can see them?

I havent added the host to the list of the array yet.. can this be the cause?

The Luns are visible in the other nodes... below is the format for the faulty and the good node respectively:

FAULTY:
AVAILABLE DISK SELECTIONS:
       0. c0t5000CCA0227ACB00d0 <HITACHI-H109030SESUN300G-A31A cyl 46873 alt 2 hd 20 sec 625>  solaris
          /scsi_vhci/disk@g5000cca0227acb00
       1. c0t5000CCA02277EBF4d0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>  rootmirr
          /scsi_vhci/disk@g5000cca02277ebf4
       2. c0t60080E50003ED5B80000052B52A633CFd0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed5b80000052b52a633cf
       3. c0t60080E50003ED5B80000052F52A633E7d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed5b80000052f52a633e7
       4. c0t60080E50003ED5B80000052752A633B7d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed5b80000052752a633b7
       5. c0t60080E50003ED5B80000053552AA44D3d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed5b80000053552aa44d3
       6. c0t60080E50003ED7380000058052A63408d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed7380000058052a63408
       7. c0t60080E50003ED7380000057C52A633DAd0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed7380000057c52a633da
       8. c0t60080E50003ED7380000057E52A633F1d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed7380000057e52a633f1
       9. c5t203400A0B849350Fd31 <SUN-UniversalXport-0670 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w203400a0b849350f,1f
      10. c5t20360080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,1f
      11. c5t20370080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,1f
      12. c6t203400A0B849350Fd31 <SUN-UniversalXport-0670 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w203400a0b849350f,1f
      13. c6t20360080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w20360080e53ed738,1f
      14. c6t20370080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w20370080e53ed738,1f


GOOD:
AVAILABLE DISK SELECTIONS:
       0. c0t5000CCA015393DACd0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>
          /scsi_vhci/disk@g5000cca015393dac
       1. c0t5000CCA0153684F4d0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>
          /scsi_vhci/disk@g5000cca0153684f4
       2. c5t203400A0B849350Fd31 <SUN-UniversalXport-0670 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w203400a0b849350f,1f
       3. c5t20360080E53ED738d0 <drive type unknown>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,0
       4. c5t20360080E53ED738d1 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,1
       5. c5t20360080E53ED738d2 <drive type unknown>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,2
       6. c5t20360080E53ED738d3 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,3
       7. c5t20360080E53ED738d4 <drive type unknown>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,4
       8. c5t20360080E53ED738d5 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,5
       9. c5t20360080E53ED738d6 <drive type unknown>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,6
      10. c5t20360080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,1f
      11. c5t20370080E53ED738d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,0
      12. c5t20370080E53ED738d1 <drive type unknown>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,1
      13. c5t20370080E53ED738d2 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,2
      14. c5t20370080E53ED738d3 <drive type unknown>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,3
      15. c5t20370080E53ED738d4 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,4
      16. c5t20370080E53ED738d5 <drive type unknown>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,5
      17. c5t20370080E53ED738d6 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,6
      18. c5t20370080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,1f
      19. c6t202400A0B84930EDd0 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,0
      20. c6t202400A0B84930EDd1 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,1
      21. c6t202400A0B84930EDd2 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,2
      22. c6t202400A0B84930EDd3 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,3
      23. c6t202400A0B84930EDd4 <SUN-LCSM100_F-0670 cyl 25598 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,4
      24. c6t202400A0B84930EDd5 <SUN-LCSM100_F-0670 cyl 25598 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,5
      25. c6t202400A0B84930EDd6 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,6
      26. c6t202400A0B84930EDd7 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,7
      27. c6t202400A0B84930EDd8 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,8
      28. c6t202400A0B84930EDd9 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,9
      29. c6t202400A0B84930EDd10 <SUN-LCSM100_F-0670 cyl 15358 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,a
      30. c6t202400A0B84930EDd11 <SUN-LCSM100_F-0670 cyl 25598 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,b
      31. c6t202400A0B84930EDd12 <SUN-LCSM100_F-0670 cyl 25598 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,c
      32. c6t202400A0B84930EDd13 <SUN-LCSM100_F-0670 cyl 25598 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,d
      33. c6t202400A0B84930EDd14 <SUN-LCSM100_F-0670 cyl 25598 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,e
      34. c6t202400A0B84930EDd15 <SUN-LCSM100_F-0670 cyl 25598 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,f
      35. c6t202400A0B84930EDd16 <SUN-LCSM100_F-0670 cyl 51198 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,10
      36. c6t202400A0B849350Fd31 <SUN-UniversalXport-0670 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b849350f,1f

I did - see below.

I think its the host list in the array that is prevent the node to see the LUNs.
Let me know what you think.

 # cfgadm
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             scsi-sas     connected    configured   unknown
c3                             scsi-sas     connected    unconfigured unknown
c4                             scsi-sas     connected    unconfigured unknown
c5                             fc-fabric    connected    configured   unknown
c6                             fc-fabric    connected    configured   unknown
c7                             scsi-sas     connected    unconfigured unknown
c8                             scsi-sas     connected    unconfigured unknown
c9                             fc-fabric    connected    configured   unknown
c10                            fc           connected    unconfigured unknown
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         unknown      empty        unconfigured ok
usb0/3                         unknown      empty        unconfigured ok
usb1/1                         unknown      empty        unconfigured ok
usb1/2                         unknown      empty        unconfigured ok
usb2/1                         unknown      empty        unconfigured ok
usb2/2                         usb-hub      connected    configured   ok
usb2/2.1                       unknown      empty        unconfigured ok
usb2/2.2                       unknown      empty        unconfigured ok
usb2/2.3                       usb-hub      connected    configured   ok
usb2/2.3.1                     unknown      empty        unconfigured ok
usb2/2.3.2                     usb-storage  connected    configured   ok
usb2/2.3.3                     usb-communi  connected    configured   ok
usb2/2.4                       usb-device   connected    configured   ok
usb2/3                         unknown      empty        unconfigured ok
usb2/4                         usb-hub      connected    configured   ok
usb2/4.1                       unknown      empty        unconfigured ok
usb2/4.2                       unknown      empty        unconfigured ok
usb2/4.3                       unknown      empty        unconfigured ok
usb2/4.4                       unknown      empty        unconfigured ok
usb2/5                         unknown      empty        unconfigured ok

# devfsadm
# format
Searching for disks...done

c5t203400A0B849350Fd31: configured with capacity of 16.00MB
c5t20360080E53ED738d31: configured with capacity of 16.00MB
c5t20370080E53ED738d31: configured with capacity of 16.00MB
c6t203400A0B849350Fd31: configured with capacity of 16.00MB
c6t20360080E53ED738d31: configured with capacity of 16.00MB
c6t20370080E53ED738d31: configured with capacity of 16.00MB


AVAILABLE DISK SELECTIONS:
       0. c0t5000CCA0227ACB00d0 <HITACHI-H109030SESUN300G-A31A cyl 46873 alt 2 hd 20 sec 625>  solaris
          /scsi_vhci/disk@g5000cca0227acb00
       1. c0t5000CCA02277EBF4d0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>  rootmirr
          /scsi_vhci/disk@g5000cca02277ebf4
       2. c0t60080E50003ED5B80000052B52A633CFd0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed5b80000052b52a633cf
       3. c0t60080E50003ED5B80000052F52A633E7d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed5b80000052f52a633e7
       4. c0t60080E50003ED5B80000052752A633B7d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed5b80000052752a633b7
       5. c0t60080E50003ED5B80000053552AA44D3d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed5b80000053552aa44d3
       6. c0t60080E50003ED7380000058052A63408d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed7380000058052a63408
       7. c0t60080E50003ED7380000057C52A633DAd0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed7380000057c52a633da
       8. c0t60080E50003ED7380000057E52A633F1d0 <SUN-LCSM100_F-0784 cyl 32766 alt 2 hd 512 sec 64>
          /scsi_vhci/ssd@g60080e50003ed7380000057e52a633f1
       9. c5t203400A0B849350Fd31 <SUN-UniversalXport-0670 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w203400a0b849350f,1f
      10. c5t20360080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,1f
      11. c5t20370080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,1f
      12. c6t203400A0B849350Fd31 <SUN-UniversalXport-0670 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w203400a0b849350f,1f
      13. c6t20360080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w20360080e53ed738,1f
      14. c6t20370080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w20370080e53ed738,1f
Specify disk (enter its number):

I will check them and let you know.

Tks again for the help!

From this output:
# cfgadm
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             scsi-sas     connected    configured   unknown
c3                             scsi-sas     connected    unconfigured unknown
c4                             scsi-sas     connected    unconfigured unknown
c5                             fc-fabric    connected    configured   unknown
c6                             fc-fabric    connected    configured   unknown
c7                             scsi-sas     connected    unconfigured unknown
c8                             scsi-sas     connected    unconfigured unknown
c9                             fc-fabric    connected    configured   unknown
c10                            fc           connected    unconfigured unknown

You still have three not two fabric connections, why is that?

I have no idea.. but the luxadm shows only two ports connected which is how it is now.

# luxadm -e port
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    CONNECTED
/devices/pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0:devctl    NOT CONNECTED
/devices/pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0/fp@0,0:devctl      NOT CONNECTED

And also:
HBA Port WWN: 10000090fa346bbe
        OS Device Name: /dev/cfg/c9
        Manufacturer: Emulex
        Model: LPe12002-S
        Firmware Version: 2.01a4 (U3D2.01A4)
        FCode/BIOS Version: Boot:5.03a0 Fcode:3.01a1
        Serial Number: 4925382+13180000P2
        Driver Name: emlxs
        Driver Version: 2.80.8.0 (2012.09.17.15.45)
        Type: N-port
        State: offline
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: not established
        Node WWN: 20000090fa346bbe


But in the cfgadm:
# cfgadm
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             scsi-sas     connected    configured   unknown
c3                             scsi-sas     connected    unconfigured unknown
c4                             scsi-sas     connected    unconfigured unknown
c5                             fc-fabric    connected    configured   unknown
c6                             fc-fabric    connected    configured   unknown
c7                             scsi-sas     connected    unconfigured unknown
c8                             scsi-sas     connected    unconfigured unknown
c9                             fc-fabric    connected    configured   unknown
c10                            fc           connected    unconfigured unknown

=====

Anything to be done about this?

I will be checking each of the arrays storage pools masking list and add the faulty node to it.... hopefully after this the LUNs appear with the format command.

If no cable connected at c9, you can manually unconfigure by:
# cfgadm -c unconfigure c9

Any harm on leaving it like this?

What exactly does this configured means? Because its not connected... is it a bug or something like this?

No harm if no cable connected, but better to clean up in some stage, not a bug.

Hi again,

I made conf changes in the array and restarted the faulty node .. now I see this in the var/adm/messages..

Jan 21 12:45:40 dp-node9 scsi: [ID 107833 kern.warning] WARNING: /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20370080e53ed738,1f (ssd2):
Jan 21 12:45:40 dp-node9        Corrupt label; wrong magic number
Jan 21 12:45:40 dp-node9 scsi: [ID 107833 kern.warning] WARNING: /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w203400a0b849350f,1f (ssd3):
Jan 21 12:45:40 dp-node9        Corrupt label; wrong magic number
Jan 21 12:45:40 dp-node9 scsi: [ID 107833 kern.warning] WARNING: /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w203400a0b849350f,1f (ssd3):
Jan 21 12:45:40 dp-node9        Corrupt label; wrong magic number
Jan 21 12:45:40 dp-node9 scsi: [ID 107833 kern.warning] WARNING: /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,1f (ssd0):
Jan 21 12:45:40 dp-node9        Corrupt label; wrong magic number
Jan 21 12:45:40 dp-node9 scsi: [ID 107833 kern.warning] WARNING: /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0/fp@0,0/ssd@w20360080e53ed738,1f (ssd0):
Jan 21 12:45:40 dp-node9        Corrupt label; wrong magic number


Anyway to solve this?

I think this the cause for the LUNs not be in the format command....

I also found that the onnly disk the faulty node can see from the array is the one with SUN-UniversalXport-0670.. see below:

14. c6t20370080E53ED738d31 <SUN-UniversalXport-0784 cyl 8 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w20370080e53ed738,1f

And the good node with all the LUNs has disks with SUN-LCSM100_F-0670:

      35. c6t202400A0B84930EDd16 <SUN-LCSM100_F-0670 cyl 51198 alt 2 hd 64 sec 64>
          /pci@400/pci@1/pci@0/pci@8/SUNW,emlxs@0,1/fp@0,0/ssd@w202400a0b84930ed,10

And also this one disk that the faulty node can see as well.... not sure what this means...

Tks for all the help so far,
Jaoo

Wrong magic number means the LUN is un configured /formated un partitioned.

This is normal for a new hard drive/LUN added to a system.

Is the UniverXport LUN supposed to be one from which this system will boot?

Compare your definition/settings/configuration of this LUN versus your other LUN.

Since you currently use comparisons to determine correct/incorrect/unexpected functionality, using the same method evaluating/comparing SAN level LUN definition as well as switch level port settings/configuration.

I did now wa reboot -- -r to rebuild the device table and it worked... I can see the LUNs now :)

I read that The magic number really isn't a big problem it just means that there are pointers for disks that aren't being used.. so I will not worry about it.

Tks.

The magic number is not a problem when you are adding disk resources (hard drives, LUNS) (expected). It is a problem when an existing drive has this error which usually means the drive is dead/malfunctioning (unexpected).