Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 3186
  • Last Modified:

How to configure sun server to utilize SAN storage; volumn manager Veritas

Hi,

We have two sun servers that just got some space from a SAN storage system(Hitachi Thunder 9500V series). Now we need to configure storage in sun to utilize this space. Some existing volumns will be expanded. I know we probably need to use command vxdiskadm, luxadm but I am not sure the exact steps. Can anyone help? Thanks
0
ebi168
Asked:
ebi168
  • 15
  • 10
9 Solutions
 
pshattuckCommented:
You need to install SUN SAN 4.x

Get the s/w from your tech

Follow the instructions ( if your server is not allowed traffic from SUN then set proxy to local)

SAN 4.x will come with PatchPro so that patches can be installed with it

If these are not the latest patches then you will have to update them ( Patches will be 113039 -113042 and 113046)

      NOTE: I have installed this many times and I was using SAN 4.2 .. sometimes I had to uninstall the old patches then install the new patches

You will now have to edit the /kernel/drv/scsi_vhci.conf file

Depending on how you have your server setup will depend on your values

1) mpxio Global

If you have only one Fiber Card then set this value for disable to yes

If you have 2 cards and they are being used for redundancy then set this value for disable to no

2) Load-balancing

If you have only one Fiber Card then set this value for load-balance to none

If you have 2 cards and they are being used for redundancy then set this value load-balasnce to round-robin


We need to know what Fiber Channel Cards you are using, the server model,  and what version of OS

I will Qlogic Fiber Cards and Solaris 9 as an example for right now so you can follow along


3) Now say you have Qlogic cards

You would type luxadm qlgc

you will get an output as something like this

Found path to X FC100/P, ISP2200, ISP23xx, Dervices

Open Device: /devices/pci@9,600000/SUNW,qlc@2/fp@0,0:devctl
Detected FCode Version:    ISP2200 FC-AL:Host Adapter Driver: 1.xx
01/11/20

Open Device: /devices/pci@8,600000/SUNW,qlc@1/fp@0,0:devctl
Detected FCode Version:    ISP2200 FC-AL:Host Adapter Driver: 1.xx  
04/04/20

A THIRD WILL APPEAR IF YOU HAVE 2 FIBER CARDS

Open Device: /devices/pci@8,600000/SUNW,qlc@2/fp@0,0:devctl
Detected FCode Version:    ISP2200 FC-AL:Host Adapter Driver: 1.xx  
04/04/20


Generally /devices/pci@9,600000 is the fiber controller for your disk drives -- IF YOU HAVE FIBER CONTROLLED HARD DRIVES IN YOUR SERVER

/devices/pci@8,600000 is usually your Fiber Cards

YOU MUST VERIFY THIS BECUASE YOU CAN SERIOUSLY SCREW UP YOUR SYSTEM!!!!!!


So now we have identified the Qlogic Fiber Cards

4) You will now type luxadm -e forcelip /devices/pci@8,600000/SUNW,qlc@1/fp@0,0:devctl

and if you have a 2nd fiber card

luxadm -e forcelip /devices/pci@8,600000/SUNW,qlc@2/fp@0,0:devctl

5) Now at the prompt type format

You should now see the LUNs accessible via the port your SAM administrator has your servers connected to.

For HITACHI the LUNs will look like this

/dev/dsk/c4t50060E80000000000000A8430000009Cd0s0

If you have failover enabled then you should only see 1 entry for each LUN .. if you see dupe entries then you did not follw the instruction number 1

If you do not see any LUNs there are 2 things you can do (Becuase SUN is flaky wioth 1st time connection with HITACHI)

a) remove the fiber cables from the fiber cards on the server for 2 minutes, put them back in and perform steps 4 and 5 again

b) if a does not work then do a boot -r with the cables connected.


Once you are happy with what you see in the format command , edit the /etc/vfdstab file accordingly and then mount the LUNs
0
 
pshattuckCommented:
As for Veritas ... I can't help you .. I avoid Veritas like the plague ......
0
 
ebi168Author Commented:
Thank you for your detailed desc. Let me look into it.
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 
ebi168Author Commented:
I see we have SAN foundation kit, about version: PKG_CLIENT_VERSION=8. Is this the SUN SAN s/w you talked about. So how can I check on the fiber channel card we used.

0
 
pshattuckCommented:
" I see we have SAN foundation kit, about version: PKG_CLIENT_VERSION=8. Is this the SUN SAN s/w you talked about."

That sounds right... It should have an install script called install_it or something close to it. You should check to make sure it's not old .. if it is .. then get the latest version for you OS version from your SUN tech. It should not say 8 .. I should say something like 4.0, 4.01, 4.1, 4.2, etc...



So how can I check on the fiber channel card we used.

Well not knowing what version of Solaris you are using you can:

sysdef | more and look at eahc "pci, instance #"

pkginfo | grep iber OR pkginfor | grep ibre

Look at the packing list that came with the server

run dmesg if you can do a reboot

open the server up and check :)
0
 
ebi168Author Commented:
Looks like we have two SUN JNI FIBER HBA ADAPTERs:

FCE-6460-N HBA 0 and FCE-6460-N HBA 1

Will the steps you listed be any difference. So what parameter should I feed to luxadm, yours is qlgc.

Additionally, could you also provide some resources that have related information on how to explore SAN storage in sun.
0
 
yuzhCommented:
0
 
pshattuckCommented:
Ok for JNI cards do this


 You need to install SUN SAN 4.x

Get the s/w from your tech

Follow the instructions ( if your server is not allowed traffic from SUN then set proxy to local)

SAN 4.x will come with PatchPro so that patches can be installed with it

If these are not the latest patches then you will have to update them ( Patches will be 113039 -113042 and 113046)

      NOTE: I have installed this many times and I was using SAN 4.2 .. sometimes I had to uninstall the old patches then install the new patches

You will now have to edit the /kernel/drv/scsi_vhci.conf file

Depending on how you have your server setup will depend on your values

1) mpxio Global

If you have only one Fiber Card then set this value for disable to yes

If you have 2 cards and they are being used for redundancy then set this value for disable to no

2) Load-balancing

If you have only one Fiber Card then set this value for load-balance to none

If you have 2 cards and they are being used for redundancy then set this value load-balasnce to round-robin

3) For JNI cards do a

luxadm probe -p

You should get something like this

/devices/pci@8,600000/SUNW,qlc@2/fp@0,0:devctl CONNECTED

There may be a second card, if it does not say connected then you will need to use the cfgadm command to get it connected

4) You will now type luxadm -e forcelip /devices/pci@8,600000/SUNW,qlc@2/fp@0,0:devctl

MAKE SURE THAT DEVICE IS THE JNI!!!!!

5) Now at the prompt type format

You should now see the LUNs accessible via the port your SAM administrator has your servers connected to.

For HITACHI the LUNs will look like this

/dev/dsk/c4t50060E80000000000000A8430000009Cd0s0

If you have failover enabled then you should only see 1 entry for each LUN .. if you see dupe entries then you did not follw the instruction number 1

If you do not see any LUNs there are 2 things you can do (Becuase SUN is flaky wioth 1st time connection with HITACHI)

a) remove the fiber cables from the fiber cards on the server for 2 minutes, put them back in and perform steps 4 and 5 again

b) if a does not work then do a boot -r with the cables connected.


Once you are happy with what you see in the format command , edit the /etc/vfdstab file accordingly and then mount the LUNs






Additionally, could you also provide some resources that have related information on how to explore SAN storage in sun.

You don't explore the HITACHi on the SUN. What happens is that LUNs on the HITACHI are assigned to a port(s) for access. You server will be connected at those port(s). So you will only see the LUNs assigned to the ports where your SUN server is connected. If you attached via a fabric then it will be up to the Administrator of the fabric to determine which ports you are allowed to look at.


To explore the HITACHI fully you must be a the DCU laptop that is in the HITACHI itself. Depending on your SAN administrator, access could be restricted.
0
 
ebi168Author Commented:

Our mpxio-disable="yes"; however our load-balance="round-robin"; How do you explain this? Based yours, it looks like contradictary. And since we are just expanding space; these setting must have been used in initial storage configuration from SAN.

Also how will I be certain that both cards are used.

Is it a must that I download the latest patch. Also what I mean resource is web links that have useful info regard to storage settings.
0
 
pshattuckCommented:
"Our mpxio-disable="yes"; however our load-balance="round-robin"; How do you explain this? Based yours, it looks like contradictary"

It is :) .. and I have no explanation.

"Also how will I be certain that both cards are used."

do a luxadm -e display_map of each fiber card

Do they show a WWN? if so then they are talking to the Hitachi, mpxio will make sure the cards are being used. If you are doing the redundancy routine with the cards then set

mpxio-disable=no

load-balance=round-robin

If you want to test, the here's how.

.....Mount a small test LUN with no critical data on it

.....Confirm the sytem see the mount.

..... mv a large non-critical file over to the new mount (something that will take a few minutes)

..... while the mv is happening, remove one of the fiber cavles from one of the fiber cards (simulating a card/cab/eport malfunction)

..... see if the mv completes, if so then you have confirmed that both cards are working


Is it a must that I download the latest patch

Personally I have seen stability and performance gains with every new patch SUN has come out for their SAN application, so yes I do recommend it



Also what I mean resource is web links that have useful info regard to storage settings

What exaclty are you looking  for?
0
 
ebi168Author Commented:
do a luxadm -e display_map of each fiber card

--so what is the display_map of a fiber card

What exaclty are you looking  for?

-- for example, what you are telling me here, is there doc/article about it.

0
 
pshattuckCommented:
-e    Expert mode. This option is not  recommended  for  the  novice user.

I was wrong on display_map ..  it's dump_map (display is an upper luxadm syntax)

.....Display WWN data for  a  target  device  or  host  bus adapter  on the specified fibre channel port. If there
           are no target devices on the specified port, an  error is   returned.

So do a luxadm -e dump_map



" for example, what you are telling me here, is there doc/article about it"

Nope, none that I ever found (and I asked my SUN Tech). I knew enough about luxadm and HITACHI SANs to muddle my way through. Once I got my servers connected, I had pretty much memorized each step. I should write it down some day.
0
 
ebi168Author Commented:
I'll look into it.
0
 
ebi168Author Commented:
Hi,

I did a luxadm -e dump_map FCloop and get:

Pos AL_PA ID Hard_Addr Port WWN         Node WWN         Type
0     1   7d    0      21000003ba0b42e1 20000003ba0b42e1 0x1f (Unknown Type,Host Bus Adapt
er)
1     ef  0     ef     21000004cf837202 20000004cf837202 0x0  (Disk device)
2     e8  1     e8     21000004cf8371cd 20000004cf8371cd 0x0  (Disk device)
3     e4  2     e4     21000004cf837189 20000004cf837189 0x0  (Disk device)
4     dc  6     dc     508002000016cfd9 508002000016cfd8 0xd  (SES device)
5     e2  3     e2     21000004cf8371f3 20000004cf8371f3 0x0  (Disk device)
6     e1  4     e1     21000004cf836965 20000004cf836965 0x0  (Disk device)
7     e0  5     e0     2100000c50ed5b15 2000000c50ed5b15 0x0  (Disk device)
8     d9  8     d9     21000004cf781bf0 20000004cf781bf0 0x0  (Disk device)
9     d6  9     d6     2100000c50ed5ff1 2000000c50ed5ff1 0x0  (Disk device)
10    d5  a     d5     21000004cf781b63 20000004cf781b63 0x0  (Disk device)
11    d4  b     d4     21000004cf781652 20000004cf781652 0x0  (Disk device)
12    d3  c     d3     21000004cf781994 20000004cf781994 0x0  (Disk device)
13    d2  d     d2     21000004cf781a87 20000004cf781a87 0x0  (Disk device)

We have 12 disks on sun machine, and the other two, Unknown Type,Host Bus Adapter, SES device do you what they mean. Two adapter cards is used for redundency, so it will show up only once here, right?

Thanks
0
 
pshattuckCommented:
0     1   7d    0      21000003ba0b42e1 20000003ba0b42e1 0x1f (Unknown Type,Host Bus Adapter)

That's the port on the Hitachi


4     dc  6     dc     508002000016cfd9 508002000016cfd8 0xd  (SES device)

That's the port on your JNI Fiber Card.

The rest are the Hitachi LUNs


If you are configured for redundancy, then perform a format and you should see the 12 drives (listed once) along with whatever drives exist on the server
0
 
ebi168Author Commented:
I am still not sure:

1. how will the 12 disks on sun associated with hitachi storage. It looks like they are mixed together, but in what way?
2. since there is only one line for hitachi port/JNI port, does it mean we only have one HBA card used?
3. I am not sure if we are configurated for redundency, I only know we have two adapter cards. How can I check?
0
 
ebi168Author Commented:
Format shows these two additional items, do you know what does each mean. Also what is the difference between a hitachi port and JIN port?

 12. c8t24d0 <HITACHI-DF600F-0000 cyl 3001 alt 2 hd 50 sec 768>
     /pci@9,700000/JNI,FCR@3/sd@18,0
 13. c9t24d0 <HITACHI-DF600F-0000 cyl 3001 alt 2 hd 50 sec 768>
     /pci@9,600000/JNI,FCR@2/sd@18,0

Thanks
0
 
ebi168Author Commented:
yuzh: thanks for the links
0
 
pshattuckCommented:
1. how will the 12 disks on sun associated with hitachi storage. It looks like they are mixed together, but in what way?

It's just a list of the LUNs .. they are not associated with each other . other then they are on ports that you can access on the Hitachi


2.since there is only one line for hitachi port/JNI port, does it mean we only have one HBA card used?

From what you have posted you have only dump_map'd one card .. not the other.


3. I am not sure if we are configurated for redundency, I only know we have two adapter cards. How can I check?

As I posted above .

.....Mount a small test LUN with no critical data on it

.....Confirm the sytem see the mount.

..... mv a large non-critical file over to the new mount (something that will take a few minutes)

..... while the mv is happening, remove one of the fiber cavles from one of the fiber cards (simulating a card/cab/eport malfunction)

..... see if the mv completes, if so then you have confirmed that both cards are working


Format shows these two additional items, do you know what does each mean.

So Drives 0-11 are all internal or cennected to the server by something other then the Hitachi?


 Also what is the difference between a hitachi port and JIN port?

That one is physically located on a JNI card on your server and the other is physically located on the Hitachi

0
 
ebi168Author Commented:
1. how will the 12 disks on sun associated with hitachi storage. It looks like they are mixed together, but in what way?

It's just a list of the LUNs .. they are not associated with each other . other then they are on ports that you can access on the Hitachi

-- is it a list of LUNs or disks? we have 12 disks on the server, to me these are for those disks. Why you say they are LUNs. If they are not associated, maybe veritas put them together in volumns.

2. Format shows these two additional items, do you know what does each mean.

So Drives 0-11 are all internal or cennected to the server by something other then the Hitachi?

--0-11 are the 12 disks on the sun server itself, I believe.

AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf837202,0
       1. c1t1d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf8371cd,0
       2. c1t2d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf837189,0
       3. c1t3d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf8371f3,0
       4. c1t4d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf836965,0
       5. c1t5d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100000c50ed5b15,0
       6. c1t8d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf781bf0,0
       7. c1t9d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100000c50ed5ff1,0
       8. c1t10d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf781b63,0
       9. c1t11d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf781652,0
      10. c1t12d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf781994,0
      11. c1t13d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf781a87,0

0
 
ebi168Author Commented:
.....Mount a small test LUN with no critical data on it

.....Confirm the sytem see the mount.

..... mv a large non-critical file over to the new mount (something that will take a few minutes)

..... while the mv is happening, remove one of the fiber cavles from one of the fiber cards (simulating a card/cab/eport malfunction)

..... see if the mv completes, if so then you have confirmed that both cards are working

--not sure what you mean? If we only have one line shown by luxadm -e dump_map, and you said this means only one card used, do we need to test for redundency.
0
 
pshattuckCommented:
-- is it a list of LUNs or disks?

It's disks .. now that you have shown me some output , I am seeing a better picture of your setup. It was hard to not make assumptions when I don't have an output to look at, it sounded like you were connecting 12 LUNs from the Hitachi.

The 12 are the disks that the format command is stating are on your server..  

not sure what you mean? If we only have one line shown by luxadm -e dump_map, and you said this means only one card used, do we need to test for redundency.

What is the output to

luxadm probe -p ?


0
 
ebi168Author Commented:
Ok, I eventually got a good picture:

0-11 are those disks on the sun system (SENA enclosure)
12-13 are for Hitachi, 13 is the redundency of 12.

I just heard it is not necessary do SAN stuff. Let's wait and see. I will post our final solution when it is done.
0
 
pshattuckCommented:
You need the SAN software for SUN to properly talk to the Hitachi,

Tasks like creating LUNs, changing the emulation of the LUNs, port access and configuration, and running appz on the Hitachi like ShadowImage can only be done on the Hitachi


The only thing you do from the server is to mount, umount the LUNs provided to you and use something like Veritas or even better QFS to grow or shrink the filesystems or setup a RAID evironment with the LUNs that are combined as a filesystem on your server.

0
 
ebi168Author Commented:
Is it for first time setup that we use SAN software? Once everything is setup and Sun is already talking to Hitachi, wouldn't sun and veritas command be enough?
0
 
ebi168Author Commented:
Thanks for your help even there is no final solution.
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 15
  • 10
Tackle projects and never again get stuck behind a technical roadblock.
Join Now