How to setup LDOM redundancy for multiple Guest Domains connected to SAN

I have a T5240 server with 2 fiber channel cards.  This server has two PCI buses, so I am looking into setting up two I/O domains -- one for each PCI bus.  I am confused on how to setup the FC cards and multipathing to support multiple guest domains.  If each I/O domain controls one FC card then I'm assuming multipathing can't be setup at the I/O domain level but I'm not sure it can be setup at the guest domain level either.  In addition, if there are multiple guest domains then I'm assuming I would need NPIV to be able to use the same FC cards for all the guest domains.  So how would this work?  Without NPIV would I need a separate set of FC cards for each guest domain?  
macklinjAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

RowleyCommented:
In this case, your secondary i/o domain will also become a service domain once you create your vdiskserver service and share vdisk devices to other hosts. You multi-pathing software within the guest that is using resources from these services domains (your primary + i/o domain) will work out which devices are available.

Because only the service domains need to have control over your hba's - remember, you're creating vdiskserver services in each i/o domain - then any guests using these services only need to ever worry about the disk devices presented to them.

Any clearer?
0
macklinjAuthor Commented:
Hi Rowley,

It's getting clearer.  I am working on setting this up so I may have an additional question in a couple of days.
0
macklinjAuthor Commented:
The Sun Blue Print, "LDOMs I/O Best Practices", states "If the server’s two PCI buses are allocated to each of two I/O domains, however, two host bus adapters can be configured to provide access to external storage through two PCI Express slots via two I/O domains. Redundancy and resiliency can then be established in the guest domain with software mirroring such as that provided by Solaris Volume Manager software."  

Hence, multipathing cannot be done in the Guest domain.  I'm guessing when NPIV support is added to Solaris you will be able to do multipathing in the Guest domain.
0
macklinjAuthor Commented:
From Octave Orgeron's "An Introduction to Logical Domains":

SAN storage has many benefits, including performance, reliability, and
portability among servers. Solaris 10 and above include support for SAN
connectivity. Although third-party drivers and FC HBAs may work on standalone
servers, they may not work with LDoms. This is also true with SAN
multi-pathing software. STMS (a.k.a. MPXIO) should be utilized. STMS can
only be utilized in a service domain that has direct access to the FC HBAs,
such as the primary domain. The SAN infrastructure and multi-pathing are
completely transparent to guest domains. This situation will change in the
future when FC HBAs can be virtualized using NPIV, which provides guest
domains with a virtualized FC HBA instance [3]. Here is an example of virtualizing
SAN storage.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
RowleyCommented:
Thanks for sharing, useful to know next time I'm asked to multipath within a guest.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.