Question

ISCSI SAN configuration questions

Asked by: SashcoIT

OK, this should be an easy one, but pretty important that I get a very good explanation!

We recently ordered an EMC Clariion AX4 with an extra DAE loaded with all the disks we need.  Because we are working on a budget, we could not afford Fiber Channel and have therefore opted to go with ISCSI.  I am finding that ISCSI seems to be a little out of EMC's comfort level and I am getting answers that are conflicting with other answers...

We bought this SAN expecting to virtualize ALL of our network and move all backups, several databases, file storeage and eventually all of our VM VHD files over to the SAN.  We would be consolidating about 15 physical servers into about 3 physical servers running 12+ VM's.  This pretty much means that ALL of our current network traffic is going to be consolidated between the 3 physical machines and the SAN.  Based on this, to keep from having any bottlenecks, we wanted to make sure that we had a good network connection between these devices.  We worked with EMC who assured us that we didnt need to go into a Clariion CX4 with 6 network ports, as an AX4 would provide plenty of bandwidth with the 4, 1GB network connections it had.  We then purchased a new very expensive server with 10, 1GB ports so that we could then bind/team some of these ports and dedicate traffic to some of our more recourse intensive applications/VMs.  

Now the time has come, and we have been working with an EMC install tech to get this setup.  First of all, we are using Hyper-V (as we already had MS Enterprise licensing, so our licensing is essentially free) and it seems that EMC doesnt really know a thing outside of VMWare, even though they claim to.  Also we have found that to meet our budget they have left out PowerPath VE (VE= Virtual Environment) which means that we have no load balancing abilities on this SAN.  This is very much an issue, as we fully disclosed to them what we expected to connect to this SAN, Hyper-V being a major part of this.  Also, it is now showing that our tech, nor anyone I can now get ahold of, seems to know a thing about ISCSI.

Here is where the real problem is coming in:  

I am told that the 4 NIC's on the back of the SAN are setup so that there are 2 ports going to SPA and 2 going to SPB.  Also, these ports are only setup to be redundant of eachother for each SP, and each SP is really only setup to be redundant of eachother... this means that essentially at any point, we will only actually have a 1GB connection into this SAN.  Now maybe I am underestimating a 1GB connection, however I have doubts that we can run 3+ databases, all of our file storeage, backups, VMM library and eventually VM's over a 1GB connection.

The reason I am here is to find out 2 things;  1) does what they are telling me sound correct? -I have my doubts, seeing how they told me differently during the sell, and that they dont seem too knowledgable on anything but Fiber, and 2) if this is the case, is this common to all SAN solutions?  -We essentially thought of the 4 ports on the SAN as being teamed similar to an HP team or Linux Bind, which would allow us to offer up a 4 x 1GB team, giving us a 4GB connection.... Does any SAN work this way?

Thanks for any help you can provide!

This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.

Subscribe now for full access to Experts Exchange and get

Instant Access to this Solution

  • Plus...
  • 30 Day FREE access, no risk, no obligation
  • Collaborate with the world's top tech experts
  • Unlimited access to our exclusive solution database
  • Never be left without tech help again

Subscribe Now

Asked On
2009-10-20 at 08:03:01ID24827149
Tags

EMC Clariion

,

SAN

Topics

Storage Technology

,

Hard Drives & Storage

,

Computer Servers

Participating Experts
1
Points
500
Comments
6

Trusted by hundreds of thousands everyday for fast, accurate and reliable tech support.

  • "The time we save is the biggest benefit of Experts Exchange to Warner Bros. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange." Mike Kapnisakis, Warner Bros.
  • "Our team likes having a resource that is more secure than just using Google and most experts using this service really know their stuff. It's nice to look here first versus using Google." Dayna Sellner, Lockheed Martin
  • "Anytime that I've been stumped with a problem, 9 out of 10 times Experts Exchange has either the accepted solution or an open discussion of the potential solution to the problem." Kenny Red, eBay Inc.

See what Experts Exchange can do for you.

Got a question?

We've got the answer.

Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.

Screenshot of Experts Exchange Knowledgebase

Need individual assistance?

Our experts are ready to help.

If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.

Screenshot of Experts Exchange Knowledgebase

Want to learn from the best?

Read articles from industry experts.

Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.

Screenshot of an Article

Working on a long term project?

Store your work and research.

Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.

Screenshot of Experts Exchange Knowledgebase

Access the answers to your technology questions today.

Subscribe Now

30-day free trial. Register in 60 seconds.

What Makes Experts Exchange Unique?

Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.

Trusted by the world's most respected brands.

image of each brand's logo

Faithfully serving IT professionals since 1996.

Experts Exchange Logo

Try it out and discover for yourself.

Subscribe Now

30-day free trial. Register in 60 seconds.

Related Solutions

  1. iSCSI on VM?
    I have a Dell iSCSI SAN that is hosted by a physical server. The physical server is having hardware problems and I was thinking of replacing it with a ESX VM (hosted on another SAN - NOT the iSCSI SAN). Is this possible? In other words, the VM will host the iSCSI SAN with ...
  2. VMware iSCSI SAN and File Sharing
    Hello, I am in the process of putting together a proposal for VMWare Infrastructure using an ISCSI SAN for storage. The SAN will consist of AIO1200r SAS (NAS/iSCSI) or Dell MD3000i (iSCSI) or similar product. My problem is this I (will) have multiple Windows VM's runni...
  3. ISCSI or FIber
    Ok, We are going to be virtualizing our entire system here over the next year and we obviously need a SAN. I engaged Dell (equalogic) and IBM (DS4700) to provide me with solutions. Now i have worked the vendors over for about 3 months and have them practically giving me the...
  4. iSCSI and VMware
    I have a host configured with a vSwitch for iSCSI traffic. A VM guest is set up with RDM access to a volume on an iSCSI SAN. The connectiviy to the iSCSI SAN is available and functioning. However, connectivity from our Enterprise storage solution fails. From the VM we can pin...
  5. iScsi san reccomendation for vmware
    I'm helping a small company that runs vmware ESX3.5 on an Equallogic SAN. The SAN is 2 TB and has 10 vms running on it. The company is growing and needs at least another 3TB over the next few years. Any reccomendations for an economical iscsi SAN that works with vshpere4 and ...
  6. How to expand a VM disk that is on ISCSI SAN
    I have a VMWARE cluster which uses ISCSI SAN for DATASTORES and I have a VM MS2003 which I need to add disk space to . The VM was Thick provisioned . What is the bast wat to do this? Also if it was thin provisioned what would be the best way for axpansion?

Free Tech Articles

  1. WARNING: 5 Reasons why you should NEVER fix a computer for free.
    It is in our nature to love the puzzle. We are obsessed. The lot of us. We love puzzles. We love the challenge. We thrive on finding the answer. We hate disarray. It bothers us deep in our soul. W...
  2. SCCM OSD Basic troubleshooting
    SCCM 2007 OSD is a fantastic way to deploy operating systems, however, like most things SCCM issues can sometimes be difficult to resolve due to the sheer volume of logs to sift through and the dispe...
  3. Migrate Small Business Server 2003 to Exchange 2010 and Windows 2008 R2
    This guide is intended to provide step by step instructions on how to migrate from Small Business Server 2003 to Windows 2008 R2 with Exchange 2010. For this migration to work you will need the fo...
  4. Create a Win7 Gadget
    This article shows you how to create a simple "Gadget" -- a sort of mini-application supported by Windows 7 and Vista. Gadgets can be dropped anywhere on the desktop to provide instant information, ...
  5. Outlook continually prompting for username and password
    There have been a lot of questions recently regarding Outlook prompting for a username and password whilst using Exchange 2007. There are a few reasons why this would happen and I will try to cover t...
  6. Backup Exchange 2010 Information Store using Windows Backup
    There seems to be quite a lot of confusion around the ability to backup Exchange 2010 using the built in Windows Backup feature. This stems from the omission of this feature prior to Exchange 2007 s...

Cloud Class Webinars

  1. Avoiding Bugs in Microsoft Access
    Alison Balter takes and in-depth look at avoiding bugs in Access. In this webinar you will learn about using the immediate window to debug your applications, invoking the debugger, using breakpoints to troubleshoot, stepping through code, setting the next statement to execute, ...
  2. Top 10 Best New Features in Visio 2010
    Scott Helmers gives live demonstrations of the top 10 new features in Visio 2010. This webinar will teach you how to create compelling diagrams by adding shapes to the page with a single click, linking the shapes in a diagram to data in Excel (or SQL Server, or SharePoint), ...
  3. IT Consultant Business Secrets Revealed
    Michael Munger, Experts Exchange tech pro and IT consultant, pulls back the curtain on his very successful businesses and answers question on every IT consultant and business owner should know about. He shares secrets on what he did to solve the 5 most common problems in IT, ...
  4. Disaster Recovery and Business Continuity
    Quest CTO, Mike Billon, gives an overview of the steps involved in building a dunamic disaster recovery plan. Through case studies and an examination of software/hardware tooles for monitoring and testing, you'll gain a better understandin of where you are, where you want ...
  5. Organize Your Visio Diagrams with Containers and Lists
    Scott Helmers uses cross functional flowcharts, wireframe diagrams, data graphic legends and seating charts to teach you: how to ustilize all three new structured diagram components in Visio 2010, the best practices for organizeing shapes in previous version of Visio, how to organize ...
  6. How to Us Objects, Properties, Events and Methods in Microsoft Access
    Alison Dalter gives an in-depbth look at objects, properties, events and methods in Microsoft Access. In this webinar you will learn about using the object browser, referring to objects, working with properties and methods, working with object variables, understanding the ...

Join the Community

Give a Little. Get a Lot.

Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.

Join the Community

Answers

 

by: mastooPosted on 2009-10-20 at 09:55:12ID: 25616078

I can't pretend to be an expert in this area, but I recently lived through this same experience: moving a smallish server room to an ESX San environment.  San throughput (oh and cost and ease of use) was my main concern.  I only briefly looked at the AX4, but as I recall it was as you describe: active/passive with 4 x 1 Gbs.  I focused more on the CX4.  Everybody gave me different answers when it came to a single virtual machine  getting throughput combined over nics, which is what our database servers need.  This won't help you but I ended up switching at the last minute to EqualLogic because the sales guy was pretty helpful and with two boxes you have 8 active (and 8 passive) x 1 Gbs.  My main requirement was 150 MB/sec on a single database connection.  Combining throughputs wasn't as smooth as I'd hoped, but we're able to get the 150 so I'm happy at this point.  Without the combined throughputs I got the 80 MB/sec you'd expect on a single cable.

Keep in mind you might care more about iops rather than sustained bandwidth, depending on what you do.  You assessed your current i/o activity?  All those spindles supports gobs of iops, and the iScsi is less of an issue.

 

by: SashcoITPosted on 2009-10-20 at 10:08:30ID: 25616217

Great information Mastoo.  I do not have a good assesment of our i/o, but I did send over some perfmon results for EMC, at which point they never gave us back the kind of exact metrics that we were hoping to get... I called them up a few weeks later and simply got a "you will be fine" answer.  

I assume that when you are speaking of the active/passive you are referring to the procs and the passive essentially being a backup... is this correct?  (I am very much a beginner in the SAN world)

So, it is sounding as though I will only ever have a single cable connection at any point, therefore my question is this; assuming that we stick with this setup and dont look at returning it, and assuming that we detirmine we can live on 80MB throughput, it really doesnt make since to team 4, 1GB NIC's on the server side to the switch... do you agree?  I might as well break the teams and offer up more single cable connections to the different hosts....

Thanks!

 

by: mastooPosted on 2009-10-20 at 10:38:53ID: 25616528

I was a complete newbie just 3 months back - the usual baptism by fire.  Yes, my active/passive was referring to a controller for redundancy.

Since we're doing ESX and you're Hyper-V the teaming thing might be different.  In ESX we set up multipath i/o on the host actually at the iScsi level, so I'm not sure we're talking the same thing.  Are you actually referring to teaming or bonding at the nic level?  I don't recall any EMC people mentioning that as an option when I was starting to sweat the bandwidth limitations.  We set the mpio to use a round robin as a means of distributing the i/o across all of the wires.  The possible benefits are: 1) A single virtual machine connection can get throughput spread across multiple wires; 2) The vm's rotate through the paths so they kind of are load balanced and less likely to step on each other; 3) Redundancy.  I think for your question #1 doesn't apply.  #2 and/or #3 might still justify some kind of pooling of the connections if that is how you accomplish that in Hyper-V.

We just plunged in with ESX so I never even looked at Hyper-V

 

by: SashcoITPosted on 2009-10-20 at 11:25:32ID: 25616970

I do think that we are running somewhat of a different configuration which may be causing a little confusion as well.  See in our environment, while we could present the SAN Luns to the VM's via the Hyper-V host, we have chosen to instead install ISCSI initiators on the VM's themselves, as this allows us to easily move our VM's from one host to another...  That said, the Hyper-V host that is running the hypervisor is not essentially aware of the SAN even.

When I refer to the teaming/binding, yes I mean at the NIC level... the original assumptions was to take say 2 ports and bind them to get a 2GB connection over to our swtich that goes into the SAN, and then present this teamed NIC to a VM as a dedicated connection to the SAN.  This was how we hoped to ensure that we did not have ANY extra network traffic interfering with our main SQL database.  

So in this situation, we would have our main SQL box with a normal network connection to the rest of our network and then would have a team of 4 NICs acting as 1 (giving it a 4GB connection) over to the SAN.... This team of 4 would not be presented to any other VM, therefore there would be nothing for the hypervisor to load balance.  This same setup would translate over to a few other machines.

I guess there are several ways to skin this, but the main thing that I am finding in reference to my original question is that it sounds like it can be done on some models but not the AX4.... I have a conf call in a bit with someone at EMC and Ill update with any other info.

 

by: mastooPosted on 2009-10-20 at 12:46:57ID: 25617765

Ah yes, that was one of the options somebody gave me - present different LUNs to each VM and each VM has its iScsi initiator.  FWIW, I was told no nic teaming with the AX4, which steered us towards FC on the CX4.  You might also clarify whether the VM's iScsi initiator might support simultaneous mpio as that would accomplish the same thing.

 

by: SashcoITPosted on 2009-10-26 at 06:14:52ID: 31643416

Thanks for your help.  

In the end, we have revisited our perfmon results that we gave to EMC, and the ISCSI as is will work just fine:

With the EMC, there is SPA and SPB, each having 2 NICs connecting into them.  The 2nd NIC is nothing more than failover unless you have their PowerPath VE software which enables a type of loadbalancing.... Funny how they mentioned loadbalancing throughout the whole sales process but forgot to mention that in a Virtual Enviroment (VE) you need to spend another $12k to get that feature.   Ultimately, the 2 procs each own their own Luns, therefore you technically have 2, 1GB (80MB throughput) channels to the SAN at any point, unless 1 of the procs goes down.  This fact, combined with revisiting our IO, looks like we will be just fine with this setup.

Thanks for following this one, and offering your help!

20120131-EE-VQP-002

3 Ways to Join

30-Day Free Trial

The Experts

98% positive feedback on 31,087 answers since March 2000. angeliii is a Microsoft Most Valuable Professional for his work with MS SQL Server & Develoment.

He has also proven his knowledge of Visual Basic Programming, PHP Scripting and Oracle Databases.

The Experts

97% positive feedback on 10,752 answers since July 2000. lrmoore has more than 18 years experience in the networking industry.

The six-time Mircosoft MVPs specialties include firewalls, virtual private networking, and network management.

Testimonials

"...and excellent source for support... Kind of like having your very own IT dept." Electriciansnet

Testimonials

"I was apprehensive at signing up at first. However... it has already made my life as an IT administrator much easier." JaCrews

Testimonials

"WOW! You guys have great, active, and knowledgeable people on here." moore50

Business Clients

Business Clients

In the Press

"If you’ve got a question... Experts Exchange can supply an answer.”

In the Press

"...an invaluable aid for both IT professionals and those who require tech support."

In the Press

"where IT professionals provide quick answers on just about any topic"

Business Account Plans

Loading Advertisement...