VMware vSphere Essentials Plus Kit with vSphere Storage Appliance Recommend Hardware and Setup -- Need Advice?

Hi There,

I am building a vmware solution for my customer and need some advice to make sure I have covered all my bases and have choosen the right hardware / Software for the solution.

We are looking at running:
Vmware VMware vSphere Essentials Plus Kit with vSphere Storage Appliance.
We will have 3 hosts with 2 processors present in each server, and will vSphere Storage Appliance accross the servers.

With the following Hardware for each host:
PowerEdge R710 E5620 2.4GHz NO RAM NO HDD H700 RAID DVD-RW RPD - 3yr Pro NBD      
With Additional Processor - Intel Xeon E5620 Processor
4GB Dual Rank LV RDIMM 1333MHz - Kit  x 12 - Total 48 GB's of Memory per server
600GB 6GBps SAS 15k 3.5" HD Hot Plug Fully Assembled - Kit      X 6 per server
Intel Gigabit ET Dual Port Server Adapter PCIe x4 - Kit       X 2 per server
Warranty Uplift from 3year NBD to 5yr Next Business Day - R710      1
We will have three of these servers for the configuration, and will run VSA accross the servers each with 6 x 600GB Drives.

For connecting the Vmware hosts to the LAN- We will have a HP HP 2910G - 24 port 10/100/1000 switch with 4 x GBICS, which is linking to each patch room in the building via Fibre.
We will possibly add another HP 2910G - 24 port 10/100/1000 switch, for the vSphere Storage appliance -- Not sure if this is required?


We will be running the following virtual servers on Software Option 1 (Linux PDC and Zimbra Setup):
Main Linux Mail /  Zimbra Server -- Red Hat Enterprise Linux with Zimbra Professional (250 Mail Users) -- 1.5TB Hard drive Space 24GB's Memory Allocated.
Centos Linux Server -- Samba File Server Running Zimbra 120 users --  1.2TB Hard Drive Space, 16GB Memory
Centos Linux Server -- Hyla Fax Server --  30GB Hard drive space 4GB Memory (4 fax lines)
Group Buildsmart Database Server -- Windows Server 2008 - SQL Server 2008 R2 (40 concurrent Users) -- 300GB Hard drive space, (Current Database is 14GB's), 16GB memory.
Group Buildsmart Webserver (Connects to SQL Server) and TS Server -- Windows Server 2008 R2 with TS CALS (15 TS Users, and 30 Web Server Users)      -- 120GB Hard Drive Space, 16GB memory
Document Server, running Mircofile Software for Document scanning an storage -- Windows Server 2003 -- SQL Server 2005 -- 500GB Hard Drive Space, 12GB Memory
Access Control Machine -- Windows XP -- 50GB Hard drive space, 4GB Memory
Antivirus Server - Windows 2003 or XP -- 50GB Hard Drive space, 4GB Memory
Windows Server 2003 Running Blackberry Enterprise Server (+-100 Users over the air sync) -- 150GB Hard drive space, 8GB Memory


Total Hard drive usage : 3.9TB
Total Memory allocated : 104GB

We will be running the following virtual servers on Software Option 2 (Microsoft 2008 PDC and Excahnge 2010 Server):


Main Microsoft Exchange 2010 Server -- Microsoft Server 2008 R2 (BDC) with Exchange 2010 (250 Mail Users) -- 1.5TB Hard drive Space 24GB's Memory Allocated.
Microsoft 2008 R2 File Server -- Primary Domain Contoller, AD and File Server --  1.2TB Hard Drive Space, 16GB Memory
Centos Linux Server -- Hyla Fax Server --  30GB Hard drive space 4GB Memory (4 fax lines)
Group Buildsmart Database Server -- Windows Server 2008 - SQL Server 2008 R2 (40 concurrent Users) -- 300GB Hard drive space, (Current Database is 14GB's), 16GB memory.
Group Buildsmart Webserver (Connects to SQL Server) and TS Server -- Windows Server 2008 R2 with TS CALS (15 TS Users, and 30 Web Server Users)      -- 120GB Hard Drive Space, 16GB memory
Document Server, running Mircofile Software for Document scanning an storage -- Windows Server 2003 -- SQL Server 2005 -- 500GB Hard Drive Space, 12GB Memory
Access Control Machine -- Windows XP -- 50GB Hard drive space, 4GB Memory
Antivirus Server - Windows 2003 or XP -- 50GB Hard Drive space, 4GB Memory
Windows Server 2003 Running Blackberry Enterprise Server (+-100 Users over the air sync) -- 150GB Hard drive space, 8GB Memory
Centos Linux Running ScopServ / Asterisk VOIP switchboard appoximately 130 Users, with Voice recording, VOicemail and IVR with Music on hold -- 150GBs hard drive space, 16GB of Memory.  -- Using a Quintum Bri Gate to terminate to PSTN / Telecom network, as well as some SIP channels with second operator over Fibre.

Total Hard drive usage : 3.9TB
Total Memory allocated : 104GB


Just some background I am new to vmware, and we have only setup vmware ESXI 5.0 the free licence, we have done a few basic Microsoft Hyper-V server setups.
Currently we are running a linux enviroment at this site, Red hat Enterprise 5 and the File and Mail server are combined. We are using POP3 running Dovecot, with a Ldap / Samba setup.
We are running a WAN with 2 other sites replicating the ldap database and if we were to switch to Microsoft we would then change these two sites to Microsoft AD as well.
These are my questions, but any comments or idea's to assist would be appreciated:

1a: From what I understand I will lose half of my space running RAID 10 with VSA. -- Total 10.8TB, 5.4TB Usable space ---- Is this correct?
1b: If I am incorrect, what will my available storage space be and is it fine to run one very large RAID 10 array (5.4TB), or should I devide it up?
1c: Should I run the ESXi 5.0 host OS from a USB stick in the server, or should I just install it on the array?
2a: Will this hardware handle the servers which I have listed, or do you think that it would be better to remove some off the vmware infrastructure?
2b: Is there any issues running a VOIP Switchboard / Asterisk server on this sort of infrastructure?
3a: Do I have enough memory and Hard drives to allow me to loose on host and still be able to keep the servers running?
3b: Estimating if only two servers are online because of some sort of failure their full memory capacity is 48GB x 2, 96GB of memory I would be short will or can I configure it to shrink a servers memory allocation if we loose a host?
4a: Would we require a seperate switch for the VSA or is it fine to run it on the one HP 2910G - 24 port 10/100/1000 switch with 4 x GBICS?
4b: Do you think that this switch (HP 2910G - 24 port 10/100/1000 switch with 4 x GBICS) is ok for what we are trying to do?
4c: I was going to setup 3 seperate vLan's on the network, 1 for the VSA connectivity, 1 for the VOIP, 1 for all data traffic for the entire network.
4d: Will 6 nic's per server be enough for my Guests and VSA? (2 oboard and 2 x 2 Dual Intel as per server hardware above)
5: On the Microsoft software option, can I run a single Exchange server or should I run more and split up the roles? (Total 200-250 users, only 120 users onsite/local. The remaining users are site users and will connect in intermittantly via 3G)
6: Please any other information or advise on right or wrong hardware would be good, I am pitching this to my customer on Wednesday the 7th of Decemebr 2011, so will hopefully finalize costings tomorrow.


Thanks,
Spencer
StarrateITAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

StarrateITAuthor Commented:
I am now looking into a SAN using ISCSI rather then using VSA.

Vmware were saying that I will not get the required storage with my configuation using VSA.
So looking at removing all disks out of the servers and then having the ISCSI setup to a Dell SAN. Will post what product I am looking.
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I'm glad you've completed some research, and maybe changed your mind with vSphere VSA.

VSA has some major scalability limitations. For example, the product doesn’t scale beyond two or three nodes, whichever a user starts with, according to VMware’s VSA documentation, and it supports a maximum of eight disks per VSA cluster.

Also VMware Admins must determine the storage capacity of the VSA cluster prior to installation. VSA 1.0 does not support adding storage to data stores and their replicas after you configure the cluster. Plus, this capacity will be limited by a 75% storage overhead requirement for RAID data protection. Thus, a VSA consisting of eight 2 TBs would have a raw capacity of 16 TB, but the 75% redundancy overhead would result in a maximum usable capacity of 4 TB.

But there are alternatives, HP LeftHand, VSA for years, as well as StorMagic, StarWind, and Datacore, offer Virtual SAN solutions.

1a. Yes, RAID 10 used in vSphere VSA - VMware documentation cites high availability as the reason behind VSA’s capacity limitations: “The VSA cluster requires RAID10 virtual disks created from the physical disks, and the vSphere Storage Appliance uses RAID1 to maintain the VSA datastores’ replicas,” resulting in effective capacity of just 25% of the total physical hard disk capacity.

1b. You would have 25% of the total physical storage. This could be further devided into smaller LUNs. I would recommend between 500 - 800 LUNs, being presented to ESXi, unless you require much larger LUNs.

1c. Yes, it should.

Here is the VMware KB on installing 5.0 on USB/SD:
http://kb.vmware.com/kb/2004784

It then removes it from your disks, and then there is no requirement for local disks in the servers.

2a.The hardware specification seems appropriate.

2c. VOIP - Test it, there have been some issues with response and networking on the hosts, VMware recommend avoiding VOIP based applications on their hardware platform.

3a. No requirement for hard drives on the servers, spend money on the SAN.

3b. Should be okay.

4a. You would need two switches for a SAN for resilience for iSCSI.

4b. Yes, we use these edge switches.

4c. Yes, ESXi supports VLANs.

4d. Yes, 6 NICs is adequate.

5. Single Exchange server would be fin.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
StarrateITAuthor Commented:
Thanks Hanccoka..

I am looking at using the Dell Power Vault MD3200i External iSCSI.
Are Near line SAS drives (7.2k RPM) fast enough for my entire enviroment or do I need to mix the array up with 15k SAS drives in a seperate array and only use the nearline for file storage and email database storage (Excahnge MDB)?

The Dell Power Vault comes with 2 x 300GB 15k SAS drives which I guess I could use for holding anything that needs higher disk speeds in a RAID 1 configuration, SQL database Server not sure what else would require it.

Was thinking to populate the iSCSI device with 6 x 2TB drives. ( 400-19343      2TB Near Line SAS 6Gbps 7.2k 3.5in HD Hot Plug).
Do you think 6 x 6Gps 7.2k drives in a RAID 10 array would run fast enough to hold my VMs?
Setup all my VM's virtual disks on the single (12TB -- 6TB usable space) RAID 10 Arrage?

I would then need to look at a NAS device to hold backups of the VM's. I am setting up the backup system with Veeam.
0
The 7 Worst Nightmares of a Sysadmin

Fear not! To defend your business’ IT systems we’re going to shine a light on the seven most sinister terrors that haunt sysadmins. That way you can be sure there’s nothing in your stack waiting to go bump in the night.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
RAID 1 performance is poor. RAID 10 iks what you should be aiming for, which has good Read and Write performance.

Veeam Backup and Replication is a good choice for backup to a NAS.

Personally, I would not use 7.2k drives, and we would use SAS 15k drives in RAID 10. I would purchase at least an additional 4 or 6 SAS 15k drives.
0
StarrateITAuthor Commented:
I would not be able to reach my required data store size with one PowerVault MD3200i External iSCSI RAID 2x300GB 15k SAS - 12 Bays Array with Dual Controllers (4 Ports per Controller) in RAID 10.

If I put in an extra controller it will give me the full 12 bays and then add in 12 x 600GB SAS 15k Drives (Removing the drives it comes populated with, I would only reach 7.2TB of space and then 3.6TB's of usable space in a RAID 10 array?

I am reluctant to run this in a RAID 5 configuartion because if I lose two disks I will loose everything?
Eish, it is hard to get to my required storage size at a reasonable value.

Any suggestions of where to go from here :(
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
How much storage do you need?

Exchange, SQL, VOIP all require good performance. For performance you need 10k or 15k SAS drives.
0
StarrateITAuthor Commented:
Need to have around 4TB's as a minimum and possibly some room to grow.
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
You could look at other VSA alternatives, as before just not vSphere VSA, if there is a budget constraint.

or HP P2000.
0
kevinhsiehCommented:
That amount of RAM that you are assigning to your VMs is way too high. For example, your Exchange server would do very well with 12 GB, not 24. For your DC/file server, 2 GB would be okay, 4 GB would be great, but 16 GB RAM is way overboard. You access control VM is probably good with 512 MB to 1 GB RAM.

Why are you looking at 3 hosts? You could run everything on 1, so you only need a second host for failover. Change your DIMMs from 4 to 8 GB DIMMs. The cost is about the same per GB, and you can put a lot more RAM in each server. You could easily load 96 GB RAM into each host. In fact, I think that my last R710 had 144 GB of RAM using 18 x 8 GB DIMMs. I know that VMware has stupid RAM licensing limits in vSphere 5, but that's another problem...

Do not run Exchange on a domain controller. Setup AD on an additional or different VM. If you go option 2 with Microsoft servers, buy Windows Datacenter processor licenses for each host. It will cover unlimited number of Windows server VMs running on the hardware.

Asterisk may or may not run properly virtualized. I have heard that it can have issues under load, and with things like conference calling. My memory is a few years old, so things might have changed.

Exchange 2010 isn't IOPS intensive, and VoIP should barely tough the disk, but I agree that 6 7.2K drives isn't enough. How about using 10K drives? Can you hit your capacity requirements using those? I know that there are 900 GB 2.5" 10K SAS drives.
0
StarrateITAuthor Commented:
I met with my customer and have been working on this proposal, below is where I am headed now, thank you so far for the advice. I need as much as I can get so I can spec this right from the start..

Hardware:
PowerEdge R710 E5620 2.4GHz NO RAM NO HDD H700 RAID DVD-RW RPD - 3yr Pro NBD      
With Additional Processor - Intel Xeon E5620 Processor
8GB Dual Rank LV RDIMM 1333MHz - Kit  x 6 - Total 48 GB's of Memory per server
Intel Gigabit ET Dual Port Server Adapter PCIe x4 - Kit       X 3 per server
Dell ESXi USB Module built in (No disks)
Warranty Uplift from 3year 4 hours response time -- Mission Critical

Dell Main iSCSI Storage
PowerVault MD3200i External iSCSI RAID 2x300GB 15k SAS - 12 Bays Array with Dual Controllers (4 Ports per Controller)
600GB 6GBps SAS 15k 3.5" HD Hot Plug Fully Assembled - Kit x 12 RAID 10 -- +-3.5TB usable space
Warranty Uplift from 3year NBD to 3yr 4 hour response - Mission Critical - PowerVault

Dell Local backup Storage
PowerVault NX3100 Optimal Base 1 x Intel E5620 (2.4GHz), 12GB Memory
No Second Processor 1 S
146GB SAS 6Gbps 10k 2.5" Internal HD Cabled x 2 to Run OS
2TB Near Line SAS 6Gbps 7.2k 3.5" HD Hot Plug x 8 Raid 10 -- 8TB’s usable space
PERC H700 Integrated RAID Controller, 512MB Cache, For 12x HDD Chassis
Integrated Config, English WSS 2008 R2 X64 Standard Edition
Warranty Uplift from 3year NBD to 3yr 4 hour response - Mission Critical

Infrastructure Software
VMware vSphere 5 Essentials Plus Kit (Limited 3 hosts, limited to 2 processors and 192GB’s memory)
V-VESTP/S-AP000-00-E  - Veeam Essentials Plus bundle for VMware Tier A (6 Processor Licence, Back up and Replication, Monitor, Reporter, Business View as well as FastSCP)

Network and UPS
APC Smart-UPS 10 KVA 3 phase In / Out
2 x HP 2910G - 24 port 10/100/1000 switch with 4 x GBICS
All other Switches linked in via fibre and each having an APC 1200VA UPS

•      One single Microsoft Exchange 2010 standard Server  --  Microsoft Server 2008 R2 with Exchange 2010 (250 Mail Users will be growing to 500-600 users) – 1.2TB Hard drive Space 16GB's Memory Allocated. +- 3.5-4GB per mailbox allowance.
•      Microsoft 2008 R2 Primary Domain Contoller, AD and DNS server
•      Microsoft 2008 R2 File Server – 1TB Hard Drive Space, 16GB Memory
•      Centos Linux Server -- Hyla Fax Server --  30GB Hard drive space 2GB Memory (4 fax lines)
•      Group Buildsmart Database Server -- Windows Server 2008 - SQL Server 2008 R2 (45 concurrent webUsers) -- 50GB OS VDisk + 80GB Database, (Current Database is 14GB's, Max 40GB’s), 16GB memory.
•      Group Buildsmart Webserver (Connects to SQL Server) and TS Server -- Windows Server 2008 R2 with TS CALS (15 TS Users, and 40 Web Server Users)      -- 120GB Hard Drive Space, 16GB memory
•      Document Server, running Mircofile Software for Document scanning an storage -- Windows Server 2003 -- SQL Server 2005 -- 500GB Hard Drive Space, 12GB Memory
•      Access Control Machine -- Windows XP -- 50GB Hard drive space, 2GB Memory
•      Antivirus Server - Windows 2003 or XP -- 50GB Hard Drive space, 2GB Memory
•      Windows Server 2003 Running Blackberry Enterprise Server (+-300 Users over the air sync) -- 150GB Hard drive space, 8GB Memory


I met with the customer about the solution and they want to structure it as a managed service deal where I include the hardware on a rental basis. I will finance / lease the hardware from my bank and then sell this on to the customer on a 3 year rental contract with a managed service for support added in. The critical thing here is up time, the customer wants a guaranteed 99% uptime with penalties in place where I do not meet the SLA.
This is the first time I am doing a contract like this and want to make sure I have 110% covered all my bases and need to make sure that this solution is scalable to 600 users over the 3 years. I would need to buy an additional storage device which I will build in on a per user basis, over and above 300 users. I will not be able to spring a large hardware bill on the customer so I need to make sure before we start that the system can be upgraded using the more of the same hardware to scale up.

My idea here is as they grow I will add in an additional PowerVault MD3200i External iSCSI with more 15K SAS drives and ideally by then these disks will be much bigger making it a little easier. I would imagine this would only be needed by the end of next year. I would also think that I do not need more hosts all the way up to the 600 users, these three hosts should be enough I will just add more memory where needed up to the max 192GB’s.

1.      Does what I am putting forward sound like a workable solution where I can achieve my required SLA requirements and will support the required expansion?
2.      Would the scaling be as stated above where I could just add a second iSCSI device to this configuration to allow the required expansion for storage?
3.      Would I need an additional host, if they do not require any more services just more users (600 Users max) or could I live with my three hosts just more memory?
4.      Should I look at a redundant UPS for the server room?
5.      Do I need to have more than one exchange server to handle the initial load (250-350 users)?
a.      Do you see any potential problems with this exchange configuration, single server?
b.      Do I need to look at DAG to make sure I will keep to my SLA?
c.      How should I split up my databases?
i.      I was thinking of having 70 users per mail store allowing 350 users on this first standard server?
d.      Once I reach 350 users I would guess it would be best practice to now bring up another Exchange standard server just to manage the next 200 odd users?
e.      Or should I start with two standard exchange servers HT+CAS and then Mailbox on second?
i.      Then add another two standard servers when they grow above 350 users?
6.      Will I get the required IOPS out of my chosen storage device running one large 12 disk 15k RAID 10 array though a Gigabit adapter?
7.      How easily would I be able to backup all the vm’s to my Dell NAS using Veeam twice a day over the LAN, would I get the required through put to do this and doing it during the day (Midday live backup) would my users suffer, I am sure the night time one would not be an issue?
8.      My biggest issue now is finding a simple cost effective offsite backup, in South Africa we battle with internet connectivity but we should be putting in a 10-15mbit fibre connection (1:10 contention ratio) but then I still need to find a location to backup too and there is a fair use policy on this connection where we are rate limited after 2.25TB’s of traffic per month.
a.      I would be backing up approximately 2TB’s at the get go which would be growing, is this possible / feasible to backup over a 10mbit/s connection?
b.      I will be speaking to the ISP supplying the service to see if I can utilize their Data centre to store backups and if they will not include this usage in my Fair use policy?
i.      I would imagine that with a 15mbit connection this should be doable?
c.      Should I just forget offsite over a WAN and look at using 5 x 4TB USB 3 (Seagate) backup disks which are rotated on a daily basis?
i.      Would I manage to do a veeam backup at midday and then every night to the NAS (Dell NX3100) and then also backup over night to the USB 4TB drive as well, which is rotated (Monday-Friday)?
d.      Any other advise would be appreciated, please comment on anything else you think I have forgotten?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I think you need to start a new Related Question, posted to Exchange, VMware and Storage which would give your Question, more visibility to the Experts.
0
StarrateITAuthor Commented:
okay thanks, will do this now.
0
StarrateITAuthor Commented:
Opening a new question to continue as this is a complicated question to get to a final answer.
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Make sure, you post in VMware, Storage and Exchange, to get ALL Experts Views.
0
StarrateITAuthor Commented:
Already posted it, to vmware, Server software, server hardware. I see you cant change it once it is posted.

Should close and repost it?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Request Assistance on the Question, and Ask the Moderators to add for you.
0
StarrateITAuthor Commented:
How to I request assistance on a question, looked around and can't find it anywhere?
0
StarrateITAuthor Commented:
Think I might have found it in contact us, chatting to someone now?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
it's on the Question Description.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Virtualization

From novice to tech pro — start learning today.