Need speed improvement between ESXi hosts?

I'm running into some serious performance issues with a couple of VMware hosts I have in my datacenter and it's actually due to the physical hardware not quite being up to par. My question is is it better to buy a new beast of a server to combine my 2 hosts into a single ultra fast solution or should I just get 4 decent servers and try to connect them with fiber or 10GbE?

If I'm reading correctly I'm thinking I should migrate these VM's to a single new host so the connectivity between the VM's will be faster? Or, do they actually just talk with 1GB connections when they're on the same host? 10GbE stuff is incredibly expensive so I'm hoping that a single ESXi host will greatly improve performance so I only have to buy the 1 machine and not all this other stuff.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

How are the two hosts currently networked?  How many VMs per host?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Lets have some more information, host specifications, CPU, Memory, Datastore, Shared Storage Fibre Channel, iSCSI, Networking, ESXi versions, as jimmyray7 has stated how many VMs.

how have you configured the virtual switches?
andyalderSaggar maker's framemakerCommented:
If you put all your VMs on a single server then network communication between them through the virtual switch is really fast, see the note on page 4 of :-

Note: The speed and duplex settings found in physical networking are not relevant in the virtual network, because all the data transfer takes place in the host system’s RAM, nearly instantaneously and without the possibility of collisions or other signaling-related errors.

Tht doesn't mean you can transfer a file from one VM to another at RAM speed of course because that file has to be read from one VMDK file and written to another and those files are both on relatively slow disks.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Active Protection takes the fight to cryptojacking

While there were several headline-grabbing ransomware attacks during in 2017, another big threat started appearing at the same time that didn’t get the same coverage – illicit cryptomining.

BigDeerAuthor Commented:
Currently, there are 4 servers on 2 hosts.

Server A hosts:
a single SQL server that holds a pretty heavy database

Server B hosts:
a heavily used terminal server that runs the software that talks to the database server on host A.
a document scan database server with some software called laser fische (the terminal server has the front end software for this as well)
a time card server that is light and isn't having trouble

Both server A and B are PowerEdge 1950 III's with dual quad X5460's and 16GB RAM. The main issue comes from the terminal server hanging up when trying to communicate with the scan server and the queries to pull the images taking forever. Both the scan server and the terminal server produce alerts saying the CPU is running at over 75% for more than 20 minutes at a time on a regular basis. The owners also complain about the query speed to the database server on the other host.

Both hosts are networked with standard gigabit...I've never looked into this internal switching since I'm new to ESXi. Thanks for the sites and the PDF. I'll look these over and see if that makes any difference in the speed between the local host machines and I'll definitely use this if we go with a single box for all 4 virtuals.

Based on the machines that need to run on this single ESXi server do you think it could run on a single R710 or R510 loaded with 15k RPM drives and say 24GB RAM?
Did you enable VT in the BIOS of each host? That will improve performance. How many vCPU have you assigned to the SQL and terminal server. If they are CPU bound you should add additional vCPUs to them. How much RAM does your SQL server have? I would look at increasing it. Since your 1950s are currently maxed on RAM, you can look at upgrading them with higher capacity DIMMs or upgrade to a new server. I would be surprised if the physical CPUs are maxed; the bottleneck is probably somewhere else.

If you do want to replace a host, I would go with more than 24 GB RAM which is less than the 32 you currently have. Look at 48 or more, which allows you to give a lot more RAM to your 64 bit guests. If you are running 32 bit guests I would look at upgrading them after looking at vCPU and such.
BigDeerAuthor Commented:
Sorry about the delay on this fellas...slammed at work.

I didn't set these boxes up and I'll have to make a trip to the datacenter to see if VT is enabled. The terminal server has 4 and the database server does too. Each of these has 8GB RAM allocated and I'm going to bump them up to 10 for the terminal server and 12 for the database server. I haven't seen the CPU spikes since I added an extra GB of RAM to another database server that's for a different APP so I think it's mostly RAM related at this point.

Good point on the amount of RAM in the single server...the servers are all 64-bit.

Thanks for the articles...I discovered I can use the VMXNET3 adapters and run 10GB between the virtual machines on the same host which will be a big boost.

I'll report back after the changes. Thanks.
BigDeerAuthor Commented:
Many thanks guys...these are 3.5 boxes and I changed the NICs to VMEXNET2 which surprisingly helped a good deal. I allocated as much RAM and CPU as possible to each of the machines that needed it most. Now, we no longer get the CPU usage warnings and the machines are running much better. It's still not as power as they want, but it bought us some needed time, so we're going to upgrade to a new single box with much more power in the near future.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.