Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 981
  • Last Modified:

VMWare NIC setup

I have a Dell R610 server running VMWare 4.1 with all internal storage.  I am also running Lab Manager which allows me to create and setup systems for training purposes which I move from location to location for training.  The server has 2 x 4GB NICs and I am using a Dell PowerConnect 6224 switch.  

I am not doing any time of VMotion.  I have 3 virtual machines running (DC, VCenter and LabManager) and then each attendee gets 4 virtual machines to work with (2 Servers/ 2 Clients).  Performance seems to be a bit slow and I just want to make sure I have the best possible setup.  Each class has a total of 16 attendee's, so I can use all 8 ports to connect from the server to the switch if needed.

What would be the optimal setup on the ESX side within Networking as well as on the switch?  I thought that ESX would automatically load balance across all NICs plugged into the switch, but I just want to be sure.  The ESX Node, DC, VCenter and LabManager boxes are all running on a 172.16.x.y network and the DC is giving out IP Addresses via DHCP to the 16 users laptops when they connect.  The LabManager systems are running in fenced mode and are configured, but they cannot communicate with the other systems or the host laptops for security.  Each client connects to LabManager via Web Browser only.

Thanks,
0
itg_admin
Asked:
itg_admin
  • 10
  • 5
  • 2
  • +2
1 Solution
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
what is slow?

ESX does load balance if setup correctly on ESX host and network switch.

But where is the performance bottleneck?

have yo checked network utilization, if you suspect this is slow, using perfomance charts on the host server.

if you wanted more bandwidth you could trunk two or more ports, and bond or etherchannel them to increase throughput bandwidth, if you are suffering network performance issues.
0
 
itg_adminAuthor Commented:
The ESX server and all the clients laptops connect to the same switch.  There is nothing else running and there are no other network devices.  I'm asking for assistance with the ESX networking setup and what it should look like.  Should I have VCenter and all virtual machines running over all 8 NICs and setup a Channel Group on the Dell switch?  
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
connecting 8 nics seems a bit over the top, two or four nics connected from the vSwitch to the physical switch should give you enough resources. Ive never seen virtual machines on a single host swamp the network connections of 4 nics.
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
IanThCommented:
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
ive been thinking about your configuration.

how about two nics dedicated for service console, bonded or in a channel group load balanced for failover.

four nics bonded, channel group

leaving two spare.

i dont know if you have and storage requirements like nfs or iscsi.
0
 
eeRootCommented:
Verify everything is connecting at full duplex and the proper speed.  Then see if link aggregation is being used.  If not, you have to manually set it up on both the server and switch
0
 
itg_adminAuthor Commented:
@hanccocka - during a typical class I would have as many as 16 users connected to the switch and each user would be running 4 virtual machines each.  As we move on in the course they could use as many as 5 virtual machines each, so a total of 80 virtual machines.  I just want to give as much bandwidth as I could and went with 4.  I did create a channel group on the Dell switch.  I am using internal storage, so no storage like iSCSI is needed for this.

@eeRoot - I can see from the switch that Channel-group 1 is up, but is there a way or a need to check this on the ESX node?  What should the proper setup be on ESX?  I do not need to access the console or VCenter of this channel group, I will physically plug a cable into another NIC to my laptop to connect to the ESX host.  
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
well if you want to give as much bandwidth as possible connect all eight nics to the physical switch, if the physical switch backplane can cope. How do the candidates access the servers, and do there servers interact with each other. Also check the switch doesnt use over subscribed ports.

here is a good article, on how you should setup esx host and switch

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004048

there no real sanity check on esx, other than checking the vswitch iproperties and nics are setup correctly.

you could also divide all nics between the 16 attendees, so a pair of attendees per pair of ports.
0
 
itg_adminAuthor Commented:
hanccocka - On the ESX host I am running a virtual DC, VCenter server and then a LabManager server.  The DC is giving out DHCP and when each client connects their laptop to the switch they should pull and IP from me.  This switch is not plugged into the network at any location, it is private.  Once they are connected they http to the LabManager server and login and are presented with a configuration of machines for each days lesson.  Those virtual machines can only interact with the others in their group, which is a configuration within LabManager called "fenced mode".  This way each attendee has the same server names, client names, domain name and IP addresses and follow the same set of labs.

I'm guessing that this question might be best answered  by someone with experience working with VMWare LabManager.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
we do!
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
can you identify what is slow?
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
e.g. web browser sessions, machine deployment, general VM usage?
0
 
itg_adminAuthor Commented:
1.   Non bandwidth related - How much CPU and RAM did you allocate to your LabManager system?  The time to clone to workspace and then Deploy with Defaults seems long.  And if I dont clone to workspace before the class starts and try to do it when the users login on their own it takes about 45 minutes or so.  Is this normal?  Should I allocate all remaining CPU cores and memory to the LabManager server which is hosting the virtual machines or is LabManager pulling that memory and cpu from the ESX host?

2.   When the users switch to the Pop out Console, it sometimes takes 20 seconds or so to switch between the machines using the Configuration button along the top.  I'm not sure if this is bandwidth related or not.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Our Lab Manager server is of similar specification to our vCenter Servers, VMware View Connection Broker, and SQL Servers templates.

We use Windows 2008 R2 Enterprise, 64 bit, 8GB RAM, and Dual Processors, (host CPUs are 2.6GHz or 3.0GHz).

The deployment time is a function of read and write (IOPS) on your datastores, we used to use FC SAN for storage, but dropped it in favour of local Ultra 320 disks, because it was too slow, but multiple concurrent deployments of Servers for the Lab and Training were taking too long, so we've moved all our rapid deployment systems including Lab Manager and VMware View Desktops to SSDs. (Fusion IO drives), but due to cost implications of these, we have been experimenting with Intel and Kingston consumer SSDs on our IBM Blades.

Lab Manager is installed on a virtual machine in our case, so it uses the resources that have been granted to it in the virtual machine, it just acts as a tool, to interface (via VIM) with vCenter/ESX to administer jobs, and deployments.

When you've got these issues, how many VMs are you running, and what are the VM specs, Host Specifications, we had a situation, where a Server was misconfigured and a stupid amount of memory was over allocated, someone got Megs a Higs mudduled, and allocation 4000Gigs thinking it was Megs to a server, and deployed the all the servers for the class, and it flatlined the server!

Because memory was over comitted, and the server, started swapping to disk, everything went very slow....for a while until we could re-gain control of the server, by shutting down VM servers. After memory stopped swapping, things returned to normal.

Are you deploying your stored templates from the same datastore, as the destination datastore, or if different datastores, they don't exist on the same disk array or LUN, same same of disk/spindles?

VM Consoles, and Access, Web Console access is just plain https over TCP/IP to the Lab Manager server, but this spawns a console access request direct to the ESX server, (TCP 902/903), this is the same function, as using vCenter Console access (TCP 902/903).

Do you experience similar delays if you use a vSphere GUI on the laptop - for test purposes.

20 seconds seems long, but I've never timed it.
0
 
itg_adminAuthor Commented:
The server is a Dell R610 with 2 x 2.8 Nehelam processors and 96GB RAM.  I am using 600GB 15K SAS drives and not sure but I believe its a RAID10.  ESX 4.1 is installed on the box and I have configured the 3 servers as follows:

TrainingDC - 2GB RAM and 1 CPU running Server 2008
VCenter - 2GB RAM and 1 CPU running Server 2008
LabManager - 4GB RAM and 1 CPU running Server 2003

Each user for the current labs gets 2 x Server 2003 system (1GB RAM and 1 CPU) and 2 XP Client systems (1GB RAM and 1 CPU).  I could have as many as 16 attendees in a given class connected.  I have not been told by anyone in the classes that they have latency when working with their given virtual machines, only when switching between machines.  I usually will Clone to Workspace each group of systems for all users before they arrive.  Then I will begin Deploying them as needed about 20 minutes before the start of class.  I would say that when I login as the Administrator and see all configurations, I can right click on each user (default S01 - S16) and Deploy with Defaults and the time before I can click on the next machine is about 10 seconds.  If I click on 5 users to Deploy at a time, it takes about 5 minutes before they are all deployed.  Sounds like what your saying is this is more based off the I/O of disk more so than CPU and RAM.

I guess what I wanted to get an answer to is if LabManager is using only those resources assigned to it to build and run those virtual machines or is it pulling from the ESX host?  I could easily assign another 64GB RAM and 3 more CPUs to the LabManager if that is what needs to be done to optimize the setup.  

When I go into the Networking area of the ESX host, I do see a series of virtual switches created for what I am assuming is each group of virtual machines that have been used.  I dont have enough NICs to go 1 to 1 that way and not sure if that will speed things up since the traffic from the clients to LabManager is https over TCP/IP.  Thoughts?

0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Deployment is disk bound I/O.

Lab Manager doesn't really require much resource, unless handling many current connections from clients. It just handles the request, and creates jobs and passes them to vCenter. vCenter then communicats with the ESX hosts or hosts.

ESX host has very little to do with it, other than host the Virtual Machines, (CPU, Memory, NICs and Storage).

Task Creation accepted and handled by Lab Manager, which is communicated to vCenter, this is very similar to the deployment process by VMware View Connection broker server. Deployment jobs are communication to vCenter server. It is then based on how quick it can deploy templates, snapshots, which is ESX server CPU, Read/Write time on the datastore.

Do you check the overall performance of host, CPU, Memory, DIsk, Networking, when the "Lab" is running? (though vCenter)

Check performance of Lab Manager, is it processor or memory starved? (inside the VM, and from vCenter performance). If you have standard Windows 2003, 4GB is your limit, unless you have Enterprise of 64 bit version.

Check network latency between clients and Lab Manager, ESX servers at load, any request timeouts from pings, or retries.
0
 
vivigattCommented:
You may also want to make sure that all your VMs are using the "best" virtual HW for their NICs.
Usually, flexible or vmxnet is the best. e1000 is to be used if you have Intel Pro 1000/10M NICs in teh physical host.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Waiting upon Author of question to return further information?
0
 
IanThCommented:
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

  • 10
  • 5
  • 2
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now