ESX 3.5 network delays

we have an ESX 3.5/update 5 server with 1Gbit network and SCSI U320 hdds.

when copying files between vitrual machines and a neighbour servers (they are in the same 1Gb network segment) we encounter a speed limit of ~100Mb instead of 1G.

- The ifconfig on ESX host shows 1G connected, and the same speed is on VM's network card
- no collisions detected on a hosted network card
- no traffic shaper is applied to a VM's virtual switch
- network card is a "VMXNET"
- no problems was found with IRQ sharing
- VM's HDD is a "RAW-type"
- no limits on shared resources
- disk shares is set to "HIGH"
Y0sh1Asked:
Who is Participating?
 
Y0sh1Connect With a Mentor Author Commented:
It was FC HBA's that is causing delays. The problem was solved by Updating the HBA's BIOS to the latest version.
0
 
za_mkhCommented:
When you say a speed limit of 100mb ...  do you mean it is transfering at 100 Megabytes a second or a 100 Megabits a second?
How do you know there is speed limitation?
Apologies in advance if these are stupid questions ...
0
 
bbnp2006Commented:
vmware tools installed?
0
Cloud Class® Course: Microsoft Azure 2017

Azure has a changed a lot since it was originally introduce by adding new services and features. Do you know everything you need to about Azure? This course will teach you about the Azure App Service, monitoring and application insights, DevOps, and Team Services.

 
Y0sh1Author Commented:
The speed is about 7-9 Megabytes per\sec which is near 100Mbits/sec networking

vmware tools installed and is up to date
0
 
bbnp2006Commented:
what OS is your VM running?
0
 
Y0sh1Author Commented:
windows server 2003 x64
0
 
bbnp2006Commented:
VMXNET 2 (enhanced) or VMXNET 3?
0
 
Y0sh1Author Commented:
Enhanced vmxnet. Also tried on E1000 - no differences where found in delays. So i thing with vmxnet 3 there will be no diffs also.
0
 
bbnp2006Commented:
all our windows 2003 server VMs are using VMXNET 3 type vNIC and it works perfectly.
0
 
Y0sh1Author Commented:
when i added a vNIC there where only two choices of E1000 and Enhanced vmxnet.  No VMXNET3 where suggested. To add VMXNET3 i followed these steps:
1.Upgraded VMware tools (from VMWare infrastructure console i'v choosed upgrade tools by right-click on VM)
2. Shutdown VM
3. Tryed to find an 'upgrade hardware' by right clicking with no success.

The version of VM is 4, while VMXNET3 wants in to be at least  7.
The version of Vmware infrastructure client i use is  2.5.0 build  62276.  Seems like i need to upgrade the client to create a version 7 VMs
0
 
bbnp2006Commented:
Sounds about right :) VMXNET 3 is available for VMs on hardware ver7
0
 
Y0sh1Author Commented:
well, i'v upgraded client to 2.5.0 build 204931, but it still creates version 4 VMs, no upgrade hardware is available by right click.
0
 
vmwarun - ArunCommented:
Set the speed 1000 Mbps and duplex to Full manually instead of Auto config on both the ESX Host and physical switch port and check the transfer speeds.
0
 
Y0sh1Author Commented:
setting the speed 1000 Mbps and duplex to Full manually not helped
0
 
vmwarun - ArunCommented:
I suggest that you check your network cables which interconnect ESX Hosts and physical switches. (CAT 5 would provide 100 Mbps while CAT 5E or CAT 6 would provide 1000 Mbps)
0
 
Y0sh1Author Commented:
I'll check the cable tomorrow, but if we'd have a CAT 5,  the link status wouldnt be a 1000G/Full ,right?
0
 
Paul SolovyovskySenior IT AdvisorCommented:
Also check to see if there are switches in the between the two segments that are 100mbps
0
 
Y0sh1Author Commented:
well, just checked - cable is CAT 5E and there is no 100Mb switches in between.
0
 
Y0sh1Author Commented:
have to add that when networking files between VMs (hosted on the same ESX host) the speed is about 20Mbytes/sec
0
 
Paul SolovyovskySenior IT AdvisorCommented:
have you tried to put a physical system on the same source and destination switch, perhaps a port setting or uplink?  How many nics do you have on the vswitch and how is it setup?
0
 
Y0sh1Author Commented:
ESX host and other servers are on the same 1G switch with no shapers  applied on its ports. The link status on both ends is 1G/Full

There is only one NIC (Broadcom NetXtreme II BCM5708) on the vswitch. with 1G/Full (manual)

There is another NIC (BCM5708) that is dedicated to a service console.
0
 
vmwarun - ArunCommented:
When you transfer files between two virtual machines that are a part of the same port group in a vSwitch, then the traffic does not go out and is routed within the vSwitch so transfer speed would be higher.
Try changing the cable to a different CAT5E or CAT 6 and check.
0
 
Y0sh1Author Commented:
have changed the cable to a different CAT5 E - not helped.
0
 
kugaconsultantsCommented:
When you say copying files from VMs to other servers, you're talking about copying to physical servers?  How about copying files from physical servers to VMs?  Do you still have the same slow copy performance?

I'm thinking that the write cache on the server's HDDs or controllers you're copying to is not sufficient to handle higher transfers through the network.  Also, you may want to check the processor on both servers copying because remember TCP/IP is not offloaded to the NIC unless you have TOE enabled or an HBA.  Your server's processor may be running near capacity which will be the reason why trasnfers speeds are not optimal.
0
 
Y0sh1Author Commented:
I am talking about copying files between VMs and other physical servers that are locating on the same physical 1Gbit switch.
When copying files from VM to Physical server the speed is 31Megabytes/sec
When copying files from Physical server to VM the speed is about 5Megabytes/sec

No matter where files are copying to - whether it be a local ESX host storage or a NAS disk attached via lun.
CPU utilization is at 17 percents at maximum

i'll check the write buffers later
0
 
Y0sh1Author Commented:
i have to notice (again) that copying files between VMs have no any delays and the speed is 30Megs/sec. So i can conclude that nothing is wrong with the storage write cache. It is something else...
0
 
Y0sh1Author Commented:
when copying files via FTP from external server into COS's partition, the delays also persist. I think it has to be something with tuning of the  host physical networking. possible iptables or other stuff...maybe some unnoticed default ESX network shaper...dont know yet
0
 
Paul SolovyovskySenior IT AdvisorCommented:
What is the setting on the vswitch that is going out to your production network, how are you doing it? MAC, IP Hash, etc...?  Are you running trunking on your vswitch?
0
 
Y0sh1Author Commented:
On vswitch i have only one physical NIC.  NIC teaming is configured by default and the settings are:
Load Balancing: Route based on the originating virtual port ID
Network failover detection: Link status only
Notify switches: yes
Failback: Yes

Traffic shaping is disabled
Security is set to Accept all

Configured Speed, Duplex: 1000Mb, Full Duplex

On the "virtual machine network" properties the VLAN ID is set to "none", so no trunkin is configured
0
 
Vijay kumar MohanrajCloud ArchitechCommented:
If your VM is only reporting 100Mbps network speed, that is because you are using the VLANCE VMware NIC instead of the VMXNET NIC. Don't worry, it's a false indicator, your virtual NIC is still transmitting packets as fast as possible based on the PNIC or system bus speed, but you should upgrade to the VMXNET NIC wherever and whenever possible as it is a more efficient and slightly faster NIC than the slower-more-compatible VLANCE which is a virtualized old AMD PCNET Ethernet standby.
0
 
Y0sh1Author Commented:
(referring to my 1st post)  all NICs are "VMXNET"
0
 
vmwarun - ArunCommented:
Its nice to see that your problem got resolved.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.