studios
asked on
Windows Server 2003 seems to be limiting bandwidth
I recently acquired a HP ProLiant DL-380 G3 server and moved my Windows SBS 2003 server into this hardware. Today, while copying large files from the server to the workstation, I'm seeing that something is limiting or throttling the amount of bandwidth between the server and the workstations. I can copy a 400,000 mb file from my laptop to my desktop workstation through the same switch, same subnet, same everything, in about 45 seconds. It took about 12 minutes copying the same file from the server to either the desktop or the laptop. When I look at the network traffic with a sniffer-type application (Network Instruments Observer) I see a definite limiting as the traffic curve looks very much like a table-top at the 1k/s value.
Since the network is capable of transferring the files far faster, and the PCs can transfer files between themselves, I'm left thinking the server is the source of the throttling. There must be something on the HP ProLiant DL380 G3 server, it's installed drivers, management software, or other HP server management that is limiting the transfer rate.
I have checked - QoS is not installed in the network properties. I have looked through the network connection properties and found nothing that would seem to be about limiting bandwidth or throttling a connection.
It is limited in both directions. When I started another copy from the server, the amount of traffic traveling over the network increases, so there is capacity. But the speed of sending the data is still pretty slow. I'm using the 1gb LAN connection on the DL380 server and it says its connected at 100. I'm using an old 3COM 10/100 hub/switch that lets me sniff the network.
Since the network is capable of transferring the files far faster, and the PCs can transfer files between themselves, I'm left thinking the server is the source of the throttling. There must be something on the HP ProLiant DL380 G3 server, it's installed drivers, management software, or other HP server management that is limiting the transfer rate.
I have checked - QoS is not installed in the network properties. I have looked through the network connection properties and found nothing that would seem to be about limiting bandwidth or throttling a connection.
It is limited in both directions. When I started another copy from the server, the amount of traffic traveling over the network increases, so there is capacity. But the speed of sending the data is still pretty slow. I'm using the 1gb LAN connection on the DL380 server and it says its connected at 100. I'm using an old 3COM 10/100 hub/switch that lets me sniff the network.
This really does sound like a port duplex mismatching issue.
- What is the brand of your networks switch to which your server is connected? DO you have more you have more than one switch?
- Is your switch managed or unmanaged?
- If it is unmanaged, make sure all your devices are connected and set to auto for speed and duplex
- If your switch is managed, I would force the network switch ports to be 100 or 1000MB full duplex
ASKER
I didn't say 400gb, I said 400 mb, as in about 2/3 of a CD-ROM.
I agree with all you say about 10/100 hubs, etc. except that the same hub, same network, same other cards are able to transfer that 400mb file from one machine to the other through the same hub in about 1/20th the time. That actually makes the collision light on the hub blink. The transfers that are taking so long are not even making the collision light blink.
My sniffer trace shows very few packet resends.
I had put all the latest and greatest HP stuff on the server about a month ago. This is the first time I've been trying to move larger files to and from the shares and actually paying attention.
I agree with all you say about 10/100 hubs, etc. except that the same hub, same network, same other cards are able to transfer that 400mb file from one machine to the other through the same hub in about 1/20th the time. That actually makes the collision light on the hub blink. The transfers that are taking so long are not even making the collision light blink.
My sniffer trace shows very few packet resends.
I had put all the latest and greatest HP stuff on the server about a month ago. This is the first time I've been trying to move larger files to and from the shares and actually paying attention.
what is the model of your network switch?
Is this SBS server a clean install of Windows 2003 ?
My initial thought would be speed/duplex issue, but your use of the HUB should force switch and server into a consistent state.
Make sure the port is set to auto-negotiate speed and duplex on both the switch and the server. If they disagree, or your network cable is bad, autonegotiate on one side is your worst nightmare.
Duplex and speed on both sides of the connection need to agree, they need to be either hard-set to 1000/full on both sides, or auto/auto on both sides, or auto (if switch is unmanaged).
I'm concerned about the possibility of a problem with your Windows install, it could be some sort of AV scanner or other software on the server causing an issue.
Recommendation would normally be to try a boot CD i.e. an ultimate boot cd or Knoppix Linux boot disk, during a maintenance window
Something you can boot the SBS server from to a different OS and perform some raw download tests of a big file from a web server on a XP machine or other 2003 server that doesn't have issues, to help rule out a NIC issue.
My initial thought would be speed/duplex issue, but your use of the HUB should force switch and server into a consistent state.
Make sure the port is set to auto-negotiate speed and duplex on both the switch and the server. If they disagree, or your network cable is bad, autonegotiate on one side is your worst nightmare.
Duplex and speed on both sides of the connection need to agree, they need to be either hard-set to 1000/full on both sides, or auto/auto on both sides, or auto (if switch is unmanaged).
I'm concerned about the possibility of a problem with your Windows install, it could be some sort of AV scanner or other software on the server causing an issue.
Recommendation would normally be to try a boot CD i.e. an ultimate boot cd or Knoppix Linux boot disk, during a maintenance window
Something you can boot the SBS server from to a different OS and perform some raw download tests of a big file from a web server on a XP machine or other 2003 server that doesn't have issues, to help rule out a NIC issue.
ASKER
3COM OfficeConnect DualSpeed Hub 16
This is a link to a page that shows the product, but then the FAQ seems to be about a different unit. The picture looks just like it, though.
http://www.3com.com/products/en_US/detail.jsp?tab=support&pathtype=support&sku=3C16751B-US
The fact that it was a hub makes network sniffing much easier, and this is a small network.
I have plugged things into a NetGear FS526T which has two gig ports on it and I'm getting ready to see if that changes things.
This is a link to a page that shows the product, but then the FAQ seems to be about a different unit. The picture looks just like it, though.
http://www.3com.com/products/en_US/detail.jsp?tab=support&pathtype=support&sku=3C16751B-US
The fact that it was a hub makes network sniffing much easier, and this is a small network.
I have plugged things into a NetGear FS526T which has two gig ports on it and I'm getting ready to see if that changes things.
ASKER
It does look like there is a problem between my desktop PC and the server, and not between a Thinkpad laptop and the same server. I'm transferring a 44gb file from the server to the laptop with both plugged into the gig ports on the Netgear FS526T. It looks like its going to take about 3 hours for the 44gb. This is a transfer that was going to take 221 hours between the server and the desktop when I first started.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Today I put in a 64-bit Intel NIC ( Intel Pro/1000-MT) and changed nothing else. I get much better transfer speeds. My sample 3 GB file ( 3,072,703,488 bytes) copied from the server to my laptop in 3 minutes, 26 seconds. According to the bandwidth calculators I find online, such as http://www.ibeast.com/content/tools/band-calc.asp this is about 140Mbps transfer speed.
According to the bandwidth calculator at http://www.numion.com/Calculators/Time.html this file "should" copy in 26 seconds if there were no overhead. 36 seconds with 50% overhead.
In my first post, the file that took 12 minutes to copy (which is actual 354,673,152 bytes) copies from the server to the laptop in 53 seconds. (Both the laptop and the server say they're connected at 1gb). Interestingly, it takes just 38 seconds to copy it to my desktop, which is only connected at 100 Mbps. My desktop has better performance disk drives, though.
According to the bandwidth calculator at http://www.numion.com/Calculators/Time.html this file "should" copy in 26 seconds if there were no overhead. 36 seconds with 50% overhead.
In my first post, the file that took 12 minutes to copy (which is actual 354,673,152 bytes) copies from the server to the laptop in 53 seconds. (Both the laptop and the server say they're connected at 1gb). Interestingly, it takes just 38 seconds to copy it to my desktop, which is only connected at 100 Mbps. My desktop has better performance disk drives, though.
There has to be some issue with the hub. Upgrade to a swithc and there are more advantages and less collisions. Also, there is no way you can get 1000Mb on that hub.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks for your help. I still don't know why the HP cards seem to be limiting themselves, but dropping the Intel card "solves" the problem and its time to move on!
If you were only getting 1 kilobyte a second, it would take a lot lonegr than 12 minutes to transfer a 400 gigabyte file.
What exactly is the speed you're getting?
Getting only 100 megabits to the server, when you should be linking up at a gigabit severely limits your ability to test correctly, you should be using a managed gigabit switch with port mirror or SPAN for throughput testing at this scale.
If you plug that server into a 10/100 hub, and you're transferring amounts of data that massive, your hub itself is going to severely restrict the transfer.
The maximum you can transfer while plugged into a 10/100 hub is 100 megabits half-duplex (50 megabits bidirectional Ethernet), which comes to less than 30 megabit after IP overhead.
Check your normal link-up speed on the server.
Use the HP Firmware maintenance CD, latest version downloaded from HP website and burned for your specific model of server
To check for updates to NIC firmware.
HP Servers do have an optional throttling feature, but it should not be enabled by default and requires a license to enable, it's called HPVT (HP Virus Throttle), which is designed to slow network activity in response to a virus infection.
It's more likely you have a problem with your cable, or link-up between server and switch..
I suggest also checking your switch for errors, link-speed, etc. on the interface attached to your server's main NIC.