Link to home
Start Free TrialLog in
Avatar of Pancake_Effect
Pancake_EffectFlag for United States of America

asked on

What is a normal speed for internal transfer of files?

I noticed that we were having internal file transfer speeds of about 20mbps, when I thought since we put Cat6 in we would be receiving gig speed instead.

After looking around a bit I've read that Cat6 doesn't give you gig speed, it allows for multiple clients running at about 20mbps to ALL TOGETHER reach up to a gig of throughput on a line.

Is this true?
Avatar of IanTh
IanTh
Flag of United Kingdom of Great Britain and Northern Ireland image

cat 5e and cat6 can do gig but how many clients are on that cat6 cable

if you has switches see if you can use multiple ports in a group done by lag (logical port group) that will give you multiple gig between switches.
There are many other things to put into consideration eg, CPU speed, memory, hardware capacity

I haven't come across a single host that transmits at gig speed yet (not saying it does not exist). It may exist but that means it is not common yet if it does.

Yes, there are gig nic ports available but still....

Check the NIC specification on your device

Gig ports for now are advantageous only on uplinks (between switches)
yes it is true.

In case of file transfer, the speed limit is usually due to physical hard drive speed.
Avatar of Don Johnston
The cable type is but one factor in the speed equation.  Cat 6 supports UP TO 1gbp/s.

Is your switch capable of 1gbp/s?
Is the switchport that the PC is connected to configured for 1gbp/s?
Is your NIC capable of 1gbp/s?
Is the NIC configured for 1gbp/s?
Mbs or MBs?  One is bits/second, one is bytes/second.

Also, the cabling probably won't be your bottleneck - that would be the hard drives at either end.  

Switches determine network speed, generally, but there a many things that can contribute to something less than maximum speed, including network cards, drivers, environmental conditions, etc.
have you set the network on the clients to 1g
There are servers than can do gigabit speeds, in fact some that can exceed gigabit speeds.

However most desktop computers peak out at about 400 Mbps and most multi-core/cpu Intel based servers will peak at about 800-900 Mbps.

CAT6 will support connections up to 1 Gbps.  However all the networking components between the two end points in the connection must also support 1 Gbps.  This includes any switches/routers and all NIC's.

Typically to get above 400 Mbps you also have to use jumbo frames of at least 4K.

As other have said, typically the disks or disk subsystems can't sustain the transfer rates needed in order to get 1 Gbps.

However, what you heard about maxing out at 20mbps is false.
Avatar of Pancake_Effect

ASKER

So let's say if I have a gig switch, a gig nic on both ends, all using cat6 for transport, what would a typical speed from a typical SBS Server (we have a dell power edge) sending to a another standard dell user machine such as a dell 755 PC?

Because right now I'm only receiving about 20mbps when pulling from the file shares which doesn't make sense. The connection status states 1g on both ends. I guess that was my main concern with, should I be using more link aggregations?

Or do you believe that it's simply the hard-drive write speeds as mentioned?
SOLUTION
Avatar of giltjr
giltjr
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Yes, not knowing more about the whole configuration, I'd be inclined to look at the hard drives at either end.  

This is testable using a network testing product that either checks the connection between the two servers you're talking about, or by using an Internet speed test site, that will tell you connection speeds to some point on the Internet.  

You can also set up Performance Monitor on each server to watch Disk I/O and Network I/O while performing a file transfer.  This should give you a clearer idea of where the problem lies.  If it does indeed turn out to be the network, try changing ports, cables, switches, etc, to see if any improvement is realized.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
all switches do have a cache if the cache is full you will see slowness no matter what settings are in place.
As a data point...

I've got a Dell PowerEdge 1950 with a 2TB SATA drive running Windows Server 2012 in a VMware VM connected to a DGS-1008G/RE switch which then connects to an ASUS M4A88TD-V EVO/USB3 running Windows 7 64-bit.

Using QCheck, I'm seeing 266mbp/s.
Thanks for the tip about Qcheck I used it and was surprised by our horrible speeds! Headquarters configured and installed some new switches and it's gotten a lot slower. Our old switches were slow, but not this slow. This is even after redoing all the wiring to use Cat6. The wireless is least to say dismal.  I'm going over a lot of the configs and honestly nothing stands out, I'll continue looking unless any of you might know. What's odd is why is UDP so slow...that's suppose to be faster.

Results of QCheck:

Client to Server
TCP
New Switches Throughput: 11MB
Old Switches throughput: 90MB


UDP
New Switches Throughput: 6MB
Old Switches throughput: 14MB


From Server to Client
TCP
New Switches Throughput: 10MB
Old Switches throughput: 30MB

UDP
New Switches Throughput: 6MB
Old Switches throughput: 14MB

Wireless Network, (Aruba-24 APs and great signal), but throughput is terrible.

TCP
0.875MB

UDP
1.0MB
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I agree about the bad throughput.  

Are there other devices using the switch while you're testing?  That can have an impact too.
The good news is that you know exactly what device is creating the symptom.

What is the make/model of the old switch and the new switch?

As previously suggested, check the speed and duplex settings.
Awesome tool!

Here are the results:

New Switches:
User generated image

Old Switches:


Being that both of them are saying the slowest connection is a 10mbps link, that's not a duplex setting is it, or is that our Internet speed? So in other words is that something on our side or something between the two points such as a ISP? I'm guessing our network? I can't think of any links on our network that uses only 10mbps duplex except one port we have dedicated for a legacy device.

If it sounds like it's on our side, it sounds like since it's on both the old and new switches it would be a one going to a router or something. I don't know if I should start checking link and make sure it's not running at a duplex setting of 10mbps.

What are your thoughts on this test?

PS: If you want the statistics or extended details let me know thanks.
I thought you were testing between two devices on the same switch?  If you looking at speeds to the internet, that's a whole different ballgame.
I 2nd donjohnston's post.

Can you please give us the details of the complete network path between the two computers in question?
Ha!  Yeah, we thought we were troubleshooting a local network, not an Internet connection...
You guys are correct, just the LAN..  

However using the pgm554's network check, I just posted the results of what it stated. The only reason why I spoke about the Internet is because I didn't know where the "slowest link" result was coming from...my network or somewhere between ISP's? I wasn't quite sure what that testing tool all test. I'm assuming just lan based off the description maybe.
Gotta deal with one thing at a time...

Let's focus on the local issue first.

What's the make/model of the old and new switch?
O.K.  

Can you please verify the speed you are getting.  Is it 20 million bits per second or is it 20 million BYTES per second?

How are you testing?  
Doing a file copy of a single large file over to/from a file share?  
Doing a file copy of a bunch of little files to/from a file share?
Using FTP?
Hopefully this helps, the old switches are just there for the time being, it will be replaced by the new switches. The new switches are just a extension of the old ones basically for the time being until they take over (the topology is better then what it sounds lol.)

No matter what I plug into..the new or old switches directly, it still has bad throughput. As shown above I used QCheck to check the bandwidth. I also checked directly by doing large file transfers from the SBS to my laptop, the speed correlates with what QCheck results are saying I'll post them again below.

User generated image.



Results of QCheck:

Laptop to Server
TCP
New Switches Throughput: 11MB
Old Switches throughput: 90MB


UDP
New Switches Throughput: 6MB
Old Switches throughput: 14MB


From Server to laptop
TCP
New Switches Throughput: 10MB
Old Switches throughput: 30MB

UDP
New Switches Throughput: 6MB
Old Switches throughput: 14MB

Wireless Network, (24 APs and great signal), but throughput is terrible.

TCP
0.875MB

UDP
1.0MB

Again these are all Lan speeds.
So...even the new switches go through the old switches to get to the server?
You really need to post a topology diagram.
Correct they do go through them, it's only a temporary situation for reasoning beyond the scope of this, but granted they will be phased out soon as we complete a second project related to it. But the problem happened to the old switches too before the new switches were even attached to them. (the new switches are very recent) I just figured it was worth mentioning.

So really you can just imagine it just as the left side of the above topology and forget the new switches if it helps troubleshoot the root of the problem.
Sorry... Cross post. :-(
"So really you can just imagine it just as the left side of the above topology and forget the new switches if it helps troubleshoot the root of the problem. "
Well who knows?  Maybe the new switches have no problem at all, but seem slow because the traffic is still going through the old switches.
Well that's a possibility, but there lies the problem...something is up with the old switches regardless. By old switches they're not actually very old..only about 1.5 years old actually so there shouldn't be much issue with them. We just are putting in a new closet, hence why we have the new ones to replace them.

So the big question is, why are the 2960s so slow? When I connect my laptop to it it, the NIC status states 1g speed. However as you can see above from QCheck, the throughput is terrible when transferring files from Server to Client or Client to Client even.
Um,

New Switches Throughput: 11MB
Old Switches throughput: 90MB

Unless I am missing something at one time the only way to get above 30-40 MB second on Gbe was to have jumbo frames enabled and have a frame size of at least 4K.

Can you check to see if you are using jumbo frames on the old switches?

If hosts are configured to try and use jumbo frames, but the switches are not performance will be impacted.
Let me check to see what ours is set at
So based off of donjohnston's findings he's getting about 32MB, I'm getting about 11MB on the new switches...BUT let's take that out of the picture since they are not fully set up yet to be alone, which may increase the speed once they're directly connected to a router...instead trunking over.

The old switches are receiving 90MB throughput based off of QCheck which is good? That was kind of my original question I guess, "what is a good speed?" or normal speed.

But for some reason the file transfers (not QCheck) actual file transfers are only about 20mbps....so maybe it's just crappy client hard-drives is the root of all of this?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
If you were getting 90MB on the old switches and getting 11MB on the new switch and you are using the same computers, then its not the hard drives.  

Again, I would check jumbo frames settings.
The jumbo seems to be set at 1500 which is the default for the MTU size.

Here are the results of the test pgm554:

These are the old switches (the main ones I'm concerned with):
User generated image
New Switches
User generated image
See the excessive packet loss stat on the diag,you need to check that everything is set to auto,not full ,not half,AUTO.
I set up a separate question about light indicators on a different thread located here, I was originally just asking about light indicators..but it's become more involved and maybe this connection is the root of these problems with the packet loss? It really seems like it could be related..

Thread located here:

Second Thread

Edit: The only line I see not set to auto auto is the line out to ISP. It's set to 100 full, but I imagine that was coordinated with the ISP...and that line shouldn't have much to do with lan speeds.
Your internet speed sucks when hooked up to the new switches(you are in the kb range),so if you can set auto up on ISP, do that too.

Also ,if you notice ,my slowest link is a 622 mb OC12,yours is a 10 meg Ethernet.
Something is definitely misconfigured.

Is your IOS up to date?
Cisco is always looking to lock you into an upsell.
Unless you are running the Web100 stuff on your own internal network it will NOT tell you if you are using jumbo frames internally.  Everything on the Internet is 1500.

You need to look at your NIC configuration and your switch configuration.

Since your problem, as you describe it, is between two computers within your building going over switches within your building, can we PLEASE leave anything that deals with the Internet out of the picture?
As I mentioned before,the Stanford NDT gives info that says his slowest link is 10 meg Ethernet,which is another clue that something is misconfigured internally.
Usually that's a half duplex fall back thing.

I manage several sites and all of them use gig and that 10 meg figure tells me there's a bottle neck in his LAN that is reporting that speed.

I'm not sure why his packet size is reported at 1380 bytes though.
Maybe a Cisco thing.

You running vlans?

Check this out for trouble shooting:

http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a0080ba9521.shtml
pgm554, yes it does show that.

However, unless they know some magic I'm not aware of, that is the slowest link between the server running  Stanford NDT and his computer. It can't tell you the slowest link between any two computers within his network.  Example for me it says the slowest link is 100 Mbps.  Which happens to be my link speed to our ISP link.

However internally everything between my computer and all servers is 1 Gbps.
pgm554
I agree, it seems like something in regards to the 10mbps connection somewhere internally that is causing all of the LAN issues.

We do have a lot of vlans here, but it's mostly for the wireless switches.

We have just one main vlan for the client machines that are wired. We don't have many computers here..only about 55'ish are turned on at a given time, and used only for basic task. We do not have VOIP or IP Video.


giltjr
Here is a pic of my NIC jumbo frames. I know you said 1500 is set for the Internet but here's what I'm seeing:



The switch, I guess which command is used to view that? I was just doing a "show int" command and saw this:



But like you said this is just for the the Internet? is there a separate command to see if it's used internally? I thought that was a special command you had to apply to a interface for jumbo packets.
I did a show interface and saw a lot of dropped packets on some of the interfaces. I know it's probably normal depending on the ACL. But here is a list after doing the "show asp drop command"

Frame drop:
  No valid adjacency (no-adjacency)                                          113
  No route to host (no-route)                                                 80
  Flow is denied by configured rule (acl-drop)                           1380798
  Invalid SPI (np-sp-invalid-spi)                                             29
  NAT-T keepalive message (natt-keepalive)                                     1
  First TCP packet not SYN (tcp-not-syn)                                  117020
  Bad TCP flags (bad-tcp-flags)                                               86
  TCP data send after FIN (tcp-data-past-fin)                                 17
  TCP failed 3 way handshake (tcp-3whs-failed)                              3410
  TCP RST/FIN out of order (tcp-rstfin-ooo)                                24971
  TCP SEQ in SYN/SYNACK invalid (tcp-seq-syn-diff)                           501
  TCP SYNACK on established conn (tcp-synack-ooo)                            143
  TCP packet SEQ past window (tcp-seq-past-win)                             3861
  TCP invalid ACK (tcp-invalid-ack)                                           76
  TCP RST/SYN in window (tcp-rst-syn-in-win)                                  53
  TCP packet failed PAWS test (tcp-paws-fail)                               1372
  IPSEC tunnel is down (ipsec-tun-down)                                      240
  Slowpath security checks failed (sp-security-failed)                       695
  Expired flow (flow-expired)                                                185
  ICMP Inspect seq num not matched (inspect-icmp-seq-num-not-matched)          3
  ICMP Error Inspect no existing conn (inspect-icmp-error-no-existing-conn)                                                                                                                           87
  DNS Inspect invalid domain label (inspect-dns-invalid-domain-label)         19
  DNS Inspect id not matched (inspect-dns-id-not-matched)                    494
  FP L2 rule drop (l2_acl)                                               5757074
  Interface is down (interface-down)                                          22
  Dropped pending packets in a closed socket (np-socket-closed)             7056

Open in new window


Don't know if that would effect any of the lan transfer speeds.
So you have jumbo frames turned on?

Turn them off and test with new switches.
Turn them off on the NIC? I don't seem to have the option, the lowest it goes is 1500 with no on or off option. I have noticed that some computers do have a on and off in the past (this was one of the first things I looked into), but not all. Some of them like this computer doesn't seem to have it, only can set the size..

User generated image
Well,then I guess not.

Here's a little product called jperf.
You can test nodes on your network for speed and see if it's a certain port or setting.
Pretty simple to use.
You set up one node as server and the other as workstation.
It then will give you stats in mbit or gbits or whatever.

I would go round robin and see if it's just a driver or a switch port setting.

http://code.google.com/p/xjperf/downloads/detail?name=jperf-2.0.2.zip&can=2&q=

http://www.smallnetbuilder.com/lanwan/lanwan-howto/30408-measuring-network-performance-jperf
No, the show command on the switch shows what the switch is configured for.  If all the interfaces involved and your PC shows 1500, then it is using 1500.

Some switches it is at the system level and some it is at the "interface" level.  The interface can be a physical port, port channel, or vlan interface.

Just to make sure, with the old switch you get 90MB, that would be over 900Mbit/sec or close to wire speed for 1Gbe.  

You replace the switch with the new one and you only get 11MB, or just over 100Mbit/sec.

I do know the 2960 can do 1 Gbps, I have a few in our network.
Might be a dumb question, but you said that 90MB is close to GB speed..shouldn't it technically be closer to 1024MB (theoretical speed) ( being 1024 in 1 GB? But as mentioned above I should be seeing something like 200MB after the overhead and all the other things combined that come into play.

On a side note:

I noticed that on the ASA 5510.. port 0/0 (our line out to ISP) is running at 100 full BUT the port has the capability for gig speed. The ISP must have 100 full on their side. Port 0/1 is running full 1000..which is our gateway. And we have Port 0/2 which has all of our wireless and about 4 different vlans with different SSIDs. I notice this port is running at only 100 full.

This is because of the ASA and it's limitations of only have 0/0 and 0/1 gig capable as I learned in another thread.

The wireless is barely being used, but with that many vlans and the 23 or so APs broadcasting out, might have some overhead?

Do you believe that this might be possibly causing issues for the other networks on the other ports?

It looks like the ISP line is auto negotiating anyways at 100 full...should I just move that port over to 0/3 and move the wireless network to the gig capable port 0/0?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
>The wireless is barely being used, but with that many vlans and the 23 or so APs broadcasting out, might have some overhead?

Wireless is another whole ball of wax.

Depending upon draft spec g, n or a ,wireless speeds can be very unpredictable.

Take g as an example,the best you will get is in the 20ish mbits range.
Wireless acts more like a hub than a switch using a method almost like CSMA/CA.
Depending upon how many folks attach to the AP,and that number can go way down.
What is your ISP link speed on the WAN side?
It's purchased at 20mbps, but I've only seen about 15 on a good day which you are to expect loss from the ISP usually...especially since we're rural.

Our wired connection has about 13-15 normally...the wireless we're lucky to get 1 even with great signal.  I've had Aruba techs look at it, and they say everything is okay with it. Maybe that has to deal with the 100mbps connection instead of the full on the ASA?

But I guess it sounds like 90 MB is a good throughput based off your numbers, I'm more concerned about the old switches, because the new switches will basically take the place of them, but 10-20mbps is all I get on transfer speeds...even with a crappy HD should it really be that slow? This is even trying it on a newer computer with sata 6.

 
Unless your ISP connection on the WAN side is more than 100 Mbps, then you can leave it right there.  You can only go as fast as the slowest link between the two computers that are talking to each other.  Based on your other Internet tests, your ISP link is 10 Mbps, so your 100 Mbps connection to your firewall is not causing a problem.



Wireless is another whole ball of wax.

Depending upon draft spec g, n or a ,wireless speeds can be very unpredictable.

Take g as an example,the best you will get is in the 20ish mbits range.
Wireless acts more like a hub than a switch using a method almost like CSMA/CA.
Depending upon how many folks attach to the AP,and that number can go way down.
So I guess would you say that typically on a port running at only 100 full, with 4 vlans..2 switches..23 APs..would cause much issue on the ASA if it only has about 8 clients attached to it at a given time?



PS: The web100 site is working for me again. I wasn't able to do the test for a while because it was saying it was doing one already for the past few hours. But here are the results of after doing a auto auto on the ISP line out. It changed it from 100 full manual to 100 full auto.

Here are the results after:
Old Switches
User generated image
New Switches
User generated image
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Take a typical wireless G connection.one device attached 20 mb,2 dev 10 mb ,3 dev 3 mb and so on.
It's shared bandwidth as opposed to switched.
O.K. we need to keep everything on track and focus on one thing at a time.

The ASA should not enter the conversion unless computerA and computerB are on different VLANs/IP subnets and you are routing the traffic through the ASA.  

Are the two computers on different or the same VLAN/IP subnet?  If so, then lets leave the ASA out of the conversation.

If the two computers are on different VLANs/IP subnets, do you route them through the ASA?  If not, please leave the ASA out of the conversation.

Are either of the two computers using wireless as its connection?
If the answer is no, then please leave anything dealing with the wireless network out of the conversation.

Basically what needs to be focused on are the two computers and the logical and physical network path between the two computers.  Anything else being introduced into the conversation just takes us off track.

So:

Your original issue seems to be that copying a file from comptuerA to computerB got a througput of 90MB/sec.  Which means if you had a 100MB file it would take just over 1 second to copy.

Is that correct?

Then you added a new switch, moved computerA to the new switch and now when you copy a file from computerA to computerB you only get 11MB/second.  Which means the same 100MB file now takes 10 seconds.

Is that right?

If both of those statements are correct, then we can start looking at the individual networking components to see what is causing the drop in performance.
I made reference in one of my posts to essentially do what you have outlined using jperf as a tool  testing between nodes to for for speed.

Easy to use and easy to tweak.

Mike Pennacchi from Network Protocol Specialists turned me on to it a couple of years ago and it's been my go to for troubleshooting LAN speed problems before hitting Wireshark and such.
Yes, jperf would be great for him to install on the two computers to do various testing with.
Here's some results:

User generated image
User generated image

When dragging and dropping files from the SBS, they both however have about the same speed...20mbps.

The second one looks like it's a firewall dropping it based on the sudden drop...I think the default was 5001. They both use the same centrally managed Firewall for the local host, not sure why it's dropping the second.

Only difference is one is XP where the drop happens, client 1 is win7. But maybe Client1 will show you what your looking for?
So client 1 is W7 and client 2 is XP?

C1 looks fine.
C2  is horrible.

Are these on different subnets?
I assume then you have something like:

"SBS2011" < -----. OLD SWITCH (???)  <------> NEW SWITCH (2960)
                                   /\                                    /\
                                   |                                     |
                                   \/                                    \/
                                WinV7                            WinXP

What happens if you move WinV7 to new switch and WinXP to old switch?  You should really use the same client for all tests, that way the only thing changing is the network connection.  

Windows V7 by default is configured for TCP window scaling size and uses newer version of SMB protocol, which SBS2011 also uses.

Window XP by default is not configured for TCP window scaling and uses older version of SMB.  You can make changes to XP to enable window scaling.

Can you sanitize the switch configurations and post? Also let us know what the old switch model/brand is.
These are both tested on the old switches, sorry I thought you guys wanted me to try a few clients. The windows 7 PC is my main laptop that I've been using test from, so I'll probably stick with that one.

The old switches are the 2960's. I can get you those configs here soon.

As long as I can get the old ones going that's my main concern (being the servers are on them.) The new ones I'm not too worried about (for now).

As far is the auto-tuning level, do you recommend them on or off for the client machines?
My point is if it's only the XP machines that are having the latency issue and not the W7,then we have narrowed it down and can proceed from there.
O.K, I thought that the network performance on the "old" switches was "fast" and that the performance on the new switches was "slow."

Maybe I need to go back and re-read the original question, be cause it now seems that you are saying that when doing a file copy to/from the server you are getting 20MB/second period.  Which although not great, its not bad.  I get 25-30MB, sometimes 40, when doing SMB copies to/from Windows V7.
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I'll give that a try, sorry I haven't been able to reply for a while, (just had eye surgery). I'll report back.
Turned out to be a duplex setting, turned it to auto, then bam started working. Thanks everyone for the help!