asked on

Gigabit not real Gigabit transfer

I have two machines hooked directy together through cat 5 both gigabit nics. I am trying some scp transfer and can only get 28MB/s. what is 224 Mb/s. That is only 22%. I tryied to adjust my sysctrl file from a few different sites. Did not change the number at all. Any ideas what would be slowing this to 28 MB/s.

I know that i am suppose to test this from the ram and not hard disk but the hard disk still rights faster then 28M/s

hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 144 MB in 3.03 seconds = 47.58 MB/sec

Any help would be great, thanks!

ravenpl

> hdparm -t /dev/sda
This is linear speed of reading the device. Try testing filesystem file write, ususally it gives ~30M on SATA drives.
Also, scp does intensive encryption/decryption, which may be the couse also.
Then, most of cheap 1G chips has so little buffers, that it's impossible to reach the 1Gbps.
Then, most of cheap mobos puts the 1Gbps NIC on same PCI bus along with other devices (like hdd). They altogether share 133MBytes/s PCI bandwidth.

Please try something from http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html

aashishkunte

Dear clintonm9,

Your both NIC's have Full Duplex Mode ?
You can check this using ethtool and find out the Duplex if NIC. For Optimum Data Transfer Needs Exact TX & RX speeds.

Regards,
Aashish

clintonm9

ASKER

[root@yourspasite ~]# ethtool -a eth0
Pause parameters for eth0:
Autonegotiate: on
RX: on
TX: off

clintonm9

ASKER

is there anyway to test the filesystem file write speed?

ravenpl

try simplest, the following writes 100M file /tmp/FILE and shows the execution time
sync; time { dd if=/dev/zero of=/tmp/FILE bs=10240 count=10240; sync; }

clintonm9

ASKER

sync; time { dd if=/dev/sda of=/root/home.tgz bs=10240 count=10240; sync; }
10240+0 records in
10240+0 records out

real 0m5.212s
user 0m0.033s
sys 0m0.634s

home.tgz is 160 mb

is this correct

ravenpl

It's possible(and gives < 20MB/s), but
why haven't You use my command?
The command I provided reads from memory, writes to file, therefore tests write speed only.

clintonm9

ASKER

sorry i was confused

sync; time { dd if=/dev/zero of=/tmp/FILE bs=10240 count=10240; sync; }
10240+0 records in
10240+0 records out

real 0m2.736s
user 0m0.046s
sys 0m0.541s

ASKER CERTIFIED SOLUTION

ravenpl

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

clintonm9

ASKER

So how do you improve that with SATA?

ravenpl

You will not. that's the common limit. In fact 36M is not so bad.
Ok, You can improve this by creating raid0 (striping) either with hardware (high end controllers) or with linux software raid.

clintonm9

ASKER

Thanks for your help, why do drive advertise 300 MB/s SATA is you are saying 32MB/s is good?

ravenpl

Please try one of http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html
Will give you more realistic results on network speed.
Bye.

ravenpl

> Thanks for your help, why do drive advertise 300 MB/s SATA is you are saying 32MB/s is good?
That's what They advertise ;)
You have to ask Them, but it's commmon, that They are saying, that the drive have speed of 150MB/s becouse sata controller can do this speed. But then, controller is connected to PCI, which speed is only 133MB/s, and so on ;)

clintonm9

ASKER

what if you use 66mhz pci?

ravenpl

Then its faster. But is Your controller connected to pci64/66 ?
OK, there is high end HW an ow end. I've seen SIL3114 sata controllers that were for pci64/66 (at least You could plug them into), but reading from four connected to it drives (it has 4 channels) concurrently coudn't beat the 90MB/s speed.

clintonm9

ASKER

what about this?

http://www.3ware.com/products/serial_ata2-9000.asp

over 800MB/sec RAID 5 reads and exceeds 380MB/sec RAID 5

ravenpl

3ware is know as the only really hardware sata raid. I would call it high end sata hardware...