slow file transfer over NFS

ok, I read the man page which says:

=======
Especially useful options include:
rsize=32768,wsize=32768
    This causes the NFS client to try to negotiate a buffer size up to the size specified. A large buffer size does improve performance, but both the server and client have to support it. In the case where one of these does not support the size specified, the size negotiated will be the largest that both support.
=======

So I used those values in my mount command. How do I know what value was negotiated ?

It "seems" to be running faster, but won't know until it hits a large file.

is there a better way to transfer files from one machine to the other ?

I tried FTP, but got similarly dissapointing speeds:

226 Transfer complete.
1995529 bytes sent in 12.84 seconds (151.83 Kbytes/s)

How can I check whether the disks are being flogged on the old box ? It's quite a few years old now, so it could just be hardware is being maxed out by the disk copy (I think it has whopping 9GB SCSI disks in an external enclosure)

I realised that if I just type "mount" then it tells me what parameters it used. It was already using 32768, so it was only perception that is was faster.

You could do the mount in the other direction - mount a dir on the new server to a point on the old and then do a compressed tar with  the target on the nfs mount.  You would be trying to use the horsepower of the old server in that case though.

Another thought would be the compress flag on the rsync - again, you would need to run it in the other direction (push from the old to the new) in order to see any benefit (which could be negated by the speed of the old server).

You might need to experiment with the rsize/wsize and specify them directly,  bigger is not always better.  You could spec them at 2K 4K 8K 16K... and test speed at each level.

This site is meant for Linux, but the speed test stuff is applicable for you:

http://nfs.sourceforge.net/nfs-howto/ar01s05.html

thanks for all the comments, I'll try to address all of the questions that have been asked.

The backup takes about 90 minutes. I'm not sure how much data is being written to tape. The result from the backup command is:

fbackup(3056): total blocks written to output file /dev/rmt/0mb: 23721866

If the blocks were 1KB each, then that is 23.7GB onto the tape. Considering it's a DDS3 tape, thats fairly impressive. If the blocks are half the size (ie. 512 bytes) then it's 12GB onto the tape, which is probably a lot more reasonable.

I did check, network card on old & new server are both set to 100 full. As I said, I looked at the stats on the router that joins the two networks and the utilisation of the interfaces is only 2-3 Mbps max.

I can't install stuff on the old box, so rsync pushed from old to new isn't an option, I did think about this so that I could use compression with rsync. I can try a file copy the other way though.

I'm not sure about doing a compress/zip on the files using the old server. When I say old, I mean OLD. I'm led to believe it's probably 8-10 years old, so using CPU on it for compression may not give much improvement.

On restoring from tape, as you can see from the above, the backup is being done using "fbackup". From what I can see the equivalent utility is "frecover" to restore the backups. Can anyone give advice on whether I will be able to restore the backup to an NFS mount with this utility ?

I've inherited this situation and I pretty much can't change anything on the old box. I'm a linux person (RHCE), so I don't know too much about HPUX, but luckily most of this isn't HPUX specific stuff.

I'll be trying to restore from tape to an NFS mount today. I'll update this when I either manage to trash the old system or successfully get the parameters of frecover correct and restore to a different path ;)

cd /mnt/recovery_dir;frecover -xXf /dev/rmt/0mb

Test it with `-i /file/that/can/be/safely/overwritten`

Just an update, I haven't had a chance to test the restore from tape to an NFS mount. I'm still planning on it, just that another project has been shoved in front of the HPUX migration and so it will have to wait for another week or so. Thanks for your patience, I will update again when I have more to report.

I ran frecover with the options given and it says:

frecover -xXf /dev/rmt/0mb -i /usr/local/log/shutdown.msg
frecover(1024): can't create directory ./local
frecover(1024): can't create directory ./usr/local
frecover(1015): cannot chdir to directory ./usr/local
frecover(1024): can't create directory ./usr/local
frecover(1024): can't create directory ./usr/local/log
frecover(1015): cannot chdir to directory ./usr/local/log

Which tells me that it's not creating the directory tree structure for where it wants to put the file. Is there a way of making frecover create the directory structure as it goes ?

For my test file (/usr/local/log/shutdown.msg) I've just created the directory for it and it worked fine after that, but I don't want to have to do that for everything.

I suspect this has something to do with NFS and permissions and the like, but I'm not totally sure. When I run the full restore it is able to create some of the file structures, but not the others ?

I am exporting with:
/oldbox -anon=65534,rw=oldbox

I managed to get my restore across NFS working, but it was still boringly slow.

Even when I connected the second NIC from newbox onto the same LAN as the oldbox (and disconnected the other NIC, so it could only be using the correct one), I still got really SLOW restore speeds.

I was restoring using the frecover as suggested above and to restore a fraction of the tape (a DB backup) still took about twice as long as it takes for the full backup to run and write the data to the tape in the first place.

Seems like the slowness is being caused by writing the files across NFS still ?

Any other suggestions ?

My experience is that "pushing" with NFS is slow (permissions?) while "pulling"  is quite snappy.  I think, when "pushing" it checks to see if you are allowed before letting you copy each file, while if you are pulling you, by default, have permission. I already (half) stated that above...