VMware ESX and NFS problems

I have recently setup a VMware ESX 3.5 environment consisting of two hosts.  They have an NFS Datastore that is served up via NFS from a physical Linux server with a RAID 10 of 4 1 TB SATA drives. Each of the VM's (4 total) also connect to an NFS share off of the same RAID 10 (though on a different LVM logical volume).  The VM's are all RHEL 5 as well as the NFS server.

I can scp files between VM's at 20-30 MB/s, scp files from the Datastore (VM's local storage) to the NFS mount at 20-30 MB/s and back again at the same speed.  I can scp files back and forth to the NFS server directly at 20-30 MB/s.  I can dd if=/dev/zero at greater than 100 MB/s to either the Datastore or the NFS mount.  However, if I dd if=/dev/urandom it pegs out at a max of roughly 5-6 MB/s.  The same occurs with the dd using urandom to an NFS volume on a 0 NetApp filer as well from the same VM's.  With the VM that has 4 vCPU's and 24 GB or RAM all of those 20-30's go up to about 60 MB/s but no higher.

It seems that anything actually generating new non-zero data hits a wall.

dd with /dev/zero to NFS mount:
dd if=/dev/zero of=/storage/testfile2 bs=8k count=10000
10000+0 records in
10000+0 records out
81920000 bytes (82 MB) copied, 1.17408 seconds, 69.8 MB/s

dd with /dev/zero to Datastore:
dd if=/dev/zero of=/testfile1 bs=8k count=10000
10000+0 records in
10000+0 records out
81920000 bytes (82 MB) copied, 0.446165 seconds, 184 MB/s

dd with /dev/urandom to NFS mount:
dd if=/dev/urandom of=/storage/testfile3 bs=8k count=10000
10000+0 records in
10000+0 records out
81920000 bytes (82 MB) copied, 16.4113 seconds, 5.0 MB/s

dd with /dev/urandom to Datastore:
dd if=/dev/urandom of=testfile2 bs=8k count=10000
10000+0 records in
10000+0 records out
81920000 bytes (82 MB) copied, 15.3982 seconds, 5.3 MB/s

scp from Datastore to NFS mount:
scp testfile2 XXX.XXX.XXX.XXX:/storage/testfile3
testfile2                                                                                                                              100%   78MB  26.0MB/s   00:03

scp from NFS mount to Datastore:
scp XXX.XXX.XXX.XXX:/storage/testfile3 testfile5
testfile3                                                                                                                              100%   78MB  26.0MB/s   00:03

nfsstat -c:
nfsstat -c
Client rpc stats:
calls      retrans    authrefrsh
143587     0          0

Client nfs v3:
null         getattr      setattr      lookup       access       readlink
0         0% 7647      5% 34        0% 71439    49% 6473      4% 0         0%
read         write        create       mkdir        symlink      mknod
1017      0% 45415    31% 35        0% 0         0% 0         0% 0         0%
remove       rmdir        rename       link         readdir      readdirplus
37        0% 0         0% 0         0% 0         0% 719       0% 10075     7%
fsstat       fsinfo       pathconf     commit
4         0% 396       0% 0         0% 97        0%