I have 2 freenas systems configured as follows:
Build: FreeNAS-126.96.36.199-RELEASE-x64 (e625626)
Platform : Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
PUSH - 1x28T raidz pool (2xSSD x 480GB for zilog, 1xSSD x 240GB + 13x3TB for data + 1x3TB for hotspare) Compression and deduplication are off.
PULL - 1x28T raidz pool (2xSSD x 240GB for zilog, 1xSSD x 280GB + 13x3TB for data + 1x3TB for hotspare) Compression and deduplication are off.
1x22TB zvol is created on PUSH
zfs replication from PUSH to PULL is setup according freenas manual.
Doing initial replication which supposed to transfer about 22TBs of data.
filesystem ~85% full.
Machines are geographically separated with BW about 40 megabits/s over MPLS.
zfs replication takes too long to complete, it running only about 8 megabits/s (0.8-1 megabytes/s). The network is not the bottleneck.
Disabling cypher and compression for the ssh transport increased the speed to around 10 megabits/s (1-1.2 megabytes/s) which is obviously slow too.
Am I alone with this or is it a known issue? Does anyone even solved this kind of issues before?
Should I file a bug report?