We are redesigning out current ZFS file server. It currently only has one pool of over 70TB.
I see this is a problem since, if there are enough disk failures within the pool to make it non-functional, we have to restore all 60TB of currently existing data.
A debate has risen as to how to divide up the storage into smaller pools, and how big/small each of those pools should be.
As I see it, the only drawback of using pools that are "too small" is a lot of wasted disk space. We are using `4TB 7500 spinning disks` with `4TB SSD cache disks`. So, I'm leaning towards 20TB raidz vdisks. That means we get about 77% of the disk space usable.
Q: Are there any other advantages/disadvantages to making the pool sizes too large/too small?