Suggestions: Building a NAS

Hello Experts,

I am evaluating the possibility to build a NAS appliance from existing hardware we have in our company. I need some advice and input on what to evaluate further and what caveats to keep in mind. Also, though I am firm with handling Linux, I am not that far with UNIX (FreeBSD, OmniOS, OpenIndiana, Solaris).

The appliance would mainly be used as backing iSCSI storage for a few VM hosts/nodes (2-4). Also, it might be possible that we provision some classical network shares with it.

Here are scope statements, 'must have':
- Cut costs (as an alternative to a new QNAP)
- iSCSI, 10GbE
- Thin Provisioning
- LUN Backup and snapshotting (via API / shell script)

'Nice to have':
- Hierarchical storage
- Compression
- Dedup
- Replication
- High Availability

From the specs, I really like to use ZFS, which I am somewhat familiar with and can do all this.

This in mind, I came across two distros:
napp-it on OmniOS:

I already have FreeNAS running in a VM and from what I can do it looks promising.

Special Questions:
- From what I read Oracle pulled ZFS from Open Source, so the FreeBSD port is not very up to date any more. Is it therefore better to use a Solaris fork like OmniOS / OpenIndiana?

- What matters the most to me is the iSCSI performance. I read the whole iSCSI layer is not such a good performance in FreeBSD then the other distros I mentioned?

- Can you suggest more distros / appliances or even whole different approaches which make better sense?

Thanks in advance,
LVL 13
Daniel HelgenbergerAsked:
Who is Participating?
kevinhsiehConnect With a Mentor Commented:
Since you are looking at options without anyway, take a look at Nexenta Community Edition which is good for 18 TB.

I am having problems with copying the url but just do a search for Nexenta Community Edition and you will find it. It is based on ZFS and is under active commercial development.
Aaron TomoskyConnect With a Mentor Technology ConsultantCommented:
Personally I use and love it. It's FreeBSD based but I had almost no Linux/unix experience when I started with it and got along just fine. I use Nfs vs iscsi but I easily added samba shares and even tied to active directory. The best thing about this setup is you make a second and use zfs snapshot send to the second one.

If you really want, you can use zfs on Linux but I don't think there are any easy distros like you have for FreeBSD/Solaris
arnoldConnect With a Mentor Commented: might be another option to consider it has a HA configuration using drbd as the means by which data is replicated from one node to the other.
Complexity lies in configuring drbd in such a way that when the nodes change status that the drbd replication direction transitions as well.

I'm aware that Openfiler development seems to have .........
Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Daniel HelgenbergerAuthor Commented:
Hello, thank you for your input so far!
Personally I use and love it.
Right! I read about zfsguru. It is still beta? Any problems? What is your usecase? I know it is about the GUI; zfs as basis is considered stable.

I'm aware that Openfiler development seems to have .........
Hello arnold, your comment seems to be truncated? I know about openfiler. I dropped it from my list though because it is Linux, and ZFS support in Linux is experimental. Also, I would need the advanced iSCSI license I think. In this case, I rather by a QNAP.

Can anyone tell me something about the general performance of the FreeBSD ports of ZFS and COMSTAR compared to Solaris (clones like OmniOS, OpenIndiana)?

Also, I now have a second VM with Napp-It. I consider FreeNAS more comfortable and with a larger user base, but it seems to be very powerful as well as it features the latest ZFS. Also, I was able to create a thin provisioned iSCSI LUN in almost the same time as in FreeNAS; usability is ok for me I suppose. Does anyone here use Napp it?
Not sure what license are you talking about, FC based connection?
Think that openfiler indicated no further development beyond version 2.99
Daniel HelgenbergerAuthor Commented:
Hello arnold,
I mean this:

As I understand it, it is some advanced GUI settings. Since I expect to use this feature a lot, I rather use the GUI than command line.
Its 1000 Euro. I do not need the FC target licence; I would only deploy iSCSI luns. Also, I think I would need a commercial license?
Since you will mainly use this setup for backup purposes, why do you think you need the plugin?

I could understand the need for the plugin if you use this storage for continuous and time sensitive IO such as a storage for a database.

Try the VM version of openfiler, and see if it meets your needs.
Aaron TomoskyConnect With a Mentor Technology ConsultantCommented:
My last setup was 8 drives in a raid 10 (zfs calls this mirrored pools) with an intel 320 slog, samsung 840 l2arc and 24gb ram on a workstation I decommissioned, first gen i7. This was starting to slow down with more than 20 vms on 5 low power hosts. I had two 1gb nics, one for Esxi Nfs and one for samba over the LAN.

So you can do quite a bit without a lot of horsepower. The problem running vms becomes iops way before throughput. For My new setup it was easier to just get 4 samsung evo 1tb ssd for my vm pool. More iops on less drives.
Daniel HelgenbergerAuthor Commented:
Thank you, I did not know that Nexenta has a community edition. I will def. give it a try.

Apart from support and the 18TB cap, are there any restrictions worth noting? I briefly looked over their site and did not find anything.
I haven't used Nexenta as we have always decided that we would rather pay a vendor to do the integration rather than have separate hardware and software vendors with us stuck in the middle. For what it is worth, the paid version is pretty inexpensive.
Daniel HelgenbergerAuthor Commented:
True, this is how I like it best, too.
For what it is worth, the paid version is pretty inexpensive.
Before I get a quote, can you tell me what that means? And is it licensed per raw storrage or net storage? The net storage I need in the beginning would be little more than 5TB - but mirrored.

Aaron TomoskyTechnology ConsultantCommented:
last I checked it's licensed as raw. So if you have 4 3tb drives in a raid 10, you get 6tb but licensed for 12tb
Daniel HelgenbergerAuthor Commented:
Has any of you an estimate on how much they charge?
Daniel HelgenbergerAuthor Commented:

thanks for all the input on that matter. In the end I decided for NexentaStor. I also want to thank smckeown777 for the insight:

If anyone is interested, a 18TB commercial license for NexentaStor is about 1.800.00 Euro atm.
Aaron TomoskyTechnology ConsultantCommented:
Remember it's raw storage for cost purposes. So with raid 10 18tb is 9tb...
All Courses

From novice to tech pro — start learning today.