ZFS Server Build

Great...knew this was the place to get answers!!

Thanks for the replies
@meyersd - yes this will be on all day every day - plus I just saw those Seagate models only have 1 Year warranty which doesn't help either...so thanks for that reminder...

@dlethe - going to grill you if you don't mind ;)

The main reason I asked the question was I read on another forum that 'TLER interferes with ZFS and you shouldn't use drives that have this option' - I thought it didn't sound correct so I was doubting myself...from your answer I can take it that TLER isn't anything to worry about?

This box is my 2nd build with ZFS - the first one was simply a storage build - i.e. I have a number of NAS devices(ReadyNAS/QNAP) that I use already and wanted to have a 'backup' of those(since we all know HD's fail no matter what RAID is used) - so I built it purely as a backup storage device and I rsync from the existing NAS units to this ZFS machine once a week

Now I want to build something with a bit more kick since I want to use it for storage and possibly experiment with NFS for VM storage to an ESXi machine...if that makes sense...

So yes I want to get a decent performance out of the new build...at this stage I'm starting with what drives to get...have yet to pick out a motherboard or anything else...

Can I ask if you don't mind what you are running your NAPP-IT system on? Hardware wise I mean? Motherboard make?

When I built the first model I used an ASRock Z77 Pro4-M - but when I installed NAS4Free it didn't recognise the onboard NIC, so had to get another card to fix that...what I'd like this time is to get a board that is going to run Napp-It out of the box without more cards needed...so wondering what you are running on? Or any recommendations?

Am I right in saying I could realistically use multiple drives in this machine - i.e. couple of SAS for the high performance pool and a couple of SATA for storage pool etc etc...?

This is good info thanks for the inputs folks...

P.S . Those 4 green drives used to be my /data pool, which I replaced with the SAS-based mirrored drives.  3X-5X faster.  VERY easy to migrate from the 4 SATA disks to the 2 SAS drives with ZFS also. Just a few zpool commands

Those 2 Torx SSDs are also tied with a rubber band and duct taped to the power supply.  No bays or room for them, but not a problem with SSDs.  What can I say, I'm a tightwad  ;)

Excellent...more good info to add to the solution ;)

I have a Dell T110 server that isn't being used for much...so I could in reality use it as the base and build up since its got the 'server class' board etc...is that what you are saying?

Reason I wanted to build from scratch was drive space - nice to be able to add more drives at a later time to the same unit...

For talk sake lets say I can't get a 'used server' - what is the best board I can go with? By best I mean give me an example of 'server' boards? I'm not used to spec'ing that type of thing - I always purchase branded servers for clients so this is where I fall over

A $3000 high end ZEUS SSD

- nice...when i win the lotto ;)

Thought the TLER thing was a mistake alright...my current box has WD RED's already and no issues so far!!

tied with a rubber band and duct taped to the power supply

Didn't know that about SSD's either...must keep in mind ;)

Don't get just a board. Get a whole working system.  You certainly have to have 64bit CPU, dual core, but anything more than that will be a waste.  You should really be able to pick up an old HP or Dell server that sold for $5000 or more several years ago for a few hundred dollars.

They're noisy but rock solid. Check out your local used computer sellers. If you are in a major city then you should have no problems getting them.   If you must create your own, get a true server motherboard.  One designed for use specifically as a server.  Go to the intel.com site to educate yourself.   ECC memory is highly recommended, but only if you want 24x7x365 and never any problems.

(But watch out for needing 220VAC.   When I built my home I had them put in some 220 plugs, so not an issue for me)

Ok...yep I can pick up a used one somewhere...I supply servers to clients so there's always a few being replaced...cool

So realistically I don't need high speed cpu then? When you say

dual core, but anything more than that will be a waste

is that what you mean? No need for high end or did I misread that bit?

Power isn't an issue(Ireland I'm from) so no worries on that end...
Will check out the Intel site as well cheers...

I do have old Dell servers but they are VERY old...like Pentium based ones...and loud...so will avoid those for this run...

This is all great info appreciate the time to help sir...

No, not the Pentium, you need 64bit architecture.  Old XEONs would be fine.  

Here is a top from my system

last pid: 21076;  load avg:  0.12,  0.13,  0.13;  up 79+19:46:36                                                                                                                               08:15:59
92 processes: 91 sleeping, 1 on cpu
CPU states: 98.1% idle,  0.6% user,  1.3% kernel,  0.0% iowait,  0.0% swap
Kernel: 440 ctxsw, 310 trap, 691 intr, 1614 syscall, 244 flt
Memory: 8186M phys mem, 946M free mem, 4093M total swap, 4093M free swap

Select allOpen in new window

Now after kicking off a zpool scrub ...

last pid: 21132;  load avg:  0.22,  0.15,  0.14;  up 79+19:47:54                                                                                                                               08:17:17
92 processes: 91 sleeping, 1 on cpu
CPU states: 88.0% idle,  1.1% user, 10.9% kernel,  0.0% iowait,  0.0% swap
Kernel: 1121 ctxsw, 255 trap, 3406 intr, 1126 syscall, 192 flt
Memory: 8186M phys mem, 931M free mem, 4093M total swap, 4093M free swap

Select allOpen in new window

You'll note that I only have one processor turned on also!

Sweet...have a bit of looking to do then, will get cracking on this and see how it goes!

Again appreciate all the assistance as this will def help with getting this build right...

Legend as always sir ;)

Will close this one out...

This is what makes this the best forum on the internet...period

Great help as always folks...much appreciated

Shane

Last advice ... scrimp on the server, but not on the disks.  Go SAS and let ZFS do dedup & compression to get usable space.  Do mirroring or RAIDZ2, but not RAIDZ1. You don't want to sweat catastrophic data loss for what may be a 24 hour or longer rebuild with RAIDZ1 in event of a 2nd HDD failure.

Forgot to mention the hell I went through when using those green SATA drives when It was degraded due to lack of TLER.  I often had to rebuild it because ZFS killed a perfectly good drive due to how long it took to do error recovery.

If you use SATA, you MUST use enterprise class, but for the money SAS is so much better.

Will do...cheers again...no doubt I'll be back with more questions when I hit the next wall ;)

One last thing...apologies

You mentioned 'SAS Backplane' - am I to assume that this means to use Enterprise SAS drives I need this? Meaning I can't just connect those drives to a regular SATA port on a motherboard?

If not then that means yes i need a proper server based setup as they usually come with all in...just wanting to be clear on that bit before I continue...cheers

SAS drives need a SAS controller.

Hello smckeown777,

Sorry for posting on a stale thread.

I am evaluating the exact same things now,  Napp It, nas4free, FreeNAS, NexentaStor.

Though nice, I tent to decide against FreeNAS (FreeBSD zfs and Comstar ports).

Can you give me a brief rundown of your experience with NappIt and nas4free - or what you used in the end?

Thanks a lot,
Daniel

Hi Daniel

Sure...I went with the Nexenta build for this project...working out fine as well, picked this one since it was more up to date with the ZFS stuff etc...

So no issues to report all working very well, I've also built another machine with Nas4Free as well using the WD RED sata drives in that one so have both up and running and all fine

NappIT I didn't experiment with really...spun up a VM version just to have a look but the Nas4free and Nexenta interfaces I got used to quicker so stuck with those...

Didn't look at FreeNAS at all either

Hope that helps!
Shane

Hello Shane,

from what I've seen so far (still in the 'VM stage') I think I'll go the same road with NexentaStor. NappIt also works really well nowadays; but for production use I feel better with the broader support of Nexenta
Did you go with the Commutiy or enterprise version of Nextenta? And to give me an idea, what is your approach in backing up the iSCSI LUNs you provisioned?

Thanks, much appreciated!

Hi Daniel...

I went with the community edition as my data requirements are below the 18TB it supports...

Don't use iSCSI so can't comment on that bit...as for backups I use rsync...so simple setup

Hello Shane,
thanks again; I move to the lab now!

Have a nice weekend,
Daniel