Hot Plug Enclosure/Setup for ESXi box

Hi folks...need some assistance with a number of questions, hope someone can help...

I've built a new server that is running ESXi 5.1 with 2 VM's - Nexenta(for ZFS) and Windows 7

This is the hardware I've used to get this working

Motherboard - SuperMicro X8ST3
Hard drives - 16GB USB(for ESXi), 250GB Sata(for datastore), 2 x 3TB Seagate Constellation E3's - Model ST33000650SS

How I've set this up is I have setup 'Passthrough mode' on the onboard LSI1068E controller to pass to the Nexenta VM and access the 2 3TB drives for ZFS...

This motherboard has 8 SAS ports and 6 SATA ports - the SAS are passing to the Nexenta VM so all I'm left with for passthrough to the Windows 7 VM are the SATA ports...

Ok here's the issue

I want to pass drives through to the Windows 7 VM - these drives I want to be 'hotplug''s why

I am using this box for both storage and diagnostics
The Nexenta setup with ZFS running as storage(already shared out)
Now I want to be able to use Windows 7 to run various hard drive diags applications and things like Runtime's Raid Reconstructor for with the 6 onboard SATA ports this should work I assume...but I then realised what I'd really like is a hotpluggable setup - where I've the controller passing direct to the VM but able to handle hotplug drives...

This is where I am lost through and need assistance...not sure about hotplug and what is needed to make this work? Do I need an external enclosure that connects to a sata port?
I assume the onboard sata ports will not work with hotplug drives? Or can anybody give me details on what makes it 'hotplugable'?

Anybody with any experience with something like this? Or is this even feasible?

Any other info you need just ask...

All ideas welcome...thanks
LVL 24
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

I'm not sure I'd be going this route for what you're trying to do.

If your primary goal is to be able to do data recovery - raid reconstruction, etc. or arbitrary disks that a client gives you, you really want to be plugging those disks directly into the SATA ports on the machine you're working on - a direct hardware connection gives the best chance of success especially when your disks are in questionable condition.  Putting too many layers of abstraction - virtualization, passthrough etc. - is going to cause issues when you're dealing with disks that are in questionable condition.

There's a bazillion hot-swap bays on Newegg. They all pretty much work the same way. They're just a physical tray / carriage which lets you slot the disk in and out easily. Nothing special to them, really. If your motherboard's sata ports supports hot-swap (if you're AHCI, it probably does), then you can use a hot-swap bay:

But... you want to install this in a physical desktop workstation.
For your Nexenta NAS with the 2x SATA disks... you probably want this to be a physical piece of hardware, not virtual. You can run this even on commodity hardware.

The NAS provides the storage to your ESXi host. The ESXi host uses that storage to store the data for it's VMs.

While it is technically possible to make a VM and install Nexenta on it, and then use THAT for storage of your VMs, it's a bit of a chicken-and-egg problem and not really ideal. Nexenta works best when it connects talks to the disks, uninhibited by virtualization, RAID configuration etc.

If you just want to play with ZFS a bit and get a feel for it, sure, make a VM and play with it, but in that case just provision half a dozen small virtual disks as VMDK files and use those as your disks in Nexenta to get a feel for what you can do with it, creating, extending, and destroying various zpools and playing with it.

But when you're ready to use it for real, you want it to be installed physically on separate hardware. Besides, Nexenta is going to gobble up at least 4GB out of your 16GB available on your host. That's memory that could have been better used to power your VMs.
smckeown777Author Commented:

The reason I am doing this like I've built it is to take advantage of this hardware to do 2 things - storage and diags...

The Nexenta is just creating a simple share(and running on ZFS of course) to allow me to store recovery data/misc data - not for storing VM's or anything like just simple cifs share

As for the Win7 VM I figured since passthrough means passing the hardware layer to the VM - is this not correct?

If this isn't going to work then cool...this is an experiment so I wanted to get some thoughts/feedback on the setup

As for your enclosures - so from what I understand to get this working all I need is a motherboard that is AHCI compatible and nothing else? If so great...just want to check

I don't really need it to be internal...can be an external if that makes life easier...
Newly released Acronis True Image 2019

In announcing the release of the 15th Anniversary Edition of Acronis True Image 2019, the company revealed that its artificial intelligence-based anti-ransomware technology – stopped more than 200,000 ransomware attacks on 150,000 customers last year.

If Nexenta is being used simply for it's fileserver capabilities, I'd suggest you set up RAID in hardware, and provide a single VMDK to your Nexenta VM, which will be seen as a single disk. If you're virtualization your fileserver, redundancy should be provided by your hardware, rather than the VM. You can use ZFS but it won't provide any additional benefits over a regular, less memory hungry filesystem.

Any enclosure will do, internal or external. The enclosure in general is just a piece of plastic that positions the connectors and cradles the disk. The only requirement, as you said, is a motherboard which supports AHCI.

As for the Win7 VM I figured since passthrough means passing the hardware layer to the VM - is this not correct?
You are correct, but this works reliably only if your hardware is also working reliably, and makes assumptions about what kinds of responses it will be getting from the hardware. If you're working with potentially-defective disks which you are doing data recovery on, every layer you put inbetween your recovery application and the hardware is a potential place where problems could occur.

This is the same reason why connecting your disk directly to the IDE header on the motherboard is preferable to connecting it to an IDE->USB adapter / enclosure.
smckeown777Author Commented:
Thanks Frosty555...

Reason I setup Nexenta like I did is to allow ZFS to be the raid(since ZFS doesn't play with underlying raid controllers very well) so that's already taken care of

Main question was about hotplug - wasn't sure what all was needed to make that part work

I've now passed the onboard SATA ports using 'Passthrough' and its working like I expected...all I need now is the enclosure to make plugging in the drives easier...something with 4 drive bays should do the trick - any recommendations on what I could use(if you've worked with enclosures before?)

As for the setup...this is not in production...this is a simple experiment to see if things work...I'm in learning mode and want to see what is possible

I fully understand that in 'data recovery' mode that connecting directly to the ports is arguements there...

Thanks for the assistance.
"Hobbyist" enclosures are, unfortunately, notoriously cheap and flimsy... but they should work well enough for your purposes.

I've personally bought this one for my own FreeNAS server:

It's okay, and works as expected, and isn't too flimsy. The fan is nice. However, it is not a "trayless" hot swap system. You have to screw the mounting trays onto the hard drives, and it uses these tiny proprietary screws... don't lose them!

If I had to do it again, I'd probably buy this one:

Make sure you measure your case! These things are surprisingly deep, especially with the fan.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
smckeown777Author Commented:
Least I was looking at the right product line then! I was thinking of getting this one...

Good stuff, I'll see where I can source one and go from there...thanks for the input, will close this one out for now
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Server Hardware

From novice to tech pro — start learning today.