VMware vSAN - Need feedback on its drawbacks/limitations

Here are my frustrations with vSAN. Will like feedbacks from others.

1. You cannot hot-swap or add a disk while an ESX server is up – like you can do with other storage systems. You have to shut down the server to format the disk RAID0 to be able to make it part of vSAN.
2. Often you cannot upgrade vSAN software without replacing the physical controllers on the ESXi servers. For example, I cannot go from 6.0 to 6.5 vSAN unless we replace PERC H710 controllers. (Not looking forward to it.)
3. About 20 to 25% of disk space goes to waste to make up for the fact that 1 out of 4/5 servers may go down.
4. Shutting down a ESXi server becomes much more complicated since data sitting on its disks has to be ‘evacuated’ (off-loaded) to disks of other servers, sometimes taking hours.
5. Booting up an ESXi server also takes much longer, since data on its disks has to reconciled ('initialized') with data already in production.
6. After rebooting an ESXi server, you need to wait for almost a day – while all data is resynced – before you can reboot the next ESXi server.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Ajay ChananaMCSE-2003/08|RHCSA| VCP5/6 |vExpert2018Commented:
Please find the limitations below.

  1. vSAN does not support hosts participating in multiple vSAN clusters.
  2. vSAN does not support vSphere DPM and Storage I/O Control.
  3. vSAN does not support SCSI reservations.
  4. vSAN does not support RDM, VMFS, diagnostic partition, and other device access features.
AkulshAuthor Commented:
Dear Ajay Chanana,

I wished to know if you agree or disagree with 6 limitations I listed and why. Thanks.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I agree. those are the facts as you have stated.

are you using 10GBe, and All SSDs ?

Is version 6.0 your first vSAN implementation ?

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

AkulshAuthor Commented:
Dear Andrew,

Thanks for responding.
- Yes we are using 10 Gb Ethernet NICs with twinax Cu cables.
- All disks are not SSD, only one each is, in two disk groups.
- Yes vSAN 6.0 is our first (and probably last) vSAN implementation.

Let me add, there are further problems with vSAN
7. When I try to put an ESXi host in Maintenance mode with ‘Ensure data availability from other hosts’ option, sometimes it may get stuck, at say, 72%, for days even though no resynching is shown involving this host.
8. In Cluster > Monitor > vSAN > Resynching Components, 'ETA to Compliance' time keeps increasing for some components, which makes no sense to me. One 32 GB data set is not done in 24 hours. (No errors or congestion anywhere)

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
VSAN 6.6 is now the current version and it gets better with every version release but you have been trail blazers

So what are you going to replace it with?

Did you follow correct implementation procedure and have VMware sign off design?
AkulshAuthor Commented:
Dear Andrew,

We did follow correct implementation and in fact, things went fine for more than a year. (We rebooted ESXi hosts only once or twice in that time.) This month we had to expand the vSAN to increase capacity, and in the process I discovered those 6 shortcomings. (After reboots, HA is not working on two hosts and so frustrations are multiplying.)

Going to 6.6 would involve replacing PERC controllers, and I am not looking forward to it. Most of the above limitations will remain in future versions, though mechanism of resync is certainly going to improve. (#7 and 8 issue will likely go away.)

Luckily this is only a Disaster Recovery site for us, so will stick with vSAN for now.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
With all versions of VMware vSphere certified hardware changes or drops off the list

What server nodes are you using?

If you have VMware sign off for current design nag them.
AkulshAuthor Commented:
We have 4 Dell PE R820 with PERC H710 controllers.
After a VMware Support call, vSAN is finally working fine after many days.
But 2 hosts are refusing to participate in HA now -- nothing was changed on them other than new vSAN disks -- and so headaches continue.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
In the past 2-3 years we have deliberately dropped Server Nodes, and the PERC H710 and deliberately deployed vSAN on PERC H730.

e.g. because Dell R710, R720 are not officially supported by PERC H730 they were dropped in favour of R730 for vSAN.

PERC H730 also has better support for vSAN that the older H710.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.