Why the CSV volume size shows double than the actual in S2D cluster?

S2D CSV volumes has only 20 TB but in Failover cluster manager it shows 40 TB been consumed, Any idea it is windows 2016 with 4 nodes cluster with 2SSD and 10 SCSI on each nodes. The total size it shows 140 TB of disk size and used is 40 TB but actually it is only 20 TB. In capacity tier there is no space left and performance tier there is space but unable to create any new CSV volume. Any idea what caused the issue?

4 SSD x 745 GB = 3 TB for caching per node

10 SAS x 3.46 TB = 34 TB per per node

Total nodes are 4 x 37 TB = 146 TB.

In failover cluster manager it shows 146 TB but actually it doesn't allows 70 TB in total including both capacity and performance tier. But when checking the data size and consumable amount of space should be less but it actually allows even less than expected. Doesn't makes any sense again just for 30 TB we have S2D solution.

Is there any way to break the performance tier and assign only to the capacity tier, as a layman we expect hyperconverged solution but actually speaking it doesn't even give what is expected. Is there any way we can convert or make 70 TB usable. Client is not happy with the storage part. Please share your inputs.
LVL 12
Ganesh Kumar ASr Infrastructure SpecialistAsked:
Who is Participating?
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
Mirror Accelerated Parity yields 50% of available storage less a small reserve that S2D sets aside for its own uses.
0
 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
What is the resiliency setup for the volumes? Mirror or Mirror Accelerated Parity (LRC)?
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
There are two options it shows, Performance Tier (Mirror), Capacity Tier with Dual Parity. Incase if i want to change the parity how i will change the tier. Is it possible in Windows 2016 S2D
0
Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Get-StoragePool | FL
Get-VirtualDisk | FL
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
Please refer the attachment too.

==================================
Get-Storagepool | fl


ObjectId                          : {1}\\S2DCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_StoragePool.Obje
                                    ctId="{185d51fe-cb40-11e7-9244-806e6f6e6963}:SP:{185d51fd-cb40-11e7-9244-806e6f6e69
                                    63}"
PassThroughClass                  :
PassThroughIds                    :
PassThroughNamespace              :
PassThroughServer                 :
UniqueId                          : {185d51fd-cb40-11e7-9244-806e6f6e6963}
AllocatedSize                     : 120419795337216
ClearOnDeallocate                 : False
EnclosureAwareDefault             : False
FaultDomainAwarenessDefault       : PhysicalDisk
FriendlyName                      : Primordial
HealthStatus                      : Healthy
IsClustered                       : False
IsPowerProtected                  : False
IsPrimordial                      : True
IsReadOnly                        : False
LogicalSectorSize                 :
MediaTypeDefault                  : Unspecified
Name                              :
OperationalStatus                 : OK
OtherOperationalStatusDescription :
OtherUsageDescription             :
PhysicalSectorSize                :
ProvisioningTypeDefault           : Fixed
ReadOnlyReason                    : None
RepairPolicy                      : Parallel
ResiliencySettingNameDefault      : Mirror
RetireMissingPhysicalDisks        : Auto
Size                              : 173433137463296
SupportedProvisioningTypes        : {Thin, Fixed}
SupportsDeduplication             : True
ThinProvisioningAlertThresholds   : {70}
Usage                             : Other
Version                           : Windows Server 2016
WriteCacheSizeDefault             : Auto
WriteCacheSizeMax                 : 18446744073709551614
WriteCacheSizeMin                 : 0
PSComputerName                    :

ObjectId                          : {1}\\S2DCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_StoragePool.Obje
                                    ctId="{68efa95b-8b4b-4547-a25b-0b2f1b6b10e6}:SP:{db05af96-c7f1-46cf-bef2-6f1511eb11
                                    c1}"
PassThroughClass                  :
PassThroughIds                    :
PassThroughNamespace              :
PassThroughServer                 :
UniqueId                          : {db05af96-c7f1-46cf-bef2-6f1511eb11c1}
AllocatedSize                     : 120419795337216
ClearOnDeallocate                 : False
EnclosureAwareDefault             : False
FaultDomainAwarenessDefault       : PhysicalDisk
FriendlyName                      : Primordial
HealthStatus                      : Healthy
IsClustered                       : True
IsPowerProtected                  : False
IsPrimordial                      : True
IsReadOnly                        : False
LogicalSectorSize                 :
MediaTypeDefault                  : Unspecified
Name                              :
OperationalStatus                 : OK
OtherOperationalStatusDescription :
OtherUsageDescription             :
PhysicalSectorSize                :
ProvisioningTypeDefault           : Fixed
ReadOnlyReason                    : None
RepairPolicy                      : Parallel
ResiliencySettingNameDefault      : Mirror
RetireMissingPhysicalDisks        : Auto
Size                              : 173433137463296
SupportedProvisioningTypes        : Fixed
SupportsDeduplication             : True
ThinProvisioningAlertThresholds   : {70}
Usage                             : Other
Version                           : Windows Server 2016
WriteCacheSizeDefault             : Auto
WriteCacheSizeMax                 : 18446744073709551614
WriteCacheSizeMin                 : 0
PSComputerName                    :

ObjectId                          : {1}\\S2DCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_StoragePool.Obje
                                    ctId="{68efa95b-8b4b-4547-a25b-0b2f1b6b10e6}:SP:{e310a53e-bd50-4d0f-8ac6-45c0126314
                                    b0}"
PassThroughClass                  :
PassThroughIds                    :
PassThroughNamespace              :
PassThroughServer                 :
UniqueId                          : {e310a53e-bd50-4d0f-8ac6-45c0126314b0}
AllocatedSize                     : 43997913415680
ClearOnDeallocate                 : False
EnclosureAwareDefault             : False
FaultDomainAwarenessDefault       : StorageScaleUnit
FriendlyName                      : S2D on S2DCluster
HealthStatus                      : Healthy
IsClustered                       : True
IsPowerProtected                  : False
IsPrimordial                      : False
IsReadOnly                        : False
LogicalSectorSize                 : 4096
MediaTypeDefault                  : Unspecified
Name                              :
OperationalStatus                 : OK
OtherOperationalStatusDescription :
OtherUsageDescription             : Reserved for S2D
PhysicalSectorSize                : 4096
ProvisioningTypeDefault           : Fixed
ReadOnlyReason                    : None
RepairPolicy                      : Parallel
ResiliencySettingNameDefault      : Mirror
RetireMissingPhysicalDisks        : Never
Size                              : 160528697655296
SupportedProvisioningTypes        : Fixed
SupportsDeduplication             : True
ThinProvisioningAlertThresholds   : {70}
Usage                             : Other
Version                           : Windows Server 2016
WriteCacheSizeDefault             : 0
WriteCacheSizeMax                 : 18446744073709551614
WriteCacheSizeMin                 : 0
PSComputerName                    :

==================================================

Get-virtualdisk | fl



ObjectId                          : {1}\\S2DCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_VirtualDisk.Obje
                                    ctId="{68efa95b-8b4b-4547-a25b-0b2f1b6b10e6}:VD:{e310a53e-bd50-4d0f-8ac6-45c0126314
                                    b0}{15519cbf-232f-4689-932d-24e272501c5d}"
PassThroughClass                  :
PassThroughIds                    :
PassThroughNamespace              :
PassThroughServer                 :
UniqueId                          : BF9C51152F238946932D24E272501C5D
Access                            : Read/Write
AllocatedSize                     : 16492674416640
AllocationUnitSize                :
ColumnIsolation                   :
DetachedReason                    : None
FaultDomainAwareness              :
FootprintOnPool                   : 32986154139648
FriendlyName                      : Data
HealthStatus                      : Warning
Interleave                        :
IsDeduplicationEnabled            : False
IsEnclosureAware                  :
IsManualAttach                    : True
IsSnapshot                        : False
IsTiered                          : True
LogicalSectorSize                 : 4096
MediaType                         :
Name                              :
NameFormat                        :
NumberOfAvailableCopies           :
NumberOfColumns                   :
NumberOfDataCopies                :
NumberOfGroups                    :
OperationalStatus                 : Degraded
OtherOperationalStatusDescription :
OtherUsageDescription             :  
ParityLayout                      :
PhysicalDiskRedundancy            :
PhysicalSectorSize                : 4096
ProvisioningType                  :
ReadCacheSize                     : 0
RequestNoSinglePointOfFailure     : False
ResiliencySettingName             :
Size                              : 16492674416640
UniqueIdFormat                    : Vendor Specific
UniqueIdFormatDescription         :
Usage                             : Other
WriteCacheSize                    : 33554432
PSComputerName                    :

ObjectId                          : {1}\\S2DCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_VirtualDisk.Obje
                                    ctId="{68efa95b-8b4b-4547-a25b-0b2f1b6b10e6}:VD:{e310a53e-bd50-4d0f-8ac6-45c0126314
                                    b0}{4f5bf386-ee22-429b-b9f8-16311226eac1}"
PassThroughClass                  :
PassThroughIds                    :
PassThroughNamespace              :
PassThroughServer                 :
UniqueId                          : 86F35B4F22EE9B42B9F816311226EAC1
Access                            : Read/Write
AllocatedSize                     : 2199023255552
AllocationUnitSize                :
ColumnIsolation                   :
DetachedReason                    : None
FaultDomainAwareness              :
FootprintOnPool                   : 4398851817472
FriendlyName                      : File Servers Disk Data
HealthStatus                      : Warning
Interleave                        :
IsDeduplicationEnabled            : False
IsEnclosureAware                  :
IsManualAttach                    : True
IsSnapshot                        : False
IsTiered                          : True
LogicalSectorSize                 : 4096
MediaType                         :
Name                              :
NameFormat                        :
NumberOfAvailableCopies           :
NumberOfColumns                   :
NumberOfDataCopies                :
NumberOfGroups                    :
OperationalStatus                 : Degraded
OtherOperationalStatusDescription :
OtherUsageDescription             :  
ParityLayout                      :
PhysicalDiskRedundancy            :
PhysicalSectorSize                : 4096
ProvisioningType                  :
ReadCacheSize                     : 0
RequestNoSinglePointOfFailure     : False
ResiliencySettingName             :
Size                              : 2199023255552
UniqueIdFormat                    : Vendor Specific
UniqueIdFormatDescription         :
Usage                             : Other
WriteCacheSize                    : 33554432
PSComputerName                    :

ObjectId                          : {1}\\S2DCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_VirtualDisk.Obje
                                    ctId="{68efa95b-8b4b-4547-a25b-0b2f1b6b10e6}:VD:{e310a53e-bd50-4d0f-8ac6-45c0126314
                                    b0}{b5e62b9a-5aa7-4875-881a-b738c0c1989f}"
PassThroughClass                  :
PassThroughIds                    :
PassThroughNamespace              :
PassThroughServer                 :
UniqueId                          : 9A2BE6B5A75A7548881AB738C0C1989F
Access                            : Read/Write
AllocatedSize                     : 3298534883328
AllocationUnitSize                :
ColumnIsolation                   :
DetachedReason                    : None
FaultDomainAwareness              :
FootprintOnPool                   : 6597875073024
FriendlyName                      : OS
HealthStatus                      : Warning
Interleave                        :
IsDeduplicationEnabled            : False
IsEnclosureAware                  :
IsManualAttach                    : True
IsSnapshot                        : False
IsTiered                          : True
LogicalSectorSize                 : 4096
MediaType                         :
Name                              :
NameFormat                        :
NumberOfAvailableCopies           :
NumberOfColumns                   :
NumberOfDataCopies                :
NumberOfGroups                    :
OperationalStatus                 : Degraded
OtherOperationalStatusDescription :
OtherUsageDescription             :  
ParityLayout                      :
PhysicalDiskRedundancy            :
PhysicalSectorSize                : 4096
ProvisioningType                  :
ReadCacheSize                     : 0
RequestNoSinglePointOfFailure     : False
ResiliencySettingName             :
Size                              : 3298534883328
UniqueIdFormat                    : Vendor Specific
UniqueIdFormatDescription         :
Usage                             : Other
WriteCacheSize                    : 33554432
PSComputerName                    :
s2d.JPG
0
 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Please post your PowerShell used to configure the virtual drives New-Volume ?
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
We didn't try anything in the powershell, we used FCM to create virtual disk pool, clustered shared volumes and vms placed on top of it.

PS C:\Users\s2d\Desktop> Get-StoragePool -FriendlyName S2* | Get-ResiliencySetting

Name   NumberOfDataCopies PhysicalDiskRedundancy NumberOfColumns Interleave NumberOfGroups
----   ------------------ ---------------------- --------------- ---------- --------------
Simple      1                   0                      Auto           262144     1
Mirror       3                  2                      Auto            262144     1
Parity        1                  2                      Auto            262144     Auto
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
Any idea, how to convert or change the tier and increase more space with HA. Now we have mail servers, and other critical application servers running on the S2D cluster. We need to know how can we reclaim the space and this was recommended by the vendor so we did it and right now we need some recommendation to change and suggest what mode we should change, probably  we will think about moving the volumes from one to another. Please help!
0
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
The general rule of thumb for S2D is # Nodes x 2 = # of CSVs to run on the cluster.

At four nodes, 50% is about the best one can do. More nodes with Mirror Accelerated Parity can yield around 80% with all-flash setups. 3-Way mirror is always 33% while 2-Way Mirror is 50%.

Storage efficiency in S2D is explained here: Fault tolerance and storage efficiency in Storage Spaces Direct.
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
Is there any way to convert from 3-way mirror to 2 way mirror to gain 50% of space, The disk setup already been described in the first post. Since we are running high intensive application, is it recommended to remove the CSV volume and use only the direct disk space, will the VM failover will be similar like one in the CSV volumes?
0
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
CSVs are required.

If there is free space in the pool create a 2-way mirror virtual disk and migrate existing storage in. Delete the 3-way mirror. Repeat.

NOTE: Make sure there is at least 2x Capacity + 250GB free in the Pool and Retire Disk is sent to ENABLED to allow the virtual disks that are degraded by a disk loss get rebuilt into free pool space. 2-Way Mirror means 1 disk or 1 node failure and then DANGER.
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
Strangely when trying to create a pool on the mirrored it is not allowing and throwing error like below snapshot. I am unable to create / add additional disk in the existing pool. Any help?
s2derror01.JPG
s2derror02.JPG
s2derror03.JPG
0
 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
There's no free space in the Capacity Tier.
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
Yes there is no free space, is it because of that i am unable to create the pool. When i did before i was able to create on both pool. Any help will be appreciated on the same. Please advice.
0
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
Ganesh,

As mentioned earlier, the only way out of this mess is to back everything up, delete the CSVs, set them up again, and then restore the VMs.

An export and import process for the VMs would also be another method. A repository for the exports would be required that is large enough to hold all of the data.
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
The issue is there is no storage to store the data to move the VMs, i need to use any one of the node to move all VMs and then remove the CSV volume and recreate it. Is there any way to repair the CSV volume or Storage spaces direct to fix the CSV volume.

i believe it is due to the node1 connectivity failure, To correct it i tried removed from the cluster and reset the disk, set canpool eq true and disks shows healthy. But after adding the node1 to the cluster, the disks shows unhealthy with transient error.
0
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
That was a serious error. Now there is a discrepancy between the cluster and that node.

Storage Spaces writes metadata _to the disks_ so now there's a real mess there. :(

At this point, either hire a professional or wipe and reload the setup according to the desired free space outcome.

Are the workloads for a tenant environment or environments?

And, if I haven't already mentioned this: Make sure there is enough free space in the Pool to allow for a lost disk to be rebuilt into it when using 2-Way Mirror. Otherwise, one more disk dies and boom it's all gone.
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
No there is no free space available under mirrored volume it shows 0 space available, the issue is 20TB  been actually used CSV consumes double so 40 TB gone, In FSM it shows 106 Free space available. There are workloads for the customer with 15 critical applicaiton VMS's running on this S2D. Could this because of the Access Edge configuration on the port or trunk mode causing the issue not sure.
0
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
The error made has compromised the setup. It's time to make a tough decision as per the choices given above.
0
 
Ganesh Kumar ASr Infrastructure SpecialistAuthor Commented:
Though the issue is not resolved but got some idea, will raise the case with Microsoft. The S2D was created based on the Lenovo solution as per their document.  I also have SCVMM not sure if that help to fix the issue. Let us see.
0
 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Please keep us posted.
1
 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Any update?

One thought: The disks indicating DISCONNECTED for the node that had them cleaned could have them RETIRED on the cluster side. Then, make sure the disks themselves on the evicted node are cleaned properly before rejoining the cluster.

This is the script to clean the disks on the evicted node:

icm (Get-Cluster -Name <cluster or node name> | Get-ClusterNode) {

Update-StorageProviderCache

Get-StoragePool | ? IsPrimordial -eq $false | Set-StoragePool -IsReadOnly:$false -ErrorAction SilentlyContinue

Get-StoragePool | ? IsPrimordial -eq $false | Get-VirtualDisk | Remove-VirtualDisk -Confirm:$false -ErrorAction SilentlyContinue

Get-StoragePool | ? IsPrimordial -eq $false | Remove-StoragePool -Confirm:$false -ErrorAction SilentlyContinue

Get-PhysicalDisk | Reset-PhysicalDisk -ErrorAction SilentlyContinue

Get-Disk | ? Number -ne $null | ? IsBoot -ne $true | ? IsSystem -ne $true | ? PartitionStyle -ne RAW | % {

$_ | Set-Disk -isoffline:$false

$_ | Set-Disk -isreadonly:$false

$_ | Clear-Disk -RemoveData -RemoveOEM -Confirm:$false

$_ | Set-Disk -isreadonly:$true

$_ | Set-Disk -isoffline:$true

}

Get-Disk |? Number -ne $null |? IsBoot -ne $true |? IsSystem -ne $true |? PartitionStyle -eq RAW | Group -NoElement -Property FriendlyName

} | Sort -Property PsComputerName,Count

Open in new window


Do not, and I repeat DO NOT, run this script on any in production Storage Spaces Direct Node!

DATA LOSS WILL OCCUR!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.