Difference in Storage Usage from ShareFile Reporting to Array Reporting
I have an issue where I'm seeing a discrepancy between what ShareFile is stating the storage consumption is to what is actually being consumed on our storage array. ShareFile states 4 TB of usage while the storage array is reporting 27 TB.
Citrix support has told me that our version, 4.1 has an issue with deletion and suggested we upgrade to 5 to which we are working on new servers to replace the old. We have been through 3 different versions from 3.0 upwards.
What concerns me is looking at the array I see multiple files of the same size but none of them equal to the highest size file we have shared. It appears to me that the files are being broken up into pieces during the encryption process.
I'm new to ShareFile but not storage; my apologies if this is scatter brained...
Has ShareFile always had an issue with file deletion prior to 5.0?
Could ShareFile encryption be causing the overhead due to splitting files?
The NTFS cluster size is set for 15K and not 4K; this would cause overhead but could it account for that large of a difference?
Is it possible that the NTFS cluster size and file splitting with encryption is causing it?
Is there a way to correlate the file ID to a set of split files?
Any help is greatly appreciated. Please query me for additional information as needed. 23 TB is a considerable amount of space to be waisted and has an associated cost.
Citrix support has told me that our version, 4.1 has an issue with deletion and suggested we upgrade to 5 to which we are working on new servers to replace the old. We have been through 3 different versions from 3.0 upwards.
What concerns me is looking at the array I see multiple files of the same size but none of them equal to the highest size file we have shared. It appears to me that the files are being broken up into pieces during the encryption process.
I'm new to ShareFile but not storage; my apologies if this is scatter brained...
Has ShareFile always had an issue with file deletion prior to 5.0?
Could ShareFile encryption be causing the overhead due to splitting files?
The NTFS cluster size is set for 15K and not 4K; this would cause overhead but could it account for that large of a difference?
Is it possible that the NTFS cluster size and file splitting with encryption is causing it?
Is there a way to correlate the file ID to a set of split files?
Any help is greatly appreciated. Please query me for additional information as needed. 23 TB is a considerable amount of space to be waisted and has an associated cost.
What storage array are you using and is it thin or thick provisioned?
ASKER
It's on an Isilon array which is thin provisioned. Are you thinking it's a zero space reclaim?
*** update ***
There is a quota with an advisory and hard limit set.
*** update ***
There is a quota with an advisory and hard limit set.
Yes, I think it is probably unused space not being zeroed out, sdelete from sysinternals or similar can fill the free space with nulls. I don't know whether Insilon automatically reclaim zeroed out space or if you have to run some command to get it back on your SAN.
Usage may still show as higher on the SAN if the stats are raw usage without taking parity into account.
Usage may still show as higher on the SAN if the stats are raw usage without taking parity into account.
ASKER
It was a good thought; however, I ran a TreeSize against the target and it's showing the space as consumed by files.
Guess you'll have to wait until Sharefile is upgraded for the fix. I suspect the individual file size difference isn't Sharefile breaking them up but compressing them during encryption, after all it's hardly any more effort to do both at once rather than just encrypting. Tape drives do that, but they use an ASIC for it rather than the CPU.
ASKER
Just an update...
I've found that yes, the files are being split into chunks. The log files actually state "chunk = yes". I don't know if this is a static or dynamic chunk. What I've noticed is anything over 512 MB is being split into those size chunks.
I've also upgraded to 5.3.1 but yet to see any relief from the storage perspective.
I'll continue to update events progress.
Anyone with additional information, please chime in!
I've found that yes, the files are being split into chunks. The log files actually state "chunk = yes". I don't know if this is a static or dynamic chunk. What I've noticed is anything over 512 MB is being split into those size chunks.
I've also upgraded to 5.3.1 but yet to see any relief from the storage perspective.
I'll continue to update events progress.
Anyone with additional information, please chime in!
ASKER
It appears I've found the issue. While doing some debugging with high log levels for the FileCleanSvc I noticed that the "persistentstorage" location wasn't being queued at all for any type of deletion. Searching for this I ran across an article here:
https://support.citrix.com/article/CTX233141
It was exactly my problem in that the list of queues was set with no value. I've set the value as per the instructions above and adjusted the time intervals and alas my storage usage is now dropping.
https://support.citrix.com/article/CTX233141
It was exactly my problem in that the list of queues was set with no value. I've set the value as per the instructions above and adjusted the time intervals and alas my storage usage is now dropping.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.