Corrupt size cluster shared volume (CSV) on Hyper-V 2012 R2

Unable to use extra disk space on my cluster shared volume (CSV) after extending.

Setup:
EMC storage: VNXe3150
6 physical servers configured in a Hyper-V 2012 R2 cluster
connected to the storage through iSCSI
CSV is formatted with REFS

Events leading up to the issue:
CSV initially was 1 TB
I was copying data to one of the VMs (using robocopy)
more data then I expected
I had assigned more space to the VM then was actually available
During the robocopy process, suddenly all VMs on the cluster grinded to a halt

My response:
* Increased LUN size on EMC from 1024GB to 1950GB
* Sorted out which hyper-v host was in control of the CSV
* RDPed to that particular hyper-v host
* Opened an elevated CMD
* started 'diskpart':
screen shot diskpartAs you can see, I tried to 'extend' the volume. Executing the extend command returned an error but also reported a successfully extended volume.

The best way to explain the current problem is by a screen shot of windows disk manager:
screen shot disk managerJudging from the information in the top panel, the capacity of the volume is still the old size of 1023.87 GB
But the bottom panel shows a disk and volume size of 1996.67 GB
Most importantly the newly claimed disk space is assigned from the storage but unavailable to servers on the cluster!
Fortunately the data on CSV (Hyper-V disk images) is still accessible. So no data has been lost (yet).

Tried (but failed) so far:
Chkdsk /scan (finds nothing wrong)
Chkdsk /fix
Defrag
Repeating the diskpart process (reports not enough space available to extend)
shrink volume (unrecognized FS error, threatened to remove all data when proceeding)
Re-attaching in iSCSI initiator
Re-scan in disk manager

PLAN B:
If I'm unable to 'fix' the problem within the next few hours I'll be forced to take these (time consuming) steps:
* backup data from CSV
* Destroy current volume
* Re-create larger volume
* Restore data
* resume migration

Circumstances / considerations:
* Not enough space on local media at the customer to backup the CSV data
* Customer is a high school of with 2000 students (end-users)
* Their holiday ends and school starts within next 4 days (no pressure)

Does anybody have an alternative suggestion to my plan B?

Thank you in advance!
LVL 2
Joost KuinSenior system administratorAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

MaheshArchitectCommented:
I think you should turn off all VMs placed on CSV

Then remove volume from csv

Then check from disk manager and command prompt if its showing proper size

Reboot all Hyper-V servers one by one

Readd the volume as CSV in Hyper-V cluster and check.

This should not imapct your data stored on CSV, you might want to take CSV volume backup 1st
0
Joost KuinSenior system administratorAuthor Commented:
Hi Mahesh,

Thanx for replying! Your suggestion sounds very do-able and at worse pretty harmless to the data.
I'm currently backing up the data to a NAS that my colleague dropped on-site. It is estimated to finish in about 3 hours from now. I'll be sure to give your suggestion a go before I waste more time restoring the data to a new volume.

PS: can you elaborate on why you think that this might work? Have you had a similar experience?

Kind regards,
Joost
0
Joost KuinSenior system administratorAuthor Commented:
@Mahesh:
I tried everything you instructed but the difference between the two size values would not be corrected.

With the pressure of getting this project finished on time, Idecided to go with plan B (starting over), which is currently in progress.

I wish that I would have had a bit more time to figure out the issue. ...
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Joost KuinSenior system administratorAuthor Commented:
Check my last comment.
I appreciated Mahesh's input but unfortunately his suggestions have not contributed to the solution.

I feel the need for an additional grading option. Just to indicate, that in the interest of time, I have decided to not await further advice and canceled the rescue process.
0
Marc BauerCommented:
I had the same issue. Just increased the volume by 1GB, extended again and it recovered.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.