Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

Corrupt size cluster shared volume (CSV) on Hyper-V 2012 R2

Posted on 2014-08-18
4
Medium Priority
?
2,284 Views
Last Modified: 2014-11-12
Unable to use extra disk space on my cluster shared volume (CSV) after extending.

Setup:
EMC storage: VNXe3150
6 physical servers configured in a Hyper-V 2012 R2 cluster
connected to the storage through iSCSI
CSV is formatted with REFS

Events leading up to the issue:
CSV initially was 1 TB
I was copying data to one of the VMs (using robocopy)
more data then I expected
I had assigned more space to the VM then was actually available
During the robocopy process, suddenly all VMs on the cluster grinded to a halt

My response:
* Increased LUN size on EMC from 1024GB to 1950GB
* Sorted out which hyper-v host was in control of the CSV
* RDPed to that particular hyper-v host
* Opened an elevated CMD
* started 'diskpart':
screen shot diskpartAs you can see, I tried to 'extend' the volume. Executing the extend command returned an error but also reported a successfully extended volume.

The best way to explain the current problem is by a screen shot of windows disk manager:
screen shot disk managerJudging from the information in the top panel, the capacity of the volume is still the old size of 1023.87 GB
But the bottom panel shows a disk and volume size of 1996.67 GB
Most importantly the newly claimed disk space is assigned from the storage but unavailable to servers on the cluster!
Fortunately the data on CSV (Hyper-V disk images) is still accessible. So no data has been lost (yet).

Tried (but failed) so far:
Chkdsk /scan (finds nothing wrong)
Chkdsk /fix
Defrag
Repeating the diskpart process (reports not enough space available to extend)
shrink volume (unrecognized FS error, threatened to remove all data when proceeding)
Re-attaching in iSCSI initiator
Re-scan in disk manager

PLAN B:
If I'm unable to 'fix' the problem within the next few hours I'll be forced to take these (time consuming) steps:
* backup data from CSV
* Destroy current volume
* Re-create larger volume
* Restore data
* resume migration

Circumstances / considerations:
* Not enough space on local media at the customer to backup the CSV data
* Customer is a high school of with 2000 students (end-users)
* Their holiday ends and school starts within next 4 days (no pressure)

Does anybody have an alternative suggestion to my plan B?

Thank you in advance!
0
Comment
Question by:Joost Kuin
  • 3
4 Comments
 
LVL 39

Expert Comment

by:Mahesh
ID: 40269756
I think you should turn off all VMs placed on CSV

Then remove volume from csv

Then check from disk manager and command prompt if its showing proper size

Reboot all Hyper-V servers one by one

Readd the volume as CSV in Hyper-V cluster and check.

This should not imapct your data stored on CSV, you might want to take CSV volume backup 1st
0
 
LVL 1

Author Comment

by:Joost Kuin
ID: 40269901
Hi Mahesh,

Thanx for replying! Your suggestion sounds very do-able and at worse pretty harmless to the data.
I'm currently backing up the data to a NAS that my colleague dropped on-site. It is estimated to finish in about 3 hours from now. I'll be sure to give your suggestion a go before I waste more time restoring the data to a new volume.

PS: can you elaborate on why you think that this might work? Have you had a similar experience?

Kind regards,
Joost
0
 
LVL 1

Accepted Solution

by:
Joost Kuin earned 0 total points
ID: 40270437
@Mahesh:
I tried everything you instructed but the difference between the two size values would not be corrected.

With the pressure of getting this project finished on time, Idecided to go with plan B (starting over), which is currently in progress.

I wish that I would have had a bit more time to figure out the issue. ...
0
 
LVL 1

Author Closing Comment

by:Joost Kuin
ID: 40281353
Check my last comment.
I appreciated Mahesh's input but unfortunately his suggestions have not contributed to the solution.

I feel the need for an additional grading option. Just to indicate, that in the interest of time, I have decided to not await further advice and canceled the rescue process.
0

Featured Post

Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell® is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Article by: evilrix
Looking for a way to avoid searching through large data sets for data that doesn't exist? A Bloom Filter might be what you need. This data structure is a probabilistic filter that allows you to avoid unnecessary searches when you know the data defin…
Herein one will find an aggregate of some of my experience building and deploying virtualization stacks both in standalone, clustered Hyper-V, clustered Hyper-V with a Scale-Out File Server (SOFS) backend, and Storage Spaces Direct (S2D).
In this Micro Tutorial viewers will learn how to use Windows Server Backup to create full image of their system. Tutorial shows how to install Windows Server Backup Feature on Windows 2012R2 and how to configure scheduled Bare Metal Recovery backup.…
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
Suggested Courses

580 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question