Link to home
Start Free TrialLog in
Avatar of IT_Crowd
IT_CrowdFlag for Heard Island and McDonald Islands

asked on

SCVMM 2012 Crashes When Trying to Remove a Hyper-V Host

I am trying to remove a server that is no longer part of our environment. We were unable to remove it from VMM properly, before it needed to be reimaged, though. Now, when I try to force remove it 

remove-vmhost <Host_Server_Name> –force

Open in new window


I get the following error message:
remove-vmhost :

Unable to retrieve the result of the job because the
connection to the Virtual Machine Manager management server localhost was
lost. The job will continue running on localhost until it completes. (Error
ID: 1611)

1) Verify that localhost is online and can be remotely accessed from your
computer.
2) Verify that the Virtual Machine Manager service is started on localhost. If
the problem persists, restart the Virtual Machine Manager service.
To find out the result of the job, re-connect to the server, and run the
following cmdlet:
PS> Get-VMMServer localhost | Get-Job | where { $_.ID -eq
"bb62e4bb-43f9-4f04-9afb-f5579a70ae09"}
At line:1 char:1
+ remove-vmhost MYSERVER -force
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ReadError: (:) [Remove-SCVMHost], CarmineExcepti
   on
    + FullyQualifiedErrorId : 1611,Microsoft.SystemCenter.VirtualMachineManage
   r.Cmdlets.RemoveHostCmdlet

The service is started at the time of the command, but it stops after the command fails. I will also post the two errors in the event log... Any ideas?
Avatar of IT_Crowd
IT_Crowd
Flag of Heard Island and McDonald Islands image

ASKER

Log Name:      Application
Source:        Application Error
Date:          4/24/2014 3:05:22 PM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      SAIMAD59v.strand.com
Description:
Faulting application name: vmmservice.exe, version: 3.1.6027.0, time stamp: 0x51bf69cb
Faulting module name: KERNELBASE.dll, version: 6.2.9200.16857, time stamp: 0x530e784b
Exception code: 0xe0434352
Fault offset: 0x0000000000047b8c
Faulting process id: 0x6e8
Faulting application start time: 0x01cf5ff72ba67631
Faulting application path: C:\Program Files\Microsoft System Center 2012\Virtual Machine Manager\Bin\vmmservice.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: c3c2d006-cbeb-11e3-9410-00155d0b2204
Faulting package full name:
Faulting package-relative application ID:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-04-24T20:05:22.000000000Z" />
    <EventRecordID>170976</EventRecordID>
    <Channel>Application</Channel>
    <Computer>SAIMAD59v.strand.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data>vmmservice.exe</Data>
    <Data>3.1.6027.0</Data>
    <Data>51bf69cb</Data>
    <Data>KERNELBASE.dll</Data>
    <Data>6.2.9200.16857</Data>
    <Data>530e784b</Data>
    <Data>e0434352</Data>
    <Data>0000000000047b8c</Data>
    <Data>6e8</Data>
    <Data>01cf5ff72ba67631</Data>
    <Data>C:\Program Files\Microsoft System Center 2012\Virtual Machine Manager\Bin\vmmservice.exe</Data>
    <Data>C:\Windows\system32\KERNELBASE.dll</Data>
    <Data>c3c2d006-cbeb-11e3-9410-00155d0b2204</Data>
    <Data>
    </Data>
    <Data>
    </Data>
  </EventData>
</Event>
Avatar of IT_Crowd
IT_Crowd
Flag of Heard Island and McDonald Islands image

ASKER

Log Name:      Application
Source:        .NET Runtime
Date:          4/24/2014 3:05:22 PM
Event ID:      1026
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      SAIMAD59v.strand.com
Description:
Application: vmmservice.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: Microsoft.VirtualManager.DB.CarmineSqlException
Stack:
   at Microsoft.VirtualManager.Engine.TaskRepository.Task`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].SubtaskRun(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name=".NET Runtime" />
    <EventID Qualifiers="0">1026</EventID>
    <Level>2</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-04-24T20:05:22.000000000Z" />
    <EventRecordID>170975</EventRecordID>
    <Channel>Application</Channel>
    <Computer>SAIMAD59v.strand.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Application: vmmservice.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: Microsoft.VirtualManager.DB.CarmineSqlException
Stack:
   at Microsoft.VirtualManager.Engine.TaskRepository.Task`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].SubtaskRun(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
</Data>
  </EventData>
</Event>
Avatar of Shabarinath TR
Shabarinath TR
Flag of India image
I had seen a similar issue with my SCVMM 2012 instance. Couldnt re-collect if the vmm was crashing or not. However, one Host machine was stuck in the removal process. Had a case with Microsoft for this issue and after several tracing, it was identified that the DB had some corruption which is causing the issue.

The steps which I got for manual deletion from DB is as below. Please note - its a try and make sure you have proper backup. In case of an issues, you should be able to revert back.

1.  First, stop the VMMservice.  
2.  Then, take a backup of the existing DB using SQL Management studio. (Right click on DB name, choose tasks, click on Back up, chose a full backup).  
3.  Now, right click on table tbl_ADHC_Host and script table as > Select to > New query window  
4.  Execute now the query and you will get the list of Hosts  
5.  In there, start pasting the commands below one by one and verifying they executed correctly. (To execute, you can highlight the command and press F5 or press the “!” symbol).

Change now the the IDs within the below commands and run them. Below it’s a 2 Node Cluster example.

Examples:
DELETE FROM tbl_ADHC_HostNetworkAdapter WHERE (HostID = '85099616-fdc8-4f85-a702-979a4251f050')
DELETE FROM tbl_ADHC_HostNetworkAdapter WHERE HostID='8cfff5ad-0990-4a92-bd6d-4d24190e5d5a '
DELETE FROM tbl_ADHC_VirtualNetwork WHERE (HostID = '85099616-fdc8-4f85-a702-979a4251f050')
DELETE FROM tbl_ADHC_VirtualNetwork WHERE (HostID = '8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ')
DELETE FROM tbl_ADHC_HostVolume WHERE (HostID = '85099616-fdc8-4f85-a702-979a4251f050') OR (HostID = '8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ')
DELETE FROM tbl_ADHC_HostDisk WHERE (HostID = '85099616-fdc8-4f85-a702-979a4251f050') OR (HostID = '8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ')
DELETE FROM tbl_WLC_PhysicalObject WHERE (HostId = '85099616-fdc8-4f85-a702-979a4251f050') OR (HostId = '8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ')
DELETE FROM tbl_WLC_VObject WHERE (HostId = '85099616-fdc8-4f85-a702-979a4251f050') OR  (HostId = '8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ')
DELETE FROM tbl_ADHC_Host WHERE (HostID = '85099616-fdc8-4f85-a702-979a4251f050') OR  (HostID = '8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ')
DELETE FROM tbl_ADHC_ClusterDisk WHERE (ClusterID = '1148853c-cde3-45d2-82ee-bb04259d6a79')
DELETE FROM tbl_ADHC_HostCluster WHERE (ClusterID = '1148853c-cde3-45d2-82ee-bb04259d6a79')
DELETE FROM tbl_ADHC_AgentServerRelation WHERE (AgentServerID = '648e977e-d10e-40b6-aca2-d9d2dc63b790')
DELETE FROM tbl_ADHC_AgentServer WHERE (ComputerName = 'Server_Node1')
DELETE FROM tbl_ADHC_AgentServer WHERE (ComputerName = 'Server_Node2')  

6.  Now start the vmmservice and open the VMM console.
7. Remove the scvmm agents from Nodes and restart them


Another option is on this blog.

http://helshabini.wordpress.com/2011/11/14/manually-remove-hyper-v-host-cluster-from-scvmm-2012-database/

Good luck !
ASKER CERTIFIED SOLUTION
Avatar of IT_Crowd
IT_Crowd
Flag of Heard Island and McDonald Islands image
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of IT_Crowd
IT_Crowd
Flag of Heard Island and McDonald Islands image

ASKER

Figured it out myself.