asked on

How to install KB3045318 patch on AlwaysOn two node multi-subnet failover cluster

Problem: KB3045318 patch fails on AlwaysOn two node multi-subnet failover cluster

Environment:

Two node multi-subnet failover cluster (no shared storage)
Sql Server 2012 SP1
Windows Server 2008 R2 SP1

Prior to initiating the windows update process (it is set to manual), I paused the node (the node was in secondary status). From all I have read, this is the correct procedure for applying sql patches and/or service packs to instances participating in an AlwaysOn cluster of this type.

Error:
The machine is clustered, but the cluster is not online or cannot be accessed from one of its nodes. To continue determine why the cluster is not online and rerun setup instead of rerunning the rule since the rule can no longer detect a cluster environment correctly.
I located instructions for running a service pack using a switch that will bypass this check. The problem here is that this KB is running from windows update and I am not aware of an override option to the check. Nor am I sure that I should be overriding a check placed into the KB by Microsoft.

From the Bootstrap Log’s ‘SystemConfigurationCheck_Report’ file

From the Summary text file:
Overall summary:
Final result: The patch installer has failed to update the following instance:. To determine the reason for failure, review the log files.
Exit code (Decimal): -2067919922
Exit facility code: 1214
Exit error code: 3022
Exit message: The machine is clustered, but the cluster is not online or cannot be accessed from one of its nodes. To continue determine why the cluster is not online and rerun setup instead of rerunning the rule since the rule can no longer detect a cluster environment correctly.
Start time: 2015-10-20 10:37:21
End time: 2015-10-20 10:40:01
Requested action: Patch

From ‘Detail’ file

(01) 2015-10-20 10:39:06 Slp: Initializing rule : Not clustered or the cluster service is up and online.
(01) 2015-10-20 10:39:06 Slp: Rule is will be executed : True
(01) 2015-10-20 10:39:06 Slp: Init rule target object: Microsoft.SqlServer.Configuration.Cluster.Rules.ClusterServiceFacet
(01) 2015-10-20 10:39:12 Slp: Detected a patch / remove patch operation on a paused node.
(01) 2015-10-20 10:39:12 Slp: Sco: Attempting to create base registry key HKEY_LOCAL_MACHINE, machine
(01) 2015-10-20 10:39:12 Slp: Sco: Attempting to open registry subkey Software\Microsoft\Windows\CurrentVersion\ClusterAwareUpdating
(01) 2015-10-20 10:39:12 Slp: Rule 'Cluster_IsOnlineIfClustered' detection result: Is Cluster Online Results = False; Is Cluster Verfication complete = False; Verfication Has Warnings = True; Verification Has Errors = True; on Machine XXXXYYYY
(01) 2015-10-20 10:39:12 Slp: Evaluating rule : Cluster_IsOnlineIfClustered
(01) 2015-10-20 10:39:12 Slp: Rule running on machine: XXXXYYYY
(01) 2015-10-20 10:39:12 Slp: Rule evaluation done : Failed
(01) 2015-10-20 10:39:12 Slp: Rule evaluation message: The machine is clustered, but the cluster is not online or cannot be accessed from one of its nodes. To continue determine why the cluster is not online and rerun setup instead of rerunning the rule since the rule can no longer detect a cluster environment correctly.
(01) 2015-10-20 10:39:12 Slp: Initializing rule : Cluster group owned by local computer
(01) 2015-10-20 10:39:12 Slp: Rule is will be executed : True
(01) 2015-10-20 10:39:12 Slp: Init rule target object: Microsoft.SqlServer.Configuration.Cluster.Rules.NodeClusterGroupFacet
(01) 2015-10-20 10:39:13 Slp: Rule 'Cluster_IsLocalNodeGroupOwner' detection result: NodeOwnsClusterGroup=False
(01) 2015-10-20 10:39:13 Slp: Evaluating rule : Cluster_IsLocalNodeGroupOwner
(01) 2015-10-20 10:39:13 Slp: Rule running on machine: XXXXYYYY
(01) 2015-10-20 10:39:13 Slp: Rule evaluation done : Succeeded
(01) 2015-10-20 10:39:13 Slp: Rule evaluation message: The local computer is not an owner of an online cluster group for a failover cluster instance that contains the SQL Server service, Analysis Services service or a generic service. The SQL Server instances do not need to be taken offline.
(01) 2015-10-20 10:39:13 Slp: Saved to directory.......C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\20151020_103702\SystemConfigurationCheck_Report.htm
(01) 2015-10-20 10:39:13 Slp: Sco: Attempting to create directory C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\20151020_103702\resources
(01) 2015-10-20 10:39:13 Slp: Sco: Attempting to create directory C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\20151020_103702\resources, security descriptor
(01) 2015-10-20 10:39:13 Slp: Sco: Directory 'C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\20151020_103702\resources' already exists, skipping creation, moving to set security descriptor.
(01) 2015-10-20 10:39:14 Slp: Sco: Attempting to set security descriptor for directory C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\20151020_103702\resources, security descriptor
(01) 2015-10-20 10:39:14 Slp: Sco: SDDL is NULL/empty and overwriteSecurityDescriptor is false, so nothing to do, return.
(01) 2015-10-20 10:39:15 Slp: Rules evaluation report is available now at C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\20151020_103702\SystemConfigurationCheck_Report.htm
(01) 2015-10-20 10:39:15 Slp: Error: Action "Microsoft.SqlServer.Configuration.RulesEngineExtension.RunRulesAction" threw an exception during execution.
(01) 2015-10-20 10:39:15 Slp: Microsoft.SqlServer.Setup.Chainer.Workflow.ActionExecutionException: The machine is clustered, but the cluster is not online or cannot be accessed from one of its nodes. To continue determine why the cluster is not online and rerun setup instead of rerunning the rule since the rule can no longer detect a cluster environment correctly. ---> Microsoft.SqlServer.Configuration.RulesEngineExtension.RulesEngineRuleFailureException: The machine is clustered, but the cluster is not online or cannot be accessed from one of its nodes. To continue determine why the cluster is not online and rerun setup instead of rerunning the rule since the rule can no longer detect a cluster environment correctly.
(01) 2015-10-20 10:39:15 Slp: at Microsoft.SqlServer.Configuration.RulesEngineExtension.RunRulesAction.ExecuteAction(String actionId)
(01) 2015-10-20 10:39:15 Slp: at Microsoft.SqlServer.Chainer.Infrastructure.Action.Execute(String actionId, TextWriter errorStream)
(01) 2015-10-20 10:39:15 Slp: at Microsoft.SqlServer.Setup.Chainer.Workflow.ActionInvocation.ExecuteActionHelper(TextWriter statusStream, ISequencedAction actionToRun, ServiceContainer context)
(01) 2015-10-20 10:39:15 Slp: --- End of inner exception stack trace ---
(01) 2015-10-20 10:39:15 Slp: at Microsoft.SqlServer.Setup.Chainer.Workflow.ActionInvocation.ExecuteActionHelper(TextWriter statusStream, ISequencedAction actionToRun, ServiceContainer context) (01) 2015-10-20 10:39:24 Slp: Received request to add the following file to Watson reporting: C:\Windows\TEMP\tmp5388.tmp
(01) 2015-10-20 10:39:24 Slp: The following is an exception stack listing the exceptions in outermost to innermost order
(01) 2015-10-20 10:39:24 Slp: Inner exceptions are being indented
(01) 2015-10-20 10:39:24 Slp:
(01) 2015-10-20 10:39:24 Slp: Exception type: Microsoft.SqlServer.Configuration.RulesEngineExtension.RulesEngineRuleFailureException
(01) 2015-10-20 10:39:24 Slp: Message:
(01) 2015-10-20 10:39:24 Slp: The machine is clustered, but the cluster is not online or cannot be accessed from one of its nodes. To continue determine why the cluster is not online and rerun setup instead of rerunning the rule since the rule can no longer detect a cluster environment correctly.
(01) 2015-10-20 10:39:24 Slp: HResult : 0x84be0bce
(01) 2015-10-20 10:39:24 Slp: FacilityCode : 1214 (4be)
(01) 2015-10-20 10:39:24 Slp: ErrorCode : 3022 (0bce)
(01) 2015-10-20 10:39:24 Slp: Data:
(01) 2015-10-20 10:39:24 Slp: SQL.Setup.FailureCategory = RuleViolationFailure
(01) 2015-10-20 10:39:24 Slp: DisableWatson = true
(01) 2015-10-20 10:39:24 Slp: Stack:
(01) 2015-10-20 10:39:24 Slp: at Microsoft.SqlServer.Configuration.RulesEngineExtension.RunRulesAction.ExecuteAction(String actionId)
(01) 2015-10-20 10:39:24 Slp: at Microsoft.SqlServer.Chainer.Infrastructure.Action.Execute(String actionId, TextWriter errorStream)
(01) 2015-10-20 10:39:24 Slp: at Microsoft.SqlServer.Setup.Chainer.Workflow.ActionInvocation.ExecuteActionHelper(TextWriter statusStream, ISequencedAction actionToRun, ServiceContainer context)
(01) 2015-10-20 10:39:24 Slp: Collecting Cluster Logs:
(01) 2015-10-20 10:39:24 Slp: Running: C:\Windows\system32\Cluster.exe log /g /copy:"C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\20151020_103702" /node:"XXXXYYYY" /span:"3"
(01) 2015-10-20 10:39:48 Slp: Process returned 0
(01) 2015-10-20 10:39:48 Slp: Cluster log generation completed successfully
(01) 2015-10-20 10:39:48 Slp: Dumping final cluster state.
(01) 2015-10-20 10:40:00 Slp: Completed dumping final cluster state.
(01) 2015-10-20 10:40:03 Slp:
(01) 2015-10-20 10:40:03 Slp: ----------------------------------------------------------------------
(01) 2015-10-20 10:40:03 Slp:
(01) 2015-10-20 10:40:03 Slp: Error result: -2067919922
(01) 2015-10-20 10:40:03 Slp: Result facility code: 1214
(01) 2015-10-20 10:40:03 Slp: Result error code: 3022
(01) 2015-10-20 10:40:03 Slp: SQM Service: Sqm does not have active session.

Zberteoc

have you tried to apply the patch with the node active?

ASKER CERTIFIED SOLUTION

Zberteoc

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

SJCFL-Admin

ASKER

What is missing in the write-up is detailed instructions on how best to accomplish step 1.

1.Remove automatic failover on all synchronous-commit replicas.

Here are the missing instructions:

Expand AlwaysOn HighAvailability
Expand Availability Groups
Right-Click on your availability group name
Change the Failover Mode to MANUAL for BOTH INSTANCES

And then after the patch/upgrade is applied follow the same process and Change the Failover Mode to MANUAL for BOTH INSTANCES

SJCFL-Admin

ASKER

The expert provided a write-up from Microsoft, but I still needed to create an incident with them understand what process needed to be followed to accomplish the step that was recommended. But the expert did point me to the correct article and I do very much appreciate the assist.

Zberteoc

1.Remove automatic failover on all synchronous-commit replicas.

In order to change this you will have to open the Management STudio on the primary replicat and do this:

1. Expand the server node
2. Expand the AwaysON High Availability node
3. Expand Availability Groups node
4. Right click on the Availability Group node > Properties

You will get this screen:

You can see the at the bottom panel where you can change the Availability Mode from Synchronous commit to Asynchronous commit and the Failover Mode from Automatic to Manual. Afte the change both nodes should say Async and Manual.

Zberteoc

Actually you only need to change the Failover Mode from Automatic to Manual and you can leave the Availability Mode in Synchronous commit. This is only to make sure that the primary will not failover by itself to secondary while you do the update. After update you can switch back to Automatic.

Zberteoc

Just let me know how it went if is not too much trouble. I myself need to update from SP1 to last SP2 CP pretty soon.

SJCFL-Admin

ASKER

I am scheduled to reattempt patch on Monday. Will provide update.

Zberteoc

Any news? :)