techdrive
asked on
Exchange 2010 sp2 after patching issue
I have an issue after patching my exchange 2010 environment. I ran all the updates on the 3 servers. I confirmed that all updates installed successfully and after a series of test I determined that mailflow is working and all databases are mounted. My only problem is
test-replicationhealth
Server : server1
Check : ClusterNetwork
CheckDescription : Checks that the networks are healthy.
Result : *FAILED*
Error : Node 'server3' has a network interface that is down. The IP address is '192.168.45.57'. Current
state is 'Failed'.
Identity :
IsValid : True
This message appears in the system section of event viewer
Cluster network interface 'server3 - LAN-NIC1' for cluster node 'server3' on network 'Cluster Network 1' failed. Run the Validate a Configuration wizard to check your network configuration. If the condition persists, check for hardware or software errors related to the network adapter. Also check for failures in any other network components to which the node is connected such as hubs, switches, or bridges.
[PS] C:\Windows\system32>ping 192.168.45.57
Pinging 192.168.45.57 with 32 bytes of data:
Reply from 192.168.45.57: bytes=32 time<1ms TTL=128
Reply from 192.168.45.57: bytes=32 time<1ms TTL=128
Reply from 192.168.45.57: bytes=32 time<1ms TTL=128
Reply from 192.168.45.57: bytes=32 time<1ms TTL=128
Ping statistics for 192.168.45.57:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
I validated that the cluster test ran successfully on all components and pinged test passed
Mail flow runs fine and tested.
Mapi connectivity runs fine
When I run the cluster manager there are no red or yellow signs
Inside of the cluster manager under network 2 it does list the card and down on network 1 it is up.
I can RDP to the IP address.
However, when I run the command I have above test-replicationhealth or view this info under database availability tab under the mailbox icon at the organizational level the ipv6 address appears and only for this IP address does it state that it has failed.
test-replicationhealth
Server : server1
Check : ClusterNetwork
CheckDescription : Checks that the networks are healthy.
Result : *FAILED*
Error : Node 'server3' has a network interface that is down. The IP address is '192.168.45.57'. Current
state is 'Failed'.
Identity :
IsValid : True
This message appears in the system section of event viewer
Cluster network interface 'server3 - LAN-NIC1' for cluster node 'server3' on network 'Cluster Network 1' failed. Run the Validate a Configuration wizard to check your network configuration. If the condition persists, check for hardware or software errors related to the network adapter. Also check for failures in any other network components to which the node is connected such as hubs, switches, or bridges.
[PS] C:\Windows\system32>ping 192.168.45.57
Pinging 192.168.45.57 with 32 bytes of data:
Reply from 192.168.45.57: bytes=32 time<1ms TTL=128
Reply from 192.168.45.57: bytes=32 time<1ms TTL=128
Reply from 192.168.45.57: bytes=32 time<1ms TTL=128
Reply from 192.168.45.57: bytes=32 time<1ms TTL=128
Ping statistics for 192.168.45.57:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
I validated that the cluster test ran successfully on all components and pinged test passed
Mail flow runs fine and tested.
Mapi connectivity runs fine
When I run the cluster manager there are no red or yellow signs
Inside of the cluster manager under network 2 it does list the card and down on network 1 it is up.
I can RDP to the IP address.
However, when I run the command I have above test-replicationhealth or view this info under database availability tab under the mailbox icon at the organizational level the ipv6 address appears and only for this IP address does it state that it has failed.
Can you run test-servicehealth on all server.
ASKER
All 3 servers do not have any servers that have services that are not started.
Role : Mailbox Server Role
RequiredServicesRunning : True
ServicesRunning : {IISAdmin, MSExchangeADTopology, MSExchangeIS, MSExchangeMailboxAssistant s, MSExchangeMailSub
mission, MSExchangeRepl, MSExchangeRPC, MSExchangeSA, MSExchangeSearch, MSExchangeServiceHost
, MSExchangeThrottling, MSExchangeTransportLogSear ch, W3Svc, WinRM}
ServicesNotRunning : {}
Role : Client Access Server Role
RequiredServicesRunning : True
ServicesRunning : {IISAdmin, MSExchangeAB, MSExchangeADTopology, MSExchangeFBA, MSExchangeFDS, MSExchangeIMAP4,
MSExchangeMailboxReplicati on, MSExchangePOP3, MSExchangeProtectedService Host, MSExchangeRPC,
MSExchangeServiceHost, W3Svc, WinRM}
ServicesNotRunning : {}
Role : Hub Transport Server Role
RequiredServicesRunning : True
ServicesRunning : {IISAdmin, MSExchangeADTopology, MSExchangeEdgeSync, MSExchangeServiceHost, MSExchangeTranspo
rt, MSExchangeTransportLogSear ch, W3Svc, WinRM}
ServicesNotRunning : {}
Role : Mailbox Server Role
RequiredServicesRunning : True
ServicesRunning : {IISAdmin, MSExchangeADTopology, MSExchangeIS, MSExchangeMailboxAssistant
mission, MSExchangeRepl, MSExchangeRPC, MSExchangeSA, MSExchangeSearch, MSExchangeServiceHost
, MSExchangeThrottling, MSExchangeTransportLogSear
ServicesNotRunning : {}
Role : Client Access Server Role
RequiredServicesRunning : True
ServicesRunning : {IISAdmin, MSExchangeAB, MSExchangeADTopology, MSExchangeFBA, MSExchangeFDS, MSExchangeIMAP4,
MSExchangeMailboxReplicati
MSExchangeServiceHost, W3Svc, WinRM}
ServicesNotRunning : {}
Role : Hub Transport Server Role
RequiredServicesRunning : True
ServicesRunning : {IISAdmin, MSExchangeADTopology, MSExchangeEdgeSync, MSExchangeServiceHost, MSExchangeTranspo
rt, MSExchangeTransportLogSear
ServicesNotRunning : {}
Can you reboot the server for which it is showing error and test-replication again.
ASKER
So is rebooting the server the fix...Your final answer.?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
kicking myself for all that work and research I did.
Did reboot fixed or not? Also note SP2 is very old. You should upgrade to SP3 RU2. Below are the steps, might help you to do smooth upgrade
Steps Upgrading Exchange 2010 SP1 or SP2 to SP3:
My Lab Setup: 2 node Multi-role server with DAG.
1) Upgrade the schema for Exchange 2010 SP3. If you have Enterprise and Schema Admin rights, setup will do it automatically, however if not, then AD team is required to perform this task.
Purpose of this Schema Update:
===================
In Order to have the Exchange 2010 SP3 deployed across the environment, it is a prerequisite to have the Active Directory Schema to be extended for supporting Exchange 2010 SP3. During this upgrade, the Active Directory Schema will be extended to support Exchange 2010 SP3.
Steps to be performed for this upgrade:
========================== ======
Reference Article –
http://technet.microsoft.com/en-in/library/bb629560(v=exchg.141).aspx
Prerequisites: AD account need to be member of following groups
Member of: Domain Admin, Enterprise Admin, Schema Admin
Install Active Directory Management Tools (Run this Command, if Required)
ServerManagerCmd -i RSAT-ADDS
Software Location Exchange 2010 SP3
http://www.microsoft.com/en-us/download/details.aspx?id=36768
Download it and extracting it run below command for preparing the schema.
setup /PrepareLegacyExchangePerm issions (Optional):
Check C:\ExchangeSetupLogs\Excha ngeSetup.l og for successful completion
Setup /PrepareSchema
Check C:\ExchangeSetupLogs\Excha ngeSetup.l og for successful completion
Setup /PrepareAD /OrganizationName:"Your Org Name"
Check C:\ExchangeSetupLogs\Excha ngeSetup.l og for successful completion
How to verify schema version?
http://technet.microsoft.com/en-in/library/bb125224%28v=exchg.141%29.aspx
2) Steps for Upgrading Exchange 2010 to SP3
Software Location:
1) Exchange 2010 SP3: http://www.microsoft.com/en-in/download/details.aspx?id=36768
Note: Any Interim Updates for Exchange Server 2010 must be uninstalled prior to installing this update.
As I have all roles in one server in a 2 node DAG. Kindly perform the below steps.
Phase 1: Upgrading Database Availability Groups to Exchange 2010 SP3 (Start with Passive)
Note: Make sure to take full Exchange backup, including system state or any custom setting related to CAS role. Do save the OWA folder under bin path too. During SP3 installation, Exchange setup removes all SP2 related folders; in case you have any proxying it might fail, so make sure to upgrade all servers, however one by one.
1) Open Exchange Management Shell, browse to script folder and run below command
.\StartDagServerMaintenanc e.ps1 -serverName servername
Note: This script move all Exchange related component including database to another server.
2) Disable the Forefront by using below command (It might ask you to stop few Exchange services)
fscutility /disable
3) Stop the OS Antivirus services and Monitoring Agent for Microsoft Exchange Server
4) Verify Backups are completed.
5) Run the SP3 setup files using GUI or goto run setup /m:upgrade. If you are getting IIS component error goto run setup /mode:upgrade /installwindowscomponents
Note: If you are getting this error "A Restart from a Previous Installation is Pending"
Open regedit and delete the HKLM\SYSTEM\CurrentControl Set\Contro l\Session Manager\PendingFileRenameO perations key and Rerun Setup.
Ref: http://technet.microsoft.com/en-us/library/cc164360(v=exchg.80).aspx
6) Reboot the server
7) After reboot verify below logs for any error:
Event Logs
Setup Logs (C:\ExchangeSetupLogs)
Services - All Exchange related services are running.
Check IIS Settings are not changed.
Check Authentication settings from EMC for OWA/OMA/ESW. If you find any change revert back to old.
8) Enable Forefront Protection for Exchange. Command fscutility /enable (It might ask you to stop few Exchange services)
9) Start OS Antivirus and Exchange Monitoring Agent Services
10) .\StopDagServerMaintenance .ps1 -serverName servername
11) Now use same steps on another server in DAG.
-------------------------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- -
Phase 2: Upgrade Management Tools
If you have installed management tools in any other machine make sure to upgrade those also.
-------------------------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- -
Finally: - how to verify the upgrade.
Goto Powershell and type below command.
GCM exsetup |%{$_.Fileversioninfo}
Steps Upgrading Exchange 2010 SP1 or SP2 to SP3:
My Lab Setup: 2 node Multi-role server with DAG.
1) Upgrade the schema for Exchange 2010 SP3. If you have Enterprise and Schema Admin rights, setup will do it automatically, however if not, then AD team is required to perform this task.
Purpose of this Schema Update:
===================
In Order to have the Exchange 2010 SP3 deployed across the environment, it is a prerequisite to have the Active Directory Schema to be extended for supporting Exchange 2010 SP3. During this upgrade, the Active Directory Schema will be extended to support Exchange 2010 SP3.
Steps to be performed for this upgrade:
==========================
Reference Article –
http://technet.microsoft.com/en-in/library/bb629560(v=exchg.141).aspx
Prerequisites: AD account need to be member of following groups
Member of: Domain Admin, Enterprise Admin, Schema Admin
Install Active Directory Management Tools (Run this Command, if Required)
ServerManagerCmd -i RSAT-ADDS
Software Location Exchange 2010 SP3
http://www.microsoft.com/en-us/download/details.aspx?id=36768
Download it and extracting it run below command for preparing the schema.
setup /PrepareLegacyExchangePerm
Check C:\ExchangeSetupLogs\Excha
Setup /PrepareSchema
Check C:\ExchangeSetupLogs\Excha
Setup /PrepareAD /OrganizationName:"Your Org Name"
Check C:\ExchangeSetupLogs\Excha
How to verify schema version?
http://technet.microsoft.com/en-in/library/bb125224%28v=exchg.141%29.aspx
2) Steps for Upgrading Exchange 2010 to SP3
Software Location:
1) Exchange 2010 SP3: http://www.microsoft.com/en-in/download/details.aspx?id=36768
Note: Any Interim Updates for Exchange Server 2010 must be uninstalled prior to installing this update.
As I have all roles in one server in a 2 node DAG. Kindly perform the below steps.
Phase 1: Upgrading Database Availability Groups to Exchange 2010 SP3 (Start with Passive)
Note: Make sure to take full Exchange backup, including system state or any custom setting related to CAS role. Do save the OWA folder under bin path too. During SP3 installation, Exchange setup removes all SP2 related folders; in case you have any proxying it might fail, so make sure to upgrade all servers, however one by one.
1) Open Exchange Management Shell, browse to script folder and run below command
.\StartDagServerMaintenanc
Note: This script move all Exchange related component including database to another server.
2) Disable the Forefront by using below command (It might ask you to stop few Exchange services)
fscutility /disable
3) Stop the OS Antivirus services and Monitoring Agent for Microsoft Exchange Server
4) Verify Backups are completed.
5) Run the SP3 setup files using GUI or goto run setup /m:upgrade. If you are getting IIS component error goto run setup /mode:upgrade /installwindowscomponents
Note: If you are getting this error "A Restart from a Previous Installation is Pending"
Open regedit and delete the HKLM\SYSTEM\CurrentControl
Ref: http://technet.microsoft.com/en-us/library/cc164360(v=exchg.80).aspx
6) Reboot the server
7) After reboot verify below logs for any error:
Event Logs
Setup Logs (C:\ExchangeSetupLogs)
Services - All Exchange related services are running.
Check IIS Settings are not changed.
Check Authentication settings from EMC for OWA/OMA/ESW. If you find any change revert back to old.
8) Enable Forefront Protection for Exchange. Command fscutility /enable (It might ask you to stop few Exchange services)
9) Start OS Antivirus and Exchange Monitoring Agent Services
10) .\StopDagServerMaintenance
11) Now use same steps on another server in DAG.
--------------------------
Phase 2: Upgrade Management Tools
If you have installed management tools in any other machine make sure to upgrade those also.
--------------------------
Finally: - how to verify the upgrade.
Goto Powershell and type below command.
GCM exsetup |%{$_.Fileversioninfo}
ASKER
thank you thank you thank you...Sorry I could not reboot the server until last night. I just wanted to thank you for your help.
ASKER
I have a 3 node DAG. I notice the cluster is on the first server. Is it necessary for me to fail this over to do a successful patch upgrade. I did not do this the first time and now wonder if I should have done this.