Exchange cluster fail

this morning i noticed bunch of cluster errors on almost all exchange servers that had happened about 3 am last night. so kept digging and finally i pulled the cluster log files from all exchange servers via
Get-ClusterLog -TimeSpan 600 -Destination C:\temp\ClusterLog

and i noticed bellow errors that shows entire cluster went down.

00016538.00016480::2018/03/02-05:44:05.417 INFO  [CHM] Received notification for two consecutive missed HBs to the remote endpoint 172.19.17.202:~3343~ from 172.19.16.149:~3343~
00016538.00016480::2018/03/02-11:46:05.102 INFO  [IM] Marking Route from 172.19.16.149:~3343~ to 172.17.166.38:~3343~ as down
00016538.00016480::2018/03/02-11:46:05.103 INFO  [IM] got event: Remote endpoint 172.17.166.39:~3343~ unreachable from 172.19.16.149:~3343~
00016538.00016480::2018/03/02-11:46:05.103 INFO  [NDP] Checking to see if all routes for route (virtual) local 169.254.1.95:~0~ to remote 169.254.6.31:~0~ are down

now the interesting part is that the network team investigataed everything and they confirmed there was no network connectivity of any kind around that time between any of the exchange severs.

my question is that if there was no network connectivity what else might have caused all this? everything has come back online and normal on its own. but i wana know the reason for this.
LVL 2
Riaz Alexander AnsaryEnterprise Infrastructure Systems EngineerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

timgreen7077Exchange EngineerCommented:
There could have been a network drop. In regards to what caused it, that is something your network team needs to figure out.  If this happened on most all your Exchange servers then I would think something happened with the company network at 3am in the morning. Exchange doesnt control the network but it relies on it heavily and if there is a blip or issue on the network side exchange starts disconnecting from DBs and DAGs and so forth.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Riaz Alexander AnsaryEnterprise Infrastructure Systems EngineerAuthor Commented:
Ended up that Our new backup system Comvault took snapshots of each server and that caused the blip. not sure why backup team had the snapshot setup for exchange servers. they are excluded now. thanks for the response
0
timgreen7077Exchange EngineerCommented:
Good deal
0
timgreen7077Exchange EngineerCommented:
Assisted author in troubleshooting the issue.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Database Availability Group (DAG)

From novice to tech pro — start learning today.