• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2762
  • Last Modified:

Exchange 2010 Server hang, can't find global catalog server


Hello all we are stumped on an issue we have been having with a new Windows 2008 R2 and Exchange 2010 standard server that hangs about once a week. We had it installed and running for a month before the migration from Exchange 2003 to Exchange 2010. We added the server to our network, following the recommended procedures for upgrading the forest/domain. We had both Exchange 2010 & 2003 working in tandem for roughly a month, with all our mailboxes residing on the 2003 server with no problems.

After a month, we where ready to migrate the mailboxes to the 2010 server. And, planned to run the servers in tandem.
The migration went fine. All the mail routing is working correctly. Webmail and auto discover are all working. And, the Exchange server is acting as an additional DNS server as well.

So, the problem is that about every 5 days or so, the server totally hangs and all services stop to function. We are unable to ping the server. But, when we log in through the DRAC port we see the login screen. The server is completely unresponsive, but no Blue screen. This has occurred over 5 times. And, every time, its the same error messages.

Server details:
Windows 2008 R2 service pack 1
Exchange 2010 Service pack 1 rollup 5
Symantec Backup Exec 2010 R3
DNS services
Domain Controller
Global Catalog server
Dell R710 4 SATA drives RAID 6 - 24 GB Ram (Always using 20 GB with 4 GB free) - 2 Quad core processors

I believe that this is a Active Directory and DNS issue. But, I can't track down a resolution. And, I don't understand why the Exchange server can't see itself as a Global Catalog server. I don't see any NIC errors. But, it seems like the server just stops being able to communicate with any other AD servers, including itself.

Other notes of interest are - The Microsoft Exchange System Attendant service stops by itself and Offline address book stops updating on Clients Outlook.

Here are the errors we see in event viewer right before the server goes offline
Application Log:

Log Name:      Application
Source:        MSExchange ADAccess
Date:          10/10/2011 12:18:22 PM
Event ID:      2102
Task Category: Topology
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      EXCH2blabla.corp.blabla.com
Description:
Process MAD.EXE (PID=7512). All Domain Controller Servers in use are not responding:
DC2blabla.corp.blabla.com
exchblabla.corp.blabla.com
EXCH2blabla.corp.blabla.com

Log Name:      Application
Source:        MSExchange ADAccess
Date:          10/10/2011 12:18:43 PM
Event ID:      2106
Task Category: Topology
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      EXCH2blabla.corp.blabla.com
Description:
Process MAD.EXE (PID=7512). Exchange Active Directory Provider failed to obtain DNS records for forest DC=corp,DC=blabla,DC=com. DNS Priority and Weight for the Global Catalog servers in this forest will be set to the default values 0 (priority) and 100 (weight).  

Log Name:      Application
Source:        MSExchange ADAccess
Date:          10/10/2011 12:19:03 PM
Event ID:      2105
Task Category: Topology
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      EXCH2blabla.corp.blabla.com
Description:
Process MAD.EXE (PID=7512). Exchange Active Directory Provider failed to obtain DNS records for domain corp.blabla.com. DNS Priority and Weight for the Domain Controllers in this domain will be set to the default values 0 (priority) and 100 (weight).

Log Name:      Application
Source:        MSExchangeSA
Date:          10/10/2011 12:21:22 PM
Event ID:      9154
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      EXCH2blabla.corp.blabla.com
Description:
DSAccess returned an error '0x80040934' on DS notification. Microsoft Exchange System Attendant will reset DS notification later.


System Log:

Log Name:      System
Source:        NETLOGON
Date:          10/10/2011 12:36:26 PM
Event ID:      5782
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      EXCH2blabla.corp.blabla.com
Description:
Dynamic registration or deregistration of one or more DNS records failed with the following error:
No DNS servers configured for local system.

Log Name:      Application
Source:        MSExchange ADAccess
Date:          10/13/2011 1:29:04 PM
Event ID:      2080
Task Category: Topology
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      EXCH2blabla.corp.blabla.com
Description:
Process MAD.EXE (PID=2224). Exchange Active Directory Provider has discovered the following servers with the following characteristics:
 (Server name | Roles | Enabled | Reachability | Synchronized | GC capable | PDC | SACL right | Critical Data | Netlogon | OS Version)
In-site:
DC2blahblahcorp.blah.com      CDG 1 7 7 1 0 1 1 7 1
exchblabla.corp.blabla.com      CDG 1 7 7 1 0 1 1 7 1
EXCH2blabla.corp.blabla.com      CDG 1 0 0 1 0 0 0 0 0
 Out-of-site:
0
pjinlaok
Asked:
pjinlaok
  • 3
1 Solution
 
SuperTacoCommented:
I would suggest PROPERLY, disabling IPv6. on the Exchange server.  also, make sure you DC is the primary DNS server and the Exchange server points to its IP address as the second server, NOT 172.0.0.1.  Please find the below link to disable IPv6 via the registry.  Please also remove the IPv6 loopback entries form the hostfile.  (::1)

http://support.microsoft.com/kb/929852
0
 
pjinlaokAuthor Commented:
Hi SuperTaco -

We went' through the IPv6 disable that you recommend and verified the DNS settings were all correct.

The server continues to crash every few days with no logs indicating what the problem is.

We have a drac card installed so we can remotely see what is going on with the server when it is hung. We were able to console in to the server and see that the clock was still ticking, but the server was completely unresponsive. No PING on either of the IP addresses.

The server isn't blue screening so we don't get any memory dumps either.

Where do you sugest we look next?
0
 
pjinlaokAuthor Commented:
This is been a very odd issue and we are still drilling down on a resolution.

I believe that the the AD warnings are a symptoms of a bigger issue. Even after following the recommendations of SuperTaco the server continually randomly hang.

We have a DRAC card on the server which allows us to remotely see the console (Great investment!). When the server HANGS it quickly loses all functionality: First tcp/ip drops, then mouse and keyboard, but the clock keeps ticking. The server never actually crashes (which means no log messages), it just hangs in a frozen state.

Since this is our first Windows 2008 server in our domain it needed to be a Domain controller - and - since we use autodiscover in Exchange 2010 it needed 2 IP addresses. One for https www. xxx.com and one for autodiscover.xxx.com.

We removed that IP for troubleshooting.

We removed any external USB drives.

I ran a hardware diagnostic scan from the Dell BIOS - everything PASSED

We are investigating the BROADCOM 4 port gigabit NIC - We are going to install a pci express NIC to take it out of the equation.

We kept searching and we did find an issue with ESET NOD32 and we have uninstalled and are continuing to monitor.

Here are the link I was given that describe the problem with ESET NODS32 and Windows 2008 server

Link about Eset v4.x Issue with server 2008.
http://smokeys.wordpress.com/2009/05/26/updated-2010-09-04-warning-dont-use-any-eset-nod32-v4-x-antivirus-product-together-with-windows-vista-sp2-or-windows-server-2008-sp2/
and
http://kb.eset.com/esetkb/index?page=content&id=SOLN2523

Has anyone else come across the same issue?

Curious because we have ESET running on many other server and haven't seen this problem. I also checked the versions that they are referring to in these articles and we are running a much newer code rev than that.

Till the next update

pjinlaok

0
 
pjinlaokAuthor Commented:
This turned out to be an ESET NOD32 4 issue,
There active directory errors where a symptom of the real problem.
0

Featured Post

 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now