Inbound mails not received (*and inbox not updating on activesync devices)

[Win 2003 SBS]

Emails are suddenly not received from external senders.
Inbox not updating on iPhone. Everything else works
Internal emails work fine and everyone can send.
OWA works fine
RPC/HTTP works (except, you know, receiving emails)

Around the time inbound emails stopped working the only events were:
Event Type: Error
Event Source:      .NET Runtime 2.0 Error Reporting
Event Category:      None
Event ID:      5000
Date:            2012-04-13
Time:            13.35.04
User:            N/A
Computer:      SERVER-SBS
EventType clr20r3, P1 exchangemon.exe, P2, P3 4863dc3c, P4 mscorlib, P5, P6 4e154d36, P7 344a, P8 15a, P9 system.unauthorizedaccess, P10 NIL.

Half an hour later a handful of these events show:

Event Type:      Error
Event Source:      MSExchangeAL
Event Category:      LDAP Operations
Event ID:      8026
Date:            2012-04-13
Time:            15.15.16
User:            N/A
Computer:      SERVER-SBS
LDAP Bind was unsuccessful on directory server-sbs.DOMAIN.local for distinguished name ''. Directory returned error:[0x51] Server Down.

Event Type:      Error
Event Source:      MSExchangeDSAccess
Event Category:      Topology
Event ID:      2102
Date:            2012-04-13
Time:            15.15.10
User:            N/A
Computer:      SERVER-SBS
Process MAD.EXE (PID=3680). All Domain Controller Servers in use are not responding:

Microsoft Exchange ActiveSync Connectivity Tests
Attempting the FolderSync command on the Exchange ActiveSync session.
       The test of the FolderSync command failed.
      Additional Details
       An HTTP 403 forbidden response was received. The response appears to have come from Unknown. Body of the response: <body><h2>HTTP/1.1 403 Forbidden</h2></body>

Microsoft Office Outlook Connectivity Tests
No issues.
For the activesync issue I've gone through alanhardisty's activesync check list and checked all settings. All is fine.

I hardly know where to go. Any ideas?
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

are you blacklisted? check and make sure your domain mx record correctly points to your mail server. check with your isp
musrasAuthor Commented:
MX ok. Not blacklisted anywhere :/

Thanks for your suggestions.
Are those emails stuck in the queue and for some reason not being delivered to the users? Have you checked? Or are they not coming in at all? A couple of those errors look like possible DNS issues, though the fact that everything else seems to work doesn't support that. Check the queue when that's happening and see if they are in there and let us know.
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

musrasAuthor Commented:
Not a single item in any queue and nothing showing inbound in Message Tracking Center after 1pm (local time), but plenty outbound, which still works.

I'm uncertain on how to troubleshoot possible server dns setup, but it does seem that might be worth exploring...
So if nothing is showing in the queue, could you try a telnet to port 25 at the exact time this is happening? That way you can see if your server is responding to requests on that port or if something is blocking them somehow.
I meant a telnet to port 25 from an outside connection... just to be clear.
musrasAuthor Commented:
There we go. "Could not open connection to the host on port 25: Connect failed"

Although I don't see where the issue is.

interface ethernet0 auto
interface ethernet1 100full
nameif ethernet0 outside security0
nameif ethernet1 inside security100
hostname pixfirewall
fixup protocol dns maximum-length 512
fixup protocol ftp 21
fixup protocol h323 h225 1720
fixup protocol h323 ras 1718-1719
fixup protocol http 80
fixup protocol rsh 514
fixup protocol rtsp 554
fixup protocol sip 5060
fixup protocol sip udp 5060
fixup protocol skinny 2000
no fixup protocol smtp 25
fixup protocol sqlnet 1521
fixup protocol tftp 69
access-list inside_outbound_nat0_acl permit ip 
access-list outside_cryptomap_dyn_20 permit ip any 
access-list outside_acl permit tcp any host eq 3389 
access-list outside_acl permit tcp any interface outside eq ftp 
access-list outside_acl permit tcp any host eq https 
access-list outside_acl permit tcp any host eq www 
access-list outside_acl permit tcp any interface outside eq 993 
access-list outside_acl permit tcp any interface outside eq imap4 
access-list outside_acl permit tcp any interface outside eq 465 
access-list outside_acl permit tcp any host eq smtp 
access-list outside_cryptomap_dyn_40 permit ip any 
access-list COMPANYVPN_splitTunnelAcl permit ip any 
access-list COMPANY_splitTunnelAcl permit ip any 
access-list outside_cryptomap_dyn_60 permit ip any 
access-list COMPANY_VPN_splitTunnelAcl permit ip any 
access-list outside_cryptomap_dyn_80 permit ip any 
pager lines 24
icmp permit host outside
mtu outside 1500
mtu inside 1500
ip address outside
ip address inside
ip audit info action alarm
ip audit attack action alarm
ip local pool VPN
pdm location outside
pdm location inside
pdm location 217.yyy.yyy.yyy outside
pdm location SERVER inside
pdm logging informational 100
pdm history enable
arp timeout 14400
global (outside) 1 interface
nat (inside) 0 access-list inside_outbound_nat0_acl
nat (inside) 1 0 0
static (inside,outside) tcp 3389 SERVER 3389 netmask 0 0 
static (inside,outside) tcp smtp SERVER smtp netmask 0 0 
static (inside,outside) tcp https SERVER https netmask 0 0 
static (inside,outside) tcp www SERVER www netmask 0 0 
static (inside,outside) tcp interface imap4 SERVER imap4 netmask 0 0 
static (inside,outside) tcp interface 993 SERVER 993 netmask 0 0 
static (inside,outside) tcp interface 465 SERVER 465 netmask 0 0 
static (inside,outside) tcp interface ftp SERVER ftp netmask 0 0 
access-group outside_acl in interface outside
route outside 213.zzz.zzz.zzz
timeout xlate 0:05:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h225 1:00:00
timeout h323 0:05:00 mgcp 0:05:00 sip 0:30:00 sip_media 0:02:00
timeout sip-disconnect 0:02:00 sip-invite 0:03:00
timeout uauth 0:05:00 absolute
aaa-server TACACS+ protocol tacacs+ 
aaa-server TACACS+ max-failed-attempts 3 
aaa-server TACACS+ deadtime 10 
aaa-server RADIUS protocol radius 
aaa-server RADIUS max-failed-attempts 3 
aaa-server RADIUS deadtime 10 
aaa-server RADIUS (inside) host SERVER *** timeout 10
aaa-server LOCAL protocol local 
http server enable
http 217.yyy.yyy.yyy outside
http inside
no snmp-server location
no snmp-server contact
snmp-server community public
no snmp-server enable traps
floodguard enable
sysopt connection permit-ipsec
crypto ipsec transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac 
crypto dynamic-map outside_dyn_map 20 match address outside_cryptomap_dyn_20
crypto dynamic-map outside_dyn_map 20 set transform-set ESP-3DES-MD5
crypto dynamic-map outside_dyn_map 40 match address outside_cryptomap_dyn_40
crypto dynamic-map outside_dyn_map 40 set transform-set ESP-3DES-MD5
crypto dynamic-map outside_dyn_map 60 match address outside_cryptomap_dyn_60
crypto dynamic-map outside_dyn_map 60 set transform-set ESP-3DES-MD5
crypto dynamic-map outside_dyn_map 80 match address outside_cryptomap_dyn_80
crypto dynamic-map outside_dyn_map 80 set transform-set ESP-3DES-MD5
crypto map outside_map 65535 ipsec-isakmp dynamic outside_dyn_map
crypto map outside_map client authentication RADIUS LOCAL
crypto map outside_map interface outside
isakmp enable outside
isakmp policy 20 authentication pre-share
isakmp policy 20 encryption 3des
isakmp policy 20 hash md5
isakmp policy 20 group 2
isakmp policy 20 lifetime 86400
telnet 217.yyy.yyy.yyy outside
telnet inside
telnet timeout 5
ssh 217.yyy.yyy.yyy outside
ssh inside
ssh timeout 5
management-access inside
console timeout 0
dhcpd address inside
dhcpd dns SERVER
dhcpd wins SERVER 
dhcpd lease 3600
dhcpd ping_timeout 750
dhcpd auto_config outside
dhcpd enable inside
: end

Open in new window

So this is your configuration but do you see anything in the logs at the time that you tried to telnet? Or when the problem is happening? You don't have any time of email filter like Postini or a barracuda where the emails go through before coming into your network, right?
musrasAuthor Commented:
No filters, straight PIX to SBS2003.

I have Kiwi syslog running, but could use some pointers on how to properly filter the torrent of log messages. If someone has the experience I suspect it could cut hours off tinkering...
Yeah, now a bit outside my comfort zone, perhaps someone else can jump in.
If you telnet from the inside at the same time when this is happening it works, right? So it's only the outside connections that are being blocked? So it's probably something on the PIX...
If nobody else has a better solution you could look into re-running the SBS internet connection wizard (it may not be named exactly that I don't remember). But this may not be necessary as it's most likely something on the PIX.
Also look for any errors on the exchange server logs just to make sure there's nothing there.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
So what was it? What fixed it?
musrasAuthor Commented:
Not actually fixed, but I think this thread has outlived its time with respect to the question title. I award you the points for pointing in the right direction (on second thought that might be entirely the wrong thing to do on EE...)
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.