• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2214
  • Last Modified:

SBS 2011 / Exchange 2010 : Random emails arriving late or not at all.

I've got a weird problem with out SBS 2011/Exchange 2010 system. For the last few days,  a handful of emails for seeming random users are either being delayed by over an hour or are not getting through at all.  There's been no changes to the server other than updates.

There's nothing in the event logs to suggest a problem. The headers of the delayed messages look like this..

Received: from st11p00mm-asmtp003.mac.com ( by office.mycompany.org.uk
 ( with Microsoft SMTP Server id 14.1.438.0; Sat, 1 Feb 2014
 15:32:08 +0000
Received: from []
 (hostxx-xxx-xxx-xxx.range86-157.btcentralplus.com [86.157.xxx.xxx]) by
 st11p00mm-asmtp003.mac.com (Oracle Communications Messaging Server
 7u4-27.08( 64bit (built Aug 22 2013)) with ESMTPSA id
 <0N0B00JIOMXZD620@st11p00mm-asmtp003.mac.com> for vick@mycompany.org.uk; Sat, 01
 Feb 2014 14:30:05 +0000 (GMT)
X-Proofpoint-Virus-Version: vendor=fsecure
 definitions=2014-02-01_01:2014-01-31,2014-02-01,1970-01-01 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0
 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0
 reason=mlx scancount=1 engine=7.0.1-1308280000 definitions=main-1402010070
User-Agent: Microsoft-Entourage/
Date: Sat, 1 Feb 2014 14:29:58 +0000
Subject: You're going to hate me but ...
From: Gary<gary@mac.com>
To: Alan<alan@btinternet.com>, Simon
CC: Vick<vick@mycompany.org.uk>, RICHARD
	<richard@richard.com>, Paul<paul@paul.com>,
	Sarah<sarah@sarah.co.uk>, David<david@gmail.com>, Tim<tim@tim.co.uk>, Mick<mick@aol.com>
Message-ID: <CF12B566.5F3DD%gary@mac.com>
Thread-topic: You're going to hate me but ...
Thread-index: Ac8fWhUvvYv8eHkDUk+N0J2io1fYzg==
In-Reply-To: <C45DCB72-5610-4D84-A9F0-455DEB35458F@btinternet.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="B_3474109804_1209807"
X-Cyberoam-AV-Policy: Zip
X-CTCH-Error: Unable to connect local ctasd
Return-Path: gary@mac.com
X-MS-Exchange-Organization-AuthSource: SBS-Server.mycompany-net.local
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-PRD: mac.com
X-MS-Exchange-Organization-SenderIdResult: Pass
Received-SPF: Pass (SBS-Server.mycompany-net.local: domain of
 gary@mac.com designates as permitted sender)
 receiver=BASCA-SBS.mycompany-net.local; client-ip=;
X-MS-Exchange-Organization-SCL: 3
X-MS-Exchange-Organization-PCL: 2
X-MS-Exchange-Organization-Antispam-Report: DV:3.3.13320.464;SID:SenderIDStatus

Open in new window

We have a Cyberoam CR25ia as firewall/AV/AS. Nothing in those logs suggest much either (although I'm not entirely sure if I'm looking the right place there).

Various tests and monitoring suggests that things are running fairly well other than the odd warning about lack of RAM. We've got 8GB RAM for about 8 concurrent users. CPU is trundling at about 8% load. 156GB free on system Drive. Server is mostly just for email with light fileserver duties. No other SQL servers or other heavy load stuff on it.

Not sure where to go from here. Any ideas?
  • 9
  • 5
  • 5
  • +1
3 Solutions
I would take a look at your exchange queues, when the problem occurs, to make sure they are not getting overloaded.  
Where is your anti-spam located? If you do your own anti spam filtering, after the email arrives, your exchange server queues could be filling up with spam.  (This is why I always recommend using an email gateway service like GFI to scan your mails, THEN forward them to your exchange server)
Ah, re-read question and I see your cyberoam is doing the anti-spam.  I guess that would eliminate my possible SPAM suggestion, unless the spam is using up THAT much internet bandwidth (possible I guess, but I'd think you'd have noticed that).
Dale303Author Commented:
I've been keeping an eye on the internet bandwidth. It's not being stressed at all. Just the odd spike every now and then
Creating Active Directory Users from a Text File

If your organization has a need to mass-create AD user accounts, watch this video to see how its done without the need for scripting or other unnecessary complexities.

Guess it wouldn't hurt to check the exchange queues anyway...though it's just a shot in the dark.
Dale303Author Commented:
nothing there. there's a queue called 'submission' but there's nothing in it.

I'm wondering if it's something to do with lack of RAM. The pagefile does seem to be constantly being R/W to. And Exchange itself seems to be busy even late on a weekend

Excahnge Disk Activity
Simon Butler (Sembee)ConsultantCommented:
Lack of memory wouldn't stop emails from being delivered at all.
Processing of email is very low resource intensive.

Do the messages appear in message tracking at all?
Do you have logging on that other antispam service? If you do, ensure it is enabled so you can see if the traffic is coming in.

Due to the nature of email, there isn't much more that you can do, because until the email is delivered to a service under your control it is out of your hands. However if it was me, the first thing I would be looking at as the cause is the anti-spam service. Most email delivery problems are caused by third party tools in my experience.

Gary ColtharpSr. Systems EngineerCommented:
Have you checked your AS appliance? Rebooted it? This is the line that leads me to believe it is related to your issue:

X-CTCH-Error: Unable to connect local ctasd

Dale303Author Commented:
OK. Sorry for the delay. I've been away. The problem kind of cleared itself up for a couple of weeks but now it's back. I added more RAM just in case so the server now has 16GB.

The Cyberoam is a UTM device. It sits between the router and our server and basically deals with all the Firewall/Antivirus/Antispam/VPN duties. There's no realtime antispam or antivirus on the server itself other than what is built into SBS. A couple of tweaks to the  Hub Transport section including adding an anonymous SMTP receive connector to a specific IP address so that an external webapp could send emails via it but that's it.

The messages do eventually arrive. Some people get NDRs though. Most emails get through pretty quickly, others take 30mins, a few a couple of hours and in the extreme cases, it's been 2 days.

What logs should I look at for 'first point of contact' with the server? This should help me work out if it's the server of the Cyberoam device is at fault. How are the services layered? Is there even one log that can show the lot? The odd thing is, the delays are  completely random.  I suspect something is choking the connection so that in effect at certain points, for the sender, the server appears offline.

It's entirely possible that it's the UTM that's causing the issues but I had to start somewhere and eliminate other possibilities first. I've been running Cyberoam devices for some time and I've never had an issue with them before.

"X-CTCH-Error: Unable to connect local ctasd" I have no idea what this means. I'll give Cyberoam a call to see if they can help.
Simon Butler (Sembee)ConsultantCommented:
Do you have logging enabled on the Receive Connectors? It isn't enabled by default. If not, then enable it and start there.
Also look at the headers of a message that was delayed and see where the delay occurred.

" I suspect something is choking the connection so that in effect at certain points, for the sender, the server appears offline."
You may wish to see if the cyberoam has a bandwidth monitor,  this will help you determine if the issue is indeed due to a choked up internet connection.

Other tests:  from your home, or other offsite location, setup some kind of test like "ping [yourIP] /t".  If you record timing with the tests, you can determine if communication to your server is down, or very slow, at the time when the inbound email issue occurs.

General advice:  We were always having problems like this due to huge amounts of inbound spam choking up our connection.  It doesn't sound like this is actually happening to you YET, based on your queues(unless your cyberoam is indeed blocking the spam first).  However, I still want to suggest that you use a third party email gateway.  These services provide an initial level of anti-spam, and THEN forward only non-spam messages to your server.  The biggest difference you will notice, is reduced spam traffic to your server, because the provider's IP address is listed in the public MX record, not yours.  (so your IP gets hidden from those spammer bots).  we used: http://www.gfimax.com/mail/mail-protection
Dale303Author Commented:
@Sembee. From 'MX toolbox analyze headers' a typical delayed email will be...

1      *      mx.google.com      ESMTPSA      2/25/2014 7:15:25 PM
2      1 hour            mail-we0-f172.google.com      SMTP      2/25/2014 8:22:37 PM
3      36 seconds      mail-we0-f172.google.com      office.org.uk      Microsoft SMTP Server      2/25/2014 8:23:13 PM
Dale303Author Commented:
@Korbus Bandwidth is not a problem. I have PRTG monitoring at the router and the server. They both appear very normal. A few spikes here and there but mostly well below 10% of capacity. I checked with our ISP and they confirmed bandwidth is not the problem and there's been very little packet loss. I can keep an RDP session up for hours without a glitch.

As for your other advice, I would normally agree but we've had no issues in the past andthe Cyberoam box really does kill 99% of spam and viruses before they reach the server. Also, it was all working totally fine up until a couple of weeks ago.
Certian clients prefer cloud based email but these guys really like to keep things in house for a variety of reasons.

I get the feeling it's Exchange (or the Cyberoam box) that is 'falling asleep at the job' and for whatever reason isn't ready to pick up emails every now and then. Not 'offline' as such but maybe too busy or something. Is there a way to check this?

Please tell me if I'm wrong but if a the server is not ready to accept emails, would the sending server just retry a bit later? Could this be the reason for the delay? Is there any way to check this?
Simon Butler (Sembee)ConsultantCommented:
That looks like the delay is between delivery from Google to your server.
That also means that most of the causes are outside of your control, plus you cannot see the queues etc. Once the email is delivered to the server it is then delivered to the mailbox quickly.

If a server isn't ready, then it does retry, however as the recipient, it is impossible to know what is happening. Most servers will retry every 15 minutes initially, then as the email gets older they will try at increasing intervals.

If you suspect that you may have problems, sign up for a free monitoring account at mxalerts.com and see whether that flags anything.

Dale303Author Commented:
That's what I thought at first but it's not just Google, it's Yahoo and me.com as well.

I used SLcheck and set up a scheduled task to 'ping' port 25 and this is what I got. back (not the most thorough test as the connection wasn't always live but the log does show there are times when the server is not accessible....

25/02/2014,22:38:58,mail.company.co.uk, 25,220,50,Yes
25/02/2014,22:41:01,mail.company.co.uk, 25,220,40,Yes
25/02/2014,22:43:47,mail.company.co.uk, 25,220,40,Yes
25/02/2014,22:48:58,mail.company.co.uk, 25,220,40,Yes
25/02/2014,22:59:59,mail.company.co.uk, 25,220,40,Yes
25/02/2014,23:16:21,mail.company.co.uk, 25,220,40,Yes
25/02/2014,23:16:34,mail.company.co.uk, 25,220,40,Yes
25/02/2014,23:17:31,mail.company.co.uk, 25,220,30,Yes
25/02/2014,23:19:35,mail.company.co.uk, 25,220,40,Yes
25/02/2014,23:26:45,mail.company.co.uk, 25,220,40,Yes
25/02/2014,23:43:17,mail.company.co.uk, 25,220,50,Yes
25/02/2014,23:43:40,mail.company.co.uk, 25,220,40,Yes
25/02/2014,23:47:01,mail.company.co.uk, 25,220,60,Yes
25/02/2014,23:48:17,mail.company.co.uk, 25,220,50,Yes
25/02/2014,23:51:55,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:01:22,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:16:21,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:20:50,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:21:48,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:27:40,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:40:15,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:45:36,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:47:04,mail.company.co.uk, 25,220,40,Yes
26/02/2014,00:47:08,mail.company.co.uk, 25,220,50,Yes
26/02/2014,01:09:02,mail.company.co.uk, 25,220,40,Yes
26/02/2014,01:11:16,mail.company.co.uk, 25,220,60,Yes
26/02/2014,01:23:56,mail.company.co.uk, 25,220,40,Yes
26/02/2014,01:24:39,mail.company.co.uk, 25,220,90,Yes
26/02/2014,01:24:55,mail.company.co.uk, 25,220,50,Yes
26/02/2014,01:25:44,mail.company.co.uk, 25,220,50,Yes
26/02/2014,01:26:23,mail.company.co.uk, 25,220,60,Yes
26/02/2014,01:27:03,mail.company.co.uk, 25,220,60,Yes
26/02/2014,01:28:10,mail.company.co.uk, 25,220,60,Yes
26/02/2014,01:29:29,mail.company.co.uk, 25,220,70,Yes
26/02/2014,01:34:58,mail.company.co.uk, 25,220,40,Yes
26/02/2014,01:36:28,mail.company.co.uk, 25,220,50,Yes
26/02/2014,01:51:56,mail.company.co.uk, 25,220,50,Yes
26/02/2014,02:11:30,mail.company.co.uk, 25,220,40,Yes
26/02/2014,02:13:56,mail.company.co.uk, 25,220,30,Yes
26/02/2014,02:19:32,mail.company.co.uk, 25,220,50,Yes
26/02/2014,02:24:07,mail.company.co.uk, 25,220,60,Yes
26/02/2014,02:29:40,mail.company.co.uk, 25,220,70,Yes
26/02/2014,02:31:45,mail.company.co.uk, 25,220,70,Yes
26/02/2014,02:39:37,mail.company.co.uk, 25,220,50,Yes
26/02/2014,02:43:10,mail.company.co.uk, 25,220,80,Yes
26/02/2014,02:53:34,mail.company.co.uk, 25,220,60,Yes
26/02/2014,03:03:24,mail.company.co.uk, 25,220,220,Yes
26/02/2014,03:05:07,mail.company.co.uk, 25,220,150,Yes
26/02/2014,03:06:36,mail.company.co.uk, 25,220,760,Yes
26/02/2014,03:07:12,mail.company.co.uk, 25,n/a,Timeout,No
26/02/2014,03:15:39,mail.company.co.uk, 25,220,120,Yes
26/02/2014,03:18:16,mail.company.co.uk, 25,220,100,Yes
26/02/2014,03:20:19,mail.company.co.uk, 25,220,50,Yes
26/02/2014,03:25:57,mail.company.co.uk, 25,220,130,Yes
26/02/2014,03:31:23,mail.company.co.uk, 25,220,210,Yes
26/02/2014,03:33:53,mail.company.co.uk, 25,220,140,Yes
26/02/2014,03:37:45,mail.company.co.uk, 25,220,150,Yes
26/02/2014,03:42:17,mail.company.co.uk, 25,220,70,Yes
26/02/2014,03:45:40,mail.company.co.uk, 25,220,90,Yes
26/02/2014,03:47:22,mail.company.co.uk, 25,220,100,Yes
26/02/2014,04:02:17,mail.company.co.uk, 25,220,170,Yes
26/02/2014,04:08:29,mail.company.co.uk, 25,220,160,Yes
26/02/2014,04:12:33,mail.company.co.uk, 25,220,180,Yes
26/02/2014,04:15:34,mail.company.co.uk, 25,220,70,Yes
26/02/2014,04:20:42,mail.company.co.uk, 25,220,270,Yes
26/02/2014,04:27:22,mail.company.co.uk, 25,220,170,Yes
26/02/2014,04:35:07,mail.company.co.uk, 25,220,120,Yes
26/02/2014,04:42:23,mail.company.co.uk, 25,220,160,Yes
26/02/2014,04:52:10,mail.company.co.uk, 25,220,190,Yes
26/02/2014,04:52:22,mail.company.co.uk, 25,220,300,Yes
26/02/2014,04:53:44,mail.company.co.uk, 25,220,140,Yes
26/02/2014,04:54:09,mail.company.co.uk, 25,220,200,Yes
26/02/2014,11:42:43,mail.company.co.uk, 25,n/a,Timeout,No
26/02/2014,11:43:06,mail.company.co.uk, 25,220,40,Yes
26/02/2014,11:43:27,mail.company.co.uk, 25,220,40,Yes
26/02/2014,11:48:40,mail.company.co.uk, 25,220,40,Yes
26/02/2014,11:52:56,mail.company.co.uk, 25,220,40,Yes
26/02/2014,11:57:59,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:00:37,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:16:00,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:21:48,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:30:57,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:36:21,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:40:04,mail.company.co.uk, 25,220,50,Yes
26/02/2014,12:40:13,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:44:35,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:45:01,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:49:15,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:49:33,mail.company.co.uk, 25,220,40,Yes
26/02/2014,12:50:32,mail.company.co.uk, 25,220,40,Yes
26/02/2014,13:00:50,mail.company.co.uk, 25,220,40,Yes
26/02/2014,13:01:51,mail.company.co.uk, 25,220,40,Yes
26/02/2014,13:07:47,mail.company.co.uk, 25,220,40,Yes
26/02/2014,13:08:10,mail.company.co.uk, 25,220,30,Yes
26/02/2014,13:16:54,mail.company.co.uk, 25,220,10,Yes
26/02/2014,13:40:30,mail.company.co.uk, 25,220,10,Yes
26/02/2014,13:41:19,mail.company.co.uk, 25,220,40,Yes
26/02/2014,13:41:46,mail.company.co.uk, 25,220,40,Yes
26/02/2014,13:44:21,mail.company.co.uk, 25,220,10,Yes
26/02/2014,13:51:05,mail.company.co.uk, 25,220,40,Yes
26/02/2014,13:54:20,mail.company.co.uk, 25,220,40,Yes
26/02/2014,13:57:55,mail.company.co.uk, 25,220,40,Yes
26/02/2014,14:01:19,mail.company.co.uk, 25,220,10,Yes
26/02/2014,14:06:14,mail.company.co.uk, 25,220,10,Yes
26/02/2014,14:19:29,mail.company.co.uk, 25,220,10,Yes
26/02/2014,14:27:56,mail.company.co.uk, 25,220,40,Yes
26/02/2014,14:30:49,mail.company.co.uk, 25,220,10,Yes
26/02/2014,14:37:43,mail.company.co.uk, 25,220,10,Yes
26/02/2014,14:40:33,mail.company.co.uk, 25,220,10,Yes
26/02/2014,14:47:43,mail.company.co.uk, 25,220,10,Yes
26/02/2014,14:57:07,mail.company.co.uk, 25,220,10,Yes
26/02/2014,14:58:13,mail.company.co.uk, 25,220,60,Yes
26/02/2014,14:59:10,mail.company.co.uk, 25,220,10,Yes
26/02/2014,15:01:02,mail.company.co.uk, 25,220,130,Yes
26/02/2014,15:03:01,mail.company.co.uk, 25,220,10,Yes
26/02/2014,15:03:36,mail.company.co.uk, 25,220,10,Yes
26/02/2014,15:07:27,mail.company.co.uk, 25,220,40,Yes
26/02/2014,15:10:12,mail.company.co.uk, 25,220,40,Yes
26/02/2014,15:11:54,mail.company.co.uk, 25,220,40,Yes
26/02/2014,15:18:43,mail.company.co.uk, 25,220,10,Yes
26/02/2014,15:24:15,mail.company.co.uk, 25,220,40,Yes
26/02/2014,15:29:52,mail.company.co.uk, 25,220,30,Yes
26/02/2014,18:26:55,mail.company.co.uk, 25,220,Timeout,No
26/02/2014,18:34:02,mail.company.co.uk, 25,220,70,Yes
26/02/2014,18:34:08,mail.company.co.uk, 25,220,40,Yes
26/02/2014,18:45:34,mail.company.co.uk, 25,220,40,Yes
26/02/2014,18:46:11,mail.company.co.uk, 25,220,40,Yes
26/02/2014,18:50:26,mail.company.co.uk, 25,220,40,Yes
26/02/2014,18:55:49,mail.company.co.uk, 25,220,40,Yes
26/02/2014,19:00:56,mail.company.co.uk, 25,220,40,Yes
26/02/2014,19:02:29,mail.company.co.uk, 25,220,40,Yes
26/02/2014,19:03:56,mail.company.co.uk, 25,220,40,Yes
26/02/2014,19:05:22,mail.company.co.uk, 25,220,40,Yes

Open in new window

There's nothing obvious in the server logs to say why though, especially around the suspect time.

Ping worked fine throughout, looking at the PRTG logs of both the server and the UTM device there were no bandwidth issues at the time.

CPU peaked at 20%, most of the time it was under 5%, no peaks at all the times when the timeouts occurred.

I'll take a look at MXalerts to see if that can give me any more info.
Simon Butler (Sembee)ConsultantCommented:
MX Alerts will just tell you when there is a timeout, it doesn't really help past that. As a troubleshooting tool, ping is close to useless.

What else is on the server? What is between the server and the internet? You may have to start looking outside of Exchange for the cause.

Dale303Author Commented:
I wasn't actually using ping other than to confirm that the internet as a whole was running. SLcheck was the little app I used to report on port 25 at regular intervals.

MXalerts over the last couple of hours...

Probe Date/Time, Response Date/Time, Response Time, Results
2014-02-26 22:31:12 2014-02-26 22:31:25 13 Seconds Success
2014-02-26 22:12:01 - - Failed
2014-02-26 21:31:17 2014-02-26 21:32:04 47 Seconds Success
2014-02-26 21:12:01 - - Failed
2014-02-26 20:31:39 2014-02-26 20:32:32 53 Seconds Success

I've already replaced the router . I've also called the ISP and they say nothing is wrong

Here's some more things I've just done. I'll give it until the morning and check the logs. If it's all clear I'll add them back to see which one breaks it.

I've disabled AV/AV scanning on the UTM to see if that effects anything at all.
There's an anonymous receive connector I set up for a cms to send messages through our server. I've temporarily disabled that too
I've limited the amount of RAM Exchange uses to 6GB (on 16GB machine)

After that I think it's a support call to MS.
Dale303Author Commented:
I just thought. There's another Exchange server I can use to send messages to it (no smarthost so it would be a fairly direct connection). Would the logs from that be able to tell me anything useful? If so, which logs would be the most useful?
Simon Butler (Sembee)ConsultantCommented:
The only thing the logs would tell you is when there is a delay and if there was an NDR. You would have to route all of your email through the other server for that to happen though.

That SLcheck program is pretty neat.  Where were you running that from when you got those results?
If you were running in on your local LAN, that would eliminate any router and ISP issues from the equation.
If you were running it from offsite, say at home, and you see those delays-  BUT a copy running on the LAN does NOT see any delays:  that would implicate the router/ISP and NOT the exchange server.
Dale303Author Commented:
Hi Guys.

It turns out that switching lots of things off for a weekend soak test revealed it's the cyberoam UTM playing up. If I switch on the AS/AV on it the problems start occurring. It's not all the time and the cpu/disk/bandwidth logs from the Cyberoam don't suggest a bottleneck. It must be something else wrong. I paid for support for that so I'll get them to fix now I can prove it's playing up,

Well done SLcheck (MX alerts worked too but the 30 min refresh was a bit long for this test). I would never have been able to work out from users random complaints of emails taking ages.

Anyway, thanks for sticking through this. Your help at tell me what it isn't is almost as good as what it is.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 9
  • 5
  • 5
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now