asked on

Exchange SMTP fails to receive data line if message includes attachment.

I've got a client running SBS 2003 with updates all current. This is a new client to me and the prior technician never got their e-mail system working properly. The problem is in seemingly randomly dropped inbound mail. The guaranteed method of dropping an inbound mail item is to have a large attachment (roughly 200kb plus). When I send test messages from my exchange server to theirs, I can sometimes get simple messages through with no attachments but if the message follows one that fails, the next messages will often fail as well. I experience the same thing when sending from my hotmail account. I have removed the ip inspect rules and any firewall from the Cisco 1811 Router this Server sits behind and have only a simple ACL and NAT applied which seems to route fine and "should" not be the problem.

I've got message tracking enabled on both servers and when a message successfully sends to the client's server, message tracking on our server says "message transferred to mail.customer.com through smtp." When they fail, I get "message transferred to (blank) through smtp" as if it wanted to say to who and it thinks the process went properly but it clearly failed on the other end.

OK, on the customer server, the failure occurs as soon as the e-mail data is to be sent. It is after the "xexch50" line with exchange to exchange communication and after the "RCPT" line with Hotmail as the sending domain. Here are some log entries...

4 e-mails sent with e-mail # 3 failing.
2007-04-26 21:04:55 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 EHLO +mail.axeontech.com 250 0 312 23 0 SMTP - - - -
2007-04-26 21:04:55 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 MAIL +FROM:<jason@axeontech.com> 250 0 44 31 0 SMTP - - - -
2007-04-26 21:04:56 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 RCPT +TO:<administrator@towntoyota.com> 250 0 41 38 0 SMTP - - - -
2007-04-26 21:04:56 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 xexch50 +2128+2 504 0 32 14 0 SMTP - - - -
2007-04-26 21:05:04 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 BDAT +<7D24ED99711E8D4587588A6F4A5A9D36024B25@axeonserver.AxeonTech.local> 250 0 106 8931 6140 SMTP - - - -
2007-04-26 21:05:04 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 QUIT mail.axeontech.com 240 9969 68 4 0 SMTP - - - -
2007-04-26 21:05:11 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 EHLO +mail.axeontech.com 250 0 312 23 0 SMTP - - - -
2007-04-26 21:05:11 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 MAIL +FROM:<jason@axeontech.com> 250 0 44 31 0 SMTP - - - -
2007-04-26 21:05:13 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 RCPT +TO:<administrator@towntoyota.net> 250 0 41 38 0 SMTP - - - -
2007-04-26 21:05:13 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 xexch50 +2128+2 504 0 32 14 0 SMTP - - - -
2007-04-26 21:05:19 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 BDAT +<7D24ED99711E8D4587588A6F4A5A9D36024B26@axeonserver.AxeonTech.local> 250 0 106 8931 5093 SMTP - - - -
2007-04-26 21:05:19 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 QUIT mail.axeontech.com 240 8922 68 4 0 SMTP - - - -
2007-04-26 21:05:31 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 EHLO +mail.axeontech.com 250 0 312 23 0 SMTP - - - -
2007-04-26 21:05:31 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 MAIL +FROM:<jason@axeontech.com> 250 0 44 31 0 SMTP - - - -
2007-04-26 21:05:32 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 RCPT +TO:<administrator@towntoyota.com> 250 0 41 38 0 SMTP - - - -
2007-04-26 21:05:32 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 xexch50 +2092+2 504 0 32 14 0 SMTP - - - -
2007-04-26 21:06:18 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 EHLO +mail.axeontech.com 250 0 312 23 0 SMTP - - - -
2007-04-26 21:06:18 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 MAIL +FROM:<jason@axeontech.com> 250 0 44 31 0 SMTP - - - -
2007-04-26 21:06:19 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 RCPT +TO:<administrator@towntoyota.net> 250 0 41 38 0 SMTP - - - -
2007-04-26 21:06:19 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 xexch50 +2092+2 504 0 32 14 0 SMTP - - - -
2007-04-26 21:06:47 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 BDAT +<7D24ED99711E8D4587588A6F4A5A9D36024B28@axeonserver.AxeonTech.local> 250 0 106 49230 26641 SMTP - - - -
2007-04-26 21:06:47 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 QUIT mail.axeontech.com 240 30375 68 4 0 SMTP - - - -

With the Quit line for the failed mail occuring 7 minutes later...
2007-04-26 21:13:21 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 QUIT mail.axeontech.com 240 470328 0 16 467015 SMTP - - - -

Here is the Hotmail communication...

2007-04-27 15:54:38 65.54.246.93 bay0-omc1-s21.bay0.hotmail.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 EHLO +bay0-omc1-s21.bay0.hotmail.com 250 0 311 35 0 SMTP - - - -
2007-04-27 15:54:38 65.54.246.93 bay0-omc1-s21.bay0.hotmail.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 MAIL +FROM:<reachjasonhere@hotmail.com> 250 0 51 38 0 SMTP - - - -
2007-04-27 15:54:40 65.54.246.93 bay0-omc1-s21.bay0.hotmail.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 RCPT +TO:<administrator@towntoyota.com> 250 0 41 38 0 SMTP - - - -

With the Quit line appearing 8 minutes later...
2007-04-27 16:02:30 65.54.246.93 bay0-omc1-s21.bay0.hotmail.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 QUIT bay0-omc1-s21.bay0.hotmail.com 240 477844 0 16 469516 SMTP - - - -

I did in one test this morning see a "timeout" in the meassage logging, looked like this...

2007-04-27 15:33:30 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 EHLO +mail.axeontech.com 250 0 312 23 0 SMTP - - - -
2007-04-27 15:33:30 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 MAIL +FROM:<jason@axeontech.com> 250 0 44 31 0 SMTP - - - -
2007-04-27 15:33:32 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 RCPT +TO:<administrator@towntoyota.net> 250 0 41 38 0 SMTP - - - -
2007-04-27 15:33:32 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 xexch50 +2176+2 504 0 32 14 0 SMTP - - - -

then
2007-04-27 15:47:09 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 TIMEOUT mail.axeontech.com 121 497137728 38 16 815250 SMTP - - - -
2007-04-27 15:47:09 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 QUIT mail.axeontech.com 240 824734 38 16 815250 SMTP - - - -

I've got the Intelligent Message Filter running and it is archiving everything it blocks so I know that's not where the messages are going and they are not in junk mail folders.

Big mystery to me at this point. 500 Points.

Jason

Sembee

What else is on the server?
AV, Antispam etc?

The DNS looks fine, so it isn't that, so you have to start looking at SMTP traffic interference.

Simon.

AxeonTech

ASKER

They were having DNS errors with their ".net" domain but all that is ironed out now. There is no AntiSpam solution currently installed and the AntiVirus is Symantec Corporate 10.1 at this point. The previous tech was supposed to install Enterprise with Exchange protection and AntiSpam but did not. It is my intention to install those products but not until this is worked out. I really don't see how Corporate AV could be the cause but I don't see how this could be happening in the first place either.

Sembee

Based on my experience with Symantec's products, I would immediately point the finger at the AV software from that company. Whether or not it is supposed to be causing a problem, it usually does. I would suggest removing it and seeing if everything works correctly. If it does, replace Symantec with something that works.

Simon.

AxeonTech

ASKER

Boy do I wish that were the case. I've removed every shred of Symantec software and ran the latest version of their thorough "NoNav" tool with no improvement in performance. Now I have a "naked" server and about 50 clients with no AV "mama" to run home to.

Here is the lates test of one e-mail with no attachment and the next with a 400kb .XLS file...

2007-04-28 05:14:08 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 EHLO +mail.axeontech.com 250 0 312 23 0 SMTP - - - -
2007-04-28 05:14:08 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 MAIL +FROM:<jason@axeontech.com> 250 0 44 31 0 SMTP - - - -
2007-04-28 05:14:09 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 RCPT +TO:<administrator@towntoyota.com> 250 0 41 38 0 SMTP - - - -
2007-04-28 05:14:09 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 xexch50 +2128+2 504 0 32 14 0 SMTP - - - -
2007-04-28 05:14:18 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 BDAT +<7D24ED99711E8D4587588A6F4A5A9D36024B3E@axeonserver.AxeonTech.local> 250 0 106 8931 4922 SMTP - - - -
2007-04-28 05:14:25 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 MAIL +FROM:<jason@axeontech.com> 250 0 44 31 0 SMTP - - - -
2007-04-28 05:14:25 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 RCPT +TO:<administrator@towntoyota.com> 250 0 41 38 0 SMTP - - - -
2007-04-28 05:14:26 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 xexch50 +2092+2 504 0 32 14 0 SMTP - - - -

and 7 minutes later...
2007-04-28 05:21:30 64.184.161.29 mail.axeontech.com SMTPSVC1 TOYOTASERVER 10.17.162.205 0 QUIT mail.axeontech.com 240 483921 0 16 423234 SMTP - - - -

Notice the lack of a "quit" line from the 1st e-mail? There seems to be something fundamentally wrong with SMTP communications here. Perhaps I need to create a new Virtual SMTP server and see how that works? Any other ideas?

AxeonTech

ASKER

I created a new Virtual SMTP Server in case the default instance was corrupted in any way. Nothing improved and symptoms remain the same. So, no Firewall, no AV or AS, still no attachments.

Sembee

What do you mean by "no firewall"?
Have you got your servers connected to the internet with no protection at all?

Simon.

AxeonTech

ASKER

Cisco 1811 running NAT and ACL -that's it right now as I'm troubleshooting this problem. Guess you could call that a firewall but there's no IP/Packet Inspection going on that could interfere.

AxeonTech

ASKER

I've now tried varying the MTU settings with no improvement and am getting ready to just reload the server which is way more effort than I even want to consider. Any other ideas?

AxeonTech

ASKER

OK, here we are 2 weeks later and I still don't have a solution. Anyone have any ideas on this one? It appears we're losing info at the packet level and I'm trying to figure out just where in the "chain" that loss is coming. The device chain is DSL Modem to Cisco 1811 Router to Cisco Switches to Network Devices (clients) including the E-Mail server. Any larger attachments just don't come it (over 100kb or so). It's like a black hole with no NDR and lots of head-scratching. I'm goin to try replacing network hardware tomorrow and troubleshoot that way.

AxeonTech

ASKER

I'm still working on the exact solution but after further troubleshooting I started replacing hardware. The cause is the Cisco 1811 Router and the application of one ACL and a series of inspect rules. Thanks for the input of others.

AxeonTech

ASKER

All is well with SMTP. Can't believe the amount of hours I put into it. It was the ACL config on the Cisco router in spite of several Cisco Techs saying it couldn't be and even testing it in their lab. We created new ACLs and replaced the original one and voila! Thanks for your feedback everyone but unless someone sees it differently, I don't see a points winner other than dumb luck on my part.

ASKER CERTIFIED SOLUTION

Computer101

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Ben Hart

I'd like to ask, what were the specific problems with your ACL.. did you ever find out?

AxeonTech

ASKER

Never found what was wrong with it. Just went to the new ACL and never looked back. Sorry.

Ben Hart

Hmm.. do you possibly have the old and new for comparison?