no outbound smtp on PIX 525.

The issue we have is that we no longer have the ability to send email via SMTP from behind our Firewall.
Inbound email is unaffected.

From outside we can telnet to port 25 of the mail server and we do receive a response as follows

220 nostromo.******.co.uk VPOP3 ESMTP Server Ready
helo
250 nostromo.******.co.uk VPOP3 SMTP Server - Hello <nohostname> (87.194.180.52), pleased to meet you.

However, if we attempt to connect to a mailserver on the internet from our network then we get no response, although before yesterday we could perform this test and get a response.

We have checked that the MX record for the domain is correct and that the mail server is not blacklisted anywhere that we can see.


The firewall in question is a Pix 525 running version 7.2.3.

Up unti yesterday we were able to send outbound email via smtp using direct MX record lookups.  However we now have every outbound connection terminated by the remote host with error 10054.

We have rebooted all public facing equipment , on the off chance this may help.  We have checked the configuration of all public facing devices and cannot see anything that has obviously changed to prevent mail delivery.  We have, as far as we are aware, not made any changes to the configuration of the PIX firewall.  

Below are a sample of the logfile regarding smtp traffic and a copy of the PIX configuration.

6|Oct 08 2007|18:20:21|302014|194.214.217.19|192.168.40.9|Teardown TCP connection 35610 for outside:194.214.217.19/25 to inside:192.168.40.9/5000 duration 0:00:00 bytes 0 TCP Reset-I
6|Oct 08 2007|18:20:14|106015|82.68.251.18|aaa.bbb.ccc.9|Deny TCP (no connection) from 82.68.251.18/1884 to aaa.bbb.ccc.9/25 flags RST  on interface outside

: Saved
:
PIX Version 7.2(3)
!
hostname gateway
domain-name ******.co.uk
enable password .********* encrypted
names
dns-guard
!
interface Ethernet0
 speed 100
 duplex full
 nameif outside
 security-level 0
 ip address aaa.bbb.ccc.3 255.255.255.0
!
interface Ethernet1
 speed 100
 duplex full
 nameif inside
 security-level 100
 ip address 192.168.0.1 255.255.255.0
!
interface GigabitEthernet0
 shutdown
 nameif intf2
 security-level 4
 no ip address
!
passwd 2KFQnbNIdI.2KYOU encrypted
banner exec Disconnect NOW if you are not authorised to access this system.  By Continuing, you consent to your keystrokes and data content being monitored for lawful purposes.  Unauthorised use is a criminal offence under the Computer Misuse Act 1990.
banner motd Disconnect NOW if you are not authorised to access this system.  By Continuing, you consent to your keystrokes and data content being monitored for lawful purposes.  Unauthorised use is a criminal offence under the Computer Misuse Act 1990.
boot system flash:/image.bin
ftp mode passive
clock timezone GMT/BST 0
clock summer-time GMT/BDT recurring last Sun Mar 1:00 last Sun Oct 2:00
dns domain-lookup inside
dns server-group DefaultDNS
 name-server 192.168.0.31
 domain-name ******.co.uk
object-group service web-access tcp
 port-object eq ftp
 port-object eq www
 port-object eq https
object-group service vpop3 tcp
 description vpop3 combination smtp and webmail
 port-object range 5108 5108
 port-object eq smtp
access-list inside_nat0_outbound extended permit ip any 192.168.99.64 255.255.255.192
access-list outside_access_in extended permit tcp any aaa.bbb.ccc.128 255.255.255.128 object-group web-access
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.19 eq https
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.9 object-group vpop3
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.51 eq smtp
pager lines 24
logging enable
logging timestamp
logging list exactvpn level informational class vpnc
logging buffered errors
logging trap warnings
logging asdm informational
logging queue 0
logging host inside 192.168.0.70
logging class vpn trap informational
mtu outside 1500
mtu inside 1500
mtu intf2 1500
ip local pool exactvpn 192.168.99.64-192.168.99.127 mask 255.255.255.192
no failover
monitor-interface outside
monitor-interface inside
monitor-interface intf2
icmp unreachable rate-limit 1 burst-size 1
asdm image flash:/asdm.bin
asdm history enable
arp timeout 14400
nat-control
global (outside) 1 interface
global (outside) 10 aaa.bbb.ccc.6 netmask 255.255.255.0
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 10 0.0.0.0 0.0.0.0
static (inside,outside) aaa.bbb.ccc.128 192.168.40.128 netmask 255.255.255.128
static (inside,outside) aaa.bbb.ccc.30 192.168.40.30 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.19 192.168.40.19 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.9 192.168.40.9 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.51 192.168.40.70 netmask 255.255.255.255
access-group outside_access_in in interface outside
route outside 0.0.0.0 0.0.0.0 aaa.bbb.ccc.1 1
route inside 192.168.40.0 255.255.255.0 192.168.0.21 1
route inside 192.168.11.0 255.255.255.0 192.168.0.21 1
route inside 192.168.15.0 255.255.255.0 192.168.0.21 1
route inside 192.168.3.0 255.255.255.0 192.168.0.21 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout uauth 0:05:00 absolute
aaa-server TACACS+ protocol tacacs+
aaa-server RADIUS protocol radius
http server enable
http 192.168.3.70 255.255.255.255 inside
http 192.168.0.70 255.255.255.255 inside
http 192.168.10.70 255.255.255.255 inside
snmp-server host inside 192.168.3.1 poll community public
no snmp-server location
no snmp-server contact
snmp-server community public
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec transform-set ESP-DES-SHA esp-des esp-sha-hmac
crypto dynamic-map outside_dyn_map 20 set pfs
crypto dynamic-map outside_dyn_map 20 set transform-set ESP-DES-SHA
crypto map outside_map 65535 ipsec-isakmp dynamic outside_dyn_map
crypto map outside_map interface outside
crypto isakmp identity hostname
crypto isakmp enable outside
crypto isakmp enable inside
crypto isakmp policy 10
 authentication pre-share
 encryption des
 hash sha
 group 1
 lifetime 86400
crypto isakmp policy 30
 authentication pre-share
 encryption des
 hash sha
 group 2
 lifetime 86400
crypto isakmp policy 65535
 authentication pre-share
 encryption des
 hash sha
 group 2
 lifetime 86400
telnet 192.168.0.70 255.255.255.255 inside
telnet 192.168.10.70 255.255.255.255 inside
telnet timeout 50
ssh timeout 5
ssh version 1
console timeout 0
!
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns migrated_dns_map_1
 parameters
  message-length maximum 512
policy-map global_policy
 class inspection_default
  inspect dns migrated_dns_map_1
  inspect ftp
  inspect h323 h225
  inspect h323 ras
  inspect netbios
  inspect rsh
  inspect rtsp
  inspect skinny
  inspect sqlnet
  inspect sunrpc
  inspect tftp
  inspect sip
  inspect xdmcp
  inspect http
  inspect pptp
  inspect esmtp
!
service-policy global_policy global
ntp authenticate
ntp server 192.168.0.227 source inside prefer
group-policy exact-staff internal
group-policy exact-staff attributes
 dns-server value 192.168.0.19 192.168.0.31
 vpn-tunnel-protocol IPSec
 default-domain value exact.ds
username spencer password ********* encrypted privilege 15
username spencer attributes
 vpn-group-policy DfltGrpPolicy
username tricky password ****************** encrypted privilege 15
username tricky attributes
 vpn-group-policy DfltGrpPolicy
username rdent password ***************** encrypted privilege 0
username rdent attributes
 vpn-group-policy DfltGrpPolicy
username tripitaka password ***************** encrypted privilege 15
username tripitaka attributes
 vpn-group-policy DfltGrpPolicy
username djg3ex password ********************. encrypted privilege 0
username djg3ex attributes
 vpn-group-policy DfltGrpPolicy
tunnel-group DefaultRAGroup ipsec-attributes
 isakmp ikev1-user-authentication (outside) none
tunnel-group exact-staff type ipsec-ra
tunnel-group exact-staff general-attributes
 address-pool exactvpn
 default-group-policy exact-staff
tunnel-group exact-staff ipsec-attributes
 pre-shared-key *
prompt hostname context
asdm image flash:/asdm.bin
asdm location 192.168.0.70 255.255.255.255 inside
asdm history enable


Regards


Spencer Clark
vodyanoiAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

grbladesCommented:
Add the following to the configuration.

policy-map global_policy
 class inspection_default
  no inspect esmtp
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
grbladesCommented:
The esmtp inspection forces the mail transfer to be 100% standards compliant which not all mail server software is. The esmtp inspection is pretty useless imho as most people need to turn it off anyway.

As to why it was working previously perhaps some software has been upgraded?
0
lrmooreCommented:
>Up unti yesterday we were able to send outbound email via smtp using direct MX record lookups
What changed yesterday? If it works at all, it is not typically a configuration problem.
0
Get Certified for a Job in Cybersecurity

Want an exciting career in an emerging field? Earn your MS in Cybersecurity and get certified in ethical hacking or computer forensic investigation. WGU’s MSCSIA degree program was designed to meet the most recent U.S. Department of Homeland Security (DHS) and NSA guidelines.  

vodyanoiAuthor Commented:
Thank you for the prompt reply.  Unfortunately, however it appears to have made no difference. We are still experiencing the 10054 server disconnect errors.

Following up on this I guess further inspection of our network is needed.  

In answer to the question - what has changed since yesterday? As far as I am aware no changes have been made to any network components for over a week ?

0
grbladesCommented:
What mail server software are you using?
From your 1st post I take it is VPOP3

Have you tried telenting to a destination mail server on port 25 to see if you can establish a connection?
0
vodyanoiAuthor Commented:
Yes the mailserver is VPOP3.

No we cannot telnet to port 25 of an external mailserver. If we attempt this through windows the screen stays balnk for a few seconds then returns to a prompt.

If we do the same from a unix server (SCO Openserver 5.0.7) we get the following result

[callunix] # telnet mailcluster.zen.co.uk 25
Trying 212.23.3.230...
Connected to mailcluster.zen.co.uk.
Escape character is '^]'.
SetSockOpt: Connection reset by peer
Connection closed by foreign host.

If I try this from a machine on a separate DSL connection to the internet, which does not use the PIX firewall, I get a normal response

220 schroedinger.zen.co.uk ESMTP Exim 4.50 Mon, 08 Oct 2007 20:03:00 +0000
421 schroedinger.zen.co.uk: SMTP command timeout - closing connection

Also if I try our own mail server from the DSL connection I can connect and receive a response

220 nostromo.exact3ex.co.uk VPOP3 ESMTP Server Ready
helo
250 nostromo.exact3ex.co.uk VPOP3 SMTP Server - Hello <nohostname> (87.194.180.52), pleased to meet you

regards


Spencer Clark



0
lrmooreCommented:
>access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.51 eq smtp
>static (inside,outside) aaa.bbb.ccc.51 192.168.40.70
>route inside 192.168.40.0 255.255.255.0 192.168.0.21 1
Looks like your mail server is not local to your PIX, but routes through another router. It could be something on that router?
What is out in front of  your PIX? Another router with T1? What about its config? Anything on it that might be blocking? What about active IPS system? Anything on the LAN that could be intercepting?
0
vodyanoiAuthor Commented:
After much investigation and redirecting our outbound email to our ISP we have discovered the following

Having asked our ISP to monitor this connection we received the following response from them.
-------
Our engineer has detrmined that your mail server is trying to set TCP keepalives which our servers don't like.

sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315]
setsockopt(SO_KEEPALIVE) on
connection from 194.143.179.9 failed: Invalid argument
sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315] SMTP connection from mail.exact3ex.co.uk [194.143.179.9]:2280 I=[0.0.0.0]:0 lost
sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315] no MAIL in SMTP connection from mail.exact3ex.co.uk [194.143.179.9]:2280 I=[0.0.0.0]:0 D=0s
-------
This being the case we took the decision to migrate the smtp service off the Pix Firewall to a Watchguard X2500 which had been in storage.  As soon as we migrated outbound smtp to this new firewall the smtp service worked correctly and we were able to send outbound email.

Moving forwards I will setup a test mail server to run through the Pix Firewall and see if I can get to the cause of this issue
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Email Protocols

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.