vodyanoi
asked on
no outbound smtp on PIX 525.
The issue we have is that we no longer have the ability to send email via SMTP from behind our Firewall.
Inbound email is unaffected.
From outside we can telnet to port 25 of the mail server and we do receive a response as follows
220 nostromo.******.co.uk VPOP3 ESMTP Server Ready
helo
250 nostromo.******.co.uk VPOP3 SMTP Server - Hello <nohostname> (87.194.180.52), pleased to meet you.
However, if we attempt to connect to a mailserver on the internet from our network then we get no response, although before yesterday we could perform this test and get a response.
We have checked that the MX record for the domain is correct and that the mail server is not blacklisted anywhere that we can see.
The firewall in question is a Pix 525 running version 7.2.3.
Up unti yesterday we were able to send outbound email via smtp using direct MX record lookups. However we now have every outbound connection terminated by the remote host with error 10054.
We have rebooted all public facing equipment , on the off chance this may help. We have checked the configuration of all public facing devices and cannot see anything that has obviously changed to prevent mail delivery. We have, as far as we are aware, not made any changes to the configuration of the PIX firewall.
Below are a sample of the logfile regarding smtp traffic and a copy of the PIX configuration.
6|Oct 08 2007|18:20:21|302014|194.2
6|Oct 08 2007|18:20:14|106015|82.68
: Saved
:
PIX Version 7.2(3)
!
hostname gateway
domain-name ******.co.uk
enable password .********* encrypted
names
dns-guard
!
interface Ethernet0
speed 100
duplex full
nameif outside
security-level 0
ip address aaa.bbb.ccc.3 255.255.255.0
!
interface Ethernet1
speed 100
duplex full
nameif inside
security-level 100
ip address 192.168.0.1 255.255.255.0
!
interface GigabitEthernet0
shutdown
nameif intf2
security-level 4
no ip address
!
passwd 2KFQnbNIdI.2KYOU encrypted
banner exec Disconnect NOW if you are not authorised to access this system. By Continuing, you consent to your keystrokes and data content being monitored for lawful purposes. Unauthorised use is a criminal offence under the Computer Misuse Act 1990.
banner motd Disconnect NOW if you are not authorised to access this system. By Continuing, you consent to your keystrokes and data content being monitored for lawful purposes. Unauthorised use is a criminal offence under the Computer Misuse Act 1990.
boot system flash:/image.bin
ftp mode passive
clock timezone GMT/BST 0
clock summer-time GMT/BDT recurring last Sun Mar 1:00 last Sun Oct 2:00
dns domain-lookup inside
dns server-group DefaultDNS
name-server 192.168.0.31
domain-name ******.co.uk
object-group service web-access tcp
port-object eq ftp
port-object eq www
port-object eq https
object-group service vpop3 tcp
description vpop3 combination smtp and webmail
port-object range 5108 5108
port-object eq smtp
access-list inside_nat0_outbound extended permit ip any 192.168.99.64 255.255.255.192
access-list outside_access_in extended permit tcp any aaa.bbb.ccc.128 255.255.255.128 object-group web-access
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.19 eq https
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.9 object-group vpop3
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.51 eq smtp
pager lines 24
logging enable
logging timestamp
logging list exactvpn level informational class vpnc
logging buffered errors
logging trap warnings
logging asdm informational
logging queue 0
logging host inside 192.168.0.70
logging class vpn trap informational
mtu outside 1500
mtu inside 1500
mtu intf2 1500
ip local pool exactvpn 192.168.99.64-192.168.99.1
no failover
monitor-interface outside
monitor-interface inside
monitor-interface intf2
icmp unreachable rate-limit 1 burst-size 1
asdm image flash:/asdm.bin
asdm history enable
arp timeout 14400
nat-control
global (outside) 1 interface
global (outside) 10 aaa.bbb.ccc.6 netmask 255.255.255.0
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 10 0.0.0.0 0.0.0.0
static (inside,outside) aaa.bbb.ccc.128 192.168.40.128 netmask 255.255.255.128
static (inside,outside) aaa.bbb.ccc.30 192.168.40.30 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.19 192.168.40.19 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.9 192.168.40.9 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.51 192.168.40.70 netmask 255.255.255.255
access-group outside_access_in in interface outside
route outside 0.0.0.0 0.0.0.0 aaa.bbb.ccc.1 1
route inside 192.168.40.0 255.255.255.0 192.168.0.21 1
route inside 192.168.11.0 255.255.255.0 192.168.0.21 1
route inside 192.168.15.0 255.255.255.0 192.168.0.21 1
route inside 192.168.3.0 255.255.255.0 192.168.0.21 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout uauth 0:05:00 absolute
aaa-server TACACS+ protocol tacacs+
aaa-server RADIUS protocol radius
http server enable
http 192.168.3.70 255.255.255.255 inside
http 192.168.0.70 255.255.255.255 inside
http 192.168.10.70 255.255.255.255 inside
snmp-server host inside 192.168.3.1 poll community public
no snmp-server location
no snmp-server contact
snmp-server community public
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec transform-set ESP-DES-SHA esp-des esp-sha-hmac
crypto dynamic-map outside_dyn_map 20 set pfs
crypto dynamic-map outside_dyn_map 20 set transform-set ESP-DES-SHA
crypto map outside_map 65535 ipsec-isakmp dynamic outside_dyn_map
crypto map outside_map interface outside
crypto isakmp identity hostname
crypto isakmp enable outside
crypto isakmp enable inside
crypto isakmp policy 10
authentication pre-share
encryption des
hash sha
group 1
lifetime 86400
crypto isakmp policy 30
authentication pre-share
encryption des
hash sha
group 2
lifetime 86400
crypto isakmp policy 65535
authentication pre-share
encryption des
hash sha
group 2
lifetime 86400
telnet 192.168.0.70 255.255.255.255 inside
telnet 192.168.10.70 255.255.255.255 inside
telnet timeout 50
ssh timeout 5
ssh version 1
console timeout 0
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns migrated_dns_map_1
parameters
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns migrated_dns_map_1
inspect ftp
inspect h323 h225
inspect h323 ras
inspect netbios
inspect rsh
inspect rtsp
inspect skinny
inspect sqlnet
inspect sunrpc
inspect tftp
inspect sip
inspect xdmcp
inspect http
inspect pptp
inspect esmtp
!
service-policy global_policy global
ntp authenticate
ntp server 192.168.0.227 source inside prefer
group-policy exact-staff internal
group-policy exact-staff attributes
dns-server value 192.168.0.19 192.168.0.31
vpn-tunnel-protocol IPSec
default-domain value exact.ds
username spencer password ********* encrypted privilege 15
username spencer attributes
vpn-group-policy DfltGrpPolicy
username tricky password ****************** encrypted privilege 15
username tricky attributes
vpn-group-policy DfltGrpPolicy
username rdent password ***************** encrypted privilege 0
username rdent attributes
vpn-group-policy DfltGrpPolicy
username tripitaka password ***************** encrypted privilege 15
username tripitaka attributes
vpn-group-policy DfltGrpPolicy
username djg3ex password ********************. encrypted privilege 0
username djg3ex attributes
vpn-group-policy DfltGrpPolicy
tunnel-group DefaultRAGroup ipsec-attributes
isakmp ikev1-user-authentication (outside) none
tunnel-group exact-staff type ipsec-ra
tunnel-group exact-staff general-attributes
address-pool exactvpn
default-group-policy exact-staff
tunnel-group exact-staff ipsec-attributes
pre-shared-key *
prompt hostname context
asdm image flash:/asdm.bin
asdm location 192.168.0.70 255.255.255.255 inside
asdm history enable
Regards
Spencer Clark
Inbound email is unaffected.
From outside we can telnet to port 25 of the mail server and we do receive a response as follows
220 nostromo.******.co.uk VPOP3 ESMTP Server Ready
helo
250 nostromo.******.co.uk VPOP3 SMTP Server - Hello <nohostname> (87.194.180.52), pleased to meet you.
However, if we attempt to connect to a mailserver on the internet from our network then we get no response, although before yesterday we could perform this test and get a response.
We have checked that the MX record for the domain is correct and that the mail server is not blacklisted anywhere that we can see.
The firewall in question is a Pix 525 running version 7.2.3.
Up unti yesterday we were able to send outbound email via smtp using direct MX record lookups. However we now have every outbound connection terminated by the remote host with error 10054.
We have rebooted all public facing equipment , on the off chance this may help. We have checked the configuration of all public facing devices and cannot see anything that has obviously changed to prevent mail delivery. We have, as far as we are aware, not made any changes to the configuration of the PIX firewall.
Below are a sample of the logfile regarding smtp traffic and a copy of the PIX configuration.
6|Oct 08 2007|18:20:21|302014|194.2
6|Oct 08 2007|18:20:14|106015|82.68
: Saved
:
PIX Version 7.2(3)
!
hostname gateway
domain-name ******.co.uk
enable password .********* encrypted
names
dns-guard
!
interface Ethernet0
speed 100
duplex full
nameif outside
security-level 0
ip address aaa.bbb.ccc.3 255.255.255.0
!
interface Ethernet1
speed 100
duplex full
nameif inside
security-level 100
ip address 192.168.0.1 255.255.255.0
!
interface GigabitEthernet0
shutdown
nameif intf2
security-level 4
no ip address
!
passwd 2KFQnbNIdI.2KYOU encrypted
banner exec Disconnect NOW if you are not authorised to access this system. By Continuing, you consent to your keystrokes and data content being monitored for lawful purposes. Unauthorised use is a criminal offence under the Computer Misuse Act 1990.
banner motd Disconnect NOW if you are not authorised to access this system. By Continuing, you consent to your keystrokes and data content being monitored for lawful purposes. Unauthorised use is a criminal offence under the Computer Misuse Act 1990.
boot system flash:/image.bin
ftp mode passive
clock timezone GMT/BST 0
clock summer-time GMT/BDT recurring last Sun Mar 1:00 last Sun Oct 2:00
dns domain-lookup inside
dns server-group DefaultDNS
name-server 192.168.0.31
domain-name ******.co.uk
object-group service web-access tcp
port-object eq ftp
port-object eq www
port-object eq https
object-group service vpop3 tcp
description vpop3 combination smtp and webmail
port-object range 5108 5108
port-object eq smtp
access-list inside_nat0_outbound extended permit ip any 192.168.99.64 255.255.255.192
access-list outside_access_in extended permit tcp any aaa.bbb.ccc.128 255.255.255.128 object-group web-access
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.19 eq https
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.9 object-group vpop3
access-list outside_access_in extended permit tcp any host aaa.bbb.ccc.51 eq smtp
pager lines 24
logging enable
logging timestamp
logging list exactvpn level informational class vpnc
logging buffered errors
logging trap warnings
logging asdm informational
logging queue 0
logging host inside 192.168.0.70
logging class vpn trap informational
mtu outside 1500
mtu inside 1500
mtu intf2 1500
ip local pool exactvpn 192.168.99.64-192.168.99.1
no failover
monitor-interface outside
monitor-interface inside
monitor-interface intf2
icmp unreachable rate-limit 1 burst-size 1
asdm image flash:/asdm.bin
asdm history enable
arp timeout 14400
nat-control
global (outside) 1 interface
global (outside) 10 aaa.bbb.ccc.6 netmask 255.255.255.0
nat (inside) 0 access-list inside_nat0_outbound
nat (inside) 10 0.0.0.0 0.0.0.0
static (inside,outside) aaa.bbb.ccc.128 192.168.40.128 netmask 255.255.255.128
static (inside,outside) aaa.bbb.ccc.30 192.168.40.30 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.19 192.168.40.19 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.9 192.168.40.9 netmask 255.255.255.255
static (inside,outside) aaa.bbb.ccc.51 192.168.40.70 netmask 255.255.255.255
access-group outside_access_in in interface outside
route outside 0.0.0.0 0.0.0.0 aaa.bbb.ccc.1 1
route inside 192.168.40.0 255.255.255.0 192.168.0.21 1
route inside 192.168.11.0 255.255.255.0 192.168.0.21 1
route inside 192.168.15.0 255.255.255.0 192.168.0.21 1
route inside 192.168.3.0 255.255.255.0 192.168.0.21 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout uauth 0:05:00 absolute
aaa-server TACACS+ protocol tacacs+
aaa-server RADIUS protocol radius
http server enable
http 192.168.3.70 255.255.255.255 inside
http 192.168.0.70 255.255.255.255 inside
http 192.168.10.70 255.255.255.255 inside
snmp-server host inside 192.168.3.1 poll community public
no snmp-server location
no snmp-server contact
snmp-server community public
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec transform-set ESP-DES-SHA esp-des esp-sha-hmac
crypto dynamic-map outside_dyn_map 20 set pfs
crypto dynamic-map outside_dyn_map 20 set transform-set ESP-DES-SHA
crypto map outside_map 65535 ipsec-isakmp dynamic outside_dyn_map
crypto map outside_map interface outside
crypto isakmp identity hostname
crypto isakmp enable outside
crypto isakmp enable inside
crypto isakmp policy 10
authentication pre-share
encryption des
hash sha
group 1
lifetime 86400
crypto isakmp policy 30
authentication pre-share
encryption des
hash sha
group 2
lifetime 86400
crypto isakmp policy 65535
authentication pre-share
encryption des
hash sha
group 2
lifetime 86400
telnet 192.168.0.70 255.255.255.255 inside
telnet 192.168.10.70 255.255.255.255 inside
telnet timeout 50
ssh timeout 5
ssh version 1
console timeout 0
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns migrated_dns_map_1
parameters
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns migrated_dns_map_1
inspect ftp
inspect h323 h225
inspect h323 ras
inspect netbios
inspect rsh
inspect rtsp
inspect skinny
inspect sqlnet
inspect sunrpc
inspect tftp
inspect sip
inspect xdmcp
inspect http
inspect pptp
inspect esmtp
!
service-policy global_policy global
ntp authenticate
ntp server 192.168.0.227 source inside prefer
group-policy exact-staff internal
group-policy exact-staff attributes
dns-server value 192.168.0.19 192.168.0.31
vpn-tunnel-protocol IPSec
default-domain value exact.ds
username spencer password ********* encrypted privilege 15
username spencer attributes
vpn-group-policy DfltGrpPolicy
username tricky password ****************** encrypted privilege 15
username tricky attributes
vpn-group-policy DfltGrpPolicy
username rdent password ***************** encrypted privilege 0
username rdent attributes
vpn-group-policy DfltGrpPolicy
username tripitaka password ***************** encrypted privilege 15
username tripitaka attributes
vpn-group-policy DfltGrpPolicy
username djg3ex password ********************. encrypted privilege 0
username djg3ex attributes
vpn-group-policy DfltGrpPolicy
tunnel-group DefaultRAGroup ipsec-attributes
isakmp ikev1-user-authentication (outside) none
tunnel-group exact-staff type ipsec-ra
tunnel-group exact-staff general-attributes
address-pool exactvpn
default-group-policy exact-staff
tunnel-group exact-staff ipsec-attributes
pre-shared-key *
prompt hostname context
asdm image flash:/asdm.bin
asdm location 192.168.0.70 255.255.255.255 inside
asdm history enable
Regards
Spencer Clark
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
>Up unti yesterday we were able to send outbound email via smtp using direct MX record lookups
What changed yesterday? If it works at all, it is not typically a configuration problem.
What changed yesterday? If it works at all, it is not typically a configuration problem.
ASKER
Thank you for the prompt reply. Unfortunately, however it appears to have made no difference. We are still experiencing the 10054 server disconnect errors.
Following up on this I guess further inspection of our network is needed.
In answer to the question - what has changed since yesterday? As far as I am aware no changes have been made to any network components for over a week ?
Following up on this I guess further inspection of our network is needed.
In answer to the question - what has changed since yesterday? As far as I am aware no changes have been made to any network components for over a week ?
What mail server software are you using?
From your 1st post I take it is VPOP3
Have you tried telenting to a destination mail server on port 25 to see if you can establish a connection?
From your 1st post I take it is VPOP3
Have you tried telenting to a destination mail server on port 25 to see if you can establish a connection?
ASKER
Yes the mailserver is VPOP3.
No we cannot telnet to port 25 of an external mailserver. If we attempt this through windows the screen stays balnk for a few seconds then returns to a prompt.
If we do the same from a unix server (SCO Openserver 5.0.7) we get the following result
[callunix] # telnet mailcluster.zen.co.uk 25
Trying 212.23.3.230...
Connected to mailcluster.zen.co.uk.
Escape character is '^]'.
SetSockOpt: Connection reset by peer
Connection closed by foreign host.
If I try this from a machine on a separate DSL connection to the internet, which does not use the PIX firewall, I get a normal response
220 schroedinger.zen.co.uk ESMTP Exim 4.50 Mon, 08 Oct 2007 20:03:00 +0000
421 schroedinger.zen.co.uk: SMTP command timeout - closing connection
Also if I try our own mail server from the DSL connection I can connect and receive a response
220 nostromo.exact3ex.co.uk VPOP3 ESMTP Server Ready
helo
250 nostromo.exact3ex.co.uk VPOP3 SMTP Server - Hello <nohostname> (87.194.180.52), pleased to meet you
regards
Spencer Clark
No we cannot telnet to port 25 of an external mailserver. If we attempt this through windows the screen stays balnk for a few seconds then returns to a prompt.
If we do the same from a unix server (SCO Openserver 5.0.7) we get the following result
[callunix] # telnet mailcluster.zen.co.uk 25
Trying 212.23.3.230...
Connected to mailcluster.zen.co.uk.
Escape character is '^]'.
SetSockOpt: Connection reset by peer
Connection closed by foreign host.
If I try this from a machine on a separate DSL connection to the internet, which does not use the PIX firewall, I get a normal response
220 schroedinger.zen.co.uk ESMTP Exim 4.50 Mon, 08 Oct 2007 20:03:00 +0000
421 schroedinger.zen.co.uk: SMTP command timeout - closing connection
Also if I try our own mail server from the DSL connection I can connect and receive a response
220 nostromo.exact3ex.co.uk VPOP3 ESMTP Server Ready
helo
250 nostromo.exact3ex.co.uk VPOP3 SMTP Server - Hello <nohostname> (87.194.180.52), pleased to meet you
regards
Spencer Clark
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
After much investigation and redirecting our outbound email to our ISP we have discovered the following
Having asked our ISP to monitor this connection we received the following response from them.
-------
Our engineer has detrmined that your mail server is trying to set TCP keepalives which our servers don't like.
sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315]
setsockopt(SO_KEEPALIVE) on
connection from 194.143.179.9 failed: Invalid argument
sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315] SMTP connection from mail.exact3ex.co.uk [194.143.179.9]:2280 I=[0.0.0.0]:0 lost
sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315] no MAIL in SMTP connection from mail.exact3ex.co.uk [194.143.179.9]:2280 I=[0.0.0.0]:0 D=0s
-------
This being the case we took the decision to migrate the smtp service off the Pix Firewall to a Watchguard X2500 which had been in storage. As soon as we migrated outbound smtp to this new firewall the smtp service worked correctly and we were able to send outbound email.
Moving forwards I will setup a test mail server to run through the Pix Firewall and see if I can get to the cause of this issue
Having asked our ISP to monitor this connection we received the following response from them.
-------
Our engineer has detrmined that your mail server is trying to set TCP keepalives which our servers don't like.
sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315]
setsockopt(SO_KEEPALIVE) on
connection from 194.143.179.9 failed: Invalid argument
sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315] SMTP connection from mail.exact3ex.co.uk [194.143.179.9]:2280 I=[0.0.0.0]:0 lost
sand1 - exim[29315]: [ID 197553 mail.info] 2007-10-08 14:01:33 [29315] no MAIL in SMTP connection from mail.exact3ex.co.uk [194.143.179.9]:2280 I=[0.0.0.0]:0 D=0s
-------
This being the case we took the decision to migrate the smtp service off the Pix Firewall to a Watchguard X2500 which had been in storage. As soon as we migrated outbound smtp to this new firewall the smtp service worked correctly and we were able to send outbound email.
Moving forwards I will setup a test mail server to run through the Pix Firewall and see if I can get to the cause of this issue
As to why it was working previously perhaps some software has been upgraded?