i2mc
asked on
How do I configure Sendmail to Queue when using SMART_HOST
Hi
I have a linux server sitting behind a firewall that makes it unable to send mail directly to the outside world. Instead it is configured to relay all mail through an approved smtp relay host. This works - except that sending mail is *really* slow - typically 30 - 40 secs per message.
The mails are sent from a CGI application, so this delay is not acceptable and I need another solution. I know I can change my application to workaround the problem, but I feel that sendmail should be able to handle my requirement.
What I'd like to do is have the sendmail running on my application server to accept the messages instantly and then hand them off to the smtp relay host as and when it can.
I've been reading the docs and surfing the web for a day now and seem no closer to finding a solution - yet this must be a common requirement and I must be missing something obvious.
All help appreciated
Thanks
Roger
UK
I have a linux server sitting behind a firewall that makes it unable to send mail directly to the outside world. Instead it is configured to relay all mail through an approved smtp relay host. This works - except that sending mail is *really* slow - typically 30 - 40 secs per message.
The mails are sent from a CGI application, so this delay is not acceptable and I need another solution. I know I can change my application to workaround the problem, but I feel that sendmail should be able to handle my requirement.
What I'd like to do is have the sendmail running on my application server to accept the messages instantly and then hand them off to the smtp relay host as and when it can.
I've been reading the docs and surfing the web for a day now and seem no closer to finding a solution - yet this must be a common requirement and I must be missing something obvious.
All help appreciated
Thanks
Roger
UK
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
>Sounds like a DNS issue to me. Does your CGI connect to localhost or the system's host name?
>Or is it invoking sendmail directly?
I'm using the Perl Mail::Sender module to run my smtp session as this module does the business nicely on my other servers which are able to send mail directly, rather than having to relay through someone else. I'd prefer to keep to this method as it gives me windows portability.
I connect to "localhost", my hosts file contains a valid definition as you say.
If I SSH to my application server and invoke sendmail directly, I get the same problem, but maybe WHERE the delay is will help you folk pin down where the issue is [ I've changed sensitive info but left the intention clear I hope ] ;
[root@vault root]# sendmail -v someone@somewhere
hello from vault
.
[ after submitting this, the following output happens almost instantly - which I presume is my local sendmail starting the conversation with the relay I have to use ]
someone@somewhere... Connecting to [127.0.0.1] via relay...
220 myhost.mydomain ESMTP Sendmail 8.12.8/8.12.8; Fri, 6 Aug 2004 11:24:03 +0100
>>> EHLO mydomain
250-myhost.mydomain Hello localhost.localdomain [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-DELIVERBY
250 HELP
>>> MAIL From:<root@mydomain> SIZE=18
[ At this point there is a huge delay - 30/40 secs as I mentioned before. Then I get the rest of the output as shown below. ]
250 2.1.0 <root@mydomain>... Sender ok
>>> RCPT To:<someone@somewhere>
>>> DATA
250 2.1.5 <someone@somewhere>... Recipient ok
354 Enter mail, end with "." on a line by itself
>>> .
250 2.0.0 i76AO3Zc015016 Message accepted for delivery
someone@somewhere... Sent (i76AO3Zc015016 Message accepted for delivery)
Closing connection to [127.0.0.1]
>>> QUIT
221 2.0.0 myhost.mydomain closing connection
does that help clarify which part of the process is causing me problems?
Many Thanks
Roger
PS. I'm not ignoring the sendmail.cf advice - I'd just like to be a bit clearer as to what the problem is before I start doing low-level magic.
>Or is it invoking sendmail directly?
I'm using the Perl Mail::Sender module to run my smtp session as this module does the business nicely on my other servers which are able to send mail directly, rather than having to relay through someone else. I'd prefer to keep to this method as it gives me windows portability.
I connect to "localhost", my hosts file contains a valid definition as you say.
If I SSH to my application server and invoke sendmail directly, I get the same problem, but maybe WHERE the delay is will help you folk pin down where the issue is [ I've changed sensitive info but left the intention clear I hope ] ;
[root@vault root]# sendmail -v someone@somewhere
hello from vault
.
[ after submitting this, the following output happens almost instantly - which I presume is my local sendmail starting the conversation with the relay I have to use ]
someone@somewhere... Connecting to [127.0.0.1] via relay...
220 myhost.mydomain ESMTP Sendmail 8.12.8/8.12.8; Fri, 6 Aug 2004 11:24:03 +0100
>>> EHLO mydomain
250-myhost.mydomain Hello localhost.localdomain [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-DELIVERBY
250 HELP
>>> MAIL From:<root@mydomain> SIZE=18
[ At this point there is a huge delay - 30/40 secs as I mentioned before. Then I get the rest of the output as shown below. ]
250 2.1.0 <root@mydomain>... Sender ok
>>> RCPT To:<someone@somewhere>
>>> DATA
250 2.1.5 <someone@somewhere>... Recipient ok
354 Enter mail, end with "." on a line by itself
>>> .
250 2.0.0 i76AO3Zc015016 Message accepted for delivery
someone@somewhere... Sent (i76AO3Zc015016 Message accepted for delivery)
Closing connection to [127.0.0.1]
>>> QUIT
221 2.0.0 myhost.mydomain closing connection
does that help clarify which part of the process is causing me problems?
Many Thanks
Roger
PS. I'm not ignoring the sendmail.cf advice - I'd just like to be a bit clearer as to what the problem is before I start doing low-level magic.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks for the help jlevie
OK so the situation now is that I have changed *nothing* and yet it now works fine. The problem seemed to be the DNS lookup of the recipient that was taking time and what's frustrating is that a) I've changed nothing and b) don't know if the administrator of the dns servers I have to use has done anything either. My application runs on a server inside the most protected network I've ever worked with and in addition to firewalls, appliances and acl filters, I have content filters and all sorts to contend with ... so there is much that is out of my control :(
All that said, I'd still prefer a solution that lets my cgi program hand off mail to my local sendmail instantly so that the user can get on with other stuff. Whether or not the mail gets delivered is sendmails problem and I can parse mail logs to get this data later on. Can I perhaps control WHEN my local sendmail attempts to relay through the smart host?
I think this takes me back to the realm of pablouruguay's response and I'd appreciate some more assistance with configuring sendmail in this manner. I'm surprised it isn't covered in the sendmail docs but maybe I've missed it.
Thanks again
Roger
OK so the situation now is that I have changed *nothing* and yet it now works fine. The problem seemed to be the DNS lookup of the recipient that was taking time and what's frustrating is that a) I've changed nothing and b) don't know if the administrator of the dns servers I have to use has done anything either. My application runs on a server inside the most protected network I've ever worked with and in addition to firewalls, appliances and acl filters, I have content filters and all sorts to contend with ... so there is much that is out of my control :(
All that said, I'd still prefer a solution that lets my cgi program hand off mail to my local sendmail instantly so that the user can get on with other stuff. Whether or not the mail gets delivered is sendmails problem and I can parse mail logs to get this data later on. Can I perhaps control WHEN my local sendmail attempts to relay through the smart host?
I think this takes me back to the realm of pablouruguay's response and I'd appreciate some more assistance with configuring sendmail in this manner. I'm surprised it isn't covered in the sendmail docs but maybe I've missed it.
Thanks again
Roger
"I have changed *nothing* and yet it now works fine." tends to support it being a DNS issue. Changes in upstream equipment or DNS server could make the problem better or worse and be outside of your view or control. You haven't said what version of Linux this is, but assuming it is similar to a RedHat system I'd like to see the contents of:
/etc/hosts
/etc/resolv.conf
/etc/sysconfig/network
and the output from 'ifconfig -a' and 'netstat -nr'. I'm fairly sure that something in that data will reveal the cause of the problem. Given that this syetm is behind firewalls & etc. There's really no need to obfuscate the data and doing so may make it impossible to determine where the problem lies.
The normal behaviour of sendmail is to accept a message, place it in queue, and then start a delivery process for the message. Sendmail will do checks on the sender of the message (as part of anti-spam control) when the sender issues a "MAIL From" command. That's were you are getting the delay and it has nothing, per se, to do with the fact that you are using a SMART HOST. Including 'FEATURE(`delay_checks') in the sendmail.mc may help in this sort of situation. However, the real solution is to find an fix the basic problem that causes the delay.
/etc/hosts
/etc/resolv.conf
/etc/sysconfig/network
and the output from 'ifconfig -a' and 'netstat -nr'. I'm fairly sure that something in that data will reveal the cause of the problem. Given that this syetm is behind firewalls & etc. There's really no need to obfuscate the data and doing so may make it impossible to determine where the problem lies.
The normal behaviour of sendmail is to accept a message, place it in queue, and then start a delivery process for the message. Sendmail will do checks on the sender of the message (as part of anti-spam control) when the sender issues a "MAIL From" command. That's were you are getting the delay and it has nothing, per se, to do with the fact that you are using a SMART HOST. Including 'FEATURE(`delay_checks') in the sendmail.mc may help in this sort of situation. However, the real solution is to find an fix the basic problem that causes the delay.
ASKER
>You haven't said what version of Linux this is
Sorry - Red Hat 9.1
>I'd like to see the contents of: /etc/hosts /etc/resolv.conf /etc/sysconfig/network and the output from 'ifconfig -a' and 'netstat -nr'
As requested. Apologies for maintaining a degree of obfuscation, but I have been consistent such that xxx and yyy are the same values everywhere mentioned and "somewhere" is consistent as well. I'm happy to run other checks if this seems to throw up anything odd. FWIW I've run my domains (sending and receiving) through dnsreport.com and nothing fails at that level.
[root@vault root]# more /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
xxx.yyy.171.148 vault.somewhere.tv
xxx.yyy.171.147 stream.somewhere.tv
xxx.yyy.171.149 snap.somewhere.tv
[root@vault root]# more /etc/resolv.conf
search somewhereeurope.com somewhere.tv
nameserver xxx.yyy.171.139
nameserver xxx.yyy.171.15
nameserver xxx.yyy.171.16
[root@vault root]# more /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=vault.somewhere.t
GATEWAY=xxx.yyy.171.253
[root@vault root]# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:0D:60:14:31:F0
inet addr:xxx.yyy.171.148 Bcast:xxx.yyy.171.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:26688435 errors:0 dropped:0 overruns:0 frame:0
TX packets:26151985 errors:11 dropped:0 overruns:0 carrier:0
collisions:7864 txqueuelen:100
RX bytes:1354148019 (1291.4 Mb) TX bytes:4245220561 (4048.5 Mb)
Interrupt:24 Memory:fe010000-fe020000
eth1 Link encap:Ethernet HWaddr 00:0D:60:14:31:F1
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:25 Memory:fe030000-fe040000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:138219 errors:0 dropped:0 overruns:0 frame:0
TX packets:138219 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:83171388 (79.3 Mb) TX bytes:83171388 (79.3 Mb)
[root@vault root]# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
xxx.yyy.171.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 xxx.yyy.171.253 0.0.0.0 UG 0 0 0 eth0
[root@vault root]#
Does that help at all?
I like the sound of FEATURE ('delay_checks'), I'll read the docs on that one - I also appreciate that the underlying problem needs to be resolved ... but I'd still prefer sendmail not to hang the cgi app. At the end of the day I can solve this by routing email through an asynchronous agent I run on another server - but it's windows and I've had problems with sending email from windows servers in the past due to some low level perl sockets malarchy so going down that route may open a new can of worms...
Many thanks for the assistance thus far, I'll increase the points for this Q as it seems not to be quite as easy as I first thought.
Cheers
Roger
Sorry - Red Hat 9.1
>I'd like to see the contents of: /etc/hosts /etc/resolv.conf /etc/sysconfig/network and the output from 'ifconfig -a' and 'netstat -nr'
As requested. Apologies for maintaining a degree of obfuscation, but I have been consistent such that xxx and yyy are the same values everywhere mentioned and "somewhere" is consistent as well. I'm happy to run other checks if this seems to throw up anything odd. FWIW I've run my domains (sending and receiving) through dnsreport.com and nothing fails at that level.
[root@vault root]# more /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
xxx.yyy.171.148 vault.somewhere.tv
xxx.yyy.171.147 stream.somewhere.tv
xxx.yyy.171.149 snap.somewhere.tv
[root@vault root]# more /etc/resolv.conf
search somewhereeurope.com somewhere.tv
nameserver xxx.yyy.171.139
nameserver xxx.yyy.171.15
nameserver xxx.yyy.171.16
[root@vault root]# more /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=vault.somewhere.t
GATEWAY=xxx.yyy.171.253
[root@vault root]# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:0D:60:14:31:F0
inet addr:xxx.yyy.171.148 Bcast:xxx.yyy.171.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:26688435 errors:0 dropped:0 overruns:0 frame:0
TX packets:26151985 errors:11 dropped:0 overruns:0 carrier:0
collisions:7864 txqueuelen:100
RX bytes:1354148019 (1291.4 Mb) TX bytes:4245220561 (4048.5 Mb)
Interrupt:24 Memory:fe010000-fe020000
eth1 Link encap:Ethernet HWaddr 00:0D:60:14:31:F1
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:25 Memory:fe030000-fe040000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:138219 errors:0 dropped:0 overruns:0 frame:0
TX packets:138219 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:83171388 (79.3 Mb) TX bytes:83171388 (79.3 Mb)
[root@vault root]# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
xxx.yyy.171.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 xxx.yyy.171.253 0.0.0.0 UG 0 0 0 eth0
[root@vault root]#
Does that help at all?
I like the sound of FEATURE ('delay_checks'), I'll read the docs on that one - I also appreciate that the underlying problem needs to be resolved ... but I'd still prefer sendmail not to hang the cgi app. At the end of the day I can solve this by routing email through an asynchronous agent I run on another server - but it's windows and I've had problems with sending email from windows servers in the past due to some low level perl sockets malarchy so going down that route may open a new can of worms...
Many thanks for the assistance thus far, I'll increase the points for this Q as it seems not to be quite as easy as I first thought.
Cheers
Roger
ASKER
point increase
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Sounds like a DNS issue to me. Does your CGI connect to localhost or the system's host name? Or is it invoking sendmail directly?
In general a CGI should be executing sendmail to send messages. On a recent Linux with Sendmail 8.12 or later that will use the MSA instance of Sendmail, which in turn will use the localhost IP to pass the message to the MTA. For this to work properly /etc/hosts must contain a valid definition for the localhost, something like:
127.0.0.1 localhost.localdomain localhost
A side benefit of this method is that your CGI can set the sender with the -f option to Sendmail. That allows bounces to go to a sane place instead of just to the postmaster.