Solved

Postfix Spamassassin Amavisd Clamav: Slow down

Posted on 2006-11-11
20
784 Views
Last Modified: 2011-09-20
We are using Postfix Spamassassin Amavisd ClamAV under Redhat 9 as an email gateway to an Exchange Server.
During heavy loads the system becomes is very slow (many minutes) to respond to a command promt and the mail just stacks up in the Postfix queue. Oftentimes a reboot will help.

The maillog shows an inordinate amount of time is spent during the AMAVIS fwd-data-end stage.
What does "fwd-data-end" indicate?
Where do I go to check/fix whatever controls  "fwd-data-end"?
Is this the cause of the severe slowdowns or should I bark up another tree (I am already howling at the moon)?
Thanks,
Jerry

For example, some TIMING entries:

Nov 10 13:59:25 ISISDSPAM amavis[4563]: (04563-07-19) TIMING [total 14410 ms] - SMTP pre-DATA-flush: 10 (0%)0, SMTP DATA: 107 (1%)1, body_digest: 1 (0%)1, gen_mail_id: 0 (0%)1, mime_decode: 9 (0%)1, get-file-type2: 108 (1%)2, decompose_part: 1 (0%)2, parts_decode: 0 (0%)2, AV-scan-1: 11 (0%)2, spam-wb-list: 1 (0%)2, SA msg read: 1 (0%)2, SA parse: 1 (0%)2, SA check: 4149 (29%)31, SA finish: 2 (0%)31, update_cache: 1 (0%)31, decide_mail_destiny: 1 (0%)31, fwd-connect: 5 (0%)31, fwd-mail-from: 1 (0%)31, fwd-rcpt-to: 1 (0%)31, fwd-data-cmd: 0 (0%)31, write-header: 0 (0%)31, fwd-data-contents: 0 (0%)31, fwd-data-end: 9578 (66%)97, fwd-rundown: 108 (1%)98, prepare-dsn: 68 (0%)98, main_log_entry: 246 (2%)100, unlink-2-files: 1 (0%)100, rundown: 0 (0%)100

Nov 10 13:59:26 ISISDSPAM amavis[4562]: (04562-07-22) TIMING [total 6196 ms] - SMTP pre-DATA-flush: 2 (0%)0, SMTP DATA: 29 (0%)1, body_digest: 1 (0%)1, gen_mail_id: 0 (0%)1, mime_decode: 194 (3%)4, get-file-type1: 10 (0%)4, decompose_part: 1 (0%)4, parts_decode: 0 (0%)4, spam-wb-list: 3 (0%)4, update_cache: 0 (0%)4, decide_mail_destiny: 1 (0%)4, fwd-connect: 399 (6%)10, fwd-mail-from: 1 (0%)10, fwd-rcpt-to: 303 (5%)15, fwd-data-cmd: 0 (0%)15, write-header: 0 (0%)15, fwd-data-contents: 0 (0%)15, fwd-data-end: 5194 (84%)99, fwd-rundown: 21 (0%)99, prepare-dsn: 12 (0%)100, main_log_entry: 23 (0%)100, unlink-1-files: 1 (0%)100, rundown: 0 (0%)100

Nov 10 13:59:37 ISISDSPAM amavis[4562]: (04562-07-23) TIMING [total 10596 ms] - SMTP pre-DATA-flush: 9 (0%)0, SMTP DATA: 32 (0%)0, body_digest: 1 (0%)0, gen_mail_id: 0 (0%)0, mime_decode: 5 (0%)0, get-file-type1: 10 (0%)1, decompose_part: 1 (0%)1, parts_decode: 0 (0%)1, spam-wb-list: 3 (0%)1, update_cache: 0 (0%)1, decide_mail_destiny: 1 (0%)1, fwd-connect: 5 (0%)1, fwd-mail-from: 1 (0%)1, fwd-rcpt-to: 1 (0%)1, fwd-data-cmd: 0 (0%)1, write-header: 1 (0%)1, fwd-data-contents: 0 (0%)1, fwd-data-end: 10520 (99%)100, fwd-rundown: 1 (0%)100, prepare-dsn: 1 (0%)100, main_log_entry: 4 (0%)100, unlink-1-files: 0 (0%)100, rundown: 0 (0%)100

Nov 10 13:59:49 ISISDSPAM amavis[4562]: (04562-07-24) TIMING [total 11917 ms] - SMTP pre-DATA-flush: 2 (0%)0, SMTP DATA: 69 (1%)1, body_digest: 1 (0%)1, gen_mail_id: 0 (0%)1, mime_decode: 438 (4%)4, get-file-type2: 10 (0%)4, parts_decode: 0 (0%)4, AV-scan-1: 11 (0%)4, spam-wb-list: 2 (0%)4, SA msg read: 1 (0%)4, SA parse: 2 (0%)4, SA check: 1988 (17%)21, SA finish: 2 (0%)21, update_cache: 1 (0%)21, decide_mail_destiny: 2 (0%)21, fwd-connect: 5 (0%)21, fwd-mail-from: 1 (0%)21, fwd-rcpt-to: 1 (0%)21, fwd-data-cmd: 0 (0%)21, write-header: 0 (0%)21, fwd-data-contents: 1 (0%)21, fwd-data-end: 9372 (79%)100, fwd-rundown: 1 (0%)100, prepare-dsn: 1 (0%)100, main_log_entry: 6 (0%)100, unlink-2-files: 1 (0%)100, rundown: 0 (0%)100

Nov 10 13:59:50 ISISDSPAM amavis[4563]: (04563-07-20) TIMING [total 24444 ms] - SMTP pre-DATA-flush: 106 (0%)0, SMTP DATA: 133 (1%)1, body_digest: 1 (0%)1, gen_mail_id: 0 (0%)1, mime_decode: 275 (1%)2, get-file-type3: 16 (0%)2, parts_decode: 0 (0%)2, AV-scan-1: 36 (0%)2, spam-wb-list: 1 (0%)2, SA msg read: 3 (0%)2, SA parse: 5 (0%)2, SA check: 3527 (14%)17, SA finish: 2 (0%)17, update_cache: 1 (0%)17, decide_mail_destiny: 1 (0%)17, fwd-connect: 5 (0%)17, fwd-mail-from: 1 (0%)17, fwd-rcpt-to: 1 (0%)17, fwd-data-cmd: 0 (0%)17, write-header: 0 (0%)17, fwd-data-contents: 3 (0%)17, fwd-data-end: 20211 (83%)100, fwd-rundown: 24 (0%)100, prepare-dsn: 0 (0%)100, main_log_entry: 90 (0%)100, unlink-3-files: 1 (0%)100, rundown: 0 (0%)100
0
Comment
Question by:jinfeld
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 12
  • 8
20 Comments
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17929673
could you give us specs on the machine, the /etc/hosts, /etc/resolv.conf files. Is this machine on a dmz zone? Is it behind a firewall. Can dns resolve this host in a timely manner?
post back the results
0
 

Author Comment

by:jinfeld
ID: 17931387
How do I check if DNS (our local Windows 2003 Server is located at 192.168.254.4) resolves the host in a timely manner?

CPU: AMD Sempron 64 3400+ Palermo in
Memory: 1 Gb 400MHz DDR Non-ECC
MBd: ASUS K8V-MX Socket 754 VIA K8M800
HD: Seagate 200GB IDE
No firewall
No DMZ

/etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1       ISISDSPAM       localhost.localdomain   localhost

/etc/resolv.conf
; generated by /sbin/dhclient-script
;search isisDSPAM
nameserver 192.168.254.4
nameserver 206.13.28.12

Thanks,
Jerry
0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17931472
what is the ip of the RH machine, is it fixed?
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 5

Expert Comment

by:ircpamanager
ID: 17931512
your postfix server needs a hostname. to see if it resolves it, type in command prompt   ping RH Hostname<<< or whatever th ename of the box is. if it returns the ip address of RH box with reply message then it is resolving it in the dns.
0
 

Author Comment

by:jinfeld
ID: 17931604
[root@ISISDSPAM root]# ping ISISDSPAM
PING ISISDSPAM (127.0.0.1) 56(84) bytes of data.
64 bytes from ISISDSPAM (127.0.0.1): icmp_seq=1 ttl=64 time=0.021 ms
64 bytes from ISISDSPAM (127.0.0.1): icmp_seq=2 ttl=64 time=0.015 ms
64 bytes from ISISDSPAM (127.0.0.1): icmp_seq=3 ttl=64 time=0.011 ms

The static IP address fo rthe RH box:
 inet addr:192.168.254.3

Thanks,
Jerry
0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17932036
if you ping ISISDSPAM from anohter machine does it return 192.168.254.3
0
 

Author Comment

by:jinfeld
ID: 17932435
No
Should I create an HIST record in the Win 2003 DNS server to point ISISDSPAM to 192.168.254.3?
Thanks
0
 
LVL 5

Accepted Solution

by:
ircpamanager earned 250 total points
ID: 17932612
you should create an A record in Win 2003 DNS. Then you should add this to your /etc/hosts file
/etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1       ISISDSPAM       localhost.localdomain   localhost
192.168.254.3 ISISDSPAM.yourdomain.com  ISISDSPAM

then type    service network restart as root or sudo.

then try pinging it again from other machine
0
 

Author Comment

by:jinfeld
ID: 17932852
/etc/hosts  now looks like this:
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1       ISISDSPAM       localhost.localdomain   localhost
192.168.254.3   ISISDSPAM       MotionPro2.local        ISISDSPAM

The DNS "A" record on the Win2003 server now looks like this:
HOST: ISISDSPAM
FQDN: ISISDSPAM.MotionPro2.local
IP: 192.168.254.3

Now I can ping ISISDSPAM from any Windows PC on the network.
How does this affect the Spam box since all the Postfix to AMAVSID to SA to Postfix is handled internally on the solo RH box?

Thanks,
Jerry

0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17933110
/etc/hosts  now looks like this:
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1       ISISDSPAM       localhost.localdomain   localhost
192.168.254.3   ISISDSPAM       MotionPro2.local        ISISDSPAM

chnage 192.168.254.3   ISISDSPAM       MotionPro2.local        ISISDSPAM
to 192.168.254.3   ISISDSPAM.MotionPro2.local        ISISDSPAM
you are just using this as a realy correct? so in your main.cf you have exchange server as relay?
If exchange cant resolve RH box it will take forever.
0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17933285
do you have content_filter line in main.cf?
0
 

Author Comment

by:jinfeld
ID: 17933392
/etc/postfix/main.cf
content_filter = smtp-amavis:[127.0.0.1]:10024

Should the Win2003 DNS "A" reference the Windows Domain: ISISDSPAM.MotionPro2.local        
Or should it rerence the RH Postfix domain (this is the domain list in the email header):  isisdspam.spam.motionpro.com

Jerry
0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17933611
try commenting out content_filter and check the speed
0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17933649
"Or should it rerence the RH Postfix domain (this is the domain list in the email header):  isisdspam.spam.motionpro.com"
is the RH box under a different domain?If it is why do you have it this way?If they are both(exchange and postfix box)on same subnet then same domain makes more sense.
The "A" record should point to the same domain the Win 2003 box is apart of ISISDSPAM.MotionPro2.local
0
 

Author Comment

by:jinfeld
ID: 17933659
Will that stop any email checking by ClamAV and SA?
Jerry
0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17933772
yes and no, it will turn off advanced content filter
have a look here
http://www.postfix.org/FILTER_README.html
0
 

Author Comment

by:jinfeld
ID: 17934770
Hi,
A ping on the Exchange server box now resolves to the spam box IP.
In /etc/postfix/main.cf  I commented out
#content_filter = smtp-amavis:[127.0.0.1]:10024

ran postfix reload

Was there anything else to reload or restart?

Timing seems to be unaffected with the bulk going to fwd-data-end:

Nov 13 15:15:52 ISISDSPAM amavis[5126]: (05126-02-3) TIMING [total 28317 ms] - SMTP pre-DATA-flush: 1 (0%)0, SMTP DATA: 39 (0%)0, body_hash: 1 (0%)0, gen_mail_id: 0 (0%)0, mime_decode: 17 (0%)0, get-file-type2: 13 (0%)0, parts_decode: 0 (0%)0, AV-scan-1: 76 (0%)1, spam-wb-list: 1 (0%)1, SA msg read: 1 (0%)1, SA parse: 3 (0%)1, SA check: 5392 (19%)20, update_cache: 1 (0%)20, fwd-connect: 8 (0%)20, fwd-mail-from: 1 (0%)20, fwd-rcpt-to: 83 (0%)20, write-header: 2 (0%)20, fwd-data: 1 (0%)20, fwd-data-end: 22668 (80%)100, fwd-rundown: 1 (0%)100, post-do_spam: 1 (0%)100, main_log_entry: 6 (0%)100, unlink-2-files: 1 (0%)100, rundown: 0 (0%)100

Nov 13 15:16:06 ISISDSPAM amavis[5441]: (05441-01-2) TIMING [total 11530 ms] - SMTP pre-DATA-flush: 1 (0%)0, SMTP DATA: 77 (1%)1, body_hash: 1 (0%)1, gen_mail_id: 0 (0%)1, mime_decode: 12 (0%)1, get-file-type2: 14 (0%)1, parts_decode: 0 (0%)1, AV-scan-1: 16 (0%)1, spam-wb-list: 1 (0%)1, SA msg read: 1 (0%)1, SA parse: 2 (0%)1, SA check: 4174 (36%)37, update_cache: 1 (0%)37, fwd-connect: 4 (0%)37, fwd-mail-from: 1 (0%)37, fwd-rcpt-to: 2 (0%)37, write-header: 2 (0%)37, fwd-data: 0 (0%)37, fwd-data-end: 7214 (63%)100, fwd-rundown: 1 (0%)100, main_log_entry: 6 (0%)100, unlink-2-files: 1 (0%)100, rundown: 0 (0%)100
0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17938476
do a postfix reload
0
 

Author Comment

by:jinfeld
ID: 17942504
I believe the DNS issue helped the SPam box throug-put greatly. The true fix for the "fwd-data-end" slowup was what you alluded to earlier: double checking on the second pass-through Postfix.
Using the recommendations on:
http://www.ijs.si/software/amavisd/README.postfix.txt 

I used this settings in /etc/postfix/master.cf
 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
to stop the second pass filtering. (see the full web page for this discussion).

Thanks for your thorough consideration and help!
Jerry
0
 
LVL 5

Expert Comment

by:ircpamanager
ID: 17946433
no problem I am glad I could help.
0

Featured Post

NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I am a long time windows user and for me it is normal to have spaces in directory and file names. Changing to Linux I found myself frustrated when I moved my windows data over to my new Linux computer. The problem occurs when at the command line.…
It’s 2016. Password authentication should be dead — or at least close to dying. But, unfortunately, it has not traversed Quagga stage yet. Using password authentication is like laundering hotel guest linens with a washboard — it’s Passé.
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

687 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question