Solved

Cannot read procmail generated malboxes

Posted on 2004-03-20
22
476 Views
Last Modified: 2013-12-15
I am using fetchmail to retrieve mail from an ISP with the -m option to pass it to procmail.  These are my environment settings in .procmailrc
SHELL=/bin/sh
PMDIR=/mail
LOGFILE=/mail/pmlogs/pmlog
LOG="
"
DEFAULT=/spool/var/mail/andrew
VERBOSE=yes
MAILDIR=/var/spool/mail

This works fine. It sorts according to the recipes and delivers to the appropriate files in /var/spool/mail.  If I cat these files they appear to contain the mail they should. Each message seems to start with the line " STATUS:  R"

However:
using the linux mail command  (even -f <filenname> to ensure I am reading the right file) there are no messages.  

If I use a Windows box and Outlook to retreive using POP3 (which is the eventual aim) I get an authentication error but  /var/log/messages reports:Mar 21 15:45:05 Server ipop3d[10004]: Login failed user=<userid> auth=<userid> host=<hostname> [192.168.1.2]
Mar 21 15:45:08 Server ipop3d[10004]: Command stream end of file while reading line user=<userid> host=<hostname> [192.168.1.2]

I can login remotely using the same userid and password
Why are my mail files unreadable?
Thanks
0
Comment
Question by:ajcocker
  • 11
  • 6
  • 5
22 Comments
 
LVL 17

Expert Comment

by:owensleftfoot
ID: 10645636
What is your actual smtp program? Procmail isnt an mta - it just sorts out delivery. You should be running an smtp server such as sendmail,exim,postfix or qmail. Which one are you using?
0
 

Author Comment

by:ajcocker
ID: 10646880
None.

At this stage I only want to use fetchmail to retrieve mail from several ISP accounts, sort it with procmail and store it until it is read it using POP3 from WinXP/Outlook cliients on my home LAN.

All mail is sent directly from Outlook to my ADSL ISP providers mail service. It all works except I can't read the mailboxes procmail creates; either with Windows or Linux

The reason is partly because I don't have a static IP address and partly because of low confidence in the reliabilty of my ADSL service whcih make me reluctant to be wholly dependent upon it.
0
 
LVL 17

Expert Comment

by:owensleftfoot
ID: 10647530
The "Command stream end of file" error can sometimes happen when the pop server is trying to do an ident query on the user and it times out. Try putting an entry for each client machine in /etc/hosts on the linux box.
0
 
LVL 17

Expert Comment

by:owensleftfoot
ID: 10647553
The other thing to check are the file permissions on the /var/spool/mail/usename files - dont forget the windows users should own these files or at least have read and write access to them.
0
 

Author Comment

by:ajcocker
ID: 10655360
I don't think there is a problem with hosts.  I can ping every box on the lan by either IP address or name. /etc/hosts reads:
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1      Server      localhost.localdomain      localhost
192.168.1.5      Portable      
192.168.1.4      Living      
192.168.1.3      Rumpus      
192.168.1.2      Andrew      

I can log in either locally or remotely (using putty) as a user and cat the mail box.  
/var/spool/mail lists, in part:
-rw-rw----   1 guest    mail     0 Mar  6 12:19 guest
-rw-------   1 melanie  mail 33636 Mar 21 12:50 melanie
-rw-------   1 root     mail 51825 Mar 20 15:22 root
If  log in as melanie I get the message "You have mail."  If execute the command mail I get the message "You have no mail"

The file 'melanie' contains mail
If I execute the command mail -f  /var/spool/mail/melanie mail opens with "0 messages"
0
 

Author Comment

by:ajcocker
ID: 10655960
I have sent a sungle test message and copied below the full content of /var/spool/mail /<user>, replacing all '@' with 'at'

Status: R
Return-path: <andrewatcocker.id.au>
Received: from pop-server.bigpond.net.au [144.135.24.13]
      by localhost with POP3 (fetchmail-6.2.0)
      for rootatlocalhost (single-drop); Tue, 23 Mar 2004 19:37:30 +1100 (EST)
Received: from mta07bw.bigpond.com ([192.168.114.14])
 by mailms4ps.email.bigpond.com
 (iPlanet Messaging Server 5.2 HotFix 1.14 (built Mar 18 2003))
 with ESMTP id <0HV000EALT4RRPatmailms4ps.email.bigpond.com> for
 ajcocker%bigpond.net.auatims-ms-daemon; Tue, 23 Mar 2004 18:34:03 +1000 (EST)
Received: from cofe-daemon.mta07bw.email.bigpond.com by
 mta07bw.email.bigpond.com
 (iPlanet Messaging Server 5.2 HotFix 1.14 (built Mar 18 2003))
 id <0HV0004QAT4REYatmta07bw.email.bigpond.com> for ajcockeratbigpond.net.au
 (ORCPT ajcockeratbigpond.net.au); Tue, 23 Mar 2004 18:34:03 +1000 (EST)
Received: from tbf-daemon.mta07bw.email.bigpond.com by
 mta07bw.email.bigpond.com
 (iPlanet Messaging Server 5.2 HotFix 1.14 (built Mar 18 2003))
 id <0HV00040RT3YMCatmta07bw.email.bigpond.com> for ajcockeratbigpond.net.au;
 Tue, 23 Mar 2004 18:33:34 +1000 (EST)
Received: from bwmam12.bigpond.com ([144.135.24.103])
 by mta07bw.email.bigpond.com
 (iPlanet Messaging Server 5.2 HotFix 1.14 (built Mar 18 2003))
 with SMTP id <0HV0001NYT3XGRatmta07bw.email.bigpond.com> for
 ajcockeratbigpond.net.au; Tue, 23 Mar 2004 18:33:34 +1000 (EST)
Received: from mail.dnsvr.com ([69.10.136.60])
 by bwmam12.bigpond.com(MAM REL_3_4_2 189/12233906) with SMTP id 12233906; Tue,
 23 Mar 2004 18:33:33 +1000
Received: from gizmo04bw.bigpond.com (gizmo04bw.bigpond.com [144.140.70.14])
 by mail.dnsvr.com (Postfix) with SMTP id F0D461DE99E      for
 <melanieatcocker.id.au>; Tue, 23 Mar 2004 03:31:29 -0500
Received: (qmail 30974 invoked from network); Tue, 23 Mar 2004 08:25:59 +0000
Received: from unknown (HELO bwmam10.bigpond.com) (144.135.24.97)
 by gizmo04bw.bigpond.com with SMTP; Tue, 23 Mar 2004 08:25:59 +0000
Received: from cpe-139-168-63-173.vic.bigpond.net.au ([139.168.63.173])
 by bwmam10.bigpond.com(MAM REL_3_4_2 171/7871976) with SMTP id 7871976; Tue,
 23 Mar 2004 18:31:27 +1000
Date: Tue, 23 Mar 2004 19:31:34 +1100
From: Andrew Cocker <andrewatcocker.id.au>
Subject: Test message
To: melanieatcocker.id.au
Reply-to: andrewatcocker.id.au
Message-id: <002f01c410b1$40b6e3c0$0201a8c0atandrew>
MIME-version: 1.0
X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
X-Mailer: Microsoft Outlook, Build 10.0.4510
Content-type: multipart/alternative;
 boundary="----=_NextPart_000_0030_01C4110D.74275BC0"
Importance: Normal
X-Priority: 3 (Normal)
X-MSMail-priority: Normal
Original-recipient: rfc822;ajcockeratbigpond.net.au

This is a multi-part message in MIME format.

------=_NextPart_000_0030_01C4110D.74275BC0
Content-Type: text/plain;
      charset="us-ascii"
Content-Transfer-Encoding: 7bit

This is a test message

------=_NextPart_000_0030_01C4110D.74275BC0
Content-Type: text/html;
      charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.0.4630.0">
<TITLE>Test message</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P><FONT COLOR=3D"#000000" FACE=3D"Times New Roman">This is a test =
message </FONT>
</P>

</BODY>
</HTML>
------=_NextPart_000_0030_01C4110D.74275BC0--


0
 
LVL 17

Expert Comment

by:owensleftfoot
ID: 10656040
I dont know whether its your problem or not but "mail" will only report on new messages - if the message has already been read it will tell you you have no messages. Try using mutt or pine to open the mailbox.
0
 

Author Comment

by:ajcocker
ID: 10656388
Pine is not installed
mutt -f /var/spool/mail/melanie offers to create /home/melanie/Mail.  When I say no it exits and reports "/var/spool/mail/melanie is not a mailbox."

Don't think I said before, but I am using Fedora Core 1 with all updates applied
0
 
LVL 17

Expert Comment

by:owensleftfoot
ID: 10656416
Try deleting the line "Status: R" in the mailbox.
0
 

Author Comment

by:ajcocker
ID: 10656807
No difference.
Other mail received from my ISP using Outlook starts the same way, although I haven't yet read it. eg

Status: R
Return-path: <qna@experts-exchange.com>
Received: from mta11bw.bigpond.com ([192.168.114.143])
etc etc
0
 
LVL 9

Expert Comment

by:Alf666
ID: 10665498
The problem is that your mails should all begin with a line "From something" to be recognizable as valid mails.

Try using the "-f me@localhost" option for procmail. It should work much better.
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 
LVL 9

Expert Comment

by:Alf666
ID: 10682059
Have you tested this solution ? Do you need any more infos ?
0
 

Author Comment

by:ajcocker
ID: 10684602
I have been slow to respond because I have been testing things.  -f doesn't seem to work but I have confirmed that the problem is in the procmail options.  Trouble is I don’t really understand the manual.  

The option “/usr/sbin/procmail/ -d %T" is suggested in the manual but I don’t understand it. I have not been able to find an explanation of %T anywhere. It think it actually works OK but there is a recipe and a locking error which I think is why I get log messages like “Unable to write to var/spool/mail/_Wfd.4T8YAB.Server”

It is tiime consuming and I guess that tomorrow (Saturday) taken care of
0
 
LVL 9

Expert Comment

by:Alf666
ID: 10685987
What I can confirm is that your problem is definitely located in this leading from line. The %T is just a variable which fetchmail will replace by your "To" address

Could you post your fetchmail / procmail config (hiding the real user/password) ?
0
 

Author Comment

by:ajcocker
ID: 10692774
I have corrected one typing error which fixed the unable to write problem.  Fetchmail now seems to work OK but doesn't pass the mail onto procmail.  It just stores it according to whatever users are specified as 'here'  (andrew in this case)  The maik that ends up in /var/spool/mail/andrew is formatted OK and is readable as well as retrievable remotely using Outlook/POP3.

while diagnosing I am invoking fetchmail with
fetchmail -avvk "/usr/bin/procmail -m "/usr/bin/procmail -d %T"
FETCHMAILRC
-rw-------  1 root root 267 Mar 27 12:32 .fetchmailrc
#.fetchmailrc
set postmaster andrew
#TPG account
poll mail.tpg.com.au protocol pop3 user username there with password secret is andrew here
#Bigpond account
poll pop-server.bigpond.net.au protocol pop3 user username password secret is andrew here
# smtphost 127.0.0.1

PROCMAILRC
-rwxr--r--  1 root root 160 Mar 27 12:27 .procmailrc

SHELL=/bin/sh
PMDIR=/mail
LOGFILE=/mail/pmlogs/pmlog
LOG="
"
DEFAULT=/var/spool/mail/andrew
VERBOSE=yes
MAILDIR=/var/spool/mail
#
:0
* ^TOandrew
{
:0c
andrew
:0
/mail/backup
}
:0
* ^TOros
{
:0c
ros
:0
/mail/backup
}
:0
* ^TOannabel
{
:0c
annabel
:0
/mail/backup
}
:0
* ^TOmelanie
{
:0c
melanie
:0
/mail/backup
}
:0
* ^TOmary
{
:0c
!marys email address
:0
/mail/backup
}


0
 
LVL 9

Expert Comment

by:Alf666
ID: 10694737
Try invoking fetchmail normally, and change the following in your .fetchmailrc :

poll mail.tpg.com.au protocol pop3 user username there with password secret is andrew here and wants mda "/usr/bin/procmail -d %T"

0
 

Author Comment

by:ajcocker
ID: 10696375
It made no difference
It still worked as before but procmail was not invoked.
I also tried putting errors in the .procmailrc file to see if it reported them but it didn't.
0
 

Author Comment

by:ajcocker
ID: 10697390
I did have a copying error above, sorry. I was invoking it as:
fetchmail -avvk -m "/usr/bin/procmail -d %T"
0
 
LVL 9

Expert Comment

by:Alf666
ID: 10697940
Yes, I thought it was a typing mistake. It would not have worked either.

Stupid question, but are you sure that procmail is in /usr/bin ?

Then, using -vv should give you a hint of why procmail is not invoked. There should be an error message output by fetchmail.
0
 

Author Comment

by:ajcocker
ID: 10698280
yes it is in /usr/bin (confirmed with whereis)
 
Iimmdiately before it downloads each mail item it reports that it is going to rewrite the return path:, from: and to:
and then
Fetchmail: about to deliver with /use/bin/procmail -d andrew

It then downloads the mail OK, no other messages

I also tried this because I thought it should work:

Cat <large mail file> | /usr/bin/procmail -d <user>
If <user> is invalid it reports an error (means procmail is working) .  With a valid user it returns to the prompt almost immediately with no other action, far too quickly to have processed it.  Just like it does with fetchmail.  Is there some, other way of testing what  procmail is doing or stepping through it?.

It seems to me that it is not reading the .procmailrc file
0
 
LVL 9

Accepted Solution

by:
Alf666 earned 125 total points
ID: 10698321
Argl. i'm stupid. Try the following :

copy your .procmailrc to, /etc/procmailrc (without the dot). And try :

cat <large mail file> | /usr/bin/procmail -d <user>

It should work much better. In fact, i think it's looking for the .procmailrc inside the recipient's home directory.

You can trace procmail like this :

cat <large mail file> | strace -o /tmp/fic /usr/bin/procmail -d <user>

Then, in /tmp/fic, you will have all system calls, including stat, fstat, open (which are the ones interesting you in the search of a valid procmail config file).

0
 

Author Comment

by:ajcocker
ID: 10701797
Thanks that did it. I feel dumb.   i riginally set it up to run as user root but changed that when I realisedd that was a security risk but I didn't change the location of the .procmailrc file from /root to /home/andrew

Thanks for your help
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

How many times have you wanted to quickly do the same thing to a list but found yourself typing it again and again? I first figured out a small time saver with the up arrow to recall the last command but that can only get you so far if you have a bi…
Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now