Avatar of ujoe
 asked on

Help with shell script to count logins from different IPs (to same account) in exim_mainlog.

I can figure out parts of this, but am having a difficult time putting it all together.

Here's the scenario:

This is a Redhat Linux server running a newer version of exim, with WHM/cPanel. I am the administrator with full root/shell access.

The exim_mainlog file contains multiple logins to the same email account from different IP addresses. This is an indication that the email user's password has been stolen and is being used to broadcast spam, depending of course on the number of different IP addresses logging in, within a short period of time. Here is a short example list of entries, just to illustrate this situation:

2012-03-10 16:31:19 1S6Uoo-0002d2-17 <= sales@customerdomain.com.au H=(customerdomain.com.au) [] P=esmtpa A=courier_login:sales@customerdomain.com.au S=830
2012-03-10 16:31:24 1S6Uor-0002cc-V2 <= sales@customerdomain.com.au H=(customerdomain.com.au) [] P=esmtpa A=courier_login:sales@customerdomain.com.au S=838
2012-03-10 16:31:27 1S6Uov-0002e3-KX <= sales@customerdomain.com.au H=(customerdomain.com.au) [] P=esmtpa A=courier_login:sales@customerdomain.com.au S=839

Of course, these entries would be more numerous and mixed in with many other logins to many other email accounts on the server.

What I need to come up with is a shell script that counts the number of DIFFERENT IPs logging into the same account, e.g. within a tail of something like the last 100 entries.

I've found this line, which works great to count the number of exim logins per account:
egrep -o 'login[^ ]+' /var/log/exim_mainlog | sort|uniq -c|sort -nk 1

... and I can even get it to work for the last 100 entires, like this:
egrep -o 'login[^ ]+' /var/log/exim_mainlog | tail -100 | sort|uniq -c|sort -nk 1

But what I need is output that tells me how many DIFFERENT IPs have logged in to that account.

In other words, what I get from the script lines above is output like this:

     14 login:cccc@domainuber.com
     15 login:aaaa@domainlanka.com
     15 login:dddd@domaingreat.com

The leading number indicates the number of logins to each account.

But what I need to see instead is output something like the following:

14   2     login:jbs+domainuber.com
15   1     login:cmesa@domainlanka.com
15   12   login:rjc@domaingreat.com

... where the second number in each entry indicates the number of different IPs involved with the logins for each specific account.

I have also tried to work this out in a "for" loop, e.g.

for i in `egrep -o 'login[^ ]+' /var/log/exim_mainlog | tail -100 | sort|uniq -c|sort -nk 1 | awk '{print $1}'`
if [ $i -gt 12 ]
echo $i

Trying to establish 12 as a maximum threshold, but with this I am getting even less info than before, certainly not the number of DIFFERENT IP addresses logging into each account.

Any help would be greatly appreciated.

Web ServersApache Web ServerShell Scripting

Avatar of undefined
Last Comment

8/22/2022 - Mon

A loop seems the easiest way to go.  I've modified your existing line to write the tail output to a temporary file, so that we use it once to get the list of user logins, and then later on count the number of different IP addresses.
egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | sort|uniq -c|while read num email; do
  echo $num $(awk "/$email/{print \$7}" tmp_exim|sort -u|wc -l) $email
done; rm tmp_exim

Open in new window

As requested, each output line gives the number of logins, the number of unique IP addresses, and the email address.

Thanks for your work. However, I am not seeing accurate results.

When I ran the script:

egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | sort|uniq -c|while read num email; do
  echo $num $(awk "/$email/{print \$7}" tmp_exim|sort -u|wc -l) $email
done; rm tmp_exim

... one of the email accounts in approximately the middle of the result list (I have only checked this one result.) Shows this (I changed the email address for privacy):

10 10 login:info@domain.com

Then I carefully isolated the last 100 lines of the exim_mainlog file, put it into another file named tester.txt Then when I did a grep on the /var/log/exim_mainlog file, like this:

grep info@domain.com tester.txt | grep login

I get no output.

Just to make sure I also did this:

grep info@domain.com tester.txt

still no output.

Okay, so thinking that the exim_mainlog might have updated since I pulled the last 100 lines, I ran the script again, and got the exact same results.

Then I ran this, on the entire exim_mainlog file:

grep info@domain.com exim_mainlog | grep login

Then I got 39 lines/exim_mainlog entries with this domain, but there were only 7 different IPs in the entire set, as opposed to 10 as per your script.

Regretfully I am not really good enough to troubleshoot your work. Can you suggest any refinements I might try?

Update in an attempt to trouble shoot, I removed the "rm tmp_exim" part at the very end.

Then I ran the script again, and ran pico on the tmp_exim file.

Sure enough, there were exactly 100 log entry lines from exim_mainlog, but when I carefully counted the example domain logins, and the number of IPs accessing, I got this result:

11 logins, with 2 different IPs.

As opposed to the result that I found for this domain, when last running the script, which indicated 11 logins with 11 different IPs.

So I believe something is not quite right about the tally with regard to the number of different accessing IPs.
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy

Hmm, might be the IP address part that's breaking.  If you remove the "|wc -l" part from the middle line, the output will show all of the unique values it is trying to compare - it *should* be the unique IP addresses.  On my test box, with ths change, I get:

5 [] [] [] login:sales@customerdomain.com.au
3 [] [] login:work@customerdomain.com.au

(I have an exim_mainlog with just two email addresses, and a total of 4 different IP addresses)

I removed "|wc -l" part and sorry to say, I cant make any sense out of the output. I get stuff like this:

11 []:49233 []:49270 []:49556 []:49578 []:49888 []:49899 []:49901 []:50155 []:50184 []:50188 []:50213 login:info@domain.com
7 (enduserPC) login:jane@r-domain.org
4 (stevemnb) login:janet@domain.com
1 (localhost) login:j.ann@domain.com
1 login:jeff+domain.com
4 (JoycePC) login:joyce@domain.com

(I replaced the word "domain" with the actual domain names, there were actually different domains in every output line.)

View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.

And I wanted to add this to my previous post. With "|wc -l" I get this, taking the same example output as I posted previously:

11 11 login:info@domain.com
7 1 login:jane@r-domain.org
4 1 login:janet@domain.com
1 1 login:j.ann@domain.com
2 0 login:jeff+domain.com
4 1 login:joyce@domain.com

I'll try the middle line replacement now.
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.

Bingo, the output looks a whole not more accurate now.

Thanks very much for your help!

Ah, spotted the problem.  The examples you gave originally just had IP addresses - some of the ones in the "without wc -l" post show the IP address immediately followed by a port number, or some other field (presumably a machine name).  My "sed" version of the middle line extracts just the IP address.

Glad it works now!

Cool thanks!

It also become apparent that the last 100 logins are counted initially, but I only wanted the last 100 exim_entries to be "looked at", then counting the logins within that initial range of the newest 100 total entries. Might sound strange but the whole thing has more to do with "what's happening to the exim system right now" kind of thing. I'll cron this script to run every minute.

But this should be easy to adjust, I would assume I change the top line from:

egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | -c|while read num email; do

To this:

tail -100 /var/log/exim_mainlog | tee tmp_exim | egrep -o 'login[^ ]+' | -c|while read num email; do
This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.

Yes, that looks like a valid change (though you'll need to add back in the "sort | uniq -c" bit in the lines above - cut-n-paste error, I think!).