Link to home
Start Free TrialLog in
Avatar of ujoe
ujoe

asked on

Help with shell script to count logins from different IPs (to same account) in exim_mainlog.

I can figure out parts of this, but am having a difficult time putting it all together.

Here's the scenario:

This is a Redhat Linux server running a newer version of exim, with WHM/cPanel. I am the administrator with full root/shell access.

The exim_mainlog file contains multiple logins to the same email account from different IP addresses. This is an indication that the email user's password has been stolen and is being used to broadcast spam, depending of course on the number of different IP addresses logging in, within a short period of time. Here is a short example list of entries, just to illustrate this situation:

2012-03-10 16:31:19 1S6Uoo-0002d2-17 <= sales@customerdomain.com.au H=(customerdomain.com.au) [92.47.137.191] P=esmtpa A=courier_login:sales@customerdomain.com.au S=830
2012-03-10 16:31:24 1S6Uor-0002cc-V2 <= sales@customerdomain.com.au H=(customerdomain.com.au) [186.123.22.104] P=esmtpa A=courier_login:sales@customerdomain.com.au S=838
2012-03-10 16:31:27 1S6Uov-0002e3-KX <= sales@customerdomain.com.au H=(customerdomain.com.au) [201.29.210.105] P=esmtpa A=courier_login:sales@customerdomain.com.au S=839

Of course, these entries would be more numerous and mixed in with many other logins to many other email accounts on the server.

What I need to come up with is a shell script that counts the number of DIFFERENT IPs logging into the same account, e.g. within a tail of something like the last 100 entries.

I've found this line, which works great to count the number of exim logins per account:
egrep -o 'login[^ ]+' /var/log/exim_mainlog | sort|uniq -c|sort -nk 1

... and I can even get it to work for the last 100 entires, like this:
egrep -o 'login[^ ]+' /var/log/exim_mainlog | tail -100 | sort|uniq -c|sort -nk 1

But what I need is output that tells me how many DIFFERENT IPs have logged in to that account.

In other words, what I get from the script lines above is output like this:

     14 login:cccc@domainuber.com
     15 login:aaaa@domainlanka.com
     15 login:dddd@domaingreat.com

The leading number indicates the number of logins to each account.

But what I need to see instead is output something like the following:

14   2     login:jbs+domainuber.com
15   1     login:cmesa@domainlanka.com
15   12   login:rjc@domaingreat.com

... where the second number in each entry indicates the number of different IPs involved with the logins for each specific account.

I have also tried to work this out in a "for" loop, e.g.

#!/bin/bash
for i in `egrep -o 'login[^ ]+' /var/log/exim_mainlog | tail -100 | sort|uniq -c|sort -nk 1 | awk '{print $1}'`
do
if [ $i -gt 12 ]
then
echo $i
fi
done

Trying to establish 12 as a maximum threshold, but with this I am getting even less info than before, certainly not the number of DIFFERENT IP addresses logging into each account.

Any help would be greatly appreciated.

Thanks!
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

A loop seems the easiest way to go.  I've modified your existing line to write the tail output to a temporary file, so that we use it once to get the list of user logins, and then later on count the number of different IP addresses.
egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | sort|uniq -c|while read num email; do
  echo $num $(awk "/$email/{print \$7}" tmp_exim|sort -u|wc -l) $email
done; rm tmp_exim

Open in new window

As requested, each output line gives the number of logins, the number of unique IP addresses, and the email address.
Avatar of ujoe
ujoe

ASKER

Thanks for your work. However, I am not seeing accurate results.

When I ran the script:

#!/bin/bash
egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | sort|uniq -c|while read num email; do
  echo $num $(awk "/$email/{print \$7}" tmp_exim|sort -u|wc -l) $email
done; rm tmp_exim

... one of the email accounts in approximately the middle of the result list (I have only checked this one result.) Shows this (I changed the email address for privacy):

10 10 login:info@domain.com

Then I carefully isolated the last 100 lines of the exim_mainlog file, put it into another file named tester.txt Then when I did a grep on the /var/log/exim_mainlog file, like this:

grep info@domain.com tester.txt | grep login

I get no output.

Just to make sure I also did this:

grep info@domain.com tester.txt

still no output.

Okay, so thinking that the exim_mainlog might have updated since I pulled the last 100 lines, I ran the script again, and got the exact same results.

Then I ran this, on the entire exim_mainlog file:

grep info@domain.com exim_mainlog | grep login

Then I got 39 lines/exim_mainlog entries with this domain, but there were only 7 different IPs in the entire set, as opposed to 10 as per your script.

Regretfully I am not really good enough to troubleshoot your work. Can you suggest any refinements I might try?
Avatar of ujoe

ASKER

Update in an attempt to trouble shoot, I removed the "rm tmp_exim" part at the very end.

Then I ran the script again, and ran pico on the tmp_exim file.

Sure enough, there were exactly 100 log entry lines from exim_mainlog, but when I carefully counted the example domain logins, and the number of IPs accessing, I got this result:

11 logins, with 2 different IPs.

As opposed to the result that I found for this domain, when last running the script, which indicated 11 logins with 11 different IPs.

So I believe something is not quite right about the tally with regard to the number of different accessing IPs.
Hmm, might be the IP address part that's breaking.  If you remove the "|wc -l" part from the middle line, the output will show all of the unique values it is trying to compare - it *should* be the unique IP addresses.  On my test box, with ths change, I get:

5 [186.123.22.104] [201.29.210.105] [92.47.137.191] login:sales@customerdomain.com.au
3 [201.29.210.105] [201.29.210.106] login:work@customerdomain.com.au

(I have an exim_mainlog with just two email addresses, and a total of 4 different IP addresses)
Avatar of ujoe

ASKER

I removed "|wc -l" part and sorry to say, I cant make any sense out of the output. I get stuff like this:

11 [2.146.211.124]:49233 [2.146.211.124]:49270 [2.146.34.175]:49556 [2.146.34.175]:49578 [2.146.34.175]:49888 [2.146.34.175]:49899 [2.146.34.175]:49901 [2.146.34.175]:50155 [2.146.34.175]:50184 [2.146.34.175]:50188 [2.146.34.175]:50213 login:info@domain.com
7 (enduserPC) login:jane@r-domain.org
4 (stevemnb) login:janet@domain.com
1 (localhost) login:j.ann@domain.com
1 login:jeff+domain.com
4 (JoycePC) login:joyce@domain.com

(I replaced the word "domain" with the actual domain names, there were actually different domains in every output line.)
ASKER CERTIFIED SOLUTION
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of ujoe

ASKER

And I wanted to add this to my previous post. With "|wc -l" I get this, taking the same example output as I posted previously:

11 11 login:info@domain.com
7 1 login:jane@r-domain.org
4 1 login:janet@domain.com
1 1 login:j.ann@domain.com
2 0 login:jeff+domain.com
4 1 login:joyce@domain.com

I'll try the middle line replacement now.
Avatar of ujoe

ASKER

Bingo, the output looks a whole not more accurate now.

Thanks very much for your help!
Ah, spotted the problem.  The examples you gave originally just had IP addresses - some of the ones in the "without wc -l" post show the IP address immediately followed by a port number, or some other field (presumably a machine name).  My "sed" version of the middle line extracts just the IP address.

Glad it works now!
Avatar of ujoe

ASKER

Cool thanks!

It also become apparent that the last 100 logins are counted initially, but I only wanted the last 100 exim_entries to be "looked at", then counting the logins within that initial range of the newest 100 total entries. Might sound strange but the whole thing has more to do with "what's happening to the exim system right now" kind of thing. I'll cron this script to run every minute.

But this should be easy to adjust, I would assume I change the top line from:

egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | -c|while read num email; do

To this:

tail -100 /var/log/exim_mainlog | tee tmp_exim | egrep -o 'login[^ ]+' | -c|while read num email; do
Yes, that looks like a valid change (though you'll need to add back in the "sort | uniq -c" bit in the lines above - cut-n-paste error, I think!).