asked on
Help with shell script to count logins from different IPs (to same account) in exim_mainlog.
I can figure out parts of this, but am having a difficult time putting it all together.
Here's the scenario:
This is a Redhat Linux server running a newer version of exim, with WHM/cPanel. I am the administrator with full root/shell access.
The exim_mainlog file contains multiple logins to the same email account from different IP addresses. This is an indication that the email user's password has been stolen and is being used to broadcast spam, depending of course on the number of different IP addresses logging in, within a short period of time. Here is a short example list of entries, just to illustrate this situation:
2012-03-10 16:31:19 1S6Uoo-0002d2-17 <= sales@customerdomain.com.a
2012-03-10 16:31:24 1S6Uor-0002cc-V2 <= sales@customerdomain.com.a
2012-03-10 16:31:27 1S6Uov-0002e3-KX <= sales@customerdomain.com.a
Of course, these entries would be more numerous and mixed in with many other logins to many other email accounts on the server.
What I need to come up with is a shell script that counts the number of DIFFERENT IPs logging into the same account, e.g. within a tail of something like the last 100 entries.
I've found this line, which works great to count the number of exim logins per account:
egrep -o 'login[^ ]+' /var/log/exim_mainlog | sort|uniq -c|sort -nk 1
... and I can even get it to work for the last 100 entires, like this:
egrep -o 'login[^ ]+' /var/log/exim_mainlog | tail -100 | sort|uniq -c|sort -nk 1
But what I need is output that tells me how many DIFFERENT IPs have logged in to that account.
In other words, what I get from the script lines above is output like this:
14 login:cccc@domainuber.com
15 login:aaaa@domainlanka.com
15 login:dddd@domaingreat.com
The leading number indicates the number of logins to each account.
But what I need to see instead is output something like the following:
14 2 login:jbs+domainuber.com
15 1 login:cmesa@domainlanka.co
15 12 login:rjc@domaingreat.com
... where the second number in each entry indicates the number of different IPs involved with the logins for each specific account.
I have also tried to work this out in a "for" loop, e.g.
#!/bin/bash
for i in `egrep -o 'login[^ ]+' /var/log/exim_mainlog | tail -100 | sort|uniq -c|sort -nk 1 | awk '{print $1}'`
do
if [ $i -gt 12 ]
then
echo $i
fi
done
Trying to establish 12 as a maximum threshold, but with this I am getting even less info than before, certainly not the number of DIFFERENT IP addresses logging into each account.
Any help would be greatly appreciated.
Thanks!
Here's the scenario:
This is a Redhat Linux server running a newer version of exim, with WHM/cPanel. I am the administrator with full root/shell access.
The exim_mainlog file contains multiple logins to the same email account from different IP addresses. This is an indication that the email user's password has been stolen and is being used to broadcast spam, depending of course on the number of different IP addresses logging in, within a short period of time. Here is a short example list of entries, just to illustrate this situation:
2012-03-10 16:31:19 1S6Uoo-0002d2-17 <= sales@customerdomain.com.a
2012-03-10 16:31:24 1S6Uor-0002cc-V2 <= sales@customerdomain.com.a
2012-03-10 16:31:27 1S6Uov-0002e3-KX <= sales@customerdomain.com.a
Of course, these entries would be more numerous and mixed in with many other logins to many other email accounts on the server.
What I need to come up with is a shell script that counts the number of DIFFERENT IPs logging into the same account, e.g. within a tail of something like the last 100 entries.
I've found this line, which works great to count the number of exim logins per account:
egrep -o 'login[^ ]+' /var/log/exim_mainlog | sort|uniq -c|sort -nk 1
... and I can even get it to work for the last 100 entires, like this:
egrep -o 'login[^ ]+' /var/log/exim_mainlog | tail -100 | sort|uniq -c|sort -nk 1
But what I need is output that tells me how many DIFFERENT IPs have logged in to that account.
In other words, what I get from the script lines above is output like this:
14 login:cccc@domainuber.com
15 login:aaaa@domainlanka.com
15 login:dddd@domaingreat.com
The leading number indicates the number of logins to each account.
But what I need to see instead is output something like the following:
14 2 login:jbs+domainuber.com
15 1 login:cmesa@domainlanka.co
15 12 login:rjc@domaingreat.com
... where the second number in each entry indicates the number of different IPs involved with the logins for each specific account.
I have also tried to work this out in a "for" loop, e.g.
#!/bin/bash
for i in `egrep -o 'login[^ ]+' /var/log/exim_mainlog | tail -100 | sort|uniq -c|sort -nk 1 | awk '{print $1}'`
do
if [ $i -gt 12 ]
then
echo $i
fi
done
Trying to establish 12 as a maximum threshold, but with this I am getting even less info than before, certainly not the number of DIFFERENT IP addresses logging into each account.
Any help would be greatly appreciated.
Thanks!
Web ServersApache Web ServerShell Scripting
Last Comment
simon3270
ASKER
Thanks for your work. However, I am not seeing accurate results.
When I ran the script:
#!/bin/bash
egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | sort|uniq -c|while read num email; do
echo $num $(awk "/$email/{print \$7}" tmp_exim|sort -u|wc -l) $email
done; rm tmp_exim
... one of the email accounts in approximately the middle of the result list (I have only checked this one result.) Shows this (I changed the email address for privacy):
10 10 login:info@domain.com
Then I carefully isolated the last 100 lines of the exim_mainlog file, put it into another file named tester.txt Then when I did a grep on the /var/log/exim_mainlog file, like this:
grep info@domain.com tester.txt | grep login
I get no output.
Just to make sure I also did this:
grep info@domain.com tester.txt
still no output.
Okay, so thinking that the exim_mainlog might have updated since I pulled the last 100 lines, I ran the script again, and got the exact same results.
Then I ran this, on the entire exim_mainlog file:
grep info@domain.com exim_mainlog | grep login
Then I got 39 lines/exim_mainlog entries with this domain, but there were only 7 different IPs in the entire set, as opposed to 10 as per your script.
Regretfully I am not really good enough to troubleshoot your work. Can you suggest any refinements I might try?
When I ran the script:
#!/bin/bash
egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | sort|uniq -c|while read num email; do
echo $num $(awk "/$email/{print \$7}" tmp_exim|sort -u|wc -l) $email
done; rm tmp_exim
... one of the email accounts in approximately the middle of the result list (I have only checked this one result.) Shows this (I changed the email address for privacy):
10 10 login:info@domain.com
Then I carefully isolated the last 100 lines of the exim_mainlog file, put it into another file named tester.txt Then when I did a grep on the /var/log/exim_mainlog file, like this:
grep info@domain.com tester.txt | grep login
I get no output.
Just to make sure I also did this:
grep info@domain.com tester.txt
still no output.
Okay, so thinking that the exim_mainlog might have updated since I pulled the last 100 lines, I ran the script again, and got the exact same results.
Then I ran this, on the entire exim_mainlog file:
grep info@domain.com exim_mainlog | grep login
Then I got 39 lines/exim_mainlog entries with this domain, but there were only 7 different IPs in the entire set, as opposed to 10 as per your script.
Regretfully I am not really good enough to troubleshoot your work. Can you suggest any refinements I might try?
ASKER
Update in an attempt to trouble shoot, I removed the "rm tmp_exim" part at the very end.
Then I ran the script again, and ran pico on the tmp_exim file.
Sure enough, there were exactly 100 log entry lines from exim_mainlog, but when I carefully counted the example domain logins, and the number of IPs accessing, I got this result:
11 logins, with 2 different IPs.
As opposed to the result that I found for this domain, when last running the script, which indicated 11 logins with 11 different IPs.
So I believe something is not quite right about the tally with regard to the number of different accessing IPs.
Then I ran the script again, and ran pico on the tmp_exim file.
Sure enough, there were exactly 100 log entry lines from exim_mainlog, but when I carefully counted the example domain logins, and the number of IPs accessing, I got this result:
11 logins, with 2 different IPs.
As opposed to the result that I found for this domain, when last running the script, which indicated 11 logins with 11 different IPs.
So I believe something is not quite right about the tally with regard to the number of different accessing IPs.
Hmm, might be the IP address part that's breaking. If you remove the "|wc -l" part from the middle line, the output will show all of the unique values it is trying to compare - it *should* be the unique IP addresses. On my test box, with ths change, I get:
5 [186.123.22.104] [201.29.210.105] [92.47.137.191] login:sales@customerdomain
3 [201.29.210.105] [201.29.210.106] login:work@customerdomain.
(I have an exim_mainlog with just two email addresses, and a total of 4 different IP addresses)
5 [186.123.22.104] [201.29.210.105] [92.47.137.191] login:sales@customerdomain
3 [201.29.210.105] [201.29.210.106] login:work@customerdomain.
(I have an exim_mainlog with just two email addresses, and a total of 4 different IP addresses)
ASKER
I removed "|wc -l" part and sorry to say, I cant make any sense out of the output. I get stuff like this:
11 [2.146.211.124]:49233 [2.146.211.124]:49270 [2.146.34.175]:49556 [2.146.34.175]:49578 [2.146.34.175]:49888 [2.146.34.175]:49899 [2.146.34.175]:49901 [2.146.34.175]:50155 [2.146.34.175]:50184 [2.146.34.175]:50188 [2.146.34.175]:50213 login:info@domain.com
7 (enduserPC) login:jane@r-domain.org
4 (stevemnb) login:janet@domain.com
1 (localhost) login:j.ann@domain.com
1 login:jeff+domain.com
4 (JoycePC) login:joyce@domain.com
(I replaced the word "domain" with the actual domain names, there were actually different domains in every output line.)
11 [2.146.211.124]:49233 [2.146.211.124]:49270 [2.146.34.175]:49556 [2.146.34.175]:49578 [2.146.34.175]:49888 [2.146.34.175]:49899 [2.146.34.175]:49901 [2.146.34.175]:50155 [2.146.34.175]:50184 [2.146.34.175]:50188 [2.146.34.175]:50213 login:info@domain.com
7 (enduserPC) login:jane@r-domain.org
4 (stevemnb) login:janet@domain.com
1 (localhost) login:j.ann@domain.com
1 login:jeff+domain.com
4 (JoycePC) login:joyce@domain.com
(I replaced the word "domain" with the actual domain names, there were actually different domains in every output line.)
ASKER CERTIFIED SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
ASKER
And I wanted to add this to my previous post. With "|wc -l" I get this, taking the same example output as I posted previously:
11 11 login:info@domain.com
7 1 login:jane@r-domain.org
4 1 login:janet@domain.com
1 1 login:j.ann@domain.com
2 0 login:jeff+domain.com
4 1 login:joyce@domain.com
I'll try the middle line replacement now.
11 11 login:info@domain.com
7 1 login:jane@r-domain.org
4 1 login:janet@domain.com
1 1 login:j.ann@domain.com
2 0 login:jeff+domain.com
4 1 login:joyce@domain.com
I'll try the middle line replacement now.
ASKER
Bingo, the output looks a whole not more accurate now.
Thanks very much for your help!
Thanks very much for your help!
Ah, spotted the problem. The examples you gave originally just had IP addresses - some of the ones in the "without wc -l" post show the IP address immediately followed by a port number, or some other field (presumably a machine name). My "sed" version of the middle line extracts just the IP address.
Glad it works now!
Glad it works now!
ASKER
Cool thanks!
It also become apparent that the last 100 logins are counted initially, but I only wanted the last 100 exim_entries to be "looked at", then counting the logins within that initial range of the newest 100 total entries. Might sound strange but the whole thing has more to do with "what's happening to the exim system right now" kind of thing. I'll cron this script to run every minute.
But this should be easy to adjust, I would assume I change the top line from:
egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | -c|while read num email; do
To this:
tail -100 /var/log/exim_mainlog | tee tmp_exim | egrep -o 'login[^ ]+' | -c|while read num email; do
It also become apparent that the last 100 logins are counted initially, but I only wanted the last 100 exim_entries to be "looked at", then counting the logins within that initial range of the newest 100 total entries. Might sound strange but the whole thing has more to do with "what's happening to the exim system right now" kind of thing. I'll cron this script to run every minute.
But this should be easy to adjust, I would assume I change the top line from:
egrep 'login[^ ]' /var/log/exim_mainlog | tail -100 | tee tmp_exim | egrep -o 'login[^ ]+' | -c|while read num email; do
To this:
tail -100 /var/log/exim_mainlog | tee tmp_exim | egrep -o 'login[^ ]+' | -c|while read num email; do
Yes, that looks like a valid change (though you'll need to add back in the "sort | uniq -c" bit in the lines above - cut-n-paste error, I think!).
Open in new window
As requested, each output line gives the number of logins, the number of unique IP addresses, and the email address.