chrisdodds
asked on
How to find the infected machine on my network?
Hey friends, I'm having quite a troublesome issue I'd appreciate help on. I have a network running a Windows SBS 2008 server (running Exchange 2007) with 6 client machines. Within the last couple of weeks I've had some users complain of sent mail being "Delayed". Upon looking at the Exchange queues I found multiple connectors with email waiting to be sent with any number of different reasons for their delay. Upon looking up our static external IP with Spamhaus.org, I found that we were listed in the "XBL". Further research lead me to the Composite Blocking List where I delisted us and all was fine.
A couple of weeks go by and we're having the same issue. At this point I ran multiple scans using multiple utilities looking for all kinds of baddies - rootkits in particular. Thing is, there wasn't that much detected (which I know doesn't mean that one or more of these machines isn't still infected), but the network is pretty clean and I rarely have issues with these folks. After scanning all my network PC's, I delisted us again hoping that the infection had been found and removed from whatever machine it was that was causing my trouble.
This morning I see more mail in the queue and once again, we're blocked. I downloaded WireShark and I'm currently looking through packets but I'm not very well-versed in using this product or any other similar network/packet sniffing products. Here is the result of looking up our external IP at http://cbl.abuseat.org/lookup.cgi
*********
IP Address 216.177.180.90 is listed in the CBL. It appears to be infected with a spam sending trojan or proxy.
It was last detected at 2011-09-10 10:00 GMT (+/- 30 minutes), approximately 3 days, 2 hours ago.
It has been relisted following a previous removal at 2011-09-06 21:22 GMT (6 days, 15 hours, 3 minutes ago)
This IP is infected with, or is NATting for a machine infected with Torpig, also known by Symantec as Anserin.
This was detected by observing this IP attempting to make contact to a Torpig Command and Control server at 91.19.43.205, with contents unique to Torpig C&C command protocols.
Torpig is a banking trojan, specializing in stealing personal information (passwords, account information, etc) from interactions with banking sites.
Torpig is normally dropped by Mebroot. Mebroot is a Rootkit that installs itself into the MBR (Master Boot Record).
With Mebroot or any other rootkit that installs itself into the MBR, you will either have to use a "MBR cleaner" or reformat the drive completely - even if you manage to remove Torpig, the MBR infection will cause it to be reinfected again.
The best way to find the machine responsible is to look for connections to the Torpig C&C server. This detection was made through a connection to 91.19.43.205, but this changes periodically. To find these infections, we suggest you search for TCP/IP connections to the range 91.19.0.0/16 and 91.20.0.0/16 (in other words: 91.19.0.0-91.20.255.255) usually destination port 80 or 443, but you should look for all ports. This detection corresponds to a connection at 2011-09-10 10:12:30 (GMT - this timestamp is believed accurate to within one second).
These infections are rated as a "severe threat" by Microsoft. It is a trojan downloader, and can download and execute ANY software on the infected computer.
You will need to find and eradicate the infection before delisting the IP address.
We strongly recommend that you DO NOT simply firewall off connections to the sinkhole IP addresses given above. Those IP addresses are of sinkholes operated by malware researchers. In other words, it's a "sensor" (only) run by "the good guys". The bot "thinks" its a command and control server run by the spambot operators but it isn't. It DOES NOT actually download anything, and is not a threat. If you firewall it, your IPs will remain infected, and they will still be able to download from real command & control servers run by the bot operators.
If you do choose to firewall these IPs, PLEASE instrument your firewall to tell you which internal machine is connecting to them so that you can identify the infected machine yourself and fix it.
We are enhancing the instructions on how to find these infections, and more information will be given here as it becomes available.
Virtually all detections made by the CBL are of infections that do NOT leave any "tracks" for you to find in your mail server logs. This is even more important for the viruses described here - these detections are made on network-level detections of malicious behaviour and may NOT involve malicious email being sent.
This means: if you have port 25 blocking enabled, do not take this as indication that your port 25 blocking isn't working.
The links above may help you find this infection. You can also consult Advanced Techniques for other options and alternatives. NOTE: the Advanced Techniques link focuses on finding port 25(SMTP) traffic. With "sinkhole malware" detections such as this listing, we aren't detecting port 25 traffic, we're detecting traffic on other ports. Therefore, when reading Advanced Techniques, you will need to consider all ports, not just SMTP.
Pay very close attention: Most of these trojans have extremely poor detection rates in current Anti-Virus software. For example, Ponmocup is only detected by 3 out of 49 AV tools queried at Virus Total.
Thus: having your anti-virus software doesn't find anything doesn't prove that you're not infected.
While we regret having to say this, downloaders will generally download many different malicious payloads. Even if an Anti-Virus product finds and removes the direct threat, they will not have detected or removed the other malicious payloads. For that reason, we recommend recloning the machine - meaning: reformatting the disks on the infected machine, and re-installing all software from known-good sources.
--------------------------
WARNING: If you continually delist 216.177.180.90 without fixing the problem, the CBL will eventually stop allowing the delisting of 216.177.180.90.
If you have resolved the problem shown above and delisted the IP yourself, there is no need to contact us.
Click on this link to delist 216.177.180.90.
*********
My question now is, how can I "turn on" monitoring to see what machine it is that's infected with this malware? There must be something I can institute through Group Policies to figure this out? Or to block traffic to these sites at my SBS 2008 server level as is suggested? I'm at a loss because at this point I don't know if one of my network PC's is infected or if my server may be infected. Any help in trouibleshooting and resolving this issue is greatly appreciated.
- Chris in San Antonio
A couple of weeks go by and we're having the same issue. At this point I ran multiple scans using multiple utilities looking for all kinds of baddies - rootkits in particular. Thing is, there wasn't that much detected (which I know doesn't mean that one or more of these machines isn't still infected), but the network is pretty clean and I rarely have issues with these folks. After scanning all my network PC's, I delisted us again hoping that the infection had been found and removed from whatever machine it was that was causing my trouble.
This morning I see more mail in the queue and once again, we're blocked. I downloaded WireShark and I'm currently looking through packets but I'm not very well-versed in using this product or any other similar network/packet sniffing products. Here is the result of looking up our external IP at http://cbl.abuseat.org/lookup.cgi
*********
IP Address 216.177.180.90 is listed in the CBL. It appears to be infected with a spam sending trojan or proxy.
It was last detected at 2011-09-10 10:00 GMT (+/- 30 minutes), approximately 3 days, 2 hours ago.
It has been relisted following a previous removal at 2011-09-06 21:22 GMT (6 days, 15 hours, 3 minutes ago)
This IP is infected with, or is NATting for a machine infected with Torpig, also known by Symantec as Anserin.
This was detected by observing this IP attempting to make contact to a Torpig Command and Control server at 91.19.43.205, with contents unique to Torpig C&C command protocols.
Torpig is a banking trojan, specializing in stealing personal information (passwords, account information, etc) from interactions with banking sites.
Torpig is normally dropped by Mebroot. Mebroot is a Rootkit that installs itself into the MBR (Master Boot Record).
With Mebroot or any other rootkit that installs itself into the MBR, you will either have to use a "MBR cleaner" or reformat the drive completely - even if you manage to remove Torpig, the MBR infection will cause it to be reinfected again.
The best way to find the machine responsible is to look for connections to the Torpig C&C server. This detection was made through a connection to 91.19.43.205, but this changes periodically. To find these infections, we suggest you search for TCP/IP connections to the range 91.19.0.0/16 and 91.20.0.0/16 (in other words: 91.19.0.0-91.20.255.255) usually destination port 80 or 443, but you should look for all ports. This detection corresponds to a connection at 2011-09-10 10:12:30 (GMT - this timestamp is believed accurate to within one second).
These infections are rated as a "severe threat" by Microsoft. It is a trojan downloader, and can download and execute ANY software on the infected computer.
You will need to find and eradicate the infection before delisting the IP address.
We strongly recommend that you DO NOT simply firewall off connections to the sinkhole IP addresses given above. Those IP addresses are of sinkholes operated by malware researchers. In other words, it's a "sensor" (only) run by "the good guys". The bot "thinks" its a command and control server run by the spambot operators but it isn't. It DOES NOT actually download anything, and is not a threat. If you firewall it, your IPs will remain infected, and they will still be able to download from real command & control servers run by the bot operators.
If you do choose to firewall these IPs, PLEASE instrument your firewall to tell you which internal machine is connecting to them so that you can identify the infected machine yourself and fix it.
We are enhancing the instructions on how to find these infections, and more information will be given here as it becomes available.
Virtually all detections made by the CBL are of infections that do NOT leave any "tracks" for you to find in your mail server logs. This is even more important for the viruses described here - these detections are made on network-level detections of malicious behaviour and may NOT involve malicious email being sent.
This means: if you have port 25 blocking enabled, do not take this as indication that your port 25 blocking isn't working.
The links above may help you find this infection. You can also consult Advanced Techniques for other options and alternatives. NOTE: the Advanced Techniques link focuses on finding port 25(SMTP) traffic. With "sinkhole malware" detections such as this listing, we aren't detecting port 25 traffic, we're detecting traffic on other ports. Therefore, when reading Advanced Techniques, you will need to consider all ports, not just SMTP.
Pay very close attention: Most of these trojans have extremely poor detection rates in current Anti-Virus software. For example, Ponmocup is only detected by 3 out of 49 AV tools queried at Virus Total.
Thus: having your anti-virus software doesn't find anything doesn't prove that you're not infected.
While we regret having to say this, downloaders will generally download many different malicious payloads. Even if an Anti-Virus product finds and removes the direct threat, they will not have detected or removed the other malicious payloads. For that reason, we recommend recloning the machine - meaning: reformatting the disks on the infected machine, and re-installing all software from known-good sources.
--------------------------
WARNING: If you continually delist 216.177.180.90 without fixing the problem, the CBL will eventually stop allowing the delisting of 216.177.180.90.
If you have resolved the problem shown above and delisted the IP yourself, there is no need to contact us.
Click on this link to delist 216.177.180.90.
*********
My question now is, how can I "turn on" monitoring to see what machine it is that's infected with this malware? There must be something I can institute through Group Policies to figure this out? Or to block traffic to these sites at my SBS 2008 server level as is suggested? I'm at a loss because at this point I don't know if one of my network PC's is infected or if my server may be infected. Any help in trouibleshooting and resolving this issue is greatly appreciated.
- Chris in San Antonio
ASKER
You're exactly right - I need to find the machine on my network that's causing this trouble! My fear now is that it's the server itself. Thank you so much. I'll check out the link here within the next 2 hours and see what I can get setup. I should know how to do this - but I don't (yet). I'll repost here with more soon.
- Chris
- Chris
Have you scanned your server for infection?
You may be dealing with backscatter and none of your computers are infected. If you receive alot of spam and your server accepts is, it will determine there is no one here by that name and send it back, thus making it loook like you're spamming:
http://spamlinks.net/prevent-secure-backscatter.htm
You may be dealing with backscatter and none of your computers are infected. If you receive alot of spam and your server accepts is, it will determine there is no one here by that name and send it back, thus making it loook like you're spamming:
http://spamlinks.net/prevent-secure-backscatter.htm
This detection was made through a connection to 91.19.43.205, but this changes periodically. To find these infections, we suggest you search for TCP/IP connections to the range 91.19.0.0/16 and 91.20.0.0/16 (in other words: 91.19.0.0-91.20.255.255) usually destination port 80 or 443, but you should look for all ports.
I would also set up a rule on your firewall to allow outbound traffic to 91.19.0.0/16 and 91.20.0.0/16 with logging enabled. Set the rule at the top of your rule chain so that it catches all the traffic to those networks. That way you can positively identify which hosts are infected.
ASKER
mustang83 - I checked out both Part 1 & 2 and I am much more familiar now with the Exchange message tracking logs and how to manipulate/view the contents of these files and will be monitoring this as I need to.
notacomputergeek - I checked out your link as well on the possibility of the backscatting and I'm a little confused by all of the DSN codes and how to only generate the NDR's for specific accounts but will continue to read on this. I don't suspect that is the case, but I'll stop right there because I'm just not sure!
da3ve - I implemented your suggestion and will be checking these logs as well.
Funny thing is, this is the first chance I've had to revisit this and the first thing I did this morning was check the SpamHaus/CBL sites to see if our external static IP was listed (just knowing of course that we would be). To my surprise, we weren't. I was just slightly confused. I spent a great deal of time scanning all office PC's for malware/rootkits, etc. and it was after THAT I saw that we were listed again. I left us "listed" until I could implement some of these changes but mysteriously, we're no longer listed. That was on the 12 of September that we were logged as being listed in the CBL (http://cbl.abuseat.org/lookup.cgi). I posted for help here on the 13th, and now it appears that it may not be warranted. I'll keep my eyes peeled but for now I'm going to keep this question open before awarding points. I appreciate all your suggections and I hope this doesn't plague me anymore!
- Dodds in SA
notacomputergeek - I checked out your link as well on the possibility of the backscatting and I'm a little confused by all of the DSN codes and how to only generate the NDR's for specific accounts but will continue to read on this. I don't suspect that is the case, but I'll stop right there because I'm just not sure!
da3ve - I implemented your suggestion and will be checking these logs as well.
Funny thing is, this is the first chance I've had to revisit this and the first thing I did this morning was check the SpamHaus/CBL sites to see if our external static IP was listed (just knowing of course that we would be). To my surprise, we weren't. I was just slightly confused. I spent a great deal of time scanning all office PC's for malware/rootkits, etc. and it was after THAT I saw that we were listed again. I left us "listed" until I could implement some of these changes but mysteriously, we're no longer listed. That was on the 12 of September that we were logged as being listed in the CBL (http://cbl.abuseat.org/lookup.cgi). I posted for help here on the 13th, and now it appears that it may not be warranted. I'll keep my eyes peeled but for now I'm going to keep this question open before awarding points. I appreciate all your suggections and I hope this doesn't plague me anymore!
- Dodds in SA
have you had any further problems with this?
ASKER
Hey friends, still having trouble with this and now that I have my Exchange Message Trackin enabled, when looking through these logs, I'm not exactly sure what I'm even looking at or should be looking for? I see some things that look a little strange, but I can't tell if what I'm looking at is malicious? Any further help to interpret these logs is appreciated. I've done everything I can think of and still find my server listed off and on...
- Chris
- Chris
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I appreciate everybody's assistance in troubleshooting this issue, but unfortunately none of the solutions presented ultimately lead to the final resolution.
You need to enable message tracking on your exchange. You priority is to work out which machine is causing you to be blacklisted.
In my environment i have a linux box setup as a smart host which logs every outgoing mail into a sql database. Just the headers not the the content.
You can do something similar by looking through the logs of your exchange.
To set up view this site for help.
http://www.msexchange.org/tutorials/exchange-2007-message-tracking-part1.html
I have tested your gateway and its not something silly like a open relay. It must be something sending from your machines.
Wireshark will show smtp traffic but you may not be able to determine if its real traffic or spam traffic. Your logs are the best bet.
Graham