Help Locating Spam Source

Hello Experts!

I have dozens of WordPress sites on a CentOS 6.4 server, all operating through one IP address, and it turns out that that IP has just been blacklisted because the server has been generating spam.

This is a two part question:

1.  How do I locate the source of the spam?
2.  What can I do in the future to catch this type of thing before I get blacklisted?

Thank you all in advance for any help you can offer in this area.
LVL 17
OmniUnlimitedAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jason C. LevineNo oneCommented:
1) You will need to go through the logs to see where the spam is coming from.  Is it email spam or pingback spam or something else?  Don't throw WordPress under the bus right away...it may turn out to be something else (unlikely, but you have to rule it out).

2) Depends on what the true problem is.  If WordPress is indeed compromised on one or more installations, then you can follow the steps in this article to harden sites:

http://www.experts-exchange.com/Web_Development/Blogs/WordPress/A_10806-Recovering-From-and-Preventing-WordPress-Site-Hacks.html
0
OmniUnlimitedAuthor Commented:
@jason1178: Thank you so much for your assistance.  I read your article, nice read, but it made me grimace to think one of the solutions you suggested was delete everything and restore it again.

I would really, really like to pinpoint the exact source of the spam, if that is at all possible.  Knowing where the spam is coming from would make me more confident in making changes rather than, "let's just try this and see if the spam stops."

My hosting company is not being to helpful in helping me find the logs for the emails (this is email spam, by the way.)  Do you have enough server experience to help me track down the logs?
0
Jason C. LevineNo oneCommented:
but it made me grimace to think one of the solutions you suggested was delete everything and restore it again.

That's really only applicable to WordPress sites when you have a good backup of the content handy.  Since installing WordPress is very fast and replacing plugins and themes is not a backbreaking task you can quickly kill and restore a site.  In this specific case, it doesn't mean to reformat the box and start a fresh server instance.

Do you have enough server experience to help me track down the logs?

Some, but a lot depends on how the hosting company has you configured and how restricted you are in moving about the server.

I would assume your mail logs are located at:

/var/log/maillog

and running

tail -f /var/log/maillog

will show you what you need to know to start pinpointing things.  Do you have a sample message with headers?  That can also give us some clues.
0
Newly released Acronis True Image 2019

In announcing the release of the 15th Anniversary Edition of Acronis True Image 2019, the company revealed that its artificial intelligence-based anti-ransomware technology – stopped more than 200,000 ransomware attacks on 150,000 customers last year.

OmniUnlimitedAuthor Commented:
Some, but a lot depends on how the hosting company has you configured and how restricted you are in moving about the server.

I have root access to the server.

I did locate maillogs at /var/log/maillog, but it does not appear that these would be the proper logs as I see no spam entries in this log.  Is it possible that the hacker could have accessed another mail service?

Do you have a sample message with headers?

Unfortunately, none of the spam messages got sent to me.  According to what I see in WHM, the spam got sent to email addresses around the globe.  A lot to Russia.
0
Jason C. LevineNo oneCommented:
Is it possible that the hacker could have accessed another mail service?

That, or the ISP has you configured a little differently than the norm or the hacker is spoofing your IP address and your server isn't even hacked or some other thing.  So. Many. Possibilities.

Your ISP support should be able to pinpoint this in seconds and it's in their best interests to do so.  Having one server spewing spam is a short step to having the whole block of IPs banned worldwide so they should be assisting you more than they are.

You could also hire a security service to sweep the server looking for injected backdoors.  While I'm starting to doubt this is WordPress related, we still haven't ruled it out completely and it may be worth some money to Sucuri or Stop The Hacker to get a better idea of what is going on.
0
OmniUnlimitedAuthor Commented:
or the hacker is spoofing your IP address and your server isn't even hacked or some other thing.

You know, just to see whether it could have been a php script generating these emails, I went ahead and changed php.ini to disable the email settings.  I checked the logs this morning through WHM and the emails are still going out.  But, you are right.  This may be a case where I am not even hacked, because the emails don't follow a pattern like they were being automatically generated or something.  The emails are going out one at a time, and at irregularly spaced intervals.

Check out the send times from this morning:

7:10:07 AM to madeiraxxn@mail.ru
6:55:06 AM to canumndawds@airjordanoutletusa.us
6:25:06 AM to jackiunksmtn@hotmail.com
5:45:05 AM to referijdalmer34@hotmail.com
4:32:04 AM to yourmailstetrr@gmail.com
3:29:03 AM to facelift.yxhd@gmail.com
3:22:03 AM to b.eij.in.g2.22.w.f@gmail.com
3:03:03 AM to timberlandbootsous@gmail.com
2:54:02 AM to chsrcqc@bestspmall.com
2:48:02 AM to jdisa.j.i.s.j.04.5.51.0@gmail.com
1:34:01 AM to gorlumes@mail.ru
1:17:01 AM to swaggybear23@hotmail.com
1:08:01 AM to adm2x@studiabialystok.eu
1:01:01 AM to plugov1986@mail.ru
...

These people are not in any of our contacts or databases.  I don't know where these email addresses are coming from.

I went ahead and installed ClamAV through the WHM which my host said should help stem the tide of the spam emails, but apparently it has had little effect.
0
OmniUnlimitedAuthor Commented:
UPDATE -

It turns out that my CPanel overrode the email settings I disabled and that appears to be the reason the spam still kept going out.  I finally managed to disable the php email and I have not seen a single spam email go out yet.

Now, my only problem is, how do I find out where among the literally thousands of php files on this server is the offending mail() function?
0
OmniUnlimitedAuthor Commented:
SUCCESS -

Well, for the books, I found this outstanding resource:

http://mcquarrie.com.au/wordpress/2012/10/tracking-down-malicious-php-spam-scripts/

which showed me how to enable script logging into the mail system and was able to locate and eliminate the problem out of a vulnerable php file.  This little script saved me untold amounts of hours in trying to locate the guilty file.  I would heartily endorse it for anyone who ends up with the same problem as I.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
OmniUnlimitedAuthor Commented:
Sorry Jason1178, I really appreciate the help, but none of what you suggested really provided me with the solution to my problem, i.e. finding the source of the spam.

Thank you for your assistance anyway.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.