troubleshooting Question

Has my server been hacked? How do I get it back?

Avatar of razzmatazz
razzmatazz asked on
Web ApplicationsWeb ServersApache Web Server
4 Comments1 Solution205 ViewsLast Modified:
Hi everyone!  In the midst of a ginormous crisis here, hoping someone can help.

A couple of months ago I was put in charge of a LAMP-driven website hosted on a dedicated server at OVH.  Right from the start, I noticed numerous attempts (in the http error logs) to log in to the system using various different passwords so I installed fail2ban but didn't really know what else I could do aside from that.

Well, a couple of days ago the situation took a drastic turn for the worse, with thousands of attempts to access various files on the system and log in with bizarre user names resembling urls, and so forth.  The site crashed a bunch of times, I updated fail2ban, but every time I added a rule, some new syntax popped up in the http error logs.

Finally, I tried to access the site a couple of hours ago and a window popped up asking me for a username and password!  When I tried to close the window I got a 401 error message.  How could this happen?  I was still able to log on to the machine with Putty and noticed all kinds of weird processes and also a bunch of files which had seemingly just been deposited on the machine all at once (but apparently 0 bytes in size - don't know if this is relevant).  I tried to kill the processes but more sprung up in their place (some referring to "agetty" - don't know what this is) and was unable to delete any of the files.  In a state of panic, I attempted to change the root password, and it seemed to work, only I seem to be able to log in using the old password as well!

What is going on?!  And what, if anything, can I do to get the site back?
ASKER CERTIFIED SOLUTION
kennethfine

Our community of experts have been thoroughly vetted for their expertise and industry experience.

Join our community to see this answer!
Unlock 1 Answer and 4 Comments.
Start Free Trial
Learn from the best

Network and collaborate with thousands of CTOs, CISOs, and IT Pros rooting for you and your success.

Andrew Hancock - VMware vExpert
See if this solution works for you by signing up for a 7 day free trial.
Unlock 1 Answer and 4 Comments.
Try for 7 days

”The time we save is the biggest benefit of E-E to our team. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange.

-Mike Kapnisakis, Warner Bros