Hi everyone! In the midst of a ginormous crisis here, hoping someone can help.
A couple of months ago I was put in charge of a LAMP-driven website hosted on a dedicated server at OVH. Right from the start, I noticed numerous attempts (in the http error logs) to log in to the system using various different passwords so I installed fail2ban but didn't really know what else I could do aside from that.
Well, a couple of days ago the situation took a drastic turn for the worse, with thousands of attempts to access various files on the system and log in with bizarre user names resembling urls, and so forth. The site crashed a bunch of times, I updated fail2ban, but every time I added a rule, some new syntax popped up in the http error logs.
Finally, I tried to access the site a couple of hours ago and a window popped up asking me for a username and password! When I tried to close the window I got a 401 error message. How could this happen? I was still able to log on to the machine with Putty and noticed all kinds of weird processes and also a bunch of files which had seemingly just been deposited on the machine all at once (but apparently 0 bytes in size - don't know if this is relevant). I tried to kill the processes but more sprung up in their place (some referring to "agetty" - don't know what this is) and was unable to delete any of the files. In a state of panic, I attempted to change the root password, and it seemed to work, only I seem to be able to log in using the old password as well!
What is going on?! And what, if anything, can I do to get the site back?
Install an application firewall (which is a different animal than a regular port-blocking firewall). Everyone tells you that good programming is the only protection, but if you're not sure if all of your programming is perfect, a good application firewall will protect you from much badness (SQL Injection attacks and so forth.)