Solved

Server stopped working all of the sudden, possible attack?

Posted on 2004-08-18
7
315 Views
Last Modified: 2010-04-20
Hello All,

We have a server running with RHE 3, Apache, Php, Mysql and Bind. It's our main production server.

Now the problem we faced day before was something very weird. All of the sudden all the services on the server stopped responding, we couldn't ssh into the server, we couldn't see the sites, couldn't fetch mails and so on.

We could ping the server though, ping was going fine, but we just couldn't access the server in any way.

Now the question I would like to ask you guys is,  if such problem occurs again, how to trace it. I tried to check /var/log/messages but all I found were portsentry's messages.

If we were under attack, or if the server was hacked, how do we trace it? Are there any guidelines or links through which we can possibly know what cause our server to respond all of the sudden?


Thanks in Advance.
0
Comment
Question by:manojoswal
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 40

Expert Comment

by:jlevie
ID: 11832964
> but we just couldn't access the server in any way.

Was the console still responsive? From the console could you initiate a network connection out to some other box (e.g., ssh some-local-host)? What was the load average on the box?

Is the server up to date (as in running up2date and allowing it to update the kernel)?
0
 
LVL 22

Accepted Solution

by:
pjedmond earned 150 total points
ID: 11835251
I recommend having a look at tripwire:

http://sourceforge.net/projects/tripwire/

It enables you to setup up an index of all the important files on your system, and if anyone messes with them you are notified. You can check the integrity of your files at any time that you want. This will give you the confidence to know that you javen't been hacked.

As for tracing the fault. First you need to check whether you can access the PC in anyway at all. Hopefully you should be able  to access at least a login prompt from the console?

If you can, then you need to try:

ps -ef               What processes are running
top                  Is anything using up the processor time


You may find that killing off/restarting certain rogue processes solves the problem, in which case you will know where to look next time.

You've done the obvious thing of looking in the logs. If there is nothing there, then you may need to increase the amount of logging that occurs. Linux software often has a -v (verbose), or -d  (debugging) mode.

It's also woth learning a little about klogd (the kernel logging daemon):

 http://linux.about.com/library/cmd/blcmdl8_klogd.htm

HTH:)
0
 
LVL 1

Assisted Solution

by:bestondoa
bestondoa earned 150 total points
ID: 11843279
Hello,

You should run a check on your systemany "rootkit". Rootkits are a collection of modified program sources or binaries which replace a selection of system binaries. For example, an hacker would enter your system and replace \bin\login with a modified one that has a secret username/password that will allways let him enter as root.

Go to http://www.chkrootkit.org/ and download chkrootkit. chkrootkit localy scans your system binaries for rootkit modification.

Cheers,

Bestondoa

0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 

Author Comment

by:manojoswal
ID: 11843478
1. We then got the server rebooted, since then its running fine.
2. We keep a good vigil on it, till the moment it stopped responding, we had adequate memory and CPU free. (its a dual xeon with 1 gb ram, SCSI and RAID) so the process that caused it was a sudden one.
3. There are no signs of DDOS or DOS.
4. We need to know exactly how we could go about finding the cause of this failure. What logs, what we should look for in it.
5. We have trip wire, it shows really nothing serious.

are there some links or guidlines somewhere that will guide us to track activity and find all possible traces of the cause of the problem.

regards

manoj



0
 

Author Comment

by:manojoswal
ID: 11843518
we need to report to the client the cause of this failure, any guidelines or papers on the net that have clues to foresnic analsysis of an attack.

regards

manoj oswal
0
 
LVL 40

Assisted Solution

by:jlevie
jlevie earned 100 total points
ID: 11843704
Is the server (and the kernel) up to date w/respect to the RHEL errata?

Is the BIOS on the system current?

What is the typical and peak load average?
0
 
LVL 1

Assisted Solution

by:LieutenantLefsa
LieutenantLefsa earned 100 total points
ID: 12171255
Did any of the disks fill up?
0

Featured Post

Why Off-Site Backups Are The Only Way To Go

You are probably backing up your data—but how and where? Ransomware is on the rise and there are variants that specifically target backups. Read on to discover why off-site is the way to go.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

rdate is a Linux command and the network time protocol for immediate date and time setup from another machine. The clocks are synchronized by entering rdate with the -s switch (command without switch just checks the time but does not set anything). …
Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

729 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question