Solved

**URGENT ** Massive unknown server activity (or attack?)

Posted on 2003-11-13
35
636 Views
Last Modified: 2013-12-04
Go easy on me since I am a Web Programmer and not a Network or Server Engineer. :-)

I have a server running WIndows 2000 Server, IIS 5.0, Exchange 2000, SQL Server 2000, and Norton Corporate and Exchange Anti-virus. My Broadband connection goes through a Router that has basic blocking set-up but nothing specific.  I only have 2 active e-mail addresses. I have 5 or 6 websites but they are mostly "Test" sites in which clients come and view progress on their Web Project. So, this server gets very little traffic.

I have been running this server for something like 2 years and never had a problem. Now, all of the sudden, starting yesterday morning, my server is working away. Hard Drive spinng and a constant noise. She is just working and working.

All of my internal apps and websites are very slow because the server is busy doing something else.

How do I find out what it is doing? Where do I even begin to look for a cause so that I can fix this. I have NO CLUE where to start.

Thanks in advance!
0
Comment
Question by:DLockwood
  • 12
  • 11
  • 6
  • +4
35 Comments
 
LVL 49

Expert Comment

by:sunray_2003
ID: 9741026
0
 
LVL 49

Expert Comment

by:sunray_2003
ID: 9741029
0
 
LVL 49

Expert Comment

by:sunray_2003
ID: 9741031
Use a good firewall

Sunray
0
 
LVL 49

Expert Comment

by:sunray_2003
ID: 9741034
0
 
LVL 49

Expert Comment

by:sunray_2003
ID: 9741043
0
 

Author Comment

by:DLockwood
ID: 9741059
I am scanning for viruses as we speak, but I doubt that is the problem. I think I just figured it out....tell me what you think.

The other day I wanted to listen to some music and didn't want to tie up my personal machione because I was programming. So I pulled up www.365live.com on the server and listened to music for 8 hours os so. When I went back to the server there were something like 20 pop-up ads. Needless to say, I stopped doing that, but would they or could they have dropped something down on my machine?
0
 
LVL 3

Expert Comment

by:zamoti
ID: 9741322
Give her the 'ol Ctrl-Alt-Delete, get your task manager and look at the processes tab.  If you click on the "CPU" column header, it will sort them by CPU usage.  If there is an unknown or strange process you've never heard of topping the charts, you could try ending the process.  Beware, because if you end a fairly important process you might end up having to reboot.  If it's a terribly important one it just won't let you.  
If the CPU isn't that busy, you could check out the hard disk usage by using the Performance Monitor (click start, run, type "perfmon" without quotes).  Right click in the main graph window and select "Add Counters."  From here you have MANY choices of the events that you can monitor.  Sadly, they won't tell you the source of the use, but perhaps you could get an idea here.  You can track CPU usage, disk usage, RAM usage and network usage.

Good luck!

0
 
LVL 49

Expert Comment

by:sunray_2003
ID: 9741356
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9742424
Check for Zombies:

taken from: http://www.grc.com/dos/grcdos.htm
A Quick & Easy Check for IRC Zombie/Bots
-------------------------------------------------------------------------------------------------

If you have managed to read all the way through this lengthy and detailed adventure, I am sure you will agree that you do NOT want any of these nasty Zombies or their relatives running around loose inside your PC. Fortunately, it's quite easy to verify that your system is not currently infected by one of these IRC Zombie/Bots.

All of the IRC Zombie/Bots open and maintain static connections to remote IRC chat servers whenever the host PC is connected to the Internet. Although it is possible for an IRC chat server to be configured to run on a port other than "6667", every instance I have seen has used the IRC default port of "6667".

Consequently, an active connection to an IRC server can be detected with the following command:


netstat -an | find ":6667"
Open an MS-DOS Prompt window and type the command line above, then press the "Enter" key. If a line resembling the one shown below is NOT displayed, your computer does not have an open connection to an IRC server running on the standard IRC port. If, however, you see something like this:
 



TCP   192.168.1.101:1026   70.13.215.89:6667  ESTABLISHED
 . . . then the only question remaining is how quickly you can disconnect your PC from the Internet!
A second and equally useful test can also be performed. Since IRC servers generally require the presence of an "Ident" server on the client machine, IRC clients almost always include a local "Ident server" to keep the remote IRC server happy. Every one of the Zombie/Bots I have examined does this. Therefore, the detection of an Ident server running in your machine would be another good cause for alarm. To quickly check for an Ident server, type the following command at an MS-DOS Prompt:


netstat -an | find ":113 "
As before, a blank line indicates that there is no Ident server running on the default Ident port of "113". (Note the "space" after the 113 and before the closing double-quote.) If, however, you see something like this:
 



TCP     0.0.0.0:113     0.0.0.0:0     LISTENING
 . . . then it's probably time to pull the plug on your cable-modem!  

-------------------------------------------------------------------------------------------------

LucF
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9742488
If this is not the case, be a lucky man!! ;-)
But then you should install a network sniffer to find out if you can see where the data is comming from.
Check the event-viewer if you can see anything strange.
Check firewall logs.

Btw, also try http://www.eicar.org/anti_virus_test_file.htm and try downloading the first of the four files, your virusscanner should react right away. If it doesn't, something has disabled your virusscanner, do an online-scan or boot from a bootable cd-rom wich has a virusscanner.
0
 

Author Comment

by:DLockwood
ID: 9742606
LucF - thanks for all of your input. I tried everything you wrote and I have good news and bad news.

Good news - no Zombies - nothing on port 6667 or on 113. Also, my virus protection responded instantly to the very first fiel I even tried.

I have run Ad-Aware and it removed lots of objects and items from the registry.

Now the Bad News - Server is still chugging away and working very hard on something.
0
 

Author Comment

by:DLockwood
ID: 9742770
My Router says it has detected a Syn Flood attack Attack and blocked it. It says it blocked it but maybe this has something to do with it since the detection was at around the same time that this all started.
0
 

Author Comment

by:DLockwood
ID: 9742825
I just ran the following from a DOS Command: netstat -n -p tcp

There were entries that said "Syn Sent" when I expected them to say "Syn Received". Does this mean that someone has "hijacked" my machine and is now using mine to send out Syn Attacks?
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9742921
>Does this mean that someone has "hijacked" my machine and is now using mine to send out Syn Attacks?
not for sure, but I do suggest you block and drop ICMP packages on your router, not on your server. Take a look at: http://www.grc.com/dos/drdos.htm and scroll towards:
Distributed Reflection
A Next-Generation DDoS Attack

So you might be unaware, but some kind of script kiddies can still use your server for a DDoS attack.

If you really want to make sure you don't have a zombie running in your system, download zonealarm (free) and install it, if something strange is trying to make an outbound connection, you know what to do: BLOCK IT!

LucF
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9742939
Btw, also check your users machines for this.... Use a network sniffer to find out from wich computer these packages come. Just to be sure.
0
 

Author Comment

by:DLockwood
ID: 9743631
I have "discovered" another piece of info for the puzzle, which, btw, still has not been solved.

I looked under WinNT\System32\Logfiles\SMTPSVC1 and the log files in here are growing.
They were and have been typically 20k in size per day. This "Attack" started yesterday afternoon and the log for yesterday was 20,554k and todays log is 40,256 k and growing.

Running Ad-Aware and Spyware detection software appears to have slowed the attack down substantially, but my server hard drive is still chugging away.

****** HELP ************

What now?
0
 
LVL 13

Expert Comment

by:Gnart
ID: 9745425
1) you are Sync DOS attacked.  TCP three-way handshake.  TCP Sync packet is being send to you, it sounds as if your router is handling the Sync DOS attack.

2) you are running SQL so check your UDP for the sql-slammer..
SQL-Slammer worm attacks on port 1433 and 1434...... see here:

http://www.cert.org/advisories/CA-2003-04.html
I don't like flooding folks with lots of URLs.

Modify your router or firewall to drop TCP/UDP on those ports.....

Your WinNT\System32\Logfiles\SMTPSVC1 is filling up fast - are you also running SMTP?
You didn't mention that.  Check your Exchange store to see if you have an email worm that is mass mailing.  Most likely not, since you are running an antivirus.  I suspect that your mail server is being used to relay mail by spammers..... so check and secure your email server.

Monitor SMTP port to see if there is an increase in activity here....

cheers
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 32

Accepted Solution

by:
Luc Franken earned 500 total points
ID: 9747149
> I suspect that your mail server is being used to relay mail by spammers
Good thinking... ;-)

Take a look here if you want to sort that out:
http://support.microsoft.com/default.aspx?scid=kb;en-us;310380
0
 

Author Comment

by:DLockwood
ID: 9748751
Well gentlemen, my server has gotten very quiet again and I would say it is almost back to normal.

I have learned a lot even though it was training under duress :-)

I downloaded and installed a program called CommView onto my server so that I could try and see what is going on. The only thing that concerns me know is that there is soooooo much traffic coming through my server. I guess that I do not understand it all enough to know if this is normal.

When I view the packet statistics or IP connections, it moves so fast that I cannot even see it. Is this normal? What types of packets and from whom should I even see?
0
 

Author Comment

by:DLockwood
ID: 9748959
Well, I have changed my mind and still think that the saga continues. I noticed that all of the actions that I took seemed to improve things, but now I do not get any inbound e-mail.

So I went into exchange and started looking around. In my SMTP Virtual Server Queue there are thousands and thousands of messages that are queued up trying to be sent. I noticed 2 things that seem suspicious/interesting.
1.) After the domain name it says "(Remote Delivery)"
2.) that are set to "retry" all throughout the day.

Any ideas??
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9749305
Delete those messages, it seems your mailserver has been used to send spam messages. (you've checked the things on the link I gave you??)

>When I view the packet statistics or IP connections, it moves so fast that I cannot even see it. Is this normal?
Could happen, depends on what your server is doing normally, and if there's some kind of attack, you should create a log file (just log about an hour) to see if there are a lot of packages going some way (and where they come from, could be an infected machine inside your network)

LucF
0
 

Author Comment

by:DLockwood
ID: 9749526
LucF,

Yes I did all of the things that were on the link you gave me. The interesting thing is that I had already blocked relaying about 2 years ago. When I set up Exchange the first time, relaying was turned on and I got added to a group of black hole lists as a known spammer. Once I cleaned that up it ended.

I do have 1 Windows XP machine on my network that seems to be talking CONSTANTLY to the server. When you say it migtht be infected - do you mean by a virus or what?
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9749585
Yep, I mean a virus, just scan it, just to sort it out. Also look for spyware etc.
0
 

Author Comment

by:DLockwood
ID: 9750189
No viruses found.

I'll check for Spyware now.
0
 
LVL 1

Expert Comment

by:cubicleslave
ID: 9751103
Suggestion... try doing a netstat (or use TCPView, free software tool from sysinternals.com) on your XP machine, to see what processes are running and what ports are open/active connections to your server.  TCPView is like a continous, live netstat window.  Sounds like your XP machine may have been compromised.  Check the event logs on your XP machine.  Someone also suggested a software-based or desktop firewall (sunray_2003 and LucF)... if you don't have one installed on your XP machine, either turn on the ICF (internet connection firewall, built into XP) or install a 3rd party firewall program like ZoneAlarm, as a stopgap... to stop an illegitimate process on the XP system from communicating with your server temporarily, until you can get it sorted out.  (of course the additional logging of the firewall program is helpful too)  Check for a zombie or trojan process, as LucF suggested.  I would go with Gnart's suggestion, turn off SMTP if you don't need it (on both XP workstation and the server).
0
 

Author Comment

by:DLockwood
ID: 9751465
OK - Great. I have installed TCPView, but I don't know what any of it means. It doesn't look bad, but what do I know.

I'll check the logs and install ZoneAlarm.

BTW- I have followed lots of steps and do believe it was some sort of a syn flood attack. I have blocked the necessary ports and cleaned up anything that I could. Then I went and deleted ALL of the SMTP queued messages, which were all rogue anyway, and after rebooting my server, she is quiet as can be. Certainly back to normal.

Of course, now that I have lost 2 days of work, I would like to get to the root of the problem and try and prevent it from happening again (sure, like that's really goping to happen!).

When this sort of thing happens, is it usually done by someone that is out to get me or was I a random victim?

DL
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9751530
>When this sort of thing happens, is it usually done by someone that is out to get me or was I a random victim?
Usually you're a random victim, otherwise you'd have a massive DDoS attack wich would completely blow you off the internet, this seems to be just some kind of script kiddy wich just wants to annoy someone.

>after rebooting my server, she is quiet as can be.
Check it again tomorrow (or monday) If there are still not any strange things happening, I think this was the end of it.

>I would like to get to the root of the problem and try and prevent it from happening again (sure, like that's really goping to happen!).
Make sure your firewall settings are right, keep your virusscanners up-to-date, you know the drill ;-)
Once you get struck by a real DDoS attack, you should inform your ISP, they might be able to block it (you won't)

LucF
0
 

Author Comment

by:DLockwood
ID: 9751723
LucF,

Thanks for sticking with me on this - you'll never know how much I appreciate it. When you are dealing with something that is so foreign and you feel clueless, it is nice to know someone is following along and trying to assist through each new piece of info and/or problem.

It was hard to award the points because it was a combination of multiple answers and suggestions that ultimately led me to fixing the problem.

In the end however, it was an e-mail based attack and LucF was the one to provide that answer as well as sticking with me all the way.

Thanks to all.

DL
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9753291
You're welcome, I'm really glad this problems is solved.

Take care,

LucF
0
 

Expert Comment

by:sneak_nakata
ID: 9822750
great tutorial LucF... and great guru..
do u have any website LucF ?
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9822771
;-) Not at the moment, I'm still building one.. Once I have one, I'll post the link in my profile.

LucF
0
 

Expert Comment

by:sneak_nakata
ID: 9834669
Thanks..
do u have any Certified certificate LucF ?
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9835459
nope, to be honest, I'm a school dropout, so I had to teach myself, bought around 60 books to teach myself the basics of TCP/IP, different versions of windows (technet is now my homepage ;-)), etc. I do study now to get some certificates (Cisco/MCSA) hoping to pass exams in march.
0
 

Expert Comment

by:sneak_nakata
ID: 9836962
glad to heard from u...
Wow!! u're very intelligent person.. how old are u LucF ? ;)
0
 
LVL 32

Expert Comment

by:Luc Franken
ID: 9839545
ThanQ, birthday: 16-03-1981 (dd-mm-yyyy) so I'm 22 years old at the moment.
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

Recently, a new law in my state forced us to get a top-to-bottom analysis of all of our contract client's networks. While we have documentation, it was spotty at best for some - and in any event it needed to be checked against reality. That was m…
In today's information driven age, entrepreneurs have so many great tools and options at their disposal to help turn good ideas into a thriving business. With cloud-based online services, such as Amazon's Web Services (AWS) or Microsoft's Azure, bus…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now