Solved

How to identify which web service is causing a DoS (Windows 2003 server)

Posted on 2010-11-17
25
269 Views
Last Modified: 2012-05-10
Recently, one of our Windows 2003 servers has been pushing out it's ethernet maximium 100mbps due to what we believe to be a DoS hack. I think the problem comes from a web service. This is a shared hosting server. When the peak occurs, if I turn off the IIS web server, the peak instantly stops. I tried doing this on a per domain approach when the peak happens as well but it's difficult to find the exact culprite as it seems the hack can tell i'm problem solving this and just when I think one specific Web service is the source, reactivating it doesn't start up the peak again ...

My questions:

Is there a simple Windows tools that will allow me to individually monitor Web service output ? I tried the Performance tuning console with all Web services added but this was non conclusive as the peaks had no effect on the "total bytes/sec" charts of all Web services ?

Any thoughts on this ?

Thanks,

Danny
6degres.ca
0
Comment
Question by:iDanny
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
25 Comments
 
LVL 31

Expert Comment

by:Justin Owens
ID: 34155293
Wireshark would be your friend on this.  It is a freeware network/port scanner which will give you details about incoming and outgoing network traffic.  Combine that with SysInternals ProcMon to get a PID level distinction of usage.  It can tell you at the PID level what each process is doing.
0
 

Author Comment

by:iDanny
ID: 34156023
Hi DrUltima,

Will either of these tools detail by web service ? As mentionned, this is a shared hosting server and there are about 20 different domains (web services) running in IIS. I'm looking to identify which one of these is causing the traffic peak.

Thanks,

Danny
0
 
LVL 31

Expert Comment

by:Justin Owens
ID: 34156122
Wireshark will see all incoming requests to specific ports and specific addresses.  It would see a difference in www.myaddress.com:80 and www.myaddress.com:8080.  It would also see a difference in www.myaddress1.com and www.myaddress2.com even if both are on the same port.  That is why I suggested it as a start.

The IIS service has multiple threads running under its service.  ProcMon will help you identify which thread within the service is actually using your resources.
0
Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

 

Author Comment

by:iDanny
ID: 34157615
Oooof ... I downloaded and installed both!

Wireshark seem to monitor all traffic on a specific interface, However, i'm pretty confused as to filter data on a per domain basis ? Also, you mentioned it monitors all "incoming requests", what about outgoing traffic ? Does outdoing traffic necesseraly imply an incoming request ?

Yes, i'm a newbie when it comes to ethernet analysis !

Thanks,

Danny
0
 

Author Comment

by:iDanny
ID: 34157947
Ok, let’s see if I can’t shed some clearer light into what’s happening exactly and perhaps you can guide me to a tool that will effectively show me the culprit without requiring a doctorate in ethernet protocols to configure it ;-)

I know something in the IIS structure is causing my traffic peaks (DoS like).  I know this as stopping IIS during a peak will stop the traffic peak immediately. Now I need to figure out in what website (web service) the peak is generate from so I can look deeper into that specific website for any hacked scripts.

As mentioned, I tried to close the sites one by one during a peak all while monitoring network total traffic but the hack seems works around this effort: the peaks become intermittent and further more apart when I stop some web services ...

Essentially, I need a tool that will measure an adapter’s traffic and match the source of the traffic to specific web services.

Danny
0
 
LVL 2

Expert Comment

by:mitrum
ID: 34166296
Wireshark &  Microsoft Network Monitor  will show to inbound and outbound traffic.

 for more detail you can see

http://support.microsoft.com/kb/812953

http://portforward.com/networking/wireshark.htm

you can use TCPDump / TCPEye , network monitoring software that displays the list of all currently opened TCP/IP and UDP ports on your local computer
0
 

Author Comment

by:iDanny
ID: 34197042
Well, it seems the problem originated from an IIS exploit. This explains why closing IIS would stop the traffic peaks but closing indivisual domains didn't.

Due to the urgency of the situation, I migrated all the domains to a new Windows server and will reformat this server.

Thanks,
Danny
0
 

Author Comment

by:iDanny
ID: 34247677
MMM ???

I had a new Windows 2008 server installed and migrated all the sites/domains over to it.

I seems the exploit has crossed over too ???

I really can't tell where it's coming from. I  am pretty sure this is in IIS as clsing IIS stops the surge of traffic. However, as mentionned, closing site by site doesn't seem to have an effect ?

Does anyone know how to monitor this to find the culprit ? I was thinking of a tool that would analyse the network traffic, find the port where the surge is and most importantly, identify the  process (and ideally the domain source assuming it's IIS) ?

Thanks,

Danny
0
 

Author Comment

by:iDanny
ID: 34247743
My toughts are that a file in a domain is exploiting IIS somehow. Just by saying that makes me realise that by closing the IIS domains, this should prevent the file from doings it's job ?

Help ...
0
 
LVL 31

Expert Comment

by:Justin Owens
ID: 34248680
Is the excess traffic incoming or outgoing?
0
 

Author Comment

by:iDanny
ID: 34249122
outgoing (udp)
0
 
LVL 31

Expert Comment

by:Justin Owens
ID: 34249174
Is the outgoing all going to the same address, or is is spread out?
0
 

Author Comment

by:iDanny
ID: 34249206
same ip
0
 
LVL 31

Expert Comment

by:Justin Owens
ID: 34249232
OK... and I am sorry for individual questions, I am just thinking as I ask... Is the IP address internal to your network or external?
0
 

Author Comment

by:iDanny
ID: 34249281
no problem.

It's a public IP. This is a web server (shared).

Danny
0
 
LVL 31

Expert Comment

by:Justin Owens
ID: 34249529
Can you use your firewall (software or hardware) to block input/output on that address and see if the traffic generation on the server stops?  It sounds like you are being targeted by someone.
0
 

Author Comment

by:iDanny
ID: 34249576
i can, but this is a web server and i need it's regular traffic to keep going through. This surge of traffic seems to be boincing from port to port (udp), I really need to fnd the culprit source.
0
 

Author Comment

by:iDanny
ID: 34250979
Essentially, i'm looking of a networking monitoring solution that will link outgoing UDP traffic to IIS domains.
0
 
LVL 31

Expert Comment

by:Justin Owens
ID: 34251021
I am unaware of something so specific.   Do you have any incoming requests from that IP, or is it 100% outgoing?
0
 

Author Comment

by:iDanny
ID: 34251115
i will run wireshark for a whlie to see if i can find anything worht sharing.
0
 
LVL 31

Expert Comment

by:Justin Owens
ID: 34251222
As I think about it, if an exploit is being ... well ... exploited, then stopping one site at a time until the traffic stops probably won't yield anything.  How many different sites is this server hosting?  Also, since you built a new machine, can you rebuild your old server and transfer the sites to it one at a time?  You should be able to identify the offending site at that time.  Also, what does the traffic contain which is being sent to that external IP?
0
 

Author Comment

by:iDanny
ID: 34253062
The server is hosting about 25 domains.

I have the ability to do what you are suggesting but to be honnest, this is something i'd like to keep as a very last resort. I just transfered all of these domain (took a good 2 days) and the key to this was keeping the same IP! You see, in shared hosting environments, we don't akways control the relative zone files and that becomes a nightmare when moving sites around.

I will check to see what the packets contain. I need to catch this while it happens with wireshark.



0
 

Accepted Solution

by:
iDanny earned 0 total points
ID: 34476652
I solved this by assigning individual .NET pools to each domain and then stopping them one by one to find the culprit. I found him :-)
0
 
LVL 27

Expert Comment

by:Tolomir
ID: 35225108
This question has been classified as abandoned and is being closed as part of the Cleanup Program. See my comment at the end of the question for more details.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Learn about cloud computing and its benefits for small business owners.
Ransomware continues to be a growing problem for both personal and business users alike and Antivirus companies are still struggling to find a reliable way to protect you from this dangerous threat.
Attackers love to prey on accounts that have privileges. Reducing privileged accounts and protecting privileged accounts therefore is paramount. Users, groups, and service accounts need to be protected to help protect the entire Active Directory …

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question