Solved

Network Unresponsive/timeout Issues

Posted on 2003-11-06
12
520 Views
Last Modified: 2013-12-07
Recently our ethernet network began experiencing a problem in which connection timeouts are occuring between the workstations and multiple servers.  We are using mulitiple Win2k server and one Novell 3 server, all over which are losing connectivity.  I had thought the the problem was narrowed down to a bad cisco switch, however after replacing the switch the problem still exists.  I can ping each server and watch as it replys for about 5 minutes strait, then will begin timing out. It will timeout for anywhere from 5 to 60 seconds before "coming back". I have tried just about every network analyzer I can find and do not notice any strange activity .  Any help, including anything suspicious to look for would be helpful.
0
Comment
Question by:fakir420
  • 3
  • 2
  • 2
  • +5
12 Comments
 
LVL 7

Assisted Solution

by:Robing66066
Robing66066 earned 100 total points
ID: 9696950
Sounds like it might be a broadcast storm brought on by a cabling loop.  Check your connections between switches.  If you have a situation where you can reach the same switch by two different paths, you may have a cabling loop.  (You may not, the lines may be trunked, but you can check this on the switch).  If you find such a loop, remove one of the paths and see if your problem goes away.

Good luck.  
0
 
LVL 35

Assisted Solution

by:ShineOn
ShineOn earned 100 total points
ID: 9697143
It could also be broadcast storm brought about by a worm like Welchia or MSBlaster.  Are all of your servers and workstations patched to current service packs and security hotfixes?  Do you run an antivirus realtime scanner?  Are the antivirus signature files kept up-to-date?

If the answer to ANY of those questions is "no" then you should suspect an infection and start tracking it down and cleaning it up.
0
 
LVL 16

Expert Comment

by:SteveJ
ID: 9697423
Any other switches on the network . . . running spanning tree?

When the ping fails, does the switch show a valid mac address for the machine that's plugged into it? Does the port show UP?

Check for stuff ShineOn and RObing66066 mention.

Good luck,
Steve
0
 
LVL 1

Author Comment

by:fakir420
ID: 9697592
I do not believe it is an RPC worm as I have looked at netowrk traffic and have not seen any signs of this type of traffic.  No changes have been made to cabling so I do not believe it could be a looping issues.  I first beleieved it had something to do with DNS, is there any kind of DNS specific broadcast/attack?  I can also supply a network trace file if needed.
0
 
LVL 1

Assisted Solution

by:rogue_phoenix
rogue_phoenix earned 100 total points
ID: 9698274


When the timeouts happen, do they occur on the clients simultaneously or each on its own?

are your interfaces/switch ports hard set, or do you rely on autonegotiation?
http://www.cisco.com/warp/public/473/3.html

If it's DNS related 1). it'll show up in the trace, 2).  it'd be a timeout more likely than a broadcast attack. Are there DNS proxies in the network that clients would be going through?

If none of the above leads anywhere, I'd like to see a network diagram and a wide open (no filters) trace (snoop, tcpdump or ethereal are preferable to netmon). Preferably from multiple clients & servers simultaneously, with either sync'd clocks or notes on the time differential so I can tell whats happening on each end, but at least one pair of a client-server mismatch would be the minimum.


0
 

Expert Comment

by:DRVV
ID: 9698556
Just for reference, I just posted a similiar problem on my network (Question Q_20790421 sporadic major packet loss).

It will be interesting to see if the cause of the problem is the same for both of our networks.
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 41

Expert Comment

by:stevenlewis
ID: 9698567
rerun the ping test, and make note of the time, and then check event viewer on the machines to see if anything unusual shows (like services stopping and starting)
0
 
LVL 1

Author Comment

by:fakir420
ID: 9698746
I see also now that on at least one server when doing netstat there is an established connection to unkown.level3.net port 80.  I have read about an exploit that alters registry settings/creates new hosts file, but I did find this on that machine.  I don't know why there would otherwise be an connection or if this has anything to do with the problem.

One software package shows traffic on the MS/TCP Loopback Interace on the servers rise to very high levels at the time of unresponsiveness.  I'm not sure what this shows as I don't really see that much traffic on the ethernet interface.
0
 
LVL 18

Accepted Solution

by:
chicagoan earned 200 total points
ID: 9699462
although incidents.org doesn't show any reports of probes from unknown.level3.net, it did ring a bell with me and there are lots of folks in the group reporting INCOMING connections from unkown.level3.net:80 which certainly should be suspicious.

This has all the earmarks of a worm. What you set a span port up on your switch and does your sniffer have an adaptor it can get into promiscuous mode? Have you set your firewall to log at debug level and analyzed the syslog from that?
0
 
LVL 1

Author Comment

by:fakir420
ID: 9702676
I've posted trace files from two seperate stations (made with Ethereal) in libpcap format.  The compressed file is about 25mb and can be downloaded from http://tdec-fish.dyndns.org/traces.rar
0
 
LVL 18

Assisted Solution

by:chicagoan
chicagoan earned 200 total points
ID: 9703980
took a look and other than suggesting you ban e-donkey there's not much to go on...
If you could put the sniffer on a span port or the wan segment it might help
 
0
 
LVL 35

Expert Comment

by:ShineOn
ID: 9704106
What're you allowing edonkey for anyway?  You should likewise ban kazaa, WinMX, IRC and any other sharing stuff... beyond being a source of unecessary traffic, it can expose you to legal issues.
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

I eventually solved a perplexing problem setting up telnet for a new switch.  I installed a new Cisco WS-03560X-24P switch connected to an existing Cisco 4506 running a WS-X4013-10GE Sup II-Plus. After configuring vlans and trunking,  I could no…
#Citrix #Citrix Netscaler #HTTP Compression #Load Balance
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now