Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Network Unresponsive/timeout Issues

Posted on 2003-11-06
12
Medium Priority
?
529 Views
Last Modified: 2013-12-07
Recently our ethernet network began experiencing a problem in which connection timeouts are occuring between the workstations and multiple servers.  We are using mulitiple Win2k server and one Novell 3 server, all over which are losing connectivity.  I had thought the the problem was narrowed down to a bad cisco switch, however after replacing the switch the problem still exists.  I can ping each server and watch as it replys for about 5 minutes strait, then will begin timing out. It will timeout for anywhere from 5 to 60 seconds before "coming back". I have tried just about every network analyzer I can find and do not notice any strange activity .  Any help, including anything suspicious to look for would be helpful.
0
Comment
Question by:fakir420
  • 3
  • 2
  • 2
  • +5
12 Comments
 
LVL 7

Assisted Solution

by:Robing66066
Robing66066 earned 300 total points
ID: 9696950
Sounds like it might be a broadcast storm brought on by a cabling loop.  Check your connections between switches.  If you have a situation where you can reach the same switch by two different paths, you may have a cabling loop.  (You may not, the lines may be trunked, but you can check this on the switch).  If you find such a loop, remove one of the paths and see if your problem goes away.

Good luck.  
0
 
LVL 35

Assisted Solution

by:ShineOn
ShineOn earned 300 total points
ID: 9697143
It could also be broadcast storm brought about by a worm like Welchia or MSBlaster.  Are all of your servers and workstations patched to current service packs and security hotfixes?  Do you run an antivirus realtime scanner?  Are the antivirus signature files kept up-to-date?

If the answer to ANY of those questions is "no" then you should suspect an infection and start tracking it down and cleaning it up.
0
 
LVL 16

Expert Comment

by:SteveJ
ID: 9697423
Any other switches on the network . . . running spanning tree?

When the ping fails, does the switch show a valid mac address for the machine that's plugged into it? Does the port show UP?

Check for stuff ShineOn and RObing66066 mention.

Good luck,
Steve
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 1

Author Comment

by:fakir420
ID: 9697592
I do not believe it is an RPC worm as I have looked at netowrk traffic and have not seen any signs of this type of traffic.  No changes have been made to cabling so I do not believe it could be a looping issues.  I first beleieved it had something to do with DNS, is there any kind of DNS specific broadcast/attack?  I can also supply a network trace file if needed.
0
 
LVL 1

Assisted Solution

by:rogue_phoenix
rogue_phoenix earned 300 total points
ID: 9698274


When the timeouts happen, do they occur on the clients simultaneously or each on its own?

are your interfaces/switch ports hard set, or do you rely on autonegotiation?
http://www.cisco.com/warp/public/473/3.html

If it's DNS related 1). it'll show up in the trace, 2).  it'd be a timeout more likely than a broadcast attack. Are there DNS proxies in the network that clients would be going through?

If none of the above leads anywhere, I'd like to see a network diagram and a wide open (no filters) trace (snoop, tcpdump or ethereal are preferable to netmon). Preferably from multiple clients & servers simultaneously, with either sync'd clocks or notes on the time differential so I can tell whats happening on each end, but at least one pair of a client-server mismatch would be the minimum.


0
 

Expert Comment

by:DRVV
ID: 9698556
Just for reference, I just posted a similiar problem on my network (Question Q_20790421 sporadic major packet loss).

It will be interesting to see if the cause of the problem is the same for both of our networks.
0
 
LVL 41

Expert Comment

by:stevenlewis
ID: 9698567
rerun the ping test, and make note of the time, and then check event viewer on the machines to see if anything unusual shows (like services stopping and starting)
0
 
LVL 1

Author Comment

by:fakir420
ID: 9698746
I see also now that on at least one server when doing netstat there is an established connection to unkown.level3.net port 80.  I have read about an exploit that alters registry settings/creates new hosts file, but I did find this on that machine.  I don't know why there would otherwise be an connection or if this has anything to do with the problem.

One software package shows traffic on the MS/TCP Loopback Interace on the servers rise to very high levels at the time of unresponsiveness.  I'm not sure what this shows as I don't really see that much traffic on the ethernet interface.
0
 
LVL 18

Accepted Solution

by:
chicagoan earned 600 total points
ID: 9699462
although incidents.org doesn't show any reports of probes from unknown.level3.net, it did ring a bell with me and there are lots of folks in the group reporting INCOMING connections from unkown.level3.net:80 which certainly should be suspicious.

This has all the earmarks of a worm. What you set a span port up on your switch and does your sniffer have an adaptor it can get into promiscuous mode? Have you set your firewall to log at debug level and analyzed the syslog from that?
0
 
LVL 1

Author Comment

by:fakir420
ID: 9702676
I've posted trace files from two seperate stations (made with Ethereal) in libpcap format.  The compressed file is about 25mb and can be downloaded from http://tdec-fish.dyndns.org/traces.rar
0
 
LVL 18

Assisted Solution

by:chicagoan
chicagoan earned 600 total points
ID: 9703980
took a look and other than suggesting you ban e-donkey there's not much to go on...
If you could put the sniffer on a span port or the wan segment it might help
 
0
 
LVL 35

Expert Comment

by:ShineOn
ID: 9704106
What're you allowing edonkey for anyway?  You should likewise ban kazaa, WinMX, IRC and any other sharing stuff... beyond being a source of unecessary traffic, it can expose you to legal issues.
0

Featured Post

[Webinar] Cloud and Mobile-First Strategy

Maybe you’ve fully adopted the cloud since the beginning. Or maybe you started with on-prem resources but are pursuing a “cloud and mobile first” strategy. Getting to that end state has its challenges. Discover how to build out a 100% cloud and mobile IT strategy in this webinar.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I had an issue with InstallShield not being able to use Computer Browser service on Windows Server 2012. Here is the solution I found.
Make the most of your online learning experience.
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…
Monitoring a network: why having a policy is the best policy? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the enormous benefits of having a policy-based approach when monitoring medium and large networks. Software utilized in this v…
Suggested Courses

926 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question