Solved

Network Unresponsive/timeout Issues

Posted on 2003-11-06
12
527 Views
Last Modified: 2013-12-07
Recently our ethernet network began experiencing a problem in which connection timeouts are occuring between the workstations and multiple servers.  We are using mulitiple Win2k server and one Novell 3 server, all over which are losing connectivity.  I had thought the the problem was narrowed down to a bad cisco switch, however after replacing the switch the problem still exists.  I can ping each server and watch as it replys for about 5 minutes strait, then will begin timing out. It will timeout for anywhere from 5 to 60 seconds before "coming back". I have tried just about every network analyzer I can find and do not notice any strange activity .  Any help, including anything suspicious to look for would be helpful.
0
Comment
Question by:fakir420
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
  • 2
  • +5
12 Comments
 
LVL 7

Assisted Solution

by:Robing66066
Robing66066 earned 100 total points
ID: 9696950
Sounds like it might be a broadcast storm brought on by a cabling loop.  Check your connections between switches.  If you have a situation where you can reach the same switch by two different paths, you may have a cabling loop.  (You may not, the lines may be trunked, but you can check this on the switch).  If you find such a loop, remove one of the paths and see if your problem goes away.

Good luck.  
0
 
LVL 35

Assisted Solution

by:ShineOn
ShineOn earned 100 total points
ID: 9697143
It could also be broadcast storm brought about by a worm like Welchia or MSBlaster.  Are all of your servers and workstations patched to current service packs and security hotfixes?  Do you run an antivirus realtime scanner?  Are the antivirus signature files kept up-to-date?

If the answer to ANY of those questions is "no" then you should suspect an infection and start tracking it down and cleaning it up.
0
 
LVL 16

Expert Comment

by:SteveJ
ID: 9697423
Any other switches on the network . . . running spanning tree?

When the ping fails, does the switch show a valid mac address for the machine that's plugged into it? Does the port show UP?

Check for stuff ShineOn and RObing66066 mention.

Good luck,
Steve
0
Building an interactive eFuture classroom

Watch and learn how ATEN provided a total control system solution including seamless switching matrix switch, HDBaseT extenders, PDU, lighting control to build an interactive eFuture classroom.

 
LVL 1

Author Comment

by:fakir420
ID: 9697592
I do not believe it is an RPC worm as I have looked at netowrk traffic and have not seen any signs of this type of traffic.  No changes have been made to cabling so I do not believe it could be a looping issues.  I first beleieved it had something to do with DNS, is there any kind of DNS specific broadcast/attack?  I can also supply a network trace file if needed.
0
 
LVL 1

Assisted Solution

by:rogue_phoenix
rogue_phoenix earned 100 total points
ID: 9698274


When the timeouts happen, do they occur on the clients simultaneously or each on its own?

are your interfaces/switch ports hard set, or do you rely on autonegotiation?
http://www.cisco.com/warp/public/473/3.html

If it's DNS related 1). it'll show up in the trace, 2).  it'd be a timeout more likely than a broadcast attack. Are there DNS proxies in the network that clients would be going through?

If none of the above leads anywhere, I'd like to see a network diagram and a wide open (no filters) trace (snoop, tcpdump or ethereal are preferable to netmon). Preferably from multiple clients & servers simultaneously, with either sync'd clocks or notes on the time differential so I can tell whats happening on each end, but at least one pair of a client-server mismatch would be the minimum.


0
 

Expert Comment

by:DRVV
ID: 9698556
Just for reference, I just posted a similiar problem on my network (Question Q_20790421 sporadic major packet loss).

It will be interesting to see if the cause of the problem is the same for both of our networks.
0
 
LVL 41

Expert Comment

by:stevenlewis
ID: 9698567
rerun the ping test, and make note of the time, and then check event viewer on the machines to see if anything unusual shows (like services stopping and starting)
0
 
LVL 1

Author Comment

by:fakir420
ID: 9698746
I see also now that on at least one server when doing netstat there is an established connection to unkown.level3.net port 80.  I have read about an exploit that alters registry settings/creates new hosts file, but I did find this on that machine.  I don't know why there would otherwise be an connection or if this has anything to do with the problem.

One software package shows traffic on the MS/TCP Loopback Interace on the servers rise to very high levels at the time of unresponsiveness.  I'm not sure what this shows as I don't really see that much traffic on the ethernet interface.
0
 
LVL 18

Accepted Solution

by:
chicagoan earned 200 total points
ID: 9699462
although incidents.org doesn't show any reports of probes from unknown.level3.net, it did ring a bell with me and there are lots of folks in the group reporting INCOMING connections from unkown.level3.net:80 which certainly should be suspicious.

This has all the earmarks of a worm. What you set a span port up on your switch and does your sniffer have an adaptor it can get into promiscuous mode? Have you set your firewall to log at debug level and analyzed the syslog from that?
0
 
LVL 1

Author Comment

by:fakir420
ID: 9702676
I've posted trace files from two seperate stations (made with Ethereal) in libpcap format.  The compressed file is about 25mb and can be downloaded from http://tdec-fish.dyndns.org/traces.rar
0
 
LVL 18

Assisted Solution

by:chicagoan
chicagoan earned 200 total points
ID: 9703980
took a look and other than suggesting you ban e-donkey there's not much to go on...
If you could put the sniffer on a span port or the wan segment it might help
 
0
 
LVL 35

Expert Comment

by:ShineOn
ID: 9704106
What're you allowing edonkey for anyway?  You should likewise ban kazaa, WinMX, IRC and any other sharing stuff... beyond being a source of unecessary traffic, it can expose you to legal issues.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When it comes to security, there are always trade-offs between security and convenience/ease of administration. This article examines some of the main pros and cons of using key authentication vs password authentication for hosting an SFTP server.
During and after that shift to cloud, one area that still poses a struggle for many organizations is what to do with their department file shares.
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…
There's a multitude of different network monitoring solutions out there, and you're probably wondering what makes NetCrunch so special. It's completely agentless, but does let you create an agent, if you desire. It offers powerful scalability …
Suggested Courses

617 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question