Solved

Network Unresponsive/timeout Issues

Posted on 2003-11-06
12
525 Views
Last Modified: 2013-12-07
Recently our ethernet network began experiencing a problem in which connection timeouts are occuring between the workstations and multiple servers.  We are using mulitiple Win2k server and one Novell 3 server, all over which are losing connectivity.  I had thought the the problem was narrowed down to a bad cisco switch, however after replacing the switch the problem still exists.  I can ping each server and watch as it replys for about 5 minutes strait, then will begin timing out. It will timeout for anywhere from 5 to 60 seconds before "coming back". I have tried just about every network analyzer I can find and do not notice any strange activity .  Any help, including anything suspicious to look for would be helpful.
0
Comment
Question by:fakir420
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
  • 2
  • +5
12 Comments
 
LVL 7

Assisted Solution

by:Robing66066
Robing66066 earned 100 total points
ID: 9696950
Sounds like it might be a broadcast storm brought on by a cabling loop.  Check your connections between switches.  If you have a situation where you can reach the same switch by two different paths, you may have a cabling loop.  (You may not, the lines may be trunked, but you can check this on the switch).  If you find such a loop, remove one of the paths and see if your problem goes away.

Good luck.  
0
 
LVL 35

Assisted Solution

by:ShineOn
ShineOn earned 100 total points
ID: 9697143
It could also be broadcast storm brought about by a worm like Welchia or MSBlaster.  Are all of your servers and workstations patched to current service packs and security hotfixes?  Do you run an antivirus realtime scanner?  Are the antivirus signature files kept up-to-date?

If the answer to ANY of those questions is "no" then you should suspect an infection and start tracking it down and cleaning it up.
0
 
LVL 16

Expert Comment

by:SteveJ
ID: 9697423
Any other switches on the network . . . running spanning tree?

When the ping fails, does the switch show a valid mac address for the machine that's plugged into it? Does the port show UP?

Check for stuff ShineOn and RObing66066 mention.

Good luck,
Steve
0
Save the day with this special offer from ATEN!

Save 30% on the CV211 using promo code EXPERTS30 now through April 30th. The ATEN CV211 connects a laptop directly to any server allowing you instant access to perform data maintenance and local operations, for quick troubleshooting, updating, service and repair.

 
LVL 1

Author Comment

by:fakir420
ID: 9697592
I do not believe it is an RPC worm as I have looked at netowrk traffic and have not seen any signs of this type of traffic.  No changes have been made to cabling so I do not believe it could be a looping issues.  I first beleieved it had something to do with DNS, is there any kind of DNS specific broadcast/attack?  I can also supply a network trace file if needed.
0
 
LVL 1

Assisted Solution

by:rogue_phoenix
rogue_phoenix earned 100 total points
ID: 9698274


When the timeouts happen, do they occur on the clients simultaneously or each on its own?

are your interfaces/switch ports hard set, or do you rely on autonegotiation?
http://www.cisco.com/warp/public/473/3.html

If it's DNS related 1). it'll show up in the trace, 2).  it'd be a timeout more likely than a broadcast attack. Are there DNS proxies in the network that clients would be going through?

If none of the above leads anywhere, I'd like to see a network diagram and a wide open (no filters) trace (snoop, tcpdump or ethereal are preferable to netmon). Preferably from multiple clients & servers simultaneously, with either sync'd clocks or notes on the time differential so I can tell whats happening on each end, but at least one pair of a client-server mismatch would be the minimum.


0
 

Expert Comment

by:DRVV
ID: 9698556
Just for reference, I just posted a similiar problem on my network (Question Q_20790421 sporadic major packet loss).

It will be interesting to see if the cause of the problem is the same for both of our networks.
0
 
LVL 41

Expert Comment

by:stevenlewis
ID: 9698567
rerun the ping test, and make note of the time, and then check event viewer on the machines to see if anything unusual shows (like services stopping and starting)
0
 
LVL 1

Author Comment

by:fakir420
ID: 9698746
I see also now that on at least one server when doing netstat there is an established connection to unkown.level3.net port 80.  I have read about an exploit that alters registry settings/creates new hosts file, but I did find this on that machine.  I don't know why there would otherwise be an connection or if this has anything to do with the problem.

One software package shows traffic on the MS/TCP Loopback Interace on the servers rise to very high levels at the time of unresponsiveness.  I'm not sure what this shows as I don't really see that much traffic on the ethernet interface.
0
 
LVL 18

Accepted Solution

by:
chicagoan earned 200 total points
ID: 9699462
although incidents.org doesn't show any reports of probes from unknown.level3.net, it did ring a bell with me and there are lots of folks in the group reporting INCOMING connections from unkown.level3.net:80 which certainly should be suspicious.

This has all the earmarks of a worm. What you set a span port up on your switch and does your sniffer have an adaptor it can get into promiscuous mode? Have you set your firewall to log at debug level and analyzed the syslog from that?
0
 
LVL 1

Author Comment

by:fakir420
ID: 9702676
I've posted trace files from two seperate stations (made with Ethereal) in libpcap format.  The compressed file is about 25mb and can be downloaded from http://tdec-fish.dyndns.org/traces.rar
0
 
LVL 18

Assisted Solution

by:chicagoan
chicagoan earned 200 total points
ID: 9703980
took a look and other than suggesting you ban e-donkey there's not much to go on...
If you could put the sniffer on a span port or the wan segment it might help
 
0
 
LVL 35

Expert Comment

by:ShineOn
ID: 9704106
What're you allowing edonkey for anyway?  You should likewise ban kazaa, WinMX, IRC and any other sharing stuff... beyond being a source of unecessary traffic, it can expose you to legal issues.
0

Featured Post

Secure Your Active Directory - April 20, 2017

Active Directory plays a critical role in your company’s IT infrastructure and keeping it secure in today’s hacker-infested world is a must.
Microsoft published 300+ pages of guidance, but who has the time, money, and resources to implement? Register now to find an easier way.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
igmp snooping in layer 2 switch 4 26
Home firewall recommendations 11 86
VLAN Issue 4 68
Windows update hosed the internet connection to my VMs. 9 61
Data center, now-a-days, is referred as the home of all the advanced technologies. In-fact, most of the businesses are now establishing their entire organizational structure around the IT capabilities.
When you try to share a printer , you may receive one of the following error messages. Error message when you use the Add Printer Wizard to share a printer: Windows could not share your printer. Operation could not be completed (Error 0x000006…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…

735 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question