Please help - Network was down.

jsctechy
jsctechy used Ask the Experts™
on
Hi all,
Had a big problem last night and was wondering if there is something I can do to prevent from happening again.  
Yesterday, at 5:30PM my router crashed.  Port FA0/1 (which connects to my LAN) was being hammered and showing almost 5,000 packets a second.  It is a Cisco 3640 router.  So I consoled to the router, and it kept locking up on me.  So if I would do a 'sh run' it would print out, maybe 1 screen, and then hang...  If I disconnected the FA0/1 port then the rest would print out.  Anyway, I tracked it down to a server that had mIRC on it... don't know how it got on there, but I checked my firewall.  I see rules in there, allow any source to 'THIS SERVER'.  Also I saw another one, but wasn't sure why it was there either.  Anyway, I turned off the server and everything started functioning correctly.  This morning I came in, and connected another server to the network (I used this servers network cable, the night before to attach to my laptop) and it went down again....  I disconnected that network cable and everything worked again.
Has anyone heard of something like this going around, or is there something I can do on my servers to prevent this?
I removed 1 rule from my firewall, but do not plan to remove the other until 4:30 today.  I cannot risk my network going down during trading hours.
Any advice would be helpful.  
PS: I called Cisco and talked to their tech for over an hour, which didn't help me for crap, halfway through the call he said 'oh, fa0/1 is the port with the problem?'............

Thanks,
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Sr. Systems Engineer
Top Expert 2008
Commented:
Sounds exactly like the old MSBlaster or Slammer worm outbreak.
Highly suggest you scrub all servers with online AV, latest patches, etc.
Several IRC worms went around a while back, so if these servers are not patched and updated, they could still be vulnerable.
You should also perform a firewall configuration review and take out any lines that allow any source IP addresses to touch your servers for services that shouldn't be allowed.  For example, if you have a mail server that allows any Internet host to send SMTP traffic to it, that's OK...but if it allows all IP or RDP traffic or some other type that shouldn't be there, then that should be taken out.

In my experience, if you look at your firewall rules and can't identify the purpose behind a rule that allows inbound traffic, it is better to take it out and see who screams than to leave it in and wonder what type of traffic is being allowed into your network...
jsctechyInfrastructure Team Lead

Author

Commented:
Thanks guys.  I'll look into both.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial