Solved

No Internet Acess: PAT seems to be reaching max limit, PIX 501

Posted on 2006-10-30
10
303 Views
Last Modified: 2013-11-16
Hi all,

Recently users have been complaining about not being able to access the internet. I checked our firewall, but nothing seemed to be out of the ordinary, and still no one could access the internet. Then I checked the cpu usage, and it fluctuated between 79 - 96%. I did a show xlate and found a LOT of translations taking place (numerous were from one or two pc's, which says virus, but AV hasnt found anything). However, when I did a clear xlate, internet connectivity seemed to be restored. The cpu usage is still pretty high, and I have to do a clear xlate every 15-20 minutes to keep connectivity up. I am not sure what's going on, but from what I have seen, it seems that the Pix might be reaching the limit of how many PATs it can do.  

Any help is appreciated on this.
0
Comment
Question by:pgreetis
10 Comments
 
LVL 20

Expert Comment

by:calvinetter
ID: 17840036
Yes, definitely smells like an infested PC or 2 - malware itself, or malware that's installed an SMTP engine, trying to spam the world.  Mini SMTP engines on PCs are popular w/ malware these days.

As a temporary measure only, you can lower the timeout for NAT translations (default is 3 hrs):
timeout xlate 0:15:00   <- sets it to 15 min limit
clear xlate

Capture the output of "sh xlate" to a file, & see what source ports & what source internal IPs are accounting for the most traffic.  Then track down those PCs & take them off the network while you scan w/ both AV tools & anti-spyware tools.

cheers
0
 

Author Comment

by:pgreetis
ID: 17851551
Hi, thank you for that...I did lower the timelimit to 15, and that seemed to hold up for a day, but this morning, some people were complaining again, so I checked the firewall again. And this time, there weren't that many PATs (maybe 50 or so) and people were still having problems. Additionally, i did a show cpu usage, and it was at 1% (everytime internet connectivity goes dows- so does the cpu usage). Also, I cannot telent into the pix anymore for some odd reason.

So, here are the symptoms of the firewall as soon as connecetivity goes down (these were there before as well, the only difference is that PATs werent this low):

CPU usage goes down to 1%
Cannot telnet at all
PATs USED to reach 6000+, currently there are only like 50 (from one pc).
All computers that try to open connections are on port 80- they're contacting advertising, spam websites, and some come up as page not found, or "nothing to see here" (we tracked down the websites).
Operation is normal only after I do a clear xlate (as in I can telnet back in, connectivity goes up).


So yes, this is definitely a virus and its spreading, however, what confuses me, is why the pix stops responding. I have currently re-routed all traffic to a different pix, however some pc's are still on the faulty pix (the ones with viruses), and they open numerous connections, but nowhere near the limit of PAT.

Thoughts?

Thank you for your response and help on this.
0
 
LVL 20

Expert Comment

by:calvinetter
ID: 17853661
 Regarding the PIX, I'd recommend some preventative measures:
- If the PIX isn't running the latest 6.3(5) code, & you have current Cisco "SmartNet" support on it, I strongly suggest upgrading the firmware to 6.3(5).  Same goes for any other 501 or 506 PIX you may have there.
- If you're not using the PDM web interface to manage/configure your PIX, you should disable the web server:
  no http server enable

  Regarding the workstations, take the offending PCs off the local LAN, clean out all temp files (ccleaner is good for this: www.ccleaner.com), then do a full hd scan with antivirus & anti-spyware apps; Adaware & Spybot used together are quite good at finding spyware.  AVG Free is a good freeware antivirus app (free.grisoft.com), finds some malware that commercial ones might miss - use it in conjunction with whatever commercial AV app you've got.

cheers
0
 

Author Comment

by:pgreetis
ID: 17868843
Hello, once again, thank you for your continued help, and my apologies for not updating this sooner. We finally figured out what was going on. Seems like one of the machines was sending orders to the other ones, and the other ones at that time started opening connections to spam websites. The affected machine itself was trying to connect to two specific IPs. As soon as I blocked those IP's everything went back to normal.

The last question that was bothering me a bit was when this happens (too many xlates), when we try to telnet, now the pix does something different...it gives a screen that has:

Username:

Password:


EDIT, EXECUTE (I think) OTHER OPTIONS.

(All this is on one screen, at the same time).

Essentially if I pick edit, it lets me enter a username and password, but thats it...nothing else happens..Im not sure if this behaviour is normal, or why it is heppening. I can console in fine though.

So, thats the last of it, I think. We have the issue figured out, and also reduced the xlate to 15 mins(which helped a lot). Currently scanning machines and keep an eye on the ones that were opening, and know for a fact which one machine is responsible for this mess. I will try and get a support contarct with Cisco and upgrade the PIX as well.

Once again, thank you for all your help.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 20

Expert Comment

by:calvinetter
ID: 17870859
>EDIT, EXECUTE (I think) OTHER OPTIONS.
   That's definitely not normal, never seen that from a PIX.  Sounds like maybe one of the infected PCs were hijacking the telnet session?? Or just your telnet client barfing when it can't connect??

  I strongly suggest NOT using telnet to your PIX, since everything (including the enable password) are sent in the clear!
   Use SSH instead (if it has DES or 3DES licensed):
sh ver   <-- look for  "VPN-DES:    Enabled"   If so, procede below:
pix(config)# ca zero rsa
pix(config)# ca gen rsa key 1024   <-- wait, or go & get coffee
pix(config)# ca save all   <-- very important! without this it won't save the key after a reload
pix(config)# ssh 0.0.0.0 0.0.0.0 inside

  Then use an SSH client (putty is a great, free one - just Google it) to connect to the PIX.  Use the same username/password if using AAA, or "pix" as username, & telnet password.  ** Be sure to configure 'putty' or other SSH client to use SSH v1 - PIX 6.x doesn't support SSH v2 connections!

cheers
0
 

Author Comment

by:pgreetis
ID: 17962555
calvinetter....Im an idiot. We figured out what the problem was, and I should be ashamed of myself for making such a stupid mistake....we installed a new linksys switch, and without checking its default setting I plugged it into the core- turns out its default IP out of the box was exactly the same as our PIX...the interface we were getting was of the switch.

However, we did have a virus, and the above was actually happening with one of the servers opening all the connections. You suggestions helped a LOT, and did keep us up and running till we found the infected machine.

So, problem is solved, and I hang my head in shame for making such a stupid mistake. ^_^

Thanks for all your help!
0
 
LVL 20

Expert Comment

by:calvinetter
ID: 17965944
Hey, don't worry about it - stuff happens!  Glad you got it going.

cheers
0
 
LVL 51

Expert Comment

by:Keith Alabaster
ID: 18108597
Calvinetter, Can you have a look at this one for me?
http://www.experts-exchange.com/Security/Firewalls/Q_22063144.html

Thanks
keith
0
 
LVL 1

Accepted Solution

by:
Computer101 earned 0 total points
ID: 18136465
PAQed with points refunded (400)

Computer101
EE Admin
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Overview The Cisco PIX 501, PIX 506e, ASA 5505 and ASA 5510 (most if not all of this information will be relevant to the PIX 515e but I do not have a working configuration handy to verify the validity) are primarily used within small to medium busi…
Have you experienced traffic destined through a Cisco ASA firewall disappears and you do not know if the traffic stops in the firewall or somewhere else? The solution is the capture feature. This feature was released in 6.2(1) and works in all firew…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
This video discusses moving either the default database or any database to a new volume.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now