Solved

PIX Firewall Syslog Messages

Posted on 2009-07-07
5
870 Views
Last Modified: 2012-08-14
Hello,

I was going through our PIX syslogs and noticed that we are getting a large volume of 106015 messages. See example below:

2009-07-06 02:39:42      Lpr.Info      10.10.16.228      %PIX-6-106015: Deny TCP (no connection) from 208.70.187.70/80 to 64.122.142.138/31632 flags ACK  on interface outside

Here is Cisco's explanation:

106015

Error Message    %PIX-6-106015: Deny TCP (no connection) from IP_address/port to
IP_address/port flags tcp_flags on interface interface_name.

Explanation    This message is logged when the firewall discards a TCP packet that has no associated connection in the firewall unit's connection table. The firewall looks for a SYN flag in the packet, which indicates a request to establish a new connection. If the SYN flag is not set, and there is not an existing connection, the firewall discards the packet.

Recommended Action    None required unless the firewall receives a large volume of these invalid TCP packets. If this is the case, trace the packets to the source and determine the reason these packets were sent.


In our case, we are getting a large volume of these. I would like to know if this is something I should look into further? I am not sure what the best way is to trace the packets from the source to the destination. Is there software that can do this for me? Any recommendations would be appreciated.

Thanks, T
0
Comment
Question by:Streppa09
  • 3
  • 2
5 Comments
 
LVL 8

Expert Comment

by:Nothing_Changed
ID: 24796171
The source of that traffic is www.freshloc.net. the .com side of that is a temperature control and monitoring system company, the .net side is (im assuming) their customer/dealer support site nad requires login. The source port is 80, implying a SYN ACK to an ack that someone in your site sent into their webserver. You may be getting these log entries if there is a poorly written app on their side sourcing traffic back to you on a different IP than you are establishing a connection to (a kind of assymetrical routing). If freshloc.net or freshloc.com is NOT a place you or your people would be connecting to, it's an attack someone out in the wild is either bouncing off that company's compromised server, or they are source spoofing the traffic just to see if they can get into your network.

You can ignore it safely if it is not impacting site function, and if it is not enough traffic inbound to negatively impact your internet bandwidth.

You can verify that it is a bad response to a connection request coming out of your network in a number of ways. The simplest would be to tap the link between your firewall and your outside network, and watch a Sniffer for connection requests out that retry again and again to connect to port 80 on www.freshloc.com or .net that correlate with these log entries occuring. You could also turn your PIX logging up to debug for a while, and look for outbound SYNs to port 80 that dont get answered, then compare the destination port of these log entries to those and look for correlation. For instance, in your example 208.70.187.70/80 to 64.122.142.138/31632 shows your side's high port to be 31632 connected to the other side's port 80. If this is a bad web app and not an attack, you would expect to see traffic that never gets answered from your inside or outside IP heading to port 80, then an answer coming back to that same ip and port but FROM a different source on port 80, and that triggers the log entry.

0
 

Author Comment

by:Streppa09
ID: 24797034
Freshloc is a program that we are using internally to monitor temperature for food coolers and freezers. It sends information to their website, were users can login to view sensor readings.

You mentioned putting a sniffer between our firewall and outside connection. Do you have any software that you would recommend to do this? How would you go about this?

Thanks, T
0
 
LVL 8

Accepted Solution

by:
Nothing_Changed earned 250 total points
ID: 24806751
The best Sniffer is the actual Sniffer from Netscout (formerly Network General), but it's kind of pricey for a small organization. A really good free tool is Wireshark, they have pre compiled binaries for windows and linux available. Check out their wiki here if you want to --> http://wiki.wireshark.org/

But since you have a PIX, you also have a limited packet capture capability there as well. For this learning exercise, I think we'd do well to try that first. NOTE: If your PIX is hammered on CPU or memory, let me know and DO NOT do this. As long as your CPU utilization is under about 60% and memory is under about 75% or so you'll be totally fine.

First, we need to know if all of the log entries you are seeing are using the exact same IPs above, or are there other hosts? Of particualr importance is what host(s) are coming from the freshloc side.

Then, make an access list that will match all of the traffic you are interested in seeing. For our purposes here, that is your inside host(s) that could be natted behind your IP you mentioned above, any destination servers at freshloc you may use, and the IPs returning from freshloc that are incorrect.

For instance, lets say that you are natting all of your outbound traffic behind that one IP, and from nslookup I see that that freshloc is showing me only 208.70.187.70 for freshloc.net and www.freshloc.net, and 69.56.149.114 for freshloc.com and www.freshloc.com. If there are other hosts you use at freshloc, like someapplication.freshloc.something, nslookup or dig those too , and repeat it a few times to see if you always get the same IPs, then stick those IPs in the FRESHLOC object group, the access list will rebuild itself automagically.

I suggest using object groups so if you discover new interesting hosts, its far easier to add them. So, your PIX access lists would look something like this (paste this in at hte config prompt):
object-group network FRESHLOC
 network-object host 208.70.187.70
 network-object host 64.122.142.138

object-group network APPERTS
 network-object host 64.122.142.138

access-list CAP-FRESHLOC permit ip object-group FRESHLOC object-group APPERTS
access-list CAP-FRESHLOC permit ip object-group APPERTS object-group FRESHLOC

Then we create the capture on the PIX using this access list, from the enable prompt (you don't need config for a cap), like this:
capture FRESHLOC access-list CAP-FRESHLOC buffer 512000 packet-length 256 interface outside circular-buffer

NOTE: in case of problems with cpu or memory (im an over cautious network guy) , have the "no" version of this ready to paste to the command line, which would be "no capture FRESHLOC" without quotes.

This will set up a half Mb buffer and capture the first 256 bits of any packets between these sets of hosts, and wrap the buffer. I'm slicing the packets to only 256 bits so we do not see all of the http or app traffic at this point, we are only trying to correlate the request out to one host being answered by another host. Once we isolate that traffic, we can send a capture file to the freshloc guys so they can fix their (application/load balancers/routing/firewalls/etc.). If you REALLY want to see whats in the packets, make that packet-length 1522 instead of 256 and you'll get everything up to 802.1Q tags and complete app traffic. But do that after this is figured out :)

AGAIN NOTE: in case of problems with cpu or memory (im an over cautious network guy) , if your firewall starts chugging and not working well, have the "no" version of this ready to paste to the command line, which would be "no capture FRESHLOC" without quotes. paste it in and press enter, the firewall will catch up to the command shortly and disable the capture.

Then, to see if there are hits on the capture, from enable prompt just type this and you can see that stuff is or is not being captured:
HomePix# sh cap FRESHLOC
0 packet captured
0 packet shown
as you can see my house is NOT connected to freshloc using your IPs :)

Now, watch your firewall log for your errors to crop up, then the moment you see one, show the cap again like this (i made an ICMP cap as an example):
HomePix# sh cap ICMP detail
4 packets captured
14:27:17.371394 0030.8801.3db0 000c.ce7d.67d5 0x0800 74: 12.9.9.83 > 6.18.19.20: icmp: echo request (ttl 119, id 12235)
14:27:22.467276 0030.8801.3db0 000c.ce7d.67d5 0x0800 74: 12.9.9.83 > 6.18.19.20: icmp: echo request (ttl 119, id 12278)
14:27:27.967509 0030.8801.3db0 000c.ce7d.67d5 0x0800 74: 12.9.9.83 > 6.18.19.20: icmp: echo request (ttl 119, id 12329)
14:27:33.467398 0030.8801.3db0 000c.ce7d.67d5 0x0800 74: 12.9.9.83 > 6.18.19.20: icmp: echo request (ttl 119, id 12376)
4 packets shown

Then capture that capture dump text and the log entries, paste it in your next post, and I'll show you how to read it to see what I suspect your problem is.

0
 

Author Comment

by:Streppa09
ID: 24816857
I looked at a few things. First, CPU usage is at 0%. Second, memory utilized is less than 75%. So, we're good there. I also looked my syslogs again and I have over 7000 of the 106015 syslog messages, with many different ip addresses associated in each message. My question is, before I get hot and heavy with your procedure, which I very much appreciate you sharing your knowledge, will I have to repeat this for all 7000 messages? I am not sure if I have the time to slowly weed through all of the messages. Do you know if there an easier way to do this, or is this just the nature of the beast?

Thanks, T
0
 
LVL 8

Expert Comment

by:Nothing_Changed
ID: 24824081
No no, we only need to really catch one to verify the problem. Without even seeing traces, I'd wager that your service provider is answering requests you are sending to one ip/server from aonther one, due most likely to a misconfigured content switch(load balancer), or a misconfigured NAT  at tehir edge.
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

I recently updated from an old PIX platform to the new ASA platform.  While upgrading, I was tremendously confused about how the VPN and AnyConnect licensing works.  It turns out that the ASA has 3 different VPN licensing schemes. "site-to-site" …
I recently attended Cisco Live! in Las Vegas, a conference that boasted over 28,000 techies in attendance, and a week of hands-on learning hosted by a solid partner with which Concerto goes to market.  Every year, Cisco displays cutting-edge technol…
This video discusses moving either the default database or any database to a new volume.
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now