Solved

IPS signatures that block web crawling

Posted on 2014-11-04
3
287 Views
Last Modified: 2014-11-16
I got a case where the following network IPS blocks a genuine web crawling application:
" HTTP Embedded Open Type / True Type Font Download "

Currently, it's found that the above network IPS signature blocks the crawling activity.

Q1:
Can I safely say that such 'crawling' activity is unlikely to be hindered by endpoint IPS
(ie IPS with agent sitting inside the VMs / servers) ?

Q2:
As we have both network IPS & endpoint based IPS, suppose the crawling still fails
after the network signature is lifted, should I also lift/disable the following endpoint
signatures (pls indicate which of them are likely to hinder ie likely to have same
effect as the above listed network IPS signature) :
1005154 - Adobe Flash Player Remote Code Execution Vulnerability
1005155 - Adobe Flash Player Remote Code Execution Vulnerability (CVE-2012-1535)
1005158 - Restrict Microsoft Office Files With Embedded SWF - 2
1004850 - Identified TTF File/OTF File Download
1004853 - Identified Suspicious Microsoft Office Files With Embedded Font
1004855 - Identified EOT File With Embedded TrueType Font File
1004858 - Identified Suspicious Microsoft Office Files With Embedded Dexter Font
1005250 - Identified Suspicious EOT File With Embedded Dexter Font
1003624 - Embedded OpenType Font Integer Overflow Vulnerability
1003623 - Embedded OpenType Font Heap Overflow Vulnerability

Q3:
Is there any other way to allow the crawling without compromising (ie
without lifting the signature), say whitelist the source of the known
crawling server?
0
Comment
Question by:sunhux
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 63

Accepted Solution

by:
btan earned 500 total points
ID: 40423746
Q1. Based on this vulnerability, it is targeting, for example, in the font parsing subsystem of the win32.sys driver -- provides an entry point for hackers to take complete control of an unpatched machine without any user action beyond normal browsing or opening a rigged document file.
http://blogs.technet.com/b/srd/archive/2009/11/10/font-directory-entry-parsing-vulnerability-in-win32k-sys.aspx

Specific cases includes
- Malicious fonts (TTF’s) delivered within .eot files hosted on malicious web sites which are rendered in all versions of Internet Explorer by default.
- Malicious office documents e-mailed to victims with social engineering to entice the victim to open the document which contains a malformed embedded font which would then be rendered upon opening the Office document (PowerPoint and Word documents are the most likely attack vectors).

The best protection from likely attacks is for all affected users to download and apply the patch. also at network level, there is still the signature too..e.g. http://tools.cisco.com/security/center/viewAlert.x?alertId=21470

remember the target is the endpoint client/server and not the network device in this case, hence the last defence like HIPS is critical as well ...

Q2. The key words is defense in depth and set up deterence on any form of atatck, adversary also can obfuscate such that the exploit which the signature is looking for is not able to detect. There can be encoding, re-odering or any other form of manipulation to ensure signature breaks and evade the detection to reach the endpoint.

Likewise not all the rigged document with exploit comes from network, there are channel directly into the machine by user carrier or email or password protected attachment that eventually still need to be reside in machine and start their modus operandi. In short, HIPS and NIPS are critical but they are not silver bullet for all attacks and known one if patch is not available...or not pushed down to reach the machine etc...


Q3. If it is authorised source then whitelisting is the best effort to not cause false alarm to ops team. prior informing is required and especially the start to end period to allow these. You should not whitelist ip as long term basis unless necessary and required. Bot and robots and crawler are the form of recon in the cyber kill chain, do not leave this as mild and neglect that.

..of course you can disable directory browsing and other form of recon as layer but it defeat the legit spidering objective. If there are WAF, IPS, NGFW, FW etc, you may want to still let it pass but do track it closely and validate no harm or anomalous activities during the period of sanction such crawling.
0
 

Author Comment

by:sunhux
ID: 40445602
My netadmin colleague has ack'ed it is possible to whitelist specific IP while still allow the above listed network ips signatures to be enabled.  This helps.  As always thanks for the usual impressive responses
0
 
LVL 63

Expert Comment

by:btan
ID: 40445605
thanks for sharing
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you thought ransomware was bad, think again! Doxware has the potential to be even more damaging.
Use of TCL script on Cisco devices:  - create file and merge it with running configuration to apply configuration changes
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question