IPS signatures that block web crawling

Posted on 2014-11-04
Last Modified: 2014-11-16
I got a case where the following network IPS blocks a genuine web crawling application:
" HTTP Embedded Open Type / True Type Font Download "

Currently, it's found that the above network IPS signature blocks the crawling activity.

Can I safely say that such 'crawling' activity is unlikely to be hindered by endpoint IPS
(ie IPS with agent sitting inside the VMs / servers) ?

As we have both network IPS & endpoint based IPS, suppose the crawling still fails
after the network signature is lifted, should I also lift/disable the following endpoint
signatures (pls indicate which of them are likely to hinder ie likely to have same
effect as the above listed network IPS signature) :
1005154 - Adobe Flash Player Remote Code Execution Vulnerability
1005155 - Adobe Flash Player Remote Code Execution Vulnerability (CVE-2012-1535)
1005158 - Restrict Microsoft Office Files With Embedded SWF - 2
1004850 - Identified TTF File/OTF File Download
1004853 - Identified Suspicious Microsoft Office Files With Embedded Font
1004855 - Identified EOT File With Embedded TrueType Font File
1004858 - Identified Suspicious Microsoft Office Files With Embedded Dexter Font
1005250 - Identified Suspicious EOT File With Embedded Dexter Font
1003624 - Embedded OpenType Font Integer Overflow Vulnerability
1003623 - Embedded OpenType Font Heap Overflow Vulnerability

Is there any other way to allow the crawling without compromising (ie
without lifting the signature), say whitelist the source of the known
crawling server?
Question by:sunhux
  • 2
LVL 62

Accepted Solution

btan earned 500 total points
ID: 40423746
Q1. Based on this vulnerability, it is targeting, for example, in the font parsing subsystem of the win32.sys driver -- provides an entry point for hackers to take complete control of an unpatched machine without any user action beyond normal browsing or opening a rigged document file.

Specific cases includes
- Malicious fonts (TTF’s) delivered within .eot files hosted on malicious web sites which are rendered in all versions of Internet Explorer by default.
- Malicious office documents e-mailed to victims with social engineering to entice the victim to open the document which contains a malformed embedded font which would then be rendered upon opening the Office document (PowerPoint and Word documents are the most likely attack vectors).

The best protection from likely attacks is for all affected users to download and apply the patch. also at network level, there is still the signature too..e.g.

remember the target is the endpoint client/server and not the network device in this case, hence the last defence like HIPS is critical as well ...

Q2. The key words is defense in depth and set up deterence on any form of atatck, adversary also can obfuscate such that the exploit which the signature is looking for is not able to detect. There can be encoding, re-odering or any other form of manipulation to ensure signature breaks and evade the detection to reach the endpoint.

Likewise not all the rigged document with exploit comes from network, there are channel directly into the machine by user carrier or email or password protected attachment that eventually still need to be reside in machine and start their modus operandi. In short, HIPS and NIPS are critical but they are not silver bullet for all attacks and known one if patch is not available...or not pushed down to reach the machine etc...

Q3. If it is authorised source then whitelisting is the best effort to not cause false alarm to ops team. prior informing is required and especially the start to end period to allow these. You should not whitelist ip as long term basis unless necessary and required. Bot and robots and crawler are the form of recon in the cyber kill chain, do not leave this as mild and neglect that.

..of course you can disable directory browsing and other form of recon as layer but it defeat the legit spidering objective. If there are WAF, IPS, NGFW, FW etc, you may want to still let it pass but do track it closely and validate no harm or anomalous activities during the period of sanction such crawling.

Author Comment

ID: 40445602
My netadmin colleague has ack'ed it is possible to whitelist specific IP while still allow the above listed network ips signatures to be enabled.  This helps.  As always thanks for the usual impressive responses
LVL 62

Expert Comment

ID: 40445605
thanks for sharing

Featured Post

Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Phishing is at the top of most security top 10 efforts you should be pursuing in 2016 and beyond. If you don't have phishing incorporated into your Security Awareness Program yet, now is the time. Phishers, and the scams they use, are only going to …
If you're not part of the solution, you're part of the problem.   Tips on how to secure IoT devices, even the dumbest ones, so they can't be used as part of a DDoS botnet.  Use PRTG Network Monitor as one of the building blocks, to detect unusual…
After creating this article (, I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
Here's a very brief overview of the methods PRTG Network Monitor ( offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…

932 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now