IPS signatures that block web crawling
Posted on 2014-11-04
I got a case where the following network IPS blocks a genuine web crawling application:
" HTTP Embedded Open Type / True Type Font Download "
Currently, it's found that the above network IPS signature blocks the crawling activity.
Can I safely say that such 'crawling' activity is unlikely to be hindered by endpoint IPS
(ie IPS with agent sitting inside the VMs / servers) ?
As we have both network IPS & endpoint based IPS, suppose the crawling still fails
after the network signature is lifted, should I also lift/disable the following endpoint
signatures (pls indicate which of them are likely to hinder ie likely to have same
effect as the above listed network IPS signature) :
1005154 - Adobe Flash Player Remote Code Execution Vulnerability
1005155 - Adobe Flash Player Remote Code Execution Vulnerability (CVE-2012-1535)
1005158 - Restrict Microsoft Office Files With Embedded SWF - 2
1004850 - Identified TTF File/OTF File Download
1004853 - Identified Suspicious Microsoft Office Files With Embedded Font
1004855 - Identified EOT File With Embedded TrueType Font File
1004858 - Identified Suspicious Microsoft Office Files With Embedded Dexter Font
1005250 - Identified Suspicious EOT File With Embedded Dexter Font
1003624 - Embedded OpenType Font Integer Overflow Vulnerability
1003623 - Embedded OpenType Font Heap Overflow Vulnerability
Is there any other way to allow the crawling without compromising (ie
without lifting the signature), say whitelist the source of the known