Solved

should web bots / crawlers (like baidu) be blocked?

Posted on 2015-02-19
3
155 Views
Last Modified: 2015-02-19
Hello Web experts,

Should web bots / crawlers (like baidu) be blocked?

Thanks for your help.
0
Comment
Question by:epifanio67
  • 2
3 Comments
 
LVL 32

Accepted Solution

by:
_ earned 500 total points
ID: 40619398
Depends on whether you want your content Public, how easily you want it found, or if you want it Private.
Do you have a specific scenario in mind?

This has a blurb about the top 10  good bots, and you can decide if you want them on or not:
http://www.incapsula.com/blog/know-your-top-10-bots.html
0
 

Author Closing Comment

by:epifanio67
ID: 40619432
Thank you coral47....

good article.... it helps...

Regards
0
 
LVL 32

Expert Comment

by:_
ID: 40619541
Thank you much.    : )
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
nexus filter logs 3 43
VPS for routing recomendations 3 49
No internet connection message when there is - multiple software 4 35
Lightweight Networking 9 36
If your business is like most, chances are you still need to maintain a fax infrastructure for your staff. It’s hard to believe that a communication technology that was thriving in the mid-80s could still be an essential part of your team’s modern I…
ADCs have gained traction within the last decade, largely due to increased demand for legacy load balancing appliances to handle more advanced application delivery requirements and improve application performance.
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

776 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question