Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

should web bots / crawlers (like baidu) be blocked?

Posted on 2015-02-19
3
Medium Priority
?
174 Views
Last Modified: 2015-02-19
Hello Web experts,

Should web bots / crawlers (like baidu) be blocked?

Thanks for your help.
0
Comment
Question by:epifanio67
  • 2
3 Comments
 
LVL 32

Accepted Solution

by:
_ earned 2000 total points
ID: 40619398
Depends on whether you want your content Public, how easily you want it found, or if you want it Private.
Do you have a specific scenario in mind?

This has a blurb about the top 10  good bots, and you can decide if you want them on or not:
http://www.incapsula.com/blog/know-your-top-10-bots.html
0
 

Author Closing Comment

by:epifanio67
ID: 40619432
Thank you coral47....

good article.... it helps...

Regards
0
 
LVL 32

Expert Comment

by:_
ID: 40619541
Thank you much.    : )
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article, we’ll look at how to deploy ProxySQL.
In this article, the configuration steps in Zabbix to monitor devices via SNMP will be discussed with some real examples on Cisco Router/Switch, Catalyst Switch, NAS Synology device.
Learn how to set-up custom confirmation messages to users who complete your Wufoo form. Include inputs from fields in your form, webpage redirects, and more with Wufoo’s confirmation options.
If you're a developer or IT admin, you’re probably tasked with managing multiple websites, servers, applications, and levels of security on a daily basis. While this can be extremely time consuming, it can also be frustrating when systems aren't wor…
Suggested Courses

926 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question