Solved

should web bots / crawlers (like baidu) be blocked?

Posted on 2015-02-19
3
138 Views
Last Modified: 2015-02-19
Hello Web experts,

Should web bots / crawlers (like baidu) be blocked?

Thanks for your help.
0
Comment
Question by:epifanio67
  • 2
3 Comments
 
LVL 32

Accepted Solution

by:
_ earned 500 total points
ID: 40619398
Depends on whether you want your content Public, how easily you want it found, or if you want it Private.
Do you have a specific scenario in mind?

This has a blurb about the top 10  good bots, and you can decide if you want them on or not:
http://www.incapsula.com/blog/know-your-top-10-bots.html
0
 

Author Closing Comment

by:epifanio67
ID: 40619432
Thank you coral47....

good article.... it helps...

Regards
0
 
LVL 32

Expert Comment

by:_
ID: 40619541
Thank you much.    : )
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

Let’s list some of the technologies that enable smooth teleworking. 
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now