Solved

should web bots / crawlers (like baidu) be blocked?

Posted on 2015-02-19
3
146 Views
Last Modified: 2015-02-19
Hello Web experts,

Should web bots / crawlers (like baidu) be blocked?

Thanks for your help.
0
Comment
Question by:epifanio67
  • 2
3 Comments
 
LVL 32

Accepted Solution

by:
_ earned 500 total points
ID: 40619398
Depends on whether you want your content Public, how easily you want it found, or if you want it Private.
Do you have a specific scenario in mind?

This has a blurb about the top 10  good bots, and you can decide if you want them on or not:
http://www.incapsula.com/blog/know-your-top-10-bots.html
0
 

Author Closing Comment

by:epifanio67
ID: 40619432
Thank you coral47....

good article.... it helps...

Regards
0
 
LVL 32

Expert Comment

by:_
ID: 40619541
Thank you much.    : )
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Short answer to this question: there is no effective WiFi manager in iOS devices as seen in Windows WiFi or Macbook OSx WiFi management, but this article will try and provide some amicable solutions to better suite your needs.
If you're not part of the solution, you're part of the problem.   Tips on how to secure IoT devices, even the dumbest ones, so they can't be used as part of a DDoS botnet.  Use PRTG Network Monitor as one of the building blocks, to detect unusual…
Learn how to set-up custom confirmation messages to users who complete your Wufoo form. Include inputs from fields in your form, webpage redirects, and more with Wufoo’s confirmation options.
Learn how to set-up PayPal payment integration in your Wufoo form. Allow your users to remit payment through PayPal upon completion of your online form. This is helpful for collecting membership payments, customer payments, donations, and more.

919 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now