Baidu.com crawler ip-addresses?

peps03
peps03 used Ask the Experts™
on
Hi,

I can't find Baidu.com crawlers' ip-addresses?
Can anybody help me with this?

Thanks!
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2013
Commented:
Baidu has thousands of IPs that it uses.  

If you want to know which they are using, then you will probably need to capture the user agent for all requests to robots.txt, and then save the ip.

The user agents: and details of the ip for each user agent can be found here:
http://www.useragentstring.com/pages/Baiduspider/

Cd&

Author

Commented:
Thanks!! Thousands is too much to gather :)

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial