cbonnock
asked on
Powerful web crawler/spider application
I am looking for the most powerful web crawler program I can find at any cost. I would like to spider a list of websites from a mysql database and return, metas, text, etc, back to the database. I would also like it be able to handle as many different types of docs as possible. Has anyone had good experiences with any of them? I have used a number of freeware versions, but they are far too unstable... many returning only 75% of sites back. If it helps, I don't need to go beyond one page with the crawler either.
Thanks for the input,
cbonnock
Thanks for the input,
cbonnock
ASKER CERTIFIED SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
ASKER
These are not quite what I am looking for...
$19.90 - iPageSaver 1.0
Free to Try
http://www.spidersoft.com/