Link to home
Create AccountLog in
Avatar of cbonnock
cbonnock

asked on

Powerful web crawler/spider application

I am looking for the most powerful web crawler program I can find at any cost. I would like to spider a list of websites from a mysql database and return, metas, text, etc, back to the database. I would also like it be able to handle as many different types of docs as possible. Has anyone had good experiences with any of them? I have used a number of freeware versions, but they are far too unstable... many returning only 75% of sites back. If it helps, I don't need to go beyond one page with the crawler either.

Thanks for the input,
 cbonnock
ASKER CERTIFIED SOLUTION
Avatar of nep1
nep1

Link to home
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
See answer
Avatar of nep1
nep1

"If it helps, I don't need to go beyond one page with the crawler either." Try this>>>

$19.90 - iPageSaver 1.0
Free to Try
http://www.spidersoft.com/
Avatar of cbonnock

ASKER

These are not quite what I am looking for...