• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 314
  • Last Modified:

spider

what is spider? how do it works?
Can you teach me to write spider programmes?
0
a_flying
Asked:
a_flying
1 Solution
 
a_flyingAuthor Commented:
Edited text of question
0
 
MasseyMCommented:
A spider is a piece of software that will "weave" its was across the World Wide web in search of specfic items... It is a very difficult thing to write, but basically, it goes like this:

you have a list of sites... you search the pages on those site for "keywords" and you then log that info... Obviously the ones with the higher occurances of the keywords, the better the page... (some disagree... this is especially true if it is a sex site...)  
A lot of spiders get their sites from a proxy server or by random guessing... A lot of search engines use "spider" technology... ie.. WWW.METACRAWLER.COM and WWW.HOTBOT.COM .

0

Featured Post

Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now