[Webinar] Streamline your web hosting managementRegister Today

  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 316
  • Last Modified:

URL Submission to multiple search engines

Is anyone familiar with those sites on the web that charge you X amount of dollars for submitting your URL to a 100 (or whatever)
search-engines? Example - worldsubmit.com
My question is: Do you know how they do this ? I am assuming they use a CGI interface for each search-engine - and have their own script that simply executes the CGI for each of the 100 search engines.
 I am really looking for a script that will let me submit a URL
to multiple search engines. I have heard of some off-the-shelf
software that lets you do this  - but am really more interested
in something that I can interface to my web-application - like
a CGI script.

Thanks in advance

Anuj Varma (avarma@adversite.com)
1 Solution
avarmaAuthor Commented:
Any pointers would be very much appreciated.
to be fair - i dont think it's really worth registering to 200 search engines - but 20-30 yes..

...Register-It! will do 10-15 or so of the most popular ones in one gor for free...  ...i tihnk they're at www.register-it.com

there are a coupla other free ones out there but i dunno where

i think they just basically fake a response from the form on each search engine's site - the "Add My site" form, but with a different one for each engine (cos they all ask for slightly different stuff)
How THEY do it I'm not sure. How I'd do it is relatively simple. You start a cgi program which starts several threads. Each thread connects to a search engine by posting the query. (You'd need to know the exact format of the query for each server). The thread reads the response and, in simple cases, writes the data it got to the cgi output (ie: std_out). The thread then proceeds with the next engine until no engines are left. This executes the query in parallel for a fast response (the response is the slowest of the engines, assuming one thread per engine) (More that 10 threads is not really on).
   A more sophisticated approach is to parse the html reply from the engines separating out the answers from the advertising and other junk and merging the results into one result. This is made easier if you can agree on a search interface with the provider of the search engine.
It is very simple.  What they have done is determine what sequence the submission pages reach require.  You do this by manually doing a submission and seeing what is actually sent.  The vast majority of those that I have looked at do a GET type transmission so you can see it in the location bar, once you have pressed submit.

They then just execute requests to each of these engines using the get commands, for example, to submit to excite, you need to access:


You will need to replace the URL and eMail in the above.  

You can do these various submissions from a perl script, or you can even use a simple web page with Javascript and the submit() function of Javascript.


Featured Post

The 14th Annual Expert Award Winners

The results are in! Meet the top members of our 2017 Expert Awards. Congratulations to all who qualified!

Tackle projects and never again get stuck behind a technical roadblock.
Join Now