URL Submission to multiple search engines

Is anyone familiar with those sites on the web that charge you X amount of dollars for submitting your URL to a 100 (or whatever)
search-engines? Example - worldsubmit.com
My question is: Do you know how they do this ? I am assuming they use a CGI interface for each search-engine - and have their own script that simply executes the CGI for each of the 100 search engines.
 I am really looking for a script that will let me submit a URL
to multiple search engines. I have heard of some off-the-shelf
software that lets you do this  - but am really more interested
in something that I can interface to my web-application - like
a CGI script.

Thanks in advance

Anuj Varma (avarma@adversite.com)
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

avarmaAuthor Commented:
Any pointers would be very much appreciated.
to be fair - i dont think it's really worth registering to 200 search engines - but 20-30 yes..

...Register-It! will do 10-15 or so of the most popular ones in one gor for free...  ...i tihnk they're at www.register-it.com

there are a coupla other free ones out there but i dunno where

i think they just basically fake a response from the form on each search engine's site - the "Add My site" form, but with a different one for each engine (cos they all ask for slightly different stuff)
How THEY do it I'm not sure. How I'd do it is relatively simple. You start a cgi program which starts several threads. Each thread connects to a search engine by posting the query. (You'd need to know the exact format of the query for each server). The thread reads the response and, in simple cases, writes the data it got to the cgi output (ie: std_out). The thread then proceeds with the next engine until no engines are left. This executes the query in parallel for a fast response (the response is the slowest of the engines, assuming one thread per engine) (More that 10 threads is not really on).
   A more sophisticated approach is to parse the html reply from the engines separating out the answers from the advertising and other junk and merging the results into one result. This is made easier if you can agree on a search interface with the provider of the search engine.
It is very simple.  What they have done is determine what sequence the submission pages reach require.  You do this by manually doing a submission and seeing what is actually sent.  The vast majority of those that I have looked at do a GET type transmission so you can see it in the location bar, once you have pressed submit.

They then just execute requests to each of these engines using the get commands, for example, to submit to excite, you need to access:


You will need to replace the URL and eMail in the above.  

You can do these various submissions from a perl script, or you can even use a simple web page with Javascript and the submit() function of Javascript.


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Scripting Languages

From novice to tech pro — start learning today.