We help IT Professionals succeed at work.

how to create meta search engine ?

savsoft
savsoft asked
on
916 Views
Last Modified: 2013-12-13
Meta search engine pass the quires through many search engines like google and yahoo but i want to know how their programming work ?
does it possible to make in php?
is google and other search engine gives rights to use their search engine for meta search engine ?


Comment
Watch Question

Marco GasiFreelancer
CERTIFIED EXPERT
Top Expert 2010

Commented:
Yes they do. You can start to read these pages for Google and Yahoo:

http://code.google.com/intl/it-IT/apis/ajax/
http://developer.yahoo.com/everything.html

You have to learn about curl also: http://php.net/curl

What you want to do is not so trivial to can be done with few lines of code. Good luck.

Author

Commented:
Thank you,
ok, i will read this..
Most Valuable Expert 2011
Author of the Year 2014
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION

Author

Commented:
Thank you ray and margusG.

I am reading your refered page and found very useful.
Actually i want to start my own search engine to show better search result then existing search engine. But i know that i can't crawl whole web like google nd bing. So i select meta search technology.
Please can you suggest me more if you have any better idea.
I have dedicated server of 500gb space 4gb ram

Most Valuable Expert 2011
Author of the Year 2014

Commented:
Not to discourage you, but what you're working on puts you in direct competition against Google, Yahoo, and Bing -- all of them are spending millions of dollars each month trying to get better search results.  They constantly study each other, and they have unlimited access to the top scientists and engineers.

I think you're in fine shape just using their search results through their APIs.  Just be careful of the terms of service - you may need to pay them if you use their data for commercial purposes.

Author

Commented:
is there any service where we can pay for their data usage?
I have also found an amazon web information service
http://aws.amazon.com
There we use all information of any website.
Can i use this information for my search engine. It charges $0.00015 per request. Alexa also powered by it.


Author

Commented:
As i understand search technology, according to it all existing search engine have web spider/web crawler program which start visiting website through some initializing url ( known as seeds) and store all website in their database to further index or page rank use. then crawler detect all hyper links and also add it to their seeds list.it need very large space to store these information. it is one type of downloading of whole web. i think it repeat all this process atleast once in 10 days.
if its true then i think its not an advance method. it makes large data transfer.

There is need to develop new technology of search engine....

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Thanks for using Experts Exchange.

Please provide your email to receive a sample view!

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.