We help IT Professionals succeed at work.

How to programmatically search google and process results?

Yurich
Yurich asked
on
Hello,
I'm trying to do the following:

Send a request to google search, passing search parameters, like

http://www.google.co.nz/search?hl=en&q=search+terms&btnG=Search

Then get the result and process each result record by record.

I understand that there was Development Kit on Google, but first, I can't find anymore, and then I found some references that Google is not supporting it any more. The class name was: GoogleSearchService

I understand that I can look at http://google.com/apis and I have, but there're way too many apis and their description is not precise, so I'm not sure which one I can use.

Would appreciate your help,
Yurich
Comment
Watch Question

BRONZE EXPERT

Commented:
I started at this point:
http://code.google.com/apis/codesearch/

Author

Commented:
Thank you,

I've seen that page as well, but description they give on the page:

The Google Code Search data API allows client and web applications to search public source code for function definitions and sample code.

is absolutely NOT what I'm after or what I asked in this question. I don't need to search for function definitions, I need to perform google search. Unless I misunderstand something of course.

Regards,
Yurich

Éric MoreauSenior .Net Consultant
BRONZE EXPERT
Top Expert 2016

Commented:
BRONZE EXPERT
Commented:

Author

Commented:
Thank you guys,

2 eric: I've seen your article, this link doesn't work :( http://www.google.com/apis/download.html - I think I've seen this link before, that's why I mentioned that I can't find it...

2 chilltern: It's a nice piece of code, but I can't use it as is :( I need a library from Google and apparently I can't find where to get it from... They offer to modify the code on the page and try it again, but the code is actually read-only, so don't see the point in that...

What I want to do is something like this:
http://www.shoemoney.com:80/serps.php

I found about 20 programs like that but they ALL are not accurate :( for some reason, so I wondered if I could do something similar - it can't be THAT difficult :|

Still need some help,
Regards,
Yurich
Senior .Net Consultant
BRONZE EXPERT
Top Expert 2016
Commented:
Google has left the SOAP technology in favor of AJAX!
"As of December 5, 2006, we are no longer issuing new API keys for the SOAP Search API. Developers with existing SOAP Search API keys will not be affected."

See http://code.google.com/apis/ajaxsearch/

Author

Commented:
Thank you guys, it is actually more complex than I expected at the beginning, so I just used whatever tools I could find online.

Cheers,
Yurich

Commented:
I tried to do somthing like this and used plain old screen scraping on the normal SERP.

They locked me out after about 500 requests, so that's not possible either. Just so you know.