Solved

Data scrapping of google results in C#

Posted on 2009-03-31
2
508 Views
Last Modified: 2012-05-06
Hi,
Can anyone please suggest a tutorial or good resource for data mining of Google results.
Actually I want to search a particular string in a Google and want to store them in my Access/MSSQL database.

Regards,
Mushi
0
Comment
Question by:Mushq
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
2 Comments
 
LVL 31

Assisted Solution

by:RiteshShah
RiteshShah earned 150 total points
ID: 24036762
0
 
LVL 23

Accepted Solution

by:
Tony McCreath earned 350 total points
ID: 24037155
Google has discontinued registering for the soap api, however they do provide a new AJAX api in its place:

http://code.google.com/apis/ajaxsearch/

Here's some code I've played with...
// rsz=small    4
// rsz=large    8
// start=4      index based
 
string url = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&hl=en&rsz=large&start=0";
 
url += "&q=" + HttpUtility.UrlEncode("the search phrase");
 
url += "&key=XXXXXXXXXXXXXX";
 
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Referer = "http://www.mydomain.com/"; // Google requests you do this
 
WebResponse response = request.GetResponse();
 
StreamReader reader = new StreamReader(response.GetResponseStream());
 
string jsonResponse = reader.ReadToEnd();
 
// then parse the jsonResponse 

Open in new window

0

Featured Post

Creating Instructional Tutorials  

For Any Use & On Any Platform

Contextual Guidance at the moment of need helps your employees/users adopt software o& achieve even the most complex tasks instantly. Boost knowledge retention, software adoption & employee engagement with easy solution.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article introduced a TextBox that supports transparent background.   Introduction TextBox is the most widely used control component in GUI design. Most GUI controls do not support transparent background and more or less do not have the…
Introduction Hi all and welcome to my first article on Experts Exchange. A while ago, someone asked me if i could do some tutorials on object oriented programming. I decided to do them on C#. Now you may ask me, why's that? Well, one of the re…
Michael from AdRem Software explains how to view the most utilized and worst performing nodes in your network, by accessing the Top Charts view in NetCrunch network monitor (https://www.adremsoft.com/). Top Charts is a view in which you can set seve…
In this brief tutorial Pawel from AdRem Software explains how you can quickly find out which services are running on your network, or what are the IP addresses of servers responsible for each service. Software used is freeware NetCrunch Tools (https…

724 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question