Solved

Data scrapping of google results in C#

Posted on 2009-03-31
2
502 Views
Last Modified: 2012-05-06
Hi,
Can anyone please suggest a tutorial or good resource for data mining of Google results.
Actually I want to search a particular string in a Google and want to store them in my Access/MSSQL database.

Regards,
Mushi
0
Comment
Question by:Mushq
2 Comments
 
LVL 31

Assisted Solution

by:RiteshShah
RiteshShah earned 150 total points
ID: 24036762
0
 
LVL 23

Accepted Solution

by:
Tiggerito earned 350 total points
ID: 24037155
Google has discontinued registering for the soap api, however they do provide a new AJAX api in its place:

http://code.google.com/apis/ajaxsearch/

Here's some code I've played with...
// rsz=small    4

// rsz=large    8

// start=4      index based
 

string url = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&hl=en&rsz=large&start=0";
 

url += "&q=" + HttpUtility.UrlEncode("the search phrase");
 

url += "&key=XXXXXXXXXXXXXX";
 

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

request.Referer = "http://www.mydomain.com/"; // Google requests you do this
 

WebResponse response = request.GetResponse();
 

StreamReader reader = new StreamReader(response.GetResponseStream());
 

string jsonResponse = reader.ReadToEnd();
 

// then parse the jsonResponse 

Open in new window

0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction Although it is an old technology, serial ports are still being used by many hardware manufacturers. If you develop applications in C#, Microsoft .NET framework has SerialPort class to communicate with the serial ports.  I needed to…
This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.

920 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now