Solved

Data scrapping of google results in C#

Posted on 2009-03-31
2
501 Views
Last Modified: 2012-05-06
Hi,
Can anyone please suggest a tutorial or good resource for data mining of Google results.
Actually I want to search a particular string in a Google and want to store them in my Access/MSSQL database.

Regards,
Mushi
0
Comment
Question by:Mushq
2 Comments
 
LVL 31

Assisted Solution

by:RiteshShah
RiteshShah earned 150 total points
ID: 24036762
0
 
LVL 23

Accepted Solution

by:
Tiggerito earned 350 total points
ID: 24037155
Google has discontinued registering for the soap api, however they do provide a new AJAX api in its place:

http://code.google.com/apis/ajaxsearch/

Here's some code I've played with...
// rsz=small    4

// rsz=large    8

// start=4      index based
 

string url = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&hl=en&rsz=large&start=0";
 

url += "&q=" + HttpUtility.UrlEncode("the search phrase");
 

url += "&key=XXXXXXXXXXXXXX";
 

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

request.Referer = "http://www.mydomain.com/"; // Google requests you do this
 

WebResponse response = request.GetResponse();
 

StreamReader reader = new StreamReader(response.GetResponseStream());
 

string jsonResponse = reader.ReadToEnd();
 

// then parse the jsonResponse 

Open in new window

0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
Illustrator's Shape Builder tool will let you combine shapes visually and interactively. This video shows the Mac version, but the tool works the same way in Windows. To follow along with this video, you can draw your own shapes or download the file…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now