Solved

Check for a word in Google's first page of results

Posted on 2010-09-03
14
348 Views
Last Modified: 2012-08-13
I have a few different domain names and i'm trying to build a small screen to check to see if each of the domains appear on googles homepage for a particular search phrase.  I have tried this same technique with Yahoo and it works fine.  When I try it with google I get this message:

Forbidden
Your client does not have permission to get URL /search?hl=en&q=flowers&btnG=Google+Search from this server. (Client IP address: xx.xxx.xx.xx)

Please see Google's Terms of Service posted at http://www.google.com/terms_of_service.html ...

I'm passing two variables into the search one is the text to look for on the page and the other is the search phrase.

Any ideas why I can not do this or how this,  surely the http request is the same as me searching in my browser?

<%


strInputURL = "http://www.google.com/search?hl=en&q="& request("search")&"&btnG=Google+Search"


websiteurl = request("findtext")
Set objXMLHTTP = Server.CreateObject("MSXML2.ServerXMLHTTP.3.0")
objXMLHTTP.Open "GET", strInputURL, False
objXMLHTTP.Send
strPageText = objXMLHTTP.responseText
Set objXMLHTTP = Nothing
if instr(strPageText,websiteurl) then
response.Write("Website is on the first page!!<br>") 

else

response.Write("Sorry your website is not on the first page :(<br><br>") 

end if

%>

Open in new window

0
Comment
Question by:djpazza
  • 5
  • 5
  • 2
  • +1
14 Comments
 
LVL 22

Expert Comment

by:Om Prakash
ID: 33594762
When i added this code in asp page, i get following message:
"Website is on the first page"

please check the following thread:
http://www.google.com/support/forum/p/customsearch/thread?tid=77f6f7c75926ec20&hl=en
0
 
LVL 9

Author Comment

by:djpazza
ID: 33595266
Your would need to pass some variables to it see it working I have another page doing this.  I have updated the code to submit the data on the same page.  So you type in the search word and a word to look for on the results page.

So when I search google.co.uk for flowers I know the words "Stunning Flowers" appear but I get this google permission error come up using the same on  my example code.
<%@LANGUAGE="VBSCRIPT" CODEPAGE="65001"%>
<%
if request("button") <>"" then

strInputURL = "http://www.google.com/search?hl=en&q=" & request("search")&"&btnG=Google+Search"


websiteurl = request("findtext")
Set objXMLHTTP = Server.CreateObject("MSXML2.ServerXMLHTTP.3.0")
objXMLHTTP.Open "GET", strInputURL, False
objXMLHTTP.Send
strPageText = objXMLHTTP.responseText
Set objXMLHTTP = Nothing
if instr(strPageText,websiteurl) then
response.Write("Website is on the first page!!<br>") 

else

response.Write("Sorry your website is not on the first page :(<br><br>") 
response.Write strPageText
end if
end if

%>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Untitled Document</title>
</head>

<body>
<form id="form1" name="form1" method="get" action="">
  <table width="500" border="0" cellspacing="0" cellpadding="0">
    <tr>
      <td width="280" scope="col">Search phrase</td>
      <td width="220" scope="col"><label>
        <input type="text" name="search" id="search" />
      </label></td>
    </tr>
    <tr>
      <td>Keyword or website address to search in results page</td>
      <td><input type="text" name="findtext" id="findtext" /></td>
    </tr>
    <tr>
      <td>&nbsp;</td>
      <td><label>
        <input type="submit" name="button" id="button" value="Submit" />
      </label></td>
    </tr>
  </table>
</form>
</body>
</html>

Open in new window

0
 
LVL 22

Assisted Solution

by:Om Prakash
Om Prakash earned 166 total points
ID: 33595328
I think the server doesn't allow automated searches.
Read Google's Terms of Service page.
http://www.google.com/terms_of_service.html 
0
Gigs: Get Your Project Delivered by an Expert

Select from freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely and get projects done right.

 
LVL 82

Expert Comment

by:leakim971
ID: 33596763
0
 
LVL 9

Author Comment

by:djpazza
ID: 33597231
I have made a website using the search control / search api and tried querying that page but that does not work as the search results are not actually on the page.  As in when I view source the search results are not in the code.
0
 
LVL 82

Accepted Solution

by:
leakim971 earned 334 total points
ID: 33597279
Chek the following :

(you will need to customize it to get same result as an usual google search)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8"/>
<script src="http://www.google.com/jsapi?key=AIzaSyA5m1Nc8ws2BbmPRwKu5gFradvD_hgq6G0" type="text/javascript"></script>
<script type="text/javascript">

	google.load('search', '1');

	function searchComplete() {
		if(webSearch.results && webSearch.results.length>0) {
			var r = webSearch.results;
			for(var i=0;i<r.length;i++) {
				result = r[i];
				if( document.getElementById("findtext").value.indexOf(result.visibleUrl)>=0 ) {
					alert("your rank is : " + parseInt(i+1))
				}
			}
		}
		else {
			alert("no results!");
		}
		search.disabled = false;
		findtext.disabled = false;
	}

	var webSearch;
	
	function getRankFirstPage() {
		var search = document.getElementById("search");
		var findtext = document.getElementById("findtext");
		search.disabled = true;
		findtext.disabled = true;
		webSearch = new google.search.WebSearch();
		webSearch.setSearchCompleteCallback(this, searchComplete, null);
		webSearch.execute(search.value);
	}

	function OnLoad() {
	}

	google.setOnLoadCallback(OnLoad);

</script>
</head>
<body style="font-family: Arial;border: 0 none;">
<!--
<div id="content">Loading...</div>
-->
<form id="form1" name="form1" method="get" action="">
<table width="500" border="0" cellspacing="0" cellpadding="0">
    <tr>
    	<td width="280" scope="col">Search phrase</td>
	    <td width="220" scope="col"><label><input type="text" name="search" id="search" value="paulroach" /></label></td>
    </tr>
    <tr>
	    <td>Keyword or website address to search in results page</td>
    	<td><input type="text" name="findtext" id="findtext" value="www.paulroach.com" /></td>
    </tr>
    <tr>
	    <td>&nbsp;</td>
    	<td><label><input type="button" name="button" id="button" value="get ranking" onclick="getRankFirstPage()" /></label></td>
    </tr>
</table>
</form>
</body>
</html>

Open in new window

0
 
LVL 82

Expert Comment

by:leakim971
ID: 33597383
0
 
LVL 9

Author Comment

by:djpazza
ID: 33616489
Thanks leakim971 that looks like it may do the job however i need the result to print to the page not in alert box and need to be able to query it via a querystring.

I've had a bash but its not working correctly when i check the page result for a value over 0 it says found everytime.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8"/>
<script src="http://www.google.com/jsapi?key=AIzaSyA5m1Nc8ws2BbmPRwKu5gFradvD_hgq6G0" type="text/javascript"></script>
<script type="text/javascript">

	google.load('search', '1');

	function searchComplete() {
		if(webSearch.results && webSearch.results.length>0) {
			var r = webSearch.results;
			for(var i=0;i<r.length;i++) {
				result = r[i];
				if( document.getElementById("findtext").value.indexOf(result.visibleUrl)>=0 ) {
					/*alert("your rank is : " + parseInt(i+1))*/
					document.getElementById("test").textContent = parseInt(i+1);
				}
			}
		}
		else {
			alert("no results!");
		}
		/*search.disabled = false;
		findtext.disabled = false;*/
	}

	var webSearch;
	
	function getRankFirstPage() {
		
		var search = document.getElementById("search");
		var findtext = document.getElementById("findtext");
		/*search.disabled = true;
		findtext.disabled = true;*/
		webSearch = new google.search.WebSearch();
		webSearch.setSearchCompleteCallback(this, searchComplete, null);
		webSearch.execute(search.value);
	}

	function OnLoad() {
	}

	google.setOnLoadCallback(OnLoad);

</script>
</head>
<body style="font-family: Arial;border: 0 none;" onload="getRankFirstPage();">
<!--
<div id="content">Loading...</div>
-->
<form id="form1" name="form1" method="get" action="">
<table width="500" border="0" cellspacing="0" cellpadding="0">
    <tr>
    	<td width="280" scope="col"></td>
      <td width="220" scope="col"><input name="findtext" type="hidden" id="findtext" value="<%=request("findtext")%>" />
        <input name="search" type="hidden" id="search" value="<%=request("search")%>" /></td>
    </tr>
    <tr>
	    <td></td>
    	<td></td>
    </tr>
    <tr>
	    <td><span id="test"></span>&nbsp;</td>
    	<td></td>
    </tr>
</table>
</form>
</body>
</html>

Open in new window

0
 
LVL 82

Expert Comment

by:leakim971
ID: 33632537
Replace : document.getElementById("test").textContent = parseInt(i+1);
By: document.getElementById("test").innerHTML = parseInt(i+1);

0
 
LVL 9

Author Comment

by:djpazza
ID: 33658480
Ok thanks will be back at work Tuesday will try that
0
 
LVL 9

Author Comment

by:djpazza
ID: 33670867
Is that not the same as the code i'm already using just that .textcontent is limited to firefox ?
0
 
LVL 82

Assisted Solution

by:leakim971
leakim971 earned 334 total points
ID: 33687358
0
 
LVL 142

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 35126296
This question has been classified as abandoned and is being closed as part of the Cleanup Program. See my comment at the end of the question for more details.
0

Featured Post

Gigs: Get Your Project Delivered by an Expert

Select from freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely and get projects done right.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
javascript question 6 49
How to show result of a NULL value at random? 32 50
2 delete statements in one query, one of which has multiple rows to delete 9 59
PHP AJAX JSON 2 57
Hello, all! I just recently started using Microsoft's IIS 7.5 within Windows 7, as I just downloaded and installed the 90 day trial of Windows 7. (Got to love Microsoft for allowing 90 days) The main reason for downloading and testing Windows 7 is t…
Have you ever needed to get an ASP script to wait for a while? I have, just to let something else happen. Or in my case, to allow other stuff to happen while I was murdering my MySQL database with an update. The Original Issue This was written…
Migrating to Microsoft Office 365 is becoming increasingly popular for organizations both large and small. If you have made the leap to Microsoft’s cloud platform, you know that you will need to create a corporate email signature for your Office 365…
In this video I am going to show you how to back up and restore Office 365 mailboxes using CodeTwo Backup for Office 365. Learn more about the tool used in this video here: http://www.codetwo.com/backup-for-office-365/ (http://www.codetwo.com/ba…

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now