If you are linking those pages via href like this ...

<a href="http://www.abcd.com/anypage.html">Click here</a>
or
<a href="http://www.abcd.com/anypage.html"><img src="SomeImage.gif"</a>

then change your href to

<a href="#" onclick="GoToSite('http://www.abcd.com/anypage.html')">Click here</a>

and have a function

<script language="javascript">
function GoToSite(URL)
{
    location.href = URL; // If you open the linked page in the current window;
    //window.open(URL, "NewWindow"); // If you open the linked page in the new window;
}
</script>

This way the linked site will not receive the HTTP_REFERER.

Change

<a href="#" onclick="GoToSite('http://www.abcd.com/anypage.html')">Click here</a>

to

<a href="#" onclick="GoToSite('http://www.abcd.com/anypage.html');return false;">Click here</a>

in the last post.

there is!

<a href="http://safeurl.de/?http://www.website.com/secret/data/"> ... </a>

amit_g is quite correct. You might go for a slightly simpler version:

<a href="http://www.abcd.com/anypage.html" onclick="self.location.href=this.href;return false;">Click here</a>

easily added to an existing link, and it gives the usual advance indication of where the user is going in the status line.

I just ran a test - using JavaScript to change self.location.href still sends the referrer, at least in my browser.

???  where exactly do you find that referrer value ?

I created two webpages, linked from one to the other using the suggested method, loaded the page, clicked the link, then looked at my server logs. The referrer was displayed in them.

I hope I didn't mislead anyone by oversimplifying how I worded my original question. I said I was 'linking' to other websites, thinking that if I could hide the referer [sic] from a link, it would apply in all cases. In reality, I am retrieving the HTML for the page using ActiveX in the parent, and then writing the HTML code to a child window. The reason for this is to suppress any script execution in the child window, thereby preventing pop-ups ands other script-driven distractions.

So here is the script I use to GET the HTML of the target page, and it is this action that always sends the 'referer' to the visited server's access log:  

childWindow[i] = window.open("",thisWindowName,windowProps,false);
var xmlHttp = new ActiveXObject("MICROSOFT.XMLHttp");
xmlHttp.open("GET", targetURL, false);
xmlHttp.send(null);
Then the ResponseText is written to childWindow[i] to display the page

What I notice, is that if I open the target URL using a normal 'window.open', no referer is logged in the access log of the visited server. Referer is blank, as if I typed the URL into the 'Address' of the browser. This is the bahavior that I am trying to make happen when I retrieve using ActiveX as above.

I keep reading snippets about how you can set the 'referer' for a web page using HTTP_REFERER in the HEAD, and that value will appear in the web access logs of any visited sites, not the actual Address of the page. However, it's not clear to me how or where you set that variable.

Thanks again.
Sam G

You could try

xmlHttp.setRequestHeader("Referer", "http://SomeSite");

but this doesn't work.

http://www.perfectxml.com/msxmlAnswers.asp?Row_ID=61

Good suggestion, and it had me hopeful. But as you point out, the Referer seems to be ignored, and the URL in the Address bar of Internet Explorer is always the Referer value that is sent to the server.

Is there a way to see the exact header that is constructed and sent to the server? That way we may be able to figure out how the Referer information is sent (or not).

Sam

I am quite lost Now.  The 'not set when jumping using javascript' is one of those truths I have always believed but never tested. And testing now it is is apparently false. referrer gets set anyway :(
sorry.

Why do you want to do this anyway?

The only reason I can think of is if you wish to hide your site from a third party you are taking content from (thereby breaching their copyright).  If so you should probably check the member guidelines.

You might break the chain with a serverside redirect. in php this could be:

<?php            // file named  redirect.php
header( "Location: " .$_GET['file'] );
exit;
?>

and your link rewritten as:
<a href="redirect.php?file=http://somesite.com/path/page.html">somesite</a>

Have you the resources to test something like that ?

There is nothing we can do at least in this object to set the request header. I would believe that this header reset is ignored because of browser security. I would like to see it documented somewhere but I could not find anything.

JakobA, this header is not set if we do a location.href="http://SomeSite.com/Somepage.html" so what you have believed is true. It is only that this object is somehow not allowing to reset the header.

dorward, I can appreciate your suggestion, but if anyone wanted to steal HTML code from a website, it is as easy as loading the page into IE and hitting  View|Source, and voila, there is all the source code!  I don't need to write any script. I must confess that I have often looked at other's source code to see how things are done, and then adapted pieces of it it to my own purpose. I think that dissemination of knowledge is at the heart of the internet.

I want to hide the referer because most of the linked sites are the worst kind of junk merchants (which I am NOT promoting, quite the opposite), and they could use the referer to go back to the web page on which resides my email address. I can't just do a normal window.open because any script in the opened page would execute and create a mess of pop-ups or worse.

Sam

You are not promoting them, but you are linking to them.
This question is becoming very weird :-))

There is a rather a lot of difference between "stealing" HTML code, and breaching the copyright on _content_ (i.e. the text that the HTML is structuring).

If your email address is in clear text on a page then address harvesting bots will find it. Trying to hide a link isn't going to do much good. Try a decent spam filter instead. I get very good results from Spam Assassin.

JakobA, I don't mean to be weird, but there are lots of sites out there that are critical of unethical sites that are populating the web, especially those promoting bogus medical products, pornography, get-rich-quick schemes, etc. In many cases, the critical site will provide links to the latest set of offenders.  That's all it is, along with a wrinkle that the viewer is not threatened by malicious scripts that are often running in such pages.

Please help me find a way to block the referer! According to all that I read it should be possible, and in fact, security is enhanced when a website does not know if a person came direct or through a link in another page. Why should any website know about any other pages that I have visited?

Weird Sam

Alas in using ActiveX objects You go beyond my area of expertize. And also beynd the capacity of my Netscape browser :-))

Actually I would not worry too much about beeing backtracked. I have seen a lot of such SPAM-sites and my impression is that the great majority of such siteowners are less than competent to read their own weblog, let alone understand the data in it.

best JakobA

> Why should any website know about any other pages that I have visited?

There are plenty of reasons, but if you want to stop sites finding out where YOU came from then configure your browser not to send a referrer or filter it out with a proxy server.

amit, it seems that your suggestion should work, namely 'xmlHttp.setRequestHeader("Referer", "http://SomeSite");'. but for some reason, it doesn't.

I performed a small test that proved that the Referer value sent to the server is an identical copy of the string in the Address bar of IE. For example, if my site is 'http://MyMainSite.com', and I change it to 'http://MyMainSite.com#MiddleOfPage', then the value of Referer in the web access log is 'http://MyMainSite.com#MiddleOfPage'. I guess that's not too startling, is it?

However, is there a way in which we can change the location value of the parent window without causing it to do a reload? If so, then we could change location to, for example, blank before the GET and then change it back to the correct value after the HTML is loaded. Whenever I have tried this, IE immediately reloads upon a change of location and screws everything up. Is there a way to set location and supporess reload, just while the GET is being executed?

Sam

I don't think it would be possible because as soon as you change the location, IE will reload the page. If it were possible, there will be no security on the browser. Anyone can write a malicious code to display something in URL and have content from somewhere else.

amit, I think you are right about not being able to change the location, as it would be a potential security breach. However, suppressing the Referer value seems like a way to IMPROVE privacy, by denying the server any of your browsing history, especially when you come to a server as a result of a Google search and all your search terms are part of the referer URL.

I just know there has to be a way to do it, but clearly it isn't obvious!

Thanks for your persistence,
Sam

You can get rid of the HTTP_REFERER by using just a few lines of php (placed wherever on the net):

makeSafe.php

<?
$url = $HTTP_GET_VARS['url'];
header("Location: $url");
?>


and create your links like this:

href="http://somewhere.com/makeSafe.php?url=http://www.site.net/"


I can't see why it wouldn't work even with your special controls in ActiveX;


You could make your "window.open" related function add this prefix (....makeSafe.php) to each href passed as argument;


Any progress with that?

// I tested this on both by local and external server supporting php and it worked...

Sure, it is possible to change or remove the HTTP_REFERER header. Many proxies and anonymizer do exactly that. But they don't do it in JavaScript. When programming for the browser, the capabilities are limited because of security constraints. In this case we are limited by what XMLHttp object provides. It sure does provides a method to add a header or modify a header but it doesn't allow that while it is instantiated in browser context.

Put this code in a .vbs file

set xmlHttp = CreateObject("Microsoft.XMLHTTP")
call xmlHttp.open("GET", "http://AnySite", false)
call xmlHttp.setRequestHeader("Referer", "http://localhost/Test/Junk.htm")
call xmlHttp.send()
responseText = xmlHttp.responseText
MsgBox(Right(responseText, 1000))

and you will see that the referer is changed.

amit, I think I implemented your suggestion, and I still get the Address of the parent page as the Referer in the server access log.

In the parent page, I created a function in VBscript:

<SCRIPT LANGUAGE="VBScript">
Function getHTML(targetURL)
  set xmlHttp = CreateObject("Microsoft.XMLHTTP")
  call xmlHttp.open("GET", targetURL, false)
  call xmlHttp.setRequestHeader("Referer", "NONE")
  call xmlHttp.send()
End Function
</SCRIPT>

Then I called the function from JavaScript, also in the parent page. And again the location of the parent window was sent as the referer.

I found a mention on some other website that "NONE" was a special vlaue that would suppress the referer, but it comes through no matter what. I tried several other values, including real URLs and "".

This is a very persistent problem!

Sam

I am sorry I did not mention it properly. I wanted to demonstrate that if you put the code in vbs file, the header would be changed. This happens because vbs file is executed on the desktop and not with the browser. The same code will not change the header when it is hosted inside the browser. So the same object and same method isn't letting us change the header once the object is hosted inside the browser and the reason could only be security.

amit, You're absolutely right! Executing VBscript on the desktop worked perfectly! Now, is there any way that I can execute external VBscript from JavaScript within IE?

I think we're getting there!

Thanks
Sam

amit, Some thoughts on Microsoft's security logic:

1. I could understand IE preventing the assignment of another domain to Referer, since it would allow JavaScript to pin the "blame" for a referral on a totally uninvolved domain. However, it should be possible to BLOCK sending the Referer in JavaScript, since that would add to privacy by preventing the server discovering the visitor's recent browsing history.

2. If JavaScript uses 'window.open', then no Referer is sent. Why should one method block the Referer and another method make it impossible to block the Referer?

Of course, it isn't the first time that Microsoft has left glaring contradictions and inconsistencies in its products ;-(

I see 2 possibilities, although I don't know how to do either:

1. Could server-side JavaScript or VBscript help us? I am completely ignorant in this area, but I could learn quickly.
2. Could I donwload a .VBS file along with the web page, and execute it from within IE so that it is able to create a null Referer as above?

I know we'll solve this probelm, since we are not trying to do anything that is fundamentally pernicious.

Thanks
Sam

Call me obsessive-compulsive, and I'll plead guilty. I really feel the need to solve this problem, and everyone can breathe a sigh of relief ;-)

I found a Microsoft article that seems to comtradict what we are seeing here:
http://support.microsoft.com/default.aspx?scid=KB;EN-US;Q178066&

The article states "This prevents local file names from being sent inadvertently to Web servers when linking from local content to Web sites that might snoop on such information." And yet when I go from my web page on local hard disk to the server, I see the entire path to the file in the Referer field ("file://C:\\Documents%20and%20Settings\\....."). So I am forced to divulge the path when MS says it explicitly prevents that!

Can anyone make any sense of this?

Sam

Amit, that sounds like a good lead. A couple of questions (again, forgive my ignorance):

1. The website is running on Linux/Apache, which I assume means that I don't have IIS. Is that correct? I can relocate to an NT/IIS server if necessary.

2. I am not familiar with ASP (although by tonight I will be!). How do I run the server-side ASP script from client-side JavaScript? How do I pass parameters to it, such as the target URL? How do I know when it has completed its job and I can advance to the next URL?

I'll start working on this approach and let you know what happens.

Cheers,
Sam

1) You are correct, you don't have IIS on a Linux box.
2) Learning ASP is not that big a deal but I am not sure if that would really help you in what you are trying to do.

Whatever you use for server side ASP/JSP/Perl/PHP..., please note that all server side script does is to produce HTML. The only difference is that it can change that HTML in each request. So a simple server side script like

<html>
<body>
...
My name is <% =GetMyName() %>.
...
</body>
</html>

will call GetMyName function and get the name from database/file or compute it (if it can!) and then generate

<html>
<body>
...
My name is Generated Name.
...
</body>
</html>

which it will send to the browser. So you do not call server side script from client side or vice versa. All server side script do is to generate HTML (and may be javascript code also) at run time. I hope I gave you enough insight of how server side script works and now it is up to you to decide if it can help you in doing what you are doing. Generally every solution involves a good mix of server side and client side scripting.

That looks like my only hope, Amit. I am now in the process of signing up for a Windows 2000 / IIS web hosting plan so that I can try it out.

I'll let you know what happens.

Please let me know if you have any more ideas.

Cheers
Sam

Amit, the prize goes to you once again! Many thanks for your knowledge and persistence. I nearly have it working. The method you showed me will, with a little programming, do the trick.

I am now struggling with getting the ASP child window to tell the parent window when it has finished GETting all the HTML. However, the big problem has been solved.

Thanks again.
Sam