PHP Referrer not detect if redirect from another domain using .htaccess

Note: Change immediately to using 302s, as 301s cache permanently in browsers, so any typo or redirect target change will never effect any previous traffic.

Many small fortunes have been lost because a 301 was used, rather than a 302.

Better to use real URLs for all this for testing.

A simple example is your domain.com Webserver config may prohibit forwarding any referrer data, so you'll never have any referrer data chain through redirects.

To debug this.

You'll require access to both domain.com + iwhost.com Webserver configs, then review all headers involved.

For example...

imac> curl -I -L https://DavidFavor.com/
HTTP/1.1 200 OK
Date: Sat, 28 Dec 2019 17:03:40 GMT
Server: Apache/2.4.41 (Ubuntu)
Strict-Transport-Security: max-age=63072000; preload
Content-Security-Policy: default-src https:; font-src https: data:; img-src https: data:; script-src https:; style-src https:; report-uri https://davidfavor.com/tracker?key=OyU4KBEERJwD1OwvZClvncSoVIsszo4C&type=csp
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Referrer-Policy: no-referrer-when-downgrade
X-XSS-Protection: 1; mode=block; report=https://davidfavor.com/tracker?key=OyU4KBEERJwD1OwvZClvncSoVIsszo4C&type=xss
Feature-Policy: geolocation 'none'; midi 'none'; camera 'none'; usb 'none'; magnetometer 'none'; accelerometer 'none'; vr 'none'; speaker 'none'; ambient-light-sensor 'none'; gyroscope 'none'; microphone 'none'
Upgrade: h2,h2c
Connection: Upgrade
Last-Modified: Thu, 01 Aug 2019 11:31:47 GMT
Accept-Ranges: bytes
Content-Length: 11685
Vary: Accept-Encoding
Cache-Control: max-age=0, public, no-cache, no-store, must-revalidate
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
X-Edge-Location: YUL
Content-Type: text/html; charset=UTF-8

Select allOpen in new window

Sets Referrer-Policy to pass through referrer details only to HTTPS sites, so any downgrades to HTTP strip all referrer data.

This is common. In fact, this is required these days for basic security.

To answer your question requires working with your actual URLs.

There are just to many variations to beginning making theoretical guesses about this.

Best to test + know, rather than guess.

Hi.. Thanks.

The actual domain is redirect from https://internet-webhosting.com to https://iwhost.com .

Anybody can help ?

The reason for this problem is as I guessed.

imac> curl -s -I -L https://internet-webhosting.com
HTTP/1.1 301 Moved Permanently
Date: Sun, 29 Dec 2019 21:24:57 GMT
Server: Apache
Referrer-Policy: unsafe-url
Location: https://iwhost.com/
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 200 OK
Date: Sun, 29 Dec 2019 21:24:58 GMT
Server: Apache
Cache-Control: max-age=2592000
Expires: Tue, 28 Jan 2020 21:24:58 GMT
Content-Type: text/html; charset=UTF-8

Select allOpen in new window

Notice the header Referrer-Policy: unsafe-url is set.

The unsafe-url specifically strips off all referrer data to close several potential security holes.

You must change your Referrer-Policy policy to forward along Referrer data.

Try Referrer-Policy: no-referrer-when-downgrade which is a good compromise.

Aside: Your site iwhost.com is running very slow.

To fix this refer to https://www.webpagetest.org/result/191229_1D_3eb6a0f84e368f2ff9e37c90a9b58585/1/details/#waterfall_view_step1 which suggests several changes...

1) Looks like you're using some sort of custom code which has poor caching design. Best fix this.

Tip: WordPress can easily be tuned to 1M+ requests/minute. Target this type throughput for your site speed.

2) Enable HTTP/2 protocol support.

3) Enable Brotli compression support.

4) https://www.ssllabs.com/ssltest/analyze.html?d=iwhost.com fails because site is so unstable... meaning site is unable to handle minor traffic flow of the SSL tester.

Best fix all these problems prior to running any traffic to this site.

Also best to change the 301 to a 302 else you'll eventually this will likely cause problems.

Hi,

I have changed the referrer policy to no-referrer-when-downgrade .

Can you help to check now ?

I need to use 301 redirection to ensure google carry the ranking to new domain. 302 may strip my existing ranking.

Hi,

I have checked at https://securityheaders.com/?q=https%3A%2F%2Finternet-webhosting.com .

It shows as follows :

Raw Headers
HTTP/1.1      301 Moved Permanently
Date      Mon, 30 Dec 2019 13:21:15 GMT
Server      Apache
Referrer-Policy      no-referrer-when-downgrade
Location      https://iwhost.com/
Content-Length      227
Content-Type      text/html; charset=iso-8859-1

But why my PHP still not able to detect the referrer ?

1) I have changed the referrer policy to no-referrer-when-downgrade.

This should fix your problem.

2) Can you help to check now ?

Only you can check this, via logging the environment data flowing through to your iwhost.com site.

3) I need to use 301 redirection to ensure google carry the ranking to new domain. 302 may strip my existing ranking.

To start answering this, open a new question, as this relates to how the terminal/final/canonical URL is chosen, which can be a bit complex.

Forget about question No. 3.

The main problem is about http referrer not being pass using .htaccess redirect.

My PHP still not able to detect the referrer, it still shows empty like internet-webhosting.com did not pass referrer data at all.

Any idea where else to check ?

$_SERVER['HTTP_REFERER'] is empty.

Be clear about Referrer versus query parameters.

Referrer forwarding data should pass through, independent of .htaccess rules, because the environment variable HTTP_REFERER passes based on the Referrer-Policy setting.

From a quick scan of .htaccess docs, effecting/changing pass through of HTTP_REFERER is complex, so if this variable isn't passing through, likely something else is stripping it out.

If I were debugging this, I'd start by removing any complex rules from my Apache config.

The above mentioned .htaccess file should pass through HTTP_REFERER with no changes, so requests arriving at iwhost.com (before entering your .php file) should be intact.

Likely good debug starting point will be just to dump the entire environment prior to any PHP code running.

Something like creating an env.php file with the following...

<?php

$array = $_SERVER;
ksort($array);

while (list($var,$value) = each ($array)) {
   $value = trim(preg_replace("/<address>|<\/address>/", '', $value));
   echo $var . " = " . $value . "<br />\n";
}

?>

Select allOpen in new window

Then https://iwhost.com/env.php will dump your environment as it's coming across the wire.

This will tell you if the problem relates to the domain.com or iwhost.com config, to better isolate where to start debugging.

After first redirection from internet-webhosting.com .

The env.php produce empty result.

But if refresh the data is there but without referrer.

Then the problem seemingly relates to the https://internet-webhosting.com site hosting.

Most likely something in your config if stripping out most/all the environment to enforce some security policy.

This is a common side effect of... overly aggressive tuning guides + some Distro default install configurations.

You'll likely require using ssh to get into your HTTPS config files + log files, to debug + fix this problem.

No overly complex. Just time consuming to figure out + fix.

At least now you know the exact source of the problem.

Any recomended company or maybe you that i can hire to fix this ?

I'll PM you.

Anybody else can help me on this issue ?

Just got notified to look at this. To my knowledge "Referrer-Policy" headers do not have any impact on redirects. By default, a browser will send a Referrer header when it follows a normal transition from Page A to Page B. That Referrer-Policy header simply allows you to tell the browser to NOT send along the referrer in certain circumstances. So it's a subtractive policy, not additive. It will not add a referrer header if the browser wasn't planning on adding one.

A redirect (regardless of 301 or 302) will never, by default, preserve the original Referrer. There might be some browser override options, but that's not something the server controls. In the past, if I wanted to preserve the referrer, the simplest solution was to pass it along in the redirect URL:

ORIGINAL:
Source URL: https://internet-webhosting.com
Referrer: https://www.experts-exchange.com
Redirect To URL: https://iwhost.com

ADJUSTED:
Source URL: https://internet-webhosting.com
Referrer: https://www.experts-exchange.com
Redirect To URL: https://iwhost.com?ref=https://www.experts-exchange.com

Actually, I take that back. I went back and read the RFC on Referrer-Policy and I was wrong in my initial comment.

Try changing your Referrer-Policy to "strict-origin" instead.That should allow the referer to come through.

Also, you'll need to completely restart your browser in most cases to see changes you've made since you're using a 301. Otherwise, you'll probably end up using the same cached redirect and you won't get the updated referer policy header.

Hi,

I have tried strict-origin , you can verify here at :

https://securityheaders.com/?q=https%3A%2F%2Finternet-webhosting.com

But, still iwhost unable to detect the referrer .

Tip: Remove all additional headers, then retest to verify correct information is coming across.

Then start adding additional headers, ensuring information continues to flow.

If your browser has cached the 301, then it will not be acting upon the updated referrer policy. The security headers site (which is a great one, by the way - I'm glad you run your site through it), does not cache anything so it will see the updated header while your browser's cache will not.

Before I made the recommendation, I ran a test and used Fiddler to modify the referrer policy to strict-origin for my local browser. I was able to see the correct referrer pass through to iwhost.com. If you go look at the access logs for iwhost.com for 1/2/2020 sometimes around 11 AM Mountain Time (give or take 30 minutes), you should see an entry from me with the referrer coming from this Experts Exchange page. So I'm pretty confident it works - you just need to properly reset the cache and fully restart your browser.

I would suggest using 302 redirects until you get everything working as expected, and then use 301 once you want it permanent. It will make for easier and less-confusing testing.

On a side note, I was going to reproduce the same test right now and take a screenshot to show you the referrer being passed in when using strict-origin, but the internet-webhosting.com site seems to be down.

Okay, now that internet-webhosting.com is responding, I was able to use Fiddler to capture the request and show confirmation that Referer is now indeed passing. To perform the test, I opened up a new instance of Firefox, cleared the cache, and all history, then came to this Experts Exchange question page and clicked on one of the links to internet-webhosting.com. The result:

REQUEST #1:

GET https://internet-webhosting.com/ HTTP/1.1
Host: internet-webhosting.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://www.experts-exchange.com/questions/29168151/PHP-Referrer-not-detect-if-redirect-from-another-domain-using-htaccess.htmlhtaccess.html
Connection: keep-alive
Upgrade-Insecure-Requests: 1

RESPONSE HEADERS #1:

HTTP/1.1 301 Moved Permanently
Date: Sun, 05 Jan 2020 22:27:55 GMT
Server: Apache
Referrer-Policy: strict-origin
Location: https://iwhost.com/
Content-Length: 227
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

REQUEST #2:

GET https://iwhost.com/ HTTP/1.1
Host: iwhost.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://www.experts-exchange.com/questions/29168151/PHP-Referrer-not-detect-if-redirect-from-another-domain-using-htaccess.html
Connection: keep-alive
Upgrade-Insecure-Requests: 1

So you can see that the referer has successfully been passed along now with the strict-origin policy.

Hi,

I have placed the following scripts at the header of iwhost.com .

$ref = $_SERVER['HTTP_REFERER'];
echo "$ref";

When clicking internet-webhosting.com link in the previous post, nothing is shown at iwhost.com . But when i click the iwhost.com link directly  above, the script is working.

I don't see the referrer echoed anywhere in your header right now - not even if I go straight to iwhost.com.

Perhaps you still have a cached 302 redirect when you were testing.

I only enable it when doing testing for few seconds, this is very active website.

I'm using incognito and private browsing. Remove cache also.

Can you echo the referer in an HTML comment block?

Note: Referrer-Policy: strict-origin specifically says...

1) Only return Referrer data to client initiating the request to https://internet-webhosting.com

2) The strip all Referrer header data from any 30X redirects.

3) This means any redirects to https://iwhost.com/ will have all Referrer header data stripped.

Be sure you look at your Referrer Policy carefully, if you require pass through of Referrer header data this along in all 30X requests.

As mentioned above, first entirely remove Referrer Policy, verify all's working correctly, then begin testing Referrer Policy changes.

David, where is this stated?

i have placed the script to show the referrer at the bottom of the site.

Please refer to the attached.

Let me know what referrer-policy should i changed from internet-webhosting.com so that you guys can check.

I have tried empty referrer, unsafe-url, no-referrer-when-downgrade and strict-origin all no luck.

It is working exactly fine how it is. I tested with a brand-new instance of Firefox, as well as a new instance of Chrome. I clicked on the https://internet-webhosting.com link from this page, and the result at the bottom of the page in BOTH browsers was:


However, I noticed that when I refreshed iwhost.com or went to it without clicking on a link (no referer sent), I still saw that same footer. So I think the problem that you're having is simply cache. You are testing different things but if you do not completely clear your cache and history related to both of those hosts, then the browser will simply show you the results from your first test every single time.

I notice that when click the internet-webhosting.com link from experts-exchange it will shows experts-exchange as referrer. But not when type directly internet-webhosting.com .

May i know how to make it shows as its from internet-webhosting.com instead ?

If you directly type internet-webhosting.com, then that is not supposed to generate a referrer. What you are seeing is correct. It is correct behavior for there to be no referrer if you type in an address or if you go directly to an address in any way (e.g. via browser bookmark).

It doesn't matter that internet-webhosting.com redirects to iwhost.com. That redirect is not the definition of a referrer for the browser.

If you need to redirect from internet-webhosting.com to iwhost.com when someone types in the address directly or uses a bookmark, and know that they were redirected that way, then the only way to do that is to send some kind of flag through the query string:

Location: https://iwhost.com/?r=internetwebhosting

How to modify my current .htaccess to have that query string and maintain current feature ?

My current .htaccess are as follows :

#REDIRECT TO IWHOST

RewriteCond %{HTTP_HOST} ^internet-webhosting.com [NC,OR]
RewriteCond %{HTTP_HOST} ^www.internet-webhosting.com [NC]
RewriteRule ^(.*)$ https://iwhost.com/$1 [L,R=301,NC]

The simplest approach would be to just add it after the $1:
RewriteRule ^(.*)$ https://iwhost.com/$1?r=iwh [L,R=301,NC]

But that's based on the premise that your originating URL doesn't already have a query string and you're just redirecting straight URLs. If you need to handle original query strings, it can get trickier, but give the above a try first.

The above works, but i have query string that need to maintained, eg : https://www.internet-webhosting.com/?aid=xxxx should also redirect to https://iwhost.com/?aid=xxx .

I need to cover both straight URL and with existing query string.

So far, there's been no way to test your entire 1st -> 2nd site redirect chain, so only guesses can be made.

Create some random redirect URL from 1st site -> 2nd site for testing + publish the URL.

This way a simple test can be done to make better sense of what's occurring.

Thank you all. You guys are helpful.

You're welcome!

Be great if you can post what you did to fix the problem.