IIS 10 - URL Rewrite rule for stopping Image Hotlinking - not working properly.

I have been messing around with this for the last 48 hours and cannot, for the life of me, figure out why it is not working as it is supposed to work.
I've looked at a video showing how to set it up and how it works, and I've mimicked what was done in the video, and nothing, still cannot get it to work.

This is what happens.
Using this code below will work; as you notice, it has the pattern="Pattern: "
However, the leading site that hosts the images has its images replaced with the stop-hotlinking.png image.
So, in theory, it works, but the site has no access to its images to display.
If I remove (pattern="Pattern: "), it does not work.

To sum it up.
The below code will display the "stop-hotlinking.png" image on all outside sites, as well as the hosting site.

         <rewrite>
            <rules>
                <rule name="STOP-Hot-Linking" enabled="true" stopProcessing="true">
                    <match url=".*\.(gif|jpg|png)$" />
                    <conditions>
                      <add input="{HTTPS}" pattern="on" />
                      <add input="{HTTP_REFERER}" pattern="Pattern: ^$" negate="true"/>
                      <add input="{HTTP_REFERER}" pattern="Pattern: ^https?://(www\.)?domain\.com/.*$" negate="true"/>
                    </conditions>
                    <action type="Rewrite" url="/graph/stop-hotlinking.png" />
                </rule>
            </rules>
        </rewrite>

Select allOpen in new window

I even tried it with the pattern as this, and it will display the stop-hotlinking.png on all sites including the hosting site.

<add input="{HTTP_REFERER}" pattern="^http://(.*\.)?domain\.com/.*$" negate="true"/>

Select allOpen in new window

Okay so let's convert this stuff into English.

First of all, <conditions> will default to requiring that EVERY condition inside it be matched. So:

<conditions>
  A
Means "A and B and C" must all be met in order to perform the rewrite.

So your first condition:

<add input="{HTTPS}" pattern="on" />

Select allOpen in new window

Just means "The request for the image is using HTTPS" so if the request was for http://domain.com/image.jpg, then the condition would not be met and the rewrite wouldn't be performed.

Personally, I'd just take out this condition. It's restrictive with no benefit.

The second condition:

<add input="{HTTP_REFERER}" pattern="Pattern: ^$" negate="true"/>

Select allOpen in new window

Is a bit weird. You have the word "Pattern: " in the actual pattern value. Maybe this is an IIS thing I'm not aware of, but I think that's a mistake. Basically, it will never match because of that "Pattern: " at the  beginning. What's funny is that it's kind of okay because you have negate="true" at the end, which says, "this condition is successfully met if the match fails." So this is kind of a useless rule because the pattern will never match but the negate says, "aaaand that's okay."

Ultimately you probably want to block requests that have no referrer at all, which is -probably- what this condition was trying to achieve but it just didn't do it correctly. Overall, you could probably take this condition out to simplify things.

If you take out the top two conditions, that should leave you with just the third condition:

<add input="{HTTP_REFERER}" pattern="Pattern: ^https?://(www\.)?domain\.com/.*$" negate="true"/>

Select allOpen in new window

<add input="{HTTP_REFERER}" pattern="^https?://(www\.)?domain\.com/.*$" negate="true"/>

Select allOpen in new window

I haven't tested that out but I think that should be correct, assuming my phone didn't muck up anything. 

The following code will work.
But it also shows the stop-hotlinking.png on the hosting website as well.

<rule name="Prevent Image Hotlinking" enabled="true" stopProcessing="true">
<match url=".*\.(jpg|jpeg|png|gif|bmp)$" />
<conditions>
    <add input="{HTTP_REFERER}" pattern="^https://domain.com/.*$" />
</conditions>
<action type="Rewrite" url="/graph/stop-hotlinking.png" logRewrittenUrl="true" />
</rule>

Select allOpen in new window

I need to show the images on the domain the images belong to.
And stop the images from showing on other websites.

I found and tried this as well, Blocking Image Hotlinking, Leeching and Evil Sploggers with IIS Url Rewrite
However, I was unsuccessful at getting it to work.
My version of what is provided in the linked article above.
Keeping in mind, I tried to get it to work with code identical to what is in the article, but with my domain(s) in place, it still would not work.

      <rule name="Blacklist block" stopProcessing="true" enabled="true">
          <match url="(?:jpg|jpeg|png|gif|bmp)$" />
          <conditions>
			  <add input="{HTTP_REFERER}" pattern="^https://www.domain.com/.*$" />
              <add input="{DomainsBlackList:{C:1}}" pattern="^block$" />
          </conditions>
          <action type="Rewrite" url="https://www.domain.com/graph/stop-hotlinking.png" logRewrittenUrl="true"/>
      </rule>
    </rules>
    <rewriteMaps>
              <rewriteMap name="DomainsBlackList" defaultValue="allow">
                  <add key="www.hotlinking-BAD.com" value="block" />
		  <add key="www.whitelist-GOOD.com" value="allow" />
              </rewriteMap>
    </rewriteMaps>

Select allOpen in new window

Hi Wayne,

Just for now, let's keep working with this snippet from your most recent comment:

<rule name="Prevent Image Hotlinking" enabled="true" stopProcessing="true">
<match url=".*\.(jpg|jpeg|png|gif|bmp)$" />
<conditions>
    <add input="{HTTP_REFERER}" pattern="^https://domain.com/.*$" />
</conditions>
<action type="Rewrite" url="/graph/stop-hotlinking.png" logRewrittenUrl="true" />
</rule>

Select allOpen in new window

I feel that's a pretty close starting point to what you want, and we can work from there.

Currently, that rule says, "If the visitor is trying to access an image from a page whose URL starts with https://domain.com/, then rewrite the image URL to the stop-hotlinking.png"

So there's a couple issues:

1. Most importantly, you need the negate="true" on that <add input> condition because you want to ALLOW image requests from that domain, not BLOCK them. So by adding negate="true" you're saying when the referrer is NOT this domain...

2. Your pattern:

^https://domain.com/.*$

Select allOpen in new window

...doesn't check for subdomains nor situations where you're calling it from regular http instead of https.

If you add a ? after a single character in a pattern (technically speaking, it's a "regular expression"), it says the preceding character or group is optional, so:

https?:

Select allOpen in new window

...will match http: or https:

You can use ( ) to group stuff together, so:

(www\.)?domain

Select allOpen in new window

...will match www.domain or domain.

The ^ at the beginning just says, "Start matching at the beginning of the value". So this:

pattern="foo"

Select allOpen in new window

...would match a value like "hello foo" and "foo bar". But this:

pattern="^foo"

Select allOpen in new window

...would only match "foo bar" because "foo" is at the beginning.

The $ is the inverse of that, indicating the end of the value. So this:

pattern="foo$"

Select allOpen in new window

...would only match "hello foo" because "foo" is at the very end. And this:

pattern = "^foo$"

Select allOpen in new window

...would only match a value that was EXACTLY "foo".

And finally . matches every character, while * says "zero or more instances of the preceding character" so .* means "anything and everything". So:

pattern="^https?://(www\.)?domain\.com/.*$"

Will match all of these URLs:
http://domain.com/
http://www.domain.com/foo/bar/licious?and=some
https://domain.com/page.aspx
https://www.domain.com/subfolder/

And the negate="true" will make this condition match every URL that does NOT match that pattern, so it will match:
http://google.com
https://blahblah.org
...etc...

So this condition that I had provided in my previous comment -should- be accurate if you copy and paste it (and change the domain.com part (and subdomain, if appropriate) to match your situation:

<add input="{HTTP_REFERER}" pattern="^https?://(www\.)?domain\.com/.*$" negate="true"/>

Select allOpen in new window

All that said, it should be helpful for you to actually see the referrer, so when testing, use your developer tools (hit F12 in your browser), go to the Network tab, and then load up your web page where you're doing the tests. Then click on the row in the network tab that tries to load an image, and you can scroll down to the Request Headers section, and see the Referer value that is being sent by the browser (this will be the value being matched by HTTP_REFERER):

Using what you have provided has put me back in the same place as before.
All domains are loading the stop image, including the hosting site.

The header information for each referer points to the domain I am loading the page.
So,
(This would be the image hosting domain)

authority: DomainOne (This is the same across all of them)
DomainOne is showing
referer: DomainOne/test.asp  (This is the only one that shows the test.asp page)

authority: DomainOne (This is the same across all of them)
DomainTwo is showing
referer: DomainTwo

authority: DomainOne (This is the same across all of them)
DomainThree is showing
referer: DomainThree

Why would this be affecting the Image Hosting domain?
Could there be something else that is causing it?

UPDATE
I removed nearly everything from the web.config file and tried it, and it posted the image to every domain, including the hosting domain.

I think some important piece of information is getting lost in all the abstraction, so let's start by confirming that you have three different domains that should be allowed to load images.

The only domains that matter for the sake of the rule are the ones that are in the Referer headers (so if the image-hosting domain doesn't have any web pages itself that load images, then you don't have to worry about that domain).

Anyway, let's say that you have these three domains:
images.foobar.com
www.foobar.com
subsite.foobar.com

Let's say you go to https://www.foobar.com/test.asp, and it has an IMG tag like:
<img src="https://images.foobar.com/photo.jpg">
...and maybe it has an iframe that points to https://subsite.foobar.com/subpage.asp, and that subpage.asp page ALSO has an IMG tag loading the same photo.jpg.

In that scenario, your condition would only need to account for https://www.foobar.com and https://subsite.foobar.com. You don't need to account for images.foobar.com in your condition because it's just the destination - not the referer.

So the next big question is - are all three domains on the same primary domain (e.g. foobar.com) or is it like:
www.foobar.com
subsite.anothersite.com

If everything's on the same domain, you could change the primary pattern to:

^https?://[^/]*\.?foobar\.com/.*

Select allOpen in new window

...which would match foobar.com and every subdomain of foobar.com. (And again, you would have negate="true")

If there are different domains like foobar.com and anothersite.com, then you'll need to have individual rules or multiple negated conditions.

The hosting site DOES host the images to the pages to allow people to see them.
So, every time I enable the rule, all images go to the STOP image.
So, I quickly set it and then disabled it.

Here are the three test sites. (The RULE is disabled at the moment)
If you refresh the page(s), you can cycle through the live servers, as this is on a farm of 3 web servers, there is 5, but two are down)
https://www.cffpics.com/test.asp (This is the image hosting site)
https://www.cffspotlight.com/test.asp
https://www.cffcs.com/test.asp

Here is the code for the rule with the exact domain in place.

     <rewrite>
        <rules>
<rule name="Prevent Image Hotlinking" enabled="true" stopProcessing="true">
<match url=".*\.(jpg|jpeg|png|gif|bmp)$" />
<conditions>
	<add input="{HTTP_REFERER}" pattern="^https?://(www\.)?cffpics\.com/.*$" negate="true"/>
</conditions>
<action type="Rewrite" url="/graph/stop-hotlinking.png" logRewrittenUrl="true" />
</rule>
</rules>
    </rewrite>

Select allOpen in new window

Now, this is what the plan is.
Several of my domains get their images from cffpics.com
So, right now. I am just testing to make it work by blocking cffcs and cffspotilight, BUT having images still load in cffpics.
Once we can get this rule to work as needed, I want to Whitelist all allowed domains.
(The two listed domains above and Google, Facebook, Twitter, etc.)

I hope this will help you better.

Ah, ok. Try using this input instead:

<add input="{HTTP_REFERER}" pattern="^https?://(www\.)?(?:cffpics\.com|cffspotlight\.com|cffcs\.com)/.*$" negate="true"/>

Select allOpen in new window

What exactly is the code supposed to do? Allow?
If so, I added another domain page, to check, and it is also seeing the image, and not seeing the STOP image.
So far, does not seem as if it is working.

Domain which is not listed.
https://www.cffkb.com/test.asp

<rule name="Prevent Image Hotlinking" enabled="true" stopProcessing="true">
<match url=".*\.(jpg|jpeg|png|gif|bmp)$" />
<conditions>
	<add input="{HTTP_REFERER}" pattern="^https?://(www\.)?(?:cffpics\.com|cffspotlight\.com|cffcs\.com)/.*$" negate="true"/>
</conditions>
<action type="Rewrite" url="/graph/stop-hotlinking.png" logRewrittenUrl="true" />
</rule>

Select allOpen in new window

Oh and you asked:
> What exactly is the code supposed to do? Allow?

So the pattern is pretty much the same as before except it should allow the first 3 domains you mentioned.

The regular expression (?:foo|bar) means "foo" OR "bar", so this:

(?:cffpics\.com|cffspotlight\.com|cffcs\.com)

Select allOpen in new window

...means "cffpics.com" OR "cffspotlight.com" OR "cffcs.com"

Regular expressions can start becoming a little hairy to read if you're new to them. It's definitely something that gets easier with some practice. If you want the whole tutorial on them, there's a great site here:

Regular Expression Tutorial - Learn How to Use Regular Expressions (regular-expressions.info) 

There's a lot of content out there for them, so I'm trying to stick with mostly-simple patterns.

Sadly, it is not working.
I placed the meta tag in each file and added in the web.config code, and it is the only RULE in the web.config, and I tried refreshing with tools opened so no caching, and then open an incognito, and still the cffkb shows the image, and it does not have the meta tag in it.

The script is running on the site if you want to test them out on your end.
As it sits right now, it is not working.

I tried in three different browsers.
Chrome - Shows all
IE - Shows all
Edge - Blocks all, except when I view images on the cffpics site, I can see them. On the test page, it is blocked. Weird.

Okay - give me a moment.

First test with Edge looks like it's working:

Second test with Chrome gives me the same results as Edge (all 3 "good" domains showed the image while my localhost copy got blocked).

Third test with Firefox initially looked like it was going to fail but it turned out that FF had cached the good image from the first test hit from the cffspotslights.com domain, and was serving up the real image from disk cache. Once I went into the developer tools and disabled cache, I saw the correct results - the real image showed up on all 3 domains and I got blocked on the localhost.

Fourth test with IE11 is -technically- working. However, IE apparently doesn't like your particular STOP image:




I'm going to guess it's because the original file requested was .jpg, but the image returned is a PNG, so even though the right content-type is coming back, IE is probably just flustered. Or maybe there's something about the image content it doesn't like.

However, all 3 domains showed the real image correctly.

I also did a quick test from one of my other real domains (instead of localhost) and it also failed appropriately (I saw the STOP image).

I just went back for another re-test (I think I had caching turned on for the cffkb test in Chrome) and I'm getting the STOP images for everything now, so I'm assuming you maybe changed something?

I wonder if it is me being inside my network that is causing me this issue.
A few minutes ago, I was thinking that I might want to use the hotspot on my cellphone to my other laptop and check it out on there.

Thanks for all the testing.
It would be awesome if this is actually working, but just not seen properly by me.

Looking back at your test, it seems like cffspotlight is the one that is showing the STOP image.
cffkb should be showing it, while the rest should be good.

It seems to be working.
I did a google search and found where someone had shared one of the images on a forum.
www.kissfaq.com
I looked through the forum posting, and all images from other sites were visible. However, when I saw the one from cffpics, it was not. Instead, when I clicked on the link to the image, it gave me the STOP image.
So, it looks as if it is working.

I need to add the following to the list.
Google, facebook, twitter,

Not working.
All images on Facebook are showing the STOP image.
And when I go to the image page, I get the STOP image as well.

https://www.cffpics.com/graph/CFF-Pics-Logo.gif

So, I am going to disable it until it can be fixed to work.

Code with Facebook and others added in.

   <add input="{HTTP_REFERER}" pattern="^https?://(www\.)?(?:cffpics\.com|cffspotlight\.com|cffcs\.com|google\.com|facebook\.com|twitter\.com)/.*$" negate="true"/>

Select allOpen in new window

> Looking back at your test, it seems like cffspotlight is the one that is showing the STOP image.

I had copied the html from one of the pages (might have been cffspotlight) so the text at the top might say one thing but I included the address bar in all the screenshots so you could see the testing urls along with the results. I didn't want to change anything about the html in order to not taint the test results. The one that shows the stop image is the localhost page.

Can you share the Facebook url where you're seeing the stop images?

> And when I go to the image page, I get the STOP image as well.

If you go -directly- to the image then you will definitely get the stop image because you're not sending a referrer in that scenario, so it would be treated the same as if someone was using a bot to download the image from your site.

A referrer header is only sent when you're viewing the image from a web page or if you've clicked on a link from a web page to go to the image.

Also services like FB and Twitter will often use other kinds of subdomains (which are not always easy to see without looking at the raw headers). And in some cases, FB won't directly link to the image but they'll use an intermediate bot to do it

OK.
So, I messed around with the Facebook Scrape Tool
With several different image pages from cffpics and everything worked.
So, I placed another image URL to the cffkb test page, and it showed the STOP image to where the other pages worked.
THEN, I added the cffkb to the list in web.config, and after about a minute, I refreshed the page and wa-la. Everything works.

There is enough information on this thread that anyone who has an issue will come in and get it up and running in no time flat.

Thank you so much for the time you have taken to get this working.
I would have never caught on to the meta tag information for the referral.

Thank you, gr8gonzo. You genuinely are Great, my friend.
Keep it up.

Wayne

Awesome news! Glad you got it working, Wayne! Thanks for sticking with it - have a good rest of the weekend.

You as well. I learned a lot, and this is going to work across multiple sites.
Take care.