Link to home
Start Free TrialLog in
Avatar of Robb Hill
Robb HillFlag for United States of America

asked on

How to protect a folder from being crawled or accessed via a url....its used privately in the enterprise.

I have a wordpress site hosted on ngnix.  

We have a need to store  4 jpg images in the site so our sharepoint online page can pull these images into an email.  Ultimately we wanted  a storage place that outlook could get this and build this prescripted email.

The issue though is we some concerned that if someone got the url they could access this.

Is there a way to prevent this folder from being accessed by the outside world, crawled etc...or prevent if someone in theory could get this url?
Avatar of Robb Hill
Robb Hill
Flag of United States of America image

ASKER

this is the idea i was thinking..but not sure if this is best.

https://stackoverflow.com/questions/10236717/how-to-prevent-a-file-from-direct-url-access
The only 100% prevention is to restrict access to some content (in your case a folder with 4x images) to users which have done a successful login + now have a session established.

Any other way can always be defeated with minimal effort.
I dont think that works as these images are just being rendered on a prebuilt email from Sharepoint Online.

If you had to authenticate ...I believe this would cause the email to not show the image.
No.

In the email you place a link to an image.

Then when a visitor visits, they must authenticate for access to the directory.

How you arrange to generate the contents of the directory is an entirely different matter.

Your question was about, "How to protect a folder from being crawled or accessed via a url", rather than about how to generate images on the fly.

And... generating images on the fly is done exactly the same way... Whether access is to a directory or a script generating data/images, content restriction, via login role(s), is the only 100% sure way to control access.

Any other game played, can always be worked around.
ok If I am on my company email...outlook 365......

I have an email being created.

The email is autosent to recipients..and upon doing that it renders an image...

But because the email is an html email the image needs to be accessable.

I dont see how you authenticate against that as you propose.  Would it not have to be public ..or atleast accessable over the web for those users of the email.

With that in mind how do you setup the storage of these images...if we were using our public website as the place to store these images.

We had put on our sharepoint online directory..but due to the fact that it requires authentication on our network....it was not always working.  Perhaps we have some filter on this content..

Previously when we had sharepoint 2013 on our network..it worked just fine..but in this case it was stored differntly.
The bottom line is that if you want to embed those images in your email, then they need to be publicly accessible - that's just how it works.

You can prevent the Search Engines from indexing them with a simple robots.txt file, but that's only advisory - there's no way to force the search engines to NOT index the folder. All self-respecting Search Engines will honour the robots.txt file but it's not guaranteed. And of course if someone knows the url, then they can view the image.
what if I manually add it into a cdn zone,  that is only in the private dns. Then the images would be visible to anyone in the network,  but not the public internet.
Also I guess I am confused too...why do so many articles talk about protecting items in folders via htaccess files.
ASKER CERTIFIED SOLUTION
Avatar of Chris Stanyon
Chris Stanyon
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks to both of you for your comments!