Solved

Why does browser look for cgi-bin for url?

Posted on 2016-08-21
17
68 Views
Last Modified: 2016-08-23
Hi-

I have a website www.w2w3w2w3.com where index.html is doing a redirect in a meta tag:
<meta http-equiv="refresh" content="2;url=http://www.w2w3w2w3.com/cgi-bin/">

Now, I added an entry to my Hosts file:
127.0.0.1      www.w2w3w2w3.com

But instead of resolving to my localhost (root) folder where there is an index.html that merely displays "Hello World!", the browser instead resolves to (on my local pc) http://www.w2w3w2w3.com/cgi-bin/
i.e. in my case, to a non-existent cgi-bin folder on my local pc.

I can use various tools to confirm that the browser is in fact resolving to localhost on my pc (ping, fiddler, tracert...).

So I figure there's some cache that is responsible for this behavior.  I suspect that if I go into Chrome and clear all the caches, the browser will no longer resolve to the cgi-bin folder (which is the case if I "switch person" in Chrome to a profile that has never navigated to www.w2w3w2w3.com before, or if I use an incognito window to navigate to the url).

But before I clear all the Chrome browsing caches, I want to know exactly which cache is responsible for this behavior and whether or not I can examine a cache file to confirm the entry causing this behavior.

Any gurus out there that can help me?

Thanks--
Sam
0
Comment
Question by:SAbboushi
  • 7
  • 6
  • 4
17 Comments
 
LVL 21

Expert Comment

by:Kim Walker
ID: 41764731
But instead of resolving to my localhost (root) folder where there is an index.html that merely displays "Hello World!", the browser instead resolves to (on my local pc) http://www.w2w3w2w3.com/cgi-bin/
This is because...
I have a website www.w2w3w2w3.com where index.html is doing a redirect in a meta tag:
<meta http-equiv="refresh" content="2;url=http://www.w2w3w2w3.com/cgi-bin/">
The browser is doing exactly what you told it to do in your index.html file, redirect to your cgi-bin folder.
0
 

Author Comment

by:SAbboushi
ID: 41764763
Thanks Kim - please forgive me if I'm missing something, but your response leads me to ask whether you know what the Hosts file does?

Also, I said: which is the case if I "switch person" in Chrome to a profile that has never navigated to www.w2w3w2w3.com before, or if I use an incognito window to navigate to the url

By this I mean that it does NOT redirect in those two cases which to me demonstrates it is a cache issue.  Note that the index.html with redirect is on the website (i.e. the domain); the index.html with "Hello World!" is on localhost.
0
 
LVL 21

Expert Comment

by:Kim Walker
ID: 41764774
(which is the case if I "switch person" in Chrome to a profile that has never navigated to www.w2w3w2w3.com before, or if I use an incognito window to navigate to the url
Sorry, I didn't realize that the case you were referring to was that it no longer acknowledged the hosts file which tells the computer to ignore the domain name server and resolve the domain name to the localhost (127.0.0.1).

So now I'm not sure what the question is. Are you asking why switching the person in Chrome doesn't acknowledge the hosts file? What happens in other browsers?
0
 
LVL 21

Expert Comment

by:Kim Walker
ID: 41764775
Here's a thread on Google Chrome Help Forum. It's 5 years old, but the search function of the address bar in Chrome still applies. There's a part at the end that addresses a page that has already been cached.
0
 

Author Comment

by:SAbboushi
ID: 41764785
>> There's a part at the end that addresses a page that has already been cached.
Closing out all instances of chrome.exe didn't clear it, which seems in alignment with my suspicion re: Chrome cache(s)

>> So now I'm not sure what the question is.
But before I clear all the Chrome browsing caches, I want to know exactly which cache is responsible for this behavior and whether or not I can examine a cache file to confirm the entry causing this behavior.
0
 
LVL 21

Expert Comment

by:Kim Walker
ID: 41764795
I don't know which behavior you're talking about. Correct behavior would be to acknowledge the hosts file and redirect to the localhost. However, Chrome will apparently ignore the hosts file unless you include the http:// protocol in your address or if the page is cached.

Although there are several areas of browsing data, there is only one cache -- cached images and files.
0
 
LVL 51

Assisted Solution

by:Julian Hansen
Julian Hansen earned 150 total points
ID: 41764835
I am confused by this question. You say in your index.html you have

<meta http-equiv="refresh" content="2;url=http://www.w2w3w2w3.com/cgi-bin/">

Open in new window


That is an instruction to the browser to go to the URL http://www.w2w3w2w3.com/cgi-bin which you say it is in fact doing but you are confused as to why?

The question is - is it going there immediately or only after 2 seconds - if after 2 seconds then the behaviour is as expected - if immediately then we need to look further.

hosts has nothing to do with the cgi-bin folder - that is a http server decision - all hosts does is get you to the server - after that anything that appears after the domain name is for the server to decide what to do with.

In your subsequent posts you are saying (I think - it is not clear) that in some cases it does not browse to the cgi-bin folder?

Have you eliminated caching by doing a hard refresh (Shift + F5)?
0
 

Author Comment

by:SAbboushi
ID: 41765646
Thanks for the responses

>> I don't know which behavior you're talking about. Correct behavior would be to acknowledge the hosts file and redirect to the localhost.

Why is browser redirecting to cgi-bin folder on localhost when:
1) url in browser is www.w2w3w2w3.com
2) index.html on localhost does NOT redirect - it only displays "Hello World!"

Note: Hosts file is correctly redirecting to localhost (i.e. my pc).
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 51

Expert Comment

by:Julian Hansen
ID: 41765716
First thing first - why do you have the meta refresh tag in your code at all - that is confusing the issue. Can you take that out and lets go from there.

Remove the line
<meta http-equiv="refresh" content="2;url=http://www.w2w3w2w3.com/cgi-bin/">

Open in new window

from index.html

Then try the localhost and www.w2w3w1w3.com again.
0
 

Author Comment

by:SAbboushi
ID: 41765729
Julian - the meta tag is NOT on localhost, so unclear on why you see this to be relevant ; )
0
 
LVL 51

Expert Comment

by:Julian Hansen
ID: 41765935
Ok I understand
w2w3w2w3.com -> index.html with redirect
localhost ->index.html "Hello World" (sans redirect)

hosts -> localhost -> w2w3w2w3.com

Browser shows w2w3w2w3.com/cgi-bin - suggesting that it is going to the original website instead of resolving to the plain old "Hello World"

Am I on the same page?

have you tried running ipconfig /flushdns

What do you get if you ping the domain?
0
 

Author Comment

by:SAbboushi
ID: 41766028
>> Browser shows w2w3w2w3.com/cgi-bin - suggesting that it is going to the original website instead of resolving to the plain old "Hello World".  Am I on the same page?
Yes -

>> have you tried running ipconfig /flushdns
Yes. Note: I've dismissed this as a dns cache issue per above "By this I mean that it does NOT redirect in those two cases which to me demonstrates it is a cache issue."  (i.e. browser cache)
0
 
LVL 21

Expert Comment

by:Kim Walker
ID: 41766079
You still haven't answered my question how other browsers behave. What happens in Firefox? What happens in IE? I still believe this is a Chrome issue where it is bypassing the hosts file and searching the internet in other personalities. Did you clear the cache for the personality that is acknowledging the hosts file? Did you do a hard refresh (shift + F5) as Julian suggested?
0
 

Author Comment

by:SAbboushi
ID: 41766110
>> I still believe this is a Chrome issue where it is bypassing the hosts file and searching the internet in other personalities.
Thanks for your thoughts Kim.  My tests (above) has already demonstrated to me that Chrome is not bypassing the hosts file.  It seems that some entry in some Chrome browsing cache is responsible for updating the url (I get the same behavior if I'm not connected to the internet)

>> You still haven't answered my question how other browsers behave. What happens in Firefox? What happens in IE?
Same behavior as I reported re: different Chrome profile / incognito mode: if I visited the site before I add it to Hosts file, then url automatically changes in address bar from www.w2w3w2w3.com  to www.w2w3w2w3.com/cgi-bin when I navigate to that address.  If I had not previously visited www.w2w3w2w3.com, then "/cgi-bin" is not appended.

Clearing all the caches in Edge stopped the automatic appending of "/cgi-bin", so back to my question:

before I clear all the Chrome browsing caches, I want to know exactly which cache is responsible for this behavior and whether or not I can examine a cache file to confirm the entry causing this behavior.
0
 
LVL 21

Accepted Solution

by:
Kim Walker earned 350 total points
ID: 41766282
before I clear all the Chrome browsing caches, I want to know exactly which cache is responsible for this behavior and whether or not I can examine a cache file to confirm the entry causing this behavior.
As I said before:
Although there are several areas of browsing data, there is only one cache -- cached images and files.
You have mentioned changing Chrome profiles and what happens in Chrome. You mentioned clearing all caches in Edge. I haven't seen Edge yet but according to Microsoft's support site, there is still only one cache -- cached data and files. But you still have not told me what happens in Firefox and you haven't mentioned what happens when you do a hard refresh (shift + F5).

However, it has become clear to me, finally, that Chrome is definitely loading the cached page from the remote site and executing the meta refresh which now attempts to load from the local disk because of the hosts file. It has been difficult for me to get the big picture on this question for some reason. This should not be related to any of the other browsing data except the one and only cache. Unfortunately Chrome encrypts the cached pages so that even if you navigate to the folder they're stored in, you cannot determine which file is which nor view the contents of those files. I did see a mention of a Chrome Cache Viewer add-on when I was researching this question, though.

I can also see that a hard refresh will probably attempt to refresh the non-existing redirected page unless you can do it in the first 2 seconds before it has an opportunity to try to redirect. Do a shift+F5 immediately after entering and executing the domain name in the address bar. If that doesn't work, you will have to clear the cache unless the Cache Viewer add-on will allow you to selectively delete cached files.
0
 
LVL 51

Expert Comment

by:Julian Hansen
ID: 41766407
I have tried to replicate your scenario. Created a file with the meta refresh - pointing to a cgi-bin folder on the server. That works.
Setup a hosts entry to point domain to local host - went to the same url - it loaded the local version. Tried in all browsers with no issue.

Are you sure you are not using a proxy server?
0
 

Author Comment

by:SAbboushi
ID: 41766883
Hey Kim!  Thanks - your post helped me figure out the following:

I can view the list of cached pages in Chrome using: "chrome://cache"
CTRL/F to search for "w2w3w2w3" -> Click on the link for "http://www.w2w3w2w3.com/"

This opens up the cached page where I can see:
1) Content-Location: http://www.w2w3w2w3.com/index.html
2) The contents of index.html showing the meta redirect

Shortcut to seeing any cached page without having to search:
chrome://view-http-cache/http://www.w2w3w2w3.com/

That's what I've been looking for!

So when Chrome is pulling from the cache when I don't want it to, I can:
1) delete the cache (which I'd rather not do to remove only 1 offending entry), or
2) I can try to do a hard refresh (Thanks Julian - didn't know about SHIFT/F5), but that would only work with a delayed redirect (worked for me this time)
3) My preferred solution: Open Chrome DevTools>Network> check: Disable cache; navigate to http://www.w2w3w2w3.com/ which now displays "Hello World!"; uncheck: Disable cache.  Cached page is now replaced with the one I want. (btw: ChromeCacheView doesn't seem to have option for deleting)

And thanks for correcting my improper use of the word "cache"; I mistakenly have been saying cache when what I was referring to was "Browsing Data", and (as you've pointed out repeatedly) there's only one cache: "Cached images and files"; all those other entries under "Browsing Data" are not caches ; )

No Julian - no proxy server (except temporarily when I was using Fiddler to examine what was going on)

Thanks folks for all the time you spent on this.  Much appreciated!
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

When you start your Windows 10 PC and got an "Operating system not found" error or just saw  "Auto repair for startup". After a while, you have entered a loop for Auto repair which does not fix anything and you will be in a  panic as all your work w…
Possible fixes for Windows 7 and Windows Server 2008 updating problem. Solutions mentioned are from Microsoft themselves. I started a case with them from our Microsoft Silver Partner option to open a case and get direct support from Microsoft. If s…
The viewer will learn the basics of jQuery including how to code hide show and toggles. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery…
Google currently has a new report that is in beta and coming soon to Webmaster Tool accounts. This Micro Tutorial will highlight new features for Google Webmaster Tools.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now