asked on

Problem with home page indexing on my live server, but not on my test server...???

I have a site I just launched, but for some reason my home page isn't getting picked up by indexing correctly.

If you enter site:www.givemobiley.org into Google you'll see it has indexed our home page like this...

Index of
www.givemobiley.org/
Index of /. Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 Server at ...

Also, if I click the Facebook like button on my home page it ends up on Facebook as "Index of /" instead of using an actual page name.

What's strange, is I have the exact same versions of the site files on my local test server and it's working just fine. When I was building it I forgot to disable crawlers on my test sever, so it got indexed in Google. As of right now, if you enter site:sandbox.givemobiley.org into Google you'll see the home page was indexed just fine. I did enter a removal request, though, so if it's gone by the time you see this that's why.

Also, if I hit the Facebook like button on http://sandbox.givemobiley.org it works just fine. It's only my live site giving me issues, and it's only the home page.

My web hosting company can't give me any answers as to why this is happening. I have lots of other sites hosted with them, too, and I don't seem to be having the issue on anything but this one site. It's a dedicated virtual machine, and again, they can't seem to fix this for me.

Any information on why this might be happening would be greatly appreciated. Thanks!

James Williams

You are aware you page should start off with a heading like so:

DOCTYPE html
     PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="content-type" content="text/html; charset=UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="robots" content="index, follow, noarchive" />
    <meta name="description" content="I have a site I just launched, but for some reason my home page isn't getting picked up by indexing correctly. If you enter site:www.givemobiley.org into Google you'll see it has indexed our home..." />
    <meta name="keywords" content="apache, indexing, Apache Web Server, Internet Search Engine Optimization" />

Open in new window

Besides it taking 85 seconds to load.

the page started off with this.

	
			
	
<div id="side-tabs">
	<ul>
								<li class="clearfix"><a href="#"  class="activeTab"><img src="http://www.givemobiley.org/wp-content/uploads/2012/04/features-icon.png" alt="" width="36" height="37" /><span>Features</span></a></li>
								<li class="clearfix"><a href="#" ><img src="http://www.givemobiley.org/wp-content/uploads/2012/04/specs-icon.png" alt="" width="36" height="37" /><span>How it Works</span></a></li>
								<li class="clearfix"><a href="#" ><img src="http://www.givemobiley.org/wp-content/uploads/2012/04/reviews-icon.png" alt="" width="36" height="37" /><span>Reviews</span></a></li>
								<li class="clearfix"><a href="#" ><img src="http://www.givemobiley.org/wp-content/uploads/2012/05/usbswiper-icon.png" alt="" width="36" height="37" /><span>USBSwiper</span></a></li>
			</ul>

Open in new window

Selvol

Andrew Angell

ASKER

My server is screwed up right now and I'm waiting on a response from my managed hosting company on why. I posted a ticket to them about this, too, and I think they're got it all screwed it up trying to troubleshoot. All my sites on this server are jacked right now, and I'm not happy about it. Just got done griping to them about it before I came back here.

Please hold off for now. I'll update again once I know the server is running like it should.

Andrew Angell

ASKER

Ok, it's loading again now, so you'll see what you should see. I definitely create my files with good tags. They just had a bad version of a file up there causing a bunch of garbage.

Andrew Angell

ASKER

In fact, here's a new page with nothing but the facebook like button on it: http://www.givemobiley.org/index-test.php

The same thing is happening with that. It shows up on facebook as "Index of /".

That same exact code, though, at the bottom of most my subpages, so for example: http://www.givemobiley.org/portfolio/

That works just fine.

Andrew Angell

ASKER

And then here's the exact same file copied to another site on the same exact server and it works fine here, too: http://www.angelleye.com/index-test.php

James Williams

Yes the site looks good.
I am a bit lost now.
Besides google most likely just needing to re-index your site
I cant see a real problem.

Selvol

Andrew Angell

ASKER

Selvol,

Do you have Facebook? If so, hit the Like button on my home page and then look at your Facebook timeline. You'll see it in there as "Index of /" instead of a nice Page name like it should be.

Then if you do the same thing from my angelleye.com site you'll see it works nicely there. Of course, you can simply unlike them again to get them back off your timeline.

I noticed this first and thought it was an issue with my Facebook button or something, but then I realized Google was indexing the home page the same way Facebook sees it.

So the fact that Facebook is still seeing it that way leads me to believe Google is going to continue seeing it that way, too, until whatever the problem is gets fixed.

Andrew Angell

ASKER

My web host has been able to confirm that there's an issue, but they're struggling to help me figure it out, too.

James Williams

I liked it from your home page. www.givemobiley.org/

Andrew Angell

ASKER

So does it show up nicely in your Facebook or does it just say "Index of /"?

Andrew Angell

ASKER

Here's what it looks like for me, which is the same way Google sees it...

James Williams

Yes I got the index of same as you.

Go back to face book and Create another Button,

It should pick up the new Title.

Selvol

Andrew Angell

ASKER

It's not, though. I've done it numerous times, and the page value in the button code is dynamic, anyway. It just sends the URL of the current page, and then they pull the title, images it can find, etc. to display in Facebook...when it's working correctly.

Both Google and Facebook (and probably anything else that reads site header info) is picking up garbage from pages on the root of this site.

It's only happening with this 1 website account on this server. I can move the same exact button code to other sites and it works just fine. Also the same exact code is also on my local test server still : http://sandbox.givemobiley.org. If you hit the like button there it'll show up nicely in your Facebook like it should. Same exact code, synced with my live server.

Andrew Angell

ASKER

This is just getting weirder and weirder. My Google index seems to be better now, so that's good. Facebook, though, has got weird things going on with this website for some reason.

If I type my site URL, www.givemobiley.org into a fresh Facebook update, I still get "Index of /" on Facebook instead of the actual site name. No description, no logo pulled.

However, if I type the URL into a fresh Facebook update with capital letters like this, www.GiveMobiley.org, then it pulls the name and description just fine, but still won't pull a logo at all.

Now, if I make a comment on an existing post, and I use www.GiveMobiley.org, it pulls everything perfectly! Nice title, nice description, and the logo. Looks great.

However, if I make a comment on an existing post with www.givemobiley.org, then it just pulls the "Index of /" thing again.

This is crazy, and I'm at a loss for what to try to do to fix it.

ASKER CERTIFIED SOLUTION

Bernard Savonet

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Andrew Angell

ASKER

Interesting information. Thank you for that. I'm going to play with that and see what happens.

I guess that falls in line with what I was about to update with, too.

What I noticed is that on sites where I don't have an index page of any kind it just loads of up the directory browsing. Well, the title of this ends up being "Index of /" so I was going to come here and say that for some reason my site must not be allowing Facebook to see my index page.

But now with your info it seems that maybe this was indeed happening, and that's what Facebook cached, but now that Apache is serving up the index correctly Facebook just needs to stop using the cache and it would work.

I would think that hopefully they'll dump the cache at some point..?? Again, thanks for the info. I'll play around and come back with more info or some points before long.

Bernard Savonet

By default, Apache displays the index of the directory
.... but there is another default which says that this index must be splayed "through" the page index.htm, or html, or php etc.
This depends on the order in which these files are listed on the DirectoryIndex directive in the httpd.conf file, which can be overridden thru directives in the .htaccess file of this directory

See http://httpd.apache.org/docs/2.0/mod/mod_dir.html then http://www.htaccess-guide.com/directoryindex-uses/

Andrew Angell

ASKER

This lead me to start searching for Facebook caching issues and how to reset cache. I wound up finding another article which led me to this: https://developers.facebook.com/tools/debug

When you use that tool it apparently resets the Facebook cache for the URL being tested.

Bernard Savonet

B-) Thx for the grade and points.
And for the Facebook trick!

Bernard Savonet

B-) Thx for the grade and points.
And for the Facebook trick!