Google not indexing homepage

The last time Google indexed our homepage - www.2dplay.com was in July 2005.
We are also not coming up on the keyword - "2dplay" on google, although we have 100's of inbound links. We have a blank robots.txt in the root folder.
Can anyone help us find out what's causing this to occur? Why is our site not being indexed properly?
bigwigmediaAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

weikelbobCommented:
puzzling...
rivusglobalCommented:
Could you tell me a bit about your web server configuration and what changes you have made to your web server configuration and service providers lately? It can be the smallest thing that causes this type of thing to happen.

It appears you were running Apache/2.0.46 (Red Hat) up until sometime in December 2005, but now your running Apache/1.3.33 (Unix) with various mods. It also appears as though your IP address has also changed recently.

Let us know some of the things your code can't tell us, and I will do my best to try and help you discover the problem, and figure out how to correct it.
rivusglobalCommented:
Also... Is it just your homepage which is not being indexed?
CompTIA Network+

Prepare for the CompTIA Network+ exam by learning how to troubleshoot, configure, and manage both wired and wireless networks.

bigwigmediaAuthor Commented:
No many pages are not being indexed.

We are running 3 servers (hence 3 different IP addresses).  We implement round-robin DNS.

Let me emphasize that how home page hasn't been indexed since July 2005!!
However some pages are being spidered and get PR like http://www.2dplay.com/action-games.htm

Very strange!

If anyone helps solve the problem we will make a donation of $200 through paypal or western union as well as the points.

rivusglobalCommented:
Have you tried to use Google Sitemaps to force the indexing?
desertcitiesCommented:
The problem is in your "robots.txt" file.  You have it setup to "exclude" all visiting spiders.  

Currently your robots.txt file says:

User-agent: *
Disallow: /

The Disallow command you have now tells all visiting spiders "not" to index "any" of your web pages.

Try changing your robots.txt file to something like this, unless of course you want to exclude some directories:

# Robots.txt file created by Mark Bishop
# For domain: http://www.2dayplay.com

# All robots will spider the domain
User-agent: *
Disallow:

You can read more about the robots.txt file here: <http://www.seoconsultants.com/robots-text-file/>

Also to jump start things with Google, do make a sitemap and upload it to Google.  You can read more about it here: <https://www.google.com/webmasters/sitemaps/docs/en/protocol.html>

There are other scripts out there to allow you to quickly make a sitemap for Google.  Some free, some for a few dollars.  

Good Luck!

Mark

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
desertcitiesCommented:
OOPS, apologies.

I was looking at the wrong website when I commented above.

I found no robots.txt file for 2dplay.com.  I will investigate it further.

Mark
bigwigmediaAuthor Commented:
Here is our robots text file:

www.2dplay.com/robots.txt
rivusglobalCommented:
As far as I can tell, the above robots.txt is not visible, or empty.
desertcitiesCommented:
"Here is our robots text file:"

www.2dplay.com/robots.txt

I'm not seeing anything.  

If you don't have any robots.txt file, then all spiders are welcome to index any and all pages.

However, if you do have a robots.txt file, place it in your root directory and it must also include (1) Disallow field in the robots.txt file.  It can be empty, but should exist.  

User-agent: *
Disallow:

While Meta Tags are no longer considered important, some say even necessary, I did notice you used the word "games" several times and to some engines this can still be considered spamming.  Try to limit your keywords to 3.  

On the other hand, you do have some pages indexed (cached) just a few days ago using the same number of keywords in your other tags.  

I'm not sure if the use of your iFrame advertising script is not allowing the spiders to crawl your other links and content pages, but it's possible.  

But first get the robots.txt file in the right location and that it's worded correctly and you have (1) Disallow (empty if you want all things crawled) or not have a robots.txt at all.  And do the Google Sitemap asap and uploaded to Google.

Mark






bigwigmediaAuthor Commented:
Can you see the robots.txt file now? We've included what you said we should.
rivusglobalCommented:
I can see it now bigwigmedia... Looks alright. Did you implement the Google Sitemaps?
bigwigmediaAuthor Commented:
We have now uploaded the sitemap - it may take a few hours to process. Please check back then. Thanks!
desertcitiesCommented:
Super.

A nice 2006 present!

Mark
rivusglobalCommented:
Indeed... Looking forward to hearing your results. Happy New Year!
bigwigmediaAuthor Commented:
Hi, The sitemap has been uploaded to google - the status has come back OK. But it still look as though google still has the homepage as an old page. Any more ideas?
Happy new year guys/gals!
desertcitiesCommented:
Google's Googlebot can visit a website quite often.  I see it in visiting many sites at least 1 time a week.  Their cache though can sometimes stick around for months unless you have new, or updated, content.

So if any of your web pages have been updated, or new pages added, just sit-back and wait to see if Google updates your site contents.  Also, if you can check your website's raw log files that should show you what visiting spiders, including Googlebot, are showing-up at your site for spidering.

Your Google sitemap should help a lot too.

Mark
rivusglobalCommented:
An additional suggestion would be to compare your homepage against the Google Webmaster Guidelines to make sure everything checks out.

http://www.google.ca/intl/en/webmasters/guidelines.html, let us know if you have any questions about individual items.

You could also create an RSS feed of your site. This would allow you to quickly syndicate your content and keywords shortly after it is updated.

In your page code, I see you using item numbers in your Anchor text.

Example: <li><h2><a href="asteroid-field/asteroid-field-info.htm">4. Asteroid Field</a></h2></li>

It would be more productive to use code such as,

<li><h2>4. <a href="asteroid-field/asteroid-field-info.htm">Asteroid Field</a></h2></li>

to ensure you are capturing the keywords.

A couple additional suggestions anyhow. Let us know where your at with these.

Owen
rivusglobalCommented:
btw... good to see your Sitemaps status has come back OK. I may take some time, but this will definitely help to increase your coverage.
duzCommented:
bigwigmedia -

>If anyone helps solve the problem we will make a donation of $200 through paypal

< hence duz makes a brief appearance :) >

I think the problem is probably inadvertent hijacking caused by the meta-refresh on virtualtoy.nm.ru being treated as a 302 redirect. If you have control of this domain put 'redirect 301 / http://www.2dplay.com/' into the .htaccess and be patient. If you are in a hurry get virtualtoy.nm.ru removed from the index with http://services.google.com:8882/urlconsole/controller

< the $200 will be donated to charity should this solve the problem >

- duz
desertcitiesCommented:
It's really nice to meet you, Duz!

I feel honored you dropped by to help. I read so much about you, and your superior help.

Wishing you and everyone on this post a super 2006!

Mark
rivusglobalCommented:
Nice return Duz... I trust the 301 Moved will fix this problem.

I would also remove your URL from the Google index to speed up the results as suggested.

Happy New Year Duz!

Let us know the results bigwig.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Internet Marketing

From novice to tech pro — start learning today.