Progress made optimizing site for Yahoo but not Google

Michael -

Some things with Google can take a long time before you see the results.  Think months not days :)  

The first thing is that Google does not yet seem to be aware of all the pages in your site. You need to check the server logs to see if Google is spidering your site without any problems. A couple of the page urls have 'spaces' in them which can confuse bots and browsers, change these to 'hyphens'.  Also your meta keyword tag looks to have over 1000+ characters in it which is spam territory. Either remove it altogether (it is practically useless) or use just a few words. I think when I was looking you had the same title and metas on some pages. Remember Google does not index websites it indexes webpages so each page should be optimized independently for your target keywords.  You might like to consider externalizing your css to increase the text to code ratio on the page.

You need many more backlinks for your site. Make sure you have an entry in Dmoz which will automatically get you and entry in the Google directory. Be sure to read this http://dmoz.org/add.html carefully first because they are ruthless if the rules are not followed. Look for specialist directories to list your site in and look for sites that might want to link to yours.

- duz

humeniuk -

>You can also look through <url of large site deleted> - it may give you some insight into where Google and Yahoo differ.

Let's try to keep the standards up at this end of EE by answering the questions directly and not sending questioners on surfing trips.  Particularly if you are not sure if the answer is even going to be there!

Nice to have you on board recently in this channel anyway :)

- duz

Hi duz,

Thanks for the greeting and the constructive criticism.  Both are always welcome.

My inclusion of the links was meant to be supplemental to the answer I gave and not to be taken as the answer itself.  To quote a wise fellow in another thread: "There is no magic wand and you are going to have to read :)".
That is good advice in virtually any of the SEO-related questions asked here because it deflates the magic bullet myth about SEO.  My intent in providing that links was to offer a couple of  information resource that would allow Michael to look further into areas relevant to his question.

Nevertheless, having given my justification, I will take your counsel to heart.  I have only been a regular here for a couple of months and if some part of my answer is considered 'bad form', I will be more careful in the future.  I would far prefer to make a contribution here than to take something away.

Apologies to Michael for going off-topic in his thread.

Thanks for the advice.
I have taken the space out of one of the file names.
Also, have reduced the meta tag keywords on all the pages.
I will (eventually) externalize the CSS.

What did you mean by "check the server logs to see if Google is spidering your site without any problems"? How do I check the server logs?

Also, I know I need more links coming in. Is it also useful to have more links from one page to another inside the web site i.e. in addition to my navigation banner at the top of each page?

As usual, thank you both for the excellent, timely, invaluable feedback.

Michael

Michael -

>How do I check the server logs?

You need to contact support at your hosting service provider because the process varies. Maybe you can do it through the control panel they provide?  In any event you will find the logfiles as downloadable files when you ftp your site.  They are text files and you can eyeball them or create the most complex cross tabulations with logfile analysis software. It's a whole new world inside a log file!

>Is it also useful to have more links from one page to another inside the web site.

Not on a site of your size.  Just design the navigation for the users.

- duz

I have now reviewed the server logs - very interesting.
Have made some mods to the site in response to their Failure Report.

Question: one line item on the Failure Report refers to "/robots.txt".
I am guessing that this failure item is related to the following meta tag in my site:
<meta name="robots" content="index,follow">

Have I got the syntax wrong in this line? (I borrowed this line from something I saw in an EE question/answer)

Michael

michael -

>I am guessing that this failure item is related to the following meta tag in my site...

No it isn't, it is referring to the robots.txt file that most search engine robots look at/for first. The robots.txt file is an exclusion file and as far as there is a standard it is described here http://www.robotstxt.org/wc/norobots.html

It can be used to stop specific crawlers or specific parts of your site from being crawled. I suggest you have one even if you have no bot to exclude in which case it will have two lines and look like this:

User-agent: *
Disallow:

Upload it as a text file in root /robots.txt

When it is there check it with http://www.sxw.org.uk/computing/robots/check.html

>Have I got the syntax wrong in this line?
<meta name="robots" content="index,follow">

That's fine but you don't actually need it because again it is the default.

- duz

Thank you.
I added the robots.txt file and tested it.
Although there were no errors, I did get this warning:

Warning at line number 2:
Disallow:
Paths should be absolute. A number of robots require that the path in a Disallow directive should be absolute. For example with the URL http://example.org/blah the Disallow line should read Disallow: /blah

Are they suggesting that I use the /filename format?

Or should I leave the line as it currently is i.e. "Disallow: "?

If it makes no difference, I'll leave it as it is.

Michael