Link to home
Start Free TrialLog in
Avatar of damianb123
damianb123Flag for United Kingdom of Great Britain and Northern Ireland

asked on

Block website review pages with no content from search engines

Hi,
    I have an e-commerce site which has a product review for every item, however not all items have reviews, so I have been advised by my SEO company to add META ROBOTS to stop the reviews with no content from being crawled, how can I achieve this?

Thanks in advance.
Avatar of Scott Fell
Scott Fell
Flag of United States of America image

I don't understand the thinking behind this.  What search engines look for is content.  Your content is your products and product description.  Comments can possibly hurt you if they are not moderated.  This is also something your seo company should know how to do.  It would be interesting for you to create a new question and post a link to your site and ask here on EE for SEO suggestions.

The answer to your question is
 <meta name="robots" content="noindex" />

Open in new window

And a little info https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
The SEO consultant sounds like they have a limited understanding of how Google works. If there is actually a problem with some pages (and itis not clear there is a problem), then the approach should be to improve page quality instead of trying to hide the pages from the spiders.  Bad advice from the consultant IMO.

Cd&
Avatar of damianb123

ASKER

Hi,
   Thanks for your responses....  Basically google have indexed 43,100 pages from our website which contain no valid data, so your theory of not crawling pages with no content is not accurate - I have the proof within Google's Webmaster Tools.  I can see from clicking many of these pages, that they are review pages with no reviews, however 'some' do have reviews and I want these to stay - hence my requirements to create a noindex command so only review pages with a review are crawled and the rest not!

I don't think it can be done.

Any ideas?

Cheers
If you are talking about "blank" pages, then you should be able to remove those links in webmaster tools.  You don't have to do it one at a time, you can upload a file.

You can also disavow links in webmaster tools.

What happens if reviews show up?  I would also look at why those pages are getting indexed if they don't have content?  I think a better option would be to remove any link to show reviews if there are none.  Then when a review hits, the link will be there and google will crawl it.
Our product pages do contain content i.e. the product at hand, and on the bottom have a review section.  The main page gets indexed which is fine, but also Google indexes every review section on the bottom as a seperate page... the easy solution would be to say:

disallow: www.mydomain.com/reviews

However this would black them all, when I actually want any review which has been made to be indexed.  

I hope that makes more sense....
How do you have products and reviews on the same page but google is indexing the reviews as a separate page?  Is it a frame? or a hide/unhide link?

What is the url of your site?
The reviews are on the same page as the content yes, and Google is indexing the reviews as a seperate page....  I can't disclose link though, sorry!  I can tell you it's an OSCommerce 2.2 site.

Cheers for all your help
Based on this sample, http://demo.oscommerce.com/product_info.php?cPath=1_5&products_id=27 there is a button for reviews that goes to another page http://demo.oscommerce.com/product_reviews.php?cPath=1_5&products_id=27

Therefor you have product_info.php and product_reviews.php.  It's the product_reviews.php that you don't want indexed if there is not any content.  

Since you want to do this dynamically, you will have to do a little php scripting on that template to look for reviews.  If no reviews, then add the no index to the meta tag.  You can find out how to modify the templates here.
http://library.oscommerce.com/public/sites/Library/pdf/oscom22-old.pdf about 3/4 the way down, look for the heading, (AC) (NEW) Introduce Template Layout and Modular Header Tags. From there you can see a list of pages including

catalog/product_info.php
catalog/product_reviews.php
catalog/product_reviews_info.php
catalog/product_reviews_write.php
catalog/products_new.php
catalog/reviews.php

Although it might be easy and tempting to use query/js to look for content and if no content, add to the head section, there is a good chance the search engines will ignore the js on the client side and you will need to do this server side with php.
I think the better option would be to NOT have a review button if there are no reviews and instead, have the Write A Review button linking to product_reviews_write.php  and that page should be no index.  That will solve your problems.
hi Padas,
    Can you provide more details and possibly some code on your thinking above, mainly your last post.  So are they indexing this because of the button link on the main product page?

Cheers
ASKER CERTIFIED SOLUTION
Avatar of Scott Fell
Scott Fell
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial