Solved

Block website review pages with no content from search engines

Posted on 2013-06-15
11
346 Views
Last Modified: 2013-08-05
Hi,
    I have an e-commerce site which has a product review for every item, however not all items have reviews, so I have been advised by my SEO company to add META ROBOTS to stop the reviews with no content from being crawled, how can I achieve this?

Thanks in advance.
0
Comment
Question by:damianb123
  • 6
  • 4
11 Comments
 
LVL 52

Expert Comment

by:Scott Fell, EE MVE
ID: 39251359
I don't understand the thinking behind this.  What search engines look for is content.  Your content is your products and product description.  Comments can possibly hurt you if they are not moderated.  This is also something your seo company should know how to do.  It would be interesting for you to create a new question and post a link to your site and ask here on EE for SEO suggestions.

The answer to your question is
 <meta name="robots" content="noindex" />

Open in new window

And a little info https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
0
 
LVL 53

Expert Comment

by:COBOLdinosaur
ID: 39251396
The SEO consultant sounds like they have a limited understanding of how Google works. If there is actually a problem with some pages (and itis not clear there is a problem), then the approach should be to improve page quality instead of trying to hide the pages from the spiders.  Bad advice from the consultant IMO.

Cd&
0
 

Author Comment

by:damianb123
ID: 39253491
Hi,
   Thanks for your responses....  Basically google have indexed 43,100 pages from our website which contain no valid data, so your theory of not crawling pages with no content is not accurate - I have the proof within Google's Webmaster Tools.  I can see from clicking many of these pages, that they are review pages with no reviews, however 'some' do have reviews and I want these to stay - hence my requirements to create a noindex command so only review pages with a review are crawled and the rest not!

I don't think it can be done.

Any ideas?

Cheers
0
 
LVL 52

Expert Comment

by:Scott Fell, EE MVE
ID: 39253578
If you are talking about "blank" pages, then you should be able to remove those links in webmaster tools.  You don't have to do it one at a time, you can upload a file.

You can also disavow links in webmaster tools.

What happens if reviews show up?  I would also look at why those pages are getting indexed if they don't have content?  I think a better option would be to remove any link to show reviews if there are none.  Then when a review hits, the link will be there and google will crawl it.
0
 

Author Comment

by:damianb123
ID: 39253594
Our product pages do contain content i.e. the product at hand, and on the bottom have a review section.  The main page gets indexed which is fine, but also Google indexes every review section on the bottom as a seperate page... the easy solution would be to say:

disallow: www.mydomain.com/reviews

However this would black them all, when I actually want any review which has been made to be indexed.  

I hope that makes more sense....
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 52

Expert Comment

by:Scott Fell, EE MVE
ID: 39253604
How do you have products and reviews on the same page but google is indexing the reviews as a separate page?  Is it a frame? or a hide/unhide link?

What is the url of your site?
0
 

Author Comment

by:damianb123
ID: 39253614
The reviews are on the same page as the content yes, and Google is indexing the reviews as a seperate page....  I can't disclose link though, sorry!  I can tell you it's an OSCommerce 2.2 site.

Cheers for all your help
0
 
LVL 52

Expert Comment

by:Scott Fell, EE MVE
ID: 39253723
Based on this sample, http://demo.oscommerce.com/product_info.php?cPath=1_5&products_id=27 there is a button for reviews that goes to another page http://demo.oscommerce.com/product_reviews.php?cPath=1_5&products_id=27

Therefor you have product_info.php and product_reviews.php.  It's the product_reviews.php that you don't want indexed if there is not any content.  

Since you want to do this dynamically, you will have to do a little php scripting on that template to look for reviews.  If no reviews, then add the no index to the meta tag.  You can find out how to modify the templates here.
http://library.oscommerce.com/public/sites/Library/pdf/oscom22-old.pdf about 3/4 the way down, look for the heading, (AC) (NEW) Introduce Template Layout and Modular Header Tags. From there you can see a list of pages including

catalog/product_info.php
catalog/product_reviews.php
catalog/product_reviews_info.php
catalog/product_reviews_write.php
catalog/products_new.php
catalog/reviews.php

Although it might be easy and tempting to use query/js to look for content and if no content, add to the head section, there is a good chance the search engines will ignore the js on the client side and you will need to do this server side with php.
0
 
LVL 52

Expert Comment

by:Scott Fell, EE MVE
ID: 39253726
I think the better option would be to NOT have a review button if there are no reviews and instead, have the Write A Review button linking to product_reviews_write.php  and that page should be no index.  That will solve your problems.
0
 

Author Comment

by:damianb123
ID: 39253780
hi Padas,
    Can you provide more details and possibly some code on your thinking above, mainly your last post.  So are they indexing this because of the button link on the main product page?

Cheers
0
 
LVL 52

Accepted Solution

by:
Scott Fell,  EE MVE earned 500 total points
ID: 39253824
It is indexing because there is probably a button that links to one of the reviews pages.  Just remove the link/button when there are no reviews.  You will then loose the "Write A Review" button so just replace the view review button/link with the write a review link.

Otherwise, you would just add the no index as a meta tag just for the product without a review.  But then the review may take some time to show up.  

I think your original question is already answered here

Now we are getting into something else.  If you want to go down the new route, close out this question and start a new one.  However, make sure you have a good example of your database and code.  Make it easy for us to help you.  This is a great resource on what is needed to create a Short, Self Contained, Correct (Compilable), Example http://sscce.org/

In short, you will test for a product review by the id in the url. If there are no reviews, then show the Write Review button, otherwise, show the View Review button.  

Include your data table layout only for what is needed, and simplified page of code. http://sscce.org/
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Building a website can seem like a daunting task to the uninitiated but it really only requires knowledge of two basic languages: HTML and CSS.
Is your Office 365 signature not working the way you want it to? Are signature updates taking up too much of your time? Let's run through the most common problems that an IT administrator can encounter when dealing with Office 365 email signatures.
This tutorial demonstrates how to identify and create boundary or building outlines in Google Maps. In this example, I outline the boundaries of an enclosed skatepark within a community park.  Login to your Google Account, then  Google for "Google M…
The viewer will learn how to dynamically set the form action using jQuery.

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now