Partial page search engine indexing

Is there some code that can be used to hide part of a web page from indexing. (i.e. to hide flash content that has layers which seems to cause problems with google's indexing)
Who is Participating?
AndyBeardConnect With a Mentor Commented:
Google has to crawl a page to see any meta or x-robots instructions - the documents you refer to say Google doesn't crawl, so your suggestion to use meta robots in combination with robots.txt is in error.

Any use of robots.txt to block content is still creating a dangling node/page that accumulates PageRank and can show in results

This article predates the supposed changes with the handling on nofollow links, we don't know what is happening with the "reset vector" but the same is true with pages blocked by robots.txt - the juice goes into some kind of reset vector, possibly being handled in the same way.

As Sebastian stated in the comments of your referred post

hence the safe way to keep stuff out of search engine indexes is:

robots.txt: Allow: /path or Disallow:

Per URI in /path: “noindex” meta elements / x-robots-tags
You have to let the search engines in to read the page in order to then tell them not to index, use a technique such as noindex  in robots.txt which is only an unofficial Google option, or use a technique such as I suggested with javascript.

The best of those options with flash is in my opinion the javascript route, as long as any navigation elements are also present with text links.
No there isn't.

You can either hide the whole page or none of it - you can not hide part of a page.

Google has no issue with flash files, and has recently announced plans to index the text with in them

Use robots.txt and <META NAME="ROBOTS" CONTENT="NOINDEX"> robots.txt alone will not stop Google indexing the page, it wills till crawl and display a link to the pages in the search results.
The new generation of project management tools

With’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

Tony McCreathTechnical SEO ConsultantCommented:
You could move the bit you want hidden into a sperate file and include it via an iframe. Use the meta tag above to ensure the contents of the iframe are ignored.
Google doesn't crawl pages blocked with robots.txt which means that you can't combine it with meta noindex as Google won't see the noindex.

The best solution currently is to render the flash with external javascript such as using Jquery, and to block the javascript using robot.txt
That will work for all major engines

If you only care about Google, you might try their robots.txt noindex directive, thogh I believe this is still a little unofficial, or serving all flash content with X-Robots noindex in the header.

Noindex: /
From Google help documents

"While Google won't crawl or index the content of pages blocked by robots.txt, we may still index the URLs if we find them on other pages on the web"

If you only use the robots.txt and do not also include the rpbots meta tag with content=noindex you could still see your supposedly blocked URL's in Google search results - there will be no description or cache but the URL's will be indexed.

For a thorough run down of the difference between blocking and crawling see Matt Cutts video 
Definitely award for Andy's last comment
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.