Partial page search engine indexing

Is there some code that can be used to hide part of a web page from indexing. (i.e. to hide flash content that has layers which seems to cause problems with google's indexing)
LVL 1
lallen30Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Graceful_PenguinCommented:
0
andrewbleakleyCommented:
No there isn't.

You can either hide the whole page or none of it - you can not hide part of a page.

Google has no issue with flash files, and has recently announced plans to index the text with in them

Use robots.txt and <META NAME="ROBOTS" CONTENT="NOINDEX"> robots.txt alone will not stop Google indexing the page, it wills till crawl and display a link to the pages in the search results.
0
Tony McCreathTechnical SEO ConsultantCommented:
You could move the bit you want hidden into a sperate file and include it via an iframe. Use the meta tag above to ensure the contents of the iframe are ignored.
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

AndyBeardCommented:
Google doesn't crawl pages blocked with robots.txt which means that you can't combine it with meta noindex as Google won't see the noindex.

The best solution currently is to render the flash with external javascript such as using Jquery, and to block the javascript using robot.txt
That will work for all major engines

If you only care about Google, you might try their robots.txt noindex directive, thogh I believe this is still a little unofficial, or serving all flash content with X-Robots noindex in the header.

Noindex: /
0
andrewbleakleyCommented:
From Google help documents http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

"While Google won't crawl or index the content of pages blocked by robots.txt, we may still index the URLs if we find them on other pages on the web"

If you only use the robots.txt and do not also include the rpbots meta tag with content=noindex you could still see your supposedly blocked URL's in Google search results - there will be no description or cache but the URL's will be indexed.

For a thorough run down of the difference between blocking and crawling see Matt Cutts video http://www.mattcutts.com/blog/robots-txt-remove-url/ 
0
AndyBeardCommented:
Google has to crawl a page to see any meta or x-robots instructions - the documents you refer to say Google doesn't crawl, so your suggestion to use meta robots in combination with robots.txt is in error.

Any use of robots.txt to block content is still creating a dangling node/page that accumulates PageRank and can show in results

This article predates the supposed changes with the handling on nofollow links, we don't know what is happening with the "reset vector" but the same is true with pages blocked by robots.txt - the juice goes into some kind of reset vector, possibly being handled in the same way.

http://andybeard.eu/1121/seo-linking-gotchas-even-the-pros-make.html

As Sebastian stated in the comments of your referred post

http://www.mattcutts.com/blog/robots-txt-remove-url/#comment-400703

hence the safe way to keep stuff out of search engine indexes is:

robots.txt: Allow: /path or Disallow:

Per URI in /path: “noindex” meta elements / x-robots-tags
You have to let the search engines in to read the page in order to then tell them not to index, use a technique such as noindex  in robots.txt which is only an unofficial Google option, or use a technique such as I suggested with javascript.

The best of those options with flash is in my opinion the javascript route, as long as any navigation elements are also present with text links.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
andrewbleakleyCommented:
Definitely award for Andy's last comment
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Web Development

From novice to tech pro — start learning today.