Solved

SEO BLock Url to get indexed in Google.

Posted on 2011-03-03
11
515 Views
Last Modified: 2012-05-11
I want to block url from getting indexed in Google. Please help me in doing that. I know how to use robots.txt for restricting folders but how can I block URL indexing.

And How to check if SSL certificate is having right info or not.

Please help.
0
Comment
Question by:ravigosai
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +1
11 Comments
 
LVL 14

Expert Comment

by:Dhanasekaran Sengodan
ID: 35025644
place this line  in robots.txt file

Disallow: /file-name.html


0
 
LVL 14

Expert Comment

by:Dhanasekaran Sengodan
ID: 35025923
place this line  in robots.txt file

User-agent: *
Disallow: /
0
 

Author Comment

by:ravigosai
ID: 35029852
How can I block URL indexing by using Disallow:/

I didnt get it. The URL is not main website. still the URL is being redirected to the main landing page of the site. I dont know how to make sure that google does not index this particular URL.
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 

Author Comment

by:ravigosai
ID: 35030323
The URL I am trying to block lands on the same page as our main URL lands. is there a way to block indexing on it without redirecting it to 404?
0
 
LVL 14

Expert Comment

by:Dhanasekaran Sengodan
ID: 35033949
0
 
LVL 23

Expert Comment

by:Tony McCreath
ID: 35043481
Could you clarify what you want.

Do you really want to stop Google from indexing your home page?

It sounds like a very bad move. Could you explain why you want to do that.
0
 
LVL 29

Accepted Solution

by:
fibo earned 500 total points
ID: 35043542
B-) I must confess I am slightly confused at what is your problem and the solution you are seeking for.

Here is what I understand:
You have one file, say index.html
The content of this file can be displayed by 2 URLs, say mydomain.com/ (with or without the trailing slash) and /index.html
For SEO purpose, you would want search engines to keep the url / but not /index.html

Since it is the same file in 2 URLs disguises, you cannot use the otherwise ad-hoc tools, such has a <robot noindex> metatag in the html code.

There are 2 different problems
1 - Why are there references to index.html?
2 - How can I remove the url from the known urls?

1. Remove from YOUR site links to the bad URL

Point 1 is usually, and in this case most probably, created by you: your site itself references /index.html instead of /
So this is really your first item on the check list: change your source code so that "home" is / and not /index.html
ANY OTHER ACTION IS USELESS IF THIS IS NOT DONE, since you are nurturing the problem!
Since it is not that easy to check that there remains no link to the bad url, use one tool like Xenu http://home.snafu.de/tilman/xenulink.html which will make a list of the URLs linked to from within your site, as well as (this is the useful part here!) which pages are linking to this URL

At the same time, you might also consider using the canonical metatag to reinforce the above work. See page
http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html
Note that this is not an alternative to tidying your own site from references to the "bad" url

2. Remove the URL from the search indexes

Once this is solved, you now need to remove the url /index.html from the indexes of Google + Bing + Yahoo.
Just connect to their webmasters tools, validate you site (my preferred solution is one extra file, but if fails in some situations). While you are at it, communicate them any sitemap you might have done --after checking that the "bad" url is absent from the map!--
Wait 2 days, then come back to the tool, check if they have recorded the "bad" url and if yes, use the tools to remove it.

Finally, you might use some url_rewrite... but I would not do that unless after 4 weeks the problem is still there.
0
 

Author Comment

by:ravigosai
ID: 35072454
Basically the site was using the old url before. and then we rebuilded the site and also we changed the URL. so now we were required to redirect the old URL also to new site so that was being done. so now google is indexing the old ULR as duplicate URL. and that is creating issue.
0
 
LVL 23

Expert Comment

by:Tony McCreath
ID: 35079662
Still not clear.

Could you provide some example urls with that description?

If you are redirecting the old url to the new url it should be a 301 redirect. This tells search engines to transfer their index from old to new. It your using any other form of redirect the transfer may not happen.
0

Featured Post

[Webinar] Code, Load, and Grow

Managing multiple websites, servers, applications, and security on a daily basis? Join us for a webinar on May 25th to learn how to simplify administration and management of virtual hosts for IT admins, create a secure environment, and deploy code more effectively and frequently.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will inform Clients about common and important expectations from the freelancers (Experts) who are looking at your Gig.
Although a lot of people devote their energy toward marketing for specific industries, there are some basic principles that can be applied to any sector imaginable. We’ll look at four steps to take and examine how those steps were put into action fo…
The viewer will get a basic understanding of what section 508 compliance can entail, learn about skip navigation links, alt text, transcripts, and font size controls.
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question