Solved

SEO BLock Url to get indexed in Google.

Posted on 2011-03-03
11
514 Views
Last Modified: 2012-05-11
I want to block url from getting indexed in Google. Please help me in doing that. I know how to use robots.txt for restricting folders but how can I block URL indexing.

And How to check if SSL certificate is having right info or not.

Please help.
0
Comment
Question by:ravigosai
  • 3
  • 3
  • 2
  • +1
11 Comments
 
LVL 14

Expert Comment

by:Dhanasekaran Sengodan
ID: 35025644
place this line  in robots.txt file

Disallow: /file-name.html


0
 
LVL 14

Expert Comment

by:Dhanasekaran Sengodan
ID: 35025923
place this line  in robots.txt file

User-agent: *
Disallow: /
0
 

Author Comment

by:ravigosai
ID: 35029852
How can I block URL indexing by using Disallow:/

I didnt get it. The URL is not main website. still the URL is being redirected to the main landing page of the site. I dont know how to make sure that google does not index this particular URL.
0
Online Training Solution

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action. Forget about retraining and skyrocket knowledge retention rates.

 

Author Comment

by:ravigosai
ID: 35030323
The URL I am trying to block lands on the same page as our main URL lands. is there a way to block indexing on it without redirecting it to 404?
0
 
LVL 14

Expert Comment

by:Dhanasekaran Sengodan
ID: 35033949
0
 
LVL 23

Expert Comment

by:Tony McCreath
ID: 35043481
Could you clarify what you want.

Do you really want to stop Google from indexing your home page?

It sounds like a very bad move. Could you explain why you want to do that.
0
 
LVL 29

Accepted Solution

by:
fibo earned 500 total points
ID: 35043542
B-) I must confess I am slightly confused at what is your problem and the solution you are seeking for.

Here is what I understand:
You have one file, say index.html
The content of this file can be displayed by 2 URLs, say mydomain.com/ (with or without the trailing slash) and /index.html
For SEO purpose, you would want search engines to keep the url / but not /index.html

Since it is the same file in 2 URLs disguises, you cannot use the otherwise ad-hoc tools, such has a <robot noindex> metatag in the html code.

There are 2 different problems
1 - Why are there references to index.html?
2 - How can I remove the url from the known urls?

1. Remove from YOUR site links to the bad URL

Point 1 is usually, and in this case most probably, created by you: your site itself references /index.html instead of /
So this is really your first item on the check list: change your source code so that "home" is / and not /index.html
ANY OTHER ACTION IS USELESS IF THIS IS NOT DONE, since you are nurturing the problem!
Since it is not that easy to check that there remains no link to the bad url, use one tool like Xenu http://home.snafu.de/tilman/xenulink.html which will make a list of the URLs linked to from within your site, as well as (this is the useful part here!) which pages are linking to this URL

At the same time, you might also consider using the canonical metatag to reinforce the above work. See page
http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html
Note that this is not an alternative to tidying your own site from references to the "bad" url

2. Remove the URL from the search indexes

Once this is solved, you now need to remove the url /index.html from the indexes of Google + Bing + Yahoo.
Just connect to their webmasters tools, validate you site (my preferred solution is one extra file, but if fails in some situations). While you are at it, communicate them any sitemap you might have done --after checking that the "bad" url is absent from the map!--
Wait 2 days, then come back to the tool, check if they have recorded the "bad" url and if yes, use the tools to remove it.

Finally, you might use some url_rewrite... but I would not do that unless after 4 weeks the problem is still there.
0
 

Author Comment

by:ravigosai
ID: 35072454
Basically the site was using the old url before. and then we rebuilded the site and also we changed the URL. so now we were required to redirect the old URL also to new site so that was being done. so now google is indexing the old ULR as duplicate URL. and that is creating issue.
0
 
LVL 23

Expert Comment

by:Tony McCreath
ID: 35079662
Still not clear.

Could you provide some example urls with that description?

If you are redirecting the old url to the new url it should be a 301 redirect. This tells search engines to transfer their index from old to new. It your using any other form of redirect the transfer may not happen.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Learn about the eCommerce marketing trends for the year ahead.
This article will inform Clients about common and important expectations from the freelancers (Experts) who are looking at your Gig.
Any person in technology especially those working for big companies should at least know about the basics of web accessibility. Believe it or not there are even laws in place that require businesses to provide such means for the disabled and aging p…
The is a quite short video tutorial. In this video, I'm going to show you how to create self-host WordPress blog with free hosting service.

685 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question