SEO BLock Url to get indexed in Google.

I want to block url from getting indexed in Google. Please help me in doing that. I know how to use robots.txt for restricting folders but how can I block URL indexing.

And How to check if SSL certificate is having right info or not.

Please help.
Who is Participating?
Bernard S.Connect With a Mentor CTOCommented:
B-) I must confess I am slightly confused at what is your problem and the solution you are seeking for.

Here is what I understand:
You have one file, say index.html
The content of this file can be displayed by 2 URLs, say (with or without the trailing slash) and /index.html
For SEO purpose, you would want search engines to keep the url / but not /index.html

Since it is the same file in 2 URLs disguises, you cannot use the otherwise ad-hoc tools, such has a <robot noindex> metatag in the html code.

There are 2 different problems
1 - Why are there references to index.html?
2 - How can I remove the url from the known urls?

1. Remove from YOUR site links to the bad URL

Point 1 is usually, and in this case most probably, created by you: your site itself references /index.html instead of /
So this is really your first item on the check list: change your source code so that "home" is / and not /index.html
ANY OTHER ACTION IS USELESS IF THIS IS NOT DONE, since you are nurturing the problem!
Since it is not that easy to check that there remains no link to the bad url, use one tool like Xenu which will make a list of the URLs linked to from within your site, as well as (this is the useful part here!) which pages are linking to this URL

At the same time, you might also consider using the canonical metatag to reinforce the above work. See page
Note that this is not an alternative to tidying your own site from references to the "bad" url

2. Remove the URL from the search indexes

Once this is solved, you now need to remove the url /index.html from the indexes of Google + Bing + Yahoo.
Just connect to their webmasters tools, validate you site (my preferred solution is one extra file, but if fails in some situations). While you are at it, communicate them any sitemap you might have done --after checking that the "bad" url is absent from the map!--
Wait 2 days, then come back to the tool, check if they have recorded the "bad" url and if yes, use the tools to remove it.

Finally, you might use some url_rewrite... but I would not do that unless after 4 weeks the problem is still there.
Dhanasekaran SengodanCommented:
place this line  in robots.txt file

Disallow: /file-name.html

Dhanasekaran SengodanCommented:
place this line  in robots.txt file

User-agent: *
Disallow: /
Worried about phishing attacks?

90% of attacks start with a phish. It’s critical that IT admins and MSSPs have the right security in place to protect their end users from these phishing attacks. Check out our latest feature brief for tips and tricks to keep your employees off a hackers line!

ravigosaiAuthor Commented:
How can I block URL indexing by using Disallow:/

I didnt get it. The URL is not main website. still the URL is being redirected to the main landing page of the site. I dont know how to make sure that google does not index this particular URL.
ravigosaiAuthor Commented:
The URL I am trying to block lands on the same page as our main URL lands. is there a way to block indexing on it without redirecting it to 404?
Tony McCreathTechnical SEO ConsultantCommented:
Could you clarify what you want.

Do you really want to stop Google from indexing your home page?

It sounds like a very bad move. Could you explain why you want to do that.
ravigosaiAuthor Commented:
Basically the site was using the old url before. and then we rebuilded the site and also we changed the URL. so now we were required to redirect the old URL also to new site so that was being done. so now google is indexing the old ULR as duplicate URL. and that is creating issue.
Tony McCreathTechnical SEO ConsultantCommented:
Still not clear.

Could you provide some example urls with that description?

If you are redirecting the old url to the new url it should be a 301 redirect. This tells search engines to transfer their index from old to new. It your using any other form of redirect the transfer may not happen.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.