[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 180
  • Last Modified:

Stop search bots from cataloging site

Hi,
I'm building a few sites in which I do not want search bots to catalog the site Or put another way I would like them to ignore the site. How do I do this?
0
markloessi
Asked:
markloessi
  • 3
1 Solution
 
gripeCommented:
create a robots.txt file in your root directory with the following line:

User-agent: *
Disallow: /
0
 
gripeCommented:
Here is a link to a tutorial explaining all of the features of robots.txt in detail:

http://www.searchengineworld.com/robots/robots_tutorial.htm
0
 
gripeCommented:
> create a robots.txt file in your root directory with the following line:

This should read 'lines'.. so two separate lines - one for 'User-agent: *' and one for 'Disallow: /'
0

Featured Post

Important Lessons on Recovering from Petya

In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.

  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now