asked on

google only: the use of the word allow in robots.txt

robots.txt

I see keywords
'user-agent'
'allow'
'disallow'

is 'allow' used by google

I am only looking for a google answer

My client is upset that many pages are visible in google but does not care about other search engines at this moment

I could explain to client about rogue webcrawlers that gather personal information, but all they care about now is google

So could you answer question
yes google follows the word 'allow'
no google does not follow the word 'allow'

and provide the reasons.

Client wants to block many pages and future pages from google.

SOLUTION

QuinnDex

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

SOLUTION

Anwar Saiah

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ASKER CERTIFIED SOLUTION

Gary

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

QuinnDex

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

will make google drop the index of a page in time...

you can stop google from showing pages much sooner by going to google webmaster tools and telling google what pages not to show

SOLUTION

Loganathan Natarajan

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

rgb192

ASKER

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
and removal with webmaster tools

thanks