Link to home
Start Free TrialLog in
Avatar of linuxraja
linuxrajaFlag for India

asked on

robots file web

Issue: Preventing the search engine indexing for the URL https://xxx.com/dir1/

1. The actual URL is http://demo.xxx.linuxraja.com/site1/dir1/dir2/index.html
2. The https redirection below is mentioned in the above site configuration file

    RedirectMatch   ^/context1/$        /context1/dir2/home/index.html
    RedirectMatch   ^/context1/dir2/$   /context1/dir2/home/index.html

    RedirectMatch   ^(/context1/dir2/account/.*)$        https://xxx.com$1
    RedirectMatch   ^(/context1/dir2/deposit/.*)$        https://xxx.com$1
    RedirectMatch   ^(/context1/dir2/login/.*)$          https://xxx.com$1
    RedirectMatch   ^(/context1/dir2/play/.*)$           https://xxx.com$1
    RedirectMatch   ^(/context1/dir2/registration/.*)$   https://xxx.com$1
    RedirectMatch   ^(/context1/dir2/withdraw/.*)$       https://xxx.com$1
    RedirectMatch   ^(/newsys/details.jsp.*)$            https://xxx.com$1


3. Document root for demo.xxx.linuxraja.com is ROOT1
4. Document root for xxx.com is ROOT2 under ROOT2 there are lot of context like context1,context2....
5. I need to prevent indexing only the context1 contents. that is https://xxx.com/context1
6. I hope you will understand the senario
7. Can anyone give  sample the robots.txt file contents and where to place robots file
8. I already refered all site form google.Please give me the solution.
ASKER CERTIFIED SOLUTION
Avatar of dikshanttyagi
dikshanttyagi
Flag of India image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Member_2_231077
Member_2_231077

Here's one that stops CS and the Lounge from being indexed by all of them, https://www.experts-exchange.com/robots.txt .

Not sure why you posted this under hardware/servers though.
Why on earth type all that lot in when you can use "user-agent *" ?