view a hidden robots.txt file

Posted on 2009-02-23
Last Modified: 2012-05-06
When I go to

the robots.txt file is hidden

how can a person hide a robots.txt file

how can a person read a hidden robots.txt file
Question by:jerseyguy0
    LVL 20

    Expert Comment

    Sure it's hidden and not just empty?
    LVL 1

    Author Comment

    its hidden
    LVL 16

    Expert Comment

    How do you know it really is a robots.txt ?
    Hiding it makes its existence useless.

    Maybe it a script that only shows the contect on certain browser version, search engine, etc.
    LVL 20

    Expert Comment

    Of course they might evaluate the user agent string and send the full robots.txt only if the agent is among the list of known-to-thems.
    This will cause any new robot to see an empty robots.txt and hence it will not preventd from indexing even if it obeys the robots protocol.
    LVL 12

    Accepted Solution

    Hi here;

    how can a person hide a robots.txt file:

    A very nice tutorial on this...

    Best regards...
    LVL 31

    Assisted Solution

    Here is another article, with a more advanced script with more user agents


    Featured Post

    How to run any project with ease

    Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
    - Combine task lists, docs, spreadsheets, and chat in one
    - View and edit from mobile/offline
    - Cut down on emails

    Join & Write a Comment

    You cannot be 100% sure that you can protect your organization against crypto ransomware but you can lower down the risk and impact of the infection.
    These days socially coordinated efforts have turned into a critical requirement for enterprises.
    The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
    This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

    754 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    15 Experts available now in Live!

    Get 1:1 Help Now